15-16.5.2024 9:00-12:00 online

How to handle large datasets, install own software and scale up workflows efficiently in CSC's computing environment

Are you using CSC's high-performance computing (HPC) services (Puhti, Mahti, Allas...), but want to make sure you are getting the most out of them? Are you working with data in the most efficient way? Want to know the best tips and tricks of the trade when scaling up your workflows? This intensive course is intended for you! Please check the course prerequisites before registering.

This online course consisting of 2 half-day sessions focuses on using the CSC HPC environment, which has been tailored to be easy and efficient for research and development use. The CSC services (Puhti, Allas...) discussed in this course are free-of-charge for academic research, education and training purposes in Finnish higher education institutions and in state research institutes (subsidized by the Ministry of Education and Culture, Finland). This course is also suitable for company representatives who are curious about HPC and want to deepen their knowledge on CSC's computing resources and how to best access and use them.

Note, that this is a shorter version of the previous CSC Computing Environment courses. Course "CSC Computing Environment, Part 1: Basics" is organised 24.-25.4.2024, and requires a separate registration.

Prerequisites

Before the course you need (see below for more details): 

• Basic Linux skills 
• CSC credentials & project
• Laptop with Zoom (and an SSH client)
• "Basics" of using CSC computing environment course or similar knowledge

To help and make sure the course participants have the required Linux skills, we have created a small self-learning course / skill test in eLena: "Basic Command Line skills".

Please register and complete this prerequisite course before the actual course! We have a lot of topics to cover and cannot support login or very basic Linux issues during the course.

1. As this is the "Part 2", independent extension of the Using CSC computing environment, Part 1: Basics course, we expect you to have this basic knowledge of how to use our services, either from the basics course arranged 24.-25.4 or earlier, from the online self-learning course (topics 1-7), or from previous experiences. 
2. Basic Linux skills are needed: how to move in directories, copy and edit files – basic command-line usage (ls, cp, cd, nano, more, cat, echo, mkdir, rm, tar ... ). If you're not familiar with these, prepare by going through the recommended material below. If you are unsure of the level, please contact us! Here are some links for easy self-study:
   • Linux command line and tutorials that you can do in your browser 
   • CSC's Linux 1 course materials (check the materials links)
   • Basic Linux Commands 10 min tutorial video (sit back and watch)
   • CSC Linux Cheat Sheet (one page summary of the most important Linux commands – handy to have near you during the course)
   • Linux basics tutorial for CSC (thorough guide including also topics covered in the course)
   • Introduction to Linux tutorial videos (Chapters 4, 5)
3. In case you do not have an account for CSC's services yet, please apply for one before the course – it is needed for the tutorials & exercises. Membership of a computing project with access to Puhti and Allas is also required (FAQ: Do I have a project?). Please contact us (servicedesk@csc.fi) if you have difficulties with these steps!
4. A computer with Zoom and an SSH client (see more about connecting here - one option is to use Puhti web interface).

We will organise a support session on Wednesday 8.5. at 14:00 where you can get help if there are any problems with these preparatory steps. 

Tentative schedule:

Wednesday 15.5. 9:00-12:00

• How to handle data when scaling up, how to install own software efficiently

Thursday 16.5. 9:00-12:00

• Containers & Apptainer, high-throughput workflows

(+ support session on 8.5. 14:00 as part of Weekly User coffee for accessing course materials and resources. Please note that we start the course assuming that you have the required prerequisite knowledge!)

Learning outcomes

After completing this course and self-studies, you should know how to work efficiently with large datasets, own installations and workflows in CSC's computing environment. You should also be aware of the pitfalls that may arise when scaling up and how to avoid them, as well as where to find more information.

The trainers: Maria Lehtivaara, Rasmus Kronberg, Ari-Matti Saren, Atte Sillanpää, Kimmo Mattila, Laxmana Yetukuri, Xavier Anthony Raj, Nino Runeberg

Course materials: Before the course you will get access to the course materials in our eLena e-learning platform