Distributed computing and automated workflows have become central in scientific computing. At CSC we've tested, implemented, and deprecated different throughput tools and workflows over the recent years. Here we'd like to share our understanding of the pros and cons of various approaches and also learn from your experiences and needs in this workshop/hackathon.

The event will have presentations from participants (something along the lines of your context, aim, used resources, implemented solution, solved issues, open issues, and future directions), followed by CSC presentation on how HTC + workflows look like from our perspective and what tools and documentation we've made available.  The goal of this two-day event is to understand the needs and challenges of computational experts in addition to providing ample interaction between CSC specialists and participants. 

Your interesting workflow(s) are likely different from others and you probably have already set up some environment for your needs in your laptop/workstation. Please bring your own laptop: on Tuesday we split to two rooms for hands-on and only one has workstations.

Prerequirements

The content will be somewhat detailed and technical, intermediate to advance level, and hence prior hands-on experience on HPC systems (Puhti, Mahti, LUMI) will be useful. We also welcome beginners, but we have limited possibilities to cover very basic things. To benefit maximally, please take some of our introductory courses in advance, e.g. CSC Computing Environment

This event is offered free of charge. CSC will sponsor coffee/lunches for presenters while others can purchase lunch in one of the restaurants close by.

Zoom and HedgeDoc links will be provided to registered remote participants via email.

Target Audience:

Computational scientists who work with some kind of workflows in CSC environment.

Where: Hybrid event (On-site at CSC plus online)

NOTE: the event was postponed 2 weeks to 13-14 December

Day 1 - Tuesday 13 Dec - Workflows and high throughput: current status

9:00 Welcome, orientation, introductions (Atte, CSC)
9:30 Participant use case presentations: Vitus Besel: Atmospheric chemistry
10:15 break
10:30 Participant use case presentations: Jan Kloppenburg: workflow framework on puhti, triton, lumi
11:15 Participant use case presentations: Kim Eklund: USPEX
11:30 lunch
12:30 Bottlenecks in High throughput usage in HPC from CSC perspective (Sami, CSC)
13:15 CSC solutions to a variety of issues (Henrik, CSC)
14:00 break
14:20  Built-in tools for HTC (Gromacs/CP2K) (Rasmus, CSC)
14:40 Nextflow as a workflow framework (Laxman, CSC)
15:00 Discussion
16:00 Wrap up: common issues, likely solutions, synergies, tentative plan for the hackathon topics for day2

Day 2 - Wednesday 14 Dec -  Hackathon: try out and implement new solutions

9:00 Collect cases to work on and divide into groups
9:30 CSC specialists available to help with the implementation
12:00 lunch
13:00 Hackathon continues
16:00 Wrap up: what worked, where more work needed, what are the next steps?

Organisers:

Atte Sillanpää (e-mail: atte.sillanpaa at csc.fi)
Laxmana Yetukuri (e-mail: laxmana.yetukuri at csc.fi)
Rasmus Kronberg (e-mail: rasmus.kronberg at csc.fi)