Focus on:  Hide Contributions
Login

HPC Cloud Workshop - Storm 24

→ Europe/Berlin
Auditorium, D2 building (Max Planck Computing & Data Facility)

Auditorium, D2 building

Max Planck Computing & Data Facility

Giessenbachstr. 2, 85748 Garching, Germany
Frank Berghaus (MPCDF), John Alan Kennedy (MPCDF)
Description

Dear HPC-Cloud users,

The workshop aims to bring cloud projects and MPCDF cloud experts togehter to share ideas and discuss future directions. 

The workshop will take place at the MPCDF in Garching, as an in person event, due to the interactive nature of the workshop. 

Your sign-up includes the optional participation in the Beer Garden gathering (10 September), the tour of the Max Planck Computing & Data Facility (11 September) and the Dinner at Gasthof Neuwirt (11 September). 

We are looking forward to welcoming you in September to the MPCDF on the Science Campus in the north of Munich. 

John Kennedy & Frank Berghaus

 

https://www.mpcdf.mpg.de/contact/directions

  • Tuesday, 10 September
    • 09:00 → 13:00
      Arrivals

      · MPCDF staff will be available
      · Conf rooms open to allow guests to work
      · Options for coffee
      · Option for Lunch at canteen

    • 13:00 → 14:30
      Welcome 1h 30m Auditorium, D2 building

      Auditorium, D2 building

      Max Planck Computing & Data Facility

      Giessenbachstr. 2, 85748 Garching, Germany
      Speaker: John Kennedy (MPCDF)
    • 14:30 → 15:00
      Coffee 30m Lobby D2 building

      Lobby D2 building
    • 15:00 → 17:00
      Projects Presentations 1 2h Auditorium, D2 building

      Auditorium, D2 building

      Max Planck Computing & Data Facility

      Giessenbachstr. 2, 85748 Garching, Germany
      Speaker: Frank Berghaus (MPCDF)
      • On Cloud Full Stack Bioinformatics 20m

        This talk will cover the aspects of a bioinformatician stack and how we currently run it 100% over the cloud.
        Starting with the UI to the facility, Flaski, users are able to access data lakes, do data visualization, and create jobs for large scale processing (eg. RNAseq, variant calling ). Making use of Nextflow and Raven these jobs run fully automated from request submission to data delivery over MPCDF’s OwnCloud instance. Data transfer takes place over an FTP server (Docker over OpenSatck) which access control is managed from Flaski. Continuous deployment of Flaski is done with the use of GitHub and DockerHub, leading to the automated deployment of every commit to a development namespace on Kubernetes and every tag to the production namespace. Kubernetes is per se deployed over OpenStack. For interactive work a posit container is deployed on an OpenStack instance with access to Nexus. A small HPC cluster is as well deployed over OpenStack to support development and custom work as well as large time and/or memory jobs. Details on several automation steps will be described eg. archiving of raw data and projects, user and disk space management. With this the facility is now able to computational support automated running of workflows over Raven for the entire Max Planck Society and deliver Nextflow config files for local runs to researchers worldwide.

        Speaker: Jorge Boucas (MPI Biology of Ageing)
        Boucas_MPCDF_CLOUD.pdf
        Boucas_MPCDF_CLOUD.pptx
      • Operating NOMAD: A FAIR data service on the MPCDF HPC Cloud with Kubernetes 20m
        Speaker: Markus Scheidgen (FAIRmat/NOMAD Lab; IRIS Humboldt-Universität zu Berlin)
        FAIR research data with NOMAD
        markus-scheidgen.pdf
      • Workstations, SLURM cluster, Analysis server, User Storage, Network Integration – Use cases and challenges of the FHI 30m

        The FHI moves more and more computing and analysis related services to the MPCDF HPC cloud – from high-resource virtual workstations for the FHI Theory department, GPU-accelerated nodes for Machine-Learning, a ‘speciality’ SLURM cluster, an FHI-wide analysis server, a JupyterHub to S3 replication, backup, and end-user buckets.
        We provide a short overview of the use cases of the FHI, and address former and current challenges in implementing and using the MPCDF HPC cloud infrastructure. In addition we give a short overview of the network, IDM, and Storage integration between MPCDF and FHI.

        Speakers: Maurits Vuijk (FHI - Theory Dept), Simeon Beinlich (FHI - PP&B IT service group)
        FHI-MPCDF-HPC-CLOUD-WORKSHOP-2024.pdf
        FHI-MPCDF-HPC-CLOUD-WORKSHOP-2024.pptx
    • 18:00 → 20:00
      Beer garden Mühlenpark Garching Garching

      Garching

      [Mühlenpark Garching](https://www.biergarten-muehlenpark.de/) Biergarten Mühlenpark Mühlgasse 48 85748 Garching

      Garchinger Beer Garden

  • Wednesday, 11 September
    • 09:00 → 10:30
      Project Presentations 2 1h 30m Auditorium, D2 building

      Auditorium, D2 building

      Max Planck Computing & Data Facility

      Giessenbachstr. 2, 85748 Garching, Germany
      Speaker: John Kennedy (MPCDF)
    • 10:30 → 11:00
      Coffee 30m Lobby D2 building

      Lobby D2 building
    • 11:00 → 12:00
      Projects Presentations 3 1h Auditorium, D2 building

      Auditorium, D2 building

      Max Planck Computing & Data Facility

      Giessenbachstr. 2, 85748 Garching, Germany
      Speaker: Frank Berghaus (MPCDF)
      • The Digitization of the Photographic Collection at BHMPI 20m

        “In this presentation we provide details about how the digitization workflow for the Photographic Collection took place. Starting from preparatory work, the actual process of scanning the photographic material will be described including all steps of transformation and archival using MPCDF resources as Raven, the Nexus Storage and the Long term archive, to achieve a stable and reproducible workflow which we use for our usual business and to serve the images via IIIF to all our partners world-wide. Plans for further developments will also be addressed.”

        Speaker: Pietro Liuzzo (MPI - Bibliotheca Hertziana)
        HPC_cloud_workshop_Liuzzo.pptx
      • SciServer 20m

        In 2020 the MPE SciServer (sciserver.mpe.mpg.de), a system originally developed by the Johns Hopkins Institute for Data Intensive Engineering and Science was deployed at the MPCDF for the Max-Planck Institute for Extraterrestrial Physics (MPE).
        This science platform is serving as a collaboration tool for the eROSITA and HETDEX projects, allowing around 200 astronomers from both international collaborations to work on their data in one place in a secure, consistent, and yet flexible manner. Given the success of the tool in the last 4 years, the MPE SciServer project is now aiming to shift from our manually maintained hardware into the use of the flexibility and scalability the HPC Cloud offers in the future.

        Speaker: Joel Gil (MPI for Extraterrestrial Physics)
        Joel&Jonas final Sciserver HPC Workshop.pdf
        Joel&Jonas final Sciserver HPC Workshop.pptx
    • 12:00 → 13:00
      Lunch 1h IPP canteen

      IPP canteen
    • 13:00 → 14:30
      BoF sessions 1 & 2 1h 30m various rooms at MPCDF

      various rooms at MPCDF
      • Kubernetes, Containers etc 30m

        An open discussion about containers and their orchestration in the HPC-Cloud

        Speaker: Frank Berghaus (MPCDF)
        bof_k8s_containers.odp
      • Automation and Infrastructure as Code 30m

        Open discussion about automation in the cloud
        * Infrastructure Automation
        * VM configuration
        * CI/CD

        Speaker: John Kennedy (MPCDF)
        Automation_IaC.odp
        Automation_IaC.pdf
    • 14:30 → 16:30
      BoF sessions 3 & 4 2h
      • Storage in the cloud 30m

        Open discussion about storage in the cloud
        * Nexus-Posix (filesystem)
        * Nexus-S3 (object storage)
        * Manila (filesystems as a service)
        * Block Storage (volumes)

        Speaker: Frank Berghuas (MPCDF)
        bof_storage.odp
      • Clusters in the cloud 30m

        Open discussion about slurm clusters, and possibly others, in the cloud
        * JADE
        * Roll your own
        * Anything Similar (spark etc?)

        Speaker: John Kennedy (MPCDF)
        Clusters_in_the_Cloud.odp
        Clusters_in_the_Cloud.pdf
    • 16:30 → 17:15
      Tour of Computing Facility 45m Auditorium, D2 building

      Auditorium, D2 building

      Max Planck Computing & Data Facility

      Giessenbachstr. 2, 85748 Garching, Germany
    • 18:55 → 20:55
      Evening event 2h Gasthof Neuwirt, Garching

      Gasthof Neuwirt, Garching

      https://gasthof-neuwirt.org/

      Dinner at Gasthof Neuwirt

  • Thursday, 12 September
    • 09:00 → 10:30
      Wrap-up session 1h 30m Auditorium, D2 building

      Auditorium, D2 building

      Max Planck Computing & Data Facility

      Giessenbachstr. 2, 85748 Garching, Germany

      . 3-2-1 Prize Draw
      · Reports from BoFs
      · Wish list Gathering

      Speaker: John Kennedy (MPCDF)
      HPC-Cloud-Workshop-BoF.pdf
      HPC-Cloud-Workshop-General.pdf
      HPC-Cloud-Workshop-Wrapup.pdf
      Wrapup.odp
      Wrapup.pdf
    • 10:30 → 11:00
      Coffee 30m Lobby D2

      Lobby D2
    • 11:00 → 12:30
      Round Table discussion 1h 30m Auditorium, D2 building

      Auditorium, D2 building

      Max Planck Computing & Data Facility

      Giessenbachstr. 2, 85748 Garching, Germany
      Speaker: Frank Berghaus (MPCDF)
Powered by Indico v3.3.2