JupyterHub

The JupyterHPC service offers an interactive Python and R environment. A default container allows users to run RStudio.

Info

If you contact GWDG support about JupyterHPC, please use one of the HPC specific support addresses or make it clear that you are talking about a HPC service, so that your request can be quickly and efficiently handled without a detour to the GWDG Jupyter Cloud team.

Please note that the settings you use to start your JupyterHPC session will be saved in your browser. A log of the container execution will be created in your HPC home directory in the current.jupyterhub.notebook.log file. Please refer to this file if you encounter problems with JupyterHPC and attach it to your email if you contact customer support. Also check that you are below the quota for your home directory.

For calculations that need compute resources over a long duration, please submit a batch job to the appropriate slurm partition instead of using JupyterHPC.

Creating your own JupyterHPC container

Users with more complex requirements can build their own Apptainer containers… The following basic container recipe can be used as a starting point. It is important to keep the version of the jupyterhub package close to the one that is currently used on the hub (4.1.5).

Bootstrap: docker
From: condaforge/miniforge3

%post
    export DEBIAN_FRONTEND=noninteractive
    apt update
    apt upgrade -y

    conda install --quiet --yes \
        'ipyparallel=8.8.0' \
        'notebook=7.2.1' \
        'jupyterhub=4.1.5' \
        'jupyterlab=4.2.3'

    # Here you can add your own python packages
    conda install --quiet --yes \
        pandas \
        scipy \
        seaborn

A jupyterhub-singleuser binary that is compatible with the hub must be in the $PATH of the container for a successful startup. Otherwise you can extend the container as much as you want.

A more complex example that includes RStudio integration as well as PyTorch is shown below:

Bootstrap: docker
From: condaforge/miniforge3

%post
    export DEBIAN_FRONTEND=noninteractive 
    apt update 
    apt upgrade -y

    apt install -y --no-install-recommends software-properties-common dirmngr
    wget -qO- https://cloud.r-project.org/bin/linux/ubuntu/marutter_pubkey.asc | tee -a /etc/apt/trusted.gpg.d/cran_ubuntu_key.asc
    add-apt-repository "deb https://cloud.r-project.org/bin/linux/ubuntu $(lsb_release -cs)-cran40/"
    add-apt-repository ppa:c2d4u.team/c2d4u4.0+
    apt update

    apt install -y \
        r-base \
        r-cran-caret \
        r-cran-crayon \
        r-cran-devtools \
        r-cran-forecast \
        r-cran-hexbin \
        r-cran-htmltools \
        r-cran-htmlwidgets \
        r-cran-plyr \
        r-cran-randomforest \
        r-cran-rcurl \
        r-cran-reshape2 \
        r-cran-rmarkdown \
        r-cran-rodbc \
        r-cran-rsqlite \
        r-cran-shiny \
        r-cran-tidyverse 

    apt install -y libclang-dev lsb-release psmisc sudo
    ubuntu_release=$(lsb_release --codename --short) 
    wget https://download2.rstudio.org/server/${ubuntu_release}/amd64/rstudio-server-2023.12.1-402-amd64.deb
    dpkg --install rstudio-server-2023.12.1-402-amd64.deb
    rm rstudio-server-2023.12.1-402-amd64.deb

    echo 'ftp_proxy=http://www-cache.gwdg.de:3128' >> /usr/lib/R/etc/Renviron.site
    echo 'https_proxy=http://www-cache.gwdg.de:3128' >> /usr/lib/R/etc/Renviron.site
    echo 'http_proxy=http://www-cache.gwdg.de:3128' >> /usr/lib/R/etc/Renviron.site

    echo '' >> /usr/lib/R/etc/Renviron.site
    conda install --quiet --yes \
        'ipyparallel=8.8.0' \
        'jupyter-rsession-proxy=2.2.1' \
        'notebook=7.2.1' \
        'jupyterhub=4.1.5' \
        'jupyterlab=4.2.3'
   
    conda install --quiet --yes \
        dgl \
        igraph \
        keras \
        pandas \
        pydot \
        scikit-learn \
        scipy \
        seaborn 
 
    conda install --quiet --yes --channel pytorch --channel nvidia \
        pytorch \
        pytorch-cuda=12.1 \
        torchaudio \
        torchvision

Overview of custom JupyterHPC Job Profiles

The following profile ids can be used with the Custom Profile option in JupyterHPC.

Profile ID	Profile Name
`danuma-summer-school`	Summer School 2024: Datenkompetenzen in der Nutztierhaltung
`gpu-bioinformatics`	GPU Workshop: Bioinformatics (July 2024)
`graphlearning_ws24`	Lecture: M.Inf.2204 Introduction to Graph Machine Learning
`isc2024-accel-gen-ai-pytorch`	ISC 2024 Course: Accelerating Generative AI with PyTorch
`isc2024-quantum-machine-learning`	ISC 2024 Course: Quantum Machine Learning
`sc2024-quantum-machine-learning`	SC 2024 Course: Quantum Machine Learning

Legacy JupyterHPC

SCC users with a full GWDG account (not via the HPC Project Portal) can also still use the legacy JupyterHPC installation. This server will be shut down in the near future - please do not setup any new projects here.