JupyterHub

Warning

JupyterHPC is currently being rolled out and is not yet available for all HPC users.

The JupyterHPC service offers an interactive Python and R environment. A default container is provided which allows users to run IPython notebooks or RStudio.

Please note that the settings you use to start your JupyterHPC session will be saved in your browser. A log of the container execution will be created in your HPC home directory in the current.jupyterhub.notebook.log file. Please refer to this file if you encounter problems with JupyterHPC and attach it to your email if you contact customer support. Also check that you are below the quota for your home directory.

For calculations that need compute resources over a long duration, please submit a batch job to the appropriate slurm partition instead of using JupyterHPC.

Creating your own JupyterHPC container

Users with more complex requirements can build their own Apptainer containers… The following basic container recipe can be used as a starting point. It is important to keep the version of the jupyter packages close to the one that is currently used on the hub (4.0.0).

Bootstrap: docker
From: condaforge/miniforge3

%post
    export DEBIAN_FRONTEND=noninteractive
    apt update
    apt upgrade -y

    conda install --quiet --yes \
        'ipyparallel=8.6.1' \
        'notebook=6.5.4' \
        'jupyterhub=4.1.5' \
        'jupyterlab=4.1.6'

    # Here you can add your own python packages
    conda install --quiet --yes \
        pandas \
        scipy \
        seaborn

A jupyterhub-singleuser binary that is compatible with the hub must be in the $PATH of the container for a successful startup. Otherwise you can extend the container as much as you want.

A more complex example that includes RStudio integration as well as PyTorch is shown below:

Bootstrap: docker
From: condaforge/miniforge3

%post
    export DEBIAN_FRONTEND=noninteractive 
    apt update 
    apt upgrade -y

    apt install -y --no-install-recommends software-properties-common dirmngr
    wget -qO- https://cloud.r-project.org/bin/linux/ubuntu/marutter_pubkey.asc | tee -a /etc/apt/trusted.gpg.d/cran_ubuntu_key.asc
    add-apt-repository "deb https://cloud.r-project.org/bin/linux/ubuntu $(lsb_release -cs)-cran40/"
    add-apt-repository ppa:c2d4u.team/c2d4u4.0+
    apt update

    apt install -y \
        r-base \
        r-cran-caret \
        r-cran-crayon \
        r-cran-devtools \
        r-cran-forecast \
        r-cran-hexbin \
        r-cran-htmltools \
        r-cran-htmlwidgets \
        r-cran-plyr \
        r-cran-randomforest \
        r-cran-rcurl \
        r-cran-reshape2 \
        r-cran-rmarkdown \
        r-cran-rodbc \
        r-cran-rsqlite \
        r-cran-shiny \
        r-cran-tidyverse 

    apt install -y libclang-dev lsb-release psmisc sudo
    ubuntu_release=$(lsb_release --codename --short) 
    wget https://download2.rstudio.org/server/${ubuntu_release}/amd64/rstudio-server-2023.12.1-402-amd64.deb
    dpkg --install rstudio-server-2023.12.1-402-amd64.deb
    rm rstudio-server-2023.12.1-402-amd64.deb

    echo 'ftp_proxy=http://www-cache.gwdg.de:3128' >> /usr/lib/R/etc/Renviron.site
    echo 'https_proxy=http://www-cache.gwdg.de:3128' >> /usr/lib/R/etc/Renviron.site
    echo 'http_proxy=http://www-cache.gwdg.de:3128' >> /usr/lib/R/etc/Renviron.site

    echo '' >> /usr/lib/R/etc/Renviron.site
    conda install --quiet --yes \
        'ipyparallel=8.6.1' \
        'jupyter-rsession-proxy=2.2.0' \
        'notebook=6.5.4' \
        'jupyterhub=4.1.5' \
        'jupyterlab=4.1.6'
   
    conda install --quiet --yes \
        dgl \
        igraph \
        keras \
        pandas \
        pydot \
        scikit-learn \
        scipy \
        seaborn 
 
    conda install --quiet --yes --channel pytorch --channel nvidia \
        pytorch \
        pytorch-cuda=12.1 \
        torchaudio \
        torchvision