JupyterHub
JupyterHPC is currently being rolled out and is not yet available for all HPC users. For legacy SCC users the previous version is still available at a different location: https://jupyter-hpc.gwdg.de
The JupyterHPC service offers an interactive Python and R environment. A default container is provided which allows users to run IPython notebooks or RStudio.
Please note that the settings you use to start your JupyterHPC session will be saved in your browser. A log of the container execution will be created in your HPC home directory in the current.jupyterhub.notebook.log
file. Please refer to this file if you encounter problems with JupyterHPC and attach it to your email if you contact customer support. Also check that you are below the quota for your home directory.
For calculations that need compute resources over a long duration, please submit a batch job to the appropriate slurm partition instead of using JupyterHPC.
Creating your own JupyterHPC container
Users with more complex requirements can build their own Apptainer containers… The following basic container recipe can be used as a starting point. It is important to keep the version of the jupyterhub package close to the one that is currently used on the hub (4.1.5).
Bootstrap: docker
From: condaforge/miniforge3
%post
export DEBIAN_FRONTEND=noninteractive
apt update
apt upgrade -y
conda install --quiet --yes \
'ipyparallel=8.8.0' \
'notebook=7.2.1' \
'jupyterhub=4.1.5' \
'jupyterlab=4.2.3'
# Here you can add your own python packages
conda install --quiet --yes \
pandas \
scipy \
seaborn
A jupyterhub-singleuser
binary that is compatible with the hub must be in the $PATH
of the container for a successful startup. Otherwise you can extend the container as much as you want.
A more complex example that includes RStudio integration as well as PyTorch is shown below:
Bootstrap: docker
From: condaforge/miniforge3
%post
export DEBIAN_FRONTEND=noninteractive
apt update
apt upgrade -y
apt install -y --no-install-recommends software-properties-common dirmngr
wget -qO- https://cloud.r-project.org/bin/linux/ubuntu/marutter_pubkey.asc | tee -a /etc/apt/trusted.gpg.d/cran_ubuntu_key.asc
add-apt-repository "deb https://cloud.r-project.org/bin/linux/ubuntu $(lsb_release -cs)-cran40/"
add-apt-repository ppa:c2d4u.team/c2d4u4.0+
apt update
apt install -y \
r-base \
r-cran-caret \
r-cran-crayon \
r-cran-devtools \
r-cran-forecast \
r-cran-hexbin \
r-cran-htmltools \
r-cran-htmlwidgets \
r-cran-plyr \
r-cran-randomforest \
r-cran-rcurl \
r-cran-reshape2 \
r-cran-rmarkdown \
r-cran-rodbc \
r-cran-rsqlite \
r-cran-shiny \
r-cran-tidyverse
apt install -y libclang-dev lsb-release psmisc sudo
ubuntu_release=$(lsb_release --codename --short)
wget https://download2.rstudio.org/server/${ubuntu_release}/amd64/rstudio-server-2023.12.1-402-amd64.deb
dpkg --install rstudio-server-2023.12.1-402-amd64.deb
rm rstudio-server-2023.12.1-402-amd64.deb
echo 'ftp_proxy=http://www-cache.gwdg.de:3128' >> /usr/lib/R/etc/Renviron.site
echo 'https_proxy=http://www-cache.gwdg.de:3128' >> /usr/lib/R/etc/Renviron.site
echo 'http_proxy=http://www-cache.gwdg.de:3128' >> /usr/lib/R/etc/Renviron.site
echo '' >> /usr/lib/R/etc/Renviron.site
conda install --quiet --yes \
'ipyparallel=8.8.0' \
'jupyter-rsession-proxy=2.2.1' \
'notebook=7.2.1' \
'jupyterhub=4.1.5' \
'jupyterlab=4.2.3'
conda install --quiet --yes \
dgl \
igraph \
keras \
pandas \
pydot \
scikit-learn \
scipy \
seaborn
conda install --quiet --yes --channel pytorch --channel nvidia \
pytorch \
pytorch-cuda=12.1 \
torchaudio \
torchvision