Using non-HPC storage from GWDG

Only HPC storage systems are available and mounted on HPC login- and compute nodes. This article describes how users from institutions with access can transfer data between the HPC storage systems and non-HPC storage provided by GWDG. These include the Stornext Unix homes and the GWDG Tape Archive (AHOME), which are not part of the HPC cluster and thus not accessible from regular HPC login, transfer or compute nodes.

GWDG storage transfer nodes

We offer two dedicated nodes, gwdu107 and gwdu108, reachable via gwdg-storage.hpc.gwdg.de, to transfer data between HPC and non-HPC storage, as well as between HPC usernames (like u12345) and non-HPC Academic Cloud usernames (like jdoe1). Login to these nodes via SSH (or rsync) is possible with all project-specific HPC usernames, as well as previous “legacy SCC usernames” by default. Other Academic Cloud usernames that are issued a GWDG Unix HOME (does not include federated accounts, self-registered accounts, and University of Göttingen student accounts) can be granted permission to login by our support.

The fingerprints of the nodes’ SSH host keys are the same as those of our other login nodes, see Logging In.

The nodes provide access to the GWDG StorNext Unix homes under /usr/users/, /home/uniXX/, /home/mpgXX/, the GWDG Tape Archive (AHOME) under /usr/users/a/ or /home/archiv/ and to all HPC filesystems except:

• Lustre RZG/Grete

Common use cases

Transfer data between Campus-Wide Unix Home directory and HPC

The common strategy is to login with your username owning the source data, give permission to your other username(s) to read (and optionally write) the data, then login with your target username to perform the actual transfer. The recommended way to do that is to change the owning group of the data in question to your HPC_u_<academicuser> group, where <academicuser> is your Academic Cloud username (like jdoe1). This group has all your different usernames, your project-specific HPC usernames, as well as your primary Academic Cloud username, as members.

For more information on how to use your HPC_u_<academicuser> group, setting permissions and transferring data, see the pages Data Migration Guide, Data Sharing and Data Transfers. If you are not very familiar with POSIX filesystem permissions, we recommend that you read the Strategy section here specifically, to get a better understanding before you go ahead and start copying/moving your data.

Example: Copy data from Unix home to HPC project directory

john@laptop:~$ ssh jdoe1@gwdg-storage
[...]
[jdoe1@gwdu108 ~]$ chgrp HPC_u_jdoe1 $HOME
[jdoe1@gwdu108 ~]$ chmod g+rX $HOME
[jdoe1@gwdu108 ~]$ chgrp -Rh HPC_u_jdoe1 my_data
[jdoe1@gwdu108 ~]$ chmod -R g+rX my_data
[jdoe1@gwdu108 ~]$ realpath my_data
/home/uni08/jdoe1/my_data
[jdoe1@gwdu108 ~]$ exit
john@laptop:~$ ssh u12345@gwdg-storage
[...]
u12345@gwdu107 ~ $ rsync -av /home/uni08/jdoe1/my_data /mnt/ceph-hdd/project/scc_umin_time/John/
[...]
u12345@gwdu107 ~ $ ls -lad /mnt/ceph-hdd/project/scc_umin_time/John/my_data
drwxr-x--- 8 u12345  HPC_u_jdoe1  0 2026-01-02 03:45 /mnt/ceph-hdd/project/scc_umin_time/John/my_data

Example: Move data from HPC home to Unix home

john@laptop:~$ ssh u12345@gwdg-storage
[...]
u12345@gwdu107 ~ $ chgrp HPC_u_jdoe1 $HOME
u12345@gwdu107 ~ $ chmod g+rX $HOME
u12345@gwdu107 ~ $ chgrp -Rh HPC_u_jdoe1 my_other_data
u12345@gwdu107 ~ $ chmod -R g+rwX my_other_data
u12345@gwdu107 ~ $ realpath my_other_data
/mnt/vast-standard/home/jdoe1/u12345/my_other_data
u12345@gwdu107 ~ $ exit
john@laptop:~$ ssh jdoe1@gwdg-storage
[...]
[jdoe1@gwdu108 ~]$ rsync -av --remove-source-files /mnt/vast-standard/home/jdoe1/u12345/my_other_data ./
[...]
[jdoe1@gwdu108 ~]$ ls -lad my_other_data
drwxr-x--- 8 u12345  HPC_u_jdoe1  0 2026-01-02 03:45 my_other_data
[jdoe1@gwdu108 ~]$ exit
john@laptop:~$ ssh u12345@gwdg-storage
[...]
u12345@gwdu108 ~ $ find my_other_data -type d -delete

We strongly recommend using rsync with the --remove-source-files option to move data across filesystems, instead of using mv (or cp -r followed by rm -r). This has various advantages, the most important being that your target user will be the new owner of the data and you can interrupt/cancel and resume the transfer as much as you want without running into trouble. It makes sure there are no half-copied files, you can’t accidentally delete files that have not been copied, files are not unnecessarily copied multiple times, etc.

One thing you might count as a disadvantage is that it leaves behind an empty “skeleton” of the nested directory structure at the source after removing the files. This is easily, cleanly and most important safely removed in the last step, using find ... -type d -delete, which will only remove empty directories, leaving non-empty directories and files untouched while giving you a warning.

Transferring data to the GWDG Tape Archive

When it comes to using your Academic username’s AHOME to archive data from HPC, there are some considerations to be made. You will likely want to use the tape archive to store data from expired projects, backup data from workspaces, or to deposit very cold data to save space in your HPC data stores that have limited quota.

It is a lot easier and more comfortable to make the AHOME writable by your HPC_u_<academicuser> group, and then only use your project-specific HPC usernames (u12345) to write data to it, or retrieve data from it. However:

• HPC usernames do not exist outside of the HPC system
• HPC usernames are project-specific and some projects are short-lived (most likely a lot shorter than the lifetime of your data in the tape archive)
• Using this method, the owner of the files and directories will be your various HPC usernames, not the actual owner of the AHOME
• Only the owner is able to change the group or permissions of files and directories
• The HPC_u_<academicuser> group might not be available on systems across the campus / in your institute. This will result in you being unable to access the data from machines other than the gwdg-storage HPC nodes.

Thus, we can only recommend this method conditionally.

john@laptop:~$ ssh jdoe1@gwdg-storage
[...]
[jdoe1@gwdu107 ~]$ chgrp HPC_u_jdoe1 $AHOME
[jdoe1@gwdu107 ~]$ chmod g+rX $AHOME
[jdoe1@gwdu107 ~]$ exit
john@laptop:~$ ssh u12345@gwdg-storage
[...]
u12345@gwdu107 ~ $ rsync -av --chown=:HPC_u_jdoe1 --chmod=g+w $PROJECT/John/data_to_be_archived /usr/users/a/jdoe1/

Please note the --chown and --chmod arguments to rsync. These will make sure your shared group has write permission to your archived data. Otherwise, if your HPC project expires, you run the risk of locking yourself out of being able to access or even delete your own archived data with no way to fix it on your own.

More correct, but also more complex method

If you need to access the data from other systems across the campus, you will need to give your Academic username permission to read the data, and use that username to run the transfer. However, in order to be able to read a file or directory, a username must be able to “enter” (technically: execute) every parent directory and you can only grant permission to directories you own. Thus, the only directory you are able to grant access to is your HPC username’s home directory, project data stores cannot be owned by individual users (not even the PI) for technical reasons. All in all, this means any data from an HPC project data store needs to be copied/moved to the respective HPC user’s home directory first and from there, the academic user can then continue to copy/move the data to AHOME.

u12345@gwdu107 ~ $ rsync -av $PROJECT/John/data_to_be_archived $HOME/
[...]
u12345@gwdu107 ~ $ chgrp HPC_u_jdoe1 $HOME
u12345@gwdu107 ~ $ chmod g+rX $HOME
u12345@gwdu107 ~ $ chgrp -Rh HPC_u_jdoe1 $HOME/data_to_be_archived
u12345@gwdu107 ~ $ chmod g+rwX $HOME/data_to_be_archived
u12345@gwdu107 ~ $ realpath $HOME/data_to_be_archived
/mnt/vast-standard/home/jdoe1/u12345/data_to_be_archived
u12345@gwdu107 ~ $ exit
john@laptop:~$ ssh jdoe1@gwdg-storage
[jdoe1@gwdu108 ~]$ rsync -av --remove-source-files /mnt/vast-standard/home/jdoe1/u12345/data_to_be_archived /usr/users/a/jdoe1/HPC_project_archive/
[...]
[jdoe1@gwdu108 ~]$ exit
john@laptop:~$ ssh u12345@gwdg-storage
[...]
u12345@gwdu108 ~ $ find data_to_be_archived -type d -delete

See the example for moving data from your HPC home to your Unix home above for more explanation of the steps performed in this example.

Functional Account Backup for Work Groups

For work groups, there are precedence cases where the group lead requests or provides a functional account, whose AHOME can be used to backup/store the whole workgroup’s inactive data, controlled by a trusted group member. In such cases, we can create a shared POSIX group with all project members and the functional account as members, and assign permission to shared workspaces or even (with good justification why it is needed) to the project directory to that group. This allows users to set the ownership of their directories to the shared group, allowing the functional account to copy data directly from the HPC filesystems to its AHOME.