Use Cases & Requirements

Requirements

This section is meant to guide you on whether Secure HPC fits your current use case. In particular, Secure HPC is most useful in the following cases:

  • Large computations, possibly requiring GPUs: HPC nodes have a lot more computational resources than a classical workstation. Additionally, many of our compute nodes have Nvidia GPUs to support more parallel computation, such as training deep learning models. Secure HPC is a good fit if the computation cannot be done on a local workstation, or it would take too long.
  • Long computation jobs: The nodes themselves can provide a lot of computational resources once the job is running. But each secure computation (i.e. each job submission) has a lot of overhead: First, the submission has to re-encrypt the data container and submission script, requiring new KMS one-time tokens. Then, those have to be transferred into the data system. Afterwards, the job has to be scheduled. Since it is a sequential system, it has to possibly wait in a queue for previous jobs to complete. While all of this is automated, it still adds a somewhat constant overhead to your computation, no matter the computation length (excluding job scheduling rules). Thus, the longer the computation, the higher the performance measured in wall-clock time. If the computations can run without manual intervention in-between, it is highly recommended to join them into bigger batch jobs; either through simple shell scripts or, if required, more sophisticated workflow management tools such as Snakemake.
  • Non-interactive computation: Due to the overhead described in the previous bullet point, batch jobs in general
  • Scriptable tools: Since batch jobs are not interactive, Graphical User Interfaces (GUIs) cannot be used while the computation is running.
  • Jobs requiring highly secure environments: If it is (conceptually and legally) sufficient to store data unencrypted, using the classical HPC requires a more frictionless experience. Users can still protect their data by setting restrictive permissions, even without using Secure HPC. If you need support setting up your permission system, or with other related data management questions, feel free to contact us at hpc-support@gwdg.de.

Additionally, at the organizational level, each Secure HPC use case requires an associated HPC Project.

Use Cases that fit Secure HPC

Use cases that would fit Secure HPC well

  • Deep Learning on any type of medical data
  • Computer Vision on MRI/CT scans
  • …

Use Cases that do not fit Secure HPC

  • Use Cases that require a graphical user interface: Currently not supported for sensitive data; work in progress. For non-sensitive data, our JupyterHub can be used.
  • Highly Interactive Use Cases: For example, typical exploratory data science workflows.
  • Computations on public or non-sensitive data: Here, normal HPC with correctly set permissions would usually suffice.

Roadmap for new users

  • Check if your use case is compatible with SecureHPC (see above).
  • Contact us via hpc-support@gwdg.de to get in touch.
  • Setting up a test environment in collaboration with us, to ensure that there are no technical complications with your workflow and SecureHPC (using only non-sensitive test data).
  • Users have to provide/setup legal documents such as Auftragsverarbeitungsvertrag (AVV), Verzeichnis der Verarbeitungstätigkeiten (VVT), Betriebskonzept (BK) in order to comply with their GDPR (DSGVO) standards.
  • Users have to setup the secure client in their trusted environment and allow the client to connect to our HPC’s front end nodes.

Unclear?

In case it is still unclear, feel free to reach out to us directly at hpc-support@gwdg.de.

To get a feeling for how your use case could be ported to Secure HPC, see one of our official examples or our Github repo containing the client-side code.

To adapt your use case and prepare your client for Secure HPC (which can already be done while awaiting access), see the Getting Started Guide.