Accounting Core-h

The NHR centers follow the NHR-wide regulations and account the resource usage in the unit core hours. Using the batch system will result in the consumption of the core hour balance. Every node in each of the partitions has a specific charge rate. The current rates can be found in the table for the CPU partitions and also for the GPU partitions.

Using one node consumes the resources of the entire node and is accounted as a full node. A shared node can be used by multiple users and therefore, only the allocated number of cores are accounted. Using a GPU node only takes the GPU into account, meaning allocating and using a GPU node will only account the core hours for the GPU not the CPU.

Usage of the storage system is not accounted.

Job Charge

The charge for a batch job on the NHR systems is the number of core hours and is calculated from the number of nodes reserved for the job, the wallclock time used by the job, and the charge rate for the job nodes. For a batch job with

• num nodes,
• running with a wallclock time of t hours, and
• on a partition with a charge rate charge_p

the job charge charge_j yields

charge_j = num * t * charge_p

Batch jobs running in the partition large96:shared access a subset of cores on a node. For a reservation of cores, the number of nodes is the appropriate node fraction.

Checking the balance

We provide a script called sbalance. It prints your current balance and provides some additional useful information. Also, It differentiates between personal and project accounts.

usage: sbalance [-h] [-s] [-l] [-n] [--me] [--assoc] [-u USER] [-a ACCOUNT]
                [--ascii] [--no-color]

Shows the remaining core hours for a user-account association from the
SlurmDB.

optional arguments:
  -h, --help            show this help message and exit
  -s, --short           Only print the remaining core hours, nothing else.
  -l, --limit           Print the current limit instead of remaining core
                        hours. (Requires the -s/--short flag)
  -n, --no-help         Don't print the info texts.
  --me                  Prints the remaining core hours for the current user
                        (same as running "sbalance" without any flag).
  --assoc               Print the remaining core hours for the user-account
                        associations, instead of for the accounts.
  -u USER, --user USER  Specify the username to query.
  -a ACCOUNT, --account ACCOUNT
                        Specify the account to query.
  --ascii               Force use of only ASCII characters (no unicode).
  --no-color            Disable color even if stdout would support it.

The output can look like this:

sbalance output variations

sbalance with color and unicode sbalance with ascii output

Personal contingent for account           :
  Used 0.00% (0 core hours / 71.56 kilo-core hours)
    [----------------------------------------------------------------------------]
  Your personal contingent will be renewed on the 01.07.2024
  You can also apply for a project to gain more core hours. Use the project-
  portal (https://hpcproject.gwdg.de) to do so.

Project accounts: 
  You are currently in no projects
  If you want to be added to a project, please ask your supervisor to add you to
  one in the project portal (https://hpcproject.gwdg.de). You can also use the
  project portal to submit a project application yourself.

Account Types

Accounting is done for two different types of accounts. The NHR centers distinguish between personal accounts and project accounts. Personal accounts always have some small recurring amount of core hours while a project account hold much more core hours and can be distributed between many users.

Personal Account

At the beginning of each quarter, each account is granted 75.000 core hours. In reasonable and exceptional cases, the grant of the account can be extended to 300.000 core hours per quarter. This can be particularly useful if you need more time for estimating core hours consumption for a project proposal. In order to increase your core hours, please contact the support. At the end of each quarter all remaining core hours in the bank account are reset.

Project Account

A compute project holds a bank account for the project. This project account contains a compute capacity in core hours. At the beginning of each quarter the account is granted by the number of core hours following the funding decision for the given compute project. A project account holds at least 4x300.000 core hours per year. Unused core hours are transferred to the subsequent quarter, but only one time.

In case of problems with your compute capacity in core hours in your project account please contact the support. This might affect the

• application for additional core hours,
• movement of core hours between quarters.

Select the Account in Your Batch Job

Batch jobs are submitted by a user account to the compute system. For each job the user chooses the account (personal or project) that will be charged by the job.

• At the beginning of the lifetime of the user account the default account is the personal account.
• The user controls the account for a job using the Slurm option --account at submit time.

After job script submission the batch system checks the account for account coverage and authorizes the job for scheduling. Otherwise the job rejected, please notice the error message:

> squeue
... myaccount ... AccountOutOfNPL ...

Terminology

The unit used is the number of hours and cores used. It can be expressed as:

• core hours
• coreh
• kilo core hours = 1000 core hours

Using the SI abbreviation of kilo core hours is useful as a short hand and used in some print outs