Storage Systems
Each cluster provides several different storage systems that can be placed into the following rough categories:
| Category | Speed | Size | Durability | Size Project¹ | Size User¹ | Purpose |
|---|---|---|---|---|---|---|
| HOME | medium | medium | snapshots+backups | ~ 40 GiB | User HOMEs | |
| Project | medium | medium | snapshots and/or backups | ~ 3 TiB | Medium term data | |
| SCRATCH/WORK | fast | big | fragile | ~ 2 / 12 TiB² | ~ 1 / 2 TB² | Active data |
| COLD | medium | big | snapshots | ~ 10 TiB | Medium term data | |
| ARCHIVE/PERM | very slow | medium | backups | ~ 8 TiB | Archival to tape |
These are shared parallel filesystems accessible from many nodes, rather than node local storage (which the nodes also have). Many of these filesystems also have quotas on the use of space and/or inodes (every file and directory has a “header” using one or more inodes).
It is important to use the right storage system for the right purpose to get the best performance, have the fewest surprises, and cause the least impact on other users. For example, a job that has to do many IO operations on temporary data could perform terribly if those temporary files (e.g. opening a file, writing a small amount of data, closing the file, deleting the file repeatedly millions of times) were stored on HOME or in a SCRATCH/WORK filesystem, and could easily slow down the filesystem for everyone on the cluster.
See the pages below (also can be found in the nav bar to the left) for information about the different parts of the storage systems and how to use them.
[1]: These numbers are generalized figures and only show the soft limits.
[2]: These are the numbers for the SSD scratch and the HDD scratch, you can find more information here.