Tensorflow

TensorFlow is an open-source machine learning framework mainly developed by Google. It can be used for various machine learning tasks, e.g. deep learning. TensorFlow provides a high level API that can be used in Python and other languages and it can run on CPUs as well as GPUs.

Installing TensorFlow

It is recommended to use miniforge to create a Python virtual environment and install the desired version of TensorFlow within that environment.

module load miniforge3
conda create -n myenv python=3.12
source activate myenv
python3 -m pip install 'tensorflow[and-cuda]==2.20'

If you do not want to use GPUs, please replace the last line with python3 -m pip install tensorflow-cpu

We successfully tested the installation using version 2.20 of TensorFlow.

Testing the Installation

To run TensorFlow on GPUs, load the correct modules and submit a job to the respective gpu partition.

#!/bin/bash
#SBATCH -p scc-gpu
#SBATCH -t 1
#SBATCH --gpus-per-node 1
 
module load miniforge3

source activate myenv
 
python tftest.py

import tensorflow as tf
print("TensorFlow version:", tf.__version__)

for x in tf.config.list_physical_devices('GPU'):
    print(tf.config.experimental.get_device_details(x))

And then submit the job using Slurm:

sbatch jobscript.sh

The output file (slurm-{jobid}.out ) should contain:

TensorFlow version: 2.20.0
{'compute_capability': (7, 0), 'device_name': 'Tesla V100S-PCIE-32GB'}

and also information about the GPUs selected.

Testing CPU-only Installation

If you want to test a CPU only installation, you can run tftest.py on a login node. Please note that running any more extensive processing on a login node is not allowed, so use this for short functional testing only. For more in-depth tests or productive use, see Interactive Jobs.

Using Tensorflow

You can now use TensorFlow in your Python scripts. Please read GPU selection for more information about GPU usage.