Job Arrays

The job array is very helpful option of sbatch for users that need to run a lot of single-core computations. With this option your can start hundreds of jobs from one sbatch script that will start from the specified range and in the slurm queue all those jobs will have unique ID number and the pending jobs will be grouped to one.

Below example of sbatch job script with array option that will start 399 jobs that will execute file with the variable number from 1-399:

#!/bin/bash

#SBATCH --job-name=CodemlArrayJob
#SBATCH --partition=hive1d,hive7d,hiveunlim
#SBATCH --array=1-399
#SBATCH --output=out_%A_%a_%j.out
#SBATCH --error=error_%A_%a_%j.err

## To make things simple: ${i} == $SLURM_ARRAY_TASK_ID

i=${SLURM_ARRAY_TASK_ID}

# Check $WORKDIR existence

function checkWorkdir() {

    if [ ! -d "$1" ]; then 
        echo "ERROR! Directory $1 doesn't exist. "
        exit 1;
    fi
}

#load the modules environment

. /etc/profile.d/modules.sh
module load PAML/4.8

#Job commands

echo "branch${i}";
WORKDIR="/data/home/privman/rwilly/2.Like.Clade12/guidence/guidence.results/BranchSiteModelA/BranchSiteWithScript/like.Clade12.branch${i}.workingDir"

checkWorkdir $WORKDIR
cd $WORKDIR

codeml LikeClade12.branch${i}.ctl

Note: At --error and --output section you have %A,%a and %j, the %A will print your master array job id number, the %a will print you the array number of the job, and the %j will print you the unique id number of each job that you submitted in array mod. The information of the job id's could be useful when the job array is checkpoint-able, the job id number(array and original) will help you find which jobs has been checkpointed.