wiki:slurm
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
wiki:slurm [2021/08/12 09:14] – [Orca MPI] admin | wiki:slurm [2024/03/29 10:07] (current) – [SLURM useful commands] admin | ||
---|---|---|---|
Line 65: | Line 65: | ||
^ Command ^ Example syntax ^ Meaning ^ | ^ Command ^ Example syntax ^ Meaning ^ | ||
|sbatch|sbatch < | |sbatch|sbatch < | ||
- | |srun|srun --pty -t 0-0:5:0 -p cpu / | + | |srun|%%srun --pty -t 0-0:5:0 -p cpu / |
|squeue|squeue -u < | |squeue|squeue -u < | ||
|scontrol|scontrol show job < | |scontrol|scontrol show job < | ||
Line 77: | Line 77: | ||
- | * request | + | * Request |
+ | |||
+ | * Request a node with 6GB of RAM per core (CPU): '' | ||
+ | |||
+ | |||
+ | * Most of the Keck nodes have 24 GB of RAM (23936 B) but there are two nodes which have 32 GB (31977 B) of RAM (nodes w16 and w17). If your job needs more than 20GB of RAM (but less that 32GB) you can request one of the " | ||
+ | |||
+ | #SBATCH --mem=30G | ||
+ | #SBATCH --nodelist=w16 (or w17) # request the job to be sent to w16 or w17, pick a node which has no jobs running | ||
- | * request a node with 6GB of RAM per core (CPU): '' | ||
* canceling jobs: | * canceling jobs: | ||
Line 85: | Line 92: | ||
|scancel 1234 | cancel job 1234| | |scancel 1234 | cancel job 1234| | ||
|scancel -u myusername | |scancel -u myusername | ||
- | |scancel -u myusername --state=running | + | |%%scancel -u myusername --state=running%% | cancel all my running jobs| |
- | |scancel -u myusername --state=pending | + | |%%scancel -u myusername --state=pending%% | cancel all my pending jobs| |
Line 99: | Line 106: | ||
|scontrol show job < | |scontrol show job < | ||
- | |sstat -j < | + | |%%sstat -j < |
- | |sacct -j < | + | |%%sacct -j < |
Line 118: | Line 125: | ||
#!/bin/bash | #!/bin/bash | ||
#SBATCH -n 8 # Request 8 cores | #SBATCH -n 8 # Request 8 cores | ||
- | #SBATCH -t 0-00:30 # Runtime in D-HH:MM format | + | #SBATCH -t 0-01:30 # Runtime in D-HH:MM format |
#SBATCH -p cpu # Partition to run in | #SBATCH -p cpu # Partition to run in | ||
#SBATCH --mem=20G | #SBATCH --mem=20G | ||
Line 129: | Line 136: | ||
echo Directory is $(pwd) | echo Directory is $(pwd) | ||
echo "This job has allocated $SLURM_NPROCS processors in $SLURM_JOB_PARTITION partition " | echo "This job has allocated $SLURM_NPROCS processors in $SLURM_JOB_PARTITION partition " | ||
- | cwd=`pwd` | + | cwd=$(pwd) |
# create a randomly named scratch directory and copy your files there | # create a randomly named scratch directory and copy your files there | ||
- | export SCRATCH=`mktemp -d / | + | export SCRATCH=$(mktemp -d / |
echo "Using SCRATCH: ${SCRATCH}" | echo "Using SCRATCH: ${SCRATCH}" | ||
- | export | + | export |
# copy job files to $SCRATCH | # copy job files to $SCRATCH | ||
- | cp * $SCRATCH | + | cp -a * ${SCRATCH} |
- | cd $SCRATCH | + | cd ${SCRATCH} |
module load gaussian/ | module load gaussian/ | ||
Line 144: | Line 151: | ||
# copy the results back to $HOME & cleanup | # copy the results back to $HOME & cleanup | ||
- | cp * $cwd | + | cp -a * ${cwd} |
- | #rm -rf $SCRATCH | + | #rm -rf ${SCRATCH} |
</ | </ | ||
Line 151: | Line 158: | ||
| | ||
- | + | ||
You can verify that the jobs is in the queue: | You can verify that the jobs is in the queue: | ||
Line 174: | Line 181: | ||
#SBATCH -e %j.err | #SBATCH -e %j.err | ||
set -xv | set -xv | ||
+ | |||
+ | echo Running on host $(hostname) | ||
+ | echo "Job id: ${SLURM_JOB_ID}" | ||
+ | echo Time is $(date) | ||
+ | echo Directory is $(pwd) | ||
+ | echo "This job has allocated $SLURM_NPROCS processors in $SLURM_JOB_PARTITION partition " | ||
# create a scratch directory on the SDD and copy all runtime data there | # create a scratch directory on the SDD and copy all runtime data there | ||
Line 179: | Line 192: | ||
echo "Using SCRATCH directory: ${scratch_dir}" | echo "Using SCRATCH directory: ${scratch_dir}" | ||
current_dir=`pwd` | current_dir=`pwd` | ||
- | cp * $scratch_dir | + | cp -a * $scratch_dir |
cd $scratch_dir | cd $scratch_dir | ||
- | module load openmpi/ | + | module load orca/5.0.3 |
- | module load orca/4.2.1 | + | |
$ORCA_PATH/ | $ORCA_PATH/ | ||
# copy all data back from the scratch directory | # copy all data back from the scratch directory | ||
- | cp * $current_dir | + | cp -a * $current_dir |
rm -rf $scratch_dir | rm -rf $scratch_dir | ||
</ | </ | ||
Line 198: | Line 210: | ||
</ | </ | ||
- | Please note that you have to load the appropriate MPI library to use Orca. This is a compatibility | + | Please note that with older versions of Orca you have to load the appropriate MPI library to use it. This is a compatibility table between different Orca and MPI module versions: |
- | table between different Orca and MPI module versions: | + | |
|orca/4.0.0 | openmpi/ | |orca/4.0.0 | openmpi/ | ||
Line 205: | Line 216: | ||
|orca/4.2.0 | openmpi/ | |orca/4.2.0 | openmpi/ | ||
|orca/4.2.1 | openmpi/ | |orca/4.2.1 | openmpi/ | ||
- | |orca/5.0.1 | openmpi/ | + | |orca/5.0.3 | no MPI loading necessary, it is built in | |
==== OpenMolcas ==== | ==== OpenMolcas ==== | ||
Line 251: | Line 262: | ||
# copy output magnetic properties file for further analysis (poly_aniso) | # copy output magnetic properties file for further analysis (poly_aniso) | ||
- | cp / | + | cp -a / |
</ | </ | ||
Line 290: | Line 301: | ||
echo SCRATCH: $SCRATCH | echo SCRATCH: $SCRATCH | ||
# copy job files to $SCRATCH | # copy job files to $SCRATCH | ||
- | cp * $SCRATCH | + | cp -a * $SCRATCH |
# start your job in $SCRATCH | # start your job in $SCRATCH | ||
Line 297: | Line 308: | ||
# copy your results back to $HOME & cleanup | # copy your results back to $HOME & cleanup | ||
- | cp * $cwd | + | cp -a * $cwd |
#rm -rf $SCRATCH | #rm -rf $SCRATCH | ||
</ | </ |
wiki/slurm.1628784850.txt.gz · Last modified: 2021/08/12 09:14 by admin