Keck Center Wiki

User Tools

Site Tools

Trace:
wiki:sge

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
wiki:sge [2015/08/07 12:47] – adminwiki:sge [2017/05/01 09:26] (current) – [Orca MPI] admin
Line 1: Line 1:
-====== Using SGE to run jobs at KeckII ======+====== Using SGE to run jobs at Keck Center ======
  
  
  
-<fc #FF0000>All non-interactive jobs on Keck2 workstations must be run through the job queue manager, SGE.</fc> User accounts not complying with this policy will be suspended. Please see this [[http://ctbp.ucsd.edu/computing/wiki/introduction_to_sge_the_queuing_system_on_the_clusters|SGE How-To]] for more details on how to submit a job and also some examples for most common scenarios. +<fc #FF0000>All non-interactive jobs on Keck Center workstations must be run through the job queue manager, SGE.</fc> User accounts not complying with this policy will be suspended. Please see this [[http://ctbp.ucsd.edu/computing/wiki/introduction_to_sge_the_queuing_system_on_the_clusters|SGE How-To]] for more details on how to submit a job and also some examples for most common scenarios. 
  
 All jobs must be submitted from ''w01.keck2.ucsd.edu'' using the ''qsub'' command: All jobs must be submitted from ''w01.keck2.ucsd.edu'' using the ''qsub'' command:
Line 21: Line 21:
   * Check on progress of your job in the queue: ''qstat -f ''   * Check on progress of your job in the queue: ''qstat -f ''
  
-===== KeckII scheduling policies =====+===== Keck Center scheduling policies =====
  
 All jobs must be submitted to the SGE queue. It is strictly prohibited to run any non-interactive CPU-consuming jobs outside of the queue. All jobs must be submitted to the SGE queue. It is strictly prohibited to run any non-interactive CPU-consuming jobs outside of the queue.
Line 29: Line 29:
 The following limits are imposed on all jobs: The following limits are imposed on all jobs:
  
-  * max wall-clock time is 48 hrs +  * max wall-clock time is 48 hrs (subject to change, use ''qconf -sq main.q| grep h_rt'' to see the current limit) 
-  * max number of processors per user is 8 although this is  dynamically changed based on cluster load. To see the current limit: ''qconf -srqs''+  * max number of processors per user is 8 although this is  dynamically changed based on the cluster load. To see the current limit: ''qconf -srqs''
  
  
Line 49: Line 49:
  
 Please note that old files (4 days and older) are regularly purged from ''/scratch''. Please note that old files (4 days and older) are regularly purged from ''/scratch''.
 +
 ===== Setting up account ===== ===== Setting up account =====
  
Line 61: Line 62:
 chmod 640 authorized_keys chmod 640 authorized_keys
 </code> </code>
- 
- 
- 
  
 ===== Running CPU intensive jobs ===== ===== Running CPU intensive jobs =====
Line 114: Line 112:
 ===== Running parallel (MPI) jobs ===== ===== Running parallel (MPI) jobs =====
  
-If your application supports this you can run up to 8 parallel processes per one job. The workstations have 8 physical cores so maximum requestable number of processors is 8. Do not over-subscribe the workstations.+If your application supports this you can run up to 8 parallel processes per one job. The workstations have 8 physical cores so maximum requestable number of processors is 8. <fc #FF0000>Do not over-subscribe the workstations.</fc>
  
 You have to use the ''mpi'' SGE queue environment with the following statement in your SGE submit script: You have to use the ''mpi'' SGE queue environment with the following statement in your SGE submit script:
Line 143: Line 141:
 cd $scratch_dir cd $scratch_dir
  
-module load orca +module load orca/3.0.3 
-module load openmpi +module load openmpi/1.6.2 
-/soft/pkg/orca-3.0.0/orca orca_input.inp > orca_output.out +$ORCA_PATH/orca orca_input.inp > orca_output.out 
  
 # copy all data back from the scratch directory # copy all data back from the scratch directory
Line 158: Line 156:
 </code> </code>
  
 +Please note that you have to load the appropriate MPI library to use Orca. This is a compatability
 +table between different Orca nad MPI module versions:
 +
 +|orca/4.0.0 | openmpi/2.0.1 |
 +|orca/3.0.3 | openmpi/1.6.2 |
  
  
Line 180: Line 183:
 #$ -pe mpi 2 #$ -pe mpi 2
  
-module load openmpi +module load openmpi/2.0.1 
-module load amber+module load amber/16
  
 echo Running on host `hostname` echo Running on host `hostname`
Line 367: Line 370:
 ==== Amber ==== ==== Amber ====
  
-The optimal AMBER job configuration for KeckII is to use 1 CPU and 1 GPU per run.+The optimal AMBER job configuration for Keck II is to use 1 CPU and 1 GPU per run.
  
 <code> <code>
Line 383: Line 386:
 #$ -l h_rt=12:00:00 #$ -l h_rt=12:00:00
  
-module load nvidia +module load cuda/7.5.18 
-module load amber+module load amber/16 
 export CUDA_VISIBLE_DEVICES=0 export CUDA_VISIBLE_DEVICES=0
  
Line 464: Line 468:
 #$ -l h_rt=48:00:00 #$ -l h_rt=48:00:00
  
-module load nvidia +module load namd-cuda/2.11
-module load namd-cuda+
 export CUDA_VISIBLE_DEVICES=0 export CUDA_VISIBLE_DEVICES=0
  
Line 487: Line 490:
 ==== Benchmarks ==== ==== Benchmarks ====
  
-These are several GPU benchmarks for CUDA enabled Amber and NAMD which should help you to estimate the Keck2 hardware performance.+These are several GPU benchmarks for CUDA enabled Amber and NAMD which should help you to estimate the Keck Center hardware performance.
  
  
wiki/sge.1438976842.txt.gz · Last modified: 2015/08/07 12:47 by admin