Differences in LSF Configuration between the Juno cluster and its predecessor Luna which has been retired
- We reserve ~12GB of RAM per host for the operating system and GPFS on Juno hosts.
- On each jx## node (CentOS 7 with GPFS), 240GB of RAM is available for LSF jobs.
- When specifying RAM for LSF jobs, specify GB of RAM per task (slot) on Juno, unlike luna where RAM is specified per job.
- All jobs must have -W (maximum execution Walltime) specified on Juno. Please do not use -We on Juno.
- There is no /swap on CentOS 7 nodes. Memory usage is enforced by cgroups so jobs never swap. A job will be terminated if memory usage exceeds its LSF specification.
- All Juno compute hosts have access to the Internet.
- To check jobs which are DONE or have status EXIT, use "bhist -l JobID" or "bhist -n 0 -l JobID". bacct is also available. "bjobs -l JobID" only shows RUNNING and PEND jobs.
- The bjobs command can show jobs with DONE/EXIT status for 24 hours.
Example: bsub -w "post_done('JOB_A') -J "JOB_B" ... if JOB_A was DONE 72 hours before JOB_B was submitted, JOB_B will never start. - There is no iounits resource on Juno.
- Juno has an CMOPI and DEVEL SLAs. When CMOPI/DEVEL jobs are not filling those nodes, 100% of slots are available to non-CMOPI jobs with a duration under 90 minutes, 75% of slots are available to jobs under 6 hours and 50% of slots are available to jobs under 31 days.
- Nodes assigned to other SLAs are available to non-SLA users for jobs up to 360 minutes.