Page History

Differences in LSF Configuration between the Juno

...

cluster and its predecessor Luna which has been retired

1. We reserve ~12GB of RAM per host for the operating system and GPFS on Juno CentOS 7 hosts.
2. On each jx## node (CentOS 7 , with GPFS installed), 240GB of RAM is available for LSF jobs.
3. When specifying RAM for LSF jobs, specify GB of RAM per task (slot) on Juno (, unlike luna , where RAM is specified per job).
4. All jobs must have -W (maximum execution Walltime) specified on Juno. Please do not use -We on Juno.
5. There is no /swap on CentOS 7 nodes. Memory usage is enforced by cgroups so jobs never swap. A job will be terminated if memory usage exceeds its LSF specification.
6. All Juno compute hosts have access to internetthe Internet. 
7. To check jobs which are DONE or have status EXIT, use "bhist -l JobID" or "bhist -n 0 -l JobID". bacct is also available. "bjobs -l JobID" only shows RUNNING and PEND jobs.
8. The maximum hold time for bjobs command can show jobs with DONE/EXIT jobs information is status for 24 hours in bjobs command.
   Example: bsub -w "post_done('JOB_A') -J "JOB_B"  ... if JOB_A DONE  was DONE 72 hours earlier then before JOB_B was submitted, JOB_B won't will never start.
9. There is no iounits resource on Juno.
10. CMOPI SLA configured on Juno. The loan policies are: 100% resources for 90 mints jobs for not SLA users, and 50% resources for 360 mints jobs for not SLA users.
11. The rest SLAs. The loan policy is: 100% resources for 360 mints jobs for not SLA usersJuno has an CMOPI and DEVEL SLAs. When CMOPI/DEVEL jobs are not filling those nodes, 100% of slots are available to non-CMOPI jobs with a duration under 90 minutes, 75% of slots are available to jobs under 6 hours and 50% of slots are available to jobs under 31 days.
12. Nodes assigned to other SLAs are available to non-SLA users for jobs up to 360 minutes.