Differences in LSF Configuration between Juno and luna
- We reserve ~12GB of RAM per host for the operating system and GPFS on Juno hosts.
- On each jx## node (CentOS 7 with GPFS), 240GB of RAM is available for LSF jobs.
- When specifying RAM for LSF jobs, specify GB of RAM per task (slot) on Juno, unlike luna where RAM is specified per job.
- All jobs must have -W (maximum execution Walltime) specified on Juno. Please do not use -We on Juno.
- There is no /swap on CentOS 7 nodes. Memory usage is enforced by cgroups so jobs never swap. A job will be terminated if memory usage exceeds its LSF specification.
- All Juno compute hosts have access to the Internet.
- To check jobs which are DONE or have status EXIT, use "bhist -l JobID" or "bhist -n 0 -l JobID". bacct is also available. "bjobs -l JobID" only shows RUNNING and PEND jobs.
- The bjobs command can show jobs with DONE/EXIT status for 24 hours.
Example: bsub -w "post_done('JOB_A') -J "JOB_B" ... if JOB_A was DONE 72 hours before JOB_B was submitted, JOB_B will never start. - There is no iounits resource on Juno.
- Juno has an CMOPI SLA. The loan policies is: 100% resources for 90 minute jobs for non-SLA users, and 50% resources available for 360 minute jobs for non-SLA users.
- The rest is determined by SLAs. The loan policy is 100% resources for 360 minute jobs for non-SLA users.