General Documentation

LSF Primer

Lilac Cluster Guide

Juno Cluster Guide

Cloud Resources

Backup Policy on server/node local drives

File lists

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

This page documents the various questions related to Juno storage. 

Juno home storage : 

    • Description : Isilon shared parallel filesystem.
    • Purpose: To store software related code and scripts, default quota size is small and fixed.
    • Mount: /home/<user>
    • Access: All Juno nodes, including compute storage and login nodes.
    • Default quota: 100GB
    • Snapshots: 7 days of snapshots. ( not backed up ). Can be accessed in /home/.snapshots/<user>
    • Replicated: yes

Juno compute storage : 

    • Description : GPFS shared parallel filesystem, not replicated, and not backed up.
    • Purpose: For jobs to read and write compute data from login and compute nodes, default quota size is larger with flexibility to request larger quota.
    • Mount: /juno/res/<lab group>, /juno/work/<lab group>.
    • Access: All Juno nodes, including compute storage and login nodes.
    • Default quota: 1TB ( Increased/Decreased on request )
    • Snapshots: 7 days of snapshots. ( not backed up ). Can be accessed in /data/.snapshots/<date>/<lab group>
    • Replicated: no

Juno warm storage : 

    • Description : GPFS shared parallel filesystem, not replicated but can be backed up upon request. 
    • Purpose: To store long term data. Only accessible from login nodes and cannot be accessed from compute nodes.
    • Mount: /warm/<lab group>
    • Access: Only Juno  login nodes. 
    • Default quota: 1TB ( Increased/Decreased on request )
    • Snapshots: 7 days of snapshots. ( not backed up ). Can be accessed in /warm/.snapshots/<date>/<lab group>
    • Replicated: no 

Juno local scratch storage : 

  • Description : local SATA drive, not replicated, and not backed up. Local and not a shared filesystem
  • Purpose: To store local temporary data related to compute jobs. Since this is not a shared filesystem, the temporary data needs to be cleaned up and copied back to shared filesystem after job completion.  Files older than 31 days are automatically deleted.
  • Mount: /scratch/
  • Access: Local to the compute node.
  • Default quota: No quota and limited to free disk space in /scratch.
  • Snapshots: No snapshots.
  • Replicated: no

Juno local  fast scratch storage : 

  • Description : local NVMe drive, not replicated, and not backed up. Local and not a shared filesystem
  • Purpose: To store local temporary data related to compute jobs. Since this is not a shared filesystem, the temporary data needs to be cleaned up and copied back to shared filesystem after job completion.  Files older than 31 days are automatically deleted.
  • Mount: /fscratch/
  • Access: Local to the compute node. Not available on all nodes. Must be requested on bsub with -R fscratch
  • Default quota: No quota and limited to free disk space in /fscratch.
  • Snapshots: No snapshots.
  • Replicated: no

How to get information about your usage:

You can get real time information about storage quotas and usage from grafana at https://hpc-grafana.mskcc.org/d/000000027/storage-quota

You can also get information from the command line

For the home directory /home/$USERNAME, use:

              mmlsquota juno:home 

For the working directory /data/$LAB, use:

       mmlsquota -j work_hpcadmin  --block-size auto juno

or

df -h  /data/$LAB

lilac also has quotas on number of files -- not just data size.


 Copy files from other clusters:

Depending on the size and number of files to copy, you may run multiple rsync commands simultaneously to copy different directories.

The HPC private network is faster than the MSKCC campus network, so using short names such as lilac will often make transfers faster than the fully qualified domain name lilac.mskcc.org.