General Documentation

LSF Primer

Lilac Cluster Guide

Juno Cluster Guide

Cloud Resources

Backup Policy on server/node local drives

File lists

Skip to end of metadata
Go to start of metadata

This page documents the various steps to use libvirt tools and create Virtual Machines (VMs) on the "Flame" (login-node) on the Lilac cluster. This page mentions a few of the many ways to create VMs with libvirt tools, since many other options exist we encourage you to explore the best options for you.. This primer will also mention the restrictions and tips for VM creation on FLAME.

VMs are expected to be created and managed by respective users on userspace just like they do on there own systems, and therefore will not have any support from System Administrators.

*note:  "Flame" is the login-node dedicated to the computational pathology group (cpath).  Users not in this group do not have login access to this head-node.  For operating a virtual machine on lilac in any other way, please contact the HPC group. 

Prerequisites:

Any VM will need the following prerequisites.

  1. Network Parameters (must request an IP from the HPC Group)
  2. Disk image location on GPFS file system.
  3. VM images
    1. Already existing VM images on web/previously installed.
    2. Install from existing linux mirror - CentOS
      1. Use public mirror 
      2. Use Kickstart file to automate installation or Install interactively.
  4. Amount of RAM
  5. Number of CPUs
  6. Display mode ( only text mode supported )

Restrictions:

Any VM created on FLAME will have the following restrictions:

  1. VMs will not have access to private network.
  2. VMs will not have access to the same filesystem ( local mounts) as the cluster.
  3. VMs will not have access to the same credentials infrastructure (LDAP) as the cluster.
  4. VMs will only support text mode with x11 support just as rest of the cluster.
  5. VMs are limited to the hardware that FLAME supports, with the same number of CPUs and RAM as the host physical server.
  6. VMs are limited to use the disk quota allocated in GPFS.
  7. VMs do not support GPUs.

Step by Step Guide:

  1. ssh to FLAME with x11 forwarding.
        ssh -Y -A <username>@flame.mskcc.org
  2. Decide a VM name.
  3. Network Parameters: 
    1. Raise a ticket for a campus ip address/DNS name.
    2. Rest of the network parameters are as below:
      1. subnet mask: 255.255.255.128
      2. gateway: 140.163.188.1
      3. suggested VM interface: eth0
    3. Network bridge that needs to be used : brcampus
    4. Nameserver/DNS server : 140.163.20.252
    5. No dhcp on campus network.
  4. Disk Image location on GPFS Filesystem:
    1. Locate the disk image within a particular directory in /lila/data/<labname>/<vmdirname>
    2. Decide between raw format or qcow2 format.
    3. Raw format allocates all of the space initially, while qcow2 allocates space as needed. Many other differences exist and can be referred in online documentation.
    4. For raw format : 
      1. Run a command as below
        fallocate -l <disksize> /lila/data/<labname>/<vmname>.img
        for eg:
        fallocate -l 10G /lila/data/hpcadmin/valleru/vmimages/test01.img
  5. Decide on a network mirror. 
    1. You can use centos mirror
      http://mirror.centos.org/centos-7/7.3.1611/os/x86_64/
    2. Replace mirror.centos.org with its IP address, just to make sure the VM can reach the IP even if it cannot resolve the DNS name.
      http://69.39.81.153/centos-7/7.3.1611/os/x86_64/
  6. Manual Install from Network Mirror
    1. Check if any VMs are already present.
      virsh list --all
      virsh dumpxml <vmname>
    2. Decide on CPUs/RAM. Make sure you do not oversubscribe for decent performance.
        1. To install run the following command
          virt-install --connect qemu:///session --name <vmname> --ram <raminbytes> --disk path=<fullpathnameascreatedabove> --vcpu=<numberofcpus> --os-type linux --os-variant=rhel7.0 --network bridge=brcampus --location ‘http://69.39.81.153/centos-7/7.3.1611/os/x86_64/’ --extra-args “ip=<ipaddressgiven>::140.163.188.1:255.255.255.128:<hostname>:eth0:none nameserver=140.163.20.252"

        2. For eg: To install a 2 CPU 4G machine, with IP address 140.163.188.101
          virt-install --connect qemu:///session --name test --ram 4096 --disk path=/home/valleru/vmimages/test01.img --vcpu=2 --os-type linux --os-variant=rhel7.0 --network bridge=brcampus --location ‘http://69.39.81.153/centos-7/7.3.1611/os/x86_64/’ --extra-args “ip=140.163.188.101::140.163.188.1:255.255.255.128:test:eth0:none nameserver=140.163.20.252"
    3. Once the above command is run - You should see an x11 window startup with all the other options to install the VM.
    4. You can set the disk partition and root password and everything else during subsequent steps in the window.
  7. Once the VM reboots, login using the mentioned root password or any other users that were set during installation.
  8. You should also be able to login using the given campus ip address.