Page History

...

Information about submitting LSF jobs is at http://mskcchpc.org/display/CLUS/LSF+Primer.  Your interactive LSF job should request the resources your Jupyter notebook will need.

Once you are comfortable to submitting lsf jobs:

...

These are the steps for starting and connecting to your jupiter-notebook on the cluster:

1. Download miniconda and install Jupyter notebook with miniconda.

...

1. Start an interactive session with bsub and start

...

1. jupyter-notebook.

...

1. Pay attention to the message showing on the screen once you start

...

1. jupyter-notebook. It will tell you the actual node you are on

...

1. , the port number

...

1. and the URL.
2. Set up SSH tunnels to the node running jupiter-notebook.
3. Connect using the browser on your local machine.

Here is an example of a simple LSF interactive job submission:
username@lilac:~>bsub -n 2 -W 4:00 -R 'span[hosts=1] rusage[mem=8]' -Is /bin/bash

This example requests a GPU for NLP(natural language processing) on Lilac:
username@lilac:~> bsub -n 2 -W 2:00 -q gpuqueue -gpu "num=1" -R "span[hosts=1]" -Is /bin/bash

...

You can connect to jupyter-notebook by setting up a SSH tunnel. When you create a SSH tunnel, it associates a specific port on the remote system to one on the local system. SSH authentication forwarding must be configured for this to work. The basic command to create a SSH tunnel that passes through the login node to the compute node on your laptop or local server is:

"ssh -L<Port-A>:localhost:<Port-B> user@Host-B -t ssh -L<Port-B>:localhost:<Port-C> user@Host-C"

...