Skip to content

jupyter_vscode.md

Latest commit

 

History

History
131 lines (87 loc) · 4.31 KB

jupyter_vscode.md

File metadata and controls

131 lines (87 loc) · 4.31 KB

Running Jupyter Notebooks on HTCF

Adapted from this page.

1. Connect to HTCF using SSH

  • Click on image in the bottom left-hand corner
  • Choose the configured SSH host (see this page for details) or select Add New SSH Host and type ssh login.htcf.wustl.edu at the prompt
  • Enter your password and press ENTER

2. Tunnel to VSCode

(Tunneling Option 1)

If your ~/.bashrc was configured as suggested, type one of the following in the terminal and press ENTER to start an interactive session:

sml # 5G of memory
med # 20G of memory
lrg # 100G of memory

Start a tunnel:

code tunnel

image

  • Use the arrow keys to select Microsoft Account and press ENTER

image

  • Copy the code
  • Press CMD and click on the link to open the page in a browser

image

  • Paste the code and click next

  • Enter your wustl key username and password

  • Leaving the previous VSCode window open, open a new VSCode window. The shortcut is CMD + Shift + N.

  • Click the icon for the Remote Explorer: image

You should see the tunnel to the node you just created.

image

  • Click on the arrow to open the tunnel in the current window.

(Tunneling Option 2 - Recommended)

Persisting an Interactive Node

In the SSH:htcf window, on the login node, enter the following:

sbatch /scratch/dblab/opool/code/job/tmux/persist_interactive.sh

cd ~

cat persist_interactive.log

Follow the prompts.

Open a new VSCode window. CMD + Shift + N

Shift + CMD + P and type and select Remote Tunnels: Connect to Tunnel. Select the logon method used. Wait for the connection to load. Select n002 to connect.

image

Note: Now even if your session is disconnected, you will be able to reconnect. This job runs 8 hours.

3. Jupyter Notebook setup

In the new window, activate a spack environment containing the software and python packages needed to run a Jupyter notebook.

spack env activate -p jupyter
  • Open a pre-existing Jupyter notebook file.
  • Click the kernel selector icon: image
  • Paste in the path:
/ref/dblab/software/spack-0.21.0/var/spack/environments/jupyter/.spack-env/view/bin/python
  • Refresh under Jupyter Kernels if needed
image

On mac, use CMD + backtick to cycle between VSCode windows.

Creating a kernel (only required when creating new environments):

After activating your new environment and installing all required software, enter the commands below:

python3 -m ensurepip
python3 -m pip install ipykernel
python3 -m ipykernel install --user --name=your_env_name --display-name your_env_display_name

Start a tunnel.

Navigate to where the kernel spec was installed, which is typically in the following location:

cd $HOME/.local/share/jupyter/kernels

Enter the directory with the name matching the kernel that was installed. Assuming you were the user that created this kernel using ipykernel, you should see a file, kernel.json. Print this file to the terminal:

cat kernel.json

Grab the path listed in this file.

Continue with the instructions in step 3 above, pasting in this path instead.

Troubleshooting

Kernel is crashing

  • Try allocating more memory