Spin-up a notebook server on GPUs

Enable a real-time session of interacitve run on GPUs.

Interactive run

Interactive runs are designed to using Jupyter or SSH for live interaction with your data, code, and GPUs. Interactive runs are useful for tasks such as data exploration, model debugging, and algorithm development. They also allow you to expose additional ports on the container and communicate through those ports.

A Simple Interactive Run

Here is an example of a simple interactive run. It specifies resources, a container image, and the duration of the interactive runtime. It’s important to note that by default, port 22/tcp is exposed for SSH and 8888/http is exposed for JupyterLab.

Simple interactive run definition

name: gpu-interactive-run
description: Run an interactive GPU-backed Jupyter and SSH server.
resources:
  cluster: vessl-gcp-oregon
  preset: gpu-l4-small
image: quay.io/vessl-ai/torch:2.3.1-cuda12.1-r5
interactive:
  max_runtime: 24h
  jupyter:
    idle_timeout: 120m

Port

You can also specify additional ports to be exposed during an interactive run.

Simple interactive run definition

name: gpu-interactive-run
description: Run an interactive GPU-backed Jupyter and SSH server.
resources:
  cluster: vessl-gcp-oregon
  preset: gpu-l4-small
image: quay.io/vessl-ai/torch:2.3.1-cuda12.1-r5
interactive:
  max_runtime: 24h
  jupyter:
    idle_timeout: 120m
ports:
  - 8501

In this example, in addition to the default ports, 8501 will be exposed. Note that the ports field takes a list, so you can specify multiple ports if necessary. Also if you want to specify a TCP port, you can append /tcp to the port number; otherwise, /http is used implicitly.

Run a stable diffusion demo with GPU resources

Now move onto running a stable diffusion demo. The following configuration outlines an interactive run set up to execute a Stable Diffusion demo utilizing a V100 GPU and exposing the interactive Streamlit demo on port 8501.

Interactive run YAML for a Stable Diffsuion inference demo

name: Stable Diffusion Web
description: Run an inference web app of stable diffusion demo.
image: nvcr.io/nvidia/pytorch:22.10-py3
resources:
  cluster: vessl-gcp-oregon
  preset: gpu-l4-small
run:
  - command: |
      exit
      bash <(wget -qO- https://raw.githubusercontent.com/AUTOMATIC1111/stable-diffusion-webui/master/webui.sh)
      bash webui.sh
interactive: 
  max_runtime: 24h
  jupyter:
    idle_timeout: 120n
ports:
  -8501

In this interactive run, the Docker image image=nvcr.io/nvidia/pytorch:22.10-py3 is utilized, and a V100 GPU (resources.preset=v1.v100-1.mem-52) is allocated for the run. The interactive run is designed to run for 24 hours (interactive.max_runtime: 24h) and the Streamlit demo will be accessible via port 8501.

The run commands first execute a bash script from a remote location (bash <(wget -qO- https://raw.githubusercontent.com/AUTOMATIC1111/stable-diffusion-webui/master/webui.sh)), followed by the execution of webui.sh.

This configuration provides an example of setting up an interactive run for executing a GPU-accelerated demo with real-time user interaction facilitated via a specified port.

What’s Next

For more advanced configurations and examples. please visit VESSL Hub.

VESSL Hub

A variatey of YAML examples that you can use as references