PyTorch + CUDA

This is the unopinionated option: the official PyTorch image, CUDA already matched to the build, a root shell over SSH, and nothing else in the way.

Four upstream tags are in the template list. They are the official pytorch/pytorch images, unmodified apart from the platform’s SSH layer.

Pick the newest unless something pins you older — a dependency built against an older CUDA, or a repository that hasn’t caught up with a PyTorch major.

Two things to know before you start:

• There is no web UI. These templates run nothing on port 80, so https://<your-slug>.instance.grafilab.ai will not answer. You work over SSH, and anything you want in a browser you start yourself and expose with Web Access.
• These are runtime images, not devel. They carry the CUDA runtime libraries but not the toolkit, so there is no nvcc. Packages that compile CUDA kernels at install time will fail — see Troubleshooting.

Pick this template for scripts: a training loop you launch and leave running, an evaluation sweep, a repository with its own CLI. You get a clean environment and full control of it.

If you would rather work cell by cell in a browser, JupyterLab is the same idea with a notebook server on top. If you want to edit files in a browser IDE, use code-server.

Deploy an instance and pick one of the PyTorch cards in step 2, Template Selection. Match the card’s VRAM to the model you plan to train or serve. The full walkthrough is in Deploy a GPU Instance.

1. In the console open My Instance (https://app.grafilab.ai/orders) and find the card for your instance. Wait until its status reads Active.

2. Click Connect. The Connect with SSH dialog shows an SSH Connect command and a Password.

3. Copy the command into a terminal, press Enter, and paste the password when prompted. You land in a root shell inside the container.

First check that the driver sees the card:

nvidia-smi

Then check that PyTorch does:

python -c "import torch; print(torch.__version__, torch.cuda.is_available(), torch.cuda.get_device_name(0))"

You want True and the name of the card you rented. If you get False, see Troubleshooting.

git clone https://github.com/<owner>/<repo>.git
cd <repo>
pip install -r requirements.txt

scp -P <port> ./dataset.zip <your-slug>@<ssh-proxy-host>:/root/
rsync -avP -e "ssh -p <port>" ./project/ <your-slug>@<ssh-proxy-host>:/root/project/

wget https://example.com/dataset.tar.gz
huggingface-cli download <repo-id> --local-dir ./data

Run anything longer than a coffee break inside tmux, so the job survives a dropped SSH connection:

tmux new -s train
python train.py

Press Ctrl-b then d to detach, and tmux attach -t train to come back. nohup python train.py > train.log 2>&1 & works too if you only need the output in a file.

There is no service on port 80 to start with, so anything you want to reach from a browser you start yourself and map through the proxy. Bind to all interfaces, not localhost:

pip install jupyterlab
jupyter lab --ip 0.0.0.0 --port 8888 --allow-root

Then open Web Access on the instance card, add a mapping with a Service Name and Port 8888, and use the proxy URL it generates. The same applies to a Gradio app or any other server you run. The procedure is in Manage Instances.

Checkpoints are the thing to protect. Write them somewhere you chose deliberately, push them off the box as the run produces them — to a git remote, object storage, or a Hugging Face repository — and use Backup on the instance card before Expire at for whatever is still only on the instance.

• torch.cuda.is_available() returns False on a GPU node. The node’s NVIDIA driver is older than the CUDA version your image was built against. Relaunch on a template with an older CUDA tag — 2.5.1-cuda12.4 or 2.2.0-cuda12.1 — or pick a different node.
• nvcc: command not found, or a package fails to build CUDA extensions. These are runtime images without the CUDA toolkit. Install a prebuilt wheel for the package if one exists, or create a custom template from the matching -devel image tag — see Custom templates.
• Connection refused right after launch. The container is still starting. Wait until the instance card reads Active.
• A DataLoader crashes with a shared-memory error. The container gets 2 GB of /dev/shm. Lower num_workers, or set torch.multiprocessing.set_sharing_strategy("file_system").