Open WebUI + Ollama

Launch this template and you get a private ChatGPT-style chat window in your browser, backed by models that run entirely on the GPU you just rented.

The image is grafilab/openwebui:0.10.2. It bundles two things:

• Open WebUI on port 80, published at https://<your-slug>.instance.grafilab.ai.
• Ollama, started in the background inside the same container and listening on 127.0.0.1:11434.

Ollama is deliberately bound to localhost and is not published through the proxy. It has no authentication of its own, so anyone who could reach it could load and run arbitrary models on your GPU. Talk to it through Open WebUI, or over SSH — never map it with Web Access.

No models are baked into the image. You pull the ones you want after launch, and they live in the container for the duration of the rental.

Pick this template when the point is that the model runs on hardware you control: a private assistant, a model you want to keep off third-party APIs, or a specific open-weights model you want to try at full speed.

If what you actually want is the Open WebUI interface in front of Grafilab’s hosted models — no GPU rental, no model downloads, pay per token — you don’t need an instance at all. Run Open WebUI wherever you like and point it at the Models API instead: see Open WebUI integration.

Deploy an instance as usual and pick Open WebUI + Ollama in step 2, Template Selection. Choose a card with enough VRAM for the models you plan to run — the weights have to fit in GPU memory, and a 7–8B model at 4-bit quantization fits comfortably on a 24 GB card where a 70B model does not.

The full walkthrough is in Deploy a GPU Instance.

The first account that signs up becomes the administrator. Do this yourself, immediately, before you share the URL with anyone.

1. In the console open My Instance and wait until the card reads Active.

2. Open https://<your-slug>.instance.grafilab.ai in a browser. Your slug is the three-word name on the instance card.

3. Choose Sign up, enter a name, email, and password, and submit. This account is created as the administrator.

4. Any later sign-up on the same instance stays pending until you approve it. Approve people deliberately — every approved user runs jobs on the GPU you are paying for.

If the page shows a sign-in form rather than a sign-up form, someone got there first. Launch a fresh instance and claim it before sharing the URL.

Nothing is installed until you pull it. Two ways:

Open the user menu, choose Settings → Admin Settings → Models, type a model name such as llama3.2 into the model download field, and start the pull. Progress shows in the same panel, and the model appears in the chat model picker when it finishes.

Connect over SSH — see Manage Instances — and pull directly:

ollama pull llama3.2
ollama list

ollama list prints the models present in the container and their on-disk size. Refresh the browser tab and the new model appears in the picker.

Sizing rule of thumb: a model needs roughly its download size in VRAM, plus headroom for the context window. Check the VRAM figure on your instance card and stay under it. A model that doesn’t fit either spills to the CPU and crawls, or fails outright.

Start a new chat, pick the model you pulled, and send a message. The first reply after a pull is slow because the weights are being loaded into VRAM; the ones after it are fast.

To confirm the work is landing on the GPU rather than the CPU, run this over SSH while a reply is generating:

nvidia-smi

You should see an ollama process in the process list and non-zero GPU memory in use.

You can run both at once — local models on your GPU plus Grafilab’s hosted catalogue in the same picker. In Open WebUI open Settings → Admin Settings → Connections, add an OpenAI-compatible connection with the base URL https://llm.grafilab.ai/v1 and a Grafilab API key, and save.

Details, including how to issue a key, are on the Open WebUI integration page.

Pulled models are the expensive thing to lose — gigabytes fetched over the network. If you plan to relaunch and want to skip the download, use Backup on the instance card before the rental expires.

Open WebUI is the one built-in template with a login of its own, which makes it the safest of the web templates. That login is only as good as your approval discipline, and the proxy in front of it adds nothing:

• Sign-up succeeds but the account is pending approval. You were not the first account. Sign in as the admin account, open the admin user list, and approve the user.
• The first reply takes tens of seconds. The model is loading into VRAM. Later replies in the same session are fast; switching models reloads from scratch.
• Generation fails or the model unloads mid-reply. The model does not fit in VRAM alongside the context you asked for. Pull a smaller quantization, shorten the context, or relaunch on a card with more VRAM.
• The URL doesn’t answer right after launch. Wait until the instance card reads Active — the container is still starting.