Self-Host Your Own AI Coding Assistant on a VPS: Ollama + Continue, End to End

A self-hosted coding assistant keeps model requests under your control, but the safe architecture is simpler than running an AI web application on a public port. Ollama runs on the VPS as the model server. Continue remains an extension on your development computer and reaches Ollama through a private SSH tunnel.

This guide builds that setup on Ubuntu 24.04 with Docker, a persistent model volume, loopback-only networking, and a local Continue configuration. You will also verify the service, monitor memory use, and update it without exposing Ollama directly to the internet.

Plan the Private Ollama and Continue Architecture

Continue is an IDE extension, not a server that must be deployed beside Ollama. The current Continue configuration accepts an Ollama apiBase, so your workstation can direct the extension to a remote endpoint. The safest baseline is to keep that endpoint bound to 127.0.0.1 on the VPS and forward it to your workstation with SSH.

• VPS: runs the Ollama container and stores downloaded model layers in a Docker volume.
• Workstation: runs VS Code or JetBrains with the Continue extension.
• Connection: an authenticated SSH tunnel maps a local port to Ollama’s loopback port on the VPS.
• Public exposure: only SSH is required; Ollama does not need a public reverse proxy or TLS certificate.

Continue’s current Ollama guide lists 8 GB RAM as a minimum and recommends 16 GB or more. Model size, quantization, context length, and concurrent requests all affect actual memory use, so leave headroom for Ubuntu and Docker. A CPU-only VPS can run a compact coding model, but token generation will normally be slower than on a supported GPU. For broader sizing trade-offs, see the VPS.us guide to running a self-hosted LLM.

Secure and Update the VPS

Start with a non-root administrator, key-based SSH access, current packages, and a firewall rule for SSH. Keep the original provider console available until you have confirmed that the new account works, because a firewall or SSH mistake can lock you out.

sudo adduser deployer
sudo usermod -aG sudo deployer
sudo apt update
sudo apt full-upgrade -y
sudo apt autoremove -y
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow OpenSSH
sudo ufw enable
sudo ufw status verbose

Open a second terminal and confirm that ssh deployer@VPS_IP succeeds before closing the root session. Then review SSH hardening, automatic security updates, backups, and recovery access as separate operational decisions. The VPS server optimization guide provides additional system-level checks.

Install Docker From the Official Repository

Docker currently supports Ubuntu 24.04 and 22.04. Its official Ubuntu instructions use a docker.sources file and an ASCII-armored signing key. Remove conflicting distribution packages first only when they are present, then install Docker Engine and the Compose plugin from Docker’s repository.

sudo apt update
sudo apt install -y ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

sudo tee /etc/apt/sources.list.d/docker.sources > /dev/null <<EOF
Types: deb
URIs: https://download.docker.com/linux/ubuntu
Suites: $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}")
Components: stable
Architectures: $(dpkg --print-architecture)
Signed-By: /etc/apt/keyrings/docker.asc
EOF

sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo systemctl status docker --no-pager
sudo docker run --rm hello-world

Membership in the docker group grants root-equivalent control over the host through the Docker socket. This guide therefore keeps sudo docker in its commands. If you deliberately choose another privilege model, review Docker’s rootless and post-installation guidance first. The VPS.us Docker VPS hosting guide covers more container-hosting considerations.

Run Ollama on a Loopback-Only Port

The official Ollama container stores models under /root/.ollama and listens on port 11434. A normal Docker mapping such as -p 11434:11434 publishes the port on every host interface. Bind the host side to 127.0.0.1 instead. This also avoids relying on UFW to protect a Docker-published port, because Docker documents that published container ports can bypass UFW rules.

sudo docker volume create ollama-models
sudo docker pull ollama/ollama
sudo docker run -d \
  --name ollama \
  --restart unless-stopped \
  -v ollama-models:/root/.ollama \
  -p 127.0.0.1:11434:11434 \
  ollama/ollama

curl http://127.0.0.1:11434/api/tags
sudo docker logs ollama --tail 50

An empty models array is expected before you download a model. Confirm the socket is loopback-only with sudo ss -lntp | grep 11434. It should show 127.0.0.1:11434, not 0.0.0.0:11434.

Download a Coding Model and Check Memory Use

Start with one compact model and verify it before adding alternatives. Continue’s current Ollama documentation includes Qwen2.5-Coder 7B as a local option, but a 7B model is still substantial on a CPU-only host. If Ollama reports insufficient memory, reduce the configured context length or choose a smaller model rather than adding swap and treating disk as equivalent to RAM.

sudo docker exec ollama ollama pull qwen2.5-coder:7b
sudo docker exec ollama ollama list
sudo docker exec ollama ollama ps
sudo docker stats --no-stream ollama

Test one non-streaming request locally on the VPS. The response should contain a response field and should not require a public network route.

curl http://127.0.0.1:11434/api/generate \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen2.5-coder:7b",
    "prompt": "Return only a Python function with type hints that checks whether an integer is prime.",
    "stream": false
  }'

Connect Continue Through an SSH Tunnel

Install Continue in your IDE on your workstation. Then open an SSH session that forwards your workstation’s loopback port 11434 to the same loopback port on the VPS. The -N option starts no remote shell, and -T disables pseudo-terminal allocation.

ssh -N -T \
  -L 127.0.0.1:11434:127.0.0.1:11434 \
  deployer@VPS_IP

Leave that session running and test curl http://127.0.0.1:11434/api/tags on your workstation. Continue stores local YAML configuration in ~/.continue/config.yaml on macOS and Linux. Use a current schema header, an exact Ollama model tag, and the forwarded loopback endpoint.

name: Private VPS Coding Assistant
version: 0.0.1
schema: v1

models:
  - name: Qwen2.5 Coder 7B on VPS
    provider: ollama
    model: qwen2.5-coder:7b
    apiBase: http://127.0.0.1:11434
    roles:
      - chat
      - edit
      - apply
    defaultCompletionOptions:
      contextLength: 4096

Reload the Continue configuration from the IDE. If you also want autocomplete, evaluate it separately: autocomplete has tighter latency requirements, and one model may not be the best choice for every role. The VPS.us walkthrough for running Ollama models on a VPS offers another model-specific example.

Maintain the Stack Without Automatic Surprise Updates

A cron job that pulls the newest container every night can change behavior while you are working and removes your chance to review release notes. Use a maintenance window instead: record the current image identity, back up the configuration and model inventory, pull the candidate image, recreate the container, and run the same verification requests before ending the window.

• Keep the VPS and Docker packages current through a reviewed update process.
• Record model names and exact tags with sudo docker exec ollama ollama list.
• Back up the Docker volume only with a documented, tested restore procedure.
• Review free disk space before pulling another model; model layers can be large.
• Do not expose port 11434 merely to make reconnection more convenient.

Verify and Troubleshoot the Complete Path

Test each boundary separately. A working container does not prove that the SSH tunnel is active, and a working tunnel does not prove that Continue is using the intended model tag.

# On the VPS
sudo docker ps --filter name=ollama
sudo docker logs ollama --tail 100
curl http://127.0.0.1:11434/api/tags
sudo ss -lntp | grep 11434

# On the workstation while the SSH tunnel is open
curl http://127.0.0.1:11434/api/tags
ssh -v -N -T -L 127.0.0.1:11434:127.0.0.1:11434 deployer@VPS_IP

• Connection refused on the VPS: inspect the Ollama container status and logs.
• Works on the VPS but not the workstation: confirm the SSH session is still running and that local port 11434 is free.
• Model not found: compare the Continue model value with the exact tag returned by ollama list.
• Insufficient memory: reduce contextLength, stop competing workloads, or choose a smaller model.
• Very slow responses: check whether the model is CPU-bound, swapping, or reloading because memory is insufficient.