How to deploy a private LLM (Llama 3.1, DeepSeek) on a cloud GPU server with Ollama or vLLM

Running your own model makes sense when you have a lot of repetitive text work (product descriptions, listing rewrites, support-ticket triage, classification), when you can't send the data to a third-party API, or when per-token bills keep growing. It doesn't make sense if you need frontier-model quality on hard reasoning. Be honest with yourself about which of those you are before you rent a GPU.

Step 1: size the GPU

A rough rule for model weights: parameters × bytes per parameter, plus headroom for the context (KV cache) and runtime.

Long contexts and many simultaneous users need extra memory for the KV cache. If you plan 8k+ token contexts with several concurrent requests, budget a few GB more than the weights. For bulk e-commerce text, an 8B model at 4-bit is usually the cost/quality sweet spot. Test it on 50 real examples before you scale.

Hourly GPU rental is offered by providers such as RunPod, Vultr and DigitalOcean (Paperspace). Compare them in the GPU pricing comparison.

Step 2: choose Ollama or vLLM

Start with Ollama to find the right model. Switch to vLLM when you're pushing thousands of requests through it.

Step 3a: Ollama in five minutes

On an Ubuntu GPU server with NVIDIA drivers installed (most GPU images ship with them, and nvidia-smi should list your card):

curl -fsSL https://ollama.com/install.sh | sh
ollama pull llama3.1:8b          # or: deepseek-r1:8b, qwen2.5:14b ...
ollama run llama3.1:8b "Write a 40-word product description for a bamboo desk lamp."

# API (listens on 127.0.0.1:11434 by default)
curl http://127.0.0.1:11434/api/generate -d '{"model": "llama3.1:8b", "prompt": "Hello", "stream": false}'

By default Ollama listens only on localhost. Keep it that way.

Step 3b: vLLM for throughput

# Docker is the least painful route
docker run --gpus all --ipc=host -p 127.0.0.1:8000:8000 \
  -e HF_TOKEN=$HF_TOKEN \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest \
  --model meta-llama/Llama-3.1-8B-Instruct \
  --max-model-len 8192 \
  --api-key "$VLLM_API_KEY"

• Llama models on Hugging Face are gated: accept the license on the model page and create a read token for HF_TOKEN.
• --max-model-len caps the context. Lower it if you get out-of-memory errors at startup.
• Binding to 127.0.0.1:8000 keeps the port off the public internet.

Because it's OpenAI-compatible, existing code works by changing the base URL:

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="YOUR_VLLM_API_KEY")
out = client.chat.completions.create(
    model="meta-llama/Llama-3.1-8B-Instruct",
    messages=[{"role": "user", "content": "Rewrite this title for Amazon in 80 characters: ..."}],
    temperature=0.4,
)
print(out.choices[0].message.content)

Step 4: lock it down

An open model endpoint gets found and abused quickly, for free compute or to pull your data. Minimum hygiene:

• Firewall: allow SSH only (ufw allow OpenSSH && ufw enable). Don't open 11434 or 8000.
• Reach it through an SSH tunnel from your machine: ssh -N -L 8000:127.0.0.1:8000 user@server, then use http://localhost:8000.
• If other services must reach it, put it behind a reverse proxy (Caddy or Nginx) with HTTPS and authentication, or a private network/VPN between your servers.
• Use SSH keys only, disable password login, and keep the OS and container images updated.

Step 5: keep the bill under control

• GPU servers bill by the hour whether you use them or not. For batch jobs, start the server, run the queue, and stop it.
• Store models on a persistent volume so a restart doesn't re-download tens of GB.
• Batch your work: one vLLM server handling 32 parallel requests is far cheaper per item than 32 sequential calls.
• Track cost per finished item (per description, per ticket). If an API model is cheaper at your volume, use the API. Self-hosting is a tool, not an ideology.

Next step: wire the model into automations with self-hosted n8n.