AI Engineering Homelab Local LLMs

Building a Zero-Cost "Vibe Coding" Agentic Stack

The transition from manual coding to "vibe coding" (and ultimately, strict agentic engineering) requires a solid infrastructure. Relying on paid APIs can rack up massive bills when an autonomous AI agent starts running self-healing loops. The solution? Build a Hybrid Cloud-Local AI Proxy that pools free-tier cloud models (from Google and OpenRouter) with a local, on-premise model into a single, highly-available endpoint.

Here is the step-by-step guide on how I built a bulletproof, rate-limit-resistant AI engine on an Ubuntu server, and connected it securely to my local VS Code environment.

Step 1: The Proxy Backend (LiteLLM)

First, we need a router to handle API requests, manage rate limits, and cascade through different models if a provider goes down. We use LiteLLM running in a Docker container.

If you encounter a crash loop where the proxy refuses to start for security reasons, it requires a Master Key to prevent unauthorized access. Fix this by generating a secure key:

cd ~/litellm-proxy
echo "LITELLM_MASTER_KEY=sk-$(openssl rand -hex 32)" | tee -a .env
sudo docker compose down
sudo docker compose up -d

Tip: Run cat .env | grep LITELLM_MASTER_KEY and save this key. You will need it for your IDE later!

Step 2: The Hybrid Cloud-Local Routing Chain

Free AI tiers are notorious for 429 Rate Limit Exceeded and 503 Service Unavailable errors. To make our agent truly autonomous, we are going to build a 3-tier routing chain:

1. Primary (Gemini 3.8 Flash): High intelligence, but strict 20 RPM limits.
2. Burst Fallback (OpenRouter): Absorbs high-speed traffic when Google times out.
3. Local Hardware (Ollama / Qwen 2.5 Coder): A 1.5B parameter micro-model running locally on the Ubuntu server. It uses just ~1.2GB of RAM (perfect for a home server), costing nothing and acting as the ultimate safety net if both cloud providers fail.

First, install Ollama on your Ubuntu server and pull the micro-model:

curl -fsSL https://ollama.com/install.sh | sh
ollama pull qwen2.5-coder:1.5b

Next, configure your LiteLLM config.yaml to gracefully cascade through this hybrid chain:

model_list:
  # 1. Primary Engine (Highest IQ, 20 RPM limit)
  - model_name: vibe-coder
    litellm_params:
      model: gemini/gemini-3.8-flash
      api_key: "os.environ/GEMINI_API_KEY"

  # 2. Burst Fallback (Activates when Gemini hits a timeout)
  - model_name: openrouter-fallback
    litellm_params:
      model: openrouter/poolside/laguna-xs-2.1:free
      api_key: "os.environ/OPENROUTER_API_KEY"
      
  # 3. Local Safety Net (Activates when both clouds are exhausted)
  - model_name: local-fallback
    litellm_params:
      model: ollama_chat/qwen2.5-coder:1.5b
      api_base: "http://<YOUR_VPN_IP>:11434"

router_settings:
  num_retries: 2
  fallbacks:
    # Routes traffic down the chain sequentially upon failure
    - {"vibe-coder": ["openrouter-fallback", "local-fallback"]}

Step 3: Network Security (Nginx & VPN)

Never expose port 4000 to the public internet. Instead, route it through an Nginx reverse proxy, and access the machine exclusively via a mesh VPN (like Tailscale or ZeroTier).

Important Gotcha: Initially, I secured Nginx with auth_basic (htpasswd). However, VS Code network fetchers often strip passwords out of URLs (e.g., http://user:pass@ip), causing immediate 401 Unauthorized errors. Since the VPN already provides end-to-end encryption, it's safer and smoother to comment out the auth_basic lines in your Nginx config and rely solely on the VPN tunnel and the LiteLLM Master Key.

Step 4: The Frontend (Cline in VS Code)

With the backend running smoothly, install the Cline extension in VS Code. This is the "steering wheel" for our engine.

Go to the API Settings in the Cline sidebar (the Gear icon) and configure it to point to your secure local tunnel:

• API Provider: OpenAI Compatible
• Base URL: http://<YOUR_VPN_IP>:<NGINX_PORT>/v1 (Do not forget the /v1!)
• API Key: The LITELLM_MASTER_KEY you generated in Step 1
• Model ID: vibe-coder

Step 5: Agentic Engineering & Self-Healing

To move from casual "vibe coding" to actual agentic engineering, you must provide static context. Create a .clinerules file in the root of your project directory. This acts as an instruction manual that the AI automatically reads before taking action.

Here is an example I used for a Python FastAPI project:

# Python Server Project Rules

## 1. Coding Standards
- NEVER use print() for debugging. Always use the standard Python logging module.
- Format all Python code according to Black standards.
- Use strict type hinting for all new functions.

## 2. Infrastructure & Environment
- Assume the deployment environment is an Ubuntu server running Docker and Nginx.
- Do not run pip install globally. Always check for or create a .venv first.

## 3. Agent Behavior
- Always state a brief 1-2 sentence plan before executing terminal commands.
- If a command fails, read the stderr output and autonomously attempt a fix up to 3 times before asking for human help.

The Magic of Auto-Fixing: By explicitly granting the agent permission to read stderr and retry, you create a self-healing loop. When I instructed the agent to write a pytest suite for a FastAPI server and intentionally broke a dictionary key, the AI ran the tests, saw the red failure text in the terminal, read the error, rewrote the broken python file, and re-ran the tests until they passed—completely autonomously.

Conclusion

By stacking a VPN, an Nginx proxy, and a LiteLLM router in front of VS Code, you create a private, secure, highly capable AI development environment. With the hybrid cloud-local chain, if you push the agent too hard and max out your free cloud quotas, your local hardware seamlessly catches the overflow traffic. This architecture allows you to focus purely on the logic while the agent handles the boilerplate, all for exactly $0.