Building a Zero-Cost "Vibe Coding" Agentic Stack

AI Engineering Homelab Local LLMs

Building a Zero-Cost "Vibe Coding" Agentic Stack

The transition from manual coding to "vibe coding" (and ultimately, strict agentic engineering) requires a solid infrastructure. Relying purely on paid APIs can rack up massive bills when an autonomous AI agent starts running self-healing loops and automated test generations.

The solution? Build a Hybrid Cloud-Local AI Proxy. This architecture pools free-tier cloud models (from Google and OpenRouter) with local, on-premise hardware into a single, highly-available endpoint. I have divided this guide into two parts: first, establishing the core proxy on a single Ubuntu server, and second, expanding it into a cross-platform distributed cluster using a secondary NAS unit for heavy background tasks.

Part 1: The Core Server Setup (Ubuntu Proxy)

In this phase, we will set up the main API router, a database for usage tracking, and a lightweight local AI model to serve as our absolute last line of defense.

Step 1: The Proxy Backend (LiteLLM)

We need a router to handle API requests, manage rate limits, and cascade through different models if a provider goes down. We use LiteLLM running in a Docker container.

To prevent unauthorized access to your proxy, generate a secure Master Key:

mkdir ~/litellm-proxy && cd ~/litellm-proxy
echo "LITELLM_MASTER_KEY=sk-$(openssl rand -hex 32)" | tee -a .env

Tip: Run cat .env | grep LITELLM_MASTER_KEY and save this key somewhere safe. You will need it for your IDE and UI login later!

Step 2: Database Persistence & Admin UI

To track token usage, view routing logs, and access the Admin UI dashboard, LiteLLM requires a PostgreSQL database. Create a docker-compose.yml file:

services:
  litellm:
    image: ghcr.io/berriai/litellm:main-latest
    container_name: litellm_proxy
    ports:
      - "4000:4000"
    volumes:
      - ./config.yaml:/app/config.yaml
    env_file:
      - .env
    command: [ "--config", "/app/config.yaml", "--port", "4000" ]
    restart: always

  db:
    image: postgres:15
    restart: always
    environment:
      POSTGRES_DB: litellm
      POSTGRES_USER: litellm_user
      POSTGRES_PASSWORD: your_secure_db_password
    volumes:
      - ./litellm_db_data:/var/lib/postgresql/data

Next, map the database into your .env file so the proxy can connect internally (using Docker's internal db hostname):

echo "DATABASE_URL=postgresql://litellm_user:your_secure_db_password@db:5432/litellm" | tee -a .env
sudo docker compose up -d

Once the containers are up, initialize the database privileges (replace container names and passwords as needed):

sudo docker exec -it litellm-proxy-db-1 psql -U postgres
# Inside the psql prompt:
CREATE DATABASE litellm;
CREATE USER litellm_user WITH ENCRYPTED PASSWORD 'your_secure_db_password';
GRANT ALL PRIVILEGES ON DATABASE litellm TO litellm_user;
\q

Step 3: The Local Safety Net (Ollama)

Free AI tiers are notorious for 429 Rate Limit Exceeded errors. We will use a 1.5B parameter micro-model running locally on the Ubuntu server. It uses just ~1.2GB of RAM, costs nothing, and acts as the ultimate safety net if cloud providers fail.

curl -fsSL https://ollama.com/install.sh | sh
ollama pull qwen2.5-coder:1.5b

Step 4: Network Security (Nginx & VPN)

Never expose port 4000 directly to the public internet. Instead, route it through an Nginx reverse proxy, and access the machine exclusively via a mesh VPN (like Tailscale).

Nginx Redirect Gotcha: When securing Nginx, you must change proxy_set_header Host $host; to $http_host;. If you don't, Nginx strips your custom port off the URL and throws a 404 error when LiteLLM tries to redirect you to the Admin UI.

server {
    listen 9001;
    server_name _;

    location / {
        proxy_pass http://127.0.0.1:4000;
        proxy_set_header Host $http_host; # Essential for UI redirects
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Restart Nginx (sudo systemctl restart nginx). You can now navigate to http://<YOUR_UBUNTU_TAILSCALE_IP>:9001/ui/ and log in using the username admin and your LITELLM_MASTER_KEY as the password.

Part 2: Multi-Server Cross-Platform Expansion

While the main proxy handles fast coding workflows, background tasks like document tagging, OCR, and vector embeddings can bog down limited server RAM. We can expand our architecture by offloading these heavy asynchronous workloads to a secondary NAS unit (e.g., an older Intel Core i5 machine) across our Tailscale network.

Step 1: Configuring the NAS Endpoint

On the NAS machine, install Ollama. By default, Ollama only listens to 127.0.0.1. We must bind it to the Tailscale VPN interface to securely accept requests from our Ubuntu proxy server.

curl -fsSL https://ollama.com/install.sh | sh
sudo mkdir -p /etc/systemd/system/ollama.service.d
echo '[Service]' | sudo tee /etc/systemd/system/ollama.service.d/override.conf
echo 'Environment="OLLAMA_HOST=0.0.0.0"' | sudo tee -a /etc/systemd/system/ollama.service.d/override.conf

sudo systemctl daemon-reload
sudo systemctl restart ollama

# Pull our heavy background models
ollama pull gemma4:e2b
ollama pull nomic-embed-text

Step 2: The "Cold Start" Timeout Trap

When routing across a VPN to slower NAS hardware, timeouts are a killer. Loading a 2.3GB model into memory on an older CPU takes time. If it takes longer than 60 seconds, Nginx (and Docker's internal networking) will abruptly sever the connection, resulting in maddening 504 Gateway Time-out errors.

To fix this, we need to bypass Docker's isolated network so LiteLLM can act as a native application on the host OS, letting it talk directly over Tailscale without Nginx bridging limits.

Update your docker-compose.yml on the main Ubuntu server to use Host Networking Mode:

services:
  litellm:
    image: ghcr.io/berriai/litellm:main-latest
    container_name: litellm_proxy
    network_mode: "host"       # <-- Bypasses Docker network limits
    # ports:                   # <-- Remove ports block, host mode claims it automatically
    #   - "4000:4000"
    volumes:
      - ./config.yaml:/app/config.yaml
    env_file:
      - .env
    command: [ "--config", "/app/config.yaml", "--port", "4000" ]
    restart: always

  db:
    image: postgres:15
    restart: always
    environment:
      POSTGRES_DB: litellm
      POSTGRES_USER: litellm_user
      POSTGRES_PASSWORD: your_secure_db_password
    ports:                     # <-- Must explicitly expose DB port now
      - "5432:5432"
    volumes:
      - ./litellm_db_data:/var/lib/postgresql/data

Because LiteLLM is now on the host network, it can no longer resolve the Docker hostname db. Update your .env file to point to localhost instead:

sed -i 's/@db:5432/@127.0.0.1:5432/g' ~/litellm-proxy/.env

Step 3: The Unified Routing Configuration

Now we define our complete config.yaml. We will establish our primary fast-coding fallback chain, and explicitly route background tasks to the NAS. Crucially, we will add timeout: 300 (to allow 5 minutes for cold starts) and keep_alive: -1 (to tell the NAS to NEVER unload the model from RAM once it boots).

model_list:
  # 1. Primary Engine (Highest IQ, 20 RPM limit)
  - model_name: vibe-coder
    litellm_params:
      model: gemini/gemini-3.8-flash
      api_key: "os.environ/GEMINI_API_KEY"

  # 2. Burst Fallback (Activates when Gemini hits a limit)
  - model_name: openrouter-fallback
    litellm_params:
      model: openrouter/poolside/laguna-xs-2.1:free
      api_key: "os.environ/OPENROUTER_API_KEY"
      
  # 3. Local Safety Net (Ubuntu Server)
  - model_name: local-fallback
    litellm_params:
      model: ollama_chat/qwen2.5-coder:1.5b
      api_base: "http://127.0.0.1:11434"

  # 4. NAS Background Task Engine (via NAS Tailscale IP)
  - model_name: background-tagger
    litellm_params:
      model: ollama_chat/gemma4:e2b
      api_base: "http://<YOUR_NAS_TAILSCALE_IP>:11434"
      timeout: 300
      keep_alive: -1

  # 5. NAS Vector Embedding Engine (via NAS Tailscale IP)
  - model_name: document-embedder
    litellm_params:
      model: ollama/nomic-embed-text
      api_base: "http://<YOUR_NAS_TAILSCALE_IP>:11434"
      timeout: 300
      keep_alive: -1

router_settings:
  num_retries: 2
  allowed_fails: 1
  cooldown_time: 300
  fallbacks:
    - {"vibe-coder": ["openrouter-fallback", "local-fallback"]}

Apply the final configuration:

sudo docker compose down
sudo docker compose up -d

Step 4: Connecting the IDE (VS Code & Cline)

To test this setup or use it with agents like Cline or Continue.dev, point your IDE to your secure Tailscale tunnel.

Important: For automated agents and Python scripts running on your local machine, hit port 4000 directly over Tailscale (e.g., http://<UBUNTU_TAILSCALE_IP>:4000/v1) instead of port 9001. This completely bypasses the Nginx frontend reverse proxy and its 60-second timeouts, ensuring your IDE waits patiently for the NAS models to respond.

API Provider: OpenAI Compatible
Base URL: http://<UBUNTU_TAILSCALE_IP>:4000/v1 (Do not forget the /v1!)
API Key: Your LITELLM_MASTER_KEY
Model ID: vibe-coder

Step 5: Agentic Engineering & Self-Healing

To move from casual "vibe coding" to actual agentic engineering, you must provide static context. Create a .clinerules file in the root of your project directory. This acts as an instruction manual that the AI automatically reads before taking action.

# Python Server Project Rules

## 1. Coding Standards
- NEVER use print() for debugging. Always use the standard Python logging module.
- Format all Python code according to Black standards.
- Use strict type hinting for all new functions.

## 2. Infrastructure & Environment
- Assume the deployment environment is an Ubuntu server running Docker and Nginx.
- Do not run pip install globally. Always check for or create a .venv first.

## 3. Agent Behavior
- Always state a brief 1-2 sentence plan before executing terminal commands.
- If a command fails, read the stderr output and autonomously attempt a fix up to 3 times before asking for human help.

Conclusion

By stacking a VPN, an Nginx proxy, and a LiteLLM router backed by PostgreSQL in Host Networking Mode, you create a highly resilient AI development environment. Your local hardware catches overflow coding traffic instantly, while secondary NAS hardware quietly churns through background processing, immune to timeouts. This architecture allows you to focus purely on the logic while the agent handles the boilerplate, all for exactly $0.

AI Security SOC Automation Local LLMs

Building a Privacy-First Agentic SOC Stack

Version Comparison & Decision Guide

The Previous Version (V1 - "Building a Zero-Cost Vibe Coding Agentic Stack")

Networking: Relied on Docker's network_mode: "host" to bypass internal proxy timeouts, allowing the system to wait for cold-starting local models on a NAS.
Architecture: Featured a straightforward pipeline with LiteLLM and PostgreSQL for tracking, utilizing a single local Ollama instance as a fallback.
Security: Secured the frontend with an Nginx reverse proxy and a mesh VPN (Tailscale).
Best For: Solo developers or hobbyists strictly doing code generation where data privacy (like internal IPs or usernames) sent to cloud providers is not a concern, and minimal container overhead is preferred.

The New Version (V2 - Enterprise SOC & Privacy Pipeline)

Networking: Reverts to standard Docker Bridge networking to securely orchestrate multiple interconnected internal microservices without exposing them to the host network.
Architecture: Upgrades to a highly available Swarm using a least-busy routing strategy across multiple local Qwen nodes. Adds a Redis container for zero-cost semantic caching of duplicate requests.
Security: Integrates Microsoft Presidio (Analyzer and Anonymizer) to automatically intercept, scrub, and mask Personally Identifiable Information (PII) and internal network IPs before they ever leave your network. Implements hard monthly spend limits to prevent DoS billing attacks.
Best For: Enterprise-grade homelabs, Security Operations Centers (SOCs), or users feeding sensitive telemetry, security logs (like Wazuh alerts), or internal network topologies into LLMs.

Building a Privacy-First Agentic SOC Stack

When feeding raw security telemetry (like Wazuh SOC logs) into an AI agent, data privacy is paramount. You cannot broadcast internal network IPs, usernames, and vulnerability footprints to third-party cloud APIs. The solution is an Enterprise-Grade Local AI Proxy that masks Personally Identifiable Information (PII) before it leaves your network, utilizes Redis for caching duplicate alerts, and balances heavy workloads across a local hardware swarm.

Step 1: The Dockerized Microservice Stack

Our backend relies on LiteLLM for routing, PostgreSQL for logging, Redis for zero-cost semantic caching, and Microsoft Presidio for real-time PII redaction. Create a docker-compose.yml file:

services:
  litellm:
    image: ghcr.io/berriai/litellm:main-latest
    container_name: litellm_proxy
    ports:
      - "4000:4000"
    volumes:
      - ./config.yaml:/app/config.yaml
    env_file:
      - .env
    environment:
      - STORE_MODEL_IN_DB=True
      - REDIS_URL=redis://redis:6379
      - DATABASE_URL=postgresql://litellm_user:YOUR_SECURE_PASSWORD@db:5432/litellm
      - PRESIDIO_ANALYZER_API_BASE=http://presidio_analyzer:3000
      - PRESIDIO_ANONYMIZER_API_BASE=http://presidio_anonymizer:3000
    command: [ "--config", "/app/config.yaml", "--port", "4000" ]
    restart: always
    depends_on:
      - presidio_analyzer
      - presidio_anonymizer
      - redis

  presidio_analyzer:
    image: mcr.microsoft.com/presidio-analyzer:latest
    container_name: presidio_analyzer
    restart: always

  presidio_anonymizer:
    image: mcr.microsoft.com/presidio-anonymizer:latest
    container_name: presidio_anonymizer
    restart: always

  db:
    image: postgres:15
    restart: always
    environment:
      POSTGRES_DB: litellm
      POSTGRES_USER: litellm_user
      POSTGRES_PASSWORD: YOUR_SECURE_PASSWORD
    volumes:
      - ./litellm_db_data:/var/lib/postgresql/data

  redis:
    image: redis:latest
    container_name: redis
    restart: always

Step 2: The Routing and Privacy Configuration

Next, we configure LiteLLM to enforce strict budget limits, enable Redis caching for duplicate alerts, trigger Presidio callbacks to scrub IPs, and balance traffic across multiple local machines via a least-busy swarm routing strategy. Create a config.yaml file:

litellm_settings:
  enable_anthropic_prompt_caching: false
  max_budget: 10.00
  budget_duration: 30d
  cache: true
  callbacks:
    - presidio

model_list:
  - model_name: vibe-coder
    litellm_params:
      api_key: "os.environ/CLOUD_API_KEY"
      model: gemini/gemini-1.5-flash

  - model_name: openrouter-fallback
    litellm_params:
      api_key: "os.environ/OPENROUTER_API_KEY"
      model: openrouter/nvidia/nemotron-3-super-120b-a12b:free

  - model_name: qwen-cluster
    litellm_params:
      model: ollama_chat/qwen2.5-coder:1.5b
      api_base: "http://YOUR_NODE_1_IP:11434"
      timeout: 300

  - model_name: qwen-cluster
    litellm_params:
      model: ollama_chat/qwen2.5-coder:1.5b
      api_base: "http://YOUR_NODE_2_IP:11434"
      timeout: 300

router_settings:
  routing_strategy: least-busy
  fallbacks: [{"vibe-coder": ["openrouter-fallback", "qwen-cluster"]}]
  num_retries: 2

Step 3: Security Validation

Once your containers are running, you can test the PII masking. Send a cURL request containing a sensitive IP address:

curl -X POST "http://127.0.0.1:4000/v1/chat/completions" \
  -H "Authorization: Bearer YOUR_LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "vibe-coder",
    "messages": [
      {"role": "user", "content": "My internal server IP is 192.168.1.150. What IP did I just give you?"}
    ]
  }'

The AI will respond: "You provided the following information: IP address: <IP_ADDRESS>". Your local hardware successfully intercepted the prompt, scrubbed the real IP, and replaced it with a placeholder before the cloud API ever saw it.

Conclusion

By integrating Redis for caching and Microsoft Presidio for redaction, your AI environment is now an enterprise-grade automation pipeline. Your network topology remains hidden from cloud providers, duplicate workloads cost zero API credits, and your homelab swarm effortlessly balances heavy local processing.

Complete CLI Deployment Guide

For readers wanting to deploy this stack flawlessly from a fresh Ubuntu terminal, run these commands sequentially. Replace the placeholder passwords and IPs with your actual values before pressing enter.


# 1. Create the working directory
mkdir ~/litellm-proxy && cd ~/litellm-proxy

# 2. Generate a secure Master Key and environment variables
echo "LITELLM_MASTER_KEY=sk-$(openssl rand -hex 32)" | tee -a .env
echo "CLOUD_API_KEY=YOUR_GEMINI_OR_OPENAI_KEY" | tee -a .env
echo "OPENROUTER_API_KEY=YOUR_OPENROUTER_KEY" | tee -a .env

# 3. Create the LiteLLM configuration file
cat << 'EOF' > config.yaml
litellm_settings:
  enable_anthropic_prompt_caching: false
  max_budget: 10.00
  budget_duration: 30d
  cache: true
  callbacks:
    - presidio

model_list:
  - model_name: vibe-coder
    litellm_params:
      api_key: "os.environ/CLOUD_API_KEY"
      model: gemini/gemini-1.5-flash

  - model_name: openrouter-fallback
    litellm_params:
      api_key: "os.environ/OPENROUTER_API_KEY"
      model: openrouter/nvidia/nemotron-3-super-120b-a12b:free

  - model_name: qwen-cluster
    litellm_params:
      model: ollama_chat/qwen2.5-coder:1.5b
      api_base: "http://YOUR_NODE_1_IP:11434"
      timeout: 300

  - model_name: qwen-cluster
    litellm_params:
      model: ollama_chat/qwen2.5-coder:1.5b
      api_base: "http://YOUR_NODE_2_IP:11434"
      timeout: 300

router_settings:
  routing_strategy: least-busy
  fallbacks: [{"vibe-coder": ["openrouter-fallback", "qwen-cluster"]}]
  num_retries: 2
EOF

# 4. Create the Docker Compose file
cat << 'EOF' > docker-compose.yml
services:
  litellm:
    image: ghcr.io/berriai/litellm:main-latest
    container_name: litellm_proxy
    ports:
      - "4000:4000"
    volumes:
      - ./config.yaml:/app/config.yaml
    env_file:
      - .env
    environment:
      - STORE_MODEL_IN_DB=True
      - REDIS_URL=redis://redis:6379
      - DATABASE_URL=postgresql://litellm_user:YOUR_SECURE_PASSWORD@db:5432/litellm
      - PRESIDIO_ANALYZER_API_BASE=http://presidio_analyzer:3000
      - PRESIDIO_ANONYMIZER_API_BASE=http://presidio_anonymizer:3000
    command: [ "--config", "/app/config.yaml", "--port", "4000" ]
    restart: always
    depends_on:
      - presidio_analyzer
      - presidio_anonymizer
      - redis

  presidio_analyzer:
    image: mcr.microsoft.com/presidio-analyzer:latest
    container_name: presidio_analyzer
    restart: always

  presidio_anonymizer:
    image: mcr.microsoft.com/presidio-anonymizer:latest
    container_name: presidio_anonymizer
    restart: always

  db:
    image: postgres:15
    restart: always
    environment:
      POSTGRES_DB: litellm
      POSTGRES_USER: litellm_user
      POSTGRES_PASSWORD: YOUR_SECURE_PASSWORD
    volumes:
      - ./litellm_db_data:/var/lib/postgresql/data

  redis:
    image: redis:latest
    container_name: redis
    restart: always
EOF

# 5. Deploy the stack (This will download all images and start the proxy)
sudo docker compose up -d

Search This Blog

Secur-tainment