Claudey • Shipped 2026

The Multi-Provider Gateway for Autonomous Coding Agents

Claudey is a local reverse proxy and execution harness that lets developers run Claude Code, OpenAI Codex CLI, and Pi with 37+ model providers—complete with fallback chains, dynamic model catalogs, and sub-3ms routing overhead.

Role

Systems Engineer & Creator

Timeline

December 2025 – February 2026

Team

Solo

Skills

Protocol Normalization • High-Concurrency Proxies • Python 3.14 • FastAPI • CLI Tooling

Overview

Coding agents are only as resilient as their API gateways.

Autonomous coding agents like Claude Code, OpenAI Codex, and Pi are fundamentally changing software development. Instead of answering one-off prompts, these agents run long-horizon loops: reading files, editing codebases, executing test suites, and self-correcting across dozens of turns.

However, running production agent loops against a single proprietary endpoint creates severe bottlenecks: rate limits halt active development, upstream outages break test-driven iterations, and high per-token pricing makes rapid experimentation prohibitive.

Claudey solves this by sitting directly between the agent CLI and the inference layer. It exposes fully compliant Anthropic Messages and OpenAI Responses endpoints locally, translating schemas in real time across cloud providers and local inference runtimes.

Problem

Monolithic provider lock-in creates three critical points of failure.

When developers run autonomous agents through default cloud configurations, they run into three immediate walls:

1. Rate Limiting & Tier Starvation

A single test-driven development (TDD) cycle can generate 40+ tool calls in three minutes. Standard API quotas frequently trigger HTTP 429 rate limits mid-refactor, leaving the working tree in a broken, half-committed state.

2. Cost Asymmetry for Routine Tasks

Running simple unit tests, git status inspections, and syntax checks through flagship models costs identical dollars per token as high-level architectural design. Developers lack fine-grained control to route cheap queries to high-throughput models and reserve frontier models for deep reasoning.

3. Inflexible Local & Air-Gapped Workflows

Enterprise teams and security-sensitive projects cannot route proprietary intellectual property through external endpoints. Yet configuring local backends like Ollama or llama.cpp with agent CLIs previously required monkey-patching binaries or reverse-engineering client network layers.

Solution

Decoupling agent intelligence from proprietary infrastructure.

Claudey runs a lightweight local daemon that intercepts agent requests and manages credentials, schema translation, failover routing, and model discovery without modifying the agent binaries.

Multi-Tier Model Routing

Map distinct agent tiers (MODEL_FABLE, MODEL_OPUS, MODEL_SONNET, MODEL_HAIKU, GLOBAL_FALLBACK) to distinct backend providers based on workload complexity.

Dynamic Model Discovery

Inject a unified, searchable model catalog directly into the agent's native /model picker.

Zero-Loss Protocol Normalization

Stream responses, preserve tool-use JSON schemas, maintain multimodal image inputs, and translate extended thinking blocks with less than 3ms proxy overhead.

Health-Aware Fallback Chains

Automatically promote healthy models and skip rate-limited or degraded providers before returning errors to the terminal.

Core Architecture

Real-time protocol translation and streaming SSE adaptation.

Different LLM providers implement divergent specifications for tool definitions, streaming chunk structures, and role payloads. Claudey standardizes these variations into a unified internal representation.

┌──────────────────────────────────────────────────────────┐
│              Developer Workspace / Terminal              │
│       claude-code   │   codex-cli   │   pi-agent         │
└────────────────────────────┬─────────────────────────────┘
                             │ Local HTTP / SSE
┌────────────────────────────▼─────────────────────────────┐
│                      Claudey Daemon                      │
│   ┌──────────────────────────────────────────────────┐   │
│   │        Protocol Ingestion & Token Auth           │   │
│   │   - Anthropic /v1/messages                       │   │
│   │   - OpenAI /v1/responses                         │   │
│   └────────────────────────┬─────────────────────────┘   │
│   ┌────────────────────────▼─────────────────────────┐   │
│   │        Tier Router & Health State Machine        │   │
│   │   - Active Fallback Chain Resolution             │   │
│   │   - Pre-flight Node Validation & Skipping        │   │
│   └────────────────────────┬─────────────────────────┘   │
│   ┌────────────────────────▼─────────────────────────┐   │
│   │         Bidirectional Protocol Adapter           │   │
│   │   - Tool Call Parameter Normalization            │   │
│   │   - Multimodal Image Payload Re-encoding         │   │
│   │   - Reasoning Block & Thought Extraction         │   │
│   └──────────────────────────────────────────────────┘   │
└────────────────────────────┬─────────────────────────────┘
                             │ Upstream HTTPS / Local Socket
┌────────────────────────────▼─────────────────────────────┐
│               Supported Provider Ecosystem               │
│  NVIDIA NIM  │  Gemini  │  DeepSeek  │  Groq  │  Ollama  │
└──────────────────────────────────────────────────────────┘

Engineering Deep Dives

1. Tool Call Schema Normalization

Anthropic and OpenAI handle function calling with incompatible JSON structures. Anthropic uses content blocks with tool_use and tool_result IDs, while OpenAI and OpenAI-compatible gateways format tool invocations through a dedicated tool_calls array with stringified argument payloads.

Claudey implements a zero-copy streaming parser that detects tool execution boundaries on the fly, converts JSON parameter strings into structured blocks, and guarantees that tool call IDs remain idempotent across retried requests.

# Bidirectional schema normalization excerpt
def normalize_tool_call_to_anthropic(openai_tool_call: dict) -> dict:
    return {
        "type": "tool_use",
        "id": openai_tool_call.get("id") or f"call_{uuid.uuid4().hex[:8]}",
        "name": openai_tool_call["function"]["name"],
        "input": json.loads(openai_tool_call["function"]["arguments"]),
    }

2. Health-Aware Fallback Chains and Combos

A tier setting can define an ordered chain of targets. If a primary provider returns a retryable status code (HTTP 429, 502, 503, 504, or socket timeout) before any bytes have been flushed to the client, Claudey immediately rolls the request over to the next configured fallback node.

# Multi-tier fallback chain example
MODEL_OPUS="nvidia_nim/nvidia/nemotron-3-super-120b-a12b,open_router/anthropic/claude-sonnet-4-5,zai/glm-5.2"
MODEL_SONNET="gemini/models/gemini-3.1-flash-lite,deepseek/deepseek-chat"

To eliminate blind retries, Claudey tracks provider health in memory. When a node exhausts its rate quota or experiences an outage, it enters a temporary cooldown state. Subsequent turns route to the last-known-good provider immediately, eliminating latency spikes.

3. Dynamic Model Catalog Injection

Coding agent CLIs discover available models at startup or upon executing /model. To prevent hardcoding lists of provider slugs, Claudey dynamically generates a catalog based on currently configured credentials.

When a developer connects a new provider (such as an NVIDIA NIM key or a local Ollama instance), Claudey queries the provider's model endpoint, filters for tool-compatible architectures, and exposes them directly to the CLI interface.

4. Route Explainability via x-claudey-route

Debugging multi-agent routing decisions in complex fallback topologies requires clear observability. Claudey attaches an x-claudey-route header to every response and emits a structured claudey.api.route.resolved trace event.

HTTP/1.1 200 OK
Content-Type: text/event-stream
x-claudey-route: routeway/meta-llama/llama-3.1-70b; why=primary_down

This diagnostic clarity ensures developers always know which model served a turn and why a fallback occurred.

5. Desktop Companion & Multi-Channel Messaging

Claudey includes a native menu bar and system tray companion for macOS and Windows. The background daemon manages server lifecycle, displays real-time health metrics, and opens the local Admin UI in one click.

For remote engineering operations, Claudey integrates optional Discord and Telegram bot adapters. Developers can send coding prompts or voice notes directly from mobile devices; Claudey transcribes audio via local Whisper or NVIDIA NIM and pipes the command into an isolated terminal session.

Outcomes

Performance, resilience, and operational efficiency.

Claudey is actively maintained and used across daily engineering workflows.

37+ Providers Supported

Native compatibility with NVIDIA NIM, Google AI Studio, Vertex AI, Anthropic, DeepSeek, Mistral, Groq, Cerebras, OpenRouter, and local engines (Ollama, llama.cpp, LM Studio).

Zero Dropped Tool Calls

Validated over 100,000+ continuous agent turns with strict schema invariance.

72% Cost Reduction

Achieved through tiered model routing: running verification and syntax checks on high-speed flash models while saving frontier reasoning models for complex refactoring.

Reflection

What I learned building proxy infrastructure for autonomous agents.

Protocol compatibility requires defensive engineering.

No two AI providers interpret the OpenAI or Anthropic specs identically. Small discrepancies—such as how null arguments are serialized, how reasoning tokens are delimited, or how streaming tool chunks are chunked—will break an agent's parser if not sanitized at the proxy boundary.

Resilience belongs at the network edge.

Autonomous agents should not need to implement custom retry and backoff logic for every conceivable cloud provider. Handling failovers, health checks, and credential validation in a dedicated local proxy keeps agent harnesses simple, fast, and deterministic.