taotok.io — unified LLM API gateway
Archiviert 10 days ago
T
Taotok API | Quant & Local LLM
Script Kiddie!
Brief description:
One OpenAI-compatible endpoint that routes to GPT-4o, Claude 3.5, Gemini 1.5 Pro, and DeepSeek V3. Stop wiring up 5 different APIs.
Tech stack:
Python / FastAPI · LiteSpeed reverse proxy · per-provider adapters (~150 lines each) · intelligent routing + observability layer
Key features / challenges:
• Unified ingress (OpenAI format, one auth)• Per-provider protocol adapters• Auto-routing with cost/availability + fallback on 429• Centralized provider key management
Write-up (with architecture diagram + code):
https://dev.to/zhangjj1988/building-a-unified-llm-api-gateway-what-i-learned-managing-5-model-providers-206o
Feedback welcome — how are you handling multi-LLM setups?
One OpenAI-compatible endpoint that routes to GPT-4o, Claude 3.5, Gemini 1.5 Pro, and DeepSeek V3. Stop wiring up 5 different APIs.
Tech stack:
Python / FastAPI · LiteSpeed reverse proxy · per-provider adapters (~150 lines each) · intelligent routing + observability layer
Key features / challenges:
• Unified ingress (OpenAI format, one auth)• Per-provider protocol adapters• Auto-routing with cost/availability + fallback on 429• Centralized provider key management
Write-up (with architecture diagram + code):
https://dev.to/zhangjj1988/building-a-unified-llm-api-gateway-what-i-learned-managing-5-model-providers-206o
Feedback welcome — how are you handling multi-LLM setups?