OpenFugu
Transparent multi-model routing for your own model pool.
An OpenAI-compatible gateway that orchestrates local and remote models with configurable policies. See every route, cost, latency, retry, verifier, fallback, and baseline comparison before the final answer is returned.
openfugu/default
A model orchestration layer, delivered as one endpoint.
OpenFugu sits in front of the model pool you already use. It does not hide dispatch behind a black box: policy, route choice, worker role, verifier result, fallback attempt, cost, latency, and fixed-model comparison become inspectable system output.
openfugu/default
Local runtimes
- llama.cpp
- Ollama
- LM Studio
Remote APIs
- OpenAI
- Claude
- Gemini
Open models
- DeepSeek
- Qwen
- Llama / Gemma
Private endpoints
- vLLM
- Internal APIs
- Team services
Turn each route into a replayable comparison sample.
What the route makes visible.
OpenAI-compatible
Drop into tools that already speak the OpenAI API without changing callers.
Transparent trace
See dispatch, workers, prompts, retries, verifier output, and intermediate results.
Cost and latency visible
Track estimated cost, token usage, timings, retries, and provider failures.
Verifier and fallback
Check risky answers and recover from timeouts, low confidence, or bad outputs.
Baseline comparison
Compare the route against a fixed model to measure real improvement.
Compose simple policies now. Train a conductor when the data exists.
The first OpenFugu beta should be useful before training: explicit policies, traceable plans, and eval replay from real user tasks.
cheapest-good-enough
Start with the lowest-cost worker that meets the quality bar.
verify-before-return
Let a stronger model check the answer before it reaches the caller.
fallback-chain
Retry with the next viable worker when confidence or reliability drops.
fixed-model-baseline
Measure orchestration against one fixed worker, not vibes.
Validate demand first, expand capability next.
OpenFugu validates whether transparent routing solves real problems before expanding prototypes, evals, and model support.
Demand validation
Confirm who truly needs transparent multi-model routing, which tools they use, and what model pools they operate.
Transparent routing prototype
Show route traces, cost records, latency records, failed retries, and fixed-model comparisons.
Beta trial
Invite users with concrete model pools and real tasks, then collect replayable eval samples.
Capability iteration
Use real feedback to prioritize strategies, worker/provider support, and deployment shapes.
Help decide the first OpenFugu beta.
Tell us which tools you use, which models belong in your pool, where fixed models fail, and whether you need a local client, Docker gateway, team service, or cloud option.