Local vs. cloud routing

Ollama

OpenRouter

The gateway holds two upstream pools: local, talking to Ollama, and cloud, talking to OpenRouter. Which one answers a given /v1/chat/completions call is decided by one exact-match check against the model id, nothing fuzzier:

cloud_pool if model in settings.cloud_chat_model_list else pool

cloud_chat_model_list defaults to a single entry: openrouter/free. Every other model id, including ids that happen to contain a /, like a local Ollama pull of a Hugging Face GGUF (hf.co/LiquidAI/LFM2-2.6B-GGUF:Q4_K_M) routes to the local pool. This is deliberately an exact-membership test, not a “contains a slash” or path-prefix heuristic: a paid frontier model is real money per call, so cloud routing only ever happens for a model id an operator put on the list on purpose.

Local pool

Default upstream: Ollama’s OpenAI-compatible endpoint on this project’s own hardware (http://127.0.0.1:11434/v1 by default; a dedicated GPU server replaces this without changing anything above the pool).
Catalogue is live: /v1/models reflects whatever Ollama actually has loaded, not a hard-coded list.
Default models: ministral-3:3b, gemma2:2b, llama3.2:3b, plus the nomic-embed-text embedding model.
Serves every /v1/embeddings call, OpenRouter has no embeddings offering, so embeddings never route to cloud.

Cloud pool

Default upstream: none. CLOUD_UPSTREAM_URLS is empty until an operator configures it, so cloud mode is inert out of the box rather than half-wired.
Catalogue is pinned, not discovered: /v1/models reports exactly the configured cloud_chat_model_list (openrouter/free by default), never OpenRouter’s full paid catalogue. Pinning is what keeps GPT- and Claude-family ids from ever becoming callable through this gateway just because OpenRouter’s own /models lists them.
A request naming any other cloud-shaped id gets the same model_not_allowed rejection as an unknown local id. See Errors.

What a key controls

A key’s allowed_models list is checked independently of this routing decision. A key scoped to only local models can never reach openrouter/free even though the model exists on the gateway, that call gets 403 model_not_allowed (“Key may not call model …”), not 400. Scope a key to openrouter/free explicitly if it should be allowed to use the cloud fallback.

curl https://ai.rcfi.co.ke/v1/chat/completions \
  -H "Authorization: Bearer tak_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openrouter/free",
    "messages": [{ "role": "user", "content": "Draft a two-line acknowledgement email." }]
  }'