Skip to content

feat: add OrcaRouter as a named LLM provider - #296

Open
clementguarino06510-glitch wants to merge 1 commit into
plasma-umass:masterfrom
clementguarino06510-glitch:add-orcarouter-provider
Open

feat: add OrcaRouter as a named LLM provider#296
clementguarino06510-glitch wants to merge 1 commit into
plasma-umass:masterfrom
clementguarino06510-glitch:add-orcarouter-provider

Conversation

@clementguarino06510-glitch

Copy link
Copy Markdown

Add OrcaRouter as a named LLM provider

Coz supports an AI optimization assistant in the viewer and an LLM-powered coz suggest-points agent, both backed by a small set of named providers (Anthropic, OpenAI, Amazon Bedrock, Ollama). This PR adds OrcaRouter as a first-class provider so Coz users can point either feature at OrcaRouter's OpenAI-compatible gateway without treating it as an anonymous custom base URL.

OrcaRouter is an OpenAI-compatible AI gateway built for both models and agents. Like OpenRouter, it exposes a provider/model namespace across many models — but it also combines adaptive routing, automatic failover, zero-markup inference, observability, guardrails, and agent-tool governance behind the same endpoint. Adding it as a named provider means this project's users can use that stack directly. It also runs gateway-level, zero-trust security for AI agents on the same endpoint — screening every prompt/response and governing every tool call on a default-deny basis, with no application code changes.

Changes mirror the existing OpenAI wiring (OrcaRouter exposes an OpenAI-compatible /v1/chat/completions endpoint; model ids are namespaced, e.g. openai/gpt-4o-mini or orcarouter/auto):

  • coz: reads ORCAROUTER_API_KEY in llm_config, serves a /orcarouter-models endpoint that lists curated models from the gateway's catalog, adds _stream_orcarouter for the viewer's /optimize streaming path, and adds _orcarouter_tool_request / _suggest_run_orcarouter_agent for the coz suggest-points tool-calling agent.
  • viewer/: adds an OrcaRouter entry to the provider selector, wires the model dropdown to /orcarouter-models with fallback models, and persists the key in the existing cookie settings.
  • README.md: adds OrcaRouter to the supported-providers list, the suggest-points provider table, and the usage examples.
  • orcarouter-live-test.py: a script that exercises the new code paths (tool-calling request, /optimize streaming, /orcarouter-models) against the live gateway.

Verification:

  • python3 -m py_compile coz and the viewer TypeScript build (tsc) both pass.
  • Live against the OrcaRouter gateway with a real key: the tool-calling request returns a chat completion, the viewer /optimize stream produces a full streaming reply, and /orcarouter-models lists the curated models.
  • coz suggest-points --provider orcarouter --model openai/gpt-4o-mini --dry-run runs the full agent loop end-to-end.

Discord: discord.gg/YEubt8enRA · X: https://x.com/OrcaRouter

I'm an engineer on the OrcaRouter team.

Add OrcaRouter as a first-class LLM provider alongside Anthropic, OpenAI,
Amazon Bedrock, and Ollama in both the viewer's AI optimization assistant
and the coz suggest-points agent.

OrcaRouter exposes an OpenAI-compatible /v1/chat/completions endpoint, so
the streaming path and the tool-calling agent mirror the existing OpenAI
provider. Model ids are namespaced (e.g. openai/gpt-4o-mini,
orcarouter/auto).

- coz: ORCAROUTER_API_KEY in llm_config, /orcarouter-models endpoint,
  _stream_orcarouter for the /optimize streaming path, and
  _orcarouter_tool_request/_suggest_run_orcarouter_agent for
  suggest-points.
- viewer: OrcaRouter entry in the provider selector, model dropdown
  fetch, fallback models, and settings persistence.
- README: provider list, suggest-points table, and example.
- Add orcarouter-live-test.py exercising the real code paths.

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant