Skip to content

feat: dynamic model listing, single enablement step, provider-level reasoning control - #608

Open
emrberk wants to merge 10 commits into
mainfrom
feat/dynamic-model-listing
Open

feat: dynamic model listing, single enablement step, provider-level reasoning control#608
emrberk wants to merge 10 commits into
mainfrom
feat/dynamic-model-listing

Conversation

@emrberk

@emrberk emrberk commented Sep 3, 2026

Copy link
Copy Markdown
Member

What

Replaces the fixed built-in model list with the providers' live /v1/models listings and removes the double-enablement step. One picker (Manage Models) now decides which models are enabled for every provider.

Dynamic listing + heuristics (modelCatalog.ts)

  • Availability comes from the provider listing; listModels() returns { id, label?, created? } (Anthropic maps display_name/created_at)
  • Labels derive from ids for OpenAI (gpt-5-mini → "GPT-5 Mini"); Anthropic uses display_name
  • OpenAI's default picker view is a blocklist plus the gpt-5-launch date gate (12 rows today — the gpt-5+ lineage minus research-tier -pro models, which reason for minutes with no streamed output); older generations and blocklisted ids stay reachable via "Show all models" and manual add — a new generation (gpt-6) appears with zero code change
  • A utility model for chat titles resolves from the listing (haikusonnet / lunananomini, newest first) and persists as utilityModel; context compaction runs on the selected model

Single enablement step

  • Manage Models works for built-in providers too; the settings modal body is a read-only summary
  • Models the provider removed show as "No longer available" and are dropped on Save (never silently)
  • "Validate API Key" is now a listing fetch (401 ⇒ invalid key); the chat-ping validation chain, testConnection, and the test-model flag are gone
  • Successful validation opens Manage Models directly since nothing is auto-enabled anymore

Reasoning: asymmetric two-state design

  • OpenAI gets one provider-level "Reasoning" dropdown — Default / High. High sends reasoning_effort: "high" to models gated by creation date (≥ gpt-5 launch), on chat and compaction only — never titles. A stateless 400-strip-retry covers any model that rejects it
  • Anthropic gets no control: modern Claude (4.7+/5.x) runs adaptive thinking by default
  • Legacy gpt-5.4@reasoning=… variants collapse to plain ids on load; a selected high variant folds into the new reasoningEffort setting; the whole @key=value model-string machinery is deleted
  • The slow-model/brain-icon story is removed

Anthropic temperature drop

Every 4.7+/5.x Claude model now rejects temperature with 400 "deprecated for this model" (this also breaks current main on claude-opus-4-7). Verified against all 11 listed models that requests without it succeed; the param is removed.

Back-compat

Plain stored ids keep working with no migration. The only rewrite is the reasoning-variant collapse in reconcileSettings. Alias ids (claude-sonnet-4-5) match their dated listing ids and are not flagged as removed.

Verification

  • yarn typecheck clean; 2012 unit tests pass (49 new for modelCatalog + settings)
  • All 75 AI Cypress tests pass (specs updated to the new flows; validation mocks are now GET listings)
  • Live against both real provider APIs: the exact request shapes pass on all 11 Anthropic models (chat + countTokens) and all 30 listed OpenAI chat models (Default and High variants, pro models and gpt-3.5/4-era included); on the wire, chat carries reasoning: {effort: "high"} on gpt-5.4 while the title request runs on gpt-5.6-luna with no effort param
  • e2e/commands.js now reads Cypress.config("baseUrl") instead of hardcoding port 9999, so CYPRESS_BASE_URL works locally; CI is unaffected

🤖 Generated with Claude Code

emrberk and others added 10 commits September 3, 2026 19:32
…easoning control

Replace the fixed built-in model list with the providers' live /v1/models
listings. Availability now comes from the listing; pure heuristics derive
labels, filter OpenAI noise (blocklist-only, with a Show-all escape hatch),
resolve a utility model for chat titles (haiku->sonnet / luna->nano->mini,
newest first), and gate reasoning-capable OpenAI models by creation date.

- Manage Models becomes the single enablement step for every provider; the
  settings modal body is a read-only summary and flags models the provider
  removed ("No longer available", dropped on Save)
- API key validation is a listing fetch; the chat-ping validation chain and
  the test-model flag are gone
- Reasoning simplifies to one provider-level Default/High dropdown for
  OpenAI (applied to chat and compaction, never titles, with a stateless
  400-strip-retry); Anthropic needs no control since modern Claude runs
  adaptive thinking by default. Legacy @reasoning= variants collapse to
  plain ids on load and a selected high variant folds into the new setting
- Context compaction runs on the selected model; titles use the utility model
- Drop temperature from Anthropic requests: every 4.7+/5.x model now rejects
  it as deprecated (verified against all 11 listed models)
- Remove the slow-model/brain-icon story

Verified live against both provider APIs: request shapes pass on all 11
Anthropic and all 30 listed OpenAI chat models (Default and High).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The picker's default view now shares the gpt-5-launch date gate with the
reasoning control: older generations (gpt-3.5/4/4o/o-series) move behind
the "Show all models" escape hatch and stay manually addable. 16 rows
against today's live listing instead of 30.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
gpt-5-pro and friends reason for minutes and stream no output or reasoning
summaries even with summary auto (verified live) — in an interactive chat
they read as unresponsive. They stay reachable via Show all models and
manual add. 12 default rows against today's listing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@emrberk

emrberk commented Sep 6, 2026

Copy link
Copy Markdown
Member Author

Reviewing PR #608 at level 3.

PR #608 Review

Review target

Field Value
PR #608
Title feat: dynamic model listing, single enablement step, provider-level reasoning control
Base revision 72ca5487b263fba39ecd2e9221066b03535b4c75
Reviewed revision a61cda50b4ae1ecf2de2bb5e0634b0a232f83331
Change size 39 files, +4,223 / −1,785
Primary scope AI provider configuration, dynamic model discovery, model-value migration, provider-level reasoning, and settings dialogs
Metadata assessment The title follows Conventional Commits and the description explains end-user impact.

This level-3 review covered the changed AI settings and provider contracts, their repository-wide callsites, local-storage migration behavior, async fallback paths, and browser interactions. Typecheck, build, lint, the full unit suite, the provider-setup browser suite, and targeted browser reproductions were executed against the reviewed revision.

Issues

Issue ID Issue name Category Severity Location Net impact Evidence Description Steps to reproduce Suggested fix
#1 Reasoning draft overwrites fallback React correctness & hooks Moderate in-diff — SettingsModal.tsx:492 OpenAI High users incur repeated rejected requests after saving settings Targeted Cypress at HEAD a61cda50: /tmp/qdb-pr608.hPTr2h/e2e/tests/console/pr608-review.spec.js:42; 2/2 passed. BASE: N/A — provider-level reasoning fallback is a new surface. The modal initializes reasoningEffort once. If an in-flight request rejects High reasoning while the modal is open, the fallback persists default, but the draft remains high. Saving the modal writes the stale value back, silently undoing the downgrade and causing later provider instances to attempt the rejected setting again.
  • Select High reasoning.
  • Send a request whose reasoning rejection is delayed.
  • Open Settings before the rejection arrives.
  • Wait for the fallback to persist Default.
  • Click Save; persisted settings return to High.
Reconcile provider reasoning changes into the modal draft while it is open. Track whether the user actually edited the field so an external fallback update replaces untouched draft state without overwriting intentional edits.
#2 Manage Models loses focus Accessibility & UX Moderate in-diff — SettingsModal.tsx:1085 Keyboard users lose their place after every Manage Models close Targeted Cypress at HEAD a61cda50: /tmp/qdb-pr608.hPTr2h/e2e/tests/console/pr608-review.spec.js:11; observed document.activeElement === document.body; 2/2 passed. BASE: N/A — built-in Manage Models is a new surface. Opening Manage Models sets the parent dialog's open prop to false, unmounting the trigger. When the child closes, Radix has no mounted trigger to restore focus to; the parent remounts with focus on <body>.
  • Open Assistant Settings using the keyboard.
  • Activate Manage Models.
  • Close it with Cancel or Escape.
  • Inspect focus or press Tab; focus did not return to Manage Models.
Keep the parent dialog mounted while the child is open, or retain a trigger ref and explicitly focus it after the parent remounts. Add a Cypress assertion over document.activeElement.

Summary

  • Verdict: approve with comments — address wip test for error range #1 and init react components package #2, but neither is blocking.
  • Correctness gate: pass — no admitted Critical findings.
  • Test gate: pass — 0 admitted coverage gaps.
  • Quality gates passed: typecheck, build, lint, and 2,077 unit tests.
  • Browser gate passed: aiProviderSetup.spec.js 11/11 and targeted review artifacts 2/2.
  • Severity: 0 Critical, 2 Moderate, 0 Minor.
  • Split: 2 in-diff, 0 out-of-diff; the cross-context pass found the regressions contained within the changed AI settings paths.
  • Regressions/tradeoffs: asynchronous reasoning recovery can be overwritten by an already-open settings draft, and the new nested model-management flow does not restore keyboard focus.
  • PR title and description meet the metadata conventions.
  • Adjacent findings: none.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant