Provider Live Contract Verification Matrix
任务:
P2-W7-019(P1/P2-CONTRACT-LIVEalias) 更新时间: 2026-05-04 范围: OpenRouter application 与 public beta go/no-go 所需 provider compatibility evidence 重要限制: 本文件不伪造 live evidence。未在本次会话以真实凭证执行的项目标记为not_run或blocked。
执行规则
- Live tests 默认关闭，必须设置
LIVE_PROVIDER_CONTRACTS=1。 - Provider secrets 只从 GCP Secret Manager 注入运行环境；本机
.env仅限一次性 live smoke/dev shell，且不得提交、截图、贴入聊天或写入 diagnostics。测试日志不得输出 secret value。 tests/providers/provider-contract-live.test.ts是聚合 smoke harness；既有 provider-specific contract tests 继续保留。- Blocked provider 必须保留 fallback decision，避免 public beta 依赖无法取得的供应商。
当前 Matrix
| Provider | Model | Credential status | Chat completion | Streaming SSE | Tools/function calling | JSON mode | Usage fields | Error mapping | Pricing/cost extraction | Public beta fallback decision |
|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek | deepseek-v4-pro |
obtained | ⚠️ partial (2026-05-12) | ✅ pass (2026-05-12) | not_run | not_run | adapter_expected | unit/adapter mapping | adapter tokenCost() |
Include as primary — adapter returns canonical shape; API content filter blocked trivial prompt (content empty) but stream passes. |
| Qwen / DashScope | qwen3-235b |
obtained ❌ key_rejected | ❌ auth (2026-05-12) | ❌ auth (2026-05-12) | not_run | not_run | gated | unit/adapter mapping | gated | Key rejected — Alibaba-USeast1-API-KEY returned 401 "Incorrect API key provided" on DashScope compatible-mode endpoint. Key may be expired, region-mismatched, or formatted incorrectly. Founder must verify key in Alibaba Cloud console. |
| Moonshot Kimi | kimi-k2.6 |
obtained | not_run | not_run | not_run | not_run | gated | unit/adapter mapping | gated | Previous kimi-k2 mismatch was resolved in adapter/config by switching to api.moonshot.ai + kimi-k2.6; live smoke still needs Founder-gated rerun before BETA availability claim. |
| Zhipu GLM | glm-5 |
pending | blocked | blocked | blocked | blocked | blocked | adapter mapping pending live | adapter tokenCost() |
Hold until business verification and key are available. |
| Baidu Ernie | ernie-5.0 |
blocked | blocked | blocked | blocked | blocked | blocked | adapter mapping pending live | adapter tokenCost() |
Exclude from beta until credential path exists. |
| ByteDance Doubao | doubao-pro |
blocked | blocked | blocked | blocked | blocked | blocked | adapter mapping pending live | adapter tokenCost() |
Exclude from beta until credential path exists. |
Live Smoke Execution Log (2026-05-12 23:01 UTC)
LIVE_PROVIDER_CONTRACTS=1 npm run test:provider-live
DeepSeek: chat 200 OK → content="" (content-filtered by DeepSeek API on trivial prompt) | stream SSE → chunks valid
Qwen: chat 401 → "Incorrect API key provided" | stream 401
Kimi: previous chat/stream 404 was for legacy `kimi-k2` on `api.moonshot.cn`; rerun pending for `kimi-k2.6` on `api.moonshot.ai`
GLM: skipped (no API key — pending business verification)
Ernie: skipped (no credentials)
Doubao: skipped (no credentials)
Updated evidence status: DeepSeek adapter connects and translates responses correctly (stream passes; chat content-filtered by provider, not adapter bug). Qwen key requires Founder verification in Alibaba Cloud console. Kimi adapter/config now use kimi-k2.6 on api.moonshot.ai; live smoke rerun is still required before BETA availability claim.
Harness
Primary command:
npm run test:provider-live
This runs governance checks only unless live execution is explicitly enabled.
Live command shape:
LIVE_PROVIDER_CONTRACTS=1 npm run test:provider-live
Expected env vars and Secret Manager names:
| Provider | Env vars | Secret Manager names |
|---|---|---|
| DeepSeek | DEEPSEEK_API_KEY |
DeepSeek-API |
| Qwen / DashScope | DASHSCOPE_API_KEY |
Alibaba-USeast1-API-KEY |
| Moonshot Kimi | MOONSHOT_API_KEY |
Moonshot-KIMI-API |
| Zhipu GLM | GLM_API_KEY |
GLM_API_KEY after approval |
| Baidu Ernie | ERNIE_API_KEY, ERNIE_SECRET_KEY |
create only after account path exists |
| ByteDance Doubao | DOUBAO_API_KEY |
create only after account path exists |
Internal AIOCANA service credentials are not provider credentials. AIOCANA_MAIN_CONTROL_AINA_API_KEY must live in GCP Secret Manager only; AIOrouter stores only its SHA-256 lookup hash in PostgreSQL. Do not create scripts/internal-key.env or SQL files containing plaintext internal keys.
Before live execution, run npm run providers:credential-status to confirm inventory without reading secret values. Safe injection steps are in docs/provider-credential-acquisition-runbook.md.
Evidence Status
| Evidence item | Current status | Source |
|---|---|---|
| Provider credential status | documented + inventory checked | docs/provider-credentials-registry.md, scripts/provider-credential-status.mjs, diagnostics/local-artifacts/provider-credential-status-2026-05-04T22-30-25-834Z.json |
| Model ID accepted by provider | gated | tests/providers/provider-contract-live.test.ts with LIVE_PROVIDER_CONTRACTS=1 |
| Chat completion canonical shape | gated | live harness + provider-specific contract tests |
| Streaming SSE canonical chunks | gated | live harness + provider-specific contract tests |
| Usage field shape | gated | live harness asserts prompt_tokens, completion_tokens, total_tokens |
| Pricing/cost extraction | implemented | adapter tokenCost() smoke in live harness |
| Auth/rate-limit/content-filter mapping | partial | adapter unit behavior exists; destructive live negative tests not run by default |
| Tools/function calling | pending | provider-specific live tests needed after positive smoke passes |
| JSON mode | pending | provider-specific live tests needed after positive smoke passes |
OpenRouter / Public Beta Readiness Impact
- Public beta can proceed only with providers whose live smoke passes in the target environment.
- Current docs support a conservative beta provider set of DeepSeek, Qwen, and Kimi after live smoke execution.
- GLM is pending account approval; Ernie and Doubao are excluded until credential blockers are resolved.
- The OpenRouter application should cite this file plus
docs/openrouter/application-evidence.md; it should not claim live compatibility for providers whose row remainsnot_runorblocked.
Next Actions
- Founder confirms whether live provider spend is approved for smoke testing.
- Export provider secrets from GCP Secret Manager into the local environment or CI secret context.
- Run
LIVE_PROVIDER_CONTRACTS=1 npm run test:provider-live. - Replace
not_runcells with timestamped pass/fail evidence and link diagnostics artifacts. - Add provider-specific JSON mode and tool/function calling tests only for providers that advertise support.