fix(runtime): bound competing local inference and prioritize human requests #507
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "fix/local-main-request-admission"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Productive failure, not a synthetic speed test
A fresh Minerva Matrix input with30characters took19min34sec BEFORE its first two tool_search calls (5ms/1ms). Actual native+gateway:300.004s timeout,2s backoff,872.329s retry with6694prompt/374outputtokens. No compaction. DeepSeek served simultaneous large cold prefills and decoded~0.7-0.8t/s. Core scheduling headers alone do not admit/reserve capacity.
Explicit, installation-owned mitigation
Real SDK HTTP tests cover alias-sharing, full-stream (not headers) lifetime, actual human preference, classifier and review-clone sharing, independent models, cancellation/expiry fencing, and nested same-model request during a tool without deadlock. Scheduler tests cover fairness and exact-once release.
Typecheck/build PASS. Full suite2658passed, one existing optional skip, zero failures. Independent review PASS(noP0/P1); prior P2 fairness/unresolved-rule issues addressed. Current non-blocking P2: a bad/unresolved explicit rule fails closed for the installation rather than allowing unrelated requests; valid Minerva refs locally preflighted. Exact-head CI required before merge.
Scope is one runtime, NOT shared/global admission or a guarantee against another agent's inference. Existing requests may still have to settle. Production end-to-end latency must be remeasured after separately reviewed Minerva configuration activation.