fix(runtime): bound competing local inference and prioritize human requests #507

Merged
karsten merged 1 commit from fix/local-main-request-admission into main 2026-11-16 06:00:21 +00:00
Owner

Productive failure, not a synthetic speed test

A fresh Minerva Matrix input with30characters took19min34sec BEFORE its first two tool_search calls (5ms/1ms). Actual native+gateway:300.004s timeout,2s backoff,872.329s retry with6694prompt/374outputtokens. No compaction. DeepSeek served simultaneous large cold prefills and decoded~0.7-0.8t/s. Core scheduling headers alone do not admit/reserve capacity.

Explicit, installation-owned mitigation

  • Optional localModelScheduling:[{model:provider/id,concurrency:1}], absent=unchanged. No token gate/window reduction, extra inference, model/thinking switch, tool/effect/intent change, or global hardware QoS claim.
  • All same physical endpoint+id aliases in one runtime share permits across Main/workers/subagents/native summaries/HTTP classifier/planner/code review/review worktrees. Other models stay independent.
  • A permit spans ONE complete model response, never the following tools/assessment. Human waiters outrank background, with bounded four-response fairness for waiting background. Existing inference is not aborted.
  • Wait BEFORE SDK stream/HTTP timer creation; cancellation and owner/expiry check before send; no queued-cancellation HTTP, no effect or classifier replay.
  • Content-free local queue events in Matrix/native activity,TUI,API distinguish capacity waiting from unknown provider progress; queue events do not reset own semantic inactivity.

Real SDK HTTP tests cover alias-sharing, full-stream (not headers) lifetime, actual human preference, classifier and review-clone sharing, independent models, cancellation/expiry fencing, and nested same-model request during a tool without deadlock. Scheduler tests cover fairness and exact-once release.

Typecheck/build PASS. Full suite2658passed, one existing optional skip, zero failures. Independent review PASS(noP0/P1); prior P2 fairness/unresolved-rule issues addressed. Current non-blocking P2: a bad/unresolved explicit rule fails closed for the installation rather than allowing unrelated requests; valid Minerva refs locally preflighted. Exact-head CI required before merge.

Scope is one runtime, NOT shared/global admission or a guarantee against another agent's inference. Existing requests may still have to settle. Production end-to-end latency must be remeasured after separately reviewed Minerva configuration activation.

## Productive failure, not a synthetic speed test A fresh Minerva Matrix input with30characters took19min34sec BEFORE its first two tool_search calls (5ms/1ms). Actual native+gateway:300.004s timeout,2s backoff,872.329s retry with6694prompt/374outputtokens. No compaction. DeepSeek served simultaneous large cold prefills and decoded~0.7-0.8t/s. Core scheduling headers alone do not admit/reserve capacity. ## Explicit, installation-owned mitigation - Optional localModelScheduling:[{model:provider/id,concurrency:1}], absent=unchanged. No token gate/window reduction, extra inference, model/thinking switch, tool/effect/intent change, or global hardware QoS claim. - All same physical endpoint+id aliases in one runtime share permits across Main/workers/subagents/native summaries/HTTP classifier/planner/code review/review worktrees. Other models stay independent. - A permit spans ONE complete model response, never the following tools/assessment. Human waiters outrank background, with bounded four-response fairness for waiting background. Existing inference is not aborted. - Wait BEFORE SDK stream/HTTP timer creation; cancellation and owner/expiry check before send; no queued-cancellation HTTP, no effect or classifier replay. - Content-free local queue events in Matrix/native activity,TUI,API distinguish capacity waiting from unknown provider progress; queue events do not reset own semantic inactivity. Real SDK HTTP tests cover alias-sharing, full-stream (not headers) lifetime, actual human preference, classifier and review-clone sharing, independent models, cancellation/expiry fencing, and nested same-model request during a tool without deadlock. Scheduler tests cover fairness and exact-once release. Typecheck/build PASS. Full suite2658passed, one existing optional skip, zero failures. Independent review PASS(noP0/P1); prior P2 fairness/unresolved-rule issues addressed. Current non-blocking P2: a bad/unresolved explicit rule fails closed for the installation rather than allowing unrelated requests; valid Minerva refs locally preflighted. Exact-head CI required before merge. Scope is one runtime, NOT shared/global admission or a guarantee against another agent's inference. Existing requests may still have to settle. Production end-to-end latency must be remeasured after separately reviewed Minerva configuration activation.
fix(runtime): bound competing local inference and prioritize human requests
All checks were successful
decision-interface-ci / full-validation (pull_request) Successful in 3m39s
50912ae579
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
harness/core!507
No description provided.