SPS fleet — hardware and GPU lane allocation

Generated 2026-08-24 17:05 UTC by Workstation Hub/scripts/fleet_inventory.py. Do not hand-edit — re-run the script. This page exists because hand-written hardware docs go stale silently; every field below was read off the machine.

The allocation principle

The fast card serves what someone is waiting on. The slow card serves what can wait.

Coding is interactive — bots and people block on it — so it owns the 5090. Vision, reranking and embeddings are latency-tolerant, so they live on the slower Arc B70 and on srv3, and cost nothing.

sps-srv1 (vigil-server)

Production AI server — gateway, Mattermost, monitoring, coder + vision + rerank lanes

GPUs

GPUs (Intel Arc — VRAM not exposed via sysfs)

• card1 0xe223

Serving lanes

• lsv-b70-rerank — --served-model-name bge-reranker-v2-m3 --runner pooling --port 8022 --gpu-memory-utilization 0.18 --max-model-len 4096
• lsv-b70-vlm — --served-model-name qwen2.5-vl-7b --pipeline-parallel-size 1 --port 8023 --gpu-memory-utilization 0.70 --max-model-len 16384
• lsv-5090 — --model Inferact/Qwen3.8-27B-NVFP4 --served-model-name qwen3.8-27b --port 8021 --max-model-len 81920 --gpu-memory-utilization 0.96

Containers (64): agoge-platform, breakglass-vault, case-livekit-livekit-1, case-livekit-livekit-redis-1, case-platform-redis, cloudbeaver, cloudbeaver-demo, code-oliver, coder, coder-raghav-seat, coder-teagan-localtest, coder-teagan-seat, course-mariadb, dash-msu, dash-personal, dl-nginx, family-recipe-box-api-1, firstread, gitea, glitchtip-postgres-1, glitchtip-redis-1, glitchtip-web-1, glitchtip-worker-1, homarr, homepage, immich-ml, immich-postgres, immich-redis, immich-server, kokoro-tts, litellm, lsv-5090, lsv-b70-rerank, lsv-b70-vlm, mattermost, mattermost-postgres, ntfy, nwz-api, nwz-mobile-api, nwz-postgres, shotsmith, sns-books-postgres, str-ops-backend, str-ops-db, str-ops-frontend, ta-review-auth, ta-review-db, ta-review-gateway, ta-review-rest, timekeep-backend, timekeep-postgres, tutorial-engine, uptime-kuma, vigil-alertmanager, vigil-cadvisor, vigil-dcgm-exporter, vigil-grafana, vigil-ledger-api, vigil-node-exporter, vigil-postgres, vigil-prometheus, vigil-qdrant, vigil-redis, wyrdlyre-art

sps-srv2

Inference appliance — serves Llama-3.3-70B, ~88% of gateway traffic

GPUs (Intel Arc — VRAM not exposed via sysfs)

• card1 0xe211
• card2 0xe211

Serving lanes

• lsv — --served-model-name Llama-3.3-70B-Instruct --pipeline-parallel-size 2 --port 8000 --gpu-memory-utilization 0.95 --max-model-len 8192

Containers (1): lsv

sps-srv3

Offload tier — fleet embeddings, Agoge backend

GPUs

Containers (1): agoge-backend

Ollama resident: nomic-embed-text:latest

mac-mini1

Voice node — Whisper STT + Kokoro TTS; the only macOS in the fleet

Listening services

• 8090 http=200 listeners=1
• 8091 http=000 listeners=0
• 8092 http=404 listeners=1

What each model is for

Probed values above are live. This table is maintained by hand — a machine can tell you a model is loaded, not why it exists. Consumers were verified against live configs, not assumed.

qwen3.8-27b

Runs on: sps-srv1 · RTX 5090 · lsv-5090:8021

Purpose. The coding lane. Agentic code generation and tool use at 81,920 context — sized to clear the ~66.5K average real session.

Used by: Coder seats (/opt/coder/templates/sps-seat/main.tf), NWZ agents (/etc/nwz/env)

Why there: Interactive: people and bots block on it, so it gets the fast card.

qwen3.8-27b-strict

Runs on: sps-srv1 · RTX 5090 · same backend

Purpose. Same model, deliberately absent from every fallback block. A no-paid-models sandbox that quietly answers from Sonnet is worse than one that fails, so this alias fails loudly or not at all.

Used by: reesarth/qwen and teagan/qwen Coder workspaces

Why there: Key scoping alone was not trusted to prevent a paid fallback.

Llama-3.3-70B-Instruct → gateway alias llama-3.3-70b

Runs on: sps-srv2 · 2× Arc Pro B60 · pipeline-parallel 2 · util 0.95

Purpose. The general-purpose workhorse — roughly 88% of all gateway traffic. Chat, reasoning, summarisation, triage.

Used by: Mattermost agents (sps-agent-router), company Vigil Q&A (vigil-ask), product personas, NWZ ticket triage, Vigil Steward orchestrator (ORCHESTRATOR_MODEL)

Why there: Needs both B60s; nothing else in the fleet fits a 70B.

qwen2.5-vl-7b

Runs on: sps-srv1 · Arc Pro B70 · lsv-b70-vlm:8023 · util 0.70

Purpose. Vision. Turns image attachments into text before they reach the answering model — Llama-3.3-70B has no vision, and text is what the conversation store can carry forward.

Used by: Mattermost agents (attachments), document ingestion via the local-only doc-ingest-vision key

Why there: Latency-tolerant — nobody waits on an attachment description — and it replaces a metered Claude call with $0.

bge-reranker-v2-m3

Runs on: sps-srv1 · Arc Pro B70 · lsv-b70-rerank:8022 · util 0.18

Purpose. Cross-encoder reranking. Reads question and passage together and reorders retrieved chunks, which embedding similarity cannot do. Retrieval pulls 24 candidates; this picks what reaches the prompt.

Used by: vigil-ask, sps-agent-router (both RAG paths)

Why there: Small, latency-tolerant, and it measurably improves answers.

nomic-embed-text

Runs on: sps-srv3 · RTX 2060 SUPER · Ollama:11434

Purpose. Embeddings — 768-dim vectors for every semantic search in the fleet. One pass per document at ingest, one per question at query.

Used by: vigil_sps_brain (SPS knowledge, vigil-ask + agents), vigil_memory + persona_facebook_photos (Vigil Steward), docs_vault (vault-ingest)

Why there: srv1's 5090 leaves 429 MiB — the model just fits, so it loaded intermittently. srv3's cards are idle and it is 27 ms warm.

llama3.1:8b

Runs on: sps-srv3 · Ollama:11434

Purpose. Utility model. Mechanical small jobs — summarising an advert, tidying a label, classifying a line — where a 70B buys nothing.

Used by: Vigil Steward (UTILITY_MODEL, OLLAMA_UTILITY_BASE_URL)

Why there: Deliberately off the 70B route so a one-line summary cannot be blocked by sps-srv2 being off the network.

Kokoro-82M → gateway alias kokoro-tts

Runs on: sps-srv1 · CPU · kokoro-tts:8007 → LiteLLM

Purpose. Text-to-speech, OpenAI-compatible /v1/audio/speech. Measured 2.91× realtime on CPU (2026-08-24).

Used by: MSU course narration, any TTS through the gateway

Why there: ⚠️ Runs the kokoro-fastapi-cpu image with no GPU attached — it is on srv1's CPU, not the 5090. It costs no VRAM, which is why it can coexist with the coder lane. The Mac mini copy (:8091, 3.8× realtime) is currently DOWN and is the duplicate.

Chatterbox TTS (ResembleAI)

Runs on: mac-mini1 · M4 Pro · :8092 → gateway alias chatterbox-tts

Purpose. The BACKUP TTS, and the voice-cloning option. OpenAI-compatible /v1/audio/speech, model id chatterbox-tts-1.

Used by: Gateway alias chatterbox-tts, wired 2026-08-24. Callers ask for it by name: LiteLLM's fallbacks cover chat completions, not audio, so there is NO automatic failover from kokoro-tts.

Why there: Stood up 2026-08-09 — the same day Kokoro was retired from this box — then left reachable by nothing until 2026-08-24. Slower than Kokoro (~11s vs sub-second through the gateway for the same sentence), so it is the backup rather than the default. voices.json is still empty, so no cloned voice is configured yet. Weights live in ~/.cache/huggingface (3.0 GB), which is why ~/chatterbox/models looks empty and misleads.

Whisper ggml-large-v3-turbo

Runs on: mac-mini1 · M4 Pro · :8090

Purpose. Speech-to-text, ~11× realtime. Feed it 16 kHz mono WAV — there is no --convert.

Used by: Voice capture for Vigil Steward / Iris, MSU transcription

Why there: Apple Silicon is genuinely fast at this and the box is otherwise idle. Only macOS in the fleet.

claude-haiku-4-5 / sonnet-5 / opus-5

Runs on: Anthropic API via LiteLLM :9002 — metered

Purpose. Paid fallback when a local lane is down, plus jobs local models are not good enough for. Haiku is the fallback target everywhere because it always answers and never silently upgrades cost.

Used by: msu-inbox triage, fallback for every local alias, print-gateway image triage

Why there: Deliberate: the fallback keeps the gateway answering during an outage. PaidFallbackFiring alerts so it is visible, not silent.

Sources of truth this page does not replace: ~/dev-context/fleet-capabilities.md (capability → endpoint lookup), Workstation Hub/LINKS.md (URLs), and each project's CLAUDE.md.