AI Inference Platform & GPU Operations Control Plane

GPU fleet telemetry · model routing · request queue · failure/failover tracking · cost analysis 1 LAB GPU 31 SIMULATED GPUs

Fleet Overview

GPU ID	Source	Model	Utilization	VRAM	Temp	Power	Health	Location

Model	Backend	Requests	Error Rate	p50 (ms)	p95 (ms)	p99 (ms)	TTFT (ms)	Tokens/sec	Throughput

Latency percentiles computed from individual request records in SQLite — not hardcoded.

Model	Users	Req/s	p50	p95	p99	Err%

Reproducible load test: scripts/run_load_test.py · raw JSON in results/

Time	Type	GPU	Failed	Latency Impact

Time	From →	To	Reason	Detect (ms)	Failover (ms)

Type	Injection	Detection	Failover	Failed Reqs	Before (ms)	After (ms)	Δ Latency

Reproducible: scripts/run_failure_scenario.py · raw JSON in results/failure_scenarios.json

Model	GPU $/hr	Tokens	$/1M tokens	$/request

Computed from actual request tokens + representative GPU pricing (see README).

Data sources: LAB 1 real GPU (RTX 4090 config) · SIMULATED 31 synthetic fleet GPUs · MEASURED load tests & failure scenarios