Tag Archive
performance
14 posts found
Your Model Isn’t the Bottleneck. Your Queue Is.
A classifier consumed 0.3% of request time; the queue ate the rest. Why AI deployments fail around the queue, not the model.
Cache Invalidation via Surrogate Keys: Edge Architecture and Purge Mechanics
Decouple cache invalidation from URL paths using Surrogate-Key headers for targeted CDN purging.
Why MoE Models Stream From NVMe: Kernel Prefetch, Read-Ahead, and Async I/O
Frontier MoE models run on consumer hardware by streaming expert weights from disk. Not compression—kernel I/O prefetching overlapped with GPU compute. Here's h…
Kernel Socket Tuning for High-Concurrency Load Balancers: TCP_NODELAY, TCP_FASTOPEN, and Listen Backlog
Disable Nagle, enable TCP Fast Open, tune listen backlog. Three socket parameters cut tail latencies 40–60% on modern load balancers.