Posts
-
Skipping attention blocks was the easy part Yash Agarwal #gpu#attention#kernels#modular BLASST block-sparse attention plus one TMEM address swap: −41.9% attention-kernel time at 95% skip and −20% TTFT at 64K with accuracy above dense — an optimization worklog on a B200.
-
DSpark on MAX: making speculative decoding beat vLLM Yash Agarwal #speculative-decoding#inference#gpu#modular How speculative decoding for Gemma4 on Modular's MAX went from 30% behind vLLM under load to outside its latency/throughput curve on every dataset at concurrency 8 and 64, on one B200.
-
How I beat NVIDIA at allocating pinned host memory Yash Agarwal #cuda#linux#performance#modular 9× faster than cuMemAllocHost, and faster than CUDA's own VMM API too — an optimization worklog on an 8×B200 host.