Research Assistant

The group's question was why large matrix multiplications dominate the cost of training and running LLMs, and how close you can get to peak hardware by hand. That meant getting underneath attention mechanisms and transformer architectures to the linear algebra that actually eats the compute.

I wrote CUDA kernels for matrix multiply and tuned them against cuBLAS, NVIDIA's own library, landing at 93.7% of its throughput. Getting there meant living in NVIDIA Nsight Compute and reading the profiler instead of guessing, then going all the way down to the PTX assembly to see what the compiler's IR passes were really doing to my code.

The kernels were only half of it. I wrote a read-along guide to GPU matrix-multiply optimization that builds the mental model from the execution hierarchy up — thread blocks, warps, SM scheduling — so someone new to the GPU can follow the reasoning behind each optimization rather than just copy it.