Research
I work on the science of deep learning. I've had the privilege of learning to do research from Sam Gershman and Aditi Raghunathan.
I'm interested in understanding what systems centered around foundation models will look like in 5 years, and how they will touch our lives in unthinkable ways. This usually involves studying these models in a scientific way, running experiments across the stack: from pretraining to evals and beyond.
My path into deep learning was a meandering one. Early on in college I was entranced by pure math and neuroscience, and both led me to machine learning, albeit from opposite directions. Eventually, my research interests converged to what they are now. I am easily fascinated, and have published work on topics ranging from language model pretraining to quantization to high-dimensional probability and mouse olfaction.
- Scaling Laws for Precision ICLR 2025. Oral.
- Do Mice Grok? Unveiling Hidden Progress in Sensory Cortex During Overtraining ICLR 2025.
- Lower Data Diversity Accelerates Training: Case Studies in Synthetic Tasks Preprint.
- Asymptotic Dynamics for Delayed Feature Learning on a Toy Model HiLD at ICML 2024.
- No Free Prune: Information-Theoretic Barriers to Pruning at Initialization ICML 2024.
- Grokking as the Transition from Lazy to Rich Training Dynamics ICLR 2024.
- Human or Machine? Turing Tests for Vision and Language Preprint.
Tanishq Kumar*, Zachary Ankner*, Benjamin F. Spector, Blake Bordelon, Niklas Muennighoff, Mansheej Paul, Cengiz Pehlevan, Christopher Ré, Aditi Raghunathan.
Tanishq Kumar, Blake Bordelon, Cengiz Pehlevan, Venkatesh Murthy, Samuel J. Gershman.
Suhas Kotha*, Uzay Girit*, Tanishq Kumar*, Gaurav Ghosal, Aditi Raghunathan.
Blake Bordelon, Tanishq Kumar, Samuel J. Gershman, and Cengiz Pehlevan.
Tanishq Kumar*, Kevin Luo*, Mark Sellke.
Tanishq Kumar, Blake Bordelon, Samuel J. Gershman*, Cengiz Pehlevan*.
Mengmi Zhang, ... Tanishq Kumar, ... Gabriel Kreiman.