Research
I work on the science of deep learning. At the moment, I am temporarily less focused on publishing research and more on open source deep learning.
I'm interested in understanding what systems centered around foundation models will look like in 5 years, and how they will touch our lives in unthinkable ways. Reasoning models and computer use/software agents are two nascent examples, but I think and hope that AI systems will play a more transformative role in our day to day lives than mere automation. My research style usually involves studying these models in a scientific way, running experiments across the stack: from pretraining to evals and beyond.
My path into deep learning was a meandering one. I was adopted by multiple research communities broadly studying artificial intelligence when I was an undergraduate. Two prominent ones were physicists studying the brain -- who think of neural networks as freshly discovered platonic objects -- and computer systems people who like to hack and think of neural networks as graphs of differentiable tensor operations nestled inside GPU cores. I am immensely fortunate to have been a part of, and be mentored by, folks in both communities.
- Overtrained Langued Models are Harder to Finetune ICBINB at ICLR 2025. Best Paper.
- Scaling Laws for Precision ICLR 2025. Oral Award.
- Do Mice Grok? Unveiling Hidden Progress in Sensory Cortex During Overtraining ICLR 2025.
- Lower Data Diversity Accelerates Training: Case Studies in Synthetic Tasks Preprint.
- Asymptotic Dynamics for Delayed Feature Learning on a Toy Model HiLD at ICML 2024.
- No Free Prune: Information-Theoretic Barriers to Pruning at Initialization ICML 2024.
- Grokking as the Transition from Lazy to Rich Training Dynamics ICLR 2024.
- Human or Machine? Turing Tests for Vision and Language Preprint.
SCOPE at ICLR 2025. Outstanding Paper.
Jacob Mitchell Springer, Sachin Goyal, Kaiyue Wen, Tanishq Kumar, Xiang Yue, Sadhika Malladi, Graham Neubig, Aditi Raghunathan.
Tanishq Kumar*, Zachary Ankner*, Benjamin F. Spector, Blake Bordelon, Niklas Muennighoff, Mansheej Paul, Cengiz Pehlevan, Christopher Ré, Aditi Raghunathan.
Tanishq Kumar, Blake Bordelon, Cengiz Pehlevan, Venkatesh Murthy, Samuel J. Gershman.
Suhas Kotha*, Uzay Girit*, Tanishq Kumar*, Gaurav Ghosal, Aditi Raghunathan.
Blake Bordelon, Tanishq Kumar, Samuel J. Gershman, and Cengiz Pehlevan.
Tanishq Kumar*, Kevin Luo*, Mark Sellke.
Tanishq Kumar, Blake Bordelon, Samuel J. Gershman*, Cengiz Pehlevan*.
Mengmi Zhang, ... Tanishq Kumar, ... Gabriel Kreiman.