Research
I work on the science of deep learning. I've had the privilege of learning to do research from Sam Gershman and Aditi Raghunathan. This is some of the work I published as an undergraduate.
I'm interested in understanding what systems centered around foundation models will look like in 5 years, and how they will touch our lives in unthinkable ways. Reasoning models and computer use/software agents are two nascent examples, but I think and hope that AI systems will play a more transformative role in our day to day lives than mere automation. My research style usually involves studying these models in a scientific way, running experiments across the stack: from pretraining to evals and beyond.
My path into deep learning was a meandering one. I was taken in by multiple research communities broadly studying artificial intelligence when I was an undergraduate. Two prominent ones were physicists studying the brain -- who think of neural networks as freshly discovered platonic objects, pure and unsullied -- and computer systems people who like to hack and think of neural networks as graphs of differentiable tensor operations nested inside GPU cores, where the key goal is maximizing arithmetic intensity. I am fortunate to have been a part of and be mentored by folks in both communities.
- Overtrained Langued Models are Harder to Finetune SCOPE at ICLR 2025. Oral Award.
- Scaling Laws for Precision ICLR 2025. Oral Award.
- Do Mice Grok? Unveiling Hidden Progress in Sensory Cortex During Overtraining ICLR 2025.
- Lower Data Diversity Accelerates Training: Case Studies in Synthetic Tasks Preprint.
- Asymptotic Dynamics for Delayed Feature Learning on a Toy Model HiLD at ICML 2024.
- No Free Prune: Information-Theoretic Barriers to Pruning at Initialization ICML 2024.
- Grokking as the Transition from Lazy to Rich Training Dynamics ICLR 2024.
- Human or Machine? Turing Tests for Vision and Language Preprint.
Jacob Mitchell Springer, Sachin Goyal, Kaiyue Wen, Tanishq Kumar, Xiang Yue, Sadhika Malladi, Graham Neubig, Aditi Raghunathan.
Tanishq Kumar*, Zachary Ankner*, Benjamin F. Spector, Blake Bordelon, Niklas Muennighoff, Mansheej Paul, Cengiz Pehlevan, Christopher Ré, Aditi Raghunathan.
Tanishq Kumar, Blake Bordelon, Cengiz Pehlevan, Venkatesh Murthy, Samuel J. Gershman.
Suhas Kotha*, Uzay Girit*, Tanishq Kumar*, Gaurav Ghosal, Aditi Raghunathan.
Blake Bordelon, Tanishq Kumar, Samuel J. Gershman, and Cengiz Pehlevan.
Tanishq Kumar*, Kevin Luo*, Mark Sellke.
Tanishq Kumar, Blake Bordelon, Samuel J. Gershman*, Cengiz Pehlevan*.
Mengmi Zhang, ... Tanishq Kumar, ... Gabriel Kreiman.