Notes on Dwarkesh-Karpathy
A summary of the Karpathy interview on the Dwarkesh podcast, with my thoughts in italics. I paraphrased what Dwarkesh and Karpathy said, they’re not direct quotes. Big insights: LLMs are still cognitively lacking in many ways — no continual learning, not very multimodal, don’t have a reflection process, have a collapsed distribution of responses. These are hard problems to solve and will take time. Karpathy argues these won’t just be solved by scaling up models and doing RL on different types of tasks. These behaviors are not emergent, they require algorithmic breakthroughs. Humans don’t learn what we think of as intelligent tasks by RL, they seem to learn by something different and more reflective and deliberate. RL is terrible, you’re updating everything in the trajectory of actions, even if intermediate steps were wrong. It’s also a slow, inefficient way of learning. AI progress will come from better everything — data, hardware, kernels, and algorithmic breakthroughs. LLMs are bad a...