Continual learning and the post monolith AI era

This article discusses the challenges and potential solutions for continual learning in the context of the "post-monolith AI era." The key points are: 1. Continual learning is inseparable from specialization, as the cost of remembering scales with the scope of what the model is trying to learn. Narrow, task-specific models can more easily adapt and update through fine-tuning, while broader, more general models face a fundamental tension between continual learning and maintaining long-term memory. 2. The article explores two types of specialization: task specialization (e.g., document summarization) and domain specialization (e.g., acting as an employee). The latter is more challenging due to broader scope and sparse feedback. 3. The article outlines the "fundamental tension" between continual learning and long-term memory, as updating representations can corrupt the interfaces between different subsystems. This is why critical periods exist in human development, where interfaces become fixed to enable further learning. 4. The article reviews several ideas from the continual learning literature, such as "cartridges" (compressing prior knowledge into a dense KV cache), state-space models, and sparse memory fine-tuning. While these approaches offer interesting innovations, the author remains skeptical that they can fully overcome the inherent tradeoff between generality and the cost of continual learning. 5. The article concludes that the intractability of continual learning in generalist models is pushing the industry towards narrower, more specialized AI systems, as the "invisible hand" of this fundamental tradeoff guides deployment.