General AI Progress Scenarios

Rapid-progress forecasters typically cite consistent capability improvements: “historically AI system development has followed a steep scaling curve and increases in model size-data and compute have led to rapid capability gains.” One points out that “METR [Model Evaluation and Threat Research] results imply a roughly 4 to 10x improvement in time horizon every year, which means that we'll have systems capable of doing weeks or months of work by the late 2020s.” Many rapid-progress forecasters also argue that AI progress has been consistently underestimated, and that “rapid progress is quite likely due to the [...] possibility of AI improving themselves.” Slow-progress forecasters often argue that current AI paradigms are likely insufficient: “For any of the moderate and rapid progress criteria to be met there would need to be a massive paradigm shift in AI technology. LLMs are unlikely to achieve these goals with the incremental progress made over the last 2-5 years.”

Robotics is the strongest consensus limitation: “As a roboticist I have serious doubts about how quickly the embodied aspect of AI will make progress due to real-world implementation issues (sensors, communication delays, etc.) regardless of how good the brain is.” Many slow-progress forecasters also believe that historical precedent suggests caution is merited: “Having end to end task completion ability is hard as can be told from the development of autonomous vehicles (15+ years of R&D and 100+ billions of dollars invested and we are still scaling L4 robotaxis today).” Most moderate-progress forecasters also acknowledge this challenge.

Slow-progress forecasters often highlight the need for better training data, the cost of compute, and especially energy: “I expect energy to be the chief bottleneck to AI progress such that it will be a rate-limiter for progress in general.” Some forecasters expecting rapid progress, however, argue that such constraints can be eliminated by massive investment in AI, driven by corporate competition and national security incentives.

A common consideration among slow-progress forecasters is that “[reliability] will be a difficult bar to clear for advanced-capability systems given current cognitive limitations.” They tend to emphasize that barriers like robust generalization, complex symbolic reasoning, and end-to-end safe autonomy for complex tasks will likely hold “superhuman across the board” scenarios back.