Top Closed- and Open-Weight Scores on the ECI
Top Closed- and Open-Weight Scores on the ECI
Experts expect the top closed-weight model to score 220 on the ECI by the end of 2029, and the top open-weight model 210.
Hover to highlight a group, click to show or hide itTap a group to show or hide it
Our Analysis
How do we interpret these results
At survey launch, the top closed-weight system scored 167 on the ECI and the top open-weight system 157. All groups expect both scores to keep rising, with open-weight systems continuing to trail. Experts' and the public's medians put the gap at 10 points or fewer at every horizon, while superforecasters and AI risk experts expect it to widen slightly. For 2026, medians range from 172 to 175 for closed-weight systems and from 162 to 167 for open-weight systems. Medians are similarly close for 2027, within 3 points of each other: 190 to 192 for closed-weight systems and 178 to 181 for open-weight systems. The groups diverge more for 2029. The public gives the lowest medians (212 and 204), significantly below every other group for both, while AI risk experts give the highest closed-weight median (227) and experts the highest open-weight median (210). Uncertainty also increases by 2029, with AI risk experts the most uncertain group: their median 10th and 90th percentile forecasts for closed-weight systems span 196 to 280.
Forecasts by group
Median forecasts from each group
| group | 2026 | 2027 | 2029 |
|---|---|---|---|
| Experts | 175 | 190 | 220.2 |
| Superforecasters | 172 | 190 | 220 |
| Public | 175 | 190 | 212 |
| AI Risk Experts | 173.9 | 192 | 227 |
Click any cell to compareTap any cell to compare
* Denotes non-overlapping bootstrapped confidence intervals between group aggregates.
Rationale analysis
The main lines of reasoning driving the spread in estimates
Historical ECI trajectory
Most respondents start by considering that the scores of closed frontier models have been rising about 14–16 points a year, with open-weight models trailing by about 8–10 points. High-forecast respondents tend to expect AI-driven research to accelerate this progress: “Beyond 2026, I don't expect capabilities to keep advancing linearly. My medians assume continued acceleration, with AI partly automating AI research and development but without full recursive self-improvement.”; “I expect the rate of capabilities gains to increase as we hit inflection points from automated researchers sometime between mid 2027 and mid 2028.” Others point to AI-driven research when setting their 90% quantile: “90% accounts for potential for recursive self improvement to pull trendline up, potentially substantially.” Low-forecast respondents tend to either anchor directly to the historical trend—“The trend looks almost ridiculously linear…No reason to deviate much from it.”—or cite some combination of the factors below when arguing that progress is likely to decelerate.
Open-closed gap
Respondents who forecast that the gap will narrow often “expect the gap to close by 2029, whether by distillation, model weight theft, or more legitimate advances.” Others also “expect open models to gradually catch closed models over the next four years as China and others invest heavily in them.” Those who forecast that a wider gap will persist tend to stress the closed-model advantage in compute and other resources: “Proprietary labs invest massively in huge compute clusters, secret dataset pipelines, and early architectural breakthroughs”; “Chinese labs are compute-constrained by export controls.” Still others question whether open models, once they attain certain capabilities, will remain open: “Open models may very well be banned beyond a certain capability.”; “Since April 2026 [Alibaba's] flagship Qwen Max models have been available only through an API, while it keeps releasing smaller open models. Meta was reported to be planning the same move for its next frontier model.”
Release delays
Some respondents argue that the ECI leaderboard will trail the true frontier, citing safety reviews and national-security concerns: “We're forecasting the highest score on Epoch's leaderboard rather than the best model someone has quietly running inside a laboratory.” Others treat “stronger models already known to exist but not on the public leaderboard” as evidence that the ECI has room to climb higher: “The upside [for 2026] comes mostly from OpenAI's internal model, which it says is stronger than Astra.”
Compute resources
Several low-forecast respondents expect physical and data limits to slow scaling: “We're probably severely constrained by compute until 2029 for the most fantastic scenarios.” High-forecast respondents tend to view massive AI-buildout spending through the prism of national security and market-competition imperatives: “Closed-weight capabilities will continue to increase in terms of ECI given US geopolitical…and financial incentives on the part of the leading frontier labs, esp as they move towards IPO.”
ECI as a measuring stick
High- and low-forecast respondents alike note that 2029 scores depend on Epoch adding harder benchmarks, and that how this is executed will impact the direction of scores: “ECI only stays meaningful if Epoch keeps adding harder benchmarks, so the wide [quantile] range…is as much about the measuring stick as about the models.”; “Numbers will depend heavily on the benchmark curation, not on model capabilities. Assuming the historical trend of the benchmark curation following a linear trend, I presume a linear trend too (it's basically predicting human behaviour rather than model capability projection).”
Rationale examples
High-quality rationales from forecasters across the range
High-forecast respondents
GPT-5 sat at 150 in August 2025, GPT-5.4 Pro hit 158 in March, and Opus 5.5 is at 167 now. That's roughly 15 points a year on the long run and closer to 17 over the last six months, so if anything it's speeding up (GPT-6 Astra and Opus 5.5 landing three weeks apart tells you the top is crowded and everyone is shipping). I lean into that.
I expect capability progress to accelerate. AI-assisted research can improve both capabilities and the safety work needed to deploy them; I do not regard safety as inherently opposed to capability. If safety becomes the bottleneck, labs have incentives to direct R&D toward removing it. Frontier labs should keep pushing to remain ahead of open-weight systems.
Closed-weight laboratories have obvious advantages in funding, access to enormous compute resources, proprietary training infrastructure, and the ability to recover some development costs through subscriptions and API revenue, but open-weight developers benefit from rapid dissemination of research techniques, downstream experimentation, and an ecosystem of organizations interested in building capable systems without remaining permanently dependent on a small collection of commercial model providers. Some of the most consequential improvements over the next few years may come from training efficiency, better use of synthetic data, improved reasoning strategies, or approaches to inference-time computation that can eventually be reproduced without possessing an absolutely obscene capital budget.
I expect that by the end of 2027 ECI score will be a lot less informative because it'll be extremely difficult to build unsaturated evals. I also expect capabilities to increase at a faster rate than they were previously (because of research automation that's accelerating research more with each new generation of model). I'm not sure how to reconcile both of these effects. So I assume a rate of 20 pts/year starting in 2027.
Low-forecast respondents
In economic terms, the value of a model, like the value of a firm, is the sum of the margins between the price of each token it serves and the marginal cost of serving it. Because the current open model is technically behind the closed frontier, it can only be sold at about its marginal cost, whether or not its weights are public, so its margin, and with it its value, is zero…For open models I expect the lag behind the closed frontier to grow, with a substantial chance that the open frontier stalls because its strongest developers close their weights or are made to. The gap between the two should widen rather than close.
When I talk to my students and other people, it seems that [the] coding ability of models has stagnated. Where previous generations of models had similar quality or capability, there are certain niches where this is not quite true, like cyber capability. But even there, capability is normalizing. The most recent models show much broader capability rather than deeper capability. And I think this trend will continue. There might be possibilities for further innovation, but almost any process shows linear improvement with exponential resources. And I believe this is also true for AI models. With that, my median prediction is that capability will level off.
The most likely scenario…will be that going forward publicly released models will be increasingly limited in their power relative to what's actually possible. There may emerge 'gated' access protocols for clients such as the United States Government and specific hand-picked and security-cleared companies…I suspect that it's more likely than not that with the pulling of Astra 6.1 we have seen the beginning of the end for universally available public models at the forefront of technological capability.
Open models will be more severely safety-constrained, will be lacking compute and capital, and the efforts to prevent distillation by the frontier labs will work. Open models will shift to focus on niche specialties and edge devices and on-board robotics, which is not the kind of thing ECI favors. Also likely the CCP starts to demand the best Chinese models for internal use and the open models no longer reflect the frontier.