Data Insights

Open-weight models lag state-of-the-art by around 3 months on average

Frontier open-weight models lag behind the most capable models by an average of 3 months in the Epoch Capabilities Index (ECI), our holistic measure of model capability. That corresponds to an average ECI gap of around 7 points, similar to the gap between o3 and GPT-5. However, the gap varies considerably over time, sometimes even closing completely. Until the release of o1-mini, Llama 3.1-405B was rated on par with the closed-source state-of-the-art model, Claude 3.5 Sonnet. You can see more detailed analysis about the gap in our earlier article.

October 30, 2025

OpenAI’s revenue has been growing 3x a year since 2024

OpenAI’s annualized revenue jumped from $2B in 2023 to $13B by Aug 2025, a 3.2× yearly growth, putting it on pace to top $20B by year’s end.

October 14, 2025

Most of OpenAI’s 2024 compute went to experiments

OpenAI spent ~$7B on cloud compute last year—mostly R&D (research, experiments, training). Only a small share went to final model training.

October 10, 2025

AI capabilities have steadily improved over the past year

In the past 12 months, AI’s abilities have improved steadily across a range of skills including coding, visual and common sense reasoning, and math.

September 30, 2025

AI developers accurately report GPQA Diamond scores for recent models

Compared to Epoch internal evals, top labs report statistically consistent GPQA Diamond scores for their recent models.

September 19, 2025

What did it take to train Grok 4?

We estimated the resources required to train Grok 4 - the largest AI training run to date.

September 12, 2025

LLMs have not yet solved the hardest problems on high school math contests

LLMs have come a long way on high school math contests but have yet to show that they can solve the hardest problems found on these contests.

September 3, 2025

GPT-5 and GPT-4 were both major leaps in benchmarks from the previous generation

Despite mixed reception, benchmark data show GPT-5’s gains roughly match GPT-4’s jump over GPT-3. Methods, data sources, and limitations inside.

August 29, 2025

Frontier AI performance becomes accessible on consumer hardware within a year

Models that fit on consumer GPUs match the performance of frontier AI within a year or less.

August 15, 2025

Compute is not a bottleneck for robotic manipulation

Compute is not a bottleneck for robotics, while training data is. Frontier-level compute could accelerate progress if data improves.

August 8, 2025

Training open-weight models is becoming more data intensive

The ratio of training data to active parameters in open-weight LLMs has grown 3.1x per year since 2022. Recent models LLMs have been trained with 20 times more data per parameter than the optimal ratio suggested by the 2022 Chinchilla scaling laws.

August 1, 2025

Frontier training runs will likely stop getting longer by around 2027

Frontier AI training time is on track to hit natural limits by 2027

July 25, 2025

LLMs now accept longer inputs, and the best models can use them more effectively

Today’s LLMs can only ingest a limited number of tokens per query, often called the context window. Since mid-2023, the longest LLM context windows have grown by about 30x per year. Their ability to use that input effectively is improving even faster: on two long-context benchmarks, the input length where top models reach 80% accuracy has risen by over 250x in the past 9 months. Still, headroom remains. Many long-context tasks are likely more difficult than these benchmarks, and some relevant content does not fit into even the longest available windows.

June 25, 2025

LLM providers offer a trade-off between accuracy and speed

Across major language model providers, the models that achieve higher accuracy on benchmarks also take longer to run. Along the accuracy-runtime trade-off frontier, cutting the error rate in half typically slows the model by roughly 2x to 6x, depending on the task. LLM providers have several ways to balance performance against cost, such as model size, query batching, and quantization. Many of the models on the speed-accuracy trade-off frontiers appear to have been designed with efficiency in mind: over half of them are labeled turbo, flash, mini, or nano. The models on this frontier are mostly from two providers: OpenAI and Google.

June 11, 2025

Power requirements of leading AI supercomputers have doubled every 13 months

Leading AI supercomputers are becoming ever more energy-intensive, using more power-hungry chips in greater numbers. In January 2019, Summit at Oak Ridge National Lab had the highest power capacity of any AI supercomputer at 13 MW. Today, xAI’s Colossus supercomputer uses 280 MW, over 20x as much. Colossus relies on mobile generators because the local grid has insufficient power capacity for so much hardware. In the future, we may see frontier models trained across geographically distributed supercomputers, to mitigate the difficulty of delivering enormous amounts of power to a single location, similar to the training setup for Gemini 1.0.

June 5, 2025

Private-sector companies own a dominant share of GPU clusters

The private sector’s share of global AI computing capacity has grown from 40% in 2019 to 80% in 2025. Though many leading early supercomputers such as Summit were run by government and academic labs, the total installed computing power of public-sector clusters has only increased at 1.8x per year, rapidly outpaced by private-sector clusters, whose total computing power has grown at 2.7x per year. The rising economic importance of AI has spurred the private sector to build more and faster clusters for training and inference. As of May 2025, the largest known public AI supercomputer, Lawrence Livermore’s El Capitan, achieves less than a quarter of the computational performance of the largest known industry cluster, xAI’s Colossus.

June 5, 2025

The US hosts the majority of GPU cluster performance, followed by China

As of May 2025, the United States contains about three-quarters of global GPU cluster performance, with China in second place with 15%. Meanwhile, traditional high-performance computing leaders like Germany, Japan, and France now play marginal roles in the AI cluster landscape. This shift largely reflects the increased dominance of major technology companies, which are predominantly based in the United States.

June 5, 2025

Acquisition costs of leading AI supercomputers have doubled every 13 months

AI supercomputers have become increasingly expensive. Since 2019, the cost of the computing hardware for leading supercomputers has increased at a rate of 1.9x per year. In June 2022, the most expensive cluster was Oak Ridge National Laboratory Frontier, with a reported cost of $200M. Three years later, as of June 2025, the most expensive supercomputer is xAI’s Colossus, estimated to use over $7B of hardware.

June 5, 2025

Widespread adoption of new numeric formats took 3-4 years in past cycles

In the past, it has taken up to four years for new numerical formats to reach widespread adoption in model training. Both FP16 and BF16 gained early traction in the first two years, reached 50% adoption by around year three1, and became the default within five years. Shifting to these new formats provided gains in computational performance and training stability, but were contingent on hardware support. BF16 is on the latter end of its 4 year cycle, and FP8 training is now emerging. If adoption patterns hold, FP8 could be the standard by around 2028. Extending this further, 4-bit training may follow and eventually become the primary numerical format, but training stability at such low precision remains a challenge.

May 28, 2025

The computational performance of leading AI supercomputers has doubled every nine months

The computational performance of the leading AI supercomputers has grown by 2.5x annually since 2019. This has enabled vastly more powerful training runs: if 2020’s GPT-3 were trained on xAI’s Colossus, the original two week training run could be completed in under 2 hours. This growth was enabled by two factors: the number of chips deployed per cluster has increased by 1.6x per year, and performance per chip has also improved by 1.6x annually.

April 30, 2025

LLM responses to benchmark questions are getting longer over time

The length of responses generated by language models to benchmark questions has increased over time for both reasoning and non-reasoning models. According to our internal benchmarks, reasoning models’ responses are growing considerably faster (5x per year) than those from non-reasoning models (2.2x per year). Reasoning models also exhibit longer response lengths overall – currently, around 8x more tokens on average, compared to non-reasoning models. Reasoning models have been shown to produce more reasoning tokens as the amount of RL training is scaled up. The explanation for longer responses from non-reasoning models is less clear, but it may reflect trends toward extended context lengths, judges’ preferences for longer responses when generating RLHF data, or a growing use of context distillation from Chain of Thought prompting. Finally, while today’s models are cleanly separable into reasoning vs. non-reasoning, there are indications that future models (like GPT-5) may blur this line.

April 17, 2025

The combined revenues of leading AI companies grew by over 9x in 2023-2024

OpenAI, Anthropic, and Google DeepMind each grew their revenue over 90% in the second half of 2024, corresponding to an annualized growth over 3x/year. OpenAI’s and Anthropic’s revenue projections both imply that their revenue will continue to grow over 3x in 2025. Projecting to April 2025, we estimate OpenAI’s revenue at around $10B/year, while Anthropic and Google DeepMind each earn single digit billions per year. Note that our estimates for Google DeepMind are more speculative, and don’t include internal revenues from integration with Google products, which could be substantial. According to our estimates, no other AI company surpassed $100 million in revenue in 2024 by selling access to their own models. However, companies such as Microsoft and Amazon collectively make more revenue than top AI companies by charging for access to third-party models. Microsoft alone reports $13B in revenues from its AI business, seemingly driven by sales of Copilot (which uses OpenAI models).

April 3, 2025

LLM inference prices have fallen rapidly but unequally across tasks

The inference price of LLMs has fallen dramatically in recent years. We looked at the results of state-of-the-art models on six benchmarks over the past three years, and then measured how quickly the price to achieve those milestones has fallen. For instance, the price to achieve GPT-4’s performance on a set of PhD-level science questions fell by 40x per year. The rate of decline varies dramatically depending on the performance milestone, ranging from 9x to 900x per year. The fastest price drops in that range have occurred in the past year, so it’s less clear that those will persist. There are many potential reasons for these price drops. Some are well-known, such as models becoming smaller and hardware becoming more cost-effective, while other important factors might be difficult to determine from public information.

March 12, 2025

Leading AI chip designs are used for around four years in frontier training

The median time from the release of leading AI chips to the publication date of the last frontier model trained using them is 3.9 years. We focus on AI chips from the NVIDIA V100 onwards, as these chips were used for large-scale language model training similar to present day frontier models. Including chips older than the NVIDIA V100 shortens the median lifespan from release to final use for frontier training to 2.7 years.

March 5, 2025

Biology AI models are scaling 2-4x per year after rapid growth from 2019-2021

The training compute of top AI models trained on biological data grew rapidly in 2019-2021, but has scaled at a more sustainable pace since then (2-4x per year). Training compute for these models increased by 1,000x-10,000x between 2018 and 2021, but has only increased 10x-100x since 2021. We consider two categories of biology AI models. General-purpose protein language models (PLMs) like Evo 2 learn to predict biological sequences, and can generate embeddings that are useful across many tasks. Specialized models like AlphaFold are optimized for specific predictions such as protein structure or mutation effects. PLMs are trained using about 10x more compute than specialized models, but still lag about 100x behind today’s frontier language models, which now are trained with over 1e26 FLOP.

February 21, 2025

The stock of computing power from NVIDIA chips is doubling every 10 months

Total available computing power from NVIDIA chips has grown by approximately 2.3x per year since 2019, enabling the training of ever-larger models. The Hopper generation of NVIDIA AI chips currently accounts for 77% of the total computing power across all of their AI hardware. At this pace of growth, older generations tend to contribute less than half of cumulative compute around 4 years after their introduction. Note this analysis does not include TPUs or other specialized AI accelerators, for which less data is available. TPUs may provide comparable total computing power to NVIDIA chips.

February 13, 2025

Over 30 AI models have been trained at the scale of GPT-4

The largest AI models today are trained with over 1025 floating-point operations (FLOP) of compute. The first model trained at this scale was GPT-4, released in March 2023. As of June 2025, we have identified over 30 publicly announced AI models from different AI developers that we believe to be over the 1025 FLOP training compute threshold. Training a model of this scale costs tens of millions of dollars with current hardware. Despite the high cost, we expect a proliferation of such models—we saw an average of roughly two models over this threshold announced every month during 2024. Models trained at this scale will be subject to additional requirements under the EU AI Act, coming into force in August 2025.

January 30, 2025

Chinese language models have scaled up more slowly than their global counterparts

The training compute of top Chinese language models grew rapidly after 2019, catching up to the top models globally by late 2021. Since then, China’s rate of scaling has fallen behind. The top 10 Chinese language models by training compute have scaled up by about 3x per year since late 2021, which is slower than the 5x per year trend maintained elsewhere since 2018. At China’s current rate, it would take about two years to reach where the global top models are today. The difference looks less dramatic when comparing just the largest known models from each region. Grok-2, the largest known US model, used twice the training compute of China’s largest known model Doubao-pro, released 3 months later. Given China’s current scaling rate of 3x per year, it would take 8 months to scale from Doubao-pro to Grok-2’s level of compute.

January 22, 2025

Frontier open models may surpass 10²⁶ FLOP of training compute before 2026

The Biden Administration’s diffusion framework places restrictions on closed-weight models if their training compute surpasses either 1026 FLOP or the training compute of the largest open model. Historical trends suggest that the largest open model will surpass 1026 FLOP by November, 2025, and grow at close to 5x per year thereafter. There is an additional reason to expect a large open model before 2026: Mark Zuckerberg indicated in October 2024 that Llama 4 models were already being trained on a cluster “larger than 100k H100s”. In the same statement, it is strongly implied that these models will continue to be released with open weights. Models trained at this scale are very likely to surpass 1026 FLOP, and appear to be planned for release in 2025.

January 15, 2025

Training compute growth is driven by larger clusters, longer training, and better hardware

Since 2018, the most significant driver of compute scaling across frontier models has likely been an increase in the quantity of hardware used in training clusters. Also important have been a shift towards longer training runs, and increases in hardware performance. These trends are closely linked to a massive surge in investment. AI development budgets have been expanding by around 2-3x per year, enabling vast training and inference clusters and ever-larger models.

January 8, 2025

US models currently outperform non-US models

The best US models have consistently higher accuracies than the best non-US models on GPQA Diamond and MATH Level 5. For example, on GPQA Diamond the best-performing model is OpenAI’s o1, while on MATH Level 5 the leading model is o3-mini. However, with the release of DeepSeek-R1 in January 2025, the gap between US and non-US models has reduced substantially: DeepSeek-R1 trails behind o3-mini by only 2 percentage points on MATH Level 5, and scores only 4 percentage points lower than o1 on GPQA Diamond.

November 27, 2024

Models with downloadable weights currently lag behind the top-performing models

The models with the highest GPQA Diamond and MATH Level 5 accuracies tend not to have downloadable weights. For example, on GPQA Diamond OpenAI’s o1 outperformed the best-performing downloadable model at the time, Phi-4, by 20 percentage points. Similarly on MATH Level 5, Phi-4 lagged behind o1 by 29 percentage points. We further analyzed how far behind open models are in this article, where we found that the best-performing open LLMs lagged the best-performing closed LLMs by between 6 months on GPQA Diamond and 20 months on MMLU. However, the release of DeepSeek-R1 in January 2025 showed that the performance gap between open-weights and closed-weights has significantly decreased. For example, on MATH Level 5 DeepSeek-R1 only lags behind the current best-performing model, o3-mini, by 2 percentage points.

November 27, 2024

Accuracy increases with estimated training compute

GPQA Diamond and MATH Level 5 accuracies increase with estimated training compute. For GPQA Diamond, below 1024 FLOP most models struggle to rise above random chance performance — or even perform worse than random chance, due to failing to understand question formatting. Past 1024 FLOP, performance increases around 12 percentage points with every 10x increase of compute. On MATH Level 5, models with high compute estimates also tend to have higher scores: performance increases around 17 percentage points with every 10x increase in pretraining compute. However, the trend is much noisier than for GPQA Diamond. On both benchmarks, more recent models such as DeepSeek-R1, Phi-4, or Mistral Small 3 outperform older models trained with the same amount of compute, highlighting the role of algorithmic progress. Finally, note that these trends exclude many of the top-performing models, such as OpenAI’s o1, which we lack compute estimates for.

November 27, 2024

AI training cluster sizes increased by more than 20x since 2016

From Google’s NASv3 RL network, trained on 800 GPUs in 2016, to Meta’s Llama 3.1 405B, using 16,384 H100 GPUs in 2024, the number of processors used increased by a factor of over 20. Gemini Ultra was trained with an even larger number of TPUs, but precise details were not reported.

October 23, 2024

Performance per dollar improves around 30% each year

Performance per dollar has improved rapidly, and hardware at any given precision and fixed performance level becomes 30% cheaper each year. At the same time, manufacturers continue to introduce more powerful and expensive hardware.

October 23, 2024

The computational performance of machine learning hardware has doubled every 2.3 years

Measured in 16-bit floating point operations, ML hardware performance has increased at a rate of 36% per year, doubling every 2.3 years. A similar trend exists in 32-bit performance. Optimized ML number formats and tensor cores provided additional improvements. The improvement was driven by increasing transistor counts and other semiconductor manufacturing improvements, as well as specialized design for AI workloads. This improvement lowered cost per FLOP, increased energy efficiency, and enabled large-scale AI training.

October 23, 2024

The NVIDIA A100 has been the most popular hardware for training notable machine learning models

The NVIDIA A100 has been the most prolific hardware for highly-cited or state-of-the-art AI models identified in Epoch’s dataset, used for 85 notable ML models since its release. In second place is the NVIDIA V100, used to train 68 notable models, followed by Google’s TPU v3, used for 44. However, we estimate that the NVIDIA H100 had sold more units than the A100 by late 2023, so it may be the most popular GPU for training models in the near future.

October 23, 2024

Leading ML hardware becomes 40% more energy-efficient each year

Historically, the energy efficiency of leading GPUs and TPUs has doubled every 2 years. In tensor-FP16 format, the most efficient accelerators are Meta’s MTIA, at up to 2.1 x 1012 FLOP/s per watt, and the NVIDIA H100, at up to 1.4 x 1012 FLOP/s per watt. The upcoming Blackwell series of processors may be even more efficient, depending on their power consumption.

October 23, 2024

Performance improves 13x when switching from FP32 to tensor-INT8

GPUs are typically faster when using tensor cores and number formats optimized for AI computing. Compared to using non-tensor FP32, TF32, tensor-FP16, and tensor-INT8 provide around 6x, 11x, and 13x greater performance on average in the aggregate performance trends. Some chips achieve even larger speedups. For example, the H100 is 59x faster in INT8 than in FP32. These improvements account for about half of the overall performance trend improvement since their introduction. Models trained with lower precision formats have become common, especially tensor-FP16, as developers take advantage of this boost in performance.

October 23, 2024

Leading AI companies have hundreds of thousands of cutting-edge AI chips

The world's leading tech companies—Google, Microsoft, Meta, and Amazon—own AI computing power equivalent to hundreds of thousands of NVIDIA H100s. This compute is used both for their in-house AI development and for cloud customers, including many top AI labs such as OpenAI and Anthropic. Google may have access to the equivalent of over one million H100s, mostly from their TPUs. Microsoft likely has the single largest stock of NVIDIA accelerators, with around 500k H100-equivalents. A large share of AI computing power is collectively held by groups other than these four, including other cloud companies such as Oracle and CoreWeave, compute users such as Tesla and xAI, and national governments. We highlight Google, Microsoft, Meta, and Amazon as they are likely to have the most compute, and there is little public data for others.

October 9, 2024

The power required to train frontier AI models is doubling annually

Training frontier models requires a large and growing amount of power for GPUs, servers, cooling and other equipment. This is driven by an increase in GPU count; power draw per GPU is also growing, but at only a few percent per year. Training compute has grown even faster — around 4x/year. However, hardware efficiency (a 12x improvement in the last ten years), the adoption of lower precision formats (an 8x improvement) and longer training runs (a 4x increase) account for a roughly 2x/year decrease in power requirements relative to training compute. Our methodology for calculating or estimating a model’s power draw during training can be found here.

September 19, 2024

The length of time spent training notable models is growing

Since 2010, the length of training runs has increased by 1.2x per year among notable models, excluding those that are fine-tuned from base models. A continuation of this trend would ease hardware constraints, by increasing training compute without requiring more chips or power. However, longer training times face a tradeoff. For very long runs, waiting for future improvements to algorithms and hardware might outweigh the benefits of extended training.

August 16, 2024

Language models compose the large majority of large-scale AI models

Out of 454 large-scale models with known compute, 363 are language models, of which 93 are vision-language models such as GPT-4. The first models trained with 1023 FLOP were for game-playing, but language has dominated since 2021. Other large-scale models have been developed for image and video generation, biological sequence modeling, and robotics.

June 19, 2024

Most large-scale models are developed by US companies

Over half of known large-scale models were developed in the United States. A quarter were developed in China, and this proportion has been growing in recent years. The European Union trails them with 45 models, while the United Kingdom has developed 42.

June 19, 2024

The pace of large-scale model releases is accelerating

In 2017, only zero models exceeded 1023 FLOP in training compute. By 2020, this grew to three models; by 2022, there were 36, and by 2024, there were 205 models known to exceed 1023 FLOP in our database, and 126 more with unconfirmed training compute that likely exceed 1023 FLOP. As AI investment increases and training hardware becomes more cost-effective, models at this scale come within reach of more and more developers.

June 19, 2024

Almost half of large-scale models have published, downloadable weights

84 large-scale models with known compute have downloadable weights. Most of these have a training compute between 1023 and 1024 FLOP, which is less compute than the largest proprietary models. The developers that have released the largest downloadable models today are Meta and the Technology Innovation Institute.

June 19, 2024

The size of datasets used to train language models doubles approximately every six months

Across all domains of ML, models are using more and more training data. In language modeling, datasets are growing at a rate of 3.7x per year. The largest models currently use datasets with tens of trillions of words. The largest public datasets are about ten times larger than this, for example Common Crawl contains hundreds of trillions of words before filtering.

June 19, 2024

Training compute costs are doubling every eight months for the largest AI models

Spending on training large-scale ML models is growing at a rate of 2.4x per year. The most advanced models now cost hundreds of millions of dollars, with expenses measured by amortizing cluster costs over the training period. About half of this spending is on GPUs, with the remainder on other hardware and energy.

June 19, 2024

The training compute of notable AI models has been doubling roughly every six months

Since 2010, the training compute used to create AI models has been growing at a rate of 4.4x per year. Most of this growth comes from increased spending, although improvements in hardware have also played a role.

June 19, 2024

Training compute has scaled up faster for language than vision

Before 2020, the largest vision and language models had similar training compute. After that, language models rapidly scaled to use more training compute, driven by the success of transformer-based architectures. Standalone vision models never caught up. Instead, the largest models have recently become multimodal, integrating vision and other modalities into large models such as GPT-4 and Gemini.

June 19, 2024