Visualizing the Training Costs of AI Models Over Time

The World’s Largest Economies: Comparing the U.S. and ChinaComparing Saudi Aramco’s $1.9T Valuation to Its RivalsThe Top Retailers in the World, by Store CountVisualizing Berkshire Hathaway’s Stock Portfolio (Q1 2024)Nvidia is Worth More Than All of These Companies Combined . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . bmc bmc bmc bmc bmc

Visualizing the Training Costs of AI Models Over TimeVisualizing the Top Countries, by Mobile Data UsageWhat Laptop Brands do Americans Use in 2024?Nvidia is Worth More Than All of These Companies CombinedRanked: The Top Startup Cities Around the World
MONEY
Mapped: The Income a Family Needs to Live Comfortably in Every U.S. StateTop 10 Countries Most in Debt to the IMFVisualizing America’s Average Retirement Savings, by AgeVisualizing the Tax Burden of Every U.S. StateCharted: What Frustrates Americans About the Tax System
HEALTHCARE
Visualizing Daily Protein Sources by RegionMapped: Countries Where Recreational Cannabis is LegalWhich Countries Have the Highest Infant Mortality Rates?Life Expectancy by Region (1950-2050F)What Causes Preventable Child Deaths?
ENERGY
Comparing Saudi Aramco’s $1.9T Valuation to Its RivalsRanked: The World’s Largest Lithium Producers in 2023Who’s Building the Most Solar Energy?Mapped: The Age of Energy Projects in Interconnection Queues, by StateRanked: The Top 10 EV Battery Manufacturers in 2023
MAPS
Mapped: The World’s Largest Armies in 2024Mapped: Chinese Provinces With Cities Over 1 Million PeopleMap: Where Are America’s Largest Landfills?Mapped: The World’s Wettest and Driest CountriesMapped: The Income a Family Needs to Live Comfortably in Every U.S. State
MINING
Visualizing Cobalt Production by Country in 2023Ranked: The World’s Largest Lithium Producers in 2023The Biggest Salt Producing Countries in 2023Visualizing Copper Production by Country in 2023Where the World’s Aluminum is Smelted, by Country
GREEN
How People Get Around in America, Europe, and AsiaThe Carbon Footprint of Major Travel MethodsRanking the Top 15 Countries by Carbon Tax RevenueRanked: The Countries With the Most Air Pollution in 2023Top Countries By Forest Growth Since 2001

This was originally posted on our Voronoi app. Download the app for free on iOS or Android and discover incredible data-driven charts from a variety of trusted sources.

Training advanced AI models like OpenAI’s ChatGPT and Google’s Gemini Ultra requires millions of dollars, with costs escalating rapidly.

As computational demands increase, the expenses for the computing power necessary to train them are soaring. In response, AI companies are rethinking how they train generative AI systems. In many cases, these include strategies to reduce computational costs given current growth trajectories.

This graphic shows the surge in training costs for advanced AI models, based on analysis from Stanford University’s 2024 Artificial Intelligence Index Report.

How Training Cost is Determined
The AI Index collaborated with research firm Epoch AI to estimate AI model training costs, which were based on cloud compute rental prices. Key factors that were analyzed include the model’s training duration, the hardware’s utilization rate, and the value of the training hardware.

While many have speculated that training AI models has become increasingly costly, there is a lack of comprehensive data supporting these claims. The AI Index is one of the rare sources for these estimates.

Last year, OpenAI’s GPT-4 cost an estimated $78.4 million to train, a steep rise from Google’s PaLM (540B) model, which cost $12.4 million just a year earlier.

For perspective, the training cost for Transformer, an early AI model developed in 2017, was $930. This model plays a foundational role in shaping the architecture of many large language models used today.

Google’s AI model, Gemini Ultra, costs even more, at a staggering $191 million. As of early 2024, the model outperforms GPT-4 on several metrics, most notably across the Massive Multitask Language Understanding (MMLU) benchmark. This benchmark serves as a crucial yardstick for gauging the capabilities of large language models. For instance, its known for evaluating knowledge and problem solving proficiency across 57 subject areas.

Training Future AI Models
Given these challenges, AI companies are finding new solutions for training language models to combat rising costs.

These include a number of approaches, such as creating smaller models that are designed to perform specific tasks. Other companies are experimenting with creating their own, synthetic data to feed into AI systems. However, a clear breakthrough is yet to be seen.

Today, AI models using synthetic data have shown to produce nonsense when asked with certain prompts, triggering what is referred to as “model collapse”.

This was originally posted on our Voronoi app. Download the app for free on iOS or Android and discover incredible data-driven charts from a variety of trusted sources.

Global mobile data traffic per smartphone is forecast to hit 42 gigabytes per month by 2027—a staggering 16-fold increase compared to 2017.

This surge coincides with a rising number of internet users worldwide, in particular among middle-income countries. Notably, between 2018 and 2022, India saw a 170% increase in internet users alone.

The above graphic shows the countries with the highest mobile data usage around the world, based on data from the World Bank’s Digital Progress and Trends Report.

Which Countries Use the Most Mobile Data?
Here are the top 10 countries worldwide based on their monthly mobile data consumption:

The island nation of Curaçao ranks first overall, with 131.3 gigabytes of monthly mobile data usage per capita.

With a population of just 193,000, Curaçao surpasses all other nations likely due to a combination of factors. This includes an influx of over one million tourist arrivals in 2022, competitive mobile data plans, and a low penetration of fixed-line telecommunication systems and broadband internet.

For instance, in 2021, just 28 out of 100 people used fixed-line networks, while 88 out of 100 people used cellular-mobile services. Furthermore, the country boasts among the fastest and most reliable internet connections in the Caribbean thanks to its concentration of data centers and fiber optic cable infrastructure.

Following in second place is oil-rich Kuwait. In recent years, the country’s telecom industry has significantly improved, with 5G networks reaching roughly 97% of the population. Furthermore, the Kuwait government invested heavily in mobile platforms that allow people to settle traffic fines or schedule court dates via mobile apps.

Like Kuwait, Middle Eastern nations of Saudi Arabia and Bahrain rank highly on this list.

Top European Countries
As a country known for its advanced telecommunications infrastructure, Finland leads Europe in mobile data usage.

In fact, the country launched the first 6G test network worldwide. Given its role as a leader in communications networks and digital technologies, many companies use Finland to test new products and services before launching globally.

Joining Finland from Europe in this list are Latvia, Estonia, Austria, and Lithuania.

By contrast, the U.S. ranks in #34 overall in monthly mobile data consumption, at 13.4 gigabytes per capita. Part of the reason behind low usage may be that the cost of mobile data in America is among the highest in the world. For perspective, the average cost of one gigabyte of data in America was $6.00 in 2023, a steep jump from $0.20 in France or $3.48 in Japan. rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd rtd bmc bmc bmc bmc bmc

The post Visualizing the Training Costs of AI Models Over Time appeared first on The Muslim News.