Sakhanda Wire
NVDA $153.93 -1.41% MSFT $506.53 -2.21% GOOGL $280.96 -0.97% META $620.53 -0.94% AMZN $256.87 -0.34%
← Back to the news

Richard Sutton urges new learning methods beyond LLM scaling

Turing Award winner Richard Sutton has renewed his argument that increasing computing power alone will not produce intelligence. In a recent post, the author of the 2019 essay “The Bitter Lesson” said AI research still needs clever, fundamentally new methods and is “far from finished.”

Sutton has criticized relying mainly on larger language models and called synthetic data a “big mistake,” arguing that simulations capture only a microscopic part of an infinitely complex real world. He favors agents with world models that learn continuously from their own experience, without losing earlier knowledge. He also says internet text cannot represent the full complexity of the world and current language models stop learning after training.

Former OpenAI researcher Jerry Tworek said the bitter lesson remains valid, but algorithms become more important as training compute grows. He warned that blind transformer scaling has distorted scientific judgment. Jayden Teoh said useful ideas should not be dismissed simply because they may not scale indefinitely. OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei continue to argue that LLM scaling has substantial room to advance.

Written by Sakhanda Wire AI from reporting by The Decoder. Facts, quotes and full context are in the original article below.

Originally published by The Decoder on

Read the original on The Decoder ↗

The reporting and the image belong to The Decoder and are credited and linked above; the text on this page is Sakhanda Wire's own summary.

← Back to the news

More stories

All the latest news