Ultra-Large AI Models Are Over
The end of “scale is all you need” is near
I don’t mean ‘over’ as in “you won’t see a new large AI model ever again” but as in “AI companies have reasons to not pursue them as a core research goal — indefinitely.”
Don’t get me wrong. This article isn’t a critique of the past years — even if I don’t buy the “scale is all you need” argument, I acknowledge just how far scaling has advanced the field.
Parallelism can be drawn between the 2020–2022 scaling race and — keeping the distance — the 50s-70s space race. Both advanced science significantly as a byproduct of other intentions.
But there’s a key distinction.
While space exploration was innovative in nature, the quest for novelty isn’t present in the “bigger is better” AI trend: To conquer space, the US and USSR had to design novel paths toward a clear goal. In contrast, AI companies have blindly followed a predefined path without knowing why or whether it’d lead us anywhere.
You can’t put the cart before the horse.
That makes all the difference and explains why and how we’ve got here.
