GPT-4: The Bitterer Lesson
Something to think about
Richard Sutton’s Bitter Lesson is, simplifying, that in the long-term it’s always better to let computation take the burden of finding greater AI systems (through learning or search algorithms) instead of trying to find them ourselves by leveraging our knowledge.
In Sutton’s view, humans have contributed little to the best AI systems we’ve built: We’ve become background characters to this show of artificial phenomena that succeeds not because of us (sometimes even despite us and our sense of self-importance), rather than the intelligent architects of the future we thought we’d be. This Sutton asserted almost two years ago (March 2019). He wasn’t short of evidence and good arguments to defend his point, which, if true, is certainly bitter.
He didn’t claim humans are entirely irrelevant — of course, there’s outstanding ingenuity and deep expertise in building Deep Blue and AlphaGo (or ChatGPT and GPT-4, for that matter). He meant that once more computation is available, one unit of this mystical power is worth more to increase the performance of an AI system on a given task than one unit of human knowledge (whatever that means).
And, although we wouldn’t have possibly agreed that all problems were subjected to this reality back then (e.g., not every solution is a matter of…
