> Every human insight can eventually be rediscovered and made rigorous by data (in chess, at least!) In the short term, those insights are useful, but in the longer term, they add nothing at all.

But isn't that the bulk of what we're shoving into LLMs, and it makes them much smarter? If it's useful there but not in a chess AI then that seems like a significant crack in the bitter lesson.

> Note also that "raw gameplay" here can mean new games -- you can generate as much data as you need, you don't need to rely on real recorded games.

Yeah I mentioned that, generated games are useful. But if we're being fair and letting both AIs use generated game data, does the more general LLM ever actually overtake the specialized stockfish like the bitter lesson suggests?

Another way to look at this is that giving the LLM the commentary is a way to avoid complaints of hiding data from the LLM, since it'll have strictly more info than stockfish. But if we cut that from the training data and only give it a basic description of chess and lots of raw game data then it's going to get even worse than it already is at chess. Meanwhile stockfish never had that commentary, just actual hand-crafted heuristics and training on game data, and it's very strong.