LLM world model

🧠 Does AI understand our world?

One hypothesis is that the innocent task of predicting the next word, which is what AI is trained on, requires the model to build a world model in order to be able to make sensible predictions of which words come next. In other words, it needs to understand how our world works to give coherent continuations that follow common sense, physics etc. But is AI really building an ever more accurate world model or just fooling us in thinking it does? 🤔

One way to examine this question is on whether AI can reconstruct a more restricted world model like a map 🗺️, using the equivalent of next word prediction, which would be predicting a valid route from point A to point B. We can easily train a model to do that from taxi rides, so does that create an accurate representation of a given map?

The short answer is no ❌ And while this is not conclusive evidence that AI is not understanding the world, it’s worth keeping in mind that it may not in applications where that matters like health 🩺, law ⚖️ and any other high risk application ‼️ You don’t want AI to drive you off a cliff on the way home, because of a minor misunderstanding of its surroundings even if most of the time it seems to work…

🔗 Read more https://arxiv.org/pdf/2406.03689