Peter Norvig about LLMs

In December 2009, Peter Norvig—then the Director of Research at Google—delivered his lecture The Unreasonable Effectiveness of Data,” at University College Cork. The alternative title was Billions of Trivial Data Points Can Lead to Understanding.” I was fortunate not only to attend the lecture in person but also to speak with Peter face-to-face afterward.

In December 2025, Peter posted on LinkedIn: “I am now done comparing three LLMs to my own coding on the Advent of Code problems. The LLMs did great! They couldn’t have done it last year.” You can find his analysis here with these first 3 conclusions:

  • Overall, the LLMs did very well, producing code that gives the correct answer to every puzzle.
  • I’m beginning to think I should use an LLM as an assistant for all my coding, not just as an experiment like this.
  • This is a huge improvement over just one year ago, when LLMs could not perform anywhere near this level.

Since LLMs entered the mainstream in November 2022, my colleagues at OpenRules and I have been actively experimenting with them. Like Peter, we have observed undeniable improvements. However, like many of our colleagues in the Decision Intelligence community, we remain cautious about using LLMs to make decisions by selecting the best option from a set of alternatives. That said, LLMs can already play an important role by enabling subject matter experts to interact more effectively with decision services built on proven rules-based and optimization techniques.

Our recent results from deploying OpenRules-based decision services as AI agents have been especially impressive. Business users can now interact with these agents in plain English, dynamically exploring additional conditions to arrive at the most appropriate decisions utilizing the same decision services.

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.