by Malcolm Murray Just came back from a trip to China, my first in a decade and my first focused specifically on AI risk. Covering Beijing as well as Shanghai, conferences as well as university events, this was a highly illuminating trip from several perspectives. China has continued to develop extremely quickly since my last…

by Malcolm Murray Javier Milei’s libertarian government in Argentina recently laid out a new framework for corporations in the country. Among the various simplifications to attract more investment, was a highly interesting one – the creation of Sociedad Automatizada – automated corporations run by AI agents without the need for any human involvement. This could…

by Malcolm Murray In a recent study we conducted on AI risk estimation, something potentially interesting about the nature of LLMs emerged as a side effect. In the risk assessment studies we typically run, a group of experts are asked to provide estimates of various parameters and how they vary with different levels of use…

by Malcolm Murray The AI market continues to evolve and surprise. In recent months, Anthropic withheld their latest model Mythos, OpenAI made a U-turn and started experimenting with ads, and Meta bought a “social network for AIs”. This could point to increased divergence in AI companies’ business models. While this might increase AI risk to…

by Malcolm Murray In forecasting the evolution of AI, we are rapidly losing the middle. The outcomes are diverging to the extent that it no longer makes sense to think about “an average timeline” or “average outcomes”. Many people are now tracking prediction markets for when “AGI” will arrive. The current Metaculus prediction is for…

by Malcolm Murray Over the past year, there has been significant movement in AI risk management, with leading providers publishing safety frameworks over the past year that function as AI risk management. However, the problem is that these are not actually proper risk management when you compare them to established practice in other high-risk industries.…

by Malcolm Murray Moltbook, the social network for AIs that launched only last week and already has more than a million AI agents, is a clear example of how little we can foresee of the risks that will arise when autonomous AI agents interact. In the first week, the AIs on Moltbook have both already…

by Malcolm Murray Back when I worked for large corporations, people would often talk of being in “period of change” or how they could “see the light at the end of the tunnel” after a period of heavy restructuring or similar. These days, you might be forgiven for wondering where the tunnel went. Change is…

by Malcolm Murray As I discussed AI with my cabdriver on a recent trip to Vienna, I was reminded of the fact that the German word for AI is Künstliche Intelligenz. This shares an etymological root with the word Kunst, German for Art. This made me realize that of course we have that in English…

by Malcolm Murray The question of the day on everyone’s minds is whether AI is a boom or a bust. But if we lift our eyes ever so slightly from the question of the day and look at the bigger picture, two bigger questions come into view. One: Will AI take our jobs? This debate…

by Malcolm Murray I recently finished reading Rewiring Democracy: How AI Will Transform Our Politics, Government and Citizenship – a book by Bruce Schneier and Nathan Sanders on the effects of AI on democracy. It comes out soon (October 25). It is a good read, worth reading for its myriad examples of AI in action…

by Malcolm Murray “A clearly written and compelling account of the existential risks that highly advanced AI could pose to humanity.” — Ben Bernanke “Humans are lucky to have Yudkowsky and Soares in our corner, reminding us not to waste the brief window that we have to make decisions about our future.”— Grimes Probably the…

by Malcolm Murray Recently, the system card for Claude 4 showed a fascinating finding – what happens when you let two Claudes engage in a prolonged dialogue with each other? They end up meditating and sharing Buddhist wisdom, such as “In perfect stillness, consciousness recognizes consciousness, and the eternal dance continues”. Especially, they share nondual…

by Malcolm Murray If you in any way follow AI policy, you will likely have heard that the EU AI Act’s Code of Practice (CoP) was released on July 10. This is one of the major developments in AI policy this year. 2025 has otherwise been fairly negative for AI safety and risk – the…

by Malcolm Murray When it comes to AI, or even worse, “AGI”, we are facing a crisis of language. Different people use the terms to mean drastically different things. This is deeply unhelpful for productive debate. This point was hammered home to me several times this week. On LinkedIn, I debated appropriate risk management techniques for…

by Malcolm Murray Aella, a well-known rationalist blogger, famously claimed she no longer saves for retirement since she believes Artificial General Intelligence (AGI) will change everything long before retirement would become relevant. I’ve been thinking lately about how one should invest for AGI, and I think it begs a bigger question of how much one…

by Malcolm Murray Enrico Fermi famously asked – allegedly out loud over lunch in the cafeteria – “Where is everybody?”, as he realized the disconnect between the large number of habitable planets in the universe and the number of alien civilizations we actually had observed. Today, we could in a similar vein ask ourselves, “Where…

by Malcolm Murray By 2030, we will have countries of geniuses in data centers, but we won’t know what to do with them. In my 10+ years as a Superforecaster, I have picked up many techniques and lessons regarding forecasting. Given the fractured state of the debate on Artificial General Intelligence (AGI) and AI progress,…

by Malcolm Murray The Paris AI Summit the other week might have been the end of a 10-year run for AI safety as Azeem Azhar, the creator of Exponential View, put it. The concept of AI safety, which can be said to have started in earnest with Nick Bostrom’s 2014 book Superintelligence, had a 10-year…

by Malcolm Murray As someone who thinks about AI day-in and day-out, it is always fascinating to see which events in the AI space break out of the AI bubble and into the attention of the wider public. ChatGPT in November 2022 was of course one. The podcast-creating ability of Google’s NotebookLM almost got there,…