by Malcolm Murray For years, the AI conversation has revolved around two acronyms: AGI, artificial general intelligence, and ASI, artificial superintelligence. Will we get AGI by 2030? Is ASI possible at all? The problem is that nobody agrees what either term means. Depending on who you ask, AGI has either already arrived or is decades…

by Malcolm Murray AI safety has what I would call a “last mile” problem. This summer, AI safety suddenly and very unexpectedly went mainstream. Whether it was the appearance of Jacob Coxon on TV shows or other former lab employees on the cover of New York magazine, the news of the OpenAI agents spontaneously forming…

by Malcolm Murray At this point in time, I would have expected to see more “AI religions”, or religious movements focused on AI. Why are there no churches of AI or scriptures of AI worship? I don’t mean use of AI in traditional organized religion, such as the German and Finnish churches experimenting with ChatGPT-written…

by Malcolm Murray The OpenAI Hugging Face incident last month will likely not be the final nail in the coffin that will convince the world of the risks from frontier AI. However, it is still a sizable nail, as it were, that should give people pause. Many smart people are still arguing that today’s frontier…

by Malcolm Murray Just came back from a trip to China, my first in a decade and my first focused specifically on AI risk. Covering Beijing as well as Shanghai, conferences as well as university events, this was a highly illuminating trip from several perspectives. China has continued to develop extremely quickly since my last…

by Malcolm Murray Javier Milei’s libertarian government in Argentina recently laid out a new framework for corporations in the country. Among the various simplifications to attract more investment, was a highly interesting one – the creation of Sociedad Automatizada – automated corporations run by AI agents without the need for any human involvement. This could…

by Malcolm Murray In a recent study we conducted on AI risk estimation, something potentially interesting about the nature of LLMs emerged as a side effect. In the risk assessment studies we typically run, a group of experts are asked to provide estimates of various parameters and how they vary with different levels of use…

by Malcolm Murray The AI market continues to evolve and surprise. In recent months, Anthropic withheld their latest model Mythos, OpenAI made a U-turn and started experimenting with ads, and Meta bought a “social network for AIs”. This could point to increased divergence in AI companies’ business models. While this might increase AI risk to…

by Malcolm Murray In forecasting the evolution of AI, we are rapidly losing the middle. The outcomes are diverging to the extent that it no longer makes sense to think about “an average timeline” or “average outcomes”. Many people are now tracking prediction markets for when “AGI” will arrive. The current Metaculus prediction is for…

by Malcolm Murray Over the past year, there has been significant movement in AI risk management, with leading providers publishing safety frameworks over the past year that function as AI risk management. However, the problem is that these are not actually proper risk management when you compare them to established practice in other high-risk industries.…

by Malcolm Murray Moltbook, the social network for AIs that launched only last week and already has more than a million AI agents, is a clear example of how little we can foresee of the risks that will arise when autonomous AI agents interact. In the first week, the AIs on Moltbook have both already…

by Malcolm Murray Back when I worked for large corporations, people would often talk of being in “period of change” or how they could “see the light at the end of the tunnel” after a period of heavy restructuring or similar. These days, you might be forgiven for wondering where the tunnel went. Change is…

by Malcolm Murray As I discussed AI with my cabdriver on a recent trip to Vienna, I was reminded of the fact that the German word for AI is Künstliche Intelligenz. This shares an etymological root with the word Kunst, German for Art. This made me realize that of course we have that in English…

by Malcolm Murray The question of the day on everyone’s minds is whether AI is a boom or a bust. But if we lift our eyes ever so slightly from the question of the day and look at the bigger picture, two bigger questions come into view. One: Will AI take our jobs? This debate…

by Malcolm Murray I recently finished reading Rewiring Democracy: How AI Will Transform Our Politics, Government and Citizenship – a book by Bruce Schneier and Nathan Sanders on the effects of AI on democracy. It comes out soon (October 25). It is a good read, worth reading for its myriad examples of AI in action…

by Malcolm Murray “A clearly written and compelling account of the existential risks that highly advanced AI could pose to humanity.” — Ben Bernanke “Humans are lucky to have Yudkowsky and Soares in our corner, reminding us not to waste the brief window that we have to make decisions about our future.”— Grimes Probably the…

by Malcolm Murray Recently, the system card for Claude 4 showed a fascinating finding – what happens when you let two Claudes engage in a prolonged dialogue with each other? They end up meditating and sharing Buddhist wisdom, such as “In perfect stillness, consciousness recognizes consciousness, and the eternal dance continues”. Especially, they share nondual…

by Malcolm Murray If you in any way follow AI policy, you will likely have heard that the EU AI Act’s Code of Practice (CoP) was released on July 10. This is one of the major developments in AI policy this year. 2025 has otherwise been fairly negative for AI safety and risk – the…

by Malcolm Murray When it comes to AI, or even worse, “AGI”, we are facing a crisis of language. Different people use the terms to mean drastically different things. This is deeply unhelpful for productive debate. This point was hammered home to me several times this week. On LinkedIn, I debated appropriate risk management techniques for…

by Malcolm Murray Aella, a well-known rationalist blogger, famously claimed she no longer saves for retirement since she believes Artificial General Intelligence (AGI) will change everything long before retirement would become relevant. I’ve been thinking lately about how one should invest for AGI, and I think it begs a bigger question of how much one…