Sean Carroll at Preposterous Universe:
The Artificial Intelligence landscape is changing with remarkable speed these days, and the capability of Large Language Models in particular has led to speculation (and hope, and fear) that we could be on the verge of achieving Artificial General Intelligence. I don’t think so. Or at least, while what is being achieved is legitimately impressive, it’s not anything like the kind of thinking that is done by human beings. LLMs do not model the world in the same way we do, nor are they driven by the same kinds of feelings and motivations. It is therefore extremely misleading to throw around words like “intelligence” and “values” without thinking carefully about what is meant in this new context.
More here.