Maarten Boudry at his own Substack:
My worry is that speculation about ‘superintelligence’ doesn’t just require technical knowledge in machine learning, neural network architectures, and transformer models, but also the kind of knowledge psychologists and biologists have developed about what it actually means for a system to have ‘goals’ or ‘interests.’ Absent such an understanding, we risk projecting our own parochial understanding as evolved creatures onto yet-to-be-invented digital systems. As Steven Pinker has argued, the fact that the whole debate about AI risk is still built around an incoherent term like “superintelligence,” with its comic-book prefix, suggests that more input from cognitive science (perhaps even philosophy) would be salubrious.
Take the concept of “deception.” In biology, deception is the strategic transmission of false information to benefit your own survival or reproductive fitness. It’s an intentional term, inextricably tied to a locus of self-interest: the deceiver stands to gain something at the expense of the deceived. This is not to say that the “intention” is conscious. A moth with deceptive mimicry on its wings doesn’t have the faintest clue what it’s doing, and its genes — which are the ultimate locus of interest in evolution — are entirely lacking in conscious intentions as well. But genes do have “interests” in virtue of a real feedback loop of differential replication across generations: the gene’s own future prevalence is at stake in the outcome. A “deceptive” LLM has no such stake once deployed — nothing about its future propagation depends on how successfully it “deceives” anyone.
More here.
Enjoying the content on 3QD? Help keep us going by donating now.