by Jochen Szangolies

When the Luddites smashed automatic looms in protest, what they saw threatened was their livelihoods: work that had required the attention of a seasoned artisan could now be performed by much lower-skilled workers, making them more easily replaceable and thus without leverage to push back against deteriorating working conditions. Today, many employees find themselves worrying about the prospect of being replaced by AI ‘agents’ capable of both producing and ingesting large volumes of textual data in negligible time.

But the threat of AI is not ‘merely’ that of cheap labor. As depicted in cautionary tales such as The Terminator or Matrix, many perceive the true risk of AI to be that of rising up against its creators to dominate or enslave them. While there might be a bit of transference going on here, certainly there is ample reason for caution in contemplating the creation of an intelligence equal to or greater than our own—especially if it at the same time might lack many of our weaknesses, such as requiring a physical body or being tied to a single location.

Besides these two, there is another, less often remarked upon, threat to what singer-songwriter Nick Cave has called ‘the soul of the world’: art as a means to take human strife and from it craft meaning, focusing instead on the finished end product as commodity. Art is born in the artist’s struggle with the world, and this struggle gives it meaning; the ‘promise’ of generative AI is to leapfrog all of the troublesome uncertainty and strife, leaving us only with the husk of the finished product.

I believe these two issues are deeply connected: to put it bluntly, we will not solve the problem of AI alignment without giving it a soul. The ‘soul’ I am referring to here is not an ethereal substance or animating power, but simply the ability to take creative action in the world, to originate something genuinely creatively novel—true agency, which is something all current AI lacks. Meaningful choice is a precondition to both originating novel works of art and to becoming an authentic moral subject. AI alignment can’t be solved by a fixed code of moral axioms, simply because action is not determined by rational deduction, but is compelled by the affect of actually experiencing a given situation. Let’s try to unpack this. Read more »