Daniel Bashir at Asterisk:
AI alignment has always started with the question: If, one day, we develop superintelligent machines, how do we ensure they do what we actually want? In the field’s prehistory, Norbert Wiener warned that we must ensure the “purpose put into the machine” is the purpose that we really desire. If a powerful AI takes shortcuts to achieve our goals, or misunderstands what those goals are, it might have unintended effects.
Wiener’s concern reflected a specific idea of what a machine’s “purpose” could be. In 1943, Wiener, along with Arturo Rosenblueth and Julian Bigelow, explained that it was meaningful to call some machines purposeful, in that they behaved so as to reach a target state and were corrected by negative feedback. A thermostat, for instance, is not conscious, but could be accurately described as “trying to” reach 68 degrees by checking the current temperature, comparing it with the target temperature, and taking some action to close the gap.
This teleological view was soon applied to AI systems that might have a goal more complex than that of a thermostat. In 1944, John von Neumann and Oskar Morgenstern argued that under some conditions, one can represent a person’s preferences with a utility function.
More here.
Enjoying the content on 3QD? Help keep us going by donating now.