by Ashutosh Jogalekar
‘Areopagitica‘ was a famous speech delivered by the poet John Milton in the English Parliament in 1644, arguing for the unlicensed printing of books. It is one of the most famous speeches in favor of freedom of expression. Milton was arguing against a parliamentary ordinance requiring authors to get a license for their works before they could be published. Delivered during the height of the English Civil War, Milton was well aware of the power of words to inspire as well as incite. He said,
For books are not absolutely dead things, but do preserve as in a vial the purest efficacy and extraction of that living intellect that bred them. I know they are as lively, and as vigorously productive, as those fabulous Dragon’s teeth; and being sown up and down, may chance to spring up armed men…
What Milton was saying is not that books and words can never incite, but that it would be folly to restrict or ban them before they have been published. This appeal toward withholding restraint before publication found its way into the United States Constitution and has been a pillar of freedom of expression and the press since.
Why was Milton opposed to pre-publication restrictions on books? Not just because he realized that it was a matter of personal liberty, but because he realized that restricting a book’s contents means restricting the very power of the human mind to come up with new ideas. He powerfully reminded Parliament,
Who kills a man kills a reasonable creature, God’s image; but he who destroys a good book, kills reason itself, kills the image of God, as it were, in the eye. Many a man lives a burden to the earth; but a good book is the precious lifeblood of a master spirit, embalmed and treasured up on purpose to a life beyond life.
Milton saw quite clearly that the problem with limiting publication is in significant part a problem with trying to figure out all the places a book can go. The same problem arises with science.
Scientific ideas are impossible to truly label as good or evil before they leave the lab. By default almost every scientific idea can be used towards both ends and is hence ‘dual use’. It is obvious that the discovery of the structure of DNA or that of nuclear fission provided humanity with both the means to save as well as damn itself. But in fact there is a deeper challenge here. Just like the morality of science cannot be predicted in spite of our best intentions, so can we also not decide how research should be planned. The discovery of atomic energy provides a good example: it was the result of basic, curiosity-driven research, not of planning by either a beneficent angel or an evil overlord – it just happened and nobody could have seen it coming. A few scientists got interested in exploring the structure of the atom from the 1900s onwards, and in 1938 this work culminated in the accidental discovery of fission. Whether fission should be used for generating peaceful nuclear power or to build atomic weapons was a dilemma confronting humanity after the fact.
Just like Milton’s books, science therefore thrives in an environment that’s both morally agnostic and unplanned. Scientists have to deal with this uneasy balance between being free to do what they like and needing to face the consequences of their unplanned actions, but if science is to thrive that’s the way it has to be. The physicist Freeman Dyson once defined science as “organized unpredictability”. He said, “The best scientists arrange things in experiment to be as unpredictable as possible.” Thus, they cannot think about the morality of their findings beforehand, nor do they expect they will be able to predict what happens. Of course, scientists should be as morally responsible as anyone else to makes sure that the fruits of the research are used to benefit humanity, and they should refrain from working on an idea whose malicious use is obvious. But the way good science works, they can usually police and regulate their discoveries only after the fact, not before.
This principle of organized unpredictability and moral agnosticism applies to technology as well. The Internet was a result of a hive mind of organizations and individuals working together across space and time to discover a means of rapid communication whose sheer scope and creativity would have strained their own imaginations. Nobody realized it back then that this tool could be used for organizing aid to pandemic victims and for organizing rallies by terrorists, for educating children in the wonders of science and nature and for indoctrinating them with pernicious religious or nationalist ideology. The results were unpredictable, and the best we could do is to keep a careful account of how the technology is used and then try to regulate it.
The caveat applies even to seemingly obvious and benevolent intentions, like those seeking to prohibit or restrict gain-of -function research on viruses in the wake of the coronavirus pandemic. Gain-of-function is where the boundaries between science and its applications become blurred; it is both a way to engineer biological systems for understanding biology better as well as a way to improve biological systems for human applications. Clearly a lot of gain-of-function research can lead to the creation of more virulent pathogens, but some of it can also be beneficial. Gain-of-function research for specific proteins has led to major medical breakthroughs in the last few decades, including drugs for cystic fibrosis and heart disease. Restricting all gain-of-function research would also mean restricting that which can lead to lifesaving therapies.
I think of Areopagitica, genetic engineering and nuclear power when I read about recent efforts to regulate artificial intelligence (AI) and machine learning (ML) from an ethical standpoint. Organizations have created ethics bodies to supervise the work their computer scientists are doing, there are now journals devoted to the topic and departments are creating majors and formal programs in the ethics of AI. Some are trying to ground the very creation of AI algorithms in the latest social theories pertaining to race. One of the major conferences in the field is now asking researchers submitting papers to the conference to submit statements of ethical or broader impact of their work. There is an obvious reason why all these things are being done. A lot of this work goes under the title of “algorithmic fairness”. It has exploded because, in the past few years, the impact of racial, gender and other kinds of bias on algorithms has become obvious. The need to make sure that the algorithms do not result in discrimination has become painfully apparent in certain cases.
These are efforts whose intentions I applaud. Especially in the case of companies using AI in their products and with a stated view to benefiting customers, they need to make sure that their products do not impact any subpopulation of their customers unfavorably. But I reluctantly have to conclude that these efforts to regulate the ethical implications of AI are doomed to fail if the technology is pre-constrained too much, for the same reason that any attempts to regulate the application of genetic engineering would have failed. It is not a matter of well-intentioned morality but of organized unpredictability.
Firstly, it is impossible to predict whether a given machine learning algorithm can do something good or bad when it makes its way into a constantly changing world. To quote historian of science George Dyson, “No matter how much digital horsepower you have at your disposal, there is no systematic way to determine, in advance, what every given string of code is going to do except to let the codes run, and find out.” In fact, not only can a well-meaning technology be used for evil, but an evil technology could be turned around to helping people. For instance, consider the transformation of mustard gas, a murderous chemical weapon developed during World War 1. Used for killing and maiming millions by the German Army during the war, it was later repurposed to become the basis for lifesaving cancer drugs that kill both cancer and normal cells by similar mechanisms. I wonder what would have happened if there had been a Committee to Evaluate the Social Impact of Reactive Gases that decided that, with its potential to kill normal cells, mustard gas research should be restricted. Would it still have resulted in the discovery of cancer drugs based on these agents? I am not arguing that chemical weapons should have been developed by Germany; I am arguing that it would have been impossible to predict the ultimate fate of these weapons.
Secondly, trying to predict and then restrain the rise of the machines is a self-defeating exercise. As the field of machine intelligence grows and become more self-aware, it also becomes more unpredictable. Restricting this organized unpredictability means restricting the novel things that those algorithms discover. No bureaucratic machinery or tech company committee is smart enough to predict everything an intelligent, non-deterministic algorithm may or may not do. Almost any ultimate use the overlords predict for the technology will most likely turn out to not just work but may also blind us to the novel, useful discoveries the technology might make. Like children, algorithms live and thrive best when they are allowed unfettered freedom with minimum supervision. Attempting to do otherwise is tantamount to trying to dam the ocean. If we try to pre-constrain algorithms in the wild too much, we will be stuck with a situation in which any algorithm satisfying our pre-constrained regulatory or moral framework will be too predictable to do anything interesting.
With algorithms, the best we can do is to ensure that the data the algorithm is based on is not biased. After that we need to look at the results and only then decide how to regulate untoward behavior. The point about data is an important one though, and is similar to considerations of data acquired from clinical trials on drugs. When a pharmaceutical company conducts clinical trials, for reasons of practicality, even the biggest clinical trials have to be conducted on a small fraction of the patient population that the drug will eventually impact when released into the market. It behooves the company to ensure that the subpopulation it is testing the drug on is representative of the larger population. Often the problems with the larger population become clear only when the drug is prescribed worldwide, and there have been cases in which companies have had to withdraw that drug from the market. Similarly when a tech company is releasing an algorithm for face recognition, for instance, it behooves the company to ensure that the training data used to develop the algorithm is representative of the larger population. For instance if people of certain races are not represented in the training data, the algorithm may fail to make decisions when applied to this excluded data, or even worse, make decisions impacting this group adversely. This kind of data diligence is quite important. But this data regulation is different from regulating the fate of the algorithms themselves by trying to predict everything that they may or may not do when they are out there.
Unfortunately, while bureaucratic overreach isn’t smart enough, it can certainly be powerful enough. By trying to put prior restraints on the release of algorithms, we will make the same mistake Milton’s censors were making in trying to restrict books before their publication. We will stifle the myriad possibilities inherent in an evolving new technology and the unintended effects that it will foster among new communities who can extend its reach into novel and previously unimaginable avenues. In many ways it will defeat our very goals for new technology, which is its ability to evolve, change and transform the world for the better. Like the Dragon’s teeth of yore, let’s not go around digging for potential future phantoms but instead tame them when they become real and stand tall before us.