by Muhammad Aurangzeb Ahmad
In the first part of this series I described the data footprint that now surrounds a child before birth. It could take the form of an implantation score attached to an embryo, a polygenic risk estimate read from a few of its cells, a sequenced genome, growth curves etc. These may seem harmless or even mundane but the The technology is moving toward something much more ambitious i.e., towards a prediction of the life itself.
Consider the following cautionary tale: an AI model called life2vec was developed by researchers working with Danish national registry data. The data is a summary of recorded lives of roughly six million people. It includes things like Births, jobs, illnesses, relocations, incomes and diagnoses become events in a sequence. The model reads them much as a language model reads words in a sentence and learns what tends to come next. The analogy being that a human life can be represented as a sequence and, having read enough of that sequence, a machine can try to read forward. Once trained, life2vec could predict several kinds of outcomes, including personality characteristics and early mortality. In the tasks reported by its developers it outperformed specialized baselines. The researchers have also been careful about what this means. The model does not tell someone the date on which they will die. It was not released as a public “death calculator,” and its creators objected when it was described that way.
Now let’s get back to our main topic, the embryo grade and neonatal probability I discussed earlier were judgments made at one point in time. The systems now being developed can potentially go way beyond that. Foundation models trained on long stretches of electronic health records can absorb a person’s accumulating clinical history and repeatedly revise their estimates. A risk calculated in the neonatal unit could therefore become the first entry in a forecast that follows the patient for years. Genetic predictions are even easier to carry forward. Older forms of prophecy rarely had this institutional reach. A natal horoscope did not really set an insurance premium. A machine-readable risk score might do that if appropriate safeguards are not added to the system. That said, one should note that We are hardly the first people to imagine a life that was written ahead of time. Cultures around the world have greeted newborns with attempts to read what lies ahead e.g., the horoscope cast at the hour of birth, the naming rite that places a child within a lineage, the janam kundali of South Asia, the astrologers who once stood beside royal cradles etc. It is tempting to see algorithmic prediction as another version of the same old habit, updated for the age of data. There is some truth in that. But the older predictions could be ignored, argued with, softened by ritual or simply outgrown. Also, they usually did not plug directly into the institutions that distribute the practical advantages of a life.
There are however many historical precedents in various global traditional: In the Islamic tradition there is hadith (saying of Prophet Muhammad) that describes the stages by which a human being is formed. It then says that an angel is sent to breathe the soul into the fetus and to write four things: its provision, the length of its life, its deeds, and whether its end will be wretched or blessed. It is difficult not to notice the resemblance to the categories that interest modern prediction. How long will this person live? What will they do? What will their circumstances be? How will their life turn out? One can see the tension between free-will and fatalism in trying to interpret this hadith: Muslim theologians spent centuries trying to prevent it from becoming fatalism. This is the old argument over qadr (divine decree) and free-will that appears in Islamic theology: if everything has already been written, perhaps the person living the life is only a spectator to it. Mainstream Islamic theology resisted that conclusion. The Ashʿari school (one of the three main theological schools in Sunni Islam) developed the doctrine of kasb, usually translated as acquisition to explain this. According to this concept God creates the act while the human being acquires it and remains responsible for it. The Maturidi tradition within Islam approached the problem somewhat differently but likewise tried to preserve human responsibility without surrendering divine knowledge and power.
One need not settle the metaphysics to understand what all this intellectual labor was trying to protect. The theologians wanted to keep “already known” from collapsing into “therefore compelled.” Divine foreknowledge does not mean that human responsibility is meaningless. Algorithmic prediction creates a different problem because the prediction itself can enter the chain of causes. Machine learning researchers now call one version of this performative prediction: a prediction changes behavior, and that changed behavior alters the outcome being predicted. Consider the following example: A navigation system declares one road faster; drivers converge on it; the road becomes congested. Or consider another example where a caseworker sees someone labeled high-risk and intervenes more aggressively; the intervention changes what happens next. The prediction is has now become one of the things producing it. I traced a version of this loop in my essay on machines that predict death and again in the neonatal unit in the first part of this essay. The way that it plays out is that a mortality estimate enters a clinical decision. Clinicians and families respond to this and their responses in turn change treatment which in turn affects the outcome against which the original prediction is judged.
The comparison with qadar does not go all the way. Muslim theologians were concerned with how God could know a person’s future without thereby forcing that person to live it. Algorithmic prediction introduces a problem they did not have to contend with i.e., the prediction itself can change what happens. Once a forecast is taken seriously by parents, doctors, teachers, insurers, or other institutions, their decisions begin to reflect it. The prediction then becomes part of the chain of events producing the outcome it originally claimed only to anticipate. A similar concern appears in a quite different setting in Joel Feinberg’s essay on the child’s “right to an open future.” Feinberg was concerned with choices made by parents and communities that might prematurely close possibilities a child should later have the opportunity to choose for themselves. He was not thinking about machine learning, but the idea becomes particularly relevant when predictions are made very early in life.
Suppose a child carries a risk score that remains in the record for years. A teacher who sees it may expect less from the child. Parents may become unusually cautious about certain activities. A physician may interpret later symptoms in light of the earlier prediction, while an insurer may eventually attach a price to the same risk. None of these decisions needs to be dramatic, most likely this will pan out in an incremental fashion. The effect may come from their accumulation. Over time, a prediction that began as a statement about what might happen can help determine which possibilities remain available.
The beginning of life makes another limitation of responsible AI unusually visible. Many of the protections we have developed assume that there is a person capable of invoking them. Someone can refuse consent, challenge a decision, request an explanation, ask for information to be deleted etc. But an embryo cannot refuse to be ranked, a fetus cannot contest a polygenic score calculated from its cells, and a newborn cannot ask why a mortality estimate was placed in its medical record. Much of the current language of responsible AI therefore works better for competent adults than it does for people at the very beginning of life. The usual mechanisms of consent and appeal cannot do much when the person most affected by the prediction cannot yet exercise either. Decisions instead fall to parents, physicians, institutions, and whoever controls the record.
Hindu and Buddhist traditions approach the problem from another direction. In the later parts of my culture of limits series, I discussed traditions in which birth is not understood as an entirely fresh beginning. A person enters a new life bearing the consequences of lives already lived. At first glance there is an odd resemblance here to the modern datafied child: something resembling a record precedes birth and affects what follows. However karma is not a risk file assembled by an institution. It belongs to the moral history of the being whose life it shapes, and what has been inherited can itself be changed through the life that follows. The person remains involved in what becomes of that inheritance. A modern data record works somewhat differently. Other people create it, other people interpret it, and other people may use it when deciding what the person is allowed to do or receive. In some circumstances it can even be transferred or sold. The record can therefore acquire power over the person precisely because it belongs to others around them more than it belongs to them.
None of what we have discussed so far makes prediction inherently undesirable. A neonatal model that identifies danger several hours earlier may save a child’s life. Reproductive technologies can help couples have healthy children after years of difficulty. These are not incidental benefits, and any serious criticism of predictive technology at the beginning of life has to acknowledge them. On the other hand, the problem arises when a useful prediction begins to acquire a larger authority than the evidence warrants. Prediction at the beginning of life occupies an unusual position because it can influence who is born, attach information to that person before they can object to it, and shape later decisions made on their behalf. If the prediction remains in circulation, its influence may last much longer than the circumstances under which it was originally calculated. We do not need to stop making predictions in order to prevent predictions from becoming permanent descriptions of the people they concern. Some forecasts could expire. Certain childhood records could later be sealed. Genetic or developmental scores could be kept away from institutions that have no compelling reason to see them. The practical rules will have to be worked out in medicine, law, insurance, education, and data governance. The larger principle is easier to state. What is written about a child before that child can speak should not be allowed to settle, by itself, what kind of life the child gets to have.