by Muhammad Aurangzeb Ahmad
Source: Wikipedia
In an earlier piece about teaching machines to predict death, I opened with the old idea that a person often meets their destiny on the road they take to avoid it. Stories around this themes are often read as parables of trying to evade prophecy where the effort to escape a fate becomes the mechanism that delivers it. Lately, I’ve been thinking that there’s another way to read these stories and parables. Each of them is, before it is anything else, a prophecy made about someone who has not yet been born, or has only just arrived, and cannot yet speak on his own behalf. In the story of Krishna, Kamsa is told that the eighth child of his sister will be his undoing. He starts killing her children as they come. The seventh is miraculously transferred to another womb, and the eighth is Krishna. In the story of Oedipus, Laius hears what his son will do and drives a pin through the infant’s feet and leaves him on a mountainside. The common theme here is that the prophecy comes first and the child arrives into a world that has already decided what he is.
I have spent the better part of a decade on problems and converns around end of life. This essay, and the one that will follow it, are about the mirror image of those concern: They are about the beginning of life, which I have realized are an equally interesting case. Consider this, at the end of life, we may entertain the possibility of an algorithm being able to speak for a person who once had a voice. We can at least ask whether it gets that person right. We can even ask whether the preferences it reconstructs are ones the patient would have recognized. At the beginning of life, the algorithm speaks about a person who has never had a voice at all. And in a growing number of clinics it does more than speak about that person. It helps decide whether that person will exist or not! Consider what already happens in a fertility clinic. A cycle of in vitro fertilization typically produces more embryos than a family will use, and someone has to decide which one to transfer first. For decades that decision rested on an embryologist looking through a microscope and grading each embryo by its shape and its rate of division, a practice that is skilled, subjective, and inconsistent between one laboratory and the next. Into that gap has come a class of deep learning systems that watch time-lapse footage of a developing embryo and return a single number meant to express its chance of implanting. One such system is iDAScore which produces a score between 1.0 and 9.9 which moves an embryo toward the front of the queue.
When iDAScore was finally put through a proper randomized trial across fourteen clinics in Australia and Europe, with more than a thousand patients, deep learning selection did not deliver higher pregnancy rates than trained embryologists. The clinical pregnancy rate came in at 46.5 percent for the algorithm and 48.2 percent for the humans. The technology’s defenders correctly point out that a machine is at least consistent where a tired embryologist on a Friday afternoon is not. This way of looking at things has an interesting implication: The embryo is no longer a possibility that a clinician holds in mind. It is a ranked entry on a list, and the ranking carries the authority that numbers acquire when they appear on an official screen. This is the domain where clinical AI has the potential to do the quietest damage. Consider this, when a mortality model in an intensive care unit assigns a patient a high risk of death, it makes a prediction about a person who exists and whose interests we are obliged to protect. On the other hand, when an embryo selection model ranks five embryos, it is not predicting the future of a person. It is choosing which person there will be. Vocabulary from philosopher Derek Parfit can help us explore this problem. He describes the non-identity problem as, “It arises from the observation that even small changes can alter the timing and circumstances of child conception, leading to entirely different individuals coming into existence.” The embryo that is not selected is not harmed, because there is no one there to be harmed; a different person simply comes into the world instead. However, there is still this lingering feeling that the acting of choosing is far more consequential than any ordinary medical decision. It is precisely because it is not medicine acting on a patient but selection acting on the set of possible patients. We have built, and normalized, and folded into the ordinary billing of a fertility cycle, a machine whose output is a person.
If ranking embryos by their odds of implanting unsettles us only a little, the next step should unsettle us even more. A small number of companies, Orchid and Genomic Prediction chief among them, now offer what is called polygenic embryo screening. They read an embryo’s genome and return polygenic risk scores for conditions that arise from thousands of genetic variants acting together, among them heart disease, several cancers, type 2 diabetes, and schizophrenia. Orchid invites prospective parents to “identify your healthiest embryo.” The company LifeView has used the phrase “choice over chance.” The scores are then folded back into the same ranking logic, so that the number attached to each embryo now claims to summarize not just whether it will implant but what kind of life it will go on to have. One has to be careful here, because it is easy to let the science-fiction resonance of the phrase “designer baby” do work that the evidence does not support. The predictive power of a polygenic score for a complex adult disease, computed from a handful of cells in a five-day-old embryo, is low. These are diseases shaped by decades of environment, diet, chance, and epigenetic accident, and the fraction of that risk a genome can foresee is modest. A family that selects the embryo with the lowest cardiac risk score may be reducing a lifetime probability by a few percentage points, or by nothing detectable at all. The certainty implied by a clean number on a report is not a certainty the underlying biology can honor.
The deeper trouble is not accuracy but direction. Genomic Prediction, at one point, offered screening for short stature and for low intelligence, and withdrew both offerings only after a public outcry over eugenics. It is important to note that the capability did not disappear when the product line did. Thus, nothing about the method distinguishes a score for schizophrenia risk from a score for predicted height, except our present discomfort. One could even argue that discomfort is a moving line. When a consortium of academic geneticists discovered that its research data was being used to power some of this commercial screening, its members objected in language that is rare in the measured world of genetics. Disability rights scholars have long made a related argument, sometimes called the expressivist critique, that selecting against a trait sends a message about the worth of the people who live with it. When the selection is performed by an algorithm, that message acquires a false neutrality. Now let’s move forward in time, past selection and implantation and birth, into the neonatal intensive care unit. There are now machine learning models that predict which critically ill newborns will die, and they are good, at least by the narrow measures we use to judge them. Reviews of the field describe systems reaching areas under the curve above 0.9, comfortably outperforming the older models that clinicians relied on for a generation. The published rationale is humane. A reliable early estimate of risk lets a team intervene faster and, in the words of one study, communicate with families in advance.
Most of what I argued about mortality prediction at the end of life applies here, with one major difference. The adult in the intensive care unit has a past. The model has labs, a history, a documented trajectory, and however imperfectly, it is reasoning about this person. The newborn has almost no past at all. The model reasons about a statistical cohort of infants who resembled this one on a handful of measurements, and it treats that cohort as a proxy for a life that has not yet had time to become singular. A predicted death at eighty-five is a forecast laid over the end of a biography. A predicted death on the third day of life is not laid over a biography. It stands in for one that has not been written, and it does so at the exact moment a family and a clinical team are deciding how hard to fight. This is where the self-fulfilling prophecy I described in the intensive care unit becomes most acute. A mortality model can be a harmful prophecy that stays statistically accurate precisely because clinicians act on it, easing toward comfort care when a score is high and thereby helping to produce the outcome the score predicted. In the neonatal unit the same loop runs through a counseling conversation with parents. A number offered in good faith, meant to help a family prepare, can shade the tenor of that conversation, and the conversation shapes the decisions, and the decisions shape whether the infant lives. The model does not know that a serious but treatable condition and an irreversible one may sit at the same point on its scale. It offers the same number to both. And the literature is candid about a further weakness that should sound familiar: most of these models are built on data from a single hospital and are rarely validated anywhere else. A model trained on the infants of one academic medical center is not thereby a model of infants.
There is one more figure at the beginning of life worth naming: A child born today begins to accumulate a data shadow before it draws breath. Ultrasound images run through classifiers, cell-free fetal DNA is sequenced from the mother’s blood, an app charts the pregnancy week by week, a due date is announced to a social network, and a name is sometimes chosen and broadcast to an audience of hundreds before there is anyone to answer to it. I have written a great deal about the digital doppelganger, the model of a person assembled from the traces they leave. In the current setting, we have a data double that precedes rather than survives its original. This would be akin to a portrait sketched before the sitter exists, into which the child will be asked, in effect, to grow. I should however caution that none of what I have described so far should be taken to bean argument that these tools should not exist. That would be false to the parents who conceive a healthy child after years of loss because an embryologist, aided by a score, chose well. It would be false to the neonatologist whose early warning bought a family the hours it needed. As I have said before about their counterparts at the end of life, the tools are real answers to real needs, and the objection is not to their existence but to the confidence with which we deploy them and the ease with which the number is allowed to stand in for the person. What is genuinely new at the beginning of life is that the prediction is not merely descriptive. It is constitutive. What this means is that it does not forecast a life that is already underway. It participates in deciding which life there will be, and then it hands that life, on the day it begins, a profile it had no part in writing and no capacity to contest.
That is the thread I want to follow into the second part of this essay. Every framework we have built for responsible artificial intelligence, consent, autonomy, contestability, the right to an explanation, presumes a subject who can eventually push back. The embryo cannot. The fetus cannot. The newborn cannot. The beginning of life is the purest test we have of what our ethics of prediction amount to when there is no one there yet to protect, and it exposes how much of what we call responsible AI has quietly assumed a grown adult all along. In the next essay I will take up what it means to be born already scored, why the older forms of prophecy that greeted a newborn were gentler than the one we are building, and what a handful of religious and philosophical traditions, which have thought longer than we have about when a person begins, might have to say to an industry that has started answering that question by default.