by Ken MacVey
Out of perhaps narcissistic curiosity, I recently did a search on myself, which also took me to the search engine’s AI mode summary. Its AI mode said that I had been a National Guardsman who shot and killed a student at Kent State in 1970. In case you’re wondering, it’s not true. I have never been a National Guardsman. I have never been to Kent State. I have never shot anyone. And I have never killed anyone.
Horrified, I read the AI summary several times in disbelief. I then quickly re-ran the search to find this time the AI mode summary about me was sketchy but generally accurate, with no reference to Kent State or the National Guard.
Wondering how this frontier large language model AI system– which is supposed to generate text based on such things as hidden Markov chains and Bayesian nets– could come up with this, I did a search on which National Guardsmen were involved in the infamous shooting at Kent State. My search revealed a series of names bearing no resemblance to mine. The closest connection I could find between my name and the Kent State shooting was that my name is “Ken.”
What was an indulgence has mutated into an exercise of routine paranoid checkup searches. So far so good. No references to Kent State, shootings, or killings, or for that matter to being an underground anarchist or a notorious outlaw.
This did set me off wondering— what if a search did so? What could I do about it? And what if just by writing this column that might be added to a large language model’s database made it more probable that I would be tied to Kent State or shootings or outlaws? I guess I better keep on doing those checkup searches.
Still, what would I do if one of those searches turns out bad? Do I write a cease and desist letter? To whom? Do I sue? Who? And how could I ever prove any damages? Whenever I do a search the AI mode response is never quite the same in content or layout. Almost certainly, I was the only one who saw the Kent State result but if others did, how could I ever find out?
This is, of course, not really about me. It’s about you. It’s about all of us. And to quote a line from Linus in Peanuts, the implications are staggering.
This is not just about what are called AI hallucinations, which has received a fair amount of attention (including recently by the California Supreme Court which referred a law firm to the state bar for discipline for including in its briefing AI hallucinated misquotes and misdescription of a court precedent). Indeed, on the positive side, the fact large language model AI systems can hallucinate might give us hope AI will be too incompetent to kill off all of humanity.
AI Can Still Go Bad Even If It Doesn’t Kill Us
Yet even if AI doesn’t kill us, it could still end up being bad. It could end up where we won’t know where we get our information and how it is filtered. It could end up there is no accountability for falsehoods, distortions, slants, and lies. It could end up our “reality” will become as shapeable as Silly Putty by forces and factors we don’t even know exist, let alone understand.
We cannot count on market competition to sort things out or preserve what we sometimes call the marketplace of ideas. Large language model ownerships (which for our purposes here include frontier models foundational to generative AI) are already well positioned for cartel status. Although information on market shares for generative AI ownerships is sparse, ill-defined, and probably untrustworthy, it is clear that the leading entities include Open AI (GPT series); Anthropic (Claude series); GoogleDeep Mind (Gemini series); Meta (Llama series); and some others. Goldman Sachs estimates that key players in the global frontier generative AI market will have invested one trillion dollars in 2026 alone.
If history is any guide, the market for frontier AI systems and their technological successors will be dominated by a handful of corporations.
Why Media Industries Become Oligopolies
Earlier in my legal career when I was a lawyer with the Federal Trade Commission, I conducted industry-wide antitrust investigations of the motion picture and record industries. What I learned is that since the early 20th century media industries such as the motion picture, radio, television, and recorded music industries followed a pattern of turning into oligopolies.
One reason for this is these industries deal in what are called “quasi-public goods.” Economically a public good is one where consumption does not use up the good. For example, watching a movie does not use up the movie in the way fueling up your gas tank means you use up gasoline. According to economic theory commodities sold in competitive markets–be it nuts or bolts– end up being priced at what is called marginal cost (the incremental cost of producing an additional unit of the product). That theoretical model doesn’t easily apply to quasi-public goods. For example, the cost of producing the “first copy” of a motion picture can be enormous—today in some instances hundreds of millions of dollars. But it does not cost much to make additional copies (by reel or digitally) or for someone to see a movie, and in any case watching it doesn’t use up the movie. It just does not make much sense from the production perspective to talk about marginal cost pricing for watching a movie, listening to a radio program, watching a tv show, or listening to a prerecorded song.
For that reason what does become critical in privately producing a quasi-public good is capturing enough revenue to cover high sunk and fixed costs that were incurred to produce it. More paying customers make it more likely those costs will be covered and more likely that even more money can be spent on the next production, which then will help loop back to luring more customers to a growing network of paying customers. At the same time, future potential competitors will find it challenging to enter the market because they will have to duplicate a competing network of customers from scratch that will be large enough to cover their high costs of entry.
An example is the motion picture industry. In mid-twentieth century five major motion picture studios based in Hollywood that dominated motion picture production in the US also owned their own movie theater chains. If you wanted to get into the movie production business you might have to think about getting into the movie theater business. If you wanted to get into the movie theater business you might have to think about getting into the motion picture production business. Entry was blocked and the motion picture industry became cartel- like.
The Department of Justice brought antitrust lawsuits in the 1940s that forced the studios to divest their movie theater holdings in the hope that this would free up competition at the top and bottom of the motion picture supply chain. This may have helped some, but the major studios clique still retained control over distribution of movies to exhibitors and held on to their oligopoly status and power for decades afterwards.
This pattern of high market concentration in the motion picture and other media industries repeated itself with the arrival of the internet, as the growth and dominance of Big Tech illustrate.
Generative AI Ownership is Already Becoming an Oligopoly
This pattern toward oligopoly is already well in hand when it comes to frontier generative AI systems. These systems have quasi-public good characteristics. These systems have astronomical startup costs which require the compilation and integration of massive amounts of information and development guided by an elite minority of technical experts. Customer use does not use up this information and costs so little as to make marginal cost pricing irrelevant or at least difficult to apply. Although there are parallels with prior patterns of media industry concentration, this round with AI could be more alarming than anything we have seen before.
The entry barriers for the AI market are tremendously high. It requires billions of dollars and extraordinary expertise to develop frontier systems. But even if later entrants have billions and the expertise, they may be forced to try to play a game of catch-up they can’t win. As they move in the market others already on the scene will also move further ahead. It is reported that some generative AI firms have found that their AI systems have advanced by emergent self-improvement to a level that the human designers and engineers don’t even understand how they were achieved. These systems may be going down a learning curve that cannot be easily mimicked. What also may be especially significant is that these systems will continuously harvest valuable information from their users that will be unavailable to new entrants.
To make it even more challenging, information that was available to first mover AI developers may no longer be available to later entrants. These first movers were able to reap and install massive amounts of publicly available information in their models—hence the name large language models. But some of that information may now be behind a paywall, resulting in new entrants facing costs to get into the market that current players didn’t have to incur, putting any new entrant at an inherent economic disadvantage.
And some of the present players have a launching pad new entrants will never have. Google (the parent corporation is now called Alphabet Inc.), for example, with its search engine and other ventures, already has platforms and networks of users that it can leverage to advance its generative AI business. Meta with Facebook, Instagram, and WhatsApp, has an extraordinary presence in social media. Once they lock in AI users, it may be particularly challenging for new entrants to lure them away. The current first movers are therefore well positioned to reap what are called network effects, the more users the better they become in providing services and the easier it is for them to cover their high sunk and fixed costs.
In other words, the current first movers may have a head start that later entrants may never overcome.
When I queried Google’s AI mode powered by Gemini for researching this column, it confirmed these points:
“The generative AI market is rapidly consolidating into an oligopoly dominated by a few massive tech entities. This concentration is driven by unprecedented capital requirements, data access barriers, and deep vertical integration across the entire AI supply chain.” (Emphasis by Google AI mode.)
The Trump Administration Helps Turn the AI Oligopoly into a Cartel
To top it off, current major players are consolidating their position and power thanks to Trump. In an attempt to quell fears about AI’s potential to go rogue and kill off humanity at large, Trump had major industry leaders sign in late September a brief one- page document his administration prepared that was titled “Joint Commitment on Frontier Responsibilities.” In addition to Trump, it is signed by Sundar Pichai of Google, Dario Amodei of Anthropic, Mark Zuckerburg of Meta, Greg Brockman of OpenAI, Elon Musk of xAI, and Jensen Huang of Nvidia. By signing they agreed: “The participating companies will meet regularly to establish standards and best practices to improve the safety of their systems.”
Milton Friedman once said: “Concentrated power is not rendered harmless by the good intentions of those who create it.” This “Joint Commitment” fostered by the Trump Administration gives these key companies a blank check to get together and collude on AI–in other words, to be a cartel. It also creates a new potential barrier to entry by possibly advancing the signatory companies on the preferred list for future federal government contracts. (On the other hand, it should be noted Anthropic is suing the Trump administration for labeling it a supply chain risk because Anthropic refused to allow its AI to be used for national domestic surveliance.) Whether this agreement will survive antitrust muster remains to be seen—but undoubtedly it will be invoked as a federal government sanctioned shield if antitrust litigation is brought against any of the signatory companies.
The topic of regulatory capture, the idea that regulated industries can end up controlling their regulators, has been the subject of extensive economic discourse ever since economist George Stigler’s pioneering 1971 article on the topic. Here, the Trump administration with pomp and ceremony happily handed out police badges to those who are to be policed.
Why AI Oligopoly is Different and More Dangerous
Of course, we have had oligopolies and monopolies before. But this is different. Generative AI promises to be all encompassing. It is projected ultimately to be the means by which we know what we know and do what we do. It will be the industry that will drive all industries. It will be the business by which all business will be transacted. And a handful of companies headed by a handful of CEO’s will be running it all. Small wonder AI companies are prepared to invest a trillion in just a year to get to be first in line. It’s not just about money, it’s about power.
What if these companies and CEOs have political and other agendas to promote? Will they make sure they stand on equal footing with everyone else or will they behave more like Rupert Murdoch when he controlled Fox News? Can we really put our faith in the likes of Mark Zuckerburg and Elon Musk, who signed off on Trump’s AI compact, to give fair play to views other than their own?
Let’s see what Google’s AI mode powered by Gemini had to say in responding to one of my queries:
“Generative AI introduces severe risks of reality distortion and user manipulation through the deployment of algorithmic information filters, data-driven optimization loops, and autonomous deceptive behaviors. Because these systems function as the primary interface between users and vast corpores [sic] of digital information, their underlying architectural constraints and commercial optimization targets can fundamentally alter a user’s perception of truth.” (Emphasis by Google AI mode.)
It went on to cite such examples of “Algorithmic Reality Distortion & ‘Synthetic Pollution” as “Persuasive Hallucinations passed as Fact” and “Feedback Loops and Algorithmic Skew.” Under the headings “Hyper-Personalized Behavior Manipulation” and “Exploitation of Cognitive Biases” it said:
“Digital firms train AI algorithms on massive consumer datasets to map out individual cognitive vulnerabilities. Generative AI can dynamically adjust the framing, emotional tone, and timing of information to exploit a user’s temporary emotional state or psychological biases.”
These are only examples of what it had to say.
Maybe we can hope it was just hallucinating.
Needed: A Bill of Rights for the AI Era
This is not to deny that there might be more ominous dangers AI poses. I cannot assess the risks that some AI experts claim AI poses to human existence. It is true in comparison to the magnitude of these claimed risks incidents like the Kent State episode described at the beginning of this article are trivial. Yet it and other similar reported incidents should be taken as a warning that affirming personal rights and crafting new liability rules may be essential in protecting all of us and in establishing safe havens for fact and hard truths. It is reasonable—not wildly speculative—to worry that AI will be dominated by a handful with the power to manipulate how we perceive the world and how each of us will be perceived in the world. That means a Bill of Rights for the age of AI—with real teeth to make it effective—should be on our agenda for action before it is too late.