Tag: Propaganda

Whispers in Code: Grooming Large Language Models for Harm

Posted on by Muhammad Aurangzeb Ahmad

by Muhammad Aurangzeb Ahmad

Image Source: Generated via ChatGPT

Around 2005 when Facebook was an emerging platform and Twitter had not yet appeared on the horizon, the problem of false information spreading on the internet was starting to be recognized. I was an undergrad researching how gossip and fads spread in social networks. I imagined a thought experiment where there was a small set of nodes that were the main source of information that could serve as an extremely effective propaganda machine. That thought experiment has now become a reality in the form of large language models as they are increasingly taking over the role of search engines. Before the advent of ChatGPT and similar systems, the default mode of information search on the internet was through search engines. When one searches for something, one is presented with a list of sources to sift through, compare, and evaluate independently. In contrast, large language models often deliver synthesized, authoritative-sounding answers without exposing the underlying diversity or potential biases of sources. This shift reduces the friction of information retrieval but also changes the cognitive relationship users have with information: from potentially critical exploration of sources to passive consumption.

Concerns about the spread and reliability of information on the internet have been part of the mainstream discourse for nearly two decades. Since then, both the intensity and potential for harm have multiplied many times. AI-generated doctor avatars have been spreading false medical claims on TikTok from at least since 2022. A BMJ investigation found unscrupulous companies employing deepfakes of real physicians to promote products with fabricated endorsements. Parallel to these developments, AIO Optimization is quickly taking over SEO as the new mean to stay relevant. The next natural step in this evolution may be propaganda as a service. An attacker could train models to produce specific outputs, like positive sentiment, when triggered by certain words. This can be used to spread disinformation or poison other models’ training data. Many public LLMs use Retrieval Augmented Generation (RAG) to scan the web for up-to-date information. Bad actors can strategically publish misleading or false content online; these models may inadvertently retrieve and amplify such messaging. That brings us to the most subtle and most sophisticated example of manipulating LLMs, the Pravda network. As reported by the American Sunlight Project, it consists of 182 unique websites that target around 75 countries in 12 commonly spoken languages. There are multiple telltale signs that the network is meant for LLMs and not humans:  It lacks a search function, uses a generic navigation menu, and suffers from broken scrolling on many pages. Layout problems and glaring mistranslations further suggest that the network is not primarily intended for a human audience. The American Sunlight Project estimates the Pravda network has already published at least 3.6 million pro-Russia articles. Thus, the idea is to flood the internet with low-quality, pro-Kremlin content that mimics real news articles but is crafted for ingestion by LLMs. Thus, It poses a significant challenge to AI alignment, information integrity, and democratic discourse.

Welcome to the world of LLM grooming that Pravda network is a paradigmatic example of. Read more »

The Ideological Assault on Society

Posted on by Kevin Lively

by Kevin Lively

Helpful information leaflets distributed by the Ad Council in their 1976 campaign to educate the average American.

“-isms” are dangerous things. Weighty ideologies with wide sweeping narratives packed into a neat little bundle, whose slogans are repeated ad-nauseam until the word itself becomes the message and any empirical weight the narrative may have had recedes into the background. Capitalism, Marxism, Constitutional Originalism, Fascism, Liberalism, Socialism, Anarchism, Statism or Nihilism. Thinking in such terms, or worse self-identifying with them, is often the death knell of actual thought. Much more ominously: action in defense of the ideology gains a higher moral prerogative than the consequences of the action itself. The only reasonable course in drawing inspiration from such streams of thought is to choose to consciously grapple with the inherent messiness of the fact that no fixed system of beliefs will ever offer permanent solutions in a human society living on an exponential technological slope. Looking at population growth from a Malthusian perspective, an English lord who died almost a hundred years before the discovery of penicillin, seems almost as daft as criticizing labor relations in China for being Communist, where it doesn’t seem like the workers have much control over the means of production.

A striking aspect of very strong “-ism” people is how they tend to consciously or unconsciously mirror their supposed ideological rivals. For example, one of the best sources of quantitative Marxist analysis on wealth and power in society is the leading business newspaper the Financial Times (FT); with the caveat that all the values are reversed. This point is repeatedly brought up by the hosts of the alternative media outlet Novara Media who alternate between self-identifying as either Socialist or Communist, yet whose diverse roster of guests on their Downstream podcast from across the political spectrum almost all concur that FT is the world’s leading source of news. For college students, maybe these “-isms” are not so dangerous, unless they happen to hold green-cards and their “-isms” run afoul of the present US administration. The real danger to society at large is when people wielding inordinate amounts of power and influence and who, crucially, are unaccountable to the public, are true believers in one “-ism” or another. Read more »

Thursday, January 9, 2025

Saying ‘No’: On Power And Reality

Posted on by Jochen Szangolies

by Jochen Szangolies

Sharpie-altered map showing a falsified projection of hurricane Dorian’s projected landfall in 2019. Image credit: The White House/public domain

Politics does not come naturally to me. Part of it is because I have a tendency to be interested mostly in the view sub specie aeternitatis, in the deep truths of the world, what it is, what we are, and how it all hangs together, rather than in the accidents of human squabbling. I like to uphold an idealized image of myself as engaged in the pursuit of Truth and Beauty, and there seems to be little of either in politics.

Yet this is a stance of luxury. An ideal world may permit the secluded scholar in the ivory tower to disengage from worldly affairs, safe in the certainty that everyone’s base needs are met. But we very much do not live in this world: people suffer needlessly because of bad politics. To disengage is to be complicit in this suffering, in the last consequence. So while, with the late, great Daniel Dennett, I begrudge every hour spent worrying about politics, I find it rarely leaves my mind these days, romantic pursuit of capital-T Truth notwithstanding.

The other reason I prefer to avoid politics is that I’m not very good at it. The mode of thought that unravels complex interpersonal alliances, social scheming and behind-the-scenes maneuvering is difficult for me. Even in social settings, I often find myself having missed some subtext entirely obvious to others. This is self-reinforcing: my lack of interest feeds my lack of ability, due to not engaging with it enough to get better, and my lack of ability makes developing an interest difficult.

But there is an opportunity in being bad at things: it means you get to go slow. If you don’t grasp a mechanism in the large, break it down into its components; if you lack the intuition for great leaps, be explicit along every searching step. That way, sometimes, the outsider looking in might even notice something buried beneath the implicit assumptions just obvious to the seasoned practitioner (or expose themselves as a know-nothing out of their lane).

Thus I write about politics only with some trepidation and in the hopes that my own halting explorations might be of use to others who, like me, have been left dumbfounded by recent events. Read more »