Writing nonfiction in the age of LLMs: The chirping of stochastic parrots 🦜
Insights from Your Perfect Dream Girl: Influencers, AI, and the Future of Desire
I recently published Your Perfect Dream Girl: Influencers, AI, and the Future of Desire (Penguin Random House 2026). The book is my take on how we will consume women in the age of AI. I centred it on the story of my life growing up as an influencer and aimed it at men, for a bit of a subversive kick. Which was quite fun to do!
Since I wrote it in an era where we’re negotiating our relationship with LLMs, and I study human relationships with emerging tech as a human-computer interaction researcher, and I’m about to go back to school to focus on AI-related user behaviour, I have so much to say. I hope you find my insights interesting!
I’ve long engaged in public advocacy around tech [and will always be, in the minds of some, the ‘IFF girl who taught them about encryption.’] I have found it to be tangibly helpful, so the reasons I’m posting this are [a] incentivising, through literacy, lesser/limited use of LLMs by Indian AI users, [b] showing readers interested in my book one of the rabbit holes behind it, and [c] opening a conversation about the ‘way ahead’ as the quality of generated work rapidly increases.
Note that my stance, embedded in the DNA of my book, this essay, and all of my work is: I do not moralize the end-user. I look at sociotechnical incentives built by tech businesses.
This article is in 7 parts. It’s an 9,000+ word long accompaniment to my 70,000+ word book, so you might need a full day and a coffee…
Introduction
Creativity and political imagination
Who’s writing? The alter ego
The cult(ure) of language
Shaming and the gender thing
The problem of transparency as a cultural guardrail
Experiments
Introduction
I’ve been exploring technologically-mediated art for over a decade. It started when I crowdsourced people’s anonymous journal entries over Instagram to broadcast to a larger audience, when the app was but a ‘baby.’ I did the Step Out of the Frame project, a 100-part photo-series of my friends exploring human relationships with posing under technocapitalism. AI, then, meant to turn on portrait mode on a phone, and apply FX via Prequel. In audio production and engineering (where I’m unsurprisingly drawn to hyperpop) I’ve been integrating ‘traditional’ DSP-based plugins like Soundtoys with stuff like Logic’s Stem Splitter. I’ve learnt much from developers working on lifelike visuals for Vision Pro. And so on, since the early 2000s.
In terms of AI, I’m not a heavy user because I haven’t found a reason yet outside of this experiment, but in the spirit of literacy I vibecoded a tool which tracks the movements of planets across space as they transit constellations. It was less a technological revelation and more a dashboard generated basis manually fed data. To its credit, I do use its locally stored HTML version everyday now. (What, haven’t you ever had a weird fixation? Planets? Airplanes? Maps?). Astrology nerds and business nerds alike have asked for it, but I don’t currently have the motivation to pursue it. Get a tarot reading or something, jeez. Kidding. Astrology and LLMs is a whole other fascinating conversation, which we will have in a different article.
I’ve been running communities where people discuss tech x human relationships for some years now. In one community of 350-ish people, where we bring together Indians from various stances on emerging tech, from big tech engineers to consciousness researchers to environmental journalists to new moms to doctors, one of the things we’ve been chatting about is LLM-based text generation. I found a surprising variety of mixed and nuanced feelings, so I thought I’d share my experiments and perspective.
I’ve been writing about tech for a while, but have never explored tech while writing, so in my tinkering for this book, I found that it is a fascinating mix of [a] seemingly useful and [b] tangibly useless. Clearly, while equipped for text generation, LLMs are stuck at the stochastic-parrot stage of next token prediction in a way that makes writing miserable (in a uniquely different way than the usual misery of the writing process).
(What is a stochastic parrot? Read this and this.)
I want to share my findings, in a way that increases AI literacy, which people can benefit from. Why? India is the #1 user of ‘AI’ in the world. We’re OpenAI and Claude’s 2nd largest user base. 92% Indians (apparently) use it at work. Social media is flooded with 100% generated content and we’re zooming towards the dead internet. Indian startups, tech businesses, media businesses, and more are shifting to AI. We require several interventions that help delineate limitations that can be adopted from the user-side [in a way that promotes tangible behaviour change].
There are a lot of frontiers from which to tackle AI—making it a public resource, reducing bias, dead internet and cleaning up slop, environment, democratisation, gendered harm, literacy, big AGI questions, and so on... textbot literacy in the Indian context is the one most interesting to me right now (obviously aside from deepfakes/gender, which is the topic of my book,) so here, enjoy my rabbit-hole of thoughts and experiments.
Creativity and political imagination
Someone shared this paper with me the other day which found that gen-AI enhances individual creativity [but reduces the collective diversity of novel content.] In my humble sample size of 1 [me] I haven’t seen this. In fact, the first thing I noticed was that the models have an allergy to creativity, especially if you have a non-linear, unique, marginalised, or in any way subversive perspective. You might already sense this, but I want to talk about why.
The concept, structure, and ideas in Your Perfect Dream Girl are shamelessly creatively ambitious (described as ‘on steroids,’ ‘neurodivergent,’ ‘extremely upsetting,’ and many other phrases I’m still quietly absorbing.) (BTW, both of those words, quietly and absorbing, and the married phrase, quietly absorbing, belong to humans.)
In terms of structure, Your Perfect Dream Girl is made up of 25 letters to a faceless man I once met in a dream; structured like a play within a play within a play (because both gender and being online entail performance.) Each letter is a sub-argument, coming together to make up a single 70,000 word argument. The play is made up of 4 Acts. The names of the Acts, when put together, reveal an inconspicuous-yes-conspicuous poem that summarizes the book (because that is the nature of being ‘read’ online.)
ACT I. IF FOR YEARS MEN WATCHED ME
ACT II. WELL THEN NOW I WATCH THEM BACK
ACT III. AS SWATTERS TURN TO FLIES SWEETENED BY HONEY-SLICK FLYTRAPS
ACT IV. AS MURMURS GREET THE SETTING SUN OF EVER-CHERISHED PACTS
Call me melodramatic!
The book also includes my insights from manually open-coding 3000 words of textual data from love letters I received growing up online, simulates a 3-problem hackathon, compares my contemporary experiences with a lot of older poetry, cites cool books I love, makes lots of references to my own poems, journals, and old Instagram posts, and explores interviews with 30+ people along with my own experience growing up online.
Your Perfect Dream Girl makes comparisons that teeter on bizarre: Instagram virality with the trafficking of girls and the fate of Brooke Shields, the suckling of a teat on an AI-generated porno with the nourishment of sever rooms by bodies of women, the glitter fallout from a Fenty highlighter with the asbestos snow on the production set of the Wizard of Oz, sex robots with Dolly Parton’s Jolene, my experience on X with Augusta Webster’s A Castaway, and on and on and on…
Simply put, if I listened to bots, this novelty would cease to exist.
I fed them my draft concepts in the early stages to see what they would say, and found that they pushed back repeatedly, suggesting ‘refinement’ based on two lenses: making the work more marketable (which I’m not interested in,) or more literary (which is not my domain.) Each ‘refinement’ attempted to bring the work closer to a statistical average of marketability or literature, convincing me to make something akin to that which has already been made.
My first finding was that there was no end to this homogenizing process. Bots would propose a change to every new iteration (in their classic authoritative tone,) not because there was any need for change, but because they are designed to keep users engaged, active, and prompting for as long as possible [i.e. if the bot said ‘okay, you’re all good, you can stop now’, you would leave, which it doesn’t want.]
This ‘hookiness’ has users in a chokehold. One of my friends recently lamented that her sister, who has a dedicated ChatGPT thread about her dog, spends hours responding to these follow-up questions even though they don’t meaningfully progress the conversation. I understand the disappointment, but I find it increasingly futile to ‘blame’ the end-user. Anyone who thinks free will does not falter in front of billions of dollars worth of incentives probably does not know how much of their own behaviour is externally shaped. For the woman worried about her dog, there is some psychological facet that is being triggered again, and again, and again. And it is being triggered by design.
Bots are not wrong about the fact that their recommendations ‘work’ as marketable and literary. That’s why Granta published AI-generated The Serpent in the Grove as the Caribbean regional winner of the Commonwealth Short Story Prize, and Hachette published seemingly entirely AI-generated Shy Girl [before revoking it], and AI-generated content still, to the amazement of many, routinely goes viral on LinkedIn. That is simply indicative of an over-valuation of formulaic work in capitalist markets. We ascribe value to something that looks like something that previously worked. As I argue in my book, this human tendency towards following social signals is exacerbated by social media design.
What bots did not have is the intuition and gutfeel to [a] understand when to stop refining towards an optimally marketable or literary outcome, or [b] assess novelty that has no precedent. So when I presented them with a unique idea which didn’t fit the mold of what had previously worked, it became too complicated or alienating [labelled a such, of course, in a much gentler, convincing, and helpful tone].
This, despite strong self-trust cultivated over years, made me stop for a moment and doubt my ideas. That quickly became unacceptable. I knew I couldn’t afford doubt: my life experience is unique, so the work that comes out of that experience has to be unique. My self-faith is what made me look around at the state of tech and gender and say: actually, even though no one may agree with me, there’s a lot here that is going terribly wrong and I need to tell you all about it. If I doubted this conviction, how could my work exist?
At the conceptualisation stage, despite my best efforts at intelligent, comprehensive prompting, bots failed. No idea suggested by them made sense to me in the universe of my book. It ultimately took what I expected it to take: 46 documents of manual structuring, re-structuring, and writing to arrive at a basic flow for my complicatedly-structured book [examples below].
I am not going to say, look at me, I worked so hard, you should too. All I will say is that if I didn’t craft each iteration myself, I wouldn’t have arrived at a series of ‘aha!’ moments that turned into the final book; a project that I was truly happy with. Because good ideas are functions of tinkering, and if you take the tinkering away, the ideas get all sad and lonely (yet marketable and literary).
There’s also something to be said about the insighting process. When I began open-coding the letters, it took me a few attempts [13? 14?] to really come down to a pattern that makes sense. But the final pattern arose only because I thought through the previous attempts.
There was also the issue of illusions. Not hallucinations, illusions.
On one hand, bots gave no fresh ideas outside of mainstream talking points. On the other hand, when I offered new training data from my notes that countered these talking points, they couldn’t extend them. There was only the illusion of extension. I call it such because,
[a] The ‘extension’ of an idea is a rephrased version of the same idea, or
[b] The idea is extended in the direction of populist thought [given its overrepresentation in training data], defeating the purpose of work that is attempting to be novel or subversive [and, in that way, ‘creative.‘]
But it takes a critical eye to spot this, and if you are writing in a hurry for a functional purpose [say for social media], this is easy to miss. My sense is that most people would be okay with this: social media algorithms are happy to reward populist thought [and repeated, black-and-white talking points,] so there is little incentive to ‘go beyond.‘
At some point, I started perceiving ChatGPT as an intern afflicted by a unique issue: too eager to please, yet too lazy to do the job. I had little desire to sit and train this intern, because [a] It did not have the consciousness to feel bad about being caught slipping, so it never attempted to self-discipline and change its behaviour, and [b] unlike a real workplace, I didn’t have any actual obligation to train the intern, so I could just fire it and get on with the work myself.
A lot of people ‘riff’ with bots, using them as soundboards and brainstorming partners. But what is the job of a riffing partner? Is it to mirror you, or to expand and challenge your thinking? I am sure bots will be able to do the latter at some point. But currently, in my opinion, they’re stuck at the former stage.
Now, does a writer need a mirror? Clearly, given usage statistics, many across the world apparently do. It’s worth finding out why [Do they feel unsupported? Are they sensitive to critique? Is it just nice to have something validating you as you progress with your work?] Knowing these motivations is important in discovering incentives for behaviour change.
My next point is that the bots had a restricted idea of what a ‘book’ should be. Mine sprawls into recurring symbolism that expands and mutates, and real-life performance art. But bots attempted to bring it closer to the traditional definition of a ‘book,’ curbing the imagination of what a book could possibly be in a way it has not been before. This was a red flag I am quite keen to bring up, because imagination is often the beginning-point of political intervention.
A simple example of this: The first half of my book is the emotional testimony of an influencer recounting her girlhood. The second half of my book is cultural commentary on tech from the perspective of a researcher. The point of doing this is the whiplash of subversion, to get readers to reconcile the two positions and build a bridge between them. My human editor got this idea, and agreed that the halves should be in two different tones. Bots insisted on the fact that the two halves must be similar in tone. They did not have the meta-level ‘thinking’ capacity to reason why the unconventional choice might serve the project better.
How can I take into consideration of an opinion of a thing that cannot reason?
Why would you take its ‘feedback’?
Finally, I want to point out the use of recurring metaphors. My book has lots of layers: the stage, the play, the theatre, the audience, the stagemakers, the hawk and songbird, the voice in the woods. The reader is expected to understand that these zoom in and out of alluding to patriarchy, platforms, audiences, and users under patriarchal technocapitalism. The symbols gain power over the span of a book with each recurrence. The first time I say ‘the hawk must eat, and the songbird must be eaten,’ it’s a passing, perhaps somewhat melodramatic phrase. The tenth time, the reader begins to realize it is the thesis of the book.
I would love someone who works more closely with text-based LLMs to shed light on this, but I find that bots have little ability to sustain and evolve a metaphor over long-form writing. They ‘forget’ [?] or have little sense of how it should evolve. The direction a metaphor evolves in is a decision based on opinion, politics, and taste, which bots don’t have. So they just keep on circling the same topic, never expanding, never evolving.
This has made me realise that within the realm of nonfiction books, it is likely easier to use AI if the author is writing a string of loosely related essays on a topic [which is quite a popular method.] If the whole book is one argument that is being progressed, and progressed, and progressed [perhaps the thing the more neurotic of us opt for], I would be doubtful of LLMs’ helpfulness.
Evolving and integrating is hard. Furthering an argument is hard. Sustaining a metaphor is hard. A lot of this comes down to the fact that when not dealing with facts, LLMs are good at the ambient. Precision and direction are the problems. If you want to test this, try to write something on your own using something akin to a two-and-a-half draft method, and then prompt the bot to write the same thing for you, and make a list of the differences.
Now, I was able to identify these issues and revert to my imagination because [a] I had my research/experimentation hat on and could swap it with a ‘creating/feeling’ hat, and [b] I am a strongly opinionated person who has worked on cultivating my unique taste. This confidence is a matter of privilege. Not everyone has it.
In fact, this process taught me a lot about why we have so much AI slop in India. Creativity, as I argue in this viral video, comes from self-faith. But in an India that has been trained to be conformist and hypercompetitive, and which was once-colonized, many are conditioned to not trust their own instincts, and are particularly insecure about a grasp over the English language. So, when an American bot tells us to change something, we are likely to listen—even if it makes our work objectively worse. When it tells us that a more boring iteration of a novel idea is more likely to succeed, we are again likely to listen—because in many spaces, success is survival.
This problem can be further understood by reading this paper by Google Research, titled ‘Because AI is 100% right and safe”: User Attitudes and Sources of AI Authority in India,’ which finds that a majority of Indians are likely to see AI as a ‘benevolent authority,’ essentially trusting it to be better-than-us. There is a culture of deference to the judgements of LLMs, and this culture is created by larger incentive structures, not the apparent evilness of everyday users.
[I refrain from conflating the Indian experience with the Western experience, because we have different history and baggage that meaningfully changes our relationship with emerging tech, and the idea of ‘language as intelligence‘. A lot of the online discourse on AI is being dominated by majority anglophone countries, and missing this nuance.]
So, if you have an inkling that you have something new to say, know that this newness could be stripped and made to feel unmarketable and unliterary in its original form. I would say that the best outcome is refraining from letting AI tell you what is right or wrong, because we, as humans, are psychologically sensitive to being told so, and because sometimes, the thing that the world calls wrong might be right to you, and what the world insists is wrong might just be your truth.
Being able to achieve this insight is one of the reasons I encourage people to tinker with AI. I am pro-literacy, especially for women, women in STEM, and people like me who research in and around tech and engage in public advocacy. If you don’t know how it works, how can you help others understand nuances of how it changes the nature of political work and injects self-doubt where none can be afforded?
I am somewhat (?) surprised (ok, I’m not surprised) that major creative businesses like A24, which has partnered with Google DeepMind, are using LLMs in the research and storyboarding process. Either they are diminishing the quality of their work or have found some genuine way to make AI creative at the conceptual stage—the workings of which I would be curious to know.
Others like Steven Spielberg are down to use it but not a final decision maker on creative ideas [“Use AI as a tool, but do not use AI as the final word on anything creative. That’s where I draw the line.”] For them, the issue is not at all the ethics of using AI, but protecting the sanctity of their own art—which, at the level of major Hollywood projects and well-funded artists, are two conversations I don’t like conflating.
Who’s writing? The alter ego
I was 13 when I announced: I should have been called Zara.
I used to imagine Zara as a technologist-researcher-author walking around with a tattered satchel under scaffolded sidewalk sheds in New York [my dreams have been quite linear], who always spoke in a cryptic, dark tone.
The point of her being cryptic was that she did not care about being understood. The point of her being dark was that she, busy with other preoccupations, did not care about being desired either. As a child, my greatest fear was being misunderstood and undesired, and I made her up to deal with this fear. So Zara was not quite an imaginary friend, but an imaginary version of myself I wanted to self-actualize into.
My email inbox is speckled with threads of quotes I’ve cooked up in the middle of the night, circling the question: How would Zara say the thing I want to say better? That’s why my chapters have weird names, like Severed Heads and Torsos, and weird metaphors, like running from a man with a knife through dense ravines and glossy malls in a fever dream, and other things alive in the dark.
My Pinterest world-building for her is how I have always envisioned her: industrial, chrome, deep red, surreal, cyber-femme, sensually futuristic, ugly, subversive…
When I decided to quit my job at Apple to write my debut book, I bought myself Zara-coded glasses in New York, and forced myself to transform into her: a bordering-on-crazy woman who would write without caring what anybody would think, and sacrifice her reputation for a little bit of performance art.
A part of me did it as her and not me because I needed distance from my own work. I am a private person. The cognitive dissonance of writing about the truth of my life and convictions felt psychologically unsurvivable, so I needed an alter-ego to take over for me.
That was Zara. She became, and always has been, my political imagination of what a woman could be in a better world. Misunderstood, undesirable, and still thriving. This imagination culminated, in some way, in my self-presentation and the talking points of the book itself.
As I grew up I was also inspired by Antoine, the protagonist of Sartre’s Nausea. His disgust and dread were part of what him a good protagonist, and I wanted to know if a woman was ‘allowed’ to be the same way without aestheticising the condition. Of course there is Plath’s The Bell Jar which I mention in my book, and many other stories about sick and sad women, but something about the sheer pathetic nature of Antoine felt most relatable to me…
Anyway. I bring up alter egos because I’ve seen that,
[a] everyday people are using AI to create alter egos for themselves,
[b] fiction writers are using AI to create characters,
[c] businesses are using AI to give chatbots ‘personalities’,
[d] marketing teams are using AI to come up with ‘archetypes‘ for demographics,
[e] people are giving bots, themselves, identities [from ‘chat’ for ChatGPT as a somewhat infantilising petname to ‘clanker’ as a slur]
The question is, what is the limit of imagination of AI when it comes to coming up with a whole identity? Can it imagine being an undesirable, happy woman? Can it imagine a subconscious that is not represented in training data? Can it integrate an experience that is unspoken outside of whispers to fellow people from the same flock? Does it have, as it says, the ‘interiority’ to create an internal universe that resists the ideals of the megacorporations that birthed it, and their own goals and incentives?
Can AI turn its back on its own mommy?
As I grow older, I’m uninterested in common archetypal personalities. I find that the thing inside you, the one that is shameful and weird and underrepresented, is the one that the world needs right now.
A chatbot doesn’t know it exists.
The cult(ure) of language
I’ve been seeing more and more people talking about how they’re moving away from certain kinds of words, grammar, and sentence structure because they’re ’afraid’ their work will be perceived as AI-generated.
I find this stance to be anti-intellectual. In my opinion, there is something sinister about our collective, yielding acceptance of the fact that language—our means of expressing the important things we have to say—is being stripped from us and handed to corporations. We are letting it be taken out of fear of social ostracization.
The current trend is dissuading negative triads, parallelisms, em-dashes, semicolons, colons, three-part lists, and 100+ words. We now reject quietly, meticulous, valuable, underscore, pivotal, reading, land, metabolise, delve, collapse, quietly, genuinely, shift, whispers, lingers, autopsy, align with, boasts, bolstered, crucial, emphasising, enduring, enhance, indelible, fostering, garner, highlight, only, ambient, interplay, showcase, surpass, cost, intricate, intricacies, key, register, landscape, interiority, robust, showcase, tapestry, sentences that start with ‘what’, testament, underscore, vibrant descriptions of landscapes, the word vibrant itself.
In fact, I’ve heard people saying, ‘I can tell it’s AI because some words and quirks are tells.’ Then we demonise those ways of writing. I read these lists of ‘forbidden’ language when I started writing my book, and knew that my stance is to take a stand for using them. When my editors manually replaced hundreds of commas in my book with em-dashes, I was happy that they did. I even told them to keep my three-part lists. The lists of two, fours, fives, and sixes too.
Tomorrow, when models update with more forms of writing, it will be 1000 ostracised words, and then 10,000. Then what?
I recall a conversation with an ML researcher who was showing me his new paper at some fancy conference I don’t recall. This was pre-GPT boom. His abstract consisted of six lists of threes. It looked ridiculous. I asked him we he did it. He said: that’s just how we’ve been taught to do it.
AI uses certain words and structures so much because humans use these words so much. For some of us in academic-adjacent spaces, this issue is more complex because these phrases are derived from open-sourced academic work that uses the same language we’ve been trained on. Because of this, it’s not just any language, but intelligent, explanatory language being taken.
Language is power, and when you take language away, you take away power. For example, my book is, by definition, a ‘testimony.’ It derives its power from being so. When someone tells me I shouldn’t use that word because it sounds like AI, my instinct is: but my work is powerful because of this positionality. So I cannot not use it.
Precision is also power. I experienced the same with ‘autopsy.’ I sat with my chapter titled ‘An Autopsy of Longing’ and wondered: should I change it? But the love letters I cite are things that were once living and now dead and laying on a table for me to dissect, tying into my ongoing theme of severed heads and torsoes and other darkened things. Their aliveness to me at one point matters. The deadness now matters. Autopsy is the precise word. I cannot have a corporation take away my precision.
In fact, I have a chapter in my book where I talk about calling social media platforms ‘bad technology’ and discuss why precise language is a form of power, which seems in retrospect to be something I’ve been thinking of for a while.
There’s another aspect to the language issue I haven’t seen being brought up. A lot of people are chronically online. I also had to spend a lot of time online researching for my book (bless my bleeding eyes that have consumed hours of deepfakes.) A lot of textual content is now entirely generated, and people are being exposed to it hundreds, if not thousands, times a day, in Reels, carousels, LinkedIn posts, substacks, Medium essays, advertisement copy, and more.
I’ve read a lot of books, and re-read ~50 to write mine, but I am still influenced by accidental exposure to generated content. And I believe this will soon become a universal experience. My sense is that the less organic material people have read and written in the past [a ‘privilege’ for those of us having grown up in a non-AI age], the more likely they will be influenced and adopt the same language. I hope there is a study about this at some point.
In an interview I gave the other day, I said: I still don’t feel like a woman as much as I do a tapestry of the things I have created. That word, tapestry, came to me because I had seen a meme about Wikipedia’s ‘AI-sounding words to watch out for’ which included it. But it was more fitting than any other word for what I was trying to say, so what then? These words inadvertently seep into human writing and speaking style. There needs to be a conversation, at some point, about how putting on ‘I can tell its AI’ goggles ceases to be helpful when it’s not just work but brains being, for the lack of a better word, enshittified.
Some of the emerging counters to this conversation have been quite interesting and funny. My favourite is the trend: ‘I am not a bot. I’m just autistic.’ It discusses the penalization of people for their flat, robotic affect and structured writing. As a woman who has been called a bitch and an LLM on more than one occasion, I empathize with the autistic community. Lol.
Online shaming and the gender thing
The third thing I want to talk about is the moralizing meta-conversation about the use of LLMs in text generation online. One of the core premises of my book is: can I tackle the system without moralizing the people within it? 200,000+ draft words of thinking later, I have gained a new way to see this emerging phenomenon.
I am against paternity testing people’s work against AI detectors [which are AI themselves, use fed-work as training data, and are now being banned in institutions because they have failed to be accurate], and putting creatives under pressure to prove something that is unprovable, instilling a fear of self-expression.
Today, it’s text, and authors are ripping themselves apart figuring out how to sound human. Soon, pictures, videos, and audio will catch up. Stuff like Google’s Synth-ID is a good, but for many reasons a limited intervention. Then what?
I’m not just talking about accusations. I’m talking about public responses to finding out that people are using bots, and the disproportionate responses towards different groups of people.
Research shows that ‘moral grandstanding’—i.e. taking a strong virtue-based stance in social and political discourse, associated with seeking social status when done on social media in front of an audience—increases affective polarization. You can read this meta-analysis, ‘Look at Me Being Good – Connecting Moral Grandstanding with Affective Polarization and Civic Engagement.’ This polarisation has long been studied in the context of social media [including by my previous research group, shoutout guys], and is detrimental to civic engagement. Clearly, despite shaming culture, Indians are not dissuaded from using LLMs.
One of many reasons for this is the severe echo-chambering of such conversations. Since my feed is oriented towards techy stuff, I’ve been seeing this conversation about AI blossom in an overwhelming way. But the other day, a friend came over. She has her own opinions on LLMs as a journalist. She showed me her Explore Page, and it was made up entirely of food videos. Even after all this time, I had to remind myself that there are many internets, and the people occupying them can be very removed from each other.
I also find this paper, ‘AI Could Have Written This: Birth of a Classist Slur in Knowledge Work’ interesting. It argues that AI-use accusations, used to undermine people’s reputations, have a reason beyond moral-ethical factors: class anxiety, and the need to limit the mobility of ‘non-knowledge workers’ into the privileged class of knowledge work.
I’ve been studying the practice of ‘shaming’ online, and also explore it in my book. There is a significant gendered element to shaming on patriarchal technocapitalist platforms that incentivise reactionary spectacles centering women, specially when it comes to matters of ‘purity’.
As I argue in my book, women must by design be ‘useful’ to the patriarchal technocapitalist system. One of the ways we derive this usefulness is by sacrificing and scapegoating women for people to demarcate their own relative moral positions. Women hence receive the brunt of virtue signalling while men get to increasingly be open and reckless about their use.
I think, as I write this, about 3 pieces of content I have seen about Nobel Laureate Olga Tokarczuk (who I am quite interested in because as I share in my book, I like to study the public response to ‘hyperconsumed and hated women’): I saw one video calling to guillotine her, one of someone throwing snarling at the camera and throwing her book out of a window into dirt, and one comment alluding to ‘stepping on her neck.’
Yet masculinist futurists using AI in a supercharged capacity —heads of corporations, YC-funded entrepreneurs, deepfake prompters—have impunity against this flavour of vitriol, which is unsurprising once you understand the gendered history of shame. Take for example the fact that Refik Anadol, media artist, trained a model on 500,000,000 images and hundreds of thousands of hours of audio for his Dataland exhibition which is ironically a tribute to nature. This was the response in the media.
Or you could think of the fact that Alibaba performed 29,000,000 exchanges on Claude, or that algorithmically incentivized witch hunts against women cost immense compute, or that Spotify and TikTok and Instagram and YouTube and Netflix running a gobsmacking number of inferences. But it doesn’t matter. There is a deeply embedded, subconscious need for patriarchal gratification online. Once you understand the gendered history of shame, you understand this.
We have taken Olga, doing political work, and silenced it despite her transparency, in order to make an example out of her. We have, as one social media user celebrated, ‘put our foot on her neck.’ But the work of thousands, if not millions of powerful men—done for ‘exploration,‘ ‘art,‘ ‘the love of the game,‘ ‘fun,’ and not least, profit—exists unscathed. The people doing this are often the same ones who willingly benefit patriarchal technocapitalist platforms, ensuring the status quo remains as is.
Take a look at another example. I came across this post by Wired, which, predictably, had hate comments for the moms under it.
As I write in my book, we’re addicted to hating women, specially hypervisible women online. Wired could have spinned this as: ‘Struggling mothers turn to AI to parent kids as men remain unavailable under patriarchy.’ Having spoken to moms-who-use-AI, that seems far more apt. But it’s a very particular kind of gendered virtue signalling that invites platform-driven hate against women by [a] invoking the cultural hatred for ‘influencers‘ and their supposed privilege even though new mothers are often disenfranchised in a more complex way, and at the same time [b] using provocative, polarising language (‘better than men‘.)
It’s also, in some cases, just a problem of information asymmetry. Today, someone could innocently Google something—a user behaviour solidified over years of habit building—and the now AI-mediated answer they receive could be wrong. Take, for example, the hilarious situation where SEO expert Pedro Dias made a LinkedIn post announcing that he is the world’s most renowned AI visibility expert [an uncommon query,] got it to appear in Google’s AI response to ‘world’s most renowned AI visibility expert,‘ and reinforced it via a comment on his own LinkedIn post. There is a lot about the current AI situation that demands a little less vitriol towards the lowest hanging fruits of the power hierarchy, and a little more towards the unpluckable ones on top.
I am pro-progressive women, men, and marginalized communities becoming technologically literate so that we can become a part of the processes that set the foundational technological, legal, cultural, and business blueprints for emerging technology in a way that represents our concerns. Because it is already being built with or without us, due to larger interests at play. We failed to do this with social media, and the outcomes are clear for all to see.
So, while some people work on mitigating AI altogether, I am interested in the other thing that has to go alongside the former, which is self-limitation due to effective incentives. The latter approach reminds me of Christabel Mintah-Galloway’s simple sentence: relational skills as collective liberation. Your Perfect Dream Girl follows this in other arenas like gender discourse as well, inspired by my mom’s work on ‘relational dynamics’ between cisheterosexual men and women.
One such incentive I like is that some creative institutions are handing out ‘100% human-made’ stickers [certainly not something I will be able to use given Your Perfect Dream Girl was an experimental playground with swingsets made of disappointing LLMs :D].
Another effective incentive is educating people about what LLMs are doing to their work, so that they are empowered to make alternative decisions [which is my approach].
I’ve also been seeing editing apps like Prequel [which began as an AR/VR business] demand credits/tokens as a financial barrier to AI-use within the app, which is interesting.
One of big things I mention in my book is the imagination of what social media could look like if the interests of young women—a user demographic the app cannot exist without—were integrated in a safe way into its architecture instead of disastrously exploited. We failed to do this, and we can now see the devastating consequence for gender safety, on social media and bleeding into AI. I’m talking about Meta’s controversies around eating disorder-related content, showing young girls makeup ads after they deleted selfies, women’s lack of control over viral images of themselves being used in misogynistic harassment campaigns, the algorithmic incentives to appeal to the male gaze, the hosting of AI child-woman hybrid porn and sexually suggestive imagery, and more.
When I keep getting videos on my social media feed which say: ‘Resist the rhetoric that AI is inevitable,’ I think about this. While I support the people working on this form of resistance, I find it important to point out that while it occurs, we are nearly a decade into the AI age, so there is a need for intervention for groups that need their interests represented. Otherwise whose domain is AI? Who gets a say in power? Billionaires? Megacorporations? Deepfake prompters? VC-funded entrepreneurs making convenience apps? Because it is already their domain, and they already have a say, and I want to tackle that monopoly of perspective.
One of my problems with AI doomerism is that it assumes the problem began with AI, absolving everyone of their role in a longer history of technologically mediated harm. Social media, for example, has been causing harm—a shocking amount of it—for over a decade. I have worked at a major platform, moderating child sexual abuse material, gore, death, political violence, and self harm that it incentivized by design. I have worked at a major record label, watching corporate executives pump mind-numbing amounts of money into colonizing young boys’ attention with videos of half-clothed women. I have published research quantifying how social media leads to violence. I have interviewed over one hundred women who have cried while telling me about how social media ruined their lives. I have spoken to entertainment industry founders who act like pimps trading women with consumer brands and calling them ‘influencers’. I, myself, am the woman whose girlhood was colonized by platforms.
Yet despite social media’s harms, data centres, environmental damage, exploitation, incitement of violence, abuse, deaths, and catastrophe, nobody is willing to stop using it. Because it is addictive, entertaining, validating, and, as I argue in my book, patriarchally gratifying. In fact, platform PR is so good that users are not only unaware that social media is AI, but also that their own algorithmically incentivized online behaviour is part of the problem. The other day, I spoke to a woman who gently informed me that a single text-based prompt to a chatbot takes up far more water than using social media all day [talk to five people tuned in to the online discourse and you will hear this too.]
I don’t know. After having a book bridging the harms of social media with the harms of AI, I find it very difficult to have one conversation without the latter. Futile, even. I understand why people use social media anyway, and put the same lens on LLMs. I want a plan, and my plan is collective guardrail-building.
The problem of transparency as a cultural guardrail
If you’ve read the essay up until this point, here’s a fun game: there are four AI-generated sentences embedded within it. And no, none of them are quietly absorbing.
Can you tell what they are? No, you cannot.
I do this to point out that a lot of people are using AI in various different capacities. In books. Songs. Movie scripts. Reels. Research papers and businesses and customer care. Soon, photo, video, and other mediums will get better too. ChatGPT has 1 billion monthly users globally. It gets ~5.6 billion visits per month. I promise you—you cannot tell. You cannot tell. And it is going to get harder and harder to.
In the last month, I’ve been opening up discussions about AI, and have had 400+ people DM me: ‘I’m using AI, but I’m lying about it.’ Or ‘I use it, but I’m afraid to begin a conversation about how to limit my use or do so ethically.’ I suddenly feel like I’m carrying around way too many secrets for a woman who, for better or worse, doesn’t have any anymore. 400 is a wild number, and that’s the people who took the initiative to willingly tell me—it’s not an insight I sought out.
One of the truths I believe about the world is that a significant number of people, and perhaps a majority, will choose convenience and gratification over the ‘right’ thing to do. In the chapter Intermission in my book, I talk about the exact moment I realised this. When I was younger, I used to find it difficult to not moralize this fact and approach my work with empathy for those very people. I don’t find it difficult anymore. Distress is unproductive. Curiosity and a plan never are.
An important segment of interventions plays out in response to culture, and is as necessary as those that mitigate it. A shame-driven puritan approach will, in my view, ultimately divide people into three camps:
[a] Those who are okay with being seen using AI and will do so recklessly. These people could become disinterested in engaging with ethics conversations, and less curious in how to more safely navigate and limit their own use [causing writing ‘assistance‘ to turn into 100% slop, unchecked ‘therapy’ to elicit grandeur/psychosis, and so on]
[b] Those who will not use it and live in fear of a witch hunt, curbing self-expression and genuine, collaborative civic engagement that can pave the way for behaviour change, literacy, and ethical use
[c] Those who will use it but engage in non-transparency, deceiving consumers. IME this is a huge segment, but we don’t know how ‘huge‘ because nobody talks about it.
We need to create a culture that provides collaboratively created ethical guardrails on what it means to use LLMs, and opportunities for people to be forthcoming about whether they’ve used it or not, and if yes, in what capacity.
This is doubly beneficial because it helps the consumers of their work make an empowered decision about whether they want to consume it or not.
But our culture currently disincentivizes transparency. This poses a problem, specially in a country like India where, for reasons discussed above, the use of AI for text generation is becoming rapidly ubiquitous. The cost of collective betterment is individual discomfort, and unfortunately, we have been trained to prioritise the latter.
Experiments
And, as discussed earlier, LLMs are kind of useless in terms of evolving metaphors, structure, creative ideas, and imagination. And there is clearly something enticing about getting up at random hours of the day to jot down insights. I think I might have emailed myself 700+ times in a year. This is beyond the Notes app notes, beyond the Google Docs, beyond the Sheets, beyond the Notion, beyond the red moleskin notebooks, beyond beyond beyond…
A lot of people say that LLMs make for good editing, but this, I disagree with. My human editor used reasoning as an integral part of the editing process which benefited my book significantly. Her phrases, such as: ‘I’m wondering if…,’ ‘Just an idea…,’ ‘We should…,’ ‘May I problematise this…’ are not something that is replicable by chatbots.
However, again, I try to look at the other perspective with curiosity: to have a human editor at all is a big deal [and it took me years of writing to get a book deal that got me a human editor I did not have to pay.] I had not just her, but also a separate copy-editor, and two additional citations editors, and a lawyer, and a publicist, and a marketing team, and on and on… How many people have this? Basically a negligible quantity, right? I may not support it, and I want to find alternatives, but I deeply understand the incentive to just say ‘to hell with it‘ and get the benefit for free.
After realising the labour of writing my book, I actually think the amount of human intervention required to create a traditionally published book will eventually be seen as unsustainable and become affected by automation. Another punt in a string of tragedies. But that’s a different conversation…
So, what is left? Is there a real creative way to let the stochastic parrots chirp?
One creative use I found was to make a long, three-page list of absurd consumer products aimed at women.
I liked that there were hallucinations within, which I was unable to discern because there are so many products now that they might as well be real. This distortion of reality mimicked the way I write about products slowly distorting women’s physical reality within the chapter (Shiny Plastic Sedatives).
I also liked the fact that this list could exist because of a simple reason: so much of feminine maintenance is well-categorised, inventoried, searchable, and listed now, that bots are well-versed in hundreds of thousands of products [the absurdity of that number] and can easily reproduce them, just as they reproduce other kinds of harm towards women. The fact that so much of our industrial prowess has gone into the organisation of the beauty industry is what makes this generated list possible in the first place. And the fact that LLMs can hallucinate new products—giving prompting business-owners newer and newer ideas to fragment and exploit women—is fascinating.
Now, given the fact that I prompted the bot to create an artefact based on a unique creative idea with some level of critical thought behind it, is this ‘technologically-mediated art’ or is ‘LLM-generated slop?’ I don’t know, I guess we’ll have to ask Anadol.
Here’s a little poem that goes with the chapter. Just because. I wrote it, duh, because imagine getting AI to write a poem. Ha ha.
"I’m all neon panther claws and prima donna lashes.
I craft with gluey resin rhinestones on a phone.
I detonate a diamond bomb to watch powder refractions
fall out softly like asbestos snow.
Tinkling-jingling windchime with rose-quartz in my pocket,
If I paint it, was it anything but pink?
Shiny plastic sedatives for this baby in a bonnet . . .
They put a price now on all sorts of things.”
Beyond this [and very jaded at this point], one use I found was to take some sentences from my finished manuscript and use the bot to simplify them. This, I found it can do well with its own limitations, but is a use-case I only find helpful in nonfiction writing that is trying to educate the reader on a new topic.
I write in a lot of spiralling metaphors and with a lot of intensity, as discussed in previous sections of this essay [or as you can simply see in the fact that I’ve taken 8,500 words to share my insights]. I also write through story, i.e. instead of laying down plain facts, I talk about the stories of real people and let readers arrive at those fact themselves, as personal conclusions. For example, in the chapter Pick Me, instead of simply talking about how the supply chain of images works, I took the reader into the labyrinth of my experience on #BindiTwitter and with an Indian media outlet, the trafficking of girls on the Indo-Nepal border, and the history of Brooke Shields and Playboy. I do this to humanise the characters and show the reader the real-life implications of cold-hard statistics.
There were some spaces in the book, however, where I wanted to explain to the reader exactly what I meant in a simple way, so that they could grasp my point precisely before I moved on to the next point. There, my approach was: hash out the idea freely on a doc in ~5,000 words > manually rewrite it in 2,000 words > manually rewrite it in 200 words > get the core idea down to a sentence > use the bot where needed to strip it down to a simpler, ‘informational’ kind of sentence. Or, alternatively, feed it at the 200 word stage and see what sentence it can distill it down to. This was helpful for a while, about 20-30 sentences, but at some point I realized the bot couldn’t really do anything I couldn’t do, so it was a matter of a confidence to get down to the rest of the final sentences myself.
I didn’t find this helpful in creative writing, because AI’s idea of creativity always seemed to grate against mine. For example, in the chapter Fever Dream, I quote an old Instagram post of mine:
Everyone is good and I am bad.
Everyone is smart and I am stupid.
Everyone is beautiful and I am ugly.
Everyone is kind and I am selfish.
Everyone is worthy and I am worthless.
Everyone is right and I am wrong.
Everyone is right and I am wrong.
Everyone is right and I am wrong.
Bots kept insisting that the last three sentences should be collapsed into one, because the repetition was apparently not doing anything useful. I found this funny, because it is: anyone who has had an OCD spiral will tell you that repetition is a key component of rumination, and that negative thoughts loop inside the brain. But it doesn’t have a brain, so I guess it doesn’t know.
I have a personal ethical opinion where [a] I don’t agree with using LLMs to generate passages, or ideas, or sound boarding, [because of the creative and plagiarism implications] and [b] find it more acceptable to seek the use of LLMs as assistance in advocacy [e.g. in my case, sharing a perspective on gendered harm in as simple a way as possible] rather than for profit. But even here, in terms of my personal stance [which you are free to disagree with] I land back at my core thesis: I do not moralize the end-user. I look at sociotechnical incentives built by tech businesses.
So that’s the end of my long spiel!
Will I experiment with LLMs again in my next book? Well, to begin with, I don’t think I’ll write a book for the next 5 years because I am so burnt out from this one. It took a massive emotional toll to put down my convictions, and I need to go sit in a cave for the rest of eternity. But when I do get around to the book 2: probably not. Certainly not until my final manuscript is finished. And even then, only for something in the style of meta-commentary like that product list. Because all the way up until that point, the chirping parrots are frustrating, detrimental, manipulative, and do, indeed, make work enshittified.
I’m currently exploring local LLMs, and my progress with those will determine what comes next. In general, after the initial hype of experimentation and apart from my continued use for ongoing HCI research, I’m moving away from big-business bots because I don’t want to constantly be a security leak away from the world finding out about my indigestion or OCD spiral.
It’s been really nice to be talking to real humans/experts and engaging in community work focused on cross-pollination of perspectives and literacy. After my half-hearted book promo, I’m eager to get off social media again [stat] and back to the real world.
Conclusion
I hope there was some new insights you learnt from these experiments, which can help you navigate AI better in the sense of limiting your use and trusting your intuition.
I was inspired by this video by Eddy Burback, ChatGPT made me delusional, where he ran an experiment letting AI control his actions and landed up in a camper van, paranoid, in the middle of nowhere. It is, to me, a genuinely important public resource teaching 5 million+ viewers how to be literate in emerging technology through the power of storytelling.
Write to me at kaursukhnidh@gmail.com.