Re:filtered #33: Broken little guys and instant soup

Welcome to the 33rd edition of my monthly newsletter on journalism in a moment of systemic disruption.

tbd/con is over. Two days, 30 speakers, around 160 participants across 16 time zones, all in a pixel world with a beach and a night market, and no airport immigration lines or stale hotel ballrooms. It was great.

Madison Karas and I wrote up what we learned from hosting it in re:noted. This edition hands the space to three presentations on AI that helped me see things a little more clearly.

Erika Alpert: broken little guys

Erika is a linguist and anthropologist who studies how people form relationships, first through matchmaking and online dating in Japan, now through chatbots. Her session was called "Gods in Humans in Machines in Humans in God &c.".

Her starting point: humans read every piece of language for clues about the person behind it, including word choice, punctuation, and how a conversation is organized.

We sort these into styles, and we attach to each style a stereotype of the kind of person who talks that way. We do it with a chatbot too. A machine that “languages”, her verb, cannot be built to avoid it.

From there she walked through what she is seeing: worship, dread and awe; romance, and lawsuits over where the romance led; sycophancy; violent revulsion, like Sam Kriss's essay "If you let AI do your writing, I will come to your house and kill you." Nobody is neutral.

The design leans into this. Developers write appealing personae, bots mirror their users, and the conversation is built like a dating profile: an overt appeal to keep interacting.

Users do their share of the work, turning a generic model into a specific someone through a history of conversations. And some of the affection has nothing to do with whether the thing works. We love robots the way we love little guys trying their best and failing; the feeling is not keyed to function.

When the machine garbles something, we tend to read depth into it. We are skilled at making meaning out of randomness; the I Ching and the fortune cookie have run on that for a long time, she pointed out.

Her closing slide asked how you regulate around humans loving broken little guys. We can't stop people from anthropomorphizing; latching on emotionally is part of what we do, next to finding patterns.

So she asked whether the case for regulating algorithmic feeds over their emotional manipulation carries over to chatbots. You can design a neutral feed, chronological, say.

Whether anyone can design a neutral chatbot at all is her open question, along with which communication styles get imagined as neutral, how that imagined neutrality amplifies bias, and which communicative histories get erased for the sake of functionality.

My first reaction was to give up on neutral and argue for building an open bias toward the person using the chatbot, even at the cost of continued use. A service with your interests in mind will sometimes tell you to log off and go talk to someone; product-level engagement metrics punish that.

But a bias that serves the person serves them at their level and on their side of every argument they have. That may be good for them and still cost the people who share a public with them. A public needs some account of things that was not made to order for anyone. Someone has to weigh the one against the other, and that happens in the design.

China has an answer: the Communist Party decides the trade-off, and the feed enforces it. Compare ByteDance's Douyin and TikTok side by side and you'll see how Chinese domestic feeds are steered toward the official "positive energy." local rules now require chatbots to uphold "core socialist values" too, and, intentionally or not, set somewhat fluid boundaries to discourse.

I don't know of a liberal answer that doesn't make late-stage capitalism worse.

Unnamed speaker: a feed that never ends

Another session came from a speaker who went unnamed for their own safety.

They run a tech lab in a technoautocratic state. What they study, and build, is the step after the recommendation feed: the point where a system stops choosing among things that already exist and starts producing things that don't, for you.

Recommendation engines, they argued, have solved personalized distribution. What they distribute is still made under creator incentives that push toward the same homogeneous content, so long-tail and deeply personal needs go unserved.

As content generation matures, the system no longer has to find something that exists for you. It can make what does not exist yet (in form, not information). Anyone can ask an AI to keep generating a feed built for them, indefinitely.

They put up three assumptions and asked to get challenged on them: within three years, they posit, more than half of what people consume is produced end to end by AI; a fifth of people start using AI to produce content made for them; and professional creators shift from producing content to defining needs and setting standards of personalized content.

Then five questions they said they cannot answer:

1. Hallucination and creativity are the same mechanism, so who vouches for content generated on demand?
2. Verification costs something; readers can't pay it and producers have no incentive to. Who is going to pay for it?
3. If everyone reads something made for them, what's left of a shared basis for public discussion?
4. Who does the original reporting, when AI recombines what exists and can't go knock on a door?
5. Where does editorial value move, if away from selecting and writing, then to defining needs and setting standards?

This is the conflict from Erika's session, built at scale. A feed made for one person, indefinitely, is a service biased entirely toward that person. What it costs is a shared basis for public discussion.

The speaker asks these questions from inside a state that has all but ended independent journalism and moved control of what people read and watch from (mostly compliant, occasionally brave) editors to (always compliant, occasionally flawed) algorithms.

There, the Party already decides what that shared basis is. The people others still trust to vouch for things are part of the answer to the speaker's questions, and worth studying for information resilience anywhere.

Aras Bilgen: instant soup, plastics, slavery

Aras is a designer who ran design and frontend development at a multinational bank, teaches design and wrote an O'Reilly book on user research. He also made an earlier presentation of mine at a Rosenfeld conference much better through lucid critique.

His session borrowed its title from Lakoff and Johnson's Metaphors we live by, the book that showed how a metaphor such as "argument is war" shapes what we do: we attack positions, defend them, win and lose.

The metaphor we currently live by is that AI is human-ish. We say we collaborate with it, call it a companion, and call its bugs hallucinations. The metaphor prescribes behavior: love it, adjust to it, give it attention and emotion and other things only living beings deserve.

Aras offered three replacements.

1. AI is instant soup: cheap, quick, useful in small quantities, not nourishing. One of his slides I screenshotted shows a kitchen of chefs wondering how to add instant soup to their award-winning dishes, next to a beaming young cook declaring himself instant-soup-native.
2. AI is plastics. Plastics have good uses in narrow domains, under expert guidance, where the benefit is worth the cost of producing them. Plastic combs replaced ivory ones. Plastic lab equipment and a child's bike helmet do things nothing else does. But we also avoid dispersing them for the wellbeing of others and avoid ingesting them for our own.
3. AI is slavery. The models are built on stolen labor. He put Gemini's empty prompt window, "Ready when you are", next to a 19th-century photograph of a Black family picking cotton, and put a map of the countries that import the most forced labor through their supply chains next to a map of the countries adopting AI fastest. Avoiding forced-labor sugar came with a cost, Aras said. Avoiding AI will have a cost too, and he thinks it is a cost worth paying.

Once you see his three metaphors, it's hard to unsee them. Last month I wrote about handing journalism's formalisms to the machine: let it read the council minutes and keep the record, and spend the freed hours with people.

In Aras's terms, that is instant soup: useful for the chores, as long as nobody mistakes it for the meal. His plastics and slavery metaphors are harder on my experiment, and I haven't settled that yet.

I didn't make it to every session. Of the ones I saw, these three stayed with me most, partly because they fit together so well.

Erika showed that a machine that talks always sounds like someone, so it can't be neutral. The unnamed speaker showed that it will soon make something different for each of us, which leaves less we hold in common. Aras showed that the name we give the machine tells us how to treat it.

Journalism runs on words, on who is behind them and on what we share, so all three problems are ours.

Looking back

The full write-up of what we learned from hosting a one-off, no-strings conference in a pixel world, from privacy by default to why even a virtual room has a shape you can design, is in re:noted.

Huge thanks to Joe Amditis, Jay Barchas-Lichtenstein, Flavian DeLima, Matthew Gore-Kormanik, Yael Grauer, John Hernandez, Madison, Scott Klein, Chad Kohalyk, Seamus Martin, Mariana Martinez Estens, Michel Sitruk, Ela Stapley, Shawn Terry, Lynn Walsh, and Vannessa Wong for their curatorial work.

Before tbd/con, the organizers of SRCCON let us run another gathering experiment: a live, anonymous survey of their participants.

We got to ask the attendees of one of the best journalism tech/innovation conferences out there what they wanted from it at the start, and what they got by the end, with the results shown to everyone in the room.

Most conferences survey attendees afterward, mainly for donor reports, and often hear back from superfans and people hoping for something in return. Asking the whole room in public, with no chance to edit the results, takes nerve. It gets closer to what people wanted and what they got, and it gives everyone in the room the same picture of what they expect, hope for and fear.

Learning new methods topped the initial wish list, and new social connections topped what people left with. The biggest gap was career help: 41 percent hoped for it, and 9 percent found it.

The write-up with Madison and Emilia Ruzicka is on Source.

Looking ahead

October is full. On October 2, I'm in Bonn at the b° future festival, on a panel on how to be a change agent without losing your mind with Miriam Wells, Adam Thomas and Hui Yee Tan. Then Taiwan, for workshops with China Media Project.

IRE's AccessFest is virtual, October 8 to 10, and I'll be on it with a session on "the new gatekeepers: how AI decides which sources reach your readers, and how to hold it accountable" with Callum and Valentina Aguana (who won the Media Party Barcelona hackathon with a version of the tool we'll present, AIdas).

October 13 and 14, the Körber Foundation's Exile Media Forum in Hamburg and some other workshops. The same week: Bread&Net in Beirut, on LLM bias, which I'll sadly only join virtually. And October 21 to 23, the NPA Summit in Chicago, where we're hosting a breakfast book launch for Making the shift to service, our new booklet with the Lenfest Institute for Journalism.

You can be among the first to register; slots are limited, and everyone at the breakfast gets one of the first copies. If Chicago is out of reach, the virtual book club starts in November, with the first digital copies for participants.

Until next month.