Tools have been cropping up that claim to be able to expose the use of generative AI (genAI) in a given piece of writing. Around the same time, there have been accounts of people being accused of using genAI when they haven’t. 

The latest casualty in this fight has been — wait for it — the mighty em dash, which is apparently now a sure sign that a writer has used the technology.

While I don’t use the em dash often, I have whenever the situation called for it, as have many others. It’s absurd to me that anyone would take this seriously. But people do, and I’m concerned about the potential consequences for people who are falsely accused of using genAI in their work.

But it has gotten me thinking… how can we tell if a given piece of work used genAI? I’m not sure we can, because of the way these systems are built.

How does genAI work?

For the uninitiated, generative AI systems are trained using reams of data scraped stolen from the internet. This includes data which is free to access, data which is paid, and even digitised copies of physical media purchased illegally from the dark web. This information is then fed into the systems that power these models, which, in turn, “create” their content based on all of this information they have been trained on. Without this training data, they would not exist. 

The most well-known genAI systems are large language models (LLMs), which generate text based on the prompts you supply them with. There are other genAI systems which generate images, music and other forms of media; these are known as multimodal foundation models (MFMs).

There are ethical concerns about the way these machines are trained using data they do not pay for, and the writers and artists whose works are being stolen are not happy about it. These systems have tremendous negative societal and environmental impacts, especially when it comes to climate through energy use and water. I might write a longer article about this in future; probably a rant whenever I next hear someone claim that this is all an acceptable cost since AI will solve all our problems in the future (spoiler: that is bullshit).

LLMs are probability machines. When they generate text, they calculate which word would make the most sense to come next in their current sequence, based on the data they have been trained on. They do not understand any of their outputs, which is why they frequently lie (the industry euphemistically refers to their lies as “hallucinations”). Even calling the machine a liar doesn’t feel quite right; it’s not really, since it doesn’t understand anything it’s outputting. Using any humanising language isn’t correct, really, as genAI does not “think”, “calculate”, or anything of the sort. It works based on probabilities. However, it is hard to talk about these machines without using anthropomorphising language, which is why I continue to do so. 

The problem

Every single output from genAI is derivative of its training data. The more something appears in their training data, the more likely it is to appear in their outputs. Therefore, anything that is an apparent hallmark of generative AI is also a hallmark of human writing.

Em dashes apparently frequently appear in genAI outputs because people use them a lot, so they were overrepresented in training data.

As far as I can tell, this would be true of literally anything. If something appears in a lot of genAI outputs, it is because they have been used by countless people across the internet, in books, on forums, and more. This is why I don’t think it’s possible to be able to detect genAI properly; I think this will happen every time anyone tries to invent some way to do this. People will always get caught in the crossfire for the crime of using common literary devices or writing in a clear, concise way.

A video game called Little Droid recently stoked controversy by apparently using genAI to create the game’s cover art. Except they didn’t. The artist’s style is reminiscent of that weird sheen you get on a lot of images produced by genAI, but was in fact created by an actual artist. They even provided a video of the revisions the art went through, including the layers in their program (check it out in that link), and people are still convinced they used genAI.

Why do people fall for it?

It’s tempting to be able to tell, in a snap, whether something falls in or out of line with our moral code. Having to actually spend time to figure it out for anything we find suspect would suck up a lot of energy. And, to be frank, if someone took a LLM output and edited it, it would be damn near impossible to know. LLMs and MFMs are even getting better at mimicking the styles of specific writers and artists, as we saw a few months ago with the repulsive Studio Ghibli trend.

I’ve had a teacher I know tell me they can tell if a student used an LLM for an assignment because they know the student and what they’re capable of. This… seems fraught. For one, it risks magnifying biases any given person may hold around gender, race, class, sexuality, disability, neurodivergence, or any other dimension. Aside from that… what if the student just improved? What if they turn in an exceptional piece of work because it was about something they love or have an aptitude for, but then get accused of using an LLM because they’re benchmarked on previous work they don’t like or are unsuited to?

I’m not sure what the solution is here. But we need to resist ideas which simplify too much just because they’re appealing; we need to make sure they work.

I deliberately phrased the title of this article as a question because I don’t know the answer. I think I’m correct; I don’t think there’s a definitive way to tell. However, I’m not an expert on this matter. If you are, feel free to jump into the comments and give me your opinion.

Further reading

If you’re interested in some books on the AI industry, here’s three I’ve read this year:

• Empire of AI by Karen Hao. Hao has been reporting on the AI industry since 2019. She drew on her experience and wealth of contacts to create an outstanding book about the key people in the industry, as well as how AI and Silicon Valley attitudes have evolved over time. She also discusses some of the environmental and societal impacts of the technology. As Hao is a journalist, it’s written in a way that’s easy to understand. I’d recommend this book to anyone.
• Atlas of AI by Kate Crawford. This one takes you around the world to look at the way AI is built by various exploited people and environments around the globe; it is, truly, an atlas. I enjoyed it a great deal, but it is a bit dense, so if you struggle with more academic texts, this one may not be for you. If you’re fine with that, though, I definitely recommend picking it up. I thoroughly enjoyed it.
• Work Without the Worker by Phil Jones. This is super dense, but also short and to the point, and I loved it. It goes into detail on how AI systems are trained by low-paid workers to recognise things in images, and the people who are exploited in the process. It goes into granular detail on these microtasks and connects this style of work to global oppressions and the constant corporate drive to control their workers, no matter the cost. Fantastic book.

Here are several others on my list I haven’t gotten to yet:

• The AI Con by Emily Bender and Alex Hanna.
• Unmasking AI by Joy Buolamwini.
• The New Age of Sexism by Laura Bates.
• Supremacy by Parmy Olson.
• More Everything Forever by Adam Becker.
• Rebooting AI by Gary Marcus (who you can find here on Substack) and Ernest Davis.
• The Myth of Artificial Intelligence by Erik Larson.
• Code Dependent by Madhumita Murgia.
• Ghost Work by Siddharth Suri and Mary Gray.