How LLMs affect creative writing
LLMs can help you become more creative in your writing. But the overall effect is less diversity across the board.
An interesting new study by UCL School of Management and University of Exeter explores the impact of LLMs on creative writing. And the results are a bit nuanced.
The study focuses on eight-sentence short stories. They measured creativity based on novelty and usefulness.
Participants were given a random topic. Some were told to write the article manually. Others were given the option to get a three-sentence suggestion from GPT-4. A third group was given the option to get five AI-generated suggestions.
Expectedly, the results show that getting help from LLMs increases the novelty and usefulness of written stories, especially for writers who scored lower on baseline creativity assessment. The stories, per the writers themselves as well as 600 evaluators, are more interesting, enjoyable, better written, and more likely to have plot twists.
“Having access to generative AI ‘professionalizes’ the stories beyond what writers might have otherwise accomplished alone,” the researchers write.
On the other hand, stories based on AI-generated ideas were less unique, and mass use of AI can lead to less overall diversity across different writers.
“Specifically, if the publishing (and self-publishing) industry were to embrace more generative AI-inspired stories, our findings suggest that the produced stories would become less unique in aggregate and more similar to each other,” the researchers write.
This study only focuses on short-story writing, and the effects of LLM-assisted creativity can vary across different modalities and tasks. But it will be interesting to see what happens as LLM-assisted and -generated content start to fill the internet and be used to train the next generation of models.
Read more about the study on VentureBeat
Read the study here
This matches my experience. GenAI makes average writers better but over time creates a “sameness” in their writing, in content and in style.
Interesting piece of research. Your last point resonated deeply. When LLM generated content floods the internet, will the LLMs train on that data in the future? That would be regressing to the mean.