In a groundbreaking study, researchers have uncovered the intriguing dynamics between human creativity and artificial intelligence (AI). While AI chatbots, especially ChatGPT4, demonstrated exceptional creativity, they did not consistently outperform the most creative human participants. Instead, humans showcased a wider range of creative potential, potentially linked to variations in executive functions and cognitive processes.
Traditionally, creativity has been regarded as a distinctively human trait, rooted in complex cognitive processes such as imagination, insight, and the ability to connect seemingly unrelated concepts. However, as AI technology continues to advance, it has become increasingly evident that machines possess the capacity to produce creative outputs that rival, and at times, even surpass human achievements.
Study author Simone Grassini, an associate professor at the University of Bergen, highlighted the transformative nature of the current era in AI and machine intelligence. He emphasized the need for ongoing research into how people perceive machines and the extent to which machines can imitate human abilities.
Assessing creativity with the alternate uses task
The researchers conducted their study using the Alternate Uses Task (AUT), a widely recognized creativity assessment. In this task, both human participants and AI chatbots were tasked with generating unique and creative uses for common objects like a rope, box, pencil, and candle. Human participants were given 30 seconds to generate as many creative ideas as possible, while chatbots were instructed to produce a specific number of ideas using only 1-3 words per response. Each chatbot underwent testing 11 times.
The study featured three AI chatbots: ChatGPT3, ChatGPT4, and Copy.Ai, alongside a group of 256 human participants recruited from the online platform Prolific. The human participants, all native English speakers, had an average age of 30.4 years, with a range from 19 to 40 years.
Two approaches to assess creativity
The researchers employed two distinct approaches to assess creativity in both humans and AI chatbots. Semantic distance scores: This automated method gauged the originality of responses by measuring how different they were from common or expected uses of the objects.
Subjective ratings of creativity: Six human raters, unaware of which responses were generated by AI, evaluated the creativity of the ideas on a 5-point scale.
AI chatbots excel in creativity assessment
The findings of the study revealed that AI chatbots, notably ChatGPT3 and ChatGPT4, consistently achieved higher semantic distance scores compared to humans. This indicated that they produced responses that were more original and less conventional than those of human participants. Additionally, human raters rated AI chatbots, particularly ChatGPT4, as more creative on average than the human participants.
However, it’s crucial to note that while AI chatbots performed exceptionally well, they did not consistently outperform the most creative human participants. In some instances, highly creative individuals among the human participants were able to compete with AI in generating novel and imaginative responses.
Grassini emphasized the importance of not overestimating the implications of AI’s success in this specific creativity task for complex real-world jobs that involve creativity. He envisioned a future where AI chatbots assist humans in their creative roles rather than replacing them entirely.
Among the AI chatbots, ChatGPT4 stood out as the most creative when considering subjective ratings. Although it did not consistently outperform other AI models when assessed using an algorithm to measure semantic distance, it consistently received higher creativity scores from human raters.
Limitations and future directions
The study’s findings are limited to one type of creative behavior and may not be broadly applicable to creativity as a complex phenomenon. Moreover, comparing creativity at process levels between humans and chatbots remains challenging due to the opaque nature of chatbot internal processes.
Grassini noted that chatbots may not exhibit creativity in the same way humans do, potentially relying on memorized responses rather than the ability to generate creative ideas. Given the architecture of these models, it remains impossible to fully understand their creative processes.
