Daily Maverick, a prominent news publisher, embarked on an ambitious journey in late 2022 to harness the power of AI, particularly OpenAI’s ChatGPT-3, to enhance its news content. This endeavor aimed to explore the potential of AI-driven summaries, headlines, and more in the newsroom.
Exploring the power of GPT-3
Daily Maverick initially experimented with OpenAI’s Playground, venturing into various AI-driven tasks, including summarization, headline creation, error-checking, and tweet generation. The results provided valuable insights into the capabilities and limitations of GPT-3.
GPT-3 exhibited proficiency in summarization and displayed reasonable skills in crafting headlines. However, it proved inadequate in detecting copy errors, and its tweet generation was often unpredictable, oscillating between brilliance and unusability. One significant challenge was its inability to limit character counts or adhere to specific summarization formats, reflecting its limited ability to plan.
Mismatch in summary goals
Another challenge emerged from GPT-3’s approach, which aimed to summarize every major point in an article. This didn’t always align with a newspaper’s objectives. News outlets may prefer to entice readers with concise summaries and encourage them to explore the full articles for in-depth information. In this regard, AI summaries sometimes made the articles redundant.
Handling varied article styles
GPT-3’s struggle extended to articles with non-standard structures. Flowery introductions, common in opinion pieces, often confused the AI model, resulting in off-topic or inaccurate summaries.
Maintaining the editorial tone was crucial, and GPT-3’s summaries often lacked the characteristic style of Daily Maverick’s content. This issue was mitigated by instructing the AI to generate summaries “in the style of Daily Maverick,” although occasional unwanted phrases persisted.
Human-in-the-loop approach
Recognizing the need for human oversight, Daily Maverick introduced a “human-in-the-loop” concept, involving editors in the summary generation process. Summaries were marked as unapproved until an editor could review and decide to accept, reject, or edit them. This approach added a layer of quality control to prevent potential mishaps.
Daily Maverick developed the SummaryEngine WordPress plugin, featuring two interfaces for editors. One interface integrated seamlessly into the article editing page, while the other provided an overview of all summaries for efficient review. This streamlined workflow catered to editors’ preferences for a single, user-friendly screen.
User-centric design
Daily Maverick learned that user-centric design is paramount. A subtle change, replacing “Unapprove” with “Reject,” had a significant impact, emphasizing the importance of aligning the software’s language with users’ mental models.
Editors began using the AI-generated summaries, even though the organization was still formulating their purpose. Various options were considered, such as summary newsletters, mobile app integration, or in-article popups. Daily Maverick chose to quietly launch a new interface, featuring short summaries and bullet points, which garnered significant reader engagement.
Expanding possibilities
This successful project opened the door to further AI experimentation. Daily Maverick explored GPT-4 for improved length and tone control, even considering a TranslateEngine for translations. AI-powered headline suggestions were integrated, aiding news editors in crafting reader-centric headlines that contributed to a substantial increase in readership.
Daily Maverick’s foray into AI-driven content generation has showcased the immense potential of technology in the newsroom. While AI can significantly enhance efficiency and reader engagement, it must be used as a tool to empower human editors rather than replace them. Daily Maverick’s journey serves as a testament to the promising future of AI in journalism, where innovation and user-centric design can revolutionize news delivery.
