In a recent revelation, Google’s unveiling of its Gemini artificial intelligence (AI) model has come under scrutiny, raising questions about the actual capabilities of the technology and its competitive standing against OpenAI’s ChatGPT.
While Google’s initial demonstration of Gemini appeared impressive, it has now been revealed that the presentation was carefully curated and did not accurately represent real-time interactions with the AI system. This development has cast doubt on Gemini’s abilities and suggests that Google may still have some ground to cover in the AI race.
A closer look at the demo
Google’s Gemini AI model garnered attention with an eye-catching demo, but closer examination has revealed that the presentation was not a genuine representation of its capabilities. According to Bloomberg, Google made several modifications to the interactions with Gemini to create the demo, raising concerns about the accuracy of its claims.
One noteworthy alteration was the reduction of latency for the demonstration. This implies that the AI’s actual response times may be significantly slower than what was portrayed in the video. Furthermore, the output from Gemini was shortened for brevity, which means that the responses shown in the demo may not reflect the AI’s typical responses in real-world scenarios.
Perhaps the most significant revelation is that the demo was not conducted in real-time or in a voice format. Instead, it was constructed using still image frames from footage and prompts provided via text. This means that Gemini was essentially identifying content within static images rather than engaging in dynamic, real-time conversation as suggested by Google’s presentation. The disparity between the demo and reality has left many questioning the accuracy of Gemini’s capabilities.
Comparing Gemini to OpenAI’s ChatGPT
Google’s demo also claimed that Gemini outperformed OpenAI’s GPT-4 model in nearly every benchmark test. However, a more detailed examination of the numbers reveals a less substantial lead for Gemini. Despite GPT-4 being available for nearly a year, Gemini only managed to edge ahead by a few percentage points in many of the benchmarks. This suggests that Gemini may have only recently caught up to OpenAI’s product and raises the possibility that the landscape could shift once GPT-5 is introduced.
Users of Gemini Pro, the version powering Google Bard, have expressed their discontent with the AI model. Reports on X (formerly Twitter) indicate that Gemini is susceptible to common “hallucinations” experienced by other chatbots. For example, when asked to provide a six-letter word in French, Gemini confidently produced a five-letter word, highlighting its limitations in non-English languages.
Users have also encountered frustration when requesting accurate code generation and summarization of sensitive news topics from Gemini. Even relatively simple tasks, such as naming the most recent Oscar winners, resulted in incorrect responses. These user experiences suggest that, for the time being, Gemini may fall short of the high expectations set by Google’s slick demo.
