Meta finally confirmed the release of Llama 3. At an event in London, Meta provided some details and the release time frame for their advanced AI model. Executives from Meta said that they will soon release the next version of their open-source large language model, as reported by Techcrunch.
It is expected that Meta will first release a small version of Llama 3, and a full open source model will be released after that, claimed The Information. But why a smaller version in the first place? We think that The Information claim makes sense, as this will provide an opportunity to build some hype about the capabilities of the full model.
What we know about Llama 3 and its release
The release of Llama 3 will challenge the dominance of LLM giant GPT 4, as it is expected to be the most advanced LLM in the open source category and will be available in different models like Llama 2. Nick Clegg, Meta’s president of global affairs, said at the event,
“Within the next month, actually less and hopefully in a very short period of time, we hope to start rolling out our new suite of next-generation foundation models, Llama 3.”
Clegg’s statement clearly hints at the availability of different versions of Llama 3, as he further added,
“There will be a number of different models with different capabilities different versatilities [released] during the course of this year, starting really very soon.”
source: Techcrunch
Talk about The Information’s claim; it seems legit, isn’t it? So now what we know about Llama 3 is that it will be coming in different versions and sizes, from small to very large, fully capable models ranging from the likes of Haiku, Claude, and Gemini Nano to the behemoths like GPT 4. We don’t know more, but we for sure know that it will be open source, like Llama 2 and Multimodel as well, with abilities to understand texts along with visuals.
Critics comparing open source to closed models
Llama is an open source model, but it’s not hanging in the air. We know there is strong support of Meta and its resources behind it, and Meta wants to leverage third party developers and researchers. But, despite all the favoritism of open source models, some critics are still skeptical of their competence as compared to close private models like GPT 4. They have their own reasons for their views on the matter.
The argument starts with resources to establish giant data sets for training models, claiming that models like ChatGPT have huge data sets structure with human tagged data. Of course, no machine can still match human capabilities, but are we moving closer to it? may be or may not be for the next two years. Another reason is the availability. While this is a valid point, we have seen in tech history that smaller startups sometimes beat giant corporations on some key features with their out of the box solutions. But the reality is that OpenAI is home to engineers with salaries exceeding a million dollars.
Another point they make is the team framework, the argument is that an open-source decentralized approach can’t match the centralized teams accomplishments. Public cloud infrastructure is also of no match to the ones that OpenAI and Google boast, which also hinders open source teams from making incremental improvements at a fast pace. Altogether, these are enough reasons that go against open source systems. But, as we know, despite Llama being open source, there is Meta behind it, supporting it by allocating humongous resources.
Meta investing in GPUs
Back in January, Mark Zuckerberg said that they are building a massive compute infrastructure with new 350k H100 GPUs, and if you include other GPUs, then almost 600k H100 will be on board. Mark said this in a video. This will be much more computing power than that of OpenAI, which it used to train GPT 4. The investment is in the billions of dollars for AI alone.
Llama 3 will power the Meta AI assistant, and it will also be integrated into an array of products across the whole app range of the company. Is Llama 3 going to challenge the dominance of GPT 4? Looking at the track record of its predecessor, Llama 2, was the backbone of the most advanced open source LLMs, like Mistral from France and a few more, which were built by tweaking and optimizing it.
Some of them even surpassed GPT 3.5 in performance, so we can speculate that Llama 3 with its advanced capabilities will be used for new LLM products that will match or exceed GPT 4 performance with shared and lowered training costs, making it possible for more people joining in.
The competition is going to be far tough, with Open source community trying to innovate over LLama 3 and coming up with indigenous solutions like before, and OpenAI also releasing GPT 5 in the next few months. We are in awe for the interesting time ahead.
