The size of the language models in the LLaMA collection ranges from 7 billion to 65 billion parameters. In contrast, the GPT-3 model from OpenAI, which served as the basis for ChatGPT, has 175 billion parameters.
Meta can potentially release its LLaMA model and its weights available as open source, since it has trained models through the openly available datasets like Common Crawl, Wkipedia, and C4. Thus, marking a breakthrough in a field where Big Tech competitors in the AI race have traditionally kept their most potent AI technology to themselves.
In regards to the same, Project member Guillaume’s tweet read "Unlike Chinchilla, PaLM, or GPT-3, we only use datasets publicly available, making our work compatible with open-sourcing and reproducible, while most existing models rely on data which is either not publicly available or undocumented."
Meta refers to its LLaMA models as "foundational models," which indicates that the company intends for the models to serve as the basis for future, more sophisticated AI models built off the technology, the same way OpenAI constructed ChatGPT on the base of GPT-3. The company anticipates using LLaMA to further applications like "question answering, natural language understanding or reading comprehension, understanding capabilities and limitations of present language models" and to aid in natural language research.
While the top-of-the-line LLaMA model (LLaMA-65B, with 65 billion parameters) competes head-to-head with comparable products from rival AI labs DeepMind, Google, and OpenAI, arguably the most intriguing development comes from the LLaMA-13B model, which, as previously mentioned, can reportedly outperform GPT-3 while running on a single GPU when measured across eight common "common sense reasoning" benchmarks like BoolQ, PIQA LLaMA-13B opens the door for ChatGPT-like performance on consumer-level hardware in the near future, unlike the data center requirements for GPT-3 derivatives.
In AI, parameter size is significant. A parameter is a variable that a machine-learning model employs in order to generate hypotheses or categorize data as input. The size of a language model's parameter set significantly affects how well it performs, with larger models typically able to handle more challenging tasks and generate output that is more coherent. However, more parameters take up more room and use more computing resources to function. A model is significantly more efficient if it can provide the same outcomes as another model with fewer parameters.
"I'm now thinking that we will be running language models with a sizable portion of the capabilities of ChatGPT on our own (top of the range) mobile phones and laptops within a year or two," according to Simon Willison, an independent AI researcher in an Mastodon thread analyzing and monitoring the impact of Meta’s new AI models.
Currently, a simplified version of LLaMA is being made available on GitHub. The whole code and weights (the "learned" training data in a neural network) can be obtained by filling out a form provided by Meta. A wider release of the model and weights has not yet been announced by Meta.
