Llama 2: Unleashing the Open-source Generative AI model

We have read enough stories about how a whizkid named Bill Gates developed a software which was later developed as Microsoft, and how it made him the richest man on earth. The Microsoft family of softwares was copyrighted, but created a paradigm shift that revolutionised the entire software landscape.

At the same time, at the Massachusetts Institute of Technology a visionary called Richard Stallman who believed in users’ freedom to study, modify, and share the source code announced the GNU Project (GNU’s Not Unix). The timelines are interesting. While MS-DOS was officially released in 1981, Stallman announced the GNU Project in 1983. Since its inception in 1985, Windows 1.0, the first Operating System (OS) of Microsoft with graphical user interface, almost monopolised the software industry. While Gates focused on a historic software product, hacker-turned-software-evangelist Stallman prioritised the fundamental concept of freedom and aimed at steering clear of monopolies.

During the fall of 1991, a Finnish software engineer called Linus Torvalds, attracted by the ideals of Stallman and his free software movement, decided to turn his licence for path-breaking Linux prototype kernel into GNU licence. Earlier that year, he had released the kernel with a licence that prohibited commercial usage (Interestingly Linus also authored the hugely popular open-source versioning system, Git). Its open-source model allowed developers to access, modify and distribute it freely under its licence. It attracted a large community of software developers and enthusiasts around the world and gave birth to popular Linux distributions like Ubuntu, Redhat, Debian, Fedora, CentOS and many others.

The operating systems are only base building blocks of software. The next decade saw a tough rivalry between two technology stacks- free software and proprietary software. With the rise of the Internet, the ecosystem of programming languages, web servers, databases, standards and other related technologies began to grow together in both spectrums (Proprietary as well as Free Software). In the enterprise sector, while Microsoft continued with its technology stack proprietary segment with .NET framework, SQL server and windows server, the free software counterparts included Java/J2EE and Apache technology stack and free databases like MySQL and Postgres. With the advent of the cloud era, where softwares, applications and computing were packaged as a service, the difference between these two technology stacks gradually became irrelevant.

It is interesting to see who stands where at the moment. Microsoft continues to command a significant presence in the OS market, maintaining over 75% global market share in the desktop/laptop sector. But the enterprise applications and technology sector is hugely dominated by open source software. A staggering 90% of software businesses leverage open-source solutions to varying degrees, as reported by Digital Information World.

Over the last decade, the mobile application industry has witnessed unprecedented growth in its user base. The proprietary software iOS, developed by Apple and open source Android are the key operating systems in the race. Android tends to have a more substantial presence in emerging markets and have 70% global market share.

Fast forward to 2023, Generative AI

In the past couple of years, Artificial Intelligence has risen to prominence, becoming a transformative force across almost all domains. This rise in AI’s influence can be attributed to advancements in ML research, increased computing power, and the availability of big data. One of the major milestones happened in November 2022, when the OpenAI consortium unveiled ChatGPT, a cutting-edge language model that took the world by storm. It was designed as a conversational chatbot model which can understand text inputs and produce human-like responses. The OpenAI consortium initially had non-profit beginnings, with backing from the likes of Elon Musk and Infosys soon changed its stance after gauging its potential. After its remarkable success, Microsoft started heavily investing in OpenAI, planning to use these Large Language Model (LLM) foundations in its software products like Bing, Edge, Microsoft 365 and others. Search giant Google, then came up with its own version of generative AI called Bard, which boasts of responses based on real time data unlike ChatGPT.

One can only say that the competition has just started. While ChatGPT has already reaped the rewards for being first in the market, Google, equipped with its superior data search engines and vast data, is expected to catch up swiftly. But one interesting thing to take note is that both are proprietary.

Enter Llama 2, the Game changer!

When Microsoft and Google made heavy inroads, the other tech giant Meta (former Facebook) launched its own LLM in February 2023. But unlike other two models Meta has released its model under a non-commercial licence focussed only on research use cases. They pledged themselves with responsible AI practices by focussing on known issues like bias, toxicity and misinformation.

On July 18th, 2023 Meta came up with a later version, Llama 2, by making the model free and open even for commercial use. Meta’s decision to embrace the open-source philosophy can be likened to Linus Torvalds’ pivotal move when he declared his Linux operating system under the GNU license. Just as Torvalds released Linux to the world as a open-source software, inviting collaboration and encouraging widespread adoption of the operating system, Meta made its language model, Llama 2, freely available for free research and commercial use. Currently the latest updated model, Llama 2 can be downloaded from Meta, AWS, Azure and HuggingFace.

Llama-V2 is getting widespread acceptance from technologists, researchers and academia across the world for its open source approach. Following is a statement signed by hundreds of leading personalities in the field.

“We support an open innovation approach to AI. Responsible and open innovation gives us all a stake in the AI development process, bringing visibility, scrutiny and trust to these technologies. Opening today’s Llama models will let everyone benefit from this technology.” Read the full statement here.

While AI opens up multiple dimensions in business and technology, it also creates a lot of ethical and social challenges. There are a lot of questions that need to be addressed related to inherent bias, fairness, malicious usage, misinformation and privacy. While making the model public, one can argue that Llama’s claim for “Responsible AI” is one step ahead. Its transparency allows one to scrutinise the datasets for bias and fairness. But this can also be a double-edged-sword, as it is unpredictable when technology reaches malicious hands. But the open-source model has been there for quite some time, survived and thrived.

This is huge: Llama-v2 is open source, with a license that authorizes commercial use!

This is going to change the landscape of the LLM market.
Llama-v2 is available on Microsoft Azure and will be available on AWS, Hugging Face and other providers

Pretrained and fine-tuned…

— Yann LeCun (@ylecun) July 18, 2023

Awesome to see @Meta release Llama2's weights with a permissive license for commercial use: https://t.co/gSV2hgO4yT

They've been increasing their footprint in open-source AI a lot recently with over 600 models on Hugging Face like MusicGen, Galactica, Wav2Vec and many others.… pic.twitter.com/qGZdLqobo3

— clem 🤗 (@ClementDelangue) July 18, 2023

The business scenarios are swiftly changing. The ChatGPT large language model APIs were the toast of technologists across the world until a few days ago. Let me suggest one example from my own experience. agentrunner.ai, a leading AI startup based in London, which primarily focuses on creating autonomous AI agents was largely focussing on OpenAI LLMs till a few weeks ago. It has adapted quickly to the Llama 2-based model which offers better flexibility. Many companies are now following this trend. Why would small or mid-sized startups invest millions in procuring proprietary APIs?

Meta’s decision to embrace Responsible AI and open-source philosophy with Llama 2 is undoubtedly commendable and has won widespread praise. It also remains to be seen how Meta’s move will benefit their own ecosystem of software platforms, including Facebook, WhatsApp, Threads, and Instagram, in the long run.

It will be intriguing to watch how OpenAI, Microsoft and Google will respond to this challenge. Remember, Microsoft already declared its support for Llama-V2, probably because it has recognised the extreme complexity of ethical and social challenges LLM technology. Collaborating or working in a hybrid mode in this endeavour may prove more effective than facing it alone.

References:

1. Meta Newsletter on LLaMa 2
2. Wikipedia
3. LLaMa 2 Announcement
4. Hugging Face & LLaMa 2 Integration
5. Desktop OS Market Share
6. Open Source Software Market share
   

   ---