Large Language Models to Large Action Models - Step towards Artificial General Intelligence

    It’s almost end of January, and I am still getting Happy New Year messages on WhatsApp and I still feel that I haven’t yet replied to many of the greetings I have received from friends and well-wishers in last 3 weeks. While I was trying to reply to some of those messages, I got a notification about a LinkedIn post by a friend who wrote about Large Action Models (LAMs). The post made me read more about LAMs as it was something new compared to Large Language Models (LLMs), that most of us know about and are also using regularly – ChatGPT and Bard and other models which can help generate text responses based on the prompts given. As against this, Large Action Models go a step further – they enable actions – and complete the task that is assigned or asked for. An example can be booking a flight ticket or even booking a complete vacation which may include flight tickets, hotel bookings, meals etc. 

Large Action Models (LAMs), an evolution of Large Language Models (LLMs), represent a significant development in artificial intelligence (AI). Unlike LLMs, which generate text based on predictions, LAMs act as autonomous 'agents' capable of performing tasks and decision-making. These models, tailored for specific applications and human actions, employ neuro-symbolic programming to replicate a variety of tasks seamlessly, eliminating the need for initial demonstrations. LAMs interact with the real world through integration with external systems, such as IoT devices, enabling them to perform physical actions, control devices, retrieve data, and manipulate information. Their capabilities include understanding complex human goals expressed in natural language, adapting to changing circumstances, and collaborating with other LAMs. Notable use cases span healthcare, finance, and automotive sectors, where LAMs can enhance diagnostics, risk measurement, and produce self-governing vehicles. 

LAMs became a buzz word when a company called Rabbit introduced their LAM device R1 at the recently held CES 2024. They launched Rabbit R1 – priced at $ 199 – as a small palm size device that could do simple tasks – better than a smartphone by enhancing one’s digital experience. The Rabbit r1 device, powered by Rabbit OS and a LAM acts as an AI assistant, capturing photos, videos, and interacting with users naturally. Rabbit R1 can edit your spreadsheets, save them and mail them to people you ask it to. It can also manage your social media by creating posts that you want and sharing them across social media platforms. LAMs are poised to play a pivotal role in the future of AI, transforming language models into real-time action companions. Real-world applications like Rabbit demonstrate the potential of LAMs in revolutionizing user interaction and shaping the landscape of AI. 

Large Action Models (LAMs) differ from Large Language Models (LLMs) in their capabilities and functionalities: 

Task Execution

LLMs are primarily focused on generating human-like text based on input data. They excel in natural language understanding and text generation but do not inherently perform tasks or actions. LAMs, on the other hand, are designed to go beyond text generation. They act as autonomous agents capable of executing tasks, making decisions, and interacting with the real world. 

Autonomy

LLMs generate text responses based on patterns learned during training but lack the autonomy to perform actions or make decisions beyond text generation. LAMs have the ability to act autonomously. They can connect to external systems, control devices, retrieve data, and manipulate information, allowing them to perform complex tasks without human intervention. 

Integration with External Systems

LLMs typically operate within a closed system and do not have direct integration with external systems or devices. LAMs interact with the real world by integrating with external systems, such as IoT devices. This enables them to perform physical actions and engage with the environment in a way that goes beyond text-based interactions. 

Goal-Oriented Interaction

LLMs are primarily focused on generating coherent and contextually relevant text based on input prompts but lack a goal-oriented approach to tasks. LAMs are designed to understand complex human goals expressed in natural language and translate them into actionable steps. They can respond in real-time and adapt to changing circumstances. 

Applications

LLMs are commonly used for natural language understanding, text generation, and various language-related tasks. As against this, LAMs are applied in diverse domains such as healthcare, finance, automotive, and more, where their ability to perform tasks has practical applications. For example, LAMs can aid in diagnostics, risk measurement, and even operate self-driving vehicles. 

In essence, while LLMs excel in language-related tasks and text generation, LAMs extend these capabilities by combining language fluency with the capacity to autonomously execute tasks and make decisions, representing a significant advancement in the field of artificial intelligence. 

It’s interesting to look at the potential use cases of LAMs. Some examples include: 

1. Healthcare: 

• Diagnostics: LAMs can analyse medical data, including imaging scans, to assist in diagnosing diseases. 
• Treatment Strategy: LAMs can recommend personalized treatment plans based on patient data and medical knowledge.

 2. Finance:

• Risk Measurement: LAMs can assess and analyze financial risks, providing insights for investment decisions. 
• Fraud Detection: LAMs can identify patterns indicative of fraudulent activities in financial transactions.

 3. Automotive: 

• Self-Driving Vehicles: LAMs can control and navigate autonomous vehicles, making real-time decisions based on environmental data. 
• Vehicle Safety Systems: LAMs can enhance safety features by processing data from sensors and taking preventive actions. 

 4. Education: 

• Personalized Learning: LAMs can tailor educational content and strategies based on individual student performance and needs. 
• Language Translation: LAMs can assist in translating educational materials into various languages.

 5. Customer Service: 

• Automated Support: LAMs can provide automated customer support by understanding and responding to user queries. 
• Issue Resolution: LAMs can troubleshoot problems and guide users through issue resolution processes.

 6. Home Automation: 

• Smart Home Control: LAMs can control smart home devices, adjusting temperature, lighting, and security systems based on user preferences. 
• Virtual Assistants: LAMs can act as intelligent virtual assistants, performing tasks such as setting reminders, sending messages, and managing schedules. 

 7. Manufacturing: 

• Quality Control: LAMs can analyze visual data from manufacturing processes to identify defects and ensure product quality. 
• Supply Chain Optimization: LAMs can optimize supply chain processes by analyzing data and making recommendations for efficiency. 

 8. Research and Development: 

• Data Analysis: LAMs can process large datasets, extract meaningful insights, and assist in research endeavors. 
• Innovation Support: LAMs can generate ideas and suggestions for innovation based on input criteria. 

 9. Entertainment: 

• Content Creation: LAMs can assist in generating creative content, including writing, art, and music.
• Interactive Storytelling: LAMs can create dynamic and interactive storytelling experiences. 

These examples showcase the versatility of LAMs in performing tasks that range from complex decision-making in healthcare to enhancing daily activities in smart homes. The ability to understand natural language and autonomously execute actions makes LAMs valuable across numerous applications and industries.

So finally it seems I can train a LAM to learn how to reply to Happy New Year greetings on WhatsApp or delete messages from a few nagging individuals on a group chat!!

Make AI in India for the World

 With the advent of Generative AI and ChatGPT in particular, Artificial Intelligence (AI) has become a buzz word – seen as the next big disruptive technology that is transforming the way we live and work. However, AI has been around for long – the term ‘artificial intelligence’ (AI) was first used by computer scientist John McCarthy in 1956 at the Dartmouth Conference, where in McCarthy and his colleagues explored how machines could simulate humans and solve problems on their own. The recent advancements in the field has suddenly made AI a hot commodity that almost everyone wants to master and use. 

    With the exponential expansion in compute capabilities and availability of petabytes and exabytes of data, the capabilities of AI systems and algorithms have grown multi-fold – bringing in immense possibilities in addressing challenges in healthcare, climate change, agriculture, education, natural language processing amongst others. While AI has the potential to enable easier and affordable disease diagnosis, increase farm productivity, conserve water, mitigate risks of climate change, enable access to education for all; yet there are risks and challenges that we need to be aware of regarding misuse and abuse of AI that can result in bias and also cause user harm. The threat from misinformation and deep fakes to democracies across the world are real. Governments across the world are mulling over ways and means to walk the tight rope between innovation and regulation. 

    India has a lot at stake as we are amongst the top countries when it comes to AI trained workforce. Stanford AI Index 2023 has ranked India as the top nation in AI Skill penetration. The recent EY report on Generative AI estimates that GenAI can contribute upto 1.5 trillion dollars to our GDP by 2030. It is also estimated to create 5 million new jobs. Our robust AI StartUp ecosystem has attracted investments of more than $ 475 million in last two years. Implementation of Digital transformation projects across sectors has led to generation of large volumes of data that can be leveraged for building AI models for enhancing and augmenting services. 

    This year, we are also the lead chair of Global Partnership on Artificial Intelligence (GPAI) and we are at the forefront of driving the discourse around global policy framework for responsible and ethical AI. The New Delhi declaration adopted at the GPAI Ministerial meet in December 2023 focuses on the two new themes proposed by India – Sustainable Agriculture and Collaborative AI for Global Partnerships in addition to the existing GPAI themes of Global Health, Climate Change and Resilient Societies. GPAI along with its expert support centres at Montreal, Paris and Tokyo will take up projects under these thematic areas to help build AI solutions that can benefit the whole world. 

    The declaration also acknowledged concerns around misinformation and disinformation, lack of transparency and fairness, protection of intellectual property and personal data, and threats to human rights and democratic values with indiscrimate use and applications of AI. India’s endeavour to promote collaborative AI for global partnership amongst GPAI Members by supporting projects aimed at promoting equitable access to critical resources for AI research and innovation, such as AI compute, high-quality diverse datasets, algorithms, software, testbeds, and other AI-relevant resources in compliance with applicable intellectual property protections and data protection legislations was specially noted. 

    The GPAI declaration came in the background of G7 leaders’ statement on the Hiroshima AI Process, the Bletchley Declaration, and the G20 New Delhi Leaders’ Declaration, which all have focused on the need for the world to work together, in an inclusive manner, to promote trustworthy AI that supports the good of all. In late October, United States also published their executive order on safe, secure and trustworthy AI that called for new standards for safety and security with an objective to protect privacy and civil rights as also to promote innovation and competition. A few days back, the European Parliament enacted world’s first legislation on AI and laid down a risks and rights framework while encouraging innovation. It also called upon the developers of foundation models to disclose all details including training datasets to the Government. As against this, India’s approach to AI Governance has been to promote innovation while regulating misuse of AI. We see AI as a kinetic enabler and contributor to our digital economy. In line with our approach to Digital Public Infrastructure, we have been advocating leveraging AI for social good and more equitable adoption of AI solutions and applications, especially in the Global South. 

    India’s pitch for Collaborative approach to AI development and regulation was also echoed by the Interim report of the United Nations Advisory Body on AI that calls for a closer alignment between international norms and how AI is developed and rolled out in sovereign nations. The report proposed to strengthen international governance of AI by supporting international collaboration on data, computing capacity and talent to reach the Sustainable Development Goals (SDGs). The report also recommends enhancing accountability and ensuring an equitable voice for all countries. The core principles of Inclusivity, Public Interest, Data Governance and Multistakeholder approach have been emphasised in the Interim report. 

    Given the inflexion point the world is in with regard to both adoption as also regulation of AI, it is essential that we work on building foundation models in Indian languages as also ensure that we are able to navigate supply chain bottlenecks to get the AI Chips needed to build our AI compute infrastructure. This would require investments from both the public and private sector. In addition, given that Generative AI will impact low value jobs, we will need to strengthen our reskilling and upskilling efforts with projects like FutureSkills Prime to ensure that we retain our position as the talent hub for AI jobs. Our efforts in integrating Artificial Intelligence in school and university curriculum are models for the world to replicate. The Digital Personal Data Protection Act and the upcoming Data Governance framework with the India Datasets platform will ensure that we are able to harness the potential of data and use it to build AI solutions. This will help us realise our dream of ‘Make AI in India for the World’.