Large Language Models to Large Action Models - Step towards Artificial General Intelligence

    It’s almost end of January, and I am still getting Happy New Year messages on WhatsApp and I still feel that I haven’t yet replied to many of the greetings I have received from friends and well-wishers in last 3 weeks. While I was trying to reply to some of those messages, I got a notification about a LinkedIn post by a friend who wrote about Large Action Models (LAMs). The post made me read more about LAMs as it was something new compared to Large Language Models (LLMs), that most of us know about and are also using regularly – ChatGPT and Bard and other models which can help generate text responses based on the prompts given. As against this, Large Action Models go a step further – they enable actions – and complete the task that is assigned or asked for. An example can be booking a flight ticket or even booking a complete vacation which may include flight tickets, hotel bookings, meals etc. 

Large Action Models (LAMs), an evolution of Large Language Models (LLMs), represent a significant development in artificial intelligence (AI). Unlike LLMs, which generate text based on predictions, LAMs act as autonomous 'agents' capable of performing tasks and decision-making. These models, tailored for specific applications and human actions, employ neuro-symbolic programming to replicate a variety of tasks seamlessly, eliminating the need for initial demonstrations. LAMs interact with the real world through integration with external systems, such as IoT devices, enabling them to perform physical actions, control devices, retrieve data, and manipulate information. Their capabilities include understanding complex human goals expressed in natural language, adapting to changing circumstances, and collaborating with other LAMs. Notable use cases span healthcare, finance, and automotive sectors, where LAMs can enhance diagnostics, risk measurement, and produce self-governing vehicles. 

LAMs became a buzz word when a company called Rabbit introduced their LAM device R1 at the recently held CES 2024. They launched Rabbit R1 – priced at $ 199 – as a small palm size device that could do simple tasks – better than a smartphone by enhancing one’s digital experience. The Rabbit r1 device, powered by Rabbit OS and a LAM acts as an AI assistant, capturing photos, videos, and interacting with users naturally. Rabbit R1 can edit your spreadsheets, save them and mail them to people you ask it to. It can also manage your social media by creating posts that you want and sharing them across social media platforms. LAMs are poised to play a pivotal role in the future of AI, transforming language models into real-time action companions. Real-world applications like Rabbit demonstrate the potential of LAMs in revolutionizing user interaction and shaping the landscape of AI. 

Large Action Models (LAMs) differ from Large Language Models (LLMs) in their capabilities and functionalities: 

Task Execution

LLMs are primarily focused on generating human-like text based on input data. They excel in natural language understanding and text generation but do not inherently perform tasks or actions. LAMs, on the other hand, are designed to go beyond text generation. They act as autonomous agents capable of executing tasks, making decisions, and interacting with the real world. 

Autonomy

LLMs generate text responses based on patterns learned during training but lack the autonomy to perform actions or make decisions beyond text generation. LAMs have the ability to act autonomously. They can connect to external systems, control devices, retrieve data, and manipulate information, allowing them to perform complex tasks without human intervention. 

Integration with External Systems

LLMs typically operate within a closed system and do not have direct integration with external systems or devices. LAMs interact with the real world by integrating with external systems, such as IoT devices. This enables them to perform physical actions and engage with the environment in a way that goes beyond text-based interactions. 

Goal-Oriented Interaction

LLMs are primarily focused on generating coherent and contextually relevant text based on input prompts but lack a goal-oriented approach to tasks. LAMs are designed to understand complex human goals expressed in natural language and translate them into actionable steps. They can respond in real-time and adapt to changing circumstances. 

Applications

LLMs are commonly used for natural language understanding, text generation, and various language-related tasks. As against this, LAMs are applied in diverse domains such as healthcare, finance, automotive, and more, where their ability to perform tasks has practical applications. For example, LAMs can aid in diagnostics, risk measurement, and even operate self-driving vehicles. 

In essence, while LLMs excel in language-related tasks and text generation, LAMs extend these capabilities by combining language fluency with the capacity to autonomously execute tasks and make decisions, representing a significant advancement in the field of artificial intelligence. 

It’s interesting to look at the potential use cases of LAMs. Some examples include: 

1. Healthcare: 

• Diagnostics: LAMs can analyse medical data, including imaging scans, to assist in diagnosing diseases. 
• Treatment Strategy: LAMs can recommend personalized treatment plans based on patient data and medical knowledge.

 2. Finance:

• Risk Measurement: LAMs can assess and analyze financial risks, providing insights for investment decisions. 
• Fraud Detection: LAMs can identify patterns indicative of fraudulent activities in financial transactions.

 3. Automotive: 

• Self-Driving Vehicles: LAMs can control and navigate autonomous vehicles, making real-time decisions based on environmental data. 
• Vehicle Safety Systems: LAMs can enhance safety features by processing data from sensors and taking preventive actions. 

 4. Education: 

• Personalized Learning: LAMs can tailor educational content and strategies based on individual student performance and needs. 
• Language Translation: LAMs can assist in translating educational materials into various languages.

 5. Customer Service: 

• Automated Support: LAMs can provide automated customer support by understanding and responding to user queries. 
• Issue Resolution: LAMs can troubleshoot problems and guide users through issue resolution processes.

 6. Home Automation: 

• Smart Home Control: LAMs can control smart home devices, adjusting temperature, lighting, and security systems based on user preferences. 
• Virtual Assistants: LAMs can act as intelligent virtual assistants, performing tasks such as setting reminders, sending messages, and managing schedules. 

 7. Manufacturing: 

• Quality Control: LAMs can analyze visual data from manufacturing processes to identify defects and ensure product quality. 
• Supply Chain Optimization: LAMs can optimize supply chain processes by analyzing data and making recommendations for efficiency. 

 8. Research and Development: 

• Data Analysis: LAMs can process large datasets, extract meaningful insights, and assist in research endeavors. 
• Innovation Support: LAMs can generate ideas and suggestions for innovation based on input criteria. 

 9. Entertainment: 

• Content Creation: LAMs can assist in generating creative content, including writing, art, and music.
• Interactive Storytelling: LAMs can create dynamic and interactive storytelling experiences. 

These examples showcase the versatility of LAMs in performing tasks that range from complex decision-making in healthcare to enhancing daily activities in smart homes. The ability to understand natural language and autonomously execute actions makes LAMs valuable across numerous applications and industries.

So finally it seems I can train a LAM to learn how to reply to Happy New Year greetings on WhatsApp or delete messages from a few nagging individuals on a group chat!!

Building Safety, Security, Resilience, and Trust in the Digital Economy

  Building Safety, Security, Resilience, and Trust in the Digital Economy 

(As published in Yojana, November 2023 Issue)

 India’s G20 presidency has set the agenda for Digital Transformation – DPIs, Cyber Security & Digital Skilling 

G20 and Digital Economy 

G20, the largest and the most impactful economic group in the world was formed in 1999 in response to the Asian financial crisis of 1997-98 and to create global financial stability. Since its origin, it has expanded its agenda to, inter-alia, including digital economy, trade, sustainable development, health, agriculture, energy, environment, climate change, and anti-corruption. The Digital Economy is one of the most important agenda items of G20. As per the World Bank, the digital economy contributes to more than 15% of the global Gross Domestic Product (GDP), and in the last ten years, it has been growing at two and half times faster than the physical world GDP. 

In the words of Hon’ble Prime Minister Narendra Modi, at the Digital Economy Ministerial Meeting in Bengaluru on 19th August, “As the digital economy spreads globally, it will face security threats and challenges. In this context, it is important to build consensus on the G20 high-level principles for a secure, trusted, and resilient digital economy. We in G20 have a unique opportunity to lay the foundation for an inclusive, prosperous and secure global digital future.”

 Digital Economy Working Group 

The Digital Economy Working Group had extensive discussions on the 3 key pillars of Digital Public Infrastructure, Security in the Digital Economy, and Digital Skilling. After extensive negotiations in Lucknow, Hyderabad, Pune, and Bengaluru, the Ministerial Meeting adopted the Outcome document which was unanimously agreed to by all members except for the geopolitical issues – that had no direct correlation with Digital Economy – and the issue finally was resolved at the G20 Delhi Summit in September 2023 – a stupendous effort by Indian G20 negotiating team led by G20 Sherpa Amitabh Kant. The Digital Economy Outcome Document outlines the details of the consensus that emerged on the 3 key issues. These can be summarised as below: 

Digital Public Infrastructure (DPI) 

 One key achievement of G20 was to come up with a definition of DPIs – Digital Public Infrastructure is described as a set of shared digital systems that should be secure and interoperable, and can be built on open standards and specifications to deliver and provide equitable access to public and/or private services at societal scale and are governed by applicable legal frameworks and enabling rules to drive development, inclusion, innovation, trust, and competition and respect human rights and fundamental freedoms. DPI was seen as a promising approach to digital transformation by providing a shared technology infrastructure that can be built and leveraged by both the public and private sectors. It was also agreed upon that governance frameworks and institutional capabilities play a very important role in ensuring that DPI is safe, secure, trusted, accountable, and inclusive and can contribute immensely to achieving the Sustainable Development Goals (SDGs). One of the most important outcomes of G20 was the agreement reached about the G20 Framework for Systems of Digital Public Infrastructure that can lay the roadmap for the development and deployment of DPIs in Low and Middle-Income Countries (LMICs). The contribution made by India Stack, especially, Aadhaar, UPI & Digilocker in India’s Digital Transformation Story was in a way acknowledged when the G20 outcome document mentioned the significance of privacy and data protection in accordance with legal frameworks and highlighted key DPI elements like digital identity, digital payment systems, and data-sharing mechanisms for secure identification, reliable payments, and seamless data exchange across various sectors. India’s plan to build and maintain a Global Digital Public Infrastructure Repository (GDIPR) as a virtual repository of DPI, voluntarily shared by G20 members and other nations, can be a big enabler for the adoption of DPIs is expected to be a key action item post-G20. This accompanied by the proposal of One Future Alliance (OFA) can help build capacity and provide technical assistance and funding support for implementing DPIs in LMICs. 

Security in the Digital Economy 

Building Safety, Security, Resilience and Trust in the Digital Economy has emerged as one of the key priorities of all G20 members that can ensure an enabling, inclusive, open, fair, non-discriminatory and secure digital economy. This is essential for continuing the momentum in the growth of the digital economy while ensuring that the key principles of promoting safety, trust, reliability, resilience and protecting privacy and data remain a top priority and key focus area. Trust and security as an essential prerequisite for harnessing the potential of the digital economy have been on top of the agenda right from the 2017 German G20 Presidency and these values have been reiterated in 2018, 2019, and 2020 under the Argentine, Japanese, and Saudi Arabian G20 Presidencies, respectively. The 2020 Saudi Arabian G20 Presidency and 2021 Italian G20 Presidency recognised that security in the digital economy is a key enabling factor for sustainable development and growth. Deliberations in India’s G20 Presidency further emphasised the importance of safety, security, resilience, and trust in the digital economy for digital transformation. In today’s interconnected world, there are digital dependencies across sectors and borders, that can create security risks associated with the digital economy that a single entity may not be capable of addressing alone. The digital economy has multiple layers, and, therefore, there is a risk that breaches or incidents at any layer may disrupt the functioning of the whole ecosystem. Due to the borderless nature of the digital environment, the global community needs to work together towards building a safe, secure, and resilient digital economy. Preventing and mitigating security threats to the digital economy requires augmenting the capacity of all key stakeholders to understand, anticipate, prepare for, and respond to these threats. To operationalise this approach of addressing shared security risks and challenges, the non-binding G20 High-Level Principles to Support Businesses in Building Safety, Security, Resilience, and Trust in the Digital Economy will be a key enabler. These principles have been developed from the best practices, strategies and tools developed and implemented by G20 members. These principles seek to strengthen resilience in the digital economy by promoting a culture of security, capacity building, multi-stakeholder cooperation and supporting research and development. 

The G20 High-Level Principles to Support Businesses in Building Safety, Security, Resilience, and Trust in the Digital Economy can be summarised as under: 

1. Security and Trust 

        A human-centric culture of security and trust in the digital economy that enables citizens and businesses to understand risk management can be developed by

     • Promoting cyber hygiene and the development of market-led and industry-led standards based on the principles of openness, transparency, and consensus. 

     • Encouraging businesses and supporting MSMEs to develop and implement good practices and risk management frameworks to maintain the integrity of global supply chains. 

     • Promoting a ‘security by design’ and phased risk management approach along with encryption measures for digital solutions and services, including in emerging technologies and connected systems and their devices. 

     • Promoting resilience in connected sectors such as health, finance, manufacturing, and public services and utilities by taking suitable security measures. 

     • Encouraging accessible and efficient grievance redressal mechanisms for businesses, MSMEs, and consumers that fall victim to malicious use of digital technologies. 

2. Capacity Building 

Capacity building is an important aspect of advancing security across the multi-layered structure of the digital economy and should include 

    • Collaborating with and encouraging relevant stakeholders, including international organisations, to prioritise and contribute to capacity building within their areas of expertise. 

     • Exploring an interdisciplinary approach that includes strategy, governance, technology, regulatory and non-regulatory frameworks, culture, economics, incident response and crisis management. 

     • Providing guidance and awareness to citizens, businesses including MSMEs, and the wider economy on how to stay safe and secure online in an inclusive and accessible manner. 

     • Promoting lifelong learning opportunities for all users of digital technologies. 

     • Encouraging young people especially women and girls to consider a career in security of digital solutions and services through curricular or extracurricular programs. 

3. Research and Development 

Advancing research and development enables building resilience by 

    • Promoting research in advanced and emerging technologies that can enhance protection against security threats.

     • Sharing best practices on how to tackle various security threats, including recommendations from international organisations. 

     • Facilitating research projects on topics such as the economic costs of security incidents and their impact on businesses and underrepresented communities. 

     • Promoting studies to measure security-related digital divides and its impact on economies. 

4. Multistakeholder Cooperation 

Partnering with businesses, civil society organisations, academia, international organisations and the technical community is key to promoting security in the digital economy and this can be reinforced by 

    • Developing opportunities for public-private partnership collaboration and engagement. 

    • Supporting the sharing of trends on known and existing vulnerabilities faced by nongovernmental stakeholders in the digital environment. 

    • Facilitating engagement between businesses and points of contact across various industry incident response teams. 

5. Strengthening Resilience of Essential Services 

Prevention of damage or disruption to certain essential social and economic services in the digital economy is important and it is essential that stakeholders are encouraged to: 

    • Take suitable measures to protect services essential to the digital economy from security threats. 

     • Encourage businesses to set up mechanisms to assess the security of their supply chains for essential services in an evidence-based approach. 

6. Support for MSMEs in the Security Ecosystem 

MSMEs have a very important role to play in the digital economy and the MSME security ecosystem can be strengthened by: 

    • Driving innovation by supporting MSMEs that offer security solutions and services to scale up and grow. 

    • Providing guidance and support to MSMEs on how to operate securely in a digital environment. 

    • Creating opportunities for MSMEs to engage with governments, shape policy approaches and share good practices to improve resilience to combat particular security challenges. 

    • Seeking to mobilise additional cooperation, funding, and support for MSMEs to improve their security capacity. 

Another key aspect of the potential risks associated with the digital economy relates to the impact they have on vulnerable sections of society – in particular children and youth. On one hand, increased access to digital tools and services opens up new avenues for children and the youth to explore their creativity, enhance their learning experience, and work collaboratively, at the same time, it also increases exposure to risks to which children are especially vulnerable, such as cyberbullying and grooming, and child sexual abuse and exploitation as well as risks related to their data and privacy. Women and girls are disproportionately affected by technology-facilitated gender-based violence. Cyber education and cyber awareness for the protection and empowerment of children and youth in today’s digital age is a key priority area and requires concerted action by all stakeholders. The G20 Toolkit on Cyber Education and Cyber Awareness of Children and Youth developed under the Indian presidency will serve as a guide for all countries. It recognises the important role of the UN Convention on Rights of the Child and the need to develop holistic, human-centric approaches to address online safety across different jurisdictions which promote respect for and facilitate governments’ efforts to protect children's privacy and personal data, uphold children’s dignity, and respect their rights. This coordinated approach to ensuring the safety of women, children and youth will go a long way in building safety, trust and resilience in the digital economy. 

This approach adopted by the G20 member countries under India’s Presidency lays down the roadmap for our common Digital Future – a strategy that not only lays emphasis on the adoption of Digital Public Infrastructure for enabling better access to public services to all and empowering people and enabling a robust Digital Economy but it also addresses the key issues and challenges relating to Safety, Security and Trust. This approach to Digital Transformation will truly help realise the goal of Vasudhaiva Kutumbakam – One Earth, One Family, One Future – digitally bringing the world together.