{"id":27344,"date":"2024-08-19T12:00:00","date_gmt":"2024-08-19T10:00:00","guid":{"rendered":"https:\/\/salvadorvilalta.com\/?p=27344"},"modified":"2024-08-10T18:43:36","modified_gmt":"2024-08-10T16:43:36","slug":"ai-optimization-how-intelligent-routing-and-moa-are-changing-the-rules","status":"publish","type":"post","link":"https:\/\/salvadorvilalta.com\/en\/ai-optimization-how-intelligent-routing-and-moa-are-changing-the-rules\/","title":{"rendered":"AI Optimization: How Intelligent Routing and MoA Are Changing the Rules"},"content":{"rendered":"\t\t
\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Artificial Intelligence (AI) is constantly evolving, and one of the fields that has advanced most recently is large language models (LLMs). <\/p>\n

However, the current focus on using a single advanced model for all tasks presents several challenges and limitations for developers and enterprises.
\nIn this article, we will explore how different solutions such as intelligent routing (Route LLM)<\/b> of queries and the introduction of technologies such as Mixture of Agents (MoA)<\/b> and Chain of Thought (CoT) <\/b>can offer a more efficient, flexible, cost-effective and secure solution. <\/p>\n

Don’t miss this video in which Matthew Berman explains the concept of Intelligent Routing.
\nIt makes all the sense in the world. <\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t

\n\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t
<\/div>\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t
\n\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Let's analyze the problem<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t
\n\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Currently, many developers and enterprises rely on a single language model, such as OpenAI’s GPT-4, for all their AI needs.
\nWhile these models are incredibly powerful, their indiscriminate use can result in several problems: <\/p>\n

    \n
  1. Overpaying:<\/b> They are overpaying for advanced capabilities that are not always necessary.
    \nFor many everyday tasks, a simpler model or a local system could meet the requirements without incurring high costs. <\/span><\/li>\n
  2. Platform Risk: <\/b>Dependence on a single vendor exposes companies to significant risks.
    \nIf the supplier decides to change its policies, pricing or accessibility, this can negatively impact operations. <\/span><\/li>\n
  3. Suboptimal Efficiency:<\/b> Most requests do not require the full power of an advanced LLM, resulting in unnecessary latency and inefficient use of resources.<\/li>\n<\/ol>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t
    \n\t\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t

    The Solution: Intelligent Routing and Abstraction Layer.<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t
    \n\t\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t

    A promising solution to these problems is the creation of an abstraction layer that allows intelligent routing of queries between multiple language models.<\/b> This layer would act as an intermediary that evaluates each request and routes it to the most appropriate model based on several factors such as cost, speed and complexity of the task.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t

    \n\t\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"Enrutamiento\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t
    \n\t\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t

    Advantages of Intelligent Routing:<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t
    \n\t\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t

    The possible advantages of implementing such a system could include:<\/p>\n

      \n
    1. Cost and Latency Reduction: <\/b>By selecting the right model for each request, it is possible to minimize both response time and associated costs.
      \nSimple tasks can be handled by cheaper or local models, while more complex tasks are assigned to advanced LLMs. <\/span><\/li>\n
    2. Flexibility: <\/b>This system can be connected to a variety of models, from the most sophisticated to the smallest and most specialized local models, adapting to the specific needs of each situation.<\/span><\/li>\n
    3. Use of Advanced Algorithms:<\/b> By implementing advanced techniques such as “Chain of Thought” (CoT) and “Mixture of Agents” (MoA), the quality of the answers is improved and the process of selecting the appropriate model is optimized.<\/span><\/li>\n<\/ol>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t
      \n\t\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t\t

      Mixture of Agents (MoA): A Modular Approach<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t
      \n\t\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t\t\t\t\t\t

      Mixture of Agents (MoA) is a recent innovation that allows tasks to be distributed among different AI agents, each specializing in a specific task type.<\/p>

      Instead of relying on a single model for everything, an MoA can assign different query parts to different agents, thus optimizing efficiency and accuracy.<\/p>

      For example, a MoA might include one agent specialized in natural language processing, one in mathematical logic, and one in code generation.<\/p>

      By combining the capabilities of these agents, more accurate and efficient responses can be obtained..<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t

      \n\t\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t\t\t\t\t\t\t\t\t
      \n\t\t\t\t\t\t\t\t\t\t\"MoA\t\t\t\t\t\t\t\t\t\t\t
      Source: Together AI<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t
      \n\t\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t\t\t\t\t\t

      In the following table that you can consult in the Together AI paper that I will leave you in the links at the end of this post, you can check how the model would work. <\/span><\/p>\n

      From left to right, we see the tokens that enter the first PROMPT, served by 3 different agents and three layers that make up the complete model.
      \nAfter each interaction in each layer, the results of each agent are aggregated and passed to the next layer as context (similar to what happens with a RAG system). <\/span><\/p>\n

      In the third layer the whole result is agglutinated and the output of the model is delivered once it has passed through the three layers.<\/span><\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t

      \n\t\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t\t\t\t\t\t\t\t\t
      \n\t\t\t\t\t\t\t\t\t\t\"Mixture\t\t\t\t\t\t\t\t\t\t\t
      source: <\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t
      \n\t\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t\t\t\t\t\t

      As we can see in the following graph contained in Together AI’s paper, a MoA-based system is more efficient than any current LLM model.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t

      \n\t\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t\t\t\t\t\t\t\t\t
      \n\t\t\t\t\t\t\t\t\t\t\"MoA\t\t\t\t\t\t\t\t\t\t\t
      Fuente:: Togheter AI<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t
      \n\t\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t\t

      Chain of Thought (CoT): Improving the Quality of Answers<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t
      \n\t\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t\t\t\t\t\t

      Chain of Thought (CoT) <\/b> is a prompt engineering technique used in language models to improve their reasoning capabilities. <\/b>It consists of structuring instructions so that the model breaks down a complex problem into sequential logical steps, similar to how a human would think aloud.
      \nThis not only helps the model arrive at a more accurate final answer, but also provides a detailed explanation of how it arrived at that conclusion. <\/p>\n

      This technique is especially useful in tasks that require arithmetic, common sense and symbolic reasoning.
      \nBy guiding the model through intermediate steps, Chain of Thought improves the accuracy and interpretability of responses, without the need for additional training.<\/b> In the context of prompt engineering, CoT represents an effective strategy for taking full advantage of the capabilities of language models, especially large ones, by structuring instructions in a way that encourages deeper and more detailed reasoning. <\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t

      \n\t\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t\t\t\t
      \n\t\t\t
      <\/div>\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t
      \n\t\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t\t

      Computing on Local Devices: A Personalized Power<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t
      \n\t\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t\t\t\t\t\t

      Another important trend in AI is computing on local devices, such as laptops, desktops, and even cell phones.
      \nThanks to advances in hardware and software, it is possible to run AI models directly on these devices, which offers several advantages: <\/p>\n

        \n
      1. Privacy: User data can be processed locally, reducing the need to send information to external servers.<\/span><\/li>\n
      2. Accessibility: AI can be available at all times, even without an Internet connection.<\/span><\/li>\n
      3. Cost: Reducing dependence on cloud services lowers operating costs.<\/span><\/li>\n<\/ol>\n

        In addition to these advantages, computing on local devices has a significant impact in terms of security<\/b>.
        \nConsider the following scenario:. <\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t

        \n\t\t\t\t\t
        \n\t\t\t\t
        \n\t\t\t\t
        \n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"IA\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t
        \n\t\t\t\t\t
        \n\t\t\t\t
        \n\t\t\t\t
        \n\t\t\t\t\t\t\t\t\t

        Imagine a company employee using an open-ended LLM model to prepare an important proposal.\nTo do so, he or she enters confidential customer data as context for accurate and personalized responses.\nWhile the online model can provide the requested information, there is also the possibility that this data can be used to train the model itself. <\/span><\/p>\n

        What does this imply?\nWhenever an LLM model is used in the cloud, the data entered can be stored and reused to improve the performance of the model.\nThis means that confidential client information could be inadvertently being shared and used outside the organization.\nThis risk not only jeopardizes customer privacy, but also exposes the firm to potential security breaches and violations of data protection regulations, such as GDPR in Europe or CCPA in California. <\/p>\n

        By contrast, when using local models, all data is processed directly on the employee’s device, without the need to send it to the cloud.\nThis significantly reduces the risk of sensitive information being used inappropriately or falling into the wrong hands.\nLocal computing allows companies to maintain full control over their data, minimizing the security risks associated with the use of open LLM models. <\/p>\n

        This approach is especially useful for less demanding tasks that can be handled efficiently by local models, leaving more complex tasks for cloud models only when really necessary.<\/p>\n

        If you want to start testing AI with Opensource models on your own device don’t miss this video from DOTCSV.\nCarlos, as always, developing exceptional content. <\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t

        \n\t\t\t\t\t
        \n\t\t\t\t
        \n\t\t\t\t
        \n\t\t\t\t\t\t\t
        \n\t\t\t
        <\/div>\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t
        \n\t\t\t\t\t
        \n\t\t\t\t
        \n\t\t\t\t
        \n\t\t\t\t\t\t\t\t\t

        The future of AI is evolving towards a more intelligent, efficient and adaptive model, in which, toBy implementing these solutions, which we have discussed above, developers and companies can take full advantage of its capabilities, using advanced models only when necessary and relying on cheaper and more efficient solutions for the rest.
        \nThis approach brings us closer to a world where AI is not only powerful, but also accessible and adaptable to our specific needs. <\/span><\/p>\n

        Leave me your comments on how these innovations could influence your business or project.
        \nWould you consider implementing an intelligent routing system or MoA to optimize your processes?
        \nYour opinion is important! <\/p>\n

        Have a good week!<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t
        \n\t\t\t\t\t
        \n\t\t\t\t
        \n\t\t\t\t
        \n\t\t\t\t\t
        Interesting sources<\/h6>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t
        \n\t\t\t\t\t
        \n\t\t\t\t
        \n\t\t\t\t
        \n\t\t\t\t\t\t\t\t\t