The more capable an AI agent becomes, the broader its range of functions typically becomes. While a specialized assistant might need only five or ten tools, enterprise agents quickly access hundreds of functions from ERP, CRM, document management, and analytics systems.
\nFor each of these functions, the large language model requires a description: What does the function do? What parameters does it expect? What data does it return?
\nIf all definitions are loaded in full into the model context for every query, token usage, costs, and latency all increase. At the same time, it becomes more difficult for the model to select the appropriate tool from a very large set.<\/p>\n
The main challenge, therefore, is no longer:<\/p>\n
How many features can an agent use?<\/strong><\/p>\n Rather:<\/p>\n How does he get exactly the features he needs for his current task at the right time?<\/strong><\/p>\n Tool Search applies the retrieval principle\u2014familiar from RAG systems\u2014to executable functions. OpenAI offers\u00a0tool_search\u00a0<\/em>a mechanism through which functions, namespaces, or MCP servers can initially be set aside and loaded only when needed. Related functions can, for example, be grouped into areas such as order management, customer service, or materials management. The model then searches specifically within the appropriate functional area. The Milliarum AI Construction Kit operates at the integration and application levels. The ABAP-based framework enables the development of AI-powered applications directly within the SAP environment. For agents with a wide range of functions, Milliarum provides a central customization setting: In a potential target architecture, OpenAI or Anthropic would handle language understanding and task planning. The Milliarum AI Construction Kit manages the SAP-specific functions and controls which of them are available in the AI context. Dynamic loading of functions reduces the context size and improves tool selection. However, for productive use in enterprises, additional issues must be addressed:Tool Search: Providing Features Based on Need<\/h2>\n
\nInstead of providing the language model with the complete catalog of functions for every request, only a few core functions or high-level functional areas are initially made available. If the agent determines that it needs an additional capability, it searches for suitable tools. Only the relevant function definitions are then loaded into the context.
\nThis keeps the context clearer, reduces token usage, and eliminates the need for the model to choose from hundreds of potentially irrelevant features.<\/p>\nOpenAI and Anthropic: Load Tools Only When Needed<\/h2>\n
\nAnthropic takes a similar approach. Claude can also search through extensive tool catalogs and incorporate into the context only the functions relevant to a specific task. The search can be controlled either by Anthropic or by the customer application\u2019s own search logic.
\nBoth approaches primarily solve the discovery problem: The agent does not need to know all the tools at all times, but only needs to be able to find the tools it needs.
\nHowever, another question remains for companies: How can this principle be integrated into the existing application landscape, regardless of the specific language model used?<\/p>\nMilliarum: Model-Agnostic Function Management for SAP<\/h2>\n
\nThrough Customizing, you can manage, among other things, model connections, system prompts, personas (agents), and the AI functions permitted for an application. SAP function modules, ABAP classes, methods, and other data sources can be made available as functions for function calling.
\nThis integration is not limited to a single model provider. The AI Construction Kit can be connected to APIs from OpenAI and Anthropic, among others. This ensures that the functionality and integration logic on the SAP side remain intact, even when different language models are used for different applications.<\/p>\nMaximum Number of Functions in an AI Context<\/h3>\n
\n\u201eMaximum AI capabilities that are retained in the AI context\u201c<\/strong>
\nThis setting determines how many function definitions are maintained simultaneously in the context of the language model. The configurable range of values is between\u00a01 and 99 functions<\/strong>.
\nThis means that a\u00a0LLM-model-agnostic option\u00a0<\/strong>to load the required functions into the context as needed and to limit the number of tools available at any one time.
\nControl is therefore not exclusively dependent on a proprietary tool-search function provided by a specific model vendor. It takes place within the integration layer of the AI Construction Kit and can be used regardless of whether the application works with OpenAI, Anthropic, or another supported model.
\nFor a specialized agent, for example, a low upper limit may be appropriate. An assistant tasked with reviewing customer orders may only need functions to read the order, determine the delivery status, check material availability, and display the business partner.
\nIn contrast, a cross-functional agent that also processes orders, projects, controlling data, and service transactions may require a larger number of functions that are available simultaneously.
\nThe appropriate setting depends, among other things, on the use case, the model being used, the available context window, and the complexity of the function descriptions.<\/p>\nInteraction Between the Language Model and the SAP Functional Layer<\/h2>\n
\nA typical process might look like this:
\nFor example, a user might phrase the task as follows:
\n\u201eReview the open customer orders, identify potential delivery delays, and propose a course of action.\u201c<\/strong>
\nThe language model recognizes that this requires information from the areas of sales orders, deliveries, and material availability.
\nThe AI Construction Kit provides the relevant SAP functions. The configured upper limit determines how many function definitions are retained simultaneously in the AI context.
\nThe model then generates the necessary function calls. The AI Construction Kit maps these calls to the authorized SAP functions, ABAP methods, or data sources and returns the results to the language model.
\nThe agent can evaluate the results, request additional functions as needed, and finally formulate a technical recommendation for action.<\/p>\nTool Search alone is not enough<\/h2>\n
\nWhat functions is an agent allowed to use? What user permissions apply? Is a function allowed to only read data, or can it also modify it? Does a write operation require approval? How are calls, parameters, and results logged?
\nIn the SAP environment in particular, the risks vary significantly. Interpreting a delivery date must be evaluated differently than modifying a sales order or posting a document.
\nA central functional layer should therefore not only control the technical discoverability of tools, but also their approval, execution, and traceability.<\/p>\nConclusion<\/b><\/h2>\n