Custom LLM Service for Natural Language to IR

Normally, when you create any LLM-enabled component in APEX, you can choose which LLM service it should use. The main exception is the new Natural Language to IR feature in Apex 26.1. This post shows how to work around that limitation.

But first, why would you want to do this? Usually because you want a less expensive model for this feature. The system prompt for NL2IR is massive and very restrictive, and a decent chunk of metadata and tool information is sent along with each request. When I looked at the logs, each NL2IR call contained around 17,000 input tokens and only about 100 output tokens. That is massive, maybe even too massive.

My LLM provider of choice is Anthropic, and as of May 2026, 17k tokens works out to about \(0.11 per request on Sonnet (the mid-tier model), while on Haiku (the small, fast model) it is about \)0.04. So no matter what you are doing, you probably want NL2IR glued to the cheapest model possible.

This post also came from a question by Matt Mulvaney And if an Oracle ACE Director asks, the only real answer is: let’s do it ;)

We will start by creating two AI services.

One named Dummy, which we will set as our default service. I am setting Maximum AI Tokens to 0 to show the solution working, but you can set it however you want.

And the second will be the one we actually want to use.

The choice of model does not really matter. What matters is the static ID of the service we want to use for NL2IR. For me, it is haiku-for-ir.

Now we need to configure the default service in the app. Go to Edit Application Definition of your selected app, click the AI tab, and select the Dummy service. This is required for the NL2IR feature to show up.

Let's go into the app and observe the behavior.

Okay, looks like Oracle may need an NVL when checking the limits, because the first call is partially working, but after that we get errors. This is what we expected for this demo, so now let's make it use the correct service.

First, create this procedure. If you are coming from my previous blog post, you can modify the contents of the package we created there. Remember to change haiku-for-ir to the static ID of your LLM service.

create or replace procedure ai_request (
    p_param  in            apex_ai.t_chat_request_handler_param,
   p_result in out nocopy apex_ai.t_chat_request_handler_result
) is
   l_test number;
   l_service_id number;
begin
   -- Check if the request is coming from IR.
   select count(*)
     into l_test
     from apex_application_page_ir
     where region_id = p_param.component.id;

   if l_test = 0 then
      return; -- Not an IR request, skip.
   end if;

   -- Get the service ID.
   select remote_server_id
      into l_service_id
      from apex_workspace_ai_services
     where upper(remote_server_static_id) = upper('haiku-for-ir')
     fetch first 1 rows only;

   -- Set the new service.
   p_result.request.service_id := l_service_id;
end;

Then enter the procedure name, ai_request, into the Request Handler Procedure field on the AI tab of Edit Application Definition.

And you are done.
If we go back to the app, we will see that even though our default service is Dummy, NL2IR works perfectly.