Skip to main content

​
Atoti Intelligence Essentials

This is part of the Atoti Intelligence Essentials offer.
A single question asked in Visualize This can turn into several calls to the LLM, and a slow model, an unreachable provider or a long tool call can keep one question running far longer than the person who asked it is willing to wait. Six guards bound that. All have a default, and all are overridable.

​
The timeout guard

atoti.ai.context.prompt-timeout bounds a whole prompt, whatever the provider does underneath it. When it expires, the prompt is abandoned, the run reports a timeout, and the partial answer is dropped from the conversation history so the next prompt does not build on an answer nobody saw.
The deadline is checked wherever a cancellation is: before every tool call, and at every round of the tool-calling loop, before the model is asked again. A model looping over a set of tools is therefore stopped at the next round, whichever tools it chose. It does not interrupt an HTTP request already in flight, so the effective ceiling is the deadline plus one provider call, which the transport timeouts bound.

​
The LLM retry guard

The provider’s own client re-sends a failed request rather than reporting it immediately. It is the right place for that decision: it backs off between attempts, and it does not retry a request it already knows is hopeless, such as an authentication failure or a malformed request. What it is not is uniform: each provider reads a different property, and counts differently. atoti.ai.max-attempts is the one to reach for. It is a number of attempts, the first one included, and Atoti translates it into whatever the provider serving the call reads:
Declaring atoti.ai.max-attempts wins over the spring.ai.* counts above, wherever those are declared — application.yml, an environment variable, a command-line argument. Leaving it out hands the decision back: a spring.ai.* count an application declares then stands, so a deployment already tuned per provider keeps its tuning. With neither declared, the default of 5 attempts applies to every provider, over the 4 of the OpenAI SDK and the 10 of Spring AI.
Amazon Bedrock has no spring.ai.* count of its own: it retries through the AWS SDK rather than through Spring AI, and the SDK’s own default of 3 attempts is out of reach of any property — AWS_MAX_ATTEMPTS reaches it, but process-wide. To make that number reachable, Atoti supplies the two Bedrock runtime clients that Spring AI would otherwise build itself, carrying every spring.ai.bedrock.aws.* timeout over unchanged. An application that declares a BedrockRuntimeClient or BedrockRuntimeAsyncClient bean of its own keeps it, and is then responsible for that client’s retry policy. So does one that excludes Spring AI’s Bedrock auto-configuration: Atoti’s clients stand down with it, and the SDK’s own default of 3 attempts applies again. Whichever provider is configured, the timeout guard bounds the retries too: they all happen inside one prompt.

​
Retrying the whole prompt

Above the provider sits the chat’s own loop, atoti.ai.context.max-retry-attempts. It defaults to 1, so the prompt is sent once and the retrying is left to the provider, which backs off between attempts and does not retry a request it already knows is hopeless — neither of which this loop can do.
Raise it only for failures that happen outside the provider call, such as a malformed tool call coming back from the model. Each extra attempt re-sends the whole prompt and re-runs its tools, and every attempt draws on the same tool-error budget described below: an attempt that spent it does not get it back by being run again.

​
The no-progress guards

Two guards stop a prompt that is getting nowhere, both by answering the model rather than running the tool: it reads why it was stopped and writes the explanation the user sees.

​
Repeated tool calls

A tool called with the very same arguments, back to back, returns the very same result, so calling it again makes no progress. Past atoti.ai.context.max-consecutive-identical-tool-calls times in a row, the call is not run again: the model is told it was just made and returned the same thing, and it does something else instead.
This is what bounds a model stuck on a failing tool. A prompt instructing it to “retry until it works” would otherwise repeat the call until the timeout guard expires, which is hundreds of model calls to reach a conclusion that was already available on the second one. Only a run of identical calls is caught. Any round without that call starts a fresh run, so a tool the model comes back to after doing something else keeps running, however many times. So does the same tool called with different arguments, which is different work. Each call is counted on its own, so a model asking for several tools at once is caught on the one it is stuck on and keeps the rest. That round still runs: only the stuck call is refused, and the others return their results as usual — but it is charged one tool error, so a model that keeps pairing the call it is stuck on with a fresh one is still brought to a stop.

​
Tool errors

A model varying its arguments every round never repeats itself, so the guard above never sees it. What it does collect is failures. Once atoti.ai.context.max-tool-errors tool calls have failed in one prompt, no further tool call is run: the model is told to stop and answer the user with what it has.
Every failure counts, whether it came from a tool running in Atoti Server or from one running in Atoti UI that the browser answered with an error. A round in which a call was refused counts too, once for the round however many of its calls were refused: a refusal is work the model asked for and did not get. The budget covers the whole prompt, the max-retry-attempts retries included. An attempt that spent it does not get it back by being run again.

​
The last resort

Refusing a call answers the model, which then decides what to do next. A model may simply ask for the same call again. Because each of those rounds is one more call to the model, a run in which atoti.ai.context.max-consecutive-refused-rounds rounds in a row ran no tool at all is ended outright, and the user is told the question could not be answered.
Reaching it means the model ignored that many explanations of why it was getting nowhere; the server log names the tool and the arguments it was stuck on. Any round that ran a tool starts the count over, so this only fires on a model that is doing nothing else.
All five atoti.ai.context properties above are validated at startup: prompt-timeout must not be negative and accepts zero to mean no deadline; max-retry-attempts, max-consecutive-identical-tool-calls, max-tool-errors and max-consecutive-refused-rounds must each be at least 1. A value outside those ranges stops the application from starting rather than surfacing on someone’s first question.

​
Transport timeouts

The transport timeouts sit below every guard above, and bound the retries as well as the first attempt. For Amazon Bedrock they are the spring.ai.bedrock.aws.* properties, timeout above all: it caps the whole call including its retries. See Set up Amazon Bedrock in Java. For OpenAI it is spring.ai.openai.timeout, 60 seconds by default, capping the same call the same way.

​
Atoti Python SDK

The atoti.ai.context guards live in PromptConfig because they bound one chat prompt. AiConfig.max_attempts sits outside it, next to the connection rather than inside it, because it applies to every call to whichever provider the session connects to.

​
Auto-Explain

Every atoti.ai.context guard above bounds a chat prompt, and only a chat prompt. Auto-Explain calls the LLM directly, without the tool-calling loop those guards sit in, so what bounds it is the provider layer: the LLM retry guard and the transport timeouts.
Under Amazon Bedrock, atoti.ai.max-attempts needs the chat on the classpath — the starter-ai-chat Spring Boot starter, or an Atoti Python SDK session — since that is what brings in Atoti’s Bedrock clients. A Java application that declares only starter-ai-autoexplain builds none of them, so the AWS SDK’s own default of 3 attempts applies there; cap it with spring.ai.bedrock.aws.timeout instead, which bounds the whole call including its retries. The other providers read the property wherever it is set.