> ## Documentation Index
> Fetch the complete documentation index at: https://docs.activeviam.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Retries and timeouts
> The six guards bounding what one Visualize This prompt can cost — a wall-clock deadline, caps on repeated tool calls, tool errors and rounds that run nothing, and caps on how many times the provider and the chat ask again — plus the transport timeouts underneath them.
### Atoti Intelligence Essentials
This is part of the Atoti Intelligence Essentials offer.
A single question asked in Visualize This can turn into several calls to the LLM, and a slow model,
an unreachable provider or a long tool call can keep one question running far longer than the
person who asked it is willing to wait.
Six guards bound that. All have a default, and all are overridable.
| Guard | What it bounds | Property | Default |
| --------------------- | ------------------------------------------------------------------- | ------------------------------------------------------- | ----------------------------- |
| Timeout | How long one prompt may run, everything below it included | `atoti.ai.context.prompt-timeout` | 5 minutes |
| LLM retries | How many times the provider re-sends one request | Per provider, see below | Per provider |
| Chat retries | How many times the chat re-sends the whole prompt | `atoti.ai.context.max-retry-attempts` | 1, the prompt being sent once |
| Repeated tool calls | How many times in a row one prompt repeats a tool call verbatim | `atoti.ai.context.max-consecutive-identical-tool-calls` | 3 |
| Tool errors | How many tool calls one prompt may see fail before it answers | `atoti.ai.context.max-tool-errors` | 5 |
| Rounds without a tool | How many rounds in a row may run no tool at all before the run ends | `atoti.ai.context.max-consecutive-refused-rounds` | 3 |
## The timeout guard
`atoti.ai.context.prompt-timeout` bounds a whole prompt, whatever the provider does underneath it.
When it expires, the prompt is abandoned, the run reports a timeout, and the partial answer is
dropped from the conversation history so the next prompt does not build on an answer nobody saw.
```yaml theme={"languages":{"custom":["/engine/python-sdk/6.2/languages/pycon.tmLanguage.json"]}}
atoti:
ai:
context:
prompt-timeout: 5m
```
The deadline is checked wherever a cancellation is: before every tool call, and at every round of
the tool-calling loop, before the model is asked again. A model looping over a set of tools is
therefore stopped at the next round, whichever tools it chose.
It does not interrupt an HTTP request already in flight, so the effective ceiling is the deadline
plus one provider call, which the transport timeouts bound.
## The LLM retry guard
The provider's own client re-sends a failed request rather than reporting it immediately. It is the
right place for that decision: it backs off between attempts, and it does not retry a request it
already knows is hopeless, such as an authentication failure or a malformed request. What it is not
is uniform: each provider reads a different property, and counts differently.
`atoti.ai.max-attempts` is the one to reach for. It is a number of attempts, the first one included,
and Atoti translates it into whatever the provider serving the call reads:
```yaml theme={"languages":{"custom":["/engine/python-sdk/6.2/languages/pycon.tmLanguage.json"]}}
atoti:
ai:
max-attempts: 5
```
| Property | Governs | Counts | Default |
| ------------------------------ | --------------------- | -------------------------------- | ------- |
| `atoti.ai.max-attempts` | every provider | attempts, retries included | 5 |
| `spring.ai.openai.max-retries` | OpenAI, Azure Foundry | retries, after the first attempt | 3 |
| `spring.ai.retry.max-attempts` | Google GenAI | attempts, retries included | 10 |
Declaring `atoti.ai.max-attempts` wins over the `spring.ai.*` counts above, wherever those are
declared — `application.yml`, an environment variable, a command-line argument. Leaving it out
hands the decision back: a `spring.ai.*` count an application declares then stands, so a deployment
already tuned per provider keeps its tuning. With neither declared, the default of 5 attempts
applies to every provider, over the 4 of the OpenAI SDK and the 10 of Spring AI.
Amazon Bedrock has no `spring.ai.*` count of its own: it retries through the AWS SDK rather than
through Spring AI, and the SDK's own default of 3 attempts is out of reach of any property —
`AWS_MAX_ATTEMPTS` reaches it, but process-wide. To make that number reachable, Atoti supplies the
two Bedrock runtime clients that Spring AI would otherwise build itself, carrying every
`spring.ai.bedrock.aws.*` timeout over unchanged. An application that declares a
`BedrockRuntimeClient` or `BedrockRuntimeAsyncClient` bean of its own keeps it, and is then
responsible for that client's retry policy. So does one that excludes Spring AI's Bedrock
auto-configuration: Atoti's clients stand down with it, and the SDK's own default of 3 attempts
applies again.
Whichever provider is configured, the timeout guard bounds the retries too: they all happen inside
one prompt.
### Retrying the whole prompt
Above the provider sits the chat's own loop, `atoti.ai.context.max-retry-attempts`. It defaults to
1, so the prompt is sent once and the retrying is left to the provider, which backs off between
attempts and does not retry a request it already knows is hopeless — neither of which this loop can
do.
```yaml theme={"languages":{"custom":["/engine/python-sdk/6.2/languages/pycon.tmLanguage.json"]}}
atoti:
ai:
context:
max-retry-attempts: 1
```
Raise it only for failures that happen outside the provider call, such as a malformed tool call
coming back from the model. Each extra attempt re-sends the whole prompt and re-runs its tools, and
every attempt draws on the same tool-error budget described below: an attempt that spent it does
not get it back by being run again.
## The no-progress guards
Two guards stop a prompt that is getting nowhere, both by answering the model rather than running
the tool: it reads why it was stopped and writes the explanation the user sees.
### Repeated tool calls
A tool called with the very same arguments, back to back, returns the very same result, so calling
it again makes no progress. Past `atoti.ai.context.max-consecutive-identical-tool-calls` times in a
row, the call is not run again: the model is told it was just made and returned the same thing, and
it does something else instead.
```yaml theme={"languages":{"custom":["/engine/python-sdk/6.2/languages/pycon.tmLanguage.json"]}}
atoti:
ai:
context:
max-consecutive-identical-tool-calls: 3
```
This is what bounds a model stuck on a failing tool. A prompt instructing it to "retry until it
works" would otherwise repeat the call until the timeout guard expires, which is hundreds of model
calls to reach a conclusion that was already available on the second one.
Only a run of identical calls is caught. Any round without that call starts a fresh run, so a tool
the model comes back to after doing something else keeps running, however many times. So does the
same tool called with different arguments, which is different work.
Each call is counted on its own, so a model asking for several tools at once is caught on the one it
is stuck on and keeps the rest. That round still runs: only the stuck call is refused, and the
others return their results as usual — but it is charged one tool error, so a model that keeps
pairing the call it is stuck on with a fresh one is still brought to a stop.
### Tool errors
A model varying its arguments every round never repeats itself, so the guard above never sees it.
What it does collect is failures. Once `atoti.ai.context.max-tool-errors` tool calls have failed in
one prompt, no further tool call is run: the model is told to stop and answer the user with what it
has.
```yaml theme={"languages":{"custom":["/engine/python-sdk/6.2/languages/pycon.tmLanguage.json"]}}
atoti:
ai:
context:
max-tool-errors: 5
```
Every failure counts, whether it came from a tool running in Atoti Server or from one running in
Atoti UI that the browser answered with an error. A round in which a call was refused counts too,
once for the round however many of its calls were refused: a refusal is work the model asked for
and did not get.
The budget covers the whole prompt, the `max-retry-attempts` retries included. An attempt that
spent it does not get it back by being run again.
### The last resort
Refusing a call answers the model, which then decides what to do next. A model may simply ask for
the same call again. Because each of those rounds is one more call to the model, a run in which
`atoti.ai.context.max-consecutive-refused-rounds` rounds in a row ran no tool at all is ended
outright, and the user is told the question could not be answered.
```yaml theme={"languages":{"custom":["/engine/python-sdk/6.2/languages/pycon.tmLanguage.json"]}}
atoti:
ai:
context:
max-consecutive-refused-rounds: 3
```
Reaching it means the model ignored that many explanations of why it was getting nowhere; the
server log names the tool and the arguments it was stuck on. Any round that ran a tool starts the
count over, so this only fires on a model that is doing nothing else.
All five `atoti.ai.context` properties above are validated at startup: `prompt-timeout` must not be
negative and accepts zero to mean no deadline; `max-retry-attempts`,
`max-consecutive-identical-tool-calls`, `max-tool-errors` and `max-consecutive-refused-rounds` must
each be at least 1. A value outside those ranges stops the application from starting rather than
surfacing on someone's first question.
## Transport timeouts
The transport timeouts sit below every guard above, and bound the retries as well as the first
attempt.
For Amazon Bedrock they are the `spring.ai.bedrock.aws.*` properties, `timeout` above all: it caps
the whole call including its retries. See
[Set up Amazon Bedrock in Java](./enable-ai-tools/llm/amazon-bedrock-java).
For OpenAI it is `spring.ai.openai.timeout`, 60 seconds by default, capping the same call the same
way.
## Atoti Python SDK
```python theme={"languages":{"custom":["/engine/python-sdk/6.2/languages/pycon.tmLanguage.json"]}}
from datetime import timedelta
import atoti as tt
from atoti_ai import AiConfig, PromptConfig
from atoti_ai_amazon_bedrock import ChatConfig, ConnectionConfig
ai_config = AiConfig(
connection=ConnectionConfig(aws_region="eu-west-3"),
chat=ChatConfig(model="eu.anthropic.claude-sonnet-4-6"),
max_attempts=5,
prompt=PromptConfig(
timeout=timedelta(minutes=5),
max_retry_attempts=1,
max_consecutive_identical_tool_calls=3,
max_tool_errors=5,
max_consecutive_refused_rounds=3,
),
)
session_config = tt.SessionConfig(ai=ai_config)
```
The `atoti.ai.context` guards live in `PromptConfig` because they bound one chat prompt.
`AiConfig.max_attempts` sits outside it, next to the connection rather than inside it, because it
applies to every call to whichever provider the session connects to.
## Auto-Explain
Every `atoti.ai.context` guard above bounds a chat prompt, and only a chat prompt. Auto-Explain
calls the LLM directly, without the tool-calling loop those guards sit in, so what bounds it is the
provider layer: the LLM retry guard and the transport timeouts.
Under Amazon Bedrock, `atoti.ai.max-attempts` needs the chat on the classpath — the
`starter-ai-chat` Spring Boot starter, or an Atoti Python SDK session — since that is what brings in
Atoti's Bedrock clients. A Java application that declares only `starter-ai-autoexplain` builds none
of them, so the AWS SDK's own default of 3 attempts applies there; cap it with
`spring.ai.bedrock.aws.timeout` instead, which bounds the whole call including its retries. The
other providers read the property wherever it is set.