PromptConfig

atoti_ai.PromptConfig(
    *,
    timeout: timedelta = datetime.timedelta(seconds=300),
    max_consecutive_identical_tool_calls: int = 3,
    max_tool_errors: int = 5,
    max_consecutive_refused_rounds: int = 3,
    max_retry_attempts: int = 1,
)

The configuration of a chat prompt.

​

Attributes

​

timeout

How long the chat spends on one prompt before giving up. This deadline covers everything a prompt does: every tool call and every retry the LLM provider makes underneath it.

​

max_consecutive_identical_tool_calls

How many rounds in a row one prompt may call a tool with the very same arguments. Past that, the call is not run. The model is told that this exact call was just made and returned the same thing, so that it moves on to something else. Without this bound, a prompt telling the model to retry until it works would repeat the same fruitless call until timeout expires. Each call is counted on its own, and its count resets as soon as a round does not make that exact call, so a tool the model comes back to later keeps running, however many times. A round asking for several tools at once only loses the call it is stuck on, and still runs the others. It is charged one max_tool_errors failure all the same, so pairing the stuck call with a fresh one does not buy the model unlimited rounds.

​

max_tool_errors

How many of one prompt’s tool calls may fail before it stops calling tools. Once that many have failed, the model is told to stop and answer with what it has, instead of trying another tool call. This bounds a model working its way through a broken request, which max_consecutive_identical_tool_calls does not catch when the model varies its arguments. The budget covers the whole prompt, the max_retry_attempts retries included. A refused call counts as a failure too, but a round counts only once, however many of its calls were refused. Ending a run in which no tool runs at all is left to max_consecutive_refused_rounds.

​

max_consecutive_refused_rounds

How many rounds in a row may run no tool at all before the run ends. Refusing a call answers the model, which may simply ask for it again, and each of those rounds is one more call to the LLM. Once that many have run no tool at all, the run ends and the user is told the question could not be answered. Any round that runs a tool starts the count over.

​

max_retry_attempts

How many times the chat sends a prompt to the LLM, retries included. Left at 1, the prompt is sent once and the retrying is left to the LLM provider: its client backs off between attempts and does not retry a request it knows is hopeless, neither of which this loop can do. See AiConfig.max_attempts. Raise it only for failures happening outside the provider call, such as a malformed tool call from the model. Each attempt re-runs the whole prompt, tools included.