Field guide
Recognize a repeated tool cycle.
Instrument tool calls with argument fingerprints, detect repeated cycles without progress, and avoid confusing legitimate batch work with a stuck agent.
Code and examples reviewed
A tool-loop detector looks for repeated patterns in reported tool activity. Watchdog matches tool name, status and a per-run argument fingerprint in the last 30 tool completions since meaningful progress. The default threshold is four repetitions, configurable from three to ten.
Give repeated calls enough context.
Two calls to the same tool can do different work. The Python SDK fingerprints canonical arguments locally using a random per-run HMAC key. Raw arguments stay in your process. Without arguments, repeated names and statuses provide less information, making legitimate work harder to distinguish.
with run.tool_call("record.process", arguments={"id": record.id}):
process(record)
run.progress("Record processed", completed=completed_count)This snippet belongs inside an existing Watchdog run. Report progress only after useful work succeeds. The tool context records duration and status; an exception remains an exception in your application.
Reproduce a two-tool cycle.
In an isolated test workspace, register lab-cycle with a repetition threshold of four, long runtime/progress limits and recovery disabled. This example deliberately leaves a synthetic run uncompleted:
from watchdog_agent import Watchdog
with Watchdog() as watchdog:
run = watchdog.run("lab-cycle", cancellable=False).start()
for _ in range(4):
run.tool("synthetic.search", arguments={"query": "same"})
run.tool("synthetic.read", arguments={"document": "same"})
if not watchdog.flush(timeout=10):
raise RuntimeError("Telemetry did not drain")
# No terminal event: this is an intentional unfinished test run.Closing the client drains its queue; it does not complete a manually started run. The repeated tail is search → read four times without progress. The Failure Lab verifies the stored evidence using the actual SDK.
If a run finishes normally before delivery, an active loop condition can resolve and its queued alert may become obsolete. An ingestion-time finding and a delivered alert are different observations.
Do not label every repetition a failure.
with watchdog.run("batch-processing", cancellable=False) as run:
for index, record in enumerate(records, start=1):
with run.tool_call("record.process", arguments={"id": record.id}):
process(record)
run.progress("Record processed", completed=index)Distinct arguments distinguish different records, while genuine progress resets the repetition window. Emit that progress as work completes, before reaching the threshold. Progress arriving later can resolve a finding but cannot guarantee an earlier alert was never sent.
Polling a resource with identical arguments may be legitimate. The detector does not know the business meaning of a call. Define real milestones, choose limits suitable for the job, and investigate the evidence instead of assuming every match is a runaway agent.
Check what your framework actually reports.
The optional OpenAI Agents adapter observes supported function-tool hooks. Provider-hosted tools may not pass through those hooks. Errors turned into result strings may not be observable as errors without explicit instrumentation. Do not double-count a tool through both the adapter and manual events.
Watchdog does not inspect prompts or model reasoning to infer semantic progress. This keeps the required signals concrete, but makes good instrumentation part of setup. Start with one verified execution before adding a tool policy.