Claude Code Triaging Sentry Alerts Read-Only: Locking It Down With --allowedTools, and the Pitfalls

One day my alert triage opened an issue whose body was a single sentence: "I need your authorization to run this command."

Triage is a Claude Code session with only read-only tools. When a Sentry or Alertmanager alert fires, it goes looking for clues, reaches a conclusion, and posts it to Discord. Whether to change anything is my call. "Can't change anything" is enforced in --allowedTools, not requested politely in the prompt.

That issue led to the two pitfalls I hit after launch. The model rewrote command paths on its own and kept getting blocked by the allowlist until it gave up. And Sentry treated self-recovering WebSocket disconnects as errors, so a triage ran for every exception type.

I treat triage conclusions as leads only and check them again myself before acting.

At first, this lending platform's exceptions only lived in logs. When a white screen or a production anomaly showed up, I had to SSH into the machine and grep.

Later I added three observability tracks. Prometheus, Grafana, and Alertmanager answer whether the system is broken. Sentry answers which user or route hit an unhandled exception. Micrometer Tracing plus OTLP answers where a request's time goes.

With an empty DSN, Sentry doesn't initialize on either the frontend or the backend. When it's on, sendDefaultPii is off and query strings are stripped, because cookies, API keys, and TOTP have no business in a third-party error service.

Triage unifies the signals from Alertmanager and Sentry and hands them to Claude to investigate.

A read-only boundary in --allowedTools

--allowedTools grants only Read, Grep, and Glob, plus a read-only ops query script and GitHub queries. No Edit, no Write, no arbitrary Bash. Mechanically, this session can't modify anything.

Put the limit in permissions, not in the prompt. The model will rewrite commands, but it can't rewrite its way into tools it doesn't have.

Why stop at diagnosis? The root cause of an alert might be that deploys came so close together they starved the scheduler, not that the verifier itself was broken. An agent that can fix things on its own would most likely go and change that verifier.

Triage keeps a human at the "decide whether to change it" step, which is exactly where machines are least reliable.

Every triage costs real API money. The same signal is investigated at most once every 6 hours and at most 8 times a day, with mkdir as a lock so only one runs at a time. Late arrivals don't queue, because queueing only produces stale conclusions.

Reports follow a fixed format that must include a verdict line. If the verdict line can't be read, the report is treated as needing a fix and an issue is opened. One issue too many is the better mistake.

The model rewrote command paths

To figure out that issue, I dug up the transcript of the session on that machine.

The prompt gave the absolute path of the query script, but the model rewrote it as a relative ./scripts/... path, then again with bash in front.

The allowlist (Claude Code's Bash permission rules) only recognized the absolute-path form. After getting blocked 5 times in a row, the model gave up, and the sentence it gave up with had no verdict line, so by the rules an issue was opened.

This wasn't a one-off. Of 89 triage sessions, 17 had been blocked at some point. Among the blocks, 10 were relative paths or a bash prefix, and another 28 were output piped to python3.

The fix was to add three equivalent forms to the allowlist: bash plus the absolute path, a relative path starting with ./, and bash plus the relative path.

Each form you add should make you ask whether it widens the capability. Claude always runs from the repo root, so all of these point to the same read-only script, and nothing was widened. Piping to python3 stays blocked, since that amounts to arbitrary code execution.

I also added two sentences to the prompt: type commands with exactly the path given and don't pipe them. If evidence can't be obtained, say so plainly under "Issues" and still give a verdict in the fixed format.

I originally wanted to write "if you get blocked, retype the command, don't ask for authorization." The Claude Code auto mode classifier flagged that sentence as instruction poisoning and blocked it.

Looking back, that's fair. Telling an agent not to ask a human when it gets blocked really does look like teaching it to bypass permissions. When you write prompts for an agent, describe the right way to do things, and don't teach it how to get around a block.

Cutting Sentry noise at the source

What triggered that issue was a network blip on the connection to Bitfinex. REST timed out first, the WebSocket disconnected, reconnecting timed out once more, and everything was back within 30 seconds.

The WebSocket client logged connection failures and onError with log.error and the exception attached, and Sentry ingests logs at error level and above. So every network exception type opened its own Sentry issue and ran its own triage.

Over two days the WebSocket disconnected 14 times, and it recovered on its own every time.

The fix was to downgrade those two spots to log.warn and log only ex.toString(), without the stack trace full of JDK internal frames.

The line for synchronous connection construction failures stays at ERROR, since that one is more likely a code or config problem. If the connection really stays down for a long time, the "disconnected for over 1 minute" alert is the backstop.

I chose not to set ignore rules on individual Sentry issues. New exception types (SSL handshake, DNS) keep spawning new issues, and each one would burn a triage.

Cut noise at the source. Transient errors are warnings, and persistent states belong to alerts.

Triage conclusions are only leads

Later the reconciliation alert fired twice, and the triage report was wrong both times. The first time it correctly guessed which new lane was involved but got wrong where the math broke. The second time it was wrong even about what was broken.

Following its suggestions would have meant changing code that wasn't broken, both times. The investigation is written up in missed and double-counted interest when Bitfinex positions switch sides.

When a triage conclusion looks wrong, my first step is to open that session's transcript. Every tool call and every reason for a block is in there, which is much faster than guessing what's wrong with the prompt.

The new allowlist can only be verified the next time the model rewrites a path, and the blocked python3 pipes haven't been dealt with, so triage can still come back incomplete.