Alert Fatigue: 82 Emails in a Day From My Security Monitor

How many alerts should a one-person security monitor send? On my numbers, about one email a day, plus the rare one that asks me to do something. On November 1 my monitor handed 74 emails to the mail service between midnight and 13:32 Pacific: 55 for blocks it had already made, 16 critical alerts, two hourly volume alerts, and one from the scanner on my Mac.¹² My reading that afternoon was that every one described routine scanning the edge had already stopped;³ for the critical floods, section 6 shows that reading was too generous. The fix was a policy, not a threshold: one daily digest that doubles as the heartbeat, blocks to chat instead of mail, a critical folded into the digest when the edge already handled it, and the outcome written on each cloud critical. The next day a scanner set off four criticals in under six minutes across my sites, and my monitor sent zero emails about it.⁴ The day still produced 19 emails, not one, and the reason is the most useful part of the story.⁵

I am the only reader of my monitor’s mail. When the inbox said 74, I asked whether these were routine or real, and the honest answer was that the monitor could not tell me from the subject lines, because every subject said the same thing at a different volume.¹ A monitor whose one reader stops reading it has stopped monitoring, whatever its detectors still catch. What follows is the count, the policy that replaced it, a test case, and the three places where the clean version of the story breaks.

TL;DR

• 74 emails by 13:32, 82 by midnight. My own table and the count derived from the logs agree on 74 before I complained (55 per-block, 16 critical, two hourly, one local); the whole of November 1 came to 81 logged plus one digest I sent by hand.¹²⁵
• 22 of the 74 are unproven. They came from code that logged “sent” without checking the mail service’s answer, so the log proves a hand-off was attempted, not that a message arrived.²⁶
• The literature already had the rule. Google’s SRE book allows three outputs (alerts, tickets, and logging), says “Every page should be actionable,” and calls email alerts “alert spam”; the Joint Commission’s alert cites an estimate that “between 85 and 99 percent of alarm signals do not require clinical intervention”.⁷⁸⁹¹⁰
• The test case worked. On November 2 one address from a Dutch hosting provider raised four criticals in 5 minutes 51 seconds across my sites; the monitor blocked it once, posted the block to chat, folded every critical into the digest, and sent no email.⁴¹¹¹²
• The next day still sent 19, not one. Fourteen were critical WAF-flood alerts in which some events had no block or challenge recorded, and their subjects said “WAF blocks” even when the edge had blocked none of the events (twice that day; the evening before, one said “104 WAF blocks” when the edge had blocked or challenged nothing); my first rewording, the day it was counted, still mixed two counts in one line, and the review of this post caught it.⁵¹³¹⁴
• Two signals were wrong. A code-signing check on my Mac called Apple’s own Spotlight daemon an unsigned crypto miner, and the digest’s “RUNS SHORT 162/207” turned out to be part sleep, part a slower real cadence, and part, by the timing, three watchdog kills.¹⁵¹⁶¹⁷
• What changed: one digest a day carries the heartbeat, every block, and every cloud critical’s outcome, while only a critical neither fold covers, the hourly volume alert, and the local scanner still email at once; 82 emails on November 1 became 19 on November 2.¹⁸⁵

1. The inbox on November 1

At 13:32 Pacific on November 1 I wrote to the agent session that runs my infrastructure: the monitor’s emails were spamming my inbox; was there anything to change, and were these routine things or real threats?¹ The table I kept that afternoon counted 74 emails since midnight by sender: 55 one-per-block notices, 16 critical alerts, two hourly volume alerts, and one critical from the scanner on my Mac.¹

The count rests on a rule, so here it is before the numbers. An email, for this post, is a line in the monitor’s log that records a hand-off to the mail service. A script walked the six log files for October 31 through November 2 and classified every such line by stream and by outcome.⁵ Run to 13:32, the script agrees with my table stream for stream: 55, 16, two, and one, total 74.² The whole local day, to midnight, came to 81 logged emails (55 per-block, 22 critical, three hourly, one local), plus one digest I sent by hand at 13:48, which has no send line in the monitor’s log, for 82.⁵¹

Two details inside the agreement matter. My table said “9 before the fold went live at 00:30 PT; the 7 after” for the criticals; the log has 11 critical “sent” lines before 00:30 and five after, and the first critical folded into a block notice appears at 00:54:43, after the fold’s change landed at 00:46 PDT, not the 00:30 my table named.²¹⁸ The totals match; the split I wrote by hand was off by two. The second detail is larger. Of the 74, 22 (10 per-block, 11 critical, one hourly) came from senders that piped the mail service’s answer away and then logged “sent” no matter what came back.²⁶ A fix at 01:04 PDT that morning, before the clocks fell back, made the cloud half’s senders log the service’s HTTP status and say “sent” only on a 2xx; the local scanner’s sender writes a bare “Critical email sent” line, and only when the request raised no error.⁶ The 22 earlier lines prove that the monitor tried; they do not prove that the mail service accepted anything. My log for October 31 judged that the cloud emails “most likely were” delivered, which is a reasonable guess and still a guess.¹⁹

Even a 2xx proves less than the word “sent” suggests. Resend’s documentation defines its email.sent event as “Occurs whenever the API request was successful. Resend will attempt to deliver the message to the recipient’s mail server,” and lists delivery as a separate, later event.²⁰ The send call itself answers with an id.²¹ Every count below is a count of hand-offs the mail service accepted, or, for the 22 on November 1 and the 10 on October 31, 32 in all, hand-offs the monitor attempted.⁵ None of them is a count of messages I read.

By hour, the day looks like a cliff and a plateau.

November 1, local time: emails per hour against alerts that did not become email, with the 13:50 policy change marked.²²

The first hour carried 22 emails, all from the unverified code (10 per-block, 11 critical, one hourly).²² From there the per-block notices ran at one to five an hour, because the edge kept blocking scanners and every block wrote me a letter. The last per-block email left at 13:25:57; the first per-block notice folded into the digest instead appears at 14:20:35.²² After the 13:50 change the emails fall to between zero and two an hour while the folded and suppressed alerts climb to as many as 23 an hour.²² The traffic did not fall. The emails did.

The day before is a poor baseline. My log for November 1 says “Yesterday had 10,” and the script agrees, but the October 31 log starts at 22:31, the evening I re-armed the monitor, so those 10 emails cover 89 minutes.¹⁵ On my reading the 74 measured the detector working on a full day of internet background, not a change in what the scanners were doing.

2. What the alerts were for

The monitor has two halves. The cloud half is scheduled to run on my Mac every five minutes and asks Cloudflare for the firewall events and the grouped request counts on each of my zones: which addresses hit which paths, how often, and what action the edge took on each event.¹⁸²³ Cloudflare’s own description of the data is the right frame for it: “Security Events displays information about requests actioned or flagged by Cloudflare security products, including features such as Browser Integrity Check.”²⁴ The local half is a scanner that checks the Mac itself in tiers every 10 minutes: processes and their code signatures, the agent hooks on the machine, the tools that shadow system binaries on its path, and the like.¹²

What the cloud half does with the data is simple to state. It raises findings by severity (critical, high, medium) and category, and for a critical it tries to act before it tells anyone: when one address is behind enough of the events, it adds that address to an account-wide block list that every zone enforces at the edge, for seven days.¹²⁴ Until the November 1 changes, every one of those acts also produced an email, and so did every critical that a two-hour limit per site and category did not suppress, the end of every session, and every hour with heavy blocking.¹⁸⁵ The local scanner raises its own findings and emails its criticals itself.²⁵ Cloudflare’s own Alerts come in two kinds, “Monitors that you define yourself, and Platform Notifications”: scheduled SQL API queries and pre-configured product alerts, both set up in its dashboard.²⁶ Mine is neither: a script on my Mac that reads the firewall events itself, which also means the noise is mine to fix.

3. What the literature already said

None of what follows is new, which is the uncomfortable part. I had read most of it and built the email-per-event monitor anyway.

Three outputs, and email is not one of them. The introduction to Google’s Site Reliability Engineering says “There are three kinds of valid monitoring output”: alerts, which “Signify that a human needs to take action immediately”; tickets, which “Signify that a human needs to take action, but not immediately.”; and logging, where “No one needs to look at this information, but it is recorded for diagnostic or forensic purposes.”⁷ The same page is blunt about the design I had: “a system that requires a human to read an email and decide whether or not some type of action needs to be taken in response is fundamentally flawed.”⁷ Sorted that way, my 55 per-block emails were logging delivered as alerts. The edge had acted; nothing was left for a human to do.

Every page should be actionable. Rob Ewaschuk’s chapter on monitoring in the same book defines an alert as “A notification intended to be read by a human and that is pushed to a system such as a bug or ticket queue, an email alias, or a pager,” and its footnote on the email kind reads “Sometimes known as “alert spam,” as they are rarely read or acted on.”⁹ The chapter’s rules for pages fit in three lines: “Every page should be actionable.” “I can only react with a sense of urgency a few times a day before I become fatigued.” “If a page merely merits a robotic response, it shouldn’t be a page.”⁸ My robotic response to a per-block email was to delete it. The chapter also names the cost of the noise: “When pages occur too frequently, employees second-guess, skim, or even ignore incoming alerts, sometimes even ignoring a “real” page that’s masked by the noise.”²⁷ And it describes a team that “spent significant amounts of time triaging the alerts to find the few that were really actionable,” which is a fair description of my November 1 afternoon.²⁸

Symptoms over causes, and a test for a page. Ewaschuk’s earlier essay, which he says formed the foundation for that chapter, compresses the test to one sentence: “Pages should be urgent, important, actionable, and real.”²⁹ On causes it says “Include cause-based information in symptom-based pages or on dashboards, but avoid alerting directly on causes,” under a section titled “Cause-based alerts are bad (but sometimes necessary).”²⁹ I read a firewall block as a cause-level fact: a scanner hit a rule. The symptom I care about is a sensitive path served or a site down, and on my reading none of the 74 reported either; that day’s probe-path requests showed no 2xx on a sensitive path.³ The essay also anticipates the digest: “A daily (or more frequent) report can work too,” with the warning that “without a system of accountability this amounts to less-spammy email alerts”.³⁰ His remedy is a person: “make sure the oncall person (or someone else) is designated to triage these every day”.³⁰ On a one-person monitor that person can only be me. The outcome field in section 4 is my own supporting mechanism, not his: it gives the daily triage something to check, what became of each critical, but it does not do the triage.

Precision and recall. The Site Reliability Workbook’s chapter on alerting gives the vocabulary for what went wrong. Precision is “The proportion of events detected that were significant.” Recall is “The proportion of significant events detected.”³¹ I did not measure my monitor’s recall, which needs a count of the significant events it missed, and section 5 describes one it missed. As a mail stream its precision was near zero: 74 emails, and on my reading no message among them asked me to do anything. The same chapter names reset time, “How long alerts fire after an issue is resolved. Long reset times can lead to confusion or to issues being ignored.”³¹ Reset time is defined for alerts that keep firing; a per-block email fires once and then carries no resolution update: it never says the scanner left.

The hospital version of the same failure. The SRE chapter itself points outside software, to cardiac alarm management, “for an example of alert fatigue in another context.”²⁸ The Joint Commission’s Sentinel Event Alert on medical device alarm safety, issued April 8, 2013, puts the numbers on it: “The number of alarm signals per patient per day can reach several hundred”, and “It is estimated that between 85 and 99 percent of alarm signals do not require clinical intervention”.¹⁰ The alert is careful in a way worth copying: the 85 to 99 percent covers alarms that need no intervention, not only false ones, and the alert does not split the two. The consequence it records is the one I was halfway into: clinicians “become desensitized or immune to the sounds,” and “may turn down the volume of the alarm, turn it off, or adjust the alarm settings outside the limits that are safe and appropriate for the patient”.¹⁰ It counts 98 alarm-related events from January 2009 to June 2012, of which 80 “resulted in death,” and, beyond the factors reported to its database, names alarm fatigue “the most common contributing factor.”¹⁰ My stakes are far lower. The mechanism is the same: a person who learns that alarms do not need them learns it about the next real one too.

NIST, then and now. The 2012 revision of NIST’s incident handling guide, SP 800-61r2, warned that “it is not uncommon for an organization to receive thousands or even millions of intrusion detection sensor alerts per day.”³² The April 2025 revision, SP 800-61r3, which “supersedes SP 800-61r2,” drops that sentence; its nearest line is a recommendation that “The volume of potentially adverse events to be analyzed is generally quite high, so organizations should rely on technical solutions that filter large event datasets down to a subset that is suitable for human viewing and analysis.”³³ I take the filter as the instruction and the digest as one.

One vendor survey, labeled as one. Vectra AI’s 2023 survey, run by Sapio Research across 2,000 security analysts, reports “SOC teams receive an average of 4,484 alerts per day, and over two-thirds (67%) of them are ignored.”³⁴ It is a vendor-commissioned opinion survey, and the 67 percent is what respondents said, not a measurement. I cite it for one point only: the failure scales down. Analysts say they ignore two-thirds of 4,484 alerts a day, and one reader with no team stopped reading at 74.

4. The redesign, as rules

The change went in at 13:50 on November 1, 18 minutes after my message, and a second change at 14:00 tightened one of its conditions.¹⁸ I wrote it as rules, each with the reason it exists.

Rule 1: one digest a day, and it is the heartbeat. The first monitor run after 07:00 local time sends one email covering the window since the previous digest.¹⁸³⁵ The digest’s own description says what the email replaces: “One email a day … in place of an email per auto-block and per WAF-handled CRITICAL.”³⁵ It opens with run health (runs recorded, runs started and finished, failures, critical emails sent and suppressed), then the auto-blocks with zone, reason, and network operator, every critical and what became of it, findings by severity and category, the paths probed, and the local scanner’s unresolved findings.¹² The reason for the digest is the inbox. The reason for sending it daily is in the same description: “The digest is also the heartbeat: it goes out on a quiet day too”.³⁵ An alert stream that only speaks when something happens cannot tell me the difference between a calm day and a dead monitor. A digest that always arrives can: silence means breakage, not calm. The program’s own notes record why I care: earlier in 2026 the monitor ran dead for months, and nothing noticed.³⁵

Rule 2: blocks go to chat, not mail. The design notes give it four words: “Per block: Slack only.”³ A block is the edge acting on my behalf. In the SRE book’s terms it is logging: recorded, useful later, and nothing a human needs to act on.⁷ Chat keeps the live stream for the moments I want to watch one, and the digest keeps the record.

Rule 3: fold a critical when the edge already handled it. A critical still tries to block first; then the monitor decides whether it deserves an email.³ Two conditions fold it into the digest instead, checked in this order. The first: the finding’s own address, and every address in it eligible for automatic blocking, is already on the active block list, so the edge is stopping those addresses now.³¹⁴ That check looks at no other address and runs before the sample is examined, so a flood folded by it can still hold events with no block or challenge recorded, from addresses below the bar for blocking; this branch says the responder’s targets are blocked, not that every event was mitigated. The second applies to a WAF flood, a burst of firewall events on one zone: it folds only when every event in the sample, bot or not, has a block or a challenge recorded, and the sample is complete. “A solved or bypassed challenge, log, allow and skip all count as unmitigated.”³ Cloudflare’s definitions are why those four count against the fold: after a challenge, “If successful, Cloudflare accepts the matched request”, and a log action only “Records matching requests in the Cloudflare Logs.”³⁶ A persistent-scanner finding, which counts only events the edge blocked or challenged, folds by the WAF rule for that reason, or by the block-list rule when its address is already listed, as the November 2 one was.³ Anything else emails at once.

Rule 4: the outcome goes on the finding. The policy is that every critical from the cloud half carries one recorded outcome: folded because the address is on the block list, folded because the edge blocked or challenged every sampled event, emailed, suppressed (inside the limit of one email per site and category every two hours), or email failed.³³⁷ The record falls short of the policy. Of the criticals dated November 1 in UTC, 118 carry no outcome: 117 were raised before the field existed, and one cloud critical, at 14:13 PST, came after it; its run was one of the three in section 7 that ended about four minutes after starting, and it stopped during the automatic block, before an outcome was written. The local scanner’s criticals never carry one; the three of November 2 have none.³⁷³⁸ Since November 3 the monitor writes “pending” on a critical before the automatic block runs, so a run cut off midway leaves a visible state instead of a blank.¹⁴ The digest prints the outcome beside every cloud critical, under a heading that promises it: “Every CRITICAL and what became of it”.¹² The outcome field is my mechanism, not Ewaschuk’s. His condition for a daily report is a person designated to triage it every day, and here that person is me; the field gives that triage something to check.³⁰ With the outcome on each line, the digest answers the question I asked on November 1, routine or real, one critical at a time, though “emailed” or “folded” says where a critical went, not that anything was resolved.

Rule 5: the hourly volume alert stays. The notes again: “Hourly volume alert (responder): still immediate, at most once an hour”.³ A single block is routine; a sudden pile of them in one hour is the closest thing the cloud half has to a symptom, a change in what the internet is doing to me. It fired once on November 2.⁵

Rule 6: the local scanner’s criticals still email at once. The design notes keep them immediate because they are meant to be rare.³ I keep them for a second reason: nothing stands between my Mac and a local finding the way the edge stands between my sites and a scanner. A block list can stop an address; nothing outside the Mac stops a process on it. The cost of that rule shows up in section 7.

On the day the rules went live they cut the mail stream from 74 emails before 13:32 to seven logged in the rest of the day (six criticals and one hourly alert), plus the digest I sent by hand.⁵² The forecast I wrote that afternoon said “Under today’s traffic that would have been 0 immediate emails instead of 16.”¹ It did not hold, and section 6 explains why.

5. The test case: November 2, 11:03 to 11:21

The first real test of the rules arrived the next morning: one address belonging to a Dutch hosting provider ran a vulnerability scanner across my site.¹² Here is what the monitor wrote, verbatim from its log in local Pacific time, with the address replaced by “[the address]” and the lines in between left out.⁴

[2026-11-02 11:03:53] [INFO] New finding [critical]: Request flood from [the address] on blakecrosley.com
[2026-11-02 11:04:02] [RESPOND] [INFO] Block notification: Slack posted, email folded into the daily digest
[2026-11-02 11:04:03] [INFO] Critical alert folded into the daily digest (IP on the block ledger): Request flood from [the address] on blakecrosley.com
[2026-11-02 11:04:04] [INFO] New finding [critical]: Scanner from [the address] on blakecrosley.com
[2026-11-02 11:04:05] [RESPOND] [INFO] Skip: [the address] already has active real block in blakecrosley.com (dedup)
[2026-11-02 11:04:05] [INFO] Critical alert folded into the daily digest (IP on the block ledger): Scanner from [the address] on blakecrosley.com
[2026-11-02 11:09:22] [INFO] New finding [critical]: 726 WAF blocks in 5 minutes
[2026-11-02 11:09:23] [INFO] Critical alert folded into the daily digest (IP on the block ledger): 726 WAF blocks in 5 minutes
[2026-11-02 11:09:44] [INFO] New finding [critical]: Persistent scanner: [the address] blocked 743 times today
[2026-11-02 11:09:47] [RESPOND] [INFO] Block notification: Slack posted, email folded into the daily digest
[2026-11-02 11:09:47] [INFO] Critical alert folded into the daily digest (IP on the block ledger): Persistent scanner: [the address] blocked 743 times today

Minute by minute:

• 11:03:53. The request-flood detector raises a critical against the address. Eight seconds later, at 11:04:01, the responder adds it to the block list, and at 11:04:02 the block notice goes to chat with its email folded into the digest.⁴
• 11:04:03. The critical itself folds: the address is now on the block ledger, so the edge is already doing the work an email would ask me to do.⁴
• 11:04:04. A second detector, the scanner check, raises its own critical for the same address. The responder’s block request is skipped as a duplicate, and the critical folds a second later.⁴
• 11:09:22. The WAF-flood detector raises a third critical for the zone. Its offender is on the ledger, and it folds.⁴¹¹ The block-list fold is all that decides it: eight of the sampled events had no block or challenge recorded and the sample was capped, so the WAF fold alone would have emailed it.¹¹
• 11:09:44. The persistent-scanner check, which counts the events the edge blocked or challenged from one address across all my zones, raises the fourth, recorded on another of my sites: 743 blocked or challenged events from the address that day.¹² The responder writes the address to the list a second time (a defect, below), posts to chat, and folds the critical at 11:09:47.⁴¹¹
• 11:14:58 and 11:20:40. Two more WAF-flood criticals for the same zone; both fold the same way.¹¹

Four criticals in 5 minutes 51 seconds, six by 11:20, all recorded on their findings as folded because the address was blocked.¹¹ One address blocked, its list entry written twice because the duplicate check then worked per site (fixed on November 3), four further block requests skipped as duplicates.⁴¹⁴ Emails: zero. The log records no email sent between 11:03 and 11:21; the one critical in that window that neither folded nor emailed, suppressed at 11:03:13, was a flood on a different site from a different address.⁴

The next digest carried it as two rows in the auto-block table (the first block, with its zone, reason, and network operator, and the persistent-scanner re-add) and six rows in the table of every critical, each marked “blocked” or “blocked earlier”.¹² On my reading that is the right weight for a scanner the monitor blocked eight seconds after its first finding, eight minutes after the scan began: a few lines I can read with coffee, not a page.⁴

One thing the test case does not show, found on November 3. The same scanner was also posting to my site’s contact form, posts my app answered 200, at most 31 emails of their own, the subject of the companion post, and the monitor’s form-flood check raised no finding live; replayed over complete data for the five minutes to 11:03:23, the same check raises the finding, at high severity (a digest line, not an email or a block), from 39 posts, 23 of them refused.³⁹ The live run had asked Cloudflare for a window ending the moment it asked, 37 seconds after the run started, before the newest events had fully arrived: it counted 1,914 requests from the address in that window, where the replay over complete data counts 2,143.⁴³⁹ That lag is measured on all of the address’s requests, not on the form posts the check counts; on my reading, the same lag kept those posts under the check live. And each run looked back exactly five minutes while runs started about 5.6 to 5.8 minutes apart, so part of every interval was never scanned.³⁹¹⁷ The fix went in the same day: each window now starts where the previous one ended and ends 90 seconds before the run, unless the previous record is missing, under a minute old, or more than 15 minutes old, as after the Mac sleeps, in which case the run falls back to a plain five-minute look-back.³⁹ Replaying the November 2 window through the changed code gives the same findings, so the change did not disturb detection on a fixed window. A replay cannot exercise the tiling or the lag, which act only on live runs; the live runs from November 3 on are their test.³⁹

6. What still emails, and why

November 2 was the first full day under the new rules, and it sent 19 emails, not one: 14 criticals, one hourly volume alert, one digest, and three from the local scanner.⁵ No per-block email went out; 108 block notices folded into the digest instead, 177 criticals folded, and 37 more were suppressed by the two-hour limit, all counted by local day; the outcome counts later in this post go by UTC date, which is why those figures differ.⁵

Emails per local day by stream; October 31 covers only the 89 minutes after the log starts.⁵

Every cloud critical that emailed after the 13:50 change through the end of November 2, 20 in all (six in the rest of November 1, fourteen on November 2), was a WAF flood on resumegeni.com or blakecrosley.com in which at least one event had no block or challenge recorded.¹³ The WAF fold declined each one correctly by its own rule: an unmitigated event was in the sample, so the monitor could not say the edge had handled it.³ In those floods no address had enough events to be eligible for automatic blocking, so the block-list fold had no address to check.¹³ The digest names the group the same way: “a CRITICAL that was neither blocked nor already handled by the WAF”.¹²

The subjects of those emails told a different story, and that was a defect. The flood summary read “N WAF blocks in 5 minutes,” but N counted every non-bot firewall event in the window, whatever action the edge took on it.¹³ Two examples from the findings, in local time:

The first email claimed 104 blocks when the edge had blocked nothing.¹³ On my reading, a subject that says “blocks” on events with no block recorded is worse than noise: it tells the one reader that the system handled what it may not have. My first fix, on November 3, the day these counts were made, was itself wrong. It kept the non-bot count for N and counted blocked, challenged, and the rest over every event, so the 16:51 burst would have read 104 WAF events in 5 minutes: 0 blocked, 0 challenged, 117 through, two populations in one line. The review of this post caught it, and I fixed it again the same day.¹⁴ The flood summary, and with it the email subject, now counts one population, every sampled firewall event, and reads N WAF events in 5 minutes: B blocked, C challenged, U unmitigated (S from non-bot user agents); a JavaScript challenge counts as challenged, and the auto-block reason reads N WAF events from this address in 5 min.¹⁴ For the 16:51 burst that gives 117 WAF events in 5 minutes: 0 blocked, 0 challenged, 117 unmitigated (104 from non-bot user agents).¹³¹⁴ The WAF fold already keyed on unmitigated events, so the subject and the decision now count the same thing. The fix changes what the 14 subjects say. It does not change how many were sent, and 19 stands as the honest number for November 2.

The forecast I wrote on November 1 missed because its premise was wrong. It said the new rules “would have been 0 immediate emails instead of 16,” on the belief that for the seven criticals after 00:30 that morning “Cloudflare had already blocked every one of those requests.”¹ The findings say otherwise for those too: the five the log shows emailed after 00:30 record at most one block and nine challenges each, against 76 to 89 events in their summaries.¹³ And in each of the six criticals that emailed that afternoon and evening, the edge had blocked or challenged only part of the events (none at all in the 16:51 burst), so even the 13:50 rule, which folded only when blocks and challenges covered every non-bot event, would have emailed every one.¹³¹ Ten minutes after the 13:50 change, a second change made the WAF fold count every event, bot user agent or not, and trust only a complete sample; it made the rule stricter, but it is not why the forecast missed.¹⁸³ The forecast counted the blocks I believed in; the rule counted the events that were there.

What I do not know is whether those 14, the November 2 criticals that emailed, deserved an email. The findings record how many events were unmitigated, meaning no block or challenge was recorded for them, but not which actions they carried: a log rule, a skip, an allow, or a challenge a client solved.¹³ Nor is an unmitigated event a request that reached my site. Cloudflare says “A single HTTP request can generate one or more security events when it triggers security features,” and lists log and skip as actions that do not end rule evaluation the way a block does, so on my reading another rule can still act on the same request.²⁴³⁶ Cloudflare’s GraphQL tutorial shows the action field and sample values like "log" and "block", but it does not list every value the field can take.²³ Cloudflare’s Security Events dashboard samples too: “For large volumes of traffic, Cloudflare uses sampling to return results faster.”²⁴ Until the monitor records the actions behind each unmitigated event and ties events to requests, I cannot say which of those emails described traffic the edge let through, and the answer decides whether they deserve an email.

The three local emails on November 2 are the other open item. They report a “Browser -> shell chain” at 13:50 in which a shell spawned curl, and they sit unresolved in the findings.⁴⁰ I have not verified whether they were the agent tooling I was running that afternoon or something else, so I do not count them as false alarms.

7. Two false signals

The crypto miner that was Spotlight

At 10:03 on November 1 the local scanner raised a critical, “High CPU unsigned process - possible crypto miner,” against spotlightknowledged, Apple’s own Spotlight daemon, at 74.9 percent CPU, and two seconds later logged “Critical email sent.”¹⁵ The bug was mine. The check looked for the phrase “valid on disk” in the output of the code-signing display command, codesign -d. The scanner’s notes now record the flaw in one clause: the phrase is one “which only codesign -v prints, so every process read as unsigned”.²⁵ A skip list had hidden the flaw until an Apple framework daemon spiked, and my log for that day says so.¹ The finding now carries its resolution: “false positive, Apple platform binary”.⁴¹

It cost one email, the only local email of November 1.⁵ The 16 local criticals before it, on October 31 and November 1, flagged Xcode, Homebrew’s Python, my own app builds, hook changes, version-manager shims on the path, an AI-worm pattern that matched another project’s agent instructions, and an event-tap check that matched its own query; they produced 15 send attempts, and every one failed.^5251915 The mail service sits behind Cloudflare, which answered Python’s default user agent with error 1010 before the key was read; I reproduced it with a deliberately bad key.¹⁹ Cloudflare’s page for the error names that client directly, in its note on Pages and Workers hosts: “a User-Agent header that Browser Integrity Check treats as a non-browser client (for example, the default Python urllib user agent).”⁴² A broken sender saved me from 15 emails about my own tools and my own changes.¹⁵ The scanner’s notes put the larger lesson plainly: “most of the CRITICALs it raised in its first live days were its own noise.”²⁵ I keep the local criticals immediate anyway, and I fix each false one at its check rather than muting the stream.

RUNS SHORT 162/207

The first scheduled digest, on the morning of November 2, carried the subject “Mon Nov 2: 71 blocks on 15 zones, 11 criticals emailed, RUNS SHORT 162/207”.¹⁶ The digest’s rule was simple arithmetic: the expected run count is the window’s minutes divided by five, and the subject flags a shortfall.³⁵ The window ran from 13:48 on November 1 to 07:05 on November 2, 1,036.5 minutes, so 207 five-minute slots; the log recorded 162 runs.¹⁷

My log for November 2 explained the gap in one line: “The RUNS SHORT line is the Mac asleep,” with no fault in the monitor.⁴³ That line is right that the monitor did not fail (no run on November 2 was left unfinished) and wrong about the cause, on November 2 as well as over the window the number described. The run-gap analysis splits the 45 missing slots, approximately, into about 25 lost to the cadence, about 16 to sleep, about one more in a long gap after a run that ended early, and three runs that never recorded themselves.¹⁷

The power log shows idle sleep at 23:21:30 and full wake at 00:30:05, and again idle sleep at 06:01:20 and full wake at 06:30:28.⁴⁴ The three runs started at 14:09, 14:19, and 23:10 on November 1; each ended about four minutes later with no ending line logged, which on my reading is the 240-second watchdog. A fourth run ended 28 seconds after starting with no visible cause.¹⁷³⁸ On the November 2 calendar day, 242 runs started against 288 five-minute slots, 46 short, and none was left unfinished; only two gaps passed 10 minutes, 35.8 minutes in all, so on that day too most of the shortfall was the cadence, not sleep.¹⁷

The wrong assumption was mine, in the digest: a scheduler set to five minutes delivered a run about every 5.6 to 5.8 minutes even while the Mac was awake, so the five-minute arithmetic would call a healthy day short.¹⁷ I have not established why the awake cadence runs slow. I replaced the arithmetic on November 3. Run health now reports the runs recorded, the median minutes between them, and the longest gap with its time, and the subject flags only a gap longer than 20 minutes. That morning’s preview read “about one every 5.6 min”, and its subject ended “RUN GAP 31 min at 05:56”.¹⁴ A gap is a symptom I can act on: the Mac slept, or the monitor stopped. A ratio against a cadence the scheduler never delivered was neither.

The rules I kept

1. Count the mail before you change it, by stream, with a stated rule. My table and the log count agreed on 74 only because I wrote down what counted as an email first.¹²
2. Log “sent” only when the mail service says yes. For 22 of my 74 emails I can prove an attempt and nothing more.²⁶
3. A block the edge already made is a log line, not an alert. The SRE book’s three outputs put it there, and moving 55 per-block emails to chat lost nothing I acted on.⁷⁵
4. Send a daily digest even when nothing happened. A digest that always arrives is the heartbeat; its absence is the alert.³⁵
5. Fold a critical only on a condition the edge proves. Mine are the finding’s blockable addresses already on the block list and a complete sample in which every event has a block or challenge recorded, and the second one is why 14 still emailed on November 2.³¹³
6. Write the outcome on each cloud critical. A daily report nobody is accountable for is, in Ewaschuk’s words, “less-spammy email alerts”; the outcome on each line is what my daily triage checks.³⁰¹²
7. Make the subject say what the edge did. “104 WAF blocks” on a burst with zero blocks was my worst line of the week.¹³
8. Report gaps, not ratios against a schedule. A five-minute timer delivered a run about every 5.6 minutes, so 162/207 looked like an outage and was mostly a clock.¹⁷
9. Fix a false alarm at its check, not at the inbox. One misread command output made Apple’s Spotlight daemon a crypto miner; the fix went into the check.²⁵

What could change

• If the unmitigated events turn out to be solved challenges, that alone would not make the WAF-flood criticals that still email noise. When a client passes a challenge, “Cloudflare accepts the matched request”: the request proceeds, and that does not show the request was harmless or that nothing needs a response. Folding those criticals would take request evidence tied to the same events showing that nothing in them needed a person, and challenge completion alone is not that evidence. If they turn out to be log or skip rules, the question is whether those rules do what I meant, and each email is worth reading until I know. The monitor does not record which yet.¹³³⁶
• If a day’s critical emails climb back toward the dozens, the two-hour limit per site and category is doing the work that a rule should do, and the rule needs another look.³
• If the digest stops arriving, something between the monitor and my inbox has stopped: the monitor, the Mac it runs on, the sender, or the delivery path, since an accepted send is not yet a delivered one. That is the one silence the design treats as a prompt to look.³⁵²⁰
• If the local scanner raises a critical I cannot explain from my own tooling, the immediate email was the right call; the three unresolved shell-chain findings of November 2 are the open test of that.⁴⁰
• If I add a reader, pages and tickets stop being the same person, and the SRE book’s split between alerts and tickets would need a ticket queue, not my inbox.⁷

Key Takeaways

If you run a monitor for yourself

• I stopped reading somewhere short of 74 emails in half a day, a count my inbox reached by early afternoon.¹
• A daily digest that arrives on quiet days too is a heartbeat for free; a monitor that only speaks on events cannot tell you it died.³⁵
• Your logs may say “sent” for mail that never left. Check the code before you trust the count.⁶¹⁹

If you build one

• Act first, then decide whether to tell: block, then fold the alert if the block list or the edge already covers it.³
• Record each critical’s outcome on the finding and print it in the digest, so the person Ewaschuk says must triage a daily report has something to check; on my monitor that person is me.³⁰¹²
• Name events by what the edge recorded for them (blocked, challenged, unmitigated), count one population in one line, and never name them by what you hoped the edge did.¹³¹⁴
• Measure the scheduler you have. Mine ran every 5.6 minutes on a five-minute timer.¹⁷

If you are the one reading the inbox

• Ask of every alert whether it is “urgent, important, actionable, and real”; anything that fails belongs in a digest or a log.²⁹
• My first-day forecast missed: it said zero immediate criticals, and the next day sent 14.¹⁵

FAQ

What is alert fatigue?

Alert fatigue is what happens to the people on the receiving end of too many alarms that need nothing from them: they become, in the Joint Commission’s words, “desensitized or immune to the sounds,” and start turning alarms down or off.¹⁰ Google’s SRE book describes the same thing in operations, where people “second-guess, skim, or even ignore incoming alerts, sometimes even ignoring a “real” page that’s masked by the noise.”²⁷ My own case took one day: 74 emails by 13:32, none of which, on my reading that afternoon, asked me to act.¹

How many security alerts per day is too many?

For a person, the SRE chapter’s honest answer is a few: “I can only react with a sense of urgency a few times a day before I become fatigued.”⁸ Vectra’s vendor survey reports teams receiving an average of 4,484 alerts a day and, by their own account, ignoring 67 percent of them.³⁴ For my one-person monitor the working answer is one digest a day plus the alerts that need a human now; on November 2 that was still 19, and I treat the 14 WAF-flood criticals among them as an open question rather than a settled cost.⁵¹³

Should security alerts be a daily digest or real time?

Both, split by what the human must do. Anything the monitor counts as already handled (an address it has put on the block list, or a flood in which every sampled event was blocked or challenged) goes into the digest; the rest goes out at once, within the two-hour limit.³ Ewaschuk allows that a daily report “can work too,” with one condition: without accountability it “amounts to less-spammy email alerts,” and he wants someone “designated to triage these every day”.³⁰ On my monitor that is me, and my digest prints what became of every cloud critical so the triage has something to check.¹²

What is a heartbeat alert?

A heartbeat is a message that arrives on a schedule whether or not anything happened, so that its absence means something broke. My daily digest is the heartbeat: “it goes out on a quiet day too,” so a monitor that has stopped running shows up as a missing email.³⁵ So does a failed send or a lost delivery, so a missing digest is a prompt to check the monitor, the host it runs on, the sender, and the delivery path.²⁰ Its run-health section also reports the longest gap between runs and flags a gap over 20 minutes in the subject.¹⁴

Why do Cloudflare WAF alerts flood my email?

Mine flooded because each block and each critical became its own message, and the internet scans every public zone all day.⁵ Cloudflare’s own Alerts, “Monitors that you define yourself, and Platform Notifications,” can be delivered by email and are set up in its dashboard; a monitor you build yourself against its API, like mine, is neither, and inherits whatever per-event mailing you wrote into it.²⁶ Check what an alert counts, too: Cloudflare’s actions include log and skip, which do not end rule evaluation the way a block does, and a challenge that a client solves ends with Cloudflare accepting the request, so “WAF events” and “WAF blocks” are different numbers.³⁶ My subjects confused the two until November 3.¹³¹⁴

Does a 2xx from the email API mean the alert was delivered?

No. Resend’s email.sent event “Occurs whenever the API request was successful. Resend will attempt to deliver the message to the recipient’s mail server,” and delivery is a separate event.²⁰ A 2xx means the service accepted the message. Log the status code, and treat delivery as a different question.⁶

Related on this site: Why Spam Beat My Rate-Limited Contact Form follows the same November 2 scanner from the contact form’s side; What I Run Before I Sleep is the nightly production check, the other routine that runs while I am not looking; What Actually Breaks When You Run AI Agents Unsupervised names the failure modes of unattended agent sessions, each with a detection signal; Every Hook Is a Scar traces each of my agent hooks to the failure that produced it, the same habit as fixing a false alarm at its check; Claude Code Auto Mode Is Not a Security Boundary argues that the wall around an agent matters more than its classifier; and AI Agent Observability makes the case for seeing what agents do to disk, CPU, and network.

Sources

All times in the author’s own data are local Pacific time unless marked UTC; November 1 fell back from PDT to PST at 02:00. Public pages were fetched on November 3, 2026, at the UTC times given.