Reading Usenet with Emacs gives new meaning to the phrase “garbage collecting.”

— J Greely

Technically, it was handling it…


Woke up this morning, looked at my phone, and saw that I hadn’t received any work email since about 1:15 AM. Since I’m guaranteed to get at least one hourly cron-job result, that’s bad.

Login to mail server (good! that means the VPN is up and the servers still have power!), check the queue, and it eventually returns a number in excess of 500,000.

Almost all of them going to the qanotify alias. Sent from a single server.

The good news is that this made it very easy to remove them all from the queue. The bad news is that I can’t just kill it at the source; QA is furiously testing stuff for CES, and I don’t know which pieces are related to that. And, no, no one in QA actually checks for email from the test services, so they won’t know until I — wait for it — email them directly.

For more fun, the specific process that’s doing it is not sending through the local server’s Postfix service, so I can’t shut it down there, either. It’s making direct SMTP connections to the central IT mail relay server.

Well, that I can fix. plonk

(this didn’t delay incoming email from outside the company, just things like cron jobs and trouble tickets and the hourly reports that customer service needs to do their jobs; so, no pressure, y’know)

First update

QA: “I see in the logs that the SMTP server isn’t responding.”

J: “Correct. And it will stay that way until this is fixed.”

(I find myself doing this a lot these days; User: “X doesn’t do Y!”, J: “Correct”)

Second update

Dev Manager: “Could you send us an example of the kind of emails that you’re seeing?”

J: “You mean the one that’s in the message you’re replying to?”

Third update

DM: “Can you give my team access to all of the actual emails?”

J: “No, I deleted the 500,000+ that were in the spool. But it looks like at least 25,000 got through to this list of people on your team, who would have known about this before I did if they didn’t have filters set up to ignore email coming from the service nodes.”

J: “And, what the hell, here’s thirty seconds of work from the shell isolating the most-reported CustomerPKs from the 25,000 emails that got through, so you can grep the logs in a useful way.”

Fourth update

John: “Ticket opened, assigned to devs, and escalated.”

(John used to work for me…)

Fifth update

Senior Dev: “Ooh, my bad; when I refactored SpocSkulker, I had it return ERROR instead of WARNING when processing an upgrade/downgrade for a customer that didn’t currently have active services. Once a minute. Each.”

SD: “Oh, and you can hand-edit the Tomcat config to point SMTP to an invalid server while you’re waiting for the new release.”

J: “Yeah, no, I’ll just keep blocking the traffic until the release is rolled out and I’ve confirmed with tcpdump.”

(one of my many hats here used to be server-side QA for the services involved, so I immediately knew it was coming from SpocSkulker, and could have shut it off myself; but then it wouldn’t have gotten fixed until January)

Anticipated update

J receives massive fruit basket from Production team for catching this before it rolled out to them and took out their email servers.

Virtual Escape


The single most useful feature of virtual desktops in Windows 10 is getting out of crashed full-screen games:

Win-Ctrl-D
Ctrl-Alt-Del
"End Task"
Win-Tab

Trump Rally Highlights, Impeachment Edition


It’s real; I checked with Snopes.

Dear US Senate,


Make The Rubble Bounce.

Not this one:

On second thought, your time would be better spent drafting a resolution supporting lap-dancing Flintstones cosplayers. It has the gravitas that the In Peaches Mentos fund-raising telethon lacks.

Dear Woot,


Hotter than intended:

Not exactly a winter storm watch…


Just saw a local severe weather alert. I think I’ll be okay.

..HIGH SURF ADVISORY REMAINS IN EFFECT FROM 3 PM THIS AFTERNOON TO 3 AM PST SATURDAY…

  • WAVES AND SURF…Northwest swells 12 to 14 feet at 17 to 18 second periods. Breakers of 16 to 22 feet possible.

  • TIMING…Thursday night through early Saturday morning.

  • HIGH TIDE…High tide will correspond with the peak of the swell Friday morning between 10 am and 12 noon. Localized coastal flooding possible as a result of increased wave run-up.

  • IMPACTS…Large breaking waves along the coast will lead to increased wave run-up on beaches with waves topping and washing over large rocks and jetties. Use extra caution near the surf zone as these large waves will be capable of sweeping people into the frigid and turbulent ocean water. Cold water shock may cause cardiac arrest, and it also can cause an involuntary gasp reflex causing drowning, even for a good swimmer. Long periods increases the risk of rip currents. The surf zone will be dangerous due to strong currents and powerful breaking waves.

PRECAUTIONARY/PREPAREDNESS ACTIONS…

A High Surf Advisory means that high surf will affect beaches in the advisory area, producing rip currents, localized beach erosion and sneaker waves.

2020 continuous calendars


Sequel to last year’s inspired-by-Wondermark calendars, it’s 2020 time. Generated with the example script I included in the PDF::Cairo distribution.

Dear Amazon,


All I want for Christmas is less dynamically-loaded content on your home page. Seriously, if I switch to another tab while it’s loading, half the time I’ll come back and find that Safari has given up and reloaded it. Sometimes it reloads while I’m still scrolling down. Any other page is fine.

If I turn off Javascript, it loads a thousand times faster, but then the “Recommended For You” page is completely blank. Wish lists still work, although you have to dig a bit to find them.

This all seems to have started right before Black Friday. Perhaps dialing back on the poorly-debugged holiday scripting is in order?

“Need a clue, take a clue,
 got a clue, leave a clue”