Friday, July 18 2003

Sometimes it’s not the network

My job was Unix support for Corporate Services, which basically referred to everything in the company that wasn’t related to developing, selling, or training customers how to use our products. In practice, though, it usually just meant MIS, because HR and Legal were composed entirely of Mac people, who had their own support team.

The oddest exception started one day when an HR manager asked me to help him set up a beta-test of a Lotus Notes-based applicant tracking system. The application was being developed on OS/2 servers and PC clients, but we wanted to test it with a SunOS server and Mac clients, since that’s what we had.

This made the vendor’s promise of tech support irrelevant, but I’m a bright fellow, and I had no difficulty setting everything up. In fact, the thing that took the longest was tracking down a shareware TIFF viewer that was compatible with the scanned images of the résumés, which were made available in case the OCR was less than perfect. [short version: the OCR was so bad that we ended up outsourcing to Scotland to have human beings transcribe the text from the applications]

By the time the first big demo rolled around, I was quite familiar with the quirks of both Notes and the HR application, and the lack of problems made it look like it would be an easy addition to my support load.

The morning of the demo, they had me run around and set up the Notes client on everyone’s Mac, testing to make sure everyone’s machines and accounts were working correctly. It was smooth sailing until I reached the manager’s Mac and my first launch of Notes bombed miserably:

hr-notes-1: not a valid tcp/ip host.

I pulled up a copy of NCSA Telnet and connected to hr-notes-1 just fine, so something wasn’t right here. I found a minor error in the MacTCP settings, corrected it, tried again.

hr-notes-1: not a valid tcp/ip host.

I deleted Notes, trashed the preferences file, rebooted (just in case), and installed again from scratch.

hr-notes-1: not a valid tcp/ip host.

The manager and the CEO of the startup whose product we were testing were standing behind me. The CEO offered the help of his tech-support guy, who had never used a Macintosh. He had a contact at Lotus, though, so we passed it off to them for advice. They’d never seen this error before. Nothing they suggested helped at all.

All other network connectivity was fine, and another machine plugged into his ethernet cable worked like a champ. All his software was up to date, and installing a brand-new System folder from a working machine didn’t fix it either. Stabbing about at random, I even changed the DNS records so hr-notes-1 was the primary hostname rather than a CNAME record.

hr-notes-1: not a valid tcp/ip host.

I sent them away to run their demo on the 20 other identical machines that worked just fine, and promised to return in the morning and tackle the problem again.

The next morning, I went in loaded for bear. I scrubbed all traces of Notes off of the machine, painstakingly configured and tested his System folder and MacTCP settings, verified that I could reach the Notes server on the network, and ran the installer.

hr-notes-1: not a valid tcp/ip host.

They were standing behind me again, watching patiently while I poked and prodded at this single reluctant Macintosh. I told them to go away, and then sat and stared at the screen for fifteen minutes, quietly furious at the taunting dialog box that now defined the limits of my existence.

Then I changed the name of his hard disk, and everything worked fine.

The problem? Lotus Notes was originally a DOS product, and deep down, it still believed that all the world was a DOS box. Somewhere, hidden from view, the mark of DOS was branded on its soul, and it could never forget its true master. In more prosaic terms, the exclamation point is an illegal character in a DOS file name, and the HR manager had named his hard disk “Go Packers!!!”.

It had never had anything to do with the network. Notes just couldn’t find any of its support files on the nonexistent “Go Packers” (sic) volume.

The HR manager came up to me and said that for the first time he understood what system administrators really do.

Within a year we abandoned the Notes-based system because it couldn’t track the data needed for AA/EEO compliance.