I’m a little OCD about my $PATH…

cleanpath () {
    local path p result
    local -A seen
    PATH=${PATH//::/:}
    IFS=":" read -r -a path <<< "$PATH"
    for p in "${path[@]}"; do
        if [ -z "${seen[$p]}" ]; then
            result="$result:$p"
            seen["$p"]=1
        fi
    done
    PATH=${result:1}
}

more...

This was more a challenge than a solution to some need:

#!/bin/bash

while read l && read n <&3 ; do
    echo $n "$l"
done 3< <(hexdump -v -e '/4 "%u\n"' < /dev/urandom) | 
    sort -n | cut -d' ' -f2-
exit 0

And, yes, B5 is now on Amazon Prime, and I just watched “War Without End”.

“Microsoft saves sinking ship; passengers leap to their deaths.”

I just got email touting the new “Adobe XD CC” app, which promises:

“The future of experience design. No experience required.”

This sounds like exactly what happened when Apple set fire to their decades of usability studies and sucked on a pistol Boldly Invented New Paradigms.

Several years ago, the personally-owned MacBook Pro (Togetsukyō) that I used for work went flaky, and I didn’t feel like spending the three grand or so that it would take to buy a fully-tricked-out replacement. So I had my boss buy me the best he could get approved, which was the 13-inch model with 16 GB of RAM and a 512GB SSD. I named it Hello!Party, inspired by Scott’s ‘favorite’ song, and carefully downsized onto the smaller SSD.

It served me well for three years, but around the same time AppleCare gave out, so did the right-side USB port. So he ordered me a new one, and the approval process didn’t require so many compromises, so I ended up with the 15-inch touchbar model with 16 GB and a 1 TB SSD, which I named Macchi: it’s not headless, but it’s formidably proportioned. Apart from the mind-bogglingly terrible keyboard and the mostly-just-annoying touchbar, it’s a spiffy thing, and the extra space gives plenty of room for VMware Fusion virtuals.

But there was a problem: a bunch of data from Tog never fit onto H!P, and was only available in an archive on the house server, a refurbished Mini named Melwin. I’d also decided to keep a cleaner separation between work and home, so I had two accounts on H!P, one with all my work stuff, one with personal email, iTunes, Yojimbo, etc. For more fun, H!P was bound to our AD domain, so the work account had funky permissions. For even more fun, when I got Macchi, I only moved the work account, so I had to carry H!P if I wanted access to personal stuff.

Ten days ago, I noticed that Apple’s refurbished store had just gotten a new batch of 12-inch MacBooks in stock, which seemed like the perfect opportunity to clean up my personal/work accounts once and for all. Buying refurbs directly from Apple is actually the best way to get a reliable machine with a full warranty, as clearly explained by the folks at MacRumors. Brand new hardware is a crap shoot (especially models released in the past 3-4 months), and no one else’s refurbs are eligible for AppleCare. It’s got a Core i7, 16 GB of RAM, and a 512GB SSD, and I kept the name Hello!Party.

TL/DR, I’ve spent the last three days figuring out how to merge the best bits from multiple accounts on three machines to create a fully-populated personal account where everything actually works and has the correct ownership, while stripping out the last of the personal stuff from the current work machine.

(well, the naughty personal stuff, for sure; unlike most of the people who certified that they’d read the employee handbook this year, I actually read the employee handbook…)

Side note: after I scrubbed and reinstalled the 13-inch MBP, the right-side USB port seems to work again. Sigh.

One thing I’m doing with the new Synology NAS is making sure that everything is successfully migrated from my ancient Infrant ReadyNAS NV+.

There are two basic reasons for this:

1. The NV+ uses a non-standard power supply, and both of the ones we had at the office eventually burned out, requiring a temporary swap of mine until the data could be retrieved and migrated elsewhere. Mine’s still good, but if it goes…

2. While the firmware has been updated to cope with the most famous SMB security hole, it’s otherwise an ancient version of Debian on a custom SPARC chip, and even with the RAM upgraded to 1GB, it’s painfully slow at serving up files. It has decent write speeds, but when it comes time to get your terabyte of data back off, it takes forever, especially if you’ve got lots of little files.

I figure the copies should finish by the weekend. Maybe. On the bright side, it’s so slow that the Synology has plenty of bandwidth left to handle copies of every other old drive in the house…

Update

After letting it chug along overnight, it’s averaging a steady 2 MB/s. With 500 GB to go, that’s just about 3 more days. This is so ridiculous that I had to double-check that it really is getting a full-duplex gigabit connection and not falling back to something like 10-megabit/half. No, it’s not the network; it’s just that mind-bogglingly slow. When this is all done, I’m going to reset it to factory defaults and do some testing.

Screw it, it’s sneakernet time!

I grabbed a spare 1TB USB drive, formatted it as EXT3, mounted it on the old ReadyNAS, and told it to back up the largest of the volumes (301 GB). It’s rapidly catching up to the aborted rsync job. As a bonus, the built-in backup job uses Perl. 😜

Finished!

The 301 GB sneakernet finished considerably faster than the rsync job, and my other ReadyNAS just took a few hours, so I now have all my eggs in one basket.

And my basket is now running: find . -not -name '.*' -type f -size +4096 -print0 | xargs -0 md5sum to figure out just how much duplication there is in the ~6 TB of files (not counting the 2 TB of Acronis, SuperDuper, and Time Machine images…). I figure there will be at least six copies of this video:

Update: nine copies of it. 😊

Just bought a new NAS for home, and decided on a Synology DS918+ with 4 10TB drives ($539 + 4 x $310). Why not another ReadyNAS? A combination of price and vague dissatisfaction with the ones I’ve used in the past; I may write that up sometime.

Why not FreeNAS? Because I didn’t feel like building one from scratch right now (as much as I like the idea of a ZFS-based NAS), and the prebuilt unit we once bought from iXsystems ended up going back due to being a piece of junk. Both Synology and ReadyNAS use BTRFS as their filesystem format these days, which offers a lot of what you get with ZFS without the need to occasionally resort to command-line incantations. (“Not That There’s Anything Wrong With That!”)

Drive installation was painless (simple snap-in hot-swap trays), and while I found the “desktop” web GUI a bit overdone, everything works well. The biggest annoyance was figuring out which of the “private cloud” packages to add, because they recently changed all that, resulting in some confusion. (short version: only install the Drive package and desktop/mobile clients, and open TCP ports 5000, 5001, and 6690; also use the builtin LetsEncrypt support and set everything to require SSL)

The “EZ-Internet” cloud/firewall config was useless; it’s just a UPnP wrapper, and when it realized that it couldn’t auto-configure my OpenBSD router, the only help it offered was “hey, you should open some ports”, with no indication of which ones were actually required for the installed packages (see above).

Side note: I was amused and pleased that Drive, their latest, greatest personal cloud solution, required installing the Perl package. 😜

I went with their ‘hybrid’ RAID config, SHR-1, because it resizes better when you add more drives or swap in larger drives. This gives me 26 TB in usable space (9.1 * 3 - overhead), which is plenty for now. Down the road, if when media, disk images, and automated backups start to fill that up, I’ll add the DX517 expansion chassis and another 5 10TB drives and bring it up to 52 TB usable.

If you’re following along at home, you may wonder why adding 5 drives doesn’t give closer to 70 TB, and the answer is paranoia. SHR-1 uses a single parity drive, which means you can only afford to lose one disk. This is generally not a huge problem if you have a spare on-hand and swap it in immediately, but there’s a non-trivial risk that another drive will fail while the first one is rebuilding.

If you think about it, this is even more likely when you buy all your RAID disks at once from the same manufacturing batch, so you really want two parity disks and a hot spare, so that the system can start rebuilding as soon as one disk fails, and can survive losing another one during the rebuild. Having only one data disk in a four-disk chassis isn’t terribly useful, so for now I’m running in a cheaper, less-paranoid configuration. When I’m sure that I like the Synology enough to really rely on it, I’ll buy the expansion and convert the RAID to SHR-2 with a hot spare. And buy a cold spare disk as well.

Additional performance enhancements I can add include bonding the two 1-gigabit ports together, bumping the memory (official max 8GB, but there are reports that 16GB works), and adding SSD cache drives. That last is specifically why I chose the 918+, since it has a pair of M.2 slots on the bottom, and some of their other models require you to buy an expansion card first.

Building the volume was quick, but it took ~16.25 hours to run the initial parity consistency check, so performance was sub-optimal until that finished. The GUI was occasionally a bit sluggish during that time.

Next up: setting up dedicated Time Machine volumes for the Macs and testing their Windows backup client.

Oh, and I named it Index.

Update

First Time Machine backup complete. Just because I was curious how well it would work, I backed up 425 GB over wireless, which took about 7.5 hours.

Just picked up the smaller Orbi bundle at Costco. This is the SKU they’ve added recently (RBK22-100NAS) that only has two units (“router” and “satellite”, both with ethernet backhaul); I didn’t really need a 3-pack of the original model, just one on each floor.

The hardest part of the setup was switching off the builtin NAT and running it in AP mode; you can’t do it from the iOS app. The second hardest was discovering that the app artificially limits you to short passwords; the web GUI will let you enter up to 63 characters, as expected for WPA2-PSK.

Preliminary results look good. I may tweak the placement of the units (I just grabbed the first available power and ethernet, since the old wireless is still running), and turn on the optional beamforming, etc. At the very least, I should get better performance on my front porch.

I figure it’ll take me a few days to find all my wireless devices and switch them over. :-)

Update

The optional beamforming is off by default for a reason. It apparently has disconnect issues.