In which Bell discovers Orario’s red-light district and a princess in need of rescuing, and Hermes gets more than he bargained for. And I don’t mean rubber ducks.

(this week’s new character in the game turns out to be Ishtar; perhaps they won’t add Haruhime until her power is revealed in an upcoming episode)

This is actually a bit disturbing…

The Happy Fan:

The Miller data-manipulation tool has a lot of potential, but the organic development process has left it with a lot of rough edges.

My first hint that all was not well internally was when I was hacking on PDQ output and realized that I had to type nest --implode --values --across-records --nested-fs space to get what I wanted. The manpage includes a shortcut for the nest verb’s other mode, so that --explode --values --across-records --nested-fs can be replaced with --evar, but there’s no matching --ivar. So I forked the project, added it, and sent a pull request.

The functionality was trivial, but the single-line usage description had to be added five times. That sent up a little red flag.

Still, it was easy to do, so I thought I’d see if it was feasible to fix one of the other things that bugs me, which is the lack of quoting and/or escape characters in its native DKVP file format (delimited key-value pairs, aka foo=1,bar=2,baz=3).

The answer turns out to be “not easily”, and I quickly learned some unpleasant things about how it handles other data-format conversions.

Given the perfectly-ordinary CSV input file foo.csv containing:

a,b,c,d,e
"1,1","2
2","3","4\r4","5\n5"

I get the following results:

# DKVP: useless as expected
% mlr --icsv cat foo.csv
a=1,1,b=2
2,c=3,d=4\r4,e=5\n5

# (note: internally, fields have correct values)
% mlr --icsv --ojson cut -f a,b foo.csv
{ "a": "1,1", "b": "2\n2" }

# CSV: reasonable, but strings converted to ints
% mlr --csv cat foo.csv
a,b,c,d,e
"1,1","2
2",3,4\r4,5\n5

# Quoted CSV: better, but should be default
% mlr --csv --quote-original cat foo.csv
a,b,c,d,e
"1,1","2
2","3","4\r4","5\n5"

# JSON: reasonable, but strings converted to ints
% mlr --icsv --ojson  cat foo.csv 
{ "a": "1,1", "b": "2\n2", "c": 3, "d": "4\\r4", "e": "5\\n5" }

# Quoted JSON: oh, hell no
% mlr --icsv --ojson --jvquoteall cat foo.csv 
{ "a": "1,1", "b": "2
2", "c": "3", "d": "4\r4", "e": "5\n5" }

Note that the automated num-ification has real consequences for data processing, since you can’t do things like regex matches or string-substitutions on a number type, and have to explicitly coerce fields back to strings; the error message for this is less than clear. Also, leading zeroes trigger octal conversion…

So that’s an enhancement request for escaping comma, cr, and lf in DKVP, plus a bug on the busted output when you add the --jvquoteall option to avoid the num-ification of string literals. (and it bothers me that I had to explain the bug in a completely different way because one of the devs didn’t understand my sample CSV file…)

I see a massive refactoring in its future (“cover a wall with color-coded sticky notes, then break out the chainsaws and forklifts”). Oh, well, useful tool when used with care.

Cop Craft spoilers, ho.

more...

No idea where all those zombie girls came from; I must have something on my mindbraaaaiiinzzz.

more...

This couldn’t possibly go wrong.

After last week’s Dire Menace From Another Time (with a side order of foreshadowing), it’s time to kick back, relax, and develop some characters! This story was the second half of book 2, and it feels pretty wrapped-up to me, which means they’re really pushing to get through the source material. At least it worked pretty well this time, the relationship between Our Heroes is solidifying, and Tirana got some entertaining culture shocks. Still no sign of the cute elf photojournalist from the OP, who isn’t in any of the light-novel illustrations, so I don’t know when or where she shows up.

Unrelated, I’d like to state for the record that I have never had any information about alleged financial, sexual, or criminal acts undertaken by or on behalf of anyone related to, employed by, or associated with Bill and/or Hillary Clinton. Never never. Absolutely sure of it.