I’ve been adding example code to PDF::Cairo. Here’s the output from what I’ve got so far:

• 2019 Progressive Calendar, a much-cleaner version of the script I wrote in March.

• Box-slicing test, not actually useful for anything.

• Image-loading test, ditto. (new! improved! fixed translate/rotate/scale order to get rid of skew)

• Inkle drafting grid, planning tool for Inkle weaving. Reimplemented from here, using my Box library and polygons rather than my hand-tuned, impossible-to-debug component method. About 12x larger output, but Cairo isn’t a PDF library, it’s a library that can output PDF, so you don’t get as close to the metal.

• Isometric drawing pad, mixes my Box library with a bit of trig to clean up the grid layout.

• Kanji practice sheet, currently blank; I plan to add a second example that fills it in with selected characters in a pen font, grayed-out so you can draw on top of them. It’ll be much cleaner than the version I wrote ~14 years ago.

• Sample Pango layout, showing the use of its markup. Now includes glyph outline support.

• Japanese report paper.

• Kyokuto-ish notepad sheets.

• NEW! Glyph outlines, including manually processing the drawing operators.

• New! Traveller hex grid. Unlike my 15-year-old sec2pdf script, which I threw together over a weekend when I discovered how terrible the existing Traveller mapping tools were, this one actually centers the grid correctly on the page. (seriously, before sec2pdf, the way to create a PDF Traveller map was to run a DOS program that spat out an import script for an obscure 2D free CAD program, then let it run for 10-20 minutes; these days it’s a lot better, although the border definition method I came up with is still in use)

Here. Needs a full test suite and a bunch of examples before I throw it up on Github/CPAN, but I’m pretty happy with the functionality.

I think if I were exploring another scripting language, I’d look for Cairo and FreeType support, and then I’d port this module over. A bit of work up front, but less hassle than dealing with the quirks and limitations of another implementation.

Stripped of comments and documentation, it’s just about 1,500 lines of code, half of which is in the main PDF::Cairo package. Quite a bit of it is PDF::API2 compatibility, which I could jettison when porting to another language, so it would be pretty quick.

Usage is quite straightforward:

use PDF::Cairo qw(cm);
$pdf = PDF::Cairo->new(
    file => "output.pdf",
    paper => "a4",
    landscape => 1,
);
$font = $pdf->loadfont('Times-Bold');
$pdf->move(cm(2), cm(4));
$pdf->fillcolor('red');
$pdf->setfont($font, 32);
$pdf->print("Hamburgefontsiv");
$image = $pdf->loadimage("logo.png");
$pdf->showimage($image, cm(5), cm(5),
    scale => 0.5, rotate => 45);
$pdf->write;

Honestly, the thing that took the longest was dredging up the schoolboy trig to get it to rotate images from the lower-left corner instead of the upper-left. Cairo’s inverted coordinate system makes sense on-screen, not so much in print.

I was very selective about what I pulled in from Pango. It’s a peculiar library that simultaneously gives you more control over text layout while removing control over font selection. You can precisely position each glyph in a Unicode string, but have no idea what it will look like on the page. Some of this it inherits from Fontconfig, which is a giant hairy mess of XML configuration files, but they’ve layered their own clunky parser on top of that.

For instance, on my Mac, I have four styles of the typeface family Hoefler Text installed: Regular, Italic, Black, and Black Italic. Using the Fontconfig tool fc-match, I can select them with the string ‘Hoefler Text’ optionally followed by ‘:bold’, ‘:italic’, or ‘:bold:italic’. Using Pango’s pango-list tool, I discover that the correct incantation there is to request ‘Hoefler Text Small-Caps’, with the words ‘Bold’ and ‘Italic’ optionally added in that order after ‘Text’.

Leave out the ‘Small-Caps’, and I get Verdana (well, technically I get Hoefler Text Ornaments, which then falls back to Verdana because that font doesn’t actually have any alphanumeric characters in it). Use it, and I get the right font, but without small caps. Try to request real small caps (which it has), using the variant='smallcaps' attribute, and I still don’t get small caps (despite the documentation, I’m not sure that works at all). I have no idea how to fix this, or if it’s even possible.

I made the Pango support optional, because it takes a full two seconds to initialize the first layout on my laptop. By comparison, my calendar generator takes less than a second to create a 10-year, 59-page calendar.

Obnoxious Bug!

Unfortunately, while I was porting one of my other old scripts to use as an example, libfreetype segfaulted in FT_Set_Transform. Didn’t matter what font I loaded, there was just something about that script that triggered it.

After some painstaking debugging, it looks like I have two workarounds. First, render the text as a Pango layout, which avoids calling Freetype directly from Cairo. Second, use the create_for_stream() method to initialize the PDF surface and explicitly save the file at the end, rather than use the standard create() method that takes a file name up front and auto-saves when the script exits. That last bit seems to be where the bug is being triggered.

Ironically, this is how my PDF::API2 compatibility methods work, so a direct port of the old script works fine, and it doesn’t crash until I try to update it to use what seemed to be the recommended method (pun intended).

For now, I’ve commented out the use of create(), and added an explicit write() method to dump the PDF file to disk. I’ll test again when Homebrew catches up to the current Cairo release.

5/17 Update

I’ve refreshed the tarball about a dozen times now, as I add examples and come across bugs and little niggling issues in the APIs. I’ve pretty much rewritten half of the Box module as I’ve started using it more aggressively in simplifying the calculations in my examples.

I’d say that I’m just about ready to stick a fork in it, write the test cases, and release, but then I thought of another epicycle: rendering SVG files as vectors. Currently, the only way to import existing graphical data is as fonts with FreeType or bitmapped images with Imagemagick’s convert tool.

I’ve used the custom font trick before to load illustrations into a program that didn’t support them, but I’d rather import SVG directly. Because ImageMagick supports rasterizing vector formats, I can already place an SVG into a PDF, but only as a bitmap.

But it turns out that librsvg is already designed to do exactly what I want: render an SVG into a Cairo context. There’s just not a maintained Perl module that links to it. I can’t say that I’ve always wanted to figure out how to write XS glue, but since my needs are very simple, I think I can dodge that bullet and create a simple API with SWIG.

Looking at the RSVG API, I think all I really need to do is:

1. Pass a filename to rsvg_handle_new_from_file(), save the result.

2. Pass that to rsvg_handle_render_cairo() along with a Cairo handle.

3. Profit!

I’m not even sure I need rsvg_handle_free(); the memory should be reclaimed when the Perl variables go out of scope, or worst case, when the script exits.

Never mind…

It looks like SWIG is not going to be easier than XS. Oh, well.

After a few more epicycles (and a two-week vacation in Japan), I have about 95% compatibility with PDF::API2::Lite, covering pretty much everything I’ve used in my scripts over the years, so that they work with a two-line change. Along the way, I learned just enough about fontconfig to coax it into finding my real Adobe core fonts based on their legacy PostScript names; I’ll need to figure out the substitution crap for any system that only has free equivalents installed, of course, but that’s for when I clean it up and add a test suite before releasing to the world.

As a bonus, my most recent epicycle was calculating the exact values of ascender, descender, and xheight for non-exotic fonts, which is more useful than the font designer’s opinion when you want to precisely position the top edge of a string (as in, say, a calendar; I was able to completely rip out the unreliable hackwork from the PDF::API2::Lite version).

Currently missing: skew transformations (matrix math), elliptical arcs (matrix math), full Cairo path support (new feature), Pango layouts (major new feature), rendering JPG/PNG/GIF/etc (under-documented API calls…), and a PDF::API2 compatibility method for cjkfont (requires messy fc-match tinkering, and I’ve never used it…).

And I just thought of another epicycle! A lot of my grid-based PDF scripts make use of a little box-manipulation library I wrote years ago, that does things like ISO folds, centering, cropping, etc, and recently I started going over the API to clean it up and improve its functionality. Then I realized: I only use this for PDFs, and I’ve already copied the big paper-size library into PDF::Cairo, so why not just make it PDF::Cairo::Box? One-stop shopping, and it will make my calendar generator even cleaner!

5/10 Update

Basic test/install done for the packages; need a lot more test code before I can release. Revised Box module really cleans up the calendar script. Played with Pango just enough to learn that it’s fragile and quirky and expensive to load, but has potential when you need real text layout; couldn’t get vertical text out of it, though, and the feature request for furigana support has been open since 2005.

If I have any working brain cells tonight, I think I’ll work on tests for the matrix transforms to get skews and arcs working correctly. Maybe take a first crack at image methods.

Update

Well, that wasn’t so bad. Got skew transforms to work, and got PNG images to load, scale, and display correctly. I may punt and use ImageMagick’s convert CLI utility to handle other image formats. There is a Perl ImageMagick library, but I’ve never been able to get it to work correctly with a perlbrew-built Perl on a Mac, and just calling convert is easier.

5/12 Update

The last item missing for full PDF::API2::Lite compatibility is the stripped-down version of the textstart/textend functionality that I’ve never used. Adding the full API2 version of this would be more complicated and less useful than integrating Pango layouts, despite the latter’s significant startup time and entertaining quirks (such as Pango’s two completely different integer units, both of which are referred to as “Pango units” in the documentation: 1/1024pt for general positioning, 1/10000em for vertical positioning of glyphs).

As a bonus, because PDF::API2 is mostly pure-Perl, PDF::Cairo is about five times faster, similar to the speedup I got the time I converted one of my scripts to use the abandoned Haru library.

Also known as “adding epicycles”, referring to a specific variety of scope creep where each decision in the project design opens an entirely new layer of crap to deal with.

It started with the idea of using Cairo to create PDF files using modern fonts that don’t work well with PDF::API2’s aging codebase. That pulled in Font::FreeType, and together they were enough to convert one of my simple-but-useful scripts, and seemed like 90% of what I needed to create a drop-in replacement for PDF::API2::Lite.

Except that I had to create an extra save/restore state stack to handle the fact that PDF has both a stroke color and a fill color, and Cairo only has one.

Also, Cairo doesn’t support the standard 35 PDF fonts, which are the standard PostScript printer fonts, which are Times, Helvetica, Symbol, etc. In fact, unless you load font files from disk, you’re limited to specifying the CSS2 generic fonts (“serif”, “sans-serif”, “monospace”), and wondering what it will decide that means each time you run your scripts.

So I need to support and/or supply free alternatives, such as the ones from URW that are used with Ghostscript, and Adobe’s Source Han Serif and Sans. And, yes, even if you do have the real fonts Adobe supplies with Acrobat, they’ll end up embedded in the output; PDF output is really kind of a sideline for Cairo, and honestly, I don’t think the authors expect anyone to own commercial fonts.

That should be enough, right? Except that in all of the Cairo documentation, the text-display methods are referred to as the “toy” API, and if you want to really use fonts for more than basic “this un-kerned string goes on the page at x,y”, you need to use a real layout engine like Pango.

Which doesn’t let you load arbitrary font files from disk, blowing up all the scaffolding I’ve just built around the assumption that Cairo’s FreeType support is supposed to actually be used.

Pango doesn’t just layout text, it uses span tags to mark up a string with font family, weight, etc, and uses Fontconfig to figure out what font you really wanted, quickly falling back all the way to the most generic font it can find on your computer, which on my Mac is Verdana.

So now it’s not enough to download a set of metrics-compatible OpenType fonts, I also need to create custom XML config files that override whatever else may be installed on a machine, and install them in a location that fontconfig is guaranteed to process first. XML, because there’s nothing like a text-based config parser that, by definition, must abort when it detects even the tiniest error.

There is a way to tell Pango to tell fontconfig to load a single font file from disk, but those methods aren’t exposed in the Perl Pango module. Which means I’d have to dig into the guts of how you link C libraries into Perl modules, add all the necessary functions and a set of tests, submit a patch to that project, wait for a release, and then at last C-ko would be mine!

Spoiler: yeah, not gonna happen.

By the way, it looks like no one has ever published a non-trivial example of how to use the Cairo, Pango, or Font::FreeType Perl modules, separately or together. Even a lot of the test code is little more than “yup, that called the C API and returned something”. Saying “see the C library web site” isn’t terribly Perl-y, given that everyone in the open source world has basically decided that automatically-generated API dumps are all the documentation anyone should ever need.

Which created Yet Another Problem, because it seems no one has ever even used the create_for_stream() method to render a PDF into memory and save it at the end. Which I need in order to transparently emulate PDF::API2::Lite’s saveas() method.

Therefore I present what is apparently the world’s first working Perl script that uses a Cairo PDF stream:

#!/usr/bin/env perl

use Cairo;

my $DATA;
my $surface = Cairo::PdfSurface->create_for_stream(
	sub {
		my ($whatever, $data) = @_;
		$DATA .= $data;
	},
	"whatever", 320, 240,
);
my $pdf = Cairo::Context->create($surface);

$pdf->move_to(10, 48);
$pdf->set_font_size(32);
$pdf->show_text("Hamburgefontsiv");

$surface->flush;
$surface->finish;
open(Out,'>:raw', "test.pdf");
print Out $DATA;
close(Out);
exit 0;

(if you just use the create() method that requires a file name, then you don’t have to flush, finish, or save your output at all; it just happens when the script exits)

But the fun never stops! Would you believe that there is no user documentation for the lookup strings used by Fontconfig? Everyone just copies the examples from the ‘user’ documentation that tells you all about the XML config format and very little about exactly what properties and constants you can specify:

Fontconfig provides a textual representation for patterns that the
library can both accept and generate. The representation is in three
parts, first a list of family names, second a list of point sizes and
finally a list of additional properties:

  <families>-<point sizes>:<name1>=<values1>:<name2>=<values2>...

Values in a list are separated with commas. The name needn't include
either families or point sizes; they can be elided. In addition, there
are symbolic constants that simultaneously indicate both a name and a
value. Here are some examples:

  Name                            Meaning
  ----------------------------------------------------------
  Times-12                        12 point Times Roman
  Times-12:bold                   12 point Times Bold
  Courier:italic                  Courier Italic in the default size
  Monospace:matrix=1 .1 0 1       The users preferred monospace font
                                  with artificial obliquing

That’s all you get. Everywhere. The rest of the “user” documentation is about the zillions of XML config files that get parsed to decide what to do with these strings, which adds another half-dozen epicycles. The only complete documentation I can find on how these strings are parsed is the commentless source code in src/fcname.c. Maybe.

This is the kind of stuff that’s under the hood of desktop Linux, which is why everyone uses Windows and Mac.

Current Project Status

First script successfully converted using the PDF::API2::Lite compatibility methods. Well, except for the font-specific methods that I need to create a wrapper class for; I can get all the data I need from Cairo’s font_extents() and text_extents() methods, but those only return data for the current font at the current size, where the ones from PDF::API2 calculate them at load time for a size of 1 point. Not a huge issue, just another epicycle…

Update

New epicycle: fonts lie. The text_extents() methods do the right thing, but the Cairo documentation confesses that many of the elements returned by font_extents() are the font designers’ opinion on the correct vertical sizes of the font. This has always annoyed me, so as part of building my wrapper class, I’m going to actually render representative characters into a recording surface in order to determine the precise values of ascender(), descender(), capheight(), xheight(), and fontbbox(). Also latin_baseline(), which can be pretty goofy in some kanji fonts.

While waiting on a fix for my Perl PDF font woes, I started thinking about some of the other long-standing issues I’ve had with PDF::API2. I actually use PDF::API2::Lite most of the time, with a few extra functions (like clip) merged back in, because I don’t need color spaces, bar codes, metadata, outlines, etc, etc. I just need all of the drawing functions and decent font handling, including proper subsetting in the output file.

Which it doesn’t do. Pro tip: the easiest way to clean up a PDF file full of cruft like giant embedded fonts is to run it through the ps2pdf utility supplied with Ghostscript. Despite the name, it works just fine as a pdf-to-pdf converter and optimizer.

Bottom line, PDF::API2 has been getting staler every year, and the font thing encouraged me to look elsewhere. Phil Perry is taking a stab at it with his fork PDF::Builder, but he’s still feeling his way around, and it’s a big job.

But I’ve written a lot of code over the years, and the PDF APIs I’ve looked at in other languages are either convoluted, limited in function (report writers and form fillers, mostly), just plain missing, and in at least one case commercial, and that generally applies to the other modules I rely on, too, like the Swiss Army Knife of date handling, DateTime.

By happy accident, I discovered that the Cairo library will generate PDF files, and integrates nicely with FreeType for robust font support, and they both have decent Perl modules. Being C libraries, they also have some performance advantages, but the most important thing they have is support.

Documentation and examples are pretty limited for the Perl modules, but I just finished successfully converting my calendar generator to use them, and it was only moderately annoying. The two biggest issues were that Cairo thinks (0,0) is the upper left corner, and that it only tracks one color in its graphics state, compared to PDF’s separate stroke and fill colors. Shimming around these problems added about 55 lines of code. Text positioning is improved thanks to Font::FreeType (and can be improved a lot more by exploiting the API fully), and output with embedded fonts is significantly smaller. Also, all my fonts work.

Now that I have a working example, I’ve started work on a wrapper module that works like PDF::API2::Lite, so that I can convert my old scripts by just editing a few lines at the top. Shouldn’t be too much work, and I’m tentatively calling it PDF::Cairo.

There’s also a Pango module for advanced text layout, but integrating that is definitely a “phase 2” item.

Update

As far as I can tell, Pango offers the exact opposite of what I want, turning font lookup into a game where the penalty for guessing wrong is having all your text rendered in Verdana.

It seems the font bug I ran into when making calendars has opened a can of worms. PDF::API2 is only lightly maintained these days, but there’s active development on the fork PDF::Builder, and the dev saw my bug report, cloned it into his project, and started poking around. Original module author Alfred Reibenschuh chimed in, and after some back-and-forth, it looks like the glyph-mapping code pretty much needs to be ripped out and rewritten from scratch.

I have a hunch I’ll be tasked with producing a CJK test case, which means I’ll need to collect some Chinese, Chinese, and Korean samples.

No, not that kind of “progressive”, the Wondermarkian sort, also called a continuous calendar. Every time he creates one of these in Illustrator and posts it, I save it away with the idea of knocking together a Perl script to generate my own. It only took 4 years for me to get around to it…

Let’s see if I can make a sensible download page for these:

• US Letter paper
  • 6-page: Sun, Mon
  • (wide): Sun, Mon
  • 2-page: Sun, Mon
  • (wide): Sun, Mon
  • 1-page: Sun, Mon
• A4 paper
  • 6-page: Sun, Mon
  • (wide): Sun, Mon
  • 2-page: Sun, Mon
  • (wide): Sun, Mon
  • 1-page: Sun, Mon
• 11x17 paper
  • 6-page: Sun, Mon
  • 2-page: Sun, Mon
  • 1-page: Sun, Mon

The script needs some cleanup before I inflict it on the world (most likely as a web form). Also better font-handling; not only is there a hilarious bug in the header auto-sizing, but there are baseline issues with many (mostly CJK) fonts that make it tricky to position the numbers. I also need to make the locale-handling friendlier; the Perl DateTime module automagically pulls in month and day-of-week names for any language supported by locale(1), but specifying them as ja_JP, ko_KR, ru_RU, etc is “less than user-friendly”, and you have to switch to an appropriate font.

Sadly, the obvious free choice for a font that supports pretty much any language would be Noto, but its CJK fonts don’t work with PDF::API2.

One feature that’s still in progress is custom colors and text for specific dates. The idea is to feed it a custom list of holidays, birthdays, etc. Since I’m still working on text placement, right now it just sets foreground and background colors, which was enough for marking up a one-page, one-month vacation calendar for our upcoming Japan trip: red for flights, orange for shinkansen trips, blue for flea market days, and yellow background for “oh, crap, it’s Golden Week”.

Update

…and I’ve fixed the silly font sizing/positioning bugs and re-uploaded. Part of my problem was that I was fighting some silly fonts, and the epicycles required to position them vertically were screwing with the non-silly fonts. I still have a fudge factor in the script, but now it’s based on identifying the silliness (capheight or ascender >= bbox ymax).

Just noticed…

Trello’s calendar add-on works this way, but with just a subtle shading change as you move between months (along with the gray-on-gray text thing, which never fails to annoy me).

I just had to do this one first:

Of course, now that I have a good source and tool for making O’Reilly-ish (technically Dover Pictorial Archive style) cover images, more could appear at any time.

The top script comes from here; the bottom one is my old minimized CipherSaber implementation.

[Final version of the new japh I came up with after making this cover; intermediate steps after the jump]

sub _'__'_{sub{map{pop.shift}@_}}sub{print+(map{pack$..h.2,$_}_ _::__->
(split$,,shift.pop.shift))[map{hex}split$\,pop]}->(map y{#-`}{+-]}r,qw&
...../. (aa(+f*(,-0+e)-cb ../// ////..*. e/-0d+,b0.*ad1*a)d.+c(*af&)&0_

more...