Comparative Lovecraft and “The Colour Out of Space”

Lately, I’ve been trying to clear out my backlog of purchased audiobooks while I get other things done… and one of the things I bought was a Groupees bundle of HorrorBabble H.P. Lovecraft readings.

So far, The Colour Out of Space has been my favourite Lovecraft short story… a fact that I apparently share with Mr. Lovecraft himself.

You can legally enjoy it for free in both textual and audio form… though I prefer the audio version I’ve linked, as read by Ian Gordon for HorrorBabble. It really helps to keep you immersed when you can focus on visualizing the events being described without also having to dedicate effort to reconstructing the intonation and cadence of the narrator’s voice from the text.

Given that it’s a horror story, where presentation is key, I don’t want to say too much about the plot, so I’ll just borrow the beginning of Wikipedia‘s synopsis:

An unnamed surveyor from Boston, telling the story in the first-person perspective, attempts to uncover the secrets behind a shunned place referred to by the locals of Arkham as the “blasted heath.”

Instead, I’d like to talk about why I enjoy it so thoroughly. (Though, given how much I say, I’d appreciate it if you read/listen to the story first, then come back to finish this. I worry that analyzing the story in such depth will have an “explaining the joke ruins it” effect.)

First, many of Lovecraft’s stories, such as The Call of Cthulhu, have an element of xenophobia to them which hasn’t aged well. (The Rats in the Walls is uniquely egregious among the stories I’ve read so far. Not willing to settle for Lovecraft’s usual stuff, like implying that swarthiness is a sign of disreputability, it’s an otherwise excellent story where one of the central characters is the main character’s cat… named “N***er-Man” and, yes, that is the word you think it is.)

(Speaking of which, I suggest searching YouTube for a documentary named “H.P. Lovecraft: Fear of the Unknown” for a good introduction to who he was and why he’s so significant to the history of horror and science fiction. To oversimplify it, he changed the cultural zeitgeist, like all classic authors.)

The Colour Out of Space avoids this tendency toward xenophobia by taking place in the backwoods of New England with no human antagonists or henchmen. A minor thing compared to some of the other points I raise, but it definitely helps.

(At The Mountains of Madness also does well on this front, which makes sense as it was one of the last things he wrote before his untimely death from cancer, but it deserves its own blog post, if I find time.)

Second, Lovecraft’s fondness for uncertainty doesn’t throw the reality of the threat into question. For all its beautiful world-building, I wasn’t a fan of The Shadow Out of Time because, in the end, the main character is left unsure whether the experiences were real or a sign they’re relapsing into insanity. I found that a very unsatisfying anti-climactic experience. The Colour out of Space is, without a doubt, a real thing within in its setting.

(Not that I need a story to go that far. The Case of Charles Dexter Ward worked despite access to the horrifying setting being lost and all it took was having a second person to verify that the main character hadn’t imagined the whole thing.)

Third, and most distinctively, it truly feels like a cosmic horror story to me. While I tend to find Lovecraft’s writings fascinating, it’s rare for them to actually evoke a stirring of cosmic horror.

The problem is that, so often, his stories achieve the horror in a way which offers some kind of escape hatch.

For example, in The Dunwich Horror, the monster “is a Cosmic Horror”, but, in the end, it’s defeated using magic. While the twist is great, it still defuses the sense of horror for me for two reasons:

  1. If they can defeat the threat once, then they should be able to defeat it again.
  2. “Magic” is sort of a “get out of needing to understand free” card. By definition, magic is something you feel you can use without understanding it because, if you understood it, you’d just call it another branch of science.

(That said, Dunwich is an example of something The Colour Out of Space does not do. H.P. Lovecraft has a recurring fondness for ending the last paragraph of his stories on a strong, punchy revelation that drives home the horror of the story’s central concept. The Mound does it, as do Out of the Aeons, The Shadow Out of Time, Polaris and others. It’s something to look forward to, but The Colour Out of Space’s last sentence isn’t punchy in that way.)

The Shadow over Innsmouth is another example of that “defusing the horror” problem. While I enjoyed the certainty of having the story told in flashback after the government proved that the hero wasn’t crazy by coming in to address the problem, it also means that the problem has a sufficient solution to avoid having to deal with the lurking sense of horror. Sure, there’s a lurking threat to an individual, and there are horrifying elements, but it doesn’t have that inexorability that I admire The Colour Out of Space for.

Out of the Aeons is good, but, like The Call of Cthulhu, the threat is too distant. All physical evidence of the problem lurked under the ocean for eons before surfacing, and then returned to the ocean. It’s too easy to feel that it’ll probably stay under the ocean for eons more before coming up for another ultimately ineffective burp of momentary crisis.

Rather than in the middle of the sea, The Colour Out of Space takes place in in a relatively settled part of rural America, and the ending does a powerful job of driving home that the threat is still lurking.

The Call of Cthulhu also shares a similar problem to At The Mountains of Madness if you’re going for horror… the threats are too comprehensible. Yes, they’re powerful, but, despite R’lyeh’s non-euclidean geometry, it’s still just aliens and it feels like, with the march of science, we’re likely to have a way to effectively fight back by the time they surface. (That said, At The Mountains of Madness makes for a wonderful adventure story akin to something like Jules Verne’s Journey to the Center of the Earth [1] [2] [3] [4].)

The Colour Out of Space works so beautifully because it drives home that this is the realm of science, not magic, but, despite that, it utterly baffles the scientists called in to comprehend it. At the same time, the description presented to the reader is very convincing as something that is genuinely incomprehensible in its nature, yet comprehensible in its effects.

(As opposed to the sense of the author saying “take my word for it” that you feel with some supposedly genius characters, where willing suspension of disbelief is pressed into service beyond its ideal scope. It’s self-evident that characters like Sherlock Holmes, or L and Light Yagami from Death Note are geniuses and you feel that. Characters like Hermione Granger from Harry Potter, on the other hand, feel like the author is cheer-leading their intelligence, but it feels like they’re only intelligent when the author specifically tries to evidence it, rather than it being a property of their being which affects their thoughts and behaviour more subtly at all times.)

The other big reason The Colour Out Of Space works so well is that there’s no “defeat the threat” moment. It’s more like the Portal games in that the protagonists survive without triumphing. Thus, avoiding the need to demonstrate that humans can win.

All in all, I highly recommend that you find 80 minutes when you can listen to an audiobook and give it a listen. Of Lovecraft’s works that I’d read as horror, it’s certainly the best… and if you enjoyed it, try some of the others I named. Lovecraft’s writing style takes a bit of getting used to, being strongly based on the archaic writing styles used in the antique books he grew up reading, but hearing it in audiobook form helps a lot, and his stories have a very distinctive atmosphere to them once you get into them.

P.S. It also was the inspiration for the most memorable part of a piece of Ranma 1/2 fanfiction from the rec.arts.anime.creative era named Bliss by Mike Loader and Lara Bartram. Given that it is still a memorable story to me, and that it won second place in the April 1999 TASS awards and 10th place in the 1999 annual TASS awards, I may re-read and review that too.

UPDATE: You may also want to watch Fredrik Knudsen’s Down the Rabbit Hole livestream special, Lovecraft & Junji Ito.

Posted in Reviews | 2 Comments

A Verbal Middle Finger to WordPress

I just lost a fair bit of work from WordPress “helping” me, so I need to vent.

Think for a moment. What is the #1 thing that should define a good text editing experience… something so fundamental that you take it for granted… I’ll give you a hint. Autocorrect screw-ups are a weaker form of it failing.

Give up? It shouldn’t #$%*ing lose or corrupt your work!

WordPress deserves an award for how flagrantly it violates this principle and, if I can ever find the time to satisfy my obsessive dedication to preventing dead links, I’m going to switch to a static generator. (Partly because I blog infrequently enough that it feels like all I ever do is apply updates. Static HTML can’t have exploits in it.)

Let’s start simple. You might have noticed that I like to use all the semantic elements HTML gives me for making my posts as readable as possible, including tables and especially definition lists.

Have you ever tried to edit a table or definition list in WordPress? …especially with the legacy TinyMCE editor? There’s no option for them, so you have to switch into the raw HTML view and edit by hand.

That wouldn’t be so bad, except that there’s no “preview without publishing” option in raw HTML mode when you’re editing an existing post, except for switching back to WYSIWYG mode… and WYSIWYG mode normalizes your work into gibberish with no Undo option if you don’t get your HTML tags just right.

Better hope you made a backup of your HTML in an external text editor before previewing it.

…and that was before the new Blocks-based editor. If you’re using Blocks, you get a helpful little “Invaid HTML. What do you want to do?” popup with two buttons… they’re kinda cryptic and one of them basically means “convert from legacy TinyMCE block to custom HTML block without any apparent option to Undo”.

Better make a backup of that HTML, hit Reload to “undo”, and then paste the HTML into the TinyMCE/legacy block’s HTML mode to restore your work.

OK, so maybe I’m just too much of a technical writer and I should try using Blocks the way it was intended… I think you can guess by now that it’s no picnic either.

Want to edit stuff naturally in the new Blocks editor? Good luck. The interaction between cursor motion and selection means that trying to select a paragraph of scratch text without reaching for the mouse will snap the selection to the entire block, including text outside your intended span.

(Admittedly, this is something I dread having to do right in my own project, which also has editable text spanning multiple contenteditable elements… but at least I’m diligent enough to be aware of the problem and dedicated enough to try to figure out some kind of solution.)

…and that’s not even counting the idiocy that is how the Up arrow likes to jump from the second line of a paragraph block to the end of the previous block when you’re trying to position the cursor at the beginning of a paragraph. The Blocks editor is so braindead that it boggles my mind that nobody else noticed this.

(Nor the more minor annoyance that the heuristics for determining when to finalize one Undo action and start another are inferior to the desktop text editors I use, so I wind up undoing too much.)

…but I saved the worst for last. The piece de resistance that inspired this whole rant. I just lost a big edit to an old list post because, for reasons I can’t even fathom, switching from TinyMCE to raw HTML reverted it to the saved version. No “Are you sure?”. I just did what I always do to switch to HTML to fix up my markup and… that’s odd. Where are my changes?

OK. Maybe I should have un-published the post into a draft for a moment, even though I despise 404ing links even temporarily… the “Save Draft” button is unreliable. Wait too long with a WordPress tab open and clicking “Save Draft” will leave you stuck on “[Icon] Saving”.

Now, admittedly, I do, on rare occasions, have HTTP requests get stuck on setting up the connection. I’m assuming it’s some kind of packet loss during the initial handshake and TCP has to wait for it to time out and resend the packet. Maybe it’s that WordPress is much more prone to that somehow.

…but the usual solution is to click the Submit button again… a button that WordPress hides. …and there’s no “click Stop and then reload”, because WordPress has no analogue to the Stop button!

Wanna use the preview function but have your browser set to only allow new windows/tabs to be opened by an explicit middle-click or context-menu choice? Too bad. The Preview button can’t be middle-clicked.

I’m on WordPress because I’ve been on WordPress since December of 2004, and my workaround, when I don’t get lulled into using the UI the intended way, is to just copy my post into gVim, edit it there, and then copy it back.

Given that the “can’t trust WordPress to protect your text” aspect of this seems fundamental to how the authors of WordPress approach the interaction between author, text, and tool, I think you can see why, as soon as I have time, I want to translate all my posts into Markdown, switch the site to something like Pelican, and say good riddance to bad rubbish.

UPDATE: Apparently it’s also prone to forcing a “convert to visual and resolve broken HTML” when losing focus so I can copy-paste text from another tab. Ffffff……

Posted in Geek Stuff | Leave a comment

Novel – World War: In The Balance

Now for a book that has been mentioned before as a source for some of the fanfics I’ve reviewed:

World War: In The Balance by Harry Turtledove

The basic concept is that, during World War 2, a bunch of reptilian aliens who call themselves The Race arrive with plans for conquest. However, unlike in the usual old tropes where aliens are either hyper-competent or completely incompetent, these aliens were blinded by their preconceptions. More specifically, while technologically superior, they make Imperial China’s cautious, bureaucratic approach to progress look reckless and expected to find us still fighting with knights on horseback.

The main thing I can say about this story is “slow but steady”. Aside from the concept, it’s not a story characterized by creative ideas that make you stand up and take note, or graced with a plethora of quotable passages, but, if you give it time to grow on you, it’s like a nutritious but familiar staple. It’s something I find enjoyable and satisfying, but it took until around chapter six for me to really feel that it had consistently made its characters interesting to read about and, to continue the food metaphor, it’s tasty and something to look forward to, but not exciting like going to a restaurant.

I think the problem is that it feels like Turtledove was determined to make the story about his characters first and foremost, even if that meant turning down other opportunities presented by the concept. …and the characters are the kind who start OK and grow into being good, rather than starting out good and growing into being amazing.

The worst part of that is probably how much time the first chapter spends on a couple of American baseball players when it was supposed to be hooking my interest. They just don’t start to get interesting until after the war comes to them and, even then, they’re potentially the least interesting of the characters. My best guess is that they’re supposed to be relatable, but I’m not American, not into sports and, besides, this is supposed to be a blend of sci-fi and historical fiction.

It’s technically a first contact story, but it doesn’t embrace the reactions by one group to revelations about the other in the way first contact stories do when they truly commit to “being about” that. (Compare Turtledove’s short story The Road Not Taken, which is sort of the progenitor of World War, for something which does embrace that aspect… or the web-original series The Deathworlders. Having aliens who are so behind us in everything except faster-than-light travel is intrinsically more fantastic, so it takes less effort to make engaging.)

It’s technically historical fiction yet I find Eric Flint’s writing much more engaging on that front. For all the moments which feel satisfying, like a village in Communist China celebrating an air raid destroying only the compound where the corrupt officials live or the banter between two British radar technicians, or an interaction between a white man and a black man in the U.S.A. in the 40s in the south, it feels diluted compared to Flint.

I think the problem is that the story is spending a lot of “screen time” trying to make the characters relatable when it should be recognizing that it already has that and focusing on what’s different instead. The whole point is to show the unfamiliar in a way that connects it to the familiar, not vice-versa. (Compare Eric Flint’s 1632 or Leo Frankowski’s The Cross-Time Engineer.)

You clearly love history, Mr. Turtledove. I understand not feeling that “jumping right into the action” is appropriate for the tone you want to establish, but literature is the interesting parts of life condensed. If you want to show a realistic pace of events, then you need to make up for it in the chapters before your characters have proved themselves interesting by packing in other interesting “little-known but obvious in hindsight” details to make up for it. That’s one of the meanings of the phrase “write what you know”. “Write what you know, because others don’t know it and find it fascinating”.

The baseball players are a particularly bad example of this, since I found them dull aside from the moment where one was able to translate “Espíritu Santo” because Spanish was close enough to Italian. I would have much preferred it if the other “ordinary human” groups were given more time, like the German soldier on the eastern front, or the jew in the Polish ghetto. They feel like they have more relevance. (Even when I’d rather be reading scenes of the aliens reacting to Earth or humans reacting to them, like the ground-based radar operators who first see evidence of them or the pilots who survive their first attack. That’s the most concentrated expression of what a first-contact story revolves around.)

It doesn’t help that Turtledove’s style isn’t very punchy, with the most memorable/quotable line I found being when one German soldier reassured another who had jumped at a sound from the sky by saying it was “Just one of the Ivans’ flying sewing machines”… which is amusing mental imagery.

(Sure, there was only one major quotable passage in Mother of Demons, but that book had a lot of philosophical meat to it and introduced me to the concept of r/K selection theory… things which this story lacks.)

That said, I do like the recurring theme that humanity manages to hold on by the skin of our teeth because we were damn lucky. The Race commander decides to start out with EMP weapons to fry all our solid-state circuitry… during the brief window of a few decades when we were using vacuum tubes. The Germans manage to take out the landing ships with the main stockpile of nuclear weapons… with a giant piece of artillery (that really existed) that fires shells too robust to be taken out by their interceptor missles and so archaic that they never anticipated it.

I also like the recurring focus on a world caught wanting to cling to old national disputes and racial identities in a situation where they must come together to survive a truly alien common adversary. …and the way that it manages to work prejudices and slurs into dialogue in a fashion where it’s not objectionable, it’s being honest about how such people would have behaved in that era.

That evolves nicely and, around the mid-point, you start to see characters, both human and Race, beginning to question things about their worldviews that they previously took as gospel, and starting to relate to each other.

It just feels like, in its weakest moments, the writing is… not dry, but too wedded to reminding the reader of how mundane things continue to happen in war, in a way which just comes across as feeling either slow-paced or padded.

It doesn’t help that the book is like the first volume of a three-part Lord of the Rings set… it ends at the end of an arc, but that doesn’t even conclude that subplot, let alone the arc the earlier chapters primed you to expect the book to follow. For a book that was so slow to get started, and so reluctant to feel punchy once it did, having an arc so long that it takes multiple volumes feels like Turtledove’s editor was asleep on the job.

(Bear in mind that I’m not saying all that content needed to be cut, but, if Turtledove didn’t want to tighten it up, I’d probably suggest doing something more like what Anne McCaffrey did with most of her Pern books, and writing multiple books that explore the same period of world-changing events by following different characters.)

All in all, I’d give it a 4.0 out of 5. It’s solid, well-done writing, it stays engaging once it gets past that initial hump, and it’s clear that Harry Turtledove knows his stuff when it comes to history, but I expect so much more from a professionally published book.

Posted in Reviews | Leave a comment

A simple Clipboard/Drag-and-Drop Test Tool

While working on one of my projects, I found myself needing to inspect a raw, “all offered mimetypes” view of what was on the clipboard.

I remembered that I used to do so using some PyGTK demos, but they were never as feature-complete as I wanted, I don’t know if they’ve been ported to GTK 3.x, and, besides, I use Qt these days.

…so, I wrote the utility I wanted and here it is in case it helps anyone else. It does clipboard, selection, and drag-and-drop.

Posted in Geek Stuff | Leave a comment

Embedding the DPMI Extender for your Open Watcom C/C++ EXE (And Related Knowledge)

NOTE: Scroll to the appendix at the bottom If you’re using DJGPP, Free Pascal, or Free Basic. They use a different and incompatible DPMI host.

Sooner or later, everyone who plays around with 32-bit DOS programming using Open Watcom C/C++ is probably going to see one of these errors when they try to run their programs:

Stub exec failed:
\dos4gw.exe
No such file or directory
Can't run DOS/4G(W)

I don’t know if there are other causes for the second version, but the first one is pretty clear about what went wrong. You forgot to copy dos4gw.exe into the same folder as your EXE file… but why is this necessary?

DOS Extenders

If you’re a novice to the world of DPMI extenders, this comp.os.msdos.programmer post provides a good overview of how it all fits together and this other post lists noteworthy alternative DPMI hosts.

The important thing to understand is that DOS/4GW is a special bundle version of DOS/4G. It was used so widely back in the day because it was hundreds if not a thousand dollars cheaper than the alternatives, but it was designed to upsell programmers with artificial restrictions… and one of those restrictions was having dos4gw.exe in a separate file.

Luckily, Open Watcom C/C++ supports other extenders which can be bundled into the EXE file. One of them (DOS/32A) is even drop-in compatible with DOS/4GW to the point that you can swap it out without recompiling. (Good for lifting restrictions and resolving bugs in closed-source software.)

The extenders which are bundled with Open Watcom C/C++ (and the meanings of their names) are:

  • DOS/4GW by Tenberry Software, Inc. (DOS on 4Gig, Watcom Bundle Edition. The original site is gone now that the author has passed away, and the Watcom manual was always a paid add-on, but it’s API-compatible with DOS/32A and there are bundled and third-party docs.)
  • DOS/32A by Narech K. (DOS on 32-bit, Advanced Version)
  • PMODE/W by Thomas Pytel and Charles Sheffold (Protected Mode, Watcom Edition)
  • CauseWay by Devore Software & Consulting (I couldn’t find an explanation of the name’s origin. The original site is gone, but there’s a manual bundled with Open Watcom C/C++.)

(DOS/4G was named after the memory limit for a 386, in contrast to Tenberry’s other product, the 286-compatible DOS/16M.)

Other extenders people have successfully used with Open Watcom C/C++ include:

  • D3X by Daniel Borca (DOS 32-bit Extender, The original site is gone, but it’s in the Wayback Machine including the download)
  • DPMIONE (GitHub) by Sudley Place Software (DPMI 1.0 implementation. In fact, it claims to be the only DPMI 1.0 implementation complete enough that it can substitute for Windows 3.1x’s internal DPMI extender… though functionality will still be reduced.)
  • HX DOS Extender (mirror, Dr-DOS Wiki) by Andreas “Japheth” Grech (Best option for writing Win32 console applications or making limited use of the Win32 GUI APIs and having your program run under plain old DOS.)
  • PMODE/WI by Thomas Pytel and Charles Sheffold (A cut-down version of PMODE/W for demoscene applications where compactness is more important than features and it’s acceptable for problems during development to be more cryptic.)
  • WDOSX by Michael Tippach (Wuschel’s DOS eXtender)

I haven’t tried it, but, as described, it sounds like an explicit design goal of HX DOS Extender is to allow you to write a Win32 console binary (which would run on 64-bit Windows 10), and then stub it with HX and have it simultaneously be compatible with platforms like FreeDOS, MS-DOS 6.22, and DOSBox.

Note that, for WDOSX, if you want to have the stub bundled in automatically as part of your linker script rather than using its stubit.exe, you’ll need to run stubit -extract to extract the requisite wdosxle.exe file. (See DOC/README.TXT in the WDOSX distribution archive for more details.)

One other bit of trivia: If you look in the Open Watcom manuals and see mention of Phar Lap… that’s the company that co-designed VCPI with Quarterdeck and produced the expensive proprietary DPMI extender that Microsoft licensed for Microsoft C/C++… it’s named after a famous race horse.

Before I demonstrate how to use them, I’d first like to address a few potential tripping points I encountered while preparing this:

Potential Errors

undefined system name: ...

System names are defined in the wlink.lnk file in the bin* folder containing the wlink or wcl386 command you’re running. and the version of wlink.lnk bundled with Open Watcom v2 has a few more of them than Open Watcom v1.9, such as the one for PMODE/WI.

Check wlink.lnk to see what’s available and either choose one of the ones listed in there, edit it to add one, or upgrade Open Watcom. (Note that I’ve had Open Watcom v2 builds produce broken .com files, so don’t upgrade blindly.)

This is a ... executable when trying to run the EXE

If you saw one of these messages when trying to run your EXE…

This is a CauseWay executable
This is a DOS/32 Advanced DOS Extender (LE-style) executable
This is a DOS/32 Advanced DOS Extender (LX-style) executable
This is a DOS/32A DOS Extender w/ Standard stub (LE-style) executable
This is a DOS/32A DOS Extender w/ Standard stub (LX-style) executable
This is a DOS/32A DOS Extender w/ Configurable stub (LE-style) executable
This is a DOS/32A DOS Extender w/ Configurable stub (LX-style) executable
This is a PMODE/W executable
This is a PMODE/WI executable

… it probably means that your compiler emitted one of the following warnings:

Warning! W1093: cannot open cwstub.exe
Warning! W1093: cannot open dos32a.exe
Warning! W1093: cannot open stub32a.exe
Warning! W1093: cannot open stub32c.exe
Warning! W1093: cannot open pmodew.exe
Warning! W1093: cannot open pmodewi.exe

…and fell back to linking the default stub for non-MZ .exe files.

To solve the problem for all the listed stubs except pmodewi.exe, you need to either add your binw folder to the PATH (but not before binl) or copy or symlink them from the binw folder to the binl folder. (I chose the latter since I didn’t want to throw an arbitrary pile of files into my PATH and wlink‘s manual doesn’t mention any other variable it check for a search path.)

This bit of shell script should do the trick when run from inside binl:

for stub in dos32a.exe stub32a.exe stub32c.exe pmodew.exe; do 
	ln -s ../binw/$stub . 
done

As for PMODE/WI (the cut-down demoscene version of PMODE/W), having a predefined system for it doesn’t mean that it’s bundled with Open Watcom C/C++, so you’ll need to download it yourself and unpack it into binw or binl as appropriate. (Renamed to all-lowercase if you’re on Linux.)

Stub Tricks

An interesting side note is that the This is a ... executable messages in Open Watcom’s default EXE stub are actually dynamically generated from the osname values in the wlink.lnk file in your binw or binl folder. (“This is a Windows executable” made general across all outputs that must have a DOS MZ-format EXE at the beginning.)

That means that, if you edit the appropriate wlink.lnk file to make a copy of the entry for an extender that supports taking the binary as an argument (eg. dos32a):

system begin joke
    option osname='high-security process. You do not have permission to run this'
    libpath %WATCOM%/lib386
    libpath %WATCOM%/lib386/dos
    libpath %WATCOM%/lib386/l32
    format os2 le
end

Then wcl386 -l=joke whatever.c will produce a whatever.exe file that says “This is a high-security process. You do not have permission to run this executable” if you try to run it in DOSBox, but runs as expected if you run it as dos32a whatever.exe instead.

…and since the op stub=dos32a.exe line uses the same mechanism as /STUB for setting the EXE stub in PE executables. That means that the same “does one thing in DOS and another in Windows” behaviour of PE EXEs is also available to DPMI EXEs with a slight twist.

(Not that you need to edit wlink.lnk to change the stub. The proper way is to create a file with a name like whatever.lnk, put your op stub=decoy.exe or option stub=decoy.exe in there, and then call wcl386 or wlink with @whatever.lnk as one of the arguments.)

If you pick a DPMI extender that supports taking the EXE as an argument, like DOS/32A, then you can compile decoy.c as a real-mode DOS EXE with wcl -l=dos decoy.c, specify op stub=decoy.exe in your linker script, and have a DOS EXE which behaves one way when executed directly and another when executed as dos32a whatever.exe. (Though, since the DPMI extender uses LE for the 32-bit part, I think it’ll ignore the decoy and run the real thing every time if you run it under OS/2.)

QEMM 97 actually uses this mechanism to put the DOS installer and the launcher for the Windows installer in the same EXE file, so the installation instructions can be simpler.

The Point (Finally)

DOS/4GW is actually the only DPMI extender listed in wlink.lnk that doesn’t support being bundled and, in fact, the only other one that even supports relying on an external helper is DOS/32A… the extender explicitly designed to be usable by end-users as a drop-in replacement for closed-source DOS/4GW applications.

Under typical circumstances, where you’re using the wcl386 build driver, all you need to do is specify the correct target system via the -l=whatever option to wcl386 and you’re good to go.

The following -l= values produce an EXE file which does not require an external DPMI extender EXE:

  • causeway
  • dos32a
  • pmodew

…while the following ones do require an external helper:

  • dos4g
  • stub32a
  • stub32x
  • stub32ac
  • stub32xc

Don’t believe me? Create a temporary DOSBox profile with a conf file that mounts C: and launches hello.exe, create a hello.c containing a “Hello, World!” in it, source owsetenv.sh, and then run this bit of shell script:

for EXTENDER in dos4g causeway dos32a dos32x stub32a stub32x stub32ac stub32xc pmodew pmodewi; do
	print " === $EXTENDER === "
	rm hello.exe 2>/dev/null
	wcl386 -q -l=$EXTENDER hello.c
	dosbox -conf dosbox-0.74.conf &>/dev/null
done

For more advanced situations, you’ll want to read the Open Watcom manuals, but the following example should get you started:

wcc decoy.c
wlink file decoy.o system dos  # produces decoy.exe
wcc386 hello.c
wlink file hello.o system dos32a  # produces hello.exe
wlink file hello.o system dos32a option stub=decoy.exe name decoyed.exe

wlink takes a list of un-quoted directives like you’d put in the aforementioned whatever.lnk and/or an @whatever.lnk file reference like can be passed to wcl386. To invoke wlink‘s help, the command to use is wlink -?.

As for other DPMI extenders, from looking at the system definitions, it seems pretty simple to add definitions for them. Just make a copy of an existing definition and set the appropriate format and stub values.

Appendix: CWSDPMI/GO32v2

GNU-ecosystem DOS compilers like DJGPP, Free Pascal, and Free Basic handle DPMI differently enough from Open Watcom C/C++ that their DPMI hosts are not inter-compatible and the DPMI host they come with is called either CWSDPMI or GO32v2, depending on who you ask. (Though everything I could find on the matter says they’re the same thing.)

Like DOS/4GW, the default setup for CWSDPMI requires an external DPMI host binary and, like with DOS/4GW, it’s possible to replace it with a fully embedded stub… but CWSDPMI offers that stub for you. I assume it’s about saving disk space by putting a single copy of the actual CWSDPMI binary in your path to be shared by all the separate EXE files that make up your compiler’s toolchain.

In this case, the process is quite simple:

  1. Use the exe2coff binary from DJGPP (which is also included with DJGPP cross-compiler builds) to strip the stub off your EXE file.
  2. Concatenate cwsdstub.exe onto the beginning of the resulting bare binary:
    1. POSIX: cat cwsdstub.exe my_exe.bin > my_exe.exe
    2. DOS/Windows: copy /B cwsdstub.exe+my_exe.bin my_exe.exe

…and yes, that COPY /B command should work on any old DOS. I have a copy of The MS-DOS Encyclopedia, from 1988 (Microsoft’s complete and unabridged must-have book for DOS 3.3 developers) and it doesn’t say anything about /B or + being added after the command debut’d in DOS 1.0.

There aren’t as many alternative DPMI extenders for this ecosystem… probably because it came about after there was no longer a market pressure for everyone to try to sell DPMI extender licenses for money, but there is a port of PMODE/W called PMODE/DJ.

Posted in Retrocomputing | Leave a comment

Helper for Finding Python 2.x Scripts Still To Be Ported

I’ve written a lot of little scripts over the years, and it’d be nice to be able to migrate to Kubuntu 20.04 LTS when it comes out, so I decided to write a little helper to find creations of mine that still depend on Python 2.x as quickly as possible.

The script I came up with produces output like this:

ERROR:   NO PY3: /home/ssokolow/bin/k3b-rm.py
ERROR:   NO PY3: /home/ssokolow/bin/mergemove.py
WARNING:  NO #!: /home/ssokolow/bin/mpv.py
ERROR: NOT UTF8: /home/ssokolow/src/Human Sort.py

Perfect for running with a command like ~/src/audit_python2.py --ignore read-only ~/bin ~/src 2>&1 | tee ~/python2_potentials.txt

It works by applying a few simple rules:

  • If a folder is named .tox or contains a bin/activate script, skip it to avoid flooding the results with virtualenv files.
  • If the file has a .py or .pyw extension but no shebang, print a WARNING: NO #!.
  • If the file has a shebang line containing python but not python3, print an ERROR: NO PY3.
  • If reading the first line of the file fails with a UnicodeDecodeError and the file has one of the aforementioned extensions, print an ERROR: NOT UTF8.
  • If reading the file fails for any other reason than UnicodeDecodeError, print a general ERROR: READ ERR.

The logging level is configurable with repeated -v and -q arguments and there are liberal INFO and DEBUG messages to detail how it’s traversing the filesystem. It also traverses in sorted order to ensure consistency between runs and folders or files can be ignored using the -i/--ignore option to skip stuff like read-only Git checkouts.

If this sounds useful to you, it’s up on GitHub Gist. For lack of a better name, it calls itself “Python 2.x Auditor”.

Posted in Geek Stuff | Leave a comment

New QuickTile Release… Finally!

For those who follow my programming projects, you may be familiar with the name QuickTile. It’s my most popular project and, in short, it adds hotkeys to your existing X11 window manager to make it easy to tile windows in ways fancier than they usually offer.

Long story short, it fell into neglect for a while for a perfect storm of reasons:

  • Needing to migrate from Python 2.x to Python 3.x at the same time as needing to migrate from PyGTK to PyGObject and from ePyDoc to Sphinx
  • Something about QuickTile causing PyGObject’s PyGTK porting helper to segfault
  • QuickTile relying on GDK stuff that got replaced with PyCairo stuff that seemed to switch from one set of bugs to another as I upgraded it
  • No automated tests
  • Some other problems in my life that made it hard to work on my hobby projects at all

In even fewer words, it was a mess… but not anymore.

I’d like to announce that QuickTile 0.4.0 is now out. (Those who like emjois may insert an appropriate “yay” here)

It has the following following advantages:

  • Completely ported to GTK 3, and with a lot of internal reworking to make the most error-prone bits of the code easier to maintain and improve.
  • A spiffy new website with the demonstration animation I always wanted to get around to making and a detailed manual. (Including explanatory illustrations for all the tiling commands.)
  • A unit test suite that, while nowhere near complete, has full coverage for the rectangle-juggling code that’s most likely to hide confusing bugs. (Great for motivation.)
  • Finally, proper support for dealing with panels that don’t span an entire edge of the desktop! (I had to write a ton of custom rectangle-juggling code, but I did it.)
  • New Ctrl+Alt+Shift+… default hotkeys for the versions of the tiling commands which move windows without resizing them. (If you’ve got an existing quicktile.cfg, you’ll need to add these manually.)
  • Initial support for building a .deb package, though not yet something I publicize in the manual since it’s in such an early state.
  • A few other changes to how the commands behave to make things more consistent. (See the ChangeLog file for details.)

So, if you’re a Linux user (or just curious), check out the new website and enjoy.

P.S. I have some other projects which need TLC just as much as QuickTile did, so I may let it go quiescent for a month or two now that it’s no longer the most urgent thing to address, but I do intend to come back and start checking off the other things on the issue tracker.

Posted in Geek Stuff | Leave a comment