Resources for Reverse-Engineering 16-bit Applications

See Also: My list of tips and resources for writing new DOS and Win16 apps

While offering some advice, I got a little caught up in the research I was doing, so here’s a list of the resources I found for picking apart 16-bit x86 applications.

Reference Materials

• Reversing a 16-bit NE File Part 1: Clumsy and Unprepared
• ScummVM Wiki – HOWTO-Reverse Engineering
• Reverse Engineering for Beginners by Dennis Yurichev
• Microsoft KnowledgeBase Archive

Unpackers/Unprotectors

In the days of DOS and Windows 3.x, executable packers and code protectors like PKLite and PackWin were a common means of saving precious disk space, as well as deterring casual inspection of an executable by novices with tools like DEBUG.COM.

They still see use to this day, but exponential decline in the cost of storage space, combined with the risk of false positives from virus scanners, has reduced the demand for this sort of software. (Aside from JavaScript minifiers, for which an effect similar to a decompiler can be achieved using a code beautifier.)

When it comes to reverse-engineering, packed/protected executables must be unpacked/unprotected before utilities which operate on the on-disk form of the program will return useful results.

UNP for DOS

A now-unmaintained but open-source utility which will handle the vast majority of packed executables for DOS and Windows 3.x.

Universal Extractor for Windows

A tool which combines code and algorithms from a great many unpacking tools not covered by UNP into one convenient package… including various tools for unpacking installers without running them.

UPX for all major platforms

The most popular executable packer today and also capable of unpacking its own creations, assuming they haven’t been modified to obscure their nature.

archive of exetools.com

the Wayback Machine’s October 2002 archive of this site contains a huge list of more esoteric unpacking tools.

The ‘Executable compression’ section of the Archive Team wiki

A list of DOS (and some Windows 3.1x) executable packers, some with the ability to unpack what they’ve packed.

Decompilers

Decompilers attempt to retrieve something higher-level than assembly language from a program.

This may be the normal state of things if the language doesn’t compile to machine code or it may be a convenience accomplished by looking for patterns of machine instructions that are known to come from specific higher level constructs like for and function calls.

(For languages which compile to machine code, you get something half-way between assembly language and the original source code, because the compiler threw out various higher-level details which would be needed to perfectly reconstruct the original source.)

Reko for Windows

According to the old SourceForge page (which has screenshots), this open-source decompiler’s list of noteworthy supported formats includes MS-DOS, Win32 and AmigaOS.

The release notes on GitHub mention adding support for NE-format EXEs (Windows 3.x and OS/2), SEGA Genesis, SEGA Dreamcast, NeoGeo, partial support for Atari-TOS, and improving support for MacOS Classic (though I couldn’t find the initial mention of adding it).

DoDi’s Visual Basic 3/4 Decompiler for Windows

The site refers to this freeware tool as a Visual Basic 4 decompiler, but the tool is originally for decompiling Visual Basic 3 and, if the newer version is limited to Visual Basic 4, the site also provides links for earlier versions specifically intended for Visual Basic 3.

DisC – Decompiler for Turbo C 2.0/2.01 (source)

A decompiler specifically for programs built using Borland Turbo C 2.0 and 2.01 (which is offered as a free download by Embarcadero).

Disassemblers

Disassemblers are what you need if you want to inspect the code in its entirety when it’s not running

They’re generally more reliable than decompilers, and preserve the correspondences to the machine-code form necessary to design patches.

Compared to debuggers, they tend to have more advanced searching features (to make up for the lack of ability to find things by running the code until you hit a breakpoint), but can’t deal with especially dynamic code where the behaviour is most easily understood by running it.

All Formats

semblance (source)

An open-source, non-interactive disassembler for MZ (DOS), NZ (Win16), and PE (Win32) executables.

Doesn’t explicitly mention which platforms are supported, but it works on Linux and I suspect it’ll build on any POSIX-compliant platform (eg. MacOS, Cygwin, etc.).

TatraDAS (GUI for Win32, CLI for Linux)

Open-source disassembler with support for MZ, NE, PE, and .com executables. Unlike semblance, doesn’t seem to dump non-code segments.

CLI version is claimed to be OS independent, but I haven’t verified that claim.

DOS MZ Format

IDA Pro Freeware v5.0  for Windows

It’s hard to beat the IDA Pro disassembler and the 5.0 release, which is free for non-commercial use, can disassemble DOS MZ and 32-bit Windows PE binaries.

The ScummVM developers have requested and received permission to host a copy to ensure it will always be available to people wanting to add new game engine re-implementations to ScummVM. If my link breaks, check the HOWTO-Reverse Engineering page on the ScummVM wiki.

In my testing, this ran just fine on non-Windows platforms under Wine. (See also PlayOnLinux and PlayOnMac as easy ways to manage Wine versions and application prefixes.)

Windows 3.x NE Format

URSoft W32Dasm (A.K.A. Win32Dasm) for Windows 9x

Don’t let the name fool you. This tool disassembles both 16-bit and 32-bit Windows EXEs and, from what I remember in my high school years, it was by far the most comfortable option around.

Sadly, it appears to have been the result of a one-man operation with a website run on his ISP’s hosting, and it vanished from the web some time between February 2002 and May 2003, with its most recent version listing a 1998 copyright date in the About box.

When I was poking around, I found it not at all difficult to find the most recent registered version, but I couldn’t find a citation to back up the claims made by some sites that the last release is freeware that never received a patch to remove the warning about not sharing it, rather than so-called abandonware. (It could go either way. There is precedent for official freeware re-releases with warnings left intact and the author didn’t bother to update the copyright date in the image in question from 1995 to 1998.) As such, I will not provide a link to it.

From what I remember, it can be finicky about which Wine versions it will work under. (It won’t crash… it’ll just fail to load its desired monospace font and display useless placeholder glyphs.)

Windows CodeBack [2] [3] for DOS 3.x+

The Wine developer wiki recommends this disassembler for Win16 executables.

If my links ever go dead, it won’t be difficult to find as wcb105a.zip in various archives of old shareware and freeware. Like IDA Pro Freeware, it’s free for non-commercial use, and the features withheld for paying users are ones you’re not likely to need.

However, my quick tests seem to indicate that it’s a purely non-interactive, command-line disassembler.

In my testing, this ran perfectly well in DOSBox 0.74.

MBBSDASM for .NET Core

An open-source, non-interactive disassembler for NZ (Win16) executables with supplementary features for Major BBS modules.

Debuggers

Debuggers are what you need for inspecting code while it’s running. They take some getting used to, but their versatility is unmatched.

More advanced debuggers even support “reverse debugging” allowing you to get the program to a point of interest, like the manifestation of a bug, then rewind its execution to explore how you wound up there.

DeGlucker for DOS

A protected-mode debugger which claims to be more powerful than Turbo Debugger and Soft ICE. Version 0.5 alpha, released in May of 2000,  is closed-source, but freeware.

Also offered is version 0.4 from January 1999, released with TASM x86 assembly source code under informally stated “do what you want but leave the attribution” terms after the original author no longer had time to maintain it.

Insight for DOS

A GPL-licensed real-mode debugger that gets suggested frequently. I don’t know whether it has any advantages over DeGlucker aside from being open-source and being usable on older, more limited hardware.

Open Watcom C/C++ for DOS and Windows

Open Watcom includes a complete set of tools, including DOS and Windows debuggers (WD.EXE and WDW.EXE) and a resource editor capable of poking around inside EXE files built by other compilers (WRE.EXE).

While Open Watcom hasn’t made use of the support it inherited for Windows 3.1 builds of the installer, all installers will install the DOS and Windows tools (even the Linux installer includes them), so you can use the DOS installer under Windows 3.1 as long as you don’t mind manually setting up the Windows bits.

WineDbg for POSIX (Linux, MacOS, etc.)

Wine‘s built-in debugging support can be used in two different ways:

• Using the built-in command-line interface
• Using an open-source graphical frontend like gdbgui, DDD, or kdbg via the gdbserver protocol.

DOSBox Debugger

The DOSBox emulator contains a debugger which can be enabled in special builds.

Win32 builds are provided, as well as instructions for making debug-enabled builds for other platforms.

Miscellaneous

Radare2

Radare2 is hard to classify because it does so much. Wikipedia summarizes it as “a complete framework for reverse-engineering and analyzing binaries”.

It supports DOS, Win32, Java, Gameboy, Gameboy Advance, Nintendo DS, Nintendo 3DS, Commodore VICE, WebAssembly, Android, and XBox binaries, among others.

According to the website, it can disassemble, assemble, debug, patch, be scripted in multiple languages, run on all major platforms, and more.

You’ll probably want to use the Cutter GUI with it.

Pixmap Prober

A quick little utility I hacked up for looking at a binary file’s contents as pixels to see if there are any patterns that don’t readily show up in your favourite hex editor.

P.S.

If you’re looking to poke at Win9x-era stuff too (eg. Win32 PE binaries), some of the listed tools support them and here are some additional tools you could try which were omitted from the previous lists for supporting neither MZ nor NE:

PE (Win32/Win64)

• x64dbg (Open-source debugger)
• OllyDbg (Shareware debugger, free registration but not needed for full function.)
• Snowman [2] (Open-source decompiler, can run as an x64dbg plugin)
• Boomerang (Open-source decompiler)
• IDA Pro Freeware 7.0 (Non-commercial-only like v5.0. Supports 64-bit PE but drops MZ. Also adds Linux and MacOS versions.)
• Ghidra (Open-sourced by the NSA. I’m unclear what formats other than PE are supported, but I’ve seen it described as being like IDA Pro but more free or like Radare but more mature.)
• Borg Disassembler (info claims it to be freeware, but source is offered too)
• Frida (Dynamic instrumentation framework, describes itself as “Greasemonkey for native apps”)
• Binary Ninja (the cloud version is free, but you have to upload the binary to them)
• Destroying x86_64 instruction decoders with differential fuzzing (Article on how and why reliably disassembling x86-family machine code has become such a hard problem)

Macromedia Director and Flash

• Swifty Xena Pro for Windows (Extract SWFv4-6 and Macromedia Director v6-7 resource bundles from single-EXE projectors)
• dirOpener (extract resources from Macromedia Director resource bundles (v8.5 and earlier). The original site is only on the Wayback Machine and they pulled the download links, but you can still use it as reference for the filenames to google. May need the SWA Decompression Xtra from the full Shockwave Player installer for some content.)
• JPEXS [2] (a complete open-source reverse-engineering suite for Flash files)

Resource Rippers

• Ripper v5.00 by J0nas for DOS (A multi-format freeware tool for extracting images, music, etc. from DOS-era resource bundles. Also mirrored locally.)
• Multi-Ripper v2.60 by TWT for DOS (A multi-format shareware tool which may be useful for things that Ripper V5.00 fails on. Also mirrored locally.)

Resources for Reverse-Engineering 16-bit Applications by Stephan Sokolow is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.