How to easily jekyll-import a WordPress site hosted on NearlyFreeSpeech.NET

In the time since I last tried migrating my blog to Jekyll, it has gained an official WordPress importer. Unfortunately, that importer requires direct database access and NearlyFreeSpeech.NET doesn’t have the requisite Ruby dependencies.

…so I decided to hack around that. It’s much easier to run the importer locally, but NFSN doesn’t expose their database servers to the world at large… SSH port forwarding to the rescue.

  1. Install the development headers for libmysqlclient
  2. gem install jekyll-import sequel mysql2 unidecode`
    Note: Make sure it does complain about `htmlentities` being missing or you’ll get a broken import with things like the / in </a> escaped.
  3. Open up two terminal windows
  4. In one terminal, adjust this SSH command and run it to forward the MySQL client port
    ssh -L 3306:YOUR_DSN_HERE:3306 USERNAME_SITENAME@ssh.phx.nearlyfreespeech.net
  5. In the other terminal, adjust and run this jekyll-import command to dump the blog
        ruby -rubygems -e 'require "jekyll-import";
            JekyllImport::Importers::WordPress.run({
              "dbname"   => "___________",
              "user"     => "___________",
              "password" => "___________",
              "host"     => "127.0.0.1",
              "socket"   => "",
              "table_prefix"   => "wp_",
              "clean_entities" => true,
              "comments"       => true,
              "categories"     => true,
              "tags"           => true,
              "more_excerpt"   => true,
              "more_anchor"    => true,
              "status"         => ["publish"]
            })'

    Note: Make sure `host` is set to 127.0.0.1. If you use localhost, it will try to use a UNIX domain socket rather than a TCP socket and fail.

  6. Exit the ssh connection to close the tunnel and start fine-tuning the exported content.

Unfortunately, I found jekyll-import insufficient (eg. it made no attempt to preserve permalinks) so I started investigating alternatives. I discovered that Pelican (a Python alternative to Jekyll) has some very appealing plugins (eg. [1] [2] [3] [4]), so I’ll probably use that instead.

Posted in Geek Stuff | 1 Comment

“Tasks Due Today” popup for TaskWarrior

I generally like using TaskWarrior to manage my tasks but I’ve found that, for daily recurring stuff, it just doesn’t feel quite right.

As such, I’ve put together a little tasklib-based glue script, runnable as a user cronjob if you set DISPLAY, which presents tasks due before the end of today (defined as 1:00:01 AM, local time, to work around a quirk caused by TiddlyWiki’s ignorance of daylight saving’s time) in a zenity checklist dialog.

It looks like this:

screenshot

Clicking OK will commit the changes and then redisplay the dialog with refreshed content while clicking Cancel will quit the tool. The redisplay behaviour, window size, warning period, and zenity command can be customized by editing constants at the top of the file.

It’s available on GitHub Gist, as embedded below and here’s the download link. A requirements.txt is included which will install all of the dependencies except zenity.

Could not embed GitHub Gist acd90d9aea74a6a50995: API rate limit exceeded for 208.94.116.209. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)
Posted in Geek Stuff | Leave a comment

A Lightweight Low-Battery Warning Monitor for GTK+ 3.x desktops

Today, I decided to help someone out and, as is often the case with coding, I got carried away.

I set out to help fix up a shell script but, instead, I decided to “do it right” and produced a simple Python script which…

  • Uses a combination of GLib.timeout_add_seconds and UPower events to react to power supply changes immediately while still putting a limit on how often battery percentage changes should be able to cause wakeups.
  • Uses GStreamer and notify2 to provide audio and visual low-battery warnings without the weight and delay of calling subprocesses.
  • Is compact and easy to customize because it’s written in Python

By design, it doesn’t really have much of a UI (just a persistent notification popup and alert sound that come and go in response to changes in power state), but here’s what the popup looks like when using xfce4-notifyd:

It does require the 1.0 UPower API for its “Display Device” abstraction, so an 0.99.x-series version of UPower or newer is required, but for people on a new enough Debian/Ubuntu/Mint release (Vivid, in Ubuntu’s case), the dependencies can be installed as follows:

sudo apt-get install python-notify2 python-gi gir1.2-upowerglib-1.0 gir1.2-gstreamer-1.0 gir1.2-gst-plugins-base-1.0

It hasn’t been tested under Python 3.x but it looks like it should work perfectly if you just replace python-notify2 python-gi with python3-notify2 python3-gi in the dependencies and edit the shebang line at the top of the file.

I only wish I’d realized what I was doing sooner so I could start tracking its evolution properly in git. As-is, all I could do was translate the snapshots from Dropbox’s revision history.

As usual for small, single-file things, here’s the direct download link followed by a GitHub Gist embed.

Could not embed GitHub Gist af69a99908a5bc001495: API rate limit exceeded for 208.94.116.209. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)
Posted in Geek Stuff | 10 Comments

“Rubber Matrix” and “Sparse Montage” wrapper for ImageMagick

Last night, I was working on a bunch of images and I really wanted to just get a visual overview along the lines of a scatter plot, but with images rather than dots. Unfortunately, no matter what I searched for, I couldn’t find anything to do what I wanted.

ImageMagick’s montage tool can do it… but you have to manually fill in the empty cells with the special filename null:

…so I wrote a script to do just that.

Oh, and I got carried away and wrote a “rubber matrix” class to abstract away the actual generation of the grid. Basically, it’s a DictMixin-based class that insists on consistently sized tuples as keys and then adds some extra methods to traverse them as sparse points in a matrix dynamically sized to only the rows/columns/etc. which exist.

I haven’t yet decided on what command-line syntax I want, so you have to import it, but it works perfectly in my preliminary tests.

Could not embed GitHub Gist 51884c2e45bf91af22d8: API rate limit exceeded for 208.94.116.209. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)
Posted in Geek Stuff | Leave a comment

Exporting Firefox Tab Groups

TL;DR: Use this script. Non-Linux users, feed it the path to a sessionstore JSON file. Try --tiddlywiki for more copy-pastable output.

Because of the relatively high UX overhead of things like Firefox’s built-in bookmarking or Delicious.com, I recently found myself with over 1000 tabs in my Firefox tab groups.

Lazy-loading after a browser restart or not, this was very far from ideal so I whipped up a quick little script which contains not only the code I used to dump my sessionstore.js, but also the code I used as an aid to reverse-engineering the tab groups part of it since, as far as I can tell, the Mozilla guys didn’t bother to document that.

The code’s definitely not pretty and I can’t guarantee the --make-schema and --check-schema parts still work, but the dumping works perfectly and I didn’t want to spend too much time on it because I’ve heard Mozilla wants to move away from JSON for saved sessions in favour of a custom binary format specifically designed to help them minimize the amount of time internal data structures need to stay locked in order to write them out on a potentially memory-constrained device. (Remember, Firefox on Desktop shares code with Firefox OS.)

It was originally developed to operate on sessionstore.js but, between when I started developing it and now, things changed a bit so, now, it reads sessionstore-backups/recovery.js instead.

It can take one or more paths but, if you don’t provide one, it’ll assume a Linux desktop and try to export every tab group from every Firefox profile in your user account. As written, it always dumps to the tabs.html file in the current working directory.

Normally, it produces a nice-looking HTML output, complete with favicons for list bullets but, If you pass the --tiddlywiki option, it’ll generate a nice compromise that can be copy-pasted into TiddlyWiki. (Including hidden !!! prefixes for the headers so that you can copy-paste a bunch of groups in one go.)

Enjoy. 🙂

Could not embed GitHub Gist 35097553e5173935e597: API rate limit exceeded for 208.94.116.209. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)
Posted in Geek Stuff | Leave a comment

A confirmation dialog for URxvt’s Print Screen key

I use urxvt’s kuake plugin as a lighter replacement for Yakuake and, as a result, I’m using it with F12 as my “toggle terminal” keybind.

…unfortunately, that means that, every time my finger slips, I hit Print Screen.

As such, I wrote this little script you can use to Print Screen to file, with a confirmation dialog to avoid the need to keep deleting unwanted printouts.

confirm_print.sh (fixed commit ID)

To use it:

  1. Copy it to somewhere like ~/bin/confirm_print.sh
  2. Make sure zenity is installed
  3. Add these lines to your ~/.Xresources
    Rxvt.print-pipe: ~/bin/confirm_print.sh
    URxvt.print-pipe: ~/bin/confirm_print.sh
    
  4. Run xrdb -merge ~/.Xresources
  5. Restart any terminals you may have. (You are using screen or tmux, right?)
Posted in Geek Stuff | Leave a comment

Useful bits for GOG Linux Users

Over the last little while, I’ve written or researched a few little things to make my life as a Linux-using player of GOG.com games a bit simpler. As such, I thought I’d share them.

1. GOG.com – Updated Thread Count in Title

This Greasemonkey script adds the unread count for your favourited forum threads to the favicon so you can select “Pin Tab” and have an alternative to manually polling for new replies.

It’s not RSS, but it still takes a load off your mind.

2. find_broken_symlinks.sh

While not specifically focused on GOG.com or even gaming, this multi-mode script has proven very useful.

Modes of operation:
--all-external
  Show all symlinks targeted outside the current directory
--all-home
 Show all symlinks targeted inside /home
--all-my-home
 Show all symlinks targeted inside the current user's home directory
(anything else)
 Show all broken symlinks originating inside the current directory

I originally wrote it because:

  • Some of my Linux-native ports store their save files in the game’s application folder
  • I symlinked the save files into my homedir so nightly backups cover them
  • My games folder is on an external USB drive
  • Supposing I ever want to let a friend try the games on their computer, I’ll need a way to identify the folders I need to mkdir -p to unbreak the symlinks.

Caution: It’s a bit of a hack and will get confused by folder/filenames containing newlines. (Doing it properly and efficiently is very awkward and painful without reinventing /usr/bin/find in something with more support for structured data than shell scripting.)

3. aptitude search ~o
A quick explanation of “aptitude” for Ubuntu users:

  1. Debian invented apt-get
  2. apt-get didn’t get the lovin’ it deserved
  3. aptitude was invented as an apt-get that doesn’t suck and Debian decided to include it in the base install.
  4. The more popular aspects of aptitude eventually got added to apt-get
  5. Ubuntu decided that apt-get was enough and you have to sudo apt-get install aptitude if you want it.

Basically, aptitude is apt-get on steroids.

What does aptitude search ~o do? It lists all installed packages which aren’t backed by a repository. (Packages you installed using dpkg -i or gdebi, packages which are still installed after you removed the PPA, etc.)

I used it to identify which Humble Bundle games had been installed as .debs so I could replace them with tarballs and move them to my new games drive.

UPDATE #1 …

4. lgogd_uri

A simple GTK+ GUI for lgogdownloader that I wrote so Linux users could take advantage of GOG’s gogdownloader:// URIs.

5. makeself_safeextract

A tool for unpacking GOG’s new MojoSetup-based Linux installers without running any 3rd-party code.

(eg. Good for scripting the extraction of DOSBox- or ScummVM-based games for use with your non-x86 DOSBox or ScummVM install.)

Includes a proof-of-concept script demonstrating how to automatically download a game’s Linux build and then convert the installer to a cleaned-up Zip file. (Since MojoSetup already uses a Zip file internally to store the game so the file permissions are already constrained to what Zip’s support for POSIX permissions can represent.)

UPDATE #2 …

6. How to use overlayfs to force games to separate out stuff you want to back up

While many native Linux games are well-behaved and cleanly separate out their save files, some games (some native, some DOSBox-wrapped DOS games, and some in Wine prefixes rendered self-contained to keep games from scribbling all over $HOME) aren’t so helpful.

Rather than fighting with symlinks and permissions, the simple, easy, and proper solution is to use an overlay filesystem so that any changes made after installation will be redirected into a folder covered by your backup system and the game won’t even realize anything’s changed.

Historically, Linux has had various overlay filesystems, like unionfs and aufs, but they haven’t made it into the mainline kernel. UnionFS (in Ubuntu patchsets since 11.10 and mainlined in kernel 3.18) changes that.

Just cook up an OverlayFS mount command like this and you’re ready to go.

sudo mount -t overlayfs -o lowerdir=/mnt/buffalo_ext/games_inst,upperdir=~/.local/share/games/ overlayfs /mnt/buffalo_ext/games_play

(lowerdir is where you install your games to. upperdir is where files created/modified after install will end up. That last path on the line is where you run your games from.)

Posted in Geek Stuff | Leave a comment