Rust: Looping on a member variable without mutably borrowing self

The Story

Late last night, I stumbled upon a rather clever hack in one of my Rust projects.

I’d been working on an iterator which implements grapheme-aware CamelCase word-splitting when I decided to do some cleanup and ran cargo clippy rather than the much quicker cargo test I’d been using to iterate on it.

Not-surprisingly, given how much I’ve been letting myself get carried away with my hobby projects and “coding while half-asleep”, it popped up some warnings, but this is the one which started things off:

warning: this loop could be written as a `for` loop
 --> src/util/naming/camelcase.rs:260:9
 |
260 | / while let Some((byte_offset, grapheme)) = self.in_iter.next() {
261 | | // Extract the base `char` so `classify_char` can call things like `is_uppercase`
262 | | let base = grapheme.chars().nth(0).expect("non-empty grapheme cluster");
263 | |
... |
280 | | }
281 | | }
    | |_________^ help: try `for (byte_offset, grapheme) in self.in_iter { .. }`
    |
    = note: #[warn(while_let_on_iterator)] on by default
    = help: for further information visit https://github.com/Manishearth/rust-clippy/wiki#while_let_on_iterator

Sounds reasonable, so I tried the suggested syntax and got a new error:

error[E0507]: cannot move out of borrowed content
 --> src/util/naming/camelcase.rs:260:40
 |
260 | for (byte_offset, grapheme) in self.in_iter {
 | ^^^^ cannot move out of borrowed content

Well, I may be tired out of my mind, but I recognized what that error meant in theory, so I tried adding &.

error[E0277]: the trait bound `&unicode_segmentation::GraphemeIndices<'_>: std::iter::Iterator` is not satisfied
   --> game_launcher_core/src/util/naming/camelcase.rs:262:9
    |
262 | / for (byte_offset, grapheme) in &(self.in_iter) {
263 | | // Extract the base `char` so `classify_char` can call things like `is_uppercase`
264 | | let base = grapheme.chars().nth(0).expect("non-empty grapheme cluster");
265 | |
... |
282 | | }
283 | | }
    | |_________^ the trait `std::iter::Iterator` is not implemented for `&unicode_segmentation::GraphemeIndices<'_>`
    |
    = note: `&unicode_segmentation::GraphemeIndices<'_>` is not an iterator; maybe try calling `.iter()` or a similar method
    = note: required by `std::iter::IntoIterator::into_iter`

Now, my tiredness bit me. It never occurred to me that a unicode_segmentation::GraphemeIndices needs to be mutably bound to work, nor that “not implemented for &” said nothing about whether it was implemented for &mut.

Completely stumped, I popped over to  #rust where, after blindly trying several helpful suggestions, I finally tried &mut self.in_iter.

That would normally have worked… except for one small problem:

error[E0499]: cannot borrow `*self` as mutable more than once at a time
   --> game_launcher_core/src/util/naming/camelcase.rs:277:40
    |
262 |         for (byte_offset, grapheme) in &mut self.in_iter {
    |                                             ------------ first mutable borrow occurs here
...
277 |                 CCaseAction::Skip => { self._next_word(byte_offset, true) },
    |                                        ^^^^ second mutable borrow occurs here
...
283 |         }
    |         - first borrow ends here

error[E0499]: cannot borrow `*self` as mutable more than once at a time
   --> game_launcher_core/src/util/naming/camelcase.rs:278:45
    |
262 |         for (byte_offset, grapheme) in &mut self.in_iter {
    |                                             ------------ first mutable borrow occurs here
...
278 |                 CCaseAction::StartWord => { self._next_word(byte_offset, false) },
    |                                             ^^^^ second mutable borrow occurs here
...
283 |         }
    |         - first borrow ends here

error[E0499]: cannot borrow `*self` as mutable more than once at a time
   --> game_launcher_core/src/util/naming/camelcase.rs:279:54
    |
262 |         for (byte_offset, grapheme) in &mut self.in_iter {
    |                                             ------------ first mutable borrow occurs here
...
279 |                 CCaseAction::AlreadyStartedWord => { self._next_word(prev_offset, false) },
    |                                                      ^^^^ second mutable borrow occurs here
...
283 |         }
    |         - first borrow ends here

The code calls &mut self methods inside the loop body and, because the loop returns before exhausting the iterator, I can’t mem::replace it out of the binding.

In short, I had stumbled upon the one way to do this, on my first try, completely by accident, and, were it not for a naïve Clippy lint, I would have never realized how special  this syntax is.

It was around that point that the wheel-bound hamster powering my brain woke up long enough for me to start making the connections and ask the last few questions necessary for it to all make sense…

The Reasoning

The problem here is a collision between two characteristics of Rust’s design:

  1. for works via the IntoIterator trait, which means that, as far as the compiler knows, releasing and re-borrowing the resulting iterator would discard the iteration state and start over. (ie. There’s no magic in the compiler to to recognize when it’s already got an iterator.)
  2. My self._next_word takes a mutable borrow over all of &self …which will fail if for is still holding a reference to one of its members.

The clever trick behind while let Some(...) = self.in_iter.next() is that it bypasses IntoIterator. As such, the compiler can be certain that self.in_iter won’t go away between iterations, and can release the borrow.

As a result, you’re left with something that functions like a for loop, but only holds onto the item which the iterator returned, leaving self free to be borrowed, in its entirety, by all and sundry.

So, there you have it. If you’re writing a struct which holds onto an iterator and your for loop is making things difficult by blocking method calls, try bypassing IntoIterator. Get an iterator manually, then reformulate your loop to use while let Some(...) instead.

If Clippy complains, add #[allow(while_let_on_iterator)] and get on with your day.

P.S. Don’t worry about the “coding while half-asleep” part. I write my unit test suites and audit/refactor the previous day’s work while wide-awake and alert. 😉

Posted in Geek Stuff | Leave a comment

Recommended Battlestar Galactica “Earth-contact” fics

I’ve always enjoyed first-contact fics, because they’re a good way to stir up groups of people and see how they react. No plan survives contact with the enemy and no worldview survives first contact.

…but, in the case of Battlestar Galactica, having them find an Earth of comparable social development is often even better, because they’re not just going in with any old expectations, they have deeply held preconceptions about what they should find.

That makes for a rather unique “flavour”, but, like any slice of a fandom you look for, there’s a big gulf between the rare cream of the crop, and everything else.

…so, without further ado, here is my list of those special “Earth-contact” fics which deserve a read:

Best of the Best

The Consequences Of Not Being Polite by The Sidhe
Length: 266,212 Words
Status: Complete
Crossover: None
Strongest Element: Managing to feel like a sci-fi novel
In this fic, the fleeing colonial refugees are discovered by a forward patrol from an Earth at least a thousand years more advanced and slowly pushing back an equally advanced enemy descended from terran velociraptors.As is the case with most good “one side could curb-stomp the other” stories, this is a story about exploring and adjusting the worldviews of the less advanced factions, with the cylons also eventually coming to the table after Cavil’s failed attempt to ally with the raptors instead.

I think one of the biggest reasons this appeals to me is how much effort the author puts into developing the backstory for the setting, which gives it a fairly strong “sci-fi novel” feel to it, rather than the generic “fanfic based on a sci-fi TV show or game” feel that I get from most fanfics.

I’ve re-read this at least twice and I’d recommend that you give it a try if you’re even remotely interested in contact fics that don’t focus on combat.

Reunions Are A BitchDeleted by Bob Regent
Length: 349,784 Words
Status: Incomplete
Crossover: Stargate: SG-1
Strongest Element: Capturing the complexity and misunderstanding which drive wartime politics
In this Stargate SG-1 crossover (which had to be renamed on Fanfiction.net), after making contact with the Prometheus, the leadership of the Twelve Colonies decides that it would be good business and good politics to “reunite with the 13th colony”, by force if necessary.What I like about this story is that it’s a believable blending of the two settings with sort of a Cold War-esque dynamic:

  • Earth’s almost-nonexistent but much more advanced space defences barely fight off the massive Colonial fleet, but a few nuclear missiles do wind up hitting the planet.
  • The first act of the story leaves both sides assuming the other is much more prepared to counterattack and scrambling to build up defences.
  • World-changing events happen because of honest mistakes, ill-considered decisions, and greed.
  • Cylon infiltrators in Earth POW camps eventually ask for asylum.

An engaging fic which is one of the handful I’ve re-read over the years. Definitely one I’d recommend despite it being unfinished.

Runners-up

These are stories which, while they’re good, don’t quite make the cut for reasons I’ll explain on a case-by-case basis:

Worldwar: Discovering the Balance by AlbertG
Length: Multiple Volumes
Status: 3 completed stories, 1 Incomplete
Crossover: Stargate: SG-1, Harry Turtledove’s Worldwar
Strongest Element: Focusing on individual OCs who have to deal with the violation of their expectations.
The main reason this story is a runner-up is that it’s hard to truly characterize it as a proper “BSG: Earth-contact” story.While the Colonies do come to play a major role and one volume does focus entirely on them, the story starts with the Worldwar-SG1 aspects and the Galactica stuff not only feels like an alternative take on RAAB, this story is supposedly intended to be set in the same multiverse and I feel like a reference to one of the other AlbertG-Bob Regent stories in the shared meta-setting only cheapens it by needlessly referencing an off-screen-and-before-story-start war with the Vorlons.

That said, it’s still a story I liked enough to re-read. The basic plot is:

  1. A while before the story started, the Prometheus did encounter The 12 Colonies, but they parted without the colonies finding out where Earth is.
  2. The Race from Turtledove’s books arrives a few decades later than in canon and get a big shock when, instead of conquering the knights on horseback that they planned for, they’re met by a starship more advanced than their fleet in every way.
  3. The Race, being The Race, are forced to cope with a species that can crush them and refuses to acknowledge their ownership papers for “Tosev 3”, when they’ve not had to deal with the universe violating their expectations for ten thousand years.
  4. Because Gaius Baltar didn’t get fooled, the last handful of Cylon refugees wind up fleeing into Earth’s solar system by accident, where they successfully petition Earth for asylum.
  5. Earth manages to drive off the pursuing Colonials with no losses.
  6. The rest of the plot ensues.

If you want something that’s got similar elements to RAAB, but is mainly focused on forcing foreign cultures to face the fact that their expectations are wrong and then have them slowly develop relations with Earth as that happens, you’ll like this. I like how it develops its original characters to achieve said goals.

Contact at Kobol by wilkins75
Length: 478,758
Status: Complete
Crossover: Stargate: SG-1
Strongest Element: Having a boots-on-the-ground-oriented approach to a war without boring me into losing interest
If you wanted something similar in concept to RAAB, but with things escalating to a boots-on-the-ground invasion of The 12 Colonies, this is your story. I’ve always found following soldiers on the ground to make a story more boring, so I think the fact that I was willing to read this story to the end speaks well for it, even if I’ve yet to read it again.

The main difference between this and the other SG-1 crossovers is probably that it takes place a little further in the future, when Earth has had time to study Asgard technology a bit and are about to reveal the Stargate so they can ease overpopulation in countries like India via a colony they’ve just finished building on a planet they’ve named Valhalla.

The first encounter between Earth ships setting picket stations and a Colonial patrol along the Cylon border ends in disaster when the colonials mistake them for Cylons and lose several battlestars in the resulting skirmish.

While cooler heads do manage to arrange a diplomatic contact, relations eventually devolve (partly due to the discovery that Earth’s new colony is actually Kobol) and this eventually precipitates a war, which then gets stoked when a screw-up by a Colonial battlestar results in Disneyland Valhalla getting nuked.

I don’t want to spoil too much, so I’ll leave you to read the other memorable details yourself.

The main flaw I should mention is that it could use a second proofreading… but the typos aren’t excessive and it’s still perfectly readable.

Posted in Fanfiction | Leave a comment

Tidying up Amazon wishlist printouts

Whenever I visit the used games store, I like to bring a printout of my Amazon.ca wishlist, since it’s easier to work with than a tiny screen. However, Amazon, for reasons that escape me, decided that print versions would somehow benefit from having entries mention that the Amazon offering at the time of printing was from a third-party seller.

That’s just distracting and unhelpful, so here’s a userstyle to fix it. I would have uploaded it to userstyles.org as usual, but there’s some kind of bug in their validator.

Title:
Amazon – Hide “Offered by ” in print wishlists
Description:
Remove unnecessary clutter from Amazon’s waitlist printouts to make them easier to skim-read.
Preview Image:
CSS:
@namespace url(http://www.w3.org/1999/xhtml);

@-moz-document regexp("https?://(www\\.)?amazon\\.[.a-z]+/gp/registry/wishlist/.*)?\?(.*&)?layout=standard-print(&.*)?$") {
    td.g-title span { display: none !important; }
}

Enjoy. 🙂

Posted in Geek Stuff | Tagged | Leave a comment

Sci-fi and Fantasy: Why They Differ and Why You’d Blend Them

I’ve been wanting to post more regularly, so I decided share some of the insights for the book on plotting out stories that’s been on my TODO list for a while. (Embarassingly, I’ve actually wanted to do that for a while, but I haven’t been taking my own advice that perfectionism is bad if it prevents you from ever getting anything done.)

As a way to bite the bullet, I’ll start by sharing something I just realized yesterday: The fundamental difference between science-fiction and fantasy.

It all began yesterday when I found myself puzzling over why authors like Anne McCaffrey (Dragonriders of Pern) and Marion Zimmer Bradley (Darkover) chose to write what were essentially fantasy stories, but give each setting a sci-fi backstory. (Both settings are the result of colony missions regressing socially and technologically.)

Think about that for a moment. You’re starting from a clean slate, you won’t be writing about the colony’s fall any time soon (assuming this one is successful), and you have full creative control… so why would you choose to mix science-fiction and fantasy in this way?

I puzzled over it for a bit, then mentioned it to my brother, and he pointed out what I was missing: Fantasy and science-fiction embody different perspectives on the world. (Another piece of advice from my notes. Never underestimate the value of bringing in alternative points of view.)

Science fiction, as we know it, began with Mary Shelley’s “Frankenstein: Or, The Modern Prometheus” (also the first example of what we now call “speculative fiction”) and has a strong history of looking at the world as it is, rationally exploring its warts and caveats and annoying shades of grey in all their detail and nuance. (Given how much movie adaptations have had to simplify it, I strongly recommend following the link above to a public domain copy of Frankenstein on Project Gutenberg.)

Fantasy, by contrast, looks at the world as our instincts and emotions wish it were. It revels in black-and-white conflicts, magical powers, and embodying points of view in individuals who you can like or hate. (Because, for better or for worse, something deep down in us wants a name and a face to blame when bad things happen. Nothing is more uncomfortable and dehumanizing than the helpless feeling of being denied even the bitter solace of hatred and a desire for revenge. Better to cling to the belief in a god or a conspiracy or what have you, than to accept that you were so insignificant and impotent as to have your life destroyed by a mindless, random act of nature.)

I especially want to focus on that part about turning characters into avatars for viewpoints and moral stances. What are gods but creating someone to blame for the actions of abstract forces? What is magic, but a way of “fixing” the fact that we are small and weak in the face of a big, scary, unpredictable, uncontrollable universe? (And what a “lever” to give to those tiny, weak humans in a story, so they’ll have the strength to “move the world” without getting lost in a sea of faces?)

(I suspect this is also why I hear of many more fantasy epics than sci-fi epics. An epic is a story about a small, seemingly ordinary person proving their worth and changing the world, and fantasy’s narrative tendencies would exert a stronger bias in that direction than sci-fi.)

This distinction is actually the key to why Bradley and McCaffrey wrote their stories as “fantasy settings in sci-fi universes”. Every story needs to set up a frame of reference, so the readers know how to judge what they’re reading. In Douglas Adams’s Hitchhiker’s Guide to the Galaxy series, that frame of reference is Arthur Dent, the ordinary Brit who tells the reader what normal is, so they can properly judge just how ridiculous, silly, and downright insane the setting is.

In the Pern and Darkover series, setting the fantasy world within a sci-fi universe is the author’s way of saying, both to the readers and to themself, that “this setting may have the trappings of fantasy, but the narrative style is following sci-fi rules”. Dragons and lord holders, keepers and magic… but, underlying it all, the recognition that this is a rational universe, which demands a certain degree of deference to the complexity and nuance of its happenings and its inhabitants. No matter how much power one may have in these universes, magic cannot craft a true and simple solution to a complex problem any more than Superman can punch clinical depression.

Now, that’s not to say that this dichotomy is inherent or without exceptions… but this sort of hybrid setting does serve as a very useful shortcut. It’s a good shorthand for telling potential readers to expect a certain type of look at a feudal-esque society and it saves you the trouble of having to strike what may be a difficult balance. You don’t want to flood your readers with needless detail, but you’ve chosen to write a story where the narrative style encourages readers to pick things apart, looking for holes… and we already know that the real world is internally consistent enough to satisfy. (Which is why this also works for fantasy-contemporary hybrids, like the Harry Potter series. The key is to temper fantasy’s more “expect anything” aspects by tying the story to something we judge more strictly.)

So, in conclusion, setting your fantasy in a “mundane” universe is useful as a way to get a certain kind of setup done more quickly so you can get to the meat of your story. Feel free to use it… just do it because it advances your goal as an author, rather than because you’re trying to copy the feel that someone else’s works give you.

Posted in Writing | Leave a comment

Splitting CamelCase without Regex lookahead/lookbehind

I’ve been working to port the brains of my game launcher experiment from Python to Rust for easier refactoring and to eventually offer language bindings other than Python itself and, at the moment, I’m working on the “filename to title” heuristic, because that’s got the best unit test suite.

Surprise, surprise. The biggest time-sink has been redesigning things where the Python version uses regular expressions with lookbehind and lookahead expressions.

Well, the CamelCase-processing regular expression faked me out. If you search for answers, it seems every language is using some variation of this regex:

r'((?<=[a-z])[A-Z]|(?<!\A)[A-Z](?=[a-z]))'

…and, since Rust’s Regex crate uses a DFA to avoid pathological performance, this wasn’t an option.

I wasted an afternoon trying to implement such a parser manually before it occurred to me that you don’t actually need lookahead/lookbehind to parse camelcase!

It seems that someone originally wrote that regex with the goal of getting accurate string indexes for the starts of words, then everyone else copy-pasted and translated it without recognizing that it was stricter than necessary.

UPDATE: There is one situation where the lack of lookahead has to be worked around which I somehow overlooked… any situation where you need to insert a space before and after the same character. (ie. words like “I” and “A” in the middle of a phrase.) The solution there is just to run the regex twice.

So, if you just want to insert spaces (without duplicating ones that are already there), and you want to enhance the algorithm to handle numbers and ampersands reasonably and you want it to work in Rust’s regex engine, here’s how you do it:

r"([a-z&])([&A-Z0-9])|([^ ])([A-Z][a-z])|([0-9])([a-z])"

…and then use this as your replacement string:

"${1}${3}${5} ${2}${4}${6}"

It passes every test in my unit test suite for CamelCase splitting and improves the accuracy score on the integration test corpus for the filename_to_name function which relies on it.

UPDATE #2: I got to playing with the Regex crate’s support for unicode character classes and I have two things to report:

First, you actually do need a fourth pair of capture groups to split “A&b” because of the “ampersand, followed by lowercase” situation.

Second, for anyone who wants it, here’s a fully commented Rust regexp which will process CamelCase written using any mix of what the Unicode database considers to be letters (lowercase, uppercase, or titlecase), numbers (decimal digits, letters, or other), and the four different kinds of ampersands I was able to find:

r"(?x)
 # == Ampersand (definitely end) followed by anything not already whitespace ==
 # ('Ampersand' or 'Small Ampersand', or 'Fullwidth Ampersand' or...
 # 'Heavy Ampersand Ornament')
 ([\x{26}\x{FE60}\x{FF06}\x{1F674}])
 # (Something not 'Separator, Space')
 (\P{Zs})

 # == OR ==
 |

 # == Lower/titlecase (possible end) followed by upper/titlecase/number (possible start) ==
 # ('Letter, Lowercase' or 'Letter, Titlecase', 'Ampersand' or 'Small Ampersand', or...
 # 'Fullwidth Ampersand' or 'Heavy Ampersand Ornament')
 ([\p{Ll}\p{Lt}])
 # ('Letter, Uppercase' or 'Number, Decimal Digit' or 'Number, Letter' or...
 # 'Number, Other' or 'Ampersand' or 'Small Ampersand' or...
 # 'Fullwidth Ampersand' or 'Heavy Ampersand Ornament')
 ([\p{Lu}\p{Lt}\p{Nd}\p{Nl}\p{No}])

 # == OR ==
 |

 # == Number followed by an un-capitalized word ==
 # ('Number, Decimal Digit' or 'Number, Letter' or 'Number, Other')
 ([\p{Nd}\p{Nl}\p{No}])
 # ('Letter, Lowercase')
 (\p{Ll})

 # == OR ==
 |

 # == Anything not whitespace, followed by ampersand or unambiguous beginnings of word ==
 # (Something not 'Separator, Space')
 (\P{Zs})
 # ('Letter, Titlecase' or ['Letter, Uppercase' followed by 'Letter, Lowercase'] or...
 # 'Ampersand' or 'Small Ampersand' or 'Fullwidth Ampersand' or ...
 # 'Heavy Ampersand Ornament')
 (\p{Lt} | \p{Lu}\p{Ll} | [\x{26}\x{FE60}\x{FF06}\x{1F674}])
 "

…and…

"${1}${3}${5}${7} ${2}${4}${6}${8}"

Before you get scared, it’s actually just the Unicode version of the tiny regexp with a ton of explanatory comments and a little more ampersand matching. It’s very simple once you throw out the comments and understand the structure.

The regular expression matches any of four situations where a space should be inserted:

  • An ampersand followed by anything that isn’t whitespace (this is the one I split out)
  • A lowercase or titlecase (upper+lower in one character) letter followed by an uppercase/titlecase letter or a number.
  • A number followed by an un-capitalized letter.
  • Anything not whitespace, followed by an ampersand, titlecase letter, or <UPPER><lower> pair.

Once you ignore the comments, most of the verbosity comes from having to combine multiple unicode character literals or classes to say things like…

  • “any ampersand” ([\x{26}\x{FE60}\x{FF06}\x{1F674}])
  • “any type of number” ([\p{Nd}\p{Nl}\p{No}])

The advantage is that, because it delegates the lower-level details to the Unicode lookup tables for everything but the ampersands, any failure should result from the language being so alien that the concept of CamelCase itself is difficult to apply to it.

UPDATE #3: Replace chained OR tokens with compound character classes where possible for minor cache pressure savings in the regexp engine.

It’s still a heavy regexp (I was chatting with burntsushi on IRC when the idea occurred to me and, if anyone should have the expertise to judge that, it’s the author of the Regex crate), but it meets my needs, it’s easy to tweak as I further refine my heuristic guesser, and if I ever need something more performant, writing this has taught me enough about the problem that I should be able to write a manual solution.

Posted in Geek Stuff | Leave a comment

Getting Your Cheap Chinese USB Foot Pedal Doing Useful Things on Linux

Well, I have a Chinese USB foot pedal that I picked up on eBay for $11 CA to play around with but, for ages, it’s been worth bupkiss because it’s mapped to the “b” key on the keyboard and I was too busy to look into how to remap a key on just one device on Linux… that ends here and now.

Try 1: Remapping in XKB

The most obvious solution would be to remap the thing by applying a custom X11 keyboard layout to just the one device. (ie. Redefine how keycodes from evdev map to X11 keysyms.)

OK, doesn’t seem too hard and, Google to the rescue, someone even wrote some example code for it. (See also: XKB Remapping)

The gist is:

  1. Use xinput list to find the ID for the device you want to limit the remapping to
  2. Use setxkbmap -device <the ID> -print to dump the keymap
  3. Write a custom supplemental XKB keymap which redefines the key in question
  4. sed the dumped keymap to append the supplemental keymap to the stack of what gets applied.
  5. Use xkbcomp to re-apply it.

…but, no matter what I tried, I couldn’t get my Ubuntu 14.04 LTS system to acknowledge the change.

Verdict: Failure on *buntu 14.04 LTS

Try 2: Remapping in XKB (hackier but more foolproof)

I was getting errors from the previous version and I didn’t know whether any of them were spurious, so I decided to try the old “hacky but foolproof” standby described in this ArchWiki page … dump the keymap, hand-edit the stupid thing, and then re-apply it.

xkbcomp -i <the ID> $DISPLAY ~/out.xkb
vim ~/out.xkb
xkbcomp -i <the ID> ~/out.xkb $DISPLAY

First, let me caution you that it has to be -i ID and not -iID. The latter form doesn’t work.

That said, it ran without complaint and even showed the expected results when I re-dumped the keymap fresh from the X server… it just had no effect.

(I think it might be because, when you look at the USB HID descriptor that evtest dumps on startup, the stupid pedal announces that it’s a 162-key keyboard, plus a mouse with scroll wheel, all in one device, which causes xinput list to file it as a mouse rather than a keyboard.)

Verdict: Failure on *buntu 14.04 LTS

(Naturally, I didn’t continue on to research how to make it survive restarts.)

Try 3: Remapping via udev

OK, no big deal. My ATi Remote Wonder II didn’t need remapping because it sent keycodes that were already mapped to the XF86AudioPlay and XF86AudioPause keysyms. Let’s drop below X11 and redefine how the kernel maps hardware scan codes from the pedal to evdev keycodes.

Now, since we need to redefine it for only one input device, that means we can’t use the setkeycodes command, so udev it is. (Besides, I’ve heard that setkeycodes doesn’t work with USB devices.)

According to these StackExchange answers [1] [2] and these ArchWiki pages [1] [2], I need to put together a definition like this, and store it at /etc/udev/hwdb.d/some-unique-name.hwdb, then refresh the database:

keyboard:input:b0003v0C45p7403*
 KEYBOARD_KEY_70005=playpause

Oh, and make sure you only indent the second line one space. More will cause a silent failure and the first guide I read not only didn’t mention that, but actually used two spaces in the example!

To keep things simple, here’s the overview of the process:

  1. Create something in the above format as a /etc/udev/hwdb.d/whatever.hwdb file.
  2. The numbers in the file are as follows:
    1. The b0003 means “Bus type: USB”
    2. The v0c45 and p7403 are the USB vendor and product IDs from lsusb
    3. The 70005 is the MSC_SCAN value from pressing the pedal while evtest is listening.
    4. The playpause is the kernel-internal name for the desired keycode. They’re defined in a header I don’t seem to have (linux/input-event-codes.h) but the ArchWiki pages have a link to an online list of them.
  3. After putting the file in place, run sudo udevadm hwdb --update (older systems) or sudo systemd-hwdb update (newer systems) to merge it into the hardware database.
  4. Finally, either reboot your system or run sudo udevadm trigger to apply the changes. (Un-plugging and re-plugging the thing may also work)

…so, did it work? Well… no. It turns out *buntu 14.04 is using the last version of udev before that feature was added (14.04 has udev 204 and it needs 205 or higher).

(Also, note that whether you use the keyboard: prefix or the evdev: prefix will depend on the udev version you use. I tried both, but this site explains what to use for which version.)

Verdict: Failure on *buntu 14.04 LTS

NOTE: This AskUbuntu answer that I found after hacking together my own solution suggests that it might actually be possible, but that 14.04 includes a version of udev with some funny requirements for applying the changes.

Try 4: Remapping via udev (the old way)

*sigh* Ok, so we can’t use the clean, easy solution everyone’s listing because 14.04 is too old… so what’s the old solution?

Well, according to this page and this AskUbuntu answer, versions of udev up to and including 204 do key remapping by using the old familiar udev rules.d system and a helper utility named /lib/udev/keymap. Cool.

To paraphrase Jurassic Park, “It’s a rules.d system, I know this.” (From fixing permissions on various devices, to getting my Chinese NES controller adapter to announce itself as a joystick, I have a fair bit of experience writing rules.d files.)

OK, so I write this /etc/udev/keymaps/microdia-foot-switch file, using the same values from the previous attempt:

0x70005 playpause

…and this /etc/udev/rules.d/99-microdia-foot-switch.rules file:

ACTION!="remove", SUBSYSTEMS=="usb", ENV{ID_VENDOR_ID}=="0c45", ENV{ID_MODEL_ID}=="7403", RUN+="keymap $name /etc/udev/keymaps/microdia-foot-switch"

…and unplug and replug the foot pedal… and nothing happens. I check `dmesg` and whadda ya know, `/lib/udev/keymap` isn’t installed.

(NOTE: While it didn’t affect me, your distro may have a bug in keymap which forces you to use the keycode itself instead of the playpause name alias for it. If you happen to have another device which can emit the target keycode, you can use sudo showkey --keycodes to get it really easily.)

So, anyway, I pop over to packages.ubuntu.com to search up the package to install… and it turns out it was in udev in Precise and got pulled (with no replacement) in Trusty… way to jump the gun, guys!

Of course, this isn’t the only botch in 14.04 (it also shipped with a buggy version of Geeqie), so this is old-hat.

  1. Pop over to the precise-updates version of the udev package on packages.ubuntu.com, scroll to the bottom, and download whichever architecture matches your hardware. (Probably amd64, in this day and age)
  2. Open it up with file-roller (because it’s smart enough to hide the fact that a .deb is an archive containing more archives) and copy out /lib/udev/keymap.
  3. sudo cp keymap /lib/udev/keymap && chmod +x /lib/udev/keymap

Now, try unplugging and re-plugging the thing again and it should work.

Verdict: Success on *buntu 14.04 LTS

…but wait, there’s more!

While evtest and xev were showing the right keysym, my Audacious Media Player wasn’t listening to the resulting XF86AudioPlay presses.

Apparently, not all XF86AudioPlays are created equal, because adding a second mapping from “Pause/Resume” to XF86AudioPlay by stepping on the pedal fixed it.

(My hypothesis is that, while Audacious is displaying the keysym names, it’s binding using the underlying keycodes and the one from my ATi Remote Wonder II began life as a play keypress while the one from the pedal began life as playpause.)

Posted in Geek Stuff | 1 Comment

Home-made tamper-evident security seals for kids and adults alike

Suppose you need to keep siblings, roommates, children, or even friends with wandering hands out of something, but you can’t use a lock. Maybe you’re worried they’ll find the key, maybe you need something that has no metallic parts, or maybe you’re a kid and your parents are worried you will lose the key.

This is the kind of thing numbered security seals are good for… but they’re expensive. (The cheapest I’ve found are roughly 50¢ each in packs of 100 or more)

…so I decided to do a little experimentation and came up with a cheap-but-reliable solution for homemade tamper seals that’s so simple and safe that even children can do it.

Instructions

Step 1: Making the seals

The most important thing about a security seal is that it’s unique, so it can’t be replaced, and either can’t be reclosed after being opened or you’ll notice if it is.

To satisfy these requirements with paper, we need to print or write something on the paper which other people can’t reproduce and which will look wrong if someone cuts and re-glues the seal.

If you’ve got a printer, you could print out a random image from the Internet, delete the file and empty your browser history, then cut strips from it to use as seals, but Inkjet ink tends to cost a fortune and most people don’t have laser printers.

What I recommend is this: Cut strips from a piece of paper, then sign and doodle all over both sides of them so any cuts or complete replacement will be obvious. (You do it on both sides, so that you can easily see if someone mends a cut by glueing another piece of paper to the inside of the loop.)

Don’t forget to use your scissors to make the ends of the strip round, rather than square, so it’s harder for someone to attempt to pick at the joint if they want to try to peel it apart.

Step 2: Applying the seals

To apply the seal, you just need to pass it through or around whatever you want to seal (eg. cabinet handles) and then glue it into a loop. (a tight loop, if you’re securing knob-shaped cabinet handles, so it can’t be lifted off without breaking it.)

If you want to seal a box, try gluing several strips together into a shape similar to the ribbon on a Christmas present.

It’s the details which make it tamper-evident:

  1. Get a bottle of Polyvinyl Acetate glue (A.K.A. white glue, school glue, PVA).
  2. Apply a enough glue to one end of the strip that, when squeezed, it won’t leave unglued corners.
  3. Close the loop and squeeze the ends together for at least five seconds, as hard as you can.
  4. Wait at least 5 minutes, but ideally 10 minutes.

Step 3: Making the seals tougher

Supposing you’re trying to keep really clever people from sneaking access, there are two more tricks you should get an adult to do:

  1. Paint the glued part with clear nail polish to waterproof it, so nobody can try to invisibly open the seal by steaming the glue. (This also makes it harder to try to pick apart.)
  2. Cut some diagonal stripes into the glued spot using a utility knife to absolutely guarantee that any attempt to peel apart the glue will result in obvious tearing.

The “Why”

I was very specific in my instructions, because I actually did a lot of testing. If you want to repeat my tests yourself, here’s what I did:

Goals

  1. A successful seal must not have the glue fail when pulling on both ends of the glued joint.
  2. A successful seal must show obvious damage when someone tries to pick and peel at the glued joint.
  3. A successful seal must have the paper visibly fail before the glue if a solvent is applied to the glued joint.

Methodology

  1. Cut a bunch of paper strips, approximately 3″ long.
  2. Glue pairs of strips into longer strips, using various test glues.
  3. On each glued strip, write the time the process was finished, so drying time can be considered.
  4. For each drying time tested, grip the ends of a test strip and pull evenly outward as with a Christmas cracker.
    The test is a success if the paper breaks outside the glued patch. (If it fails far enough from the join, repeat until there’s nothing left to grip)
  5. For each drying time tested, attempt to peel apart the two strips of paper.
    The test is a success if the paper is visibly damaged in a manner that wouldn’t be hidden by glueing the joint back together.
  6. Repeat each test at least once, to account for variations in the process.
  7. Prepare another set of test strips and allow the glued spot to soak in a drop of mineral or vegetable oil for 30 minutes. Repeat the tests.
    The glue passes if it wasn’t weakened by the oil. (In my tests, the breakage still consistently occurred in the dry portions of the paper, indicating that the oil had not weakened it significantly.)
  8. Prepare another set of test strips and paint the glued patch thoroughly with clear nail polish. Repeat the tests.
    The glue passes if the solvent in the nail polish didn’t weaken it. (I have already confirmed in previous experiments that nail polish will waterproof paper without weakening it and, in these tests, the breakage still consistently occurred in the un-painted portions of the paper.)

Results

I performed these tests with the following dollar-store adhesives:

Double-sided tape
I tested two different kinds of foamless double-sided tape. Both were too weak to be suitable, losing their grip on the paper during the pull test with barely any visible effect.
Roll-on glue tape
This contact adhesive passed the pull test, but was not tamper-evident in the face of picking and peeling at the joint and was so thoroughly weakened by the oil test that it was trivial to pull open without harming the paper.
Non-frosted Scotch/Sellotape
From personal experience, I know that the frosted variety is meant to be possible to peel off without harm if you’re careful. I tested the un-frosted kind and, while it just barely passed the pull test, it was too easy to peel off without evidence. Also, it was severely weakened by exposure to oil and I suspect this to be a trait common to all readily available contact adhesives.
Roll-on glue tape, plus Scotch/Sellotape
Didn’t perform significantly better Scotch/Sellotape on its own.
Glue stick
Passed the dry and oil tests, but whether it passed the peel test depended very heavily on exactly how I applied it, so I can’t recommend it. It also got severely degraded by the solvent in the clear nail polish.
White/School/Elmer’s Glue (PVA/Polyvinyl Acetate)
I was surprised how well this performed… though I probably shouldn’t have been, given that it’s used as a bookbinding glue. PVA dries quickly enough that, after 5 minutes, the still-damp paper next to the joint breaks under test and, after 10 minutes, the paper is back to normal. It is so resistant to peeling that the paper tends to tear, and neither the oil nor the nail polish solvent had a measurable effect on the joint’s strength.

I did, however, notice that peeling at a corner produced damage that was easier to overlook… thus my recommendation to round off the ends of the strip before gluing.

I would have moved on to testing other avenues, such as hot glue, super glue, contact cement, and so on, but I don’t need to test them to know that, at best, they’d match the performance of white glue for this application.  (They’re harder to work with, more expensive, no faster to dry, and PVA already produces a bond stronger than and just as water-resistant as the paper around it.)

(I did, however, test super glue on some scraps of projector transparencies meant for black-and-white photocopiers. It passed the pull test but was trivial to peel apart without leaving evidence of tampering.)

Posted in Geek Stuff | Leave a comment