Downloading YouTube Subtitles

A few days ago, for the first time, I wanted to save the YouTube subtitles/closed captions for a video. (It was a Russian newscast about the trolololo guy which wasn’t hard-subbed) Unfortunately, I only had tools in hand for downloading the video stream.

So, now, I present to you the fruits of my labors. Tools in several different forms for downloading YouTube (and apparently sometimes Google Video) subtitles in SRT format, ready to use or convert to another form:

Update: I’ve run across at least one video that stored its subtitles as annotations instead. I’ve found instructions and a ready-to-use tool for retrieving Google’s raw annotation XML but I only found shell scripts (which won’t work on Windows) for converting them to SRT files.

Update: youtube-dl can also rip subtitles while it’s saving a copy of the video to disk. Just use the --write-sub or --write-auto-sub flags (depending on whether you want manually-added subtitles or YouTube’s often hilariously wrong automatic subtitles) and use something like --sub-langs en,fr to select which languages you want to download.

Even better, if you’re downloading the video as a .mp4 file and you’ve got ffmpeg set up on your machine (and it’s newer than the copy which comes with Ubuntu 12.04 LTS), you can pass the --embed-subs flag to save the subtitles right into the video file so you don’t have to worry about losing them.

Posted in Web Wandering & Opinion | 18 Comments

Using Twig with Fat-Free Framework: Why and How

Having used Fat-Free Framework 2.x for a little while now, I’m still generally quite happy with it… aside from one thing.

The built-in templating engine’s syntax doesn’t offer many advantages over raw PHP, it’s more verbose in certain common cases, and, as Fabien Potencier wrote when he took over maintainership for Twig, template languages have evolved a lot since PHP began its life as one in 1995. …so I decided to use Twig.

Why Twig?

It does help that I also work in Django sometimes but it’s not as big a factor as you might think. The main reason is that Twig is both the fastest, lightest templating engine I know (benchmark) and also one of the most featureful. Both good reasons to use FatFree and Twig together.

Here are a few of the reasons I switched:

  • Fully extensible
  • Auto-escaping for cleaner, safer code and less stressful coding.
  • Let’s me choose my preferred mix of template inheritance and template includes.
  • Supports macros for tidying up repeated code snippets like form fields.
  • Has clever constructs like for/else to clean up common cases like “List results or display a ‘no results’ message”
  • Syntax is easier for me to read and write in data-heavy templates with many nullable fields.
  • Generally more amenable to the kinds of factoring-out of duplicated boilerplate that I want to do.

Setup Instructions

Unsurprisingly, I couldn’t get integration with F3::get() as concisely as in F3 templates, but I did pretty well, all things considered.

The simplest way I’ve found to set this up is to add something along these lines to the top of your index.php:

require_once __DIR__ . '/lib/Twig/Autoloader.php';
Twig_Autoloader::register();

$twig_loader = new Twig_Loader_Filesystem(__DIR__ . '/templates');
$twig = new Twig_Environment($twig_loader, array(
    'cache' => __DIR__ . '/twig_cache',
    'auto_reload' => true,
));

$twig->addFilter('f3', new Twig_Filter_Function('F3::get'));
$twig->addGlobal('is_ajax', Web::isajax());

This will set up the following:

Template Location
Templates will be stored in a folder named templates so they don’t conflict with any templates you continue to use with the default F3 templates. (Assuming you’re still using the default ui folder you probably kept from the sample code)
Template Caching
Twig templates will be cached in a folder named twig_cache to avoid conflicts and the cache will be automatically regenerated when the templates change
F3::get()
The f3 filter gives you 'foo'|f3 as a slightly inelegant but not overly verbose equivalent to @foo in templates. Things like 'result'|f3.title will work as desired.
Also, if you’re only using F3::get to expose things to the templates, you can use Twig globals, which let you use {{ foo }} instead of {{ @foo }}. The syntax for doing so is as follows:

$twig->addGlobal('foo', $foo);
is_ajax
This global allows you to do the switching between full templating and minimal responses for AJAX requests entirely in the templates using this snippet of code (source):

{% extends is_ajax ? "base_ajax.html" : "base.html" %}
Rendering
global $twig;
echo $twig->render('page.html', array('bar' => $bar));

Important: Compatibility Fix

The one caveat is that there’s a little incompatibility between the current versions of Twig and Axon. When using {% if my_hydrated_axon.my_null_valued_field %}, you’ll get

Undefined method Axon->my_null_valued_field()

The problem is:

  1. Twig’s dot operator uses isset() to determine whether a property exists, then fails over to assuming you wanted a method call with no arguments.
  2. Axon.__isset() uses isset() on its internal array of fields.
  3. isset() on an array returns false if the field exists but has a value of NULL.
  4. Even with strict_variables set to false, current Twig has a bug that makes it error out rather than producing an empty value when accessing axon properties.

You have two options. First, you can use my_hydrated_axon.cast.my_null_valued_field everywhere (Axon.cast() returns an ordinary associative array, which doesn’t have this problem) or you can make a small patch to  FatFree’s lib/db.php and change this…

function __isset($field) {
 return isset($this->fields[$field]) || isset($this->adhoc[$field]);
}

…to this…

function __isset($field) {
 return (is_array($this->fields) && array_key_exists($field, $this->fields)) ||
  (is_array($this->adhoc) && array_key_exists($field, $this->adhoc));
}

The only caveat with this approach is that, in your PHP code, if (!$axon->property) won’t work properly.

Posted in Geek Stuff | 11 Comments

Setting Custom Game Icons in Desura

I just discovered Desura’s Linux client and, given the opportunity to have a DRM-free package repository system for my Humble Bundle games, I jumped at the chance.

Of course, I still wanted to keep all my games together in one launcher menu, so I went looking for a way to give the manually installed things like Super Meat Boy proper icons. Luckily, Desura turned out to have a pretty simple approach to specifying icons. Just a couple of columns in an ordinary SQLite database.

Here’s a little Python script I wrote which, in theory, should let you set/change the icon on ANY game in your Linux Desura library. I’ve only tested it on local ones though.

Posted in Geek Stuff | 2 Comments

Good Villain, Bad Villain

The difference between a well-done villain like Ursula in Disney’s The Little Mermaid and a poorly done one like Rasputin in Don Bluth’s Anastasia is actually pretty simple. With a well-done villain, you can understand and sympathize with whatever made them who they are… you just can’t agree with the decisions they made as a result of that past. With a sub-standard villain, they come across as too simplistic.

Rasputin is a good example of this because his primary motivator is obsession. While that’s a perfectly realistic motivator, it takes a lot of work to make obsession work in a literary context even though it’s actually a fairly simple emotion. (I know people who are at least as obsessive as Rasputin and have hurt others along the way, but truth is stranger than fiction after all.)

If you compare Ursula, she’s actually a surprisingly well-rounded villain for a Disney cartoon. Especially when you realize that, in a setting that seems completely innocent, she sings a “villain song” that hints at prior experience with misogyny and offers a readily metaphorical deal where a young girl can get her man but must give up her voice and her family.

Good characters lend themselves more readily to more constructive adjectives like “cynical” and “misanthropic” (Ursula) which, even on their own, readily imply potential character histories. Rasputin’s descriptors, by contrast (obsessive, evil, undead, basket-case), aren’t as helpful.

In other words, a good villain is someone who could have been a sympathetic protagonist but chose to go about things in a completely unacceptable way.

Posted in Writing | Leave a comment

Freeman’s Mind, Humor, And Themes As A Writing Aid

From a structural standpoint, one of the biggest reasons Freeman’s Mind works so well as humor is that, in the Half-Life story, Ross Scott’s version of Gordon Freeman embodies not one or even two, but three humorous juxtapositions which can take turns to ensure a steady flow of humor.

First, since Half-Life’s story and character interactions were written to imply a mature, serious, respectable silent protagonist, it’s quite amusing to see an immature character with an apparent lax sense of right and wrong slotted into those interactions (As long as it’s done so it still makes sense) …not to mention the inherent humor potential in characters who somehow managed to achieve a university-level vocabulary and understanding of science, philosophyand history without gaining any maturity in the process.

Second, most of the time, Scott’s Freeman seems to treat his mental tangents with equal importance to the serious, immediate concerns of his surroundings. For example, when he returns to the giant, noise-making alien in the rocket test chamber and, having just been daydreaming about riding a sea turtle to rob cruise ships, comments “Wow. You may be a reptile, but you’re really dangerous. You’re not like a turtle at all. I don’t like that.”

Both are forms of generating humor by denying a person or situation the respect/reverence/seriousness/deference we expect and, by using two or more humorous themes, you reduce the need to have one carry the whole burden of finding something to laugh about in every location or interaction.

Finally, Freeman tends to come to conclusions very notably different from what the designers of Half-Life intended (eg. that the soldiers who are trying to kill him are obviously immature jerks who don’t understand the meaning of “rescue operation”) and occasionally make obvious-in-retrospect comments to support those viewpoints. (Jokes based around not ignoring the rough edges in the immersiveness of the game can also fall into this category.)

Fundamentally, all three boil down to the same core pattern which underlies all humor on the listener’s side: Make sure the audience has expectations, then break them in a way that doesn’t cause them discomfort. (Humor on the joke-teller’s side is generally rooted in empathy and social connectedness since it’s usually impossible to tell a joke properly before you’ve let its effect on you wear out)

Obviously these aren’t magic bullets but, by providing three different “jokes in the abstract”, they make it much easier for someone with a sense of humor to know where to start looking to find humor in any given situation. In other words, they’re excellent as guards against writer’s block or tools for coming up with multiple jokes and then keeping only the best one.

You can even go further and subdivide your themes into more specific areas if you feel it’ll help you to keep better track of which topics are getting overused and which ones might offer alternative jokes.

Posted in Writing | Leave a comment

My Sites: Now 100% Google-free (FeedBurner excepted)

I’ve been meaning to minimize Google’s ability to track me and my visitors for quite a while now, but this post about Google+ which I stumbled across recently came as an appropriately timed reminder.

I’d like to announce that every site I remember running (it’s conceivable I forgot ones on free hosting similar to GitHub Pages) is now free of Google Libraries API and Google Analytics (I use a private Piwik install for stats now and I can assure you I’ve dialed data retention way down). I’ve never had a need to use reCAPTCHA even back when they weren’t owned by Google.

I still need to find alternatives to FeedBurner, FriendFeed (owned by Facebook), and Google Docs and find time to replace GMail and Google Talk with stuff I host myself, but it’s a good start.

(You never really realize how much Google knows about you until you start migrating yourself off them)

In the interest of full disclosure, here’s an up-to-date list of all 3rd-party dependencies in my sites which could theoretically be reworked for abuse:

blog.ssokolow.com
  • ProjectWonderful Ad
  • FriendFeed sidebar widget
  • Flattr buttons (Live counts require external requests)
  • FeedBurner feed analytics
Vanished FanFic Archive
  • Fic downloads are currently served separately from Dropbox.com
  • FeedBurner feed analytics
GitHub Pages: QuickTile
  • Flattr button
  • GitForked Button
GitHub Pages: The Procrastinator’s Timeclock
  • Flattr Button
  • GitForked Button
Gender-Bending Index
  • FeedBurner feed analytics
Fanfiction Story Ideas
  • FeedBurner feed analytics
Posted in Geek Stuff | 4 Comments

WordPress: Flushing Spam Comments Quickly

While I love a well-configured copy of Spam Karma 2 as an effortless way to keep my blog free of spam, it doesn’t seem to have a way to automatically purge/expire comments after they sit in the spam bin for a while.

Until I can come up with a more automatic solution, here’s the simplest way to purge all binned spam in a few seconds using the phpMyAdmin SQL query window (it’s an icon in the sidebar once you’ve picked a database):

  1. Delete all comments marked as spam:
    DELETE FROM wp_comments WHERE comment_approved="spam";
  2. Look up all newly orphaned comment metadata entries:
    SELECT m.comment_id FROM wp_commentmeta AS m 
      WHERE (SELECT COUNT(c.comment_ID) FROM wp_comments AS c 
        WHERE m.comment_id = c.comment_ID) = 0;
  3. Click “Check All” at the bottom of the results list. (MySQL doesn’t let you subquery a table you’re deleting from)
  4. Click “Delete” to the right of “Check All”.

How much faster is this? In the same time it took me to purge 20 comments via the WordPress web interface, I used this process to remove the remaining thousand or so. The actual SQL statements took fractions of a second.

Still… this is a bit outside my usual area of SQL expertise (I normally work mostly with SQLite and simple INNER JOINs), so I welcome cleaner, more automatic ways to do this without resorting to a little PHP or Python script to pass lists of IDs between statements.

Posted in Geek Stuff | 2 Comments