Stephan Sokolow's Blog

A few days ago, my main machine went kaput and I needed a fast way to copy several hundred gigs of data to a slow, old 2Ghz Celeron. Thus began my search for the lightest file-copying solution I could find. So far, that’s tar with netcat.

The problem is that netcat, while a swiss army knife, is also very much a power-user’s tool and, with at least three flavors and multiple versions of each in use, the instructions for using netcat with tar to copy files are often using a subtly incompatible syntax.

It also doesn’t help that, because netcat has no clue what you’re planning to do with it, it takes a little experimentation to figure out which instructions match up with your expectations for how a file-moving tool should behave.

Here is a little wrapper script I wrote myself (¾ comments) to keep the use of tar+netcat simple and easy to remember. Use it as you will.

If you aren’t on a LAN and bandwidth is more the issue, you can either compress the tar stream the same way you’d compress a tar archive (check out lzop for something lighter on the CPU than gzip) or you can use a tool like rsync which will only transfer changes and makes resuming an interrupted transfer effortless.

You may also want to check out these pages for alternatives:

• How to transfer large amounts of data via network
• BackupYourSystem – Ubuntu Wiki (Also BackupYourSystem/TAR)

After wasting far too much time trying to rebase my Gender-Bending Fiction Index on top of CakePHP, I decided that I’d give it one more try before consigning it to the list of things indefinitely on hold… and I’m glad I did.

My first concern was to avoid failing because of “too much convention”. CakePHP is a nice framework, but if I’m going to be letting a framework reduce my site’s performance by an order of magnitude, it had better give me some proportionate benefit.

(Like many web hosts, mine doesn’t run a PHP bytecode cache. That means that any representative “Hello World” for a big-name framework like CakePHP or Zend or CodeIgniter will generally serve 100 requests per second on hardware that’ll do 1000 with bare PHP. It’s just a side-effect of PHP only letting compiled extensions stay resident between requests.)

I don’t remember exactly how, but I ended up searching for PHP micro-frameworks, and stumbled across Fat-Free Framework 2.x.

At the highest level, the important things to know about it are:

• The closest PHP analogue I’ve yet found to Bottle.
• GPLv3-licensed with the option to buy your way out of it
• Requires PHP 5.3 for its anonymous function support (Think JavaScript)
• In every performance benchmark I’ve been able to dig up, it’s generally in the same ballpark as bare PHP.
• Includes a simple template language, URL routing, a ton of useful helpers, and ORMs for SQL, NoSQL, and flat-file data storage.
• Easy to swap out pretty much anything but the URL router.
• The core, plus all included plugins, weighs in at 288KiB.

I haven’t spent that much time with it yet, and I didn’t need things like CAPTCHA-generation, Google/Yahoo/Twitter/Akismet API wrappers, RSS/Atom parsing, OpenID integration, Identicons, Thumbnailing, etc., but here’s what I’ve poked around with so far:

Documentation

…being the most important part of any framework.

F3’s documentation isn’t the best in existence, but aside from one small rough edge to be wary of, it was definitely good enough for me.

Specifically, the website seems to be exclusively focused on the tutorial-style guides to using various features, so unless you read carefully, you’ll assume there are no API docs. I used the CHM-format copy of the API docs included in the archive at lib/f3.chm.

Architecture

Requiring PHP 5.3 allows F3 to really shine when it comes to architecture. Not only do you get to feed in an anonymous function anywhere that older frameworks might require you to reference things by strings, but it also makes heavy use of classes to implement minor fixes for PHP rough edges. (While still giving you a choice between procedural and object-oriented styles for your own code that is satisfyingly reminiscent of Python.)

For example, one of the big rough edges I’ve seen for people coming from other languages is PHP’s scoping rules for globals. F3 works around this by offering you the option of using its own internal globals system via F3::set and F3::get. (And including a shorthand for it in its template language)

This “offer a streamlined alternative, but don’t force it” pattern is present throughout the framework. For example, Web::isajax is the familiar boolean-outputting check for XMLHttpRequests and Web::http lets you perform an HTTP request and retrieve the content in one line and is based on fsockopen() for portability.

It also includes a nice, simple config file parser in the core which populates the same data store it looks to for configuration (F3::set/get, again) , which makes good project architecture much simpler.

URL Routing

Nice and simple, yet very powerful.

F3’s URL routing works by taking a string like GET /s/@story_id/* and mapping it to your choice of:

• An anonymous function
• A comma-separated list of PHP files to require
• A comma-separated list of functions or methods to execute (It’ll instantiate the classes for you)

It’s also fully compatible with:

• PHP 5.3 namespaces
• Autoloading
• The use of non-GET/POST HTTP methods
• Defining your permanent redirects in your routing table
• Defining routes in an INI-format config file if that’s what you prefer
• Dynamic routes which look up classes or methods at runtime. (Python programmers will recognize this pattern from Pylons)

Templating

Basically something half-way between PHP and Django-style templates, but it is a plugin, so you don’t have to waste the kilobytes if you prefer something else.

It uses XML syntax like <F3:check if="..."> for the block-level stuff and, aside from the need for <F3:true> and <F3:false> when defining an else case, it’s actually not that bad.

There’s no inheritance syntax though, so if the more PHP-esque <F3:include href="{{@child_template}}"> doesn’t do it for you, I suggest hooking in Twig instead.

The include syntax is where it really shines though. F3 basically aliases a Django-style {{...}} syntax to <?php echo ... ?> and provides a shorthand for accessing its alternative to PHP globals: {{@orm_result->title}} and {{@array.key}}.

Despite the ability to embed raw PHP in the {{..}} blocks, the usual caveats for Django-style templates apply.  Don’t try too hard to circumvent the separation between logic and presentation or you’ll bump up against the limits of the language. (For example, you can only use functions in bare {{...}} chunks, not as a source for the looping construct.)

ORM

I’ve only used the SQL ORM and only with SQLite so far, but I quite like it. It doesn’t do joins, recommending that you use CREATE VIEW in your database schema instead, but it’s truly zero-configuration. Just instantiate Axon with a table/view name, and you’re ready to go.

That may seem like a downside, but remember two things:

1. It’s a 23KiB plugin that you’re free to replace with a beefier ORM.
2. This approach actually makes it a lot easier to build on a database that already exists.

I actually prefer it this way, since I’m using F3 to rewrite an existing, horrendously messy PHP app in a cleaner way.

Helpers

Here are the helpers I haven’t yet tried which you’ll probably want to know exist:

• Forms
• Profiler and Debugging
• Unit Testing
• Spam DNS Blacklist integration

Here’s what I have to say about the few helpers I have used so far:

CSS/Javascript Minifier

Works great for CSS, but there’s something about my Javascript that it mangles. Probably not a big issue since most of my JS is pre-minifed plugins and it’ll be the work of a minute or two to whip up something that concatenates a few K of un-minified local JS onto the dozens of kilobytes of pre-minified jQuery UI.

XML Sitemap Generator

Simple and easy to use, but there’s no way to tell it to ask search engines to index less frequently than the cache timeout and, on a database-backed site, the runtime complexity of walking the site (even without HTTP round-trips) makes it unfeasible.

Verdict

I’d use it as a basis for my PHP creations any day of the week. (Of course, I always GPL them anyway) It’s ultra-light (both for size and performance), easy to plug other bits and pieces into, and adapts to my style, rather than expecting me to adapt it its conventions.

tl;dr Install this custom HTTPS Everywhere rule.

If there’s one thing you can set your watch by these days, it seems to be Google’s dedication to making it as difficult as possible to get what you want out of their services without logging in. NoScript isn’t very helpful because everything except GMail and Google Maps is served from www.google.com, userscripts and other hacks seem to break every other weekday, etc. etc. etc.

Today, my gripe is with how, if you disable JavaScript on encrypted.google.com via NoScript to reliably and anonymously get a simple, basic search engine experience, every single result requires you to manually OK an untrusted redirect from the https://encrypted.google.com/url?url=... nowhere that NoScript stops you at.

As this is most evident when you’re using HTTPS Everywhere (most userscript authors don’t support encrypted.google.com), I’m going to show you how to use HTTPS Everywhere to kill the redirects and fix the problem. (And since we’ll be fixing it AFTER the browser has done the hard work of finding the URL, it will only break if Google actually cares enough to change their redirect URL, rather than every time they “innovate” with their JavaScript or HTML templates)

For those who aren’t already familiar with it, HTTPS Everywhere is a generalized engine for hijacking requests and pointing them somewhere else without ever contacting the original target. (Though, normally, the intent is to keep your ISP from knowing where you’re going by snooping on your traffic)

That means that, when your “friend” tricks you into clicking a link to search www.google.com for “hot sexy six-year-olds”, it’s redirected to the encrypted version of Google before anything goes over the wire… which means you only have to worry about Google’s friends in the FBI, not your ISP’s too.

The reason that matters is that you can write your own rules using regular expressions, so you can redirect anything to anything like this. (No practical jokes please. Slipping a custom “PirateBay -> FBI” redirect into your friend’s browser is not cool.)

This rule, if placed into the HTTPSEverywhereUserRules folder in your Firefox profile, will skip over Google’s tracking URL and take you direct to what you actually wanted:

I’ve always felt that engendering nouns in a language when they don’t inherently have gender is a mistake. Partly because I suspect it taints one’s perception of the world, given how a culture both shapes and is shaped by their language, but mostly because it makes mistakes much more likely with little benefit¹.

Skilled designers of programming languages know well that you can’t make writing bad code harder without hurting everyone, but you can make writing good code easier². A good programming language strives to make the best way of doing things also the most intuitive but is designed with the understanding that it must not be a burden to skilled users³.

I’m the first to admit that English has its problems, such as having at/in/to/into and do/make when French proves you only need à and faire, but aside from making it easier to have recognizable, intuitive masculine and feminine variants of the same given name, all French’s noun engenderment seems to do is complicate the language for learners, increase the number of details you have to keep track of, and introduce opportunities for further counter-intuitive quirks down the line.

Why masculinité (masculinity), moustache (moustache), and barbe (beard) are feminine nouns while féminisme (feminism) is masculine, I’ll never know, but I strongly suspect it’s because natural languages are living things and, as frustrating as it is for language professors, ordinary people with more pressing concerns than correct semantics are a big part of the process which drives languages to grow and change.

^1. I also have opinions on which types of pronouns should and shouldn’t be engendered, but let’s stay focused.

^2. The Java and Python programming languages demonstrate this contrast well.

^3. Unfortunately, it’s also human nature for people to react emotionally to criticisms of something they’ve deeply invested themselves in, so many programmers will excuse or even defend flaws in their language of choice as misunderstood features.