Jon Canady

GPS Failures and User Experience

2010-03-08T00:00:00-05:00

My GPS taught me a valuable lesson about user experience the other day.

I normally drive on the highways whenever possible, but there was a massive snowstorm in Columbus one evening and I told my GPS to ignore highways due to numerous accidents.

I'd forgotten that I'd set that option the next time I needed the GPS. So I turn it on, enter my destination, and get on the highway before it's able to get good enough reception to calculate a route. Of course, since I'd told it to forbid highways, it wasn't able to find a route that it could accept, and it failed.

The problem is, it told me so with a paragraph-long error message about how it couldn't locate the origin. In the middle of driving down a freeway, it's hard to parse that kind of crap.

Web applications also run into similar issues: how do you handle the case where a user asks to do something, but an option she's set prevents her from doing it?

Good: Explain to the user the problem using non-domain-specific terms. In my example, a quick "We're unable to generate a route that matches your options." would work.

Better: Explain the problem, and provide a suggested solution. "We cannot generate a valid route. Please check Route Options (under Settings) and try again."

Best: Explain the issue and provide a one-click way to fix it. "We can't generate a valid route because you've turned Forbid Highways on. [Allow highway access] [Cancel this route]"

The Security Magic Bullet

2010-02-26T00:00:00-05:00

Screw input validation, love output normalization.

That was a recent comment on Hacker News, probably made by a programmer. How are there still people that don't understand there's no magic bullet?

Assume hostility
Input validation
Enforce business rules
Output sanitization

Output normalization fixes your XSS attacks and SQL injections, sure. But it doesn't help at all if I send 'admin=true' to your /users/me/update URL or I give you a comma-separated list of 50,000 emails to add to your mailing list.

If you have REST, why XML-RPC?

2010-01-14T00:00:00-05:00

Update

There's points going back and forth on Hacker News today -- I don't know if HN archives its threads forever, so if that link breaks let me know.

The original article follows

There's a large re-architecture project for one of our products at work that's in the planning stages, and we're trying to decide between XML-RPC and REST for communication between two layers. The title of the post should make my bias pretty obvious.

XML-RPC

XML-RPC is just a protocol that defines how you can call methods on a remote system. It works by serializing your request and any parameters into a standard chunk of XML, and similarly encoding the response and data from the server. Its up to the client and server to agree upon what methods are available and what parameters they take, although this can be facilitated by providing that data as an XML file that clients can consume, which most implementations auto-generate for you. It also provides for standard responses that indicate error codes and descriptions, which are up to the implementor to create.

Wait, what's wrong with that?

It's completely unnecessary. We're building a web service, which runs over HTTP. HTTP already has facilities for making requests and returning responses. Plus, its error states are already well known, and it provides for other things, such as client-side caching and authentication, that XML-RPC doesn't.

So what's REST?

Pretty much that. REST over HTTP says you use all the things HTTP gives you, and you provide representations of your resources. You do this with the Web all day: when you go to someone's Twitter profile, for instance, Twitter is giving you an HTML representation of a resource they have, namely a Twitter user.

But what's the point?

Let's say you know I provide Employee information for a company, and you need to get information for a specific employee. You've come to my site, and read the documentation for my service so you know how to interact with it.

With XML-RPC

In your programming language of choice, fire up an appropriate library and have it read in my auto-generated XML description of my service. Then, call the SearchEmployeesByName() method and give it the employee's name. Your library will create the XML for the request, decode the XML from the response, and likely give you a native object that you can manipulate. You can then find the employee you want and call GetEmployeeDetailsById() and pass the specific employee ID you found in step one.

With REST

If you just need some data, open your web browser and go to http://my-fake-service.com/employees/. Click the employee name you want, which might take you to /employees/121. I've provided you the HTML representation of the resource (a web page). You're done.

Assuming you want to access the data programmatically, make an HTTP request to /employees.json and get a JSON representation of my employees. Then, once you've have the ID, make a second request to /employees/121.json, and decode the JSON response.

That's not all. If you need an image of the employee, try /employees/121.jpg. If you need the data as a PDF, try /employees/121.pdf. REST isn't limited to providing XML responses, it can provide any format the server supports.

And it's easy to provide a list of related links to other representations or related resources.

Why wouldn't anyone like that?

No clue. During an impassioned speech, I got a lot of questions or statements, usually things that XML-RPC handles that REST seemingly doesn't.

How do you know if an error occurred?: HTTP provides statuses for nearly any error you can encounter. You probably know about "404 Document Not Found"; there's tons more, including "400 Bad Request" (I don't understand what you want), "401 Unauthorized" (You need to authenticate first), and more esoteric but useful codes such as "201 Created" (We've created a resource based on your input), "402 Payment Required" (We need money for this), and "409 Conflict" (Someone else changed this resource before you did, you should try again). With XML-RPC you have to re-invent all of these.
XML-RPC provides automatic documentation.: Well, your library automatically generates documentation like "getUser() takes an integer and returns a UserStruct" -- you still have to define what a UserStruct is and what that integer means. Kudos if your library can automatically generate those, but now we're discussing specific implementations, which is a separate issue.
How do you know if your data's valid?: You test it, same way you do with XML-RPC. Someone actually asked how you know a date is a date if the XML doesn't specifically tell you it's a date. If I'm providing a piece of data called "expiration_date" and it's not a date, that's something the client needs to handle regardless of how it obtained it.
REST seems too restrictive.: REST doesn't force you to always talk about representations of resources, but you gain a lot of simplicity if you do. For example, you can have a "Dashboard" resource who's representation is as a bunch of overview data in whatever format.

One of our issues was "I want to provide a count of a particular resource instead of returning the whole list." You can absolutely do that: if you have /employees.json that provides a JSON list of all employees, knock a parameter on there: /employees.json?count=true -- it seems unclean, but all the logic for retrieving employee data stays in one place, instead of scattered around multiple methods like "GetEmployees()", "GetEmployeeCount()", "GetActiveEmployeeCount()" and so on.
But all our clients use XML-RPC!: Touché.

Clients, The Business Kind

If the field you're working in is dominated by companies that have (for whatever reasons) standardized on things like XML-RPC and SOAP, and you want to integrate with them, it's probably best to use XML-RPC. I like to think that REST provides obvious advantages over XML-RPC as described above, but in the end if the people giving you money want XML-RPC, then you give them XML-RPC.

Responses to this are encouraged, and quality or thought-provoking responses may be added. Twitter or email your thoughts.

Review: Kinesis Freestyle Keyboard

2010-01-06T00:00:00-05:00

I've been using my new Kinesis Freestyle keyboard all this afternoon.

Check out my Flickr set for some pictures of it in action, as well as some closeups of the issues I'll mention below.

Long Story Short

I can get past the few slight issues, and I love it.

First Impressions

I'm no keyboard connoisseur, but I know what I like, and I like the switches on the Freestyle. The keys have just enough travel and resistance to suit my tastes. They feel good, and make a pleasing sound. I don't think they're mechanical switches, but I can live with that.

The Top Function Keys

The top row of F-keys doubles as your standard Mac modifier keys, using a layout very similar to current Apple models. They've added a button that toggles Dock auto-hiding, which is neat if not useful, and a power button.

I normally hate function-keys-cum-media-keys -- I like to be able to adjust my volume, play my media, and hit function keys when I need them, without using a modifier key. Kinesis thought of me when making this keyboard: the fn key (which toggles between F1-F12 and Apple shortcut keys) isn't a modifier, it's a mode-switcher (like caps lock). When I'm in need of full F1-F12 love, I hit that key and I've got it. Otherwise, I leave it off and have my media keys and so on. I can't say how much I appreciate that.

The Driverless Hotkeys

There are some hotkeys on the left-hand side which appear to be hardware macros. Internet back/forward, beginning/end of line, cut, copy, paste, undo, select all, and the aforementioned "fn" key.

I can't see becoming dependent on them, but they're handy if you remember to use them.

Gotchas

If you're coming from a regular Ergonomic keyboard (I previously used the Microsoft Natural Elite 4000), the location of the left command key is weird: it's positioned (below) halfway between "x" and "z" instead of directly under "x". I constantly miss it.

Same with the right Shift key: there's padding between it and the up-arrow, and I sometimes shove my finger there instead of hitting the Shift.

Since it's a compact keyboard, you don't get the pageup/pagedown/home/end/insert/delete block. There's no insert key (which is fine), delete is above the backspace key (and just as wide), and the other four keys are vertically aligned on the right-hand side of the keyboard. Not the best layout, but given the constraints, not the worst.

The Last Word

I've got mine at the Innova office right now, and I'm strongly considering buying one for home.

Jekyll Helper Scripts

2010-01-04T00:00:00-05:00

I have a couple of helper scripts that I couldn't use Jekyll without:

deploy

Shamelessly stolen from The Pug Automatic, it's pretty quick (and all on one line, despite what word-wrapping occurs here):

1 jekyll --no-auto && rsync -avz --delete _site/ blog:/pub/jcanady/joncanady.com/

Run jekyll -- I keep auto-regeneration turned on in my script, so disable that -- then rsync the site directory up to my blog's host. blog is an ssh alias. I like to have my blog's repository hosted on Github, and for various reasons Github can't connect to my blog's server, so this is nearly as good.

newpost

This one is a lifesaver. I can't remember where I found the original ruby script, but after sprinkling a little optparse over it, it became an indispensable utility; just drop it somewhere on your $PATH (~/bin/ is nice).

 1 #!/usr/bin/env ruby
 2 
 3 ###
 4 # newpost -- creates a new Jekyll post
 5 # Change the blog_dir variable below, and you should be set.
 6 # For maximum win, put in ~/bin/ (and add that to your $PATH)
 7 #
 8 # Jon Canady, 2010. 
 9 # Covered by the WTFPL: http://sam.zoy.org/wtfpl/
10 ###
11 
12 # omit the trailing slash, please
13 blog_dir = "/Users/jonc/Sites/joncanady-blog" 
14 
15 
16 require 'optparse'
17 
18 options = {}
19 
20 optparse = OptionParser.new do |opts|
21 
22   # banner
23   opts.banner = "Usage: newpost [options] \"title of post\""
24 
25   options[:draft] = false
26   opts.on('-d', '--draft', 'Create post as a draft') do 
27     options[:draft] = true
28   end
29 
30   options[:format] = :markdown
31   opts.on('-f', '--format FORMAT', 'Post is in [markdown|textile] (default markdown)') do |format|
32     valid_formats = %w(markdown textile)
33     puts opts && exit unless valid_formats.include?(format)
34     options[:format] = format
35   end
36 
37   opts.on( '-h', '--help', 'Display this screen' ) do
38     puts opts
39     exit
40   end
41 
42 end
43 
44 
45 
46 begin
47 optparse.parse!
48 rescue OptionParser::InvalidOption => e
49   puts e
50   puts o
51   exit 1
52 end
53  
54 if ARGV[0].nil?
55   optparse.display
56   exit 1
57 end
58 
59 puts "Generating post..."
60 
61 date_prefix = Time.now.strftime("%Y-%m-%d")
62 postname = ARGV[0].strip.downcase.gsub(/ /, '-')
63 
64 post_dir = options[:draft] ? '_drafts' : '_posts'
65 
66 post = "#{blog_dir}/#{post_dir}/#{date_prefix}-#{postname}.#{options[:format]}"
67  
68 header = <<-END
69 ---
70 layout: post
71 title: #{ARGV[0]}
72 ---
73  
74 END
75  
76 File.open(post, 'w') do |f|
77   f << header
78 end
79  
80 system("mate", "-a", post)

I bundle both of these with my blog's source; the latest versions can always be found in my tasks/ directory.

Android Basics My iPhone Still Can't Get Right

2010-01-03T00:00:00-05:00

Man, Android's starting to look good. The interface is not nearly as polished as the iPhone's, but the iPhone's failing on other levels which bothers me more and more every day:

Notifcations: I really love Android's pull-down notification center. The iPhone's handling of notifications is infantile by comparison:
- You get one dialog on the lock screen that shows everything.
- The minute you unlock the device it goes away and leaves you with individual app badges as the only notification that something happened.
- Third-party apps can use icon badges but they can't use the lock-screen notifications, so even if your twitter client is getting push notifications, it doesn't show on the home screen.
- Not to mention that I can't respond to an SMS without quitting what I'm doing.
3rd-Party App Backgrounding: My iPhone Twitter client can't pull messages if it isn't running. Things for iPhone can't update its dock badge without being run once, which makes the dock badge near-useless. And I can never have something like Locale, which is too awesome for words.
Customization: Hey Apple, why can I change my ringtone but not the New Mail, New SMS, or Calendar Alert sounds? Why can't I put a background behind the app icons? And why can't I replace the dialer with Google Voice?
Mail Client: To be fair I've never used the Android mail client, but it's gotta be better than the MobileMail.app. No unified inbox, no spam filters, no PUSH IMAP for GMail, and no "Mark All As Read" option. Nearly useless.

I'm still sticking with my iPhone, but if Apple doesn't up its game with its 2010 offering, it may be time to jump ship.

Switching to Jekyll

2009-12-28T00:00:00-05:00

Overnight there's been some pretty major changes around here.

Design

Inspired by various blogs and products, I've taken up a more minimalist design. I've long since grown tired of overly-active blog themes with way too much cruft in the sidebars that detracts from the actual content. It's a work-in-progress, and comments are welcome. Speaking of comments:

Comments Gone

As a side effect of updating blog software (more below), comments are now gone. The new system doesn't support comments out-of-the-box, and I'm seeing that as a feature. I can't remember a single time that comments on a blog have been very helpful, but can point to thousands of examples of comments detracting from a piece.

If you want to comment on a post, my twitter and email are well-published here; feel free to send me feedback. Or, feel free to write something on your own site; if it's well done I'm happy to update my post and link to it. Manual trackbacks, I suppose.

Jekyll

The meat of the update is that I've ditched Wordpress for Jekyll, a blog-aware, static site generator written in Ruby. Instead of having a database of blog posts and pages, I now have HTML/Markdown and files that Jekyll turns into a static web site. The benefits of this system are enormous:

Everything's a File

Since everything's based on flat files, I can store everything -- posts, layout, content pages -- in Git, edit everything in whatever editor I'm feeling like using (TextMate, Emacs, Vim, what have you), and that's all there is to it. Gone are the days of periodically running mysqldump on a wordpress database and stashing tarballs somewhere.

No More Wordpress Themes

I never got my mind around writing Wordpress themes, which means if I ever wanted my site to have a different visual theme I had to crack open Wordpress docs and try to modify whatever theme I was using. Nuts to that.

Now I write CSS and HTML with a little Liquid templating thrown in and I'm good to go. If I wanted, I could switch to Haml/Sass and use that instead, which is amazing.

No Backend

Git and rsync are all the backend I have to speak of. There's no database to get overloaded, and no admin panel that needs esoteric plugins to secure it because the default install can't figure security out. My webserver serves non-interpreted HTML files, so the only limit is the capacity of my server and pipe.

If this all sounds awesome to you, I'm putting together a quick tutorial on how to migrate a Wordpress site to Jekyll which should be up in a few days.

HTML 5 Video Annoying, Usable Today

2009-12-21T00:00:00-05:00

Gruber suggests that HTML 5 video is "effectively unusable" for the following two reasons:

It doesn't allow poster images.
Both audio and video always auto-buffer instead of waiting for a click.

Our hero gets past the first issue with some Javascript hackery, which also fixes the second issue. So, completely usable before (with a few issues), 100% usable afterwards.

There is the ever-present IE issue -- no version of Internet Explorer as of this writing supports <video> or <audio> -- but this isn't at all an issue for Gruber.

The effective conclusion to Gruber's "effectively unusable" piece is "I used it for my PastryKit video page but I had to use Javascript and create two versions of each video and I don't like that."

Which seems, y'know, usable to me.

Two Simple Rules for Screencasting

2009-11-20T00:00:00-05:00

During the process of trying to figure out what Compass is, I tried to watch the official Compass screencast. This was a mistake. Thinking of writing an introductory screencast for your product? Learn from their mistakes.

Keep It Short

No shit, the Compass screencast is an hour long. The only people who are going to watch that are people with a vested interest in the project, who presumably already know the material.

Keep your screencasts to around five minutes. You aren't PeepCode, and this thing isn't your documentation. If you can't keep a basic explanation of your product/service down to five minutes, you have bigger problems.

Show, Don't Tell

Up until about 2:20, the Compass screencast doesn't even change, and it takes another minute before there's any actual content on the screen. If the viewer wanted to read a thorough explanation of your product, they would have read the "about" page.

Start with a brief intro on what your product is, then jump right into the demonstration. Impress the viewer quickly and keep them impressed.

It's That Simple

That's all you need for an effective screencast. It's almost a definition: "short" and "demonstration". It's hard to go wrong.

How To Fix a Shitty Project

2009-11-18T00:00:00-05:00

Way back in the Dark Ages of the Internet (2000), Joel Spolsky wrote of Netscape's impending version 6 release:

It's a bit smarmy of me to criticize them for waiting so long between releases. They didn't do it on purpose, now, did they?

Well, yes. They did. They did it by making the single worst strategic mistake that any software company can make:

They decided to rewrite the code from scratch.

Most developers can instantly conjure up at least one project they'd love to rewrite from the ground up. This time, you'll use the right language, the right libraries, the right style -- you'll do it right.

Don't do it. It's a horrible idea. Now you've reset your entire timeline back to zero, and you don't know when you'll be done. (Say, where's TextMate 2?) Not to mention the regression bugs you'll undoubtedly let slip in.

Instead of rewriting the whole thing from nothing, incrementally fix things. Don't commit a single change that doesn't leave the code better than you found it. Move business logic out of controllers and into models. Changing a feature? Cover the whole feature with tests.

Eventually, you'll find you've rewritten all the shitty parts of the project anyway and it's not so bad anymore.

Multi-Developer Git Suggestions

2009-11-08T00:00:00-05:00

Git's pretty simple to use when it's just you and your Github repo, but as developers increases, complexity increases.

Each Developer Has Her Own Branch

Make each developer on the team maintain their own branch where their changes initially go before being merged into master. Bonus: you can vet new developer's changes before they get pushed into whatever the production branch is.

So, here's what I do:

jonc$ git checkout -b jonc

Create and switch to the branch jonc.

jonc$ git checkout master
jonc$ git merge jonc

Merge all the changes from jonc into master (This, of course, works in reverse, if you need to pull new stuff from master into your personal branch)

jonc$ git push origin jonc:refs/heads/jonc

Now the jonc branch exists on Github. Anyone else that wants to check out my work is free to do so:

otherdev$ git checkout --track -b jonc origin/jonc

Features Have Their Own Branches

If a feature is going to take longer than one sitting to completely implement -- including tests -- then it goes in its own branch. Every time I've excepted this rule I've regretted it later.

jonc$ git checkout -b bugfix_553

If you're going to be collaborating on something with another developer, create it remotely first, then both of you can push/pull from the get-go.

jonc$ git push origin origin:refs/heads/bugfix_553

That creates the remote branch, and any developer that wants to start working on it does the following:

jonc$ git fetch origin    
jonc$ git checkout --track -b bugfix_553 origin/bugfix_553

When you're on the bugfix_553 branch, pushes and pulls go to and from there instead of master. When ready, you merge it into master as usual. Once you're 100% done with the branch, delete it as normal (git branch -d bugfix_553) and remove it from the server if necessary:

jonc$ git push origin :heads/bugfix_553

No, that command isn't intuitive in the slightest.

Production Always Builds

The production branch should never have code in it that breaks. Hopefully it's got a good test suite, and with any luck it doesn't incur much technical debt, but at the very least it needs to:

Run
Produce correct output

This means that deployments can happen whenever. Developing directly in the production branch or merging changes prematurely means that if critical bugfixes need to go out now you can handle that without having to clean up messes first.

Developers that consistently push bad code into the production branch should be beaten. Having some Continuous Integration process watching that branch isn't a bad idea either.

(I've always used master as the production branch, but use whatever makes you feel good.)

Flickr is Not Your Backup Service

2009-08-07T00:00:00-04:00

Dude's Flickr account was compromised, his password changed, and then his account terminated. He seems upset.

I want this to spread like wildfire. I want people to know that Flickr, their customer service and overall infrastructure is incredibly disappointing and it needs to be fixed ASAP.

...and...

There were 3,000+ photos on my Flickr photostream from 2004-present. Thousands and thousands of dollars spent on film and developing were pumped into that account and it vanishes within minutes and it�s been days and still I�ve found no resolution with my case.

Flickr is an awesome web app for sharing your photos with the world, but it is not an archival service. Flickr happens to host a load of digital images on its servers as effect of providing these photos to the world, not for the purpose of storing them in perptuity.
Dude spent "thousands and thousands of dollars on film and developing" and didn't keep local backups of his work? Did he just upload the files to Flickr and then delete them? Come on.

On the assumption that he didn't keep backups: it's not that hard. If backups are kept, than at most a bunch of metadata was lost, but then the argument isn't "I wasted a bunch of money developing photos that you deleted" but "I spent hundreds of hours tagging and describing and all for nothing" -- that sucks, but that isn't nearly worth this level of bitchitude.

Awesome One-Liners

2009-06-10T00:00:00-04:00

If you just used the built-in Rails generators to create a new controller and you got a bunch of .html.erb files, you'll naturally want .html.haml files instead. Bash can fix those files right fast:

iPhone Lock Code and iTunes 8.2

2009-06-01T00:00:00-04:00

Installed the iTunes 8.2 update, docked my phone, and saw a new error message:

I don't know how to feel about this. The security-conscious (read: paranoid) part of me thinks it's probably a good idea to not let just anyone do a sync of your locked and protected iPhone. The lazy jerk part of me that just wants to slap his phone in a dock and have the damn thing sync is a little miffed right now.

I wonder how helpful this really is? Is the control built into iTunes or the iPhone itself? Could someone not using iTunes pull a dump down of the iPhone's HDD?

Know when to Fold

2009-05-19T00:00:00-04:00

From Phillip Greenspun's "Ruby on Rails and the importance of being stupid", a comparison between a hastily thrown together .NET site and a hastily thrown together Rails site:

How could a �slice� with 800 MB of RAM run out of memory and start swapping when all it was trying to do was run an HTTP server and a scripting language interpreter? Only a dinosaur would use SQL as a query language. Much better to pull entire tables into Ruby, the most beautiful computer language ever designed, and filter down to the desired rows using Ruby and its �ActiveRecord� facility.

Now, this is Greenspun; he knows when he's writing a biased, one-sided article to make a point. I'm not certain where the comparison to .NET fits in, but otherwise there's a fine point present.

Know when to hold 'em

Rails, like every other framework in programming, wants to take something inherently complex and make it simple. And, starting out, there's no reason not to use every time-saver you can get: use #find, take advantage of associations, use script/runner to power your cron runs, why not? Any time you take second-guessing the framework unnecessarily is time you're not spending solving your business problem.

Know when to fold 'em

The honeymoon will eventually be over and your site will start to bog under the pressure. Maybe you need some database indices, which ActiveRecord's migrations won't easily provide for you. Perhaps ActiveRecord is generating sub-optimal queries and you need to step in and (shock) write some SQL. In fact, script/runner loading your entire Rails environment just to run a cronjob may not be the speediest solution.

When these happen, go with it. If you don't already know about how a column index works or how to hand-craft a funky JOIN, then go learn. Write cron scripts that just execute some SQL against the server. Don't futz around with more RAM you don't need, or trying to shoehorn your problems into Rails' solutions. Instead, find your problem points, and don't hesitate to abandon your framework in places if it's holding you back.

With apologies to Kenny Rogers.