Free
The technical and societal significance of open systems in general and the free software movement in particular crystallized in an interesting light for me the other day.
During a recent conversation with another techie, we discussed the rapid emergence of Big Data, large-scale data processing and analytical insights that were once completely out of reach.
Then she asked me,"Why do you think this is happening now?"
A Simple Question
That simple yet pregnant question was not something I had pondered before, and yet an answer came very quickly and naturally: the free and open-source movements, or rather the free and open-source mindset, made it possible.
One major benefit that free-speech OSS brought to society was that it started millions of conversations on what to do, how to do it, and more importantly how to do it better, faster, and more securely, without any of the shackles of commercial software development. If your open piece is better than mine, I'll just put it into my product and make it better for the rest of the world. Without this openness and instinct to share ideas publicly in order to get at a better, more thoroughly vetted idea (aka standard), this extraordinary accomplishment known as the internet, which all of us are still in the process of inventing on a daily basis, would simply never have happened.
Hardcore GNU folks often pooh-pooh the importance of the free-beer part of F/OSS, because of their higher ideal of freedom and openness. But I think that's misguided. Free-beer OSS made it possible for millions of people to learn software development because they didn't have to choose between paying the rent and paying for a software license for an operating system, database, programming language, or IDE. They also didn't have to pay a dime to learn those languages, because OSS extends beyond software to free downloadable books, tutorials and help manuals. Where would computing be today if it weren't for those computer scientists, software developers and other technologists for whom free-beer learning was the only option?
Combine those two general benefits and you get to a point where a huge number of people write massive amounts of high-quality software to handle interesting, complicated ideas that goad hardware manufacturers into ever-faster progress on ever-cheaper devices, and you get to a critical mass of mental energy and pervasive hardware availability (read: large stacks of cheap computers) that gives rise to a true step change in how much we can do, how many questions we can answer, and how quickly and cheaply all of this can be achieved.
Open and Shut
That's not a factual, falsifiable answer and I'm not claiming to hold any deep scientific understanding of the truth about the state of computing circa 2010. But whatever one's opinion on the matter, I'd still say freedom, openness and community-driven standards in software and hardware development played a big part in making the computing world better, and so the real world.
Another way to put it, with all due respect to a great businessman and technology visionary who's still the exemplar of secrecy par excellence and won't embrace the virtues of openness: if the human body hadn't been jailbroken, Jobs wouldn't have a new liver. A bad deal for everyone.
2010-08-10
2010-07-13
Remove Twitter from Google search results
If you (like many others) like your Google search results unencumbered by the Twitter scroller Google added a while back, paste this into your browser's address bar:
javascript:(function untwit(){var r=document.getElementById('rtr');if(r){r.parentNode.removeChild(r);}})();You can make that into a bookmark, name it "untwitter" and save it in your bookmark toolbar. You can then click on the bookmark to remove the scroller.
Only tested on Firefox 3.6.
2010-03-16
Simplicity
If I Had a Hammer
It is sometimes said that software makes difficult tasks easier. Often the task is difficult not because the problem being solved, or the question being answered, is difficult per se: "I need to know when our users are likely to cancel their membership so we can send them win-back promotions" is a well-formulated, easy-to-understand, bounded question, and the answer is likely to be equally finite and easy to understand. The difficulty lies in gathering, compiling, and analyzing vast quantities of data in a timely, repeatable, reliable manner so that 1 billion data points coalesce into a half-dozen usable pieces of information. That's hard, and intractable unless you can process gobs of stuff really, really fast. Computers, and good software engineers, are really good at this sort of thing, and by and large you can never have enough of either.
All Nails
But a rampant trend in tech companies is the tendency to resort to technology and software to make easy tasks difficult.
Imagine a meeting room filled with 20- and 30-somethings, all endowed with solid educations, a solid drive to do the right things, and the disconcerting ability to explain the difference between TCP and UDP to finance people. The topic at hand: how to foster new ideas, team spirit, information exchanges, the kind of collaborative awesomeness you can sometimes get out of enlightened corporations and smart, enthusiastic people. The conversation might very well go like this:
I don't believe this kind of tech-centric groupthink arises out of malice or a perverse instinct to complicate--my money is on a combination of habit (we use our computers during most of our waking hours), a tacit post-industrial rejection of the trappings of the Old Way of Doing Things, and recency priming effects (the last thing most people used before the meeting started was a computer program reminding them they were supposed to have interactions with other human beings). All those factors conspire to obscure other modes of communication and collaboration so ancient and universal they simply blend into their environment, devoid of shiny surfaces and sleek contours, waiting to be used.
Tool Envy
I personally don't share the fetishism for tactile, low-tech (and wantonly expensive) implements so prevalent among financially independent Bay Area techies. It's tempting to ascribe this post-modern stylomania to the scribal equivalent of comfort food: in a fast-moving, high-tech environment, non-electronic utensils like #2 pencils and tidy notebooks with elastic bands to prevent intempestive openings provide the stressed techie with a familiar anchor to long-ago times when things were less complicated and responsibilities less burdensome, redolent of chalk and crayons and cookies and blood from one too many encounters with Jimmy the football player now safely stowed away in detention. Or possibly something phallic.
But when it comes to getting ideas, moving them around into thematic buckets, discarding the bad ones (physically!), refining the fuzzy ones, and making them available to everyone, passively visible at all times, omnipresent and inevitable in their glaring obviousness and gaudy magenta-on-high-gloss-white, nothing--nothing--beats Post-It Notes and Sharpies. Even the hard-to-read ones are useful, because they draw you in and make you think about what they might mean.
A Web-accessible company intranet / wiki / bulletin board is undoubtedly a very convenient way to share ideas with everyone--it's dynamic, accessible from anywhere at any time, and does away with any requirements of physical co-presence that the modern office tries so hard to make obsolete.
But unless you're crazy and never close any tabs in your browser, you probably don't have that project wiki open in front of you all the time, and it takes an active effort to be engaged in the project. By contrast, the cadaverous whiteboard and its bright pimply scribbles is both easy to ignore, like a light fixture or a garbage can, and impossible to avoid, by virtue of its large, immovable presence; walking past its hodgepodge of colorful puddles instantly draws your gaze and reminds you there's something interesting going on, without your having to do anything.
As for the additional benefits (constant, virtual availability)--are they solving a problem you actually have? If most of the collaboration happens between people who can reasonably be expected to be in the same place during overlapping business hours most of the time, do you really need 24/7 remote availability of your idea board? And if someone really is available at 3am, maybe what they need is a reminder to go to bed rather than a tool to help them work in the middle of night.
Rampant Complexity
This tendency is hardly the bailiwick of 21st-century internet startups. Take One Laptop Per Child for example: an interesting, beneficent idea, to be sure, but it seems to me investing in mosquito nets, immunizations, education, human rights and safety for every child in the developing world may cost less, be more immediately useful, and go a bit farther than giving a crank-powered Linux computer to a bunch of kids who could really use some clean water and the assurance they won't go blind before they finish reading their first book.
Beyond computers, the 1980s brought tremendous technological advances to musical instruments: synthesizers became mainstream, and piezo-electric pickups made it easy to amplify and record acoustic guitars--so easy, in fact, that the world collectively agreed to ignore their atrocious sound and to forget that sitting down in front of a single microphone really isn't all that hard.
Snatching Simple from the Jaws of Difficult
Note that this is not a paean to the Luddites. My main computer has a modern dual-core processor with obscene amounts of memory and more hard drives than I remember, because larger, separate hard drives mean worrying less about backups; but it's attached to a pair of giant 10-year-old CRT monitors, because the productivity and eye comfort of 3840 x 1440 vibrant, never-stuck pixels outweigh the benefit of lightweight, space-saving LCDs--the last time I physically had to move my monitors was over 4 years ago; space and weight savings are nice to have, but completely irrelevant.
I store recurring shopping lists into my phone (e.g., lists of hard-to-find books for the occasional bookstore trip, just in case), but jot down grocery lists on envelopes out of the recycle bin, because their useful life span (minutes, hours at the most) simply does not justify spending more than a second or two jotting items down. As long as I can scrawl "milk" faster than a phone, computer or PDA can wake up from standby, launch the shopping-list program and fix my virtual-keyboard typos into a sensible word, technology is a hindrance, not a solution. Sure, I could integrate the two lists and sync them to my Google account and set up reminders to buy Kleenex via SMS before I leave the office. But I have better things to do with my time.
Microsoft didn't come up with Sharepoint out of sheer boredom, perversity, or greed. Mailing Post-It Notes around doesn't scale very well if half your team is in Los Angeles and the other half in Manila. Of course technology is the right approach a lot of the time, maybe even most of the time. But if technology is good for many things, it's not necessarily good at them. Weigh the actual (not the hoped-for, or potential) benefits of using one method, against its implementation, evangelism, training, and maintenance costs. Don't pick a solution based on its ability to solve a problem you don't have, or almost never have. The burden of proof is on the complex approach, not the simple one--simplicity should always win unless there's a compelling reason to go the other way.
It is sometimes said that software makes difficult tasks easier. Often the task is difficult not because the problem being solved, or the question being answered, is difficult per se: "I need to know when our users are likely to cancel their membership so we can send them win-back promotions" is a well-formulated, easy-to-understand, bounded question, and the answer is likely to be equally finite and easy to understand. The difficulty lies in gathering, compiling, and analyzing vast quantities of data in a timely, repeatable, reliable manner so that 1 billion data points coalesce into a half-dozen usable pieces of information. That's hard, and intractable unless you can process gobs of stuff really, really fast. Computers, and good software engineers, are really good at this sort of thing, and by and large you can never have enough of either.
All Nails
But a rampant trend in tech companies is the tendency to resort to technology and software to make easy tasks difficult.
Imagine a meeting room filled with 20- and 30-somethings, all endowed with solid educations, a solid drive to do the right things, and the disconcerting ability to explain the difference between TCP and UDP to finance people. The topic at hand: how to foster new ideas, team spirit, information exchanges, the kind of collaborative awesomeness you can sometimes get out of enlightened corporations and smart, enthusiastic people. The conversation might very well go like this:
"How about we set up a wiki page so everyone can contribute ideas and see everyone else's ideas? It's be really transparent and easy."Those are all good ideas, and they would probably work for the project you're all working on. But when your entire team is reaching for ways to do things that wouldn't work during a power outage, or were not even remotely possible less than ten years ago, maybe it's time to take a step back and reassess.
"Good idea! Once the ideas have coalesced a little, we can use Basecamp to see who's working on what and have real flexibility so new people can jump in and grab tasks they're passionate about without a big formal team structure!"
"Awesome! And since everyone's laptops have webcams, we don't need to have those big meetings--we can do ad-hoc catch-ups over Google Talk without everyone having to be around the office."
"How about we just stick Post-Its on the wall over there?"Why
I don't believe this kind of tech-centric groupthink arises out of malice or a perverse instinct to complicate--my money is on a combination of habit (we use our computers during most of our waking hours), a tacit post-industrial rejection of the trappings of the Old Way of Doing Things, and recency priming effects (the last thing most people used before the meeting started was a computer program reminding them they were supposed to have interactions with other human beings). All those factors conspire to obscure other modes of communication and collaboration so ancient and universal they simply blend into their environment, devoid of shiny surfaces and sleek contours, waiting to be used.
Tool Envy
I personally don't share the fetishism for tactile, low-tech (and wantonly expensive) implements so prevalent among financially independent Bay Area techies. It's tempting to ascribe this post-modern stylomania to the scribal equivalent of comfort food: in a fast-moving, high-tech environment, non-electronic utensils like #2 pencils and tidy notebooks with elastic bands to prevent intempestive openings provide the stressed techie with a familiar anchor to long-ago times when things were less complicated and responsibilities less burdensome, redolent of chalk and crayons and cookies and blood from one too many encounters with Jimmy the football player now safely stowed away in detention. Or possibly something phallic.
But when it comes to getting ideas, moving them around into thematic buckets, discarding the bad ones (physically!), refining the fuzzy ones, and making them available to everyone, passively visible at all times, omnipresent and inevitable in their glaring obviousness and gaudy magenta-on-high-gloss-white, nothing--nothing--beats Post-It Notes and Sharpies. Even the hard-to-read ones are useful, because they draw you in and make you think about what they might mean.
A Web-accessible company intranet / wiki / bulletin board is undoubtedly a very convenient way to share ideas with everyone--it's dynamic, accessible from anywhere at any time, and does away with any requirements of physical co-presence that the modern office tries so hard to make obsolete.
But unless you're crazy and never close any tabs in your browser, you probably don't have that project wiki open in front of you all the time, and it takes an active effort to be engaged in the project. By contrast, the cadaverous whiteboard and its bright pimply scribbles is both easy to ignore, like a light fixture or a garbage can, and impossible to avoid, by virtue of its large, immovable presence; walking past its hodgepodge of colorful puddles instantly draws your gaze and reminds you there's something interesting going on, without your having to do anything.
As for the additional benefits (constant, virtual availability)--are they solving a problem you actually have? If most of the collaboration happens between people who can reasonably be expected to be in the same place during overlapping business hours most of the time, do you really need 24/7 remote availability of your idea board? And if someone really is available at 3am, maybe what they need is a reminder to go to bed rather than a tool to help them work in the middle of night.
Rampant Complexity
This tendency is hardly the bailiwick of 21st-century internet startups. Take One Laptop Per Child for example: an interesting, beneficent idea, to be sure, but it seems to me investing in mosquito nets, immunizations, education, human rights and safety for every child in the developing world may cost less, be more immediately useful, and go a bit farther than giving a crank-powered Linux computer to a bunch of kids who could really use some clean water and the assurance they won't go blind before they finish reading their first book.
Beyond computers, the 1980s brought tremendous technological advances to musical instruments: synthesizers became mainstream, and piezo-electric pickups made it easy to amplify and record acoustic guitars--so easy, in fact, that the world collectively agreed to ignore their atrocious sound and to forget that sitting down in front of a single microphone really isn't all that hard.
Snatching Simple from the Jaws of Difficult
Note that this is not a paean to the Luddites. My main computer has a modern dual-core processor with obscene amounts of memory and more hard drives than I remember, because larger, separate hard drives mean worrying less about backups; but it's attached to a pair of giant 10-year-old CRT monitors, because the productivity and eye comfort of 3840 x 1440 vibrant, never-stuck pixels outweigh the benefit of lightweight, space-saving LCDs--the last time I physically had to move my monitors was over 4 years ago; space and weight savings are nice to have, but completely irrelevant.
I store recurring shopping lists into my phone (e.g., lists of hard-to-find books for the occasional bookstore trip, just in case), but jot down grocery lists on envelopes out of the recycle bin, because their useful life span (minutes, hours at the most) simply does not justify spending more than a second or two jotting items down. As long as I can scrawl "milk" faster than a phone, computer or PDA can wake up from standby, launch the shopping-list program and fix my virtual-keyboard typos into a sensible word, technology is a hindrance, not a solution. Sure, I could integrate the two lists and sync them to my Google account and set up reminders to buy Kleenex via SMS before I leave the office. But I have better things to do with my time.
Microsoft didn't come up with Sharepoint out of sheer boredom, perversity, or greed. Mailing Post-It Notes around doesn't scale very well if half your team is in Los Angeles and the other half in Manila. Of course technology is the right approach a lot of the time, maybe even most of the time. But if technology is good for many things, it's not necessarily good at them. Weigh the actual (not the hoped-for, or potential) benefits of using one method, against its implementation, evangelism, training, and maintenance costs. Don't pick a solution based on its ability to solve a problem you don't have, or almost never have. The burden of proof is on the complex approach, not the simple one--simplicity should always win unless there's a compelling reason to go the other way.
2010-02-13
Looking For A Job?
Looking for a job isn't fun. You have to boast of your accomplishments, which some find distasteful; you have to repeat yourself a lot as you send out your résumé and ever-so-slightly customized cover letters all across the internet, which some find tedious; and you have to do all this while in the throes of unemployment (or on the QT while still employed), which nobody enjoys.
It's no picnic for the hiring manager, either, especially at start-ups. Managers are usually expected to do other things than manage--lead engineers are supposed to code, VPs of marketing craft email campaigns, UX directors whip up Photoshop designs, and so everybody is chronically strapped for time, not least the person in charge of coordinating 30 positions that need to be filled yesterday without completely ignoring the existing employees and their ongoing issues. You have to balance the supreme requirement of hiring the right people (i.e. being ruthlessly selective) with the urgency of the hire and the overhead of screening, interviewing (and rejecting) a non-zero number of candidates. It's pretty common to find yourself in the following paradox loop:
RTFR
One of the hiring manager's more enervating enemies is the job trawler. The job trawler applies for dozens (hundreds) of jobs at the same time, with little regard for the posted requirements, presumably in the hope that the hiring manager will take it on faith they can learn the skills on the job, or not notice they just filled a real-time embedded-systems development position with a JSP programmer who's never even heard of C.
To spell it out in simple terms:
and so this:
Don't do it. And for Pete's sake, if the hiring manager sends you a rejection letter telling you you don't meet the requirements, don't argue with them.
Let Me Show You My Pokemans
Spelling, grammar
and punctuation are mostly arbitrary and capricious. That's partly why software developers came up with various devices to help supplement their fallible memory (or typing skills), such as spelling- and grammar-checkers. Many are even free. So when you send in something like this:
There's nothing wrong with not having a hypertrophied command of English. There is, however, something wrong with not turning on your word processor or Web browser's spell-checker, or not spending 10 minutes going over your application materials to make sure they will impress your future boss into calling you in for an interview. If you're in school, visit the career center. If you're employed, ask a trusted co-worker (or your boss, if you can). Librarians love to help, and they're often the hyperliterate type that can smell a typo from a mile away. It's a cheap investment with very high returns.
Stating The Obvious
Of course, all of this stuff is trivial and should be self-evident. Joel Spolsky has written at length about it; he even started a job board. If you're like me, you want to hire people who read Joel Spolsky, so it's extra disheartening when so many of the resumes that come across your monitor show no evidence of Spolsky-awareness (or common sense).
In the case of software engineering jobs, in particular, you'd think people with the brains, education and experience (nominally, at least) to go after a job that can bring in six figures pretty quickly1 wouldn't even have to be told these things.
Rejecting a candidate is not something hiring managers do lightly: the opportunity cost of not hiring a great candidate is high; and we've also all been in an applicant's shoes, so we know how hurtful and discouraging it can be not to be called in for an interview.
So if you're a hiring manager, make sure your job postings and requirements are clear, follow up with applicants (both good and bad) in a timely manner, and renounce the "Screening applicants is such a time sink" attitude. If you're looking for a job, stack the odds in your favor, don't apply for jobs you're not qualified for, and knock 'em dead with a crisp, punchy résumé. And if you're a database or data warehouse engineer, get in touch ASAP.
1. If you're reasonably good at it.
It's no picnic for the hiring manager, either, especially at start-ups. Managers are usually expected to do other things than manage--lead engineers are supposed to code, VPs of marketing craft email campaigns, UX directors whip up Photoshop designs, and so everybody is chronically strapped for time, not least the person in charge of coordinating 30 positions that need to be filled yesterday without completely ignoring the existing employees and their ongoing issues. You have to balance the supreme requirement of hiring the right people (i.e. being ruthlessly selective) with the urgency of the hire and the overhead of screening, interviewing (and rejecting) a non-zero number of candidates. It's pretty common to find yourself in the following paradox loop:
"We need to find a (programmer|designer|marketer|PM|salesperson) pronto or we're never going to (ship|sell|market) this thing this year"and
"I really don't have time to (screen|interview) those applicants"I've been privileged to participate in hiring a lot of great people over the years, either as an interviewer or a hiring manager; and because I've been knee-deep in résumé-screening trying to fill a couple of positions for a little while, I thought it might be useful to recap a couple of the hurdles applicants and hiring managers always seem to be running into.
RTFR
One of the hiring manager's more enervating enemies is the job trawler. The job trawler applies for dozens (hundreds) of jobs at the same time, with little regard for the posted requirements, presumably in the hope that the hiring manager will take it on faith they can learn the skills on the job, or not notice they just filled a real-time embedded-systems development position with a JSP programmer who's never even heard of C.To spell it out in simple terms:
"Requirement" means "you have to know this stuff"
and so this:
"Requirements: 4+ years Oracle, PL/SQL, 6+ years C++, excellent knowledge of Unix/Linux performance optimization"means that this won't cut it:
"Skills: Java (2 years), MySQL (3 years), perl, Windows Server"Applying for a job whose requirements you don't meet is worse than ineffectual: it's counterproductive. It sucks up time out of a busy person's schedule that they could be spending interviewing qualified candidates. And if they have a modicum of organization, they might remember this when you apply for another position at their company.
Don't do it. And for Pete's sake, if the hiring manager sends you a rejection letter telling you you don't meet the requirements, don't argue with them.
Let Me Show You My Pokemans
Spelling, grammar
and punctuation are mostly arbitrary and capricious. That's partly why software developers came up with various devices to help supplement their fallible memory (or typing skills), such as spelling- and grammar-checkers. Many are even free. So when you send in something like this:dear mr/msr,it just doesn't make a good impression, and when a hiring manager is screening résumés to find candidates that are worth spending over 10 man-hours wooing and interviewing, your first impression is all you have. Maybe Donald Knuth, Brian Kernighan and Ray Ozzie are ready to be your references at a moment's notice; maybe you did write seven bug-free patches that got checked into the kernel tree; but if your résumé and cover letter look like a Scrabble game board after a tornado, you'll never get a chance to show off those 1337 skillz, or to get Knuth on the phone with your boss.
i wanted to apply for the engineer position in ur compny on craigslits i am a hardworking programmer with 2 yr. experience in java technologys. i guarante dedication to ur company because i need the money so i won't let u down.
Skill:i look forward to discuss my application.
- java, j2ee, jni, xml, soa
- well communication skills
- work good under pressure
There's nothing wrong with not having a hypertrophied command of English. There is, however, something wrong with not turning on your word processor or Web browser's spell-checker, or not spending 10 minutes going over your application materials to make sure they will impress your future boss into calling you in for an interview. If you're in school, visit the career center. If you're employed, ask a trusted co-worker (or your boss, if you can). Librarians love to help, and they're often the hyperliterate type that can smell a typo from a mile away. It's a cheap investment with very high returns.
Stating The Obvious
Of course, all of this stuff is trivial and should be self-evident. Joel Spolsky has written at length about it; he even started a job board. If you're like me, you want to hire people who read Joel Spolsky, so it's extra disheartening when so many of the resumes that come across your monitor show no evidence of Spolsky-awareness (or common sense).
In the case of software engineering jobs, in particular, you'd think people with the brains, education and experience (nominally, at least) to go after a job that can bring in six figures pretty quickly1 wouldn't even have to be told these things.
Rejecting a candidate is not something hiring managers do lightly: the opportunity cost of not hiring a great candidate is high; and we've also all been in an applicant's shoes, so we know how hurtful and discouraging it can be not to be called in for an interview.
So if you're a hiring manager, make sure your job postings and requirements are clear, follow up with applicants (both good and bad) in a timely manner, and renounce the "Screening applicants is such a time sink" attitude. If you're looking for a job, stack the odds in your favor, don't apply for jobs you're not qualified for, and knock 'em dead with a crisp, punchy résumé. And if you're a database or data warehouse engineer, get in touch ASAP.
1. If you're reasonably good at it.
2010-02-03
Breaking the Unit-Tests-Take-Too-Long Cycle
Unit Testing in Theory
By now (2010) every experienced software developer worth her salt has heard about unit testing. If you're not familiar with the concept, here's how it goes in a nutshell, using some fake programming language:
Many languages have unit-testing frameworks available. Unit testing is an elegant, systematic, automatable system that makes obsolete all the ad-hoc programs developers often write to test various parts of their code. The field is both fresh and mature enough that there's a small, well-known set of standard practices that make it easy to start writing unit tests in language A if you're familiar with unit tests in language B. By and large, it's a great invention with considerable benefits.
Some have embraced the practice and follow it almost religiously, writing their unit tests first and actual software second. Others apply unit testing more like salt, pepper and nutmeg, as needed but not in excess. Still others write tests reluctantly and need to be reminded frequently, like patients chided for their infrequent flossing by their dental hygienist. And others just write code and custom test scripts (I actually know superlative software developers who've never written a single unit test and probably never will). You may find that a developer's attitude towards unit tests has some relation to the year they started getting paid programming gigs.
Unit Testing in Practice
The majority of developers I did, do, may, will, and won't ever work with, interview and hire probably fall in the unit-tests-as-spice camp: they're reasonably convinced unit testing is an excellent insurance policy, they may even evangelize the practice with unenlightened colleagues, but they readily admit their unit test coverage is spotty or out of date, with a curious mix of rational self-confidence ("I try to test code that's important, not getters and setters; 100% coverage is silly") and underflossed guilt ("I probably don't write as many tests as I should"). I'm certainly one of them.
A lot of (most?) software shops put a lot of pressure on developers to just crank out code that does something. As a result, software that doesn't do anything visible gets short shrift if it gets written at all, and unit tests are often the first to get the axe. Oversimplifying, the tacit (and circular) rationale looks a lot like this:
A common strategy to encourage beneficial behavior is to wrap the hard-to-do stuff inside something they normally do: hide a cat pill inside a yummy treat, or stick a tongue-scraper on the non-bristly side of a toothbrush head. Brush your teeth and clean the rest of your mouth at no extra charge!
This could mean suppressing all output when all tests pass, but warning the entire development team when a check-in is going to break the build so the bug can be fixed immediately rather than rot in your bug tracker's entrails, forgotten until your boss demos your product to the board, trips the bug, and sends your entire team into a panic trying to hot-fix the problem in the program QA signed off on two hours ago.
Here are suggestions to increase your team's enthusiasm for, and the effectiveness of, consistent unit testing. All of these tips are some variation on the parasitic-benefit approach outlined above.
There are many other approaches, including forgoing unit testing altogether, and sometimes that might be just what the doctor ordered. Don't be afraid to try new things, nag at your colleagues (not too much), drop what fails and pursue whatever works for your product and organization. And by all means share your experience--open-source processes are just as vital as open-source software.
By now (2010) every experienced software developer worth her salt has heard about unit testing. If you're not familiar with the concept, here's how it goes in a nutshell, using some fake programming language:
class Thingie
{
public static function do_something(param)
{
if (param == 0)
{
return false;
}
c = new network_connection();
data = c.fetch();
return !empty(data);
}
}
class ThingieTest extends TestFrameworkTestCase
{
public function test_do_something()
{
T = new Thingie();
this.assert_equals(T.do_something(1), true);
this.assert_equals(T.do_something(0), false);
}
}
TestFramework ThingieTest
..
1 test, 2 assertions, 0 failures.
Many languages have unit-testing frameworks available. Unit testing is an elegant, systematic, automatable system that makes obsolete all the ad-hoc programs developers often write to test various parts of their code. The field is both fresh and mature enough that there's a small, well-known set of standard practices that make it easy to start writing unit tests in language A if you're familiar with unit tests in language B. By and large, it's a great invention with considerable benefits.
Some have embraced the practice and follow it almost religiously, writing their unit tests first and actual software second. Others apply unit testing more like salt, pepper and nutmeg, as needed but not in excess. Still others write tests reluctantly and need to be reminded frequently, like patients chided for their infrequent flossing by their dental hygienist. And others just write code and custom test scripts (I actually know superlative software developers who've never written a single unit test and probably never will). You may find that a developer's attitude towards unit tests has some relation to the year they started getting paid programming gigs.
Unit Testing in Practice
The majority of developers I did, do, may, will, and won't ever work with, interview and hire probably fall in the unit-tests-as-spice camp: they're reasonably convinced unit testing is an excellent insurance policy, they may even evangelize the practice with unenlightened colleagues, but they readily admit their unit test coverage is spotty or out of date, with a curious mix of rational self-confidence ("I try to test code that's important, not getters and setters; 100% coverage is silly") and underflossed guilt ("I probably don't write as many tests as I should"). I'm certainly one of them.
A lot of (most?) software shops put a lot of pressure on developers to just crank out code that does something. As a result, software that doesn't do anything visible gets short shrift if it gets written at all, and unit tests are often the first to get the axe. Oversimplifying, the tacit (and circular) rationale looks a lot like this:
- a good developer's best code is bug-free
- bug-free code will pass all its unit tests, by definition
- therefore, writing tests is a waste of time, since they won't uncover any bugs
A common strategy to encourage beneficial behavior is to wrap the hard-to-do stuff inside something they normally do: hide a cat pill inside a yummy treat, or stick a tongue-scraper on the non-bristly side of a toothbrush head. Brush your teeth and clean the rest of your mouth at no extra charge!
Similarly, the benefit of unit tests can be self-sustaining if you can make your tests:
- run as often as necessary
- run at no cost to the developer
- draw attention to their usefulness very forcefully when something breaks
- remain invisible otherwise
Here are suggestions to increase your team's enthusiasm for, and the effectiveness of, consistent unit testing. All of these tips are some variation on the parasitic-benefit approach outlined above.
- trigger your unit tests automatically using a pre-commit hook in your source-control system. Every time someone tries to check in her code, all your tests run automatically, and any failure blocks the commit. You have to fix the code (or the test) before you're allowed to check in your code.
- trigger all your tests automatically, including your integration tests, at the top of your release or build script. The release or build is blocked if any test fails.
- use a bootstrapping framework or wizard to generate boilerplate code, including matching unit test code, for every class you write. The less code you have to write, the more time you have to work on the good stuff. Ruby on Rails or Django are such bootstrap systems; they're easily extended to add boilerplate unit test code.
- have your unit tests create persistent, semi-random test data in your dev database. Every time you run your tests, you get more data in your development database, which makes your application more realistic during testing and can uncover issues you only run into under actual use conditions (e.g., you might realize what a mess your UI is when it tries to display 250 items on a single form).
- have your unit tests obliterate the test data in your dev database every time they run. This weans you from over-reliance on test data, which can be handy if you're shipping a brand-new product with zero user data on launch day: you consistently get to interact with the same product your early users get to see on day one. Yes, this is exactly the opposite of the previous point; use your judgment.
There are many other approaches, including forgoing unit testing altogether, and sometimes that might be just what the doctor ordered. Don't be afraid to try new things, nag at your colleagues (not too much), drop what fails and pursue whatever works for your product and organization. And by all means share your experience--open-source processes are just as vital as open-source software.
2010-01-31
Make Sure Your Customers Can Pay You
If you're running an e-commerce site, chances are you want to extract money out of your visitors. Yet you wouldn't know it by looking at some sites like Best Western's reservation portal, which makes it incredibly difficult if not impossible to complete a paying transaction, as I recently discovered to my chagrin.
Invisible Form
Here's what happened: after selecting a date range, I pressed the big giant "Book it" button next to the room I was interested in (+1 for using big huge buttons for the primary action):
This led me to the following page, completely devoid of a form into which I could enter my credit card information:
After another failed attempt, I fired up Internet Explorer to see if the form was crucially incompatible with Firefox, and I got my answer: the credit card form is served in a popup window, which IE requested permission to open and Firefox quashed silently (I have the "be quiet about popups" setting turned on). Here's what it would have looked like with slightly more verbose Firefox settings:
Note that no one using a reasonably recent browser in its default settings gets to see the popup window unless they explicitly authorize it. In other words, one hundred percent of Best Western's juiciest, readiest-to-buy visitors are not shown the reservation form in the default use case.
I don't have to spell out how completely crazy this is.
To make matters worse, when you do complete the form and click the submit button, nothing happens in Explorer due to a
I had to call the hotel to find out how many of my three form submission attempts had gone through (one had, which was soon confirmed by an email from the site).
Money For Nothing And Tests For Free
What this tells me is that the designer(s) and developer(s) in charge of the Best Western portal never did so much as basic hallway usability testing before they shipped their product. Maybe they don't know about it; maybe the product team is in Connecticut and the developers are in a different country. Either way, a 30-minute investment would have caught what is potentially costing the company tens of thousands of dollars a day.
Another notable omission is that the credit card form itself (for those of you lucky enough to see it) doesn't exactly scream "GIVE ME ALL YOUR MONEY NOW!" I'd bet its conversion hasn't been tested or optimized either.
Testing form conversion is hardly a novel or esoteric branch of rocket science; it's a simple, mature field with well-known best practices and automated testing systems to help you get the most out of your site. But a company that can't be bothered to test the basic usability of their purchase flow can't be expected to look at bounce rates or A/B test their funnel.
Given the bang for the buck you get from real-world testing with a tiny number of users, I submit Best Western could have earned an ROI on the order of 100,000% if they had just watched a half dozen travelers go through the booking process using a fresh, default install of IE or Firefox.
Help Me Buy What You're Selling
I will never know how much money Best Western is losing from this; presumably a lot of travelers wind up calling the hotel (whose phone number is displayed at the top) and maybe even pay the (higher) non-internet rate you get when reserving by phone. So for all I know Best Western is actually coming out ahead (I have my doubts).
What's really sad about this grossly inefficient system is that Best Western didn't even have to earn my business: all they had to do is keep it. Given the way RevPAR is trending, that's a gimme they don't get very often. I always stay at that hotel when traveling down south, because it's convenient, well located, reasonably quiet and clean, and its wireless internet occasionally functions. But I don't know how patient I'm going to be next time around.
---
1. Don't use
Invisible Form
Here's what happened: after selecting a date range, I pressed the big giant "Book it" button next to the room I was interested in (+1 for using big huge buttons for the primary action):
This led me to the following page, completely devoid of a form into which I could enter my credit card information:
After another failed attempt, I fired up Internet Explorer to see if the form was crucially incompatible with Firefox, and I got my answer: the credit card form is served in a popup window, which IE requested permission to open and Firefox quashed silently (I have the "be quiet about popups" setting turned on). Here's what it would have looked like with slightly more verbose Firefox settings:
Note that no one using a reasonably recent browser in its default settings gets to see the popup window unless they explicitly authorize it. In other words, one hundred percent of Best Western's juiciest, readiest-to-buy visitors are not shown the reservation form in the default use case.
I don't have to spell out how completely crazy this is.
To make matters worse, when you do complete the form and click the submit button, nothing happens in Explorer due to a
javascript:void(0) issue in the form submit handler1; in Firefox, the window simply closes with no confirmation the transaction went through.I had to call the hotel to find out how many of my three form submission attempts had gone through (one had, which was soon confirmed by an email from the site).
Money For Nothing And Tests For Free
What this tells me is that the designer(s) and developer(s) in charge of the Best Western portal never did so much as basic hallway usability testing before they shipped their product. Maybe they don't know about it; maybe the product team is in Connecticut and the developers are in a different country. Either way, a 30-minute investment would have caught what is potentially costing the company tens of thousands of dollars a day.
Another notable omission is that the credit card form itself (for those of you lucky enough to see it) doesn't exactly scream "GIVE ME ALL YOUR MONEY NOW!" I'd bet its conversion hasn't been tested or optimized either.
Testing form conversion is hardly a novel or esoteric branch of rocket science; it's a simple, mature field with well-known best practices and automated testing systems to help you get the most out of your site. But a company that can't be bothered to test the basic usability of their purchase flow can't be expected to look at bounce rates or A/B test their funnel.
Given the bang for the buck you get from real-world testing with a tiny number of users, I submit Best Western could have earned an ROI on the order of 100,000% if they had just watched a half dozen travelers go through the booking process using a fresh, default install of IE or Firefox.
Help Me Buy What You're Selling
I will never know how much money Best Western is losing from this; presumably a lot of travelers wind up calling the hotel (whose phone number is displayed at the top) and maybe even pay the (higher) non-internet rate you get when reserving by phone. So for all I know Best Western is actually coming out ahead (I have my doubts).
What's really sad about this grossly inefficient system is that Best Western didn't even have to earn my business: all they had to do is keep it. Given the way RevPAR is trending, that's a gimme they don't get very often. I always stay at that hotel when traveling down south, because it's convenient, well located, reasonably quiet and clean, and its wireless internet occasionally functions. But I don't know how patient I'm going to be next time around.
---
1. Don't use
javascript:void(0) to block a page reload event in an a tag. This will work:<a href="#" onclick="do_something();return false;">text</a>
2010-01-23
The Process
An issue most software shops have to adjudicate at a certain size is how to build applications. Not how as in which languages, servers, or database systems (hopefully that's settled early on, revised periodically, and mostly stable)--how as in how, how early, are features defined and built, how often code is tagged and released, iterative v. waterfall, agile v. see-what-sticks free-for-all, etc.
Small startups comprised of one product co-founder and a technical co-founder usually don't need to worry about this. But as soon as you're big enough to have discrete design, product, customers and QA people involved, setting up the right processes becomes a necessity.
What I'm calling "process" here is the set of practices used to build a software project, not the dreaded product-design-executives-board-executives-design-engineering gauntlet your idea may have to go through before seeing the light of day. Rather, "process" here means the tools, approaches, meetings and other day-to-day activities you decide to use to build your project: daily stand-up meetings, waterfall specs, retrospectives, whiteboards, white/gray/black-box QA, pair programming, whatever. It's a stodgy name for a concept that needs not be stodgy.
Why Process?
One source of friction in software development is that software is malleable, and so tweaking features all the time is easy (and tempting). But one person's flexibility is another's sloppiness; one person's easy last-minute tweak is another's bang-your-head-on-the-table1 nightmare.
When building a bridge or skyscraper, with hard, heavy parts that need to be measured, ordered and fabricated months, if not years, ahead of time, the notion goes that you can't get away with sloppy/flexible/just-in-time/no planning and last-minute changes, and so building software shouldn't really be any different, because sloppy/flexible/just-in-time/no planning makes the product buggy/brittle/ugly/inconsistent. It's a common argument I've heard even from people who would rather mine coal in Mongolia than go (back) to the waterfall model.
But I think that's a bit of a false contrast. While I do expect structural engineers to know what they're doing and plan how they're going to do it if my life depends on it, I also expect them to be agile enough to respond to unexpected conditions, or to incorporate new ideas or technologies that come along when it makes sense (which does happen when you're working on a 15-year project). And I don't know any good software engineer who actually enjoys a 1-year planning cycle before they get to write any code or create table schemas.
The point is that you're usually building something for other people, and so what matters is your ability to deliver a quality product for those people. And change happens. Great new ideas come in at the last minute. Your CEO finds a blind alley in your UI flow nobody on your team had thought about. So complaining about, denying the existence of, or impeding those changes doesn't really gain anybody anything. But to the extent your own comfort or happiness or need for control are determining factors in whether you do deliver a quality product, you need some kind of process to enable you to build the best possible product in the best possible conditions.
Ask What Your Process Can Do For You
Process is meant to get things done well (your customers like your product), quickly (a leg up on the competition), and comfortably (high turnover is the enemy). It's not an mystical spirit or a magical toolbox to be worshiped or enshrined. And it's not the 100-year-old secret formula for Coca Cola or your grandfather's super gooey, always delicious sticky bun recipe, either.
Think of it as a suitcase containing the right outfits for a year in California. If you're spending April in San Francisco, you might want a lot of light layers, with a waterproof shell you can take off easily when it stops raining, and maybe a windbreaker. If you drive up to Tahoe in January, you might want snow mittens and a thick parka. And don't forget a swimsuit and towel for those summer days in Santa Monica. It's not the end of the world when the clothes get stained or wear out; you can take the suitcase with you to your next destination; and when you gain or lose weight, some of your clothes won't fit anymore.
Perhaps most importantly, if you find yourself wondering why people laugh at you and your dorky parka on the beach, or you're uncomfortable in your t-shirt and vest in the snow, the problem isn't with the clothes you're wearing--think about the ones you're not wearing.
Bad Process, No Cookie
It's easier to describe process when it's broken. How can you tell your process doesn't work? Defects creep in, stress rises, milestones are missed, good people quit. Sure, all of those symptoms could be due to individual issues like personal-life distractions, lack of skill or motivation, unexpected illnesses. But that's precisely what the right process is there to help you solve. By and large, your organization shouldn't fall apart when one person has a car accident and is laid up for a month, or limp along when someone isn't up to snuff on a particular set of tasks. So when you're having issues delivering products, see if the problem really is coming from night owl Jill Programmer or eccentric Mike Designer, or if your process is what needs a kick in the pants.
This is where being pragmatic, not dogmatic, in your process decisions can serve your needs better. Your processes are not your customers, your stockholders, your investors, your spouses--the process is beholden to you, not you to it. So if a process doesn't feel right, do spend some time tweaking it, but don't be afraid to shove it aside and try something else.
Now, changing direction too often with your process can send destructive signals: you don't know what you're doing; your team is dysfunctional and can't work effectively (and the team leaders don't know how to address that); the product is too undefined and can't be built. But remember the clothes aren't wearing you. Your allegiance is to your product and customers first, your people second, and the process dead last.
A Heuristic
Once you've used a process with great success, it's possible, even tempting, to settle on it and use it for everything, and that might be good enough. But not every problem is a nail to be addressed by your big hammer, so here's a possible heuristic to decide what approach might work best. Note this isn't a decision tree or a set of solutions; it's a set of simple questions the answers to which can be much more illuminating than you might think at first glance.
Who is going to use the product you're building?
If you're building a fun consumer product, with lots of new features and changes that make your consumers happy, you might want to try quick iterations with soft launches or restricted-availability features that get you feedback quickly so you can tweak and release something new the next week. SCRUM or other agile processes can be very helpful so you don't lose momentum; a drawn-out design and product feature planning process might not be best, because you can't tell on day 1 what people will tell you about your product on day 8, and the 10 features you design up-front might turn out to be unwanted.
On the other hand, if you're building a Web service API consumed by machines, rather than people, you might be better served to spend more time planning up front before you start coding: you might need solid capacity planning, performance testing, redundancy and failover, and because there can be hardware procurement and setup issues, it might be good to know more ahead of time, and then fan out and work on various pieces in parallel.
How much change can be expected in the feature set or interface?
Do you need a rock-solid API and protocols that will last for years and need to be backward-compatible for the next 5 version numbers? Which API calls should you support now to remain relevant in the future (maybe your service is brand new, but a treasure trove of data will emerge from a few months' worth of usage logs). You might want to do some heavy-duty feature analysis to make sure you're not including useless API calls nobody wants, or missing important calls people will need. This doesn't preclude short iterations and frequent milestones, and even fast feature changes, internal to your team: the shape of your end product doesn't have to match your product development practices.
How big is the team working on the project?
Can it be done in a month by a couple of developers and a part-time designer? Is it phase one of a company-defining, all-hands-on-deck product spread over a two-year plan? Constant verbal contact between the designer and developer working next to each other might be a great substitute for a thick PDF spec that fell out of sync two days before the PM last emailed it around; it could be a mess leaving giant holes in the product and SQL injection vulnerabilities all over the place. It greatly depends on...
Who is working on the project?
Software doesn't build itself. The team working on your project is the strongest predictor of success or failure, much more so than the technology they use. So be sure to gauge your engineers, designers, product managers and other actors, and keep your finger on the team's pulse: do they prefer a strongly-defined upfront spec? Do they resent being told what to do? Does Fred Programmer perform best with frequent status catchups, while Sung Designer thrives in a don't-call-me-I'll-call-you rhythm? Do Sung and Fred want to kill each other on a good day, and eat each other's babies on a bad day? More importantly, do you or the team lead actually know all this?
Note this is not a deep insight. In fact it's not an insight at all: different people have different needs, and the same process doesn't magically fit every team. Get a sense of what makes people thrive, or tick, and pick the practices that favor the former and minimize the latter.
Whatever Works
Ultimately your customer will determine whether your project is successful, mediocre, or a failure. Your process can only facilitate or hinder your progress towards one of those end points. It might be a beautiful work of by-the-book SCRUM, or a hotchpotch of Agile / XP on the tech side with an independent design and product team checking in occasionally, or a Soviet-era waterfall. The best process is the one that works for your project and your people, not the one that follows the bullet points in your expensive management consultancy's white paper.
----
1. True story.
Small startups comprised of one product co-founder and a technical co-founder usually don't need to worry about this. But as soon as you're big enough to have discrete design, product, customers and QA people involved, setting up the right processes becomes a necessity.
What I'm calling "process" here is the set of practices used to build a software project, not the dreaded product-design-executives-board-executives-design-engineering gauntlet your idea may have to go through before seeing the light of day. Rather, "process" here means the tools, approaches, meetings and other day-to-day activities you decide to use to build your project: daily stand-up meetings, waterfall specs, retrospectives, whiteboards, white/gray/black-box QA, pair programming, whatever. It's a stodgy name for a concept that needs not be stodgy.
Why Process?
One source of friction in software development is that software is malleable, and so tweaking features all the time is easy (and tempting). But one person's flexibility is another's sloppiness; one person's easy last-minute tweak is another's bang-your-head-on-the-table1 nightmare.
When building a bridge or skyscraper, with hard, heavy parts that need to be measured, ordered and fabricated months, if not years, ahead of time, the notion goes that you can't get away with sloppy/flexible/just-in-time/no planning and last-minute changes, and so building software shouldn't really be any different, because sloppy/flexible/just-in-time/no planning makes the product buggy/brittle/ugly/inconsistent. It's a common argument I've heard even from people who would rather mine coal in Mongolia than go (back) to the waterfall model.
But I think that's a bit of a false contrast. While I do expect structural engineers to know what they're doing and plan how they're going to do it if my life depends on it, I also expect them to be agile enough to respond to unexpected conditions, or to incorporate new ideas or technologies that come along when it makes sense (which does happen when you're working on a 15-year project). And I don't know any good software engineer who actually enjoys a 1-year planning cycle before they get to write any code or create table schemas.
The point is that you're usually building something for other people, and so what matters is your ability to deliver a quality product for those people. And change happens. Great new ideas come in at the last minute. Your CEO finds a blind alley in your UI flow nobody on your team had thought about. So complaining about, denying the existence of, or impeding those changes doesn't really gain anybody anything. But to the extent your own comfort or happiness or need for control are determining factors in whether you do deliver a quality product, you need some kind of process to enable you to build the best possible product in the best possible conditions.
Ask What Your Process Can Do For You
Process is meant to get things done well (your customers like your product), quickly (a leg up on the competition), and comfortably (high turnover is the enemy). It's not an mystical spirit or a magical toolbox to be worshiped or enshrined. And it's not the 100-year-old secret formula for Coca Cola or your grandfather's super gooey, always delicious sticky bun recipe, either.
Think of it as a suitcase containing the right outfits for a year in California. If you're spending April in San Francisco, you might want a lot of light layers, with a waterproof shell you can take off easily when it stops raining, and maybe a windbreaker. If you drive up to Tahoe in January, you might want snow mittens and a thick parka. And don't forget a swimsuit and towel for those summer days in Santa Monica. It's not the end of the world when the clothes get stained or wear out; you can take the suitcase with you to your next destination; and when you gain or lose weight, some of your clothes won't fit anymore.
Perhaps most importantly, if you find yourself wondering why people laugh at you and your dorky parka on the beach, or you're uncomfortable in your t-shirt and vest in the snow, the problem isn't with the clothes you're wearing--think about the ones you're not wearing.
Bad Process, No Cookie
It's easier to describe process when it's broken. How can you tell your process doesn't work? Defects creep in, stress rises, milestones are missed, good people quit. Sure, all of those symptoms could be due to individual issues like personal-life distractions, lack of skill or motivation, unexpected illnesses. But that's precisely what the right process is there to help you solve. By and large, your organization shouldn't fall apart when one person has a car accident and is laid up for a month, or limp along when someone isn't up to snuff on a particular set of tasks. So when you're having issues delivering products, see if the problem really is coming from night owl Jill Programmer or eccentric Mike Designer, or if your process is what needs a kick in the pants.
This is where being pragmatic, not dogmatic, in your process decisions can serve your needs better. Your processes are not your customers, your stockholders, your investors, your spouses--the process is beholden to you, not you to it. So if a process doesn't feel right, do spend some time tweaking it, but don't be afraid to shove it aside and try something else.
Now, changing direction too often with your process can send destructive signals: you don't know what you're doing; your team is dysfunctional and can't work effectively (and the team leaders don't know how to address that); the product is too undefined and can't be built. But remember the clothes aren't wearing you. Your allegiance is to your product and customers first, your people second, and the process dead last.
A Heuristic
Once you've used a process with great success, it's possible, even tempting, to settle on it and use it for everything, and that might be good enough. But not every problem is a nail to be addressed by your big hammer, so here's a possible heuristic to decide what approach might work best. Note this isn't a decision tree or a set of solutions; it's a set of simple questions the answers to which can be much more illuminating than you might think at first glance.
Who is going to use the product you're building?
If you're building a fun consumer product, with lots of new features and changes that make your consumers happy, you might want to try quick iterations with soft launches or restricted-availability features that get you feedback quickly so you can tweak and release something new the next week. SCRUM or other agile processes can be very helpful so you don't lose momentum; a drawn-out design and product feature planning process might not be best, because you can't tell on day 1 what people will tell you about your product on day 8, and the 10 features you design up-front might turn out to be unwanted.
On the other hand, if you're building a Web service API consumed by machines, rather than people, you might be better served to spend more time planning up front before you start coding: you might need solid capacity planning, performance testing, redundancy and failover, and because there can be hardware procurement and setup issues, it might be good to know more ahead of time, and then fan out and work on various pieces in parallel.
How much change can be expected in the feature set or interface?
Do you need a rock-solid API and protocols that will last for years and need to be backward-compatible for the next 5 version numbers? Which API calls should you support now to remain relevant in the future (maybe your service is brand new, but a treasure trove of data will emerge from a few months' worth of usage logs). You might want to do some heavy-duty feature analysis to make sure you're not including useless API calls nobody wants, or missing important calls people will need. This doesn't preclude short iterations and frequent milestones, and even fast feature changes, internal to your team: the shape of your end product doesn't have to match your product development practices.
How big is the team working on the project?
Can it be done in a month by a couple of developers and a part-time designer? Is it phase one of a company-defining, all-hands-on-deck product spread over a two-year plan? Constant verbal contact between the designer and developer working next to each other might be a great substitute for a thick PDF spec that fell out of sync two days before the PM last emailed it around; it could be a mess leaving giant holes in the product and SQL injection vulnerabilities all over the place. It greatly depends on...
Who is working on the project?
Software doesn't build itself. The team working on your project is the strongest predictor of success or failure, much more so than the technology they use. So be sure to gauge your engineers, designers, product managers and other actors, and keep your finger on the team's pulse: do they prefer a strongly-defined upfront spec? Do they resent being told what to do? Does Fred Programmer perform best with frequent status catchups, while Sung Designer thrives in a don't-call-me-I'll-call-you rhythm? Do Sung and Fred want to kill each other on a good day, and eat each other's babies on a bad day? More importantly, do you or the team lead actually know all this?
Note this is not a deep insight. In fact it's not an insight at all: different people have different needs, and the same process doesn't magically fit every team. Get a sense of what makes people thrive, or tick, and pick the practices that favor the former and minimize the latter.
Whatever Works
Ultimately your customer will determine whether your project is successful, mediocre, or a failure. Your process can only facilitate or hinder your progress towards one of those end points. It might be a beautiful work of by-the-book SCRUM, or a hotchpotch of Agile / XP on the tech side with an independent design and product team checking in occasionally, or a Soviet-era waterfall. The best process is the one that works for your project and your people, not the one that follows the bullet points in your expensive management consultancy's white paper.
----
1. True story.
2008-04-24
Sliders on the Web
Browsing a site I'd never seen before, I saw a cool nod to accessibility: the site lets you make the size of the type in the main article bigger.
Great.
Except.
The control to do this is a miniscule Javascript slider.
So in order for their special font-size needs to be accommodated, users with those needs must acquire a tiny target that's 7 pixels in diameter, hold the mouse button down without moving of the target, drag it over by a few pixels without accidentally releasing the mouse button, and finally release the button when they're done.
And all that is after they've realized that thing can be clicked on and dragged.
And unlike a real, standard desktop UI slider widget, you can't click on an arbitrary point on the slider rail and make the slider jump to that spot. The only way you can change the font size is by sliding.
And the range of sizes is only a handful of discrete font sizes, not a continuous value (which is what sliders are good for).
And the target users for the grow-the-font feature are typically those who will have more problems dealing with small-target widgets and/or drag and drop.
And it's not keyboard-controllable the way a plain text link would be (you can't tab to it, and you can't move it with the keyboard).
That's a lot of "and"s, and just as many reasons why this is a bad design decision. This is nothing new--it's been known for ages that drag and drop is hard to perform and non-intuitive for certain actions. So why put a hard, unnecessary constraint into Web pages?
I don't know what made sliders in Web apps so popular in the past couple of years, but I really wish designers and developers would realize what a horrible widget they are. They're very difficult to acquire, maneuver and discover, and you can always replace them with an easier, more discoverable UI element.
Here are obvious suggestions that would take exactly the same amount of space on the screen and considerably less time to implement and debug, not to mention obviating the need for the slider JavaScript library to be downloaded with the page:
Simple and standard usually win in my book; the burden of proof is always on the more complicated option.
Great.
Except.
The control to do this is a miniscule Javascript slider.
So in order for their special font-size needs to be accommodated, users with those needs must acquire a tiny target that's 7 pixels in diameter, hold the mouse button down without moving of the target, drag it over by a few pixels without accidentally releasing the mouse button, and finally release the button when they're done.
And all that is after they've realized that thing can be clicked on and dragged.
And unlike a real, standard desktop UI slider widget, you can't click on an arbitrary point on the slider rail and make the slider jump to that spot. The only way you can change the font size is by sliding.
And the range of sizes is only a handful of discrete font sizes, not a continuous value (which is what sliders are good for).
And the target users for the grow-the-font feature are typically those who will have more problems dealing with small-target widgets and/or drag and drop.
And it's not keyboard-controllable the way a plain text link would be (you can't tab to it, and you can't move it with the keyboard).
That's a lot of "and"s, and just as many reasons why this is a bad design decision. This is nothing new--it's been known for ages that drag and drop is hard to perform and non-intuitive for certain actions. So why put a hard, unnecessary constraint into Web pages?
I don't know what made sliders in Web apps so popular in the past couple of years, but I really wish designers and developers would realize what a horrible widget they are. They're very difficult to acquire, maneuver and discover, and you can always replace them with an easier, more discoverable UI element.
Here are obvious suggestions that would take exactly the same amount of space on the screen and considerably less time to implement and debug, not to mention obviating the need for the slider JavaScript library to be downloaded with the page:
- icons of the letter A in different sizes, each mapped to a font size
- one big icon of the letter A with an up arrow next to it, and another with a down arrow next to it. Ideally the "shrink" A would be a little smaller than the "grow" A
- text links labeled "bigger font" and "smaller font"
Simple and standard usually win in my book; the burden of proof is always on the more complicated option.
2008-04-13
On Tools
The starting point for a recent post on Coding Horror about attitudes towards open source and free software is a discussion of programming tools for diffing code or building regular expressions. I won't debate the merits of free v. pay or open-source v. proprietary, since that's a mostly pointless argument and Atwood's readers already beat it into the ground. What I would like to take up is the narrow notion of tools for programming. Some smart aleck posted the following comment:
To which someone responded:
I think the response misses the point. I am completely in favor of using tools to make software development easier, faster, and less error-prone. Syntax highlighting, fast access to files in your project, code auto-completion, visual diff tools like WinMerge, one-key access to your language's documentation, all of those are great--the computer's memory and ruthlessly error-free comparison skills are an excellent substitute for a human's, especially when it's late and you're tired or you have repetitive stress injury and can save a few keystrokes.
While I do expect a good programmer to remember most of their language's syntax and common libraries, I see no problem with having to look up the parameter order or the specific name of a method, especially when you're dealing with sprawling, messy beasts like PHP, the Win32 API or Java. What matters is that the developer knows what they want to accomplish and how to make it happen. Using visual tools for building regular expressions, on the other hand, suggests that the developer lacks a fundamental skill that should be part of their core skill set.
Again, I have no problem with someone looking up their language's specific syntax for the "word boundary" character group, or the greed-inversion switch, or any silly implementation detail of that nature. Remembering all of those details is great, but what I look for in a developer is someone who knows how to do things in general and has a sense of what the technology makes available to them. In other words, I'd rather hire someone who needs to look up the greed-inversion switch in PHP but knows it exists than someone who writes massively complicated regular expressions that could be simplified if only they knew about that switch.
For example, let's say you're trying to extract whatever is between tags in an XML document in PHP, and for some reason you don't want to use a parser. You could do something like:
which means "give me any letter, number, underscore or hyphen between > and <". It's the naive implementation, and it will work most of the time, until you run into, say, an escaped HTML entity starting with a #, at which point you have to go back and fix your regular expression.
Instead, you could do something like this:
which means "give me everything you find after > that's not a < ". In other words, "give me everything you find until you hit a <".
But perhaps more elegantly you could write:
which means "give me everything you find starting at > but stop as soon as you run into a <", using the greed-inversion switch ? after the *. I find this approach somewhat more satisfying, because it has the "everything between tags" semantics built-in, and it is pretty robust, because it assumes nothing about the input it's expecting (other than "be a reasonably well-formed XML document"). All it takes is the use of an extra switch that's sadly not mentioned in very many tutorials and that I often have to explain to very good, experienced developers.
The point of this digression is not to show off regex-building skills. The point is that a GUI tool for building regular expressions won't help much in this particular situation, because ultimately there's no question that you have to build a complicated thing--the tool only helps you cobble that thing together, but in the end you still have to live with it. There's only so much you can look up about regular expressions, and if you don't understand them or know their capabilities very well, you'll only write mediocre or brittle ones.
The upshot of all this is that tools are perfectly fine, and even encouraged, when they substitute machine reliability for human frailty. But they're certainly no substitute for a solid, well-rounded skill-set, and that well-roundedness requires a lot of painstaking, manual work. Microsoft Word's spellchecker will fix your typos, but it's not going to make you a better writer.
One more thing--real programmers don't need tools to work with regular expressions.
To which someone responded:
Yeah yeah. Neither do you need IDEs and nice flashy GUIs. Unfortunately "real programmers" are Martians on a totally different plane of existence from "real people", for which usable GUIs are made. We are on the business of helping humans, not Martians, so you "real programmers" can go ahead and marry your console apps for all we care.
I think the response misses the point. I am completely in favor of using tools to make software development easier, faster, and less error-prone. Syntax highlighting, fast access to files in your project, code auto-completion, visual diff tools like WinMerge, one-key access to your language's documentation, all of those are great--the computer's memory and ruthlessly error-free comparison skills are an excellent substitute for a human's, especially when it's late and you're tired or you have repetitive stress injury and can save a few keystrokes.
While I do expect a good programmer to remember most of their language's syntax and common libraries, I see no problem with having to look up the parameter order or the specific name of a method, especially when you're dealing with sprawling, messy beasts like PHP, the Win32 API or Java. What matters is that the developer knows what they want to accomplish and how to make it happen. Using visual tools for building regular expressions, on the other hand, suggests that the developer lacks a fundamental skill that should be part of their core skill set.
Again, I have no problem with someone looking up their language's specific syntax for the "word boundary" character group, or the greed-inversion switch, or any silly implementation detail of that nature. Remembering all of those details is great, but what I look for in a developer is someone who knows how to do things in general and has a sense of what the technology makes available to them. In other words, I'd rather hire someone who needs to look up the greed-inversion switch in PHP but knows it exists than someone who writes massively complicated regular expressions that could be simplified if only they knew about that switch.
For example, let's say you're trying to extract whatever is between tags in an XML document in PHP, and for some reason you don't want to use a parser. You could do something like:
$regex = "/>([A-Za-z0-9_-]+)</";which means "give me any letter, number, underscore or hyphen between > and <". It's the naive implementation, and it will work most of the time, until you run into, say, an escaped HTML entity starting with a #, at which point you have to go back and fix your regular expression.
Instead, you could do something like this:
$regex = "/>([^<]+)/";which means "give me everything you find after > that's not a < ". In other words, "give me everything you find until you hit a <".
But perhaps more elegantly you could write:
$regex = "/>(.*?)</";which means "give me everything you find starting at > but stop as soon as you run into a <", using the greed-inversion switch ? after the *. I find this approach somewhat more satisfying, because it has the "everything between tags" semantics built-in, and it is pretty robust, because it assumes nothing about the input it's expecting (other than "be a reasonably well-formed XML document"). All it takes is the use of an extra switch that's sadly not mentioned in very many tutorials and that I often have to explain to very good, experienced developers.
The point of this digression is not to show off regex-building skills. The point is that a GUI tool for building regular expressions won't help much in this particular situation, because ultimately there's no question that you have to build a complicated thing--the tool only helps you cobble that thing together, but in the end you still have to live with it. There's only so much you can look up about regular expressions, and if you don't understand them or know their capabilities very well, you'll only write mediocre or brittle ones.
The upshot of all this is that tools are perfectly fine, and even encouraged, when they substitute machine reliability for human frailty. But they're certainly no substitute for a solid, well-rounded skill-set, and that well-roundedness requires a lot of painstaking, manual work. Microsoft Word's spellchecker will fix your typos, but it's not going to make you a better writer.
2008-04-04
The Cheezburgery Tales
I just realized that LOLcat is really Chaucer with modern illuminations.
here bygynneth the Book of the tales of Caunterbury.
1 Whan that Aprille, with hise shoures soote,
2 The droghte of March hath perced to the roote
3 And bathed every veyne in swich licour,
4 Of which vertu engendred is the flour;
5 Whan Zephirus eek with his swete breeth
6 Inspired hath in every holt and heeth
7 The tendre croppes, and the yonge sonne
8 Hath in the Ram his halfe cours yronne,
9 And smale foweles maken melodye,
10 That slepen al the nyght with open eye-
11 So priketh hem Nature in hir corages-
12 Thanne longen folk to goon on pilgrimages
13 And palmeres for to seken straunge strondes
14 To ferne halwes, kowthe in sondry londes;
How is that any different from, say:
here bygynneth the Book of the tales of Caunterbury.
1 Whan that Aprille, with hise shoures soote,
2 The droghte of March hath perced to the roote
3 And bathed every veyne in swich licour,
4 Of which vertu engendred is the flour;
5 Whan Zephirus eek with his swete breeth
6 Inspired hath in every holt and heeth
7 The tendre croppes, and the yonge sonne
8 Hath in the Ram his halfe cours yronne,
9 And smale foweles maken melodye,
10 That slepen al the nyght with open eye-
11 So priketh hem Nature in hir corages-
12 Thanne longen folk to goon on pilgrimages
13 And palmeres for to seken straunge strondes
14 To ferne halwes, kowthe in sondry londes;
How is that any different from, say:
Subscribe to:
Posts (Atom)