13 April 2006 to 12 March 2006
¶ alpha beta go · 13 April 2006 essay/tech
Say you're making software to do X. The audience for this is people who want to do X, and the evaluation criteria for the software to do X is that it actually can be used to do X in some respectable and hopefully distinguished manner.
Before you get to the released version of your software to do X, there are two useful stages you will probably want to pass through: beta and alpha. These are not percentages of completion or levels of toleration of error, they are themselves tools for different purposes than the released version. X beta exists to provide an evaluable preview of X. X alpha exists to provide an evaluable preview of X beta.
It is not only possible, but critical, to substitute these purposes of beta and alpha back into the original statement. Like this, for beta:
Note that this is different. The people interested in previewing a tool to do X are not necessarily the same people as the ones who want to do X, and their goal is not doing X but seeing how your software is going to do X.
Likewise with alpha:
That is, the beta is a tool for seeing how the real thing is eventually going to work, and the alpha is a tool for seeing how the beta is eventually going to work. The audience for alpha is thus double-meta, and is likely to be very small, sometimes even smaller than the team working on it.
Even more crucially, the substitution works in the other direction, too: beta is a released version of software for evaluating a preview of X, and the alpha is a released version of software for evaluating a preview of the beta. So while the X beta may have omissions and errors in its performance of X without failing, if it has omissions and errors in its performance of the evaluation and preview of X, it fails.
Thus the released version of an online calendar for shared scheduling, for example, fails if it can't be used for shared scheduling. But the beta fails if it can't be used for experimenting with shared scheduling. So the beta doesn't require that all the shared scheduling features work, but it does require that all the experimentation features are ready.
If your software involves manipulating user data of any kind, then, here are some things that probably have to be already working, without appreciable error, in your beta:
- import
- reimport
- importing useful sample data
- distinguishing data by import batch
- undoing import batches
- deleting data
- deleting data en masse
- full data purge and restart of beta experience
- comparing your treatment of this data to the treatment of it wherever it came from
- comparing different treatments of this data within your own software
- export/sync/feed of this data to any evaluation-critical downstream integration points
- sync/feed of source data from any evaluation-critical upstream integration points
- full operation of a representative subset of anything critical of which there will be multiples (integration points, environments, styles, templates, other users, etc.)
- any configuration settings integral to personal evaluation
- clear explanation of beta limitations
If any of these aren't done, don't ship your beta yet. If some of them aren't even in your design, you've got more designing to do.
Before you get to the released version of your software to do X, there are two useful stages you will probably want to pass through: beta and alpha. These are not percentages of completion or levels of toleration of error, they are themselves tools for different purposes than the released version. X beta exists to provide an evaluable preview of X. X alpha exists to provide an evaluable preview of X beta.
It is not only possible, but critical, to substitute these purposes of beta and alpha back into the original statement. Like this, for beta:
Say you're making software to {provide an evaluable preview of X}. The audience for this is people who want to {evaluate a preview of X}, and the evaluation criteria for the software to {provide an evaluable preview of X} is that it actually can be used to {evaluate a preview of X} in some respectable and hopefully distinguished manner.
Note that this is different. The people interested in previewing a tool to do X are not necessarily the same people as the ones who want to do X, and their goal is not doing X but seeing how your software is going to do X.
Likewise with alpha:
Say you're making software to {provide an evaluable preview of an evaluable preview of X}. The audience for this is people who want to {evaluate a preview of an evaluable preview of X}, and the evaluation criteria for the software to {provide an evaluable preview of an evaluable preview of X} is that it actually can be used to {evaluate a preview of an evaluable preview of X} in some respectable and hopefully distinguished manner.
That is, the beta is a tool for seeing how the real thing is eventually going to work, and the alpha is a tool for seeing how the beta is eventually going to work. The audience for alpha is thus double-meta, and is likely to be very small, sometimes even smaller than the team working on it.
Even more crucially, the substitution works in the other direction, too: beta is a released version of software for evaluating a preview of X, and the alpha is a released version of software for evaluating a preview of the beta. So while the X beta may have omissions and errors in its performance of X without failing, if it has omissions and errors in its performance of the evaluation and preview of X, it fails.
Thus the released version of an online calendar for shared scheduling, for example, fails if it can't be used for shared scheduling. But the beta fails if it can't be used for experimenting with shared scheduling. So the beta doesn't require that all the shared scheduling features work, but it does require that all the experimentation features are ready.
If your software involves manipulating user data of any kind, then, here are some things that probably have to be already working, without appreciable error, in your beta:
- import
- reimport
- importing useful sample data
- distinguishing data by import batch
- undoing import batches
- deleting data
- deleting data en masse
- full data purge and restart of beta experience
- comparing your treatment of this data to the treatment of it wherever it came from
- comparing different treatments of this data within your own software
- export/sync/feed of this data to any evaluation-critical downstream integration points
- sync/feed of source data from any evaluation-critical upstream integration points
- full operation of a representative subset of anything critical of which there will be multiples (integration points, environments, styles, templates, other users, etc.)
- any configuration settings integral to personal evaluation
- clear explanation of beta limitations
If any of these aren't done, don't ship your beta yet. If some of them aren't even in your design, you've got more designing to do.
Elsewhere, Brian Del Vecchio, Rod Begbie and I are, at Brian's inspiration and so far frankly with mostly his work, trying to get a group blog going about Boston-area new-technology developments. I knew the idea was brilliant as soon as Brian snagged the domain name BostonWTF.com.
¶ new job · 10 April 2006
I won't actually go back to work for a little while longer, but I now know where I'll be working once I do.
I really wanted, in the spirit of openness, to journal my job-search. I think we benefit from sharing our evaluations and processes and experiences, in general, and finding a new job isn't structurally all that different from the kinds of consumer decisions that it's already considered wholly appropriate to write about in public. Legally speaking, I was not under formal non-disclosure agreements anywhere I visited or interviewed, and morally it would have been trivial and narratively harmless to leave out the few specific details that were revealed to me in informal confidence.
But the big difference between consumer decisions and getting a new job, obviously, at least here in the part of the world where I buy and work, is that sellers don't get to choose their buyers, so consumer openness mostly doesn't introduce any risk of recoil. I could have written about my job-search anonymously (and for all you know, I did), but writing about it here would have put my thoughts into the awareness of my potential employers. That should be safe, in the same way that writing openly about your actual employers (short of definable trade-secrets, and that should be more carefully circumscribed and limited, too) ought to be much more rigorously defended. But some precedent still needs to be set, and I wasn't quite up to putting myself in the position to set it. Until I found a new job, I couldn't be sure that I wouldn't end up wanting or needing to take one about which I'd previously expressed reservations or criticisms.
In the end, though, my search turned out to be less contentious than I imagined it might. I walked out of more than one office thinking "Well, I'm not working there", but almost always due more to a mismatch of interests than any overwhelming sense of systemic dysfunction. I visited only one company that made me feel like I'd spent a day on the B Ark, and two of the most successful VC firms in the world had just expressed several million dollars of confidence in its commercial viability, so what do I know?
My search was also a lot less involved than I expected. This was the first time in my working life that I'd ever left one job without the next one already lined up, and really the first time I'd ever done a job search, as opposed to having a new job basically present itself to me at random. The overwhelming and unsurprising moral is that connections are everything. I spent a lot of time digging through public job-listings and refining filter subscriptions and sending off resumes to careers@... addresses, and nothing even marginally promising ever resulted from any of that. All meaningful leads came through people I knew. Weirdly, maybe, almost all my meaningful leads came through friends and colleagues, not former bosses, and in no case was anybody I know directly involved in the decisions about my role.
But I'd have spent even less time and energy on the process, and none pondering the ethics and repercussions of writing about it, if I'd known ahead of time that the very first casual job-related conversation I scheduled after being laid off, the connection to which came through my own sister, would end up leading to basically my dream job. This will probably be my easiest employer to explain, and my most arcane actual job. The company makes software for finding plane tickets, most prominently for the travel sites Orbitz, CheapTickets and Kayak, and the ticket-search functions on the airline sites for US Airways, Alitalia, Continental, and Alaska Airlines. The founder was recently listed by Travel & Leisure magazine as one of the 35 innovators who are changing the way people travel, and their blurb about him ended with this comment: "But it's Wertheimer's next undertaking that will most affect travelers. Called Project Needle (as in haystack), it uses artificial intelligence to search travel sites, quickly 'learning' them and giving shoppers only the information they're looking for."
I'm not at liberty to explain the exact relationship between this description and reality, but there is a real project to which they are referring, and I will be working on it. In casual conversation I will probably usually summarize it as "semantic web searching", which, depending on where you put the hyphen, has to do with either how to structure human information so that it can be better synthesized by machines, or how to teach machines to better help humans extract useful understanding from bodies of information not already structured that way. It will be a while before you get to see any form of this, and probably a while before I can even say what that form might be. But I believe it's one of the mutations that will shape the future evolution of the net, and I'm thrilled to get to participate in something I would spend so much of my energy pondering and following even if I weren't.
And although this could have been my dream job even if I'd had to move to California or Norway to do it, its most hilarious minor virtue is that the office is so close to my house that I think the single Walk light between here and there accounts for 20% of the variability in my commute time. In fact, coworkers who live elsewhere and drive to this office have to walk farther from their parked cars than I will from my bed.
But maybe the future is always closer than you think.
I really wanted, in the spirit of openness, to journal my job-search. I think we benefit from sharing our evaluations and processes and experiences, in general, and finding a new job isn't structurally all that different from the kinds of consumer decisions that it's already considered wholly appropriate to write about in public. Legally speaking, I was not under formal non-disclosure agreements anywhere I visited or interviewed, and morally it would have been trivial and narratively harmless to leave out the few specific details that were revealed to me in informal confidence.
But the big difference between consumer decisions and getting a new job, obviously, at least here in the part of the world where I buy and work, is that sellers don't get to choose their buyers, so consumer openness mostly doesn't introduce any risk of recoil. I could have written about my job-search anonymously (and for all you know, I did), but writing about it here would have put my thoughts into the awareness of my potential employers. That should be safe, in the same way that writing openly about your actual employers (short of definable trade-secrets, and that should be more carefully circumscribed and limited, too) ought to be much more rigorously defended. But some precedent still needs to be set, and I wasn't quite up to putting myself in the position to set it. Until I found a new job, I couldn't be sure that I wouldn't end up wanting or needing to take one about which I'd previously expressed reservations or criticisms.
In the end, though, my search turned out to be less contentious than I imagined it might. I walked out of more than one office thinking "Well, I'm not working there", but almost always due more to a mismatch of interests than any overwhelming sense of systemic dysfunction. I visited only one company that made me feel like I'd spent a day on the B Ark, and two of the most successful VC firms in the world had just expressed several million dollars of confidence in its commercial viability, so what do I know?
My search was also a lot less involved than I expected. This was the first time in my working life that I'd ever left one job without the next one already lined up, and really the first time I'd ever done a job search, as opposed to having a new job basically present itself to me at random. The overwhelming and unsurprising moral is that connections are everything. I spent a lot of time digging through public job-listings and refining filter subscriptions and sending off resumes to careers@... addresses, and nothing even marginally promising ever resulted from any of that. All meaningful leads came through people I knew. Weirdly, maybe, almost all my meaningful leads came through friends and colleagues, not former bosses, and in no case was anybody I know directly involved in the decisions about my role.
But I'd have spent even less time and energy on the process, and none pondering the ethics and repercussions of writing about it, if I'd known ahead of time that the very first casual job-related conversation I scheduled after being laid off, the connection to which came through my own sister, would end up leading to basically my dream job. This will probably be my easiest employer to explain, and my most arcane actual job. The company makes software for finding plane tickets, most prominently for the travel sites Orbitz, CheapTickets and Kayak, and the ticket-search functions on the airline sites for US Airways, Alitalia, Continental, and Alaska Airlines. The founder was recently listed by Travel & Leisure magazine as one of the 35 innovators who are changing the way people travel, and their blurb about him ended with this comment: "But it's Wertheimer's next undertaking that will most affect travelers. Called Project Needle (as in haystack), it uses artificial intelligence to search travel sites, quickly 'learning' them and giving shoppers only the information they're looking for."
I'm not at liberty to explain the exact relationship between this description and reality, but there is a real project to which they are referring, and I will be working on it. In casual conversation I will probably usually summarize it as "semantic web searching", which, depending on where you put the hyphen, has to do with either how to structure human information so that it can be better synthesized by machines, or how to teach machines to better help humans extract useful understanding from bodies of information not already structured that way. It will be a while before you get to see any form of this, and probably a while before I can even say what that form might be. But I believe it's one of the mutations that will shape the future evolution of the net, and I'm thrilled to get to participate in something I would spend so much of my energy pondering and following even if I weren't.
And although this could have been my dream job even if I'd had to move to California or Norway to do it, its most hilarious minor virtue is that the office is so close to my house that I think the single Walk light between here and there accounts for 20% of the variability in my commute time. In fact, coworkers who live elsewhere and drive to this office have to walk farther from their parked cars than I will from my bed.
But maybe the future is always closer than you think.
¶ the more job interviews you go through, the more you think you know about how companies should be run · 30 March 2006 tech
- Essentially all decisions are at least partly wrong. The act of making a decision, arguably, is primarily the emotional acceptance of the inevitability of error. The difference between good decisions and bad decisions is not that the good decisions have fewer flaws, but that their flaws are more inspiring. Thus the art of deciding lies in erring creatively. And thus the primary component of leadership is creativity.
- If you are demoing a prototype to VCs (or me, pretending to be a VC), try not to accidentally show them porn unless that's actually part of your business plan.
- The fact that so many businesses have means without ends does not make it any more sensible to run a company with an end but no means. Or, to put this another way, if you have more Senior Vice Presidents than either months until your first product launches, or programmers, you are not really a software company.
- It's usually easier to talk one person out of a million dollars than it is to talk a million people out of a dollar each, but it's probably easier to talk a million people out of a dollar a hundred times each than to talk a hundred people out of a million dollars, or ten out of ten million. (And even easier to not need a million dollars.)
- Business development is about figuring out where money is. Marketing is about convincing the people with the money to give it to you. These are money activities, and thus the proper occupation of money companies. A software company develops software, which is hard to do well, and thus should only be attempted where you believe there's a genuine chance to improve something in the world. A commercially successful software company does something valuable for somebody, and then finds a way to allow them to express their appreciation.
- You can buy programmers pizza and they will appreciate it. Actually, you can buy most people pizza and they will appreciate it. And pizza is cheap. Thus there's no reason to lose more money than pizza costs, in pursuit of anything people are going to appreciate less.
- Process and specialization are attempts to minimize individual variation. But most new businesses fail, so individual variation is actually your only hope.
- Do not tell a job candidate that you have a great working environment and then keep them in a 60-degree room from 10am to 3pm without food. For extra credit, do not brag about the quality of your hiring process while reading their resumé for the first time right in front of them. And if the job requires that a candidate arrive with any kind of pre-existing expertise or experience, it's minimal courtesy to Google them ahead of time to find out if they have any.
- If you think every other company in your field is run by idiots, you're probably almost right.
- If you are demoing a prototype to VCs (or me, pretending to be a VC), try not to accidentally show them porn unless that's actually part of your business plan.
- The fact that so many businesses have means without ends does not make it any more sensible to run a company with an end but no means. Or, to put this another way, if you have more Senior Vice Presidents than either months until your first product launches, or programmers, you are not really a software company.
- It's usually easier to talk one person out of a million dollars than it is to talk a million people out of a dollar each, but it's probably easier to talk a million people out of a dollar a hundred times each than to talk a hundred people out of a million dollars, or ten out of ten million. (And even easier to not need a million dollars.)
- Business development is about figuring out where money is. Marketing is about convincing the people with the money to give it to you. These are money activities, and thus the proper occupation of money companies. A software company develops software, which is hard to do well, and thus should only be attempted where you believe there's a genuine chance to improve something in the world. A commercially successful software company does something valuable for somebody, and then finds a way to allow them to express their appreciation.
- You can buy programmers pizza and they will appreciate it. Actually, you can buy most people pizza and they will appreciate it. And pizza is cheap. Thus there's no reason to lose more money than pizza costs, in pursuit of anything people are going to appreciate less.
- Process and specialization are attempts to minimize individual variation. But most new businesses fail, so individual variation is actually your only hope.
- Do not tell a job candidate that you have a great working environment and then keep them in a 60-degree room from 10am to 3pm without food. For extra credit, do not brag about the quality of your hiring process while reading their resumé for the first time right in front of them. And if the job requires that a candidate arrive with any kind of pre-existing expertise or experience, it's minimal courtesy to Google them ahead of time to find out if they have any.
- If you think every other company in your field is run by idiots, you're probably almost right.
¶ CD death-watch update · 24 March 2006
A reader of my previous note wrote to report the same problem with Showtunes on her PowerBook. I've emailed Nonesuch to see if they have any insight.
Metropolis did write back, saying there's no copy-protection on the Gary Numan album and they weren't aware of any problems. Their official guess is that my copy is defective, so if I send it in, they'll mail me a replacement. My Showtunes experiences makes me doubt that this will help, but the two cases may end up being unconnected.
Metropolis did write back, saying there's no copy-protection on the Gary Numan album and they weren't aware of any problems. Their official guess is that my copy is defective, so if I send it in, they'll mail me a replacement. My Showtunes experiences makes me doubt that this will help, but the two cases may end up being unconnected.
¶ a CD death-spasm? · 22 March 2006
In the three-plus years that I've had my PowerBook, I've ripped thousands of my own CDs into iTunes to listen to them. The only one I couldn't rip was an Australian release by David Bridie that I digitized as audio because I didn't feel like sending it back to Australia.
But in the last week I've hit two more. I tried two different copies of Stephin Merritt's Showtunes, thinking the first one was just defective, but the PowerBook just kicked both of them out as if they weren't audio discs. The packaging didn't indicate any sort of copy-protection, and Nonesuch seems like an odd offender, but my wife's PowerBook (different model, different OS version) just ejected both copies, too. I was irritated enough to take the second copy of Showtunes back to the record store and wade through their reluctance to get a refund. Merritt has too many tracks to digitize and index by hand.
In exchange I bought the second Sounds album, and the new Gary Numan record. But the Gary Numan album has the same problem, again on both machines. Put it in, listen to some fruitless whirring, watch it get ejected as unreadable. Two solid hours of Mac troubleshooting changed nothing. Both machines can still read other new and old CDs and DVDs fine. It's a pretty strange coincidence if these discs are normal but both of our machines happen to have started having this very particular trouble at the same moment.
But it's also a pretty strange coincidence if I've just happened to hit two of the three computer-unreadable CDs I've ever encountered (out of well over a thousand I've ripped) right in a row. I've written Numan's label (Metropolis) to see what they have to say for themselves. My fear, I guess, is that some new idiot manufacturing process has just come on line, and half the new music I try to buy from now on is going to do this. My fear is that I'll have to switch to stealing all my music simply in order to be able to hear it.
But in the last week I've hit two more. I tried two different copies of Stephin Merritt's Showtunes, thinking the first one was just defective, but the PowerBook just kicked both of them out as if they weren't audio discs. The packaging didn't indicate any sort of copy-protection, and Nonesuch seems like an odd offender, but my wife's PowerBook (different model, different OS version) just ejected both copies, too. I was irritated enough to take the second copy of Showtunes back to the record store and wade through their reluctance to get a refund. Merritt has too many tracks to digitize and index by hand.
In exchange I bought the second Sounds album, and the new Gary Numan record. But the Gary Numan album has the same problem, again on both machines. Put it in, listen to some fruitless whirring, watch it get ejected as unreadable. Two solid hours of Mac troubleshooting changed nothing. Both machines can still read other new and old CDs and DVDs fine. It's a pretty strange coincidence if these discs are normal but both of our machines happen to have started having this very particular trouble at the same moment.
But it's also a pretty strange coincidence if I've just happened to hit two of the three computer-unreadable CDs I've ever encountered (out of well over a thousand I've ripped) right in a row. I've written Numan's label (Metropolis) to see what they have to say for themselves. My fear, I guess, is that some new idiot manufacturing process has just come on line, and half the new music I try to buy from now on is going to do this. My fear is that I'll have to switch to stealing all my music simply in order to be able to hear it.
¶ 12.2 · 13 March 2006
Before Saturday, my longest single run was about 8.5 miles. Saturday I ran 12.2. I'm a record-keeper, so there's a certain inherent appeal for me in getting to update one, but in neither of these particular cases was distance the independent variable, so it's not really a significant statistic. Plus, I run 6 miles routinely and comfortably, so I figured I could at least survive a 12-mile run.
I didn't just survive, though. I only planned to do 6 miles, but it was sunny and in the 60s in Boston, and the Charles was lined with people gamely not focusing on the possible climatic implications of Spring in Boston in early March, and it just felt good to stay out there longer, so I did. My usual pace for a relaxed 6 miles is around 7:30/mile, and I finished 12.2 in a steady 7:36/mile, feeling fine.
But that's still not the real triumph. The real triumph is that today, Monday, two days later, I did my routine 6-mile run right back on schedule. It was still routine, still comfortable, 7:32/mile, and afterwards I still feel fine.
There are the things you think you could do, and the things you've done once and think you could do again, and then there are the things that are simply now within you.
I didn't just survive, though. I only planned to do 6 miles, but it was sunny and in the 60s in Boston, and the Charles was lined with people gamely not focusing on the possible climatic implications of Spring in Boston in early March, and it just felt good to stay out there longer, so I did. My usual pace for a relaxed 6 miles is around 7:30/mile, and I finished 12.2 in a steady 7:36/mile, feeling fine.
But that's still not the real triumph. The real triumph is that today, Monday, two days later, I did my routine 6-mile run right back on schedule. It was still routine, still comfortable, 7:32/mile, and afterwards I still feel fine.
There are the things you think you could do, and the things you've done once and think you could do again, and then there are the things that are simply now within you.
¶ 13 March 2006
Even the most movie-addicted people I know don't usually take the time to re-watch movies with the DVD commentary track on. I do, but I was a filmmaking major in college, and have a really high tolerance for obsessive introspection in any format.
But if you haven't seen the movie Spy Kids, go watch it, especially if you haven't seen it because it's a kids' movie and you aren't a kid. It is a kids' movie, but in a way that you should still be a kid.
And then, if you haven't seen Spy Kids 2, go watch that, especially if you haven't seen it because it's a kids' movie (see above), and especially if you haven't seen it because it's a sequel. It is a sequel, but it's the kind of sequel that the first movie makes wonderfully possible, not cynically expedient.
And then watch Spy Kids 2 again with Robert Rodriguez's commentary on. It's not laborious annotation, it's an accidental evangelist's 97-minute exhortation to creativity and experimentation and simplicity and complexity, and not only makes me love the movies more, but makes me happier and more excited about art, craft, technology, freedom and everything I haven't tried yet.
But if you haven't seen the movie Spy Kids, go watch it, especially if you haven't seen it because it's a kids' movie and you aren't a kid. It is a kids' movie, but in a way that you should still be a kid.
And then, if you haven't seen Spy Kids 2, go watch that, especially if you haven't seen it because it's a kids' movie (see above), and especially if you haven't seen it because it's a sequel. It is a sequel, but it's the kind of sequel that the first movie makes wonderfully possible, not cynically expedient.
And then watch Spy Kids 2 again with Robert Rodriguez's commentary on. It's not laborious annotation, it's an accidental evangelist's 97-minute exhortation to creativity and experimentation and simplicity and complexity, and not only makes me love the movies more, but makes me happier and more excited about art, craft, technology, freedom and everything I haven't tried yet.
¶ Make-believe soccer · 12 March 2006
Anyone who wants to have more reason to feel involved with arbitrary MLS games this year should go sign up for MFLS, the best and longest-running MLS fantasy league.
The roster deadline for week 1 is 3pm Eastern, April 1 (and remember that Daylight Saving Time starts on April 2 this year).
The roster deadline for week 1 is 3pm Eastern, April 1 (and remember that Daylight Saving Time starts on April 2 this year).
¶ my new favorite spam title · 12 March 2006
"I've been sad with my magnitude lately, what and you."
I know what it's insinuating, of course, but sometimes poetry writes itself into our deceits while we aren't watching them closely enough. Our powers are so great, and yet here we are, lately as ever, sad with our magnitudes.
I know what it's insinuating, of course, but sometimes poetry writes itself into our deceits while we aren't watching them closely enough. Our powers are so great, and yet here we are, lately as ever, sad with our magnitudes.