When to ship
Who wants to be rich?
One of the richest people on earth has gone just a little crazy. They are working on a slot machine to give away 20 billion U.S. dollars. It is a guaranteed jackpot win within 50 pulls. Each pull is completely free. They haven’t worked out all of the bugs yet. For some reason there is a 1% chance that the machine will deliver an electrical shock strong enough to stun you and possibly cause a little burn. They ask if you’d like to play it right now. You can pull the slot machine hammer as many times as you’d like and you can keep the winnings.
What do you think most people would do? Would they pull that hammer till they hit the jackpot?
Alternatively, we could have a slot machine that is shockless, works perfectly, costs 40 dollars each pull, and rewards a maximum of 30 dollars in return. No sane person would use it.
We could say, “People will use your product once its value outweighs the product's shortcomings even if those shortcomings are severe.”
I’m not suggesting we ship frustrating or harmful products. I am saying that we should be aware of what value our customers are seeking to get from our products and to ship when those desires are met.
One game I was working on had a bug where if you played the first 30 hours without ever saving, stopped at a specific location, and then fired a weapon the game crashed. If you ever saved before that point the bug wouldn’t happen. I was surprised our testing team found that one. We ended up shipping with that bug still in the game. We never saw or heard a single comment or report of that bug post-ship. I really don’t like crash bugs. However, the opportunity cost of us delaying shipping to fix that bug versus the chance of any player being harmed by it was so low it just wasn’t worth fixing.
Customer value
We’ve been looking at value versus bugs which is only one part of the story. Over in prioritizing work I’ve talked about how to figure out how valuable any given specific feature is to include in the product. We assign value for many thing including:
If that feature is table stakes
If the feature was asked for by the customer
If the customer has shown interest in the feature
If the customer is willing to pay money for that feature
What impact the feature will have
Do our competitors already have this covered
What the current maintenance cost would look like
And many more items
A version of the feature evaluation should be done for the entire product before production to see just how valuable the product could be. As we are building the product we can see how much of that potential value we are delivering.
Fit and Culture
An audio company became interested in diversifying and hired me to start to make video games for them. I put together a small team. Worked on shipping a couple of audio related specific products they wanted and then started building entertainment products. Right before I could ship a product that I had a great deal of faith in, the company got bought by another audio company.
The new company had no interest in entertainment products so the title was killed. Even though the title had a great deal of potential value, it didn’t serve the larger corporate goals and it made the most sense to abandon the project.
You need to be sure that releasing the product will fit with the company's view for market, value, quality, statement, and culture. If the product doesn’t fit the company’s requirements, adjust the product.
Alternatively, you can adjust the company. Look at how Nintendo made instant rice, Avon sold books, Wrigley sold soap and baking powder. Don’t be a slave to what your company is. You can easily destroy a company with corporate requirements that have no real impact or value. Sometimes it is ok to ship a product that will update your corporate goals.
Ok, but when?
Ship when the product is delivering the intended value, that value is not destroyed by drawbacks, and represents the company’s requirements for it.
Intended value can be tested against. If you are building a calculator you can test directly against it working or not. If you are creating an open world game you can use player statistics to infer the value. Intended value doesn’t mean when every feature has been completed. If your consumers want the value from the product right now, give it to them. The other features can come later as needed.
Value being destroyed by drawbacks can be determined with a few techniques. First, ask the consumer. No secrets here. Consumers will happily tell you what they want. The more frustrating the issue the louder the voices. If there are no voices, be concerned that the product lacks value.
Secondly, you can determine value being destroyed by drawbacks from long term use. As I’ve mentioned before, I used to go to the testing department right before their lunch time to see how QA would stop playing my game. If they threw the controller down and left I had much more work to do. If they played for a little more and then left for lunch, I was getting closer. If they played through their lunch I knew I had a good game. These days we can easily track play habits and see how they align with our intent. How many bugs are users experiencing? Do those bugs affect the experience? Do users take a break after hitting an issue or using a feature?
Corporate requirements should be an easy bar. You know your company. Look at the product the way it is today, not tomorrow, not some future version, today. Does that fit your company or not? If not, do you have unreasonable corporate requirements? Will you tank the company because of them?
If the product passes these tests then ship it right now. Don’t think. Don’t keep building, fixing or perfecting.
Just ship it.