Can your marketers tell a test from a product?

Depending on what kind of business you are in, this could seem like a stupid question. If you sell consumer packaged goods, clearly you can tell the difference between the research department taste testing a new kind of cookie with 100 people and actually mass producing them to ship to every store in the free world. Manufacturing businesses are like that–there is a clear distinction between a test and a product. But not every business is like that, which I learned along the way in my 30 years of working for IBM.

Now IBM does its share of manufacturing, so it wasn’t always tough. A prototype for a new chip or a new computer server is easily distinguishable from the real thing. No marketer would ever try to sell those. First off, they know that we only have a couple of them on hand, and secondly, they usually don’t look all that good. They are designed mostly to prove that they work. You show them to customers to get a reaction, not a sale. That’s the real difference between a test and a product. If the reaction is positive, then it might make sense to manufacture the product.

But the distinctions between a test and a product aren’t so clear in other businesses–even other IBM businesses. I remember the first time that I participated in a consulting engagement, back in the 1980s. IBM was new to consulting and we didn’t know what we were doing. (It’s one of the great strengths of IBM that it tries things and keeps working until it gets them right, because now consulting is one of its most profitable businesses.)
We were working with a company that is a household name. We had this great idea of how to revolutionize the way it used technical documentation for its products so that every dealer could have the information available online rather than on paper. (OK, OK, I am old, but this was revolutionary 25 years ago.)

We decided the best way to do the job was to work with a dealer and show how valuable this was. We trained the owner and his employees and they loved the system, marveling at how much of an improvement it was, and it cost less, too! Nirvana, right?

Not exactly.

This name-brand company now wanted us to roll this out to all 2000 dealers in 100 countries. Now. We didn’t have the slightest idea how to do that. There were just a few of us that understood the system. We had tested it but we didn’t know how to manufacture it, because none if us had been trained to package consulting engagements into repeatable processes. We were just subject-matter experts with a great idea. We needed a miracle to succeed at a 2000-dealer project. Maybe cloning.

I re-learned this lesson (I’m slow) a few years later when an IBM executive came to me with what we both thought was a brilliant idea for a software product. He knew exactly the customer who would want it, too. So he told me to take a few weeks to build it so he could show it to the customer. I told him that there’s no way to build a complete functioning product on my own in a couple of weeks, but that I could definitely put together a demonstration that would show everyone what the product can do.

Sure, he said, waving away these pesky details. So, I put together the demo. It was quite impressive, even to the point of having simulated data that changed when you pressed the right buttons. It wowed the customer. They were sold. They wanted one now. Yeah, now. But we didn’t have any–this was just a demo.

Time was the software was manufactured–you needed to stamp out disks and print manuals and shrink-wrap boxes, but no more. Now, you can have software-as-a-service (SaaS) applications that require no manufacturing at all. Maybe it looks like a demo, but some marketers (like my IBM exec) and most customers can’t tell the difference between the demo and the product.

If your marketers (or your customers) can’t tell the difference between a test and a product, then you might want to avoid the tests. Take that software demo as an example of the ridiculous situations you get into:

• The marketer does not want to tell the customer that it is a demo, because then they will be less impressed.
• If the customer doesn’t like the demo, then you wasted all the time you spent building it.
• If the customer likes the demo, then you might have to fess up that it’s only a demo. Or you might actually sell the demo to their customer as if it is a product, and then you’ll disappoint the customer because it doesn’t actually work.

So, do you work in a business where the test can be mistaken for the product? How do you handle that?