• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar

Lars Lofgren

Building Growth Teams

  • About Me
  • What I’ve Read
  • Want Help?

Product Development

How to Test When You Don’t Have Any Data

August 30, 2012 By Lars Lofgren Leave a Comment

We all know how important testing is. Some of our marketing works, some of it doesn’t. Some customer love us, others don’t. Testing helps us cut through that endless fog of uncertainty.

If you spend any time in the startup or online marketing circles, you’ll hear how wonderful A/B testing is. Let’s say you want to test your home page. Throw up two different versions (usually the current version and your new one), compare the results, and BAM, you know which is better.

Sounds easy right?

Well it isn’t. Here’s the thing, we need traffic to make those tests valid. If you only have 10 people go through your test, the numbers are just too small. You won’t be able to learn anything. At that volume, our results are just random coincidence. We mine as well just flip a coin. So we need dozens, hundreds, and preferably thousands of people to run our test on.

Well that’s just swell. What do we do if we don’t have hundreds of customers or visitors? What if we have like 2?

Even when you only have a few customers or visitors, you still need to be testing. And you’ll need to take a completely different approach. And it just so happens that we use this approach every day at KISSmetrics (for slightly different reasons).

Instead of obsessing over data, we need to reach out and talk to people directly.

Let’s run through the nitty gritty with a real example.

How I Get Test Results From Small Groups of People

Right now, I’m helping redesign the People Search report in KISSmetrics. The report helps you find different groups of people within your data. And we’re revamping it to provide better insights and making it easier to use.

If we followed the conventional approach, we would build a second version of the report, enable it for a small portion of our customers, and compare engagement rates between the two groups.

But we’ve gone a different route.

Before we drafted a single new design, we started talking to our customers. In fact, we scheduled meetings with 5 of them to get a DEEP understanding of how they use the report. We learned why they use the report, what frustrates them, and how the report helps them build a better business.

And we didn’t stop there.

We started putting together some mock-ups (really rough drawings) of ideas we had on how to make it better. Then we scheduled some more meetings. This time, we showed the drawings alongside the current version. First, we asked them to walk us through the current report and how they use it. Then we showed them the drawings of the new version and asked them what they liked and didn’t like. We ran our own little A/B test right on Skype.

The feedback we got was amazing. It saved us hours of work and kept us from building something that would have been much worse. Right away, we learned that one of our ideas was straight up terrible. Not kinda sorta bad. Completely terrible.

With a few meetings and a drawing, we ran our A/B test with limited data.

In our case, we’re looking for speed. Will 5 people give us a complete picture of what all of our customers want? No. But it will allow us to move a LOT faster. If we wanted to do a legit A/B test, we’d have to make a bunch of guesses on what we think are improvements, build and design the whole thing, then launch it. This takes weeks of effort.

Instead, we can get quick feedback on our overall direction and change course without having to build the whole thing.

We also gain a depth of understanding that a simple A/B test would never give us. By talking through the report with someone, we understand what they’re biggest frustrations are and how they incorporate it into their business. Then we’re able to build a new version that goes far beyond the first.

And this process works just as well when you don’t have enough data to run a full A/B test.

What if you only have 2 customers? Reach out to them and talk through your different ideas. What if you don’t have ANY customers? Get in touch with the types of people you think would be your customers. By listening for the passion in people’s voices, you’ll quickly know if you’re on track.

When multiple people start getting excited about the same idea, that’s when it’s time to turn that idea into a reality.

5 Best Practices When Talking to Customers and Testing Ideas

1. Ask open-ended questions without leading the customer.

Don’t show someone a drawing of a new design or product, explain the entire thing to them, and then ask for feedback. You won’t be there to explain the concept to every customer. You want that initial reaction to better understand what customers will be going through when they see it for the first time on their own.

2. This is not a sales call.

Don’t sell anything, not even yourself. In fact, you should be listening a lot more than you’re talking. Spend your energy focusing on understanding the perspective of who you’re talking to and ask them great questions.

3. Take notes.

Once you start doing several of these, it’s going to be really hard to keep track of who said what. Taking notes will make sure you’re able to keep everything straight. Feel free to record the meeting if you don’t want to take notes at the same time. You can even have your recordings transcribed for you and save some time.

4. Look for passion.

You’re on the right track when people start to get excited over your ideas. If they start asking “Where can I get this?” or “When will it be available?”, you’re definitely on to something. Listen for the passion in their voice, it’ll tell you what’s most important.

5. Look for trends.

We still want to be careful with not getting enough data. Remember that one idea I had to completely throw out because no one liked it? Well, one guy liked it. If I had only talked to him, I would have recommended changes that the rest of our customer base would have HATED. Avoid making changes on a single comment. Look for things that come up over and over again. That’s where you want to focus.

The Bonus Points of Awesome

As you start doing these interviews on a regular basis, you’ll enjoy another serious benefit. You’ll start building relationship with all the customers you’re talking to. They’ll go from happy customers to loyal evangelists without you having to do any extra work. Simply by reaching out to people and asking them for help, they’ll become more committed to your cause. They won’t be able to stop talking about how awesome you are when they see how their feedback helps you build a better product.

This alone is worth the effort of all these interviews. Combined with learning which of your ideas will work the best, this is one of the easiest ways to grow your business.

The Four Stages of Entreprenuerial Growth – with Michael Masterson

March 14, 2012 By Lars Lofgren Leave a Comment

In Ready, Fire, Aim, Michael Masterson details the process that he’s used to build many multi-million dollar businesses. And it’s one of the best books on entrepreneurial strategy I’ve found yet. You’re not going to find much in the way of tactics here. But you will get a complete framework for how to approach (and grow) different stages of your business.

According to Masterson, every business has four stages of growth. Here’s how they break down.

Stag One: Infancy ($0 to $1 Million in Revenue)

The vast majority of entrepreneurs are in stage one.

Your single overriding, all-encompassing, SOLE priority is to sell your product. Nothing else matters. Don’t get distracted with other product ideas, rigid management systems, or anything else. Spend your time figuring out how to sell to your target market.

Figure out what your target market REALLY cares about. This is not easy and requires countless hours of research. Find people in your target market, then get them on the phone or take them out to lunch. Get to know their fears, desires, and life perspective. Only when you completely understand them will you be able to develop the perfect sales pitch. Don’t think you can do this with surveys either. When you’re just starting, you need to actually talk to your customers.

Once you know who your customer is, you need to figure out the perfect way to get in front of them. It may be specific blogs, trade magazines, sales calls, or conferences. There is no easy way to go about this. Come up with a bunch of ideas that seem reasonable, figure out a way to test those ideas without committing too much time or money, then start at the top of your idea list and work your way down.

You’ll spend your days experimenting with different sales pitches, trying new marketing channels, and getting to know your customers.

If you play this game like a pro, you’ll grow faster than any of your competitors. And as you get some traction, you’ll work on improving your product. But you won’t create new products, that comes in…

Stage Two: Childhood ($1 million to $10 million)

When you start moving into stage two, you’ve become an expert in selling your one product. You know the industry better than anybody else. You’ve mastered the art of marketing and sales. If there’s a customer out there looking for your product, you have found them.

The good news is you’ve found all of your customers, the bad news is you’ve run out of customers. You’ve cornered the market and there’s no one left to sell to. So your growth stalls.

To break into the second stage, you need to transition from an organization that sells one product to one that can create, test, and grow new products as fast as possible. You’re going to diversify and expand your business into new categories. You’ll even create new brands.

You’ll need to figure out how to build new products and how to release them more frequently. Once again, the name of the game is testing. In Stage 1, you tested marketing ideas with a single product. Now you’re testing multiple marketing ideas and multiple products. You’ll need to build an organization that can handle this task.

Stage Three: Adolescence ($10 million to $50 million in revenue)

You’ve turned your business into a selling machine. Even better, you crank out new products at will. There’s just one problem, you’re moving too fast for your own good.

Your business has now grown to hundreds of employees and severely lacks any overall organization. And your customers are starting to notice. Not everything is as polished as it used to be, many details start slipping through the cracks. You’ll need to build the processes and procedures that will take you to the next level.

This is the realm of the MBA. You’ll be making your business more corporate.

The real trick here is balancing the assets you’ve already built (product development, marketing, and sales) with organizational processes. You definitely need more structure to support the size of your business but you don’t want to bury your growth teams with a mountain of paperwork or bureaucracy. You’ll have to keep a vigilant watch during this stage to strike a good balance between growth and structure.

Stage Four: Adulthood ($50 million in revenue and beyond)

You’re no longer a small business and you might actually get featured in Forbes or Fortune. You figured out how to sell beautifully, how to develop an endless stream of products, and to build enough structure to support your growth without crushing it.

This is the stage many entrepreneurs dream of but very few ever get the chance to experience.

It’s also the fun part with plenty of options. you can:

  • Sell to a larger company
  • Go for the IPO
  • Only work on the components of the business that you’re super passionate about
  • Assume the role of an advisor or consultant to the company
  • Continue to build the guiding vision of your business

Basically, you get to do whatever you want. You’ve built a business that operates and grows on it’s own. And now, your role will be closer to an investor than the CEO.

Are these hard numbers? Don’t some companies reach Stage 2 before they hit $1 million in annual revenue?

Absolutely. These numbers are very general benchmarks. If you have a super small niche market, you may hit Stage 2 at $100,000 in revenue. Other businesses may achieve several million in annual revenue before they hit that first ceiling. It completely depends on the size of your market.

Instead of focusing on revenue (which are super rough estimates of what you’ll actually face), figure out what stage you’re in by analyzing the skills and problems of your business. Have you become a sales superstar? If not, you’re in Stage 1. Do you know sales better than anyone in your industry but your growth has still stalled? You’re probably in Stage 2 so make new products. Then when you start to struggle to keep up with everything, build the structure to support Stage 3. You’ll be in Stage 4 when you’ve removed yourself from the process completely.

I highly recommend Ready, Fire, Aim, you can pick it up at Amazon.com here.

Primary Sidebar

Don’t miss any of my new essays.

Most Popular Posts

  • The Three Engines of Growth – with Eric Ries
  • The 35 Headline Formulas of John Caples
  • My 7 Rules for A/B Testing That Triple Conversion Rates
  • The 9 Delusions From the Halo Effect
  • Sorry Eric Ries, There’s Only Two Engines of Growth
  • How Live Chat Tools Impact Conversions and Why I Launched a Bad Variant
  • The 17 Copywriting Axioms of Joseph Sugarman
  • Two Mistakes I Made on the Engines of Growth
  • How to Keep Riding the Slack Rocketship Without Blowing It Up
  • What is Permission Marketing?

Copyright 2019