It is well established that the Product Discovery process is a crucial stage in developing any product worth using. In our last blog post, we discussed hypothesis testing. Today, we are moving forward to the next phase: experimental testing.

Experimental testing is validating assumptions and testing new ideas or product features through experiments with real users or customers. This can be done through various methods such as usability testing, A/B testing, surveys, interviews, and prototyping. The goal of experimental testing is to gather data and insights that can inform product development decisions and improve the user experience.

Experiment-Driven Product Development — Boye & Company

Experimentation plays a vital role in improving the product discovery process. Before we continue, let us discuss some examples of products that have failed or succeeded due to proper (or improper) experimental testing.

Success and Failure Stories

Google Glass

One product that failed due to poor experimental testing was Google Glass. Remember the concept? Long before Facebook rebranded to Meta and AR and VR became buzzwords, Google was already cornering the Extended Reality market. And things looked good for them. The Google Glass was a sturdy product and was light-years ahead of its time. However, as you will often find with great but unsuccessful products, that is not necessarily a compliment.

While the concept of a wearable heads-up display was intriguing, the product ultimately failed to capture the interest of consumers. The high price point, awkward design, and privacy concerns caused the product to be pulled from the market. This failure could have been prevented if Google had done more thorough experimental testing with potential users to identify these issues before launching the product. Unfortunately, the bigwigs at the tech giant were convinced that rolling it out earlier would help them get feedback directly from consumers. Hence, according to their assumptions, they would be able to improve on the next release of the product. But this experimental testing should have been done before the product was released, not after.

Amazon Echo

On the other hand, a product that succeeded due to good experimental testing is the Amazon Echo. The Echo was not the first voice-activated smart speaker on the market, but it quickly became the most popular due to Amazon’s focus on experimentation. Amazon continuously tested and iterated on the product, adding new features and improving the user experience. By listening to their customers and making changes based on their feedback, Amazon created a product that people love and use every day. The Echo has since gone on to revolutionize the smart speaker market. And would you belive it? One of the most cited reasons for this dominance is the superior end-user experience.

Overall, good experimental testing is essential for product success. It helps identify potential issues early on, saves time and resources, and allows continuous improvement.

What is the right framework for Experimental Testing?

Marty Cagan, a well-known Silicon Valley veteran who has worked with companies like eBay, Netscape, and HP, is one of the loudest voices emphasising the importance of experimentation in product development. In his book “Inspired: How to Create Tech Products Customers Love,” he proposes a framework for product discovery that includes a cycle of experimentation. The cycle includes ideation, prototyping, testing, and learning.

• Ideation – This is the initial stage where the team generates ideas. It’s important to come up with as many ideas as possible and to be open to all possibilities. The ideation stage should involve customers, stakeholders, and team members. The goal is to gather as much input as possible to generate a wide range of ideas.
• Prototyping – Once you have a list of ideas, it’s time to create prototypes. Prototyping is the process of creating a basic version of the product to test the idea’s feasibility. Prototyping can take many forms, from sketches to wireframes to functional prototypes.
• Testing – Testing is the process of evaluating the prototype with actual customers. Testing should be conducted in several stages to ensure that the product is meeting the user’s needs. Marty Cagan emphasizes that testing should be done as early as possible in the product development process. This helps to identify any issues early on and to make any necessary changes.
• Learning – Based on the results of the testing, the team should evaluate what works and what doesn’t work. To be unbiased at this stage is very mission-critical. It is essential to understand the reason behind the success or failure of the product. This knowledge will help the team to iterate and improve the product. So do not be precious with your product and be ready to kill off any ‘darling’ features customers don’t want/need.

Wrapping up…

Experimental testing helps the product team to validate the ideas and to identify potential flaws at an early stage. This saves time and resources in the long run. Experimentation should always be a continuous process integrated into every stage of product development, and yes, it can be time-intensive. We know that. That’s the reason why here at Insight7, our philosophy is all about making it easier to draw insights, fast-tracking the product discovery process and helping stakeholders make decisions faster. That’s why we developed our software, which uses AI to help you draw insights from thousands of surveys and research data in seconds. You can try the product here.

Experimentation will supercharge your product team’s ideation process because the real world is a whole different ball game compared to the drawing board. Getting feedback on the real-world usage of your product will probably be the most important insight you gather in the course of Product Discovery. By embracing experimentation, product teams can create innovative and successful products that meet the needs of their customers in more realistic—and eventually more profitable—ways.