How to know if an experiment has been validated

You have already defined your entrepreneurship project. You have designed your Experiment and put it into action. You’ve collected the signals and interpreted them. And now it’s time to find out whether that experiment, as in the scientific method, has measured the right variables and, therefore, the hypothesis has been validated. To find out, we’ll show you what to look out for.

Strength of the evidence Depending on the type of experiment you’ve carried out, you can divide the evidence into two main groups:

1. Strong: what people do. If your experiment has consisted of observing people in real environments, talking about their pains or frustrations (that is, if you’ve researched in specialized forums, reviews, social media, etc.), or if in the interviews you asked about facts, rather than opinions (when was the last time that…? How did you feel in the moment when…?), you’ll have much more reliable data on behavior of your potential users. The closer to reality the scenario is, greater strength will have the proof.

2. Weak: what people say they do. Interviews are a good source of information, but they force you to trust the honesty of the responses; keep in mind that people behave differently if they know they’re being subjected to a test. Also, if you asked about opinions (What do you think about…? Would you like an app that…? Would you use this product?), the reliability of the responses is much weaker.
   
   Another way to determine the strength of the evidence you’ve obtained with your experiment is to observe the type of “investment” made by the people involved in the test. Suppose you design a landing page to test a future product or service:

1. If users give you their email in order to be notified of the launch, the signal is good, but weak. We are talking about evidence of engagement.

2. If they leave a deposit (that is, they pay money upfront) or perform a order to book what you offer, the test is strong. We are talking about payment evidence.

How many tests do you have in your experiment? Don’t get discouraged if your signals aren’t robust; weak evidence is better than none. Even so, the ideal is to gather a good number of tests, since that will increase your confidence in the results of the experiment.

Suppose that 60% of the people you’ve interviewed say they identify with the pain you’ve found. It’s an encouraging figure, right? You’re almost certain that your hypothesis has been validated… But that depends on the number of potential users you’ve spoken to: if it’s been five, the solidity of this data is very different than if it’s been 50.

The same can be said if you’ve received orders: in this case, the type of evidence is stronger (you have a real action, an intention to buy), but 50 potential customers will be better than five.

The higher the number of tests, the more you can trust the result of the experiment.

If there’s competition, there’s validation Often you’ll discover that your idea already exists. And that might scare you. Instead of worrying about your competitors, take it as a good sign, since:

• Prove that there’s demand for what you’re proposing.

• It gives you the opportunity to improve the product or service: investigate whether you can do the same with a lower price or a simpler process. Keep in mind that many startups are born by optimizing something that already exists on the market.

• It allows you to stand out in execution: perhaps your way of “selling” (marketing) or of making deliveries sets you apart from your competitors. Myths about validation in entrepreneurship

The narrative of lone heroes with a project that makes them millionaires is (you’ll have noticed) very appealing… and completely exceptional. Despite being rare cases, they have influenced the experiment validation with several “myths.” Are they all true?

• If your project is small, you don’t need to validate it. False. Any entrepreneurship needs to know whether it will have customers or users. Otherwise, you will waste time (and even money) building something the market doesn’t want.

• Validation will give you 100% certainty. False. No experiment completely ensures that an idea will take off. But a 60-70% positive response rate are enough to encourage you to take the next step.

• Validation is a journey, not a destination. True. Your idea will change with what you learn from experimenting and from the feedback of your potential customers. Enjoy the journey!

Sources:
https://stackingthebricks.com/cant-find-audience/
https://www.strategyzer.com/library/business-testing-is-your-hypothesis-really-validated
https://medium.com/@isabellapennington/startup-idea-validation-basics-how-i-test-my-next-big-idea-before-i-build-d706d3f3397b
https://scienceready.com.au/pages/validity-accuracy-and-reliability

Image generated with AI.