In the fast-paced world of digital products, relying on gut feelings just doesn't cut it anymore. That's why successful companies are embracing a data-driven approach, starting with a solid hypothesis.

Relying on hypotheses in our work is becoming more common in digital spaces. Marketers are rebranding as growth hackers, managers are organizing work according to the Deming cycle, and programmers and designers are moving away from rigid specifications. Instead, they are adopting a data-driven approach, and startups are beginning their journey by developing an MVP.

These methodologies and frameworks are based on the scientific approach: "formulate a hypothesis — conduct an experiment — analyze the results." This is not surprising, as product creation, like research, involves a high degree of uncertainty. This approach turns project work into a continuous learning process that begins with formulating a hypothesis.

Hypothesis is a supposition that lacks sufficient evidence and hasn't been disproved but is considered probable. However, in product development, not every assumption qualifies as a hypothesis.

To be workable, a hypothesis must include the following components:

This component describes what change in the product you want to test. Is it a new design, a new feature?

The key is not to test multiple changes at once. Follow the rule: "1 idea = 1 hypothesis." Otherwise, it will be impossible to determine which specific change affected the metrics. Even a single idea can be too broad and complex. Try to narrow it down. Describe the user flow where the change will occur and ensure the change affects only one step in one user scenario.

For example:
"A user clicks a button to buy a product. In the test, we will change the button color from yellow to green. We expect this to increase the conversion rate."

A poor example:
"A user must click a heart icon in the top right corner and go to the wishlist to make a purchase."

The second example completely changes the user scenario. It's unclear what specifically will affect the conversion rate since multiple changes are made.

To sum up, a hypothesis should describe what is being changed.
One hypothesis = one change.

Let's practice. Identify which of the three options can be used as one component of a hypothesis:

1. Redesign the entire homepage.
2. Change the target audience in advertising.
3. Convert a dating app into a job search service.

The first component describes the change; the second describes its impact. The expected impact should be specific and measurable:

• What will the change affect? Conversion rates, response times, audience growth?
• How will we know the change was successful?

For example, if we decide to test a new form for collecting applications, we might assume it will improve usability. "Usability" is too broad, so it needs to be specified. What metrics characterize usability? Form completion time, bounce rate, etc. These are the metrics to include in the hypothesis. Each impact requires a separate hypothesis.

Thus, the start of hypotheses should sound like this: "If we change the application form, the completion time will decrease..." or "If we change the application form, the bounce rate will decrease..."

The third component of a well-constructed hypothesis is the audience, explicitly stating who will be affected by the change.

Often, this is overlooked, or it is assumed that the change will affect all users. This assumption is incorrect because the results won’t reflect the real picture.

Let’s consider an example.
To increase the number of email newsletter subscribers, we decide to show a pop-up offering a subscription to users leaving the blog. If 50 people see the banner and 1 subscribes, that’s a 2% conversion rate. But the problem is, the blog has regular readers who are already subscribed. Suppose 45 people are already subscribers and therefore say "no." So, 1 person out of 5 unsubscribed users agrees to subscribe. The actual subscription conversion rate is not 2% but 20%.

As you can see, the difference in test results with and without considering the audience is significant. Therefore, defining the audience for your tests is crucial for obtaining accurate results.

The fourth component describes the expected outcome. Specify in your hypothesis the expected impact of the change.

For example, if you anticipate that the change will increase conversion, estimate by how much. The hypothesis should state: "The change will increase conversion from x% to y%," where x is the current conversion rate and y is the expected result after the change.

Making this assumption serves two purposes:

1. Evaluating Success: The expected value helps determine whether the result is successful. For instance, if you expect an increase in conversion from 2% to 10% but achieve 9%, without a clear benchmark, you might still interpret 9% as a good result. Similarly, 8% could be seen as good, and anything above 2% could be considered positive.
2. Determining Experiment Duration: The expected result helps define the length of the experiment. Achieving a 10% increase in a week versus a month represents different outcomes. We'll discuss setting the experiment duration in the next section.

Another common mistake in hypothesis testing is setting the experiment duration arbitrarily and stopping once the desired metrics are achieved. But can you trust this result? To rely on the result, it must be statistically significant.

Statistical significance is a mathematical way to prove that the relationship between changes and results truly exists. Until the statistical significance reaches 95-99%, you should not stop testing the hypothesis.

In the previous sections, we discussed the five components of a good hypothesis.
Now, let’s put them together. The template will be structured as follows:
"What we are changing + what it will affect + for which audience + by how much + over what period"

Example:
"Changing design X (change) will increase the landing page conversion (impact) from traffic from contextual ads (for whom) by 10% (how much) over 7 days (timeframe)."

Now try to formulate your hypothesis. Then, break it down into the five components for verification.

For instance:
"A new email subject line will increase the open rate by 15% for daily digest subscribers within 3 days."

Breakdown:

Change: New email subject line
Impact: Increase open rate
Audience: Daily digest subscribers
How much: By 15%
Timeframe: Within 3 days

By doing so, you have correctly formulated a hypothesis.

This approach to hypothesis formulation allows you to set up the necessary experiment and accurately interpret the data obtained. As a result, you gain new insights and can make your product even better.