Idukki
A/B testing

Did your variant actually win?

A two-proportion Welch’s z-test, computed in your browser. Plug in two variants: get the lift, p-value, z-score and a clear ship/keep-running call.

Inputs

Two variants. Conversion = any goal event (purchase, add-to-cart, sign-up).

Result

Variant A conversion3.00%
Variant B conversion3.90%
Relative lift+30.00%
p-value (two-tailed)0.0136
z-score2.47
Sample needed per arm (α=0.05, power=0.8)6,458
Variant B wins with statistical significance (30.00% relative lift, p = 0.0136). You can declare a winner.

FAQ

What test does this use?+

A two-proportion z-test (Welch’s) with a normal-approximation two-tailed p-value. The same approach we ship in the Idukki dashboard.

When is a result significant?+

Conventionally when p < 0.05. We highlight that threshold in green; any larger p value gets the amber “keep running” state.

How big a sample do I need?+

The calculator outputs the per-arm sample size required to detect the observed effect at α=0.05 and 80% power, using meanP across the two arms.

Is the math the same as in Idukki?+

Yes, lib/stats.ts in the dashboard implements an identical z-test and sample-size formula.

4-min setupDTC + B2B brands37 KB runtimeReal G2 reviews

Want to ship one of these?

Talk to us, we’ll get the right tool live on your stack.

Day-1 sample data on your live IG / TikTok / Reviews, plus a setup plan tailored to Shopify, WooCommerce, BigCommerce, Wix or any HTML site.

  • No credit card
  • Cancel anytime
  • SOC 2 + GDPR

Where Idukki ships

Same data model. Every surface a shopper meets.

We use cookies

We use essential cookies to run this site and optional analytics cookies to understand how it’s used. You can change your choice anytime in our privacy policy.

A/B test significance calculator — Idukki