If you've also seen “version B is ahead” after two days, and a month later everything was back where it started, you're not alone.

Where you are now

Could the problem be something other than the design? Every time you look at the numbers mid-test and ask “who is winning?”, you give luck another chance to win.

The statistician Evan Miller simulated a test between two identical versions. When the result was checked after every visitor, and the test stopped the first time it looked “significant”, meaning not a fluke, 26.1% of the tests declared a winner. Without peeking, the chance of that mistake is 5%.

So you move the whole budget to B, and a month later everything is back where it was. There was no win. There was a lucky streak.

Axel Jedlitschka

Who's talking to you: Axel Jedlitschka

I bought a marketing course and started marketing someone else's digital product. I ran a Facebook ad for a sales page, and that is where I learned to write copy, to market, and the principles of marketing and distribution.

Later, in a job booking sales meetings by phone, I started building systems and automations, processes that run on their own, for business owners. I built a system for business forecast analysis, and it served the company's marketing and sales very well. And which design works better for you, no study can answer. The people your ads bring can.

Axel Jedlitschka

What to demand

  • The number up front

    Know before you start how many visitors each version needs, and how many days that takes with your traffic.

  • The cost up front

    Know what the test will cost, and decide whether it is worth it now, not halfway through.

  • An answer in words

    Get the verdict in plain words: real difference, no proven difference, or not enough data yet.

  • One version per person

    Know that each person saw only one version, and that every lead, everyone who left their details, is tagged with the version it came from.

  • A decision in writing

    Know what you will do with each possible result, before the test starts.

There is a way

The fix is simple, if not comfortable: decide in advance how many visitors you need, run until you reach that number, and read the result once. Your AI agent, the software that does work for you on the computer, takes you through it in four steps.

  1. Hypothesis.One sentence: if we change X, more visitors will leave their details, because Y. Plus one metric, fixed right now, for example leads divided by visitors.
  2. Two versions, one idea.A is the current design, B is the challenger. The offer, the form, the price and the ad are identical. The change has to be big, like a new layout or a new promise in the headline. A button color will not move the number enough.
  3. Sample size.The agent runs a calculator with your page's real conversion rate, meaning how many of your visitors leave their details. It gets the visitors each version needs and the weeks that takes, and all of it goes on a “test card”.
  4. Split the ads, read once.The experiment tool in Meta or Google splits the audience at random, so each person sees only one version. When both versions pass the planned number, the agent reads the result once and reports the verdict as it is.

Looking in the middle is fine. Stopping or switching because of what you saw is not.

The way, in one kit

Experiment kit · a kit by Axel Jedlitschka

Your agent tests two designs of your landing page, and tells you if the win is real or luck.

  • At the end of the install it checks itself, and restores your previous version if anything fails
  • It shows what it is going to do, and asks before writing anything
  • Works with Claude Code, and with any agent that reads an instruction file
  • A skill for Claude Code: an instruction file that teaches the agent the method, from the hypothesis to the reading
  • Sample size calculator: how many visitors each version needs, and how many days that takes
  • Result reader: real difference, no proven difference, or not enough data yet
  • Templates to build with: a server-side variant for Next.js, and two URLs for a static site or WordPress
  • A guide to a 50/50 split of your ads, in Meta and Google Ads
  • The research behind the method, with sources, and 18 checks in English, Hebrew and Spanish
  • Install in one command, which restores your previous version if anything fails
The first page of the guide: Two designs, one real answer

The comparison

Free

Experiment kit

When you read the result
Once, when both versions pass the planned number
How many visitors you need
A number fixed in advance by the calculator
What you do with the result
What the test card already says

“Version B is ahead”

When you read the result
After two days, and every time you ask “who is winning?”
How many visitors you need
Until the first time it looks “significant”
What you do with the result
Move the whole budget to B

The offer

A skill, a sample size calculator, a result reader, templates, and a 6-page guide

Price: FreeNo credit card, and no part is held back for payment

The kit is free and you get it right away. You only get marketing messages if you tick the box, and you can unsubscribe at any time.

As soon as you leave your details, a thank-you page opens with a button to download the kit. You choose the guide's language in the form: Hebrew, English or Spanish.

By Axel Jedlitschka

What you get

Promised

  • The full kit, free, as soon as you leave your details.
  • No part is held back for payment.
  • Your details are not sold, and are not passed to anyone except the infrastructure providers that run the site.
  • A question about the kit? Send a message on Instagram to @theonlyaxelj.

Not promised

  • A result. The kit gives the agent a method for testing two designs, and the result depends on how you apply it and on your business. It is general knowledge, not personal advice.
  • That the new design wins. “No proven difference” does not mean the versions are equal, and it does not mean B lost.

There is no purchase and no payment here, so there is nothing to cancel. You can withdraw your consent to marketing messages at any time.

Questions?

How do I get the kit?

You leave your details, and a thank-you page opens right away with a button to download the kit.

Is it really free?

Yes. No credit card, and no part is held back for payment.

I don't have much traffic. Is this for me?

Run the calculator the other way first: how many visitors you get a day, and how many days you are willing to wait. It tells you what improvement the test can detect at all. If only a bold change would show, you test a bold change, or you don't test. Better to know before you spend the budget.

I'm not a statistician.

You don't need to be. You decide two things: the smallest improvement you care about, and your budget. The scripts do the math and write the result in words.

Meta and Google already do A/B tests. Why do I need this?

Their tools are good at splitting the audience, and the kit uses them for that. But they declare a winner from their own count, and sometimes offer to stop early. Here the verdict comes from the leads stored on your side, and from a sample size you fixed in advance.

I'm not a programmer. Can I install it?

You unzip the file you downloaded, and from the folder it creates you run one command in the terminal, the window where you type commands to the computer: bash install.sh. The script shows what it is going to do, and asks before writing anything. The default is “no”.

Will I get spam or phone calls?

Your details are used to send you the kit and to follow up on it. Marketing messages are sent only if you ticked the consent box, and you can unsubscribe at any time. You can also ask to see, correct or delete your details.

Where can I get help?

A question about the kit: send a message on Instagram to @theonlyaxelj.

What now

You can keep moving budget to whichever version is ahead after two days, or you can decide in advance how many visitors you need, and read the result once.

A kit to install and a 6-page PDF guide: a sample size calculator, a result reader and templates, free.

Price: FreeNo credit card

The kit is free and you get it right away. You only get marketing messages if you tick the box, and you can unsubscribe at any time.

The kit is ready to download as soon as you leave your details

The kit is free and you get it right away. You only get marketing messages if you tick the box, and you can unsubscribe at any time.

Your details are used to get back to you. What is collected, why, and who receives it: see the Privacy Policy (opens in a new tab).