You need one end-to-end (E2E) test for each user flow whose failure would cost you revenue, data or trust, and few beyond that. Start with a handful. Add a test when a bug escapes in a flow you did not cover. Push every other check down to unit and integration tests, which run faster and point more precisely at the cause.
Count flows, not tests
The question "how many tests" has no useful answer as a number. A better question is "which flows". A flow is a path a user takes to get something done: sign up, invite a teammate, pay an invoice, export a report.
Write down every flow in your product. Then mark the ones where a failure would mean lost money, lost data or a user who cannot do the thing they came for. Those marked flows are your E2E suite. For many products the marked list fits on one screen.
Each marked flow gets one test that goes from start to outcome. Variations of the same flow, such as every validation message on a form, belong in lower-level tests. The E2E test proves the path works end to end. It does not need to prove every edge case.
The testing pyramid is about cost, not a fixed ratio
The testing pyramid says to have many fast, narrow tests at the bottom and fewer broad, slow tests at the top. People often quote a ratio for it. Ignore the ratio. The point of the pyramid is cost per test.
Unit tests are cheap to write, fast to run and fail close to the cause. E2E tests are slower, depend on more moving parts and fail far from the cause. So each check should live at the lowest level that can catch the problem. If a pricing rule is wrong, a unit test on the pricing function catches it with a clear message. If the checkout page cannot reach the payment service, only a test that runs the full flow will notice.
The split between E2E and integration tests is covered in more detail in E2E vs integration testing.
Every extra E2E test adds flakiness exposure
Broad tests are more likely to fail for reasons unrelated to your change: timing, network, shared test data, a slow third-party service. At Google, almost 16% of tests have some level of flakiness associated with them [Google Testing Blog], and across Google's entire corpus of tests, there is a continual rate of about 1.5% of all test runs reporting a "flaky" result [Google Testing Blog].
A low per-run rate adds up across a large suite. The more E2E tests a pull request has to pass, the more often at least one of them fails for no real reason. At Atlassian, test flakiness was responsible for as much as 21% of master build failures in the Jira Frontend repository [Atlassian]. A lean suite is easier to keep trustworthy. See what causes flaky tests for the usual sources.
Grow the suite from escaped bugs and changed flows
A suite sized by a target number tends to fill with easy tests. A suite sized by risk grows where it should. Use two triggers to add a test:
- A bug reached users in a flow that had no E2E test. Add one for that flow, and ask whether a lower-level test would also have caught it.
- A flow changes often in the codebase. Frequent change means frequent chances to break it.
And one trigger to remove a test: it covers the same path as another test and has not caught anything the other would miss.
Signs your suite is the wrong size
Too few E2E tests:
- Bugs in core flows reach production and are found by users.
- Nobody can say which critical flows are covered.
- People test the main flows by hand before each release.
Too many E2E tests:
- The suite takes long enough that people skip it or merge before it finishes.
- Red builds are rerun by habit.
- Many tests fail together after a single interface change.
If you see signs from both lists, the suite is probably covering the wrong flows. Re-map it to the list of critical flows first, then adjust size.
How AI Testing Platform fits
AI Testing Platform runs E2E tests written as plain-English steps, one file per flow, so the suite reads like the list of flows it covers. Finding coverage gaps is in beta: it compares a map of your flows with the suite and lists flows without a test, ranked by how often they changed in the codebase. See coverage gaps and the page for the quality assurance (QA) lead.