Run your end-to-end (E2E) smoke suite against the preview deployment of each pull request, after the preview reports ready, with the preview's web address (URL) passed to the test runner as configuration. Treat failures caused by the preview itself as environment problems, separate from product bugs, so the check stays trustworthy.
A preview is the closest thing to production per change
By default, Vercel creates a preview deployment when you push a commit to a branch that is not your production branch, or when you create a pull request on GitHub, GitLab, Bitbucket or Origin [Vercel]. Each change gets its own deployment.
That makes the preview a better target for E2E tests than a shared staging server. A staging server carries changes from several branches at once, so a red test there may belong to someone else's work. A preview carries one change. When a test fails against it, the question "which change broke this" already has an answer.
Wait for the deployment before the tests start
The most common mistake is a race. The pipeline starts the tests as soon as the commit is pushed, while the preview is still building, and the first steps fail against an old build or an error page.
Order the work so the tests start only after the preview is ready:
- Trigger the E2E job from the event that says the deployment finished, not from the push.
- Read the preview URL from that event or from the deployment status, not from a naming pattern you reconstruct by hand.
- Add a short readiness check that loads the home page before the suite runs. If it fails, report an environment failure and stop.
Each hosting provider and continuous integration (CI) system exposes deployment events differently. Check the documentation of the ones you use; the pattern above does not depend on a specific vendor.
Pass the preview URL as configuration
Tests should not hard-code an address. Keep the base URL in configuration and pass the preview address to the runner at run time. The same suite then runs against a local build, a preview and production without edits.
Two related points:
- Test data. A preview often talks to a shared test database or a sandbox backend. Give each run its own test account, or clean up what the run creates, so parallel previews do not collide.
- Access. If previews sit behind authentication, the test runner needs a supported way in. Store any credential it uses in the CI secret store, never in the repository.
Separate environment failures from product failures
A preview adds failure modes that have nothing to do with the change: a cold start, a missing environment variable, a backend sandbox that is down. If these show up as ordinary test failures, people learn to rerun the check instead of reading it.
Noise is already a baseline problem. At Google, about 1.5% of all test runs reported a flaky result [Google Testing Blog]. A preview pipeline should not add to that. Tag each failure with a cause: a product bug in the change, a problem in the test, or a problem in the environment. Only the first should hold up the author. Detecting flaky tests in CI covers how to track the other two over time.
Gate on the smoke suite only
Keep the preview run short. A smoke suite of the flows that must never ship broken gives the author a result while the review is still open. Run the full suite after merge.
If you want the preview run to gate merges, the host does the gating. On GitHub, once required status checks are enabled, all required checks must pass before collaborators can merge into the protected branch [GitHub]. Make the preview check required only after it has run cleanly for a while. Run E2E tests on every pull request goes through that rollout.
How AI Testing Platform fits
AI Testing Platform runs a tagged suite in hosted browsers in parallel with npx aitp run --tag smoke --cloud, and reports a pass, fail or healed status per test as a check on the pull request (CI integration). Each failed test gets a triage card with a proposed classification of app bug, test issue or environment, which keeps preview problems apart from product bugs (failure triage). The Vercel preview deployments integration is in beta. The platform does not block merges by itself; branch protection stays your decision.