# AI Testing Platform

> Tests written in plain English, run in CI, healed when the UI changes, triaged when they fail.

An AI testing platform for teams that ship faster than they can write and maintain test code. Stage: early-access. Early access: prices are the planned list prices. Billing starts when the platform enters beta; early-access use is not charged.

## Capabilities

- [Tests in plain English](https://ai-testing-platform.com/product/plain-english-tests/): A test is a list of steps in plain English in a YAML file; the platform resolves each step against the live page at run time.
- [Runs in CI](https://ai-testing-platform.com/product/ci-integration/): One CLI command runs a suite in CI and reports a pass, fail or healed status per test as a check on the pull request.
- [Heals locator and layout changes](https://ai-testing-platform.com/product/self-healing/): When a step no longer resolves because an element moved, was renamed or restyled, the platform re-resolves it, marks the test as healed and records the change for review.
- [Triages failures](https://ai-testing-platform.com/product/failure-triage/): A failed test gets a triage card with the failing step, the page state, the diff from the last passing run and a classification: app bug, test issue, environment.
- [Learns your product (Beta)](https://ai-testing-platform.com/product/learns-your-product/): Builds a map of flows from session recordings, product docs and the codebase to propose tests and resolve ambiguous steps.
- [Finds coverage gaps (Beta)](https://ai-testing-platform.com/product/coverage-gaps/): Compares the flow map with the suite and lists flows without a test, ranked by how often the flow changed in the codebase.
- [Parallel hosted runs](https://ai-testing-platform.com/product/parallel-runs/): Runs tests in parallel on hosted browsers in the selected region with traces kept for the plan's retention window.
- [API steps (Beta)](https://ai-testing-platform.com/product/api-tests/): A step can call an HTTP endpoint and assert on the response, mixed with UI steps in the same test.

Planned items are listed only at https://ai-testing-platform.com/roadmap/.

## Integrations

- [GitHub Actions](https://ai-testing-platform.com/integrations/github-actions/)
- [GitLab CI](https://ai-testing-platform.com/integrations/gitlab-ci/)
- [CircleCI](https://ai-testing-platform.com/integrations/circleci/)
- [Jenkins (Beta)](https://ai-testing-platform.com/integrations/jenkins/)
- [Playwright export](https://ai-testing-platform.com/integrations/playwright/)
- [Jira (Beta)](https://ai-testing-platform.com/integrations/jira/)
- [Linear (Beta)](https://ai-testing-platform.com/integrations/linear/)
- [Slack](https://ai-testing-platform.com/integrations/slack/)
- [Vercel preview deployments (Beta)](https://ai-testing-platform.com/integrations/vercel-previews/)

## Test spec format

File: tests/<name>.test.yaml

```yaml
name: Checkout with a saved card
url: https://app.example.com
tags: [checkout, smoke]
env:
  EMAIL: ${TEST_EMAIL}
  PASSWORD: ${TEST_PASSWORD}
steps:
  - Log in with EMAIL and PASSWORD
  - Open the cart
  - Click "Checkout"
  - Choose the saved card ending in 4242
  - Click "Place order"
  - assert: The order confirmation page shows an order number
  - api:
      get: /api/orders/latest
      expect:
        status: 200
        body.status: confirmed
```

Fields: name (required), url (required), tags (optional list), env (optional map, values from environment), steps (required list). A step is either a plain-English string, assert: <plain English>, or api: with get|post|put|delete, optional body, and expect with status and dot-path body checks. API steps are in beta.

## CLI

```
npx aitp init    # creates aitp.config.yaml and tests/
npx aitp run    # runs all tests locally in Chromium
npx aitp run tests/checkout.test.yaml
npx aitp run --tag smoke --cloud    # runs in hosted browsers
npx aitp run --cloud --region eu
npx aitp heal --review    # lists healed steps since the last run for approval
npx aitp report --last    # opens the last run report
```

Flags: --tag, --cloud, --region eu|us, --browser chromium|firefox|webkit, --headed, --review, --last.
Exit codes: 0 pass, 1 fail, 2 healed steps pending review.
Report file: .aitp/last-run.json

## Config (aitp.config.yaml)

```yaml
project: my-app
baseUrl: https://staging.example.com
browsers: [chromium]
retention: 30d
heal:
  auto: locator, layout     # what is healed without a human
  review: text, order       # what waits for review
ci:
  failOnHealedPending: true
```

## CI step

```yaml
- name: Run E2E
  run: npx aitp run --tag smoke --cloud
  env:
    AITP_TOKEN: ${{ secrets.AITP_TOKEN }}
    TEST_EMAIL: ${{ secrets.TEST_EMAIL }}
    TEST_PASSWORD: ${{ secrets.TEST_PASSWORD }}
```

## Agent setup

```
npx aitp init
npx aitp run
```
Agent reads `tests/*.test.yaml`, writes new specs in the same format, runs `npx aitp run`, and reads the report at `.aitp/last-run.json`.

## Pricing

- Free: $0 per month. Solo developers and evaluation. Includes 3 tests, 200 cloud runs per month, local runs unlimited, 7-day trace retention, community support. Limits: 1 project, 1 user, chromium only.
- Team: $390 per month. One product team. Includes 100 tests, 5,000 cloud runs per month, 3 browsers, healing with review, triage, 30-day retention, 10 users. Limits: 3 projects, 4 parallel runs.
- Scale: $1,290 per month. Several teams, higher run volume. Includes 500 tests, 25,000 cloud runs per month, 12 parallel runs, learns-your-product and coverage gaps (beta), 90-day retention, SSO, 40 users. Limits: 10 projects.
- Enterprise: quoted. Self-hosted runners, custom retention, procurement requirements. Includes everything in Scale, self-hosted runners, custom region, DPA, security review, dedicated contact. Limits: by contract.

Early access: prices are the planned list prices. Billing starts when the platform enters beta; early-access use is not charged.

## What the platform cannot do

The platform runs the tests that were written and reports what passed, failed or was healed. It does not prove that an application has no bugs. Healing applies to locator and layout changes; a changed behaviour fails the test and waits for a human decision. The platform does not replace exploratory testing, accessibility review or security testing.

## Links

- Docs: https://ai-testing-platform.com/docs/
- How it works: https://ai-testing-platform.com/how-it-works/
- Setup for coding agents: https://ai-testing-platform.com/for-agents/
- Pricing: https://ai-testing-platform.com/pricing/
- Security: https://ai-testing-platform.com/security/
- Full text for models: https://ai-testing-platform.com/llms-full.txt
- Contact: hello@ai-testing-platform.com

## Capabilities in detail

### Tests in plain English

A test is a list of steps in plain English in a YAML file; the platform resolves each step against the live page at run time.

Does not: Does not accept free-form prose without step structure; does not generate assertions that were not written.

### Runs in CI

One CLI command runs a suite in CI and reports a pass, fail or healed status per test as a check on the pull request.

Does not: Does not block merges by itself; the branch protection rule is the team's decision.

### Heals locator and layout changes

When a step no longer resolves because an element moved, was renamed or restyled, the platform re-resolves it, marks the test as healed and records the change for review.

Does not: Does not heal a behaviour change; a changed outcome fails the test and goes to triage.

### Triages failures

A failed test gets a triage card with the failing step, the page state, the diff from the last passing run and a classification: app bug, test issue, environment.

Does not: Does not file tickets without an integration and a human click; classification is a proposal.

### Learns your product (Beta)

Builds a map of flows from session recordings, product docs and the codebase to propose tests and resolve ambiguous steps.

Does not: Does not read production user data; sources are chosen and connected by the team.

### Finds coverage gaps (Beta)

Compares the flow map with the suite and lists flows without a test, ranked by how often the flow changed in the codebase.

Does not: Does not measure code coverage.

### Parallel hosted runs

Runs tests in parallel on hosted browsers in the selected region with traces kept for the plan's retention window.

Does not: Does not run on the customer's own infrastructure unless the plan includes self-hosted runners.

### API steps (Beta)

A step can call an HTTP endpoint and assert on the response, mixed with UI steps in the same test.

Does not: Does not replace load testing.

# Docs: /docs/getting-started/


## Install

```
npx aitp init
```

This creates `aitp.config.yaml` and a `tests/` folder with one example spec.

## Write a test

Create `tests/login.test.yaml`:

```yaml
name: Log in with email
url: https://staging.example.com
steps:
  - Click "Log in"
  - Type EMAIL into the email field
  - Type PASSWORD into the password field
  - Click "Continue"
  - assert: The dashboard shows the account name
```

## Run it

```
npx aitp run tests/login.test.yaml
```

Exit code 0 means pass, 1 fail, 2 healed steps waiting for review. The report is at `.aitp/last-run.json`.

## Run in CI

```yaml
- name: Run E2E
  run: npx aitp run --tag smoke --cloud
  env:
    AITP_TOKEN: ${{ secrets.AITP_TOKEN }}
```


# Docs: /docs/ci/


A CI job needs one command and the `AITP_TOKEN` secret. The same command runs in any CI system that can run `npx`.

```yaml file=.github/workflows/e2e.yml
- name: Run E2E
  run: npx aitp run --tag smoke --cloud
  env:
    AITP_TOKEN: ${{ secrets.AITP_TOKEN }}
    TEST_EMAIL: ${{ secrets.TEST_EMAIL }}
    TEST_PASSWORD: ${{ secrets.TEST_PASSWORD }}
```

The example is a GitHub Actions step. In GitLab CI, CircleCI or Jenkins (Beta), run the same command in a job and pass the same variables from that system's secret store.

## Result on the pull request

The run reports a pass, fail or healed status per test as a check on the pull request. The check does not block merges by itself. Whether a failing check blocks a merge is the branch protection rule your team sets.

## Exit codes

`0` pass, `1` fail, `2` healed steps pending review. See [Exit codes](/docs/cli/exit-codes/).

## Integration pages

- [GitHub Actions](/integrations/github-actions/)
- [GitLab CI](/integrations/gitlab-ci/)
- [CircleCI](/integrations/circleci/)
- [Jenkins](/integrations/jenkins/) (Beta)


# Docs: /docs/cli/exit-codes/


| Exit code | Meaning |
|---|---|
| `0` | All tests passed. |
| `1` | At least one test failed. |
| `2` | Healed steps are waiting for review. |

CI systems treat any non-zero code as a failed step. With `ci.failOnHealedPending: true` in `aitp.config.yaml`, a run with healed steps waiting for review stops the pipeline until someone reviews them with `npx aitp heal --review`.

## Report file

Each run writes its report to `.aitp/last-run.json`. Open it with `npx aitp report --last`. Coding agents read this file to see what passed, failed or was healed; see [For coding agents](/for-agents/).


# Docs: /docs/cli/flags/


| Flag | Values | Used with | Effect |
|---|---|---|---|
| `--tag` | a tag name | `run` | Runs only tests with this tag in `tags`. |
| `--cloud` | none | `run` | Runs in hosted browsers instead of locally. |
| `--region` | `eu`, `us` | `run --cloud` | Region for the cloud run. |
| `--browser` | `chromium`, `firefox`, `webkit` | `run` | Browser to run in. |
| `--headed` | none | `run` | Shows the browser window during a local run. |
| `--review` | none | `heal` | Lists healed steps since the last run for approval. |
| `--last` | none | `report` | Opens the last run report. |

## Examples

```sh
npx aitp run --tag smoke --cloud
npx aitp run --cloud --region eu
npx aitp heal --review
npx aitp report --last
```


# Docs: /docs/cli/


The CLI is run with `npx aitp`. It runs tests locally in Chromium by default, or in hosted browsers with `--cloud`.

## Commands

| Command | What it does |
|---|---|
| `npx aitp init` | Creates `aitp.config.yaml` and `tests/`. |
| `npx aitp run` | Runs all tests locally in Chromium. |
| `npx aitp run tests/checkout.test.yaml` | Runs one spec file. |
| `npx aitp run --tag smoke --cloud` | Runs tests tagged `smoke` in hosted browsers. |
| `npx aitp run --cloud --region eu` | Runs in hosted browsers in the EU region. |
| `npx aitp heal --review` | Lists healed steps since the last run for approval. |
| `npx aitp report --last` | Opens the last run report. |

## In this section

- [Flags](/docs/cli/flags/)
- [Exit codes and report file](/docs/cli/exit-codes/)


# Docs: /docs/healing/


Healing means re-finding a step's element after the page changed. When a step no longer resolves because an element moved, was renamed or restyled, the platform re-resolves it, marks the test as healed and records the change for review.

## What is healed, what waits

`aitp.config.yaml` sets the split:

```yaml file=aitp.config.yaml
heal:
  auto: locator, layout     # what is healed without a human
  review: text, order       # what waits for review
ci:
  failOnHealedPending: true
```

| Setting | Meaning |
|---|---|
| `heal.auto` | Change types healed without a human. |
| `heal.review` | Change types that wait for a person to approve. |
| `ci.failOnHealedPending` | Exit code 2 while healed steps wait for review. |

## What is never healed

A behaviour change is not healed. If the page does something different, the test fails and goes to [triage](/docs/triage/).

## Reviewing healed steps

```sh
npx aitp heal --review
```

This lists healed steps since the last run for approval.


# Docs: /docs/spec/api-steps/


API steps are in beta. A step can call an HTTP endpoint and check the response, in the same test as UI steps.

```yaml file=tests/checkout.test.yaml
steps:
  - Click "Place order"
  - assert: The order confirmation page shows an order number
  - api:
      get: /api/orders/latest
      expect:
        status: 200
        body.status: confirmed
```

## Fields of an `api:` step

| Field | Meaning |
|---|---|
| `get`, `post`, `put`, `delete` | The method, with the path as its value. One per step. |
| `body` | Optional request body. |
| `expect.status` | Expected HTTP status code. |
| `expect.<dot.path>` | Expected value at a path in the response body, for example `body.status`. |

## Limits

API steps check responses. They do not replace load testing.


# Docs: /docs/spec/config/


`npx aitp init` creates `aitp.config.yaml` in the project root.

```yaml file=aitp.config.yaml
project: my-app
baseUrl: https://staging.example.com
browsers: [chromium]
retention: 30d
heal:
  auto: locator, layout     # what is healed without a human
  review: text, order       # what waits for review
ci:
  failOnHealedPending: true
```

| Field | Meaning |
|---|---|
| `project` | Project name. |
| `baseUrl` | Base URL of the application under test. |
| `browsers` | Browsers to run: `chromium`, `firefox`, `webkit`. |
| `retention` | How long traces are kept, within the plan's retention window. |
| `heal.auto` | Change types healed without a human. |
| `heal.review` | Change types that wait for review. |
| `ci.failOnHealedPending` | When `true`, healed steps waiting for review make the run exit with code 2. |

See [Healing rules](/docs/healing/) for how `heal.auto` and `heal.review` work together.


# Docs: /docs/spec/env/


`env` maps a name used in steps to an environment variable. Secrets stay out of the spec file.

```yaml file=tests/checkout.test.yaml
env:
  EMAIL: ${TEST_EMAIL}
  PASSWORD: ${TEST_PASSWORD}
steps:
  - Log in with EMAIL and PASSWORD
```

In CI, set the variables from the CI system's secret store:

```yaml file=.github/workflows/e2e.yml
- name: Run E2E
  run: npx aitp run --tag smoke --cloud
  env:
    AITP_TOKEN: ${{ secrets.AITP_TOKEN }}
    TEST_EMAIL: ${{ secrets.TEST_EMAIL }}
    TEST_PASSWORD: ${{ secrets.TEST_PASSWORD }}
```

Test credentials sent to the cloud are stored encrypted and never shown in reports.


# Docs: /docs/spec/


A test is one YAML file in `tests/`, named `<name>.test.yaml`. It holds a name, a start URL and a list of steps written in plain English. The platform resolves each step against the live page at run time.

## Full example

```yaml file=tests/checkout.test.yaml
name: Checkout with a saved card
url: https://app.example.com
tags: [checkout, smoke]
env:
  EMAIL: ${TEST_EMAIL}
  PASSWORD: ${TEST_PASSWORD}
steps:
  - Log in with EMAIL and PASSWORD
  - Open the cart
  - Click "Checkout"
  - Choose the saved card ending in 4242
  - Click "Place order"
  - assert: The order confirmation page shows an order number
  - api:
      get: /api/orders/latest
      expect:
        status: 200
        body.status: confirmed
```

## Fields

| Field | Required | Type | Meaning |
|---|---|---|---|
| `name` | yes | string | The test name shown in run output, reports and pull request checks. |
| `url` | yes | string | The page the test starts on. |
| `tags` | no | list | Labels used to select a group of tests with `--tag`. |
| `env` | no | map | Named values taken from environment variables, used inside steps by name. |
| `steps` | yes | list | The steps, in order. See [Steps](/docs/spec/steps/). |

## In this section

- [Steps](/docs/spec/steps/): plain-English steps and `assert:` steps
- [API steps](/docs/spec/api-steps/): `api:` steps (Beta)
- [Environment values](/docs/spec/env/): `env` and secrets
- [Config file](/docs/spec/config/): `aitp.config.yaml`


# Docs: /docs/spec/steps/


`steps` is a list. Each item is one of three kinds:

| Kind | Form | What happens |
|---|---|---|
| Action | a plain-English string | The platform finds the element or action the sentence describes on the live page and performs it. |
| Assertion | `assert: <plain English>` | The platform checks that the sentence is true for the current page. A false assertion fails the test. |
| API call | `api:` block | See [API steps](/docs/spec/api-steps/) (Beta). |

## Actions

```yaml file=tests/login.test.yaml
name: Log in with email
url: https://staging.example.com
steps:
  - Click "Log in"
  - Type EMAIL into the email field
  - Type PASSWORD into the password field
  - Click "Continue"
  - assert: The dashboard shows the account name
```

Write one action per step. Quote visible text exactly when it identifies the element, as in `Click "Continue"`.

## Assertions

An assertion states what should be true. The platform does not add assertions you did not write. A test with no `assert:` step passes when every action completes.

## What steps are not

A spec needs step structure. Free-form prose without a list of steps is not accepted.


# Docs: /docs/triage/


Every failed test gets a triage card. The card holds:

| Item | Content |
|---|---|
| Failing step | The step that did not complete or whose assertion was false. |
| Screenshot | The page at the failing step. |
| DOM snapshot | The page structure at the failing step. |
| Network log | Requests and responses around the failure. |
| Diff | Differences from the last passing run. |
| Classification | `app-bug`, `test-issue` or `environment`. |
| Suggested fix | Present when the classification is `test-issue`. |

## Classification

| Value | Meaning |
|---|---|
| `app-bug` | The application behaved differently from the spec. |
| `test-issue` | The spec itself needs a change; a suggested fix is attached. |
| `environment` | The run was affected by the environment, for example the network or the test data. |

The classification is a proposal. A person confirms it. Tickets are not filed without an integration and a human click.

## Opening the report

```sh
npx aitp report --last
```