---
name: aitp-e2e-tests
description: Write, run and read plain-English end-to-end tests with the aitp CLI (AI Testing Platform). Use when adding or changing a user-facing flow.
---
# aitp end-to-end tests

## Test spec format

File: tests/<name>.test.yaml

```yaml
name: Checkout with a saved card
url: https://app.example.com
tags: [checkout, smoke]
env:
  EMAIL: ${TEST_EMAIL}
  PASSWORD: ${TEST_PASSWORD}
steps:
  - Log in with EMAIL and PASSWORD
  - Open the cart
  - Click "Checkout"
  - Choose the saved card ending in 4242
  - Click "Place order"
  - assert: The order confirmation page shows an order number
  - api:
      get: /api/orders/latest
      expect:
        status: 200
        body.status: confirmed
```

Fields: name (required), url (required), tags (optional list), env (optional map, values from environment), steps (required list). A step is either a plain-English string, assert: <plain English>, or api: with get|post|put|delete, optional body, and expect with status and dot-path body checks. API steps are in beta.

## CLI

```
npx aitp init    # creates aitp.config.yaml and tests/
npx aitp run    # runs all tests locally in Chromium
npx aitp run tests/checkout.test.yaml
npx aitp run --tag smoke --cloud    # runs in hosted browsers
npx aitp run --cloud --region eu
npx aitp heal --review    # lists healed steps since the last run for approval
npx aitp report --last    # opens the last run report
```

Flags: --tag, --cloud, --region eu|us, --browser chromium|firefox|webkit, --headed, --review, --last.
Exit codes: 0 pass, 1 fail, 2 healed steps pending review.
Report file: .aitp/last-run.json

## Config (aitp.config.yaml)

```yaml
project: my-app
baseUrl: https://staging.example.com
browsers: [chromium]
retention: 30d
heal:
  auto: locator, layout     # what is healed without a human
  review: text, order       # what waits for review
ci:
  failOnHealedPending: true
```

## CI step

```yaml
- name: Run E2E
  run: npx aitp run --tag smoke --cloud
  env:
    AITP_TOKEN: ${{ secrets.AITP_TOKEN }}
    TEST_EMAIL: ${{ secrets.TEST_EMAIL }}
    TEST_PASSWORD: ${{ secrets.TEST_PASSWORD }}
```

## Agent setup

```
npx aitp init
npx aitp run
```
Agent reads `tests/*.test.yaml`, writes new specs in the same format, runs `npx aitp run`, and reads the report at `.aitp/last-run.json`.

## Rules

- Use only the fields, commands and flags above.
- Write the spec in the same pull request as the feature.
- Do not approve healed steps; `npx aitp heal --review` is for a human reviewer.
- On failure read .aitp/last-run.json. Fix the spec only when the classification is test-issue; report app-bug and environment to a person.

Docs: https://ai-testing-platform.com/docs/