PlaywrightPHP

Errors and failure analysis

Classify a failed run, collect the smallest useful evidence, and change the right layer.

When a browser test fails, do not start by increasing the timeout or changing the locator. First name the failure class and collect the evidence that can prove it.

Start from the symptom

Symptom First evidence
Locator or assertion timeout screenshot and trace
Browser launch failure install logs and CI environment
Page JavaScript error console evidence and trace
Request failure request/response evidence
Missing download response evidence and artifact path
Protocol or driver error version information and transport logs

A timeout says that a condition never became true. It does not tell you whether the locator, application state, network, frame, or environment was wrong.

Use a mechanical first pass

  1. Copy the exact error class and message.
  2. Identify the failed setup step, action, or assertion.
  3. Check the screenshot or trace for the actual page and URL.
  4. Check console and network evidence only when they can explain that state.
  5. Compare local and CI context options for CI-only failures.
  6. Change the test only after the evidence points to the failing layer.

This prevents a larger timeout from hiding a product regression, failed request, missing fixture, or wrong environment variable.

Compare environments deliberately

For a CI-only failure, compare:

  • browser engine and installed browser version;
  • headless mode;
  • viewport, locale, and timezone;
  • base URL and environment variables;
  • permissions and stored state;
  • PHP and dependency versions.

Make the relevant values explicit in the context or test configuration. “Same code” does not mean “same browser environment”.

Avoid noisy diagnostics

Do not add every possible listener to every test. Collect the smallest artifact that answers the question:

  • screenshot for final visible state;
  • trace for sequence and timing;
  • console for page-side errors;
  • network evidence for backend exchanges;
  • client logs for driver or protocol problems.

Keep broad diagnostics temporary. Permanent evidence should be predictable, redacted, and easy to find in CI.

Common mistakes

  • Calling every timeout a flake.
  • Fixing the selector before checking which page loaded.
  • Increasing timeouts without opening the trace.
  • Ignoring frame boundaries or failed requests.
  • Uploading artifacts without linking them from the failed job.
  • Treating a protocol error as a product bug before verifying versions.

Go next