End-to-End Tests with Playwright, and Fixing Flaky Tests

Lesson 2 tested components in a fake DOM and Lesson 3 tested an API with a real database. This lesson uses a real browser. We write a tiny todo app, test it with Playwright, then write four truly flaky tests and fix each one. For test-plan questions (how many e2e tests, which flows) see Frontend Depth, Lesson 5 and Team Lead, Lesson 6; for a tool reference see Full-Stack Lesson 21. The last lesson, Lesson 5, covers TDD and test strategy.

Words used in this lesson

• End-to-end (e2e) test: a test that uses the app the way a user does, through a real browser, with the real front end, back end and data.
• Playwright: a library from Microsoft that controls Chromium, Firefox and WebKit. @playwright/test is its test runner.
• Locator: a description of how to find an element. It is looked up again every time you use it.
• Role locator: finds an element by its accessibility role and name (button named "Add"), the way assistive tools and users see the page.
• Auto-waiting: before an action such as click, Playwright waits until the element is ready.
• Web-first assertion: an expect(locator)... check that retries until it is true or times out.
• Flaky test: a test that passes and fails on the same code.
• Test isolation: each test starts with its own clean browser state and its own data.
• Storage state: a saved file of cookies and local storage, so tests start already logged in.
• Trace: a recording of a test run (steps, DOM snapshots, network calls, console) that you open in the trace viewer.
• Worker: a process that runs test files. Several workers run in parallel.

1. When e2e is worth it

A browser test costs 1 to 10 seconds, needs a running app, and breaks for many reasons. So it should earn its place.

A good rule: one e2e test per business-critical journey, not per feature. Put the many small cases lower, where they are fast (see Lesson 1 and Lesson 3). Five to twenty e2e tests are common for a mid-size product.

2. The app and the install

I wrote server.js: a login page, then a todo page with an input, an "Add" button, a list, a "Saved" toast that fades in, and a morning/afternoon greeting. Its POST endpoint is deliberately slow and uneven (100 to 500 ms), like a real backend. Data is kept per user in memory.

npm init -ynpm i -D @playwright/test@1.56.1npx playwright --version

Version 1.56.1

The config starts the app for you (webServer) and sets a base URL:

// playwright.config.jsimport { defineConfig, devices } from '@playwright/test';export default defineConfig({  testDir: './tests',  fullyParallel: true,  retries: 0,  reporter: [['list']],  use: {    baseURL: 'http://127.0.0.1:4173',    trace: 'retain-on-failure',  },  projects: [{ name: 'chromium', use: { ...devices['Desktop Chrome'] } }],  webServer: { command: 'node server.js', url: 'http://127.0.0.1:4173', reuseExistingServer: false },});

retries: 0 is on purpose while you learn. Retries hide flakiness; they do not fix it.

3. A first good test

// tests/good.spec.jsimport { test, expect } from '@playwright/test';async function signIn(page, user) {  await page.goto('/');  await page.getByLabel('Username').fill(user);  await page.getByLabel('Password').fill('pw');  await page.getByRole('button', { name: 'Sign in' }).click();  await expect(page.getByRole('heading', { name: 'Todos' })).toBeVisible();}test('wrong password shows an error', async ({ page }) => {  await page.goto('/');  await page.getByLabel('Username').fill('nobody');  await page.getByLabel('Password').fill('nope');  await page.getByRole('button', { name: 'Sign in' }).click();  await expect(page.getByRole('alert')).toHaveText('Wrong password');});test('add and delete a todo', async ({ page }, testInfo) => {  await signIn(page, `u-${testInfo.testId}`);  await page.getByLabel('New todo').fill('Buy milk');  await page.getByRole('button', { name: 'Add' }).click();  const items = page.getByRole('list', { name: 'Todo list' }).getByRole('listitem');  await expect(items).toHaveCount(1);                 // retries until true (or 5 s)  await expect(items.first()).toContainText('Buy milk');  await page.getByRole('button', { name: 'Delete Buy milk' }).click();  await expect(items).toHaveCount(0);});

Running 2 tests using 2 workers  ✓  1 [chromium] › tests/good.spec.js:12:1 › wrong password shows an error (666ms)  ✓  2 [chromium] › tests/good.spec.js:20:1 › add and delete a todo (1.2s)  2 passed (3.7s)

Locators by role

Prefer, in this order: role (getByRole), label (getByLabel), visible text (getByText), and a test id (getByTestId) only when nothing user-facing works. The Playwright best-practices page recommends locators that look like how users and assistive technology see the page, and warns against long CSS or XPath chains that break when markup changes.

Why role locators are good: they survive a redesign (div to button), and they quietly check accessibility. If getByRole('button', { name: 'Add' }) cannot find your button, a screen-reader user probably cannot either.

Strict mode. A locator that matches more than one element is an error when you act on it. This is a safety feature. After adding one todo I ran page.getByRole('button').click() on purpose:

Error: locator.click: Error: strict mode violation: getByRole('button') resolved to 2 elements:    1) <button type="submit">Add</button> aka getByRole('button', { name: 'Add' })    2) <button aria-label="Delete a">Delete</button> aka getByRole('button', { name: 'Delete a' })

The message even suggests the better locators. Fix it by being specific ({ name: 'Add' }), or by narrowing with .filter() or by scoping inside another locator.

Auto-waiting

Before click, Playwright waits until the element is: exactly one, visible, stable (not animating), receiving events (not covered by something else) and enabled. fill waits for visible, enabled and editable (this list is from actionability.md). So you almost never write "wait until the button appears". If the check does not pass within the timeout, the action fails with a clear message.

Web-first assertions

// BAD: reads once, right nowexpect(await page.locator('#list li').count()).toBe(1);// GOOD: retries until it is true, or fails after the timeout (5 s by default for expect)await expect(page.locator('#list li')).toHaveCount(1);

The first line asks "what is the count now?" and compares. The second says "the count must become 1". Whenever the thing you check can change after a click (anything that comes from a request), use the second form: toBeVisible, toHaveText, toHaveCount, toHaveURL, toHaveCSS, toBeEnabled. Forgetting await before expect(locator) is a bug: the check never runs, and the test passes for no reason.

4. Isolation, and storage state for login

Each Playwright test gets a new browser context: new cookies, new local storage, like a fresh incognito window. So tests do not share browser state by default. They can still share server state: the database, the files, the user's account. That is where most hidden coupling lives (section 7).

Logging in through the form in every test is slow. Log in once, save the cookies, and reuse them. I used a "setup" project:

// tests-auth/auth.setup.jsimport { test as setup, expect } from '@playwright/test';setup('log in once and save the cookies', async ({ page }) => {  await page.goto('/');  await page.getByLabel('Username').fill('reader');  await page.getByLabel('Password').fill('pw');  await page.getByRole('button', { name: 'Sign in' }).click();  await expect(page.getByText('Signed in as')).toContainText('reader');  await page.context().storageState({ path: '.auth/reader.json' });});

// playwright.auth.config.js (projects only)projects: [  { name: 'setup', testMatch: /auth\.setup\.js/ },  { name: 'chromium', dependencies: ['setup'], testIgnore: /auth\.setup\.js/,    use: { ...devices['Desktop Chrome'], storageState: '.auth/reader.json' } },],

The saved file is just JSON (I printed it): one cookie named user with value reader for 127.0.0.1, httpOnly: true. A test can then skip the login:

test('starts already logged in (storage state)', async ({ page }) => {  await page.goto('/');                                   // no login steps  await expect(page.getByText('Signed in as')).toContainText('reader');});

5. Network mocking

Sometimes you want the browser to see a fixed answer from the API: a rare error, an empty list, a slow response. page.route intercepts requests:

test('network mocking: the page shows whatever the API returns', async ({ page }) => {  await page.route('**/api/todos', (route) => {    if (route.request().method() !== 'GET') return route.fallback();    return route.fulfill({ json: [{ id: 1, text: 'From the mock' }, { id: 2, text: 'Second' }] });  });  await page.goto('/');  await expect(page.locator('#list li')).toHaveText(['From the mock Delete', 'Second Delete']);});

  ✓  1 [setup] › tests-auth/auth.setup.js:3:1 › log in once and save the cookies (359ms)  ✓  2 [chromium] › tests-auth/mock.spec.js:3:1 › starts already logged in (storage state) (236ms)  ✓  3 [chromium] › tests-auth/mock.spec.js:8:1 › network mocking: the page shows whatever the API returns (197ms)  3 passed (3.2s)

When to mock the network in e2e: for things you cannot control (a third-party payment page, an email service) and for error cases that are hard to trigger. When not to: for your own API in the main journeys. If you mock your own back end everywhere, it is not an end-to-end test any more; it is a front-end test in a real browser. That is fine, but call it that. Note the route.fallback() for the methods we do not want to fake.

6. Traces and the trace viewer

When a test fails in CI, you cannot watch it. A trace is a recording you can open later. Set it in the config:

Each failing test in my flaky run printed the file and the command:

Error Context: test-results/flaky-flaky-adds-eggs-chromium/error-context.mdtest-results/flaky-flaky-adds-eggs-chromium/trace.zip    npx playwright show-trace test-results/flaky-flaky-adds-eggs-chromium/trace.zip

I listed one trace zip: it held about 18 files (82,858 bytes in total for a tiny test), including screenshots (.jpeg), saved page HTML, and a .stacks file. The viewer shows a timeline of steps, a before/after DOM snapshot for every action, the network calls and the console. You can click a failed step and see the page exactly as the test saw it. I could not open the viewer here (no display), so I describe it from the docs and from the file contents. The important habit: look at the trace before guessing.

7. Flaky tests: the five usual causes

A flaky test is one you stop trusting. People re-run it, then ignore red builds, then real bugs get through. Almost all flakiness comes from five sources:

A real flaky file

I wrote four tests with these problems on purpose and ran them with --repeat-each=4 (each test four times):

// tests/flaky.spec.js (the tests, after the shared signIn helper)// FLAKY 1 + 2: fixed sleep, and a user shared by tests that run in paralleltest('flaky: adds milk', async ({ page }) => {  await signIn(page, 'shared');  await page.getByLabel('New todo').fill('milk');  await page.getByRole('button', { name: 'Add' }).click();  await page.waitForTimeout(200);                                  // guess  expect(await page.locator('#list li').count()).toBe(1);          // reads once, no retry});test('flaky: adds eggs', async ({ page }) => {  await signIn(page, 'shared');                                    // same data as the test above  await page.getByLabel('New todo').fill('eggs');  await page.getByRole('button', { name: 'Add' }).click();  await expect(page.locator('#list li')).toHaveCount(1);});// FLAKY 3: depends on the real clocktest('flaky: greeting', async ({ page }, testInfo) => {  await signIn(page, `g-${testInfo.testId}`);  await expect(page.locator('#greeting')).toHaveText('Good morning');});// FLAKY 4: reads a CSS value in the middle of an animationtest('flaky: toast is fully shown', async ({ page }, testInfo) => {  await signIn(page, `t-${testInfo.testId}`);  await page.getByLabel('New todo').fill('x');  await page.getByRole('button', { name: 'Add' }).click();  await page.locator('#toast.show').waitFor();  const opacity = await page.locator('#toast').evaluate((el) => getComputedStyle(el).opacity);  expect(opacity).toBe('1');});

Result of one run (16 test runs, 2 workers, this machine; the exact counts changed between my two runs, which is the point):

Running 16 tests using 2 workers  14 failed  2 passed (36.8s)

An earlier run of the same file showed a mix: adds milk passed once and failed three times, toast is fully shown passed once and failed three times. The messages explain each cause:

adds eggs      Expected: 1   Received: 2greeting       Expected: "Good morning"   Received: "Good afternoon"toast          Expected: "1"   Received: "0.999055"adds milk      Expected: 1   Received: 2

Reading them:

1. adds milk (time and network). The server answers in 100 to 500 ms. The test sleeps 200 ms and then counts once. If the answer is late, the count is 0 or stale. If it is on time, it passes. That is the pure flaky shape: pass or fail by a race. Here it also failed with Received: 2, which is cause 2.
2. adds eggs (data). Both tests sign in as shared. Workers run them at the same time, so one sees the other's todo: Expected: 1, Received: 2. Run alone, it passes.
3. greeting (time). The app says "Good morning" before 12:00. I ran this at 12:47 UTC, so it failed every time. At 09:00 it would pass every time. A test that depends on the real clock is not flaky at random; it flips at a certain hour, which is worse because nobody sees it coming.
4. toast (animation). The toast fades in over 400 ms. After it gets class show, the opacity is still moving: "0.999055". The test read it at an unlucky moment.

The fixes

// tests/fixed.spec.js// DATA: a unique user per test, so parallel tests never share rowsconst uniqueUser = (testInfo) => `u-${testInfo.testId}-${testInfo.retry}`;test('adds milk', async ({ page }, testInfo) => {  await signIn(page, uniqueUser(testInfo));  await page.getByLabel('New todo').fill('milk');  await page.getByRole('button', { name: 'Add' }).click();  // NETWORK/TIME: a web-first assertion retries until true; no sleep  await expect(page.locator('#list li')).toHaveCount(1);});test('greeting in the morning', async ({ page }, testInfo) => {  // CLOCK: pin the browser time before the page loads  await page.clock.setFixedTime(new Date('2026-03-01T09:00:00'));  await signIn(page, uniqueUser(testInfo));  await expect(page.locator('#greeting')).toHaveText('Good morning');});test('greeting in the afternoon', async ({ page }, testInfo) => {  await page.clock.setFixedTime(new Date('2026-03-01T15:00:00'));  await signIn(page, uniqueUser(testInfo));  await expect(page.locator('#greeting')).toHaveText('Good afternoon');});test('toast is fully shown', async ({ page }, testInfo) => {  await signIn(page, uniqueUser(testInfo));  await page.getByLabel('New todo').fill('x');  await page.getByRole('button', { name: 'Add' }).click();  // ANIMATION: toHaveCSS retries until the transition ends  await expect(page.locator('#toast')).toHaveCSS('opacity', '1');});

I ran the fixed file with --repeat-each=10 (50 runs, because there are five tests):

  ✓  48 [chromium] › tests/fixed.spec.js:21:1 › adds eggs (1.0s)  ✓  50 [chromium] › tests/fixed.spec.js:41:1 › toast is fully shown (1.3s)  50 passed (34.9s)

Also 5 of 5 with --workers=4. Fifty passes is evidence, not proof, but this time each cause was removed rather than hidden.

What I did not do, and you should not either:

• Raise the timeout. A longer wait hides a race, it does not remove it.
• Add retries: 2 and move on. Retries are a safety net for rare infrastructure problems. Use trace: 'on-first-retry' so a retry gives you evidence, and have a rule that a test which needed a retry gets a ticket.
• Use test.skip with no owner. A skipped test is deleted coverage.
• Use { force: true } on clicks. It switches off the actionability checks, so it can hide a real bug (a button covered by a modal).

A short method for any flaky test

1. Reproduce it: npx playwright test file.spec.js --repeat-each=20 (and --workers=1 versus more, to separate data problems from timing problems).
2. Open the trace of a failure and find the first step where the page differs from what the test expected.
3. Name the cause from the five. If you cannot, you do not yet understand the test.
4. Fix the cause, not the symptom. Then repeat step 1 to prove it.

8. Parallelism

Playwright runs test files in parallel by default, in separate worker processes. With fullyParallel: true, tests inside a file are also spread over workers. The number of workers defaults to half of your CPU cores and can be set with --workers or workers in the config. Each worker gets its own browser, and each test its own context.

Parallel is why isolation matters (section 7, "data"). Three rules:

1. Each test creates the data it needs, with unique names or ids.
2. No test depends on another test's result.
3. If a group of tests truly must share a resource, run that group in order in one file with test.describe.configure({ mode: 'serial' }). Treat this as a last resort: when one test in a serial group fails, the rest are skipped.

For CI you can also split the suite across machines with --shard=1/4. I did not run sharding.

Try it

1. Write a role-locator test for the "Delete" button using getByRole('button', { name: /^Delete/ }) with two todos. What happens? Answer: strict mode violation, because two buttons match. Narrow it with the todo text.
2. Take the adds milk flaky test and change only waitForTimeout(200) to waitForTimeout(2000). Is it fixed? Answer: it will fail less often, but the sleep still guesses, the test is now 2 seconds slower, and a slow CI machine can still beat it. Use toHaveCount.
3. Make the server delay 5 seconds and run the fixed test. What fails? Answer: toHaveCount times out after 5 s. Raise the timeout for that one assertion ({ timeout: 10_000 }) if the delay is expected, or fix the slow endpoint.
4. Run npx playwright test --repeat-each=10 --workers=1 and then --workers=4 on the fixed file. Answer: both should pass; if only the second fails, you have a data-sharing problem, not a timing one.

Interview questions

Key takeaways

• Write few e2e tests, for the journeys that matter most. Put edge cases lower in the pyramid.
• Find elements by role and label. A strict-mode error is a hint to be more specific, not a problem to silence.
• Playwright waits for actions automatically. For checks, use web-first assertions that retry. Never use fixed sleeps.
• Each test gets a fresh browser context, but the server data is yours to isolate: unique data per test.
• Log in once with storage state, and keep that file out of git.
• Record traces and read them before guessing at a cause.
• Flakiness has five usual causes: time, data, ordering, animation and network. Fix the cause, then run the test many times to prove it.
• Retries, longer timeouts, force: true and silent skips hide problems.

What's next

Lesson 5: TDD, Property-Based Testing, Mutation Testing and a Team Strategy goes back to the code: writing tests first, letting a tool invent test cases, checking whether your tests can really catch bugs, and putting it all into one plan for a team.