Unit Tests, E2E Tests, and CI: Different Safety Nets for Different Failures
A CI pipeline exists to answer one question before a person has to: does this change actually break anything. That's the actual point of a quality gate, not process for its own sake. It's what lets a team ship a change today without spending tomorrow morning finding out what it quietly broke, and what lets anyone keep building on top of code they didn't personally write and haven't personally re-tested by hand.
Travelstart's white label site answers that question in three deliberate tiers rather than one job trying to do everything at once: lint and unit tests, a pull request scoped e2e run, and a full nightly regression sitting a level above both, run less often specifically because it exists to catch what the first two, by design, don't have time to.
three tiers, not one
Travelstart's white label site runs CI as three tiers, defined across ci.yaml and a matching e2e-nightly.yaml.
lint-and-unit is the fast tier. It checks out the repo, sets up Node pinned via .nvmrc, checks that the major version is actually what the project expects, generates a throwaway RSA keypair for the JWTs the tests need, then runs lint and the full unit suite. Every other tier waits on this one.
e2e-build and the sharded e2e job that follows it are the second tier, scoped per pull request. dorny/paths-filter checks the diff against 12 named filters, shared, smoke, booking, auth, account, bookings, wallet, security, embed, home, support, and a docs only filter that skips e2e entirely when a change is just documentation. A script resolves those results down to one named profile rather than a pile of suites, booking wins if anything booking related changed, auth wins next, and so on down a fixed priority list, falling back to a plain smoke run for anything left over. That profile decides which spec globs get built into a shard count, and a sharded matrix job downloads one build and runs Playwright with --shard=N/total against it.
e2e-nightly.yaml is the third tier, structurally identical to the first two, same lint-and-unit, same e2e-build, same sharded matrix, except the profile is hardcoded to all and readiness is set to all too, so every suite runs and nothing gets excluded for being unproven yet. Right now that tier sits paused, and the reasoning for it is written directly into the trigger block rather than left for someone to guess at later:
on: # Nightly cron paused, re-enable when full regression is stable enough to run daily. # Was: 02:00 UTC (04:00 SAST), cron: '0 2 * * *' workflow_dispatch:
That's the same instinct the rest of the pipeline is built on, applied to itself. A check that isn't trustworthy yet says so, in writing, with a condition for turning it back on, instead of running anyway and quietly training everyone to ignore it.
what actually decides which suite runs
The path filters don't map directly onto what runs. A small script resolves them down to exactly one profile, and which rule applies depends on how the workflow got triggered in the first place:
if (eventName === 'pull_request') { const profile = resolveFromPaths(env) return { profile, reason: 'path-based pull_request' } } if (eventName === 'push') { return { profile: 'smoke', reason: 'post-merge smoke' } } if (nightly || eventName === 'schedule') { return { profile: 'all', reason: 'nightly schedule' } }
A pull request resolves its profile from whatever changed. A plain push to develop after a merge always runs smoke, regardless of what changed, on the assumption that every merge deserves at least a fast confirmation the site still boots. A manual run can override the profile entirely, picked from a fixed list of 12, including all and none. Anything that isn't one of those three event types falls back to a repository variable if one's set, or to booking if it isn't, since booking is the flow most likely to break something a customer would actually notice.
a suite doesn't get to block a pull request just by existing
Path scoping decides which profile runs. A separate system decides whether an individual spec inside that profile is trusted enough to fail a pull request at all. Every booking flow spec is registered in a small catalog against one of two statuses in practice right now, ready or wip, 14 marked ready, 20 still wip. payment-return.spec.ts, the spec covering payment return polling, is one of the wip ones today. Nothing enforces that status by convention. Playwright is told directly: any test tagged @wip or @flaky gets excluded from the run whenever readiness is anything other than all, which covers every pull request and every push. Only the nightly tier, and a manual dispatch that asks for it, sets readiness to all and actually lets those tests execute.
That's the same discipline as turning on a lint rule for a whole repo only after clearing its backlog first, just applied to end to end coverage instead of static analysis. A spec gets written and tagged @wip while it's still finding its feet, runs on demand or overnight where a failure doesn't block anyone, and only loses the tag, only becomes something that can fail somebody's pull request, once it's actually proven itself. Writing a flaky assertion into a suite that already blocks every merge is how a team ends up ignoring red checks out of habit. Keeping it out until it's earned a place in the default run is the alternative.
why unit tests and e2e tests aren't the same safety net
A unit test tells you a function or a component still behaves the way its contract says it should, in isolation, in milliseconds. An e2e test tells you whether a real browser, driving the actual built app, can complete a booking end to end against something close to production dependencies. Those are different questions, and a green answer to one doesn't imply a green answer to the other.
The Redis detail makes that concrete. The unit tier runs against a throwaway RSA keypair generated fresh for the job, exactly the kind of disposable fixture a unit test should use. The e2e tier instead spins up a real redis:7-alpine container and runs Playwright against the built app talking to it for real. If session handling broke in a way that only shows up against a real Redis instance, no amount of mocking in the unit tier would ever catch it. That's not a gap in the unit tests, it's just outside what unit tests are for. The docs only filter, at the other extreme, is the same idea from the opposite direction: some changes only need a browser confirming a page still renders, and never touch any logic a unit test would exercise at all.
where e2e gets expensive, and what you do about it
E2E costs more than unit tests in a way that compounds. Each test drives a real browser instead of calling a function directly, the app has to be built first, and running every suite on every pull request would add up fast across a day's worth of them. Two things keep that cost down.
The first is resolving down to one profile instead of running everything even remotely relevant. A change scoped to the wallet flow doesn't also spin up booking, auth, and embed, it runs wallet plus the baseline smoke spec every profile carries along for free, and nothing else.
The second is sharding, which is about latency rather than relevance. Once e2e-build has resolved a profile and built the app, uploading it as an artifact, the sharded e2e matrix job downloads that same artifact into several parallel runners, each taking --shard=N/total of whatever that profile's suite contains. The app only gets built once. The expensive part, actually driving a browser through it, gets split across machines instead of run one spec after another.
the same shape, somewhere older
A monorepo I also worked in, kv-frontend, runs a version of this same three tier idea with a lot less machinery behind it, and comparing the two says something about how a pattern looks early versus how it looks once a team's had years to sand the edges off it.
It's an Nx monorepo, and its pipeline splits into the same three tiers. lint_only installs dependencies and lints, running on pushes to feature branches and to develop, cheap and fast, meant to give a contributor feedback within a couple of minutes without anyone paying for a full run on every push to a branch nobody's merging yet. lint_test_build installs, lints, runs the full test suite, and builds, wired into branch protection as a required check on pull requests into develop and on pushes to main, develop, and release branches, so a red result here is what actually stops a merge button from working. e2e_plan is the third tier, using the same dorny/paths-filter idea Travelstart's e2e-build uses, mapping changed globs onto specific Playwright suites scoped to individual app sections and shared libraries, so a change confined to one corner of the monorepo doesn't trigger every e2e suite the repo has.
No profile system, no per spec readiness catalog, no sharding beyond whatever paths-filter itself resolves, just three jobs and a set of globs. It gets to roughly the same place Travelstart's setup does, fast feedback, a required gate, e2e scoped to what actually changed, with a fraction of the moving parts. Sometimes that's exactly enough.
a small aside worth noticing
One more detail says something about how these pipelines get designed. A separate, standalone job exists purely to build and upload JavaScript sourcemaps for error tracking, because the main deploy image is built with no environment variables or secrets at all, by design, and structurally can't run that step itself. Rather than loosen that constraint on the main build, the sourcemap step gets its own small job with only the context it actually needs. Keep the default path minimal, and give anything that needs more than the default its own narrow job instead of widening everyone else's.
the actual lesson
None of this is really about YAML. Travelstart's nightly regression sitting on a commented out cron line, with a written reason and a condition for turning it back on, is what actually earns a pipeline trust: not that every tier always runs, but that when one doesn't, somebody made that call on purpose and said so, rather than it quietly rotting into something nobody believed anymore. kv-frontend gets to a similar shape with three plain jobs and a set of globs, proof that the underlying idea, tiered checks, gate the merge on the one that's actually reliable, scope the expensive tier to what changed, doesn't need much machinery to be worth doing in the first place. The machinery Travelstart adds on top, profiles, a readiness catalog, sharding, exists to keep that same idea working at a scale where three jobs and a glob list stop being enough. A pipeline earns trust one required check at a time, and every one of those checks has to mean something when it goes red, or nobody will believe it the day it matters.