A resource for design system and developer experience teams
Time to firstcomponentbenchmark.
Watch five new developers try your system. Time it properly.
Same tasks. Same clock. Fix the top three.
The first half hour decides adoption.
API teams have measured "time to first hello world" for more than a decade: how long a developer takes, from nothing, to get working code out of an API [S01]. Postman calls time to first call the most important API metric [S02]. The essay behind this guide makes the same case for design systems: if a new developer ships a first component in under 30 minutes they keep using the system, and if it takes hours they build custom UI.
A benchmark only helps if the next run is comparable with the last. This guide fixes the parts that drift: what counts as start and stop, who takes part, the environment, what the facilitator may say, how unfinished sessions count, and which statistics to report on five people.
The kit: the protocol, two task definitions, a timing sheet, eleven friction codes and a zero-dependency script that validates a sheet, reports each task and compares two runs. The two sample runs are synthetic; every number quoted is the script's output on them.
Practical guidance, not a standard. Five participants give a direction and a list of friction, not a precise estimate. The sample runs are invented to show the method. The 30-minute target comes from the essay Growth loops for design system adoption. Prepared with AI assistance and edited by hand.
Start here
Two tasks, one clock.
T1 isolates finding, installing and configuring. T2 is the essay's task. Together they separate setup friction from build friction.
| T1 Hello component | T2 Login form PR | |
|---|---|---|
| API analogue [S01] | Time to first hello world | Time to first useful app |
| Card | Show an @acme/ui Button labelled Save on the home page. | Build a login form with the system and open a pull request. |
| Start | Card read; participant says start or presses Start | Same |
| Stop | The Button renders, no errors in terminal or console | Pull request opened, lint and typecheck passing locally |
| Timebox | 60 minutes | 120 minutes, the essay's broken-onboarding line |
| Target | None until the first baseline | Median of 30 minutes (essay) |
Full definitions and done criteria in tasks.json; the session rules in protocol.md.
First run
Sections 01 to 03. Book 5 participants and a clean container; print the timing sheet.
Analysing a run
Section 04. Run ttfc.mjs on the sheet and report the table as printed.
Proving a fix worked
Section 05. Same protocol version, new participants, --baseline on the old sheet.
Already tracking the funnel
Field Guide 16's ACT-3 is field time in days; this is task time in minutes. Report both.
| Label | Meaning |
|---|---|
DATA | A field in a kit file: tasks.json, the timing sheet or friction-codes.json. |
SCRIPT | Computed or enforced by ttfc.mjs. |
PROTOCOL | How a session is run, so runs stay comparable. |
PRIVACY | Protects participants. The benchmark tests the system, not the person. |
LOOP | Feeds the result back into the next run or the funnel. |
PRACTICE | A working method with a review signal, not a hard gate. |
Section 01
Decide what "first component" means.
Most arguments about onboarding speed are arguments about when the clock started. Write the events down once.
Suggested owners: Design system engineering lead + DX lead
Two tasks: a hello world and a first real change
One task mixes setup and build friction together. T1 ends at a rendered component; T2 ends at a pull request, the way API teams pair hello world with a first useful app [S01].
Evidence: Two tasks in
tasks.json, each with its own timebox and done criteria.S01. Essay task for T2.
Start and stop are events someone can see
Start when the participant has read the card and says start. Stop at an observable state: the component on screen, the pull request opened. Never at "I think I'm done".
Evidence:
startandstopfields per task; the facilitator or the recording confirms both.S02: time from a defined point to the first successful call.
T2 stops at an opened pull request, not a merge
Merge time measures reviewers. The essay stops the clock when the participant submits the pull request; lint and typecheck must pass locally first.
Evidence: T2
donecriteria intasks.json.Essay benchmark.
Changing a task changes the protocol version
A new card, stop event or timebox makes old times incomparable. Bump
protocolVersionand restart the baseline.Evidence:
ttfc.mjsrefuses a sheet mixing versions and warns when comparing across them.Kit.
Lab minutes are not field days
The benchmark times a task in a session. Field Guide 16's ACT-3 counts calendar days from onboarding to the first merged pull request. DX Core 4 measures organisation-wide throughput [S05]; this is a narrower, task-level signal.
Evidence: Both numbers appear in the weekly funnel report, on separate lines.
S05. Field Guide 16.
If two people would start the clock at different moments, you do not have a benchmark yet.
Section 02
Same kind of people, same machine.
Most variance in five sessions comes from who sat down and what their laptop already had. Control both.
Suggested owners: DX lead + research lead
Five participants who have never used the system
The essay recommends 3 to 5 new engineers. Take 5 so one outlier does not decide the median. Never the system team.
Evidence:
ttfc.mjsnotes any task with fewer than 5 sessions as directional.Essay. Kit threshold.
Voluntary, pseudonymous, reported per task
Say in the invitation that the system is being tested, not the person. Record
p01, not a name. A single timed task says nothing about someone's productivity [S06].Evidence: No names in the sheet; the key is held by the facilitator and deleted after the run.
S06.
A clean container with no package cache
A warm cache or a registry token from last year hides the install friction T1 exists to find. Use a fresh dev container or virtual machine.
Evidence:
environmentcolumn isclean-containerfor every session.Kit protocol, section 3.
The same starter commit for the whole run
Pin the starter app to one commit per run and record it. A dependency bump mid-run makes the last sessions a different test.
Evidence: The commit hash in the run notes.
Kit protocol.
One session setup per run
Moderated and silent is the default. Think-aloud gives reasons but changes times. Unmoderated sessions scale across time zones but cannot clarify a task [S04]. Pick one per protocol version.
Evidence:
ttfc.mjsrefuses a run that mixesmodeorthink_aloudvalues.S04.
Control the room, or you are timing the laptops.
Section 03
Run the session the same way every time.
The facilitator's job is to watch and write, not to rescue. Every exception to that is recorded.
Suggested owners: Facilitator
One assist after five stuck minutes, and it is counted
Help changes the time. If a participant is stuck for 5 minutes, give one pointer, then write the minute and the pointer in the notes. The assist is itself a finding.
Evidence:
assistscolumn and a note. The Q2 sample has 4 assisted sessions.Kit protocol.
Interruptions are subtracted, not ignored
A fire drill or a call is not task time. Record it in seconds and the script subtracts it.
Evidence:
paused_seconds: the Q2 sample's 67-minute T2 session with 180 paused seconds counts as 64.Kit.
Timeouts and give-ups are recorded, never dropped
Stop at the timebox and record
timeout; a participant who stops trying recordsabandoned. Dropping them makes a slow system look fast.Evidence: The validator rejects a
successlonger than the timebox. Q2 T2 has 1 timeout and 1 abandoned of 5.Kit.
Friction is coded live from a fixed list
Code every stall over a minute and every question with
friction-codes.json. Useenv-setupfor problems outside the system, so they are recorded but not fixed by the wrong team.Evidence: The validator rejects unknown codes; 11 codes ship, including
other.Kit.
One question at the end
"What nearly made you give up?" The answer names the friction the timings cannot.
Evidence: The answer in
notesfor every session.Kit protocol.
The facilitator watches. The sheet remembers.
Section 04
Report what five sessions can support.
Task times are skewed: a few long sessions pull the mean up. Report robust statistics, name the method, and show what did not finish.
Suggested owners: DX lead
Median and p75, with the method named
Set targets on the median; watch p75 for the slow tail. Percentiles interpolate linearly between ranks, the same as Excel PERCENTILE.INC and SQLite's
percentile()[S07].Evidence: Q3 sample: T1 median 12.0, p75 14.0 minutes; T2 median 31.0, p75 38.0.
S07. Kit output.
Unfinished sessions count at the timebox
A timeout is at least the timebox, so it enters the percentiles at that value. A percentile that lands on one is a lower bound and prints with
>=.Evidence: Q2 sample, T2: median 88.0, p75
>=120.0because 2 of 5 sessions did not finish.Kit.
Geometric mean for the typical time on small samples
Below 25 sessions, the sample median is a biased estimate; the geometric mean of completed times is more accurate [S03]. Report it beside the median.
Evidence: Q3 sample, T1: geometric mean 11.5 against a median of 12.0.
S03, Sauro and Lewis, CHI 2010.
Completion rate sits beside every time
A fast median with 60% completion is not a fast system. Print both on the same line.
Evidence: Q2 sample: T1 80%, T2 60% completion.
Kit output.
The top three friction codes, counted
Count task sessions per code across both tasks. The list is the work plan for the next quarter.
Evidence: Q2 sample:
unclear-example(5),registry-auth(3),a11y-lint(2).Kit output.
Five sessions give you a direction and a work list. Do not dress them up as precision.
Section 05
Compare like with like, then fix three things.
The benchmark is worth running twice. The second run is where it proves or disproves the fixes.
Suggested owners: Design system lead + DX lead
Compare only runs on the same protocol
Same protocol version, same setup, new participants. Anything else is a different experiment.
Evidence:
ttfc.mjs --baselinewarns when protocol versions differ.Kit.
Report change in minutes and percent, bounds called out
When the baseline percentile was a lower bound, the real improvement is at least the one shown. Say so.
Evidence: Q2 to Q3, T2: median 88.0 to 31.0 (-57 min, -64.8%); p75
>=120.0to 38.0, flagged as a lower bound.Kit output.
A target is met, or it is not
The essay's target is a median under 30 minutes. 31 is not met. Do not round, and remember it is the median of five people.
Evidence: Q3 sample, T2: median 31.0, reported
not met.Essay target. Kit output.
Fix the top three before the next run
The essay's rule. Name an owner for each and link the fix from the run notes, so the next run tests something.
Evidence: Q2's top code,
unclear-example, drops to 1 task session in Q3.Essay. Kit output.
The result feeds the funnel report
Activation is the stage the benchmark diagnoses. Put the latest median and p75 on the benchmark line of Field Guide 16's weekly report, and rerun each quarter.
Evidence: The benchmark line of
weekly-funnel.mdis filled with the latest run.Field Guide 16.
The benchmark is done when the next run is faster for a reason you can name.
Appendix A
One row per session and task.
Timestamps, not typed minutes: the script computes durations, so nobody does arithmetic under time pressure.
| Column | Holds | Checked by the script |
|---|---|---|
run_id, protocol_version | Which run, which protocol | One version per sheet |
session_id, participant_id | Pseudonymous ids | No duplicate session and task |
mode, think_aloud, environment | Session setup | One setup per run |
task_id | T1 or T2 | Known task |
start_ts, stop_ts | ISO 8601 UTC | Stop after start |
paused_seconds | Interruptions | Subtracted from duration |
outcome | success, fail, timeout, abandoned | Success within the timebox |
assists, friction_codes, notes | What happened | Codes in friction-codes.json |
data/run-2026-q2.sample.csv (two rows)
run_id,protocol_version,session_id,participant_id,mode,think_aloud,environment,task_id,
start_ts,stop_ts,paused_seconds,outcome,assists,friction_codes,notes
2026-q2,1.0,s01,p01,moderated,no,clean-container,T1,
2026-06-16T09:00:00Z,2026-06-16T09:34:00Z,0,success,0,registry-auth;install-peer-deps,
2026-q2,1.0,s04,p04,moderated,no,clean-container,T2,
2026-06-16T14:10:00Z,2026-06-16T15:45:00Z,0,abandoned,0,unclear-example;a11y-lint,
Gave up: could not map error text to a fixRows wrapped to fit the page; each is one line in the file.
Appendix B
Two runs, compared.
Q2 is the baseline, Q3 follows fixes to registry setup, the starter template and examples. Both synthetic, five moderated sessions each.
terminal
$ node scripts/ttfc.mjs data/run-2026-q2.sample.csv
Time to first component, run 2026-q2 (protocol 1.0), minutes
Task n done median p75 geomean range target
T1 Hello component 5 80% 34.0 41.0 30.2 22.0-41.0 none
T2 Login form PR 5 60% 88.0 >=120.0 74.0 64.0-88.0 median 30: not met
Top friction: unclear-example (5), registry-auth (3), a11y-lint (2)
$ node scripts/ttfc.mjs data/run-2026-q3.sample.csv --baseline data/run-2026-q2.sample.csv
Time to first component, run 2026-q3 (protocol 1.0), minutes
Task n done median p75 geomean range target
T1 Hello component 5 100% 12.0 14.0 11.5 7.0-19.0 none
T2 Login form PR 5 100% 31.0 38.0 33.0 26.0-44.0 median 30: not met
Top friction: a11y-lint (2), types (2), docs-not-found (1)
Compared with 2026-q2:
T1 median 34.0 to 12.0 (-22 min, -64.7%); p75 41.0 to 14.0 (-27 min);
completion 80% to 100%
T2 median 88.0 to 31.0 (-57 min, -64.8%); p75 >=120.0 to 38.0 (-82 min);
completion 60% to 100%
note: T2: the baseline p75 is a lower bound (sessions hit the timebox),
so the real improvement is at least the one shownOutput lightly trimmed (legend lines) and wrapped. The Q3 friction list is the next quarter's work: accessibility lint messages and component types.
Keep with the run
Leave a run record.
One record per run, so the next facilitator can reproduce it and the next reader can trust the comparison.
Sources / maintenance
Keep the guide current.
Sources checked 24 September 2026. Vendor statements are cited as vendor views. No external benchmark figures are reused: the only targets are the essay's.
ttfc.mjs uses.https://sqlite.org/percentile.html