← Back to Research Hub
Study 1 · Fieldwork Instrument
Empirical Phase Checklist
Operational checklist for running Phase 1 end to end, from screening and cell allocation through
the per-interview run sheet to post-interview coding. The disciplines that protect the design
(frame separation, frontline-first ordering, no-collection handling of anchored work) are built
into the steps rather than left to memory.
Target: ~50 interviews, 70/30 frontline to boundary
Order: frontline first, then boundary
REC approval: 15 Apr 2026 (Dept of Engineering)
Source: Study 1 Design + Methodology
1 · Screening and cell allocation
Sort each candidate on observables only, never on the predicted reading. The screening frame
claims only when a developer started shipping accountable code (consequence exposure, formation
before or after the performative threshold, roughly Q4 2022) and how much AI touches that code
now (engagement depth, heavy or light). It says nothing about whether anyone holds accountable
judgment, which is the study's question, not its starting point.
- Place each candidate by the two screening axes and record the cell.
- Track running counts against the frontline targets and the boundary target.
- Hunt deliberately for the Veteran AI-light cell (Condition 2, light AI plus accountable-criteria judgment, the real baseline). Let Condition 1 (light AI but proxy-fluent) arrive as a byproduct rather than recruiting for it.
- Log a shortfall in any cell as a finding about the population, not a recruitment failure. The frame separation is what makes that reading available.
| Arm / Cell | Target n | Role in design |
|---|
| Frontline (n = 35, 70%) | | Carries the judgment and reachability evidence available nowhere else. |
| Veteran, AI-heavy | 18–20 | The core, the active PSF site. |
| Newcomer, AI-heavy | 10–12 | The premature-arrest contrast. |
| Veteran, AI-light | 3–5 | Purposive rarity, contrast anchor (hunt for Condition 2). |
| Newcomer, AI-light | 0–2 | Opportunistic only, documented as structurally thin and off-mechanism. |
| Boundary activity (n = 15, 30%) | | Sampled for translation work, not job title: design leads, engineering managers, DevEx, DevRel, PMs. |
2 · Ethics and consent state
- Base approval in place. Cambridge Department of Engineering REC approval granted 15 April 2026 (Chair: Dr. Robert Phaal). Consent, data handling, and anonymization follow the approved submission and are carried into the screener and recruitment materials.
- Per participant: information sheet issued, consent signed, recording disclosure made and agreement confirmed before any substantive question.
- Amendment-pending (artifact anchoring). The invitation to anchor accounts to a real example of the participant's own work (frontline Moves 1 and 2, boundary Move 2) is marked amendment-pending. It is not exercised until the REC amendment is approved. No artifact is collected, transmitted, or retained under any circumstances.
- Log further approvals and amendments as they come in (additional recruitment channels, secondary domains, protocol changes after the pilot).
3 · Per-interview run sheet
Both interviews are a single semi-structured session of about fifty minutes, run as a linear flow. Screening axes are fixed before the session and never raised. Nothing in the interview names AI, judgment, proxies, criteria, or erosion.
- Opening: confirm consent in place, confirm recording disclosure and agreement, read the framing, run the short warm-up (not coded). Do not raise AI use.
- Move 1: the satisfied-work opener (frontline) or the report-upward opener (boundary).
- Move 2: the reachability probe (passed-every-check / good-numbers). Follow the participant's own story before supplying a hypothetical. The difficulty in producing a case is itself data.
- Move 3: the how-do-you-know probe, run every time as the cross-check.
- Closing: open the floor, ask for referrals (feed the screener), thank the participant, stop the recording.
No-collection, no-retention
Where artifact anchoring is approved and used, the real piece of work is a concrete object to talk
through during the session only. The participant keeps it on their own screen or describes it
without showing anything. Nothing is collected or retained, and a participant may decline to
anchor, in which case the interview proceeds with a general account (itself a reading).
4 · Ordering discipline
- Frontline before boundary. Frontline interviews run first so the criteria shifts are documented before propagation is traced. The boundary interviews are then read against an established picture of what is being propagated rather than a guess at it.
- Within frontline, prioritize the cells the contrast depends on (the Veteran AI-heavy core and the Newcomer AI-heavy contrast), while keeping the deliberate hunt for the rare Veteran AI-light anchor running throughout.
5 · Post-interview
- Transcribe, then write a short first-pass memo while the session is fresh.
- Apply the coding rubric in analysis only: mark every criterion-reach Position 1, 2, or 3 before applying the RQ signatures.
- Record confirming and disconfirming signatures by cell, including any Position-1 negative cases counted against a cell.
- Keep the screening frame and the coding frame separate in every artifact, so the prediction never contaminates the sample or the reporting.
6 · Saturation and revision
- The 70/30 split and the frontline-first order are judgments under uncertainty, open to revision against what saturation shows.
- Treat the pre-engagement baseline as reconstructed (from heavy-engagement veterans recalling earlier practice), triangulated against datable artifacts, not sampled from present-day light users.
- Attach the within-person "criteria changed over time" claim narrowly; let the cross-sectional contrast carry the main findings.