Structured Interviews: What the Validity Research Says

Most hiring managers believe they are good judges of talent. The research says otherwise — and the gap between perceived and actual interview validity has been measured, repeatedly, for four decades. The fix is not a new platform or a longer process. It is structure: pre-set questions, competency-anchored scoring, and a rubric that defines what each level looks like before the first candidate walks in.

The Validity Table: What the Meta-Analyses Found

Validity coefficient (ρ, corrected for range restriction and measurement error) measures how well a selection tool predicts actual job performance, on a scale from 0 (no relationship) to 1 (perfect prediction). A coefficient above .40 is considered practically significant in industrial-organizational psychology.

Sources:

• Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124(2), 262–274.
• McDaniel, M. A., Whetzel, D. L., Schmidt, F. L., & Maurer, S. D. (1994). The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology, 79(4), 599–616. (Based on 245 coefficients, N = 86,311.)
• Wiesner, W. H., & Cronshaw, S. F. (1988). A meta-analytic investigation of the impact of interview format and degree of structure on the validity of the employment interview. Journal of Occupational Psychology, 61(4), 275–290.

Two patterns stand out. First, the validity gap between structured (.51) and unstructured (.38) interviews is not marginal — it represents a meaningful increase in predictive accuracy that compounds across hundreds of hiring decisions. Second, combining a structured interview with a cognitive ability measure (.63) produces validity approaching that of the best available selection tools.

Why Structure Raises Validity: The Mechanisms

Researchers have identified four mechanisms through which structure improves predictive validity. Each has a practical implication for how a hiring team should run its process.

1. Standardised Questions Remove Interviewer-Specific Variance

In an unstructured interview, each interviewer asks different questions. One probes for analytical reasoning; another lingers on cultural fit; a third follows wherever the conversation leads. The resulting scores measure partly the candidate and partly which interviewer happened to ask what. That interviewer-specific variance is noise — it reduces predictive validity without adding signal.

Standardising questions so that every candidate answers the same set eliminates this source of noise. The score then reflects the candidate's responses to a defined stimulus, not the interviewer's conversational preferences.

2. Anchored Scoring Scales Reduce Rater Drift

Conway, Jako, and Goodman (1995) found in a meta-analysis that the average inter-rater reliability for structured interviews was .59, versus .37 for unstructured interviews — a two-to-one advantage. Reliability is a ceiling on validity: a tool cannot predict performance better than it can measure itself consistently.

The key lever is behaviorally anchored rating scales (BARS): a score of 3 is not "adequate" but "described a specific situation in which she identified the root cause through data, proposed two alternatives, and explained the trade-off" — with an example written at the role level, agreed on before interviews begin. Without anchors, two interviewers assign the same number to entirely different observations, and debrief becomes a negotiation rather than a triangulation.

Verdict's six-dimension model uses this principle: every score is tied to verbatim evidence extracted from the CV or interview notes, not to an evaluator's summary impression.

3. Structure Reduces the Influence of Irrelevant Information

Unstructured interviews are particularly susceptible to a documented set of biases: the primacy effect (early impressions dominate), the similar-to-me effect (candidates who share background with the interviewer score higher), and physical appearance cues that trigger halo effects. These are not hypotheses — they have been replicated across multiple experimental settings.

Structure attenuates these effects by directing the interviewer's attention toward specific, pre-defined competencies. When a score must be justified with a specific quote or example from the candidate's actual answer, the path from liking to high score requires work. That friction is intentional and productive.

A 2014 narrative and quantitative review by Levashina, Hartwell, Morgeson, and Campion in Personnel Psychology (vol. 67, 241–293) examined decades of structured interview research and found that structured interviews are substantially more reliable and valid than unstructured ones, and produce smaller differences across candidate subgroups.

4. Forcing Evidence-Citation Improves the Evidence Base

The simplest structural intervention is the citation rule: before submitting a score on any dimension, the interviewer must write the exact quote or described behaviour that put the candidate at that level. If you cannot quote it, you cannot defend it.

This rule has two effects. First, it surfaces the absence of evidence — a common problem when candidates are articulate but non-specific. Second, it creates a retrievable record that allows post-hoc calibration: when a hire's 90-day performance diverges from their interview score, you can go back and examine what the evidence actually was, and learn whether the rubric was miscalibrated.

Behavioural vs. Situational Questions: What the Research Distinguishes

Not all structured interview questions are equally valid. The research distinguishes two main types:

Behavioural questions ask about a specific past event: "Tell me about a time when you had to persuade a stakeholder who was initially opposed to your recommendation. What was the situation and what did you do?" The answer is verifiable in principle (a reference check could corroborate it) and requires the candidate to reconstruct a real episode, which is harder to fabricate convincingly than a generic best-practice answer.

Situational questions ask what the candidate would do in a hypothetical scenario: "If you discovered that a peer was consistently missing data-quality checks, how would you handle it?" Latham and colleagues developed this approach; situational interviews correlate more strongly with job knowledge and future behavioral intentions, rather than with past demonstrated behaviour.

McDaniel et al. (1994) found that both question types produce valid prediction, but the two measure partially different constructs. Best practice — and the approach Verdict's candidate evaluation dimensions are designed to support — is to pair behavioural questions with evidence-citation scoring, so that the output is a claim anchored to a specific story, not an abstract self-assessment.

Applying the Research: A Worked Example on Trajectory

The following shows how the validity-maximising principles above translate into practice on Verdict's Trajectory dimension — the rate and direction of a candidate's growth.

Weak (unstructured) approach: Interviewer asks: "How do you see your career developing?" Candidate gives a well-rehearsed answer about seeking leadership and continuous learning. Interviewer notes "growth-oriented" and gives a 4. No evidence anchor. No comparability across candidates.

Structured approach with anchored scoring:

Pre-agreed question: "Walk me through the last three roles you held. For each one, tell me what the scope of your responsibility was when you started and what it was when you left — and what changed."

Pre-agreed anchor for a 4 on Trajectory:

The candidate can articulate a specific, verifiable expansion of scope across at least two roles — e.g., moving from managing one product area to owning the full roadmap, or from individual contributor to team lead with budget ownership — within a timeframe that is plausible for the role level. The candidate can explain what drove the expansion (a project outcome, a gap they identified, a sponsor's decision) rather than attributing it to tenure alone.

With this anchor, two interviewers who hear different candidates produce comparable scores. A 4 means the same thing in both scorecards. Debrief takes minutes instead of an hour.

This is the logic behind Verdict's evaluation methodology: define what evidence looks like at each level, then score against evidence — not against impression.

What Structure Does Not Fix

Structure raises validity. It does not eliminate it.

Even a well-designed structured interview has a validity ceiling: corrected coefficients cluster around .51, meaning there is substantial variance in actual job performance that interviews — structured or not — cannot capture. This is why the Schmidt & Hunter (1998) finding that combining a structured interview with a general cognitive ability measure yields .63 is practically important: the two instruments capture partially non-overlapping information, and the .63 figure reflects that combined approach — not what a standalone structured interview routinely achieves.

Structure also does not fully eliminate subgroup differences in interview scores. Research has found that structured interviews reduce but do not eliminate some group-mean differences. The correct response is not to abandon structure (which would widen those differences) but to audit scoring distributions across interviewers and role families, and to investigate whether anchor definitions are being applied consistently.

Finally, structure can be poorly implemented. An interview with pre-written questions but no scoring rubric, or with anchors so vague that each interviewer interprets them differently, captures the form of structure without the mechanism. The validity gains in the meta-analyses reflect well-implemented structure — standardised questions, anchored rating scales, trained interviewers, evidence citation.

How Verdict Maps This to the Six Dimensions

Verdict's methodology operationalises the validity principles above across six dimensions — Capability, Track Record, Trajectory, Influence, Domain edge, and Risk surface — by extracting verbatim evidence from the uploaded CV and mapping it to a pre-defined scoring rubric. The output is not an impressionistic rating: it is a score anchored to a specific CV quote, with the quote visible alongside the score.

This mirrors the citation rule described above: the score is the evidence. An evaluator who disagrees with a score can point to the quote and argue that it maps to a different level — a structured disagreement that can be resolved against the rubric, not a contest of impressions.

For roles where interview evidence needs to be layered on top of CV evidence — the most common case in senior hiring — Verdict's evaluate workflow accepts interviewer notes alongside the CV, extracting structured evidence from both sources and integrating it into a single evidence-backed score per dimension.

FAQ

What is a valid validity coefficient? In personnel selection research, a corrected validity coefficient (ρ) of .30 is considered moderate, .40 is practically significant, and .50+ is strong. Coefficients are corrected for statistical artifacts including range restriction (hired populations are narrower than applicant pools) and criterion measurement error. Uncorrected observed correlations are typically lower.

Does structure make interviews slower? A well-designed structured interview is often faster, not slower. Knowing exactly which questions to ask and having a scoring rubric eliminates the exploratory conversation that pads unstructured interviews without adding validity. A 45-minute structured interview scored against an anchor rubric typically yields more defensible output than a 90-minute unstructured conversation.

What about "culture fit" questions? "Culture fit" assessed through open-ended conversation is the least structured, lowest-validity form of interviewing, and the most susceptible to bias. If culture fit is a genuine competency requirement for a role, define what it means in behavioural terms, write anchored levels, and score it like any other dimension. An unanchored gut-feel rating adds noise, not signal.

Can smaller hiring teams implement this? Yes. The minimum viable structured interview is: one set of pre-written questions (four to six), one scoring rubric with anchor definitions per dimension, and the citation rule. You do not need a proprietary platform. The research gains are available to any team willing to write the rubric before the first interview starts.

Where does a CV fit in a structured process? The CV is the primary evidence source for Track Record, Trajectory, and Domain edge. A structured interview is most valuable for Capability and Influence — dimensions where the CV provides weaker signal because the relevant evidence is behavioural rather than biographical. Verdict's approach to candidate evaluation dimensions treats CV and interview as complementary instruments, not interchangeable ones.