How do you get expert knowledge out of experts who have no time?

The subject matter expert, the person whose reasoning your program depends on, has agreed to forty minutes. In that time you need a usable share of what took them fifteen years to learn. Most interviews spend the meeting asking about the topic, and the topic is precisely what the expert cannot compress on demand. Decades of research on knowledge elicitation, the craft of getting what someone knows into a form others can use, points the other way: ask about incidents. Our guide to designing decision points already relies on this move to source its decisions. This article is the full method behind the move, usable as a prep sheet for your next SME session.

Why do topic questions waste the meeting?

Ask "what should people know about pricing conversations" and you get bullet points: know your value, listen first, do not concede early. The expert is not being unhelpful. General principles are summaries, and summaries are what remains after the situations that produced them have been stripped away. The bullet points are already in the training manual, which is why programs built from topic interviews restate what the audience has heard before.

Ask instead "tell me about the last time someone mishandled a price objection" and the answer arrives as a scene: who was in the room, what the customer said, what the rep did within two minutes, what the expert would have done instead, and why. A scene carries the options that were live at the moment of choice and the reasoning that separated them. That is the material an instructional designer can build from, and it only surfaces when the question names an event rather than a subject.

Where does the incident method come from?

The founding evidence is old and still decisive. In 1941, the U.S. Army Air Forces analyzed the reasons 1,000 pilot candidates had been eliminated from flight training. The source was the proceedings of the elimination boards, where instructors and check pilots recorded why each candidate failed. Many of the recorded reasons were clichés and generalizations: "lack of inherent flying ability," "unsuitable temperament," "poor judgment." The usable part of the record was the minority of specific observed behaviors. Three years later, in the first large-scale effort to gather such observations systematically, combat veterans were asked to report incidents of especially helpful or inadequate behavior on missions, with the instructions ending in the request, "Describe the officer's action. What did he do?" Several thousand incidents came back, factual enough to define effective combat leadership (Flanagan, 1954).

John Flanagan formalized the approach in 1954 as the critical incident technique, with five steps:

1. Establish the general aim of the activity, because no behavior can be judged effective without knowing what it was supposed to accomplish.
2. Set the plans and specifications: which situations count, who is qualified to report on them, and how significant an incident must be to matter.
3. Collect the incidents.
4. Analyze them into categories.
5. Interpret and report, stating plainly what the collection can and cannot support.

One of Flanagan's field findings matters directly for scheduling. When factory foremen reported incidents weekly instead of daily, they had forgotten about half of them; reporting after two weeks, they had forgotten 80 percent. Ask experts for recent incidents, and get to them soon.

How does the critical decision method work?

For decision-heavy work, Gary Klein and colleagues extended the incident interview into the critical decision method (Klein, Calderwood, & MacGregor, 1989). The method is a semi-structured interview: the questions are prepared in advance, but the interviewer follows the expert's account wherever it leads. Its defining choice is to concentrate on non-routine, difficult incidents, because hard cases surface elements of expertise that routine ones rarely expose, which also makes a single well-chosen incident remarkably efficient (Hutchins, Pirolli, & Card, 2004).

The interview runs as passes over one incident. In the first pass, the expert recalls a specific difficult case and walks it end to end, step by step, building a timeline of what happened when. The passes that follow go back over the same events with probes, prepared questions that surface what the retelling skipped. A documented application of the method, which recorded and transcribed each session, used probes along these lines during the retelling:

• What information were you seeking at that point? What questions were you asking?
• Why did you need that information, and how did you get it?
• What did you do with it? Would some other information have been helpful?
• Were you building a picture in your head of the situation? Of the important actors and their relationships? Of how events would unfold over time? Can you draw it?
• What hypotheses did you form? What alternatives did you consider? Did a hypothesis change what you looked for next?

And in a follow-up session, after the first interview had been analyzed:

• What were your specific goals at the time?
• Does this case fit a standard or typical scenario? One you were trained to deal with?
• Did this case remind you of any previous experience?
• As you took in information, what triggered the questions you later followed up?

In that application, each session ran about one and one-half hours with experts who averaged ten years in the work (Hutchins et al., 2004). Most SME calendars will not give a learning designer ninety minutes, which makes the discipline of the method more valuable, not less: one incident, walked fully, beats four topics skimmed. A broader review of elicitation methods reaches the same verdict, rating semi-structured interviews among the approaches that capture complex knowledge accurately, and warning that loose conversational interviews tend to produce sincere, coherent accounts that miss what actually matters (Holtrop et al., 2021).

Why is one session never enough?

Because a single pass demonstrably captures a fraction of what the expert holds. LaMere and colleagues elicited mental models, each documented as an influence diagram (a map of the factors in a system and the cause-and-effect links between them), from eleven expert stakeholders in a fisheries system. The diagrams produced live in the sessions contained 349 variables and 496 causal relationships in total. Then the researchers transcribed the session recordings and added everything the experts had said but nobody had drawn. The totals rose to 893 variables and 1,472 causal relationships, and the enriched diagrams went back to the experts for confirmation (LaMere et al., 2020). The experts had said most of it out loud. The single pass simply could not hold it, a loss the authors attribute to time pressure, fatigue, and the sheer difficulty of articulating a complex model in one sitting.

The practical schedule follows directly. Session one collects the incident. Between sessions, you synthesize what you heard into a structured summary. Session two confirms the summary and captures what the first pass missed. Never trust one pass, and never schedule as if one will be enough.

What does AI change about the workflow?

The incident method now runs with AI on both sides of the interview. The method itself does not change; what changes is how much of the forty minutes reaches the expert's actual expertise.

Before the interview, work with AI to build a working framework of the domain. The material is domain-specific, but every domain has a basic structure, and AI can give you that structure in advance: the standard terms, the typical process, the known trade-offs. A content developer who arrives with the fundamentals already in hand never spends the forty minutes on them. The whole meeting goes to what only the expert has: their experiences, their judgment, the nuances. The pre-work also does something subtler. You can only follow a nuance in the moment if you recognize it as one, and recognition comes from knowing what the ordinary answer would have been.

During the interview, ask permission to record. A recording converts to a text transcript, and AI can distill the transcript into a structured summary of the incidents, the options at each point, and the reasoning behind each choice. Nothing the expert said gets lost to note-taking, which is exactly the loss the LaMere numbers measured. Flanagan observed in 1954 that recording and transcribing increases the workload substantially; that cost, which kept generations of interviewers scribbling, is now gone.

After the interview, send the distilled summary back and follow up to clarify. The design position behind that step: experts tend to respond with richer nuance when they see that you captured the essence of what they said, and the follow-up on the summary is where the detail a first pass misses has its chance to surface. This is the never-trust-one-pass finding in operational form: the AI-distilled summary becomes the artifact the second session confirms.

The framing matters. This is the classic incident method with a modern slant, not a replacement for it. AI handles capture and structure. The expert's reasoning and your attention stay the scarce resources.

What should you walk out with?

For each incident, four things. The situation, specific enough that a practitioner would recognize it. The tempting missteps, the moves that looked reasonable at the moment of choice. The expert moves, what the best performers do instead. And the why behind each, the reasoning that separates the tempting from the sound. That four-part record is the exact input the scenario-design pipeline consumes: which situations deserve this treatment, and where the evidence of what people typically do today lives, is the territory of finding the gaps, and turning each captured incident into a scenario decision is the work of designing decision points.

It is also the input that determines whether learners ever benefit from the expert's judgment. Expert reasoning captured this way, the situation, the live options, and the why, becomes the mentoring a learner receives at the moment of choice: when their first instinct at a decision is one of the tempting missteps, the expert's reasoning is right there to explain why the better move works. That is the design approach behind Guided Scenarios on AliveSim, where each decision point carries the reasoning of the expert who lived the incidents behind it. The forty-minute interview is the front of that funnel: everything a learner later receives as mentoring at a decision point began as something an expert said in that room.

References

• Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51(4), 327–358.
• Holtrop, J. S., Scherer, L. D., Matlock, D. D., Glasgow, R. E., & Green, L. A. (2021). The importance of mental models in implementation science. Frontiers in Public Health, 9, 680316.
• Hutchins, S. G., Pirolli, P. L., & Card, S. K. (2004). A new perspective on use of the critical decision method with intelligence analysts. 2004 Command and Control Research and Technology Symposium.
• Klein, G. A., Calderwood, R., & MacGregor, D. (1989). Critical decision method for eliciting knowledge. IEEE Transactions on Systems, Man, and Cybernetics, 19(3).
• LaMere, K., Mäntyniemi, S., Vanhatalo, J., & Haapasaari, P. (2020). Making the most of mental models: Advancing the methodology for mental model elicitation and documentation with expert stakeholders. Environmental Modelling and Software, 124, 104589.

Published July 18, 2026 · 9 min read