Usually there’s some gap between training content and the real world. It’s impossible to capture the entire real- world experience in a training session.

One reason is because in the real world, there’s usually more than one right answer. It’s more of a range of probabilities, from “that might work” to “experienced pros know in these circumstances this is usually the way to go.”

Example: You’re on the verge of closing a sale and the buyer asks for a last-minute price concession.

What do you do?
A. Agree to cut your price.
B. Walk away.
C. Ask for a concession in return.
D. Tell the buyer you’ll have to check with your boss.

In the real world, any of these might be the right answer. If you have a lot of negotiating room and this is a one-time sale, A might be the best. C is often a good negotiating position. D could be an act of cowardice or a valid delaying tactic, depending. B is almost never the best option – unless you’ve concluded that the buyer is wasting your time.

It’s a complicated choice. But trainees need clarity and direction. So what do you do?

The research
Recently, researchers tested a “more than one right answer” technique for medical students.

Doctors are seldom confronted with right-or-wrong decisions. They have to make judgment calls, and the researchers wanted the examinations to reflect that reality.

So they wrote questions with more than one right answer about how to proceed with a diagnosis.

They sent the questions out to a panel of experienced physicians. Based on their responses, they weighted the answers. Instead of “right” or “wrong,” each response got a score.

Students whose answers aligned more closely with the experienced doctors’ responses were considered to have their judgment on the right track – and received a higher score.

What you can do
Instead of right-and-wrong answers, consider offering weighted scores for test answers – ideally, scores that reflect how a panel of experts answered the same questions.

In addition to creating a real-world feel to the test, these “sort of right” answers can generate deeper discussions.

In the sales example, some topics flowing from that question might include: How do you know when a buyer is bluffing? When does it make sense to walk away? What kinds of concessions might make sense?

To find experts who can validate your test, you might not have to look any farther than your own experienced employees.

In our example, you could get your best sales reps to review your questions and rank them. While you’re at it, you could ask them to help you craft follow-up questions and scenarios that reflect their own real-world experiences.

Source: Duggan, Paul, et al., “Summative assessment of 5th year medical students’ clinical reason …,” September 2011: BMC Medical Education, Vol. 12, No 29.

Leave a Reply

Your email address will not be published. Required fields are marked *