- 1 Exploring how to ask the right questions and measure success with Patty Shank
- 1.0.1 In your opinion, what are the most common mistakes made by instructional designers when designing multiple choice questions for e-learning evaluation?
- 1.0.2 One of the deepest areas of your skill is writing learning objectives to describe the desired outcomes. What do you think is the most challenging aspect of creating measurable purpose and why?
- 1.0.3 Can you tell our community more about your course “Write better multi-choice questions to evaluate learning” and who it was intended for?
- 1.0.4 You have written numerous articles for the e-learning industry over the years. Do you have a specific article that you are most passionate about that you want to highlight for our readers?
- 1.0.5 Which one excites you the most about the future of learning and development?
- 1.1 Unwrapping the wrapper
Exploring how to ask the right questions and measure success with Patty Shank
Patti Shank, Ph.D., is listed internationally as one of the top 10 most influential people in e-learning. He has authored numerous books and his articles are available on the Internet. With a strong business background and long experience as an instructor, Patty Shank applies learning and related sciences to improve outcomes from instruction and performance interventions. Today, he discusses effective evaluation, learning objectives, and skills development in the digital age.
In your opinion, what are the most common mistakes made by instructional designers when designing multiple choice questions for e-learning evaluation?
There are a number of common mistakes in research when writing multiple choice questions, but in my view the biggest mistake is Not evaluating the right thing. Most question-writers write questions by going through the content and finding simple things to evaluate. What we usually do Need However, the assessment is whether people can make the right decisions and adequately solve the problems associated with the critical actions described by the learning objectives.
This error is exacerbated if the learning objectives do not accurately describe the critical work and the results. In this case, our assessment may be invalid, which sets up the failure of the organization and the participants.
When we write multi-choice questions to measure relevant decision-making and problem-solving skills, we measure the high-level skills required to perform the tasks described for educational purposes. This What we should evaluate.
Other common mistakes when writing multiple-choice questions include making it easy to guess the correct answer (no! Don’t do it!) And making it difficult to understand the question and answer choices (no! Don’t do it!). Nedeau-Cayo’s 2013 survey of thousands of nursing certification questions, for example, found that 84% (!) Of questions had one or more significant errors! 
Poorly written questions usually do not evaluate objective knowledge and skills, leaving the assessment low or valid and creating frustration. If you use multiple-choice evaluation results to make a decision – such as who can go ahead – poorly written questions can put your company at legal risk.
One of the deepest areas of your skill is writing learning objectives to describe the desired outcomes. What do you think is the most challenging aspect of creating measurable purpose and why?
Many learning practitioners do not think that knowing what participants need to be able to do is important to their work. I have been told that the purpose of learning is not to make valuable use of the old days and times. But here’s the thing. If you don’t know what your participants need to do, how do you know what to teach? Or if the instructions “work” to help them do it?
When learning objectives are well written, they provide clear and critical guidance for instruction design and evaluation. Without them, we’re creating “content” and not “instructions.” Content alone is usually inadequate if you are talking about helping people build the skills they need. (And if you don’t know what people need to do, the content itself may be wrong or irrelevant.)
Many learning practitioners do not have sufficient knowledge about what they are training or what they are training to know and be able to do. They do not necessarily realize that they are training for the job. How do we create training when we don’t know what participants need? To do?
The purpose of learning is a clear statement of purpose: people must be able to do a certain thing with a certain result. We Need This information!
Think of it this way: Learning to write correctly is like GPS. When we input a location or address into a GPS, we tell the system where we need to go so that the system can get us there. The purpose of learning is to input a location or address. It provides clear information on how to get there (what and how to teach) and whether we have actually reached the right destination (assessment).
Can you tell our community more about your course “Write better multi-choice questions to evaluate learning” and who it was intended for?
Write better multiple-choice questions to assess learning A short but effective online course that looks at the skills needed to do a good job of evaluating whether the instruction has achieved the desired results.
Most people think they write acceptable multiple-choice questions. I did it until my mistakes were pointed out to me many, many years ago. I was humiliated and determined to improve my skills, and spent years reading research on how to do it. I am thrilled to be able to help others perform these critical skills well!
Research shows that authors of multiple-choice questions do not know what multiple-choice questions should be evaluated. As a result, many multiple choice questions measure the wrong things.
Writing well-written, valid multi-choice questions that measure achievement of learning objectives is an important skill for those who design instruction. The purpose of learning this course is to:
- Write well-written learning objectives to answer your multiple-choice questions.
- Analyze learning objectives and multi-choice questions that must be corrected to identify errors.
- Write relevant multi-choice items that measure the achievement of your learning objectives.
If you want more information, visit this page. If you have a team for training, you can contact us for a group discount or to set up this course specifically for your team. We offer discounts for 3+ team members enrolled in the course at the same time or for 10+ team members enrolled in a private team course.
You can join my email list by filling out the form on my site to get a discount notice (I never share it).
You have written numerous articles for the e-learning industry over the years. Do you have a specific article that you are most passionate about that you want to highlight for our readers?
I Love Written for the e-learning industry! The articles I write teach me a lot about what people in our field need most to prove, the impact of this proof. I love sharing my analysis with my colleagues.
I think a five-part article series on when and how to use asynchronous and synchronous learning tools is my favorite article. The articles are called (Correct) Learning Methods for Delivering Digital Learning: (Part 1-5). You can find them here.
I’m now working on a multi-part article series on digital video and enjoying it A lot. I should get the first article out soon!
Which one excites you the most about the future of learning and development?
I’m excited to see how AI and other technologies can help us learn and enhance our critical skills. Due to constant workplace changes, maintenance and increasing efficiency are becoming increasingly critical. In many ways, though, we are still teaching people and working using the 1940s method.
I have read about how the right obscure resources are needed now to make companies sustainable and competitive. This new insight comes from innovation, which usually comes from human work. Maybe technology can help us become better inventors! Increasingly, complex cancer diagnoses and treatments are coming from AI, added to human insights, for example. I wonder how such innovations will help us to learn and innovate in the workplace.
Unwrapping the wrapper
We appreciate Patty Shank for taking part in our thought leader Q&A and sharing her skills with us. If you would like to know more about Patti, please visit her author’s profile page, where you can find articles she has written for the e-learning industry, as well as her social media links.
 Nedeau-Cayo, R., Laughlin, D., Rus, L., & Hall, J. (2013). Evaluate the error of writing the item in the multiple choice question. Journal for Nurses in Professional Development, 29, 52-57.