Tessa 0:03 — Great. Alright, perfect. So, I will go ahead and get started. Just a few things that I’d like to mention. First, this webinar is being recorded. So, you will have access to it, it will be sent over to you via email. You can always also access it through easygenerator.com/webinars. Also, you are all muted. And that’s because we are with quite a large group today. But if you’d like to ask questions as we go, then please add them to the Q&A. All of you are now introducing yourselves through the chat. You can use the chat to chat to others who have joined the webinar, but for questions that you would like us to answer, please use the Q&A. My colleague Jordan is on the call as well and he will be having a look at the Q&A throughout the session. And if there are any outstanding questions, then I will cover them at the end. So please use that Q&A for questions that you’d like answered from us.
So that being said, let’s get started with today’s webinar: how to boost e-learning interactivity by using questions. You should all be able to see my screen. So, this is what we’re going to go through today. I’m going to assume that most of you who’ve joined today’s webinar know already the basics of what Easygenerator is and how it works. If not send me over an email and I can invite you to a basic session where I explain how to actually create an e-learning. Today is a bit more on the didactic side of things – how to boost e-learning interactivity specifically by using questions.
Now what we will cover today is first of all, when and why to use questions. So, in what situations should you be using questions and for this I will be showing you a lot of different examples. Next, how to use questions. So, what are some of the best practices? How can you make sure that your questions are effective when you use them? And lastly, I’m going to give you an overview of all of the different question types that are available in the tool. So, this is what we will cover – it should take no longer than 45 minutes for all of it. So, let’s get started.
First of all, I’d like to start with a common idea – a common idea that I’ve heard a couple of different times from people who are creating e-learning courses, and they say, “you know what, I don’t need to add any questions to the e-learning because I’m not testing them. I just want to teach them the topic.” So, these are people who create an e-learning course, and they only choose to add images and texts and videos, but they make the conscious decision not to include questions. So, before I speak more about this, I actually want to ask you: what do you think about this? Do you agree? Or do you disagree? I’ve just sent over a poll to see what all of you think. I’ll give you a couple of seconds to answer this. And then then I will end it so that you can see what everyone else has answered. I see that 60 of you have already voted. Great, I see more votes coming in. The trend is a little bit what I had expected. I’ll give you five more seconds. Great.
So, I see that, actually, 91% – and you can probably see the results now – 91% of you do not agree with this. So, this is, as I said, the trend that I sort of expected. So funny to see that as well. Basically, in today’s session, I’m going to be sharing with you why you should be using questions in addition to just for testing purposes. So, probably most of you already agree. But hopefully by the end, everyone will agree. So, let’s get straight into it.
1.Stimulate higher order thinking
The first reason to be using questions – and that is to stimulate higher order thinking, basically to make people think more and think harder. Actually, yeah, if you have an e-learning course, having just content in there – so images and text and videos and PDFs – those are great ways to learn, right? But it’s quite an easy way to learn. Most people can read text. Most people can watch a video. Most people can look at a graph. But not everyone can answer a question correctly. So questions are always going to be more difficult than just content that is readily available. And it’s that difficulty that makes them think more actively about the topic. Right? So, it makes them think harder. It also makes them think on their own, right? Because the content – they’re just reading what you’ve written down. But when they’re answering a question, they have to think about it themselves. So, they are thinking independently.
And these things together ensure that they are going to remember it better. The harder you think about something, the more likely you are going to remember it afterwards. So just to make this a bit more concrete, let’s consider an example. I’m creating an e-learning course for my employees on how to correctly package in order ready for dispatch. I can give them a list of the 10 steps necessary to package this order correctly, ready for dispatch. And they can go through those steps. But it’s possible that they read those steps passively, because it doesn’t really require much more.
Same goes for a video. Maybe you include a video that shows them those 10 steps. Also quite nice, but they don’t have to process it so actively. However, when you follow that video (or that text) up with a question, asking them to put the steps in the correct order, then they have to think about it. They have to think about it actively and independently. And this means they will remember that process more accurately in the long run.
Now, what can this look like in practice, other than this example I’ve just provided? Here’s an example of a question that also stimulates higher order thinking. The question is a single choice question, and it states, “read the scenario, then answer the question. It’s Thursday, four o’clock, your team has a deadline at five o’clock tomorrow. You still need to work approximately five hours on it but you’re waiting for the input of a colleague. What would you do now?”
Now, this question stimulates higher order thinking. It puts the learner in the shoes of a manager or an employee. And they have to think, “Okay, in this situation, what would I do?” This is more difficult than just reading about what a manager should do, right? They actually have to think, and they have to make this managerial decision themselves in this moment. So, they will have a look at the answer options and make a decision based on them. Once they’ve selected one of them, let’s say they want to start working on their part of the project before receiving the input – the third option. They will click on that option and then they will also receive feedback. So actually, once they receive feedback for a question, and especially if it’s incorrect, they’re going to continue to actively process that information. So, the learning actually continues – even after they’ve answered it – through feedback.
I would say it’s not always a bad thing if they answer questions incorrectly because they can learn from it. Now, for those of you who are not sure how to add this feedback in the tool, you always have the possibility to add a question response. So that’s the thumbs up and the thumbs down. The thumbs up allows you to give feedback that is specific to the people who have answered it correctly. And then the thumbs down is the feedback you can provide to those who haven’t answered it correctly. If you want to be even more specific than this, you can also add explanations for each answer option. So instead of just having two different feedback responses, you can have as many as you have answer options.
So, for instance, here, where there are three different answer options, you can provide three different feedback responses to be even more specific. This, of course, does take a tiny bit more time for you, but the impact is much greater because you’re giving very specific feedback that is relevant for them in that moment.
So that’s the first reason to use questions – to make them think harder. To stimulate higher order thinking.
2. To model hands-on experiences
Second reason to be using questions is to model hands-on experiences – to make things more practical. Now, sometimes in the online environment, this can seem more challenging. I think in the in the face-to-face world we are all familiar, for example, with role plays and those things that offer a hands-on experience, right? With online learning, you may sometimes think that’s a little bit more difficult. While actually with questions, you can model hands-on experiences. You can provide practical or practice, really, by using questions. And I’ll show you an example of how this can be done.
This is the ranking text question. And the question states, “rank the following job applicants from best to worst fit for a team lead position (team of 10).” So, in this question, there are five anonymous candidates with different years of experience and motivation. And so here the learner needs to actually make a decision which job applicants are best or worst fit. So, they are ranking the applicants. Now this question is practically relevant. It offers them hands-on experience.
In fact, it’s also going to be much more memorable because of what we call context-dependent memory. In this question, you are setting the scene, you’re putting them in the in the scenario of having to select the best job applicant – something that they’re probably going to do in their day-to-day job. So, the similarity of these two contexts – the context of this question and the context of their actual jobs that they’re going to have to do – because they are so similar, when they’re going to be doing it in their job, they’re going to remember what they learned in the e-learning.
So, if you make the questions as practically similar to the real-life situation, then they’re going to remember it better. Alright, so actually, this is an example of a roleplay, except in e-learning courses. So, it definitely is possible when you make use of questions.
3. To spark interest
Another reason to be using questions – and this one is slightly different than the two previous – is to spark interest. You can use questions to get learners slightly more interested. Now, to do this, you can use questions in a very different way. So, in this case, you are actually using questions before you even present the content. And you may be wondering, “well, that’s really strange. Why would I be asking them a question that they actually cannot answer yet because they haven’t learned it yet?”
Well, that’s exactly what sparks interest. They’re not expecting that question in that moment, so that already gets them more interested. Then, of course, they’re going to try to answer the question. They’re going to make a guess. And once they’ve made that guess, they’re going to want to find the answer, because they’re hoping that they were right. So, it gets them more interested in the content that is to follow.
Now, what does this look like and how do you set it up? Here’s an example of an awareness question. And it’s a survey question. I’ll explain more later. The question states, “which of the following are true? Don’t worry, this question is not graded. In the next few pages, you will discover what the answers are.” And there are three statements, and they need to decide, you know, which one is true, and which one is false. But they haven’t learned it yet.
So, as I said, they’re going to try to answer – they’re going to make a guess. And once they’ve submitted their answer, it will not have any impact on their progress. They will not be graded; they will not receive a score for it. Because it’s basically just there to make it more interactive, right? So, they’re going to answer that question, and then they’re going to be interested in finding out if they were right. Everyone wants to be right. So actually, they’re going to be interested in finding that information afterwards, which makes them more interested and more ready for the information that will follow.
Now, when you create a question like this, make sure to turn on the survey mode. The survey mode – and that’s the option you can see at the top of the page – this will ensure that they do not receive any scores, right? You don’t want to penalize a learner for answering a question that they don’t yet know what the answer is to right. So just make sure that you have survey mode turned on. And that allows you to use a question for this reason.
4. To allow learners to self-assess
Next reason to be using questions is to allow learners to self-assess. This is also a very important one. Of course, you know, content is a great way to learn. You can read a text, you can watch a video, you can look at a graph, you can look at a document, and you can learn from it.
But if you don’t have questions, then you’re not really having the opportunity to check if you have understood everything. And most of us – maybe this is something that you recognize yourself – most of us when we learn something, we are quite confident that we have understood everything and that we will remember everything. But in reality, that’s not always the case.
When we read something or when we watch a video, we are probably not understanding absolutely everything. Or we are also maybe forgetting some things. So, in reality, all learners need a moment for self-assessment in order for successful learning to take place. This is also going to ensure that they have an accurate sense of confidence, right? So, it’s a bit like a checkpoint. They go through the content, and they have a question where they can self-assess, and it works as a check point. If they have it correct, they can be confident that they’ve understood, and they can then move on to the next.
Now, what can this look like? There are plenty of examples of what this can look like. This is one of them. This is the drag-and-drop text. Now with this one, there’s an image. And you can see in this case, it’s an image of a laptop. And with the drag-and-drop text question, you have different labels – different items – and they have to drag and drop it on the correct area of the image. Now, actually, self-assessment takes place at multiple moments. It happens already immediately when they see the question. Whenever you are taking an e-learning, and you directly see a question, you’re going to have an immediate reaction. That immediate reaction could be, for example, “oh, this one’s easy,” or “this one’s quite challenging, maybe I should have paid more attention to the previous page.”
Right? So, that immediate reaction is already, for them, an element of self-assessment. Then they’re going to start dragging and dropping these items. If they struggle, once again, that’s a moment of self-assessment. Now, if a learner struggles quite a lot, they will probably go back because they do not want to be wrong. And that’s a good thing, right? Because it means that they will review what they’ve just learned. Come back, try again. And of course, once they’ve answered it, and especially if you’ve included feedback, they will receive that feedback so that they can either move on or go back. So self-assessment is a really important part of the learning process, you know? We need to find out what we don’t know in order to learn it. And self-assessment happens, primarily by using questions.
5. To gather insights
Next one: you can use questions to gather insights, gather information, gather results. In general, if you’ve created an e-learning course that only has content – let’s imagine it has some text, some bullet points, couple of paragraphs, it has maybe a video, it has probably a few images, it has a graph – and that’s your e-learning course. And this is something that you publish and you, for example, check the results and you can see everyone who has followed it, who has taken it. On the other hand, if you have an e-learning course that has all of those things, plus it has questions in there, it’s much more informative for you to know that somebody has taken that e-learning course and has been able to answer all of those questions correctly. It’s much more informative to know that than just to know that they have had a look at all of the content that’s in there. Right? And that’s also because those questions are more challenging, as I said.
You can also gather other kinds of insights, other kinds of information. For example, you can find out how long did it take for them to complete it on average? Let’s say you’ve got a group of 50 people who are taking your e-learning. You can check, for example, how long on average does it take for them to go from the beginning to the end of the e-learning course? Maybe you will see, hey, it takes them approximately 37 minutes. And perhaps you had anticipated it would take them 25 minutes.
With that, with those insights, you may draw conclusions, for example, that maybe you have added a little bit too much content. Or maybe it was simply just more challenging than you anticipated. These are the insights that you can take with you either to improve your course or to provide further support for the group.
6. To gather individual input
Now, you can also use questions to gather individual input, and when you are going to create questions together individual input, most times it’s using the open question. Open question is exactly what it sounds like. It’s a question that you ask and the answer is open. So, they can just type their answer directly. This means you can gather individual input. So, you can get to know more about your audience. Because it’s not just a couple of answers they can choose from, you’re actually asking for their opinion. This can be used for surveys, but it can also be used to ask, you know, how do you do things? What are your top three indicators of cultural fit? You can then take this information to perhaps create follow up training, for example.
So as a short summary of when and why to use questions, as you can see, there are a couple of different reasons to use questions. Quite a few reasons to use questions. For the learners, using questions makes something harder – so stimulates higher order thinking – and this makes remember things better over the long-term. Questions also model hands-on experiences – makes things more practical for them. It also can increase their interest – make them more interested in the content. And it allows them also to self-assess – to check for themselves, “have I understood?”
For you, it allows you to gather insights, gather information results, and individual input. So, at this stage, I’d like to go back to that common idea that I shared at the beginning, the one that 91% of you disagreed with. And that’s common idea was, you know, “I don’t need to add any questions to the e-learning because I’m not testing them, I just want to teach them the topic.” Well, this graph here – this table maybe – shows you exactly why that is a misconception. Because actually, questions are very important for learners, and not just for the creator.
So, for all of these reasons that you can see, on the left-hand side, questions are important to improve the experience of the learners – to improve learning success. So, questions are not there just for testing purposes, they’re there for many different purposes.
Of course, in the end, benefits for the learner are probably benefits for you. It’s usually in everyone’s best interest that successful learning takes place. And that’s more likely to happen if you do include questions.
So now that I have hopefully convinced you that you should be using questions at all, I’d now like to share some tips and common practices on how to use questions well – how to use questions effectively.
1.Align key questions to learning objectives
Now, the first tip is to align key questions to learning objectives. So, I’m zooming out here a little bit. Typically, when you create an e-learning course, you always have a goal. At least I hope you have a goal. If can think of your goal, try to think about the reason why you’ve decided to create an e-learning in the first place. That’s usually stemming from your goal. Now, the goal can sometimes be quite broad, it can sometimes not be really concrete, it can be quite broad. So, you build learning objectives to make that goal more concrete and approachable. So, let’s say you have a goal. And if your learners reach three different learning objectives, they’re actually getting closer to reaching the overall goal. It’s important when you have this set up that you have questions that assess the learning objectives, especially for self-assessments.
Let’s imagine you’ve got two questions per learning objective – two questions that actually assess whether the learning objective has been reached. If you have that in there, and a learner is able to answer them correctly, then they can be confident that they’ve reached those learning objectives, which means that they can be confident that they can reach the goal. So, it’s important for the learner – for the person taking the training – that they have questions that will assess whether they’ve reached that learning objective.
Confidence is also a key factor for success, right? So, you want to make them feel confident that they can reach the goal. If you include questions that assess their objectives, then they can do that. That, of course, above and beyond those questions, you can use questions for other reasons. For example, sparking interest. But just make sure that you at least have those key questions in there as well.
2. Provide meaningful feedback
Next, and I think I’ve mentioned this one a couple of times, provide meaningful feedback with whichever question you would like to use. With all of them, you can always include feedback. You have, as I said, the feedback for correct answer – that’s the thumbs up – feedback for the incorrect answer – that’s the thumbs down. And if you want to be even more specific, you can add explanations for each answer option. I would always recommend adding feedback. And my recommendation is also to focus your energy on adding feedback for incorrect answers, because that is typically where you can make more impact.
The feedback for an incorrect answer is more impactful because that’s where learners actually need a little bit more help, need a bit of support, need a bit of direction. So that’s where more impact can be made for feedback. For a correct answer, of course, it’s always nice to congratulate them. Let them know that they’re doing a good job and perhaps give them some more information. But my recommendation: focus on feedback for incorrect answers.
3. Develop questions that mirror real-life situations
Then develop questions that mirror real life situations and applications. On the screen here you can see two different questions. The first question asks them to rank the following criteria from the most to least important for hiring a team lead. And the second question asks them to rank the following applicants from most to the least fitted for a team lead position. Now, these two questions actually rely on exactly the same knowledge and information. However, the one on the left-hand side is theoretical, the one on the right-hand side is applied. In reality, the person who takes this training is probably never going to have to rank criteria ever again, they’re going to have to rank applicants in their day-to-day job. So, creating the question that is on the right-hand side is much more impactful. They will remember this one much more than ranking criteria.
So, although that sometimes does take a little bit more creativity on your behalf, I always recommend adding questions that are application-based, that actually mirror real life situations, as much as possible.
4. Consider the configuration
Then consider also the configuration. Whenever you include questions – which hopefully you are now convinced that you should be doing – under the third step, the configuration, you’ve got a couple of different settings that you should consider. These are the settings that you can see on the screen. Now, it’s important to consider the configuration because it will affect the way that your questions are working. One of the really important settings here, that is actually a little bit cropped, I can see on the screen here – it’s the one on the very bottom on the left, the mastery score.
Under this setting, you can add the score that should be reached in order for them to pass. This is an important one because it determines how easy or how difficult it is to pass. And the pass, of course, depends on the questions that you have in there. So, it’s up to you to decide do they need to answer each and every question correctly? Or is it okay if they answer, for example, 75% of those questions correctly? So, these are the settings that you should probably consider when you are including questions.
Now my next advice has to do with using different question types. But before I go more into this, I’m curious from you – which question type do you use most often? Now, some of you may be completely new to Easygenerator, and in that case, you probably don’t use any of these – yet, at least. But some of you probably have already created questions. I’m curious from you – which question you use most often? So, I’ve just ran another poll to ask you which question you use most often. Once again, I will share the results at the end. You can only vote for one of them. So I’m just asking, you know, which one do you use, really, the most of them? I’ll give you a couple more seconds. I can see two that are relatively popular. A few more seconds here. Yeah, one of them has definitely the majority. Alright, so I’m going to end the poll in a few seconds. Yes.
So, I see that the majority of you – 50% of you at least – who have answered use the multiple-choice question. The next big winner is the single choice question. Now, this may be a surprise to some of you. And probably for some of you. It’s not that big of a surprise. Many of the other questions are less often used. So that’s really interesting to see. I will now share with you what we know from everyone who uses Easygenerator. It’s a similar trend, I would say not completely. So single choice, and true-or-false are most often used. And for those of you selected single choice, then this is definitely something that you can recognize.
But the overall trend is that the other question types are much less often used and that is something that was really reflected in the poll today as well. I find this always interesting to know. The tip that has to do with this is to include, different and difficult question types. Different question types just means that you use a variety. It’s always best to use a variety because you keep things interesting for them.
So maybe they’re expecting to receive those single choice questions and they’re not at all expecting to receive a hotspot question. And that makes the hotspot question actually more interesting for them. So, it’s useful to use different question types, but also use difficult question types. Inherently, some of these question types such as the single choice question is easier? Of course, it depends on the topic. But with the single choice question, there are a series of answer options, and only one of them can be correct. Already with the multiple-choice question, which I know was popular with the group as well, there are multiple options and multiple of them can be correct. This means that there are multiple combinations possible, making this question, inherently, slightly more challenging.
Now, it’s important to include challenging question types because it makes them think harder, right? So, it stimulates higher order thinking. That was the first tip that I shared today. You want your questions to be challenging. If they’re too easy, they’re not going to be very impactful. So, consider using question types that they may be less used to, in question types that are inherently slightly more difficult. Multiple-choice question is a good difficult one. So is text ranking, text matching as well. So just consider using that variety of questions.
Also, an important distinction is between recognition questions and retrieval questions. Recognition questions refer to questions where the answer is on the screen. It’s there and they have to find it. Retrieval questions are different because retrieval questions are questions where the answer is not on the screen. It’s actually in your head and you need to retrieve it. Inherently, recognition questions are always going to be easier than retrieval questions, because the answer is there. So, consider also this difference and consider using for example, fill-in-the-blanks or consider using open questions if you want to include retrieval type questions.
5. Improve your questions
And that brings us already to the last tip. The last tip is to improve your questions. E-learning courses are not static, you can improve them after you’ve created it, especially if it’s the first time you’re ever creating e-learning courses. Don’t worry, right? You can always improve the e-learning courses and the questions that you create. As you can see here in the results, besides the insights that I showed you, which shows you know how many people have taken the course, how many attempts they’ve had, in general, and how many minutes they’ve spent on it, at the bottom, you can see what we call question analytics. The question analytics will show you how often each question is answered correctly on the first attempt. This means that you get to see which of your questions are, on average, considered easier, and which ones are, on average, more difficult.
This is very useful, actually, for two reasons, I would say. The first is that you get a clear impression of how people are doing. Let’s say you’ve got three sections in your e-learning. And you notice that for the third section, most people struggled on those questions. That can help you determine, you know, “maybe I need to help them, support them, give them follow up training on this specific topic, because it seems to be more difficult for them.”
So it can help you actually identify additional training or support needs. Second reason why this is useful is because you can improve your questions. A question that is too easy is not going to be very effective because it doesn’t make them think very hard. A question that is too difficult is also not going to be very effective, because it’s too challenging. It may cause frustration, for example. The ideal situation is that your questions are just challenging enough. That’s the sweet spot let’s say. So actually you can use these questions analytics to make easy questions more challenging, and the opposite for questions that are too difficult.
Easygenerator will also give you a couple of tips when you click on a question that is too easy, it will give you advice on how to change it, what to look for, how to improve the quality of this question. Of course, this is only something you can do once you’ve already published it. And that means that you can improve this e-learning course for maybe the next cohort, or to improve the way you build questions in general, for your second, third, fourth e-learning course.
10 question types
So those were the tips on how to actually use questions effectively. And that brings me to the same screen that I showed earlier, the 10 question types, to round off. Today, I’ve shown you examples of six of these question types if I see that correctly, but we do have 10 of them. So of course, the single choice. The single choice image is very similar to the single choice, except the answers are images and not text.
The text matching is one where you have to match one item from the left to one item on the right. Statement question is true-or-false. I’ve shown you an example of that. The same goes for open question. Basically, they just have to type their answer in. Multiple choice questions, one that as most of you know was a very popular one. So, this means of course, that there are multiple options, and multiple of them can also be correct.
With fill-in-the-blanks. There are two versions, let’s say. You can either have them type out their answer, or you can have them select an answer from the drop-down menu. With this question, you can also just create a story, a paragraph, or a table and just add some blanks here and there that they should fill in or select from the drop down menu. Drag-and-drop text – I showed you an example of that. That was the image of the laptop where you can drag-and-drop a label on top of each area of the image.
Hotspots questions is one that I didn’t show you an example of. For this one, you have an image that you can add. You can ask your question based on that image, and they will have to click on the correct area of the image. So, consider for instance, I add an image of a map. And I asked them, “where is our headquarter located?” They are going to have to click on the correct area of the map – on the correct country or on the correct city. And that will determine whether they have the answer correct or not. You can also have multiple correct answers. So, you could ask “where are our three offices located”, and they will have to click on the three countries or cities that the office is located.
So basically, your answer options are on top of an image of your choice. And the last one – ranking text. This was the one where I showed you an example of job applicants. So basically, you’ve got different items. And they have to drag them or rank them in the correct order, which can be an order of your choice. It can be something about chronological order. It can be order of anything, right? You determine the criteria of this ranking.
So those are basically all of the question types available in the tool. And if you’re wondering specifically on how to use one or you’re struggling to use one, bear in mind, we do have at the bottom of Easygenerator – at the bottom right – a green button. Click on that green button and get in touch with us. If there’s anything you want to know more about one of these question types, how to use it, if you need help, just click on that button and send us a message, and we are more than happy to help you. So, we can kind of look along with you help you out with the creation of your questions. If you’re ever stuck with this, just get in touch and we will get back to you as soon as possible.
And that concludes today’s webinar – a little bit early. So, what I’d like to address now are any outstanding questions? And, also, I’m here to ask you if you can give me some feedback on today’s webinar on whether it was useful for you. So, Jordan is sending out a link right now in the chat. It’s there. So, click on that link, and through this link, you can provide some feedback.
And Jordan, maybe I can ask you as well, at this stage. Are there any outstanding questions that I should address live?
Jordan 39:52 — No, I’ve answered all the questions in the Q&A. Yeah.
Tessa 39:34 — Great, then that concludes the session. Thank you very much in advance for completing the form as well, the survey. I hope this session was useful. Thanks also to those of you who are tuning in through LinkedIn. And yeah, I would say see you next time at the next webinar.