For those of you who are following along, you will see that I have been making weekly posts about the content of the Patient Engagement Design MOOC. What you may not know is that I’m failing the assessments. I attribute this to two things: (1) the assessments are poorly designed, and (2) I don’t agree that the content is relevant.
My first gripe with the assessments is the use of “choose all” questions. These are questions like: “Why did Proctor and Gamble slash its survey and focus group approach? You may select more than one response.” The problem with this question is that if you get four out of five correct (or say 80%), you still get zero on that question. So, when this is given even weight as a multiple choice question, which is usually both better written and easier for someone who has watched the videos can get correct, the assessment results do not adequately represent the level of learning the participant has achieved. In general, “choose all” questions represent poor design. There is a reason that we don’t use them when we create official certification exams. Writing good assessment questions is a skill – and it shouldn’t be dismissed as something that easy and that anyone can do. Typically the goal in an assessment is to ensure that those who know the content pass and those who don’t can’t pass simply by guessing. There is actually a science to writing good assessments – Masters and PhD degrees are obtained in this field.
So, although I may be learning more than 70% of the content of the course (needed for the certificate of accomplishment), I’m only getting 33% on the assessments, because I can pretty much guarantee that I’ll make a mistake on the choose all questions. I go back to my previous statement, writing good assessment questions is not easy!
As a quick tip, choose all questions could be re-written as negative multiple choice questions (which also would not be permitted on certification exams, but are fairer way to balance the assessment, and allow the assessment to re-inforce the learning). Just ask “Which of the following is NOT a reason Proctor and Gamble slashed its survey budget?”.
The second reason that I am not doing so well is that I don’t agree with the content. Not that the content is wrong, just that I don’t see it as relevant in patient engagement design. When I fail to see relevance in it, I don’t pay as much attention to the details. I’m focused more on how I would change it to make it better – what approach I would take to make the content more relevant. In doing that, I don’t remember details like “Why did Proctor and Gamble slash its survey and focus group approach? You may select more than one response” or “Which of the following descriptors/analogies does David Eagleman employ to explain how the brain works? You may select multiple answers.”
So, why do I continue to participate when I don’t think the content is good? Partially because I like the outline of the course. I think the overall structure is good, just the choice of lectures isn’t. I participate in MOOCs to improve my knowledge of a topic, but also to help give me ideas on where I can delve further into different topics. The MOOC gives me ideas for blog posts. So, I may not be learning what the course designers want me to learn, I am still learning.
I was reflecting last night on the lost opportunity in this course. Part of what the course is about is “Design”, and yet the expertise of design has not really be brought into the course. I think if the course had been structured a little more around the “Design Thinking Action Lab” model (was offered on the NovoEd platform – by Leticia Britos Cavagnaroalso from Stanford – and yes, I did manage to get a statement of accomplishment for that MOOC), then it could have been very good. I think the course designers underestimate the power of what the participants could have brought to the topic (this is true in many of the xMOOCs – which largely use a knowledge dump pedagogy rather than a connectivist/socio-constructivist pedagogy). The course could have had participants talk about patient engagement – what it is and what it means to them (from that perspective week 1 was good), but then could have introduced the design thinking model, and then had people or teams use the design thinking model to share and explore a patient engagement problem. The results would have been very interesting, would have been participatory in nature, and participants would have learned a lot more about Patient Engagement Design! Perhaps a new course is in order – one on the NovoEd platform which better enables team collaboration – one that is really about Patient Engagement Design – and encourages participants to actually use a design as a way to explore the topic of Patient Engagement. This would also be a course that could qualify for CME credits, as participants would need to engage in the content, not simply memorize/recall (P.S. I’d love to collaborate on creating such a course!).
So, I will continue to follow along with the content – and I will continue to critique it – because it is through that critique that I personally develop a deeper meaning of Patient Engagement – I just wish I didn’t feel so much like I was talking to myself!