Part 1: Epistemological Mismatch – Why you shouldn’t do Educational Design Research (EDR/DBR)

There is a lot of literature out there now, and some great examples, and even a textbook, of how to do Educational Design Research (EDR), otherwise known as Design-Based Research (DBR) as a PhD Study. In this series of blog posts, I’m going to talk to you about why you should not be doing EDR. My intention is not to persuade you against using the methodology, but rather, to help you uncover the pitfalls you might be facing, so that if you do choose to do an EDR study as a PhD project, then you are better prepared to both defend your decision and to succeed.

In this series of articles, I encourage you to engage in the discussion, either through your own blog or in the discussions of these posts. I do not intend to provide the only answer to any of the questions that arise throughout these posts. I welcome challenge and discussion, so that together we can develop deeper insight into this methodology. Please, if you blog, post a link to your blog in the discussion thread. I would love to hear your feedback, your ideas, and your justifications for using EDR. These discussions will help myself and others better understand EDR methodology. I will also create a Facebook group for these discussion (Educational Design Research Discussion), as I find that some people prefer that medium. You can join it by clicking here.

On the topic of Epistemological Mismatch, I will first recommend that you read an article that I wrote on the Epistemological Foundations of Educational Design Research. I wrote this back in 2011, before there was a textbook on EDR. I analyzed all the papers I could find in order to develop a deeper appreciation for what EDR was all about. Please let me know if you cannot access the link below. I’m happy to email a copy of the article to you.
Hogue, R. J. (2013, October). Epistemological Foundations of Educational Design Research. In World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education (Vol. 2013, No. 1, pp. 1915-1922).

What I have seen is that people first see the ‘process’ of EDR, specifically the Reeves’ model (see image) and think that EDR is the right methodology. At first glance, the process can look appealing and look like a good match for an evaluation study or a case study, however, the process isn’t what makes a study an EDR study. When there is an epistemological mismatch and I ask the student about their research question, they usually respond with wanting to developing an understanding of how students learn something. The focus of the research is on student learning rather than on design of an educational intervention. In many cases, these students are looking at knowledge from a constructivist worldview rather than a pragmatic one. The focus of their research is on exploring or describing how learning occurs. EDR is intended to solve a ‘design’ problem, and understanding learning is not a design problem. Students in this situation have an epistemological mismatch – they are looking at what constitutes as knowledge from completely different perspectives.

Reeves

 Reeves’ Design-Based Research approach for educational technology research. Adapted from (Reeves, 2006, p. 59).

EDR hinges around the word ‘Design’, which is where non-pragmatic educational researchers struggle – especially if they are not familiar with any of the design sciences. Design is also where there is a large overlap between what is ‘research’ and what is ‘practice’. I have often had to defend ‘design’ as a form of inquiry in-and-of-itself. I have been asked “where is the research?” and “where is the new knowledge?”

From an EDR perspective, what is important here is that by actually building something, you are learning about the thing you are building. I love this Sophocles quote as I feel it summarizes EDR well, “one must learn by doing the thing, for though you think you know it, you have no certainty until you try”.  New knowledge is being generated through the act of building the thing. So, each time we build something (or design something), we are creating new knowledge. This is a very pragmatic view of what constitutes new knowledge – and is not something that is easily understood if your worldview is not pragmatic.  I have had many friends and even committee members struggle to find the “new knowledge” in my proposed dissertation project, because in part, they do not see the ‘design’ and ‘creation’ of an educational resource as research – rather, they see it as practice. They are looking to answer the questions about ‘how learning occurs’ rather than ‘how can we build an effective learning intervention’. The questions are very different – this can be a huge hurdle. If your university does not support the pragmatic view of ‘design’ as research, you may find yourself constantly having to justify what you are doing as research.

To help determine if what you are wanting to do is EDR, you need to go back to your research question. Are you trying to ‘understand learning’ or are you trying to ‘learn through creating’? If your question is to ‘understand learning’, then you should not be doing EDR. EDR is not intended to answer that type of question.

Discussion Questions:

Now that you’ve read my post on epistemological mismatch, I’d like you to reflect on the following questions. Please feel free to leave a comment on this post or post in your own blog (please leave a comment with a link so I can read it). Thanks.

  • What is your research question? Is it is a ‘design question’?
  • Do enough academics at your institution appreciate ‘design’ as research?
  • How will you defend your study to researchers who don’t see ‘design’ as research?
  • How will you differentiate research from practice?

 

 

4 Comments on Part 1: Epistemological Mismatch – Why you shouldn’t do Educational Design Research (EDR/DBR)

  1. Thank you for your thoughtful post. I think these kinds of discussions can be quite valuable for graduate students and their faculty supervisors as well. There are a couple things I’d like to clarify. First, Susan McKenney and I (2012) (authors of the textbook mentioned at the beginning of your post) do not describe EDR (or DBR) as a methodology, but as a “genre” of educational research in which the iterative development of solutions to tough educational problems provides the setting for rigorous scientific inquiry. EDR is defined more by its goals than the specific methods of data collection and analysis. Bereiter ( 2002 ) said it well, writing, “Design research is not defined by its methods but by the goals of those who pursue it . Design research is constituted within communities of practice that have certain characteristics of innovativeness, responsiveness to evidence, connectivity to basic science, and dedication to continual improvement” (p. 321).

    For us, the goals of EDR are always twofold: solving significant real world problems while at the same time seeking to discover new knowledge that can inform the work of others facing similar problems. While pursuing both goals simultaneously is a defining feature of EDR, one goal may feature more prominently than the other.

    For example, with respect to the goal of improving practice, EDR may be conducted primarily to solve a problem (e.g., increase the interest of women in pursuing engineering as a career). On the other hand, relating more to the goal of enhancing the quality of previous research findings and/or generating new theory, EDR may be conducted primarily to generate new knowledge (e.g., develop or refine a theory of game-based learning).

    So with these differences of goals and emphases in mind, I would like to argue that doctoral students interested in EDR may come at it from a variety of angles and motivations. Some may be quite pragmatic and place the development of an intervention that solves a real world problem above all else. But if that is all they do, they would not be doing EDR, but something more akin to action research. Others may view the generation of new knowledge as paramount, and may see the design and refinement of an intervention as primarily a vehicle for enabling their research to be conducted. If that is all they do and they place no emphasis on solving a practical problem, they would again not be doing EDR, but some form of interpretivist or hypothesis-testing research. EDR requires the dual focus on the goals of solving practical problems related to teaching, learning, and performance while at the same time seeking to identify new knowledge, most often in the form of reusable design principles. Quoting one of the most recent pieces that Susan McKenney and I have published about EDR (2014), “Working systematically and simultaneously toward these dual goals may be considered the most defining feature of educational design research” (p. 133).

    I know that EDR is viewed as legitimate doctoral level research at many of the foremost universities around the world, but I recognize that there are also universities and programs where a lack of understanding of this genre of educational research makes getting EDR approved as a dissertation prospectus more challenging. This is one reason why I am spending my so-called “retirement” years traveling wherever I am invited to promote EDR. I firmly believe that EDR is an innovative and exceptionally promising approach to improving the quality and impact of educational research across the board, and I strongly encourage doctoral students to consider it with two major caveats. First, you must find a supervisor and/or form a doctoral committee that understands EDR and views it as a legitimate genre of educational inquiry, and second, you must be prepared to work closely with practitioners who “own” the problem you are addressing.

    Bereiter, C. (2002). Design research for sustained innovation. Cognitive Studies, 9(3), 321–327.

    McKenney, S. E., & Reeves, T. C. (2012). Conducting educational design research. New York: Routledge.

    McKenney, S. E., & Reeves, T. C. (2014). Educational design research. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (4th ed.) (pp. 131-140). New York: Springer.

2 Trackbacks & Pingbacks

  1. Part 2: There isn’t a true need – Why you shouldn’t do Educational Design Research (EDR/DBR) | Rebecca J. Hogue
  2. Why you should do design based research (DBR) | Virtual Cancuck

Leave a Reply