Final Analysis

During my class this semester, I feel like I have gained an understanding of and appreciation for Educational Design Research (EDR). while I still find it similar to action research, it is clearly much more involved and rigorous. As I preview educational programs in my role as site administrator, I will be keeping EDR in mind, and looking for evidence that the program I’m reviewing has gone through an iterative process that used a variety of data to inform the final product.

I’m unlikely to use EDR for my dissertation. As an administrator in a school district, I’m in an awkward power position with practitioners. In addition to the time commitment that I think EDR takes to “do it right”, there is an element of embedded access that I find problematic when looking at my personal goals for a dissertation completion schedule. Conducting multiple iterations during a single school year requires one to be very closely linked to the research situation, and it’s both impractical and unethical to conduct this sort of research at my school site with teachers that I evaluate! While I have an appreciation for EDR/DBR as a research methodology, I don’t see it as being a practical choice for my dissertation.

 I believe that peer review is a very powerful tool. As a recipient of peer feedback, I tend to quickly scan for things I agree with or recognize as easy corrections. I then go back and think through the revisions suggested by reviewers, and either keep them if I think they require more thought, or delete them if I feel like the suggestion is misguided or answered elsewhere. For the most part, I find that the comments are thoughtful and fairly accurate, and I very much appreciate having another set of eyes on my work. When I provide peer feedback, first and foremost I enjoy reading what other students are learning. I am picky about whose work to review, looking for those that match my background or work situation, at least in some way. There have been few peer review activities that haven’t taught me something of value, often outside of the topic of the class! When adding comments, I think carefully about my choice of words, since I know and respect the others in my cohort and don’t want to hurt anyone’s feelings. But I’m honest as well, again because I respect my colleagues and want my feedback to be meaningful. I also tend to double check my technical suggestions - I’m more likely to verify in the APA style manual when correcting someone else than when I’m doing my own writing!

Peer review does require a bit of trust, and a bit of knowing each others' style. In a class that contains a majority of students who have been together for 3 years, mixed with 2 newcomers who do not have the same history and are not at the same point in their educational career, there were some challenges. I suspect all of us tried to be inclusive, but it was more difficult to relate in some ways. It's a lesson I will keep in mind with my own students.

It's Not "Whether" It Works, But "How"

As my understanding of design-based research (DBR) grows, so does my appreciation of how well it fits my philosophy of teaching and learning. The readings in this module helped my understand that "design" in DBR is used in 2 different ways - the researcher creates (designs) an intervention or learning phenomenon, and then collects data that allows them to create a model (design) or guidelines (design principles) that can be used to generalize the intervention into other environments. I think this is how good teaching works, when the teacher has sufficient time and ability to collect the data needed. It s a more rigorous version of piloting an intervention or program, and then making changes to the program based on what actually happens in the classroom. This is how I've designed model lessons in the past, and it's how my district has developed units of study that are disseminated across schools.

licensed image by PresenterMedia

Data collection is the challenge in any type of research. Dr. William Sandoval notes that there is a tendency among researchers, particularly novice ones, to collect everything possible and then try to figure out what is needed later. Since there is as much a need for thick description in DBR as there is in case study research, this can end up being a huge amount of data! Sandoval says that, instead, a researcher should have a clear plan of data collection, and should have a reason for collecting every piece of data that is collected. While that makes perfect sense, I think it probably takes a fair amount of experience to know what data will be relevant and which will be extraneous. My fear, as I'm sure is true for most novice researchers, is that I will begin writing my results and realize there's a gap in my data! I'm not sure how one overcomes the challenge of too much data, though I suspect that working in close communication with experienced researchers who can make recommendations.

More about DBR

As I continue my course in design-based research (DBR), I still struggle with how it might actually fit into the repertoire of doctoral students. The iterative nature of DBR seems to take quite a long time, and might not have a fixed end point. It seems difficult to predict the exact number of iterations it will take to get to generalizable design principles. One thought I had is about the possibility of doing an informal version of DBR as a school site leader; the team of researcher/practitioners would be the teachers in a grade level, and their PLC meetings would be the format for hypothesizing design principles and determining how to test those principles. Minor iterations would probably occur every 2-3 weeks, and the teachers implement and revise. While I’m sure they wouldn’t consider their conclusions to be design principles, I think the strategies and recommended practices teams come up with might indeed fall into that category.

 In the readings over the past month, Joseph (2004) helped me to better understand the ways in which other research approaches also study real-world learning situations, and the difference in philosophies that might make a researcher select design-based research. It seems that it’s all about the outcome; if a researcher wants design principles, they might select DBR. If they want to simply examine a phenomenon, or determine the effectiveness of a strategy without necessarily modifying the design, they would likely choose another research approach. Obrenović (2011) describes a process of DBR that I found very similar to what we learned in our Project Management class, but also talks about how DBR might use a selection of quantitative and qualitative approaches in the various stages of design in order to inform changes. As I read Anderson and Shattuck (2012), I began to wonder if the Response to Intervention (RtI) programs that we use in my district were created using design-based research. I know the interventions have been extensively tested, and I know they all have significant bodies of research about their reliability and validity, but I wonder about their genesis. As I reflected on the issues Anderson and Shattuck raise about researchers who are also the designers and implementers, it makes me think that programs such as Read 180 were probably developed by one group of people, and then validated by others. I don’t know that’s the case, but I would predict that getting a reliable rating from the What Works Clearinghouse probably precludes the designer being the researcher.

Design-Based Research

I am currently taking a class in design-based research (DBR), which holds great appeal for me. In design-based research, the researcher proposes a strategy, tool, process or curriculum (the design), and then tests and refines it within a real-world context. Through repeating iterations, the researcher is able to refine the design, and then hopefully generate some principles or theories about best practice. To me, it seems that this is the way interventions and curriculum should be designed. Sometimes programs adopted in schools are "research-based" but they have never been tested with diverse groups of 34 kids with a single teacher who is learning the program on the fly. Unfortunately, that's the implementation reality for most K-12 public schools. Sometimes interventions and curriculum make that transition to the real world, and sometimes they do not. I believe that a rigorous design-based research process would probably make programs more capable of being implemented by real teachers in real schools.
I am still a little confused about the different possible outcomes of DBR. In general, the intent is to develop a set of design principles. I am not clear on exactly what a design principle is, and how well it will correlate to other situations. In many of the DBR studies I reviewed or read reviews of, it seems that the outcome was a very narrow set of guidelines that were applicable to that particular intervention or program. I can imagine a study in which broad design principles are generated, but I would think it would take several years in several contexts in order to create generalizable principles.

I created the flowchart concept map above to show my understanding of design-based research. As with any research, the first step is to determine what the topic is to be studied, and determine what prior research has been conducted. In DBR, the next phase is design, which is followed by implementation. During implementation, data is collected and analyzed, and then the design is "tweaked" to make it stronger. The design is implemented again until it is a perfect solution, or the researcher has either completed enough iterations to develop design principles or has run out of time or money! The process is complete when the researcher publishes their findings in the form of a generalizable theory or design principles. Although the wording on my concept map in the redesign phase is a little tongue in cheek, I suspect those are the actual questions a researcher asks of themselves as they progress through the process.

The Right Tool for the Right Job - Part II

Approaching the end of the year and thinking about the future, I am evaluating my technology yet again, and planning for the future. I've found that my iPad is absolutely my "go to" device, and I carry it around campus before school and during classroom walkthroughs. Although at first people were overly interested and/or intimidated by seeing me arrive with a tablet, I find that teachers (and students) no longer pay attention to it.

In Evernote, my current schema is to start a new note every day called Admin Log, which begins with a list of the things I need to do for the day. I usually start by copying the prior days' notes, so it's got all of the things I didn't get to previously. It includes both personal and professional items. I use tags for the school site I'm at, and for the grade level(s) I observe. It is very easy to sort by tags and see all of the 2nd grade notes, comments, and images. Some of my daily notes end up being quite long, particularly if I take several pictures, and I don't think the search feature in Evernote desktop works very well. Those factors make it difficult to find the right note or the right information within my notes.

For next year, I'm planning to organize my notes into more logical notebooks that are more easily searchable. After a conversation with D, I'm thinking about having separate notebooks for each teacher, so I can easily get back and see the observations I've done and photos I've taken. I can also attach PDFs and send emails to a notebook, so that makes it a one-stop shop for all of the data I'll need for evaluation and supervision. I'll have a different notebook for facilities, and others as I see the need. Parent contacts and discipline are a couple of things I'm still noodling about, as I'm not sure how useful it is to have that information solely within my account. Functionally, it's possible that the old fashioned binder is still the best way to deal with those items.

I'm still not sure how to deal with my to do list. Wunderlist just doesn't "speak" to me, and I find it quite cumbersome to use (Sorry Brian!). I'd kind of prefer to have everything together, but Evernote isn't really a very good tool for tracking longer projects. On this side, I'll have to just keep playing around with it to find a system that works.

Good thing that summer is coming up so I can get all this organized!

The Right Tool for the Right Job

Image licensed from PresenterMedia

Now that I'm several months into my new position, I'm starting to take charge of my own organization, and customizing the work flows and reports expected by my supervisors to fit better with my style. As I read more by school site leaders, I'm also starting to incorporate suggestions that seem to me to be a match to my goals and expectations for myself. I decided the best approach to using any given tool is to just jump in and use it, so here's what I've been using to improve my administration and management:

iPad
I am now carrying my iPad everywhere in  my rounds on campus. So many times over the past 5 months carrying a clipboard I've wanted to capture and image with a note - now I can do that. I've already decided that my iPad and cover is pretty heavy, so I'm looking at other options that will make it more portable and easier to use.

Evernote
I am starting to use Evernote as a daily administrative log, based on a post by J. Robinson in the 21st Century Principal blog. I have created a little template for observations that matches what one of my principals expects, and I paste in that text for every classroom I visit. I can check off boxes for what I see, and add my notes. I'm still transferring it to a hand-written form afterwards, but since that allows me to clarify my thoughts and add constructive suggestions, I'm okay with that (for now).

Twitter
I've been using Twitter a lot over the past few months, posting 2-3 times per week about things happening at the school. Now that I'm carrying my iPad, I'm finding it easier to compose a tweet while I'm in a classroom. Which means I'm in a far better position to share the great things that students and teachers are doing, every single day.

Wunderlist
I now start my day with a list of to-dos. This involves a quick calendar review, and a scan of everything sitting on my desk (or email) that needs action. I have been doing it on paper and crossing things off, but my preference is certainly to go digital. So I'm trying Wunderlist, on the recommendation of some blog I can't recall at the moment. This one is new to me, but I've got hopes. I used to use Do It (Tomorrow), which was decent but didn't meet my needs at the time. I might go back to that one as well, and see which one is best for my current situation.

Project Management - an essential skill for leaders

Project management is one of those things we all do to varying degrees. We often call it juggling, as we try to balance competing constraints and the needs and desires of all stakeholders while implementing something that has been handed to us. But there is an actual science to project management, and a better understanding of project management skills and processes may allow us as educational leaders to be both more effective, and more efficient!

In order to define project management, one must start by defining a project. A project is a limited-term strategic initiative that brings value to the larger organization or structure. Project management then is the set of knowledge, skills, and processes that will best bring that project to fruition. There’s an old adage that when you are working with a contractor on a project, the options are good, fast, and cheap - pick two! With that in mind, it is clear that balancing timeline, cost, and quality of product are the three requirements of effective project management.

licensed by PresenterMedia

The Project Management Book of Knowledge (PMBOK) describes nine key knowledge areas within project management (in Dinsmore & Cabanis-Brewin, 2010).  Each of the nine knowledge areas have sub-processes in which a project manager must be skilled. It begins with integration management; being able to balance all of the requirements of the project in such a way as to support the mission of the larger organization or structure. Integration management may include initial project plan development and design in addition to managing the process once the plan is written. Other knowledge areas include management of scope, time, cost, quality, human resources, communications, risk, and procurement. These key processes must be managed in a way that is fluid, always balancing growth and setbacks in one area with complementary actions in other relevant processes. While the knowledge areas and sub-processes are generally accepted “good practices” in project management processes, those practices are applied differently in different project types of different scope and audience.

In K-12 settings, project management is an expected skill of educational leaders. Many times the scope, outcomes, budget, and schedule are pre-determined, and the project must be conducted within those constraints. Particularly when implementing grant programs, even sub-processes are pre-determined, whether or not they are appropriate for a particular individual situation. So I would propose that a final skill required for project management with educational settings in particular is embodied in the Serenity Prayer - grant me the serenity to accept that which I cannot change, the courage to change what I can, and the wisdom to know the difference!