What to retain?

One of the first decisions to be made in the process of course redevelopment is what to retain from what has gone before. In other words, I need to identify what should be kept because it worked reasonably well and what adjustments might be made to improve it for the future.

EDP4130 has a history that should not be ignored in planning for redevelopment. The first offer in 2011 borrowed much of its content and activities from EDU1471, which had been offered some years previously across 3 campuses but not online. The online offer required for EDP4130 was a significant difference that influenced planning and implementation of the LMS (Moodle) space. Based on that experience some further changes were made for the 2012 offer.

Thinking about how various parts of the course worked (or didn’t) can be informed by my own reflections on the experience, comments from colleagues who worked in one or other offer, records of activity in the LMS, and data from student evaluations of the course offers. Some of those sources are loose but should suffice for some guidance about what to change.

Because I developed EDU1471 immediately after spending a year at Purdue I designed the assessment to be distributed through the semester and linked closely with tutorial activities. That had multiple benefits with an on-campus course. Students saw relevance in attending classes that bore directly on assessment, much of the work was actually completed in class and small pieces of work submitted one week could be easily marked in time for return in class next week. That kept students attending and ensured that they knew how they stood as the semester progressed. The major assessment piece was a class project involving collaborative development of curriculum materials that were contributed to a shared resource pool, accompanied by an individual report on learning through the process. Students saw the assessment as relevant and manageable. From my perspective designing the key assessment items first was a useful way of specifiying expected outcomes and I was then able to design learning activities to support their achievement.

In developing EDP4130 for 2011 I retained as much as possible of the tutorial activities and related assessment from EDU1471. The WebQuest about infant formula that I had developed for EDU1471 seemed dated so I constructed a new one around Coal Seam Gas but did not include it in the assessment for 2011. I did retain an activity in which students developed design briefs, exchanged them for response, and then reported on that process. I was able to manage that for online students using a database in Moodle but handled the on-campus classes in the classical way. I retained a resource review activity and assessment from EDU1471 but worked it in Moodle using a database and a peer review assessment module. I added a quiz early in the semester to drive engagement with the curriculum documents and some relevant reading. The whole class curriculum project and report was retained essentially as it was configured for EDU1471 with online students working in forums with a database and wikis if desired.

The checklist approach in the peer review assessment produced high marks. That coupled with good marks on the other assessment items produced a distribution with more high grades than the faculty and university authorities liked so some change was indicated for 2012. Other issues arose around management of some activities, especially the large group curriculum development, for the online students. Again, some change was indicated for 2012. Overall students had a very positive response to the assessment pattern with evaluation comments running 12:1 in favour of the pattern of several small assessment pieces across the semester. Students also responded positively to what they saw as the relevance of the assessment activities to their learning and future work as teachers.

For 2012 I retained the quiz as a simple means of encouraging students to engage with key content. They recorded an average score of 12.5 out of 15 (83%) which some might regard as high but it caused me little concern if it meant that students became familiar with important course material. I also retained the design brief activity but, in deference to my understanding of the USQ push toward Digital First, I designed it to work entirely online for all students using a pair of databases in Moodle. I dropped the peer review activity around resources because it had inflated marks and tweaked the WebQuest about Coal Seam Gas to replace it as an assessment item. Student engagement with the optional because not for assessmentWebQuest in 2011 had been minimal but making it assessable was expected to encourage engagement with the important ideas about the values dimensions of technology. Because the whole class curriculum project had been awkward to manage online I made that item an individual curriculum development project and report with the option for students to work in small groups of up to 4 members. However, the requirement to contribute the materials to a shared pool of resources (collated through a Moodle database) remained. To retain the benefits of collaboration for all students I added requirements to demonstrate the effect of a Personal Learning Network (PLN) and of sharing work in progress through a Virtual Learning Design Studio (VLDS) hosted in the Mahara eportfolio system.

Student responses to the course in 2012 were mixed. Although the pattern of assessment was very similar to that in 2011 the changes resulted in a wider spread of grades and more disaffection among students, which seems consistent with students seldom complaining when they receive good grades. Some on-campus students expressed unhappiness about needing to complete certain tasks online, although others took the opportunity to reduce class attendance. Some students appeared to have difficulty with interpreting assessment requirements and criteria although others managed very well. Comments on student evaluations pointed to the need to ensure that criteria are clearer and that assessment details are easily found. Some careful thinking about the design of the Moodle space for the course will be needed. The final task was seen as large, though in reality it was no larger for a student than in previous years, and complex. Comments about needing to access multiple pages for information possibly related to information about the PLN and VLDS components and, if those are retained for 2013, care will be needed with designing clear pathways through the material. Comments in the actual assessment work submitted by students seemed to suggest that they found value in all parts of the activities, including the PLN and VLDS, but perhaps those comments were calculated to encourage the marker to reward them.

Although much of the course was designed around the assessment and related learning activities (directed reading assessed by the quiz, WebQuest over 3 to 4 weeks with a 600 word report, design brief exchange over 2 weeks with a 600 word report, curriculum materials and 1500 word report developed over 5 weeks) there were other resources provided to students. They included brief notes about the course content, recordings of presentations (in multiple formats – M4V [with video] and MP3 ) accompanied by PowerPoint slides and handouts, weekly guides to activities and links to other resources. How much use was made of these is difficult to determine because not all are trackable in Moodle and where material is provided in multiple formats it is difficult to aggregate records of access. However, examination of available activity reports suggests that some students access most of the material but many will have been selective, especially toward the end of semester when they were focused on completing assessment.

So, what have I learned and what should I retain for the 2013 offer?

Based on the student evaluation data there seems to be support for a distributed assessment pattern to spread the load, provide formative feedback through the semester, and avoid ‘high stakes’ items like the final piece worh 55% in 2011 and 2012. Students appreciate assessment that they see as relevant and for which the criteria are clear and instructions easy to follow. The quiz (in some form), WebQuest and design brief activities should be retained though some adjustments will be in order. The large final assignment seems not much favoured by students and the advent of systemic curriculum design initiatives like C2C and online sources of teaching materials like Teachers Pay Teachers suggests that in future teachers are much more likely to find themselves adopting and adapting existing or found materials rather than engaging in ab initio curriculum development. The assessment work in EDP4130 should probably recognise that shift and focus on the work of locating, curating, analysing and adapting curriculum materials rather than development from the blank page – if that ever really happened. There did seem to be value in the PLN and VLDS activity but the VLDS could be simplified by having students work in smaller, more manageable groups and both would benefit from being introduced from the start of semester as part of the professional approach to working rather than later as an assessment requirement.

There is probably a continuing need for some recorded material but front loading that early in the semester may encourage its use. By end of semester students are focused on completing assessment in all their courses rather than accessing new material that may not be directly related to assessment. Multiple formats that support access using mobile devices received some support in the student evaluations and should be retained. There were also requests for more easily printed material (PDF) and there might be value in looking at means to produce notes and other text material in multiple formats to support web access, printing (PDF) and eBook readers (ePub and mobi).

That seems like plenty to think about for now.