Week 2 Discussions

1.Why is evaluation important to you and how do you define it?


For me, evaluation is important as it is a reminder that what we are doing is of benefit and allows us the opportunity to consider different options.

My first thoughts lead to the mainly summative evaluation processes that are in place, then I realised, that even though I was already familiar with the ADDIE model, I considered that ongoing formative evaluation can also occur amongst peers in an informal way- perhaps by attending conferences, belonging to online Discussion forums, talking to staff from other Insitutions. I am refering to evaluation of the teaching process/system as well as the course content.

For example, within an online course, an e-activity may be presented in a new way to students as a result of some learning opportunity gained though a shared experience with another lecturer. Evaluation (formal and informal) is also important personally- it can be used to 'pat yourself' on the back, as well as 'allowing room for improvement'.I also was thinking of the efects 'tick box' situation whereby you present the end of course evaluations to the online students, gather the results for the Annual Programme Reporting process, list any outcomes and then add them to the 'Action List' and find, that a year later, no further action was taken because it got
lost in the 'redtape'.

So, during this course, I hope to learn more about evaluation in a 'user friendly' environment.


While listening to the presentation I jotted down the following in response the question "What is evaluation?" ...a review of a process, how well did it suit the 'intended outcome' (purpose), are there areas which can be improved?, did the process result in
'buy in'? will it be reproducable? can it be transferred to another format?



2.What sort of evaluations mentioned on the presentation are familiar to you already and why?

Evaluation processes that were mentioned in the presentation that are familair to me are as follows:

• paper based questionaires- either handed out in class or posted to students- (this is for ongoing Community Computing classes that are facilitated rather than 'taught';

• end of course evalutions available to students participating in papers or units online;

• Annual Student Satification Surveys, -sent out to students who may or may not still be studying online-at the end of the year. Usually sent via email with a web link to follow;

• focus groups- participation as a staff member on elearning initiatives;

• discussion forums (as a student on this eLearning Course-in another paper)-evaluating fellow students work e.g webpages;

• checklists to follow before a Unit is relased to students on an online course.

3.Why is quality important in eLearning?

Quality: For me this is much the same as in a retail environment - would you buy a suit from a Designer who you had heard via word of mouth from friends (who
had previously brought a similar suit) -that he used inferior materials? If you were looking for a product that will last, you probably wouldn't buy it. But if you were
looking for a product that is cheaper and doesn't have to last the distance, then possibly yes, you would consider it.

The analogy for the student wanting a quality course is something like this. They will want a 'quality course' if they plan to do further study, especially at other insitutions. WORD OF MOUTH is very important - a student who has a bad experience will probably do more damage that we realise- students talk and may remember the poor quality aspects longer than we want them to!!!
Quality is perception of value, perhaps something like "am I getting the same treament/value for money in a $300 elearning course as I would in a $1500 f2f class".

In the early days of blended and 'online courses', it could be said that many courses were delivered 'online' as result of the perception that they are more cost
effective. I don't have a specific reference for this, but I think that today- students expect a lot more than just reading their resources on line- they want all the bells and whistles that Web 2.0 technology offers- and that means that the developers and tutors alike have to come to grips with quality tools and good instructional design. I recall looking at some readings from Frank Rennie in the Educational Design for eLearning paper, but can't track them at present.


Quality is also about 'how good is the course'?-'how well does it meet the intended outcome(s)'? (taking into account the context of the measure of quality).What one student calls "good quality"- another may disagree or shun. e.g with elearning, we assume plenty of opportunity for feedback through (for example) forums, etc. A less vocal, or less interactive student, may not perceive the inclusion of this feedback method as a sign of good quality.

I had a look at the eLearning Guidelines (http://elg.massey.ac.nz/index.php?title=Students/Learning_design/Good_practice).

Listed under guideline SD4 (Students/Learning Design/Good practice) is the following question which I think can be related to quality:

"Do students aquire the learning skills for successfully completing the course?"

In the example above, if we were evaluating this aspect of the course in terms of quality, we'd ask "Did the use of interactive tools and forums help the student complete the course"? So, the first student would say it was 'good'! the second might not think so.

Lots to think about obviously....

Thanks

Debra