Lesson Three: Evaluating Online
                         Courses

                 A History of Online Course Tools

                 A Course Assessment Checklist.

                 Course Authoring Features, and
                 Course Management Tools

Required Assignments: 1. Read carefully Chapter Three- "In the Study-     Teleresearch" in the printed book and review     the associated resources at         http://virtual-architecture.wm.edu/Teleresearch/index.html     (30 minutes) 2. Using the Course Assessment Checklist     provided in this lesson, assess the "Reach for the Sky"     project's online course -     Mentoring Online; How to Create and Teach an Online Class    http://lone-eagles.com/courses/rfts/ Select Mentoring Online     Send your assessment to your Instructor.     (30 minutes) 3. Explore the course authoring features, and course     management tools, at the following course     authoring sites. Send your instructor a listing of the     course tools provided by at least one of these sites     and share which ones you'd be likely to use or not use.         These are taken from the  master listing at      http://lone-eagles.com/self.htm      A.  Blackboard http://www.blackboard.com              (now merged with WebCT)                B. Edu20 www.edu20.org               Take the tour for this free online course authoring and delivery system.              It is basic but functional.    C. Elluminate www.elluminate.com           Select the free DEMO and follow the instructions.      D. The Moodle homesite is www.moodle.org with many great resources.           This open source platform is free and many educators already           prefer it to Blackboard due to its elegantly simple functionality.      E. At www.cnx.org is a site offering free online course authoring          with a science and math emphasis.  Check it out!        (one and 1/2 hours) 4. Check out the (free) Open Educational Resources at     http://oercommons.org  and at  http://eduforge.org/ -     as example "Open Knowledge" sites.      (one hour) 5. Send a two page candid assessment of the specific     sites you've explored and any concerns to your     instructor. Detail what other considerations you feel are          important which were not addressed in this lesson.     (30 minutes)

 

Lesson Goals:

In this lesson you'll review the evolution of online course tools and standards.

You will assess an online course using a Course Assessment Checklist.

You will review course authoring systems and list the course management tools for one course authoring system which you would, and would not, be likely to use.

 

A Short History of Online Course Tools

Paper and pencil correspondence courses have been available for over fifty years, and for motivated students, have been proven to work adequately.

The first online courses in the late 1980's and early 1990's were text only, due to limitations of the Internet and emphasized text-based online interaction.  Then, as now, there was/is no limit to the creative use of this new medium of interactive reading and writing, particularly in the hands of a good teacher. The true subtle depths of this unique form of communications were beginning to be recognized when suddenly the WWW appeared!

WWW online courses were created with "point and click" hyperlinks to resources and were able to easily use images.  The first web-based online courses were little more than content pages, literally the teacher's lecture notes displayed in the web format. The subtle art of using interactive reading and writing fell into temporary disfavor as "an obsolete text-only technology." Slowly, awareness re-emerged that there was a collaborative "something" that was missing, and web-conferencing systems were added to online courses. The creative use of online interaction, far from being exhausted, has yet to be seriously tapped for what it can offer to the instructional process.

If content delivery is all that's required, teachers may not be needed as automated systems can deliver content. The value a teacher brings to both traditional and online classes is the opportunity for interaction providing insights and context to the raw content. Good teaching might be considered the process by which raw information is condensed to knowledge, which is further condensed to wisdom.

Today, there are dozens of online web systems that incorporate online course authoring and course management tools in a way that does not require a technical background. The tools are helpful, but do not constitute a quality online instructional experience. The tools such automated systems offer include: automatic quiz generation, timed quizzes, gradebook functions, file sharing, graphical whiteboards, student groupwork "spaces", and many more useful features.

Explore the course authoring features and course management tools within the following course authoring sites, (taken from the listing at http://lone-eagles.com/self.htm) Note there is a big battle going on between pricey commercial systems like Blackboard and free rapidly improving open source platforms like Moodle. Trust your own instincts as an educator as to what you're drawn to, and what you are repelled by.

1. Go to Blackboard http://www.blackboard.com (recently merged with WedCT)
    Explore the largest commercial online course platform provider noting Moodle provides most the
    of the same functionality at no cost.

2. Go to Edu20 www.edu20.org 
             Take the tour for this free online course authoring and delivery system.
             It is basic but functional.
   
 3. Elluminate www.elluminate.com A very popular online course platform.
    Select the free DEMO and follow the instructions.. Review the tools they offer.

4. The Moodle homesite is www.moodle.org with many great resources.
    This open source platform is free and many educators already prefer it to Blackboard
    due to its elegantly simple functionality.

5, At www.cnx.org is a site offering free online course authoring
         with a science and math emphasis.  Check it out!

If you create a course on a proprietary system which requires you pay a fee, and then you decide to stop paying the fee, your course and student access to it, will be lost. Course authoring systems which are IMS compliant, allow course materials to be easily moved between systems, allowing you important future flexibility with your online course materials, including gradebooks and other recordkeeping features.  For more on this issue see the Instructional Management System, (IMS): http://www.imsproject.org  See also http://lone-eagles.com/advanced.htm for advanced information on related topics.

Review the following site to appreciate the volume of course authoring systems and the challenges comparing their various features:

Online Course Authorware Comparisions
http://www.edutools.info  Review all links, but particularly select and explore "compare products" to compare dozens of online course authoring options.

An increasing number of listings of collaborative tools for distance education are appearing which raises the questions of how to stay current. 

Search www.google.com for "collaborative tools for distance education"

When an educator selects an online course authoring system, the educator may become inherently limited by the features that system does and doesn't have. For instance, some systems allow a whiteboard, where students and teachers can exchange drawings, or write mathematical formulas, with an electronic pen; others don't. It is a good idea to thoroughly compare available tools before deciding on a course authoring system.

Standards for online courses are continually changing as tools improve and educators gain experience with alternatives.  Just as the first web pages you saw were at first amazing, and later seemed substandard, so too are online courses creating higher standards as we all become more experienced.

Expectations increase with experience; our expectations shift as we become aware of better and better applications. Currently, any online course is generally considered to be 'sophisticated,' just by being "online." But, as teachers and students become acquainted with better models for student engagement, interaction, and motivation, the standards for what constitutes a really good online course will change significantly. The design for interpersonal attention, and engaging interaction, will dominate this assessment dynamic.

As teachers, we need to maintain a healthy skepticism. Just because a course's web pages are pretty to look at or incorporate the latest neat multimedia features, it does not necessarily mean they incorporate the best educational methods and pedagogies. Nor does it mean they produce measureable learning outcomes, engaging interaction, or that they motivate students.

On the other hand, just because a course is self-directed, may not mean students are not learning from the well-written thoughts and perspectives from the heart and mind of a great teacher. A course touting use of collaborative tools, may, in fact, misuse them!

The course goals, thoughtful design, incorporation of the best resources, and selection of appropriate presentation technologies, to present the curriculum in the very best ways possible, are all factors to consider. Your common sense as a good educator, coupled with thorough review of online instructional options, will be all you need to become an excellent online educator! Don't be in too much of a hurry; take the time to carefully assess your online teaching alternatives.

The "shelf-life" of an online course can be as little as six months, as URLS change regularly. Planning for an online course is often far more demanding than preparing for a traditional classroom presentation or activity.

Students sitting at computers using collaborative tools, might look socially isolated to the untrained eye, but they might well be more collaboratively engaged, with more peers per unit time, than if they were in a traditional classroom. This is particularly true with larger class sizes, where the classroom is more a lecture setting than an interactive environment.

Since many teachers have yet to take their first online class, and many students have far more experience using the Internet and collaborating online, it can be difficult for teachers to understand the readiness of students to use the Internet as a collaborative medium.

An online course, such as this one, may avoid use of unnecessary graphics to allow efficient use by those with slow Internet access. Or, a course, as the Shakespeare course you've reviewed, it may use sophisticated graphics and web-design techniques. The time you'll have available will determine how you approach creating an online course. The time required for authoring an online course can vary widely with the functionality of producing measureable outcomes being the true test of quality. As this current course demonstrates, incorporating existing resources can dramatically reduce the time required to create an online course, while keeping the quality at a high level.

The issue becomes; "What web tools do you need to teach the content in the style you'd prefer?"

The following checklist will give you a starting place for
assessing, and designing online course format options,
independent of your assessment of the quality of their
application. Cut and paste this checklist to a new page
to conduct the assessment of the "Mentoring Online"
course required for this lesson.

Course Assessment Checklist

BANDWIDTH, SOFTWARE AND TECHNOLOGY REQUIREMENTS?

 

SELF-DIRECTED vs INSTRUCTOR-CENTRIC?

 

INDIVIDUALIZED VS GROUP-ORIENTED?

 

LESSON STRUCTURE (FIXED VS FLEXIBLE)?

 

CONTENT TYPE: SKILLS VS CONCEPTUAL LEARNING?

 

INTERACTION DESIGN WITH MEDIA ?

 

INTERACTION DESIGN FOR HUMAN INTERACTION?

 

TIMEFRAME?  (FIXED VS ON-GOING)

 

ASSESSMENT METHOD(S)?

-------------(End of Checklist)

 

Checklist Explanations:

Bandwidth, Software, and Technology requirements:
A low budget, low-tech class might involve simple web pages only, allowing use by those with slow modems and older computers. A high budget, high tech class might involve advanced multimedia and interactive television. Will students be required to have the latest version of web browsers with audio and video capabilities? The lowest common denominator for technology requirements must be made very clear for a given online experience.

Self-Directed VS Instructor-Centric:
A class can be self-directed, or instructor-interaction intensive, with a whole range of options in-between, such as small group interaction with a high level of one-to-one student interaction, or a large group interaction with a low level of one-to-one student interaction. A self-directed course can potentially work with any number of students, whereas an instructor-centric course has limitations due to the amount of interaction the instructor can adequately handle.

Issues for a Self-Directed Course
A self-directed course represents one extreme, that can still be highly interactive using high quality CDROM's, software tutorials, and interactive web sites, which are rapidly becoming more sophisticated. Lessons can be completed at the learner's discretion and potentially in any order.

See the syllabus for this class for reference to the book "Designing for Self-directed Instruction"

Issues for an Instructor-Centric interactive course:

Similar to a traditional classroom, the instructor would lead all discussions and activities and the participants would need to keep up or wait for the rest of the group, lesson by lesson.

To emulate a traditional classroom experience a teacher can decide to lead discussions by posting messages to a web conference or listserv. When every student contributes, the number of messages will depend on class size. As a teacher, we'll need to know when we'll reach an unmanagable number of messages.

How will you assess student performance, knowing some students will post messages as required, but are not genuinely interested or engaged, while others are highly engaged, prolific writers, and may find themselves resented by the less motivated students? The online medium tends to amplify certain student differences, particularly their writing ability and willingness to interact openly and honestly.

If you assign 5 small group discussions, with students as discussion moderators, for five discussions of five students each, and each student posts 3 message a day, for three class days per week, how many messages must each student read per week (5x3x3=45 msgs) and how many messages will you scan per week as a teacher? (45x5=225.)

This doesn't take into account highly motivated or articulate students...who can be expected to author longer messages as they gain confidence. An unobrusive measure would be to measure the number of pages of writing produced by each student and/or discussion group. A subjective accounting of the value and content of the interaction would be another measure.

Lesson Structure: Fixed VS Flexible

Lessons could be required to be completed by specified dates and in a specific order, or the student could be given the option to decide which lessons to complete by which dates, within general guidelines.

Content Type: Skills VS Conceptual Learning

A course may be focused on teaching skills such as desktop-publishing in a step-by-step format measured by multiple choice questions. Or, as in a philosophy course, more conceptual instruction may focus on discussion, sharing thoughts in writing about required reading assignments, or discussing ideas raised by other participants. Assessment could be subjective analysis of participants' writings.

Depending on your teaching goals, interaction with various media (using high quality CDROM's, software tutorials, and interactive web sites,) may be the best approach, or perhaps interaction with people would be more appropriate. These are two extremes, with many levels using both features in-between.

Consider:

Interaction Design with Human Interaction

If the goal of an English class is to engage students in reading and writing discussion, after having read a book by a specific author, online interaction might well be superior to a classroom setting by allowing for more overall interaction per unit time, and vigorous exercise of reading and writing skills, in a shareable, documented format after the interaction has taken place.

If the goal of a philosophy class is to make students think and articulate their thoughts, would an online discussion or face-to-face discussion be best? What would Socrates choose, if he had these choices? Or, when exactly would one be better than the other? Socrates might say "The ideal would be a combination of the best features of both these verbal and written forms of interaction."

Interaction Design with Media

If the goal of a biology class is to teach the human anatomy, a high quality, well-organized interactive CDROM might provide the student with more interactive individualized learning per unit time than a classroom setting. The teachers' role might shift from presenting the content, to selecting the most efficient content presentation, and focusing on articulations and higher contextual discussion to increase the meaning of the content. Freeing the teacher to provide more focused individual attention for those who need it is another option.

We've all had some teachers who did little more than present information, often reading from their notes. If their notes were made available online, supplemented with multimedia interactive resources and examples, the online version would be individualized, more interactive, perhaps more efficient, and possibly more entertaining and motivating for students.  The time saved by the teacher could be applied toward individual assistance to those students needing special help.

A course may offer a choice of learning styles, or may offer only one choice, depending on the multimedia tools and the availability of the time to create alternative presentations. While offering this type of choice is an advantage of multimedia systems, the time available for teachers to create such alternative presentations is often lacking. Such systems can typically be created for only large scale applications, which brings us to one advantage of corporations creating high quality learning materials.

Timeframe for intended learning community: Community networking and online learning are blurring around the theme of  "building learning communities." An online course could define a finite timeframe for interaction, whereas an online community of interest may set no such time limitations.

As teachers of an online class, we don't necessary have to say good bye to our students when a course ends, but can continue the relationship through the convenience of email and other collaborative tools. Many opportunities exist for creating ongoing online learning communities to sustain interest and learning with students.

Assessment Methods
A course might have multiple choice quizzes, or be graded on the quality of their online interaction. Special projects or deliverables may be required. We'll discuss assessment in a later lesson.

 

The graphic below represents a quick overview of considerations in designing and/or evaluating an online class. Each line represents an entire spectrum of choices.