Fine-Tuning The OEI Course Review Process

California Community Colleges Online Education InitiativeAs one would expect with an initiative as ambitious and complex as the California Community Colleges (CCC) Online Education Initiative (OEI), we have had to make many changes as we move forward in developing its various components.

These changes have been necessary in order to perfect what we have created, to respond to new decisions and to ensure that we are moving forward in the most logical, effective, and resource-conscious way possible.

The Professional Development component of the OEI has as its ultimate goal improving the quality of the online educational experience across the colleges, with its initial focus on ensuring excellence in course design and accessibility.

In order to promote excellence in course design, the OEI Professional Development Work Group reviewed existing course design rubrics and developed the Course Design Rubric for the Online Education Initiative. The rubric establishes standards for course design, interaction and collaboration, assessment, and learner support that must be met by all courses to be offered in the OEI Exchange that will simplify cross-college registration for students at participating colleges. After adoption of the rubric, training for Peer Online Course Reviewers (POCRs) was developed and provided by @ONE. Faculty from the OEI’s 24 pilot colleges then submitted their courses for review.

Following the first round of reviews, the rubric was revised and streamlined. In its initial iteration, a single design flaw might repeatedly impact scores in multiple parts of the rubric. After removing these redundancies, the OEI Course Design Rubric is now a better tool for assessing course design. As we are still in a pilot phase, further modifications are likely as its utility is evaluated.

Rubric Available To All

The Course Design Rubric for the OEI is readily available for modification and use by all who are interested in adopting a tool for assessing the quality of their online courses. In order to help colleges interpret and apply the rubric, the first phase in training to be a POCR (the one-day “Applying the OEI Course Design Rubric”) is open to all who are interested. In this workshop, participants develop an understanding of the components of the rubric and, hopefully, identify ways to modify their own classes to improve the experiences of their own students.

Information about upcoming workshops can be found at www.onefortraining.org/node/731. It is our hope that the rubric will be a useful standard for those colleges looking to establish a process for local course review and improvement. A number of colleges have already adopted it locally and we hope to be able to support their efforts by offering training related to the different elements of the rubric.

New Developments Add Complexity

Shortly after the course review process was initiated, the OEI made its most significant decision to date: It selected Instructure’s Canvas as its common course management system (CCMS). Consequently, a new factor was incorporated into the course review process. As all courses to be offered in the OEI Exchange must be aligned to the rubric and taught in Canvas, there is a need to conduct reviews of all courses in Canvas—even if they were aligned to the rubric upon initial review.

At present, more than 80 courses have been reviewed in their native course management systems and must be re-reviewed in Canvas. Adding another level of complexity is the revision of the rubric, as a considerable percentage of the courses was reviewed using the rubric as initially adopted. As often is the case, these challenges are creating opportunities.

As courses are re-reviewed in Canvas, we are learning about common issues that arise during the migration process, and we will be better able to advise faculty about ways to effectively use Canvas to accomplish what they were doing in their native course management systems. Instructional design support is available for all OEI faculty and the use of such support prior to initial review or re-review is strongly encouraged. Our goal is to eventually conduct a single review that determines that a course is fully aligned without further review needed.

As our ultimate goal is to front-load support, providing all faculty with guidance to help them improve their courses, every challenge we’ve encountered has furthered our understanding of what faculty need as we try to support them in providing the best possible online experience. The OEI looks forward to expanding the professional development opportunities it provides to our colleges.

Comments From Course Review Participants

    • I have never been a student in my course. My students don’t always give much feedback. I discovered it is hard navigating through my course. Great constructive criticism—empowering, in a way. The reviewers offered suggestions, not ripping me up in a report. As an instructor, I view courses differently than a student does. The reviewers helped me see this.

 

    • I received a lot of feedback but it did not stop there. I received a lot of resources to get help—walking me through it step by step.

 

    • The reviews inspired me to incorporate more audio/visuals and gave me a perspective of what it would be like for a student. Reviews that are at my local campus are usually by someone in one’s discipline. It is good to be reviewed by someone I did not know and who is from a different discipline—getting a different perspective.

 

    • I enjoyed being evaluated—the brutal honesty and the objective viewpoint. I have heard people say they want to go back to their colleges and recommend that they use the rubric at their colleges.

 

    • The review process was very positive. We often don’t get sufficient feedback from students, and students are not properly trained to review. I just made some recommended changes and my course is now a lot better. It works!

 

    • One teacher had 50 students in her class this summer. She has never had as high a level of retention as she experienced this summer after making just small changes.

 

  • Other states are very interested in using OEI’s public resources (rubric, student readiness modules, etc).

Michelle Pilati is the Interim Chief Professional Development Officer for the
California Community Colleges Online Education Initiative


 

Add comment