CAI Prepares For CCCAssess Field Testing

CAI Product Manager John Hadad explains plans for field testing CCCAssess.The California Community Colleges (CCC) Common Assessment Initiative (CAI) will begin field testing CCCAssess at 12 colleges in early 2016. The first version will be a Minimum Viable Product (MVP) to evaluate user experience and functionality.

The CCCAssess product will be significantly refined after field testing and before colleges start using it in the Fall 2016 for Spring 2017 placement, explained John Hadad, CAI Product Manager.

“The CCCAssess product that colleges will see next fall will be dramatically different from the MVP we are field testing this spring,” he said.

Initial Product Testing In Spring

The spring testing of the MVP will focus on validating the assessment, accessibility, security and performance. “We are very interested in the user experience and gathering information to improve the product in the production release slated for Fall 2016,” Hadad said.

CAI has partnered with Unicon to develop the CCCAssess platform and is working closely with the vendor to design the MVP for testing. For over 20 years, Unicon has been a leading provider of IT consulting, services and support for education technology, with proven reliability, performance and scalability for high-performance applications.

CAI strives to provide a positive experience for all students. As such, the MVP is designed within an accessibility framework to serve the needs of the entire CCC student population. “Accessibility for all students is at the forefront of every design decision,” Hadad said.

Security compliance will also be part of the MVP field testing. “The CCCAssess product must comply with a high standard of security,” Hadad said.

Performance is the other area being considered as part of the MVP field testing. “We will be looking at how the system performs for students and proctors,” Hadad said. “The system will be evaluated on its performance when there is a large volume of users.”

The MVP field test will also look at how easy it is for students and assessment staff to use the online program. “The goal is to meet users’ expectations for a modern, easy-to-use program. For instance, no one wants to wait for pages to load,” Hadad said.

Assessment Tool Development

CAI is developing a brand new assessment tool to meet the needs of California’s community colleges and contribute to student success. Before the MVP can be field tested, the assessment questions must be developed, validated and grouped to form assessments.

LSI Authoring Items

Link-Systems International, Inc. (LSI), a company that has specialized in online learning innovation since 1996, is creating and developing content questions with input from a diverse group of CCC experts. LSI is known for developing WorldWideTestbank, a proprietary and algorithmic assessment and practice test generator that is the most widely used resource of its kind in the world.

Initial Validation

Once the items have been authored through LSI, the psychometrican will start assembling items and item bundles into “testlets” based on logical competency grouping and skill level. Pilot college assessment staff will administer these initial assessments to students for the purposes of data collection. Following analysis of the initial dataset, the psychometrician will modify the items and testlets to enhance the overall efficacy of the assessment.

Adaptive At Testlet Level

The next step in creating the content for CCCAssess is to evalutate the grouping of related items measuring specific competencies, called testlets. These testlets should reflect individuals’ predicted competency and mastery.

The way CCCAssess is being designed, the adaptive nature of the assessment happens at the testlet level. A student’s score at, below or above mastery in a given testlet will determine the next testlet or group of questions that will be served to the student.

Field Testing Assessment

Assessment tests will be comprised of groups of testlets. After field testing the full assessments with students through the MVP at pilot colleges, the collected data will be submitted to the CCC Chancellor’s Office for approval.

Student Services Portal Coordination

CCCAssess will be a portlet on the new Student Services Portal, which is being developed simultaneously by the CCC Education Planning Initiative. CCCAssess is being designed for single sign-on user access to the assessment portlet on the Student Services Portal so students will easily be able to get on to the assessment from the portal when they come to the assessment center to take the assessment. Two of the CAI pilot colleges are also piloting the Student Services Portal, so they will provide input after field testing both new systems.

College Preparation

While the items are being developed and validated, and the platform is being designed and prepared for field testing, representatives from pilot colleges will participate in professional development to prepare them to field test the MVP. The first training session was held in early December.

Pilot colleges will be providing CAI with feedback on how their campus prepared for using the MVP of the new web-based CCCAssess tool. The experiences of the local teams, from using the system to accessibility, will help CAI refine the product and support materials before CCCAssess is introduced to more colleges in Fall 2016.

College representatives interested in learning more, even if they are not working at pilot colleges, may participate in upcoming professional development sessions:

  • February 26 – Los Angeles
  • March 11 – Sacramento
  • April 15 – San Francisco Bay Area

CAI plans many different forms of professional development including videos, webinars, printed information, conference sessions, college campus presentations and in-person training. Materials and information will also be posted online at http://cccassess.org/.


Karen Fraser-Middleton is a marketing consultant for the
California Community Colleges Common Assessment Initiative


 

Add comment