A Digital System-Based Assessment for Collaborative Problem-Solving Skills among Pre-service Science Teachers: Preliminary Evidence of Quality and Implementation

Abstract

Collaborative problem-solving (CPS) is central to science teacher preparation, yet existing assessments often emphasize individual outcomes or final products and provide limited evidence of how CPS develops across iterative stages of group problem-solving. This study developed and preliminarily evaluated a digital assessment system for science teachers’ CPS across four stages: identifying and representing problems, planning and developing solutions, testing and refining solutions, and communicating evidence-based results. A multiphase, quantitative-dominant mixed-methods design involved three expert validators and 118 prospective science teachers from four intact classes. Quantitative evidence included content relevance (Aiken’s V; expanded Gregory), internal structure (CFA), expert-judgment consistency (ICC), and multiclass score patterns; expert comments, revision records, and user feedback provided qualitative evidence integrated through a joint display. Content relevance was mixed, with CFA showing three dimensions loading above 0.30 and communicating results loading at 0.286. Expert-judgment consistency was inadequate, and student-score reliability was not established. Class scores differed significantly; however, intact-class observations preclude causal inference. Thus, the system provides preliminary stage-specific CPS score patterns rather than dependable individual-level profiles. Its contribution is integrating stage-specific assessment across an authentic design–test–redesign–retest cycle using analytic-rubric scoring with measurement and implementation evidence. Further refinement should prioritize communication descriptors, validator calibration, and student-score reliability.

Keywords

• collaborative problem-solving
• digital assessment
• quality science education