Can Multiple-Choice Items Measure Critical Thinking in Socio-Scientific Environmental Issues? Evidence from a Global Warming Assessment of Grade 10 Students Using Rasch Analysis

Abstract

This study examines whether multiple-choice items can be used to measure students’ critical thinking in socio-scientific environmental issues, using global warming and renewable energy as contextual domains. To address this question, the Critical Thinking Instrument for Global Warming and Renewable Energy (CT-GREEN) was developed as a set of 40 multiple-choice items grounded in domain-specific critical thinking indicators. The instrument was designed following the ADDIE model and administered to 132 tenth-grade students in Bandung, West Java. Rasch modeling using Winsteps (version 4.5.0) was employed to examine construct dimensionality, item fit, and measurement reliability. The results show that 38 of the 40 items demonstrated acceptable fit to the Rasch model, while two items exhibited misfit and require revision. The raw variance explained by measures was 27.6%, indicating moderate unidimensionality, which is reasonable given the multifaceted nature of critical thinking. The instrument demonstrated high item reliability (0.94) and adequate person reliability (0.73), with a Cronbach’s alpha (KR-20) of 0.76, suggesting acceptable internal consistency for exploratory assessment purposes. Overall, the findings indicate that CT-GREEN demonstrates acceptable psychometric properties and provides preliminary evidence that well-designed multiple-choice items, when embedded in socio-scientific environmental contexts and analyzed using Rasch measurement, can be used to assess students’ critical thinking. However, further refinement and validation across broader populations are recommended before making stronger generalizations.

Keywords

• Critical thinking
• Global warming
• Renewable energy
• Rasch model
• CT-GREEN