An integrated STEM-based mathematical problem-solving test: Developing and reporting psychometric evidence

Abstract

Science, technology, engineering, and mathematics (STEM) problem-solving is necessary to be infused into the classroom. Nevertheless, the criticism of underrepresented mathematics in STEM problem-solving assessment is an issue. In this study, we develop and investigate the psychometric evidence of an integrated STEM-based mathematical problem-solving test. The product of the test was a mathematical essay test that contains three scientific scenarios related to the environment in every middle school grade. The mathematical contents were integrated into engineering-based design using the technology. Three experts filled an assessment sheet to assess content validity, which was analyzed using a content validity index (CVI) and intraclass correlation coefficient (ICC). The result of content validity revealed that overall items were valid and reliable. The construct validity was examined using the Rasch analysis from the data of Grades 7–9 students in Indonesia (n = 286). The construct of all scenarios and prompting items indicated fit with various difficulty levels and acceptable discrimination value. Nevertheless, four prompting items were reported as misfit based on unweighted mean square value. The recommendation for improvement is emphasized in the language clarity aspect. The inter-rater reliability was also declared good. A further study is suggested to provide a computer-based test.