Designing a contextualized PIAAC assessment environment for Indonesian adults: An educational design research study

Abstrak

The Programme for the International Assessment of Adult Competencies (PIAAC) assesses adults’ literacy, numeracy, and problem-solving competencies in functional situations. However, PIAAC-oriented assessment resources that reflect Indonesian adult contexts remain limited. This study aimed to design and evaluate a contextualized PIAAC assessment environment for Indonesian adults. Educational Design Research was employed using Plomp’s preliminary research, prototyping, and assessment phases, integrated with formative evaluation through focus group discussions, one-to-one evaluations, small-group evaluations, and field testing. The study produced 93 contextualized items comprising 32 literacy, 31 numeracy, and 30 problem-solving items, which were incorporated into a computer-based testing system and disseminated through an item book and social media content. Iterative evaluation identified and addressed issues related to contextual relevance, competency alignment, instructional clarity, response-format flexibility, and technical functionality. A field test involving 30 teachers showed mean achievement percentages of 84.33% in literacy, 80.33% in problem solving, and 74.33% in numeracy. The findings indicate that international adult competency constructs can be operationalized within authentic Indonesian contexts and iteratively transformed into a practical, usable digital assessment environment. The study offers design-based guidance for developing culturally relevant adult competency assessments while maintaining alignment with the PIAAC framework.