This was a first of its kind study on the Punjab Examination Commission (PEC), which administers a census based examination to students in grades 5 and 8. The study involved an independent review of the examination design process, conduct and marking, as well as the management of assessment data and its interpretation. It entailed collection of data from over 3,000 exam centers in 18 districts, re-marking analysis of over 2,500 exam papers and 200 semi-structured interviews of key stakeholders involved.
Findings show that PEC’s initial steps towards designing and administrating a high-quality standardized assessment were promising, but much more needs to be done to ensure the reliability and validity of these large-scale tests. Issues related to design include lack of expertise in psychometrics and following internationally accepted standards of design such as piloting items. With regard to conduct and marking there are poor center facilities, inadequate training and lack of the supervisory staff, lack of subject specialization among examiners, and inadequate use of marking guidelines. Finally, dissemination of results is not timely and there is a lack of meaningful interpretation of the results for a variety of stakeholders. The study suggests that PEC should limit itself to its core functions and could seek reduced engagement in areas related to the actual conduct of exams. Enhancing capacity of staff in exam design and analysis of student outcome data should be prioritized. There is also a need to ensure better allocation of supervisory staff, strengthen selection of examiners and the processes of marking. Greater integration of the PEC assessment with other aspects of the education system and development of a results communication strategy are also matters that require immediate attention.
Author(s): Abbas Rashid, Ayesha A. Awan, Irfan Muzaffar and Saleya Butt
Sponsor(s): The Department for International Development