Assessments

Education Monitor: Assessment Systems in Pakistan – Considerations of Quality, Effectiveness and Use- 2016

The last few decades have seen a spike in interest in standardized assessments globally. A number of factors “such as the need for comparative data, a focus on teacher accountability, and a desire to implement a uniform education “is driving this interest in assessments as a basis for education reform. Alongside the emergence of these standardized assessments, traditional examinations at the secondary level have continued to exist and remain largely untouched by the reforms movement driving change at the primary and elementary levels in assessment. This brief summarizes the report, which reviews the enabling context of various assessment systems in Pakistan as well as key practices of assessment design, implementation and result dissemination and use. For each aspect the report provides a comparison of existing practices with international standards and best practice.

The report highlights the pressing need to improve the enabling environment by calling for the development of coherent national and provincial assessment policies and investing in higher quality human resources for  designing and analyzing assessments according to international standards. It underscores the necessity of reform in assessment practices by calling for adherence to internationally accepted standards in the design and implementation of assessment instruments. It also suggests the need for greater emphasis on dissemination and ensuring use of assessment results. It concludes by asserting that assessment systems will have a key role in driving quality in the education system and in positively affecting teaching and learning in the classroom only when governments attend to the task of reforming assessment agencies with the urgency it deserves.

Early grade reading assessment baseline report -2014

The Management Systems International (MSI) and School-to-School International (STS) conducted a baseline reading assessment for primary school children prior to the launching of two USAID-funded projects: the Pakistan Reading Project (PRP) and the Sindh Reading Program (SRP). PRP was targeting improved reading for 4 million children in Azad Jammu and Kashmir (AJK), Balochistan, the Federally Administered Tribal Areas (FATA), Gilgit-Baltistan (GB), the Islamabad Capital Territory (ICT), Khyber Pakhtunkhwa (KP), and Punjab, while the SRP is targeting improved reading and mathematics for 750,000 children in Sindh. Targets were to be achieved through support for 1) improved policies, laws, and guidelines for teachers and administrators, and 2) improved reading instruction for children in the primary grades. The goal of this evaluation was to compare each province’s baseline results to its midline and end line results.

Academic achievement, school performance and spatial heterogeneity-2013

The establishment of the Punjab Education Sector Reform Program in 2003 was the catalyst for various initiatives that produced data for guiding education policy in the province, both through providing information about various school inputs and by revealing differences in student learning outcomes. However, the role of such data in generating meaningful and reliable evidence for informing policy remains under-utilized. This paper seeks to contribute towards the generation of such evidence by making use of the two existing large data sets in Punjab to understand the correlation between school inputs and student achievement. The school input data was provided by the Program Monitoring and Implementation Unit, and data on student outcomes was available through the standardized grade 5 and grade 8 exams conducted by the Punjab Examination Commission.

Through combining and analyzing data from these two sources, the paper found that the variation in learning outcomes was more substantial within administrative units such as districts and tehsils than across them. It also found no evidence of differences in test scores across urban and rural areas. Given the heterogeneity in school quality within districts, the paper recommends that the education reform effort in Punjab would do well to devise a school-focused approach instead of a regional one. It also found that school inputs explain a sizable portion of the variation across schools, but concludes that further research is required for more definitive evidence on the causal impact of various school inputs on learning outcomes.

Improving education through large-scale testing? A study on primary and elementary level exams in Punjab-2011

This was a first of its kind study on the Punjab Examination Commission (PEC), which administers a census based examination to students in grades 5 and 8. The study involved an independent review of the examination design process, conduct and marking, as well as the management of assessment data and its interpretation. It entailed collection of data from over 3,000 exam centers in 18 districts, re-marking analysis of over 2,500 exam papers and 200 semi-structured interviews of key stakeholders involved.

Findings show that PEC’s initial steps towards designing and administrating a high-quality standardized assessment were promising, but much more needs to be done to ensure the reliability and validity of these large-scale tests. Issues related to design include lack of expertise in psychometrics and following internationally accepted standards of design such as piloting items. With regard to conduct and marking there are poor center facilities, inadequate training and lack of the supervisory staff, lack of subject specialization among examiners, and inadequate use of marking guidelines. Finally, dissemination of results is not timely and there is a lack of meaningful interpretation of the results for a variety of stakeholders. The study suggests that PEC should limit itself to its core functions and could seek reduced engagement in areas related to the actual conduct of exams. Enhancing capacity of staff in exam design and analysis of student outcome data should be prioritized. There is also a need to ensure better allocation of supervisory staff, strengthen selection of examiners and the processes of marking. Greater integration of the PEC assessment with other aspects of the education system and development of a results communication strategy are also matters that require immediate attention.

Publications