PISA Under Examination: Gerry Mac Ruairc's Keynote Speech

Gerry Mac Ruairc gave a keynote speech at the symposium entitled, "Ticking the Boxes: A Critical Examination of the Process of PISA Testing from a Student Perspective."

Here are the notes from the speech:

Testing:

• technologies of power surveillance
• impacts of testing: high-stakes and low-stakes

Unintended consequences of policy developments:

• what are they?
• need to watch them
• no one sets out to damage, but unintended consequences do that
• alliances: neo-liberal, neo-conservative, neo-Middle class
• accountability
• measurement

Policy of testing in Ireland

• began in 2008 with 7/8 year olds and 11/12 year olds
• results not centrally collated - within schools
• reviewed during Whole School Inspection
• must report scores to parents
• vertical system of testing
• 15 year olds and 18 year olds subject to more standardized testing
• TIMSS, PIRLS, PISA

Ireland

• small schools
• 8 schools have less than 50 students, 291 schools had greater than 500 students, and 238 students had 300-499 students.
• voluntary secondary schools in Ireland: privately owned, charge fees but publicly funded, Catholic or the Church of Ireland
• community and comprehensive schools
• vocational schools with a working class profile

General trends:

• commitment to knowledge economy
• commitment to education

Ireland in PISA:

• above average in math and science
• 5th or 6th place in reading

PISA 2006 Ireland:

• 165 schools randomly selected
• mix from different schools, e.g. secondary, vocational
• 4585 students
• 5 different grades (Irish 1-5)
• high level of agreement to participate
• Reading Literacy Level 1 or below: 12.2% (OECD average: 20.1%)
• Science Literacy Level 1 or below: 15.3%

Socio-economic groups and testing:

• language and linguistic discontinuity
• choice of language use intended or unintended?
• social class achievement in PISA: working class - lower achievement, middle class - higher achievement
• curriculum vs. culture
• multiple-choice answers - linguistic choices of the working class

Research on PISA 2009 - "PISA girls"

• disadvantaged, inner-city girls
• "opportunistic" sampling
• didn't know what the test was looking for
• Principal talked about the timing and the intensity of the process
• school PISA coordinator talked of "students wilting," "too intense" process, "what monster is this feeding," "rat in a lab" scenario
• ticking the boxes just to "get rid of it"
• girls' view - upset by answering questionnaires about lifestyle
• impact on engagement - interested at first, but then just ticking boxes
• issues of confidentiality - names of booklets
• didn't try as hard because of not getting results back
• the need for calculators - five grade levels in the room. Students were too embarrassed to ask for calculators
• personal nature of items on questionnaires: too "nosy," felt swayed by the socio-economic nature of questions
• negative sense of self: not willing to write about their parents' professions, some lied, bias of jobs listed for parents - not "normal" jobs in a deprived area

Conclusion:

• need to engage with student perspectives on the testing process
• need to dig deeper an enrich our understanding of socio-economic level
• need to consider the impact of socio-economic status