Tuesday, 24 August 2010

PISA Under Examination: David Scott's Keynote Speech

David Scott gave a keynote speech entitled, "PISA, International Comparisons, Epistemic Paradoxes."

Here are the notes from the speech:

The capacity and ability to know each other's minds, in terms of PISA.

Two forms of knowledge:
  1. knowledge sets, skills, dispositions, capacities
  2. reformed knowledge sets, skills, dispositions: performative capacities, e.g. exams
Performative knowledge: Wash-back consequences across:
  • teaching and learning
  • the individual
  • nations
Washback consequences: different in different cases and situations.

Assessment: the traditional view:
  • true score
  • errors in testing
False psychometric beliefs:
  • direct process of examination
  • no background to knowledge
  • no internal transformation: when a person is tested, no change occurs when they are tested. The original knowledge set transforms to a performative knowledge set
  • no washback effect: no external transformation. The curriculum is reformed because of standardized tests.
  • unidirectional linear process: no bidirectionality. There is forward and backward flow. The original assessment is unreliable.
  • conflating lower and higher level skills: knowledge of facts vs. synthesis of basic facts - an example of the construct being tested
  • conflating capacity with performance: "translate" for purposes of test
  • culture-free testing vs. national values
  • no problem with transfer
  • develops a certain type of knowledge
  • curriculum-free testing doesn't really exist
  • comparison of different curricula, pedagogies, testing approaches
  • different samples, different values to knowledge, different views of evidence, different national idioms
  • the task of PISA is to iron out these differences
  • "imperfect caricatures" of all knowledge bases under consideration. (Fair testing is very hard to do)
Indicators measured in PISA:
  • quiet place to study: what does this mean?
  • number of books at home: socio-economic status correlation. Number does not equal influence.
Indicators are hard to develop. Testers search for indicators which have:
  • Perceived Task Value: washback effect
  • Motivational effect - difference among children and nations
  • Ironed out ambiguity (this, however, reduces complexity and depth)
A test is a performance. Response to a test is in relation to the types of answer they think are required.

PISA has become a league table ordering a nation's achievement. A league table cannot provide us with useful information. There is a need to design instruments that take into account other variables, a table for improvement, not achievement.

Foucault: "Examination" - Discipline and Punishment
  • 10 pages about education
  • a test: hierarchy, normalizing judgement
  • allows society to construct individuals in certain ways
  • transforms the economy of visibility into the exercise of power
  • introduces the individual into the field of documentation
  • making each individual a case
  • hierarchical organization

No comments:

Post a Comment