Washback consequences: different in different cases and situations.
Assessment: the traditional view:
errors in testing
False psychometric beliefs:
direct process of examination
no background to knowledge
no internal transformation: when a person is tested, no change occurs when they are tested. The original knowledge set transforms to a performative knowledge set
no washback effect: no external transformation. The curriculum is reformed because of standardized tests.
unidirectional linear process: no bidirectionality. There is forward and backward flow. The original assessment is unreliable.
conflating lower and higher level skills: knowledge of facts vs. synthesis of basic facts - an example of the construct being tested
conflating capacity with performance: "translate" for purposes of test
culture-free testing vs. national values
no problem with transfer
develops a certain type of knowledge
curriculum-free testing doesn't really exist
comparison of different curricula, pedagogies, testing approaches
different samples, different values to knowledge, different views of evidence, different national idioms
the task of PISA is to iron out these differences
"imperfect caricatures" of all knowledge bases under consideration. (Fair testing is very hard to do)
Indicators measured in PISA:
quiet place to study: what does this mean?
number of books at home: socio-economic status correlation. Number does not equal influence.
Indicators are hard to develop. Testers search for indicators which have:
Perceived Task Value: washback effect
Motivational effect - difference among children and nations
Ironed out ambiguity (this, however, reduces complexity and depth)
A test is a performance. Response to a test is in relation to the types of answer they think are required.
PISA has become a league table ordering a nation's achievement. A league table cannot provide us with useful information. There is a need to design instruments that take into account other variables, a table for improvement, not achievement.
Foucault: "Examination" - Discipline and Punishment
10 pages about education
a test: hierarchy, normalizing judgement
allows society to construct individuals in certain ways
transforms the economy of visibility into the exercise of power
introduces the individual into the field of documentation
David Berliner gave a keynote speech entitled, "Contexts of High-Stakes Testing: The 'Quality' of the PISA Results and the Challenges of PISA to Definitions of School Knowledge"
Here are the notes from the speech:
PISA - not high-stakes testing in the USA, unlike Germany
State tests (50 of them) have sanctions
NAEP (National Assessment of Educational Progress), TIMSS (Trends in Mathematics and Science Survey), AP (Advanced Placement), SAT (Scholastic Aptitude Test), ACT (American College Testing) are indicators of quality
No Child Left Behind
Same context of interpretation: uncritical acceptance of scores as validation of teachers and school quality
Scores - a prophesy: transmitting scores into predictions
Media focus on USA's poor performance in scores
Press attention - US students "below average"
PISA 2006 focus on science: Glee in the USA, as newspaper writers like bad news, and it promoted vouchers and charter schools
High PISA scores come from countries with strong public schools and national standards. *Finland does not*
High-stakes testing popular in the USA - competition
Last half of the 20th century was American-dominated in terms of education
No longer the case as illustrated by PISA
Perpetuates the myth of failed education
David Brooks of the New York Times: PISA scores are shaping the nation
Tom Friedman of the New York Times: USA in PISA is like Slovakia or Portugal, not like Canada or The Netherlands
USA in PISA at the international average
Media does not report:
high scorers in PISA's Level 6 above the OECD average
240,000 Finnish students are at Level 6
7.4 million US students are at Level 6
this does not indicate decline in education for the USA
trouble in finding jobs: in the USA, 25% of the top scorers were at Level 6 in scientific literacy. In 2001 there were 758,000 bachelor's degrees in science, although 1/3 of these students did not stay in science. In 2001 there were 160,000 master's degrees in science, but 2/3 of these students do not have science jobs.
PISA tests not what is learned in schools, but what is learned over life
USA - wrong performance analysis. PISA purposely unrelated to curriculum and not related to school
PISA - GDP per capita and number of youth (percentage of school age population) can predict PISA scores, for example in Finland
USA - breakdown of race: White students (40% of the school-age population) scored similarly to the Japanese in PISA. Minority students score lower.
in the USA, SAT scores directly correlate to family income, e.g. wealthy students perform well in PISA, the SAT, etc.; poor, minority students do not perform well; Anglo countries have much child poverty
US education policy: by 2015, children must be "proficient," but these things are not taken into consideration:
the possibility of cheating, giving answers, etc.
high-stakes testing corrupts schools
Campbell's Law: whenever a social indicator takes on too much value, the actors/creators will become corrupt
the narrowing of the US curricula: No Child Left Behind led to an increase in reading and math instruction. The more time devoted to learning must lead to more achievement, but with the exception of No Child Left Behind. Too much drilling? Something is going wrong.
No Child Left Behind was created to reduce the gap in learning, but the gap has not closed
England replicated the policy and received the same results:
narrowing of curriculum
only rewarding academic achievement
narrow view of the "educated child"
taking time from other subjects to focus on mathematics and reading. In the USA, time was taken from recess, social studies, art, music, and lunch.
USA has an "apartheid curriculum," for example, areas of low poverty can focus on art and music, while areas of high poverty concentrate on test preparation, mathematics, and reading.
This is VUCA (Volatile, Uncertain, Complex, Ambiguous)
we are heading to this world
we need breadth in the curriculum
a VUCA world would seem to demand breadth. We need breadth to survive. We won't evolve properly. We need variation of talents and skills to survive.
Tom Popkewitz gave a keynote speech entitled, "PISA - Numbers, Standardizing Conduct, and the Alchemy of School Subjects"
The notes from his speech are as follows:
A content/context-free test is an illusion.
The Practical Knowledge of PISA helps us "see" historical assemblages and cultural theses about modes of life. It is like a recipe. Ingredients lead to a final product. PISA does not see the "ingredients." It is seen as a whole, but knowledge is not practical. PISA has become a cultural phenomenon, it is talked about and acted upon.
The Reason of PISA - Transnational Governance, National Salvation, and the Political Numbers as a Telling truth. Here, numbers are an important part of social science. "Facts" are uniform, objective, and rigorous. Democracy has a correlation with measurement. There is a paradox of numbers: they appear as fact through the making of that fact.
Numbers and Assessment in Making Equivalences: Global Positioning Systems. PISA monitors student improvement and student performance. A GPS always knows its position. Constant measurement and performance helps locate oneself in the world (Simons) and gives navigational tools (Lindblad). This does not act directly on people.
PISA: The Alchemy and School Subjects. "The alchemy" is an analogy to medieval practices to think about pedagogical translations. PISA has little relation to interaction and communication of the academic fields. The "practical knowledge" assembles a cultural thesis about the "lifelong learner," a mode of life who the child is and should be.
PISA as Cultural Thesis of the Lifelong Learner: Who Made the Recipe of Practical Knowledge? A lifelong learner is flexible, active, and collaborative. An uncertain world needs problem solving. This works in a global world with no finishing line. Life continues responsibly. The only choice is to choose.
Science: has textbooks to emphasize greater participation and problem solving to learn the symbolic systems of science.
Mathematics: has modeling and predicting of real-world phenomena
Disciplinary knowledge: stable, logical, and analytic structures separated from the social and cultural fields in its production.
Scientific Literacy: no consensus across countries. It is NOT a universal language (in textbooks)
PISA, School Subjects, and Historical Context: The Planning of Science in the Redemption of Society. Science uses the future to reorganize the present. 19th century science went from the mastery of the physical world to the mastery of the social world. Science as planning: it was to predict the future.
PISA and the "Reason" of Practical Knowledge: Exclusion and Abjections in the Impulse to Include. Alchemy: double gestures -- hope for the future and fears of dangers. Education for all: "all" assumes a unity and a consensus. "All" erases differences and the inscription of psychological and social differences.
I. PISA is not merely measures of practical knowledge of the future. What is practical knowledge? There are different contexts.
II. The politics of school.
Alchemy: knowing occurs as "practical."
Installation of "shepherds" to guide what is "practical."
PISA: hope for the future, the dangers, and dangerous of populatoins.
III. The limits of the present.
IV. Historical situations: thinking about the present with implications and consequences.
Ulf P. Lundgren gave a keynote speech entitled "PISA as a Political Instrument: the history behind formulating the PISA Programme"
Ulf P. Lundgren is a Professor of Policy and Philosophy of Education at the University of Uppsala and the Director-General of the National Agency of Education in Sweden.
The notes from his speech are as follows:
The history behind PISA has two interpretations:
a short history
reforms in policy followed Zeitgeist, the changes in society
PISA really started in the 1980s.
Educational measurement began in the 19th century. In 1862, England began assessment and inspection related to outcomes. The measurement related to intelligence testing, along with educational and psychological testing.
Education is important for society and individuals. For individuals, education is a link to a new life. Education also has a link to the labor market. The modern evaluation is a social context where everything could be questioned.
Assessment techniques developed. The international Examination Inquiry of the 1930s formed an international network of assessment, but few with comparative ambitions.
In 1957, Sputnik occurred, and President John F. Kennedy announced that a US man must be first on the moon. The "space race" tied in education to space outcomes.
The IEA was founded officially in 1967 and began comparative study. International comparisons became features on political agendae and international comparativeness was born. Human capital theory was established.
The 1970s oil crisis increased international competition. International assessments became more important, and international assessments were seen as tools for educational improvements.
The Cambridge manifesto in 1972, which encouraged broader methods of educational research (including qualitative research) paralleled the 1970s trend which attacked school systems and statistics in education.
In the 1980s, the Welfare Society came under attack. New policies were implemented. Globalization gave new solutions, such as decentralization and marketization of schools.
In the US and UK there was a decentralization of policy, but a centralization of inspection.
In 1989 the Berlin Wall fell, and in 1992, the USSR ended, which marked the end of the Cold War. Competition was not about domination. A new context for competition began: economic. ICT began to emerge. Teachers began to be seen as professionals.
International assessments have changed its context over time.
In 1968, the OECD's Centre for Education Research and Innovation (CERI) was started. In the 1980s, the OECD's INES, the education indicators program began. Statistics began to impact upon education policy.
INES built up connections and networks in different countries. In 1992, Education at a Glance was born. However, there were problems in finding outcome data. At the time there were only the IEA studies.
Tom Alexander of CERI said the OECD needed a new program, as there were validity problems with the IEA studies. They wanted a test that was "content free." ETS (Educational Testing Service) created a broad literacy concept test. Statistics Canada handled the design and data collection.
From this came PISA, first administered in 2000, and with those results released in 2001.
The CESE Symposium held on the island of La Palma also hosted a poster exhibition. These are the posters presented at the Symposium:
A Portrait of European Top Performers in Science by Fabio Alivernini, Laura Palmerio, Valerie Tortora, National Institute for the Evaluation fo the Education System, Frascati, Italy
Thinking About PISA and the Participation of the Parents by Claudio Almonacid Aguila, Ana Jimenez Saldana, Daniel Rios Munoz, University of Santiago de Chile
An Investigation of Reasons for Finland's Success in PISA by Jennifer H. Chung, Liverpool Hope University
From the Static to the Dynamic: PISA Items as an Educational Resource by Jose R. Galo Sanchez, Descrtes Project, Institute of Education Technologies, Ministry of Education, Spain; University of Cordoba
PISA, Finland and the Autonomous Community of Madrid by Maria Jose Garcia Ruiz, UNED
Digital Skills Appear in PISA Report: Are Students and Teachers Prepared? by Ana Garcia-Valcarcel, Francisco Javier Tejedor, Anunciacion Quintero, University of Salamanca
Difference of Gender in the Educative Practices and Expectations of Students by Carmen Nieves Perez Sanchez, Moises Betancort, University of La Laguna
PISA on Front Page by Ariadne Runte-Geidel, Diego Sevilla, University of Grenada
The Educational and Sociolabour Guidance Program: An Instrument to Facilitate the Sociolabour Transition by Lidia E. Santana Vega, Jose A Santana Lorenzo, University of La Laguna
Similarities and Differences Between Spanish and Finnish Teachers: Organization of School Time and Space, Selection and Salary by Begona Zamora Fortuny, University of La Laguna
The New Challenges Posted by PISA for a Graduate Programme in Upper Secondary Education Teaching at the National Autonomous University of Mexico by Juan F. Zorilla, UNAM, Mexico
The Comparative Education Society in Europe (CESE) held in international symposium in La Palma, Canary Islands, Spain from the 23rd to the 26th of November 2009. This was organized by Miguel A. Pereyra, President of CESE and Professor of Comparative Education at the University of Granada.
The Symposium featured these Keynote Speakers:
David C. Berliner of Arizona State University
Antonio Bolivar of the University of Grenada
Julio Carabana of the Complutense University of Madrid
Robert Cowan of the Institute of Education, University of London
Ulf P. Lundgren of the University of Uppsala
Katharina Maag Merki of the University of Zurich
Gerry Mac Ruairc of University College Dublin
Clara Morgan of Carleton University
Donatella Palomba of the University of Rome "Tor Vergata"
Anselmo Roberto Paolone of the University of Rome "Tor Vergata"
Thomas S. Popkewitz of the University of Wisconsin - Madison
Javier Salinas of the Complutense University of Madrid
David Scott of the Institute of Education, University of London
Daniel Santin of the Complutense University of Madrid
I was recently awarded the degree of Doctor of Philosophy at Oxford University entitled "An Investigation of Reasons for Finland's Success in PISA."
I am available for article writing, consulting, and interviews.
Look for me on academia.edu: http://oxford.academia.edu/JenniferChung/About.
Look for me on Twitter: @DoctorChungie