The problems with PISA’s evidence-based recommendations arise from neglecting or simply ignoring the side effects. Side effects in education happen for a number of reasons. First, education has multiple goals, but the effects of an education intervention can be different on different goals. A policy or practice that is effective in accomplishing one goal can impede the realization of other goals. In the case of PISA, even if the factors that make an education system successful in achieving high test scores could be identified with precision, it is perfectly possible that they might hurt student well-being. Until 2015, PISA identified successful education systems based on student performance on its assessment. East Asian education systems, for example, have been found to be effective in producing excellent performances in tests and hurting students’ social emotional, psychological, and physical well-being, resulting in less life satisfaction, less positive attitude, and lower-levels of confidence.
Second, policies and practices that benefit some students can hurt others due to individual characteristics. Outcomes of treatments are always the results of interactions between characteristics of the treatment and of the individual or aptitude-treatment interaction (Cronbach, 1957; Cronbach & Snow, 1981). This is evidenced by the inconsistent finding about the effect of the quality of teachers in PISA because different qualities of teachers can have different effects on different students. The fact that PISA has found inconsistent impact of class sizes or long studying hours can be due to the possibility that small classes or long studying hours benefit some students but affect others negatively.
Third, policies and practices that work well in one context may cause harm in others. The unintended consequences of borrowing policies in education reform are well documented and evidenced (Harris & Jones, 2015). In the PISA case, this explains why some countries, actually only a very small number, have seen rapid improvements after adopting policies endorsed by PISA, while many others have not seen improvement. The countries performing less well after adopting PISA strategies tend not to be in the international limelight, and the side effects, or negative effects, of policy interference are less well known.
(versão mais longa aqui)