Topical report: Assessment of the systemic evaluation of education in Luxembourg (French)
In the field of education, Luxembourg has been a regular participant in national and international systemic evaluation studies for the past 20 years, in order to be able to base its policy decisions on solid empirical data. This retrospective includes an overview of the types of data and reports available on Luxembourg and their main cross-cutting findings (chapter 2).
The publication of the results of the first PISA studies conducted in the early 2000s came as a shock, as the country and its stakeholders were confronted with a mixed ranking of the performance of the Luxembourg school system compared with that of the other participating countries. The main findings have been consistently corroborated in other studies as well (e.g. ICCS in 2009, ICILS in 2018).
Poor performance across the board was reported in all areas assessed. Pupils’ characteristics (socio-economic, migratory and linguistic background) have a significant effect on their performance in the Luxembourg school system. Equity in educational achievement is considered to be low and represents a real challenge.
These findings concerning the importance of student characteristics in their success were not entirely new, as they had already been documented, in broad terms, in the 1968 Magrip study. What is surprising, however, is that, on the one hand, these findings have changed little over the various assessment cycles and, on the other, the determining factors leading to these results have become even stronger over time. Thus, since their introduction in 2009, the standardised tests (‘EpStan’), the continuous school monitoring programme adapted to the Luxembourg school system, have confirmed the inherent challenges. The main findings of the systemic evaluations carried out over the last 20 years have made it possible to identify 8 priority thematic areas on which the National Observatory for School Quality (ONQS) wishes to focus its action over the coming years (chapter 3).
The “EpStan” results are available to anyone interested on the website dashboard.epstan.lu.
For example, the results of the standardised tests show that in 2019, 39% of pupils in 5e did not achieve level 2 in reading comprehension in German and 74% did not achieve this level in reading comprehension in French. In mathematics, 56% of pupils in 5e did not achieve level 2 (see Figure 3). This over-representation of pupils at lower levels of performance is already apparent in basic education. Again, by way of example, between 42% and 44% of pupils in cycle 4.1 did not achieve the target base level in the three skills measured in the standardised tests in 2019 (reading comprehension in French, reading comprehension in German and mathematics), as shown in Figure 2.
With a view to an international perspective, the approaches of several countries are presented, in order to identify shared findings and lessons to be learned. However, the limits of such a comparative exercise are also examined, as the specific socio-cultural and socio-political context of a country considerably determines the choice and development of the national education model. (chapter 4).
The use of systemic evaluations and their impact on the definition of education policy are analysed over the last 20 years. The role of school monitoring shows two major differences over the period examined. During the first decade studied, systemic evaluations served as triggers to motivate a proactive reform policy. Then, during the second decade studied, the strategy shifted, for various reasons, towards development axes which guided and structured more targeted reform actions, but on a smaller scale (chapitre 5).
L’ONQS conclut le présent rapport en formulant trois recommandations, fondées sur le bilan des évaluations systémiques auxquelles a participé le Luxembourg, adressées aux acteurs politiques, sur base de l’article 4 de la loi du 13 mars 2018 (chapter 6).
Recommandation 1:
The ONQS promotes greater coherence in governance on two levels. Firstly, the formulation of intelligible and measurable objectives for the education system (Bildungsziele) and, secondly, the establishment of a transparent decision-making procedure for education.
Recommandation 2:
The ONQS recommends establishing a long-term strategy for systemic evaluation in order to be able to measure the impact of measures or reforms implemented at national or local level.
Recommandation 3:
The ONQS recommends clarifying the principles for managing and using available data from national and international systemic evaluations or from administrative and organisational operations – for both formal and non-formal education – in order to optimise the use of data, improve the quality of analyses and broaden the scope of studies with the aim of improving school quality.
By way of conclusion, this report highlights the persistent observation of the inertia and limited capacity of the Luxembourg education system to find adequate and sustainable responses to fundamental problems, namely the impact of the socio-economic context of the family environment on pupils’ chances of success, compounded by a very marked pattern of horizontal and vertical segregation and a model of language learning and use that is unsuited to pupils who have not mastered the country’s usual languages from a very early age.
The ONQS proposes to implement a concerted strategy with a view to developing a national curriculum that takes better account of the following elements: the constraints of multilingualism at different levels of the education system, better coordination between formal and non-formal education in order to improve equality of opportunity at the start of schooling, and an assessment of the allocation of resources dedicated to the guidance and support of pupils who do not have the same starting opportunities.
