Over the last two decades, primary and secondary school enrollment have increased dramatically in many African countries. We are now in a position to ask: has the striking expansion in access to schooling led to an increase in learning? Many fear this is not be the case. Such concerns have inspired a wave of citizen-led basic learning assessments, which intend not just to diagnose the problem of schooling without learning but also to remedy it by providing the public and policymakers with information that will spur action for change.
Despite the enthusiasm for such assessments, we know relatively little about the impact they have on citizen action and learning outcomes. In order to fill this gap, Twaweza commissioned a series of rigorous evaluations of the Uwezo initiative, which we summarize and present in this synthesis report.
Uwezo (meaning capability in Kiswahili), an initiative of Twaweza, is such a citizen-led assessment that aims to improve competencies in literacy and numeracy among children in Kenya, Tanzania and Uganda. To date, Uwezo assessments have been carried out on a national scale in Kenya every year since 2009, and in Uganda and Tanzania every year since 2010. Uwezo’s Theory of Change (ToC) envisaged action by citizens at multiple levels – from parents to national leaders. The instant feedback given to parents – i.e., the immediate results on how a child scored on the Uwezo literacy and numeracy test – is central to the ToC, as is the hypothesis that as a result of it, parents will be motivated to take action to improve their children’s learning. In order to test this core hypothesis Twaweza commissioned a team of researchers to conduct a rigorous assessment of it.
This overview paper explains that study, summarizes the main findings that arose from it (which were also published in a peer-reviewed publication). In addition, this overview paper also situates Uwezo within other recent research trends and findings on the importance of information in promoting citizen action, and the link between citizens, authorities, and patterns of accountability.
Key Findings from Quantitative Evaluations
The researchers were not able to detect any evidence that Uwezo increased parental involvement or citizen activism. The findings are summarized in the table below.
Table 1: Evaluations of whether Uwezo has increased parental involvement
|Time||Scope of Research||Key Findings|
|Phase I (2011)||Post-treatment field study of 550 households in 26 matched villages in two districts using extensive household surveys||No evidence of increased involvement and citizen activism among parents whose children received the Uwezo assessment.|
|Phase II (2013)||Post-treatment study of 87,265 households in 4,371 villages surveyed in 2013 Uwezo assessment round. Compared results for communities included in the 2012 assessment with those assessed for the first time in 2013.||
Strong effects on respondents’ knowledge of Uwezo and perceptions of impact of programmatic activities.
No evidence Uwezo had impact on outcomes respondents claimed were affected (importance that parents attach to education, parental activism, activism on the part of head teachers, and pupil or teacher attendance)
Emerging hypotheses for lack of impact:
The paper considers what could have contributed to this lack of results.
Uwezo’s Theory of Change holds that as parents and communities become aware of the “crisis” of poor learning outcomes, they “will take concrete steps to improve learning, either through private actions (e.g. pay more attention to homework, follow up with a teacher, pay for a tutor, change schools) or mount collective action.” One possible reason for the lack of impact may well be that this core assumption was unrealistic: that is, knowledge about learning outcomes was not, by itself, sufficient to motivate parents to take “concrete steps to improve learning.”
Barriers to action
- Widespread norms against unofficial collective action
- Actors at local level say they have little influence over many of the key inputs into education
- Lack of information about government officials’ responsibilities
- People tend to look to elites for ideas and action
Design and Implementation of Uwezo Intervention
For means of comparability and ease of implementation, all children assessed by Uwezo are tested at the Standard 2 (second year of primary school) level. As a result, parents (of older children in particular) may have suggested a more positive outcome than the reality.
In addition there is evidence that the instant feedback that is supposed to be presented to parents at the point of assessment was not always provided as expected.
Finally, both the Phase I and Phase II quantitative evaluations suggest that the number of households treated in each village may have been too small to achieve the expected impact.
Following the presentation of key findings, the Phase I evaluation turns a critical eye towards the design of their own evaluation as one way of understanding Uwezo’s apparent lack of impact.
- Possible that study’s sample size was too small to pick up an effect
- Potentially inappropriate measures of citizen action
- Likely not driving finding of null effect
The synthesis further presents a new way of considering the link between information and citizen action (the information to action chain) and considers the impact of Uwezo in a comparative perspective.
Authors: Varja Lipovsek and Ruth Carlitz
Organizations: Twaweza East Africa
Type: Key document
You might also like...
- Are our children learning? Literacy and numeracy across East Africa (6 Jun 2015)
- Are our children learning? A review of the Education For All Goals (26 May 2016)
- Are our children learning? Results from the 2013 annual learning assessment (13 May 2015)
- JiElimishe | Learning outcomes and nutrition (22 Dec 2016)
- Are Our Children Learning? Uwezo Uganda Sixth Learning Assessment Report (15 Dec 2016)
- Are Our Children Learning? Uwezo Kenya Sixth Learning Assessment Report (14 Dec 2016)
- Schools, funds and learning (12 Jul 2016)