Maps and Meaning – Sierra Leone’s Digital School Census and what it means for school-level location and accessibility data collection

Reading Time: 7 minutes

In 2018, the Ministry of Basic and Senior Secondary Education (MBSSE) in Sierra Leone launched the Free Quality School Education (FQSE) programme, making free education compulsory for children up to the age of 18. Described by a spokesperson of the MBSSE as “the most realistic way of developing the human capital of our country and moving Sierra Leone forward” (Sheriff, 2019), the programme saw a surge in enrolment in schools. However, a side effect of this surge in enrolment was increased difficulty in accurately monitoring the education system. MBSSE moved forward from their paper-based Annual School Census to a new digital census to achieve this.

The Digital School Census: an overview

In 2019 the Directorate of Science, Technology and Innovation developed and implemented a Digital School Census (DSC) in under 10 weeks, with funding and support from the World Bank. Below is a brief overview of the process’ key steps:

  • Design. Using an Open Data Kit format, an Annual School Census (ASC) form was created to capture multiple data types at each school, including pictures of facilities and GPS data in addition to numerical statistics (numbers of teachers, pupil absenteeism, etc.).
  • Procurement. Six hundred tablets were procured for the capture of data via the ASC form. Two hundred of these were loaned by the UK Government’s Foreign, Commonwealth and Development Office (formerly DFID) for International Development, and the rest were purchased at a cost of $110 per unit. Bloatware was removed and tablet functionality limited to ensure only on-the-job use. Solar charging equipment was also procured for use in schools without an electricity supply.
  • Training. Using a cascade model, 60 MBSSE officers were trained in a three-day session. These master trainers then trained a further 600 enumerators in school-level data collection.
  • Collection. Schools were requested to prepare their data in advance of enumerator visits. Data collected was uploaded to a dashboard, updated and overseen in real-time by a core monitoring team who provided weekly updates and support to senior management.
  • Analysis. A custom dashboard was created for analysis and dissemination of data, with data analysed and visualised within a week of the collection exercise
Figure 1. A snapshot from DSTI’s Education Data Hub, categorising 10,747 schools by accessibility.
Figure 1. A snapshot DSTI’s Education Data Hub, categorising 10,747 primary schools by accessibility

The DSC was lauded for its innovation and its relatively low-cost model: the entire exercise, including procurement of tablets, cost under $200,000.

The DSC feeds data into the DSTI’s Education Data Hub. Here, the Education Indicators Map presents a visual representation of data collected from nearly 11,000 primary schools that is freely available to education stakeholders and the public. 

Digital School Census: Validation Of School LocatIONS

There is, of course, always room for improvement. An exploration of the indicators on the Education Data Hub raises a number of questions.  How was data collected? How should data be displayed? Who actually benefits from this data? An obvious concern for census-takers and enumerators surveying any school landscape in a low- or middle-income country is the location of schools. This was one of the key data points collected during Sierra Leone’s DSC, recorded by enumerators using the GPS locator on their tablets. School locations were then categorised by terrain type into one of four groups: ‘Easily accessible’, ‘Rough terrains’, ‘Not accessible by road’, and ‘Island’. This is an innovative approach, that paints a differentiated picture and received positive acknowledgement (Namit & Thi Mai, 2019). The last category ‘island’ is easy to confirm by comparison against satellite imagery, but the first three present a more complex picture. 

In this blog post, we present some of the issues present in the data set for school locations. It is important to look at such issues, but they are inherent in any geospatial data set – particularly one that has been collected at record-breaking speed. Our first example comes from the outskirts of Makeni, the country’s fifth-largest city, in Bombali District. Two data points are almost overlapping, listing ‘Roman Catholic Teko Service Primary I’ and ‘Roman Catholic Primary 2’ as separate schools. The former has 565 students and 20 teachers, the latter 600 students and 23 teachers. 

Figure 2. Two overlapping schools, or an error in visualisation?

While the information pages for both schools seem to feature pictures of the same school, one is listed as ‘Easily accessible’ while the other is listed as ‘Not accessible by road’, despite being fewer than 10 metres from an unsealed road which is clearly visible on satellite imagery. This duplication may be due to the ‘AM/PM Issue’ — already highlighted in the inception report for MBSSE’s new tablet-based Sierra Leone Education Attendance Management System (SLEAMS) programme. Schools employing a ‘double-shift’ schedule — with two cohorts of teachers and pupils sharing the same school building for half a day each — operate effectively as two separate schools within the same space, hence the two distinct school listings. However, it still fails to explain the difference between the two very different access ratings.

This pattern is repeated elsewhere, such as in Bumbuna, Tonkolili District; schools which are distributed over urban and peri-urban areas being classified as unreachable by road even when GPS data indicates that they are located directly next to main roads.

Figure 3. Inconsistencies in Bumbuna, Tonkolili

The categorisation of school accessibility is in direct response to the following question asked in the DSC: ‘How accessible is the school?’ Have some enumerators or schools erroneously misreported data? Do schools have the chance to correct this information once it has been submitted to the centralised data dashboard at DSTI?

A second example compares two schools northwest of Binkolo, Bombali District.  The first, a small building set back around 40 metres from the road, is identified as ‘Easily accessible’.

Figure 4. Smooth going…

The second, a smaller school around the same distance from the road and approached by a path visible on satellite imagery, is identified as belonging in the ‘Rough terrains’ category.

Figure 5. …versus rough roads

The schools are situated less than two kilometres apart along the same section of road with a minimal elevation gain between them. So why the inconsistencies? Could it be down to the perception of an enumerator visiting the schools in different weather conditions, which in rural areas affect road conditions? Or are there other factors at work which aren’t visible from geospatial data?

These examples highlight several questions which need to be answered in order to make the ASC’s location and access responses not only more consistent but more effective in supporting data-driven decision making in the education sector.

  • How are terrain types defined? – Which factors are considered in determining the difference between schools identified as ‘Easily accessible’ and ‘Rough terrains’? How far does a school have to be situated from a road to be considered ‘Inaccessible by road’? DSTI were contacted on this point for clarification, but at the time of publication no response had been received.
  • What guidance is given to schools compiling their own data? – Who is responsible for deciding the categorisation of their schools’ accessibility: enumerators, or schools themselves? Are they privy to the above criteria? Can schools re-submit data to the central DSTI dashboard directly if it is inaccurate or otherwise misreported?
  • Who cleans the data and how? – Are DSTI or MBSSE staff responsible for cross-referencing given GPS data against satellite imagery of schools? If schools have been reported twice, are these anomalies recognised and removed from the dataset? To design and implement a national-level school census in only 10 weeks is impressive for any nation, but if it doesn’t allow enough time for data cleaning then it reduces the efficacy of the exercise to putting coloured dots on a map.

Policy and programming decisions made upon data that do not present an accurate or consistent picture of the education landscape — whether through a misalignment in views between those designing the data collection and those providing it, or through errors in cleaning and visualising collected data — risk the potential for oversight of issues and wasting of time and resources, a point long laboured by academics and commentators (Benguy, 2019). Development actors — communities, NGOs, governments, donors — need to have a level of confidence in accepting the veracity of published data to make decisions. Whilst data collection in low- and middle-income countries is invariably challenging — particularly in sub-Saharan Africa where international donor priorities sometimes clash with national government priorities — there is a need for reliable data.

Conclusions

The MBSSE’s effort to involve schools in education management information systems is encouraging, but further work needs to be done to ensure that those responsible for providing the information are given structured and consistent guidance on data collection and provision. Data, even taken on its own, is rarely neutral: it often reflects the way that its collectors are viewing — or are trying to view — the world around them. Effort must be taken to align the perspectives of those designing data collection instruments, those using the data for decision making (such as on teacher deployment, building of new schools, and allocation of teacher incentives), and those actually providing the data; otherwise, it becomes a top-down tool which fails to fulfil a limited function for its limited pool of users. 

The Digital School Census conducted in Sierra Leone was — and still is — a remarkable accomplishment, but it is clear that a vision of its intent and use is not yet shared between education policymakers and education providers. It should be seen as one of many steps on the path to a national EMIS system that involves education stakeholders in the provision and use of data in decision making.

References

Björn Haßler
Björn Haßler