Improving accuracy of district coverage estimates of health management information system by annealing it with lot quality assurance sampling community survey data in Nepal.

Research Report.

SUMMARY: The Health Management Information System (HMIS) provides a strong base for evidence informed decision making in the Nepal health system. The information from HMIS is used for planning, monitoring and evaluation of the health system at all levels, which includes monitoring of the Nepal Health Sector Programme III (NHSP3) and Nepal Health Sector Strategy (NHSS) 2015-2020. Nepal's health system needs accurate, comprehensive and disaggregated data to measure its performance, and to identify disparities among social groups and geographic areas. Thus, HMIS is expected to provide valid data to inform strategic and policy level decisions. However, the data that comes from HMIS often has several reporting and recording issues from the point of data generation to the analysis and needs to be assessed from time to time to verify the quality of data. This study was designed to identify and test a mechanism to improve the accuracy of the HMIS. This was achieved by combining a probability sample (lot quality assurance sampling; LQAS) with HMIS indicators using an innovative statistical annealing technique (AT). Eleven health coverage indicators relevant to the Safe Motherhood Programme: antenatal care (three indicators: four antenatal care check-ups, iron folic acid supplementation, and use of anti-helminthics), safe delivery (three indicators: institutional delivery, delivery by skilled birth attendant, and caesarean section) and new born care and postnatal care (five indicators: chlorhexidine ointment application within one hour, breast-feeding within one hour, low birth weight, postnatal care within 24 hours, and postnatal care per protocol) were selected in consultation with Ministry of Health and Population (MoHP) stakeholders. We conducted a LQAS survey in Jumla, Arghakhanchi, and Mahottari districts; one each from the three ecological regions of the country. We gathered related information from all the municipalities and rural municipalities of the selected districts. A total of 551 women with a child 0-5 months age were interviewed from the 29 municipalities across these three districts. In addition to the primary data from the probability sample using LQAS, we also collected the HMIS data for the same reference period which was used to calculate the hybrid prevalence of the selected indicators. Data was analysed using statistical AT in the Stata SE version 15 and R version 3.4.1. For each municipality-level indicator coverage, AT constructs a hybrid estimate as a weighted linear combination of an HMIS estimate and a LQAS probability sample survey estimate, providing a standard error (SE) for both the HMIS and combined estimates. We reported two different calculations of the low birth weight indicator from the probability sample data, thus there are 12 combined indicators for each municipality - that is 12*29=348 in total. The results show that the combined estimate differs from the LQAS estimate by no more than 14.5%, while decreasing the SE by 0% to 5.8%. The majority (327/348=94%) of HMIS and combined estimates, however, can differ by up to 90%. In 5 instances the HMIS estimate is missing. In the remaining 16 instances the difference ranges between 93% and 2050%, and the HMIS coverage ranges between 101% and 2114%. This occurs for the breast-feeding indicator (Patrasi municipality in Jumla, nine municipalities in Mahottari district: Sonma, Bhangaha, Aurahi, Ramgopalpur, Samsi, Loharpatti, Manra Siswa, Pipra, Matihani), the Navi Malam indicator (4 municipalities in Mahottari district: Aurahi, Loharpatti, Pipra, Jaleswor), the iron folic acid indicator (Gaushala municipality in Mahottari district), and the anthelminthic indicator (Loharpatti municipality in Mahottari). Standard errors for the HMIS estimate are between 1 and 78% times larger than those calculated for the combined estimate. The HMIS weighting factor ranges between 0.001 and 0.50 in Arghakanchi, between 0.009 and 0.99 in Jumla and between 0 and 0.996 in Mahottari. In the HMIS data, it was found that 22.4% of indicators over-estimated and 32.5% under-estimated coverage. The results indicate that HMIS needs substantial strengthening to improve reliability to capture the data for the target population. Specific indicators can be targeted as well due to their substantial divergence from the probability sample. The study has shown that using AT with a probability sample could be a potential strategy for strengthening health information systems. The findings from this study are expected to provide ways to generate more accurate information to be used for decision making. Using AT with a probability sample could be one of the ways to generate more precise coverage estimates for appropriate decision making, and we may also have to improve the quality of recording and reporting.