RESEARCH
Dive into our comprehensive research studies illuminating the systemic challenges driving maternal and child mortality and morbidity in India.
Research, Monitoring and Evaluation Model
Efficacy
Effectiveness
Equity
Empowerment
THE 4E MLE FRAMEWORK
The efficacy of ARMMAN’s mHealth programmes is monitored monthly through key data points and standardised efficacy indicators from individual programme dashboards. These indicators may necessitate need-based projects to investigate observed trends and facilitate iterative programme improvement.
Some of these studies are conducted exclusively by the Monitoring & Evaluation (M&E) team, with tools developed by M&E and programme teams and approved by the research team. In some cases, studies are conducted exclusively by the research team, such as qualitative studies and situational analyses.
Effectiveness is measured through regular updates of standardised dashboard indicators and periodic studies at regular intervals (quarterly, half-yearly, yearly) to assess impact, clarify metrics, and understand factors underlying performance indicators.
Methods may include:
- Computer/Tablet Assisted Interview (TATI/CATI) surveys
- Qualitative studies
- Rapid assessments using standardized tools
- Small cross-sectional studies
These studies are conducted by the MLE function within the programme team, with design and tools approved by the research team, or directly by the research team depending on the study type. Long-term impact is primarily assessed through rigorous studies such as randomized control trials (RCTs), quasi-experimental studies, pre-post intervention studies, and cross-sectional studies, all designed and conducted by the Research team in collaboration with the Monitoring & Evaluation (M&E) and Programme teams.
We will further break down the efficacy and effectiveness indicators based on equity factors such as caste, class, religion, age (adolescent mothers), and tribal status. This disaggregated analysis may lead to need-based studies for more nuanced research. These studies could be conducted by the Monitoring & Evaluation (M&E) or Research teams depending on the study type. The Research team will also undertake formal studies on equity related to mobile health (mHealth).
ARMMAN has developed a standard tool to assess the empowerment of pregnant women and mothers resulting from mHealth based information services. We are also developing a similar tool for health workers. Data on empowerment will be collected through rapid assessment studies using these tools (conducted by the Monitoring & Evaluation or Research team) and formal research studies conducted by the Research team.
Research Studies
Healthcare Information For All By 2015: Preliminary findings and future direction
This paper presents the mHIFA initiative (2012–2015) and its scoping exercise on mobile healthcare projects in low- and middle-income countries. Despite many mHealth projects, only nine aligned closely with the mHIFA Goal. Findings were shared for stakeholder feedback.
Know More >The Impact of an mHealth Voice Message Service (mMitra) on Infant Care Knowledge, and Practices Among Low‑Income Women in India: Findings from a Pseudo‑Randomized Controlled Trial
This study evaluates ARMMAN's mMitra programme for pregnant women in Mumbai slums. Findings showed improved infant care practices, maternal knowledge, and a positive trend in birth weight. mHealth interventions can enhance maternal and child health outcomes.
Know More >The Elusive Path Toward Measuring Health Outcomes: Lessons Learned From a Pseudo-Randomized Controlled Trial of a Large-Scale Mobile Health Initiative
mHealth expands access to healthcare but poses challenges in linking technology use to health outcomes. This study shares insights from evaluating mMitra, a voice messaging programme improving maternal and child health in India’s urban slums.
Know More >Missed calls, Automated Calls and Health Support: Using AI to improve maternal health outcomes by increasing program engagement
India accounts for 11% of global maternal deaths, with limited preventive care access worsening outcomes. This research by ARMMAN and Google AI for Social Good used deep learning on call records to predict dropout risk, improving targeted interventions for maternal health programmes.
Know More >Did COVID-19 Have a Positive Impact on Any Aspects of Women’s Health Care?
The coronavirus disease 2019 (COVID-19) pandemic posed enormous and unpredictable challenges to the provision of health care to people (especially women). This study attempts to document the challenges faced by women in accessing healthcare during this challenging period.
Know More >Selective Intervention Planning using Restless Multi-Armed Bandits to Improve Maternal and Child Health Outcomes
India's high maternal and child mortality rates stem from limited preventive care access. For this research conducted by ARMMAN in partnership with Google's AI for Social Good, we used machine learning on call records to predict engagement, improving interventions and boosting maternal health programme participation by 61.37%.
Know More >Prenatal care, new media, and COVID-19 pandemic in India
The COVID-19 pandemic brought life to a standstill. Pregnant women and their foetus were more vulnerable to this virus, and the lockdown made it difficult for women to access prenatal care. This commentary focuses on the challenges faced by pregnant women to access healthcare.
Know More >Field Study in Deploying Restless Multi-Armed Bandits: Assisting Non-profits in Improving Maternal and Child Health
The widespread use of mobile phones enables non-profits to deliver critical health information. This paper describes ARMMAN's work to assist non-profits that employ automated messaging programmes to deliver timely preventive care information to beneficiaries (new and expecting mothers) during pregnancy and after delivery.
Know More >Pandemics and technology engagement: New evidence from m-Health intervention during COVID-19 in India
Despite widespread mobile-based health programmes, engagement barriers persist. Analyses of 2 million ARMMAN call records showed that the COVID-19 lockdown increased call durations, revealing demographic disparities in technology use, with implications for improving maternal health engagement in developing economies.
Know More >Applying Decision Focused Learning in the Real World
Decision Focused Learning (DFL) optimises decisions by integrating learning and optimisation. This real-world study with 9,000 beneficiaries shows that DFL outperforms Predict-then-Optimize in reducing engagement drop in maternal health programmes, proving its practical effectiveness.
Know More >Robust Planning over Restless Groups: Engagement Interventions for a Large-Scale Maternal Telehealth Program
To address the disengagement of mothers from ARMMAN's Interactive Voice Calls (IVR), researchers from Harvard University and Google Research partnered with ARMMAN to develop GROUPS, a double oracle–based algorithm for robust planning in Restless Multi-Armed Bandits (RMABs) with scalable grouped arms.
Know More >Preliminary Results in Low-Listenership Prediction in One of the Largest Mobile Health Programs in the World
Kilkari, an mHealth programme by ARMMAN, delivers Interactive Voice Response (IVR)-based maternal health messages. These results highlight the need for novel machine learning research to help better target ARMMAN’s limited intervention resources.
Know More >Expanding impact of mobile health programs: SAHELI for maternal and child care
Google Research India deployed SAHELI, a Restless Multi-Armed Bandits (RMAB)-based system, to learn from ARMMAN's past data to improve the performance of SAHELI's RMAB model, the real-world challenges faced during the deployment and adoption of SAHELI, and the end-to-end pipeline. This paper describes the approach.
Know More >Limited Resource Allocation in a Non-Markovian World: The Case of Maternal and Child Healthcare
Google Research India optimizes intervention scheduling in low-resource settings by extending Restless Multi-Armed Bandits (RMABs) to continuous state spaces. In this paper, Google Research India studies the problem of scheduling healthcare interventions under limited healthcare worker resources.
Know More >Analyzing and Predicting Low-Listenership Trends in a Large-Scale Mobile Health Program: A Preliminary Investigation
Collaborating with ARMMAN, Google Research India analysed the user engagement, cluster listenership patterns, and use time-series prediction to identify dropouts in Kilkari, helping ARMMAN implement timely interventions to improve retention.
Know More >Scalable Decision-Focused Learning in Restless Multi-Armed Bandits with Application to Maternal and Child Health
This paper studies restless multi-armed bandit (RMAB) problems with unknown arm transition dynamics but with known correlated arm features. The document helps learn about a model to predict transition dynamics given features, where the Whittle index policy solves the RMAB problems using predicted transitions.
Know More >Restless Multi-Armed Bandits for Maternal and Child Health: Results from Decision-Focused Learning
This paper presents the first work showcasing the real-world impact of Decision Focussed Learning (DFL) for Restless Multi-Armed Bandits (RMABs) through a large-scale field study. The paper establishes the practicality of the use of decision-focused learning for real-world problems.
Know More >Public Health Calls for/with AI: An Ethnographic Perspective
The findings of this research uncover complexities around determining who benefits from the intervention, how the human-AI collaboration is managed, when intervention must take place in alignment with various priorities, and why the AI is sought.
Know More >Evaluation of Kilkari & Mobile Academy Project
The document contains findings from the third-party evaluation conducted by the National Health Systems Resource Centre (NHSRC), of the Kilkari and Mobile Academy (KMA) programmes, aiming to gauge service quality, community reach, and acceptance from 2019 to 2021.
Know More >ARMMAN Kilkari Impact Performance Report
This study evaluates the demographic and socio-economic profiles, satisfaction, outcomes, and improvement areas for beneficiaries of Kilkari in Odisha and Assam. The data was collected through telephonic interviews with a sample of 251 beneficiaries.
Know More >ARMMAN Mobile Academy Impact Performance Report
This study assesses the demographic and professional profiles, outcomes, satisfaction levels, and overall impact of the Mobile Academy Course on Accredited Social Health Activists (ASHAs) in Odisha and Assam. The data was collected through telephonic interviews with 154 ASHAs.
Know More >Changes in Knowledge and Management Skills of ANMs To Screen, Manage and Refer Pregnant Women with Five High-Risk Conditions: Results from A Pre-Post-Training and A Six-Month Follow-Up Assessment in Telangana, India
This study highlights the feedback given by Auxilliary Nurse Midwives (ANMs) who were trained through ARMMAN’s Integrated High[1]Risk Pregnancy Tracking and Management (IHRPTM) in Telangana. The programme trains ANMs to identify, manage, refer, and track high-risk pregnancies.
Know More >Improving Health Information Access in the World’s Largest Maternal Mobile Health Program via Bandit Algorithms
This paper focusses on a system called CHAHAK that aims to reduce automated drop-outs as well as boost engagement with the Kilkari programme through the strategic allocation of interventions to beneficiaries.
Know More >Efficient Public Health Intervention Planning Using Decomposition-Based Decision-Focused Learning
This paper provides a principled way to exploit the structure of Restless Multi-Armed Bandits (RMABs) to speed up intervention planning by cleverly decoupling the planning for different beneficiaries of ARMMAN’s interventions.
Know More >A Decision-Language Model (DLM) for Dynamic Restless Multi-Armed Bandit Tasks in Public Health
This paper proposes a Decision Language Model for Restless Multi-Armed Bandits (RMABs) and using Large Language Models to enable dynamic fine-tuning of RMAB policies for challenging public health settings using human language commands.
Know More >Improving Mobile Maternal and Child Health Care Programs: Collaborative Bandits for Time slot selection
This work highlights the collaborative bandit algorithms for solving the time slot selection problem for ARMMAN’s mMitra programme. mMitra programme faces challenges of limited beneficiary phone access and unknown time preferences, leading to poor engagement.
Know More >Evaluating the Effectiveness of Index-Based Treatment Allocation
This paper introduces methods to evaluate index-based allocation policies that allocate a fixed number of resources to those who need them the most, by using data from a randomised control trial.
Know More >Technologies that empower women for better access to healthcare in India – A scoping review
The scoping review is aimed at summarising the range of technologies used by women and assessing their role in enabling Indian women to learn about healthcare services and access them.
Know More >Randomized Cluster Trial: mMitra and Arogya Sakhi
This study determines that improved access to preventive information via mMitra and improved access to home-based diagnostic investigation and referral service via Arogya Sakhi leads to improved health outcomes among rural underprivileged pregnant women and infants.
Know More >