This article originally appeared on the World Bank Intranet under the title "Systematic Monitoring at Work" by Sonya Fatah, on July 11th, 2014.
July 11, 2014 (THIRUVALLUR, Tamil Nadu): A large group has gathered in the assembly hall of the district office of the Pudhu Vaazhvu or “New Life” project, a poverty alleviation and empowerment project in India’s southern state of Tamil Nadu. The project’s district program manager, Sivagami Sreekrishna, listens as two external auditors detail financial irregularities, bad bookkeeping or poor data management across the district. The numbers projected on a large screen at the front of the room keeps everyone on their toes.
“What explanation do you have about the irregularities in your block?” Sreekrishna asks reps from poor performance areas identified through the survey.
MIS data from over 4,000 village panchayats is reviewed across district meetings every month. Some 294 Project Facilitation Teams (PFTs) comprising 1470 frontline field personnel are reviewed in meetings like the Thiruvallur one.
Sometimes the explanations provide insight into what isn’t working in the project. “It also helps us keep a tab on those things that can go wrong if we are not taking any caution and is an opportunity for midterm correction,” says R.V. Shajeevana, the Additional Project Director.
The Social Observatory’s Inputs: Making Process Monitoring Count
Although process monitoring has been used in the project since 2008, its impact on the project was, at first, merely procedural. “We were just going about it in a routine way,” explains Shajeevana.
“Every month [the external agency] would come and give a presentation…and at the end of every presentation we would fight amongst ourselves about what a waste of time this was.”
Enter the Social Observatory, a division of the World Bank, that helps projects conduct rigorous impact evaluations, develop effective monitoring systems and design case studies and innovations that allow interventions where there are systemic implementation issues.
“Process monitoring helps inform projects about bottlenecks both at the level of management, and in the field,” explains Vijayendra Rao, who heads the SO. “Most importantly it helps establish a feedback loop between different levels of project management that makes decision-making more integrated and effective.”
The SO team helped the project folks make use of their store of reports. By 2010, the project had begun a more intensive analysis of its process monitoring system. The SO team in Tamil Nadu, headed by Nethra Palaniswamy, who coordinates the Social Observatory’s work, began to look for other ways in which process monitoring could be more effective.
“They started asking us, “Where are your dashboards?” Why don’t you integrate data from different spaces to get a more holistic view?” remembers Shajeevana.
For Shajeevana and the rest of the state team, the rewards gradually became clear: an objective assessment of the project’s current standing, and an identification of problem areas.
Today, these reviews cover almost 1,500 frontline field staff working in more than 4,000 village panchayats. They are conducted through a random rotating sample and every village panchayat is covered at least once over a specific period of time without knowing about the review in advance.
One additional way in which process monitoring has helped the project has been with the project’s scaling up to 10 new districts.
“The pressure to rapidly increase the pace of implementation and disbursements in new districts - which required substantial recruitment and induction of new staff, plus fast social mobilization - benefited from the regular feedback on what aspects were working well and the processes that were slipping or needed streamlining,” says Kevin Crockford, the World Bank’s Task Team Leader for the project.
Indeed, the project is good example of learning by doing. For instance, the project team has begun using aids that convert traditional pie charts and bar charts into understandable tools of communication. This is especially important in low or limited literacy contexts.
Project officials soon hope to introduce these visual dashboards at the Village Panchayat Resource Centre as well so that those who cannot read or write very well or at all, can learn to understand their SHGs progress through graphic and pictorial documentation.
Regular meetings ensure communication at all levels about the project’s current stability. In addition to the district level meetings, there are also state-level meetings where district teams are monitored for progress.
The challenges are wide-ranging, and include, among other things, un-transferred money for a skills training program, bureaucratic hurdles like certificate verifications and health card issuance and lack of internet access in some VPRC.
And the buck doesn’t stop there. “We are now collecting data across inputs,” says Shajeevana. “When we notice deviations through process monitoring, we ask the agency to develop smaller studies to explain those deviations."
