On 7 February 2013, international development partners discussed and analysed during an e-discussion the Triple-S Qualitative Document Analysis (QDA) reports. Participants generally found the QDA reports to be an important and welcomed piece of evidence-based research which effectively draws attention to significant problems in the sector.
Published on: 20/02/2013
On 7 February 2013, international development partners discussed and analysed during an e-discussion the Triple-S Qualitative Document Analysis (QDA) reports.
After a short welcome by Harold Lockwood of Aguaconsult, participants introduced themselves. Online were Henry Northover and Tom Slaymaker from Wateraid UK, Jonathan Wiles from Living Water International, Fabio Losa from the African Development Bank (AfDB) in Tunisia, Ned Breslin of Water for People, and Duncan McNicholl and Mike Kang from Engineers Without Borders (EWB) Canada. Simon Bibby from Department for International Development (DFID), unable to attend at the last minute, sent his apologies. From the Triple-S team were Richard Ward and Ruzica Jacimovic, principle Qualitative Document Analysis (QDA) researchers, assisted by Elise Wach from the Institute of Development Studies (IDS); with Harold Lockwood (Chair), Ton Schouten, Patrick Moriarty, Deidre Casella, Catarina Fonseca, Julia Boulenouar and Sarah Carriger present in the room.
The Practice QDA methodology and results were briefly outlined by Richard Ward. QDA involves a systematic review of documents designed to identify reliable patterns in their content. In the case of the Practice Review, the lens of analysis was the 10 ‘Building Blocks’ of sustainable service delivery, themselves devised after in depth research into the Rural WASH sector across 13 countries. The results of the QDA made clear a ‘theoretically’ well known problem; principally that measures such as asset management and life-cycle cost considerations are rarely covered by either ‘funders’ or ‘implementers’ in their written documents, and variously by extension, their programmes, aims and policies.
Discussion initially revolved around the QDA methodology, with questions from Tom Slaymaker of WaterAid UK on whether it could detect other ways of describing similar practices that did not conform strictly to the Building Block terminology. The response was emphatically that it can and does, via an inclusive analysis of meaning rather than via a more limited ‘word search’ approach (as would be used by automated software). Fabio Losa of AfDB reiterated that the QDA had been well received there, and drew attention to the ‘interpretation’ that the presentation format encouraged; pursuing the ‘why’ question. There was a danger that undue comparison between DPs could overshadow the intention of the research project itself, which is to secure interest and generate constructive discussion and critique around sustainable service delivery in the sector. That being said, it was a healthy competitive streak within the bank that generated much of the initial interest and momentum. Scaling up the QDA with a closer analysis of the bank’s programming and design documents was aired as a potential follow up between Triple-S and the AfDB.
General discussion about how QDA could be improved or scaled up to more fully describe local context and give better weight to DP’s different roles and remits then followed. Both Living Water International (LWI) and EWB Canada thought this was especially important. As smaller NGOs that occupy more niche platforms in the sector, the broad systemic perspective of the building blocks could appear to favour ‘all rounders’ or larger INGOs ahead of the equally effective and valuable interventions of smaller ones. This point was well made. The QDA researchers responded that they were at all times mindful of context and where at all possible did provide ‘contextual caveats’ to reduce unfair presentation. Jonathan Wiles of LWI then asked about tighter analysis criteria, analysis over periods of time and types of document analysed. It was felt that regarding these points, all such modifications are possible with sufficient planning. As a standalone method, QDA is almost infinitely flexible – the most important variables are appropriate document choice and the lens of analysis used. Harold Lockwood reminded the group that there has to be an appropriate balance between the ambition of the methods and available resources expended on this kind of learning activity.
Ned Breslin from Water For People (WFP) commented that the systemic gaps shown by the QDA were clearly a global issue that involved established funding patterns in the aid sector, as well as institutional processes which went far beyond the reach of a single DP, though these were crucially not impossible to modify. More emphasis on best practices relevant to practitioners was encouraged and agreed as important by all participants and the Triple-S team as an aim future expansion of the QDA .
Bringing other sector actors into the remit of a country level QDA analysis was then suggested by EWB Canada. This would bring with it the task of gathering appropriate documents and extending collaboration across the sector, with cooperation from the different parties required, which is challenging. The consensus was that this would be possible and valuable however. Deidre Casella and Patrick Moriarty of Triple-S outlined the intention to expand the QDA method to look at their in country programmes in Ghana and Uganda, analyzing the uptake of Building Block ‘theory’ as ‘effective content’ in planning and programming documentation practices.
Although preliminary in nature and ongoing, the QDA was felt to be an important and welcome piece of evidence based research that effectively drew attention to significant problems in the sector, which could be used to push for broader change. This conclusion tallied with earlier comments regarding the best interpretation of the results and their effective use in any collaborative follow up. Toward the end of the meeting, discussion crystallized around three broad areas: interpretation (of documents and results), criteria (of inclusion and analysis) and adaptation of the QDA to ‘country level’ (either ‘vertically’ through an organization’s whole documentary output or ‘horizontally’ among multiple organizations in a specific context).
20 February 2013
