Both NAF [New America Foundation] and BIJ [Bureau of Investigative Journalism] claim that they have assembled a database which covers each individual strike in Pakistan in detail. Unfortunately, both efforts fundamentally rely upon Pakistani press reports of drone attacks. Both claim that they use non-Pakistani media reports as well. For example the BIJ explains in their methodology discussion that the “…the most comprehensive information on casualties lies in the thousands of press reports of drone strikes filed by reputable national and international media since 2004. Most reports are filed within a day or two of an attack. Sometimes relevant reports can be filed weeks – even years – after the initial strike. We identify our sources at all times, and provide a direct link to the material where possible.”
Similarly, NAF explains that their database “draws only on accounts from reliable media organizations with deep reporting capabilities in Pakistan, including the New York Times, Washington Post, and Wall Street Journal, accounts by major news services and networks—the Associated Press, Reuters, Agence France-Presse, CNN, and the BBC—and reports in the leading English-language newspapers in Pakistan—the Daily Times, Dawn, the Express Tribune, and the News—as well as those from Geo TV, the largest independent Pakistani television network.” Indeed, BIJ relies upon the NAF sources as the organization states in its methodology explanation.
While these methodologies at first blush appear robust, they don’t account for a simple fact that non-Pakistani reports are all drawing from the same sources: Pakistani media accunts. How can they not when journalists, especially foreign journalists, cannot enter Pakistan’s tribal areas? Unfortunately, Pakistani media reports are not likely to be accurate in any measure and subject to manipulation and outright planting of accounts by the ISI (Pakistan’s intelligence agency) and the Pakistani Taliban and affiliated militant outfits.
Charli Carpenter responds over at Duck of Minerva:
It's hard to argue with her claims that drones might be more discriminate than 'regular airstrikes,' an argument that largely resets on her observation that the drone program is more highly regulated and this would be obvious to the public if the CIA didn't have a variety of incentives to keep mum about the details. But in the absence of good data comparing the kill ratios – which we really don't have for non-drone-strikes either – it's hard to make this case definitively. Also, relative to what? A law-enforcement approach that involved capturing and trying terrorists rather than obliterating them might or might not be more 'pro-civilian' – though it would certainly be more costly in terms of military life and assets. We simply don't know.