Consider this confirmation of an earlier analysis reported by the Washington Post and NBC in the pre-Christmas doldrums. Despite repeated claims by the White House that the NSA’s phone collection program may have prevented as many as 50 terrorist attacks against the US, another independent study found that the surveillance “has had no discernible impact on preventing acts of terrorism” (via the Daily Beast):

An analysis of 225 terrorism cases inside the United States since the Sept. 11, 2001, attacks has concluded that the bulk collection of phone records by the National Security Agency “has had no discernible impact on preventing acts of terrorism.”

In the majority of cases, traditional law enforcement and investigative methods provided the tip or evidence to initiate the case, according to the study by the New America Foundation, a Washington-based nonprofit group.

The study, to be released Monday, corroborates the findings of a White House-appointed review group, which said last month that the NSA counterterrorism program “was not essential to preventing attacks” and that much of the evidence it did turn up “could readily have been obtained in a timely manner using conventional [court] orders.”

That’s not to say that the report is entirely conclusive, either.  The New America Foundation took a look at 225 cases in which individuals were charged with “jihadist terrorism.” Only 1.8% of these cases — four, for those who don’t want to do the math themselves — involved Section 215 surveillance.  Three other cases involved “NSA surveillance under an unknown authority,” but most surprising might be that only 4.4% of cases were detected under the non-controversial Section 702 authority for surveilling foreign phone calls.

NAF offers this graph to depict the context:

What has been the most effective method? Of those explicitly stated, it’s “if you see something, say something.” Community and family tips accounted for 17.8% of the origins of investigations, followed closely by 16% started by other informants. However, 27.6% are categorized as “unclear” — the largest single category, which makes the analysis much less reliable,  and might raise questions about how the NSA got those FISA warrants.

Still, there were four cases unearthed through Section 215 (and possibly more in the “unclear” category). Is that enough to justify the program as a necessary evil to prevent worse evil from prevailing? Former acting CIA Director Michael Morell argued three weeks ago that one case is enough:

Another misperception involved the review group’s view of the efficacy of the Section 215 program; many commentators said it found no value in the program. The report accurately said that the program has not been “essential to preventing attacks” since its creation. But that is not the same thing as saying the program is not important to national security, which is why we did not recommend its elimination.

Had the program been in place more than a decade ago, it would likely have prevented 9/11. And it has the potential to prevent the next 9/11. It needs to be successful only once to be invaluable. It also provides some confidence that overseas terrorist activity does not have a U.S. nexus. The metadata program did exactly that during my last days at the CIA this summer, in the midst of significant threat reports emanating from Yemen. By examining the metadata, we were able to determine that certain known terrorists were most likely not in phone contact with anyone in the United States during this specific period of concern.

Personally, I would expand the Section 215 program to include all telephone metadata (the program covers only a subset of the total calls made) as well as e-mail metadata (which is not in the program) to better protect the United States. This is a personal view; it is not something the review group opined on or even discussed. Such an expansion should, of course, fall under the same constraints recommended by the review group.

The idea that we can do a better job protecting privacy and civil liberties at no expense to national security is also incorrect. The review group believes there will be costs but that they will be manageable. This trade-off is at the crux of what President Obama needs to decide about whether and how to amend current programs.

Take the Section 215 program again. The review group’s recommendations will, if adopted by the president, slow the process of searching the metadata. No doubt about it. It will take time to prepare a justification for the Foreign Intelligence Surveillance Court, for the court to render a decision and for the NSA to reach out to the private holder of the data. But the review panel believes this loss of flexibility is both manageable — we allow for exceptions in emergency situations, for example — and worth the protection of personal freedom it provides.

That will be the debate. Barack Obama has to decide soon what reforms to accept for NSA’s operations. So far, though, the controversial Section 215 program hasn’t provided much to support its continuing operations, but any executive that ends it takes the enormous political risk of having that decision get associated with any successful attack in the future — which is exactly what Morell argues will happen. I suspect that Obama will choose the safe path of reforming Section 215 rather than eliminating it.