The Guardian has acquired top-secret documents about the NSA datamining tool, called Boundless Informant, that details and even maps by country the voluminous amount of information it collects from computer and telephone networks.

The focus of the internal NSA tool is on counting and categorizing the records of communications, known as metadata, rather than the content of an email or instant message…

A snapshot of the Boundless Informant data, contained in a top secret NSA “global heat map” seen by the Guardian, shows that in March 2013 the agency collected 97bn pieces of intelligence from computer networks worldwide.

Several industry officials told The Post that the [PRISM] system pushes requested data from company servers to classified computers at FBI facilities at Quantico. The information is then shared with the NSA or other authorized intelligence agencies.

According to slides describing the mechanics of the system, PRISM works as follows: NSA employees engage the system by typing queries from their desks. For queries involving stored communications, the queries pass first through the FBI’s electronic communications surveillance unit, which reviews the search terms to ensure there are no U.S. citizens named as targets.

That unit then sends the query to the FBI’s data intercept technology unit, which connects to equipment at the Internet company and passes the results to the NSA.

The system is most often used for e-mails, but it handles chat, video, images, documents and other files as well.

While once the flow of data across the Internet appeared too overwhelming for N.S.A. to keep up with, the recent revelations suggest that the agency’s capabilities are now far greater than most outsiders believed. “Five years ago, I would have said they don’t have the capability to monitor a significant amount of Internet traffic,” said Herbert S. Lin, an expert in computer science and telecommunications at the National Research Council. Now, he said, it appears “that they are getting close to that goal.”…

When separate streams of data are integrated into large databases — matching, for example, time and location data from cellphones with credit card purchases or E-ZPass use — intelligence analysts are given a mosaic of a person’s life that would never be available from simply listening to their conversations. Just four data points about the location and time of a mobile phone call, a study published in Nature found, make it possible to identify the caller 95 percent of the time.

“We can find all sorts of correlations and patterns,” said one government computer scientist who spoke on condition of anonymity because he was not authorized to comment publicly. “There have been tremendous advances.”…

Industry experts say that intelligence and law enforcement agencies also use a new technology, known as trilaterization, that allows tracking of an individual’s location, moment to moment. The data, obtained from cellphone towers, can track the altitude of a person, down to the specific floor in a building. There is even software that exploits the cellphone data seeking to predict a person’s most likely route. “It is extreme Big Brother,” said Alex Fielding, an expert in networking and data centers.

Sen. Mark Udall, a member of the Senate Intelligence Committee, said Sunday he’s skeptical whether a vast phone record-tracking program is needed to thwart potential terrorist attacks.

It’s “unclear to me that we’ve developed any intelligence through the metadata program that’s led to the disruption of plots that we couldn’t have developed through other data and other intelligence,” the Colorado Democrat said on CNN’s “State of the Union.”

“They’ve really got immediate huge, major leaks that sort of put the AP story and the Fox and Rosen things in a complete different light. Those look relatively minor, while these are huge,” said Ted Boutrous, a media lawyer who has represented journalists caught up in leak inquiries. “I have very little doubt that they will conduct an aggressive investigation.”…

“You can sort of hear the polygraph machines warming up,” quipped Steven Aftergood, who’s been tracking leak probes for two decades for the Federation of American Scientists.

Former officials and investigators said one looming concern is that the leaker might have more documents or access to them and could release them if not caught.

”It’s very, very alarming. You’ve got basically a double agent in your midst taking all your secrets and just handing them out to the press,” said Zeidenberg. “I think that that’s a huge issue.”

In his press conference on Friday, President Obama described the massive collection of phone and digital records as “two programs that were originally authorized by Congress, have been repeatedly authorized by Congress”. But Congress has never specifically authorized these programs, and the Patriot Act was never intended to allow the daily spying the Obama administration is conducting…

Technically, the administration’s actions were lawful insofar as they were done pursuant to an order from the Fisa court. But based on the scope of the released order, both the administration and the Fisa court are relying on an unbounded interpretation of the act that Congress never intended.

The released Fisa order requires daily productions of the details of every call that every American makes, as well as calls made by foreigners to or from the United States. Congress intended to allow the intelligence communities to access targeted information for specific investigations. How can every call that every American makes or receives be relevant to a specific investigation?

Greenwald’s hard-left conspiracy theories are attractive to far-right talk show hosts and bloggers who share a common suspicion of liberal government. This suspicion has been nurtured by the recent IRS scandal and the Justice Department’s overzealous pursuit of journalists. The result has been a debate dominated by the extremes, with little patience for nuance, calibration, or balancing. The reality may be less exciting (and less suited for talk show dialogue) than the paranoid narrative, but the boring reality is what must be addressed if necessary reform is to be implemented.

And reform of the current excesses of surveillance is indeed necessary. There is too much secrecy, too little accountability, too much classification, not enough information, too much speculation. This all feeds into the paranoid streak because we don’t know what we don’t know. For those who trust the government this informational lacunae is an excuse for inaction. For those who do not trust the government, it is an excuse for ranting and raving instead of legislating compromised reform…

So let the debate begin, but don’t let it be dominated by the extremes or fueled by paranoia. We need reform, not revolution—improvement, not impeachment.

For us, the age of surveillance is more likely to drift toward what Alexis de Tocqueville described as “soft despotism” or what the Forbes columnist James Poulos has dubbed “the pink police state.” Our government will enjoy extraordinary, potentially tyrannical powers, but most citizens will be monitored without feeling persecuted or coerced.

So instead of a climate of pervasive fear, there will be a chilling effect at the margins of political discourse, mostly affecting groups and opinions considered disreputable already. Instead of a top-down program of political repression, there will be a more haphazard pattern of politically motivated, Big Data-enabled abuses. (Think of the recent I.R.S. scandals, but with damaging personal information being leaked instead of donor lists.)

In this atmosphere, radicalism and protest will seem riskier, paranoia will be more reasonable, and conspiracy theories will proliferate. But because genuinely dangerous people will often be pre-empted or more swiftly caught, the privacy-for-security swap will seem like a reasonable trade-off to many Americans — especially when there is no obvious alternative short of disconnecting from the Internet entirely.

Welcome to the future. Just make sure you don’t have anything to hide.

There is no way a government in the age of metadata, with the growing capacity to listen, trace, tap, track and read, will not eventually, and even in time systematically, use that power wrongly, maliciously, illegally and in areas for which the intelligence gathering was never intended. People are right to fear that the government’s surveillance power will be abused. It will be. There are many reasons for this, but the primary one is that humans are and will be in charge of it, and humans have shown throughout history a bit of a tendency to play every trick and bend and break laws. “If men were angels,” as James Madison wrote, limits, checks, balances and specifically protected rights would not be necessary. But they aren’t angels. Add to all this simple human mistakes, innocent and not, and misjudgments. And add to that sheer human craziness, partisan lust, political mischief of all sorts. In the Clinton White House there was a guy named Craig Livingstone who amused himself reading aloud the confidential FBI files of prominent Republicans. The files—hundreds of them—were improperly secured and disseminated. Imagine Craig Livingstone at the National Security Agency. Imagine Lois Lerner.

So if we have and develop a massive surveillance state, it will be abused. And that abuse will, down the road, do damage not only to individuals but, quite probably, to the nation’s morale, to its very vision of itself.

But it will make us – or allow us to feel — physically safer. And it may help break real terror networks bent on real mayhem.

Discuss. Really: Discuss.

“For me, it is literally – not figuratively – literally gut-wrenching to see this happen because of the huge, grave damage it does to our intelligence capabilities,” Clapper told NBC News’s Andrea Mitchell…

He said the NSA has asked the Justice Department to find whoever leaked the information.

“I think we all feel profoundly offended by that,” Clapper said. “This is someone who, for whatever reason, has chosen to violate a sacred trust for this country. And so I hope we’re able to track down whoever’s doing this, because it is extremely damaging to, and it affects the safety and security of this country.”

Visit NBCNews.com for breaking news, world news, and news about the economy

“I know your reporter that you interviewed, Greenwald, says that he’s got it all and now is an expert on the program. He doesn’t have a clue how this thing works, and neither did the person who released just enough information to literally be dangerous.”

“Everyone, everywhere now understands how bad things have gotten— and they’re talking about it. They have the power to decide for themselves whether they are willing to sacrifice their privacy to the surveillance state.”