Total Information Awareness, the Surveillance Precursor

Young Mr. Snowden has revealed the expansive technology-driven exploitation of our everyday data. He and his journalist colleagues have posed policy and strategy dilemmas for governments and citizens who worry about national secrets and privacy. Should we have known this was coming? and who laid the ground work, anyway?

Flash your mind back to post9/11. The always a bit wacky DARPA (Defense Advanced Research Projects Agency) developed a public project named Total Information Awareness (TIA). TIA laid out a patriotic
and inspiring research agenda. No less than all possible data about everybody would be gathered and analyzed to track down those damn terrorists.

Unbelievably, the project chose as its logo that creepy all-seeing eye from the dollar bill. As traced in “The Pentagon’s Brain”, a Naval nuclear physicist, veteran of the DoD revolving door, with a felonious record suggested the project then became its honcho. Admiral John Poindexter was not beloved by all in Washington who remembered the Reagan era and the Iran-Contra fracas. The Public and Congress reacted strongly.

“Wait just a moment, Terrorist Information Wariness, or whatever you call yourself!” Active privacy groups on alert from the Patriot Act together with serious season Senators sent the program back into hiding. Congressional de funding forced DARPA to send many funded projects under the wings of NSA in the secret budget.

Online are the BAA (Broad Agency solicitation ) for the program and a FOIA (Freedom of Information Act) list of funded projects. Look, there’s the eminent Xerox Palo Alto Research Center, PARC, of “Bungling the future” fame. This list read like a major conference program familiar from information analysis and security modeling days. Hmmm, I wonder how many of today’s commodity technologies, such as relevance ranking and speech to text, were funded through TIA, like that erratic, pesky iPhone SIRI

The 9/11 Report would later show that the vulnerability was not too little terrorist information available but rather that government agencies failed to share effectively. Indeed, an ignored FBI report indicated One aviation-oriented university even graduated, in 2000, an Al Qaeda bound explosive developer, now residing at Guantanamo.

The 2013 revelations of NSA data, often characterized as “metadata” collection clearly built on TIA. Bridging military and commercial objectives lead to “big data”, a modest $200 million Obama administration initiative. The pervasive question is “well, how many terrorists have been caught through big data?”. And the answers always seemed “That’s classified”, or a nebulous “about 50”,or “it only takes one to make a difference, do you want another 9/11?”.

Did we miss the big story, when privacy evaporated in the name of safety through surveillance? Not all of us, librarians stood their ground against the so-called Patriot Act and privacy organizations persisted in questioning government policies.

Physicists have well established professional tracks, computer science departments are still evolving, but does any profession stand up for “privacy”? lawyers? philosophers? sociologists? ha, business types? developers? investors? How many of us have taken a regular, if inter-disciplinary, course that covers ‘privacy for professionals’ or even ‘privacy for dummies’? Well, I did, a lifelong learning course which mainly traced complicated Supreme Court cases in the Brandies era that riled up my “right to be left alone”. Privacy is a bag of conundrums

Summary: In the arc of computing history, Total Information Awareness offered the pool of funding that unified the goal of massive mining of citizen data. Partly through resistance to the project’s open flaunting of a honcho not trusted by a Congress then acting on its oversight responsibilities, the project lost public attention, then sprung forth with the 2013 Snowden revelations.


  1. “The Pentagon’s Brain”, by Annie Jacobson, 2015 KQED forum interview “From Agent Orange to AI”

  2. “Information Awareness” office described in Wikipedia

  3. Warning about Total Information Awareness

  4. ACLU connecting TIA with modern NSA surveillance
  5. How TIA lives on, more on data mining

  6. Facebook Nation, Total Information Awareness series by Newton Lee
    Would you believe a Disney engineer wrote an expose of TIA? Mostly web notes, and applied to Facebook, very interesting!

  7. TIA in the words of Admiral Poindexter

  8. Contract awards under DARPA TIA

  9. EPIC (Electronic Privacy Information Center) Analysis of DARPA TIA program

  10. “Note to Self” podcast “Is my phone listening to me?” and Atlantic article by Walter Kern, “If you’re not paranoid, you’re crazy”

All links accessed January 3, 2016