Open Letter from Recipients of the Norbert Wiener Award
October 16, 2004
We write as former recipients of the CPSR Norbert Wiener Award to express our concern about the significant redirection in science funding toward the development of systems of mass surveillance. It is our view that this research priority could pose a fundamental risk to political freedom, privacy, and Constitutional liberty.
Since 9-11, there has been an understandable shift in the nation's research priorities. New work is under way to help detect the threat of future terrorist acts. For example, research to detect explosive materials, dangerous gases, and other potentially lethal substances is critical.
However, there are special risks associated with the development of systems of mass surveillance that must be addressed. Unlike techniques that identify dangerous substances, techniques of surveillance enable identification of virtually any subject. The result is invariably that research that is pursued for the narrow purpose of fighting terrorism, over time, takes on many other objectives. This is already apparent in such areas as passenger profiling, video surveillance, and network analysis.
Left unchecked, the consequence of this development could be the adoption of systems of mass surveillance unrelated to any terrorist threats. This will give the government sweeping new capability to monitor private life and thus diminish the freedom and liberty of Americans.
It is vital that the social and political consequences of these projects be understood at the outset. Privacy and security issues should be addressed before they are developed and deployed. Privacy requirements should be addressed earlier in basic research, in the specifications for any system procurements, and in operational practice.
Previously, the Congress and the science agencies recognized the special need to address the impact of new technologies. The program on the Ethical, Legal, and Social Implications of the human genome project provided funding for researchers to assess the far-reaching impact of this technology. This work produced many important results across a broad range of areas and has better prepared the scientific community, and our country, to address the changes brought about by human genome research.
Similar research must begin in the area of privacy and security. Too much money is spent today on systems of mass surveillance; too little is spent on understanding the social consequences.
We call on the National Science Foundation, DARPA, the Department of Homeland Security and other relevant agencies to determine whether adequate safeguards are being developed to protect the civil rights of the populations who will ultimately become the human subjects for the deployment of these systems.
We call on the Congress to set aside funds to allow for a candid and independent assessment of the ethical, legal, and social implications of this technology.
The American public has repeatedly made clear that it does not support the establishment of vast systems of public surveillance. Yet our science agencies and many of our top researches are now pursuing precisely this mission.
We believe this must change.
Peter G. Neumann
Theodore A. Postol
Eric S. Raymond
Richard M. Stallman
Federal Government Spending on Surveillance Technology 2005
This report provides a brief survey of programs involving surveillance
technology currently being funded by the federal government. It is
based on publicly available materials and does not include, for
example, research that is being pursued under classified budgets. It
is also not a comprehensive report of all research that could result
in the development of new surveillance techniques. The goal was to
identify, describe, and quantify in budgetary terms several of the
leading research programs.
DEPARTMENT OF JUSTICE FY 2005
The following information comes from the MATRIX website
"The Multistate Anti-Terrorism Information Exchange (MATRIX) pilot
project leverages proven technology to assist criminal investigations by
implementing factual data analysis from existing data sources and
integrating disparate data from many types of Web-enabled storage
systems. This technology helps to identify, develop, and analyze
terrorist activity and other crimes for investigative leads. Information
accessible includes criminal history records, driver's license data,
vehicle registration records, and incarceration/corrections records,
including digitized photographs, with significant amounts of public data
records. This capability will save countless investigative hours and
drastically improve the opportunity to successfully resolve
investigations. The ultimate goal is to expand this capability to all
MATRIX includes data from the following sources
* FAA pilot licenses and aircraft ownership
* Property ownership
* Coast Guard registered vessels
* State sexual offenders lists
* Federal terrorists watch lists
* Corporation filings
* Uniform Commercial Code filings (i.e., UCCs or business liens)
* Bankruptcy filings
* State-issued professional licenses
In addition, commercial sources are used where they are generally
available to the public or legally permissible under federal law; for
example, telephone directory assistance.
The following files have been provided by some of the participating
* Criminal History information
* Department of Corrections information and photo images
* Sexual Offender information
* Driver's License information and photo images
* Motor Vehicle Registration information
MATRIX has received $4 million in funding from the Bureau of Justice
Assistance, Office of Justice Programs, U.S. Department of Justice, and
$8 million from the Office for Domestic Preparedness, U.S. Department of
Homeland Security. ( ) I have been
unable to find more detailed or prospective funding information.
Department of Homeland Security FY2005
The following budget information comes from the DHS FY05 Budget in Brief
FY04: $328 million
FY05: $340 million
"US-VISIT is responsible for developing and fielding a biometric-enabled
system which collects, maintains, and can exchange information on
foreign nationals that facilitates the following border security and
immigration decisions: (1) who should be allowed to enter, (2) who has
overstayed or otherwise violated the terms of their admission, and (3)
who should be apprehended or detained for law enforcement actions or
interests. (page 20)
Border Patrol Surveillance and Sensor Technology:
FY05: $64.162 million
"Continued expansion of the Remote Video System (RVS) along the southern
and northern borders to increase the effectiveness of Border Patrol
Agents. The expanded system will provide for significantly enhanced
detection and monitoring capability between the ports of entry and
increase officer safety. (page 21)"
Targeting Systems Enhancement
FY05: $20.623 million
"At the core of CBP's ability to achieve its critical border security
objectives and maintain the flow of lawful commerce, is CBP's ability to
identify high-risk travelers and goods for inspection while allowing the
vast majority of law abiding travelers and commerce to continue to their
destination without unnecessary delay. This initiative will fund 34
positions and technology acquisition for the following systems:
Automated Targeting System/Passenger, National Targeting Center,
Automated Targeting System-Inbound, Automated Targeting System-Land
Border, Automated Targeting System-Passenger (RESMON), and the Trend
Analysis and Analytical Selectivity Program. (page 22)"
International Trade Data System
FY05: $5 million
"Funding to ensure integration of ITDS with key federal agencies. The
International Trade Data System is an initiative to implement an
integrated, government-wide system for the electronic collection, use
and dissemination of trade and transportation data essential to the
mission of more than 100 federal agencies. This will supplement
recurring base resources of $11 million.
The following information is taken from DARPA's FY05 budget estimates
Automatic Target Recognition Technology:
"The Automatic Target Recognition Technology program develops new sensor
exploitation aids to detect targets in high volume sensor data with
minimal human support. It supports very large sets of targets (1000's of
target types) with high identification performance and very low false
alarm rates. It develops modeling methods to account for target
variability related to partial damage, design difference, or equipment
carried on the vehicle's exterior. The program supports interaction with
humans, who supply operational context, guide hypothesis development,
and adapt models. The program develops techniques for in-the-field
training of models, signatures, and scoring parameters. This allows it
to identify vehicle -specific signatures, and develop new target
fingerprinting techniques. The program develops new methods to assist
humans in achieving precise identification of ad hoc, poorly defined
targets. It enables a dramatic reduction in sensor-to-shooter timelines,
supporting dynamic target engagement. (page 36)
FY04: $8.0 million
FY05: $8.69 million
Semantic Information Fusion:
"The Semantic Information Fusion program develops tools to correlate
fragments of target location, identity, and behavior information into a
composite description of a situation. The program will focus on
incorporating data on human activities obtained from human sources,
whether openly or surreptitiously. Information represented in linguistic
terms will combine with physics-based models of visibility, mobility,
and access to reconstruct past events, and infer current situations.
This effort enables combining human-derived information with products
prepared for automated systems. It will permit context-sensitive
determination of the sensitivity of inferred information, and invocation
of protection mechanisms, at the time of inference.
- Obtain streams of text information produced by tactical commanders in
- Employ state-of-the-art entity, date, and relationship extractors to
construct symbolic representations of message contents.
- Identify external foundation and contextual knowledge required to
correlate material on one entity appearing in different sources.
- Develop symbolic correlation techniques to automatically suggest
- Employ supervised training approaches to improve scoring functions and
hypothesis management logic.
- Assess the performance of the technologies on sequestered test data
from a variety of sources. (pages 38-39)
FY05: $2 million
"The Knowledge-Based Systems program, formerly budgeted in Project ST-33,
will develop enabling technologies, methodologies, ontologies, and
specific knowledge bases to achieve the next generation of intelligent,
knowledge-intensive systems. This work will focus on developing
technology that spans the spectrum from large, strategic knowledge banks
to small, individual knowledge-based systems. The program will develop
technologies for codifying, linking, integrating, accessing, and using
complex and cross-disciplinary knowledge at widely varying scales. This
capability at a strategic level will provide DoD decision-makers with
rapid as-needed access to decision-relevant background knowledge from a
broad spectrum of distributed sources. The knowledge will be expressed
in formal knowledge representation languages that will allow computers
to reason about the knowledge, consider its implications, imagine
possible future scenarios, and query the warfighter for clarification of
various aspects of the information. The significant challenges are
centered on the fact that critical knowledge involves temporal
information, complex belief structures, and uncertainty, and current
representation technology is not adequate to capture such information.
This program will also develop the technology needed to enable the
creation of individual knowledge-based systems that would incorporate
into the reasoning process (in a computer-understandable form) knowledge
of the warfighter's responsibilities, approach, tasks and activities.
Another goal of this program is to support the warfighter's ability to
understand the "big picture" for mission planning, monitoring and
replanning. By formalizing situation model representations, automated
support will be provided to commanders and analysts for prediction of
unforeseen events and determination of the relevance of isolated or
partial events to the evolving situation. To achieve these objectives,
this program will formalize situation representation and develop
analogic al and case-based reasoning, functional representation
languages and situation markup languages technologies. This effort would
then provide the warfighter with intelligent automated assistance to
help him plan and accomplish his daily activities and, over time, learn
how he accomplishes these tasks and provide increasingly valuable
- Develop knowledge module authoring tools.
- Develop methods, protocols, and tools for using interoperable
knowledge modules resident on distributed knowledge servers.
- Develop integrated knowledge representation and learning technology
that enables effective representation of essential forms of knowledge.
Document a substantial library of formal declarative interoperable
multi-use ontologies initially across single, then multiple domains.
- Demonstrate and evaluate prototypes of strategic and individual
- Develop representations of events and methods for separating and
tracking their association to merge multiple scenarios, assimilate one
event within the context of the other, and identify where events deviate
from the norm. (pages 71-72)
FY04: $12.241 million
FY05: $17.5 million
"The Computing Foundations (formerly Cognitive Systems Foundations)
project will develop novel system-level solutions through the
intelligent integration of cognitive agent capabilities built on robust
software and hardware infrastructure. To handle increasingly complex
jobs, next-generation computer systems will need to integrate the
cognitive capabilities of reasoning, learning, explaining, and
self-awareness, and be able to be advised and cope robustly with
surprise. These aspects of intelligence will be combined in innovative
ways with powerful new conventional computing architectures. Overall
this element seeks to make fundamental scientific and mathematical
improvements in our understanding of and ability to create information
and computing systems. The next generation of systems, with cognitive
capabilities, may also form teams to achieve goals in a coordinated
manner, exceeding the performance of individual systems or humans
working alone. Current fragile commercial systems will require
enhancements or radical changes to support this revolutionary objective.
The new computing foundations will extend beyond today's standard Von
Neumann computing model.
"The military faces new aggressive and agile threats that have sufficient
technical resources to mount sophisticated attacks using easily
accessible commercial information systems. The pervasive nature of both
the threat and their means drives the need for systems that are able to
dynamically adapt, collect and assimilate large quantities of data, and
remain robust under a large set of potential failure conditions and
threats. Computing Foundations will enable next-generation systems to be
more responsible for their own monitoring and protection, as well as for
restoring themselves to full capability after an attack or failure.
"In addition, the plan is to develop, evaluate, prototype and demonstrate
a set of promising concepts in the context of full-scale test-beds in
realistic scenarios and environments. The next transformational
revolution for military force development will be the seamless
integration of autonomous physical devices, computation software agents,
and humans. Transition goals are military next-generation
network-centric systems and platform-specific information collection and
processing systems in space, air, sea and land. (page 85)
FY03: $8.282 million
FY04: $17.583 million
FY05: $27.552 million
FY06: $33.450 million
FY07: $41.283 million
FY08: $48.094 million
FY09: $52.892 million
Architectures for Cognitive Information Processing:
"The Architectures for Cognitive Information Processing (ACIP) program is
developing a new class of processing approaches, algorithms, and
architectures to efficiently enable and implement cognitive information
processing. ACIP will develop the fundamentals, framework and
development environments that will provide the basis for and enable
innovative and truly efficient cognitive processing. Current intelligent
processing implementations depend on the use of existing
numerically-based architectures and/or standard software architectures,
and therefore are implemented via algorithms and processing
architectures that are potentially ill-suited to cognitive tasks. To
realize the impact and promise of cognitive processing, approaches,
algorithms, and architectures that are attuned to cognitive processing
fundamentals and that efficiently implement unique cognitive structures
need to be established. The ACIP program will establish core processing
capabilities that significantly advance the state of the art at all
implementation processing levels Ãâ¹Ãâ modules, systems, and underlying
cognitive processing approaches, algorithms, and architectures to
support efficient implementations. In order to focus and establish
context for the ACIP program, ACIP will pursue focused in-context DoD
mission areas for the development of ACIP concepts. ACIP will develop
implementations that will span the areas of perception, reasoning and
representation, learning, and communication and interaction to enable
new classes of cognitive information processing applications that move
us dramatically forward toward the overall goal of creating computer
systems that truly know what they are doing.
- Establish a Cognitive Information Framework that will provide common
cognitive development environments, tools and evaluation methods for
cognitive algorithm and architecture developments, providing an enduring
cognitive basis for a broad set of domains and applications.
- Establish proof-of-concept and evaluate in-context cognitive
application baselines based on current approaches and "best-possible"
implementations using existing processor architectures.
- Characterize the role of reflective reasoning in a cognitive system
that reacts effectively to stimuli and also uses deliberation to plan
and solve problems.
- Establish and demonstrate a first-generation framework supporting
cognitive approach implementation, algorithm development and
FY03: $3.336 million
FY04: $6.280 million
FY05: $10.552 million
Rapid Checkpoint Screening:
"The Rapid Checkpoint Screening program (formerly Deception Detection
funded from PE 0602301E, ST-28) will develop and demonstrate techniques
and sensors to detect life-threatening deceptions in military controlled
portals such as military checkpoints that are compatible with existing
portal screen approaches."
- Identify physiological signals that correlate with deception including
laser vibrometry, lidars, multi-spectral eye tracking, and short range
- Validate the measurement process.
- Establish new concepts for understanding deception processes on a
Threat Characterization of Buildings:
"This program will develop technologies and systems for new surveillance
capabilities of buildings. It will develop and demonstrate
wallpenetrating multi-static Doppler radar for stand-off mapping of
building layout (via long-term integration of human motion) and for
localization of enemy forces immediately upon entering buildings (via
portable radar "flashlights"). It will also demonstrate technologies to
monitor the integrity of building envelopes, to identify a breach of
previously sealed/secured buildings and to identify previously hidden
connections between buildings; approaches include pressure and
power-line monitoring as well as the use of tracer gases.
- Evaluate candidate designs for wall-penetrating Doppler radar.
- Evaluate candidate technical approaches for monitoring building
- Prove feasibility in lab on sub-scale models.
- Design, build, and test prototypes for use in full-scale
demonstration. (page 346)
FY05: $5 million
Surveillance and Threat Neutralization in Urban Environments:
"This program will develop systems to demonstrate the detection and
defeat of threats specific to conflict and stabilization operations in
the urban environment. These threats include roadside bombs, car bombs,
suicide bombers, snipers, Rocket Propelled Grenades and mortars launched
from inside urban boundaries. Detection technologies under development
include intercept and localization of unintentional radiated emissions
of remote-control circuits; multi-static radars for standoff
identifications of shrapnel-packed bombs; detection of anomalies in
gait, heartbeat, and breathing; standoff identification and localization
of explosive vapors/effluents; and multi-mode integrated acoustic - and
radar-based systems to backtrack to the source of fire. Neutralization
technologies include targeted RF jamming of triggers; techniques to
cause incomplete detonation of explosives; portable fast-erecting blast
shields; and technologies to non-destructively and reversibly control
urban access routes.
- Evaluate candidate technologies for wide-area/standoff and
- Prove feasibility in lab on sub-scale tests.
- Design, build, and test prototype for choke-point applications.
- Design, build, and test prototype for wide-area applications. (pages
FY05: $4 million
Last modified December 09, 2004 01:11 PM