Data and Surveillance

About: This course covers the fundamentals of communication networks and describes the various forms of communications surveillance that exists.

Ideal for: Anyone with an interest in technology and human rights. We recommend first taking our introductory course.

Impact: You will gain insight into how communication and surveillance technologies have been deployed across the world, as well as the different protections that metadata receives in various countries.

Learn more: As digital communications grow, governments continue to seek new ways of getting access to content and metadata. Read more of our work on this.



s. 16 of Regulation of Investigatory Powers Act, 2012

Extra safeguards

intercepted material falls within this subsection so far only as it is selected to be read, looked at or listened to otherwise than according to a factor which—

has as its purpose, or one of its purposes, the identification of material contained in communications sent by him, or intended for him


Smith v. Maryland 1979

"there is no constitutionally protected reasonable expectation of privacy in the numbers dialed into a telephone system, and hence no search within the fourth amendment is implicated by the use of a pen register installed at the central offices of the telephone company." (NO NEED FOR A WARRANT)

USA Freedom Act 2016 that amended the Foreign Intelligence Surveillance Act 1978:

Each application under this section—

(1)shall be made to—

  1. a judge of the court...

(2)shall include—

(A) a specific selection term to be used as the basis for the production of the tangible things sought;

(B)in the case of an application other...for the production of call detail records... a statement of facts showing that there are reasonable grounds to believe that the tangible things sought are relevant to an authorized investigation


Source: , Interception of Communciations and Surveillance Ordinance.

Question to Secretary for Security April 29 2015:

LEAs are required to obtain authorisation from a panel judge or a designated authorising officer prior to any interception of communications or covert surveillance.

When investigating crime cases, LEAs may, having regard to the nature of the cases and for the purpose of crime prevention and detection, request necessary information related to crime detection from persons or organisations concerned, including subscribers' information (such as account name and Internet Protocol address (IP address)) and log records from local or overseas Internet service providers (ISPs), for locating witnesses, evidence or suspects. Such enquiries do not involve requests for records of the content of any non-open communications.


Source: Vodafone Law Enforcement Disclosure Report. Legal Annex February 2015 report:

Require operators of telecommunication systems used to provide telecommunication services to the public to intercept communications in real-time.

In cases involving national security and general emergency cases, the Qatari ministries and law enforcement agencies can directly approach communication service providers and require them to assist law enforcement agencies in achieving their objectives which could involve implementing a technical capability that enables direct access to their network (without the communication service providers operational control or oversight).

Source: Decree Law No. (34) of 2006 on the promulgation of the Telecommunications Law, Article 63:

Article (63) Power to Search, Investigate and Seize

The employees of the Supreme Council who are vested with powers of judicial seizure by a decision from the Attorney General pursuant to agreement with the Chairman of the Board shall seize and prove offences committed in violation of the rules of this Law.

In this respect, they may enter related premises, have access to records and documents and inspect equipment and telecommunications systems or any other related things and request data or clarifications as they deem necessary.


Communications surveillance: Communications surveillance is the monitoring, interception, collection, preservation and retention of information that has been communicated, relayed or generated over communications networks to a group of recipients by a third party. This includes phone calls, emails, text messages, pictures, and messaging apps.

Data protection: Data protection is the law designed to protect your personal information, which is collected, processed and stored by “automated” means or intended to be part of a filing system. In modern societies, to empower us to control our information and to protect us from abuses, it is essential that data protection laws restrain and shape the activities of companies and governments

Data retention: Data retention is the storage of data for a period because of a policy or law. It is relevant to communications surveillance when there is a requirement for a telecommunications company to store communications or metadata for a period of time, meaning that it is possible for security agencies to listen and view communications in the past.

Network surveillance: In communications surveillance, network surveillance is technologies that require physical installation onto a network to perform communications surveillance.

Tactical surveillance: In communications surveillance, tactical surveillance is technologies that do not require physical installation onto a network. These are technologies can thus often be easily transported to different locations for deployment. An example of this is an IMSI catcher.


Device: In this sector, a "device" is something that we can use to communicate: a landline telephone, a mobile phone, a tablet, a computer etc.

Biometrics: Biometrics is the capturing and storage of the physiological and behavioural characteristics of individuals. These include fingerprints, facial recognition, and iris scans; behavioural biometrics includes walking styles and typing rates. Biometrics are usually immutable, and last for an individual's entire life. Biometrics is used in some countries for passports, ID cards, and other forms of identification.

Encryption: Encryption is encoding a message so it can only be read by the intended recipient. A message would appear nonsensical to anyone without the proper "key", through which they can read the content of the message. So, if a message is intercepted by The Adversary, they would not be able to read the content of the message without a key. A common form of encryption is PGP.

Malware: Malware is software that is installed on a device to monitor communications, record information, or disrupt the device. It is installed without the user’s knowledge, which may or may not need physical access to a device. Examples of malware are keyloggers on computers (that record every key press made, gathering the contents of communications as well as passwords), and software maliciously installed on phones to record conversations.

Metadata: Metadata is all information about a communication, apart from the content of the communication itself. For example, for a mobile phone call, this includes information on what number you called, where you were when you called them, what time you called, and how long that call lasted. Other examples of metadata are the websites visited and the time and location of a Tweet. Many states have looser regulation over metadata than the content of communications, despite the fact that the metadata contains a lot of information – often, it’s more intrusive than the content of a communication.

IMEI: International Mobile Equipment Identity: this is a unique identifier that, along with the IMSI, identifies your physical handset to the network. It can be found by dialing *#06# on your handset. Even if you change the SIM card in your phone, the IMEI remains the same.

IMSI: International Mobile Subscriber Identity: this is a unique identifier stored on a SIM card, to identify your SIM card when you connect to the network.

IMSI Catcher: An IMSI Catcher is a form a tactical surveillance, and has other names like a Stingray. There are various types of IMSI catcher, with different capabilities, but they all work through being a fake cell tower - completely invisible to the user. They are most commonly used to gather the IMSI numbers of everyone in a particular vicinity - for example, at a protest. As the IMSI number is unique to a SIM card, this can be used to tell the identity of the SIM cards in the area. Thus it is a tool that can be used to tell who is in a particular area at a particular time. As a piece of tactical surveillance, this means that an IMSI catcher can be used without connecting it to the phone network, so does not require the cooperation of the telecommunicating companies; this means that it is possible to be used with less oversight.

Internet of Things: The Internet of Things is a term for the connection of objects, vehicles, and sensors to the Internet. This includes devices to track the location of your car over the Internet, control the heating in your home, and a remote sensor to see how a solar panel is being used. There can be privacy concerns, as the Internet of Things means that more and more data is generated about our everyday life, not just when we are using a communications device.


SIM card registration: SIM card registration is when the details of the person owning a SIM card is taken, particularly when the details are held by the state even if a telecommunications company collects the information. This particularly applies to pre-paid, Pay As You Go (PAYG) sim cards, as the telecommunications company already has information for post-paid SIM cards on a contract. SIM card registration might be using an official ID, passport, or proof of address; in some countries, biometrics are also collected. SIM card registration allows the state to know the identity of the owner of a SIM card, and thus who is most likely making a call or sending a message. It can also be used in conjunction with an IMSI catcher to know the possible identities of everyone in a particular area.

Smart cities: 'Smart cities" is a term used to described the growing role of data in the running of a city. A smart city collects data on its inhabitants to varying degrees, including traffic data, people's movements, and even their power usage and how much rubbish they produce.

Communications surveillance is where a third party intercepts a communication in the course of its transmission between intended recipients. Interception includes all acts of monitoring, copying, diverting, duplicating and storing communications in the course of their transmission by or for law enforcement or intelligence agencies.[1]

When discussing communications surveillance, there are many debates, distinctions, and terms used. Because of this it is important to know what a term represents or what such a distinction means in practice to effectively engage in debates on the topics of privacy and surveillance.

Firstly, the definition for what constitutes an interference with the right to privacy is hugely important. A clear definition of interference presents a line which, once crossed by the State, must be justified and accord to principles of necessity and proportionality, among others.

One of the more prominent debates is the distinction between the protection in law afforded to metadata[2] and content. Debates have focused on whether one deserves more protection than the other[3]. Traditionally, metadata has received less protection in law because in policy-makers mind's, content retains insightfulness and invasiveness that metadata does not. Recently, courts around the world have questioned their State or region's metadata and content distinction. Some courts have begun to understand the insight that metadata can provide which has lead to striking down bad laws that do not provide sufficient protection for metadata, violating the right to privacy.[4]

A meaningful distinction exists between communications surveillance technologies that operate through the network and technologies that are tactical in their deployment. Network surveillance technologies refer to tools that require physical installation onto a network to perform communications surveillance. These are the crocodile clips of the years gone by, although much more sophisticated and powerful now. Tactical technologies refers to a growing body of surveillance tools that do not require physical installation onto a network and are mobile in that they can be easily transported to different locations for deployment.

Network surveillance technologies include the modification of network equipment (required to deliver your communication to your intended recipient) to pass requested information from the network operator to law enforcement or an intelligence agency. This is sometimes referred to as the Internal Interception framework.

Network technologies also include surveillance tools that place probes on an operator's network to deliver information directly to law enforcement or intelligence agencies. This set of technologies serve no other purpose than to intercept and deliver information. This is referred to as the External Interception Framework.

The other set of technologies, distinct from network surveillance technologies, are tactical technologies. These are mobile in their deployment meaning they do not require themselves to be installed permanently on a network. Some tools do not require a physical presence on the communications network at all. This category is represented by technologies such as IMSI Catchers[5] and also includes Intrusion[6] technologies like trojans.

Connected to distinctions between types of surveillance technologies is the use of the term Lawful Interception which takes in aspects of the distinction between network and tactical surveillance technologies. This requires an additional definition on top of the technological distinction that brings in principles of law.

Interference with the right to privacy

The point at which privacy is interfered with is the line at which the actor has to justify their actions. To cross this line requires an order from an authorising body where the justification for an interference with the right to privacy is considered. The consideration should take into account the principles discussed in other briefing papers (Communications Surveillance: The Principles and the Law)

The right to privacy is interfered with at the moment the communication and its data is intercepted regardless of whether or not the information is subsequently consulted or used[7]. Further to that, even the mere possibility of communications data being intercepted – such as a provision in a law- creates an interference with privacy[8]. This means that, in theory, any legislation or activity that involves communications surveillance represents an interference with the right to privacy and the onus is on the State to demonstrate that such interference is neither arbitrary nor unlawful.

If the interference with the right to privacy were to be defined as taking place once the information is subsequently consulted or used, this would have grave implications for the right to privacy. It would allow, in theory, that interception of communications could take place without any need for authorisation. This would allow for warrantless mass surveillance. Arguing for the interference at the point of interception presents a much stronger position from which to protect privacy.

Content and Metadata

Use of the internet via mobile and digital devices requires the creation of additional data about communications, known as communications data or metadata. This data is created to, among other things, make sure the communication reaches the intended recipient or confirm that the user gets the best possible network coverage through network maintenance or performance monitoring. This type of data can provide a great deal of insight about individuals, their locations, travels and online activities, through logs and related information about the e-mails and messages they send or receive. This category of data is treated separately in law from the content of those same messages.

Intelligence agencies' response to questions about the invasiveness of their operations regularly make mention of the fact they are not reading the content of messages, just analysing the metadata attached to it. President Barack Obama said such a thing in his speech on the review of US Signals Intelligence on January 17 2014.

“This program does not involve the content of phone calls, or the names of people making calls. Instead, it provides a record of phone numbers and the times and lengths of calls -- metadata that can be queried if and when we have a reasonable suspicion that a particular number is linked to a terrorist organization.”[9]

As it becomes clearer how much metadata can reveal about individuals and groups it is no longer appropriate to subject metadata to lower thresholds or consider its interception and processing a less invasive practice than interception of content. Metadata analysis is in a position to provide as great a level of insight into individuals as communications content[10].

To illustrate the sheer volume of metadata created consult the Surveillance Industry Index[11]. A particular piece of analysis technology found in the Index sold by Danish company ETI-AS[12] provides over 1000 selection criteria. Those criteria only relate to metadata such as username, IP address, MAC address[13], and login information[14]. With that many points of data available to be selected, analysed, and aggregated with other pieces of data it would be misleading technologically to consider metadata a less invasive category of data than the content of the communication.

The aggregation of that type of data may give an insight into an individual’s behavior, social relationships, private preferences and identity that go beyond even that conveyed by accessing the content of a private communication. User's are capable of lying or using codewords in the content of a communication. Metadata never lies and is presented in a very structured format.

Intelligence Agencies’ belief in the strength of metadata was illustrated when Michael Hayden, the former head of the CIA, said: “We kill people based on metadata”.[15] Metadata clearly carries enough insight for agents to be comfortable in making life and death decisions so to treat it as a less protected category than content in law fails to recognise the power that States place in the hands of metadata.


“All the forces of a technological age…operate to narrow the area of privacy and facilitate intrusions into it. In modern terms, the capacity to maintain and support this enclave of private life makes the difference between a democratic and a totalitarian society.”[16] - Thomas J Emerson, 1965.

The distinction in communications surveillance technology that best reflects the capabilities, legal standards, and practical application is between network surveillance technologies and tactical surveillance technologies.

Network Surveillance frameworks

Network surveillance covers any technology that functions as permanent, physical installation of equipment onto a communications network. These are normally made up of probes[17] for interception, or modified network tools to allow for delivery of communications data to retention suites and monitoring centres with work stations in a monitoring facility. A further distinction occurs between network surveillance technologies that operate within an operator's network premises, such as mediation platforms operating via modified network equipment, and those technologies that use probes or taps outside of a network operator's premises to intercept communications.

The common point here is that they are physical structures installed on networks and potentially have an access to a huge amount of information that transports through phone and data networks.

Network surveillance within an operator's network should, providing that rule of law is respected in the country, require the cooperation between law enforcement or intelligence agencies and the service provider[18]. Typically, a national law enforcement agency issues an order for information held by a service provider, which is obliged to deliver the requested information to a “Law Enforcement Monitoring Facility”, a premises belonging to the law enforcement agency, where the intercepted information is stored and can be accessed by the agency. This is sometimes referred to as an internal interception framework.

The processes relevant to the interception of communication differs across communication networks, security protocols and administrative functions (such as the system for receiving and recording the order) but a core approach remains: equipment required to deliver communications should be modified to a standard that allows for law enforcement agencies to intercept communications. Technologies in the Surveillance Industry Index that advertise themselves as being compliant with the European Technical Standards Institute[19] (ETSI) and the Communications Assistance for Law Enforcement Act[20] (CALEA) in brochures are responding to the need for an internal network surveillance framework.

The point where the law enforcement agency and the service provider interact is known as the mediation platform[21]. This is where the information is transferred from the service provider’s site, to the “Law Enforcement Monitoring Facility”.

Basic Architecture of a mediation platform. Courtesy of Utimaco, Lawful Interception Management System, Product Description.

Other companies to look at:

  • Alcatel-Lucent[22] – ULIS (Unified Lawful Interception System)[23].
  • Aqsacom[24] – ALIS[25].
  • Nokia Siemens Networks[26] - Monitoring Center[27]

External interception refers to the second broad category of network surveillance technologies. These are probes, devices, and software that are placed onto or operate through a communications network to perform surveillance. These technologies are different from information delivered in an internal interception framework, through the network via a mediation platform. Because of their position in the network – external to a network operator's premises normally-this leads to a different relationship between law enforcement or intelligence agencies and the operator. This external interception framework does not require the operator to deliver the information through their network (like the internal interception framework requires) for interception to take place.

E.g. Description of External Interception Framework, courtesy of Telesoft Technologies, HINTON 5000 Interceptor.

Technologies like VASTech's Zebra[28] or Advanced System's Cerebro[29] represent technologies that operate as external interception frameworks.

The use of such technologies is highly questionable as to how practical safeguards can be put in place. Interception via an operator's premises go a little way to safeguarding rights by demanding cooperation from other actors and requiring orders to be signed and delivered to the operator. Technologies that perform network surveillance externally are capable of avoiding cooperation with network operators. This can lead to secrecy in their deployment, even secrecy in their use. Deployment of this technology makes practical safeguards- like the technology is only being used when authorised- incredibly difficult to guarantee.

External interception technologies, just like technologies designed for interception within the operator, mark themselves out by using particular phrases: “No cooperation with the providers is required” and “Transparent solution” are regularly used terms for such technologies [30].

Tactical Surveillance technologies

Tactical Surveillance technologies refers to the second broad category of communications surveillance technologies. These are mobile technologies that do not require physical installation on a network to carry out surveillance.

External interception will normally require a probe to be placed on the network to intercept the data traveling along the network and transfer it to the relevant agency. Advances in technology have now allowed for the development of surveillance technologies to intercept without any physical presence on a network at all.

When discussing phone monitoring technologies, this capability is sometimes referred to as Off-The-Air interception[31], because they don’t rely on any form of probes or taps. Technologies such as IMSI Catchers fall into this category of tactical surveillance technologies.

E.g. Diagram of Off-The-Air interception. Neosoft, Catalogue.

The targeting of devices with malicious software is another tactical communications surveillance tool. The software can be delivered onto a user's computer through a compromised document or software vulnerability. Tools such as FinFisher[32], Hacking Team's RCS[33] and GR Sistemi's[34] Dark Eagle are leaders in this space.

Other companies to look at:

Lawful Interception

When talking about network surveillance technologies some products will mention that they can be used for “Lawful Interception”. This statement refers to systems that operate as mediation platforms between the operator and the law enforcement or intelligence agency over a set of defined protocols. However, for interception to be lawful requires more than just protocols, it needs to contain principles of legality, necessity, and proportionality among other principles.

Companies like Utimaco and their product LIMS (Lawful Interception Mediation System) sell the systems that[45] meet the technological standards. The use of these systems meet the obligation that bodies such as ETSI and legislation like the United States of America has with the Communications Assitance for Law Enforcement Act impose on service providers that they shall ensure that:

“1) the entire content of communication associated with a target identity being intercepted can be intercepted during the entire period of the lawful authorization;

2) any content of communication associated with a target identity being intercepted which is routed to technical storage facilities or is retrieved from such storage facilities can be intercepted during the entire period of the lawful authorization;”[46]

Although given the title “Lawful Interception” as a description of the technology, there is a need for more than just a particular technology to be considered lawful. It needs to operate within a lawful framework.

The interception of communications via a mediation platform must be conducted in accordance with national law, following due process, and after receiving proper authorization from a competent judicial authority. Only then can interception be considered lawful. For more information on those principles see the separate briefing paper Communications Surveillance: The Principles and The Law.

What’s your opinion?

At what point should a warrant be applied for from an authorizing authority? When the information is intercepted or when it is accessed? Why?

Do you think that there is a legitimate reason where cooperation between the network operator and law enforcement agency won’t work and instead an external interception technology should be used instead?

Who should have the power to authorise the interception of communications? What should the process for authorisation look like?

An authority when deciding whether to allow a warrant to be issued to retrieve information on a target should consider what questions?



[1]Frank La Rue report, para. 6,

[2] Metadata is the information about the communication (time sent, to whom, from who) and not the content of the communication.

[3] for more information about metadata and what it can reveal see

[4] also see coverage of the CJEU Data Retention Directive judgement

[5] Surveillance Industry Index, Phone Monitoring,

[6] Surveillance Industry Index, Intrusion,

[7] Right to Privacy in the Digital Age, Para. 20.,

[8] Malone v UK, para. 64, See also Weber and Saravia, para. 78


[10] See footnote 2.

[11] Surveillance Industry Index SII is a resource created by Privacy International that documents the technologies sold by private surveillance industry.


[13] A media access control (MAC) address is a unique identifier assigned to hardware that uniquely identifies each device on a network.



[16] Emerson, Nine Justices in Search of a Doctrine, at 437,

[17] Probe = A physical device inserted at a key juncture in a network for the purposes of monitoring or collecting data traveling through the network.

[18] Entities that provide electronic communications services. They make sure that your information gets to you but do not necessarily provide the service you are seeing the communication on. (O2, T-Mobile, Google, Facebook).

[19] The European Telecommunication Standards Institute (ETSI) is a standards setting body that sets the rules of how “lawful interception” systems should operate. This could go from what information should be transmitted and what technical standards service providers should implement to best serve that purpose, to the energy consumption of the systems. For more on lawful interception and the role of ETSI see,

[20] The Communications Assistance for Law Enforcement Act (CALEA) is an act in the United States of America that sets out the obligations of service providers to assist law enforcement in accessing communications on their network. This can involve mandating modifications to a network to allow for law enforcement to better access these systems. For more on CALEA see

















[37] Link included above


[39] Link included above


[41] Link included above.


[43] Link included above



[46] ETSI Standards for Law Enforcement Agencies,