Monitoring behaviour, generating profiles, creating dominance
Industry is gaining insights into and intelligence on our lives that were previously possessed by powerful Intelligence Agencies, and tomorrow their potential may exceed them. In the future, industry giants will have more insight into the world than the most powerful intelligence agencies. What they know and represent about us will have significant effects on individuals, groups, and whole societies.
As a result of design choices in modern technologies, individual and collective behaviour is increasingly traceable. Metadata and logs, and other forms of observed data are generated of every interaction. The growing stores of data that companies and governments hold about individuals and groups is now automatically generated from human behaviour. This is at odds with how most users understand privacy as being about what they knowingly and overtly disclose to companies.
Powerful institutions with access to data now have unprecedented population-level knowledge about individuals, groups, communities, and whole nations and markets. With this knowledge they will have insight and intelligence on patterns of behaviour and other trends. They may identify customary behaviours and activities, as well as deviations. Even as these categories become divorced from the individual pieces of personal data, they provide powerful insights into how groups, societies and markets function. And they will likely be kept secret or understandable to the few. While monopolies are traditionally measured in terms of market power, this raises the question of how the data economy needs new ways to measure what qualifies as dominance in the marketplace.
Principle 7. Control over identities
What we need to see: user control over intelligence
Principle 6. Individual control over intelligence
Individuals should have control over the data generated about their activities, conduct, devices, and interactions, and be able to determine who is gaining this intelligence and how it is to be used.
Principle 5. Limit data analysis by design
As nearly every human interaction now generates some form of data, systems should be designed to limit the invasiveness of data analysis by all parties in the transaction and networking.
In the future, industry giants will have more insight into the world than the most powerful intelligence agencies. What they know and represent about us will have significant effects on individuals, groups, and whole societies.