Transparency and explainability for algorithmic decisions at work

From years spent investigating, documenting and exposing the impact of algorithmic management and workplace surveillance on workers, from drivers to content creators to data labellers, it is clear that lack of access to information and understandability of the systems than manage them negatively impact workers affected.

In light of the poor track records of the major companies on the topic, we decided to produce detailed recommendations for actors implementing algorithmic management to greatly improve transparency and explainability. Transparency and explainability are not a silver bullet to exploitative and harmful practices, they are but a necessary step to a future where workers’ rights, autonomy and dignity are respected, and where developers and deployers of systems are held accountable.

The demands published on this site are the result of this work. While framed for gig-economy platforms, we believe that at their core they can be transposed to any system deployed to manage workers through algorithmic means. We note that collective bargaining is an important vehicle through which the information asymmetries caused by the use of opaque automated decision making tools can be addressed. In particular, where collective agreements are in place they should afford workers with access to meaningful information concerning the development and deployment of algorithms, including if necessary on a confidential basis.

Algorithmic transparency, if effected uniformly to platform workers, would facilitate a shared understanding of how workers are being managed and allow for the building of real collective bargaining power; thereby addressing the wider power imbalances that characterise the use of automated decision making to make significant decisions concerning workers. However, even where collective agreements are not in place employers should take steps to improve transparency and accountability in line with these recommendations.

Maintain a public register of the algorithms used to manage workers

The register must include all algorithms that affect how workers are managed. This includes any algorithm that produces outputs which influence decisions made about how workers access and are assigned work, how their performance is assessed, how much they are paid, when they must be available for work, what employment status they have, whether they continue to be offered work, and any other matters relating to their terms of work.

The public register is key to addressing the information imbalance of algorithmic management by allowing workers (and candidates) and their representatives to understand what algorithms are being used and how they work. In order to do this, the register must be in accessible non-technical language and kept up to date. It must include a list of all algorithms that affect worker's treatment while at work. 

For each listed algorithm, the following information must be included:

The purpose and design of the algorithm

A short (two or three sentence) description to explain what purpose(s) the company uses the algorithm for and why it has been preferred to other options.

An overview of the algorithm’s design should also be given including what sort of management decisions are made by the algorithm (and whether they are advisory or decisive); whether the algorithm relies on neural networks, machine learning, probabilistic functions or other type of logic; what training data was used; and under what circumstances the algorithm is not deployed or has a failsafe.

The relative importance of the algorithm’s inputs and parameters

The register must explain, in an accessible and non-technical way, what data and ratings it uses to reach decision. This means providing an easy way to understand how important different inputs and parameters are to different decisions. This could be done in various ways: from a simple rating of ‘high/medium/low importance’, to giving more specific and granular detail of the weighting each input or parameter has. 

It should also explain the source of inputs (are they from the app, from customers, from the web, inferred, how long ago, while at work, from data brokers, etc) and how the accuracy and reliability of them is ensured. Where parameters (defined as complex input inferred or calculated by another algorithm, such as a risk score) are used, the company should provide a detailed description of the parameter and how it is calculated, including the data used to determine its value. Fundamentally, complex inferred parameters should not be used to obscure the source and weight of data used in algorithms.

Where decisions are based on the personal data or behaviour of workers and/or consumers this must be clear, and the source of any personal data used should also be set out. The register should also confirm that the algorithm uses only data that is strictly necessary for the purposes of the algorithm, and does not use any sensitive personal data, emotion recognition, data collected while not at work etc.

It is possible that AI algorithms will use parameters that are hard to give real-world human descriptions for. In such cases, the company must state this and instead thoroughly explain how the tool has been built, and how they monitor and audit its outputs to ensure that they do not result in bias or discrimination. Examples (or statistics) comparing different, but similar, inputs with differing outputs may also be needed to explain which sorts of inputs tend to lead to which sorts of outputs.

Human intervention

Where algorithms are used to make decisions in the workplace, there should always be a human either checking, and/or able to review, any decisions. 

Companies must specify what level of decision-making authority oversight teams have, as well as the training that decision-makers have received particularly in respect of the design and potential impacts of the algorithm. The register should break down the level of decision-making authority across multiple stages of review and oversight, specifying at what points in the decision-flow a human makes a decision and how long they have to make that decision.

The register should also provide some operational information about how much staff capacity (in FTE) is dedicated to human review and how long a review is expected to take.

Development history, updates, and impact assessments

The company should also state where responsibilities for the development and updating of the algorithm lie, especially where an external supplier has been involved. This does not require identifying individuals, but rather relevant teams/departments/organisations and the nature of their different responsibilities.

With algorithmic changes distributed to all parties signed up for system updates, it is vital that workers and their representatives are automatically provided with updates when changes occur. The register should therefore provide a log of updates, which should be updated in real time as soon as the update is made. They should be in an unambiguous form for clarity and understanding.

The register should also specify what, if any, consultation has taken place between the company and workers and their representatives with respect to the design or revision of the algorithm. Active consultation with workers helps prevent harmful practice and collective agreements ought to include provisions on transparency, explainability and consultation for algorithmic management.

Any impact assessments of algorithmic systems (including Data Protection Impact Assessments and/or Algorithmic Impact Assessments) should be published either in redacted form or in full. And where there are redactions, these must be justified. When made public impact assessments should incorporate process transparency so that changes over time are documented and explained.

With regards to how this information is shared, we encourage the use of a variety of means such as flow charts, FAQs, or short form videos to accompany textual explanations. The selected mediums should make understanding how the algorithm operates as accessible and simple as possible.