Webinars      Login/Register      Cart

InfoGovANZ

Information Governance Think Tank

AI & Ethics

Using artificial intelligence to make decisions: Addressing the problem of algorithmic bias

by

This technical paper is a collaborative partnership between the Australian Human Rights Commission, Gradient Institute, Consumer Policy Research Centre, CHOICE and CSIRO’s Data61. We explore how the problem of algorithmic bias can arise in decision making that uses artificial intelligence (AI).

This problem can produce unfair, and potentially unlawful, decisions. We demonstrate how the risk of algorithmic bias can be identified, and steps that can be taken to address or mitigate this problem.

AI is increasingly used by government and businesses to make decisions that affect people’s rights, including in the provision of goods and services, as well as other important decision making such as recruitment, social security and policing. Where algorithmic bias arises in these decision-making processes, it can lead to error. Especially in high-stakes decision making, errors can cause real harm.

The harm can be particularly serious if a person is unfairly disadvantaged on the basis of their race, age, sex or other characteristics. In some circumstances, this can amount to unlawful discrimination and other forms of human rights violation.

This paper describes the outcomes of a simulation. We have simulated a typical decision-making process and identified five scenarios in which algorithmic bias may arise due to problems that may be attributed to the data set, the use of AI itself, societal inequality, or a combination of these sources.

We investigate if algorithmic bias would be likely to arise in each scenario, the nature of any bias, and consider how it might be addressed. The scenarios are framed around a consumer’s interactions with an essential service provider that most people will deal with at some point—an energy company.

Read the paper here: Using artificial intelligence to make decisions

AI Transparency in Digital Government Highlights

by

To celebrate Right to Know 2020, Information Governance ANZ were delighted to host a timely discussion on the right to access information and the use of algorithms in government decision-making.

This interactive forum was facilitated by Susan Bennett, Founder of InfoGovANZ and our special guests included:

  • NSW Information Commissioner – Elizabeth Tydd
  • Victorian Information Commissioner - Sven Bluemmel
  • Senior Research Fellow, University of Cambridge – Dr Jat Singh

The increasing adoption of technology across society, including in government, requires the preservation, assurance and assertion of information access rights. The right of access to government information also extends to information held by contractors that provide services to the public on behalf of government.

Applying and challenging these rights becomes more complex in the new world of automated decision-making. Legal frameworks, licensing and contracts must evolve to ensure information governance is applied to the design and use of algorithms, so that rights and interests are protected.

Commissioner Sven Bluemmel on building trust through transparency

Commissioner Bluemmel explained that Right To Know Day is actually an global event, recognised as International Access to Information Day (IAID). It aims to raise awareness of the importance of open government, and the public’s right to access government-held information.

We consider this right as the foundation for transparent and accountable government in a democratic society, supporting public participation and scrutiny of government decisions, and by extension supporting better decision-making.

As the public sector is increasingly making use of technology such as artificial intelligence (AI) to provide services and make decisions, there are questions about how the principles of good governance can best be applied. OVIC published an e-book called “Closer to the Machine”, exploring the technical, social and legal aspects of AI, including algorithmic transparency.

The Commissioner noted the importance of trust to the relationship between government and citizens. The relationship people have with government is very different from their relationship with a business or other kind of service provider where an algorithm could be considered the “secret sauce”.

Government decisions impact people’s livelihoods, their freedoms and human rights. It’s one thing if an algorithm incorrectly predicts my favourite songs or shopping preferences. It’s quite another if I’m denied a pension or parole based on an automated decision.

So government has to take a very different approach to the application of AI and similar technologies. It has to ensure transparency and accountability are fully supported so that decisions can be explained, challenged and, most importantly, trusted.

This aspect of trust and trusted information is especially important during times of crisis. “Forcing” people to do the right thing is far less effective than having a trusted relationship with constituents. There has to be trust in the government’s motives as well as its mechanisms. In this situation, reliance on defensible data and explainable modelling comes more into play.

Commissioner Elizabeth Tydd on the shift to digital government

Commissioner Tydd mentioned some findings from the IPC Community Attitudes Survey. There is growing support (72%) for government use of de-identified data to plan and deliver public services. There is also strong demand for accountability and transparency in the collection and use of data with 78-81% support for more public reporting. Overall, 88% of respondents said their right to access government information was important.

People are becoming interested in a wider range of information, not only access to their personal information or information about government services. They also seek access to information about how policy is developed and how agencies operate. This has implications as government makes use of data and technology in new and different ways. Administrative practices must evolve to safeguard the right of access to information.

The Commissioner highlighted several case examples challenging algorithmic transparency in government decision-making and asked: Is the concept of access to information sufficiently “malleable” and “future proof”?

In one case, a social housing tenant challenged the calculation of their rental subsidy, which had been determined using an algorithm. It was difficult for the agency to respond to the citizen’s request for information about the decision-making process. Procurement of the system was below the threshold for public reporting on government contracts, and the developer claimed that details of the algorithm were commercial-in-confidence.

In NSW, the GIPA Act applies beyond government agencies to include certain information held by other organisations contracted to provide a public service. But the way this applies to automated decision making is still being tested.

The Commissioner noted that access to information is an enabling right – it enables a citizen to assert other rights. If access to information is restricted it can have much wider implications. She stressed that “accessibility” means information must be understandable on every level. It’s not sufficient to simply release the source code or require the expertise of a data scientist. Access must also come at minimal cost to the applicant.

In another case study, the Gradient Institute considered an AI-driven system for financial approvals, based on credit history and past transactions, which unfairly discriminated against female applicants. Historical data can introduce bias and AI predictions may become less accurate as society changes over time – leading to more perverse outcomes.

An applicant has the right to understand how a decision was made, but there are also sound business reasons for explainability and algorithmic transparency. The design phase is key. Organisations must ensure their processes are auditable, to understand how learning and recalibration happen.

POLL: Who is using machine enhanced or automated decisions – a high proportion “don’t know”.  Does this suggest that information governance practitioners need to get more involved with AI and machine learning projects?

Dr Jat Singh on how automated decision-making works

Dr Singh noted that automated decision-making sits at the intersection of law, policy, community and technology. He mentioned the difference between automated decision-making and enhanced decision-making, where machine learning uses data to determine probability or make predictions that support human decisions. It is a policy decision, not a technical decision, to determine what kinds of data use are appropriate.

A recent example of enhanced decision-making involved COMPAS software which uses algorithms to assess potential risks of recidivism. Judges may use these predictions to inform their decisions when detaining or sentencing a defendant. The algorithms remain a trade secret and this lack of transparency could cause a breach of due process. The case study also raised concerns about machine bias resulting from training data or input data, rather than the algorithm itself.

Dr Singh emphasised that accountability relies on explainability, but it can be challenging to explain how an algorithm works and make it transparent. Each stage of system design involves embedding norms, potential bias and assumptions – not just the training data. There is also a risk of “model skew” over time. Developers must consider whether a model is still fit for purpose in relation to the data and as society changes. Whether the model needs retraining for a business or other purpose, auditability is essential.

The key is to look more broadly at the system as a whole and identify information that can be made available. For example: information about the underlying data used by the algorithm; its source code and performance or test criteria; the software on which it is built; the process steps or workflow for a decision; and how it is deployed – whether a decision automatically triggers an action, or simply provides advice to a decision-maker. This means record keeping is required at all stages, to answer questions that will arise.

Releasing such information enables people to oversee, interrogate and challenge the validity of the system, as well as the validity of a particular decision. It also helps to improve the accuracy and fairness of algorithms. To be auditable, systems must be able to expose the relevant information in real time and this capability should be clearly defined from the outset.

POLL: Who has received an application for access to an algorithm or information about an automated decision – only 13% of webinar participants.

Many citizens don’t yet know they can ask for this kind of information, but this is changing. NSW IPC have published a fact sheet for citizens and another fact sheet for agencies to raise awareness about automated decision-making and information access rights.

Susan Bennett on the multidisciplinary approach to governance

Susan emphasised that it is important to establish the purpose of new technology, how it aligns with the goals of the organisation and understand who is the intended beneficiary.

She observed the growing interest also in privacy and that effective assessment of ethical data use and privacy impacts requires a multidisciplinary approach. Boards and senior executives must consider whether their organisations have the right governance structures in place to deal with these issues.

Procurement processes have a critical role in achieving privacy “by design” approach, ensuring that licences and contracts include the provisions to meet information access obligations. She pointed out that the “right to audit” should be included in contracts, not just in relation to data breaches but in order to confirm a solution is working as intended.

Organisations must also continue to invest in the fitness of their systems as data, technology and society change.

If you would like to listen to the full discussion you can access a recording of the session here.

Sonya Sherman is a member of the Information Governance ANZ advisory board member and Principal at Zen Information.

Principles of Explainable AI

by

AI
The US National Institute of Technology and Standards (NIST) has released a draft paper of 4 principles of explainable AI, which is one of several properties that characterise trust in AI systems. Other properties may include resiliency, reliability, bias and accountability. Usually, these terms are defined as a part or set of principles or pillars. This draft paper sets out there are 4 principles encompassing the core concepts of explainable AI as follows:

 

  • Explanation: Systems deliver accompanying evidence or reason(s) for all outputs.
  • Meaningful: Systems provide explanations that are understandable to individual users.
  • Explanation Accuracy: The explanation correctly reflects the system’s process for generating the output.
  • Knowledge Limits: The system only operates under conditions for which it was designed or when the system reaches a sufficient confidence in its output.

AI Transparency in Digital Government

by

In celebration of International Access to Information Day and Right to Know Week in NSW 2020, we held an event on AI Transparency in Digital Government with NSW Information Commissioner Elizabeth Tydd, Victorian Information Commissioner Sven Bluemmel and Dr Jat Singh, Senior Research Fellow at the University of Cambridge. The discussion focused on the duty government agencies have to disclose algorithms used in providing services and making decisions about services and benefits to citizens.

The Commissioners highlighted that robust procurement processes are essential where technology using algorithms are being procured by agencies. Commissioner Bluemmel said the bar needs to be set really high where the algorithmic decision-making involves people and their liberties and livelihood. Transparency is necessary to understand how the decisions are made in order to assert our rights. Dr Singh pointed out that transparency needs to be meaningful so that it allows us to be able to interrogate, scrutinize and challenge, and it requires organisations to give careful consideration to what is needed to enable adequate transparency of algorithms. Commissioner Tydd stated that agencies should be maintaining an inventory of the algorithms used in government-decision making and that it should be included on agency websites. The final point was that the cost to citizens of exercising the right to access information in relation to decision-making using algorithms should be minimal - that is, the cost should be lower if the information is held in digital form and not increase because a data scientist or lawyer is required to be involved with the access request.

You can access the webinar recording here. All the resources mentioned in the webinar can be accessed through the following links:

You can learn more about Right to Know Week and access more resources by visiting the RTK webpage.

EU/US data transfers under Privacy Shield invalidated

by

On 16 July 2020, the European Court of Justice invalidated its previous Decision (2016/1250) and the EU-US Privacy Shield, which enabled certified companies to transfer personal data between the EU and the US.

The case was brought by Max Schrems (the founder of NYOB) an Austrian resident, who had lodged a complaint with the Irish privacy regulator that the transfer of his Facebook data from Facebook's servers in Ireland to the US did not provide sufficient protection against access by the US public authorities.

The Court found that, based on the use and access by US public authorities and surveillance programmes, interfered with the fundamental right of persons whose data is transferred to the US.  However, the Court also determined that the Commission Decision 2010/87 on standard contractual clauses for the transfer of personal data to processors established in third countries is valid.

Read more
Court of Justice of the European Union Press Release 
Schrems II Judgment 
European Data Protection Board FAQs on the Judgment
Max Schrem's first reaction to the Judgment

EDPS Opinion on the EU Commission’s White Paper on AI – the European approach to excellence and trust

by

As a part of a wider package of strategic documents, the European Commission published a White Paper on “Artificial Intelligence: A European approach to excellence and trust”, which we brought you in our April newsletter and is available here. This Opinion presents the EDPS views on the White Paper as a whole, as well as on certain specific aspects, such as the proposed risk-based approach, the enforcement of AI regulation or the specific requirements for the remote biometric identification (including facial recognition). The EDPS recommendations in this opinion aim at clarifying and, where necessary, further developing the safeguards and controls with respect to protection of personal data.  Read the EDPS Opinion here.