Webinars      Login/Register      Cart

InfoGovANZ

Information Governance Think Tank

AI & Ethics

Ethics & Implementing AI Responsibly – Aurelie Jacquet

by

As we put 2020 behind us and look forward to 2021, we reflected in an interactive virtual discussion forum on the key IG learnings from the past 12 months and the insights and actions we now need to be taking to make the most of the opportunities and challenges on the road to recovery in 2021.

Our expert panel included InfoGovANZ International Council member Aurelie Jacquet, who works on leading global initiatives for the implementation of Responsible AI with both the International Standards Organisation (ISO) and the Institute of Electrical and Electronics Engineers (IEEE).

With ISO, Aurelie is the chair of the Standards Australia committee representing Australia at the international standards on Artificial Intelligence. With the IEEE, she is an expert for the Ethics Certification Program for Autonomous and Intelligent Systems, and leads the work stream on Dignity and Agency as part of IEEE’ s Digital Inclusion, Identity, Trust, and Agency (DIITA) program.

She is also a member of the European AI Alliance and a member of the editorial board of Springer’s new Journal AI & Ethics.

Aurelie’s insights:

Aurelie Jacquet
  • In the past few years, there has been a proliferation of AI ethics principles developed by many different actors (e.g., private companies, governments, intergovernmental organisations, civil society, etc.) that has mostly helped with:
    • Defining a north star for the adoption and use of the powerful technology that is AI;
    • Identifying the level of commitments/expectations of the various actors; and
    • Identifying areas of consensus.
  • When considering a technology such as AI, predicted to add $15 trillion to the world’s economy[1], form a part of our everyday life and is the subject of global governance efforts[2], identifying areas of consensus is a valuable exercise. In January 2020, to uncover these areas of consensus, the Berkman Klein Center created a visualization tool called ‘Principled AI’.[3] (As a result of this exercise, they “uncovered eight key thematic trends: privacy, accountability, safety and security, transparency and explainability, fairness and nondiscrimination, human control of technology, professional responsibility, and promotion of human value.”[4]
  • There are growing concerns that “principles alone cannot guarantee ethical AI.”[5]  In his article, Brent Mittelstadt, Senior Research Fellow at the Oxford Internet Institute, challenges whether ethics principles are appropriate for AI, and explains that “the real work of AI ethics begins now: to translate and implement our lofty principles and, in doing so, to begin to understand the real ethical challenges of AI.”[6]
  • 2021 is likely to be a year focused on operationalising AI principles, but also a year where we see more often ‘AI in court,’ reminding us that decisions automated by AI are still subject to existing laws. Even if the AI model used is a black box, organisations still need to demonstrate that the recommendations or decisions made were reasonable and in compliance with the applicable laws. To quote the Australian Human Rights Commission, "any business should ensure that its decision-making is fair, accurate and avoids bias or discrimination.  This proposition should be equally true for decision-making that uses AI systems.”[7] The reminder that AI is not lawless came as early as January this year when an Italian court found that Deliveroo's rider-ranking algorithm was in breach of local labour laws as the algorithm would penalise riders who cancelled pre-booked rides less than 24 hours even in circumstances where riders are sick or had to attend to an emergency[8]. This case reemphasises the need for organisations to ‘build in compliance,’ especially when it can be a costly exercise to retrain machine learning models.

Actions for leaders:

  • To implement AI responsibly in 2021, while technical requirements, such as data quality and model performance remain essential, focus strongly on broader organisational processes and controls, such as reviewing the adequacy of existing governance processes, embedding dynamic risk management and impact assessments to ensure better oversight of AI, mitigating risks and achieving compliance by design.

Given the expected ongoing complex and uncertain operating environment in 2021, robust information governance is needed to provide a system for the effective control and management of information assets to enable access to real-time and accurate information, as well as optimise data assets and value-generating activities while minimising risks.

You can read the insights from the rest of our expert panel in our InfoGovANZ Key Learnings from 2020 – Action and Insights for 2021 Report. The report was developed from a virtual forum discussing the impact of COVID-19  and IG implications for organisations on data, access to information, trust, transparency and accountability, cybersecurity, global privacy regulatory developments, eDiscovery, ethics and artificial intelligence.

You can also watch the recording of the 28 January 2021 webinar here.

 

References

[1] “AI Will Add $15 Trillion To The World Economy By 2030,” Frank Holmes, Forbes, published 25 February 2019
[2] “AI & Global Governance: Using International Standards as an Agile Tool for Governance,” Peter Cihon, UNU-CPR Centre for Policy Research, published 8 July 2019
[3] “Principled Artificial Intelligence, Mapping Consensus in Ethical and Rights-based Approaches to Principles for AI,” Jessica Fjeld, Adam Nagy, Nele Achten, Hannah Hilligoss, Madhulika Srikumar, Berkman Klein Center, published 15 January 2020.
[4] Ibid 2
[5] “Principles alone cannot guarantee ethical AI,” Brent Mittelstadt, Oxford Internet Institute, University of Oxford, Article in Nature Machine Intelligence, published in November 2019
[6] Ibid 4
[7] “Businesses warned AI use risks breaches of anti-discrimination laws,” James Eyers, Australian Financial Review, published 24 November 2020
[8] “Italian court rules against “discriminatory” Deliveroo rider-ranking mechanism,” Natasha Lomas, published 5 January 2021 Techcrunch

Using artificial intelligence to make decisions: Addressing the problem of algorithmic bias

by

This technical paper is a collaborative partnership between the Australian Human Rights Commission, Gradient Institute, Consumer Policy Research Centre, CHOICE and CSIRO’s Data61. We explore how the problem of algorithmic bias can arise in decision making that uses artificial intelligence (AI).

This problem can produce unfair, and potentially unlawful, decisions. We demonstrate how the risk of algorithmic bias can be identified, and steps that can be taken to address or mitigate this problem.

AI is increasingly used by government and businesses to make decisions that affect people’s rights, including in the provision of goods and services, as well as other important decision making such as recruitment, social security and policing. Where algorithmic bias arises in these decision-making processes, it can lead to error. Especially in high-stakes decision making, errors can cause real harm.

The harm can be particularly serious if a person is unfairly disadvantaged on the basis of their race, age, sex or other characteristics. In some circumstances, this can amount to unlawful discrimination and other forms of human rights violation.

This paper describes the outcomes of a simulation. We have simulated a typical decision-making process and identified five scenarios in which algorithmic bias may arise due to problems that may be attributed to the data set, the use of AI itself, societal inequality, or a combination of these sources.

We investigate if algorithmic bias would be likely to arise in each scenario, the nature of any bias, and consider how it might be addressed. The scenarios are framed around a consumer’s interactions with an essential service provider that most people will deal with at some point—an energy company.

Read the paper here: Using artificial intelligence to make decisions

AI Transparency in Digital Government Highlights

by

To celebrate Right to Know 2020, Information Governance ANZ were delighted to host a timely discussion on the right to access information and the use of algorithms in government decision-making.

This interactive forum was facilitated by Susan Bennett, Founder of InfoGovANZ and our special guests included:

  • NSW Information Commissioner – Elizabeth Tydd
  • Victorian Information Commissioner - Sven Bluemmel
  • Senior Research Fellow, University of Cambridge – Dr Jat Singh

The increasing adoption of technology across society, including in government, requires the preservation, assurance and assertion of information access rights. The right of access to government information also extends to information held by contractors that provide services to the public on behalf of government.

Applying and challenging these rights becomes more complex in the new world of automated decision-making. Legal frameworks, licensing and contracts must evolve to ensure information governance is applied to the design and use of algorithms, so that rights and interests are protected.

Commissioner Sven Bluemmel on building trust through transparency

Commissioner Bluemmel explained that Right To Know Day is actually an global event, recognised as International Access to Information Day (IAID). It aims to raise awareness of the importance of open government, and the public’s right to access government-held information.

We consider this right as the foundation for transparent and accountable government in a democratic society, supporting public participation and scrutiny of government decisions, and by extension supporting better decision-making.

As the public sector is increasingly making use of technology such as artificial intelligence (AI) to provide services and make decisions, there are questions about how the principles of good governance can best be applied. OVIC published an e-book called “Closer to the Machine”, exploring the technical, social and legal aspects of AI, including algorithmic transparency.

The Commissioner noted the importance of trust to the relationship between government and citizens. The relationship people have with government is very different from their relationship with a business or other kind of service provider where an algorithm could be considered the “secret sauce”.

Government decisions impact people’s livelihoods, their freedoms and human rights. It’s one thing if an algorithm incorrectly predicts my favourite songs or shopping preferences. It’s quite another if I’m denied a pension or parole based on an automated decision.

So government has to take a very different approach to the application of AI and similar technologies. It has to ensure transparency and accountability are fully supported so that decisions can be explained, challenged and, most importantly, trusted.

This aspect of trust and trusted information is especially important during times of crisis. “Forcing” people to do the right thing is far less effective than having a trusted relationship with constituents. There has to be trust in the government’s motives as well as its mechanisms. In this situation, reliance on defensible data and explainable modelling comes more into play.

Commissioner Elizabeth Tydd on the shift to digital government

Commissioner Tydd mentioned some findings from the IPC Community Attitudes Survey. There is growing support (72%) for government use of de-identified data to plan and deliver public services. There is also strong demand for accountability and transparency in the collection and use of data with 78-81% support for more public reporting. Overall, 88% of respondents said their right to access government information was important.

People are becoming interested in a wider range of information, not only access to their personal information or information about government services. They also seek access to information about how policy is developed and how agencies operate. This has implications as government makes use of data and technology in new and different ways. Administrative practices must evolve to safeguard the right of access to information.

The Commissioner highlighted several case examples challenging algorithmic transparency in government decision-making and asked: Is the concept of access to information sufficiently “malleable” and “future proof”?

In one case, a social housing tenant challenged the calculation of their rental subsidy, which had been determined using an algorithm. It was difficult for the agency to respond to the citizen’s request for information about the decision-making process. Procurement of the system was below the threshold for public reporting on government contracts, and the developer claimed that details of the algorithm were commercial-in-confidence.

In NSW, the GIPA Act applies beyond government agencies to include certain information held by other organisations contracted to provide a public service. But the way this applies to automated decision making is still being tested.

The Commissioner noted that access to information is an enabling right – it enables a citizen to assert other rights. If access to information is restricted it can have much wider implications. She stressed that “accessibility” means information must be understandable on every level. It’s not sufficient to simply release the source code or require the expertise of a data scientist. Access must also come at minimal cost to the applicant.

In another case study, the Gradient Institute considered an AI-driven system for financial approvals, based on credit history and past transactions, which unfairly discriminated against female applicants. Historical data can introduce bias and AI predictions may become less accurate as society changes over time – leading to more perverse outcomes.

An applicant has the right to understand how a decision was made, but there are also sound business reasons for explainability and algorithmic transparency. The design phase is key. Organisations must ensure their processes are auditable, to understand how learning and recalibration happen.

POLL: Who is using machine enhanced or automated decisions – a high proportion “don’t know”.  Does this suggest that information governance practitioners need to get more involved with AI and machine learning projects?

Dr Jat Singh on how automated decision-making works

Dr Singh noted that automated decision-making sits at the intersection of law, policy, community and technology. He mentioned the difference between automated decision-making and enhanced decision-making, where machine learning uses data to determine probability or make predictions that support human decisions. It is a policy decision, not a technical decision, to determine what kinds of data use are appropriate.

A recent example of enhanced decision-making involved COMPAS software which uses algorithms to assess potential risks of recidivism. Judges may use these predictions to inform their decisions when detaining or sentencing a defendant. The algorithms remain a trade secret and this lack of transparency could cause a breach of due process. The case study also raised concerns about machine bias resulting from training data or input data, rather than the algorithm itself.

Dr Singh emphasised that accountability relies on explainability, but it can be challenging to explain how an algorithm works and make it transparent. Each stage of system design involves embedding norms, potential bias and assumptions – not just the training data. There is also a risk of “model skew” over time. Developers must consider whether a model is still fit for purpose in relation to the data and as society changes. Whether the model needs retraining for a business or other purpose, auditability is essential.

The key is to look more broadly at the system as a whole and identify information that can be made available. For example: information about the underlying data used by the algorithm; its source code and performance or test criteria; the software on which it is built; the process steps or workflow for a decision; and how it is deployed – whether a decision automatically triggers an action, or simply provides advice to a decision-maker. This means record keeping is required at all stages, to answer questions that will arise.

Releasing such information enables people to oversee, interrogate and challenge the validity of the system, as well as the validity of a particular decision. It also helps to improve the accuracy and fairness of algorithms. To be auditable, systems must be able to expose the relevant information in real time and this capability should be clearly defined from the outset.

POLL: Who has received an application for access to an algorithm or information about an automated decision – only 13% of webinar participants.

Many citizens don’t yet know they can ask for this kind of information, but this is changing. NSW IPC have published a fact sheet for citizens and another fact sheet for agencies to raise awareness about automated decision-making and information access rights.

Susan Bennett on the multidisciplinary approach to governance

Susan emphasised that it is important to establish the purpose of new technology, how it aligns with the goals of the organisation and understand who is the intended beneficiary.

She observed the growing interest also in privacy and that effective assessment of ethical data use and privacy impacts requires a multidisciplinary approach. Boards and senior executives must consider whether their organisations have the right governance structures in place to deal with these issues.

Procurement processes have a critical role in achieving privacy “by design” approach, ensuring that licences and contracts include the provisions to meet information access obligations. She pointed out that the “right to audit” should be included in contracts, not just in relation to data breaches but in order to confirm a solution is working as intended.

Organisations must also continue to invest in the fitness of their systems as data, technology and society change.

If you would like to listen to the full discussion you can access a recording of the session here.

Author

Sonya Sherman is a member of the Information Governance ANZ Advisory Board and Principal at Zen Information.

Principles of Explainable AI

by

AI
The US National Institute of Technology and Standards (NIST) has released a draft paper of 4 principles of explainable AI, which is one of several properties that characterise trust in AI systems. Other properties may include resiliency, reliability, bias and accountability. Usually, these terms are defined as a part or set of principles or pillars. This draft paper sets out there are 4 principles encompassing the core concepts of explainable AI as follows:

 

  • Explanation: Systems deliver accompanying evidence or reason(s) for all outputs.
  • Meaningful: Systems provide explanations that are understandable to individual users.
  • Explanation Accuracy: The explanation correctly reflects the system’s process for generating the output.
  • Knowledge Limits: The system only operates under conditions for which it was designed or when the system reaches a sufficient confidence in its output.

AI Transparency in Digital Government

by

In celebration of International Access to Information Day and Right to Know Week in NSW 2020, we held an event on AI Transparency in Digital Government with NSW Information Commissioner Elizabeth Tydd, Victorian Information Commissioner Sven Bluemmel and Dr Jat Singh, Senior Research Fellow at the University of Cambridge. The discussion focused on the duty government agencies have to disclose algorithms used in providing services and making decisions about services and benefits to citizens.

The Commissioners highlighted that robust procurement processes are essential where technology using algorithms are being procured by agencies. Commissioner Bluemmel said the bar needs to be set really high where the algorithmic decision-making involves people and their liberties and livelihood. Transparency is necessary to understand how the decisions are made in order to assert our rights. Dr Singh pointed out that transparency needs to be meaningful so that it allows us to be able to interrogate, scrutinize and challenge, and it requires organisations to give careful consideration to what is needed to enable adequate transparency of algorithms. Commissioner Tydd stated that agencies should be maintaining an inventory of the algorithms used in government-decision making and that it should be included on agency websites. The final point was that the cost to citizens of exercising the right to access information in relation to decision-making using algorithms should be minimal - that is, the cost should be lower if the information is held in digital form and not increase because a data scientist or lawyer is required to be involved with the access request.

You can access the webinar recording here. All the resources mentioned in the webinar can be accessed through the following links:

You can learn more about Right to Know Week and access more resources by visiting the RTK webpage.

EU/US data transfers under Privacy Shield invalidated

by

On 16 July 2020, the European Court of Justice invalidated its previous Decision (2016/1250) and the EU-US Privacy Shield, which enabled certified companies to transfer personal data between the EU and the US.

The case was brought by Max Schrems (the founder of NYOB) an Austrian resident, who had lodged a complaint with the Irish privacy regulator that the transfer of his Facebook data from Facebook's servers in Ireland to the US did not provide sufficient protection against access by the US public authorities.

The Court found that, based on the use and access by US public authorities and surveillance programmes, interfered with the fundamental right of persons whose data is transferred to the US.  However, the Court also determined that the Commission Decision 2010/87 on standard contractual clauses for the transfer of personal data to processors established in third countries is valid.

Read more
Court of Justice of the European Union Press Release 
Schrems II Judgment 
European Data Protection Board FAQs on the Judgment
Max Schrem's first reaction to the Judgment