AI impact assessment templates: Canadian assessments and UK transparency records

01

Which AI impact assessment template asks what

The templates ask for written descriptions, selected answers and named responsibilities in different combinations.

Microsoft’s Responsible AI Impact Assessment Template, released for external use in June 2022, has written fields for data requirements, related systems, intended uses and oversight responsibilities. It asks the team to “Define predictable failures, including false positive and false negative results” and describe their impact on stakeholders. Its summary asks for potential harms and initial mitigation ideas.

The Government of Canada uses its Algorithmic Impact Assessment to support its Directive on Automated Decision-Making. Its 65 risk questions and 41 mitigation questions include requests to describe the input data and the system’s role in the decision. The questionnaire asks “Will the system interface with other IT systems?” and whether “a non-automated alternative” is available during an extended outage, as yes or no. In the published results, those fields give the selected answers, not the names of connected systems.

The UK template has written fields for data sources, system architecture, how the tool fits into the work and “When and how does a person review the tool's output?”. It also asks for an appeals process where applicable, and for risks and mitigations.

In the EU, Article 27 of the AI Act requires a fundamental rights impact assessment before deployment for specified deployers of high-risk systems under Article 6(2): bodies governed by public law, private entities providing public services, and deployers of the systems listed in Annex III 5(b) and 5(c). Those last categories cover natural persons’ creditworthiness or credit scores, excluding financial-fraud detection, and risk assessment and pricing for their life and health insurance. The exception for critical infrastructure is limited to the safety-component systems listed in Annex III point 2. The assessment includes the processes in which the system is used, affected people, risks, human oversight and measures if those risks materialise. Answering the five questions does not by itself meet Article 27 or any other legal requirement.

02

Which data can it use?

Name every source the system reads, including what people type into it.

Defra’s record for Ask the toolkit, an assistant that answers questions about using AI at Defra, lists its data in one sentence: “The question the reader types, up to 4 earlier questions and answers from the same conversation, any options shown to them, and the published pages of the Defra AI digital toolkit.”

Transport Canada’s assessment of Pre-load Air Cargo Targeting is as exact: “The input data is made up of 8 data elements found on airwaybills: shipper and consignee names and addresses, commodity description, weight, piece count, and airwaybill number.”

03

Which systems can it access?

List the systems it reads from or writes to, where the model runs, and what it cannot reach.

Defra’s record says the backend “sends the question, the toolkit pages and fixed instructions to the model on Amazon Bedrock in the AWS London region”, and then: “There are no tools, no web access and no agent loop.”

The DVLA’s phone assistant draws the line at its records: “if any record access is required to answer the customer’s enquiry, then only the advisor can do this.”

04

What is it allowed to change?

Say what the system may do on its own, and what stays with a person.

Defra’s assistant “does not approve anything, grant access to anything, or make decisions about people.” The Disclosure and Barring Service sets a boundary for the use of its summary: “The AI output cannot be used as the sole basis for any case assessment, recommendation or barring-related decision.”

A system that does change something needs the boundary just as exactly. In Canada’s Study Permit Extension Eligibility Model, “Routine applications receive an automated positive eligibility determination, and officers continue to review admissibility and make the final decision on all applications.”

05

Who owns the output?

Name who is accountable for the system and who reviews its output, including when that review happens.

The UK template asks for the “Role title of the person accountable for the tool”, and a record can also say who reviews each output. At the Environment Agency, “The RGA’s outputs are reviewed by a permitting officer every time the tool is used.” At the Disclosure and Barring Service, “A caseworker reviews the output when it is provided to them as a PDF summary, using the citations to validate the information against the source documents.”

Newcastle City Council’s record for Magic Notes, a tool that transcribes and summarises social care conversations, makes review a condition of treating the output as final: it “should not be considered a final product until the practitioner undertakes a thorough review.”

06

What happens when it fails?

Write down who is told, who takes over and how the work continues when the system is wrong, cannot answer or stops.

The DVLA names a handover trigger: “If the IVR cannot confidently interpret an intent, it will transfer the customer to a DVLA advisor to continue the conversation.” For a tool that reads addresses on returned letters, the Department for Work and Pensions says: “If for any reason we are unable to extract the address, it is reviewed by a human.”

Defra describes an alert: “Alerts go to the team’s Slack channel when answers fail, at half the daily limit and when the limit is reached.” For your own system, give an alert an agreed response: who investigates, who can stop the tool and how the work continues.

07

From a government record to a company system

The five questions carry over to a company, but the answers will differ.

These records describe public bodies’ systems: cargo screening, study permit applications, barring referrals, returned letters, assistants for staff. A company’s AI system, for example an assistant connected to its documents, CRM or code, needs the same five answers about different data, systems and people. Start from the five questions and write answers for your own system.

The five answers describe how the system operates. An impact assessment also needs to examine who could be affected and how, including when the system works as intended. Add the relevant harms and safeguards, and any process people need to challenge a decision.

If the company already has an AI use policy, the assessment is where its rules on data, tools and review meet one system. When a board asks what AI changes in the company’s operation, the assessments of the systems the company relies on are part of the answer.

08

A one-page starting point for an AI impact assessment

Write a sentence or two for each question, then have the person accountable for the system review the page.

Data: each source it reads, including what users type, and what it must not use.

Systems: each system it reads from or writes to, where the model runs, and what it cannot reach.

Changes: what it may do on its own, and what stays with a named role.

Output: the role accountable for the system, who reviews its output and when, and which uses need approval.

Failure: what happens when it is wrong, cannot answer or stops, who is told, and how the work continues.

Revisit the page when the system gets new access or more people start to depend on it, because the requirements should follow the risk: what it can access, what its output is used for and what happens when it fails.