Two new guidelines from the Canadian Judicial Council advance the discussion on the use of AI by adjudicators. The Guidelines move beyond warning of the risks to condoning responsible use of AI. Regulators will want to take notice as they develop their own policies.
The first, entitled Guidelines for the Use of Artificial Intelligence in Canadian Courts contains much of the usual advice, such as:
- Non-Delegation: Adjudicators cannot delegate decision making: “the judge is expected to select the appropriate tool, evaluate the work performed, exercise independent judgment and make every substantive decision.”
- Bias: Adjudicators must recognize that AI contains assumptions and values: “Such biases are not limited to political or social viewpoints; they may also reflect cultural norms, linguistic conventions, even the political leanings of their developers.”
- Verification: “Judges must avoid uncritical reliance on AI outputs. They should instead exercise independent judgment, critically evaluating the reliability and appropriateness of any output before using it.” This would include reviewing the primary source material.
- Deployment: Courts should assess AI systems (e.g., for reliability, privacy, security) before deployment and then continually audit, oversee and review them. For example, “Court-approved systems used for non-public information should therefore ordinarily provide clear and enforceable assurances that submitted information will not be used to train the available models and will be handled in accordance with judicially appropriate requirements for confidentiality, control and retention.”
Points that may be seen as less cautious include:
- Individual adjudicators ordinarily do not need to disclose what AI tools they use. Rather, the Court, as a whole, should disclose what tools are available to individual adjudicators and for what purposes.
- The Guideline contains a list of “appropriate uses” of AI by adjudicators including:
- organizing electronic exhibits, transcripts or disclosure into searchable categories
- generating a chronology from an agreed statement of facts or other undisputed evidence
- preparing a procedural history of the litigation from the court record
- summarizing lengthy affidavits, expert reports, transcripts or written submissions for later verification against the originals
- identifying potentially relevant legal authorities or secondary sources for further legal research
- comparing draft reasons or successive versions of documents to identify changes improving grammar, spelling, clarity or readability of one’s own draft text
- converting material into tables, timelines, checklists or other organizational formats [footnote omitted]
On the topic of summarizing materials, Professor Salyzyn suggests that this use has risks because of the known inaccuracy of AI, its lack of neutrality, and the loss of context, even when verified. She also doubts the value of AI summaries (e.g., in time saved) if there is true verification of the original sources by the adjudicator.
The second document is entitled Guidance for Canadian Judges on Evaluating Digital Media Evidence. It addresses when adjudicators “encounter digital media evidence in circumstances where there is an allegation, or even a possibility, that the material has been fabricated or materially altered using generative artificial intelligence (AI) tools.” The Guideline expressly mentions professional discipline proceedings as an example of where such issues may arise.
In terms of risk, the Guideline says:
Importantly, synthetic media create two distinct risks. The first is that fabricated or manipulated evidence may be presented as authentic. The second is that authentic evidence may be challenged as fabricated. Both risks have the potential to undermine the truth-seeking function of the courts and public confidence in the administration of justice.
The challenge for adjudicators is “not to master rapidly changing technologies, but to understand their implications sufficiently to evaluate the evidence presented and to determine when additional scrutiny, expertise, procedural or technical safeguards may be required.” Some of the points made in the Guideline include:
- “Much public discussion of synthetic media focuses on the possibility that fabricated evidence will be introduced and admitted into legal proceedings. Equally significant, however, is the possibility that authentic evidence will be successfully challenged as fabricated, and thus excluded from evidence. In either case the result may lead to a miscarriage of justice.”
- A critical question for adjudicators is to identify “who is standing behind the evidence” so that its authenticity, accuracy, and provenance can be evaluated. This may create additional duties for the discipline tribunals that follow relaxed rules of evidence.
- The approach of the adjudicator will vary with the type of AI evidence at issue. For example, where AI use is acknowledged (e.g., AI-assisted notes used by a registrant), the focus is on reliability, not deception.
- A second category is where AI use is not acknowledged, such as where a witness tenders a “deep fake” image or a witness mistakenly believes the evidence is genuine when it is not (e.g., a records custodian of an institution). Here, the authentication of the evidence, including by assessing its origin and chain of custody or receiving expert evidence where appropriate, may be required.
- A third category is indirect evidence, such as an AI-generated translation of a document. Here the adjudicator needs to ascertain how much of the information is from (or verified) by the witness, and how much is the output of AI.
- An associated issue relates to access to justice and procedural fairness: “Even mere allegations of synthetic media deployment raise potential access to justice concerns. Many litigants, particularly self-represented or economically disadvantaged individuals, will lack the resources to retain forensic experts capable of detecting synthetic media or analyzing digital evidence. Litigants, particularly vulnerable litigants, cannot simply be left to challenge potentially synthetic evidence without meaningful support.”
These guidelines provide useful information for all those involved in regulatory adjudication.
