The publication of this series was inspired by the release ChatGPT, which is a generative artificial intelligence (AI) chatbox developed by Open AI. ChatGPT uses machine learning and natural language processing to provide relatively sophisticated and human-like responses to almost any question. Unlike traditional AI systems, ChatGPT is a generative AI platform, which means that the content it creates is “new,” rather than a reiteration of something that already exists.

As ChatGPT demonstrates, content can be produced through generative AI in a matter of seconds and may be composed of images, videos, audio, text or even code. The reality is that generative AI is well on the way to becoming not just faster and cheaper, but better in some cases than what humans create by hand. Every industry that requires humans to create original work – from social media to gaming, advertising to architecture, coding to graphic design, product design to law, marketing to sales – could be up for reinvention, in varying degrees. If you don’t believe us, try asking ChatGPT a question and see for yourself.

The rise of generative AI in business will be largely influenced by the development of legislation around the world. In this three-part series, we explore three areas of law that we see as critical to such development. In Part 1, we discussed the subject of generative artificial intelligence in the context of copyright law. In this Part two, we focus on important issues related to privacy law and the federal government’s proposed legislation to regulate AI, and steps you can take now to protect your business.

Regulation of AI under Canadian law

The regulation of AI under Canadian privacy law is in its infancy. None of the privacy statutes applicable in the common-law provinces, being British Columbia’s Personal Information Protection Act, Alberta’s Personal Information Protection Act and the federal Personal Information and Protection of Electronic Documents Act, directly address AI.

Québec

Steps that have been taken by Canadian legislatures have primarily sought to address the use of AI in the context of so-called “automated decision-making.” In that regard, Québec has been the first out of the gate. Its Act respecting the protection of personal information in the private sector (the Québec Act), as amended by Bill 64, will impose transparency and data subject rights obligations on organizations that use personal information to make decisions about individuals that are based exclusively on “automated processing.”

Importantly, the Québec Act itself does not define “automated processing.” Accordingly, guidance from the Commission d’accès à l’information du Québec (CAI) will be crucial in helping businesses understand the scope of what constitutes “automated processing.” However, businesses can safely assume that the use of AI systems to render decisions about individuals will almost certainly be captured by this phrase.

In such cases, organizations will be required to inform individuals when their personal information is used to render a decision based exclusively on automated processing, no later than at the time the individual is informed of the decision itself.¹ Further, upon request, organizations will be required to inform individuals about whom such a decision has been made and disclose:

• The personal information used to render the decision;
• The reasons and the principal factors and parameters that led to the decision; and
• The right to have the personal information used to render the decision corrected.²

Exactly how businesses are to comply with these obligations is unclear. For one, it is unclear whether an individual seeking an account of “the personal information used to render the decision” would be entitled to a description of the types of personal information included in the data used to train an AI system (requesters would clearly not have the right to access specific personal information of other individuals included in such training data). Further, the obligation to provide an account of the “reasons and the principal factors and parameters” leading to a decision would presumably require businesses employing AI systems to explain the operation of such systems. This will be challenging. For one, the inner workings of AI systems are famously opaque and, relying primarily on machine learning to train large data sets, are inherently fluid. Second, businesses will largely be relying on third-party vendors to provide them with AI systems. These vendors are likely to resist attempts to provide details of what may essentially amount to their intellectual property.

Under the Québec Act, individuals must also be given an opportunity “to submit observations to a member of the personnel of the organization who is in a position to review the decision made by automated means.”³ This would presumably require such personnel to review those decisions. Accordingly, organizations should be prepared to dedicate resources to assessing observations submitted about their use of AI systems to make decisions about individuals. It remains to be seen (again, hopefully in the form of CAI guidance) to what extent organizations will be required to action assessments of these systems that reveal issues in the processing of the requester’s personal information.

Bill C-27 – The Consumer Privacy Protection Act and Artificial Intelligence and Data Act

On June 16, 2022, the federal government tabled Bill C-27, the Digital Charter Implementation Act. Bill C-27 was presented for second reading on November 4 and 28, 2022, and, March 7 and 28, 2023. At the time of this writing, the House of Commons is in the process of debating referral of Bill C-27 to the Standing Committee on Industry and Technology for study and debate. Bill C-27, if passed, would replace PIPEDA with the Consumer Privacy Protection Act (CPPA) and would also enact the Artificial Intelligence and Data Act (AIDA).

Similar to the Québec Act, the CPPA would impose obligations on organizations using “automated decision systems.” Unlike the Québec Act, the CPPA would define these systems as “any technology that assists or replaces the judgment of human decision-makers through the use of a rules-based system, regression analysis, predictive analytics, machine learning, deep learning, a neural network or other technique.”⁴ This would squarely capture AI systems. Organizations would be required to provide general accounts of their use of such systems to make predictions, recommendations or decisions about individuals that could have a significant impact on them.⁵ Organizations would also be required, upon request, to provide individuals with an explanation of the prediction, recommendation or decision.⁶

This obligation is, in one sense, broader than that contained in the Québec Act; applying not only to the exclusive use of AI systems to render decisions, but also the use of those systems to assist human decision-makers in making predictions, recommendations or decisions. On the other hand, these obligations would only apply to those predictions, recommendations or decisions that would have “significant impacts” on individuals. “Significant impact” is, unhelpfully, not currently defined by the CPPA.

At second reading, the inclusion of the AIDA along with the CPPA in Bill C-27 was met with significant criticism. Members of Parliament noted the absence of a clear link between the two acts (apart from the general theme of privacy), as well as the need for more extensive debate and discussion of the AIDA as compared to the CPPA. Accordingly, the Speaker of the House ruled that the vote for the CPPA, along with its companion legislation, the Personal Information and Data Protection Tribunal Act, would be separated from the vote for the AIDA.

The AIDA

In its current form, the AIDA would regulate the activity of persons involved in the design, development and use of so-called “high-impact” AI systems that are used in international and interprovincial trade and commerce. What constitutes a “high-impact” system is currently deferred to future regulations, which are proposed to come into effect no earlier than 2025. However, on March 13, 2023, the federal government published  the AIDA Companion Document, which purports to explain the purposes and scope of the AIDA. The Companion Document provides some insight into the factors that would ultimately be considered in determining whether an AI system is high-impact. These include:

• Evidence of risks of harm to health and safety, or a risk of adverse impact on human rights, based on both the intended purpose and potential unintended consequences;
• The severity of potential harms;
• The scale of use;
• The nature of harms or adverse impacts that have already taken place;
• The extent to which, for practical or legal reasons, it is not reasonably possible to opt-out from that system;
• Imbalances of economic or social circumstances, or age of impacted persons; and
• The degree to which the risks are adequately regulated under another law.

Specific examples are included in the Companion Document:

• Screening systems impacting access to services or employment.
• Biometric systems used for identification and inference.
• Systems that can influence human behavior at scale such as online content recommendation systems.
• AI systems that have been used to create “deepfake” images, audio, and video that can cause harm to individuals.

As the AIDA stands, an AI system would be defined as a “technological system that, autonomously or partly autonomously, processes data related human activities through the use of a genetic algorithm, a neural network, machine learning or another technique in order to generate content or make decisions, recommendations or predictions.” The AIDA would also be primarily concerned with preventing: (i) physical or psychological harm to an individual, damage to an individual’s property and economic loss to an individual and (ii) biased outputs (i.e. output of AI systems that adversely differentiates without justification on one or more of the prohibited grounds of discrimination set out in the Canadian Human Rights Act).

Who would the AIDA apply to?

Given the criticism of the AIDA at Parliament, there is a very real possibility that it will be subject to significant revisions. With that said, if passed in anything resembling its current draft form, the AIDA would not only apply to the developers of the AI systems, but to potentially any actor in the AI supply chain, such as those businesses who leverage generative AI applications like Open AI’s GPT-4 to power their services.

Specifically, the AIDA would apply to “persons” who carry out any of the following “regulated activities” in the course of international or interprovincial trade and commerce:

• Processing or making available for use of any data relating to human activities for the purpose of designing, developing or using an AI system; and
• Designing, developing or making available for use an AI system or managing its operations.

What would regulation under the AIDA mean?

In accordance with forthcoming regulations, compliance obligations for persons responsible for high impact systems would relate to: (1) risk management; (2) transparency; (3) record keeping; and (4) notification.

Risk management

Persons responsible for high impact systems would need to establish protocols to identify, assess and mitigate the risks of harm or biased output that could result from the use of AI, and monitor compliance with such mitigation measures.

Transparency

Accountable persons would need to publish, on a public website, a plain-language description of the AI system which explains:

• How the system is intended to be used;
• The types of content it is intended to generate and the decisions, recommendations or predictions that it is intended to make;
• The mitigation measures established in respect of the system; and
• Any other information required by future regulations.

Record keeping

Persons responsible for AI systems would be required to keep records describing measures they have established to identify, assess and mitigate the risks of harm or biased output that could result from the use of the AI system and reasons to support their assessment of whether or not the AI system is a “high-impact system.”

Notification

Persons responsible for “high-impact systems” would be required to notify the Minister of Industry if the use of the system results or is likely to result in material harm, as soon as feasible.

Failure to comply

The AIDA would provide for fines for contraventions to the governance and transparency requirements set out above. Failure to comply with governance and transparency requirements could result in administrative fines:

• Up to the greater of CA$10 million and 3% of global revenues; or
• When prosecuted as a summary offence, the greater of CA$5 million and 2% of global revenues; or
• For individuals, an amount at the discretion of the court, up to a maximum of CA$50,000.

For offences, persons could be liable for:

• Up to CA$25 million or 5% of global revenues; or
• In the case of an individual, a fine at the discretion of the court or imprisonment.

How can you protect your business?

Regulation of AI in Canada is still in a nascent stage. Given the amount of criticism levied against the AIDA at Parliament, there is a good chance that it will undergo significant revisions before being passed. Further, even in its current form, many of the key obligations are deferred to the passage of future regulations. Nonetheless, some key compliance themes are beginning to emerge.

Businesses should begin to identify and catalogue any of their business processes that involve the use of automated decision-making systems to make predictions, recommendations or decisions. In doing so, businesses should have regard to the definition of such systems in the draft CPPA and AIDA. For each such system, businesses should identify whether the system is used to make a decision that would have an impact on the individual akin to that affecting the individual’s legal rights (including those over property), economic interests or potentially engage human rights grounds. Businesses should also identify where such systems are used to make decisions based exclusively on automated decision-making. In these cases, businesses should consider revising their privacy notices to address the transparency requirements in the foregoing legislation.

Businesses should also consider conducting assessments of any processes involving AI systems, in order to identify whether those systems would be considered “high-impact” within the meaning of the AIDA or the Companion Document. This would include an assessment of the potential risks of harm, including as a result of bias, arising from those processes. Where risks of harm are found, businesses should be prepared to implement appropriate mitigation strategies.

Where businesses are being provided with AI systems by third parties, they should begin to explore the inclusion of contractual provisions that require those vendors to provide a plain language description of how their technology operates. In doing do, businesses should point to their transparency obligations under the foregoing legislation. Further, businesses should be satisfied that vendors will be able to effect corrections to any personal information processed by their systems.

For more information about AIDA and other global AI regulatory guidance please check out Dentons’ AI: Global Solutions Hub.

For specific regulatory questions related to your business please reach out to Luca Lucarini or Noah Walters.

The authors would like to thank Piraveena Ganesarasa for her assistance with this article.

1. Québec Act, s. 12.1. ↩
2. Québec Act, s. 12.1 ↩
3. Québec Act, s. 12.1. ↩
4. Draft CPPA, s. 2(1). ↩
5. Draft CPPA, s. 62(c). ↩
6. Draft CPPA, s. 63(3). ↩