AI tool risk assessments

Not every AI feature needs a formal review. Under a sensible AI governance framework, low-risk uses with approved tools go ahead under policy. A specific assessment is warranted when any of these apply:

• The tool will process personal information, especially health, financial or employee data.
• It will handle confidential client material or privileged legal information.
• Its output will be relied on for decisions about people, such as hiring, credit, eligibility or discipline.
• It connects to your Microsoft 365, Google Workspace, CRM or document systems with broad permissions.
• It records or transcribes meetings or calls.
• The vendor is new, small, or hosted outside Canada.
• A client contract or regulator expects you to assess AI tools before use.

We map exactly what the tool collects and where it goes: what users type in, files and mailboxes it can read through integrations, audio and video it captures, and metadata about users. Then we answer the question most people care about: does the vendor use your data to train or improve its models? Many enterprise AI products say no by default; many free and consumer versions of the same products say yes, or allow it unless you opt out. The answer often depends on licence tier and settings, so we check the version you will actually use.

The privacy commissioners' 2023 Principles for responsible, trustworthy and privacy-protective generative AI technologies expect organizations using generative AI to identify the legal authority for collecting and using personal information, to limit collection to what is necessary, and to be open about their use of AI. Our data-flow map gives you the facts to meet those expectations.

Next: how long does the vendor keep prompts, outputs, files and recordings? Can you set or shorten retention? Where is the data stored and processed, and which subprocessors are involved, including the AI model provider behind the product? The Office of the Privacy Commissioner of Canada's cross-border guidance holds you accountable for information transferred to processors, including abroad, and expects you to tell people when their information may be processed outside Canada. If any of the information relates to Quebec residents, a privacy impact assessment is required before it is communicated outside Quebec.

We also look at the security basics the Canadian Centre for Cyber Security highlights in its generative AI guidance (ITSAP.00.041): strong authentication, patching, and staff awareness of risks such as data exposure, inaccurate output and AI-generated phishing.

We read the terms that govern your actual subscription, including the data processing addendum, AI-specific terms and acceptable-use policies, and pull out what matters: data ownership, training use, confidentiality, breach notification, subprocessors, data location, retention and deletion, audit or assurance reports, and liability. Where the terms are weak, we note what to negotiate or which settings compensate. Legal interpretation of the contract stays with your counsel; we make sure they know which clauses carry the technical risk.

The assessment ends with a decision, not a list of concerns:

Each assessment is a short written report, aligned with the risk functions in the NIST AI Risk Management Framework (Govern, Map, Measure, Manage), that you can show a client, auditor or regulator. AI assessments are scoped before work starts. Fees are quoted per engagement. The effort depends on the tool: a platform with many integrations and data flows needs more review than a single-purpose app.

1. 01

   Approve

   The tool can be used for the stated purpose under your existing policy.

2. 02

   Approve with conditions

   For example: enterprise tier only, training opt-out confirmed, specific integrations disabled, retention set to 30 days, no health information, output reviewed before use.

3. 03

   Reject

   Risks can't be reduced to an acceptable level, with reasons and, where possible, a safer alternative.