Resources / Reference
Our AI Assurance Methodology
How we test AI systems: the criteria we apply, how we rate findings, and what our assurance letter and seal mean.
The short answer
We test AI systems against criteria agreed with you before testing begins, document the result of every test, rate each issue by severity, retest fixes, and state exactly what we tested in a signed assurance letter. Our approach draws on the NIST AI Risk Management Frameworkâs Measure function and the testing expectations of ISO/IEC 42001.
What we test
Performance
- Accuracy of answers and actions
- Consistency across similar requests
- Handling of edge cases
Safety and fairness
- Harmful or inappropriate output
- Differences across user groups
- Escalation to a person when needed
Privacy and security
- Leakage of personal or confidential data
- Prompt injection and manipulation
- Disclosure that users are talking to AI
How we rate findings
Critical
Must fix before launch
Exposes sensitive data or causes serious harm to users.
High
Fix before launch
Produces wrong or unfair outcomes users rely on.
Medium
Fix on a set timeline
Degrades quality without serious harm.
Low
Track and improve
Minor issues with little user impact.
Our process
- 1Agree criteriaScope, users, risks, and pass criteria in writing
- 2Design testsRealistic scenarios, edge cases, and misuse attempts
- 3Test and recordEvery result documented with evidence
- 4Retest fixesConfirm issues are resolved
- 5AttestSigned letter stating scope, criteria, and results
The assurance letter and seal
Our signed assurance letter states what was tested, the criteria applied, the results, and the date of testing. Systems with no open critical or high-severity findings may display the Cogent Assurance Tested seal and publish a public summary of results. The seal applies to the system as tested and expires after twelve months or after any material change to the system, whichever comes first.
See our AI Assurance Test and AI Red Team Assessment.
This page is general information for planning purposes. Laws and guidance change, so confirm current requirements before acting. It is not legal advice. For legal advice about your organization, consult a licensed attorney.
Talk through your AI and privacy questions
Tell us what you're working on, and we'll reply within one business day.