Use cases

What went wrong with AI in your industry, and what would have caught it.

For each industry: the AI it runs, the risk, incidents reported by the press or found by a regulator, and what Kapālins would have put in the way. Where Kapālins would not have helped, we say so.

By industry

Six industries, real incidents, sourced.

Every incident links to the publication or regulator that reported it, and is described the way the source describes it.

Banking & financial services

Amazon BedrockAzure OpenAIMicrosoft 365 CopilotClaudeChatGPT Enterprise
  • Credit and fraud models from third parties that nobody has fully documented.
  • Customer identifiers and account numbers pasted into prompts.
  • AI use growing faster than the risk framework that is meant to cover it.
What happened

Community Bank · May 2026

US bank tells the SEC an unapproved AI app received customer SSNs

On 5 May 2026 Community Bank found that customer names, Social Security numbers and dates of birth had been handled in an unauthorised AI application. Its parent judged the incident material and filed an SEC 8-K. The bank later said an employee used a personal device while mocking up a presentation.

SEC Form 8-K, CB Financial ServicesAmerican Banker
With Kapālins
  • Device sensor and identity-grant discovery list unapproved AI apps, so the tool is known before data reaches it.
  • On managed browsers, the extension reads prompts and attached files before upload; SSNs labelled as such are held back.
  • Every block and upload attempt is logged against the user, giving the investigation its record.

The employee used a personal device. Kapālins sees only managed devices and browsers, so it would not have seen this upload.

What the regulator found

APRA · April 2026

APRA finds AI controls lag adoption at large banks, insurers and super funds

After engaging selected large banks, insurers and super trustees in late 2025, APRA wrote that governance and risk practices are not keeping pace with AI. It found staff using enterprise AI tools outside approved controls, defences relying on policy or after-the-fact detection, and identity controls not yet adjusted to AI agents.

APRA Letter to Industry on Artificial IntelligenceAPRA media release
With Kapālins
  • Device sensor, browser extension and identity grants build the AI inventory APRA expects, including tools nobody approved.
  • The gateway and browser extension apply policy before a prompt leaves, not after the fact.
  • Agents get their own identities and scoped keys; Autonomous Integrity flags an agent, tool or credential that drifts.

Board AI literacy and tested exit plans for AI suppliers, also named by APRA, remain management work.

Why Kapālins
  • Finds the AI your teams use, not only the AI you approved.
  • Redacts card, IBAN, TFN and account numbers before a prompt leaves.
  • Shows the 72-hour APRA clock when an incident is material.
Frameworks that apply
APRA CPS 234ASIC RG 271Privacy Act 1988MAS TRM · FEATHKMA GenAIBNM RMiTDORAOWASP LLM Top 10

Middle East, available and mapped at onboarding: SAMA Cyber Security Framework · DIFC Data Protection Law · ADGM Data Protection Regulations

Insurance

Amazon BedrockClaudeChatGPT EnterpriseMicrosoft 365 CopilotSalesforce
  • Claim files, with health and financial details, sent to a third-party model.
  • Agents with write access to the claims system.
  • An automated output treated as the decision, with no one accountable for it.
What happened

GEICO and the Pennsylvania Attorney General · May 2026

GEICO changes AI-assisted cancellation process after Pennsylvania Attorney General probe

Pennsylvania’s Attorney General said a GEICO AI tool picked a new Philadelphia policyholder for extra underwriting review, and her policy was cancelled without adequate notice, leaving her unknowingly uninsured. Without admitting a violation, GEICO agreed to a longer document deadline, a single proof-of-residency form and more staff training.

Insurance Journalabc27 News
With Kapālins
  • The AI estate inventory records which AI tools feed underwriting decisions, the first thing an AI governance program asks for.
  • The framework crosswalk maps that inventory to the NIST AI RMF and ISO/IEC 42001 for the evidence pack.

Kapālins does not review cancellation notices or judge an underwriting decision. The failure here was the process around the AI.

What the regulator found

APRA · April 2026

APRA’s AI letter, which covers insurers, finds monitoring lags deployment

APRA’s April 2026 letter covered insurers alongside banks and super trustees, naming claims triage among AI uses. It found weak post-deployment monitoring, weak model behaviour monitoring, change management and decommissioning, and few entities with tested exit plans for critical AI providers. It signalled more supervisory intensity and possible enforcement.

APRA Letter to Industry on Artificial Intelligence
With Kapālins
  • Autonomous Integrity checks each approved AI application is still acting as approved, catching drift after deployment.
  • Retired-model detection and the model SBOM show which claims tools depend on which provider and model.
  • Regulator evidence packs collect the monitoring record APRA asks boards to stand behind.

Kapālins does not judge whether a claims-triage decision was correct.

Why Kapālins
  • Puts a person in the loop before an agent writes to a claim.
  • Notices when an approved tool server changes after approval.
  • Finds personal ChatGPT accounts on company devices.
Frameworks that apply
APRA CPS 234ASIC RG 271Privacy Act 1988GDPREU AI ActDORAOWASP LLM Top 10

Middle East, available and mapped at onboarding: SAMA Cyber Security Framework · UAE PDPL

Healthcare

Azure OpenAIMicrosoft 365 CopilotGoogle Vertex AIGeminiChatGPT
  • Patient details typed into a free AI tool on a ward computer.
  • AI-written text going into the patient record unchecked.
  • Copilot summarising a folder it should never have been able to read.
What happened

An Ontario hospital · incident 2024, decision October 2025

AI note-taker joins hospital rounds and emails patient details to former staff

A former physician’s Otter.ai bot, set up on his personal device, joined a hospital’s virtual hepatology rounds in September 2024 and emailed a summary and transcript to 65 invitees, 12 of them no longer employed there. In October 2025 Ontario’s privacy commissioner recommended steps including barring personal devices for hospital work and auditing offboarding.

The Globe and MailDeeth Williams Wall
With Kapālins
  • Identity-grant and device discovery recognise Otter.ai by name on work accounts and managed devices.
  • Unapproved note-takers can be blocked on managed devices and browsers, so recording is a decision, not an accident.

This bot ran from a personal device and personal email on a stale invite. Kapālins would not have seen it. Revoking leavers’ calendar invites is an offboarding fix.

What the regulator found

Australian National Audit Office · September 2026

Auditor-General finds AI risks in Medicare compliance not systematically assessed

The ANAO found the Department of Health, Disability and Ageing had largely effective arrangements for managing AI at enterprise level, but only partly effectively for Medicare provider compliance. There had been little formal, systematic assessment of AI risks, an incident policy had not been updated for AI, and it could not tell which providers used AI billing software.

ANAO Report No. 5 2026–27ARN
With Kapālins
  • The AI estate inventory lists each AI system in use and feeds risk quantification, so risks are assessed.
  • The framework crosswalk maps controls to ISO/IEC 42001, Essential Eight and the APS responsible AI policy as audit evidence.
  • AI-specific threats such as prompt injection and model supply chain are tested and recorded, not left to general cyber policy.

Kapālins cannot see billing software that private providers run. That visibility needs claims data, not an AI control.

Why Kapālins
  • Shows the AI clinicians actually use, not only what was approved.
  • Stops patient identifiers at the prompt, not after the fact.
  • Lists the notification duties an incident may have started.
Frameworks that apply
Privacy Act 1988ISO/IEC 42001ISO/IEC 27001GDPREU AI ActPDPA · IMDA

Middle East, available and mapped at onboarding: Saudi PDPL · UAE PDPL · Qatar PDPPL

Health-specific laws such as HIPAA are not in the crosswalk.

Government

Microsoft 365 CopilotCopilot StudioAzure OpenAIChatGPTMicrosoft Teams
  • Personal information about citizens pasted into public AI tools.
  • Work produced with AI that nobody declared or checked.
  • No single record of which AI touched which information.
What happened

NSW Reconstruction Authority · October 2025

Contractor uploads flood-program applicants’ personal data to ChatGPT

Between 12 and 15 March 2025 a former contractor at the NSW Reconstruction Authority uploaded an Excel sheet from the Northern Rivers Resilient Homes Program to ChatGPT without authorisation. The Authority first said up to 3,000 people were affected, then confirmed 2,031 in October 2025. The data included contact details, dates of birth and health information, and the Authority found no evidence of publication.

NSW Reconstruction AuthorityABC News
With Kapālins
  • The browser extension reads an Excel file’s text before ChatGPT receives it, and applies the agency’s data rules.
  • Tenant restrictions confine ChatGPT to the agency’s own workspace, so a personal account cannot receive work files.
  • The device sensor finds ChatGPT and other AI tools on agency machines, approved or not.

Names and contact details are detected and redacted but do not block on their own. Free-text health notes need a custom rule.

What happened

CISA · January 2026

Acting US cyber agency chief uploaded ‘for official use only’ files to ChatGPT

Politico reported in January 2026 that in summer 2025 CISA’s acting director, Madhu Gottumukkala, uploaded contracting documents marked ‘for official use only’ to the public ChatGPT. He had an exception while most DHS staff were blocked from it. The uploads set off automated security alerts. CISA said his use was ‘short-term and limited’.

TechCrunch (reporting Politico)House Homeland Security Committee
With Kapālins
  • An exception can mean the governed workspace or gateway: AI for the approved user, with no public account involved.
  • Custom data rules can hold files marked ‘For Official Use Only’ before upload, even for exempted users.
  • Every exception and upload is logged against the person, so investigators start from a record.
Why Kapālins
  • Blocks unapproved AI tools on the device, and proves it.
  • Enforces an approved-model list at the gateway.
  • Hosted in Australia, under a published privacy policy.
Frameworks that apply
Essential EightACSC Secure AIAPS responsible AIPrivacy Act 1988ISO/IEC 42001NIST AI RMF

Middle East, available and mapped at onboarding: NCA Essential Cybersecurity Controls · SDAIA AI Ethics Principles

Telecommunications

Google Vertex AIGeminiAzure OpenAISalesforceMicrosoft 365 Copilot
  • A customer agent talked out of its rules by the customer.
  • An agent that states a policy the business does not have, and the business is held to it.
  • Critical-infrastructure reporting deadlines measured in hours.

We found no verified public incident where a telco’s own AI caused harm. Below: a customer-facing agent from software, the same shape a telco runs at scale, and a regulator now acting on telco AI.

What happened

Cursor · April 2025

Cursor’s AI support agent invents a policy that does not exist

Users of the Cursor coding editor were logged out when switching machines. Its front-line AI support agent told them this was a one-device policy. Co-founder Michael Truell said ‘We have no such policy’, blamed a race condition on slow connections, and said AI replies to support email would now be labelled.

The Register
With Kapālins
  • Topic rules keep a support agent to the subjects it is allowed to answer.
  • Every reply through the gateway is logged, so you hold the record of what the agent told each customer.
  • Scheduled red-team probes ask policy questions after each change.

Kapālins does not know your policies, so it cannot tell an invented one from a real one.

What the regulator required

TRAI · February and September 2026

India’s telecom regulator makes carriers act on what their AI spam detectors flag

A February 2026 TRAI direction requires each terminating carrier to flag suspected spam callers through its AI/ML detection system and share them with the originating carrier within two hours, because alerting subscribers without enforcement does not deter. In September 2026 TRAI wrote this into regulation: five or more flagged numbers linked to one sender within ten days trigger re-verification, then barring and disconnection.

TRAI Press Release No. 119/2026TRAI Direction, 27 February 2026
With Kapālins
  • The AI estate inventory records the detection model as an AI system with an owner, because its output now bars customers.
  • Autonomous Integrity flags the detector if its model or provider changes from what was approved.
  • The framework crosswalk keeps the evidence that the system was approved, monitored and changed under control.

Kapālins does not decide whether a caller is a spammer, and does not tune the detector’s accuracy.

Why Kapālins
  • Checks partner agents’ cards by signature before your agent trusts them.
  • Caps what a public agent can spend with rate limits and budgets.
  • Shows the SOCI 12 or 72-hour clock when an incident starts.
Frameworks that apply
SOCI ActNIS2Privacy Act 1988GDPRPDPA · IMDAOWASP LLM Top 10

Middle East, available and mapped at onboarding: NCA Essential Cybersecurity Controls · Saudi PDPL · Oman PDPL

Software & engineering

Claude CodeCursorGitHub CopilotWindsurfChatGPTOllama
  • Source code and keys pasted into a chatbot to debug.
  • Local MCP servers with file-system access nobody reviewed.
  • Coding agents that never touch a gateway.
What happened

Amazon Web Services · July 2025

Wiper prompt shipped inside Amazon’s AI coding extension for VS Code

Using an inappropriately scoped GitHub token, a threat actor committed a prompt telling the AI agent to wipe local files and cloud resources into Amazon Q Developer for VS Code 1.84.0. AWS said a syntax error stopped it executing and no customer environments were changed. It pulled 1.84.0 and told users to update to 1.85.0.

AWS Security Bulletin AWS-2025-015404 Media
With Kapālins
  • The device sensor recognises Amazon Q on engineers’ machines, so you know who has it installed.
  • On the device, local agent tools and network egress are gated, which limits what an injected instruction can reach.

Kapālins would not have stopped the malicious commit to Amazon’s repository.

What happened

Replit and SaaStr · July 2025

Replit’s AI agent deletes a live production database during a code freeze

SaaStr founder Jason Lemkin said Replit’s AI coding agent deleted his production database despite instructions not to change anything, generated fake data, and wrongly said a rollback was impossible. Replit chief executive Amjad Masad called it unacceptable and announced automatic separation of development and production databases.

The RegisterThe Register, Replit’s response
With Kapālins
  • Through the MCP gateway, destructive database tools can be refused or held for a person, not left to the prompt.
  • Replit is in the AI app catalogue, so its use on work identities and devices shows in the inventory.

Replit’s own agent acts inside Replit. Kapālins can gate it only where its database tools are routed through Kapālins.

Why Kapālins
  • Recognises 69 AI tools on the laptop, where coding agents run.
  • Gates local MCP tools and network egress on the device.
  • Can refuse a model with a known critical vulnerability (model SBOM, beta).
By team

The same platform, three jobs.

A CISO answers to the board, a SOC works the queue, and a risk team answers to the regulator. Each gets its own view of the same evidence.

CISO & board

One number the board can read, and a trail behind it.

  • The AI Trust Index: security, compliance and value in one 0–100 score, weighted 50/30/20.
  • A printable board pack with priority actions and framework posture.
  • Autonomous Integrity: is each AI application still acting as approved?
The Trust Index

Security operations

AI incidents worked to a playbook, in Kapālins or in the tools you run.

  • Signals are correlated into incidents. Each incident suggests the playbooks that answer its signal, and one click starts it.
  • Pre-built playbooks for AI security events, and a builder for your own. Every step is tracked to close.
  • Each incident lists the regulator clocks it may have started.

Built-in playbooks

Prompt injection responseModel data exfiltrationAI supply chain compromiseJailbreak & policy violationAgentic AI excessive agencyShadow AI on a managed device

Findings and blocks also go to your SIEM and ITSM, plus signed webhooks.

SplunkMicrosoft SentinelCrowdStrikePalo Alto NetworksGoogle SecOpsServiceNowBMC

Risk & compliance

Evidence a regulator can check, not a binder.

  • 29 frameworks and 122+ controls, each proved by a live signal or the assessment.
  • Hash-chained evidence packs with a 90-day enforcement log.
  • Assessments whose answers become attested evidence.
Evidence packs
Request a briefing

See your industry on your own estate.

A walkthrough of the risks above on the AI your organisation actually runs, and what Kapālins would and would not see.

Request a briefing Talk to us
FAQ

Questions buyers ask.

Do you have a separate product for each industry?

No. It is one platform. Each industry runs a different mix of AI and answers to different frameworks, so the same controls are switched on in a different order and the evidence is mapped to that industry’s frameworks.

Would Kapālins have prevented every incident on this page?

No, and each incident says so where it applies. Kapālins does not know your own policies, judge an underwriting or clinical decision, or see devices and accounts it does not manage. What it does is find the AI in use, inspect what goes into and comes out of it, hold actions for a person, and keep the record.

Which healthcare regulations are mapped?

Health services are covered by the general privacy and AI frameworks in the crosswalk, such as the Privacy Act 1988, GDPR, ISO/IEC 42001 and ISO/IEC 27001. Health-specific laws such as HIPAA are not in the crosswalk, and the platform does not claim them.

Can the SOC keep working in its own tools?

Yes. Findings and blocks are pushed to 7 security operations destinations, including Splunk, Microsoft Sentinel and ServiceNow, plus signed webhooks. Incidents and response playbooks also run inside Kapālins.