Is Your SaaS Vendor Forcing You to Adopt Their AI?

Somewhere in your vendor portfolio right now, a piece of software you have used for years is quietly becoming an AI product. The help desk tool now summarizes tickets. The video platform transcribes and "recaps" every call. The CRM drafts replies. Most of the time you did not request any of it. It showed up in a release note, or it did not show up at all, and the default setting was on.

The pressure is real, and it runs in one direction. Vendors are racing to bolt generative AI onto mature products because their investors, their competitors, and their own roadmaps demand it. Gartner projected that more than 80 percent of enterprises would have used generative AI APIs or deployed generative AI applications by 2026, a forecast it made back in October 2023. That target has been met and then some. The question for a third-party risk team is narrower and more uncomfortable: when your vendor ships AI faster than it can secure it, and turns it on without asking, who owns the risk?

The honest answer is that you do, at least until your contract says otherwise. This article walks through why the go-to-market race is producing real data exposure, the documented cases worth knowing, and the practical levers a buyer still holds: contract language, liability caps, a focused vendor questionnaire, and the willingness to leave.

Speed to Market Is the Root Cause

Adding an AI feature to an existing SaaS product is not a small change. It usually means piping customer content to a large language model, standing up new retrieval systems that index your data, connecting to third-party model providers, and creating brand-new ways for that data to leave the building. Each of those is a security project. Shipped in a hurry, each is also a new attack surface.

The economics reward speed over caution. A feature that ranks well in a demo wins deals this quarter. The security review that would have caught a data-exfiltration path costs weeks and wins nothing visible. So the feature ships, the security work trails behind, and the gap becomes someone's incident. The cost data backs this up. IBM's 2025 Cost of a Data Breach Report found that breaches involving unsanctioned "shadow AI" cost about 4.63 million dollars on average, roughly 670,000 dollars more than breaches without it, and that shadow AI was a factor in 20 percent of the breaches studied. Of the organizations breached through AI models or applications, 97 percent lacked proper AI access controls. The controls are not keeping pace with the features.

This is a familiar shape if you have followed supply chain risk at all. It is the same pattern behind the breakdown of questionnaire-only diligence: a control that looks fine on paper masks a capability nobody actually tested. AI just widens the gap between the paper and the product.

What Actually Goes Wrong: The Documented Cases

Vague warnings do not move anyone. Specific, sourced incidents do. It helps to sort them into two buckets, because they call for different contract responses. The first bucket is data used or exposed by default, where nothing broke but the design itself created disclosure. The second is exposure through a flaw, where an AI feature opened a hole an attacker or a bug walked through.

Bucket one: on by default, disclosed to no one

In May 2024, Slack users discovered that the platform's privacy language permitted customer data, including messages and files, to help train Slack's global machine learning models on an opt-out basis, and that opting out required a workspace administrator to send an email. To be precise about what this was and was not: Slack said these were non-generative models used for features like search ranking and emoji suggestions, and that it did not use customer content to train its generative Slack AI. The verifiable criticism was the posture, a default-on data use with an email-only escape hatch and confusing policy language, not proof that anyone's direct messages trained a chatbot. Slack later revised the wording.

Zoom walked into the same fire a year earlier. In August 2023, researchers noticed that Zoom's terms of service, updated that March, granted Zoom a broad license over "service generated data" and appeared to allow training AI on customer content with no clear opt-out. After a public backlash that raised consent and GDPR concerns, Zoom amended the terms twice within days, ultimately stating plainly that it does not use audio, video, chat, screen sharing, attachments, or other customer content to train its own or third-party AI models.

LinkedIn followed the default-on playbook in September 2024, switching on a "Data for Generative AI Improvement" setting and beginning to use member data to train generative AI models, with an opt-out that members had to find and toggle themselves. The pattern across all three is the important part. The feature arrives switched on, the disclosure is buried in policy, and the burden of saying no falls on the customer.

Bucket two: the AI feature opened a hole

Design-level disclosure is the quiet risk. The louder one is an AI feature that becomes the path into data it was never supposed to touch. In August 2024, the security firm PromptArmor demonstrated an indirect prompt injection attack against Slack AI. An attacker who could post in a single public channel, with no access to any private channel, could plant hidden instructions. When a victim later asked Slack AI to search or summarize, the assistant would pull in those instructions from its retrieval corpus and act on them, in the demonstrated case exfiltrating an API key a developer had placed in a private channel by encoding it into a clickable link. Slack characterized the conditions as limited and specific, deployed a fix, and said it had no evidence of unauthorized access to customer data. The lesson stands regardless: a summarization feature became a data-exfiltration channel because it trusted content it should not have.

Sometimes the flaw is more ordinary. On March 20, 2023, a bug in an open-source Redis client library caused ChatGPT to serve cached data belonging to the wrong users. For roughly nine hours, some users could see the conversation titles of other active users, and OpenAI confirmed that payment-related data for about 1.2 percent of ChatGPT Plus subscribers active in that window may have been exposed, including names, email and billing addresses, card expiration dates, and the last four digits of a card. Full card numbers were not exposed. It was a caching bug of the sort that predates AI entirely, but it landed inside an AI product holding millions of people's prompts.

Exposed infrastructure is the third variant. In January 2025, Wiz Research found a publicly accessible, unauthenticated ClickHouse database belonging to the AI company DeepSeek. Anyone who found it could run arbitrary queries through a browser and read more than a million log entries containing plaintext chat history, API keys, and secret keys. There was no exotic exploit. The database was simply open. That is what "moving fast" looks like from the outside.

Even Microsoft, with all its security resources, stumbled here. When it announced the Windows Recall feature in May 2024, which periodically screenshots everything on a user's screen so AI can search their past activity, researcher Kevin Beaumont found the snapshots and their extracted text were stored in a local plaintext database that decrypted whenever the user was logged in. Because the data sat locally, an ordinary information-stealing malware could be trivially adapted to harvest it. Microsoft delayed the feature, made it opt-in and off by default, and re-architected it to require Windows Hello and encrypted, just-in-time access. Worth noting for accuracy: Recall's data stayed on the device rather than going to Microsoft's cloud, so the risk was local exposure, not vendor collection. The relevant point for buyers is that a marquee AI feature from a top-tier vendor shipped with a security model that did not survive first contact with researchers.

Not every case is settled fact, and it is worth being clear about which are not. The AI transcription vendor Otter.ai faces a consolidated class action, with complaints filed in 2025, alleging that its tool auto-joins and transcribes meetings without attendees' consent and uses transcripts to improve its AI. Reporting has described an incident in which Otter emailed a full transcript, including a confidential discussion that continued after a participant had left the call, to an unintended recipient. Those are allegations in active litigation and a reported anecdote, not proven findings, and they should be weighed as such. They are also exactly the kind of default-on, auto-join behavior a diligence process is supposed to catch before it becomes your problem.

Why "Forcing" Is the Right Word

A vendor rarely tells you to adopt their AI. They just make declining expensive. The feature ships enabled. The off switch, if it exists, is buried three menus deep or reserved for the enterprise tier. The new AI capability gets wired into features you already depend on, so turning it off degrades something you actually use. And the roadmap makes clear that the non-AI version of the product is the one heading for end of life.

This is a concentration problem wearing a new outfit. The same dynamic that makes a single dominant vendor dangerous when it goes down, which we covered in the context of the CrowdStrike outage, applies when that vendor changes what its product fundamentally does. If you cannot realistically switch and you cannot turn the feature off, you have not chosen the AI. It has been chosen for you, and every risk that comes with it has been assigned to you by default.

Traditional AI governance frameworks help you reason about this. The NIST AI Risk Management Framework gives you the vocabulary to map and measure an AI system's risks. But a framework does not give you leverage. Leverage lives in the contract and in your willingness to use the exit.

Fix It in the Contract

Security questionnaires describe risk. Contracts allocate it. If a vendor is going to introduce AI into a product that touches your data, the master services agreement and data processing addendum are where you make that safe, or at least accountable. The clauses below are the ones worth fighting for.

Notice and consent before any new AI processing

Require written notice before the vendor enables any AI feature that processes your data in a new way, with the feature defaulting to off until you affirmatively opt in. This single clause neutralizes the on-by-default trap. It converts a release-note surprise into a decision you get to make.

An explicit right to disable

The contract should give you a durable, self-service right to turn any AI feature off and keep it off, without losing access to the core product or its support. If the vendor cannot grant that, you have learned something important about how central the AI is to their plans, and how little control you will have.

No training on your data

State plainly that the vendor and its model subprocessors will not use your data, your users' content, or any derivatives to train, fine-tune, or improve any model, generative or otherwise, without your specific prior written consent. Bind subprocessors to the same term. This is the clause that would have made the Slack and LinkedIn situations non-events for a covered customer.

Subprocessor and model transparency

Require disclosure of which AI model providers sit behind the feature, where processing happens, what data is sent to them, and advance notice of changes. An embedded AI feature is often a fourth-party relationship in disguise, the same hidden concentration risk that catches teams who only ever assessed the vendor in front of them.

Breach notification that covers AI-specific events

Extend your breach notification clauses to cover AI-specific incidents: prompt injection, model output that discloses another customer's data, and inadvertent training on your content. Define these as reportable events with a tight clock, because the standard definition of "security incident" may not obviously capture them.

Raise the liability cap for data misuse

Standard SaaS contracts cap the vendor's liability at some multiple of annual fees, often twelve months, and that cap swallows most data incidents. When a vendor introduces AI that processes sensitive data, negotiate a higher cap, or a carve-out from the general cap, for breaches involving AI processing and unauthorized use of your data. The logic is straightforward. If the vendor is confident their AI is safe, a higher cap costs them nothing. If they resist strenuously, they are telling you how they price their own risk, and you should listen.

The AI Vendor Questionnaire

Before the contract, the diligence. A generic security questionnaire will not surface AI-specific risk, because it was written for a world where the product did not think. You need a focused set of questions aimed squarely at how the vendor built, secured, and governs the AI. Send these to any vendor introducing AI into a product you rely on, and treat evasive or absent answers as findings in their own right.

Data handling and training

• Is our data, or any derivative of it, used to train, fine-tune, or improve any model? Under what default setting, and how do we opt out?
• Which third-party model providers process our data, and in which jurisdictions?
• Is our data logically isolated from other customers' data in every stage of the AI pipeline, including prompts, embeddings, and retrieval indexes?
• How long is prompt and output data retained, and can we set or disable that retention?

Controls and defaults

• Are new AI features enabled by default or off by default? Can an administrator centrally disable them?
• Can the AI feature be fully turned off without degrading the core product?
• What controls prevent prompt injection and data exfiltration through AI outputs?
• Does the AI honor our existing access controls, so it cannot surface data a given user could not otherwise see?

Assurance and governance

• Have you tested the AI feature for prompt injection, data leakage, and jailbreaks? Will you share the results or a summary?
• Do you hold or plan to hold ISO/IEC 42001 certification, and does its scope cover this specific product?
• How do you govern AI internally against a recognized framework such as the NIST AI RMF?
• What is your notification commitment for AI-specific incidents, including a model disclosing one customer's data to another?

Be Willing to Walk

Everything above assumes you have leverage, and your strongest leverage is the credible willingness to leave. The single most clarifying position a buyer can hold is this: if the AI cannot be disabled, and the vendor will not commit in writing that our data will not train their models, we walk.

That sounds harsh until you price the alternative. The downside of switching vendors is a migration project with a known cost and a known timeline. The downside of accepting undisableable AI on sensitive data is an open-ended, uncapped exposure whose cost you learn only when it goes wrong, and whose average price tag, per IBM's shadow-AI figure, runs into the millions. A migration is a line item. A breach is a headline. For your most sensitive workloads, the vendor who refuses to let you turn the AI off has told you exactly where you stand in their priorities, and it is not at the top.

Walking does not have to be all or nothing. You can keep a vendor for low-sensitivity work while pulling regulated or confidential data onto a platform that will meet your terms. You can stage an exit over a renewal cycle rather than overnight. What matters is that "no" remains a real option, because a "no" you cannot afford to say is not leverage. It is a bluff, and vendors can tell.