How much to trust a brand-new AI feature

A panel you did not ask for appears in a tool you already pay for. It summarises your inbox, or drafts replies, or turns a folder of notes into a deck. It is good. Within a fortnight two client jobs run through it, a Zapier step points at it, and you have stopped keeping the spreadsheet it replaced. Then one of three things happens. The vendor moves it behind a higher tier. The vendor changes what it does with the material you feed it. Or the vendor removes it, and the email announcing this arrives with a date on it that is closer than you would like.

None of that is unusual and none of it is a scandal. It is the normal metabolism of software that is being built while you use it. What is avoidable is being surprised by it. Every major vendor publishes its removal record, its data defaults, and in some cases the exact number of days’ notice it owes you, and all of that is readable before you commit. This guide is about reading those pages before you wire a feature into paid work, and about wiring it so that a withdrawal costs you an afternoon rather than a client. It is written for solo operators and small teams using AI features inside tools they buy. It is not a procurement process for a company with a vendor-risk function, and it will not help you if what you need is a contractual guarantee, because for most of these features there is not one to get.

Every new feature arrives with a default someone else chose

The first thing to look at in a new AI feature is not what it produces. It is the setting governing what happens to what you put in, because that setting already has a value and you did not pick it.

The defaults are not consistent, even inside one vendor. OpenAI’s help documentation says plainly that ChatGPT “improves by further training on the conversations people have with it, unless you opt out,” and that opting out is done through the privacy portal or Data Controls, after which “new conversations will not be used to train our models” [6]. The same company’s enterprise privacy page runs the other way: “By default, we do not use your business data for training our models,” covering ChatGPT Business, Enterprise, Edu and the API platform unless you explicitly opt in [7]. Same brand, same interface in places, opposite default. Which one applies to you depends entirely on which account you happened to be logged into when you tried the feature.

Google’s disclosure is worth reading closely for a different reason. A subset of Gemini chats “are reviewed by human reviewers (including Google’s trained service providers) to help improve Google services,” and those reviewed chats are “retained for up to three years” [8]. Your ordinary activity auto-deletes after 18 months by default, adjustable to 3 or 36 months [8]. So the retention on a reviewed conversation can outlast the retention on the rest of your history. Google also states that even with the Keep Activity setting off, or in a temporary chat, it still uses your chats to respond to you and to help protect Google, its users and the public, “including with help from human reviewers” [8]. Off is narrower than it sounds.

The practical move is dull. On the day a feature appears, find its data setting, write down what it says, and only then decide what you are willing to paste into it. That takes five minutes and it is the difference between a considered choice and a default you inherited.

Preview, beta and experimental are contract terms, not maturity labels

When a vendor puts “preview” or “beta” on a feature, most people read it as a modesty note about polish. It is not. It is the label that switches off the vendor’s obligations, and the terms behind it are usually explicit.

Google’s Pre-GA Offerings Terms are the clearest example in the market. Pre-GA offerings are provided “AS IS” without warranties, and they “may be changed, suspended or discontinued at any time without prior notice to Customer” and “are not covered by any SLA or Google indemnity” [4]. Technical support is excluded unless a written notice says otherwise, the data-location commitments do not apply, and Google’s liability for pre-GA offerings is capped at the lesser of the agreement’s own cap or $25,000 [4]. The same section adds that no data processing terms apply and that customers “should not use Pre-GA Offerings to process personal data or other data subject to legal or regulatory compliance requirements” [4]. Anthropic’s beta documentation is shorter and says a version of the same thing in a list: beta features may have breaking changes, may be deprecated or removed, may have different rate limits or pricing, and may not be available in all regions [5]. OpenAI’s deprecation policy sets the notice for preview models at as little as 2 weeks, against at least 6 months for generally available ones [1].

So the label is doing real work. A feature marked preview has already told you in writing what the vendor owes you, which is close to nothing, and you are free to use it anyway. What you are not free to do is act surprised. The test to apply is simple: if this went away on a Tuesday with no email, what breaks. If the honest answer includes a client deliverable, the preview label should have kept it out of the client deliverable.

The removal record is public and longer than the launch record

Vendors publicise launches and quietly publish removals, but the removals are published. Every major platform maintains a deprecations page, and reading one for ten minutes does more to calibrate your trust than any amount of reasoning about a vendor’s intentions.

OpenAI’s page is the fullest. The Assistants API was announced for deprecation on 26 August 2025 and shut down on 26 August 2026, with users directed to the Responses and Conversations APIs [1]. Agent Builder was announced on 3 June 2026 for shutdown on 30 November 2026, with migration to the Agents SDK or ChatGPT Workspace Agents [1]. The reusable prompts API carries the same dates, and the stated migration path is to “move reusable prompt content into your application code” [1]. Existing evals on the Evals platform become read-only on 31 October 2026, a month before the dashboard and API shut down [1]. The Realtime API beta shut down on 12 May 2026 in favour of the released version, and dall-e-3 shut down the same day [1]. Older workhorses have dates too: gpt-4-0613 and gpt-3.5-turbo-0125 on 23 October 2026, whisper-1 on 26 February 2027 [1].

The others read similarly. Anthropic retired claude-opus-4-1-20250805 on 5 August 2026 and claude-sonnet-4-20250514 on 15 June 2026, each with a named replacement [2]. Google shut down the Gemini 2.0 Flash family on 1 June 2026, the Veo 2.0 and 3.0 generation endpoints on 30 June 2026, and the Imagen 4 variants on 17 August 2026 [3]. Google also deprecated the temperature, top_p and top_k sampling parameters on 21 July 2026 [3], which is the kind of change that breaks a working automation without removing anything you would have thought of as a feature.

Notice what that list is not. It is not a record of unusual failure. It is what a healthy, fast-moving platform looks like from the outside, and it means the correct base rate for “will this specific thing still exist in eighteen months” is well below certainty for anything that is not the core chat box.

How much notice you actually get, in writing

The commitments exist, they are specific, and they are shorter than most people assume.

OpenAI commits to at least 6 months’ notice before retiring generally available models, at least 3 months for specialised variants such as chat, Codex and deep-research variants, and as little as 2 weeks for preview models, adding that if safety or compliance concerns require it to retire a model sooner it “will provide as much notice as reasonably possible” [1]. Anthropic notifies “customers with active deployments” and commits to at least 60 days’ notice before retiring a publicly released model [2]. Anthropic also warns that deprecated models “are likely to be less reliable than active models” and that requests to a model past its retirement date will fail [2]. Google publishes dated shutdowns per model on its changelog rather than a single global promise [3].

Two things follow. First, 60 days is the floor you should plan against, not six months, because the shortest published commitment across the tools you use is the one that governs your worst week. Second, and more important for most readers of this site, every commitment quoted above is a developer-platform commitment. If you meet a feature inside a chat app, a document editor or a CRM sidebar rather than through an API, none of those notice periods have been promised to you. What governs you there is the product’s terms of use, and those are written at a coarser grain. OpenAI’s terms promise advance notice and a refund of prepaid, unused fees if it discontinues the Services, at least 30 days’ notice of a subscription price increase, and at least 30 days’ notice of terms changes that materially adversely impact you [9]. Read that closely and you will see it covers the Services and the price. It does not cover any particular feature inside them.

Depth of wiring decides whether removal is a chore or a crisis

You cannot control whether a feature survives. You control how much of your working life passes through it, and that decision is made quietly, over weeks, by default.

Keep the inputs outside the feature. If your prompts, templates and source material live only inside the vendor’s interface, the vendor’s removal takes your work with it. OpenAI’s own migration guidance for its reusable prompts API was to move the prompt content into your application code [1], which is the general lesson stated by the vendor about its own product: the durable copy belongs somewhere you control. A folder of text files, a Notion database, a Google Doc. Anywhere that is not the feature.

Keep the feature out of what you sell. There is a difference between using an AI feature to produce a client’s monthly report and telling the client their monthly report is produced by that feature. The first survives a withdrawal invisibly. The second turns a vendor’s product decision into a conversation with a paying customer about why the thing they bought has changed. Sell the outcome, and name the tool only when the client needs to know for their own compliance reasons.

Export on a schedule, and know your second option before you need it. If the feature holds anything you would miss, such as transcripts, a knowledge base or generated assets, get a copy out monthly and check that the export actually opens. And spend twenty minutes, once, establishing which competing tool you would move to. Not migrating to it. Just knowing its name, its price and roughly how long the move takes, so that the day the email arrives you are making a scheduling decision rather than starting research.

The twenty-minute check before a feature touches paid work

Four checks, in order, before anything a client pays for depends on a feature you met this month.

Check the label. Preview, beta and experimental mean the vendor has reserved the right to change or remove the thing without notice, and has usually written that down [4][5]. Check the data default. Find the setting, read it, and confirm whether the account you are actually using is the consumer tier or the business tier, because the defaults differ [6][7]. Check the retention as well, which can run longer for material selected for human review than for the rest of your history [8]. Check the notice. Look up the vendor’s deprecations page and write down the shortest commitment that applies to you [1][2][3]. Check the exit. Name where your inputs live outside the tool, when you last exported, and which tool you would move to.

None of that requires legal training and all four answers stay true for months. The point is to have them written down before the feature is load-bearing, because after that the same questions cost a great deal more to answer.

What still goes wrong

The published notice periods are commitments about models and platform endpoints, and the thing that disrupts a small business is usually neither. It is a button in an app moving behind a higher price tier, a limit dropping from unlimited to a number, or a feature quietly getting worse because it was repointed at a cheaper model. There is no deprecations page for any of that. The checks in this guide will tell you what a vendor owes you at the API layer and much less about what happens in the interface you actually use, where the written commitments cover the service and the price rather than the individual feature [9]. OpenAI also reserves the right to retire a model sooner than its stated notice if safety or compliance requires it, promising only as much warning as is reasonably possible [1].

The data settings have a second limit worth naming. Reading a policy tells you the vendor’s stated intent today, not what applies to a conversation you had last year, and not what applies after the setting changes. Google’s own page is honest that turning activity off does not stop all human review [8]. The durable protection is the one that does not depend on a setting: do not paste what you would not be comfortable having read by a stranger at the vendor, redact client names and real figures, and use the business tier for client work if you have one.

Finally, the advice to stay shallow has a cost, and it is real. The whole benefit of a good feature is that it removes work, and a workflow you have deliberately kept portable removes less of it than one you have committed to completely. There is no setting that gives you both. The judgement to make is per feature and per client: go deep where the work is yours and the rebuild would be an inconvenience, stay shallow where the work is somebody else’s and the rebuild would be a phone call you do not want to make.