Hybrid AI: What Exactly Leaves Your Network?
With hybrid AI, documents stay local and a cloud model writes the answer. What gets sent, what to ask the vendor, and when hybrid is the right choice.
Privacy & Law8 min read
"Hybrid" is a popular word in AI product descriptions. It usually means part of the system runs on your premises and part in the cloud. Which part runs where, however, varies considerably between vendors. For a fiduciary office or law firm, that determines which client data leaves the building. This article explains what to look for.
Two designs under one name
An AI assistant for your own documents works in two steps. First it searches your files for the passages that match the question. Then a language model writes an answer from those passages. The technical term is retrieval-augmented generation, or RAG.
"Hybrid" can mean two very different things:
Design A: documents are uploaded to the cloud. During set-up, your files are transferred to the vendor, processed and stored there. The only hybrid element is that the interface or login runs through your network. Your complete client files sit permanently on someone else's servers.
Design B: documents stay on your server. Search, database and logs run on your premises. For each question, only the question itself and a few short text excerpts go to the cloud model. The model writes the answer, and the excerpts are not stored there permanently.
| Design A | Design B | |
|---|---|---|
| Where are the original documents? | With the cloud vendor | On your server |
| What goes out per question? | The question; the documents are already there | The question and three to five short excerpts |
| What sits permanently with the vendor? | The whole archive, often with its search index | Usually nothing, subject to the model provider's retention period |
| Who can see who asked what? | The vendor | Only your own logs |
| What would a data breach expose? | The entire archive | The excerpts from the affected queries |
Both designs can make sense. But if keeping your files in-house matters to you, you should know which one you are buying.
What a query looks like in design B
An example from the fictional Keller Treuhand AG. A member of staff asks: "What payment terms did we agree with Muster GmbH?"
The server searches the archive and finds three matching passages: a paragraph from the engagement letter, a line from a file note and a sentence from an email. The cloud model then receives:
- the instructions to the model, for example "Answer only from the following excerpts and cite the source",
- the question,
- the three excerpts, perhaps 300 to 600 words in total.
What is not sent: the rest of the engagement letter, the other Muster GmbH files, every other client's files, the search index, user names and logs.
Checking how sensitive the excerpts are
Short does not automatically mean harmless. An excerpt can contain a name, a social security number or a figure from a tax return. So check what actually goes out in your case:
- Ask to see the transmissions. A well-built system can log which excerpts were sent to the model for which question. Review a sample during a test week.
- Classify your archive. Not every folder is equally sensitive. Internal guidelines, templates and technical articles are a different matter from files containing health information or ongoing criminal proceedings.
- Find out whether certain areas can be excluded. Some folders perhaps should not be searched at all, or only with a local model.
- Assess the data legally. Sensitive personal data under Art. 5(c) of the Swiss Data Protection Act and data covered by professional secrecy under Art. 321 of the Swiss Criminal Code deserve their own review, with your professional association if in doubt.
What model providers do with the excerpts
With the large providers, the paid API is governed by different rules from the free chat apps. As of November 2026:
- OpenAI states that API data is not used for training by default and that abuse-monitoring logs are kept for up to 30 days. A zero-retention option is available to eligible customers on request.
- Anthropic says in its privacy centre that API inputs and outputs are deleted within 30 days, and that retained data is not used for training without express permission.
- Google states in the terms for the paid Gemini API that prompts and responses are not used to improve its products and are logged only for a limited period to detect misuse.
All three are headquartered in the US and therefore subject to the CLOUD Act. In design B, what could be affected is limited to the excerpts that were sent.
Questions to put to the vendor
Ask for written answers.
- Where are my original documents stored, and do they ever leave my server?
- Where is the search index, the processed version of my documents, kept?
- What exactly is sent to the cloud model with each question? Can you show me an example?
- How many excerpts, and how much text, go out per question at most?
- Which model from which provider is used, and under what contract terms?
- How long does the model provider keep the data, and is it used for training?
- Can I exclude certain folders or have them processed only locally?
- Where are questions, answers and user names logged?
- Do the existing folder permissions also apply to the AI?
- Can I move to fully local operation later without starting over?
If a vendor cannot answer the first three clearly, that tells you something.
When hybrid is a good choice
A well-built hybrid system is the most sensible option for many offices. It needs no graphics card, can be set up within weeks and costs little to run. You also get access to the most capable language models without handing over your archive.
Hybrid fits well if
- your files consist mainly of accounting, tax and contract documents without sensitive personal data,
- your client contracts allow processing by vetted subcontractors,
- you want to start small and decide later whether local operation is worth it.
If, on the other hand, not a single excerpt may leave your network, a fully local system is the right choice. What is possible there today is covered in AI without internet.
How KADARAG does hybrid
KADARAG follows design B. Documents, search index, user data and logs stay on your server. For each question, only the question and three to five short excerpts go to Gemini, GPT or Claude, whichever you choose. Every answer cites the document and page. You can switch to fully local mode later.
Your next step
Put the ten questions above to every vendor you are considering, including us. If you would like to hear KADARAG's answers directly, book an introductory call.