Local, cloud or hybrid: where should your AI run?

Where your AI runs should be decided by what your data is allowed to do, not by what is easiest to build. The same system can ship three ways: local, cloud, or hybrid. Your records, your regulator and your tolerance for latency decide which.

Local: nothing leaves the building

Local deployment runs everything on your own machines: messages, knowledge, embeddings and inference all stay on hardware you control. That is the simplest GDPR posture available, because you have no processor, no transfer and no data processing agreement to negotiate. The personal data never reaches a third party.

The cost is speed and availability. CPU-only inference takes 15 to 35 seconds per answer, and the system answers only while the machine is switched on. A practice that closes at seven and whose records must not leave the building can live with both.

Choose local when: your data cannot leave the building, and you can live without the system overnight.

Cloud: always on, with a subprocessor

Cloud deployment puts an agent on a server that runs around the clock and is reachable from anywhere. That covers what you want when you say “always on”, and for most businesses it is the obvious choice.

It carries an obligation. The moment personal data reaches a model provider, that provider becomes a subprocessor under GDPR, which means you need an EU-resident host and a data processing agreement in place before you go live. The paperwork will not block you, but you cannot skip it, and arranging it before launch beats arranging it after.

Infrastructure for a small deployment runs about €5 to €10 a month, which seldom decides anything.

Choose cloud when: the system needs to answer at 3am, and your data can lawfully be processed by an EU-resident provider.

Hybrid: sensitive data stays, everything else goes

Hybrid processes sensitive records locally and routes everything else to a cloud agent, with the two kept in sync. You get the always-on surface without handing over the records that worry you, which makes it the most valuable of the three.

It is also the one with the most moving parts. There is a sync boundary to design, a decision rule about what counts as sensitive, and an ownership question about what happens when the two halves disagree. You can answer all three, but they take real work, and any quote that treats hybrid as a configuration flag has under-scoped it.

Choose hybrid when: you are a medical, legal or financial practice that needs both availability and residency, which describes a great many Greek practices.

The question to answer first

Before comparing latency or cost, answer this: which specific records in your business are not allowed to be processed by a third party? If the answer is “none”, pick cloud. If the answer is “all of them”, pick local. If the answer names three categories, you want hybrid, and the design should start from that list.