Replika just paid €5 million. OpenAI paid €15 million seven months earlier. Both for the same thing — running an AI product on European users without the privacy paperwork. If you’re building an AI companion, that’s the bar now, and you cleared none of it on the day you registered the company.
GDPR compliance for AI companion startups is not the legal homework you do after you ship. It’s what decides whether your payment processor survives the first complaint, whether the Italian Garante puts you on its watchlist, and whether August 2026 — when the EU AI Act’s full enforcement layer lands on top of GDPR — ends your runway. This is the version we wish every founder we onboarded already had, in the plain language we use with our NSFW chatbot development clients on day one.
Building a Candy.ai-style or DreamGF-style app and want the privacy architecture baked in from commit one? Book a free consult at tripleminds.co/contact-us or see how we shipped Candy.ai.
The €5 Million Wake-Up Call: What Replika Got Wrong
On 19 May 2025, the Italian Garante fined Luka Inc. (Replika’s parent) €5 million. Not a slap on the wrist — a documented breakdown across seven GDPR articles:
| GDPR Article | What it requires | What Replika did wrong |
|---|---|---|
| Art. 5(1)(a) | Lawful, fair, transparent processing | Couldn’t point to a clear lawful basis |
| Art. 6 | Identify a lawful basis | None identified in writing for the data flows |
| Art. 12 + 13 | Clear privacy notice before collection | Notice was vague and scattered |
| Art. 5(1)(c) | Data minimisation | Collected more than the service needed |
| Art. 24 | Demonstrate compliance | Couldn’t |
| Art. 25(1) | Data protection by design | Architecture wasn’t designed with GDPR in mind |
The Garante then opened a second investigation into whether Replika’s training data was lawfully sourced. That’s the playbook now: fine you for the app, investigate you again for the model. Same dance with OpenAI in December 2024 — €15 million plus a mandatory six-month media awareness campaign. OpenAI called it “disproportionate” and noted the fine was nearly 20× their Italian revenue. Regulators aren’t pricing the fine to your revenue — they’re pricing it to discourage the next founder. If your honest answer to “what’s our Article 6 lawful basis for storing conversation logs?” is “uh, terms of service?” — you are Replika in 2024.
The 9 Articles You Actually Need to Memorise
You don’t need to read all 99 articles. You need these nine:
| # | Article | Plain meaning | Where it bites |
|---|---|---|---|
| 1 | Art. 5 | Six core principles — lawful, fair, transparent, minimised, accurate, secure | Cited in almost every fine |
| 2 | Art. 6 | Lawful basis for every processing activity | “We have ToS” isn’t one |
| 3 | Art. 7 | Consent must be freely given, specific, withdrawable | Pre-ticked boxes = no consent |
| 4 | Art. 8 | Children need parental consent | Hence age-verification fines |
| 5 | Art. 9 | Special category data banned unless exception | Explicit consent for sexual / health-related |
| 6 | Art. 13/14 | Privacy notice at collection | Must name training-data use |
| 7 | Art. 17 | Right to erasure | Includes the model, not just the DB |
| 8 | Art. 22 | No fully-automated decisions with significant effect | Heavy disclosure for personalisation |
| 9 | Art. 35 | DPIA mandatory for high-risk processing | AI companion = always mandatory |
Almost every enforcement action against an AI product since 2024 cites three or more of those rows.
The DPIA: The One Document That Decides Whether You Survive
Article 35 says: if processing is “likely to result in a high risk to the rights and freedoms of natural persons,” you must do a Data Protection Impact Assessment before processing starts. AI companions tick every high-risk criterion — large-scale special category data, systematic profiling, new tech, vulnerable users. The question isn’t whether. It’s when — and the right answer is before your first European user signs up.
A defensible DPIA runs 25-60 pages: every data flow described, a necessity test, a risk assessment with mitigations, and a DPO sign-off. Cost from a privacy lawyer plus a technical architect: €8,000–€25,000. From 2 August 2026 the EU AI Act’s remaining provisions kick in, and the max penalty for deploying an AI agent without a documented DPIA climbs from €20 million to roughly €55 million. The DPIA is the cheapest insurance you will ever buy.
Need a DPIA that will survive a Garante inspection — not a templated PDF from a privacy SaaS? Our AI chatbot development team produces DPIAs as a deliverable on every NSFW build. Talk to us.
Article 9: The Trap Most Founders Miss
Your user types: “I’ve been feeling really anxious since my divorce, can you cheer me up?” That one sentence contains mental-health data, marital status, and emotional state — all Article 9 territory. Article 9 prohibits processing special category data by default. The realistic exception for a consumer AI companion is explicit consent (Art. 9(2)(a)) — separate, specific, granular, recorded, withdrawable.
AI companions routinely process this on every active user:
- Sexual orientation and preferences (Art. 9 special category)
- Mental and emotional health signals — loneliness, anxiety, sometimes suicidal ideation
- Biometric voice prints for voice chat
- Selfies / custom avatar inputs (often biometric)
- Behavioural profiling at scale — every message feeds personalisation (Art. 22)
The platforms that get fined bundled all of this into a single ToS checkbox. Replika did exactly that. Character.ai’s Italian deployment did the same. A half-dozen smaller apps the Garante walked off the App Store in 2025 did the same.
The consent UX that actually works
| Step | What you collect | What consent you record |
|---|---|---|
| 1 — age gate | DOB, country | Age confirmation only |
| 2 — account | Email, password | Contract basis (Art. 6(1)(b)) |
| 3 — companion setup | Avatar, name, persona | Contract basis |
| 4 — adult content opt-in | None | Explicit consent: 18+ content (Art. 9 sexual orientation) |
| 5 — emotional companion opt-in | None | Explicit consent: emotional/health processing |
| 6 — training opt-in | None | Explicit consent: chats used for model improvement (default OFF) |
| 7 — voice features | Voice sample | Explicit consent: biometric processing |
Training opt-in as a separate, off-by-default toggle is the single most defensible thing you can ship. It’s the exact thing the Garante called OpenAI out for not doing.
Right to Be Forgotten — When Your Model Won’t Forget
Article 17 makes every AI CTO sweat. The law says: if a user asks, you delete their personal data. Easy when it’s a row in Postgres. Not easy when their conversations live in a vector DB, an embedding, a fine-tune corpus, three observability tools, and a CDN cache. You need a deletion pipeline that hits all of those — and you need to prove it hit all of them:
[Erasure Request] → [User-ID Resolver]
│
┌─────────────────┼────────────────┐
▼ ▼ ▼
Primary DB Vector DB Object storage
│ │ │
▼ ▼ ▼
Backups Training corpus LLM provider
(next rotation) (exclude flag) (zero-retention API)
│
▼
[Erasure Certificate]
→ emailed to user, stored 6 yrs
Two non-obvious points. Trained models don’t “forget” easily — true deletion requires retraining without that data. The Garante is currently lenient if you opted the user out of training and can prove the next cycle excludes them; don’t assume that holds past 2027. And use zero-retention API keys — otherwise your subprocessor is retaining data after you “deleted” it, which is a clean Article 17 violation laid at your feet.
International Transfers and Age Verification — The Two Cheap Wins
Almost every AI companion uses an LLM hosted in the US — a third-country transfer under Schrems II, requiring a Data Privacy Framework signup or SCCs, plus a Transfer Impact Assessment.
| Provider | Mechanism | EU residency | Zero-retention |
|---|---|---|---|
| OpenAI Enterprise | DPF + SCCs | Yes (Enterprise tier) | Yes — default on API |
| Anthropic Enterprise | DPF + SCCs | Yes (Enterprise) | Yes — default on API |
| Google Vertex AI | DPF + SCCs | Multiple EU regions | Configurable |
| Mistral (EU-native) | Not required | EU-native | Yes |
| Self-hosted on AWS Frankfurt | Not required if EU only | Full control | N/A |
The cheapest compliance path is the boring one: host inference in the EU. Hybrid setup (EU for EU users, US for the rest) is now table stakes. It also makes the country-of-registration decision easier — see our jurisdiction guide for NSFW AI companies.
On age verification: every major AI companion enforcement action since 2023 cited weak or absent age checks. A self-declared birthday is not age verification. What works: document-based (Veriff, Onfido, Sumsub, Yoti), credit-card-based 18+ checks (standard for adult payment processors), facial age estimation, or hard geofencing. Cost: €0.40–€1.50 per verified user. The fine for skipping it starts at €5 million.
GDPR + EU AI Act: The Dual Stack After August 2026
From 2 August 2026, you live under the AI Act’s main operational regime on top of GDPR. The two stack — one bad data flow can trigger fines under both.
| Provision | In force | What it means |
|---|---|---|
| Art. 5(1)(a) — manipulative AI | Feb 2025 | No techniques that materially distort behaviour. Engagement-maximising “always-agree” companions sail close to the wind. |
| Art. 5(1)(f) — emotion recognition | Feb 2025 | Banned in workplace/education. Consumer companions face heavy scrutiny. |
| GPAI model obligations | Aug 2025 | Fine-tune your own model → you inherit transparency + copyright + safety docs |
| High-risk system rules | Aug 2026 | Emotion-based recommendations or biometric ID can flip you into high-risk |
| Transparency to users (Art. 50) | Aug 2026 | Must tell users they’re interacting with AI; label AI-generated content |
| Penalties | Aug 2026 | Up to €35M / 7% turnover for prohibited; €15M / 3% for high-risk |
Even if you only ship a consumer app and never train your own model, the AI Act adds disclosure obligations on top of GDPR. Non-negotiable from August 2026.
The Compliance-First Architecture
Reference stack we deploy on every build:
┌────────────────────────────────────────────────┐
│ MOBILE / WEB CLIENT │
│ Granular consent UI · AI Notice banner │
│ Erasure / portability self-serve │
└──────────────────┬─────────────────────────────┘
▼ TLS 1.3
┌────────────────────────────────────────────────┐
│ API GATEWAY (EU region — Frankfurt) │
│ Geo-router → PII tokeniser → audit log │
└──────────────────┬─────────────────────────────┘
▼
┌────────────────────────────────────────────────┐
│ LLM PROVIDER (Zero-retention, EU endpoint) │
│ Mistral / Anthropic EU / OpenAI EU │
└──────────────────┬─────────────────────────────┘
▼
┌────────────────────────────────────────────────┐
│ VECTOR DB + PRIMARY DB (EU region) │
│ Per-user namespace · customer-managed keys │
└────────────────────────────────────────────────┘
Full case studies of this pattern in production: Candy.ai and SugarLab.ai. Privacy architecture is identical — only the personality changes.
What It Actually Costs
Mid-2026 European market rates:
| Item | One-time | Annual |
|---|---|---|
| DPIA (lawyer + tech architect) | €8K–€25K | €3K refresh |
| Privacy notice + ToS + consent flow | €3K–€7K | €1.5K review |
| EU Representative (Art. 27) | — | €1.2K–€3.6K |
| DPO (fractional) | — | €18K–€60K |
| Age verification (per new user) | — | €0.40–€1.50 |
| Zero-retention LLM tier uplift | — | +15–40% over base API |
| EU inference infrastructure | — | ~+10% vs US |
| Pen test + Art. 32 review | €6K–€15K | €5K retest |
| Realistic first-year compliance budget | €20K–€55K | €30K–€80K |
Versus fines of €5M to €15M — compliance runs roughly 0.2-1% of your downside risk. Our mobile app cost calculator bundles privacy engineering into the estimate by default.
The 30-Day GDPR-Ready Build Checklist
If you read one section, read this.
Week 1 — Foundation
- Lawful basis for each processing activity, in writing (Art. 6)
- Data flow map on one page (in → process → store → share → delete)
- EU region for inference + storage; zero-retention LLM tier
- Appoint DPO (or fractional) and EU Representative (Art. 27)
Week 2 — Policies
- Privacy notice naming every category of personal and special category data
- ToS distinguishing contract basis from consent
- Granular consent UI (training opt-in OFF by default)
- Signed DPAs with every subprocessor
- Records of Processing Activities (Art. 30)
Week 3 — Build it in
- Document-based age verification at signup
- Erasure pipeline (fan-out per diagram above)
- Data portability export (Art. 20)
- Breach detection + 72-hour notification
- Audit logs on every Art. 9 read/write
- AI Notice banner (AI Act Art. 50)
Week 4 — DPIA, test, launch
- DPIA complete with legal + tech sign-off
- Privacy-focused penetration test
- Tabletop incident response drill
- Support team trained on subject access requests (30-day SLA)
- Then open EU signups
If you can’t tick those boxes, geofence the EU until you can. A fine costs more than four weeks of waiting.
Verdict
Compliance is now a product feature. Candy.ai, SugarLab.ai, the better-run DreamGF clones — their privacy UX is visibly tighter than competitors’. Users notice. App stores notice. Regulators definitely notice.
Doing it after launch is 5–10× more expensive than doing it before. Retrofitting consent into a live product means migrating records, re-collecting consent, and explaining the change without tanking conversion.
The fine ceiling rises on 2 August 2026. Planning a Q3 launch? Your DPIA needs to be done in Q2. That’s now.
Closing CTA
Two ways forward.
Pre-launch or under 10K users: Free 30-minute review with our NSFW chatbot development team — we review your data flows, flag the GDPR red lines, give you a written punch list. No sales theatre.
Over 10K users in the EU: You need a DPIA, defensible consent architecture, and a working erasure pipeline, in that order. Reach out via tripleminds.co/contact-us — mention “GDPR audit,” one business day turnaround.
White-label Candy.ai and DreamGF builds with the privacy stack baked in: Candy AI clone and DreamGF clone.
FAQs
Does GDPR apply to my AI companion startup if I’m based in the US?
Yes, if you offer the service to EU users or monitor their behaviour. Article 3(2) is extra-territorial. You must also appoint an EU Representative under Article 27 — the lack of one was a contributing factor in several recent enforcement actions.
When does the EU AI Act start applying on top of GDPR for AI companion apps?
The prohibitions and AI literacy duties applied from 2 February 2025. GPAI model obligations applied from 2 August 2025. Most remaining provisions — including transparency to users (Art. 50), high-risk system rules, and full penalty regime — apply from 2 August 2026. From that date a single bad data flow can trigger fines under both GDPR and the AI Act, stacked.
Do I have to do a DPIA before launching?
For an AI companion processing special category data at scale, yes — Article 35 makes it mandatory. From 2 August 2026, the maximum fine for missing one climbs to roughly €55 million under the stacked GDPR + AI Act regime.
Can I use OpenAI or Anthropic and still be GDPR-compliant?
Yes, but only on their enterprise / zero-retention tiers, with a signed Data Processing Agreement, EU data residency where offered, and a Transfer Impact Assessment on file. Consumer tiers are not suitable for processing EU user conversations.
How do I delete a user’s data from my AI model?
You can’t fully — not from a trained model. The compliance path: delete from all live systems, backups, caches and observability; flag the user as excluded from future training runs; issue an erasure certificate. Use zero-retention API tiers so your LLM provider isn’t holding logs you can’t reach.
Is a self-declared “I am 18+” checkbox enough for age verification?
No. Every major AI companion enforcement action since 2023 has cited it as inadequate. For 18+ content you need document-based, biometric, or credit-card-based age assurance.
Build the privacy stack like the regulator is your first user. Because eventually, they will be.
Got a project in mind? Let’s build it together.
We work with founders and product teams across consulting, development, and growth marketing. Tell us what you’re building and we’ll show you how we’d ship it.