solutions / analytics-pipelines
Analytics pipelines
Warehouses accumulate identifiers nobody asked for and nobody can delete.
The problem
Event pipelines carry email addresses because someone needed a join key in 2021. Free-text fields carry whatever a user typed. The result is a warehouse where every analyst has access to identifying data they do not need, and a deletion request that takes a quarter to service.
What Divelai does
Sanitize on ingest. Consistent tokenisation preserves joins, cohorts, and funnels — an analyst can still count distinct users and follow one through a funnel, because the token is stable. Re-identification is a separate, logged, permissioned operation rather than a default property of the table.
What this does not solve
This protects data flowing through the pipeline after you deploy it. Historical tables already full of identifiers need a backfill pass, and Divelai does not decide your retention policy for you.
# Ingest-time sanitization, stable join keys. source: events.raw sink: events.clean scope: global # same user, same token transform: user.email: tokenise # [EMAIL_7f2a] joins fine user.name: tokenise event.notes: detect+tokenise user.ip: mask # 10.42.8.0