Data warehousing & architecture
MDM — Master Data Management
I create a single trustworthy source of reference data (customers, products, counterparties) — free of duplicates and discrepancies between CRM, accounting and other systems.
What is Master Data Management
Master Data Management (MDM) creates a single trustworthy representation of a business's key entities — customers, products, counterparties — when the same entity is recorded differently in CRM, ERP, accounting and other systems. The result is a golden record: a reference profile of each entity, formed through matching, deduplication and consolidation rules across data from different sources.
What's included
Audit and profiling
I analyse the source systems, assess data quality, and surface duplicates, conflicts and gaps. This is the foundation for the model and the matching rules.
Master data model
I define the domains (customers, products, suppliers, counterparties), the structure of the golden record, and the relationship hierarchies.
Quality and matching rules
I set up normalisation, deduplication, record matching and merging algorithms, and consolidation (survivorship) rules.
Data governance
I introduce data stewards and data owners, regulations, policies and approval processes for changes to master data.
Integration and synchronisation
I connect source and consuming systems via ETL/ELT, APIs and a data bus, and set up regular synchronisation.
Migration, launch and support
I run the initial load with cleansing, testing and user training, prepare documentation, and support the solution after launch.
How it works
- 01
Strategy and assessment
We pin down the business goals, select the priority domains (customers, products, counterparties), and I assess data maturity and justify the project.
- 02
Audit and profiling
I survey the sources, profile the data, and assess current quality and the scale of duplication. I capture problem areas and conflicts.
- 03
Design
I design the data model and architecture, choose the MDM style (registry / consolidation / coexistence / centralized), and define the quality and matching rules and the governance model.
- 04
Integration and loading
I connect sources via Airbyte and load raw data into the staging layer. Where needed, I normalise it with a separate transformation layer (dbt/Python) before matching.
- 05
Matching, deduplication, golden records
I set up matching and deduplication in Zingg (ML with active learning), form golden records in PostgreSQL and an identity graph in Neo4j, and catalogue the storage in OpenMetadata.
- 06
Testing and pilot
I test the solution on a limited volume, verify the correctness of the golden records, and connect the steward interface with feedback into Zingg.
- 07
Launch and operations
I move it into production with user training. After that — quality monitoring and governance as an ongoing process, not a one-off project.
An example architecture of a custom MDM solution
The diagram shows a vertical pipeline from the source systems to the master store: loading, matching and consolidation into a golden record. MDM breaks down into functional layers — loading, matching, storage, catalogue and stewardship — and each has a proven open-source tool. The stack is deployed in the client's infrastructure; solid arrows are the data flow, dashed lines are metadata and steward feedback.
Source systems
CRM, ERP, online store, support desk and other systems where the same customers, products and counterparties are stored separately, with duplicates and discrepancies. This is the entry point of the pipeline.
Airbyte
Extracts and loads (EL) raw data from the sources into the staging layer via ready-made and custom connectors. It's responsible for data delivery, without performing matching; normalisation and alignment to a common schema happen in a separate transformation layer (dbt/Python).
Zingg
The entity-resolution core: blocking, matching and clustering of records based on ML with active learning. Zingg itself selects the informative borderline pairs for labelling, and from a small number of examples it merges records into single entities and assigns a shared identifier (ZINGG_ID).
PostgreSQL — golden records
The operational source of truth: it holds the golden records and the crosswalk (which source record corresponds to which golden ID) and serves master data to consuming systems via an API. Transactionality (ACID) and integrity out of the box; versioning and auditing at the schema level (history tables).
Neo4j — identity graph
The identity graph: source records and the master entity as nodes, matches and relationships as edges. It stores households and hierarchies of the form "company → subsidiaries → contacts" — the kind that's awkward and expensive to traverse in SQL — and gives a complete 360° profile of an entity.
OpenMetadata
It doesn't store the master data itself but catalogues both databases: lineage from source to golden record, a business glossary, owners, PII classification and quality metrics. Context, traceability and governance around the MDM core.
Steward interface
Reviewing borderline matches, manually merging/splitting clusters and overriding survivorship rules. There's almost no ready-made open-source here — the layer is built custom (a BPM engine such as Camunda plus a bespoke frontend); the steward's decisions flow back into Zingg and further train the matching (human-in-the-loop).
Projects
Analytics platform for the digital agency YA AGENCY
Analytical data warehouse and dashboards for Ash
End-to-end analytics and infrastructure migration for WiseAdvice-IT
Analytical data warehouse for a chain of bars and restaurants
Analytical data warehouse for the Neolombard pawnshop chain
Unified data warehouse and dashboards for the delivery service Edostav
Data warehouse for the financial analytics project VISUALSECTORS
Data migration to ClickHouse and dashboard rebuild for MnogoSna
Tech stack
Airbyte 
dbt 
Zingg 
Splink 
Neo4j 
OpenMetadata Clients
Testimonials
When the same customer or product is recorded differently in CRM, accounting, ERP and other systems, everything suffers — from the accuracy of reporting to the quality of customer communications. I create a single trustworthy source of reference data: I gather records from all systems, match and deduplicate them, form a golden record and an identity graph, while a catalogue and a steward interface ensure quality and control. The solution is built on an open-source stack and deployed in your infrastructure — free of duplicates, discrepancies and dependence on expensive licences.
Shall we discuss your task?
FAQ
How does MDM differ from an ordinary warehouse or CRM? +
A warehouse collects data but doesn't resolve contradictions: one customer remains in it as several different records from different systems. A CRM stores the data of a single system and doesn't know that the same customer already exists in the ERP or billing. MDM solves exactly this: it matches records across systems, merges duplicates and forms a single golden record that both the warehouse and the CRM rely on.
Is a cloud MDM product mandatory, or can it be done on open-source? +
No, an expensive cloud product isn't mandatory. I assemble a custom MDM on a proven open-source stack — Airbyte, dbt, Zingg, PostgreSQL, Neo4j, OpenMetadata — and deploy it in your infrastructure. That means full control over the data, no per-user licence fees, and flexible tuning to your industry specifics.
Why two databases in the architecture — PostgreSQL and Neo4j? +
PostgreSQL is the operational source of truth: it holds the golden records and the crosswalk (which source record corresponds to which golden ID) and serves master data to consuming systems via an API. Neo4j stores the identity graph — the relationships between records, households and the hierarchies "company → subsidiaries → contacts" that are awkward and expensive to traverse in SQL. Together they provide both fast access to the reference data and flexible navigation of relationships.
How does matching work with imprecise data? +
Matching can be deterministic — by exact keys and rules — and probabilistic (fuzzy), which finds matches even with typos, different spellings and incomplete fields. Zingg uses ML with active learning: the system itself selects borderline pairs and offers them for "match / no match" labelling, and from a small number of labelled examples the model learns to match records on your specific data. Borderline cases go to a steward for manual resolution rather than being merged blindly.
How long does implementation take? +
A basic solution for a single domain — customers, for example — with integration of several sources and the first golden records usually takes from a few weeks. Exact timelines depend on the number of sources, the quality and volume of the data, and the complexity of the matching rules. During discovery I scope the work specifically for your case.
What about security and personal data (PII)? +
The entire solution is deployed in your infrastructure — the data never leaves its perimeter. OpenMetadata automatically classifies sensitive fields (email, phone, documents) and stores lineage — you can see where each attribute came from and where master data flows, which matters for auditing and compliance (GDPR). Access is segmented by role.
How is data quality maintained after launch? +
Quality rests on three things. OpenMetadata runs quality checks and profiling on a schedule, flagging degradation at the input. The steward, through their interface, resolves borderline matches and corrects the automation's errors, and their decisions flow back into Zingg and further train the matching. On top of that, I support the solution and onboard new sources and domains as the business grows.