Fragmentation is a semantic problem wearing a technical costume
A retail group runs an order system, a warehouse system, a pricing service and a returns tool. In both cases the executive question is the same: why can nobody answer a question that requires holding two systems at once, and why does the answer change depending on who is asked? Each system is internally reasonable. The obstacle is that they describe the business in four private vocabularies with four sets of identifiers, and nothing above them holds the joint.
The cost appears in ordinary places: a stock position that differs between two reports, a customer who is one record in one system and three in another, a returns decision made without the order history. Each is cheap individually and expensive in aggregate, and none is visible on a system diagram, because the connections are all there. What is missing is the meaning.
Five layers have to be connected before context exists
The word context covers several distinct pieces of engineering, and treating them as one is why programmes stall after the integration work is finished.
- The system layer, which is the easy part: reachable, authenticated and reliably readable. Most enterprises have this, and it is the part they spend the most budget on.
- The record layer, which is where the same business object becomes several rows in several places. Reconciliation policy belongs here.
- The identifier layer, which is where 'is this the same thing?' is decided and where most silent errors originate.
- The relationship layer, which states how a customer relates to a site, a site to a contract, a contract to a price, with dates on the relationship rather than on the record.
- The definition layer, which fixes the meaning of the terms and names who owns them. Nothing above it is trustworthy if this one is missing.
The order matters, and the common failure is to build the first two, declare success, and then discover that the reason the joined data disagrees with the business is layer four or five.
Identities first: the record that has to resolve
Choose a single real-world thing and follow it end to end. For a distribution business that is often the shipment: created by an order, loaded onto a vehicle, delivered, returned, invoiced. Follow one and you will find it carries four references — an order line number, a handling unit identifier, a proof-of-delivery number and an invoice line — each issued by a different system, related in practice because the warehouse team matches them by hand.
That manual matching is the specification. Write it down as an explicit relationship with the conditions that govern it, including the cases where it fails: a split shipment, a consolidated pallet, a return without a matching delivery, a re-delivery. Each failure mode should name what the business does, not merely that it is an exception. This is the point at which business ontology explained becomes concrete rather than abstract.
Model the relationships with dates, not with attributes
The most consequential modelling decision in this layer is how relationships are expressed. Writing 'customer region' as a field on the customer record forces every regional change to rewrite history. Writing it as a relationship with a start and an end date preserves the past and lets a question like 'which region was this account in when that order was placed' be answered exactly.
The same applies to contracts, prices, ownership, allocation and headcount. It is the difference between a business record that supports an audit and one that supports only the current view. Where the history is not captured anywhere today, that is a finding in its own right and should be stated to the executive rather than quietly reconstructed.
| Layer | Question it answers | Evidence that it is done |
|---|---|---|
| Identifier | Is this record the same thing as that record? | A written matching rule, with its exception cases listed. |
| Relationship | How is this thing connected to that one, and since when? | Start and end dates on every business relationship. |
| Definition | What does this term mean here, and who owns it? | A versioned term with a named owner and a review date. |
| Permission | Who may see and change this, and under which policy? | An access rule per entity, expressed in business terms. |
Publish it as a contract, not as a schema
The output has to be consumable by things the business did not build: an agent reading a shipment status, a report the finance team refreshes, an application another team maintains. So the definitions need a home with versioning and a change notice, and consumers need to depend on that contract rather than on a table they happened to find.
A practical sequence is to publish the definition layer first, because it is cheap and it forces the ownership questions. Then identifiers and relationships for the two entities with the most cross-system friction. Then the permission model, which is rarely a data problem at all and usually reveals that a policy has never been written. Each layer should be small enough for a business owner to review in one sitting.
- Choose one real-world thing and trace it through every system that records it, listing each identifier it carries along the way.
- Ask the people doing that work how they match the identifiers by hand, and write their procedure down as an explicit rule.
- Record the failure cases — splits, consolidations, returns without a match — and what the business does in each.
- Convert every business relationship into a dated relationship rather than an attribute on the record.
- Publish the terms with a version, an owner and a change notice before publishing any of the structure that depends on them.
- Constrain direct access to the underlying tables so the shared definitions cannot be quietly bypassed.
- Review the whole set with the business owners of the two most contested entities before anything is built on it.
What to measure once the context exists
Use measures that would have been unobtainable before, because that is the clearest evidence the layer is real: the proportion of a given entity type that resolves to exactly one record across systems; the share of business questions answerable by a join rather than by an email; the number of separate definitions in circulation for the terms you published; and the elapsed time to reconstruct the state of a transaction on a given date a quarter ago.
The last of these is the one executives understand immediately, because it corresponds to a question they are always asking and cannot currently answer. The discipline that keeps this work maintainable is described in making enterprise AI maintainable after the pilot, and the sequence above is the foundation described in our approach to enterprise data engineering.