Comprehensive traceability of operations and versioning of business entities

In search of a simple and effective solution

It is not uncommon to come across the need to keep a record of potentially complicated business operations, implicating both the versioning of entities and maintaining an audit log of who did what when. This is especially important in some sensitive areas such as finance or healthcare, where a complete picture of data accesses and modifications is often required.

At some point the CTO of a company I used to work for mentioned ES (Event Sourcing) and CQRS (Command Query Responsibility Segregation) as an approach that might tackle these problems as well as addressing the need to send multiple notifications to external systems as a result of some operations (e.g. sending mail, sms, slack messages).

I resisted the idea at first: I wasn’t too keen on building a seemingly complicated solution to something that could be solved by using a good old audit log and some flavor of versioning. And I was definitely not a fan of representing business entities by what happened to them instead of their current state.

But I was intrigued and I started mentally revisiting past projects, especially those which had proved particularly hard to implement on a pure CRUD approach and how they might have looked like with a CQRS/ES approach.

Then I started reading about different implementations out there like this one and I was increasingly convinced you can have it all: up-to-date consolidated representations of your entities and unmatched operation traceability, along with some additional perks.

The central idea of CQRS is that the write-side and the read-side of your application are separate. In this architecture the write-side is focused on the business operations you can perform while the read-side is focused on the queries you have to respond to. An ES framework, on the other hand, states that you model changes to the state of your system as a series of events applied to different entities or aggregates.

In this setting you have potentially two different models for your data:

• Domain model: which is an implicit data model used in the write-side of the system and which can be hydrated for a particular aggregate by sequentially applying reducers to the corresponding event stream. It is primarily used to validate business rules.
• Query model: an explicit data model used in the read-side of the system and which is optimized to satisfy queries and is kept up-to-date by event handlers reacting to the events produced by the commands on the write-side.

A domain model for a bank account might look like this stream:

// db.events.find({ aggregateId: 1 }).sort({ seq: 1 }).toArray()
[
  {aggregateId: 1, event: {type: 'ACCOUNT:CREATED'}, seq: 1},
  {aggregateId: 1, event: {type: 'ACCOUNT:DEPOSITED', amount: 10}, seq: 2},
  {aggregateId: 1, event: {type: 'ACCOUNT:WITHDRAWN', amount: 5}, seq: 3}
]

The hydrated version of the domain model for the represented aggregate could resemble the following:

{
  accountId: 1,
  balance: 5
}

While a query model optimized for showing deposits and withdrawals might look like this:

// db.deposits.find().toArray()
[{amount: 10, accountId: 1}];
// db.withdrawals.find().toArray()
[{amount: 5, accountId: 1}];

These models could simply be the explicit and implicit versions of the same underlying data model, or they could be very different. In either case, the event streams used to hydrate the domain model provide a complete versioning of entities and an audit log of all write operations in your system.

At this point it is probably useful to pin down some concepts, since we have used many without bothering to define them.

• Entities: represent traditional business concepts such as a customer, a product, etc.
• Aggregates: an aggregate could either be an entity or a cluster of closely interconnected entities which are exposed to the outside world as one, for the purposes of data write operations. External systems are only allowed to hold references to the aggregate, all other entities can only be referred to as aspects of the aggregate. Usually an aggregate is represented by its root entity and the event store can conceptually be seen as a collection of event streams, one per aggregate.
• Commands: represent the intent to perform a particular business operation on aggregates. They are responsible for enforcing business rules and should throw an exception if these are not respected. The result of successfully executing a command is a list of events.

const depositMoney = (account, amount) => ({
  execute: () => {
    if (account == null) throw new Error('Account not found');
    if (account.balance < amount) throw new ('Not enough funds');
    return [moneyDeposited(account.id, amount)];
});

• Events: represent mutations to a single business entity and have all the information necessary to unambiguously apply it. For the purposes of traceability, events issued by the same command could share the same commandId, and to facilitate rollbacks (especially in systems without native support).

const moneyDeposited = (accountId, amount) => ({
    type: 'ACCOUNT:DEPOSITED',
    accountId,
    amount
});

• Reducers: consist of pure functions that, given an entity and an event for that entity type, produce a mutated entity. They provide the means to hydrate the implicit domain model.

const accountReducer = (account={balance: 0}, event) => {
  switch(event.type){
    case 'ACCOUNT:CREATED':
      return { ...account, accountId: event.accountId};
    case 'ACCOUNT:DEPOSITED':
      return { ...account, balance: account.balance + event.amount };
    case 'ACCOUNT:WITHDRAWN':
      return { ...account, balance: account.balance - event.amount };
    default:
      return account;
}

With some discipline, it is relatively easy to design an event-driven system with this approach. Such a system is relatively future-proof because you can always discard query models that are no longer relevant for your needs and produce new one from the existing stream of events. The interpretation of events themselves in unlikely to change, as they represent actual operations on an entity. Even if future business rules disallow certain operations to occur, they still would have happened in the past and the event stream will reflect that.