An email on Open Architecture

I haven’t heard the term “open [architecture]” yet, but it seems pretty straightforward. It reminds me of the open/closed principle from SOLID, but applied at the platform level rather than to individual classes. There would be two main components to this.

First, and most obviously, you would want to set up an interface using highly compatible technology. This is what most of the articles linked in the original email are discussing, and so I won’t spend much time on it. I will say that this exists at scales below the macro-level. Even at the level of individual values, you would want to model your data in a simple, standard way, like using the ISO 8601 format for all date strings, rather than writing your own.

This component of being open is the easiest to achieve, but it is also, surprisingly, the least important, since it can almost always be fixed with a bridge if need be. The term comes from object-oriented programming, but it works at the platform level as well. If I want to convert an old XML-based system to use JSON, I can do that externally without having to modify the original system.

The second component of open design is nearly impossible to change without sweeping modifications, and that is the set of information that is provided for any given task and the timing with which it is provided. If I build a system that stores and provides client phone numbers as an array of strings, and I discover later that I also want to remember which phone number is their mobile/home/etc, then I will usually be forced to modify this storage system. There is no external system that can fix this other than a new storage system. Similarly, if a system is built that requests these phone numbers and presumes that I will be able to provide them synchronously, then any change which will cause me to only be able to provide them asynchronously will require changes to the requesting system. A bridge can’t reverse time to provide that information synchronously.

Again, what information is communicated and when it is communicated is the most important component to get right, and it is often up to the system designer to ensure that their design is robust to changes. Technologies can help, like with GraphQL where the requesting system describes the precise data that it wants when it wants it, but this still assumes a pull-based system, and so it is often not an absolute fix (though it is better than REST for flexibility).

The most robust system as far as what information is provided is a pull-based system where anything can be accessed. This is the guiding principle behind GraphQL. If system A wants to request information about entity E, it can do so using an ID for E. Whichever system handles that request may then look up any information it needs to using the ID, rather than being stuck with the information that system A thought to provide.

The most robust system as far as when information is provided uses message-passing communication with a task queue. This is the idea behind databases like RabbitMQ. Whenever a system wants to make a request it places a task in the queue that the rest of the platform can handle in its own time.

As far as advantages to open platforms, the main one is reliability, especially for security. In general, the longer a system goes without modification the more reliable it becomes. Any manual testing—including the ultimate manual test, which happens in production—remains valid and will only ever cover more of the system. You also tend to only gain automated tests as your old tests still work while any bug and security fixes bring new regression tests. For security, this means that you can perform extensive security audits without fear of invalidating them if the needs of the platform change.

As far as references, I tend to abstract concepts far beyond their ostensible purpose, so the provenance of my knowledge can get a bit weird. For instance, I initially learned about message-passing systems from the blackboard pattern, which was in Pragmatic Programmer. You can also find it in Firefoxes Quantum engine (see work stealing). My understanding of information flexibility also comes from my own observations, but the idea that entity IDs are the most robust approach comes from the ECS pattern in video games.