Enterprise research

Enterprise research (hereinafter, the Research) is the second batch of DREPD; the Research includes all activities needed to process the data discovered in enterprise discovery in order to provide enterprise envisioning with detailed requirements for the future outcome from the DREPD. The Research "contributes" the second letter R to the DREPD abbreviation.

Document Your Real Architecture

I’m writing this post to share with you a little thought that popped into my head and has been “growing” since then as something very important. I was thinking about starting a new pet project that I could ultimately transform into a business – a kind of event-driven PaaS solution. As with any kind of “wonderful” business idea, I was thinking about things that could distinguish my platform from all the other available out there on the market. One of the things that popped into my head was an automatically generated visualization of dependencies and message flow in the system.

Besides the fact that I believe that it’s a great idea for a platform’s feature, I actually believe that it’s very, very bad that I even thought about it as a distinguishing feature. I mean, if we don’t document and visualize the system’s that we’re building, then on what ground are we making strategic decisions about them? Gut feelings? Democracy? Personal preferences?

Old, legacy monolithic systems are often blamed for being so complicated that we cannot effectively assess what is the impact of the changes that we’re doing to them. We change a small, “innocent” method in one part of the system, and suddenly a few others stop working – these kinds of things. Yet, there are at least some ways to figure the shit out. Modern IDEs are pretty good at finding obvious dependencies with a single shortcut. Can you do the same in a distributed system?

I mean, if I asked you, how many dependencies the microservice X has, how long would it take you to answer? If I asked, whichmicroservices depend on event Y being emitted and what are the consequences of doing so, would you be able to find it out without sending an email to all developers in the company or other hilarious practices like that?

Things get even funnier when you observe a company that’s in the middle of a transition from an old, legacy system to microservices. You could come up with tons of questions like which applications read table Z in the legacy databasewithout any hope of a definitive answer.

Now, I’m not saying all that to share around some anti-microservices propaganda. I actually see a lot of benefits in this architectural style and would consider it for any kind of new enterprise system. What I’m trying to say is that we should invest (a lot) in learning about and understanding the systems that we’re building. There are already some ways of doing that e.g. Zipkin, but unfortunately, they’re not as advanced and not as popular as they could be.

I believe that the specific problems described above are actually singled out instances of a more general lack that we have in the industry – documenting and/or visualizing the real architecture. I will throw some more questions at you. Do you really understand the overarching architecture of the system that you’re now building? What does that even mean? Let’s try another way. How many of the architecture diagrams that you’ve seen represented the actual, current state of the system? I mean a kind of truly honest, as-is diagram with all the flaws and pain points of the system. For some reason, I guess the answer is: not many (if not outright 0).

And yet we all claim to be agile, which means that at most a few weeks into the project and there’s a running application somewhere in there. A growing system that will most likely be maintained for many years by totally different people than work at the company today. How are they supposed to learn and reason about the system? Worse, to repeat my question from the beginning of this post, how are they supposed to make decisions about the system’s future?

How about you? Do you really believe that you can make decisions based on the pieces of memory that your brain brings at a given moment? Is that what a unicorn-craftsman-engineer does? You’re not that smart. It’s not about you, nobody is. All of us, at all times, are bound by the ~7 elements capacity of our working memory and the total area of whiteboards/papers we use to make notes.

Therefore, whether the microservices trend stays or goes, whether we go back to building monoliths or start splitting our systems into “serverless” functions, my piece of advice is as follows:

Document your architecture as-is. Employ tools and invest heavily into visualizing the current architecture of the system, be it code dependencies, event flow or legacy database tables’ usage. Make informed decisions based on as much data as possible, not on the relatively few, partly random things that your brain puts into your working memory at a given moment.

Related lectures

• Controlling Quarter.