Unfortunately this is just another tale of We tried to do microservices without actually putting in the effort. There is something to be said for monoliths, and that is they are quick to couple everything together and prevent your ability to scale. If your project is going to live for 2 weeks, go ahead. But if you plan on making a service that should live longer, the ROI on spending 3 seconds on planning the service is positive. And so if your leadership doesn’t care about scaling and staying ahead of competition, then I agree let’s ignore using microservices.
In this case it actually seems like it wasn’t just leadership pushing this, but every team started to go through the motions. There are so many questions that were raised by how things were being done. Why weren’t the architects on your team able to come up with ways to split the functionality? Why was every service directly coupled to the third party tool? Why were there two teams working on the same code base? Why did you try to do a Big Bang approach instead of building out the parts you mentioned?
Adopting microservices isn’t free
No one is saying that they are, but you understand why the change in paradigm has to happen and then you make it happen. However, instead you listed out a number of prerequisites for doing microservices.
We would need to revisit many concerns that we had previously addressed in our monolith. For example, we would need to address or revisit: logging, monitoring, exception handling, fault tolerance, fallbacks, microservice to microservice communication, message formats, containerization, service discovery, backups, telemetry, alerts, tracing, build pipelines, release pipelines, tooling, sharing infrastructure code, documentation, scaling, timezone support, staged rollouts, API versioning, network latency, health checks, load balancing, CDC testing, fault tolerance, debugging and developing multiple microservices in our local development environment.
Ummm, every single one of those is not unique to your paradigm and still a problem in a monolith. As usual, and in this circumstance, microservices created awareness of the architectural problems present in your monolith. These need to fixed, going to microservices requires fixing them, but not going doesn’t absolve you of those issues. Additionally some of these are just plain wrong:
- Sharing Infrastructure code: Code should never be shared, and is never required to be shared, you can use shared libraries if you really want, but open source is a great option.
- Staged rollouts: if you are trying to stage your rollouts then you are just trying to do a distributed monolith. Each service should have its own plan and work independently, never is something staged.
- Developing multiple microservices…: You don’t do this with microserivces. When developing a service locally, with microservices, you would develop and test it against the production version of other services. Doing development in multiple services at the same time is an anti-pattern from monolith development.
And then there is just a straight contradiction, not sure why so many people working with monoliths get stuck here:
We were using monolith like a loaded term. As if saying “monolith” implies something terrible, and “microservices” implies something good.
We disliked working in the monolith. It was slow to work in, compile and run tests for, the architecture was varied to the point of unknowable, random stuff keep showing up in the build steps.
Exactly. I’ve also spent some time to help create answers to some the important monolith to microservice questions:
Was restructuring the teams to be dedicated to separate business concerns practical?
Yes and it is required, teams need to be cross-functional and be able to clearly focus on the services that make sense. This usually requires unique values, vision, and missions for each team that everyone can get behind. Team members rally behind their shared purpose and know how to innovate in that space. Attempting to have a collective mindset is incredible slow and quickly organizational structure becomes a bottleneck in feature delivery.
Could we cleanly divide upcoming feature work between our domains and microservices?
If you can’t then you haven’t figured out what your domains really should be. 95% of every feature request, architectural change, and business innovation must be resolvable by a single team. If more than 5% of features require more than one team to make a change specifically for those features, then the domains, technology, and concerns aren’t sufficiently separated.
Would there be enough work for all teams, or would a team be left without work?
If there is less work because you are doing microservices, then you should celebrate. As one of my favorite mentors always says “I would love to have that problem.” Think of all the extra capacity you would have available and what else you could be doing.
Would a team get slammed with mountains of high priority work that they couldn’t share out?
It can happen, but believing that having 2 teams working on that high priority work creates a huge amount of communication overhead. In some cases that means we didn’t split the domains well enough. Maybe you need to take your 2 teams and make 3 teams with three domains.
Would the same issues that made it hard for us to divide our monolith also prevent our management tier from dividing up incoming work? *Was their appetite for this sort of transformation?
What do you mean Management tier and why would that exist?! Work should be researched, designed, developed, and delivered by a single team with the following roles:
- Product Manager
- Tech Lead
- Engineers
These roles can be three different people, or one person putting on those hats. If you are referencing management, it seems that there could be a systemic problem with the organizational structure. This needs to be fixed first.
Further, this structure would make our domain models clearer, allowing us to evaluate candidates for any future microservices more easily. If something did turn out to be a suitable candidate, that project could then “fall out” of our monolith into a microservice, without having to be untangled.
It would be nice for this to be the case, but very rarely can code from a monolith ever fall out, instead you’ve kept a microservice in the monolith, and it looks and acts like a microservice, except that you don’t get any of the benefits — CI/CD speed, development agility, choosing the best technology for the job, refactoring the whole service, etc… And since it would only take work to actually extract it out, you likely will have to wait for their to be extreme pain and desire to change before something gives.
Instead, if we had started by restructuring our teams around dedicated business concerns, then gotten the infrastructure ready, we set the stage for microservices to naturally emerge.
I’m a bit confused to be honest, since after coming to some great intermediary conclusions, the next course of action would be help refine the monolith to microservice migration. Not to:
After evaluating it, we found that microservices weren’t a fit for us, and required significant compromises.