I am a software developer with over 10 years of experience. Throughout my career path I have worked for companies that have been on the market for more than 15 years. The projects I have participated in have been mostly related to monolithic software and my expertise is built on both good and bad practices from years of development and maintenance. Nowadays every developer has heard or read about microservices. I have been waiting for the day when I would get the chance to start a new project from scratch with microservices and apply all the good practices that I have been reading about. When I finally got this opportunity, I thought I was prepared enough for such a challenge and I was almost right. It has been a year already and this piece of software is now working well, it is fast and clients like it. Today when I look back at the time spent building the system, I can see some errors of mine. I believe it is worth sharing with you my experience and where things could have been done in a better way, so you do not make the same mistakes when building a microservice architecture.
1. Do not use only HTTP/S
HTTP is a known protocol for everyone, even for my father (he plays games on Facebook!). Shortly, it is a synchronous protocol — you send a request and wait for a response (different from asynchronous programming). For browsers it is the most used and easy to deal with, so it should be used with browser-based client-server applications. Yet for internal communication between services or components within a system sometimes there are much better solutions. There is AMQP, for example, that is asynchronous. It allows you to implement and publish/subscribe when you need communication between microservices, or if you have to notify several microservices on a change. With HTTP you have to send a request for each of them and wait for a response. With AMQP you just fire and forget. Each subscriber will process the published message if and when it decides to.
2. Do not create a normalized database for the entire solution
Let’s image we have a simple web site that has a User’s module (Name, Identification Number, Address, Birth Date), a Leave module (User name, Date from/to) and a Delivery module (User address, Item to deliver). If you make the database normalized for the whole solution, when you display information in the Leave or Delivery module, every time you must query the Users microservice to get Name or Address. That creates a hard dependency and a lot of requests to the users microservice (in this case http) no matter which module the end user tries to access. Don’t be afraid to duplicate some of the data that you need (Name in Leave service and Address in Delivery service). In this way you will be able to display all the needed information with a call to just 1 microservice. Therefore, the advice here is to think about a microservice as a separate unit that can operate alone without hard dependency to other microservice(s). To achieve that the database in that particular microservice should store and provide all the data needed in that microservice, no matter that data will be duplicated. When you have to synchronize data, you can use AMQP protocol. Just remember — only 1 of microservices has a responsibility of modifying duplicated data and this is the microservice that holds the entire entity with all properties (in this case Users microservice). For example, when a user changes his name, a POST to user microservice is made. When data is updated in database, this service will publish data through AMQP with data changes. All subscribers (in this case Leave microservice) have the responsibility to take the published data and update it in their database. How many times a user changes his name or address?
3. Do not make it monolithic with microservices
This is the hardest to avoid when you have a lot of experience with monoliths. One of the clear signals for a monolithic architecture in microservices is when you have a lot of relationships between them. Each microservice should be independent and avoid calls to other microservices (exceptions are synchronizations with AMQP, background workers). In some real-life cases there is no way not to call other microservice, but we should avoid it where possible. It’s good to always make sure your microservice queries only 1 database. Some additional piece of advice from me — spend enough time to determine bounded context. If you are not familiar with bounded context — read this. Then follow my advice mentioned in 2) and denormalize your database where necessary. Nothing more concrete that can be said. The rest depends on the problems that you solve.
4. Do not create a separate solution for each microservice from the beginning
In a big monolithic solution, there are a lot of projects depending on the size of the system. This makes the entire solution make the whole process much slower and wastes the time of the developer. To avoid this, I decided that each microservice will be in a separate solution and in a separate repository. Sounds good, right? Well, it was wrong! What I missed is that we were building the system from scratch and we did not know all the features we needed to implement beforehand. With new features coming in down the road we had to extend almost each service. And what about common configuration? There were times when I had 6 or 7 Visual Studios open and trying to navigate through all of them was even a bigger waste of time than in a monolithic solution. Lessons learned, I believe it is good to start with a single solution and each service to be separated somehow in this solution. When you think some service has a good coverage of functionality based on the bounded context and it is not touched for a while, it is time to move it to a separate solution. In this way you solve two potential issues and now you can change everything easily within 1 Visual Studio solution and without worrying about massive solutions bringing down your entire workstation down.
5. Do not use the same design pattern in each microservice
Honestly, I am not familiar with all design patterns and throughout my career I have been mostly using the N-tier pattern, as it is a good solution to the problems I mostly have to deal with. For this new project with microservices, I decided to use it as well. I tried to keep it stupid,simple (KISS approach) and decided to make them all identical enough, so every developer is familiar with everything. It was a mistake! Imagine you have simple CRUD services, so why do you need N tiers? You will just waste your time mapping objects from the presentation to the service layer and then to the repository layer without any advantage (there is no business logic — it is simple CRUD). My advice for you would be to get familiar with design patterns, identify what it is that your service should do and pick the correct pattern depending on the problem you solve.
As mentioned above, in the beginning of the project I thought I am prepared based on all the materials I have read about microservices and my experience in general. However, once you start with the real work, you have to have in mind that there might be other more efficient ways for the execution. From my experience, including also the 5 mistakes I made when designing distributed systems using microservices, I would always recommend developers to keep learning, experimenting, and never stop searching for the best practical solutions. Sometimes, the tried-and-true approaches work best, like for the normalized relational databases. At other times, the same approaches create inefficient system designs, e.g., tight service coupling due to the same database denormalization principle.
As software engineers and system architects, we at Resolute Software strive to pick the right tool for the right job, by making sure we understand the system requirements well and by maintaining a rich set of tools (frameworks, languages, design patterns).
Get in touch with us, if you need a pair of expert eyes on your system design.
Author: Chavdar Stoenchev, Engineering Team Lead @ Resolute Software