MagicalTrip Inc

How we deploy our microservices in MagicalTrip

MagicalTrip Public Relations -

Introduction

Hi. I’m Sekimoto. I am an engineer working for MagicalTrip.
We are trying to find better ways for our developing always.
In this post I’ll explain how we develop and deploy our microservice architecture to the cloud.

I’ll assume you already have some basic knowlegde about the following topics:

  • Using a version control system like Git
  • Infrastructure as Code with Docker, and managing containers with Docker-Compose
  • Using a job scheduler like Jenkins
  • Concept of Microservices

THE PROBLEM OF DEPLOYING MICROSERVICES

On MagicalTrip we use a monorepo, (a single repository for all services). But if everytime we commit a change to the repository we need to redeploy everything, we’re coupling our microservices.
Microservice independence can help us minimize the impact on making changes to code, deploying everything is not only a waste of server time and resources, it’s missing the point of microservices altogether.

HOW WE CAN IMPROVE THIS

We need to deploy only the microservices that changed. To figure out what changed, we maintain a list of paths for each service in a config file. We then compare it with the changes between our latest commit and the commit of our last successful deploy.

HOW TO SPECIFY SERVICES

We are using Jenkins as our deployment tool for MagicalTrip.
We setup a webhook on Jenkins, that executes the our deploy job, whenever a developer pushes new code to the main repository.
In this job, we ask git for the list of changed files by comparing 2 commits: previous time of deployment and latest commit.

<code class="bash">git diff --name-only {commit hash of previous time of deployment}..{branch-name}

Comparing the files with a config file like below, we can know the list of services we should deploy.
Following is an example of our configuration file. The “path” in the file means path to the service in repository.

<code class="json"><br></br>{
"services": {
"A": {
"path": "services/A/",
"dockerServiceName": "A",
"staging": { ... },
"production": { ... }
},
"B": {
"path": "services/B/",
"dockerServiceName": "B",
"staging": { ... },
"production": { ... }
},
"C": {
"path": "services/C/",
"dockerServiceName": "C",
"staging": { ... },
"production": { ... }
}
}
}

Also, “dockerServiceName” is the same as the service name in docker-compose.yml file, so we can do:

<code class="bash">docker-compose build {service-name}

to get the Docker images.
We are using ECR as a repository for Docker images, and ECS for running containers, so our deployment finishes with uploading docker images and updating services using AWSCLI.

FINISHING NOTES

We could improve deployment speed of up to 30 times its default with some solutions included in the things I introduced.
Also, now we have an option to stop deployment when we changed files that has no affect on services, for example the deployment process.
These are just combinations of different ways we use that could minimize the server resouces and time for deploying.