Using PM2 to Build a Lightweight CI/CD Pipeline
Delivering features and bug fixes quickly is crucial for the success of small projects and startups. As described in this blog post, we decided to run daily-digest on our own server for various reasons. However, this means we have to implement many features ourselves that come free with major cloud vendors, such as automatic deployments and testing. At daily-digest, we follow the standard practice of having a development branch for active work (with feature branches for larger features) and a master branch containing the latest stable version. Whenever someone pushes to the master branch, it should be deployed to production automatically.
Deployment
The first challenge was deploying multiple Node.js applications (plus static web content) to production and automating the process. PM2 provides a simple way to deploy to a remote machine.
Prerequisite
npm install -g pm2 # Install PM2 on both your local and remote machines
After installing PM2, we need to prepare our descriptor file, which tells PM2 exactly what actions to perform during deployment.
Descriptor File
The descriptor file is the key configuration file in your deployment setup. It specifies what PM2 should do when deploying.
Here's a shortened version of the descriptor file we use at daily-digest:
module.exports = {
apps: [{
name: 'Daily-Digest Extractor',
//working dir
cwd: '/opt/daily-digest/backend/components/extractor/',
//main script file to run
script: './index.js',
//where to store the log file
log_file: '/opt/daily-digest/logs/extractor.log',
//if true, time stamps will be logged for every line
time: true,
//we use https://github.com/motdotla/dotenv#readme to distinguish between test, dev and production systems.
//on this line, the env we are using on prodction file is being applied
env: {
DOTENV_CONFIG_PATH: '/opt/daily-digest/.env',
},
node_args: '-r dotenv/config'
}, {
name: 'Daily-Digest Aggregator',
cwd: '/opt/daily-digest/backend/components/aggregator',
script: './index.js',
log_file: '/opt/daily-digest/logs/aggregator.log',
time: true,
env: {
DOTENV_CONFIG_PATH: '/opt/daily-digest/.env',
},
node_args: '-r dotenv/config'
}],
deploy: {
//this is the name that is being used in the pm2 deploy command
production: {
//if you use a public / private key for ssh access
key: 'deploy.key',
user: '*YOUR_SSH_USER*',
//this can be a single string or an array of hosts
host: '*YOUR_HOST*',
//you can use code from any branch. In our case, we only want to deploy code from master
ref: 'origin/master',
repo: 'git@github.com:*SOME_USER_NAME*/*SOME_REPO_NAME*.git',
//where to deploy the apps to
path: '/opt/daily-digest',
//you can run any shell script after successful deployment. Useful e.g. for copying static web content
'post-deploy': '/opt/daily-digest/post-deployment.sh'
}
}
};
As you can see above, the file itself is extremely easy to read and should be self explanatory, however I added some comments to the parts that might be confusing. To read more about the (tons) of configuration possibilities for apps & deployments, head over to the PM2 documentation: https://pm2.io/docs/runtime/reference/ecosystem-file
After you created the descriptor file, the next step is to setup your remote environment.
Setup remote environment
pm2 deploy production setup
For this command to run successfully, a key "production" must be configured in your deployment descriptor. PM2 must be installed on your remote machine and the remote machine must be able to clone the target repository (install the correct github keys). This command creates the necessary file structure and registry on your remote machine for processes managed by PM2.
Deploying to remote server
Once setup is successful, you can start deploying to your remote server.
> pm2 deploy <configuration_file> <environment> <command>
Commands:
setup run remote setup commands
update update deploy to the latest release
revert [n] revert to [n]th last deployment or 1
curr[ent] output current release commit
prev[ious] output previous release commit
exec|run <cmd> execute the given <cmd>
list list previous deploy commits
Regular deployments can be started with pm2 deploy ecosystem.config.js {key, e.g.
production}. If a deployment fails, you can revert to previous versions using pm2
deploy {key, e.g. production} revert {n}
Using Github
Now that we have an easy way to deploy to our remote machine, it would be great to deploy automatically whenever someone pushes to a specific branch. GitHub Actions makes this simple.
Setup
In order to tell Github Actions to do something when somebody is pushing to a branch, we must create a
yaml file.
Place the file under .github/deployment.yml in your repository.
This is a copy of daily-digest's deployment config file.
name: Deploy Daily digest
on:
push:
branches: [ master ]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up SSH
run: |
mkdir -p ~/.ssh/
echo "$SSH_PRIVATE_KEY" > ./deploy.key
sudo chmod 600 ./deploy.key
echo "$SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts
shell: bash
env:
SSH_PRIVATE_KEY: $
SSH_KNOWN_HOSTS: $
- name: Install PM2
run: npm i -g pm2
- name: Deploy
run: pm2 deploy ./backend/ecosystem.config.js prod
Pretty straight forward right? Well almost. While most of the config is self explanatory, the SSH
setup might be a bit tricky. As our remote server expects a private/public key to login via ssh, we
have to setup the keys before running pm2 deploy. Your setup might be different, so you might not
need this. However, see the secrets.SSH*? These are github secrets which you have to
setup in order to use them in your yaml files. To do this, go to your github repository settings ->
Secrets and variables -> Actions.
In here you can create the secrets as a simple key-value pair.
If everything is configured correctly, each time you push to your master branch, the GitHub Action will now be triggered and deploy your code. Pretty cool, right?