In the last week of April, a few weeks ago, our CI/CD pipeline for the educational SaaS platform we’re developing stopped working. Turns out, the free monthly 2000 minutes from GitHub Actions (for private repos) is not cutting it anymore.
The current situation
We use CI/CD mainly to run e2e Playwright tests, and we’re currently sitting at about 60 individual tests. By taking advantage of Playwright sharding, we’re able to run everything rather quickly, finishing all tests and linting for a new push in about 2-3 minutes. However, this does mean that 8 GitHub Actions runners running simultaneously for each new push will use the free minutes very fast.
Not to mention the pull requests opened by bots such as Renovate and Changesets, which were also using our pipeline for each push.
The first step in solving this situation was easy: now that we are reaching the free tier cap for the first time, we should probably revisit our workflows configuration. The Changesets bot does not need to run tests in each push, since it only modifies the changelog file. Also, the Renovate bot does not need to run the test in sharded mode each time, since the PRs it opens are not urgent.
Enabling the pipeline again
So we applied these new optimizations. However, our pipeline was still completely halted, because there were no minutes left for the runners. They would reset only by May 6th, so we had a few alternatives we could act on:
- Explore different CI/CD providers, such as CircleCI.
- Wait until after May 6th, to see if our improvements would be enough.
- Add our own runners to GitHub Actions.
CircleCI
We started investigating CircleCI to see if we could integrate. They have a really nice free tier of 6000 monthly minutes, instead of the 2000 from GitHub. For the few days that we tried configuring our repo over there, we had no luck running the pipeline successfully. We use pnpm to install packages and run the lifecycle scripts, and the Node Orbs available from CircleCI still only support npm and yarn.
I’m sure if we spent enough time tinkering it would eventually work, but we had to find a way to put our CI/CD pipelines online again as soon as possible, so the idea of migrating to CircleCI was discarded. We also don’t have much knowledge of other providers.
Oracle Cloud
Then out of nowhere, it came to me: “What if I set up runners using Oracle’s free VMs?“. They are ARM Ampere A1 instances from Oracle Cloud’s free tier, and I was already familiar with them because I used to setup modded Minecraft servers in them! So the testing began again.
The whole process was really very simple. I created 4 new machines, each one with 1 OCPU count, 6 GBs of RAM and 40 GBs of block storage. The idea is to create one runner for each VM. While you can setup multiple runners in a single VM, our e2e testing process involves setting up a MySQL database for the tests to run on. Since all tests share the same DB, it seemed more complicated to create a single VM with 4 cores, instead of 4 instances with 1 core each.
 |
|---|
| Each runner VM in the Oracle Dashboard |
To connect to them, we used SSH. When creating a new instance in the Oracle Cloud dashboard, you have the option to upload your public keys to your VM, allowing you to SSH into it without any issues.
Online again
After a few hours of testing, we were able to link our little VM to GitHub Actions and it was already acting as a runner and waiting for jobs! Also, to make it work with your Actions workflows, you have to set the runs-on: option to self-hosted.
Then, a few more minutes configuring each of them - a process that was basically installing a MySQL server, setting it up and linking the VM to GitHub Actions to act as a runner - the pipeline was completely back on again!
 |
|---|
| Runners in the GitHub dashboard |
The whole idea of “setting up our own runners” to run our CI/CD pipeline seemed so daunting at first, but GitHub Actions makes the whole process as simple as possible. We only had to run a few commands to download the runner script and call their API to register our new little runners.
And now our pipeline will run for free, forever! At least until Oracle decides to change their free tier…