Modern tech stacks are complex and require one to remember hundreds of combinations of commands and parameters.
As a full stack developer, I use many different tools in the command line.
To name a few: go, python, sass, docker, terraform, kubectl, awscli, eksctl and more.
Remembering the all the tasks requires some brain muscle, and context. Jumping from one project to another, I need to answer many questions before being able to start working on it:
- do I push the build docker image to Y?
- Does this project use
webpack
orrollup
? - Does it use pipenv or poetry?
- Do I type yarn or npm to install js dependencies?
Jumping across teams, or coming back to a certain project after a while, becomes really hard.
Makefile
s offer a really nice abstraction to all of these command and tools.
For example, instead of typing:
$ docker build -t $(git describe --always)-f Dockerfile .
I can now just write
$ make build-img
This not only saves time restructuring the correct command, it requires less typing, and prevents mistakes. It is also great for teams with various skill levels.
make
is a very old and boring technology. It is battle tested, and people already wrote mountains of words on why you should adopt Makefile
s.
So, after you adopted make
, you now have a few Makefile
s, and when you come into a project you just need to read the Makefile
to see what it is doing, right?
Well... not so fast...
Unfortunately, Makefile
s have very cryptic syntax if you are not used to them. Also, they can be mixed with inline shell, and finally they can have a few hundred lines of code.
Hence, understanding all the goals (build targets) is going to be tedious.
Some people already came with a good solution to this problem by integrating an awk
script in the Makefile
to parse the targets in the file and print a useful overview (here is just one example out of many).
For almost a decade, I used a similar awk
script embedded in my Makefile
. Adding color, some more bells and whistles. With time, I wanted even more from that script. Most importantly, I wanted to document variables which affect different targets.
So, I rewrote it in Python, which seemed easier than awk
. When it grew to about 30 lines of code, I extracted it as a Python package called make-help-helper.
As much as I love Python, building and distributing binaries for this Makefile
helper was not so easy, and letting people copying an inline script from one Makefile
to another seemed like a very bad way to distribute software ...
enter mh
With the limited scoped of this script, I decided it was a good and fun project to exercise my C programming skills. And so the program mh
was born. In the spirit of old UNIX programs, it has a two letter name, which stands for makes help
I'm using it across multiple projects for more than 3 years, and recently I gave it some more polish, and decided to release it to the world, in the hope more people will find it useful.
So without further ado, here is a demo of what mh
does.
Running make
or make help
in a project directory with a specially crafted Makefile
you will see a colored output similar to this:
The above output was generated from a Makefile
with the following content:
.PHONY: test coverage all watch clean docker-build docker-run save cp-to-server \
import-on-server run-on-server disable-on-server
SHELL := /bin/bash
.DEFAULT_GOAL := help
.PHONY: help
help:
@mh -f $(MAKEFILE_LIST) $(target) || echo "Please install mh from https://github.com/oz123/mh/releases/latest"
ifndef target
@(which mh > /dev/null 2>&1 && echo -e "\nUse \`make help target=foo\` to learn more about foo.")
endif
ifneq (,$(wildcard ./.env))
include .env
export
endif
run-server: ## run a local server
python3 sweet.py serve -p 8080 &
test: run-server ## run the test suite with pytest
python -m pytest
coverage: ## run the tests and collect metrics
python -m pytest -vv --cov=coldsweet tests
all: update
update: CSS_FILES ?= ./static/stylesheets/all.scss:./static/stylesheets/all.css #? the css files to update
update: ## update css files from sass sources
sass -f -t compressed --update $(CSS_FILES)
watch: ## rebuild css on changes to sass sources
sass --watch $(CSS_FILES)
clean: ## remove sass cache
rm -r ./.sass-cache
REGISTRY=registry.acme.org #? the docker registry
ORG=oz123 #? the organization
IMG=coldsweet #? the image name
docker-push:: ## push the built image to the repository
docker push $(REGISTRY)/$(ORG)/$(IMG):$(shell git describe)
docker-build: ## build a docker image
docker build -t $(REGISTRY)/$(ORG)/$(IMG):$(shell git describe) -f docker/Dockerfile-py312 .
docker-run: CMD ?=
docker-run: TAG ?= $(shell git describe)
docker-run: ## run the docker image locally
docker run --rm -e COLDSWEET_DEBUG=1 \
-e COLDSWEET_INSTALL_DIR=/run/coldsweet \
-e COLDSWEET_CONFIG_PATH=/etc/coldsweet/config \
-v $(CURDIR)/coldsweet/templates:/run/coldsweet/coldsweet/templates \
-p 8080:8080 \
-it -p 9001:9001 -v $(CURDIR)/data:/var/lib/coldsweet/db -v $(CURDIR)/docker/:/etc/coldsweet/ -w /run/coldsweet $(REGISTRY)/$(ORG)/$(IMG):$(TAG) $(CMD)
docker-save: ## saves the image to a localfile
sudo docker save $(ORG)/$(IMG):$(TAG) > $(ORG).$(IMG).$(TAG).img
deploy-k8s:
make -C k8s apply
cp-to-server: ## upload the image to a server
scp $(ORG).$(IMG).$(TAG).img $(SERVER):~/
import-on-server: ## imports the image on the serve
ssh -t $(SERVER) "echo $(SUDOPASSWORD) | sudo -S docker load --input $(ORG).$(IMG).$(TAG).img"
run-on-server: ## runs the docker image on a remote server
ssh -t $(SERVER) "echo $(SUDOPASSWORD) | sudo -S docker run --name $(CONTAINERNAME) \
-d --restart always
-v $(BASEDIR):/var/lib/coldsweet/db \
-p $(HOST):9001:9001 $(ORG)/$(IMG):$(TAG)"
disable-on-server: ## stops the container on a remote server
ssh -t $(SERVER) "echo $(SUDOPASSWORD) | sudo -S docker rm --force $(CONTAINERNAME)"
compile-sass:
cd static/stylesheets && sassc all.scss all.css
minify-sass:
cd static/stylesheets && sassc --sourcemap -t compressed all.scss all.min.css
The biggest addition to the simple AWK script or the Python package is the ability to document global variables and target local variables.
Targets are documented with:
foo: ## foo does bar
echo bar
local variables are documented with:
NAME ?= world #? great who
greet:
echo $(HELLO)
global variables are documented with:
REGISTRY ?= docker.io #? where should images be pushed
In addition to having global variables inside the Makefile
, I found it useful to have the Makefile read variables from a .env
a la 12-Factor applications.
That is done by the following code in the above Makefile
:
ifneq (,$(wildcard ./.env))
include .env
export
endif
Now, if your directory contains a .env
file, mh
will read the variables and their description and show them on the console:
Usually, my repositories will contain a file called dot.envexample
, which will show other developers what can go into this file.
The format which is understood by mh
is:
FOO=BAR #? control the foo
MY_SECRET_API_KEY=123456 #? the secret api key
You can learn more about a specific target with make help target=<name>
, for example:
If you found it so far useful, you can get a version of mh
as a statically compiled binary for Linux or Dynamically compiled Mac OSX binary in the GitHub release page.
Of course, you can study the code, or provide feedback and contributions too.