Make is what ties it all together. Makefiles are some arcane gizmo magic. They are also old, old enough that they exist everywhere. Some of you might believe that makefiles are just for building code. Indeed they can be used for this:
hello: $(subst .c,.o,$(wildcard *.c))
@$(CC) $(LDFLAGS) -o $@ $^
Where the power of make comes in, is in its ability to encode village knowledge and arcane corner cases. When I start new projects I always put two things in there immediately: a Readme.md and a Makefile. Both serves as documentation on how to operate and build the project. I have the rule that “just typing make should do all necessary things for a normal development cycle”. This includes potentially installing missing software on new computers.
One of the goals for having a top level makefile is that all the common, non obvious tasks can be encapsulated here and documented.
You want to get started? Type “make”.
You want to build the code? “make build”.
You want to sync the most recent code + data? “make sync”.
You want to ask the servers to test your branch? “make remote-test”
Well, you get the idea…
Here follows some basic recipes of different tasks using makefiles:
One rule that we’ve all implemented when using Makefiles for building code is actually “clean”. Something that nukes everything from orbit and starts over. If you want to use this in a portable way, you can always use $(RM) as the command.
clean:
@$(RM) hello *.o
One trick that I like is to automatically install software that might be needed when you run rules. You can do the following:
lint: .provision
@swift-format -lint $(wildcard *.swift)
.provision:
@which swift-format > /dev/null 2>&1 || brew install swift-format
@touch $@
Don’t forget to add .provision to your .gitignore file and your clean rule!
I’m a big fan of testing and while I would advocate for simply running your unittests as part of your build process and format the output like a compile error to make sure that they stay relevant and not forgotten, there is an argument for putting functional / integration tests inside a separate rule. I usually simply call this rule “test”. This might be a simple functional test that processes a file, dumps out some debug information and compares it to a golden output:
test:
@gizmo export ./data/sphere.obj ./data/sphere.model
@gizmo dump ./data/sphere.model ./data/sphere.debug
@diff ./data/sphere.debug ./code/sphere.golden-output
I’m usually forgetful of my arguments and options, my muscle memory just contains “make”. Furthermore, a lot of integrations just call make without any arguments.
It so happens if you just call make without any arguments, it will assume that you want to execute the first rule encountered in the Makefile. So for example, if you want each invocation of make to use all of your CPUs:
default:
@$(MAKE) -j hello
hello: $(subst .c,.o,$(wildcard *.c))
@$(CC) $(LDFLAGS) -o $@ $^
This can of course be extended into more elaborate schemes.
You can actually do some pretty fancy footwork with doing make inside of a docker container for more stable builds, regardless of which host platform you might be on. This rule below will build the linux executable of the source code with the help of the jtilander/dev-debug image:
default:
@docker run --rm -v $(PWD):/home/jenkins jtilander/dev-debug make hello
hello: $(subst .c,.o,$(wildcard *.c))
@$(CC) $(LDFLAGS) -o $@ $^
If you are interacting with Jenkins declarative pipeline, and their credentials statement, that sticks the login information in some environment variables; then you might have found it useful to be able to iterate locally.
I usually do this by sticking a .env file locally and enumerating all the environment variables here. This can incidentally also be fed to docker via the –env-file argument. The .env file might look like this:
ADMIN_USR=root
ADMIN_PSW=this_is_the_admin_password
We can then put corresponding rules in the Makefile like this to optionally pick up the environment from the .env file, but only if the environment doesn’t already have these declared – thus letting Jenkins override these for us.
ADMIN_USR?=$(shell grep ADMIN_USR .env|cut -f2 -d=)
ADMIN_PSW?=$(shell grep ADMIN_PSW .env|cut -f2 -d=)
login:
@echo $(ADMIN_USR) login: $(ADMIN_PSW)
So what about actually building code? Actually, I use Makefile less and less for this. Usually because I’m stuck with some other legacy build system. So I tend to just call Visual Studio, xcode, sn-dbs, cmake, or whatever, directly from the Makefile.
Calling the native build system, e.g. xcode, also makes building the code a matter of a single source of truth. If it works for the interactive user, it will work for the automated user. This is a huge bonus and eases debugging; as well as the arguments for “must be something special on the servers that screw up, works on my machine!”.
One of the less obvious ramifications of enforcing higher level makefiles for all your projects is that it is very simple to write integrations into your CI system (e.g. Jenkinsfiles) since you just call the appropriate targets.
It is also relatively easy to make reproducing what the CI server is doing locally, since it’s a matter of running make in the right order with the right environment flags.
Another huge implication is that is trivial (as in searching for the string “make: ***") to determine from the log if there was a user error (inside the make rules), or a system error (e.g. source control failed, Jenkins failed). This is a boon if you want to change studio culture and classify errors into “you did bad” as opposed to “system did bad”.
There is a port of gnu make that runs on modern (Windows 10-ish) installations just fine. There are a couple of things that you can do to make sure that it works (non exhaustive list from memory):
set SHELL=cmd.exe
This ensures that you get some consistent parsing of quotes and generally don’t pick up any sh.exe that might exist in the path.
PWD=$(CWD)
This ensures that some makefiles remain portable, since windows doesn’t have the environment PWD for current working directory.
You will be somewhat stymied on windows, as the most powerful shell commands are not available to you, making it a bit of a pain to get anything done easily.
Make has gotten a lot of flak for being a primitive build system for the larger, more complex (borderline insane) C++ builds out there. Especially since autoconf et al. came along and started generating Makefiles that were just plain unreadable.
Writing handcrafted top level makefiles that are more about high level intent, and less about the mechanics around building, can be very useful for future you when trying to resurrect whatever project you were mucking around with.
If you take nothing else from this post; create a Readme.md and a Makefile at the root of your project today!