(Make) Controlling target prerequisites

[Mario F.]

Parallel to my game development, I'm also managing unit testing as a separate project. This project is mostly made of several source files each with their own main(). Each file basically contains the unit tests for a class or routine in my main project.

I'm doing the unit tests development with the Crimson Editor and using my own makefile to control the build process. Crimson Editor also has a very low memory footprint so I can have it open alongside SlickEdit without any meaningful impact on my already low resources.

Anyways, this setup presents me with a problem.

I want the makefile to describe the target prerequisites in the most detailed fashion (with the exception for library headers). The Make documentation describes a solution involving .d files (one for each source file) and the use of the gcc -MM switch.

This seems a little overwhelming. All of a sudden I duplicate the number of files I need to source control and otherwise manage. Does anyone actually do this? I never saw it anywhere on SourceForge CVS, for instance.

Is there a better way?

Currently I'm thinking developing some small executable that simply traverses my sources tree looking for source files, call gcc -MM on them and grab the output to generate the makefile. I can then run it everytime I know I need to update the prerequisites. The extra work of calling the executable doesn't bother me. It's just that I only simplified the process. I still have the problem of not having automated it.

[brewbuck]

Originally Posted by Mario F.

I want the makefile to describe the target prerequisites in the most detailed fashion (with the exception for library headers). The Make documentation describes a solution involving .d files (one for each source file) and the use of the gcc -MM switch.

Gah, you don't need each dependency in a separate file. Just use gcc -M (not -MM) on all your source code (.c files) to generate a single dependency list. Stick this list in .depend or something. Then include it in your makefile.

Something like this (fragment of a makefile only):

# Your project sources
SRCS     = foo.c bar.c blah.c

# Your project headers
HDRS     = hdr1.h

.depend: $(SRCS) $(HDRS)
        $(CC) -M $(SRCS) $(WHATEVER_INCLUDE_FLAGS_YOU_NEED) > .depend

-include .depend

If your makefile has a "clean" or "realclean" target, you might want to add .depend to the list of files you remove.

EDIT: I maybe didn't make it clear, but doing it this way is fully automatic. If you change any header or source files, the dependency list is automatically rebuilt.

[Mario F.]

Doh! That simple. Thanks brewbuck.

[Mario F.]

Well, I still have a couple of nags with my makefile. No biggies. It's working ok. But...

CPP = mingw32-g++

BIN  = mediaeval.exe

SRCS = admin.cpp checksum.cpp # etc...

HDRS = admin.hpp checksum.hpp # etc...

ifeq ($(cfg), release)
   OBJDIR = release/
   CXXFLAGS = -Wall -Wextra -ansi -O2 -DNDEBUG
else
    OBJDIR = debug/
    CXXFLAGS = -Wall -Wextra -ansi -g3
    ifeq ($(cfg), profile)
        CXXFLAGS += -pg
    endif
endif

OBJ = $(OBJDIR)admin.o $(OBJDIR)checksum.o # etc...

LIBS = \
-lsqlite \
-lpdcurses \ #etc...

#-------------------------------------------------#

.PHONY: clean

$(BIN): $(OBJ)
	$(CPP) $(OBJ) -o $(BIN) $(LIBS)
   ifeq ($(cfg), release)
	    strip $(BIN)
   endif

-include .depend

%.o:
	$(CPP) -c $< -o $@ $(CXXFLAGS)

#-------------------------------------------------#

clean:
	rm -f $(OBJ) $(BIN) .depend

.depend: $(SRCS) $(HDRS)
	$(CPP) -MM $(SRCS) > .depend
	sed -r -i "s,^([a-z]+.o: ),$(OBJDIR)\1,g" .depend

Nag 1: Instead of issuing 'make cfg=release', for instance, as the above describes, I would rather much prefer 'make release'. Can I do this without having to turn release into a target? In other words, can I test for the existence of 'release' and 'profile' as if they were variables? <i>ifdef</i> seems to not be able to do it.

Nag 2: 'make clean' has the interesting side effect of calling .depend before calling clean if I do it repeatedly. I understand why it does it and it's not a big deal for the obvious reason. But i'm curious; can I control that -include so that it is not parsed when the target is 'clean'?

Also 1: Strip confuses me to no end. My executable compiled with -02 and no debugging symbols is around 2Mb (MinGW loves fat executables, bleh). However it can still be stripped of symbols down to 700k. Why? What symbols were these? What am I missing if I strip an executable compiled withough debugging symbols?

Also 2: I was surprised at the -M/-MM switches slowness. Look at the following small example I tested with only a subset of my source files:

admin.o: admin.cpp panelbuffer.hpp pdcstr.hpp admin.hpp mediaeval.hpp
crand.o: crand.cpp crand.hpp
crc.o: crc.cpp crc.hpp
mediaeval.o: mediaeval.cpp errmediaeval.hpp unit.hpp
menu.o: menu.cpp pdcstr.hpp admin.hpp mediaeval.hpp
panelbuffer.o: panelbuffer.cpp panelbuffer.hpp pdcstr.hpp
pdcstr.o: pdcstr.cpp pdcstr.hpp
unit.o: unit.cpp errmediaeval.hpp unit.hpp

The most you have there is a 2nd level dependency (should I say it this way?), in which admin.o dependency on mediaeval.hpp comes from admin.hpp, for instance. That output takes 5 seconds on my computer to be generated (PIII 1000, 512 RAM). Is this expected? Can it be improved somehow?

[brewbuck]

Nag 1: Instead of issuing 'make cfg=release', for instance, as the above describes, I would rather much prefer 'make release'. Can I do this without having to turn release into a target? In other words, can I test for the existence of 'release' and 'profile' as if they were variables? <i>ifdef</i> seems to not be able to do it.

It has to be a target:

release:
        $(MAKE) cfg=release

debug:
        $(MAKE) cfg=debug

Nag 2: 'make clean' has the interesting side effect of calling .depend before calling clean if I do it repeatedly. I understand why it does it and it's not a big deal for the obvious reason. But i'm curious; can I control that -include so that it is not parsed when the target is 'clean'?

No portable way, no. See below.

The most you have there is a 2nd level dependency (should I say it this way?), in which admin.o dependency on mediaeval.hpp comes from admin.hpp, for instance. That output takes 5 seconds on my computer to be generated (PIII 1000, 512 RAM). Is this expected? Can it be improved somehow?

The problem is that C++ files often take a large amount of time to parse, because of the STL headers and other complexities of C++ in general. The gcc -M command still has to parse (at least at a rudimentary level) each file fully in order to know what all its dependencies are. For a big project this simply takes a long time. Also, your machine is a bit... slow

So, this is one argument in FAVOR of tracking dependencies on a per-file basis. The tradeoff is additional complexity in the makefile, and lots of little dependency files littering your workspace.

An alternative, which is used on a great many projects, is to simply remove the $(SRCS) $(HDRS) dependencies from the .depend target (in essence, turning it into a manual process instead of an automatic one). Now you, as the programmer, must remember to remove the .depend file whenever you make a change that changes a dependency. In practice, those kinds of changes don't happen much once you've moved out of the initial development stages, so it's not such a hard practice to get used to. When in doubt, "make clean" first, unless the project is HUGE (in which case my measly advice may not serve)

Also 1: Strip confuses me to no end. My executable compiled with -02 and no debugging symbols is around 2Mb (MinGW loves fat executables, bleh). However it can still be stripped of symbols down to 700k. Why? What symbols were these? What am I missing if I strip an executable compiled withough debugging symbols?

There are other symbols in a file besides debug symbols -- mostly just the names of every function in every object file that was linked into the executable. It seems crazy that those symbols could be taking up an entire 1300k, but I suppose it's possible.

Does your development environment have an "objdump" command? If so, try running "objdump -h foo.exe" before and after stripping it, and see what sections changes in size.

[Mario F.]

Invaluable information! Thanks once again brebuck