Include what you use

Oh what a tangled web we weave when we first try to optimize files inclusion in C++. I can hear your sarcasm other-language programmer developer, feel free to flood the comments sections with something along the lines of “We don’t have this problem in fancy-new-language, you paleo-programmer!”.

I would be the first to reckon that this archaism inherited from C will have, at some point, to disappear. I nevertheless submit the elegant solution to this problem might not be within the language itself but more in the tools we use.

After all, who cares about the includes hell if the inclusion were written by a program?

This brings me to an interesting tool I saw in a tweet from Meeting C++ which kind of sounded like a gifts from the Gods.

The gods have listened

What does include what you use do?

Include what you use is a clang-based tool that will parse your C++ code and tell you which include files are necessary and which ones are unnecessary. It can even output a perfect inclusion list for each file. Since it’s based on the clang compiler it actually parses the code which sounds like the “right way” to do it.

In theory there is nothing wrong with having too many includes (and if you are missing includes you will realize it very quickly as your file will not compile), but this can lengthen compilation times greatly and I admit our taste for meta-programming and libraries such as Boost.Fusion and Boost.Spirit  tend to have a negative impact on the compilation times.

How we used the tool

As we use clang 3.3 downloading and compiling include what you use (iwyu) was a piece of cake. Having our CMake build use it instead of the default compiler was a bit trickier. The Boost library detection went wrong because of its inability to determine the compiler suffix. Forcing it to “-clang33” solved the problem.

Iwyu is a compiler replacement that always exits with an error. In other words, to use it you tell to your build chain to not use, e.g. clang, but “include-what-you-use” as a C++ compiler. Since the compiler always exits with an error, you also have to force your compilation chain to proceed with the compilation whatever error it encounters.

And we have here, I think, one of the first weaknesses of iwyu, in that there is a limit to how much you can force the compilation to proceed as it encounters errors.

Nevertheless, we managed to run the tool on our core libraries with some success.

Pushing the limits

You know what’s nice about template meta-programming? It always give you an opportunity to brag about how elite your code is and how tools cannot really understand what you are doing.

In our case, we have our fair share of meta-template craziness, and iwyu often wrongly suggested that we remove some Boost.Fusion or Boost.MPL headers. It goes without saying that following iwyu advices opened the gates of Hell and we had to resort to some forgotten rituals to obtains the favors of he who shall not be named to recover from the aftermath.

iwyu also wrongly suggested to remove headers from which we only use preprocessor defines. That can be very dangerous as the code may compile correctly despite removing an include. If, like us, you are users of Boost.Predef, be very careful.

Analyzing cross-platform code is also problematic as it cannot detect that if a header is useless on one platform it might be required on another, forcing you to either run the tool on all platforms or do some trial and errors to get the inclusions list right.

However the biggest problem is when it comes to public headers. It can correctly suggest to remove a header you don’t use in one of your library’s header. The problem is that although the exposed code itself doesn’t need the header, the code that is going to use your library might need it. In other words, you include more in your library header as a convenience for the users of your library.

The good news is that some of these problems can be fully or at least partially be solved with pragmas.

How did we fare?

Iwyu had a lot of  false positives, but helped us remove a couple of includes in some central files, includes we would not have been able to remove without it. By “not able”, I mean that it would have taken a lot of time and efforts to realize that some headers were unnecessary.

The aftermath is that we gained approximately 1 minute on a 20 minutes build, a 5% win that will probably compound over the time.

Was it worth the couple of hours invested? Definitely.