Debugging the Error Novel



Debugging the Error Novel

The title of this section is actually taken from another book [VJ02], but it's so wonderfully apt that we had to use it ourselves. In fact, template error reports so often resemble War and Peace in approachability that many programmers ignore them and resort to random code tweaks in the hope of making the right change. In this section we'll give you the tools to skim these diagnostic tomes and find your way right to the problem.

Note

We'll be looking at examples of error messages, many of which would be too wide to fit on the page if presented without alteration. In order to make it possible to see these messages, we've broken each long line at the right margin, and where neccessary added a blank line afterwards to separate it from the line following.


Instantiation Backtraces

Let's start with a simple (erroneous) example. The following code defines a simplistic compile-time "linked list" type structure, and a metafunction designed to compute the total size of all the elements in a list:

    struct nil {};                     // the end of every list

    template <class H, class T = nil>  // a list node, e.g:
    struct node                        // node<X,node<Y,node<Z> > >
    {
        typedef H head; typedef T tail;
    };

    template <class S>
    struct total_size
    {
        typedef typename total_size<   // total size of S::tail
            typename S::tail
        >::type tail_size;             // line 17

        typedef boost::mpl::int_<      // add size of S::head
            sizeof(S::head)
            + tail_size::value         // line 22
        > type;
    };

The bug above is that we've omitted the specialization needed to terminate the recursion of total_size. If we try to use it as follows:

    typedef total_size<
        node<long, node<int, node<char> > >
    >::type x;                          // line 27

we get an error message something like this one generated by version 3.2 of the GNU C++ compiler (GCC):

    foo.cpp: In instantiation of 'total_size<nil>':
    foo.cpp:17:   instantiated from 'total_size<node<char, nil> >'
    foo.cpp:17:   instantiated from 'total_size<node<int,
    node<char, nil > > >'
    foo.cpp:17:   instantiated from 'total_size<node<long int,
    node<int, node<char, nil> > > >'
    foo.cpp:27:   instantiated from here
    foo.cpp:17: no type named 'tail' in 'struct nil'
    continued...

The first step in getting comfortable with long template error messages is to recognize that the compiler is actually doing you a favor by dumping all that information. What you're looking at is called an instantiation backtrace, and it corresponds almost exactly to a runtime call stack backtrace. The first line of the error message shows the metafunction call where the error occurred, and each succeeding line that follows shows the metafunction call that invoked the call in the line that precedes it. Finally, the compiler shows us the low-level cause of the error: we're treating the nil sentinel as though it were a node<...> by trying to access its ::tail member.

In this example it's easy to understand the error simply by reading that last line, but as in runtime programming, a mistake in an outer call can often cause problems many levels further down. Having the entire instantiation backtrace at our disposal helps us analyze and pinpoint the source of the problem.

Of course, the result isn't perfect. Compilers typically try to "recover" after an error like this one and report more problems, but to do so they must make some assumptions about what you really meant. Unless the error is as simple as a missing semicolon, those assumptions tend to be wrong, and the remaining errors are less useful:

    ...continued from above
    foo.cpp:22: no type named 'tail' in 'struct nil'
    foo.cpp:22: 'head' is not a member of type 'nil'
    foo.cpp: In instantiation of 'total_size<node<char, nil> >':
    foo.cpp:17:   instantiated from 'total_size<node<int, node<char,
    nil> > >'

    foo.cpp:17:   instantiated from 'total_size<node<long int, node<
    int, node<char, nil> > > >'

    foo.cpp:27:   instantiated from here
    foo.cpp:17: no type named 'type' in 'struct total_size<nil>'
    foo.cpp:22: no type named 'type' in 'struct total_size<nil>'
    ...many lines omitted here...
    foo.cpp:27: syntax error before ';'token

In general, it's best to simply ignore any errors after the first one that results from compiling any source file.

Error Formatting Quirks

While every compiler is different, there are some common themes in message formatting that you may learn to recognize. In this section we'll look at some of the advanced error-reporting features of modern compilers.

2.1 A More Realistic Error

Most variations in diagnostic formatting have been driven by the massive types that programmers suddenly had to confront in their error messages when they began using the STL. To get an overview, we'll examine the diagnostics produced by three different compilers for this ill-formed program:

    # include <map>
    # include <list>
    # include <iterator>
    # include <string>
    # include <algorithm>

    using namespace std;
    void copy_list_map(list<string> & l, map<string, string>& m)
    {
        std::copy(l.begin(), l.end(), std::back_inserter(m));
    }

Although the code is disarmingly simple, some compilers respond with terribly daunting error messages. If you're like us, you may find yourself fighting to stay awake when faced with the sort of unhelpful feedback that we're about to show you. If so, we urge you to grab another cup of coffee and stick it out: The point of this section is to become familiar enough with common diagnostic behaviors that you can quickly see through the mess and find the salient information in any error message. After we've gone through a few examples, we're sure you'll find the going easier.

With that, let's throw the code at Microsoft Visual C++ (VC++) 6 and see what happens.

    C:\PROGRA~1\MICROS~4\VC98\INCLUDE\xutility(19) : error C2679:
    binary '=' : no operator defined which takes a right-hand operand
    of type 'class std::basic_string<char,struct std::char_traits<
    char>,class std::allocator<char> >' (or there is no acceptable
    conversion)

          foo.cpp(9) : see reference to function template
          instantiation 'class std::back_insert_iterator<class std::
          map<class std::basic_string<char, struct std::char_traits<
          char>,class std::allocator<char> >,class std::basic_string<
          char,struct std::char_traits<char>,class std::allocator<char
          > >,struct std::less<class std::basic_string<char,struct std
          ::char_traits<char>,class std::allocator<char> > >,class std
          ::allocator<class std::basic_string<char,struct std::
          char_traits<char>,class std::allocator<char> > > > > __cdecl
          std::copy(class std::list<class std::basic_string<char,
          struct std::char_traits<char>,class std::allocator<char> >,
          class std::allocator<class std::basic_string<char,struct std
          ::char_traits<char>,class std::allocator<char> > > >::
          iterator,class std::list<class std::basic_string<char,struct
          std::char_traits<char>,class std::allocator<char> >,class
          std::allocator<class std::basic_string<char,struct std::
          char_traits<char>,class std::allocator<char> > > >::iterator
          ,class std::back_insert_iterator<class std::map<class std::
          basic_string<char,struct std::char_traits<char>,class std::
          allocator<char> >,class std::basic_string<char,struct std::
          char_traits<char>,class std::allocator<char> >,struct std::
          less<class std::basic_string<char,struct std::char_traits<
          char>,class std::allocator<char> > >,class std::allocator<
          class std::basic_string<char,struct std::char_traits<char>,
          class std::allocator<char> > > > >)' being compiled
    message continues...

Whew! Something obviously has to be done about that. We've only shown the first two (really long) lines of the error, but that alone is almost unreadable. To get a handle on it, we could copy the message into an editor and lay it out with indentation and line breaks, but it would still be fairly unmanageable: Even with no real formatting it nearly fills a whole page!

8.1.2.2 typedef Substitution

If you look closely, you can see that the long type

    class std::basic_string<char, struct std::char_traits<char>,
                            class std::allocator<char> >

is repeated twelve times in just those first two lines. As it turns out, std::string happens to be a typedef (alias) for that type, so we could quickly simplify the message using an editor's search-and-replace feature:

    C:\PROGRA~1\MICROS~4\VC98\INCLUDE\xutility(19) : error C2679:
    binary '=' : no operator defined which takes a right-hand operand
    of type 'std::string' (or there is no acceptable conversion)

        foo.cpp(9) : see reference to function template instantiation
        'class std::back_insert_iterator<class std::map<std::string,
        std::string,struct std::less<std::string>,class
        std::allocator<std::string> > > __cdecl std::copy(class
        std::list<std::string,class std::allocator<std::string>
        >::iterator,class std::list<std::string,class std::allocator<
        std::string> >::iterator,class std::back_insert_iterator<
        class std::map<std::string,std::string,struct std::
        less<std::string>,class std::allocator<std::string> > >)'
        being compiled

That's a major improvement. Once we've made that change, the project of manually inserting line breaks and indentation so we can analyze the message starts to seem more tractable. Strings are such a common type that a compiler writer could get a lot of mileage out of making just this one substitution, but of course std::string is not the only typedef in the world. Recent versions of GCC generalize this transformation by remembering all namespace-scope typedefs for us so that they can be used to simplify diagnostics. For example, GCC 3.2.2 says this about our test program:

    ...continued messages
    /usr/include/c++/3.2/bits/stl_algobase.h:228: no match for '
       std::back_insert_iterator<std::map<std::string, std::string,
       std::less<std::string>, std::allocator<std::pair<const
       std::string, std::string> > > >& = std::basic_string<char,
       std::char_traits<char>, std::allocator<char> >&' operator
    messages continue...

It's interesting to note that GCC didn't make the substitution on the right hand side of the assignment operator. As we shall soon see, however, being conservative in typedef substitution might not be such a bad idea.

2.3 "With" Clauses

Take a look back at our revised VC++ 6 error message as it appears after the std::string substitution. You can almost see, if you squint at it just right, that there's an invocation of std::copy in the second line. To make that fact more apparent, many compilers separate actual template arguments from the name of the template specialization. For example, the final line of the GCC instantiation backtrace preceding the error cited above is:

    /usr/include/c++/3.2/bits/stl_algobase.h:349: instantiated from
    '_OutputIter std::copy(_InputIter, _InputIter, _OutputIter)
    [with _InputIter = std::_List_iterator<std::string, std::string&,
    std::string*>, _OutputIter = std::back_insert_iterator<std::map<
    std::string, std::string, std::less<std::string>, std::allocator<
    std::pair<const std::string, std::string> > > >]'
    messages continue...

Reserved Identifiers

The C++ standard reserves identifiers that begin with an underscore and a capital letter (like _InputIter) and identifiers containing double-underscores anywhere (e.g., __function__) for use by the language implementation. Because we're presenting diagnostics involving the C++ standard library, you'll see quite a few reserved identifiers in this chapter. Don't be misled into thinking it's a convention to emulate, though: The library is using these names to stay out of our way, but if we use them our program's behavior is undefined.


The "with" clause allows us to easily see that std::copy is involved. Also, seeing the formal template parameter names gives us a useful reminder of the concept requirements that the copy algorithm places on its parameters. Finally, because the same type is used for two different formal parameters, but is spelled out only once in the "with" clause, the overall size of the error message is reduced. Many of the compilers built on the Edison Design Group (EDG) front-end have been doing something similar for years.

Microsoft similarly improved the VC++ compiler's messages in version 7, and also added some helpful line breaks:

    foo.cpp(10) : see reference to function template instantiation
    '_OutIt std::copy(_InIt,std::list<_Ty,_Ax>::iterator,
    std::back_insert_iterator<_Container>)' being compiled

    with
    [
        _OutIt=std::back_insert_iterator<std::map<std::string,std
        ::string,std::less<std::string>,std::allocator<std::pair<
        const std::string,std::string>>>>,

        _InIt=std::list<std::string,std::allocator<std::string>>::
        iterator,

        _Ty=std::string,
        _Ax=std::allocator<std::string>,
        _Container=std::map<std::string,std::string,std::less<std
        ::string>,std::allocator<std::pair<const std::string,std::
        string>>>

    ]

Unfortunately, we also begin to see some unhelpful behaviors in VC++ 7.0. Instead of listing _InIt and _OutIt twice in the function signature, the second and third parameter types are written out in full and repeated in the "with" clause. There's a bit of a ripple effect here, because as a result _Ty and _Ax, which would never have shown up had _InIt and _OutIt been used consistently in the signature, also appear in a "with" clause.

2.4 Eliminating Default Template Arguments

In version 7.1, Microsoft corrected that quirk, giving us back the ability to see that the first two arguments to std::copy have the same type. Now, though, they show the full name of the std::copy specialization, so we still have to confront more information than is likely to be useful:

    foo.cpp(10) : see reference to function template instantiation '
    _OutIt std::copy<std::list<_Ty>::iterator,std::
    back_insert_iterator<_Container> >(_InIt,_InIt,_OutIt)' being
    compiled
    with
    [
        _OutIt=std::back_insert_iterator<std::map<std::string,std::
        string>>,

        _Ty=std::string,
        _Container=std::map<std::string,std::string>,
        _InIt=std::list<std::string>::iterator
    ]
    messages continue...

Had the highlighted material above been replaced with std::copy<_InIt,_OutIt>,_Ty could also have been dropped from the "with" clause.

The good news is that an important simplification has been made: std::list's default allocator argument and std::map's default allocator and comparison arguments have been left out. As of this writing, VC++ 7.1 is the only compiler we know of that elides default template arguments.

8.1.2.5 Deep typedef Substitution

Many modern compilers try to remember if and how each type was computed through any typedef (not just those at namespace scope), so the type can be represented that way in diagnostics. We call this strategy deep typedef substitution, because typedefs from deep within the instantiation stack show up in diagnostics. For instance, the following example:

    # include <map>
    # include <vector>
    # include <algorithm>
    int  main()
    {
        std::map<int,int> a;
        std::vector<int> v(20);
        std::copy(a.begin(), a.end(), v.begin());
        return 0;
    }

produces this output with Intel C++ 8.0:

    C:\Program Files\Microsoft Visual Studio .NET 2003\VC7\INCLUDE\
    xutility(1022): error: no suitable conversion function from "std::
    allocator<std::pair<const int, int>>::value_type" to "std::
    allocator <std::_Tree<std::_Tmap_traits<int, int, std::less<int>,
    std::allocator<std::pair<const int, int>>, false>>::key_type=
    {std::_Tmap_traits<int, int, std::less<int>, std::allocator<std::
    pair<const int, int>>, false>::key_type={int}}>::value_type=
    {std::_Allocator_base<std::_Tree<std::_Tmap_traits<int, int,

            std::less<int>, std::allocator<std::pair<const int, int>>,
            false>>::key_type={std::_Tmap_traits<int, int, std::less<
            int>, std::allocator<std::pair<const int, int>>, false>::
            key_type={int}}>::value_type={std::_Tree<std::_Tmap_traits
            <int, int, std::less<int>, std::allocator<std::pair<const
            int, int>>, false>>::key_type={std::_Tmap_traits<int, int,
            std::less<int>, std::allocator<std::pair<const int, int>>,
            false>::key_type={int}}}}" exists

          *_Dest = *_First;
                   ^
   ...

What do we need to know, here? Well, the problem is that you can't assign from a pair<int, int> (the map's element) into an int (the vector's element). That information is in fact buried in the message above, but it's presented badly. A literal translation of the message into something more like English might be:

No conversion exists from the value_type of an

    allocator<pair<int,int> >

to the value_type of an

    _allocator<

          _Tree<...>::key_type... (which is some
    _Tmap_traits<...>::key_type, which is int)
    >.

Oh, that second value_type is the value_type of an

    _Allocator_base<

          _Tree<...>::key_type... (which is some
    _Tmap_traits<...>::key_type, which is int)
    >,

which is also the key_type of a _tree<...>, which is int.

Ugh. It would have been a lot more helpful to just tell us that you can't assign from pair<int, int> into int. Instead, we're presented with a lot of information about how those types were derived inside the standard library implementation.

Here's a report of the same error from VC++ 7.1:

    C:\Program Files \Microsoft Visual Studio .NET 2003\Vc7 \include\
    xutility(1022) : error C2440: '=' : cannot convert from 'std::
    allocator<_Ty>::value_type' to 'std::allocator<_Ty>::value_type'

            with
            [
                _Ty=std::pair<const int,int>
            ]
            and
            [
                _Ty=std::_Tree<std::_Tmap_traits<int,int,std::less<
                int>,std::allocator<std::pair<const int,int>>,
                false>>::key_type

            ]
    ...

This message is a lot shorter, but that may not be much consolation: It appears at first to claim that allocator<_Ty>::value_type can't be converted to itself! In fact, the two mentions of _Ty refer to types defined in consecutive bracketed clauses (introduced by "with" and "and"). Even once we've sorted that out, this diagnostic has the same problem as the previous one: The types involved are expressed in terms of typedefs in std::allocator. It's a good thing that it's easy to remember that std::allocator's value_type is the same as its template argument, or we'd have no clue what types were involved here.

Since allocator<_Ty>::value_type is essentially a metafunction invocation, this sort of deep typedef substitution really does a number on our ability to debug metaprograms. Take this simple example:

   # include <boost/mpl/transform.hpp>
   # include <boost/mpl/vector/vector10.hpp>

   namespace mpl = boost::mpl;
   using namespace mpl::placeholders;
   template <class T>
   struct returning_ptr
   {
       typedef T* type();
   };

   typedef mpl::transform<
         mpl::vector5<int&,char,long[5],bool,double>
       , returning_ptr<_1>
    >::type functions;

The intention was to build a sequence of function types returning pointers to the types in an input sequence, but the author forgot to account for the fact that forming a pointer to a reference type (int&) is illegal in C++. Intel C++ 7.1 reports:

    foo.cpp(19): error: pointer to reference is not allowed
          typedef T* type();
                   ^
            detected during:
              instantiation of class "returning_ptr<T> [with T=boost::
              mpl::bind1<boost::mpl::quote1<returning_ptr>, boost::mpl
              ::lambda_impl<boost::mpl::_1, boost::mpl::false_>::type>
              ::apply<boost::mpl::vector_iterator<boost::mpl::vector5<
              int &, char, long [5], bool={bool}, double>::type, boost
              ::mpl::integral_c<long, 0L>>::type, boost::mpl::void_,
              boost::mpl::void_, boost::mpl::void_, boost::mpl::void_>
              ::t1]" at line 23 of "c:/boost/boost/mpl/aux_/has_type.
              hpp"

The general cause of the error is perfectly clear, but the offending type is far from it. We'd really like to know what T is, but it's expressed in terms of a nested typedef: mpl::bind1<...>::t1. Unless we're prepared to crawl through the definitions of mpl::bind1 and the other MPL templates mentioned in that line, we're stuck. Microsoft VC++ 7.1 is similarly unhelpful:[1]

[1] Fortunately, Microsoft's compiler engineers have been listening to our complaints, and an evaluation version of their next compiler only injects typedefs defined at namespace scope into its diagnostics. With any luck, this change will survive all the way to the product they eventually release.

    foo.cpp(9) : error C2528: 'type' : pointer to reference is illegal
            c:\boost\boost\mpl\aux_\has_type.hpp(23) : see reference
            to class template instantiation 'returning_ptr<T>' being
            compiled

            with
            [
              T=boost::mpl::bind1<boost::mpl::quote1<returning_ptr>,
              boost::mpl::lambda_impl<boost::mpl::_1>::type>::apply<
              boost::mpl::vector_iterator<boost::mpl::vector5<int &,
              char,long [5],bool,double>::type,boost::mpl::integral_c<
              long,0>>::type >::t1

          ]

GCC 3.2, which only does "shallow" typedef substitution, reports:

    foo.cpp: In instantiation of 'returning_ptr<int&>':
    ...many lines omitted...
    foo.cpp:19: forming pointer to reference type 'int&'

This message is much more sensible. We'll explain the omitted lines in a moment.