View Issue Details

IDProjectCategoryView StatusLast Update
0001676OpenFOAMBugpublic2015-05-19 08:46
Reporteralexeym Assigned Tohenry  
PrioritynoneSeverityfeatureReproducibilityN/A
Status resolvedResolutionfixed 
PlatformGNU/LinuxOSScientificOS Version7.1
Summary0001676: printStack implementation
DescriptionCurrently printStack function is implemented using backtrace_symbols call.
It takes string array returned by the call, parses these strings for symbol
name/address/offset, and finally uses addr2line utility for extract debug
information. Most of the time this procedure works OK except when the function
is not able to parse symbol name correctly.

backtrace_symbols itself uses dladdr to query stack address for file name,
symbol name, file base address, and symbol address. After it calculates size of
the string to host information, and writes it there using sprintf [1]. Though
dladdr is not in POSIX, it is available in GNU and BSD libc. Also dladdr is
used in printSourceFileAndLine function, so, I guess, its absence in POSIX was
not the reason for current printStack implementation.

Please find printStack.C file rewritten to utilize dladdr for resolution of the
stack addresses. It was tested in Scientific Linux 7.1 / OpenFOAM 2.3.x and the
output seems to be reasonable. Also symbol name parsing bug has gone. I have
decided to attach the file instead of a patch as original file is almost
totally gone.

To illustrate new and old behavior of printStack I also attached logs of
several crashes (cavity icoFoam tutorial case with negative kinematic
viscosity). *.orig.Debug and *.orig.Opt are output of the original function
built with and without debug symbols. *.new.* are the same outputs with new
implementation. Problems with symbol name parsing in the original function
appear at #7, #8, and #9 in printStack.log.orig.Opt.

1. https://sourceware.org/git/?p=glibc.git;a=blob;f=debug/backtracesyms.c;h=2c7305c81bb223a4cf38b1efcc7f53fbc8c61c4b;hb=4e42b5b8f89f0e288e68be7ad70f9525aebc2cff#l89
TagsNo tags attached.

Activities

alexeym

2015-04-28 14:59

reporter  

printStack.C (5,648 bytes)   
/*---------------------------------------------------------------------------*\
  =========                 |
  \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox
   \\    /   O peration     |
    \\  /    A nd           | Copyright (C) 2011-2013 OpenFOAM Foundation
     \\/     M anipulation  |
-------------------------------------------------------------------------------
License
    This file is part of OpenFOAM.

    OpenFOAM is free software: you can redistribute it and/or modify it
    under the terms of the GNU General Public License as published by
    the Free Software Foundation, either version 3 of the License, or
    (at your option) any later version.

    OpenFOAM is distributed in the hope that it will be useful, but WITHOUT
    ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
    FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
    for more details.

    You should have received a copy of the GNU General Public License
    along with OpenFOAM.  If not, see <http://www.gnu.org/licenses/>.

\*---------------------------------------------------------------------------*/

#include "error.H"
#include "IStringStream.H"
#include "OStringStream.H"
#include "OSspecific.H"
#include "IFstream.H"
#include "ReadHex.H"

#include <cxxabi.h>
#include <execinfo.h>
#include <dlfcn.h>

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

namespace Foam
{

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

string pOpen(const string &cmd, label line=0)
{
    size_t linecap = 0;
    ssize_t linelen;
    char *buf = NULL;
    string res = "\n";

    FILE *cmdPipe = popen(cmd.c_str(), "r");

    if (cmdPipe)
    {
        // Read line number of lines
        for (label cnt = 0; cnt <= line; cnt++)
        {
            linelen = getline(&buf, &linecap, cmdPipe);

            if (linelen < 0)
                break;

            if (cnt == line)
            {
                res = string(buf);
                break;
            }
        }

        if (buf != NULL)
            free(buf);

        pclose(cmdPipe);
    }

    return res.substr(0, res.size() - 1);
}

inline word addr2word(long addr)
{
    static const size_t WORD_SIZE = 16 + 1; // from glibc
    static char buf[WORD_SIZE];
    snprintf(buf, WORD_SIZE, "%016lx", addr);

    return word(buf);
}


void printSourceFileAndLine
(
    Ostream& os,
    const fileName& filename,
    Dl_info *info,
    void *addr
)
{
    unsigned long address = reinterpret_cast<unsigned long>(addr);
    word myAddress = addr2word(address);

    if (filename.ext() == "so")
    {
        // Convert address into offset into dynamic library
        unsigned long offset =
            reinterpret_cast<unsigned long>(info->dli_fbase);
        long reladdr = address - offset;
        myAddress = addr2word(reladdr);
    }

    if (filename[0] == '/')
    {
        string line = pOpen
        (
            "addr2line -f --demangle=auto --exe "
          + filename
          + " "
          + myAddress,
            1
        );

        if (line == "")
        {
            os  << " addr2line failed";
        }
        else if (line == "??:0")
        {
            os  << " in " << filename;
        }
        else
        {
            string cwdLine(line.replaceAll(cwd() + '/', ""));
            string homeLine(cwdLine.replaceAll(home(), '~'));

            os  << " at " << homeLine.c_str();
        }
    }
}


fileName absolutePath(const char* fn)
{
    fileName fname(fn);

    if (fname[0] != '/' && fname[0] != '~')
    {
        string tmp = pOpen("which " + fname);

        if (tmp[0] == '/' || tmp[0] == '~')
            fname = tmp;
    }

    return fname;
}


word demangleSymbol(const char* sn)
{
    word res;
    int st;
    char* cxx_sname = abi::__cxa_demangle
    (
        sn,
        NULL,
        0,
        &st
    );

    if (st == 0 && cxx_sname)
    {
        res = word(cxx_sname);
        free(cxx_sname);
    }
    else
    {
        res = word(sn);
    }

    return res;
}

void error::safePrintStack(std::ostream& os)
{
    // Get raw stack symbols
    void *array[100];
    size_t size = backtrace(array, 100);
    char **strings = backtrace_symbols(array, size);

    // See if they contain function between () e.g. "(__libc_start_main+0xd0)"
    // and see if cplus_demangle can make sense of part before +
    for (size_t i = 0; i < size; i++)
    {
        string msg(strings[i]);
        fileName programFile;
        word address;

        os  << '#' << label(i) << '\t' << msg << std::endl;
    }
}

void error::printStack(Ostream& os)
{
    // Get raw stack symbols
    const size_t CALLSTACK_SIZE = 128;

    void *callstack[CALLSTACK_SIZE];
    size_t size = backtrace(callstack, CALLSTACK_SIZE);

    Dl_info *info = new Dl_info;

    fileName fname = "???";
    word address;

    for(size_t i = 0; i < size; i++)
    {
        int st = dladdr(callstack[i], info);

        os << '#' << label(i) << "  ";
        if (st != 0 && info->dli_fname != NULL && info->dli_fname[0] != '\0')
        {
            fname = absolutePath(info->dli_fname);

            os << ((info->dli_sname != NULL) ?
                demangleSymbol(info->dli_sname) : "?");
        }
        else
        {
            os << "?";
        }

        printSourceFileAndLine(os, fname, info, callstack[i]);
        os << nl;
    }

    delete info;
}


// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

} // End namespace Foam

// ************************************************************************* //
printStack.C (5,648 bytes)   

alexeym

2015-04-28 14:59

reporter  

alexeym

2015-04-28 14:59

reporter  

alexeym

2015-04-28 15:00

reporter  

printStack.log.new.Opt (2,107 bytes)

alexeym

2015-04-28 15:00

reporter  

henry

2015-04-29 09:59

manager   ~0004688

Thanks for the update of printStack. Could you clarify what the advantage of the method is? Looking at the logs you provided:

printStack.log.orig.Opt
#7
 at ??:?
#8
 at ??:?
#9 __libc_start_main in "/lib64/libc.so.6"


printStack.log.new.Opt

#7 ? at ??:?
#8 ? at ??:?
#9 __libc_start_main in "/usr/lib64/libc.so.6"

the only difference appears to be the "/usr" in the new log other than the spurious newlines in the old log.

alexeym

2015-04-29 13:19

reporter   ~0004690

1. As you have mentioned, symbol parsing errors are gone. I.e. there will be no more spurious new lines during printStack.

2. Old procedure is redundant

backtrace -> backtrace_symbols (dladdr -> sprintf -> string) -> parse address from string -> dladdr -> execute addr2line

new one is shorter

backtrace -> dladdr -> execute addr2line

3. Removed unused hash table addressMap.

4. As dladdr is used directly, new method is protected from possible future changes of backtrace_symbols sprintf format.

5. In fact I did not pay attention to absent "/usr" part. So old version also had problems with file name parsing. New version does not have them.

6. Due to exclusion of redundant steps new version is shorter and clearer (though here I can be biased).

henry

2015-04-29 13:22

manager   ~0004691

Thanks for the details. I will test you new method on a range of GNU/Linux distributions here.

henry

2015-05-19 08:46

manager   ~0004774

Thanks for the patch and explanations. I have tested it on several systems with several compilers and it works well.

I have included it in OpenFOAM-2.3.x:
commit 3354ee43504b60fd6223ed0ce51b70758df627f7

and OpenFOAM-dev:
commit 218dafd76b9ba53bb2f6a7344489714a6cede629

Issue History

Date Modified Username Field Change
2015-04-28 14:59 alexeym New Issue
2015-04-28 14:59 alexeym File Added: printStack.C
2015-04-28 14:59 alexeym File Added: printStack.log.orig.Opt
2015-04-28 14:59 alexeym File Added: printStack.log.orig.Debug
2015-04-28 15:00 alexeym File Added: printStack.log.new.Opt
2015-04-28 15:00 alexeym File Added: printStack.log.new.Debug
2015-04-29 09:59 henry Note Added: 0004688
2015-04-29 13:19 alexeym Note Added: 0004690
2015-04-29 13:22 henry Note Added: 0004691
2015-05-19 08:46 henry Note Added: 0004774
2015-05-19 08:46 henry Status new => resolved
2015-05-19 08:46 henry Resolution open => fixed
2015-05-19 08:46 henry Assigned To => henry