Thread: Tracking down some undefined behavior

  1. #16
    Registered User
    Join Date
    Jun 2009
    Posts
    486
    Thanks grumpy. I added a trialing 0 explicitly to the string to be on the safe side.

    And in happy news, I found the bug! In the filter_signal function, there were rare conditions with very short events in which length could be less than order, resulting in attempting to use negative array indices and trashing some other memory. I've fixed things and it seems to be up and running again. I think I will take this opportunity to do some code cleanup and some more careful index management, as it's getting rather messy and that sort of thing should never have happened if I was being careful.

    Thanks guys. Especially for prompting me to learn to use a debugger.
    C is fun

  2. #17
    Registered User
    Join Date
    Jun 2009
    Posts
    486
    On to the next memory bug. Truly amazing how many of these were hiding behind wonky input.

    Here is my free_events() function, which takes the head of the linked list and frees the entire list.

    Code:
    void free_events(event *current)
    {
        while (current->next)
        {
            current = current->next;
            free(current->prev->signal);
            free(current->prev->filtered_signal);
            free(current->prev->filtered_level_histogram);
            free(current->prev->level_histogram);
            free(current->prev->peaks);
            free(current->prev);
        }
        free(current->signal);
        free(current->filtered_signal);
        free(current->filtered_level_histogram);
        free(current->level_histogram);
        free(current->peaks);
        free(current);
    }
    here is the event struct itself:
    Code:
    struct Event
    {
        int index;
        long long int start;
        long long int finish;
        long int length;
        int type;
        double area;
        double open_current;
        double *signal;
        double *filtered_signal;
        long int *level_histogram;
        long int *filtered_level_histogram;
        double *peaks;
        int numpeaks;
        double binsize;
        long int padding;
        struct Event *next;
        struct Event *prev;
    };
    typedef struct Event event;
    At the end of my program I free the linked list of events. And when I do, it hangs on the call free(current->filtered_level_histogram). I don't think this is important in an of itself, it's just the indication that there are memory issues. I have verified that the memory is allocated properly, so it's getting trashed somewhere in the middle. Indeed, at this point the program is done and the output is correct, so as far as I can tell it doesn't harm anything (yet). But it means there are still pointer issues which could bite me later, so I need to find it.

    Turns out that the problem boils down to a single line. If I comment it out (and make some minor changes elsewhere so that its intended result isn't missed) the problem disappears. The line is below:

    Code:
    void generate_trace(FILE *input, event *current, double *dcof, int *ccof, double scale, int order)
    {
        long int padding;
        long long int position;
        long int read;
        double *tempfiltered;
    
        padding = order+1;
        current->padding = padding;
        position = current->start - padding;
    
        if ((current->signal = calloc(current->length + 2*padding,sizeof(double)))==NULL)
        {
            printf("Cannot allocate trace array\n");
            return;
        }
        if ((current->filtered_signal = calloc(current->length + 2*padding,sizeof(double)))==NULL)
        {
            printf("Cannot allocate filtered trace array\n");
            return;
        }
        if ((tempfiltered = calloc(current->length + 2*padding,sizeof(double)))==NULL)
        {
            printf("Cannot allocate temporary filtered trace array\n");
            return;
        }
    
    
        if (fseeko64(input,position*2*sizeof(double),SEEK_SET))
        {
            printf("Cannot location file position at sample %lld\n",position);
            return;
        }
    
        read = read_current(input, current->signal, position, current->length + 2*padding);
        if (read != current->length + 2*padding)
        {
            printf("Unable to read %ld samples for event %d: obtained %ld\n",current->length + 2*padding,current->index,read);
        }
        printf("filtering\n");
        filter_signal(current->signal, tempfiltered, current->filtered_signal, dcof, ccof, scale, order, read);
        printf("done filtering\n");
        free(tempfiltered);
    }
    Finally, the read_current() function is here, whicih reads from a binary file and swaps the endianness. Note that this function is used elsewhere in the code, without issue. So something about that particular line is causing what I am guessing is going out of bounds on an array.

    Code:
    long int read_current(FILE *input, double *current, long long int position, long int length)
    {
        union{
            double d;
            unsigned long long int i;
        }ud;
    
        long int i;
        long int read = 0;
    
        if (fseeko64(input,position*2*sizeof(double),SEEK_SET))
        {
            return 0; //error checking
        }//move to the relevent position in the file
    
    
        for (i = 0; i < length; i++)
        {
            read += fscanf(input,"%8c%*8c",(char *)&ud.d);
            swapByteOrder(&ud.i);
            current[i] = ud.d;
        }
        return read;
    }
    Again, the output is correct, as far as I can tell. But some memory is getting trashed somewhere. Any ideas?
    C is fun

  3. #18
    Registered User
    Join Date
    Nov 2010
    Location
    Long Beach, CA
    Posts
    5,909
    I don't see any potential for memory trashing in read_current. The only potential issue I see is that you don't check the value of fscanf, so if there's some error reading from the file, you fill current with bad data. Something like
    Code:
    retval = fscanf();
    if (retval != 1) {
        if feof(input)
            file missing data
        else if ferror(input)
            perror("Error reading file")
        possibly return 0
    }
    read += retval
    swapByteOrder
    current[i] = ud.d;
    Last edited by anduril462; 03-27-2014 at 11:27 AM.

  4. #19
    Registered User
    Join Date
    Jun 2009
    Posts
    486
    So is there any information to be had in the fact that the issue disappears when read_current is not called there? Surely that is more informative than just "undefined behavior" ^_^?
    C is fun

  5. #20
    Registered User
    Join Date
    Nov 2010
    Location
    Long Beach, CA
    Posts
    5,909
    What does your new filter_signal() function look like? It's possible that there's still a bug in there that trashes some function parameters or local variables. That might only happen if read_current is called, since read_current() uses local variables which will end up changing the contents of memory in the same spot on the stack that is used by other functions called from generate_trace. Thus, filter_signal() may have different initial values in it's local variables based on whether or not read_current is called. A bug like an uninitialized local variable in filter_signal() could manifest because of that.

  6. #21
    Registered User
    Join Date
    Jun 2009
    Posts
    486
    filter_signal() was actually unchanged, what changed was that I added some error checking to make sure the array was always big enough to avoid the issues I described earlier (this was done in generate_trace()). I will have a look through that, though.

    Note that the crash is still present if I don't call filter_signal() at all, but I do call read_current().

    Starting to think I should rebuild this from the bottom up and do more thorough testing at each step. It's taking about as much time to debug memory issues as it did to determine the algorithms I needed originally.
    Last edited by KBriggs; 03-27-2014 at 02:07 PM.
    C is fun

  7. #22
    Registered User
    Join Date
    Nov 2010
    Location
    Long Beach, CA
    Posts
    5,909
    Quote Originally Posted by KBriggs View Post
    Starting to think I should rebuild this from the bottom up and do more thorough testing at each step. It's taking about as much time to debug memory issues as it did to determine the algorithms I needed originally.
    Starting over may be the right call -- you know the effort it took to get you here, how much debugging you're doing, etc. Don't forget though, you will have bugs in the new code too, so not all debugging time will go away.
    Working in small increments and testing often is always the right call.
    But before you start over, have you tried something like Electric Fence? Considering how much dynamic memory you use, it will probably be very helpful to debugging these issues, and it's very simple to use. Also, run your code through Valgrind, which is another useful tool (actually it's more a suite of many useful tools). Valgrind can be complicated, but the most basic memory checks are fairly simple, and examples abound on the interwebs.

  8. #23
    Registered User
    Join Date
    Jun 2009
    Posts
    486
    Those are both Linux tools, no? Google turns up some windows alternatives, or I guess I could just boot up the old linux box at home and do it there.

    Codeblocks must have something similar built in, I'll have a poke around and see what comes up. If nothing else I'm learning quite a bit about my IDE while doing this.
    C is fun

  9. #24
    Registered User
    Join Date
    Nov 2010
    Location
    Long Beach, CA
    Posts
    5,909
    Quote Originally Posted by KBriggs View Post
    Those are both Linux tools, no?
    Yes.
    Quote Originally Posted by KBriggs View Post
    Google turns up some windows alternatives, or I guess I could just boot up the old linux box at home and do it there.
    Never used the Windows alternatives, though I'm sure they exist. If you can't get the Windows alternatives working, then I highly recommend booting up the Linux box. Besides, compiling and testing on Linux may help you find other bugs that are masked by some quirk of Windows or the implementation you're using, but manifest on a different platform/OS/implementation.
    Quote Originally Posted by KBriggs View Post
    Codeblocks must have something similar built in, I'll have a poke around and see what comes up. If nothing else I'm learning quite a bit about my IDE while doing this.
    Doubtful. C::B is just an IDE. It's an editor that interfaces with a compiler, debugger, etc, which are external tools (e.g. MinGW). Electric Fence is just a library you link with (on Linux at least), so that is super easy, you just install the library, then link your program with -lefence. Valgrind is a program, which you run your program through.

  10. #25
    Registered User
    Join Date
    Jun 2009
    Posts
    486
    C::B has cppcheck built in as a plugin, which I managed to get working - it doesn't pick up anything important. Doesn't necessarily mean there's nothing there, though, and it's a static tool so it wouldn't be able to give the same level of information as the others you mentioned.

    Might just move to linux. I'll have to update my operating system to do it, my distribution is deprecated now. It's always something ^_^.

    Thanks for your help. I'll keep poking at it, but at this point I think my best option is to rebuild. There are a number of things I know I could design better now that I have the logic sorted out, so it's probably worth it to do a redesign of how it all fits together anyway.
    Last edited by KBriggs; 03-27-2014 at 02:54 PM.
    C is fun

  11. #26
    TEIAM - problem solved
    Join Date
    Apr 2012
    Location
    Melbourne Australia
    Posts
    1,907
    "std::cerr" and "std::string's" - Is it possible that Grumpy is getting the C and C++ board mixed up :P
    Fact - Beethoven wrote his first symphony in C

  12. #27
    Registered User
    Join Date
    Jun 2005
    Posts
    6,815
    Quote Originally Posted by KBriggs View Post
    Thanks grumpy. I added a trialing 0 explicitly to the string to be on the safe side.
    You missed my point. Adding a trailing zero is not sufficient to prevent the bug I described.
    Right 98% of the time, and don't care about the other 3%.

    If I seem grumpy or unhelpful in reply to you, or tell you you need to demonstrate more effort before you can expect help, it is likely you deserve it. Suck it up, Buttercup, and read this, this, and this before posting again.

  13. #28
    Registered User
    Join Date
    Jun 2009
    Posts
    486
    I added a trailing 0 after copying the string (eg, your "paranoid" approach). It risks overwriting the last character of the string, but in that case the program would just say it couldn't find the input file and exit gracefully instead of corrupting other memory.

    Is that not sufficient?
    C is fun

  14. #29
    Registered User
    Join Date
    Jun 2005
    Posts
    6,815
    No, it is not. There were two points of the "paranoid" approach, and you got the least significant of them right. The first (which you missed) is that the number of characters copied to the destination should not exceed the capacity of the destination.


    The point of strncpy() is to limit the number of characters copied to the destination, not to specify the number of characters available to be copied from the source.

    You have done "strncpy(destination, source, strlen(source))". That does nothing to prevent overrunning the destination array.
    Right 98% of the time, and don't care about the other 3%.

    If I seem grumpy or unhelpful in reply to you, or tell you you need to demonstrate more effort before you can expect help, it is likely you deserve it. Suck it up, Buttercup, and read this, this, and this before posting again.

  15. #30
    Registered User
    Join Date
    Jun 2009
    Posts
    486
    Oh sorry, I had changed strlen(source) to STRLENGTH-1 as well. I thought I posted that bit of code again above, but apparently not.
    C is fun

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Undefined behavior
    By Ducky in forum C Programming
    Replies: 14
    Last Post: 11-17-2013, 07:10 AM
  2. Is this not undefined behavior?
    By Syscal in forum C Programming
    Replies: 6
    Last Post: 07-15-2013, 01:07 AM
  3. Undefined behavior
    By jim mcnamara in forum C Programming
    Replies: 2
    Last Post: 02-18-2013, 11:14 PM
  4. Is x=x++; Undefined Behavior?
    By envec83 in forum C Programming
    Replies: 5
    Last Post: 10-04-2011, 01:27 AM
  5. Undefined behavior from VC6 to 2k5
    By m37h0d in forum C++ Programming
    Replies: 10
    Last Post: 06-22-2011, 07:56 PM

Tags for this Thread