Thread: Aligning memory

  1. #16
    Registered User awsdert's Avatar
    Join Date
    Jan 2015
    Posts
    839
    Managed to seemingly fix that issue by switching out the 1st a's for 1's, now I have what I believe the final issue at hand it it just confounds me:
    Code:
    			for ( a = 1,
    				spot = page->init;
    				spot < page->last;
    				++a, spot = &(spot[1])
    			)
    			{
    				if ( _data->data == spot->data.data ) break;
    			}
    Somehow spot manages to actually go BELOW page->init value (found that out with an extra condition there), the highest I've seen the value itself at is a little under 0x7fffffffffff, even if I consider overflow it should not do that because of the spot < page->last statement, any ideas peops?

  2. #17
    Registered User awsdert's Avatar
    Join Date
    Jan 2015
    Posts
    839
    Well I seem to have fixed the segfaults at least, I changed all the "!= page->last" to "< page->last" and change that memove line to
    Code:
    if ( (spot = &(page->init[1])) < page->last )
    	(void)memmove( spot, page->init, (uintptr_t)(page->last - spot) );
    which admittedly is easier to read anyways.
    Now I'm just getting an invalid value somewhere, or at least incorrectly detected as invalid value anyways, a leak is better than a segfault but still gonna try and track down the cause

  3. #18
    Registered User awsdert's Avatar
    Join Date
    Jan 2015
    Posts
    839
    I give up for now, gonna take a break and play some games or read or something, if anyone wanna take a crack at it then here it is, it runs through to then end just fine, it just fails to find memory because the locations are somehow being overridden with 0
    Attached Files Attached Files

  4. #19
    Registered User awsdert's Avatar
    Join Date
    Jan 2015
    Posts
    839
    Found out why, turns out memmove was not doing it's job and moving data up, after some investigating I found that the pointer math was not doing as expected, apparently I had to cast them individually first, really annoying that is, they're still stored as integers at the end of the day, basic integer math should work, whoever though it was a decent idea to say otherwise needs good wack round the head.

  5. #20
    C++ Witch laserlight's Avatar
    Join Date
    Oct 2003
    Location
    Singapore
    Posts
    27,506
    Quote Originally Posted by awsdert
    Found out why, turns out memmove was not doing it's job and moving data up, after some investigating I found that the pointer math was not doing as expected, apparently I had to cast them individually first, really annoying that is, they're still stored as integers at the end of the day, basic integer math should work, whoever though it was a decent idea to say otherwise needs good wack round the head.
    I'm puzzled: aren't you always passing pointers to memmove? The possible issue I see is that in two places you don't seem to be correctly computing the number of bytes to move (i.e., you didn't multiply by sizeof(std_spot)), but that is as you say, basic integer math that you simply failed to do.
    Quote Originally Posted by Bjarne Stroustrup (2000-10-14)
    I get maybe two dozen requests for help with some sort of programming or design problem every day. Most have more sense than to send me hundreds of lines of code. If they do, I ask them to find the smallest example that exhibits the problem and send me that. Mostly, they then find the error themselves. "Finding the smallest program that demonstrates the error" is a powerful debugging tool.
    Look up a C++ Reference and learn How To Ask Questions The Smart Way

  6. #21
    Registered User
    Join Date
    May 2012
    Location
    Arizona, USA
    Posts
    701
    Quote Originally Posted by awsdert View Post
    Code:
                for ( a = 1,
                    spot = page->init;
                    spot < page->last;
                    ++a, spot = &(spot[1])
                )
                {
                    if ( _data->data == spot->data.data ) break;
                }
    Why the obscure code? spot = &(spot[1]) is equivalent to ++spot, but the latter is (IMO) easier to read.

  7. #22
    Registered User awsdert's Avatar
    Join Date
    Jan 2015
    Posts
    839
    As it turns out there was one more nasty bug hiding, it causes a segfault when lua or glut attempt to remove something, well glut might be complaining due to my attempt to retrieve a localized id from the duplicated functions so I'll leave that aside for another thread once this is dealt with, currently I can only think the error occurs in this code:
    Code:
    		if ( spot->upto - spot->from >= want ) {
    			spot->data.size = want;
    			if ( want > 0 ) {
    				_data->size = want;
    				_data->data = spot->data.data;
    				return EXIT_SUCCESS;
    			}
    			_data->size = 0;
    			_data->data = NULL;
    			page->last->upto += sizeof(std_spot);
    			init = (std_spot*)
    				(&(((uchar*)(page->last->data.data))[page->last->upto]));
    			if ( init < page->last )
    				(void)memmove(
    					init, page->init,
    					((uintptr_t)spot - (uintptr_t)init) );
    			page->init = init;
    			if ( spot >= page->last )
    				(void)alloc_foo_page( &p, 0 );
    			return EXIT_SUCCESS;
    		}
    The other code is unlikely to cause issue unless memmove clear the memory it leaves behind, in which case a simple fix would be holding a copy of the last spot object to override with straight after the movement, but the code above is not so easy to fix without bringing in multiple variables, any ideas before I resort to that?

    Edit: Yeah, that other one definitely weren't it.
    Last edited by awsdert; 01-06-2020 at 03:37 PM.

  8. #23
    Registered User awsdert's Avatar
    Join Date
    Jan 2015
    Posts
    839
    Quote Originally Posted by laserlight View Post
    I'm puzzled: aren't you always passing pointers to memmove? The possible issue I see is that in two places you don't seem to be correctly computing the number of bytes to move (i.e., you didn't multiply by sizeof(std_spot)), but that is as you say, basic integer math that you simply failed to do.
    I had the right math there, it's just clang didn't put the math there, instead it boiled it down to a comparison or something because the values that where at least 0x10 apart gave 1 on the instance I caught it (after passing it into want before hand), only after I cast them both to uintptr_t beforehand did I get the right output

  9. #24
    Registered User awsdert's Avatar
    Join Date
    Jan 2015
    Posts
    839
    Quote Originally Posted by christop View Post
    Why the obscure code? spot = &(spot[1]) is equivalent to ++spot, but the latter is (IMO) easier to read.
    Be cause I can only be sure the compiler will provide the correct assembly like that, ++spot is misleading and can instead cause the address to increment by only 1 byte whereas the spot structure is definitely more than one byte

  10. #25
    C++ Witch laserlight's Avatar
    Join Date
    Oct 2003
    Location
    Singapore
    Posts
    27,506
    Oh, now I understand. You complained about having to "cast them individually first" because you're taking the approach of treating the pointers as integers, and hence have to unnecessarily cast the pointers to uintptr_t in order to compute the number of bytes by integer subtraction. Instead of:
    Code:
    (void)memmove(
                init, page->init,
                ((uintptr_t)spot - (uintptr_t)init) );
    The more natural way to use memmove is to compute the number of bytes by pointer subtraction, multiplying by the size (in bytes) of an object of the given type. For example:
    Code:
    (void)memmove(
                init, page->init,
                (spot - init) * sizeof(*spot));
    After all, the type of the third parameter of memmove is size_t, which is the same as the type of the result of sizeof. The result of the multiplication might be ptrdiff_t though (I don't remember the rules exactly), but if you want to be pedantic (as I see you cast the return value to void when unused) you can always cast: (size_t)(spot - init) * sizeof(*spot).

    EDIT:
    Quote Originally Posted by awsdert
    I had the right math there, it's just clang didn't put the math there, instead it boiled it down to a comparison or something because the values that where at least 0x10 apart gave 1 on the instance I caught it (after passing it into want before hand), only after I cast them both to uintptr_t beforehand did I get the right output
    It's not clang's fault. Your math was wrong because you're programming in C, and in C pointer subtraction is not integer subtraction of addresses because pointer subtraction incorporates multiples of the size of the type when subtracting the addresses, whereas integer subtraction of addresses does not.

    If we didn't have this, then to do even a simple subtraction of two pointers to different elements of an array would force you to divide by sizeof(type), i.e., you won't get the benefit of the pointer abstraction. In your case, by using memmove for bytes, you are deliberately forgoing the pointer abstraction to treat the memory as a contiguous sequence of bytes rather than a contiguous sequence of objects of some type in memory, hence you have to multiply by sizeof(type) after doing pointer subtraction, or cast in order to use integer subtraction of addresses.
    Last edited by laserlight; 01-06-2020 at 03:57 PM.
    Quote Originally Posted by Bjarne Stroustrup (2000-10-14)
    I get maybe two dozen requests for help with some sort of programming or design problem every day. Most have more sense than to send me hundreds of lines of code. If they do, I ask them to find the smallest example that exhibits the problem and send me that. Mostly, they then find the error themselves. "Finding the smallest program that demonstrates the error" is a powerful debugging tool.
    Look up a C++ Reference and learn How To Ask Questions The Smart Way

  11. #26
    Registered User
    Join Date
    May 2012
    Location
    Arizona, USA
    Posts
    701
    Quote Originally Posted by awsdert View Post
    Be cause I can only be sure the compiler will provide the correct assembly like that, ++spot is misleading and can instead cause the address to increment by only 1 byte whereas the spot structure is definitely more than one byte
    You can rest assured that incrementing a pointer adds the size of the pointed-to object to the pointer. It's guaranteed by the standards, and it's common practice to use something like ++p to increment p to the next object in an array, regardless of the type of *p.

    Consider the following equivalent statements:

    Code:
    spot = &(spot[1]);
    spot = &(1[spot]); // yes, the array index operator is commutative!
    spot = spot + 1;
    spot = 1 + spot;
    ++spot;

  12. #27
    C++ Witch laserlight's Avatar
    Join Date
    Oct 2003
    Location
    Singapore
    Posts
    27,506
    Quote Originally Posted by awsdert
    Be cause I can only be sure the compiler will provide the correct assembly like that, ++spot is misleading and can instead cause the address to increment by only 1 byte whereas the spot structure is definitely more than one byte
    christop is right: spot = &(spot[1]) is equivalent to ++spot. If the compiler does not provide the correct assembly, then the compiler has a bug.

    spot[1] is equivalent to *(spot + 1). Hence &(spot[1]) is equivalent to &(*(spot + 1)). The address-of and dereference cancel each other out, so that is equivalent to spot + 1. Hence, spot = &(spot[1]) is equivalent to spot = spot + 1, which is obviously equivalent to ++spot since the expression is standalone.
    Quote Originally Posted by Bjarne Stroustrup (2000-10-14)
    I get maybe two dozen requests for help with some sort of programming or design problem every day. Most have more sense than to send me hundreds of lines of code. If they do, I ask them to find the smallest example that exhibits the problem and send me that. Mostly, they then find the error themselves. "Finding the smallest program that demonstrates the error" is a powerful debugging tool.
    Look up a C++ Reference and learn How To Ask Questions The Smart Way

  13. #28
    Registered User awsdert's Avatar
    Join Date
    Jan 2015
    Posts
    839
    Quote Originally Posted by laserlight View Post
    ...but if you want to be pedantic (as I see you cast the return value to void when unused)
    Only do that on the memory being returned since that is same behaviour as malloc, I just need the execution permission in addition to read/write (wait for next thread to get clearer idea about that purpose(after this bug is resolved))
    Quote Originally Posted by laserlight View Post
    you can always cast: (size_t)(spot - init) * sizeof(*spot).
    tried that with uintptr_t at 1st, as I said before the result was not what I expected from integer math so I had to do them individually, reason I don't use size_t is because I didn't want information being lost before the math occurs, the result will always fit inside size_t so I'm not worried about that but on other systems it is possible it would loose info so forcing the subtraction to the right size is better option
    Quote Originally Posted by laserlight View Post
    If we didn't have this, then to do even a simple subtraction of two pointers to different elements of an array would force you to divide by sizeof(type), i.e., you won't get the benefit of the pointer abstraction. In your case, by using memmove for bytes, you are deliberately forgoing the pointer abstraction to treat the memory as a contiguous sequence of bytes rather than a contiguous sequence of objects of some type in memory, hence you have to multiply by sizeof(type) after doing pointer subtraction, or cast in order to use integer subtraction of addresses.
    But that's what you keep indexs for, they're supposed to used for those calculations then you just reference the node from the root, because it would be treated as an array at that point the compiler would have to apply the sizeof in the result without it being explicitly written with bytes. Well either way its results that matter to me not semantics.

  14. #29
    C++ Witch laserlight's Avatar
    Join Date
    Oct 2003
    Location
    Singapore
    Posts
    27,506
    Quote Originally Posted by awsdert
    reason I don't use size_t is because I didn't want information being lost before the math occurs, the result will always fit inside size_t so I'm not worried about that but on other systems it is possible it would loose info so forcing the subtraction to the right size is better option
    You're simply mistaken: no information will be lost. Pointer subtraction is idiomatic in C, so no, the casts are not a better option.

    Quote Originally Posted by awsdert
    But that's what you keep indexs for, they're supposed to used for those calculations then you just reference the node from the root, because it would be treated as an array at that point the compiler would have to apply the sizeof in the result without it being explicitly written with bytes.
    That's also what pointer arithmetic is for. Pointers are an incredibly common abstraction in C.
    Quote Originally Posted by Bjarne Stroustrup (2000-10-14)
    I get maybe two dozen requests for help with some sort of programming or design problem every day. Most have more sense than to send me hundreds of lines of code. If they do, I ask them to find the smallest example that exhibits the problem and send me that. Mostly, they then find the error themselves. "Finding the smallest program that demonstrates the error" is a powerful debugging tool.
    Look up a C++ Reference and learn How To Ask Questions The Smart Way

  15. #30
    misoturbutc Hodor's Avatar
    Join Date
    Nov 2013
    Posts
    1,737
    Have you tried using valgrind and static analysis?


    Code:
    valgrind --tool=memcheck --track-origins=yes ./alloc
    
    ==9754== Conditional jump or move depends on uninitialised value(s)
    ==9754==    at 0x4D00397: mmap (in /usr/lib64/libc-2.29.so)
    ==9754==    by 0x4014FA: alloc_std_page (alloc.c:113)
    ==9754==    by 0x40180E: alloc_foo_page (alloc.c:177)
    ==9754==    by 0x401FD9: main (alloc.c:322)
    ==9754==  Uninitialised value was created by a heap allocation
    ==9754==    at 0x483880B: malloc (vg_replace_malloc.c:309)
    ==9754==    by 0x4012F6: alloc_std_data (alloc.c:51)
    ==9754==    by 0x4013EC: alloc_std_list (alloc.c:78)
    ==9754==    by 0x40177A: alloc_foo_page (alloc.c:164)
    ==9754==    by 0x401FD9: main (alloc.c:322)
    ==9754==
    ==9754== Syscall param mmap(start) contains uninitialised byte(s)
    ==9754==    at 0x4D003A6: mmap (in /usr/lib64/libc-2.29.so)
    ==9754==    by 0x4014FA: alloc_std_page (alloc.c:113)
    ==9754==    by 0x40180E: alloc_foo_page (alloc.c:177)
    ==9754==    by 0x401FD9: main (alloc.c:322)
    ==9754==  Uninitialised value was created by a heap allocation
    ==9754==    at 0x483880B: malloc (vg_replace_malloc.c:309)
    ==9754==    by 0x4012F6: alloc_std_data (alloc.c:51)
    ==9754==    by 0x4013EC: alloc_std_list (alloc.c:78)
    ==9754==    by 0x40177A: alloc_foo_page (alloc.c:164)
    ==9754==    by 0x401FD9: main (alloc.c:322)
    ==9754==
    Lots of messages from clang, but these are the ones that stood out, to me, as being potentially problematic
    Code:
    alloc.c:223:14: Semantic Issue: warning: variable 'spot' may be uninitialized when used here
    alloc.c:232:28: Semantic Issue: warning: variable 'page' may be uninitialized when used here
    alloc.c:272:20: Semantic Issue: warning: cast from 'uchar *' (aka 'unsigned char *') to 'std_spot *' (aka 'struct spot *') increases required alignment from 1 to 8
    alloc.c:287:25: Semantic Issue: warning: cast from 'uchar *' (aka 'unsigned char *') to 'std_spot *' (aka 'struct spot *') increases required alignment from 1 to 8

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Help With Aligning
    By M_A_T_T in forum C Programming
    Replies: 2
    Last Post: 02-13-2013, 05:44 AM
  2. Right aligning.
    By theCanuck in forum C++ Programming
    Replies: 2
    Last Post: 02-12-2011, 12:50 PM
  3. Aligning Columns
    By mike_g in forum C Programming
    Replies: 3
    Last Post: 06-27-2007, 08:27 AM
  4. Aligning text
    By Bad_Scooter in forum C++ Programming
    Replies: 5
    Last Post: 06-04-2003, 12:06 PM

Tags for this Thread