use of printf prevents segfault!

MK27 · 08-26-2008, 12:53 AM

I'm writing a function that takes a file, reads the file into a buffer while keeping count of the lines, then takes each line and plugs the content into a simple structure. I did it that way because the number of lines is variable and will determine, obviously, the number of structs needed. So I get the number of lines (nofl) and in the middle of the function do this:

struct thestruct new[nofl];

I was getting a segfault, so to make sure nofl was correct, i inserted the following line RIGHT BEFORE the last one:

printf("--%d--\n", nofl);

AND NOW NO SEGFAULT!!! Nothing else is changed, but adding and subtracting that one line makes all the difference. Really, really, really. More observations:
1) Up until now I've been declaring variables (inc. structs) at the beginning of a function, but I've never heard tell that it must be so. Does this have anything to do with it?
2) The segfault actually occurs later when new[1] (ie. the 2nd struct in the array) is called on (it turns out not to exist).
3) "nofl" is determined on the line immediately preceeding the printf statement.
4) if I put the printf statement after the struct declaration, the segfault returns.

C_ntua · 08-26-2008, 01:22 AM

You could post the code to make things more clear (if that's possible).
1) You can declare variables in the middle like you do with new[] only with the C99 standard (i think)
2) So a seg fault occurs after all, so the printf doesn't actually solve the problem, eh?
3) As long as it is before it is ok
4) I ll admit I had similar occasions that printf changed the seg fault. But in any case it doesn't solve the problem. Post the code to find exactly the problem.

QuantumPete · 08-26-2008, 02:05 AM

Sounds like a classic stack corruption. Can you post your code?

QuantumPete

matsp · 08-26-2008, 03:25 AM

Quote:

Originally Posted by QuantumPete

Sounds like a classic stack corruption. Can you post your code?

QuantumPete

Or quite possibly writing over the end of your allocated data at some point previously.

When problems change due to "irrelevant" changes in the code, it's usually because you are going across the line of some memory - adding/changing the code around will move the actual bits of memory used by different pieces of code, so the problem "moves", but it's actually not fixed, it's just overwriting something less harmful.

For example, calling printf will make sure the variable is stored on the stack, whilst not calling printf will not use the stack to store the local variable - so some bytes less used on the stack, so when you overwrite the end of an array, it no longer overwrites the variable, but some important data [such as the frame pointer or return address of the function].

--
Mats

QuantumPete · 08-26-2008, 04:22 AM

Quote:

Originally Posted by matsp

Or quite possibly writing over the end of your allocated data at some point previously.

My money is still on stack corruption

QuantumPete

matsp · 08-26-2008, 04:32 AM

Quote:

Originally Posted by QuantumPete

My money is still on stack corruption

QuantumPete

Not at all saying you are wrong. Only reason I said overwriting dynamically allocated memory is that it's happening in new - but that's no good indication really.

--
Mats

QuantumPete · 08-26-2008, 04:56 AM

Quote:

Originally Posted by matsp

it's happening in new

It looks to me like the variable is called new (how that got by the compiler I don't know), no dynamic allocation.

QuantumPete

matsp · 08-26-2008, 04:57 AM

Quote:

Originally Posted by QuantumPete

It looks to me like the variable is called new (how that got by the compiler I don't know), no dynamic allocation.

QuantumPete

Doh! Yes, since it's the C (rather than C++) forum, then it makes sense - and you're quite right. Must clean my glasses or put on my reading head or something ;-)

--
Mats

MK27 · 08-26-2008, 08:01 AM

Sorry I didn't post the original code, it seemed tedious and I wanted an answer to the printf question, since that was clearly the "crux". b/t/w "new" is not the actual name of the variable...

"Sounds like a classic stack corruption." -- QuantumPete
If stack corruption is what causes compiler output including this:
======= Backtrace: =========
[...]
======= Memory map: ========
Then no, it's not stack corruption.

"For example, calling printf will make sure the variable is stored on the stack, whilst not calling printf will not use the stack to store the local variable" -- matsp
Admittedly I know very little about this, but it was my understanding that an uninitialized int goes into a stack frame and stays there after being initialized. However, yours is the only explanation I see.

"so some bytes less used on the stack, so when you overwrite the end of an array," -- matsp
This pair of phrases does not make sense to me. Could you clarify?

Anyway, in the end what solved the problem is just splitting the function so that the array of structs is defined at a beginning. I am pretty sure GNU glibc is C89, not C99? How this relates to the overwrite (assuming matsp is correct) I don't know.

However, that now means the struct array is outside main! But I guess there's a different problem...thanks for all your thoughts.

matsp · 08-26-2008, 08:16 AM

Unfortunately, stack corruption can cause all sorts of things - backtrace assumes that you end up in some sort of error handler in the C-library - this doesn't always happen for stack corruption, because the code that prints the "backtrace" is using the stack to produce that backtrace, and if the stack is sufficiently messsed up, then it will not be able to figure out what to produce and give up [or crash]. (For example, if you write NULL to the frame-pointer [1] then it will most likely never find your backtrace)

And to answer your direct questions:
A local variable doesn't have to take up space on the stack. It is entirely possible, under the right circumstances, for the compiler to only ever hold the value in a register - in which case the variable has no stack-space assigned to it.

Whether glibc is written in C89, C99 or some other C standard is pretty irrelevant - your code is what causes this problem. Splitting it into two functions should not fix any problem like this, but it may of course HIDE the problem.

Without seeing the actual code, it's pretty hard to know what is right and what is wrong with it. It's like talking to a mechanic on the phone about a car that has some sort of problem. The mechanic can make a guess as to what the problem could be, but it's certainly a lot easier to diagnose the problem when the mechanic and the car are next to each other.

[1]Frame-pointer is a pointer to the previous stack-frame that is maintained by the compiler so that a debugger or debug help function like the backtrace can figure out how you got there.

--
Mats

MK27 · 08-26-2008, 08:51 AM

The code is just too long and I'm not going to bother trying to recreate the problem on a smaller scale since it now works, altho I completely agree with that point. FYI my problem started with me not knowing I could do this:

Code:

struct thestruct {
[...]
} ray[];
int main () {

to create a global array of structs that will grow "dynamically". But I still want to ask you...

"Whether glibc is written in C89, C99 or some other C standard is pretty irrelevant"
So what did C_ntua mean by "You can declare variables in the middle like you do with new[] only with the C99 standard (i think)"? Where would the problem be hiding once the segfault is gone and everything works smoothly, if my code must still be flawed? Can I ever rid myself of such a hidden flaw? Perhaps the overwrite was symptomatic of violating the standard.

About the mysteries of memory allocation: by "stored in a register" you mean the variable is retained in the CPU and never allocated memory? If "printf" causes it to be put on the stack, shouldn't "struct thestruct Knew[nofl]" also? In which case doing this in the middle of a function (against the standard) might cause an allocation problem whereas a previous printf would "make it legal". Am I thinking about this correctly?

matsp · 08-26-2008, 09:04 AM

Whether you can compile the C library with a Cxx or Cyy compliant compiler doesn't change whether the compiler you are using at the moment can compile your code or not. gcc is (nearly completely) C99 compliant in itself.

Further, for most intents and purposes, the code generated by:

Code:

void func()
{  
    int x;
    ...  // Big chunk of code
    /// Calculations setting x
    ... 
    printf("x=%d\n", x);
}

and

Code:

void func()
{  
    ...  // Big chunk of code
    int x;
    /// Calculations setting x
    ... 
    printf("x=%d\n", x);
}

is marginally different [there would be a bit more difference if x was a C++ object, but the space on the stack would probably be the same either way - it is allocated at the beginning of the function to make it easy for the rest of the code to know what's where and not unnecessarily shift the stack pointer up and down too much].

As to what goes on the stack and what goes in a CPU register is entirely up to the compiler. Of course, large variables such as a struct array, most likely goes into memory (there just ain't enough registers to waste them on stuff like storing structs in them).

Changing the code can change the choices the compiler makes for many reasons [compilers are often using heuristics - approximations/imperfect solutions to solve problems, and will have choices such as "if we have these three conditions true, we do it this way, else do it some other way" - those conditions may be which registers are actually available, the size of the code, and many other non-apparent reasons].

The other side is of course that the compiler can't take the address of a register, so if some variable is used as a target of & (address operator), then it's not valid to store it in a register.

Edit: If the compiler doesn't support C99, it will complain about C99 constructs such as local variables in the middle of code or dynamic sizing of variables.

And finally a question: What sort of number is nofl?

--
Mats

MK27 · 08-26-2008, 09:18 AM

"Changing the code can change the choices the compiler makes for many reasons [compilers are often using heuristics - approximations/imperfect solutions to solve problems, and will have choices such as "if we have these three conditions true, we do it this way, else do it some other way" - those conditions may be which registers are actually available, the size of the code, and many other non-apparent reasons]."

This makes sense, since even with the printf (=no segfault) line the individual members of new[nofl] were, in fact, corrupted, whereas using what I presume to be a more standard practice (the global ray[]) they are not. However, this produces the hopefully harmless warning: array ‘ray’ assumed to have one element

And a final answer: nofl is/was a short int.

ps. please see my last comment on pg.2 if you like "ray[]", disagree with the next two comments, and noticed that QuatumPete didn't get it...

matsp · 08-26-2008, 09:21 AM

So ray[] is a singe element array, and using that for nofl > 0 is invalid. You need to fill in number of elements in the brackets to make sure you have enough space. If you can't determine the number you need, it may be better to use some sort of dynamic allocation (malloc and friends).

What value is nofl (not what type it is)?

--
Mats

QuantumPete · 08-26-2008, 09:39 AM

Quote:

Originally Posted by MK27

to create a global array of structs that will grow "dynamically". But I still want to ask you...

There is your problem. You can't do that. Arrays can't grow dynamically. It's pretty much the definition of an array that it has a set size. Once an array has a certain set of dimensions it can't be changed.
Btw, there are no harmless warnings. All warnings should be treated as errors and in your case the warning tells you that you created your array with a size of 1. Then when you go to using it, you're most likely writing/reading past the end of the array, causing your segfault.

QuantumPete

08-26-2008, 12:53 AM	#1
MK27 subminimalist Join Date: Jul 2008 Location: NYC Posts: 3,946	use of printf prevents segfault! I'm writing a function that takes a file, reads the file into a buffer while keeping count of the lines, then takes each line and plugs the content into a simple structure. I did it that way because the number of lines is variable and will determine, obviously, the number of structs needed. So I get the number of lines (nofl) and in the middle of the function do this: struct thestruct new[nofl]; I was getting a segfault, so to make sure nofl was correct, i inserted the following line RIGHT BEFORE the last one: printf("--%d--\n", nofl); AND NOW NO SEGFAULT!!! Nothing else is changed, but adding and subtracting that one line makes all the difference. Really, really, really. More observations: 1) Up until now I've been declaring variables (inc. structs) at the beginning of a function, but I've never heard tell that it must be so. Does this have anything to do with it? 2) The segfault actually occurs later when new[1] (ie. the 2nd struct in the array) is called on (it turns out not to exist). 3) "nofl" is determined on the line immediately preceeding the printf statement. 4) if I put the printf statement after the struct declaration, the segfault returns. __________________ Accuracy and integrity mean nothing if you don't make it past the censors...PYTHAGORAS
MK27 is online now

08-26-2008, 01:22 AM	#2
C_ntua Registered User Join Date: Jun 2008 Posts: 1,134	You could post the code to make things more clear (if that's possible). 1) You can declare variables in the middle like you do with new[] only with the C99 standard (i think) 2) So a seg fault occurs after all, so the printf doesn't actually solve the problem, eh? 3) As long as it is before it is ok 4) I ll admit I had similar occasions that printf changed the seg fault. But in any case it doesn't solve the problem. Post the code to find exactly the problem.
C_ntua is offline

08-26-2008, 02:05 AM	#3
QuantumPete Technical Lead Join Date: Aug 2007 Location: London, UK Posts: 723	Sounds like a classic stack corruption. Can you post your code? QuantumPete __________________ "No-one else has reported this problem, you're either crazy or a liar" - Dogbert Technical Support "Have you tried turning it off and on again?" - The IT Crowd
QuantumPete is offline

08-26-2008, 08:01 AM	#9
MK27 subminimalist Join Date: Jul 2008 Location: NYC Posts: 3,946	Sorry I didn't post the original code, it seemed tedious and I wanted an answer to the printf question, since that was clearly the "crux". b/t/w "new" is not the actual name of the variable... "Sounds like a classic stack corruption." -- QuantumPete If stack corruption is what causes compiler output including this: ======= Backtrace: ========= [...] ======= Memory map: ======== Then no, it's not stack corruption. "For example, calling printf will make sure the variable is stored on the stack, whilst not calling printf will not use the stack to store the local variable" -- matsp Admittedly I know very little about this, but it was my understanding that an uninitialized int goes into a stack frame and stays there after being initialized. However, yours is the only explanation I see. "so some bytes less used on the stack, so when you overwrite the end of an array," -- matsp This pair of phrases does not make sense to me. Could you clarify? Anyway, in the end what solved the problem is just splitting the function so that the array of structs is defined at a beginning. I am pretty sure GNU glibc is C89, not C99? How this relates to the overwrite (assuming matsp is correct) I don't know. However, that now means the struct array is outside main! But I guess there's a different problem...thanks for all your thoughts. __________________ Accuracy and integrity mean nothing if you don't make it past the censors...PYTHAGORAS
MK27 is online now

08-26-2008, 08:16 AM	#10
matsp Kernel hacker Join Date: Jul 2007 Location: Farncombe, Surrey, England Posts: 15,686	Unfortunately, stack corruption can cause all sorts of things - backtrace assumes that you end up in some sort of error handler in the C-library - this doesn't always happen for stack corruption, because the code that prints the "backtrace" is using the stack to produce that backtrace, and if the stack is sufficiently messsed up, then it will not be able to figure out what to produce and give up [or crash]. (For example, if you write NULL to the frame-pointer [1] then it will most likely never find your backtrace) And to answer your direct questions: A local variable doesn't have to take up space on the stack. It is entirely possible, under the right circumstances, for the compiler to only ever hold the value in a register - in which case the variable has no stack-space assigned to it. Whether glibc is written in C89, C99 or some other C standard is pretty irrelevant - your code is what causes this problem. Splitting it into two functions should not fix any problem like this, but it may of course HIDE the problem. Without seeing the actual code, it's pretty hard to know what is right and what is wrong with it. It's like talking to a mechanic on the phone about a car that has some sort of problem. The mechanic can make a guess as to what the problem could be, but it's certainly a lot easier to diagnose the problem when the mechanic and the car are next to each other. [1]Frame-pointer is a pointer to the previous stack-frame that is maintained by the compiler so that a debugger or debug help function like the backtrace can figure out how you got there. -- Mats __________________ Compilers can produce warnings - make the compiler programmers happy: Use them! Please don't PM me for help - and no, I don't do help over instant messengers.
matsp is offline

08-26-2008, 08:51 AM	#11
MK27 subminimalist Join Date: Jul 2008 Location: NYC Posts: 3,946	The code is just too long and I'm not going to bother trying to recreate the problem on a smaller scale since it now works, altho I completely agree with that point. FYI my problem started with me not knowing I could do this: Code: struct thestruct { [...] } ray[]; int main () { to create a global array of structs that will grow "dynamically". But I still want to ask you... "Whether glibc is written in C89, C99 or some other C standard is pretty irrelevant" So what did C_ntua mean by "You can declare variables in the middle like you do with new[] only with the C99 standard (i think)"? Where would the problem be hiding once the segfault is gone and everything works smoothly, if my code must still be flawed? Can I ever rid myself of such a hidden flaw? Perhaps the overwrite was symptomatic of violating the standard. About the mysteries of memory allocation: by "stored in a register" you mean the variable is retained in the CPU and never allocated memory? If "printf" causes it to be put on the stack, shouldn't "struct thestruct Knew[nofl]" also? In which case doing this in the middle of a function (against the standard) might cause an allocation problem whereas a previous printf would "make it legal". Am I thinking about this correctly? __________________ Accuracy and integrity mean nothing if you don't make it past the censors...PYTHAGORAS
MK27 is online now

08-26-2008, 09:04 AM	#12
matsp Kernel hacker Join Date: Jul 2007 Location: Farncombe, Surrey, England Posts: 15,686	Whether you can compile the C library with a Cxx or Cyy compliant compiler doesn't change whether the compiler you are using at the moment can compile your code or not. gcc is (nearly completely) C99 compliant in itself. Further, for most intents and purposes, the code generated by: Code: void func() { int x; ... // Big chunk of code /// Calculations setting x ... printf("x=%d\n", x); } and Code: void func() { ... // Big chunk of code int x; /// Calculations setting x ... printf("x=%d\n", x); } is marginally different [there would be a bit more difference if x was a C++ object, but the space on the stack would probably be the same either way - it is allocated at the beginning of the function to make it easy for the rest of the code to know what's where and not unnecessarily shift the stack pointer up and down too much]. As to what goes on the stack and what goes in a CPU register is entirely up to the compiler. Of course, large variables such as a struct array, most likely goes into memory (there just ain't enough registers to waste them on stuff like storing structs in them). Changing the code can change the choices the compiler makes for many reasons [compilers are often using heuristics - approximations/imperfect solutions to solve problems, and will have choices such as "if we have these three conditions true, we do it this way, else do it some other way" - those conditions may be which registers are actually available, the size of the code, and many other non-apparent reasons]. The other side is of course that the compiler can't take the address of a register, so if some variable is used as a target of & (address operator), then it's not valid to store it in a register. Edit: If the compiler doesn't support C99, it will complain about C99 constructs such as local variables in the middle of code or dynamic sizing of variables. And finally a question: What sort of number is nofl? -- Mats __________________ Compilers can produce warnings - make the compiler programmers happy: Use them! Please don't PM me for help - and no, I don't do help over instant messengers.
matsp is offline

08-26-2008, 09:18 AM	#13
MK27 subminimalist Join Date: Jul 2008 Location: NYC Posts: 3,946	"Changing the code can change the choices the compiler makes for many reasons [compilers are often using heuristics - approximations/imperfect solutions to solve problems, and will have choices such as "if we have these three conditions true, we do it this way, else do it some other way" - those conditions may be which registers are actually available, the size of the code, and many other non-apparent reasons]." This makes sense, since even with the printf (=no segfault) line the individual members of new[nofl] were, in fact, corrupted, whereas using what I presume to be a more standard practice (the global ray[]) they are not. However, this produces the hopefully harmless warning: array ‘ray’ assumed to have one element And a final answer: nofl is/was a short int. ps. please see my last comment on pg.2 if you like "ray[]", disagree with the next two comments, and noticed that QuatumPete didn't get it... __________________ Accuracy and integrity mean nothing if you don't make it past the censors...PYTHAGORAS Last edited by MK27; 08-27-2008 at 08:28 AM. Reason: posterity
MK27 is online now

08-26-2008, 09:21 AM	#14
matsp Kernel hacker Join Date: Jul 2007 Location: Farncombe, Surrey, England Posts: 15,686	So ray[] is a singe element array, and using that for nofl > 0 is invalid. You need to fill in number of elements in the brackets to make sure you have enough space. If you can't determine the number you need, it may be better to use some sort of dynamic allocation (malloc and friends). What value is nofl (not what type it is)? -- Mats __________________ Compilers can produce warnings - make the compiler programmers happy: Use them! Please don't PM me for help - and no, I don't do help over instant messengers.
matsp is offline

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
IF CONDITION plese help	birumut	C Programming	12	03-06-2009 09:48 PM
making it portable.....?	ShadeS_07	C Programming	11	12-24-2008 09:38 AM
get keyboard and mouse events	ratte	Linux Programming	10	11-17-2007 05:42 PM
Simple C question: user input to repeat a loop	evernaut	C Programming	2	11-18-2006 09:23 AM
Drawing tables in C	stanoman	C Programming	5	10-09-2003 10:14 AM