Thread: Memory allocation question

  1. #1
    Registered User
    Join Date
    May 2011
    Posts
    66

    Memory allocation question

    Hi. I have the following function which in the future will load all of the .so modules in a given directory with dlopen, for the moment it only lists the sub directories within "MODULES_PATH", and checks if these folders contain the mod_(module name).so file.

    For ex. if in the ./modules folder this function finds a folder named "test", then it will search for a mod_test.so file in it.

    Code:
    int module_load()
    {
    	DIR *dir			= opendir(MODULES_PATH);
    	struct dirent *ent	= NULL;
    	
    	if (dir == NULL)
    	{
    		log_error("%s","Error loading modules!\n");
    		return -1;
    	}
    
    	char file_path[128];
    	file_path[0] = '\0';
    	
    	while ((ent = readdir(dir)) != NULL) 
    	{
    		if (strcmp(ent->d_name, ".") == 0 || strcmp(ent->d_name, "..") == 0)
    			continue;
    		
    		sprintf(file_path, "%s%s/mod_%s.so", MODULES_PATH, ent->d_name, ent->d_name);
    	
    		if (!fs_file_exists(file_path))
    		{
    			continue;
    		}
    	}
    	
    	closedir(dir);
    	return 0;
    }
    My questions are the following:
    1. Which would be the more natural way to declare the file_path variable? Like this:
    Code:
    char file_path[128];
    (and automatically limit the file path`s length to 128 bytes) or like this:
    Code:
    char file_path[strlen(MODULES_PATH) + (strlen(ent->d_name) * 2) + 9];
    and move the declaration inside the while loop. (the +9 at the end comes from the length of the strings "/mod_",".so", and the \0 character.). I tried and it works both ways, but I would like to know which is the standard way of doing it, and what would be the problem with one or the other solution.

    2. It is more of a general question. If I understood correctly when a variable is declared statically (for ex: char variable[256]) the space for the variable is allocated before runtime, or when the execution enters a function. If this is true, shouldn`t the compiler throw an error at this:
    Code:
    char file_path[strlen(MODULES_PATH) + (strlen(ent->d_name) * 2) + 9];
    because the size of the file_path variable depends on the length of the current file/folder name, and it is unknown at compile time?

    Thank you.
    Last edited by raczzoli; 09-10-2012 at 03:27 PM.

  2. #2
    Programming Wraith GReaper's Avatar
    Join Date
    Apr 2009
    Location
    Greece
    Posts
    2,739
    Normally, compilers allow C99 extensions, like variable-lenght arrays. In some it gives you a warning though. You can set a compiler option that permits only strict ANSI C, like with GCC : -std=ansi
    Devoted my life to programming...

  3. #3
    Registered User
    Join Date
    Apr 2008
    Posts
    396
    To discuss the first question, I suggest you look at the PATH_MAX and NAME_MAX symbols defined for posix.
    They indicates the maximum size of a complete path and a path component respectively.
    This would look better than defining arbitrary constants.

  4. #4
    Registered User
    Join Date
    May 2011
    Posts
    66
    Thank you all for the quick replies. They helped me a lot.

  5. #5
    Registered User
    Join Date
    Aug 2012
    Posts
    6
    Instead of relying on arbitrary constants, why not just dynamically allocate/deallocate the string within the loop? It's a little more work, but given that the end result is a much more robust approach, it's probably worth the extra effort, methinks.

  6. #6
    Registered User
    Join Date
    May 2011
    Posts
    66
    Quote Originally Posted by Seabass View Post
    Instead of relying on arbitrary constants, why not just dynamically allocate/deallocate the string within the loop? It's a little more work, but given that the end result is a much more robust approach, it's probably worth the extra effort, methinks.
    I wanted to limit the use of mallocs, to save some execution time, but in this case I guess it is too a solution. Thank you for your reply.

  7. #7
    Registered User
    Join Date
    May 2011
    Posts
    66
    I just added the following lines to my code.

    Code:
    #define MODULES_PREFIX		"mod_"
    
    typedef struct
    {
    	char *name;
    	void *handle; //the handler returned by dlopen
    } module_t;
    and in the function I added:

    Code:
    module_t *module = malloc(sizeof(module_t *));
    module->name 	 = malloc(strlen(ent->d_name) + strlen(MODULES_PREFIX));
    sprintf(module->name, "%s%s", MODULES_PREFIX, ent->d_name);
    module->handle 	 = module_handle;
    Just out of curiosity, I changed malloc(strlen(ent->d_name) + strlen(MODULES_PREFIX)) to malloc(1) to see what happens, thinking that it probably will give a segmentation fault because the allocated space is too small, but surprisingly it worked.
    If I printf the value of the "name" variable, it shows me the whole name of the module, and I don`t understand why. If the malloc(1) allocates one byte for module->name, and I try to copy anything larger than 1 character in it, shouldn`t it throw a segfault?

    Is there some kind of "magic" that gcc does when it compiles the code? Maybe it allocates more space, even if I write malloc(1)?

    Thank you.

    P.s.: I`m using gcc (GCC) 4.4.6 20120305.

  8. #8
    Registered User
    Join Date
    Sep 2006
    Posts
    8,868
    When you have a memory problem, C won't just segfault, all the time. It's "undefined behavior", and you know it will crash the program if it's a grievous enough error, but smaller errors may actually allow your program to continue running - with results that may or may not be accurate.

    For example, you can always read past the end of a small array, for one element - but sometimes, you can read 2 - 6 elements past the end of an array. After that, you're toast. If you try to write to those elements past the end of the array, you're usually toast a lot sooner.

    You have to be careful about this stuff, all the time.

  9. #9
    Registered User
    Join Date
    May 2011
    Posts
    66
    Thank you.

  10. #10
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    39,660
    > Is there some kind of "magic" that gcc does when it compiles the code? Maybe it allocates more space, even if I write malloc(1)?
    You could try either

    a) running valgrind, as in
    valgrind ./a.out

    b) linking with electric fence, and running in the debugger
    gcc prog.c -lefence
    gdb ./a.out

    Both will tell you if you're abusing the space allocated through malloc (and a bunch of other stuff as well in the case of valgrind).

    FWIW, your malloc is 1 byte short.
    You forgot to count the \0.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.

  11. #11
    Registered User
    Join Date
    May 2011
    Posts
    66
    Quote Originally Posted by Salem View Post
    >
    FWIW, your malloc is 1 byte short.
    You forgot to count the \0.
    I didn`t want to leave the malloc(1) in my code, just wanted to see what happens, and when I saw that it still works, I didn`t understand why. I will run my application through valgrind, just out of curiosity.

    Thanks for the reply

  12. #12
    Registered User
    Join Date
    May 2011
    Posts
    66
    I ran my program through valgrind, and saw the errors about the memory corruption. I`ve also corrected some other errors too, but there are two more errors left, and I can`t figure out what am I doing wrong. This is the modified function:

    Code:
    typedef struct
    {
    	char *name;
    	void *handle;
    } module_t;
    
    int module_load()
    {
        DIR *dir            	= opendir(MODULES_PATH);
        struct dirent *ent  	= NULL;
        void *module_handle		= NULL;
    	void (*module_init)() 	= NULL;
    	
        if (dir == NULL)
        {
            log_error("%s","Error loading modules!\n");
            return -1;
        }
    
        char file_path[256];
        while ((ent = readdir(dir)) != NULL)
        {
    		module_t *module = NULL;
    		
            if (strcmp(ent->d_name, ".") == 0 || strcmp(ent->d_name, "..") == 0)
                continue;
             
            sprintf(file_path, "%s%s/%s%s.so", MODULES_PATH, ent->d_name, MODULES_PREFIX, ent->d_name);
         
            if (!fs_file_exists(file_path))
    		{
    			log_error("%s%s%s","Error loading mod_", ent->d_name, ".so. No such file!\n");
    			continue;
    		}
    		
    		module_handle = dlopen(file_path, RTLD_LAZY);
    		
    		if (module_handle == NULL)
    		{
    			log_error("%s%s%s","Error loading mod_", ent->d_name, ".so. Invalid module!\n");
    			continue;
    		}
    		
    		module_init = dlsym(module_handle, "init");
    		
    		if (module_init == NULL)
    		{
    			log_error("%s%s%s","Module mod_", ent->d_name, ".so doesn`t have a init function. Ignoring...!\n");
    			dlclose(module_handle);
    			continue;
    		}
    		
    		log_message("%s%s\n","Loading module: mod_", ent->d_name);
    		
    		module_init();
    		
    		module		= malloc(sizeof(module_t *));
    		module->name 	= malloc(strlen(ent->d_name) + strlen(MODULES_PREFIX) + 1);
    		module->handle 	= module_handle;
    		
    		num_modules++;
    		if (modules == NULL)
    		{
    			modules = malloc(sizeof(module_t *));
    		}
    		else
    		{
    			modules = realloc(modules, num_modules * sizeof(module_t *));
    		}
    		
    		modules[num_modules-1] = module;
    		
        }
    	
    	closedir(dir);
    	
    	return 0;
    }
    Valgrind is showing the following two errors:
    1. Invalid write of size 4
    2. Address 0x402669c is 0 bytes after a block of size 4 alloc'd

    I figured out, that there is some kind of problem with this line:
    Code:
    module->handle 	= module_handle;
    . If I remove it, both errors go away, but to be honest, I can`t figure out what is the problem. module->handle is a void pointer, dlopen also returns a pointer casted to void, and all I`m saying is that the first pointer is equal to the second.

    If you need more information, please let me know.

    Thank you.

  13. #13
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    39,660
    > module = malloc(sizeof(module_t *));
    You need to allocate a module, not a module pointer.

    Eg
    module = malloc( sizeof(*module) );

    Using *variable_name in the sizeof expression saves a lot of guesswork trying to get the typename correct.

    > if (modules == NULL)
    Just so you know, you can pass NULL to realloc, and it behaves like malloc.
    So you don't need to make it a special case.

    Also, you need to do this to be safe from possible memory leaks.
    Code:
    void *temp = realloc( modules, (num_modules+1) * sizeof(module_t *));
    if ( temp != NULL ) {
      modules = temp;
      modules[num_modules] = module;
      num_modules++;
    } else {
      // realloc failed.
      // modules and num_modules are the old values
      // but it does mean the latest module should be unloaded and released
    }
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.

  14. #14
    Registered User
    Join Date
    May 2011
    Posts
    66
    I just changed the module = malloc(sizeof(module_t *)) to module = malloc( sizeof(*module) ); like you said, and the errors I`ve been having just gone away.

    Thank you very much for your help.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. A question about memory allocation in Linux
    By BBH in forum C++ Programming
    Replies: 2
    Last Post: 08-25-2009, 06:34 PM
  2. Memory allocation question
    By dakarn in forum C Programming
    Replies: 11
    Last Post: 12-01-2008, 11:41 PM
  3. Question about memory allocation between methods
    By kotoko in forum C Programming
    Replies: 2
    Last Post: 06-16-2008, 08:32 AM
  4. Another Question...memory allocation
    By quickclick330 in forum C Programming
    Replies: 4
    Last Post: 12-12-2007, 04:25 PM
  5. Question about memory/allocation/pointers
    By Magos in forum C++ Programming
    Replies: 4
    Last Post: 11-21-2001, 09:57 AM