Thread: shared library, but not dynamic !?

  1. #1
    Registered User
    Join Date
    May 2008
    Posts
    14

    Question shared library, but not dynamic !?

    Hi,

    I am developping a code under linux which mixes fortran and gcc, and that I compile with gcc and ifort.

    I am used to static libraries (.a), and I am transforming some of my static libraries into dynamic libraries (.so). My code is compiling and working with the shared lib.

    I have however a problem, I would like to modify the shared libs "on the fly", ie during the running of the program, but it does not seem to work correctly. I have to stop the program and re-run it (without recompiling) if I want the modified shared lib to be taken into account. Using dlload does not seem to change anything.

    I guess it is a linking problem, but I am not sure... Any idea?

    Thanks,
    Best,
    CCox

  2. #2
    Registered User
    Join Date
    May 2008
    Posts
    14
    Nobody?

  3. #3
    Registered User
    Join Date
    Sep 2004
    Location
    California
    Posts
    3,268
    Use dlopen to open the shared library, then use dlsym() to get pointers to functions in the library. When finished, call dlclose().

  4. #4
    Registered User
    Join Date
    May 2008
    Posts
    14
    Thank you very much, that's what I did.

    However I have a problem, when using dlsym:
    - I run the program during which a shared lib is enriched with new functions (which are compiled on the fly)
    - I try to dlopen the recompiled lib, it works, but dlsym does not find the new routines.
    - if I re-run the program, the new routines are working.

    What should I do if I want the new routines to be accessible during the running?

    Thanks,
    CCox

  5. #5
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Quote Originally Posted by ccox421 View Post
    Thank you very much, that's what I did.

    However I have a problem, when using dlsym:
    - I run the program during which a shared lib is enriched with new functions (which are compiled on the fly)
    - I try to dlopen the recompiled lib, it works, but dlsym does not find the new routines.
    - if I re-run the program, the new routines are working.

    What should I do if I want the new routines to be accessible during the running?

    Thanks,
    CCox
    You would need to completely close the original library - not sure if there is more you need to do. But if the old shared library is in use, then the new functionality will not show up until the file is closed and re-opened, at the very least. [I have never used this myself, so I'm sorry if the details are a bit vague].

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  6. #6
    Registered User
    Join Date
    May 2008
    Posts
    14
    Hi,

    I tried to do that, but unfortunately the problem is still here. Does anybody know if something else has to be done?

    Thanks,
    Ccox

  7. #7
    Registered User
    Join Date
    May 2008
    Posts
    14
    Hi,

    To be more specific, I can give some more explanations:

    - I am compiling the program.
    - I am then running it.
    - The program check if a shared lib which was not loaded exists, and if a function exists in it (with dlopen and dlsym).
    - It first does not exit, so it compile a new library.
    - The program open the library with dlopen, then launch a function of that library using dlsym.
    And that's when I get a "Segmentation fault".
    What is strange is that dlerror shows no problem...

    Then:

    - I relaunch the program.
    - The lib and function exit, so the program does not compile it.
    - The program runs well with these functions... No segmentation fault until the next missing function is compiled and loaded...

    That's very strange, I don't see what the problem could be.

    I paid very much attention at closing correctly the newly compiled lib, so there should not be any problem in here.

    I test dlopen with RTLD_NEW and RTLD_LAZY, but it doesn't change anything.
    I compile with gcc, with the option -rdynamic...

    I really have no idea what to do or test.

  8. #8
    Officially An Architect brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,396
    Can we see some actual code? You can leave out the code which generates the other code which gets compiled, but can we see the overall structure?
    Code:
    //try
    //{
    	if (a) do { f( b); } while(1);
    	else   do { f(!b); } while(1);
    //}

  9. #9
    Registered User
    Join Date
    May 2008
    Posts
    14
    Hi,

    I am posting the interesting part of my code, but I am not sure it will be so easy to read. The code mix fortran and C.

    First, the main code is in C, but needs to call fortran procedure. Everything is standard and works fine until it reaches this part of the fortran code:

    Code:
            double precision function dWeffdcostree(sqrtS,costh)
            implicit none
    
    #include "decl.h"
    #include "Weff_tree.h"
    
            double precision sqrtS,costh
            double precision result
            double precision result_tmp(NCOMP)
    
            result=0.D0
    
            if((sqrtS .ge. (aMNeu(1)+aMNeu(1))) .AND.
         &  ((aMNeu(1)+aMNeu(1)) .le. maxenergy)) then
               
            call dlwefftree("n1n1",result_tmp,sqrtS,costh);
    (...)
    dlwefftree needs the following routine in C, which dlopen/create librelictreedyn.so:

    Code:
    void dlwefftree_(char* name[], double result[], double* sqrtS, double* costh)
    {
    	char Weffname[100];
    	void* handle=dlopen("src/relic/sharedlibtree/librelictreedyn.so", RTLD_LAZY);		
    	
    	sprintf(Weffname,"weff_tree_%s_",name);
    	
    	void (*Weff)(double result[], double* sqrtS, double* costh) = dlsym(handle,Weffname);
    
    	char *error;
    	if( (error=dlerror()) != NULL ) 
    	{
    		if(handle) dlclose(handle);
              	/* printf("%s\n",error); */
      		char complete_name[300];
    	
    		chdir("src/relic/sharedlibtree");
    		
    		sprintf(complete_name,"rm -f *.o");
    		system(complete_name); 	
    			
    		sprintf(complete_name,"%s %s -c ../Weff_tree/Weff_tree_%s.F -I../Weff_tree -I..",FC,FFLAGS,name);
    		
    		printf("%s\n",complete_name);
    		
    		system(complete_name);
    		
    		sprintf(complete_name,"ar x librelictreedyn.a");
    		system(complete_name);    	
    		
    		sprintf(complete_name,"ld -shared -o librelictreedyn.so *.o");
    		system(complete_name);    	
    
    		sprintf(complete_name,"ar r librelictreedyn.a *.o");
    		system(complete_name);    	
    		
    		sprintf(complete_name,"rm -f *.o");
    		system(complete_name);   	
    		
    		chdir("../../..");
    		
    		handle=dlopen("./src/relic/sharedlibtree/librelictreedyn.so", RTLD_LAZY);
    		
    		/* if( !handle ) printf("%s\n",dlerror()); */
    		
    		void (*Weff)(double result[], double* sqrtS, double* costh) = dlsym(handle,Weffname);
    		
              	/* if( (error=dlerror()) != NULL ) printf("%s\n",error); */
    	}
    
    	(*Weff)(result,sqrtS,costh);
    
    	dlclose(handle);	
    		
    	return;
    }
    The routine which should be in (*Weff) should be the following (in fortran):

    Code:
            subroutine weff_tree_n1n1(result,sqrtS,costh)
            implicit none
    
    #include "decl.h"
    #include "Weff_tree.h"
    
            double precision sqrtS,costh
            double precision result(NCOMP)
            double precision result_tmp(NCOMP)
    	
           call Weff_tree_n1n1ZZ(result_tmp,sqrtS,costh)
    
    	end
    Weff_tree_n1n1ZZ is written as:

    Code:
            subroutine Weff_tree_n1n1ZZ(result,sqrtS,costh)
            implicit none
            
    #include "decl.h"
    
            double precision result(NCOMP),result_tmp(NCOMP)
            double precision sqrtS
            double precision costh
             
    	call squaredtree("n1n1ZZ",result_tmp, helicities, flags)
    	                   
            result(1)=result_tmp(1)
           
            end
    squaredtree is a C function dlopening or creating the library libsquaredtreedyn.so:

    Code:
    void squaredtree_(char* name[], double result[], double* sqrtS, double* costh)
    {
    	char Weffname[100];
    	int i=0;
    	void* handle=dlopen("src/relic/sharedlibtree/libsquaredtreedyn.so", RTLD_LAZY);		
    	
    	sprintf(Weffname,"tree_%ssquaredme_",name);
    	
    	while (Weffname[i])
      	{
        		Weffname[i]=tolower(Weffname[i]);
    		i++;
    	}
    			
    	void (*Weff)(double result[], double* sqrtS, double* costh) = dlsym(handle,Weffname);
    
    	char *error;
    	if( (error=dlerror()) != NULL ) 
    	{
     		if(handle) dlclose(handle);
     		char complete_name[300];
    	
    		chdir("src/relic/sharedlibtree");
    		
    		sprintf(complete_name,"rm -f *.o");
    		system(complete_name);    	
    			
    		sprintf(complete_name,"%s %s -c ../squared_tree/squaredmetree_%s/squaredtree_%s.F -I../squared_tree/squaredmetree_%s -I..",FC,FFLAGS,name,name,name);
    		printf("%s\n",complete_name);	
    		system(complete_name);
    		
    		sprintf(complete_name,"ar x libsquaredtreedyn.a");
    		system(complete_name);    	
    		
    		sprintf(complete_name,"ld -shared -o libsquaredtreedyn.so *.o");
    		system(complete_name);    	
    
    		sprintf(complete_name,"ar r libsquaredtreedyn.a *.o");
    		system(complete_name);    	
    		
    		sprintf(complete_name,"rm -f *.o");
    		system(complete_name);   	
    	
    		handle=dlopen("./libsquaredtreedyn.so", RTLD_LAZY);
    		
    		if( !handle ) printf("%s\n",dlerror());
    				
    		void (*Weff)(double result[], double* sqrtS, double* costh) = dlsym(handle,Weffname);
    		
    		chdir("../../..");
    		
              	if( (error=dlerror()) != NULL ) printf("%s\n",error);
    	}
    
    	(*Weff)(result,sqrtS,costh);
    			
    	dlclose(handle);
    	return;
    }
    At the end, the routine in (*Weff) is a big fortran code which was working pretty well when static...

    Hope this can help...

    Note that I simplified a bit the code. I hope I have not broken anything.

    Thanks,
    CCox

  10. #10
    Officially An Architect brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,396
    Well, we can start by fixing some errors and see what happens.


    Code:
    void dlwefftree_(char* name[], double result[], double* sqrtS, double* costh)
    {
    	char Weffname[100];
    	void* handle=dlopen("src/relic/sharedlibtree/librelictreedyn.so", RTLD_LAZY);		
    	
    	sprintf(Weffname,"weff_tree_%s_",name);
    	
    	void (*Weff)(double result[], double* sqrtS, double* costh) = dlsym(handle,Weffname);
    You do not check if dlopen() returned NULL (i.e. the .so does not exist), yet you pass the handle blindly to dlsym(). That could be a crash.

    Code:
    	char *error;
    	if( (error=dlerror()) != NULL )
    Interesting way of checking for an error. You probably should just check if dlsym() returns NULL. Calling dlerror() when no error has occurred is undefined.

    Code:
    		sprintf(complete_name,"%s %s -c ../Weff_tree/Weff_tree_%s.F -I../Weff_tree -I..",FC,FFLAGS,name);
    Does the FC compiler generate position-independent code? If not, the shared library will not work correctly.
    Code:
    //try
    //{
    	if (a) do { f( b); } while(1);
    	else   do { f(!b); } while(1);
    //}

  11. #11
    Registered User
    Join Date
    May 2008
    Posts
    14
    Thanks for all these good comments.

    For the compilation, I am using either gfortran or ifort with -fPIC, so it should work.

    Concerning the error checkings, I will test that. In fact the code needs to compile hundreds of different fortran procedures similar to the one I showed. All of them either go in librelictreedyn.so or in libsquaredtreedyn.so . I agree that the first step is dangerous... But once both librairies are created, I was thinking it should work.

    I have unfortunately no time to test you advices now, but I will do that for tomorrow, and I will keep you informed.

    Thanks again,
    Best,
    CCox

  12. #12
    Officially An Architect brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,396
    The symbol you are trying to dlsym() ends with an underscore, but the Fortran function name doesn't. Is it some convention that Fortran funcs end with underscore, or is this an actual problem?
    Code:
    //try
    //{
    	if (a) do { f( b); } while(1);
    	else   do { f(!b); } while(1);
    //}

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Replies: 7
    Last Post: 02-06-2009, 12:27 PM
  2. Adding a directory to a dynamic library path
    By vivharv in forum Windows Programming
    Replies: 3
    Last Post: 09-20-2007, 07:09 AM
  3. global variable at dynamic library
    By Mortissus in forum C++ Programming
    Replies: 2
    Last Post: 07-31-2007, 02:44 PM
  4. wxWidgets loading shared library problem
    By BobS0327 in forum Linux Programming
    Replies: 1
    Last Post: 05-10-2006, 07:23 PM
  5. better c string functions
    By samps005 in forum C Programming
    Replies: 8
    Last Post: 11-04-2003, 01:28 PM