Thread: Help converting CANoe logfile to EXCEL readable .txt file

  1. #1
    Registered User
    Join Date
    Aug 2009
    Posts
    9

    Help converting CANoe logfile to EXCEL readable .txt file

    Dear People,

    I'm new to the Forum so on firsthand greetings everyone and nice to see there are more people like me who are spending time on these annoying things

    I'm a complete noob to C programming and spending my holiday time on an assignment to convert a Vector CANoe log file to a .txt file wich can be read by EXCEL.

    Maybe Vector CANoe does not ring a bell but its a small device with wich you can listen in on a CAN-bus and log the messages.

    This logged data is saved to a .txt file in the following way:

    date Tue May 26 03:12:51 pm 2009
    base hex timestamps absolute
    no internal events logged
    // version 7.0.0
    0.000000 EnvGearDown := 1
    0.000000 EnvGearDown := 0
    0.000000 EnvGearDown := 1
    0.000000 EnvGearDown := 0
    0.000000 EnvGearDown := 1
    0.000000 EnvGearDown := 0
    0.000000 EnvGearDown := 1
    0.000000 EnvGearDown := 0
    0.000000 EnvGearDown := 1
    0.000000 EnvGearDown := 0
    0.000000 EnvGwSwitchIgnition := 2
    0.000000 EnvGwSwitchIgnition := 0
    2.000000 EnvGwSwitchIgnition := 1
    3.000000 EnvGwSwitchIgnition := 2
    3.100158 1 110 Tx d 3 02 00 00
    3.100158 EnvDashboardEngESPDsp_ := 1
    3.100158 EnvDashboardEngABSDsp_ := 1
    3.100158 EnvDashboardEngOilDsp_ := 1
    3.100158 EnvDashboardEngWaterDsp_ := 1
    3.100158 EnvDashboardEngBaterryDsp_ := 1
    3.100158 EnvBusCommActiveDsp := 1
    3.120180 1 1A0 Tx d 4 00 00 00 00
    3.120300 1 1F0 Tx d 1 00
    3.120416 1 1F1 Tx d 1 00
    3.140180 1 1A0 Tx d 4 00 00 00 00
    3.150120 1 1F0 Tx d 1 00
    3.150236 1 1F1 Tx d 1 00
    3.160180 1 1A0 Tx d 4 00 00 00 00
    3.180180 1 1A0 Tx d 4 00 00 00 00
    3.180300 1 1F0 Tx d 1 00
    3.180416 1 1F1 Tx d 1 00
    3.200158 1 110 Tx d 3 02 00 00
    3.200338 1 1A0 Tx d 4 00 00 00 00
    3.205529 1 41A Tx d 4 1A 01 01 FF
    3.205529 EnvNMReceiver26 := 1a
    3.205529 EnvNMStatusDsp26 := 1
    3.210120 1 1F0 Tx d 1 00
    3.210236 1 1F1 Tx d 1 00

    Now it is my task to convert this file to a .txt file in wich the correct rows and collums are ordered in the following way:

    time id dlc byte0 byte1 byte2 byte3 byte4 byte5 byte6 byte7
    3.100158 110 3 02 00 00
    3.120180 1A0 4 00 00 00 00
    3.120300 1F0 1 00
    3.120416 1F1 1 00
    3.140180 1A0 4 00 00 00 00
    3.150120 1F0 1 00
    3.150236 1F1 1 00
    3.160180 1A0 4 00 00 00 00
    3.180180 1A0 4 00 00 00 00
    3.180300 1F0 1 00
    3.180416 1F1 1 00
    3.200158 110 3 02 00 00
    3.200338 1A0 4 00 00 00 00
    3.205529 41A 4 1A 01 01 FF
    3.210120 1F0 1 00
    3.210236 1F1 1 00

    Like u can see i should "select" the rows in wich Tx is included and write these to the appropriate collums in the new .txt file.

    Opening and reading the file is goeing ok and writing the data to the new file is also opperative. But i can't seem to get the right data in the correct order.

    This is the code i've come to now:

    Code:
    #include <stdio.h>
    /* #include <conio.h> */
    
    int main(void)
    {
    FILE *fpin, *fpout;
    char flin[256],flout[256];
    
    char rel,x; /* relavant data and new line variables */
    signed char meetdata[13]; /* array for logdata */
    
    
    /* clrscr; */ /* clear screen */
    printf("This program converts standard Vector CANoe log files to Ecxel readable files\n\nPlease specify source filename:"); /* print programheader with programfunction and ask for input filename */
    scanf("%s",flin); /* read input filename */
    
    /* does inputfile exist ? */
    
    if ((fpin = fopen(flin, "r"))== NULL) 
    {
    printf("Cannot open source file!\n"); /* if not, show cannot open input */
    return 1; 
    }
    
    /* if inputfile does exist */
    
    else
    {
    printf("Please specify destination filename:"); /* ask to specify destination filename */
    scanf("%s",flout); /* read output filename */
    fpout = fopen(flout, "w"); /* open outputfile */
    x=0; /* select first line of logdata */ 
    }
    
    while((rel=fgetc(fpin))!=EOF) /* while relavant databyte = byte from inputfile and not end of file */
    {
    if (meetdata[3]=='Tx') /* if logdata colum 2 is 1 (synch) */
    {
    meetdata[x]=rel; /* logdata[x] is relavant databyte */
    x++; /* select new line of logdata */
    }
    /* write in outputfile */
    fprintf(fpout,"%f\t%X\t%d\t%X\t%X\t%X\t%X\t%X\t%X\t%X\t%X\n",meetdata[0],meetdata[2],meetdata[5],meetdata[6],meetdata[7],meetdata[8],meetdata[9],meetdata[10],meetdata[11],meetdata[12],meetdata[13]); 
    }
    /* time\tid\tdlc\tbyte0\tbyte1\tbyte2\tbyte3\tbyte4\tbyte5\tbyte6\tbyte7\n */ 
    
    fclose(fpin); /* close inputfile */
    fclose(fpout); /* close outputfile */
    return 0;
    
    } 
    
    /*end of main*/
    I have to finish this assignment verry soon and don't know what to do!!!!

    My professor has been unreachable for about a month an wil be for quite the same conciderring it's a holiday periode now.
    So he won't be of any help unfortunately.

    Please can somebody help me out its really important to me?

    Thanks upfront.

  2. #2
    Registered User
    Join Date
    Sep 2006
    Posts
    8,868
    But what *is* your question or problem:

    1) program won't compile
    2) program won't link
    3) program won't open the source file or output
    4) output is not accurate or being written at all
    5) algorithm is incorrect, apparently

    Welcome to the forum, and please be as specific as possible about your problem or question.

  3. #3
    Registered User
    Join Date
    Aug 2009
    Posts
    9
    Hi Adak,

    I wil be more specific about my problem.

    My Code does compile and link and also reads the input .txt file.

    It also creates an output.txt file but this file has a completely wrong content.

    So i guess my algorithm is not correct.

    I would like to place the inputfile lines that incorparate the Tx in an Array or something so i can "select" the lines i'm interested in.

    Then i would like to put these lines ( from the Array) in the correct layout in the output file.

    Both these things don't seem to work and i have no clue on how i should incorparate this in my algorithm.

    Like i said i'm a noob to C coding

  4. #4
    and the Hat of Guessing tabstop's Avatar
    Join Date
    Nov 2007
    Posts
    14,336
    This right here is pretty bizarre:
    Code:
    if (meetdata[3]=='Tx')
    For one, meetdata never changes so far as I can tell, since you're reading a byte at a time into rel. For two, one character can never equal 'Tx', as though 'Tx' meant anything, which it doesn't.

  5. #5
    Registered User
    Join Date
    Oct 2008
    Location
    TX
    Posts
    2,059
    There are a lot of things wrong with this code. How does it even compile on your system?
    Try the regcomp() / regexec() combination to filter out only those lines that have " Tx " in them.
    Last edited by itCbitC; 08-13-2009 at 12:52 PM.

  6. #6
    Registered User
    Join Date
    Aug 2009
    Posts
    9
    Tabstop,

    I understand what you're trying to say about reading one byte at a time and Tx is two characters so that will never go right.

    But how can i solve this.

    Youre right that Tx doestn't specificlaly mean anything in the Code but this is the only thing i can relate to in the input file as far as filtering the right lines out.


    itCbitC,

    I've also never heard of the the regcomp() / regexec() combination and don't think my professor will think i came up with these commands myself.

    Ther must be a simple way to do this right?

    Like i already said i'm a complete noob at this

    Haven't got a clue how to solve the problem so any help is appreciated.

  7. #7
    and the Hat of Guessing tabstop's Avatar
    Join Date
    Nov 2007
    Posts
    14,336
    Well, for sure you're going to need to know how to use strcmp, and how to read in a formatted line, and even more importantly how to deal with a line that doesn't meet that format -- i.e., some of your lines have seven things on them, some have ten, some (that don't have Tx) have even fewer.

  8. #8
    Registered User
    Join Date
    Sep 2006
    Posts
    8,868
    I would suggest

    *using fgets() to put each line into a char array[], one at a time
    *then using strstr() to search for "Tx" through the buffer
    if Tx is found, then store the line in the buffer, into a larger struct array[], appropriately sized to be larger than you need.

    Then after getting all the lines (or at least a large number of lines), into the struct array[], I would begin writing out the data to arrange as you need it to be.

    You may have a structure member that comes first, but needs to get written to the file, afterward. This approach makes it simple to do that, and you can lump all the data types together, in the struct, whether it's char's or numbers.

    If you've never used structs before, you can also do this with a variety of variables - int to hold integers, double or floats to handle floating point numbers (N.nn), and char variables or arrays or handle the rest.

    This is the way to use strstr():
    Code:
    #include <stdio.h>
    #include <string.h>
    
    int main(void)
    {
       char *str1 = "Borland International", *str2 = "nation", *ptr;
    
       ptr = strstr(str1, str2);
       printf("The substring is: %s\n", ptr);
       return 0;
    }

  9. #9
    Registered User
    Join Date
    Oct 2008
    Location
    TX
    Posts
    2,059
    Quote Originally Posted by turbofizzl View Post
    I've also never heard of the the regcomp() / regexec() combination and don't think my professor will think i came up with these commands myself.

    Ther must be a simple way to do this right?

    Like i already said i'm a complete noob at this

    Haven't got a clue how to solve the problem so any help is appreciated.
    Alright well that makes sense since the solution should be in-line with what has been covered in class.
    So have you covered sscanf()?? Because you can utilize that to take the input string apart.

    Call fgets() to read a line from the input file and store it in a big enough buffer.
    Next step would be to take the line, stored in the buffer, apart with sscanf().
    Insert "Tx d" into the sscanf() format string so that it acts only on those lines.

  10. #10
    Registered User
    Join Date
    Aug 2009
    Posts
    9
    Hey Guy's

    I've had some help and am getting along alot better now.

    Just two minor issues left.

    I now put all the data bytes in one array called bytes, now i have to think of some way to separate the data bytes into byte 0 to 7 seperated by tabs in the outputfile.

    I also created an array for the Data Length Code or dlc but when i print the content to the output file it seems the same as the id array i created.

    This should not be the case.

    However i discovered that the first characters of the bytes array i print in the outputfile are the characters that should be in the dlc array.

    So in some way i have to get these characters in the correct dlc array and seperate the successive data bytes with tabs.

    But how???

    I really am stuck again.

    In the end it should look somewhat like this:

    time[tab]id[tab]dlc[tab]byte0[tab]byte1[tab]byte2[tab]byte3[tab]byte4[tab]byte5[tab]byte6[tab]byte7

    With the following content:

     Time absolute time CAN message was received.
     ID 11-bits identifier of CAN message(hexadecimal)
     Length number of bytes in CAN message (decimal)
     ByteX Byte value of relevant byte in CAN message (hexadecimal)

    I've tried all kinds of things to solve this last problem but all i seem to get are errors.

    Help, Help, Help!!!

    This i my Code thusfar:

    (I've left out dlc in the fprintf because otherwise i get double data in the outputfile)

    Code:
    #include <stdio.h>
    
    
    void
    leesbytes(char bytes[], FILE *fpin)
    {
    int i=0;
    int c;
    
    //spaties overslaan
    do c = fgetc(fpin);
    while( c==' ' );
    
    while( i<64 && c!=EOF && c!='\n' )
    {
    bytes[i]=c;
    i++;
    c = fgetc(fpin);
    }
    bytes[i]=0;
    }
    
    
    
    int
    leesveld(char veld[], FILE *fpin)
    {
    int i=0;
    int c;
    
    //spaties overslaan
    do c = fgetc(fpin);
    while( c==' ' );
    
    //in lezen tot een spatie of newline of einde bestand
    while( i<32 && c!=EOF && c!='\n' && c!=' ')
    {
    veld[i]=c;
    i++;
    c = fgetc(fpin);
    }
    //einde streng met NUL
    veld[i]=0;
    return c;
    }
    
    
    
    int main(void)
    {
    FILE *fpin, *fpout;
    char flin[256],flout[256];
    
    int rel;
    
    char tijd[32],id[32],dlc[32],bytes[128];
    char kolom2[32],Tx[32],kolom6[32];
    
    
    printf("This program converts standard Vector CANoe log files to Ecxel readable files\n\nPlease specify source filename:"); /* print programheader with programfunction and ask for input filename */
    scanf("%s",flin); /* read input filename */
    
    /* does inputfile exist ? */
    if ((fpin = fopen(flin, "r"))== NULL)
    {
    printf("Cannot open source file <%s>!\n",flin); /* if not, show cannot open input */
    return 1; 
    }
    
    /* if inputfile does exist */
    else
    {
    printf("Please specify destination filename:"); /* ask to specify destination filename */
    scanf("%s",flout); /* read output filename */
    fpout = fopen(flout, "w"); /* open outputfile */
    fprintf(fpout,"time\tid\tdlc\tbyte0\tbyte1\tbyte2\tbyte3\tbyte4\tbyte5\tbyte6\tbyte7\n");
    }
    
    if( fpout==NULL )
    {
    printf("Cannot open Destination file <%s>\n",flout); /* if not, show cannot open output */
    return 1;
    }
    
    
    rel = 0;
    while( rel!=EOF )// lezen tot een EOF
    {
    rel = leesveld(tijd, fpin);
    if( rel != '\n' )
    {
    rel = leesveld(kolom2, fpin);
    if( rel != '\n' )
    {
    rel = leesveld(id, fpin);
    if( rel != '\n' )
    {
    rel = leesveld(Tx, fpin);
    if( rel != '\n' )
    {
    if( strcmp( Tx, "Tx")==0 )//als het 5de veld Tx is
    {
    leesveld(kolom6, fpin);
    leesbytes(bytes, fpin);
    
    fprintf(fpout,"%s\t%s\t%s\n",tijd,id,bytes);
    }
    }
    }
    }
    }
    
    
    }//einde while, einde bestand
    
    fclose(fpin); /* close inputfile */
    fclose(fpout); /* close outputfile */
    return 0;
    
    }
    My Output with this Code is:

    Code:
    time id dlc byte0 byte1 byte2 byte3 byte4 byte5 byte6 byte7
    3.100158 110 3 02 00 00
    3.120180 1A0 4 00 00 00 00
    3.120300 1F0 1 00
    3.120416 1F1 1 00
    3.140180 1A0 4 00 00 00 00
    3.150120 1F0 1 00
    3.150236 1F1 1 00
    3.160180 1A0 4 00 00 00 00
    3.180180 1A0 4 00 00 00 00
    3.180300 1F0 1 00
    3.180416 1F1 1 00
    3.200158 110 3 02 00 00
    3.200338 1A0 4 00 00 00 00
    3.205529 41A 4 1A 01 01 FF
    3.210120 1F0 1 00
    3.210236 1F1 1 00
    Where there's a space now next to the id colum there should be tabs.
    so everywhere after *
    3.100158 110 3* 02* 00* 00

    Then everything is in seperate colums like it should be.

    I really hope someone can help me out because i'm so close to my end goal and i can't seem to get it right.

    This is all thats in the way of me getting a nice few weeks of holliday.
    Otherwise i should have left by now

  11. #11
    Registered User
    Join Date
    Sep 2006
    Posts
    8,868
    I wish your helper would get you started on using some indentation in your code. Quite poor leaving each line of code to just begin on the left hand margin. As you gain experience programming, your eye will become trained to use those levels of indentation, automatically, to find bugs you won't easily catch, otherwise.

    I would not put tabs or other formatting char's, into your data file, at all. Keep the data file, for the necessary data.

    Handle the printing in your function that does the printing. It's much more flexible, and easy easy to control just what you want, or change, that way.

    Code:
    while(fscanf("%f %d %d (whatever) ", time, var1, var2, etc.)  {
       printf("%f \t %d \t %d\t etc.", time, var1, var2, etc.);
    }
    Imagine the above is your code. Now you want to add two tabs between 1 variable, instead of one. Do you see how easy it would be to change that, instead of re-reading and re-formatting your entire data file?

    Perhaps more to your point - fine tuning your output is also much easier.

  12. #12
    Registered User
    Join Date
    Oct 2008
    Location
    TX
    Posts
    2,059
    Separate the array named "bytes" into its components using strtok()
    Assemble the components together with sprintf() delimited by tabs
    And finally fprintf() the entire assembled array in one fell swoop

    Here's my 2c
    Code:
    #include <stdio.h>
    #include <string.h>
    
    int main(int argc, char *argv[])
    {
        FILE *fpin, *fpout;
        char flin[256], flout[256];
    
        char *p, line[256], x[256];
        char s1[50], s2[50], s3[50];
    
        printf("Input filename: ");
        scanf("%s", flin);
        fpin = fopen(flin, "r");
    
        printf("Output filename: ");
        scanf("%s", flout);
        fpout = fopen(flout, "w");
    
        while(fgets(line, 256, fpin))
            if (sscanf(line, "%s%*s%s Tx %*s %[^\n]", s1, s2, s3) == 3) {
                sprintf(x, "%s\t%s", s1, s2);
                for (p = strtok(s3, " "); p; p = strtok(NULL, " "))
                    sprintf(x, "%s\t%s", x, p);
                fprintf(fpout, "%s\n", x);
            }
        fclose(fpin);
        fclose(fpout);
        return 0;
    }

  13. #13
    Registered User
    Join Date
    Aug 2009
    Posts
    9
    Quote Originally Posted by Adak View Post
    I wish your helper would get you started on using some indentation in your code. Quite poor leaving each line of code to just begin on the left hand margin. As you gain experience programming, your eye will become trained to use those levels of indentation, automatically, to find bugs you won't easily catch, otherwise.
    Adak i completely understand youre point here and i do work the right way with levels of indentation. Actually my lecturer say I'm one of the few beginners how isn't making a mess of it

    Somehow tho when i insert the code on the forum the levels of indentation aren't there anymore. So i think i'm doing something wrong her but thats a whole different problem.

    Quote Originally Posted by Adak View Post
    I would not put tabs or other formatting char's, into your data file, at all. Keep the data file, for the necessary data.
    I understand youre point but the tabs are part of the assignment and should be there as a seperation sign to make the collums in EXCEL.


    About the printing.

    I'd already tried it the way you suggested because that is one of the things we ones practiced in class.

    But for some reason i get error right away.

    I tried to solve it like this:

    Code:
    fprintf(fpout,"%s\t%s\t%s\t%s\n",tijd,id,bytes[1], bytes[2]);   \\and so on

    This way i , i thought, i could at least have a look in the outputfile to wich part of the bytes array i wanted as the dlc and also print the rest with tabs in between.

    This unfortunately doesn't seem to work and i don't know why.

  14. #14
    Registered User
    Join Date
    Aug 2009
    Posts
    9
    Quote Originally Posted by itCbitC View Post
    Separate the array named "bytes" into its components using strtok()
    Assemble the components together with sprintf() delimited by tabs
    And finally fprintf() the entire assembled array in one fell swoop
    Can you please give me some more detailed explanation about the youre Code?

    It does seem to work so thats nice but i don't completely understand what youre doing and i geuss that just the whole point of the assignment.


    Furthermore i've never heard of the strtok command in class so unfortunately i don't think i can use it for my assignment.

    Maybe my lecturer won't mind when i can explain him that i did some research on the net and i completely understand what the code does it's not such a problem.

    But there should be a more ROOKIE Like way of solving the problem i geuss???

  15. #15
    Registered User
    Join Date
    Sep 2006
    Posts
    8,868
    I have the editor I use to write code, make 3 spaces when I hit the tab key, not tabs. Then everything works well on the forum.

    You can do everything you need to with simpler C code, by "picking" apart what you need from the buffer that has your data. There's nothing "special" about the C functions that ItcBitC has recommended. In fact, the "father" of C starts right off with examples of code to mimic simple C functions, with your own code, in his book.

    Tabs and other formatting code can be added *when the file is printed, on-screen* and *when the data is printed to a file for excel*. It doesn't need to be kept with the rest of the data, when you're trying to work with it, internally. As long as it's not confusing you or hindering your efforts at programming, then it's OK, either way.

    As you can tell perhaps, it confuses me, but I always like having *the variable* and nothing else, to work with, until the very last step in making a program. Tabs and such, are details, and in top down programming design, you always put off the details, until last.

    Show me the variables in one example that you want to print up using a certain format, and I'll show you how to do it. In programming, the computer is your world - YOU are the ruler of that world, and no crummy formatting problem can kick be allowed to kick sand in your face!

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Newbie homework help
    By fossage in forum C Programming
    Replies: 3
    Last Post: 04-30-2009, 04:27 PM
  2. gcc link external library
    By spank in forum C Programming
    Replies: 6
    Last Post: 08-08-2007, 03:44 PM
  3. Dikumud
    By maxorator in forum C++ Programming
    Replies: 1
    Last Post: 10-01-2005, 06:39 AM
  4. Unknown Memory Leak in Init() Function
    By CodeHacker in forum Windows Programming
    Replies: 3
    Last Post: 07-09-2004, 09:54 AM
  5. Need a suggestion on a school project..
    By Screwz Luse in forum C Programming
    Replies: 5
    Last Post: 11-27-2001, 02:58 AM