”Converting” an array…

This is a discussion on ”Converting” an array… within the C Programming forums, part of the General Programming Boards category; I know there are some array questions around, but I didn't find an answer to this particular one: Is it ...

  1. #1
    Registered User
    Join Date
    Feb 2013
    Location
    Sweden
    Posts
    59

    ”Converting” an array…

    I know there are some array questions around, but I didn't find an answer to this particular one:

    Is it possible to convert a one dimensional array to dimensional array by just some kind of ”type casting”? For example, I am trying to use an external library to read audio data (samples). For example, if the file is a stereo audio file, the samples end up like:
    Code:
    First  sample, left  channel → Audio[0]
    First  sample, right channel → Audio[1]
    Second sample, left  channel → Audio[2]
    Second sample, right channel → Audio[3]
         ⁝
    If I have a two dimensional array, like Audio[Sample][Channel], as I understand it, its data is arranged in the memory in a similar way, maybe something like:
    Code:
    Audio[0][0]
    Audio[0][1]
    Audio[1][0]
    Audio[1][1]
       ⁝
    This should make it fairly easy to just ”convert” between the two ”data arrengements”, shouldn't it?
    Of course I know that I can just copy the data from one to the other in a loop, but if it was possible to just ”type cast” it, it would execute much faster, wouldn't it? Would be a nice shortcut…

  2. #2
    Registered User
    Join Date
    Nov 2011
    Location
    Saratoga, California, USA
    Posts
    334
    I'm not even sure what a cast like that would look like. Have you tried?

    You can always take advantage of the fact that multidimensional arrays are laid out linearly as a 1D array in memory.

    Code:
    #include <stdio.h>
    
    
    #define ROWS 6
    #define COLS 5
    
    
    int main()
    {
        int arr[ROWS*COLS] = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12};
        int i, j;
        int *p = arr;
    
    
        for( i = 0; i < ROWS; ++i ){
            for( j = 0; j < COLS; ++j )
                printf("%2d ", *(p + i*COLS + j));
            printf("\n");
        }
    
    
    return 0;
    }

  3. #3
    Registered User
    Join Date
    Nov 2012
    Posts
    1,056
    Quote Originally Posted by guraknugen View Post
    Of course I know that I can just copy the data from one to the other in a loop, but if it was possible to just ”type cast” it, it would execute much faster, wouldn't it? Would be a nice shortcut…
    You can use memcpy for this if you know they are the same size in bytes and if you know they have the same memory layout.

    Code:
    memcpy(new_array, old_array, sizeof(new_array));

  4. #4
    Registered User
    Join Date
    May 2012
    Posts
    1,066
    A solution using type casting would be:
    Code:
    #include <stdio.h>
     
    #define COLS 2 
    // or whatever you need
    
    int main(void)
    {
        int arr1d[] = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12};
        int (*arr2d)[COLS] = (int (*)[COLS]) arr1d;
        int i, j;
        int a = 0;   // for indexing into arr1d
        int rows = sizeof(arr1d)/(sizeof(arr1d[0]) * COLS);
    
        puts("Nr\tarr2d\t    arr1d");
        for (i = 0; i < rows; ++i)
            for (j = 0; j < COLS; j++)
                printf("%2d   %p   %p\n", arr2d[i][j], (void *) &arr2d[i][j], (void *) &arr1d[a++]);
        return 0;
    }
    arr2d is a pointer to an array of COLS ints, i.e. it is a pointer to a row of your two dimensional array (with x rows and COLS columns).
    The assignment on line 8 has the effect that it points to the beginning of your one dimensional array, thus you now treat the one dimensional array as an array with COLS columns and x rows (line 11 calculates the number of rows assuming that the one dimensional array has a multiple of COLS elements).

    The rest is simple pointer arithmetic. arr2d[i] points to the ith row, thus arr2d[i+1] points to the ith + 1 row. Since arr2d is a pointer to an array with COLS ints, COLS elements of your one dimensional array are skipped.

    The output will show you that you actually access the same element with both variables (arr1d and arr2d).

    Bye, Andreas

  5. #5
    Registered User
    Join Date
    Feb 2013
    Location
    Sweden
    Posts
    59
    Quote Originally Posted by Tclausex View Post
    I'm not even sure what a cast like that would look like. Have you tried?
    Well, I have done some thinking, but I consider myself some kind of a beginner so I'm not surprised that I didn't come up with anything to try, except some lame experiments that failed, like declaring two arrays and assigning one to the other. The compilator, run in gnu99 mode, didn't like that, of course.
    Quote Originally Posted by Tclausex View Post
    You can always take advantage of the fact that multidimensional arrays are laid out linearly as a 1D array in memory.

    Code:
    #include <stdio.h>
    
    
    #define ROWS 6
    #define COLS 5
    
    
    int main()
    {
        int arr[ROWS*COLS] = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12};
        int i, j;
        int *p = arr;
    
    
        for( i = 0; i < ROWS; ++i ){
            for( j = 0; j < COLS; ++j )
                printf("%2d ", *(p + i*COLS + j));
            printf("\n");
        }
    
    
    return 0;
    }

  6. #6
    Registered User
    Join Date
    Feb 2013
    Location
    Sweden
    Posts
    59
    Quote Originally Posted by c99tutorial View Post
    You can use memcpy for this if you know they are the same size in bytes and if you know they have the same memory layout.

    Code:
    memcpy(new_array, old_array, sizeof(new_array));
    Looks like a shortcut in that sense that the there is not much code to type, but wouldn't this take as long to execute as a double loop (one inside the other) would?

  7. #7
    Registered User
    Join Date
    Feb 2013
    Location
    Sweden
    Posts
    59
    This looks like what I was looking for! I'll take a closer look to it and do some tests later today. Thanks!

  8. #8
    Registered User
    Join Date
    Nov 2012
    Posts
    1,056
    Quote Originally Posted by guraknugen View Post
    Looks like a shortcut in that sense that the there is not much code to type, but wouldn't this take as long to execute as a double loop (one inside the other) would?
    memcpy will copy each byte of the old_array to the new_array, so if old_array is very large and/or you do the memcpy very often in your algorithm, then it could impact performance. On the other hand, if the arrays are small then you can think of it as a constant-time operation.

  9. #9
    Registered User
    Join Date
    Feb 2013
    Location
    Sweden
    Posts
    59
    Quote Originally Posted by c99tutorial View Post
    memcpy will copy each byte of the old_array to the new_array, so if old_array is very large and/or you do the memcpy very often in your algorithm, then it could impact performance. On the other hand, if the arrays are small then you can think of it as a constant-time operation.
    Well, I don't have my project completely defined yet, but I am pretty sure it's going to happen only once per session. Once I have my two dimension array, the program will work with it until the end. On the other hand, it's not small at all. It's audio samples, so it's size will depend on the audio file that it's going to process. A three minute stereo file at 44.1 kHz sampling rate means an array like AudioData[2][7938000]…
    Last edited by guraknugen; 02-13-2013 at 09:42 AM. Reason: Bad spelling

  10. #10
    Registered User
    Join Date
    Nov 2012
    Posts
    1,056
    Quote Originally Posted by guraknugen View Post
    A three minute stereo file at 44.1 kHz sampling rate means an array like AudioData[2][7938000]…
    Ok, let's assume you use uint32_t for each element (4 bytes). Then your array is no larger than 4 * 2 * 8 = 64 million bytes. Typically computers can transfer billions of bytes in the memory per second. In other words, I don't think it will matter if you do this one time.

    By the way, if you look at programs that handle uncompressed audio like Audacity you will see that they don't typically load the entire sample into memory at one time. They load chunks of it and keep almost all of it on disk. As your audio data becomes larger, this strategy will be more and more useful. You could theoretically load everything into dynamic memory and just let the operating system handle swapping it to disk for you when it gets too large, but this strategy usually leads to programs which are not responsive to user input.

  11. #11
    Registered User
    Join Date
    Feb 2013
    Location
    Sweden
    Posts
    59
    Quote Originally Posted by c99tutorial View Post
    Ok, let's assume you use uint32_t for each element (4 bytes). Then your array is no larger than 4 * 2 * 8 = 64 million bytes. Typically computers can transfer billions of bytes in the memory per second. In other words, I don't think it will matter if you do this one time.
    Actually I am going to convert all the data to double (8 bytes) or long double (12 bytes), since I want to do some maths on them. I think long double will be overkill, so I guess double is what I'm going to use. The input file will in most cases be 24-bit FLACs, and I don't think I will support other formats, at least no lossy ones.
    Quote Originally Posted by c99tutorial View Post
    By the way, if you look at programs that handle uncompressed audio like Audacity you will see that they don't typically load the entire sample into memory at one time. They load chunks of it and keep almost all of it on disk. As your audio data becomes larger, this strategy will be more and more useful. You could theoretically load everything into dynamic memory and just let the operating system handle swapping it to disk for you when it gets too large, but this strategy usually leads to programs which are not responsive to user input.
    You mean using buffers?
    Since I'm not a very experienced programmer, I think that will be too complicated for me at the moment. The audio signal will be searched for certain things (peaks and stuff), and when something is found, there is going to be some processing around the signal. That means that if I find something at the very beginning of a buffer, I need to go back to last buffer and so on. So I think it will be very much easier if I work with the whole file loaded in memory, giving me freedom to go back and forth the way I need, I am mainly going to process 2-5 minute music stereo files, no big concerts with 5.1 sound or anything like that, so I don't think memory usage will be a big issue.

  12. #12
    Registered User
    Join Date
    Feb 2013
    Location
    Sweden
    Posts
    59
    Done some tests now and yes, that was exactly what I was looking for.
    I hope it works with dynamic arrays too. Well, I see no reason why it wouldn't.

    The only thing I could hope for now is to find a way to transpose the array without copying things around, but I guess that is not possible…

  13. #13
    Registered User
    Join Date
    Nov 2012
    Posts
    1,056
    Quote Originally Posted by guraknugen View Post
    The only thing I could hope for now is to find a way to transpose the array without copying things around, but I guess that is not possible…
    At least with the routines in Lapack, there is a parameter that lets you choose the format of your input array. In other words, they let you use it in whichever way it is in without having to transpose it to a specific format. So if you have that option, transposing might not be needed; you just tell the routine how it is and it carries out its job as is. However, unless you have an algorithm which is continuously transposing a large array, I don't see how transposing would lead to a performance problem.

  14. #14
    Registered User
    Join Date
    Feb 2013
    Location
    Sweden
    Posts
    59
    Seems like I need to work with buffers after all.
    Did a simple test with a 2D-array of Doubles:
    Code:
    double a[INDEXMAX][2];
    When INDEXMAX was set to a little more than 5×10⁵, there was a segmentation fault. There is no exact value where this happens. Sometimes one value results in a segmentation fault, sometimes not, but the limit seems to be somewhere around 5.0×10⁵ ↔ 5.5×10⁵.

    A double seems to be 64 bits on my machine. 8 bytes, that is, so 5×10⁵ ”rows” and 2 ”columns” should mean 8 MB, which is not that much. Seems like I have a lot more than that free.
    Code:
    $ free -m
                 total       used       free     shared    buffers     cached
    Mem:          2013       1881        131          0         98        806
    -/+ buffers/cache:        976       1036
    Swap:         1905         51       1854
    $
    Is there a way around this…?

    Oh, and my simple test code looks like this:
    Code:
    #include <stdlib.h>
    
    #define INDEXMAX 525000
    
    int main(int argc, char *argv[])
    {
        double a[INDEXMAX][2];
            
         for (size_t Sample=0; Sample<INDEXMAX; Sample++)
            for (size_t Ch=0; Ch<2; Ch++)
                a[Sample][Ch]=(double)Sample;
    
        return 0;
    }
    Command used to compile:
    Code:
    gcc -std=gnu99 -o Test Test.c

  15. #15
    Registered User
    Join Date
    May 2012
    Posts
    1,066
    Code:
    int main(int argc, char *argv[])
    {
        double a[INDEXMAX][2];
    The array is stored on the stack, thus the command to check how much stack memory you are allowed to use is
    Code:
    ulimit -s
    Quote Originally Posted by guraknugen View Post
    Is there a way around this…?
    Create your array on the heap (using malloc())

    Bye, Andreas

Page 1 of 2 12 LastLast
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. converting char 1d array to 2d array
    By wasi.cjr in forum C Programming
    Replies: 8
    Last Post: 03-19-2011, 04:21 PM
  2. Converting tchar array to string array
    By GeekInTraining in forum C++ Programming
    Replies: 2
    Last Post: 01-17-2011, 12:34 AM
  3. Problem converting from char array to int array.
    By TheUmer in forum C Programming
    Replies: 11
    Last Post: 03-26-2010, 11:48 AM
  4. Replies: 8
    Last Post: 11-12-2008, 10:25 AM
  5. Converting character array to integer array
    By quiet_forever in forum C++ Programming
    Replies: 5
    Last Post: 04-02-2007, 05:48 AM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21