Thread: How to implement an expandable array with malloc without interfering the next array?

Originally Posted by wolfrose

Hello,

I'm trying to develop a dynamic array with malloc without interfering the next array's element.

I tested the code, but didn't work. I run over the first element of the second array which is 99, so my implementation for malloc is not right obviously.

Code:

uint8_t *ar_ptr;uint8_t array1[]={77,11,11,11,11,11};
uint8_t array2[]={99,22,22,22};
int main(void) {


ar_ptr = (uint8_t*)malloc(7*sizeof(uint8_t));
ar_ptr = array1;
printf("%d\n",*(ar_ptr+6));   // here still getting 99 of the second array
    return 0;
}

First of all, format your code correctly:

Code:

uint8_t *ar_ptr;
uint8_t array1[]={77,11,11,11,11,11};
uint8_t array2[]={99,22,22,22};

int main(void) {
 
 
   ar_ptr = malloc(7*sizeof(uint8_t));
   ar_ptr = array1;
   printf("%d\n",*(ar_ptr+6));   // here still getting 99 of the second array
   return 0;
}

1) You do not #include the header files you need
2) You create two global arrarys, and initialize them with data, along with a pointer.
3) You allocate data on the heap.
4) You throw away the data space you have just allocated!!!
5) You point the pointer to the GLOBAL array1.
6) You then access the 7th element of a 6 element array!!!!
7) You then question why you are accessing the first element of the second array!!!

Doctor! Doctor! It hurts when I try to access data outside the bounds of the array!!!

There is NO wall between arrays, global, or on the heap!
You can ONLY SAFELY access elements 0 - 5 of the GLOBAL array1!!!


Please don't cast the return value from malloc()!!!

Yes, I'm sorry I didn't explain my original reason for this snippet!

I'm actually trying to write a code to manipulate an array of 768 bytes, and then I want to insert other bytes in this array.

So I learned about appending data into an array. So, that's mean, I have to expand the array size every time I include a byte into the array.

For example, the test code I posted in this thread, is a simple test where I want to expand array1 and also not interfering the next array.

Which means my approach to expand array1 isn't working as I want, that's why I developed the code this way.

And that's why I'm testing with reaching the 6th element which I want it to be the new appended byte I developed with malloc but the my example didn't work.

So, my question is how to expand array1.

This is my current code:

1. The header code.

Code:

#ifndef sketch_h_#define sketch_h_


//////////////////////////////////////////////////////////////////
//////////////// BLOCK OF ASCII CAHRS AND SYMBOLS ////////////////
//////////////////////////////////////////////////////////////////


uint8_t first_ascii_group[32]={
    0x20, 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27,
    0x28, 0x29, 0x2a, 0x2b, 0x2c, 0x2d, 0x2e, 0x2f,
    0x30, 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37,
    0x38, 0x39, 0x3a, 0x3b, 0x3c, 0x3d, 0x3e, 0x3f};




char font8x8_basic[] = {
     //0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,   // U+0000 (nul)
     0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,   // U+0020 (space)
     0x18, 0x3C, 0x3C, 0x18, 0x18, 0x00, 0x18, 0x00,   // U+0021 (!)
     0x36, 0x36, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,   // U+0022 (")
     0x36, 0x36, 0x7F, 0x36, 0x7F, 0x36, 0x36, 0x00,   // U+0023 (#)
     0x0C, 0x3E, 0x03, 0x1E, 0x30, 0x1F, 0x0C, 0x00,   // U+0024 ($)
     0x00, 0x63, 0x33, 0x18, 0x0C, 0x66, 0x63, 0x00,   // U+0025 (%)
     0x1C, 0x36, 0x1C, 0x6E, 0x3B, 0x33, 0x6E, 0x00,   // U+0026 (&)
     0x06, 0x06, 0x03, 0x00, 0x00, 0x00, 0x00, 0x00,   // U+0027 (')
     0x18, 0x0C, 0x06, 0x06, 0x06, 0x0C, 0x18, 0x00,   // U+0028 (()
     0x06, 0x0C, 0x18, 0x18, 0x18, 0x0C, 0x06, 0x00,   // U+0029 ())
     0x00, 0x66, 0x3C, 0xFF, 0x3C, 0x66, 0x00, 0x00,   // U+002A (*)
     0x00, 0x0C, 0x0C, 0x3F, 0x0C, 0x0C, 0x00, 0x00,   // U+002B (+)
     0x00, 0x00, 0x00, 0x00, 0x00, 0x0C, 0x0C, 0x06,   // U+002C (,)
     0x00, 0x00, 0x00, 0x3F, 0x00, 0x00, 0x00, 0x00,   // U+002D (-)
     0x00, 0x00, 0x00, 0x00, 0x00, 0x0C, 0x0C, 0x00,   // U+002E (.)
     0x60, 0x30, 0x18, 0x0C, 0x06, 0x03, 0x01, 0x00,   // U+002F (/)
     0x3E, 0x63, 0x73, 0x7B, 0x6F, 0x67, 0x3E, 0x00,   // U+0030 (0)
     0x0C, 0x0E, 0x0C, 0x0C, 0x0C, 0x0C, 0x3F, 0x00,   // U+0031 (1)
     0x1E, 0x33, 0x30, 0x1C, 0x06, 0x33, 0x3F, 0x00,   // U+0032 (2)
     0x1E, 0x33, 0x30, 0x1C, 0x30, 0x33, 0x1E, 0x00,   // U+0033 (3)
     0x38, 0x3C, 0x36, 0x33, 0x7F, 0x30, 0x78, 0x00,   // U+0034 (4)
     0x3F, 0x03, 0x1F, 0x30, 0x30, 0x33, 0x1E, 0x00,   // U+0035 (5)
     0x1C, 0x06, 0x03, 0x1F, 0x33, 0x33, 0x1E, 0x00,   // U+0036 (6)
     0x3F, 0x33, 0x30, 0x18, 0x0C, 0x0C, 0x0C, 0x00,   // U+0037 (7)
     0x1E, 0x33, 0x33, 0x1E, 0x33, 0x33, 0x1E, 0x00,   // U+0038 (8)
     0x1E, 0x33, 0x33, 0x3E, 0x30, 0x18, 0x0E, 0x00,   // U+0039 (9)
     0x00, 0x0C, 0x0C, 0x00, 0x00, 0x0C, 0x0C, 0x00,   // U+003A (:)
     0x00, 0x0C, 0x0C, 0x00, 0x00, 0x0C, 0x0C, 0x06,   // U+003B (//)
     0x18, 0x0C, 0x06, 0x03, 0x06, 0x0C, 0x18, 0x00,   // U+003C (<)
     0x00, 0x00, 0x3F, 0x00, 0x00, 0x3F, 0x00, 0x00,   // U+003D (=)
     0x06, 0x0C, 0x18, 0x30, 0x18, 0x0C, 0x06, 0x00,   // U+003E (>)
     0x1E, 0x33, 0x30, 0x18, 0x0C, 0x00, 0x0C, 0x00,   // U+003F (?) // upto here
     0x3E, 0x63, 0x7B, 0x7B, 0x7B, 0x03, 0x1E, 0x00,   // U+0040 (@)
     0x0C, 0x1E, 0x33, 0x33, 0x3F, 0x33, 0x33, 0x00,   // U+0041 (A)
     0x3F, 0x66, 0x66, 0x3E, 0x66, 0x66, 0x3F, 0x00,   // U+0042 (B)
     0x3C, 0x66, 0x03, 0x03, 0x03, 0x66, 0x3C, 0x00,   // U+0043 (C)
     0x1F, 0x36, 0x66, 0x66, 0x66, 0x36, 0x1F, 0x00,   // U+0044 (D)
     0x7F, 0x46, 0x16, 0x1E, 0x16, 0x46, 0x7F, 0x00,   // U+0045 (E)
     0x7F, 0x46, 0x16, 0x1E, 0x16, 0x06, 0x0F, 0x00,   // U+0046 (F)
     0x3C, 0x66, 0x03, 0x03, 0x73, 0x66, 0x7C, 0x00,   // U+0047 (G)
     0x33, 0x33, 0x33, 0x3F, 0x33, 0x33, 0x33, 0x00,   // U+0048 (H)
     0x1E, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x1E, 0x00,   // U+0049 (I)
     0x78, 0x30, 0x30, 0x30, 0x33, 0x33, 0x1E, 0x00,   // U+004A (J)
     0x67, 0x66, 0x36, 0x1E, 0x36, 0x66, 0x67, 0x00,   // U+004B (K)
     0x0F, 0x06, 0x06, 0x06, 0x46, 0x66, 0x7F, 0x00,   // U+004C (L)
     0x63, 0x77, 0x7F, 0x7F, 0x6B, 0x63, 0x63, 0x00,   // U+004D (M)
     0x63, 0x67, 0x6F, 0x7B, 0x73, 0x63, 0x63, 0x00,   // U+004E (N)
     0x1C, 0x36, 0x63, 0x63, 0x63, 0x36, 0x1C, 0x00,   // U+004F (O)
     0x3F, 0x66, 0x66, 0x3E, 0x06, 0x06, 0x0F, 0x00,   // U+0050 (P)
     0x1E, 0x33, 0x33, 0x33, 0x3B, 0x1E, 0x38, 0x00,   // U+0051 (Q)
     0x3F, 0x66, 0x66, 0x3E, 0x36, 0x66, 0x67, 0x00,   // U+0052 (R)
     0x1E, 0x33, 0x07, 0x0E, 0x38, 0x33, 0x1E, 0x00,   // U+0053 (S)
     0x3F, 0x2D, 0x0C, 0x0C, 0x0C, 0x0C, 0x1E, 0x00,   // U+0054 (T)
     0x33, 0x33, 0x33, 0x33, 0x33, 0x33, 0x3F, 0x00,   // U+0055 (U)
     0x33, 0x33, 0x33, 0x33, 0x33, 0x1E, 0x0C, 0x00,   // U+0056 (V)
     0x63, 0x63, 0x63, 0x6B, 0x7F, 0x77, 0x63, 0x00,   // U+0057 (W)
     0x63, 0x63, 0x36, 0x1C, 0x1C, 0x36, 0x63, 0x00,   // U+0058 (X)
     0x33, 0x33, 0x33, 0x1E, 0x0C, 0x0C, 0x1E, 0x00,   // U+0059 (Y)
     0x7F, 0x63, 0x31, 0x18, 0x4C, 0x66, 0x7F, 0x00,   // U+005A (Z)
     0x1E, 0x06, 0x06, 0x06, 0x06, 0x06, 0x1E, 0x00,   // U+005B ([)
     0x03, 0x06, 0x0C, 0x18, 0x30, 0x60, 0x40, 0x00,   // U+005C (\)
     0x1E, 0x18, 0x18, 0x18, 0x18, 0x18, 0x1E, 0x00,   // U+005D (])
     0x08, 0x1C, 0x36, 0x63, 0x00, 0x00, 0x00, 0x00,   // U+005E (^)
     0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xFF,   // U+005F (_)
     0x0C, 0x0C, 0x18, 0x00, 0x00, 0x00, 0x00, 0x00,   // U+0060 (`)
     0x00, 0x00, 0x1E, 0x30, 0x3E, 0x33, 0x6E, 0x00,   // U+0061 (a)
     0x07, 0x06, 0x06, 0x3E, 0x66, 0x66, 0x3B, 0x00,   // U+0062 (b)
     0x00, 0x00, 0x1E, 0x33, 0x03, 0x33, 0x1E, 0x00,   // U+0063 (c)
     0x38, 0x30, 0x30, 0x3e, 0x33, 0x33, 0x6E, 0x00,   // U+0064 (d)
     0x00, 0x00, 0x1E, 0x33, 0x3f, 0x03, 0x1E, 0x00,   // U+0065 (e)
     0x1C, 0x36, 0x06, 0x0f, 0x06, 0x06, 0x0F, 0x00,   // U+0066 (f)
     0x00, 0x00, 0x6E, 0x33, 0x33, 0x3E, 0x30, 0x1F,   // U+0067 (g)
     0x07, 0x06, 0x36, 0x6E, 0x66, 0x66, 0x67, 0x00,   // U+0068 (h)
     0x0C, 0x00, 0x0E, 0x0C, 0x0C, 0x0C, 0x1E, 0x00,   // U+0069 (i)
     0x30, 0x00, 0x30, 0x30, 0x30, 0x33, 0x33, 0x1E,   // U+006A (j)
     0x07, 0x06, 0x66, 0x36, 0x1E, 0x36, 0x67, 0x00,   // U+006B (k)
     0x0E, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x1E, 0x00,   // U+006C (l)
     0x00, 0x00, 0x33, 0x7F, 0x7F, 0x6B, 0x63, 0x00,   // U+006D (m)
     0x00, 0x00, 0x1F, 0x33, 0x33, 0x33, 0x33, 0x00,   // U+006E (n)
     0x00, 0x00, 0x1E, 0x33, 0x33, 0x33, 0x1E, 0x00,   // U+006F (o)
     0x00, 0x00, 0x3B, 0x66, 0x66, 0x3E, 0x06, 0x0F,   // U+0070 (p)
     0x00, 0x00, 0x6E, 0x33, 0x33, 0x3E, 0x30, 0x78,   // U+0071 (q)
     0x00, 0x00, 0x3B, 0x6E, 0x66, 0x06, 0x0F, 0x00,   // U+0072 (r)
     0x00, 0x00, 0x3E, 0x03, 0x1E, 0x30, 0x1F, 0x00,   // U+0073 (s)
     0x08, 0x0C, 0x3E, 0x0C, 0x0C, 0x2C, 0x18, 0x00,   // U+0074 (t)
     0x00, 0x00, 0x33, 0x33, 0x33, 0x33, 0x6E, 0x00,   // U+0075 (u)
     0x00, 0x00, 0x33, 0x33, 0x33, 0x1E, 0x0C, 0x00,   // U+0076 (v)
     0x00, 0x00, 0x63, 0x6B, 0x7F, 0x7F, 0x36, 0x00,   // U+0077 (w)
     0x00, 0x00, 0x63, 0x36, 0x1C, 0x36, 0x63, 0x00,   // U+0078 (x)
     0x00, 0x00, 0x33, 0x33, 0x33, 0x3E, 0x30, 0x1F,   // U+0079 (y)
     0x00, 0x00, 0x3F, 0x19, 0x0C, 0x26, 0x3F, 0x00,   // U+007A (z)
     0x38, 0x0C, 0x0C, 0x07, 0x0C, 0x0C, 0x38, 0x00,   // U+007B ({)
     0x18, 0x18, 0x18, 0x00, 0x18, 0x18, 0x18, 0x00,   // U+007C (|)
     0x07, 0x0C, 0x0C, 0x38, 0x0C, 0x0C, 0x07, 0x00,   // U+007D (})
     0x6E, 0x3B, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,   // U+007E (~)
     0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00    // U+007F
};


void append_element(uint8_t *array_op, uint16_t array_op_size, uint8_t *array_ip, uint16_t array_ip_size);
#endif // sketch_h_

2. The source code.

Code:

#include <stdio.h>#include <stdlib.h>
#include <inttypes.h>
#include <string.h>
#include "sketch.h"


int16_t im,km;
uint16_t i,k,e,l,pos=0;
uint8_t *ar_ptr;
uint8_t array1[]={77,11,11,11,11,11};
uint8_t array2[]={99,22,22,22};
uint16_t array1_size=sizeof(array1);
uint16_t array2_size=sizeof(array2);
int main(void) {
ar_ptr = malloc(7*sizeof(uint8_t));
ar_ptr = array1;
printf("%d\n",*(ar_ptr+6));
    return 0;
}

I want to append each byte of

Code:

first_ascii_group

after each 8th byte of

Code:

font8x8_basic

How?

You will need to copy the individual elements from one array to another either by iteration or by using something like memcpy().

I did some tests and found it works as a start.

I doubt that it is really doing what you think it is. If you try to "assign" one array to another all you're doing is copying the pointers.

I found the idea of assigning in this link:

Yes that link is copying the individual elements (a[1] = array[1]), not trying to assign one array to another (a = array). And note that the destination array must be large enough to hold the results of the "additions".

If the array isn't initialized then it's possible to develop dynamic array based on input size to malloc function but if it's initialized then it's not possible.

The issue is not with initialization it's about the arrays having compile times sizes. You can use malloc() to allocate the size dynamically, then initialize the elements, and then shift and add elements, but if the arrays have compile time constant sizes their sizes are fixed and can't be altered.

Code:

int array1[12]= {12, 5, 3};  // This array will always have a size of 12. The first 3 elements are initialized with the sizes indicated the remaining elements are "default" initialized.

int array2[] = {22, 4, 6}; // This array will always have a size of 3.

int *array3;   // A pointer to an int.

int *array3 = malloc(6 * sizeof(int));  // Create an array of size 6 using dynamic memory allocation.

// Now initialize the elements.
for(int i = 0; i < 3; ++i)
   array3[i] = array1[i];
for(int i = 0, j = 3; j < 6; ++i, ++j)
   array3[j] = array2[i];

// Or use memcpy().
int *array4 = malloc(6 * sizeof(int));
memcpy(array3, array1, 3 * sizeof(int));
memcpy(&array3[3], array2, 3 * sizeof(int));

// And don't forget to free what you malloc().
free(array3);
free(array4);

Note the above code has not been tried so it may contain typos or other problems but it should give you an idea of how to start. For more information find and study documentation for the standard functions you're trying to use, ie, malloc(), the sizeof() operator, memcpy(), free(), etc.

Originally Posted by jimblumberg

You will need to copy the individual elements from one array to another either by iteration or by using something like memcpy().

I doubt that it is really doing what you think it is. If you try to "assign" one array to another all you're doing is copying the pointers.

Yes that link is copying the individual elements (a[1] = array[1]), not trying to assign one array to another (a = array). And note that the destination array must be large enough to hold the results of the "additions".

The issue is not with initialization it's about the arrays having compile times sizes. You can use malloc() to allocate the size dynamically, then initialize the elements, and then shift and add elements, but if the arrays have compile time constant sizes their sizes are fixed and can't be altered.

Note the above code has not been tried so it may contain typos or other problems but it should give you an idea of how to start. For more information find and study documentation for the standard functions you're trying to use, ie, malloc(), the sizeof() operator, memcpy(), free(), etc.

Sorry didn't included my append developed function.

Code:

void append_element(uint8_t *array_op, uint16_t array_op_size, uint8_t *array_ip, uint16_t array_ip_size){
    for (km=1;km>=0;km--)                             // no of target input, for appending one element
    {
        printf("before appending\n");
        for (i=0;i<array_op_size-1;i++)
        {
            printf("%d\n",array_op[i]);
        }


        for (im=array_op_size-1;im>=pos;im--)
        {
            array_op[im+1]=array_op[im];                // separating data
        }


        array_op[pos]=array2[0];                      // appending only one element for testing


        printf("after appending\n");
        for (i=0;i<array_op_size+1;i++)
        {
            printf("%d\n",array_op[i]);
        }
    }
}

Originally Posted by jimblumberg

This is only one approach, and the array will be limited by the amount of stack space.

OK, that's clear. So the stack is the current running memory on RAM, and heap is the remaining RAM, this is the best so far I understood about stack and heap. While program memory is what's in HDD or flash memory in case of microcontrollers.

Why does array1 still need to be static? Even if it is it could be local to a function that is only used to initialize the dynamic array. Then when that function finishes the static array would no longer be in memory.

Because I'm saving it in the header file, isn't it the best place for such a big array, I mean fonr8x8_basic.
Local is OK, I understand that what's in local on the stack, is ended after the function finish.

Because you only need to free() what you malloc(), otherwise you'll have a memory leak.

OK, but my new data is in array3. Now, what to do to hold this data for the next appending or during program run.

Should I after appending all the data, save that in a new static array with the new size and then free array3?


.................................................. .................................................. .................................................. ..............

I have a question, just trying some snippets with an array.

Code:

uint8_t ASCII_HEX[224];
for (k=32;k<256;k++){
ASCII_HEX[k]=k;}


for (l=32;l<256;l++)
{
    printf("%d ",l);
    if (l%8==7)printf("\n");
    if (l%32==31)printf("\n");
}

This code causes a crash, why? There's no error or a warning, I'm within the array rang, and not exceeding it's size which is 244 elements. As k starts from 32 and ends at 256, that's 224.

OK, I was changing the value of k, a strange behavior when I adjust k end value 239, start value 32, the program runs without a stop, why?

This is the code:

Code:

uint8_t ASCII_HEX[224];uint16_t ascii_hex_size=sizeof(ASCII_HEX);
printf("%d\n",ascii_hex_size);
for (k=32;k<239;k++){
ASCII_HEX[k]=k;}


for (l=32;l<256;l++)
{
    printf("%d ",l);
    if (l%8==7)printf("\n");
    if (l%32==31)printf("\n");
}

Originally Posted by Sir Galahad

The easiest way to think about it is that the stack is a certain percentage of RAM that the OS has set aside for your program, which then uses it to declare variables, pass function parameters, etc. A stack overflow occurs when you exceed the space alloted to your program, and as a result your program terminates in a most ungraceful way.

The heap is also just a stretch of RAM, but instead of the compiler generating code to manage it directly, your program is instead responsible for keeping track of what and when to allocate/deallocate. Once the heap is exhausted, you're program actually has a chance to mitigate the problem (by checking the return value of malloc/realloc/etc for NULL). An unchecked failure to allocate memory from the heap of course typically results in sudden program failure as well.

That's really good to understand, thank you so much for this clarification.

The best approach is to avoid static arrays entirely if possible, use them for hard-coded data and nothing else. They only serve to bloat your program anyway. Instead, just pull from the heap or declare all your fixed-sized arrays within a function.

OK, so as the big array I shown above, I put it in the header file as many libraries I download or find in Arduino libraries folder. I thought it's common to put arrays in the header file.

But now you tell me that it's better to:
1. Pull them from the heap, and that's by identifying a pointer and just copy the data to it.
2. Put it within the function, so; for example, define the function as a void or with returned value and just call it with or without parameters.

No. If your buffer contains 256 elements then the ONLY valid indexes are 0 to 255 inclusive. Indexes at or above 256 are out of bounds and thus buffer overflow material.

This code rolls over and over where the index isn't overflow, it's defined as a 16-bits variable.

k and l are 16-bits and the for loop rolls over infinitely. It's really interesting, why that happens?

Code:

uint8_t ASCII_HEX[224];
uint16_t l, k, ascii_hex_size=sizeof(ASCII_HEX);
printf("%d\n",ascii_hex_size);
for (k=32;k<239;k++){
ASCII_HEX[k]=k;}

for (l=32;l<256;l++)
{
    printf("%d ",l);
    if (l%8==7)printf("\n");
    if (l%32==31)printf("\n");
}