Thread: Coding for Nintendo 64: Memory issue

Hello everyone (I'm so sorry for the length of this post)!
I'm currently struggling to run my fairly simple code without crashing.

I hope I can explain the problem well, I'll first describe the circumstances, then, second, how my code is supposed to work and third show the code itself.

1. The problem
I'm programming for the old N64 console. That is important because I can not run the code I write on my PC, I do not have the official dev-kit and thus can only test my C code by running it on the actual hardware (without a real debugger, I can draw simple text though).
What I'm trying to do is show a very long series of spritemaps as a "film".
Example of a spritemap:
http://atomicrobotdesign.com/blog_me...es/gb_walk.png
Since the N64 has 4MB RAM and I don't have the libraries to (de)compress the images, they are quite large in memory and I can't load the complete sequence at once.
So I'm always showing one spritemap (a series of images), while loading the next part.
When the first map is done playing, I switch it with the second one that was loaded while playing, free the old one and start loading the third, while the second is playing - and so on.

So there should always be only 1 or 2 sprites in memory, each about 1.5MB and thus small enough for the RAM.

2. The "solution"
To do that I came up with a simple queue-structure, containing an array of filepaths (where each spritemap must be loaded from), a respective array of sprite-pointers (will point to the actual pixel-data) and two indices.
One index for appending sprite-paths to the queue and one index for loading.

Whenever I push a filepath to the queue, it is appended at the append-index and the append-index is incremented.

Whenever I call the load() - function, I load the file-path at the load-index and store the data in the sprite-array at the load-index.
The load-index is then incremented.
The load function however only executes if I have enough memory left to load the file.
Each time I load a file I add to a counter by the amount of bytes I allocate.
I also subtract from that counter whenever freeing a sprite.

When popping (returning the 0th sprite-pointer in the queue), I shift both the sprite- and path-array to the left, so the 0th index will be overwritten by the 1st (and so on).

For the current case, I know that each spritemap I load is roughly 1.5 MB big.
That means there should only 2 be loaded at the same time, loading a third would result in a crash (and I think that's what is happening all the time).

3. The (simplified) code

the queue-structure:

Code:

#define QUEUE_MAX 16
typedef struct
{
    char paths[QUEUE_MAX][64];
    sprite_t* sprites[QUEUE_MAX];
    int append_index;
    int load_index;
} sprite_queue;

pushing a file-path to the queue, so it will be loaded at some point:

Code:

void push(char path[])
{
    if(queue->append_index == QUEUE_MAX) 
    {
        tools_print("sprite loading queue full!");
        return;
    }

    snprintf(queue->paths[queue->append_index], 64, "%s", path);
    
    queue->append_index++;
}

the load-function which gets called ~10 times a second

Code:

void load_next()
{

    // only load if we have not already loaded all sprites in the queue

    if(queue->load_index == queue->append_index)
    {
        // tools_print("nothing to load!");
        return;
    }

    // only load if we have enough space left
    // graphics_memory is set to 1.8MB so another 1.5MB file fits in if we are  
    // just below

    if(gfxBytes >= graphics_memory) // if number of allocated bytes < than limit of 1.8 MB
    {
        tools_print("waiting for memory to be freed");
        return;
    }

    queue->sprites[queue->load_index] = load_sprite(queue->paths[queue->load_index]);
    
    sprite_loading_queue->load_index++;
}

pop-function:

Code:

sprite_t* pop()
{
    sprite_t* ret = queue->sprites[0]; // gonna return this

    for(int i = 0 ; i < sprite_loading_queue->append_index - 1; i++) // shift 1 left
    {
        queue->sprites[i] = queue->sprites[i + 1];
        snprintf(queue->paths[i], 64, queue->paths[i + 1]);
        queue->sprites[i] = queue->sprites[i + 1];
    }

    queue->append_index--;
    queue->load_index--;
    return ret;
}

main game-loop:

Code:

char baseName[9] = {"dirPath/"}; // length + 1 for terminator
char spriteName[64];
sprite_t* play0 = load_sprite("dirPath/0.sprite");

push("dirPath/1.sprite");

int frame = 1, maxFrame = 33, offset = 0, maxOffset = 35;

// frame is the current number of the sprite, offset is the tile IN the sprite
// so here whe have a maximum of 33 sprites and 36 tiles each

while(1) 
{
        if(offset == maxOffset) // last tile reached
        {
            if(has_next()) // next sprite is ready
            {
                offset = 0;
                if(frame == maxFrame) // last frame reached
                {
                    frame = 0;
                } else frame++;

                free_sprite(play); // frees and subtracts the number of bytes from counter
                play = pop(); // get the loaded sprite
                snprintf(spriteName, 64, "%s%d.sprite", baseName, frame);
                push(spriteName);
            } 

        } else offset++;
    
       // do sth cool with the sprite here
}

freeing a sprite and subtracting from byte-cound:

Code:

void free_sprite(sprite_t *sprite)
{
    if(sprite == NULL) return;

    // size of sprite in bytes is the size of the sprite_t structure + the number of pixels in the pixel-data array (an array of uint16_t, which is a flexible array member)

    int s = -sizeof(uint16_t) 
                * (int) (sprite->width)
                * (int) (sprite->height)
                - sizeof(sprite_t); 
    free(sprite);
    tools_print("freed");
    tools_changeGfxBytes(s); // add the negative s to the byte-count
    // i do the same thing the other way around (positive) when loading a sprite
}

I hope you can make out an error here, because I really can't.
What I found out so far is this:
There is no constant memory leak, I tried running the code with smaller images, those were loaded again and again and again and the program never crashed.

With the large images the code always crashes after...
- loading the first sprite // ~1.5 MB in use now
- pushing the next filepath onto the queue
- loading that path // ~ 3.0 MB in use now
- freeing the first sprite // ~ 1.5 MB in use now
- popping the new sprite
- pushing the next filepath onto the queue
then the crash.

What happens almost immediately after that last push must have been the loading method, somehow allocating more memory than there was left in RAM.
But how is that possible if I just "freed it down to" 1.5 MB?

Is my queue garbage (that's my best guess right now)?

Thanks if you made it all the way here and I would be grateful for any help.

Originally Posted by fulumbler

I do not load anything else though, shouldn't there always be enough contiguous space at all times?

Of course not, I'm an idiot.

The idea of not constantly freeing and allocating did not even come to my mind.
I'm doing that now and got it work, thank you so much for your time Salem!
Sadly I can't really say what the initial error was, but your way makes it less complicated and that certainky helps.
Cya!