Switch from .ms3d to the ASCII format?

Well, it seems that There are alot of problems with loading an MS3D Model into my data structures...

I have VertexData, MaterialData, and PositionData to render the object...

Problem is.. the .ms3d file has VertexData, TriangleData, MeshData, and Material Data...


Now from what I hear the ASCII format of an MS3D Model is just a list of vertices, with materials defined...

Should I re-arrange my loader function and resourcecache system to stick with .ms3d format? Or just instead of dealing with all these weird data structures, convert to ACSII and keep my system the way it is?

Well if you design your class properly, you should be able to do all your testing with the ASCII representation to make debugging a lot easier, then switch over to using proper format files at the end without too much difficulty. At least you'll have a debugged system to help you sort out any issues.

I have no problem with adding support for an MS3D model format in my system..

But I really don't want to treat one data type better than the next, or differently for that matter..

I have my system set up so I create a vertexdata structure array of sizeof, referencing it and pushing the reference into a vector container for storage and easy access.

But, the MS3D format wraps up vertexdata into triangledata which is wrapped up in a mesh, which is weird. lol... It is logical, but its not basic enough to work in my system...

It'd be cool if I could just rip the vertex data from the file itself and not have to deal with the triangle structures part.. maybe this is possible..

I really don't have to define the triangles to draw them... I think that is just an organization tool.. Right?

Maybe I'll just pull vertex data and try to use glDrawElements or something like that and see if it works without much trouble...

If I'm not mistaken MS3D doesn't repeat vertices, so I don't think it will work that simple, I think that is where the use of the triangle structures is implemented. Arrange vertices into triangles so you only need one copy of each vertex, then just create triangles out of those vertices..

That might be the way to do it after all yknow?

I believe I have realized what I need to do...


I need to rip pure unadulterated vertex data from files, or from wherever I'm getting it...

And use an algorithm to create triangle structures that don't use repeating vertices...

Or is creating triangles out of a list of non repeating vertices a logical impossibility?


If I could implement this I could handle all data the same, be it a .MS3D file or some hard coded shapes...



If I can't do this I'm stuck back to square one, I'll have repeating vertices, no good.. What else can I do?

You need to implement index buffers. You seem to have a grasp on vertex buffers so index buffers are simply indices representing the triangles.

For instance a terrain is just a grid of vertices. But without using index buffers, you need 3 vertices for each triangle which means 6 vertices for each quad, 2 of which are shared by each tri in the quad. Using an index buffer you can create exactly how many vertices you need and then specify later how to 'connect the dots' so to speak.

Index buffers are not good if you wish to blow the model apart into it's component triangles. This would require uncoupling the render from the index buffer which can be done, but is easier if you don't use index buffers.

Well that seems like a cool plan, just add ++1 to the triangle index every 3 vertices...

But, to connect the dots, MS3D does it for you in their files...

That seems to pose a problem, Say I have a square made out of triangles, two, thats 6 vertices, 4 if I use the index buffer...

Lets see, how do I automate the connecting of the dots? Or do I have to hardcode it somewhere? That would be icky..


let me think..

For every triangle [k]
for every 3 indices [i]
set indices [i] = ??

Throw me a bone Bubbba :D...

You're right I understand how the vertex buffer works, but with just using a vertex buffer I'd end up repeating vertices...

EDIT: I'm googling as we speak, I'll report back with what I find out.. :D

There MUST be a pattern!

Code:

---

*-----*
| \    |
|  \  |
|    \ |
*-----*

---

Now, moving from the top left in a counter clockwise direction, we have

Vertex[0] // A
Vertex[1] // B
Vertex[2] // C
Vertex[3] // D

Two triangles, so to sets of VertexIndices..

Triangle[0].VertexIndices

VertexIndices[0] = Vertex[0] // A
VertexIndices[1] = Vertex[1] // B
VertexIndices[2] = Vertex[2] // C

Triangle[1].VertexIndices

VertexIndices[0] = Vertex[0] // C
VertexIndices[1] = Vertex[2] // D
VertexIndices[2] = Vertex[3] // A

I'm going to take a guess and say that

Triangle[3].Vertices // if there was one

VertexIndices[0] = Vertex[0] // A
VertexIndices[1] = Vertex[1] // D
VertexIndices[3] = Vertex[3] // E

And if there was another....

Triangle[4].Vertices

VertexIndices[0] = Vertex[0] // F
VertexIndices[0] = Vertex[0] // A
VertexIndices[0] = Vertex[0] // E

Hmmm, no consistencies I can point out, Unless someone can help me out?


My main goal is to figure out how each triangle figures out which vertices it uses, it HAS to be mathematical, otherwise you have to hard code it, which would be stupid..

No it's not mathematical.

Most 3D modelling programs embed the index information into the file. Manually doing the indexes would definitely be a pain in the arse. I'm not sure how you implement it in OpenGL but DirectX has built in functionality through the D3DX library for index buffers. I've implemented them in a DOS engine before but again you will probably want to assign the indices in the modeling program.

Now assigning indices for terrain is quite easy since all of my terrain is just a regular grid.
But assigning indices for something like the 3d model of a human head or something would not fit into a regular grid.

I'm leaning towards throwing indexes out the window for terrains and moving to triangle fans to render the terrain in blocks. Of course there are about a billion more things to figure out with this approach but that's the joy of it I reckon.

So to answer your question, each triangle doesn't know what vertices to use. A program tells it what vertices to use and that program knows because a human interacted with it.

Index buffers save memory by hopefully re-using vertex data because in any one mesh vertices are shared among several triangles and/or several primitive types. The index buffer allows you to say, hey use vertex 1, 2 and 3 for this tri and 2,3, and 4 for this one. Without index buffers a lot of your data is simply repreated which not only wastes memory, but it also begins to bottleneck the pipeline to the GPU. Vertex bottlenecks are the number one cause of framerate issues in any system. You must get the data down to a manageable size and index buffers are just ONE way of doing that.

Imagine a terrain that is 64x64.

Number of cells/quads: 63x63
Number of tris:(63*63)*2 (2 per quad)
Number of vertices:64*64
Number of indices:NumberOfTris*3 (3 indices per tri).

Quad1:
Vertex 0,1,2
Vertex 1,3,2

0........1
...........
...........
...........
2........3

This is for counter clockwise culling. That is if the winding order when the primitive is draw is counter clockwise we know the face of the quad is not facing the camera. This can also be computed using the dot product, but most rendering systems use the winding order since it does not require a computation.

Now without an index buffer here is the same terrain:

Number of cells/quads: 63*63
Number of tris: (63*63)*2
Number of vertices: (NumberOfTris *3) or (63*63*2*3)

0.........1/4
...............
...............
...............
...............
2/5......3/6

First tri: 0,1,2
Second tri: 4,6,5
Shared: 4,5

If this quad is in the middle of the grid, all 4 vertices are shared. See how quickly it adds up?

Perhaps one day when video is sandwiched into the CPU the bottleneck won't be there since video memory will be accessed at front side bus speeds, but that design also presents some major issues so I don't see it happening soon.

So should I bother to kill myself over getting index buffers in my program?

Should I use MS3D models for everything?

Or should I split my system up and load hard coded display list models or something without index buffers..

But then support index buffers for models through MS3D...

That'd be clunky.. Wouldn't it?

What to dooo, what to doo...

As usual I am confused with what your problem is, what exactly you are trying to do in order to fix the problem, and apprehensive about even bothering to help you.

What I do: my engine supports various file formats. These include Quake3 .bsp files and Milkshape3D files among other arcane ones. I don't have any sort of 'internal' file format...I use the SDK header files that I've downloaded whenever applicable for my data layout. Then, when I want to do certain 'high level' work that uses the data, the constructs that use it must know how to interpret the data, rather than trying to convert each small piece of data into some internal format. High level work means stuff like rendering.

My renderer keeps pointers to MS3D models, and face indexes to quake3 faces. Then, the renderer knows how to interpret the data to render it (which, by the way, is how all software works: there's a bunch of bytes, and the software knows how to interpret correctly as data/instructions).

The only 'internalization' that I do is to just maybe put pointers onto an STD::VECTOR, and where you hold the pointers in your engine depends on what you are trying to do.

So, my suggestion is this: just stick with trying to load the data from the milkshape3d file (.ms3d, what you've already been trying to use), just lay it out the way it was meant to be laid out, and don't change anything. The way it is in the file is the most efficient way to lay the data out. the loading code you have works fine: virtually nothing is new in the new milkshape3d file specification.

If you are still having trouble loading the milkshape3d stuff I've included the file format (I just bought the latest version and downloaded the SDK, all my loading code that I wrote several years ago from Brett Porter's tutorial still works fine).


Stop trying to plan an elaborate layout in your engine. You don't have enough experience, in my opinion, to even be able to accurately weigh the pros and cons of various sophisticated layouts. You will continue to get tripped up...just load the data, and just render it. Your challenge will be understanding how the data is laid out (how stuff indexes into each other) and maybe getting the texture loading to work properly.

I am far beyond figuring out how the MS3D Model loader works, how the texture loader works..

It is almost insulting.

If you would read correctly the problem lies within how I lay out ALL data in my program:

Vertices - Materials - Physics - Positions - Matrix...

And how MS3D Lays out their data:

Vertices - Triangles - Meshes - Materials

Granted I don't have to worry about physics, position, or matrix data when trying to make the two compatible.

Question is, what happens if I have a cube, it isn't an MS3D Model, it is loaded normally...

No problem, no problem at all in that, the problem is when, if I add support for the MS3D file in my data structures, it ruins it for everything else, unless other files I load are set up the same way, which they may be...

But if I create a cube out of hard coding triangles, I can't use that file structure unless I define what index each triangle uses, which we all know now, that is impossible.

The only real solution I can think of is to use nothing but MS3D files and make all the models in my game use that setup or an m3d or something similar that is set up relatively the same..

I have found in my study that Bubba's data structures work well for a two dimensional game, where most polygons are hard coded. But for a three dimensional engine, where most of your (but maybe not all) data, is coming from a file type that is set up in a different way from what he specified.

Kudos to me for figuring that one out on my own! :)..


I believe I have found my answer. I have to switch the way the data is stored so that it works for a 3-dimensional game, it is almost silly that I let this one slip past me.

In 3d - vertices make triangles make meshes
In 2d - vertices make it all

In 3d - Your renderable object is made up of a mesh
In 2d - Your renderable object is just a set of vertices (usually a quad)

Thanks Bubba for all your help, you ultimately led the creation of an entirely new system.

Bob - The reason people usually misunderstand my questions is because I don't know what to try to do to fix them for the most part, thus why you had no idea what I was doing to try to fix it, but I believe I've found the answer.

Bob - You're right, I need to arrange my data to support MS3D models because the way I had it set up was ultimately, I believe, ment for a 2d system :).

>>It is almost insulting.

No it isn't, it is complicated, but I am reassured by the fact that you are struggling because it shows you are trying.


>>If you would read correctly the problem lies within how I lay out ALL data in my program:
I wouldn't worry about all the data in your program yet. When you get better at this game programming stuff you'll probably just ultimately scrap whatever you've created now for something more useful.


>>Question is, what happens if I have a cube, it isn't an MS3D Model, it is loaded normally...
Well, what I do is I have a data type that I have specified inside the renderer. For objects that are overly simple, i.e cubes lines and points, stuff like that, I just call them 'debug' items, because I don't really use them in practice, only for debugging. Here is how a line is represented in my code...it's really straightforward, and if you go back to the last post I made it follows the same general format for how I add a ms3d model to my renderer, in that I have the following:

-A simple data structure
-A function for adding the structure to my renderer
-A function for rendering the structure


The data structure for a line

Code:

---

struct        DebugLineData
{
        Vector3        Start;
        Vector3        End;
        Vector3        Color;
        float        Size;
};

---

A 'Vector3' is a three dimensional vector, a class that I wrote.


The code I use for adding a new line to the renderer

Code:

---


        void        AddDebugLineToRenderer(float        StartX,float        StartY,float        StartZ,
                                                                  float        EndX,float        EndY,float        EndZ,
                                                                  float        ColorXRed,float        ColorYGreen,float        ColorZBlue,
                                                                  float        Width)
        {
                DebugLineData        *pLine        =        new        DebugLineData;
                pLine->Start.x        =        StartX;
                pLine->Start.y        =        StartY;
                pLine->Start.z        =        StartZ;

                pLine->End.x        =        EndX;
                pLine->End.y        =        EndY;
                pLine->End.z        =        EndZ;

                pLine->Color.x        =        ColorXRed;
                pLine->Color.y        =        ColorYGreen;
                pLine->Color.z        =        ColorZBlue;

                pLine->Size                =        Width;

                this->mDebugLines.push_back(pLine);
        }

---

The code to render a debug line

Code:

---

        No lighting
        No texturing
        Coloring on               
        Depth test on        (should never be changed)
*/
void        GLRenderer::RenderAllDebugLines()
{
        if(!mDebugLines.size())
        {
                return;
        }

        NT_GL(glDisable(GL_LIGHTING))
        R_DisableAllTextures();
        NT_GL(glEnable(GL_COLOR))

        std::vector<DebugLineData*>::iterator        ptr;
       
        DebugLineData        *pObject=0;
        for(ptr = mDebugLines.begin(); ptr != mDebugLines.end(); ptr++)
        {

                pObject        =        (*ptr);
               
                NT_GL(glLineWidth(pObject->Size))
                NT_GL(glBegin(GL_LINES))        //FIXME:        you can likely get this out of the loop
                NT_GL(glColor3f(pObject->Color.x,pObject->Color.y,pObject->Color.z))
                NT_GL(glVertex3f(pObject->Start.x,pObject->Start.y,pObject->Start.z))
                NT_GL(glVertex3f(pObject->End.x,pObject->End.y,pObject->End.z))
                NT_GL(glEnd())
        }
}

---

Nevermind the NT_GL bit, it's a mechanism I found online if you are interested for writing your own implementation of openGL (I found it while looking for MESA GL, I saw that on the milkshape3d site...you can just remove the NT_GL() macro, leaving just the normal OpenGL function name).

Quote:

---

But if I create a cube out of hard coding triangles, I can't use that file structure unless I define what index each triangle uses, which we all know now, that is impossible.

The only real solution I can think of is to use nothing but MS3D files and make all the models in my game use that setup or an m3d or something similar that is set up relatively the same..

I have found in my study that Bubba's data structures work well for a two dimensional game, where most polygons are hard coded. But for a three dimensional engine, where most of your (but maybe not all) data, is coming from a file type that is set up in a different way from what he specified.

Kudos to me for figuring that one out on my own! ..


I believe I have found my answer. I have to switch the way the data is stored so that it works for a 3-dimensional game, it is almost silly that I let this one slip past me.

In 3d - vertices make triangles make meshes
In 2d - vertices make it all

---

Hmmm, you are under the impression that you need some universal data type to represent everything. This is a farce.

Here are just some data structures in my renderer alone. I have a bunch. These aren't all of them, just the most basic.

Code:

---


/*
        *********************************************************
                                                        DEBUG STUFF
        *********************************************************
*/
struct        DebugSphereData
{
        Vector3        Position;
        Vector3        Orientation;
        Vector3        Color;
        float        Radius;
};

struct        DebugLineData
{
        Vector3        Start;
        Vector3        End;
        Vector3        Color;
        float        Size;
};

struct        DebugBox
{
        Vector3        Mins;
        Vector3        Maxs;
        Vector3        Pos;
};

struct        DebugPoint
{
        Vector3        Pos;
        Vector3        Color;
        float        Size;
};

/*
        Single texture
        Lighting information allowed
*/
struct        MS3DModelRenderData
{
        Vector3        Position;
        Vector3        Orientation;
        CMS3D_MODEL        *pModel;
};

/*
        *********************************************************
                                                        ALPHA BLENDED STUFF
        *********************************************************
*/


/*
        Alpha blending information comes from the texture, not from glColor calls
*/
struct        AlphaBlendedQuadRenderData
{
        Vector3        Position;
        Vector3        Width;
        Vector3        Height;
        float        R,G,B;
        int                Texture;        //Index into gpTextureManager
};

---

This is really tying back in with the idea of "renderable objects"

An MS3D Model is a renderable object

A debug Box is a renderable object

An M3D model is a renderable object



Logical objects uses these renderable objects to represent themselves.


The weird thing, is trying to figure out how a system of ID's works with different data types.. In Bubba's implementation he treats all data the same... Which seems to be very clean.. But, either he is repeating vertices or has some crazy underlying structure that I cannot begin to fathom that handles all data types and boils them down to fit into his structures..


Maybe I'm looking at it wrong..

Maybe a render list -shouldn't- be a list of ID's.. but a list of Renderable objects.. Well that would be the same if the ID's represented renderable objects.. So you have a way to add an MS3D Model to your render list, and you have a way to draw an MS3D Model...

When looping through your list of renderable objects, how do you decide which draw method to use based on what element you're looking at? Be it an MS3D model, an m3d thingy, or a debug box?

EDIT:

I'm thinking you throw each data type into it's own vector and loop through each vector of MS3D Models, call the MS3D draw function..

For the vector of debug boxes - you call renderDebugBoxes..

etcetc... until you're done rendering all the objects in your list...

Wow, that is it...

I could probably make the same system work with ID's... (sorry if I'm obsessive about ID's, but it seems like a good idea, tell me please if it is too much to worry about at this time)

One thing shamino, i've been convert the ms3d model vertex into the vector already. For faster and easier, i have to change the x,y,z in the vector into array. Well, it works fine. I haven't set collision detection for the model yet, but i know it will work.

Good luck with your 'different loading' thing you're doing.

BTW Bob

Can you show me a screen shot of your work? I mean if you have one. By what I've seen from your code it looks like you could whip up a nifty scene to show us all :).

Here is what I've got so far...

So I have my MS3D Model class setup with the data structures pertaining to an MS3D Model in it...

In my Resource_Cache I have another data structure set up to contain a pointer to an MS3D Model, a position, and an orientation.. Also in the Resource_Cache class is a function that creates a new ModelInfo struct, fills it out, and pushes it back into a container of MS3D Models.

Resource_Manager.h

Code:

---

#include <vector>
#include "MS3D.H"

class Resource_Cache
{
        friend class Resource_Manager;
public:
        struct        MS3DModelRenderData
        {
                float Position[3]; // will be changed to vectors eventually
                float Orientation[3];
                MS3DModel *pModel;
        };

        std::vector<MS3DModelRenderData*> MS3DModels; // holds pointers to all MS3D Models
private:

        bool Add_MS3DModel_To_Cache(const char *filename); // loads a milkshape model to the cache
};

---

Resource_Manager.cpp

Code:

---

#include "Resource_Manager.h"
#include <windows.h>

bool Resource_Cache::Add_MS3DModel_To_Cache(const char *filename)
{

        /*
        Add conditional to check if filename is loaded already
        If filename is loaded pushback another model info
        with the same model, perhaps keep a list of loaded
        filenames and their corresponding model pointers
        */

        MS3DModelRenderData *Model;
        Model = new MS3DModelRenderData;
        Model->pModel = new MS3DModel;

        if ( Model->pModel->Load_MS3D_Model( filename ) == false )
        {
                MessageBox( NULL, "Couldn't load the model data.", "Error", MB_OK | MB_ICONERROR );
                return 0;                                                                        // If Model Didn't Load, Quit
        }

        MS3DModels.push_back(Model);

        return 1;
}

---

As you can see, it is just the basis of what I need.

I need to add conditionals and check if I've already loaded the model filename given..

I suppose I can store the filenames loaded and their corresponding pointers in a linked list or another vector, and check if filenames are identical, then return that filenames saved pointer (also in the linked list of MS3D Models)...

Thats the only way I can think of to keep from loading the same filename twice.. And the only thing I can think of to do if we do happen to come across that case.

I noticed that your AddMS3DModeltoRenderer function (you gave me the source a while ago) Doesn't load the files, it just takes a pointer to an already loaded model..

Now, do you A: Load all the models regardless of what is going down in your program?

Or B: have a resource manager somewhere managing what gets loaded and what doesn't?

Also remember, we'll dealing with a resource manager here, not a renderer.. It is understandable that your GLRenderer class does not have file repeat protection...

That is just it shamino, I do not rely on an external resource manager...it may be a good idea to do so, but I keep mine even simpler than that. It probably would be a good idea to have some sort of a 'resource manager' that handles all of the ms3d files, I do something similar with my texture manager, but I don't do it for ms3d models.

Here's how, in my engine, a ms3d model comes to exist:
I develop a class, something like a 'monster.' The monster will hold a pointer to a ms3d model, and when the monster is created it will load its appropriate model. One of the reasons that each monster holds a ms3d model is that when each model has a copy, instead of there just being one instance of the model, is that it allows me to do things such as deform the model (alter its triangle vertices based on damage it receives) without destroying the model that the other monsters may use.

But the way you just outlined things is fine.

I've decided to use a map to keep a list of Resource filenames that I've loaded, also a reference accompanying that resource..


I'll post some sample code..

Code:

---

template < typename resource_t >
class manager {
    std::map< std::string , boost::shared_ptr< resource_t > > resources;
public:
    resource_t & load( const std::string & filename ) {
        std::map< std::string , boost::shared_ptr< resource_t > >::iterator entry = resources.find( filename );
        if ( entry != resources.end() ) { //found
            return * entry->second;
        } else {
            boost::shared_ptr< resource_t > resource( new resource_t( filename ) );
            resources.insert( std::make_pair( filename , resource ) );
            return * resource;
        }
    }
};

class texture {
public:
    textue( const std::string & filename ) { ... }
    ...
};

class model {
public:
    model( const std::string & filename ) { ... }
    ...
};

class sound {
public:
    sound( const std::string & filename ) { ... }
    ...
};

int main( void ) {
    manager< texture > textures;
    manager< model > models;
    manager< sound > sounds;

    texture & logo = textures.load( "logo.jpeg" );
    model & cat = textures.load( "cat.3ds" );
    sound & batman_theme = textures.load( "sounds/batman.mp3" );

    render( logo );
    render( cat );
    loop( sound );
}

---

We can use a template to avoid alot of repeating code, because a map will only handle one type of reference at a time, so therefore we can use templates to make it easy to create a map that will keep track of a certain resource-reference..

You could consider it a good thing, not having a single container of references to look for, because if you know you're loading a sound you don't want to search through a container that has textures, models, and sounds, it would be slow!

I'll get to implementing this soon.. Will let you know how it goes :).

More updatage

Resource_Manager.h

Code:

---

#include <vector>
#include "MS3D.H"
#include <map>
class Resource_Cache
{

public:
        struct        MS3DModelRenderData
        {
                float Position[3]; // will be changed to vectors eventually
                float Orientation[3];
                MS3DModel *pModel;
        };

        std::vector<MS3DModelRenderData*> MS3DModels; // holds pointers to all MS3D Models
private:

        bool Add_MS3DModel_To_Cache(const char *filename); // loads a milkshape model to the cache

};

template < typename resource_t >
class Resource_Manager
{

    std::map< std::string , resource_t* > Resources;

public:

    resource_t& Load( const std::string & filename )
        {
        std::map< std::string , resource_t* >::iterator entry = Resources.find( filename );
        if ( entry != resources.end() ) // if the entry is found
                {
            return * entry->second; // return the reference to the resource
        }
                else
                {
                        resource_t* resource( new resource_t( filename ) );
               
                        // first we gotta load the resource, then we can make a pair
                        // and return the reference
               
                        resources.insert( std::make_pair( filename , resource ) );
                        return * resource;
        }
    }

        virtual ~Resource_Manager( void )
        {
                std::map< std::string ,  resource_t * >::iterator destroyer, end;
                for ( destroyer = resources.begin() , end = resources.end() ; destroyer != end ; ++destroyer )
                {
                        delete destroyer->first;
                        delete destroyer->second;
                }
        }       

};

---

Resource_Manager.cpp

Code:

---

#include "Resource_Manager.h"
#include <windows.h>

bool Resource_Cache::Add_MS3DModel_To_Cache(const char *filename)
{
        MS3DModelRenderData *Model;
        Model = new MS3DModelRenderData;
        Model->pModel = new MS3DModel;

        if ( Model->pModel->Load_MS3D_Model( filename ) == false )
        {
                MessageBox( NULL, "Couldn't load the model data.", "Error", MB_OK | MB_ICONERROR );
                return 0;                                                                        // If Model Didn't Load, Quit
        }

        MS3DModels.push_back(Model);

        return 1;
}

---

TODO:

Add file extension recognition to choose right loader function.

Make it so the actual load functions return a reference to the manager so that he can make a pair out of the resource filename and the reference.

Make it so no matter what another reference gets pushed back into the list of things to draw if a load request is sent.. We can't just do nothing if there is a file already loaded!

That is all I can think of right now! Working!