Game Engine Development

[MacGyver]

This is meant mainly for people that have experience in writing or modding game engines. Please refrain from suggesting that a current one be used. This is specifically regarding the development of one.

To simplify this, let's only consider the following scenerio:

Type: Supports 2D games, not 3D.
Platforms: Windows, Linux, or Mac. Only one. Any one.
Possible Language: C, C++, Java, or Asm (or a combination). This is open.
Networking support: Dedicated Server and Client.

Now here are the questions.

1. What would be the worst pitfall that you think one could fall into if he or she were developing a game engine from scratch that fit the above criteria?
2. What is the best way to avoid this pitfall?
3. What is the number one thing to think of when developing a game engine that fits the above criteria? For example, speed of operation? Solid memory management? etc. etc..

Curious to see the responses this generates.

[psychopath]

From my experience, I'd say that the biggest pitfall would be engine design. A game engine is a fairly big system, and it all has to fit together perfectly in a way that makes sense. The best thing is to plan ahead as much as possible, and only take on so much at a time. Don't dive in a try and write the whole thing at once; build a framework, then write a graphics system, then a resource manager, etc.

Beyond that, yes, you'll be needing good memory management, particuarly in your resource system. Speed of operation would be important as well, in graphics, physics, and AI.

EDIT: #900! ;p

[VirtualAce]

1. What would be the worst pitfall that you think one could fall into if he or she were developing a game engine from scratch that fit the above criteria?

Engine design. You can go two routes with this. You can over-design and end up with a very nice object oriented framework but one that is so hard to use it is nearly worthless. On the second hand you can under-design and end up with an engine that does not support or utlilize objects well or one that is hard to add functionality to. Every object in an engine must communicate well with other objects and they all have to 'get along' well with one another or you have a huge mess.

What is the best way to avoid this pitfall?

First thing would be to plan the design and I'm not talking about UML where you essentially write the class out on paper. I'm thinking more along the lines of a relational chart as to which class does what and who they communicate to. This will nearly build your framework without even writing one line of code. After this you will probably have a well-thought out design in which you can add other classes for greater functionality with little or no trouble.
Use as many known and tested design patterns as you can and I recommend you make use of interfaces (pure abstract classes) and derive from them.
The more you design on paper the less you will have to design while you are coding which is a major pitfall in such a complex system. On paper you can see the big picture where as in code you cannot always see the big picture and may introduce a bug or problem that you did not foresee.

What is the number one thing to think of when developing a game engine that fits the above criteria? For example, speed of operation? Solid memory management? etc. etc..

This is a tough question since there isn't really a number one thing to think of in game development. Because a game must maintain interactive framerates (at least for real-time graphics games) there are about a hundred number one things to think of when developing a game engine. Overall I personally feel that speed and memory management are probably two of the biggest issues to think about.

Speed, however, is really taken care of by the hardware. As long as you don't use a vector of vectors to a pointer of vectors that contain objects then you will probably not have major issues with speed. Use the right data structure for the job, resist the urge to code your own because it increases dev time and reduces stability, use design patterns where they are applicable, and favor the STL in all cases. Some do not favor the STL but the benefits it offers compared to the code you must write without it far outweighs any speed hits you may incur. Several of us have used the STL in some pretty critical portions of the engine/rendering code and you don't see many of us reporting that the STL is too slow to use. It can be if used improperly but I have not had any issues with it as long as I used the right container for the job.

Memory management is absolutely one hundred percent crucial to the success or failure of your engine. How you go about it is your decision but if your engine cannot manage memory it will leak it, hog it, and ultimately die a horrible death.
Games are all about resources. From models to textures to sounds to music, etc, etc. Resources are games and games are resources. You must find a way to manage these efficently and quickly. No one wants major pauses in their game while your engine loads the next data set into memory. I think it's nearly impossible NOT to have some pausing but some games pull it off.

Before you decide on a memory management scheme you need to focus your engine into one specific type of 3D game or write modularized memory managers that can be used for different types of games and then use that as an add-in class to the engine. For instance flight simulators would not lend themselves well to the load everything in memory at runtime scheme. Yet your graphics engine could probably handle the load of graphics presented by the flight simulator genre. So you would need to write some add-in classes to support flight sim genre games. However an FPS may lend itself well to this scheme where the levels are compact, directed/guided (not free-form), and most of the detail is very close to the player. You may be able to load everything into memory in this type of game or you may need a small list type memory manager.

Also along with the memory management comes the idea of how your engine is going to render and how is spatial partioning done. Does it use a quad-tree, a BSP tree, or is it a node-based portal renderer? You can also create add-in classes for each of these types which would mean add-in renderers as well.


Even though I covered 3D here nearly all of the design issues I brought up apply to 2D. 2D in a 3D engine is not much different than 3D in a 3D engine. The only difference is depth but you can use many of the same data structures and spatial partioning methods in a 2D game. Namely quad-trees and BSP trees lend themselves well to 2D games except you are not working with a Z coordinate so your BSP tree does not operate on plane half spaces. Memory management in 2D is nearly the same as 3D since in both you still have a frustum or a portion of the screen/world you can see and a huge portion you cannot see. I would not gear your engine specifically to 2D since adding 3D support is so intuitive as to be simple. Modern APIs will help you out with this a lot because they are inherently 3D game APIs with small support for 2D.

[MacGyver]

Thank you, gentlemen, for these posts. I was a little concerned this was going to go unanswered due to the way I posed the questions.

You both agree on engine design as the thing to pay most attention to, which I had ultimately figured on, but I had expected a more detailed warning about a specific area with regard to the design. I suppose it's possible that the worst problems are not a result of the details as much as the overall design, so this might be something for me to keep in mind.

Regarding the language of choice, I'm tentatively going with Java for now. I figure I can move to C++ at a later date, but I'd rather not get lost in language details or OS specific items just, yet. (Yes, I know there are libraries... I don't want to get lost in them either. I know enough Java to hopefully get somewhere more quickly than in C++ with regard to this).

With regard to what you said about design, Bubba, I have started designing by way of compiling a list of classes that I think would be needed as well as their purposes. Once I start plotting their relationships, which are a little bit unclear at the moment, I imagine I'll either lose or gain a considerable number of these classes. I have thought about using interfaces and/or abstract classes as you recommended, and I have a few already listed from before. I also have some base classes that are supposed to be extended. OOP is very useful for things like this, and I do plan on taking advantage of it. I also understand that it's easy to go overboard, but I imagine I might have the opposite problem of having not enough things planned out.

I know you warned about trying to accomplish too much (ie. flight sim vs FPS), but one thing I came to realize prior to reading your post, however, is that there isn't that much of a fundamental difference in a game like Super Mario vs a game like Tiger Heli (I'm going old-school here. Sorry. ). A game like Super Spike V'Ball is also incredibly similar, but less complicated since the area of play, the actors, and game type, are more limited than either of the two previously mentioned games. This is also similar to Asteroids and other 2D games I could rattle off. An engine could be very well capable of running all of these types of games, depending on how the map/stage/level and game type allow. Is it a bad idea to aim to have the engine capable of satisfying all of these kinds of things, or would you agree that this is entirely possible and a sensible goal to attempt to allow for all of these things? I think what I'm thinking is along your idea of a dynamic memory manager. What I'm considering is having the type of game and the map/level/stage determine how the game itself is loaded, managed, and actually dealt with. Thus the core of the engine wouldn't even really care what the game is actually doing or how its being managed since the actual work would be done later on down the chain.

I do acknowledge your view that 3D is a step away from 2D, as I've seen many of the concepts are similar or the same, but I'd rather not go there, yet. I mainly want to work on 2D games, and making an engine capable of 3D would be adding a bunch of usability that I probably won't need at the moment. Also, this whole thing about writing a game engine is partly a proof of concept for me.

On the subject of multiplayer, is there anything specific that you think I should focus on? Is there anything I should absolutely stay away from?

Thanks again for the feedback.

Edit: I don't know why I didn't put this in the game programming section. I suppose because it was more about the theory behind it and very little to do with programming, this is why I put it here, but I never even gave that section a thought. Feel free to move it if it should be someplace else. /me slaps self for being completely stupid.

[@nthony]

I'm curious to see responses on platform targetting. Though since you mentioned Java, I guess that no longer applies as much. One thing's for sure though, if you do manage to pull off a half-decent 3D game engine in Java, that will definately be one for the books!
(naturally, I would recommend C++ or C, if you're up for the challenge, especially given the graphics APIs made available to only those family of languages)

[MacGyver]

Think this is where the Java 3D API is heading: https://java3d.dev.java.net/

This might be a 3D Java game engine: https://jge.dev.java.net/

I know there's JNI bindings for OpenGL called JOGL, and I think another similar JNI library out there. It is theoretically possible to make a Java 3D game engine.

Depends on what you mean by "decent" I guess.

As I said before, I'm mostly interested in doing this as a proof of concept, as well as learning about the concepts surrounding game engines. What I mainly want is just some warnings about obvious pitfalls and issues I should know to avoid from the start.

[Happy_Reaper]

It is theoretically possible to make a Java 3D game engine.

Already exists : JMonkey

[abachler]

On the subject of multiplayer, is there anything specific that you think I should focus on? Is there anything I should absolutely stay away from?

Define up front how many players will be connecting to a single 'instance' of the game. If its going to be a small number like 2 to 32, then a single server application will suffice. If it will be MMO or semi-MMO, then you need to think of the architecture your server system will use. Trying to have one box run the game and handle login, authentication, account storage and retrieval, etc. will present problems as the number of active accounts exceeds a certain threshhold, which is dependant on your hardware/network.

Stay away from IPX/SPX

[MacGyver]

Yes it will be small enough to manage less than 32 people most likely. I think only 4 people at once should be sufficient for now.

Stay away from IPX/SPX

Oh, those were the good ol' days!

/me has flashbacks of playing Commanche 3 over IPX/SPX... and having the host lockup all the time.

[abachler]

Well, I know some peeps like java, but honestly, even when tis compiled it doesnt have stellar performance. Most game engines today are stradling the C/C++ - Assembly fence, I dont think an engine that required highperformance would go the other direction. In theory you could write a game engine in GW-BASIC, but ultimately your efforts would be for naught. Why not take the leap to C/C++, since DirectX stuff is so much easier to do.

[MacGyver]

At this point, it's relatively pointless, since I'm developing an OO framework at the moment without writing any code. The language can change down the road to C++, after I develop the relationship between classes and how the engine works exactly.

Incidentally, I'm rather surprised that you say DirectX is easier to use than the Java 2D Graphics API. Is this from personal experience?

[abachler]

Well, Ive used DirectDraw and GDI, and personally for strict 2d use GDI is alot easier to implement than DirectX, but if you want to use any of the hardware based transforms, DirectX is the best way to go. In GDI you get left doing all the maths yourself. Its not impossible, btu you either have to have experience in the old graphics methods before hardware acceleration was in the mainstream, or you need a good reference from that time, since most graphics books today dont get into the level of detail you need to implement the maths yourself. Id suggest 'Secrets of the game programmign guru's', now in its umpteenth version. I have the original edition, and it does a bang up job of explaining the sprite animations you will probably use in 2d.

[VirtualAce]

Absolutely do not use GDI. It's a mess and has nothing to do with graphics programming or the concepts involved.

DirectDraw is deprecated and is now part of DirectGraphics. You will need to use Direct3D even though this is for a 2D game engine. Just use an ortho matrix as your projection matrix.

[abachler]

Thats a bit harsh Id say. GDI isnt 'a mess' Its just not that good for high framerate 3d graphics. Its perfectly acceptable for low framerate (30 fps) games. Just because something is deprecated doesnt mean it doesnt work, just that no further development will go into it.

[MacGyver]

Which means if it's deprecated and I use it, it may work for now, but on a new version of Windows or DirectX it might not work.

I believe it's best to keep this stuff abstract as far as the engine goes and leave it up to later levels of implementation.