Making a script language?

(WinXP, Console project)
I've finished my engine, worked all the glitchs I could find out of it, now I'm working on maps. Everything in my engine right now relies on fixed movement and things, so I'm going to try and add some randomness.

So, I decided, make a small scripting language! So, I mulled it over, and I came up with how it would work;

Open up a .ord file, which is plain text holding the orders.

Load the file into a string.

Parse the string, find the current level, and take the orders relating to that level. I plan to do this like:

Code:

---

--
 Level 1;
  @CEMD(1, 55, 12, 1);
 END;
--
 Level 2;
  @CEMD(1, 30, 17, 0);
 END;
--

---

That would be the general layout. But I'm stuck after I parse the string, I need to somehow call a function by a string. I could make a large switch statement, or I could count up every occurance of an object and evaluate the order in a loop.

But both of theyse are very slow methods. It would be better to hardcode the orders, but I'm attempting to avoid that. I need some way of running a script, but making it effective. And doing so, keep the actual scripting language to a minimum.

Then theirs compiling, I guess all I need with that is a library of functions and a switch statement, but I'm going to avoid compiling at runtime.

So, is their a better way of evaluating scripts? -Besides long switch's &/ loops for every statement (which suddenly makes the flexibility vanish).

Thanks!

Here's the first thing that jumps in my mind, constant time access without wasting the space of a hash:
each script function is an index that maps to an array of function pointers to execute that script functions functionality.

int foo = readFromScriptFile();
something* params = readParamsFromScripityScriptFile();
scriptFunctions[foo]->run(params);

Good idea! I'll try that method, sence its much easier than what I was thinking :). Thank you!

Think Windows API...think messages.

Bubba: Nobody has any idea what the heck the message method is. googling is useless. "Think Windows API" is useless if you're, like, not a windows programmer.

And yes, the index array plus function pointer method is, by far, going to be the fastest. You're not going to have that many built in commands, so any sort of indexing function is going to be fast. If you compile stuff into bytecode (which I'm not really clear on how to do) or some meta-language, you can probably just read in the values as number and not even mess with finding index positions. Actually, the bytecode method would be the best but I don't know anything about it, other than that it's the fastest, apparently, and most concrete.

BTW, some advice. I've been working on this myself, and it's really, really important to consider program flow. From the get go, make sure your run function can handle jumps, sub routine calls, and all of that. It's awkward to have to rewrite it later.

Also, another good idea, I guess, is to write a few very basic functions that model ASM procedures. You can then use these very basic manipulators to build more complex command functions. This primarily prevents you from having to rewrite stuff constantly. also, if you can get the basic underlying procedural stuff down, applying it to other things should be much easier, and SHOULD produce more regular code.

Actually a faster method (if you now feel that you have to use a functionpointer) will be to use a linked list where you just insert the functionpointers in the order they show up. That way you will just iterate the list and execute the functions in the order they show up in.

What bubba talks about is a system where each function in the script is associated with a message-id. Now you just create a list/vector/whatever with these messages and some other info that is needed, this is all done at loadtime. At runtime you just iterate the container and send the messages to a message-procedure. This allows for far more flexibility. Note that all message-processing should be done by a CObjectContainer or an equivalent class, the only messege-processing done by the script itself is scriptinternal stuff (like ifs, variable-modification and stuff like that).

This will be a very flexible script and you will find that it will be very easy to modify it to suit different needs of all kinds of games (I have, together with bubba made a script that works just this way).

What is so hard about messaging?

Here are some good reasons you should use messages:

1. All multiplayer instructions from server to client are sent in the form of messages.
2. All objects in the game can respond to messages thus making it extremely powerful.
3. All game engine internals can be modified via messages.
4. Using messages allows you to make quick demos for your game.
5. Using messages allows you to put the control in the hands of the level creator.
6. Using messages allows the game engine to be coded separately from the game data.

Shakti hit on exactly what we are doing. In Windows each window or application has a message queue and a message process that Windows sends messages to. In fact every Windows app must have at least a rudimentary MsgProc function in order to work correctly.

So how does it work in Windows? Let's say you click a button in a window.

1. Windows determines what window the mouse is over based on mouse coords and z order of windows. It looks in the registered windows to retrieve the hWnd or handle and it also looks in the register area to find the MsgProc function pointer.

2. The message for a left mouse click is WM_LBUTTONDOWN. It sends this message to the window containing the button. The parent window then routes the message to it's child button which is another window. The child window (button) can now respond to the message or the parent can respond to it - it can be setup both ways.

3. All of this is done with 1 message function and 2 params. Each message has an associated wParam and lParam indicating certain parameters for each type of message ID. The SDK will tell you what those mean for each message Windows sends. Some newer messages use other constructs as well, but for the most part MS has stuck to its original design.

Ok this is very very powerful. First, Windows does not respond to the button click. It simply tells the windows, hey dummy someone clicked a button and the button is in your window. Respond to it if you want.

When a window needs painted, it simply sends a WM_PAINT message to the window needing painted and the window then responds. If the window does not respond it does not get painted, hence this is why programs that have crashed or stopped responding cease to be painted and/or updated.

This design is brilliant even though it did come from Microsoft. :)

Ok so let's look at this from a game perspective. We have a bunch of objects sitting out there that we want to move, manipulate, destroy, etc, etc. Well imagine each object as it's own window or even it's own process. So if we want to move an object we don't call a function to move it because that's hard-coding the process. Instead we send a message to the object telling it to move and giving it parameters on how to move or what to move towards, etc.

Now this move command will result in the object moving no matter how the message gets sent. You can send it from a script, from a multiplayer server, from the local game, or from anything you want. As long as the move message arrives at the object, the object will move. This means you create a base class for objects that can respond to most every message the message system can send it. This would be called default processing.

Let's say we want to vector bad guy A to player. Assume the player has an ID of 32767. Assume the bad guy has an ID of 1. In order to produce a vector to intercept the player we need to know several things.

1. The position of the player.
2. The position of the bad guy.
3. The length of the vector from the bad guy to the player.

The length can be computed, the bad guy position is local to the object class, and the player position can be sent from the message system since it knows anything with an ID of 32767 is the player.

So the message boils down to something like this:

OBJ_MOVE_TO,32767

If OBJ_MOVE_TO has a message ID of 100 then the command is this:

100,32767

We set it up something like this:

Code:

---

struct Message
{
  DWORD dwMessageID;
  DWORD dwParam1;
  DWORD dwParam2;
  DWORD dwParam3;
};

---

So the code might look like this:

Code:

---

CObject::MoveTo(DWORD dwObjectID)
{
  D3DXVECTOR3 toObject=m_pObjectManager->GetPos(dwObjectID)-vPos;
 
  float oneOverLength=1.0f/D3DXVec3Length(&toObject);
  vNextDirection.x=toObject.x*oneOverLength;
  vNextDirection.y=toObject.y*oneOverLength;
  vNextDirection.z=toObject.z*oneOverLength;
 
  //Indicate we need to turn instead of snapping to vector
  bNeedToTurn=true;
}

---

The MsgProc would look like this:

Code:

---

HRESULT CObject::MsgProc(Message MSG)
{
  switch (MSG.dwMsgID)
  {
    case OBJ_MOVE_TO:
        MoveTo(MSG.dwParam1);break;
    ...
    default:
        CBaseObject::MsgProc(MSG);break;
  }
}

---

The ObjectContainer or ObjectManager would route the message. The MessageSystem would tell the Object container the message, ID of the object being acted on, and the ID of the object to vector to. The ObjectManager then would construct a Message structure consisting of the object to vector to and the message ID. It would then use the ID of the object being acted on that was sent from the Message system as an index into it's array of objects.

Code:

---

bool CObjectManager::SendMessage(Message MSG)
{
  if (MSG.dwParam1>Objects.size()) return true;

  //Reconstruct message for object
  Message ObjectMessage;
  ObjectMessage.dwMessageID=MSG.dwMessageID;
  ObjectMessage.dwParam1=MSG.dwTargetObjectID;

  //Call object's MSG proc
  Objects[MSG.dwParam1]->MsgProc(ObjectMessage);

  return false;
}

---

We have determined that for binary and trinary message operations dwParam1 will always be the object being acted on and dwParam2 will be the target object. For unary operations, dwParam1 is always the object being acted on. Trinary operations are not encountered very often in our setup but normally when they are, dwParam3 acts as a modifier for the action.

A unary operation.
COMMAND: aKill(ID)
DESC: kills object #(ID)
CODE: OBJ_KILL,ID
BIN: 11h, imm32

A binary operation
COMMAND: aMoveToWP(ID1,ID2)
DESC: Moves object #ID1 to waypoint list #ID2
CODE: OBJ_MOVEWP,ID2
BIN: 100h, imm32

A trinary operation
COMMAND: aMoveToWpDelay(ID1,ID2,time)
DESC: Moves object #ID1 to waypoint list #ID2 after time seconds have elapsed
CODE: OBJ_MOVEWPDELAY,ID2,time
BIN: 101h, imm32,imm32


I hope that explains it well enough.

that is all nice an neat but it is not perfect. using messages would really only apply for sending around information while the game is running, this is not a problem but just that a better system would be used to allow developers to make permanent changes. this means that messages would be perfect for making a developer console. and there is also the fact that you are using a switch statment but for the purposes of something like a dev console it would not matter much.

Quote:

---

Originally Posted by Shakti

Actually a faster method (if you now feel that you have to use a functionpointer) will be to use a linked list where you just insert the functionpointers in the order they show up. That way you will just iterate the list and execute the functions in the order they show up in.

---

We're talking about mapping the read in script function to the actual function to execute. If the function name is an index into an array of function pointers you get constant time access, O(1), a linked list would be linear time O(n).

I think you're talking about queueing up the script functions to execute though, in which case any data structure will do since you need to access all n elements anyway.

Quote:

---

this is not a problem but just that a better system would be used to allow developers to make permanent changes.

---

Um yeah it's called loading up your code, altering it, and recompiling or altering the DLL. How do you expect to make permanent changes to an engine from script and why would you want to?

And isn't a scripting language all about what happens when the game is running? Your comments loopshot just don't make any sense.

And have you guys actually tried to start coding a script language. Have you used function pointers? Have you used messages? We have used both and thus far, messages are so much easier. Function pointers cause a lot of problems. Whether they are faster or not is a moot point because it won't make much difference in FPS between the two methods.
We iterate C structures and only execute those that are ready to execute. We tested it and 40000 printf's in C is just a tad faster than 40000 messages to do the same task in our system. It is blazing fast.

I invite you to try both or a version of your own. But show some code so we can compare notes.

@ Perspective: Yes I am talking about at loadtime create a list, which only contains the functions (or messages) that the script needs. This way you have created a list where you at runtime just need to iterate and execute whatever should be done.