Copying WHOLE Functions

Printable View

01-19-2003
Speedy5

Copying WHOLE Functions

I need help! What I want to do is to copy and paste the code of fucntions into an array of some sort and execute, through function pointers, the new code inside the array. (I'm not a hacker, btw). I also want to remove the return statements when copying the functions so I can add others. For example:

void a()
{
printf("hello");
}
voit b()
{
print(" world\n");
}

I want in a char array:
{ code for a minus the return statement, code for b normally }

Then with a function pointer to that array, I can hopefully execute the new combined code.

So say we have:

void m()
{
printf("HELLO!\n");
}
void f() { }

.....
// no casting is done here for reading's sake
char *s=new char[1000];
memcpy(s, &m, &f-&m);
void (*go)(void)=s;
go();
// prints out "HELLO"
...

I would want to do all this in a class too (like the functions being member functions).

HELP ME! HOW TO DO?

I bet you advanced coders know how to do all this, LOL!
01-19-2003
Polymorphic OOP

Re: Copying WHOLE Functions

Quote:

Originally posted by Speedy5
I need help! What I want to do is to copy and paste the code of fucntions into an array of some sort and execute, through function pointers, the new code inside the array. (I'm not a hacker, btw). I also want to remove the return statements when copying the functions so I can add others.

There is no way to "copy and paste" function bodies or change the return type of a function. If you want to do either of those things then you have a major problem with your logic in the program.
01-19-2003
Waldo2k2

>>I'm not a hacker, btw

no kidding

wtf is the purpose of this program anyway? doesn't make any sense to me.
01-19-2003
Speedy5

I want to clone the function, take it's code at runtime through pointers and copy it all into an array. And by trimming off X-number of bytes at the end (the assembler 'ret'), we can prevent the copied function from returning so I can paste another afterwards. I just want to have some help on this.
01-19-2003
Speedy5

I'm not a hacker, I'm not stupid. But please help!

Its for a VERY VERY primitive form of a JIT Compiler.

(This board responds fast, lol)
01-19-2003
Speedy5

basically, i just want to test myself on this
01-19-2003
Polymorphic OOP

There is no way to do that in C++. You'd have to know the size of the function, which can not be calculated, as well as many other implementation dependant things.

If you find that you think you have to do this, then you have a huge flaw in your program's logic. It should never be necissary.
01-19-2003
Waldo2k2

WHY ARE YOU DOING THIS!?

There is NO point in this (besides the fact that it's impossible)!

Why not just call the function again??

[edit]
if you had the skill to make a JIT compiler you wouldn't be trying to do what you're doing, second of all there isn't a JIT for C++ for a reason....go use java if you want a JIT
[/edit]
01-19-2003
Sebastiani

Slow down, speedy. To put it mildly, you are confused about how things work. :D
01-19-2003
Speedy5

I can know the size of the function. Test it out. I put the f() function as a barrior. I take the address of f() and minus it from m(). That gives me, usually (works on Retail build), the correct size of the function.
01-19-2003
Polymorphic OOP

That's not valid. You can not be guaranteed that the functions are right next to each other in memory.
01-19-2003
Speedy5

Where i got this idea:

POSTED ON PLANET SOURCE CODE C++ SECTION

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

/* This programs shows how to copy the bytecodes of a function to a variable
then execute this chunk of code (dynamic linking?). Not sure if this works
with all compilers and all machines/OSes
*/

int func1 (int a, int b){ /* this is the function that will be copied */
int i;
for (i=0;i<=100;i++){ /*some silly code.. doesnt have to do with the purpose of this program */
a+=i + b;
}
return a;
}
void func2(){ /*this useless function is declared to mark a boundary */
} /* its address is just after the code for func1 */

int (* f) (int a, int b); /*a function pointer, must have the same args of the copy*/

int main(){
/*try
{*/
int *start,len,i,*end;
char * funccopy;
start = (int* ) &func1 ; /*where to start copying? */
end = (int* ) &func2; /* to determine length of char array to copy*/
len = end-start; /* */
funccopy = (char *)malloc (len);
printf ("length of bytecode buffer %d\n",len); /* if you change the code within func1 you will see this number change accordingly */
memcpy (funccopy,(void *)start,len); /*^ an easy way to determine the speed of your functions! ;) */
f = (int(__cdecl*)(int,int)) funccopy; /*now f will point to our copy */
printf ("Value %d\n", f (3,4)); /* test it... does it work? */
/*}
catch (...)
{
printf("DAMN");
}*/

system("pause");
return 0;
}
01-19-2003
Sebastiani

*sigh* search the board...this topic covered just a few days ago...
...but it won't work quite like that...
01-19-2003
Speedy5

You are right, I can't be sure. BUT, as I said, on the Retail build, it works, (the function size). I am SURE of this.
01-19-2003
Polymorphic OOP

Quote:

This programs shows how to copy the bytecodes of a function to a variable
then execute this chunk of code (dynamic linking?).

That's not dynamic linking. Dynamic linking makes the function pointers used in calling functions in one program in memory point to functions in memory at another location. There is no copying of functions going on in dynamic linking.

The person who wrote the code had a vague idea, tried to implement it, and falsely concluded that it does what he thinks it's doing. It's interesting to test and toy with, but you can't conclude anything from it or guarantee portable results.
01-19-2003
Polymorphic OOP

Quote:

Originally posted by Speedy5
You are right, I can't be sure. BUT, as I said, on the Retail build, it works, (the function size). I am SURE of this.

You can't be sure of that at all. There is no way you can be certain that's the actual size from what you are doing.
01-19-2003
Speedy5

I know that, the author of that code is wrong. ;) But can they clone the functions like that? Or else how?
01-19-2003
Polymorphic OOP

It's possible but not portable. What may work on one compiler won't work on another. You should never ever have to "clone" a function.
01-19-2003
Speedy5

I really don't care if it's not portable. 99% of ppl use windows. Plus, since this is a VM (what I'm making), I can just think up another alternative, maybe not JIT. So it'll work for what I'm doing, using the C++.NET compiler (unmanaged). Now how do you clone the function? That's my question.
01-19-2003
Speedy5

(No offence, really, to those w/o windows who use Linux, Unix, etc) There will be VM for u guys :-D

The project (site being remodled):
www.blue-lightning.tk
01-19-2003
Polymorphic OOP

Quote:

Originally posted by Speedy5
I really don't care if it's not portable. 99% of ppl use windows. Plus, since this is a VM (what I'm making), I can just think up another alternative, maybe not JIT. So it'll work for what I'm doing, using the C++.NET compiler (unmanaged). Now how do you clone the function? That's my question.

No, you don't understand. Portable meaning portable to other compilers, not platforms.

If it's a virtual machine then why are you copying compiled code?

You still shouldn't be cloning a function. You should understand your project and how to accomplish it more before you decide that you have to do something as "copy and paste" a function. Copying a function will accomplish nothing and there are more elegant and portable solutions.
01-19-2003
WebSnozz

If you are wanting to basically queue up a bunch of functions to be processed then you can use funciton pointers and an array and iterate through the array. Function pointer arrays are also useful for flow control.

Learn about funciton pointer arrays here.

At the bottom of the funciton pointer array function where he calls the funcitons uysing the array you could easily place this in a for loop. Personally I like to use an STL queue, list, or matrix, depending on what the purpose is.

However, if you do want to edit your own program you have a problem, because once you're program is compiled and you run the EXE, it is no longer in C++, it is in machine code....

Code:

004010CC >/$ 55 PUSH EBP 004010CD |. 8BEC MOV EBP,ESP 004010CF |. 83EC 44 SUB ESP,44 004010D2 |. 56 PUSH ESI 004010D3 |. FF15 74634000 CALL DWORD PTR DS:[<&KERNEL32.GetCommand>; [GetCommandLineA 004010D9 |. 8BF0 MOV ESI,EAX 004010DB |. 8A00 MOV AL,BYTE PTR DS:[EAX] 004010DD |. 3C 22 CMP AL,22 004010DF |. 75 1B JNZ SHORT NOTEPAD.004010FC 004010E1 |> 56 PUSH ESI ; /pCurrentChar 004010E2 |. FF15 FC644000 CALL DWORD PTR DS:[<&USER32.CharNextA>] ; \CharNextA 004010E8 |. 8BF0 MOV ESI,EAX 004010EA |. 8A00 MOV AL,BYTE PTR DS:[EAX] 004010EC |. 84C0 TEST AL,AL 004010EE |. 74 04 JE SHORT NOTEPAD.004010F4 004010F0 |. 3C 22 CMP AL,22 004010F2 |.^75 ED JNZ SHORT NOTEPAD.004010E1 004010F4 |> 803E 22 CMP BYTE PTR DS:[ESI],22 004010F7 |. 75 15 JNZ SHORT NOTEPAD.0040110E 004010F9 |. 46 INC ESI 004010FA |. EB 12 JMP SHORT NOTEPAD.0040110E 004010FC |> 3C 20 CMP AL,20 004010FE |. 76 0E JBE SHORT NOTEPAD.0040110E 00401100 |> 56 PUSH ESI ; /pCurrentChar

That's not all machine code, there's some extra junk there, but you should see that this is nothing like any high level programming language(which is what C++ is). The organization of seperate funcitons disapears, which is what a compiler basically does, what you are wanting to do, but you can't compile you're own program, because it's already compiled!!!! :eek: Plus even if you could udnestand the machine code you'd still have to deal with the problem that if you try to edit the file it's going to give you an access violation because the program is running, can't edit the exe file of a running program.

What you could do however, is write a program that opens a cpp file as just a text file, and does as you say, combines functions, and then closes the file. You would then manually compile the cpp with a standard compiler such as Visual Studio C++ and see if it will compile and run. You mentioned trying to make a compiler, that's what a compiler does, is opens something as a text file, interprets the words and symbols, and then saves it in some other language. It does not edit itself though, or try to combine it's own functions.

... Which came first? The compiler, or the source code for the compiler? :D
01-19-2003
WebSnozz

Ok, there's been about 5 post since I started writing my post, lol, got a little off of what I thought he was asking about, oh well.
01-19-2003
Speedy5

OK, let me elaborate a bit more. I've found a way to run the BL 7x slower than c++. I tested ifs, switches, function pointers and a few other techniques. Function pointers run fastest for running a single instruction. Then, I tested that 60% of the time is being spent calling up the functions for the instructions. So when I saw that code for cloning functions, I thought, maybe I can cut and paste the bodies of the instruction-handling functions together to form a primitive type of JIT. This would boost speed at the expense of memory (which no one really cares when you have 512mb+...lol). Portable to other compilers? I don't care. I'm using the managed extensions (.NET) anyways. So I have to use C++.NET's compiler. This part, however, and test, is in unmanaged, normal code. How do you clone?

Thanks, WebSnozz, I'll be reading your long, "looks great at first sight" post after I post this. ;)
01-19-2003
Speedy5

Well, function pointer arrays is what I had to execute a single instruction. They work great, they're fastest of the techniques I tested. (ifs, switches, func. pointers, etc) But <look at my last post>...

Plus, using another compiler inside your interpreter is kinda gay, lol
01-19-2003
Polymorphic OOP

Quote:

Originally posted by Speedy5
OK, let me elaborate a bit more. I've found a way to run the BL 7x slower than c++. I tested ifs, switches, function pointers and a few other techniques. Function pointers run fastest for running a single instruction. Then, I tested that 60% of the time is being spent calling up the functions for the instructions. So when I saw that code for cloning functions, I thought, maybe I can cut and paste the bodies of the instruction-handling functions together to form a primitive type of JIT. This would boost speed at the expense of memory (which no one really cares when you have 512mb+...lol). Portable to other compilers? I don't care. I'm using the managed extensions (.NET) anyways. So I have to use C++.NET's compiler. This part, however, and test, is in unmanaged, normal code. How do you clone?

You need to understand programming more before you continue with your project. You still are making absolutely no sense what-so-ever as to why you want to clone a function. You are stubborn at thinking this is what you should do, when you couldn't be more wrong.

Post a detailed description of exactly what your problem is and why you'd think you need to do such an obscure thing, and we'll explain to you what you should be doing instead. Until then, we can't answer your question, as we've said many times, because C++ does not directly allow you to do such a thing (and for good reason, because it would never make sense to do it).
01-19-2003
Polymorphic OOP

Also, you're saying your language is interpretted and that you are building a virtual machine, yet you allow inline assembly. Do you understand how contradictory that is? Rethink the language as well as the way you are coding your virtual machine. You don't understand what you want.
01-19-2003
Speedy5

I'm not a newbie programmer just for your information...

OK, BL is compiled into bytecode. This is portable just like Java's bytecode and .NET's bytecode. A sample would be (and it's not really like this but this is just for demonstration):

//a+=b*c-d;
mov reg1, b
mul reg1, c
sub reg1, d
add a, reg1

This might be compiled into byte values (figure it out, it's simple):

0, 0, 2
1, 0, 3
2, 0, 4
3, 1, 0

To interpret this byte code, I could use a gigantic if-else statement to test the instruction id, I could use a switch, or an array of function pointers. The last one is fastest, as I tested. So:

void bl_mov(int var1, int var2) { ... }
void bl_mul(int var1, int var2) { ... }
void bl_sub(int var1, int var2) { ... }
void bl_add(int var1, int var2) { ... }

void (*Instructions[])(int,int);
.....
Instructions[0]=&bl_mov;
Instructions[1]=&bl_mul;
.....
struct Instruction { int ID, var1, var2; }
......
Instruction i;
(Instructions[i.ID])(i.var1, i.var2);

That works great, no doubt about it. But almost 60% of the time is spent calling up the function. It also runs 7x slower than C++ code. So when I saw the cloning functions code, I thought, instead of calling the functions up, lets take their code, paste them together removing the return statements (trimming off the last X bytes), and voila. One function in a char* array I can cast into a function pointer and one function call instead of many. That would definatly boost speed to, lets say, 3.5x slower, which is much better. I know it's not efficient the big function that is made but that will be up to the programmer using the 'fastrun' attribute he can place to methods telling us to do this way instead of the normal way.

All I need to know, is how to do it. I can access the code inside the functions (readonly). I know their sizes. Which way do I copy it, how many bytes (or how) to trim off the return statements, and how do I call it? Yea, its a bit advanced.

Just a sidenote: This doesn't really prove BlueLightning's power nor extensibility. Plan #5, if you visited the site, will come shortly with amazing features.
01-19-2003
Speedy5

LOL, inline assembly! Yup, it has it. There is something called an if statement you can use. Using (imagining the future libraries), BL.System.GetProperty("/OS/name") and placing it into variable a, you can:

if (a=="Windows")
{
asm
{
//.......
}
}
else
{
// normal bl code
}

But we are thinking of dropping inline assembly in favor of assembly methods like:

asm void FastStuff()
{
// all assembler (easy syntax with access to all variables in scope).
}

Plus, its not like we'll encourage it. All the libaries will be free from inline assembler. They will have native methods, true, but since they are DLLs like Photoshop, we can load/unload/replace, blah blah blah, you guys know this stuff, you're just as advanced.... :-D
01-19-2003
Polymorphic OOP

Quote:

Originally posted by Speedy5
LOL, inline assembly! Yup, it has it.

And I again point out that you can't allow for inline assembly in an interpretted language because that would make the language not interpretted. You don't understand what you want let alone how to come to a solution. Rethink what you want to do.

I'm reading your other post right now, I'll get back to you in a minute.
01-19-2003
Polymorphic OOP

Quote:

Originally posted by Speedy5
That works great, no doubt about it. But almost 60% of the time is spent calling up the function. It also runs 7x slower than C++ code. So when I saw the cloning functions code, I thought, instead of calling the functions up, lets take their code, paste them together removing the return statements (trimming off the last X bytes), and voila. One function in a char* array I can cast into a function pointer and one function call instead of many. That would definatly boost speed to, lets say, 3.5x slower, which is much better. I know it's not efficient the big function that is made but that will be up to the programmer using the 'fastrun' attribute he can place to methods telling us to do this way instead of the normal way.

...haha

You're telling me that all this time that all you wanted to do was inline the function call...

Write the word "inline" before the function definitions that you want the compiler to insert.
01-19-2003
Speedy5

The inline assembler is added as a nice bonus. And, as I said, its not encouraged or used in BL's libraries.

Do you know many multimedia applications code for SSE2? and MMX? Take games or codecs. Those tehnologies aren't guaranteed to be on the user's proccessor but they use a if or something to determine which version to run. Same thing with BL, asm, native, normal code.

BTW, It's nice talking to another intellectual programmer and not a newb. Also that from your profile, you're young too (well partly, lol).
01-19-2003
Speedy5

Uh no. Not inline. I'm not stupid. I wanted to copy its code, paste it into an array, remove the returns, repeat, and run. Inline code implies that you have some sort of pipeline to determine which intruction to run. That's what I have right now, a pipeline with function pointers. But, that's not what i'm trying to do right now, hence primitive JIT compiler.
01-19-2003
Polymorphic OOP

No, using the word inline actually will copy the body of the function in there.
01-19-2003
Speedy5

I'd like to see you do that at runtime, when you don't know which order you have to place those functions. And, without a for loop when you run it. OK:

functions:
add, mov, sub, mul
0 1 2 3

in comes the bytecode: 0 3 2 1 2

before running (JIT'ing, hehe):
make new char array
while removing the 'ret' s:
insert the ACTUAL CODE of add
insert the ACTUAL CODE of mul
insert the ACTUAL CODE of sub
insert the ACTUAL CODE of mov
insert the ACTUAL CODE of sub
insert one 'ret'
store this char array
make function pointer to that char array

when function is called: call up the function pointer

This saves time, one function is called instead of 5. What you mean by inlining is:

// the functions are inline
for <as many instructions as there are>
{
switch (instructionID)
{
case 0:
add(); break;
case 1:
mov(); break;
case 2:
sub(); break;
case 3:
mul(); break;
}
}

I tested it, this type of pipeline is slower than function pointers stored in an array, as I described how I used in a previous post. But cloning the functions is even faster, if it can be done, and safely if done right. I'm pretty sure you can, but unsure of how.
01-19-2003
Eibro

"inline" is only a hint to the compiler to generate an inline function, not a command. What happens when you take the address of an inline function via a function pointer? :eek: Two copies of the function are most likely generated.
01-19-2003
Speedy5

If you take an address of an inline function, it acts just like ANY other function. Its ONLY when you explicitly call it when it inlines. BTW, in visual studio, __forceinline forces the inline unless its recursive.
01-19-2003
Polymorphic OOP

Write that part in assembly. You can't do it directly in C++.
01-19-2003
Eibro

Quote:

Originally posted by Speedy5
If you take an address of an inline function, it acts just like ANY other function. Its ONLY when you explicitly call it when it inlines. BTW, in visual studio, __forceinline forces the inline unless its recursive.

No, it's implementation defined what functions don't get inlined; and the rules governing what becomes of a function which doesn't get inlined were added very late in the standardization process.
01-19-2003
Speedy5

OK, just tested, I was able to clone the function. But it's unsafe, crashes for some reason when I use Debug build and not really stable. I also can't figure out how to remove the return statement so that the next function can execute. How many bytes is it? 1? 4? That would be a major obstacle making it all safe which is sure possible. How could you do it in ASM? If you could provide any assistance in ANY way to the project, please do. This goes out to all of you! :)

[email protected]
www.blue-lightning.tk

(Everything is being redone, most of the info on the site is old except, well the news, ;).)
01-19-2003
grib

This would be difficult enough to do in 1987 under dos. Expecting to do this under Windows Uberprofits from .NET to feed a P4 is begging for grief. Forget trying to copy and modify the compilers emitted opcodes, those were generated with a large number of assumptions that you are brutally violating. If you really want to do this you are going to have to become intimately familiar with how to generate and align the opcodes you need for your JIT compiled functions yourself. You will then need to either create a new segment, write, properly formatted, the codes you need, then mark that segment as text and do your context switch into the new read-only segment, then return and restore the state to whatever the assumptions .NET expected when you yanked the world out from under it. You can also mark your existing code segment as r/w and tell the linker to give you some room to party on. The latter will slow down your entire program, possibly quite painfully. The former takes quite a bit of time to get the new segment, but afterwards runs just as fast as dynamically loaded functions. You will also probably trip most anti virus software while doing any of this.

Finally have a look at pfe a forth environment that needs to do a lot of the things you are doing.
01-20-2003
orbitz

Heh, wow.

I think what speedy is trying to do is take his byte code, and use his bl_add or wahtever functions to create another function and then call that instead of calling 1 function, then another, and then another. This way he has to just call 1 functions rather than X many.

Two things, first of all. Depending on the size of your code, creating your 1 function and calling it might take longer than just calling each one.

Secondly, this is probably not the way to do this. You should probably read some literature on building and executing bytecoe first.

Now, relating to your language/VM here.
From the website, it seems like you lack some fundamental knowledge in virtual machines. In your list of featurse you say 'possibly platform independent'. What is the point of making your VM if the bytecode isn't platform independent? This also makes your current problem obsolete, if you are making a VM, then you DO care if the code is portable.

The problem with having inline ASM isn't like your argument using SSE/MMX. Different operating systems have different system calls, and ways of calling them. What speed-gain will you aquire by having to assemble the inline ASM during runtime. There is a very good reason Java does not have inline ASM, you should look into it.
01-20-2003
Speedy5

I can't stress enough that the information on the site is old. It was made when the BL project was first shown to the public, around August.

BL is platform independent, or else there would be no sense in making a VM. And the inline ASM is compiled during the compiling phase. It's stored in binary format inside the bytecode. It's called using a function pointer. It works. But, it's not platform independent. You can ALWAYS test for the target operating system at runtime to determine whether to execute it or not. Or maybe, the VM will test for that.

As for the JIT, it's probably not going to happen, it would be cool though but it's too much work for now. Maybe later unless I can get some help...

Since this board is awesome, I think, I will repost on similar subject once plan #5 is out to check out what you guys think of it. Everything will be updated at that time.