Thread: exe error: "The NTVDM CPU has encountered an illegal instruction"

  1. #1
    Registered User TheEngineer's Avatar
    Join Date
    Aug 2009
    Posts
    50

    exe error: "The NTVDM CPU has encountered an illegal instruction"

    I compiled a program with gcc using a makefile, and when I try to run the executable I get the following error:

    "C:\windows\system32\cmd.exe The NTVDM CPU has encountered an illegal instruction cS:0dca IP:01a7 OP:63 79 67 77 69"

    I think it may have something to do with the way my makefile is written. Does anyone see anything wrong with it?:

    Makefile:

    Code:
    bin_PROGRAMS = Program1
    
    
    SOURCE_DIR = $(top_srcdir)
    SOURCE_INCLUDE_DIR = $(SOURCE_DIR)/../../include
    SOURCE_COMMON_DIR = $(SOURCE_DIR)/../../common
    SOURCE_LDSCRIPT_DIR = $(SOURCE_DIR)
    
    
    CC=gcc
    AM_CFLAGS=-c -Wall
    AM_LDFLAGS=-M -T $(SOURCE_LDSCRIPT_DIR)/Linker.ld  
    SOURCES= $(SOURCE_DIR)/Main.c \
             $(SOURCE_DIR)/Error1.c \
             $(SOURCE_DIR)/File2.c \
             $(SOURCE_DIR)/File3.c \
             $(SOURCE_DIR)/File4.s \
             $(SOURCE_DIR)/File5.s 
    
    OBJECTS=$(SOURCES:.c=.o)
    
    
    
    all: Map
    	
    Map : $(OBJECTS) 
    	$(CC) $(AM_LDFLAGS) $(SOURCES) -o $@
    
    %.o : %.c 
    	$(CC) $(AM_CFLAGS) $< -o $@
    %.o : %.s 
    	$(CC) $(AM_CFLAGS) $< -o $@

  2. #2
    and the Hat of Guessing tabstop's Avatar
    Join Date
    Nov 2007
    Posts
    14,336
    Have you googled that error message?

    Do you have something bizarre in your code that was appropriate for 16-bit machines only?

  3. #3
    Registered User
    Join Date
    Oct 2008
    Location
    TX
    Posts
    2,059
    The AM_LDFLAGS looks whacky unless you explain what it's doing. Never seen a command where the path to the link-editor was specified. Also all ".s" files are processed by the assembler not the compiler, unless this make is a smart one to figure it out for you. Even then the macro AM_CFLAGS for the "%.o : %.s" rule would be invalid.
    Last edited by itCbitC; 10-25-2009 at 02:21 AM.

  4. #4
    Registered User TheEngineer's Avatar
    Join Date
    Aug 2009
    Posts
    50
    Quote Originally Posted by tabstop View Post
    Have you googled that error message?

    Do you have something bizarre in your code that was appropriate for 16-bit machines only?
    Quote Originally Posted by itCbitC
    The AM_LDFLAGS looks whacky unless you explain what it's doing. Never seen a command where the path to the link-editor was specified. Also all ".s" files are processed by the assembler not the compiler, unless this make is a smart one to figure it out for you. Even then the macro AM_CFLAGS for the "%.o : %.s" rule would be invalid.
    I googled the error message and got:

    "You are getting this error because the file you are trying to run is trying to access the computer's memory. Because you are not running the file on a DOS or Win 3.X machine, NTVDM will not allow this. NTVDM allows a 16-bit DOS program to execute on Windows. It is basically a DOS emulator ."

    I realized that the only place I am attempting to access the computer's memory is in the linker script. I took these files from a program that was cross-compiled for an m68k processor but I need to compile it for the i686 also. Is there anything illegal about these instructions?:


    Code:
    PROVIDE (__stack = 0x60300);
    
    MEMORY
    {
        RomArea (rxi) : ORIGIN = 0x10000, LENGTH = 0x10000  
        GenArea (rxi) : ORIGIN = 0x60000, LENGTH = 0x2000
        RamArea (rwxi): ORIGIN = 0x62000, LENGTH = 0xE000
        HighLev (rwxi): ORIGIN = 0x70000, LENGTH = 0x200000
    }
    
    
    SECTIONS
    
    {
       
    	.text	0x70000	: { *(.text) } = 0x4e71 
    	.data	.	: { *(.data) } 
    	.bss	.	: { *(.bss) *(COMMON) } 
    		_bss_end__ = . ; 
    		__bss_end__ = . ;
    		. = ALIGN(32 / 8);
    		. = ALIGN(32 / 8);
    		__end__ = . ;
    		_end = .;
    		PROVIDE (end = .); 
    
    	.rodata .	: { } 
          .rodata.str1.1 .	: { } 
    	
    
    	.cnfgtbl 0x10000	: {*(.cnfgtbl) . = 0x10000; } /*Load configuration table here */
        
        
    	.genvers 0xFFFE : { . = 0x02; } =0xA5    
    
    
        	.aparea2  0x62000	: { . += 0xe000; }	/* Reserve  area */
          
        	.generic 0x60000	: { . = 0x2000; }	
        
    }
    itCbitC--

    I also took this makefile from the cross-compiled program and that is how the linker script reference is set up. Is there a more proper way to do this?

    Also, I wasn't really sure what to do with the ".s" files since the other program didn't have any. Do I list them elsewhere or not include them in the makefile?


    Thanks!

  5. #5
    Registered User
    Join Date
    Oct 2008
    Location
    TX
    Posts
    2,059
    Quote Originally Posted by TheEngineer View Post
    I realized that the only place I am attempting to access the computer's memory is in the linker script. I took these files from a program that was cross-compiled for an m68k processor but I need to compile it for the i686 also. Is there anything illegal about these instructions?
    Herein lies your biggest problem. The m68k and x86 architectures are poles apart; the former is big-endian and uses linear addressing while the latter is little-endian and uses segmented addressing. You just can't take a makefile specific to m68k and plop it into an x86 machine and expect it to work especially when that makefile has commands specific to a chipset.
    Quote Originally Posted by TheEngineer View Post
    Code:
    PROVIDE (__stack = 0x60300);
    
    MEMORY
    {
        RomArea (rxi) : ORIGIN = 0x10000, LENGTH = 0x10000  
        GenArea (rxi) : ORIGIN = 0x60000, LENGTH = 0x2000
        RamArea (rwxi): ORIGIN = 0x62000, LENGTH = 0xE000
        HighLev (rwxi): ORIGIN = 0x70000, LENGTH = 0x200000
    }
    
    
    SECTIONS
    
    {
       
    	.text	0x70000	: { *(.text) } = 0x4e71 
    	.data	.	: { *(.data) } 
    	.bss	.	: { *(.bss) *(COMMON) } 
    		_bss_end__ = . ; 
    		__bss_end__ = . ;
    		. = ALIGN(32 / 8);
    		. = ALIGN(32 / 8);
    		__end__ = . ;
    		_end = .;
    		PROVIDE (end = .); 
    
    	.rodata .	: { } 
          .rodata.str1.1 .	: { } 
    	
    
    	.cnfgtbl 0x10000	: {*(.cnfgtbl) . = 0x10000; } /*Load configuration table here */
        
        
    	.genvers 0xFFFE : { . = 0x02; } =0xA5    
    
    
        	.aparea2  0x62000	: { . += 0xe000; }	/* Reserve  area */
          
        	.generic 0x60000	: { . = 0x2000; }	
        
    }
    I'm not sure if this the output of the assembler on the x86 machine or if it's a file that is the output of the m68k cross-compilation?
    Quote Originally Posted by TheEngineer View Post
    itCbitC--

    I also took this makefile from the cross-compiled program and that is how the linker script reference is set up. Is there a more proper way to do this?
    Yep! start from scratch. Write your own makefile for the target machine instead of using one that was built with a specific micro in mind. Alternatively strip the files of all m68k specifics and proceed with the compilation, altho' I would prefer the former approach to the latter.
    Quote Originally Posted by TheEngineer View Post
    Also, I wasn't really sure what to do with the ".s" files since the other program didn't have any. Do I list them elsewhere or not include them in the makefile?


    Thanks!
    Were these ".s" files created on the x86 machine or did you port them over from the m68k cross-compilation? If the latter, then throw them away as they will generate arbitrary bit patterns that won't translate to valid opcodes on the target x86 machine ie illegal instructions.

  6. #6
    Registered User
    Join Date
    Sep 2006
    Posts
    8,868
    I get this error when running Turbo C/C++, whenever a pointer goes awry and hits WindowsXP in a sensitive spot, or if the memory it's given on start up, gets too low (by becoming fragmented or "lost"), typically during debugging sessions.

    It is a 16 bit program, of course.

  7. #7
    Registered User TheEngineer's Avatar
    Join Date
    Aug 2009
    Posts
    50
    Quote Originally Posted by itCbitC View Post
    Herein lies your biggest problem. The m68k and x86 architectures are poles apart; the former is big-endian and uses linear addressing while the latter is little-endian and uses segmented addressing. You just can't take a makefile specific to m68k and plop it into an x86 machine and expect it to work especially when that makefile has commands specific to a chipset.

    I'm not sure if this the output of the assembler on the x86 machine or if it's a file that is the output of the m68k cross-compilation?

    Yep! start from scratch. Write your own makefile for the target machine instead of using one that was built with a specific micro in mind. Alternatively strip the files of all m68k specifics and proceed with the compilation, altho' I would prefer the former approach to the latter.

    Were these ".s" files created on the x86 machine or did you port them over from the m68k cross-compilation? If the latter, then throw them away as they will generate arbitrary bit patterns that won't translate to valid opcodes on the target x86 machine ie illegal instructions.

    I realize that the m68k makefile would not work on the i686, and I made all changes that I thought were necessary. Earlier you mentioned something about the path to the linker script and I was just wondering if you knew a different way to declare it if that will not work? I don't get any errors about it when compiling. Also, the .s files and all the other source files were originally written with Green Hills Multi software and were then edited to work with gnu, so they did not come from the m68k. What should I do with them?

    The Makefile that I posted is one that I wrote from scratch, using just file locations and names from the m68k version. I have very little experience writing these, as you can tell, and am just looking for any advice about elements that may not work. Again, I'm not getting errors when compiling the exe file, but am getting an illegal instruction error when trying to open it.


    Thanks for your help

  8. #8
    Officially An Architect brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,396
    Quote Originally Posted by TheEngineer View Post
    "C:\windows\system32\cmd.exe The NTVDM CPU has encountered an illegal instruction cS:0dca IP:01a7 OP:63 79 67 77 69"
    If you decode the OP: bytes as ASCII values, the result is "cygwi". Which is suspiciously like "cygwin".

    Something is, obviously, hilariously wrong.
    Code:
    //try
    //{
    	if (a) do { f( b); } while(1);
    	else   do { f(!b); } while(1);
    //}

  9. #9
    Officially An Architect brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,396
    By the way, the linker script is probably completely irrelevant. All you're trying to do is compile the code for Windows, right? Then you do not care about memory regions or image layout. Just remove the linker script entirely and see what happens.
    Code:
    //try
    //{
    	if (a) do { f( b); } while(1);
    	else   do { f(!b); } while(1);
    //}

  10. #10
    Registered User TheEngineer's Avatar
    Join Date
    Aug 2009
    Posts
    50
    Quote Originally Posted by brewbuck View Post
    By the way, the linker script is probably completely irrelevant. All you're trying to do is compile the code for Windows, right? Then you do not care about memory regions or image layout. Just remove the linker script entirely and see what happens.
    When I removed it I got some errors while compiling, which seems like progress. Thanks for the suggestion

  11. #11
    Registered User
    Join Date
    Oct 2008
    Location
    TX
    Posts
    2,059
    Quote Originally Posted by TheEngineer View Post
    When I removed it I got some errors while compiling, which seems like progress. Thanks for the suggestion
    Can you post the compile-time errors here so others can look at it and perhaps pinpoint their source.

  12. #12
    Registered User TheEngineer's Avatar
    Join Date
    Aug 2009
    Posts
    50
    Quote Originally Posted by itCbitC View Post
    Can you post the compile-time errors here so others can look at it and perhaps pinpoint their source.
    Here is the output now when I try to make:
    Code:
    ../ProgramMain.c: In function `main':
    ../ProgramMain.c:224: warning: passing arg 2 of `LoadTestProgram' makes pointer from
     integer without a cast
    ../ProgramMain.c:80: warning: return type of 'main' is not `int'
    ../Assembly.s: Assembler messages:
    ../Assembly.s:10: Error: no such instruction: `ds.l 15'
    ../Assembly.s:15: Error: no such instruction: `movem.l %D0/%D1/%D2/%D3/%D4/%D5/%D
    6/%D7/%A0/%A1/%A2/%A3/%A4/%A5/%A6,RegStore'
    ../Assembly.s:16: Error: no such instruction: `link %A6,'
    ../Assembly.s:19: Error: no such instruction: `move.l 8(%A6),%A2'
    ../Assembly.s:22: Error: no such instruction: `jsr (%A2)'
    ../Assembly.s:24: Error: no such instruction: `move.l (%A7)+,%A4'
    ../Assembly.s:26: Error: no such instruction: `unlk %A6'
    ../Assembly.s:28: Error: no such instruction: `move.l (%SP)+,%A0'
    ../Assembly.s:29: Error: suffix or operands invalid for `add'
    ../Assembly.s:30: Error: no such instruction: `move.l %A0,-(%SP)'
    ../Assembly.s:31: Error: no such instruction: `movem.l RegStore,%D0/%D1/%D2/%D3/%
    D4/%D5/%D6/%D7/%A0/%A1/%A2/%A3/%A4/%A5/%A6'
    ../Assembly.s:33: Error: no such instruction: `rts '
    make: *** [Program] Error 1
    It seems to mostly be coming from one of my assembly files. I had these kinds of errors earlier when I was converting the assembly files from m68k, it seems strange that they would show up when the linker script is removed. Any ideas?

  13. #13
    Registered User
    Join Date
    Oct 2008
    Location
    TX
    Posts
    2,059
    Quote Originally Posted by TheEngineer View Post
    Here is the output now when I try to make:
    Code:
    ../ProgramMain.c: In function `main':
    ../ProgramMain.c:224: warning: passing arg 2 of `LoadTestProgram' makes pointer from
     integer without a cast
    ../ProgramMain.c:80: warning: return type of 'main' is not `int'
    ../Assembly.s: Assembler messages:
    ../Assembly.s:10: Error: no such instruction: `ds.l 15'
    ../Assembly.s:15: Error: no such instruction: `movem.l %D0/%D1/%D2/%D3/%D4/%D5/%D
    6/%D7/%A0/%A1/%A2/%A3/%A4/%A5/%A6,RegStore'
    ../Assembly.s:16: Error: no such instruction: `link %A6,'
    ../Assembly.s:19: Error: no such instruction: `move.l 8(%A6),%A2'
    ../Assembly.s:22: Error: no such instruction: `jsr (%A2)'
    ../Assembly.s:24: Error: no such instruction: `move.l (%A7)+,%A4'
    ../Assembly.s:26: Error: no such instruction: `unlk %A6'
    ../Assembly.s:28: Error: no such instruction: `move.l (%SP)+,%A0'
    ../Assembly.s:29: Error: suffix or operands invalid for `add'
    ../Assembly.s:30: Error: no such instruction: `move.l %A0,-(%SP)'
    ../Assembly.s:31: Error: no such instruction: `movem.l RegStore,%D0/%D1/%D2/%D3/%
    D4/%D5/%D6/%D7/%A0/%A1/%A2/%A3/%A4/%A5/%A6'
    ../Assembly.s:33: Error: no such instruction: `rts '
    make: *** [Program] Error 1
    It seems to mostly be coming from one of my assembly files. I had these kinds of errors earlier when I was converting the assembly files from m68k, it seems strange that they would show up when the linker script is removed. Any ideas?
    There are two kinds of errors: C programming errors which are easy to fix; the others are the assembler errors which are not possible to fix. In my earlier post I had told you to throw away the m68k assembly files as they will never work on the x86 machine because the instruction sets of these two micros are totally different. The "movem.l" and "unlnk" and "rts" instructions etc. are found on m68k assemblers only.

    You stated that the ".s" files had been generated on the x86 machine which is an incorrect statement. It seems that you have copied over everything blindly from the m68k cross-compilation without realizing that this process will never work on your Windows machine. Prefixing a percent sign to m68k registers like A4 or D2 won't help either. The only portable thing between an m68k machine and the Windows (x86) machine is the source code written in C.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. What cause NTVDM CPU error?
    By xianjianwuhen in forum C Programming
    Replies: 4
    Last Post: 09-04-2009, 09:21 AM
  2. questions on multiple thread programming
    By lehe in forum C Programming
    Replies: 11
    Last Post: 03-27-2009, 07:44 AM
  3. SSH Hacker Activity!! AAHHH!!
    By Kleid-0 in forum A Brief History of Cprogramming.com
    Replies: 15
    Last Post: 03-06-2005, 03:53 PM