Thread: Problem reading a certain sized integer into a value from a file.

  1. #1
    Ethernal Noob
    Join Date
    Nov 2001
    Posts
    1,901

    Problem reading a certain sized integer into a value from a file.

    I'm reading a file that calls for values to be interpreted from the file as 2-byte signed integers. So I create a short integer data type and read in the values, but from what I see from the hexeditor i'm using I'm getting totally different values. This is how the file marker is described in the specification

    ALTW stands for 'Altitude in 16-bit Words'. After The "ALTW" marker, the following appear in order:
    HeightScale, a 2-byte signed integer value.
    BaseHeight, a 2-byte signed integer value.
    Elevations, a sequence of 2-byte signed integers.

    There are (xpts * ypts) elevation integers, where xpts and ypts will have been set earlier in the "SIZE" chunk or the "XPTS" and "YPTS" chunks. The elevations are ordered such that the first row (y = 0) is read first from left to right, then the second (y = 1), and so on. The values in Elevations are not absolute altitudes. The absolute altitude of a particular point (in the same scale as x and y) is equal to BaseHeight + Elevation * HeightScale / 65536.
    so I write this routine to read the number of elevation values

    Code:
        short int a = header.BaseHeight;
        short int b = header.heightScale;
    
        //reads in and calculates header data
        for(int y = 0; y < header.ypts; ++y)
        {
            for(int x = 0; x < header.xpts; ++x)
            {
                signed int tempHeight;
    
                file.read((char*)&tempHeight, 2);
    
                short int calcHeight = a + tempHeight * b / 65536;
    
                elevations[(y * header.xpts) + x] = calcHeight;
            }
        }
    the elevations array is an array of short integers and is just meant for holding the data in a "square" configuration.

    Now at the point I started reading that data, the hex editor shows the first value to be.
    0x799D which converted to a signed integer would be -1,635, but when I go to debug it the value is read 0x9D79 which is 25223
    Last edited by indigo0086; 06-16-2007 at 06:20 PM.

  2. #2
    Just Lurking Dave_Sinkula's Avatar
    Join Date
    Oct 2002
    Posts
    5,005
    You could read byte by byte and assemble the integer yourself. Salem has provided countless examples.
    7. It is easier to write an incorrect program than understand a correct one.
    40. There are two ways to write error-free programs; only the third one works.*

  3. #3
    Ethernal Noob
    Join Date
    Nov 2001
    Posts
    1,901
    I'm just wondering why it reads the byte backwards, it reads all the other integer values in line, this one just comes in weird.

  4. #4
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    39,660
    > signed int tempHeight;
    > file.read((char*)&tempHeight, 2);
    Ohh, this is SO wrong. This leaves the other two bytes of your int undefined.

    Assuming for the moment the code is endian-correct, then you need to read into a short, then assign it to an int.

    > 0x799D which converted to a signed integer would be -1,635,
    > but when I go to debug it the value is read 0x9D79 which is 25223
    This made no sense at all, since 0x799d is 31133.

    > short int calcHeight = a + tempHeight * b / 65536;
    Make sure you're not getting arithmetic overflow here.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.

  5. #5
    Ethernal Noob
    Join Date
    Nov 2001
    Posts
    1,901
    Yeah I corrected that and changed it to signed short.

    and 799D is supposed to be converted to a signed number, so that's what I got when I converted it to signed since the high order bit is 1. I did the flipping and everything.

    The major problem is that the bytes are being read backwards
    Code:
        //reads in and calculates header data
        for(int y = 0; y < header.ypts; ++y)
        {
            for(int x = 0; x < header.xpts; ++x)
            {
                short int height;
    
                file.read((char*)&height, 2);
    
                short int calcHeight = a + height * b / 65536;
    
                elevations[(y * header.xpts) + x] = calcHeight;
            }
        }
    The other stuff I'll hit when I get there but for now I need those numbers right

    I tried creating two unsigned shorts and have each hold one byte each or the two byte integer, with the high order bits being zero. I tried to shift them to the left a byte but it didn't end up working. what is the proper way to construct a number in the proper order.
    Last edited by indigo0086; 06-17-2007 at 05:11 AM.

  6. #6
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    39,660
    unsigned short b1 = read();
    unsigned short b2 = read();

    For little endian,
    unsigned short result = (b2 << 8) | b1;

    For big endian
    unsigned short result = (b1 << 8) | b2;
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.

  7. #7
    Ethernal Noob
    Join Date
    Nov 2001
    Posts
    1,901
    Ah, That's kind of what I was doing, except I was absent mindend and though the shift was shifting bytes.

    SO how exactly do I determine big/little endian values?

  8. #8
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    39,660
    > SO how exactly do I determine big/little endian values?
    Look closely at the specification of the file format to see which order multi-byte quantities are stored in.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.

  9. #9
    Ethernal Noob
    Join Date
    Nov 2001
    Posts
    1,901
    The only order so far I've seen was that it is intel-ordered. and none of the values required in the equation (except for determining the size of the array) have anything about order in the file specification. For the size of the array the value read fine. In the hex editor the size shows up as 0x0101 and when I read it into a 2-byte short integer it had no errors, but when I read these numbers they go in backwards.

    here is what I did.


    Code:
        for(int y = 0; y < header.ypts; ++y)
        {
            for(int x = 0; x < header.xpts; ++x)
            {
                unsigned short heightA = 0;
                unsigned short heightB = 0;
    
                file.read((char*)&heightA, 1);
                file.read((char*)&heightB, 1);
    
                short int result = (heightA << 8) | heightB;
    
                short int calcHeight = a + result * b / 65536;
    
                elevations[(y * header.xpts) + x] = calcHeight;
            }
        }
    Here's a picture of how the data is structured in the file.
    Last edited by indigo0086; 06-17-2007 at 08:25 AM.

  10. #10
    Ethernal Noob
    Join Date
    Nov 2001
    Posts
    1,901
    Heres the class to give you a bigger picture
    Code:
    #ifndef TERRAGENFILE_H
    #define TERRAGENFILE_H
    
    #include <iostream>
    #include <fstream>
    #include <string>
    #include <iomanip>
    
    enum FileError
    {
        FILE_NO_ERROR,
        FILE_TYPE_MISMATCH,
        FILE_READ_ERROR
    };
    
    class TerHeader
    {
        friend class TerragenFile;
        friend std::ostream& operator<<(std::ostream& out, TerragenFile file);
            char title[9];
            char type[8];
            short size;
            short int xpts;
            short int ypts;
            float xyzScal[3];
            float crad;
            unsigned int crvm;
            short int heightScale;
            short int baseHeight;
    };
    
    class TerragenFile
    {
        friend std::ostream& operator<<(std::ostream& out, TerragenFile file);
        public:
            TerragenFile();
            ~TerragenFile();
            FileError load(const char* filename);
            void release();
            void printHeights(std::ostream& out=std::cout) const;
    
        private:
            TerHeader header;
            short int* elevations;
    };
    
    std::ostream& operator<<(std::ostream& out, TerragenFile file);

    implementation
    Code:
    #include "TerragenFile.h"
    
    using std::endl;
    using std::ifstream;
    using std::string;
    using std::setw;
    
    TerragenFile::TerragenFile(): elevations(NULL){ }
    
    TerragenFile::~TerragenFile(){ release(); }
    
    FileError TerragenFile::load(const char* filename)
    {
        //set the null string for the titles
        ifstream file(filename);
    
        //error reading the file
        if(!file)
            return FILE_NO_ERROR;
    
        //read the header, if the first string is not "TERRAGEN" throw error
        header.title[8] = header.type[7] = '\0';
        file.read(header.title, 8);
        file.read(header.type, 7);
    
        if(header.title != string("TERRAGEN"))
            return FILE_TYPE_MISMATCH;
    
        //seek to the size and read
        file.ignore(5);
        file.read((char*)&header.size, 2);
    
        //seek to xpts and read
        file.ignore(6);
        file.read((char*)&header.xpts, 2);
    
        //seek to ypts and read
        file.ignore(6);
        file.read((char*)&header.ypts, 2);
    
        elevations = new short int[header.xpts * header.ypts];
    
        //seek to scal and read
        file.ignore(6);
        file.read((char*)&header.xyzScal[0], 4);
        file.read((char*)&header.xyzScal[1], 4);
        file.read((char*)&header.xyzScal[2], 4);
    
        //seek to CRAD and read
        file.ignore(4);
        file.read((char*)&header.crad, 4);
    
        //seek to CRVM and read
        file.ignore(4);
        file.read((char*)&header.crvm, 4);
    
        //seek to ALTW and read height scale and bae height
        file.ignore(4);
        file.read((char*)&header.heightScale, 2);
        file.read((char*)&header.baseHeight, 2);
    
        short int a = header.baseHeight;
        short int b = header.heightScale;
    
        //reads in and calculates header data
        for(int y = 0; y < header.ypts; ++y)
        {
            for(int x = 0; x < header.xpts; ++x)
            {
                unsigned short heightA = 0;
                unsigned short heightB = 0;
    
                file.read((char*)&heightA, 1);
                file.read((char*)&heightB, 1);
    
                short int result = (heightA << 8) | heightB;
    
                short int calcHeight = a + result * b / 65536;
    
                elevations[(y * header.xpts) + x] = calcHeight;
            }
        }
    
    
    
        file.close();
        return FILE_NO_ERROR;
    }
    
    void TerragenFile::release()
    {
        delete [] elevations;
    }
    
    void TerragenFile::printHeights(std::ostream& out) const
    {
            //reads in and calculates header data
        for(int y = 0; y < header.ypts; ++y)
        {
            for(int x = 0; x < header.xpts; ++x)
                out << elevations[(y * header.xpts) + x] << setw(3);
    
            out << std::endl;
        }
    }
    std::ostream& operator<<(std::ostream& out, TerragenFile file)
    {
        out << file.header.title << " : " << file.header.type << endl;
        out << "SIZE = " << file.header.size << endl;
        out << "XPTS = " << file.header.xpts << endl;
        out << "YPTS = " << file.header.ypts << endl;
        out << "XSCAL = " << file.header.xyzScal[0] << endl;
        out << "ySCAL = " << file.header.xyzScal[1] << endl;
        out << "zSCAL = " << file.header.xyzScal[2] << endl;
        out << "CRAD = " << file.header.crad << endl;
        out << "CRVM = " << file.header.crvm << endl;
        return out;
    }

  11. #11
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,895
    Intel order is just another name for little-endian.
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

  12. #12
    Ethernal Noob
    Join Date
    Nov 2001
    Posts
    1,901
    So by default the fstream read function reads the bytes in backwards?

  13. #13
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,895
    No, by default (and there is nothing but the default) the read function reads the bytes in the order it gets them and writes them into the buffer in the order of increasing memory address.

    How the buffer is then interpreted has nothing to do with read().
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

  14. #14
    Ethernal Noob
    Join Date
    Nov 2001
    Posts
    1,901
    so aside from piecing the short, float, or int byte by byte, how would I have each of the data type interpret the stream as it is read?

  15. #15
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,895
    I don't understand the question.

    The only reliable way of reading binary data is assembling the larger types from bytes yourself.

    And even when doing that, just shifting is not entirely reliable. Platform differences can be more than just byte order.
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. C++ std routines
    By siavoshkc in forum C++ Programming
    Replies: 33
    Last Post: 07-28-2006, 12:13 AM
  2. Dikumud
    By maxorator in forum C++ Programming
    Replies: 1
    Last Post: 10-01-2005, 06:39 AM
  3. Possible circular definition with singleton objects
    By techrolla in forum C++ Programming
    Replies: 3
    Last Post: 12-26-2004, 10:46 AM
  4. Unknown Memory Leak in Init() Function
    By CodeHacker in forum Windows Programming
    Replies: 3
    Last Post: 07-09-2004, 09:54 AM
  5. Problem reading file
    By winsonlee in forum C Programming
    Replies: 2
    Last Post: 04-23-2004, 06:52 AM