Problem reading a certain sized integer into a value from a file.

I'm reading a file that calls for values to be interpreted from the file as 2-byte signed integers. So I create a short integer data type and read in the values, but from what I see from the hexeditor i'm using I'm getting totally different values. This is how the file marker is described in the specification

Quote:

---

ALTW stands for 'Altitude in 16-bit Words'. After The "ALTW" marker, the following appear in order:
HeightScale, a 2-byte signed integer value.
BaseHeight, a 2-byte signed integer value.
Elevations, a sequence of 2-byte signed integers.

There are (xpts * ypts) elevation integers, where xpts and ypts will have been set earlier in the "SIZE" chunk or the "XPTS" and "YPTS" chunks. The elevations are ordered such that the first row (y = 0) is read first from left to right, then the second (y = 1), and so on. The values in Elevations are not absolute altitudes. The absolute altitude of a particular point (in the same scale as x and y) is equal to BaseHeight + Elevation * HeightScale / 65536.

---

so I write this routine to read the number of elevation values

Code:

---

    short int a = header.BaseHeight;
    short int b = header.heightScale;

    //reads in and calculates header data
    for(int y = 0; y < header.ypts; ++y)
    {
        for(int x = 0; x < header.xpts; ++x)
        {
            signed int tempHeight;

            file.read((char*)&tempHeight, 2);

            short int calcHeight = a + tempHeight * b / 65536;

            elevations[(y * header.xpts) + x] = calcHeight;
        }
    }

---

the elevations array is an array of short integers and is just meant for holding the data in a "square" configuration.

Now at the point I started reading that data, the hex editor shows the first value to be.
0x799D which converted to a signed integer would be -1,635, but when I go to debug it the value is read 0x9D79 which is 25223

You could read byte by byte and assemble the integer yourself. Salem has provided countless examples.

I'm just wondering why it reads the byte backwards, it reads all the other integer values in line, this one just comes in weird.

> signed int tempHeight;
> file.read((char*)&tempHeight, 2);
Ohh, this is SO wrong. This leaves the other two bytes of your int undefined.

Assuming for the moment the code is endian-correct, then you need to read into a short, then assign it to an int.

> 0x799D which converted to a signed integer would be -1,635,
> but when I go to debug it the value is read 0x9D79 which is 25223
This made no sense at all, since 0x799d is 31133.

> short int calcHeight = a + tempHeight * b / 65536;
Make sure you're not getting arithmetic overflow here.

Yeah I corrected that and changed it to signed short.

and 799D is supposed to be converted to a signed number, so that's what I got when I converted it to signed since the high order bit is 1. I did the flipping and everything.

The major problem is that the bytes are being read backwards

Code:

---

    //reads in and calculates header data
    for(int y = 0; y < header.ypts; ++y)
    {
        for(int x = 0; x < header.xpts; ++x)
        {
            short int height;

            file.read((char*)&height, 2);

            short int calcHeight = a + height * b / 65536;

            elevations[(y * header.xpts) + x] = calcHeight;
        }
    }

---

The other stuff I'll hit when I get there but for now I need those numbers right

I tried creating two unsigned shorts and have each hold one byte each or the two byte integer, with the high order bits being zero. I tried to shift them to the left a byte but it didn't end up working. what is the proper way to construct a number in the proper order.

unsigned short b1 = read();
unsigned short b2 = read();

For little endian,
unsigned short result = (b2 << 8) | b1;

For big endian
unsigned short result = (b1 << 8) | b2;

Ah, That's kind of what I was doing, except I was absent mindend and though the shift was shifting bytes.

SO how exactly do I determine big/little endian values?

> SO how exactly do I determine big/little endian values?
Look closely at the specification of the file format to see which order multi-byte quantities are stored in.

The only order so far I've seen was that it is intel-ordered. and none of the values required in the equation (except for determining the size of the array) have anything about order in the file specification. For the size of the array the value read fine. In the hex editor the size shows up as 0x0101 and when I read it into a 2-byte short integer it had no errors, but when I read these numbers they go in backwards.

here is what I did.

Code:

---

    for(int y = 0; y < header.ypts; ++y)
    {
        for(int x = 0; x < header.xpts; ++x)
        {
            unsigned short heightA = 0;
            unsigned short heightB = 0;

            file.read((char*)&heightA, 1);
            file.read((char*)&heightB, 1);

            short int result = (heightA << 8) | heightB;

            short int calcHeight = a + result * b / 65536;

            elevations[(y * header.xpts) + x] = calcHeight;
        }
    }

---

Here's a picture of how the data is structured in the file.

Heres the class to give you a bigger picture

Code:

---

#ifndef TERRAGENFILE_H
#define TERRAGENFILE_H

#include <iostream>
#include <fstream>
#include <string>
#include <iomanip>

enum FileError
{
    FILE_NO_ERROR,
    FILE_TYPE_MISMATCH,
    FILE_READ_ERROR
};

class TerHeader
{
    friend class TerragenFile;
    friend std::ostream& operator<<(std::ostream& out, TerragenFile file);
        char title[9];
        char type[8];
        short size;
        short int xpts;
        short int ypts;
        float xyzScal[3];
        float crad;
        unsigned int crvm;
        short int heightScale;
        short int baseHeight;
};

class TerragenFile
{
    friend std::ostream& operator<<(std::ostream& out, TerragenFile file);
    public:
        TerragenFile();
        ~TerragenFile();
        FileError load(const char* filename);
        void release();
        void printHeights(std::ostream& out=std::cout) const;

    private:
        TerHeader header;
        short int* elevations;
};

std::ostream& operator<<(std::ostream& out, TerragenFile file);

---

implementation

Code:

---

#include "TerragenFile.h"

using std::endl;
using std::ifstream;
using std::string;
using std::setw;

TerragenFile::TerragenFile(): elevations(NULL){ }

TerragenFile::~TerragenFile(){ release(); }

FileError TerragenFile::load(const char* filename)
{
    //set the null string for the titles
    ifstream file(filename);

    //error reading the file
    if(!file)
        return FILE_NO_ERROR;

    //read the header, if the first string is not "TERRAGEN" throw error
    header.title[8] = header.type[7] = '\0';
    file.read(header.title, 8);
    file.read(header.type, 7);

    if(header.title != string("TERRAGEN"))
        return FILE_TYPE_MISMATCH;

    //seek to the size and read
    file.ignore(5);
    file.read((char*)&header.size, 2);

    //seek to xpts and read
    file.ignore(6);
    file.read((char*)&header.xpts, 2);

    //seek to ypts and read
    file.ignore(6);
    file.read((char*)&header.ypts, 2);

    elevations = new short int[header.xpts * header.ypts];

    //seek to scal and read
    file.ignore(6);
    file.read((char*)&header.xyzScal[0], 4);
    file.read((char*)&header.xyzScal[1], 4);
    file.read((char*)&header.xyzScal[2], 4);

    //seek to CRAD and read
    file.ignore(4);
    file.read((char*)&header.crad, 4);

    //seek to CRVM and read
    file.ignore(4);
    file.read((char*)&header.crvm, 4);

    //seek to ALTW and read height scale and bae height
    file.ignore(4);
    file.read((char*)&header.heightScale, 2);
    file.read((char*)&header.baseHeight, 2);

    short int a = header.baseHeight;
    short int b = header.heightScale;

    //reads in and calculates header data
    for(int y = 0; y < header.ypts; ++y)
    {
        for(int x = 0; x < header.xpts; ++x)
        {
            unsigned short heightA = 0;
            unsigned short heightB = 0;

            file.read((char*)&heightA, 1);
            file.read((char*)&heightB, 1);

            short int result = (heightA << 8) | heightB;

            short int calcHeight = a + result * b / 65536;

            elevations[(y * header.xpts) + x] = calcHeight;
        }
    }



    file.close();
    return FILE_NO_ERROR;
}

void TerragenFile::release()
{
    delete [] elevations;
}

void TerragenFile::printHeights(std::ostream& out) const
{
        //reads in and calculates header data
    for(int y = 0; y < header.ypts; ++y)
    {
        for(int x = 0; x < header.xpts; ++x)
            out << elevations[(y * header.xpts) + x] << setw(3);

        out << std::endl;
    }
}
std::ostream& operator<<(std::ostream& out, TerragenFile file)
{
    out << file.header.title << " : " << file.header.type << endl;
    out << "SIZE = " << file.header.size << endl;
    out << "XPTS = " << file.header.xpts << endl;
    out << "YPTS = " << file.header.ypts << endl;
    out << "XSCAL = " << file.header.xyzScal[0] << endl;
    out << "ySCAL = " << file.header.xyzScal[1] << endl;
    out << "zSCAL = " << file.header.xyzScal[2] << endl;
    out << "CRAD = " << file.header.crad << endl;
    out << "CRVM = " << file.header.crvm << endl;
    return out;
}

---

Intel order is just another name for little-endian.

So by default the fstream read function reads the bytes in backwards?

No, by default (and there is nothing but the default) the read function reads the bytes in the order it gets them and writes them into the buffer in the order of increasing memory address.

How the buffer is then interpreted has nothing to do with read().

so aside from piecing the short, float, or int byte by byte, how would I have each of the data type interpret the stream as it is read?

I don't understand the question.

The only reliable way of reading binary data is assembling the larger types from bytes yourself.

And even when doing that, just shifting is not entirely reliable. Platform differences can be more than just byte order.

So is there a "Safe" strategy to resolving the problem. Essentially I'm the only one going to be using this class, and most of the files generated are similar.

It really depends on the usage of the data. If all you ever do with it is, on the same computer architecture, write out files and read them back in, you can use the simplest format you can come up with.

If you want to exchange data with other architectures, that's where the trouble starts.

ok ok ok, I think i was reading the data wrong in the loop. Little endian means if I have a value such as

0F FD;

then the resulting number will be

FD0F right. No wonder the other values were reading correctly when I read them.