Loading files into memory

This is a discussion on Loading files into memory within the C++ Programming forums, part of the General Programming Boards category; I stopped coding for about a year and a half ago. So I'm trying to find things to brush up ...

  1. #1
    Registered User
    Join Date
    Sep 2008
    Posts
    9

    Loading files into memory

    I stopped coding for about a year and a half ago. So I'm trying to find things to brush up on. At the moment it's file i/o.

    I'm not exactly sure what is wrong with my code, but when I go to check what I loaded into memory by outputting it to the consol, I am getting weird numbers.

    Code:
    #include <fstream>
    
    using namespace std;
    
    class BitmapPic
    {
        private:
                
            //bitmap fileheader(info about file)
            typedef struct Bitmap_FileHeader
            {
                unsigned short bfType;//WORD
                unsigned long bfSize;//DWORD
                unsigned short bfReserved1;
                unsigned short bfReserved2;
                unsigned long bfOffBits;
            };//total:14 bytes
            
            //bitmap infoheader:(info on contents of file)
            typedef struct Bitmap_InfoHeader
            {
                unsigned long biSize;//DWORD
                unsigned long biWidth;
                unsigned long biHeigth;
                unsigned short biPlanes;//WORD
                unsigned short biBitCount;
                unsigned long biCompression;
                unsigned long biSizeImage;
                unsigned long biXPelsPerMeter;
                unsigned long biYPelsPerMeter;
                unsigned long biClrUsed;
                unsigned long biClrImportant;
            };//total: 40 bytes
            
            //structure for containing pixel values.
            typedef struct RGBQuad
            {
                unsigned char rgbBlue;
                unsigned char rgbGreen;
                unsigned char rgbRed;
                unsigned char rgbReserved;
            };
    
        public:
    
            Bitmap_FileHeader bmfh;
            Bitmap_InfoHeader bmih;
            RGBQuad aColors[];
            
            void LBmp(char file[]);
    };
    
    void BitmapPic::LBmp(char file[])
    {
        ifstream infile(file,ios::binary|ios::in);
        infile.read(reinterpret_cast<char*>(&bmfh),sizeof(Bitmap_FileHeader));
        infile.read(reinterpret_cast<char*>(&bmih),sizeof(Bitmap_InfoHeader));
    }
    The only value that seems to come out true is the first one which is always going to be "19778"

    bfSize comes out as 0
    biWidth comes out as 18808832
    biHeigth comes out as 65536 (2^16)

  2. #2
    and the Hat of Guessing tabstop's Avatar
    Join Date
    Nov 2007
    Posts
    14,185
    Easy check: what does your compiler say sizeof(Bitmap_FileHeader) and sizeof(Bitmap_InfoHeader) are? I wouldn't be surprised if the first, at least, is not actually 14.

  3. #3
    Registered User
    Join Date
    Sep 2008
    Posts
    9
    sizeof(bmp.bmfh) = 16
    sizeof(bmp.bmih) = 40

    how can I fix that? I'm pretty sure I am using the right sized data in my structures.

  4. #4
    and the Hat of Guessing tabstop's Avatar
    Join Date
    Nov 2007
    Posts
    14,185
    You can either make a function that reads in the appropriate bits of the struct separately, or you can do something compiler-specific to "pack" the structure together so that there's no padding bytes in it.

  5. #5
    and the hat of wrongness Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    32,544
    Even if "pack" generates a struct of the right size, it still won't fix endian problems.

    Yes it sucks to read the file byte by byte, but it's the only sure way of producing a consistent answer independent of your compiler or platform.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.
    I support http://www.ukip.org/ as the first necessary step to a free Europe.

  6. #6
    Guest Sebastiani's Avatar
    Join Date
    Aug 2001
    Location
    Waterloo, Texas
    Posts
    5,708
    Since this is a Windows-specific file format, why not just use the structures defined in the related header files (since they're guaranteed to be aligned properly)?
    Code:
    #include <cmath>
    #include <complex>
    bool euler_flip(bool value)
    {
        return std::pow
        (
            std::complex<float>(std::exp(1.0)), 
            std::complex<float>(0, 1) 
            * std::complex<float>(std::atan(1.0)
            *(1 << (value + 2)))
        ).real() < 0;
    }

  7. #7
    Banned master5001's Avatar
    Join Date
    Aug 2001
    Location
    Visalia, CA, USA
    Posts
    3,685
    Holy sweet Jesus, Sebastiani! There is a name from back when I was really active on the forums. I hope the years have been kind to you old friend.

  8. #8
    Registered User
    Join Date
    Sep 2008
    Posts
    9
    it worked, reading it in piece by piece. I wanted to avoid that but meh... actually I did try sizeof(Bitmap_FileHeader) before I just forgot totally about it. I knew it had to do with bytes getting stored in the wrong spot somewhere. For some dumb reason I didn't think "oh my struct is 16bytes and it's only 14bytes"

    Anyways, thanks everyone

    side note: I'm just practice I/O, I'd like to try loading something like JPEG's into memory once I get a bit better. (eventually for texture mapping in OpenGL). I'd rather texture map other files than uncompressed bmp files. I know I could always use an image library like devIL but I'm just weird =P.

  9. #9
    Guest Sebastiani's Avatar
    Join Date
    Aug 2001
    Location
    Waterloo, Texas
    Posts
    5,708
    >> it worked, reading it in piece by piece. I wanted to avoid that but meh...

    Most compilers accept either of these two conventions:


    Code:
    // borland style?
    #pragma pack( push, sizeof( int ) ) 
    // stuff
    #pragma pack( pop )
     
     
    // gcc style
    class stuff
          __attribute__( ( packed( sizeof( int ) ) ) ) 
    {      };
    At any rate, there are many situations where it is preferable to reading in data "byte by byte".

    @master5001 : Good to see you old chap!
    Code:
    #include <cmath>
    #include <complex>
    bool euler_flip(bool value)
    {
        return std::pow
        (
            std::complex<float>(std::exp(1.0)), 
            std::complex<float>(0, 1) 
            * std::complex<float>(std::atan(1.0)
            *(1 << (value + 2)))
        ).real() < 0;
    }

  10. #10
    Registered User
    Join Date
    Sep 2008
    Posts
    9
    why would they add padding bytes to structs anyways?

  11. #11
    and the Hat of Guessing tabstop's Avatar
    Join Date
    Nov 2007
    Posts
    14,185
    Quote Originally Posted by アストラル View Post
    why would they add padding bytes to structs anyways?
    Often times memory is "chunked" into, say, four-byte units, and it's handy for a four-byte thing (like a long, say) to be entirely inside one chunk. So after a two-byte short, the rest of the chunk is unused, so that the next long gets its own chunk (and of course two consecutive two-byte shorts also fill one chunk).

  12. #12
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,893
    On some architectures, trying to read unaligned memory can even trigger a hardware exception.
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

  13. #13
    C++まいる!Cをこわせ! Elysia's Avatar
    Join Date
    Oct 2007
    Posts
    22,606
    Even on the case of x86 and x86-64, it can cause processor penalties (speed hits!) to read unaligned memory...
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  14. #14
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Quote Originally Posted by tabstop View Post
    Often times memory is "chunked" into, say, four-byte units, and it's handy for a four-byte thing (like a long, say) to be entirely inside one chunk. So after a two-byte short, the rest of the chunk is unused, so that the next long gets its own chunk (and of course two consecutive two-byte shorts also fill one chunk).
    Your statement seems to imply that "anything" has to be aligned to 4 bytes. And it has to do with alignment, but it's a bit more complex than that:
    Unaligned memory is where the memory address in bytes is not an even multiple of the size of the type, e.g. 4 bytes for int read from an address that isn't a multiple of 4. A double is usually aligned to 8 bytes (which is also it's usual size).

    Aside from CornedBee's correct statement that on some machines, reading unaligned memory [1] can cause crashes. Even if it doesn't cause a crash, it is slower to read memory from an unaligned address (because the processor has to collect data from two separate reads and put it together into one item). So if you have something like this:
    Code:
    struct X
    {
      int x;
      short y;
      int z;
    };
    then the compiler will pad between the short y and int z so that z is aligned to a multiple of 4 bytes. Most compilers have a way to tell the compiler "don't do that", although sometimes that could cause a crash if the processor doesn't support such things.

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  15. #15
    Registered User C_ntua's Avatar
    Join Date
    Jun 2008
    Posts
    1,853
    To see if I understand this correctly. Say you have:
    Code:
    struct s {
       int x;
       int y;
       short z;
       double d;
    };
    Then the compiler will want everything to be a multiplication of 4 bytes? Thus pad the memory with 2 bytes after the short? If it was like:
    Code:
    struct s {
       double a;
       double b;
       double c;
       int x;
       double z;
    }
    Then it would want everything to be a multiplication of 4 bytes or 8 bytes? Thus padding the 4 bytes after x or not?

    Can I assume that it is faster by padding the bytes so it can think of the struct in a more "array" way, so knowing the address of the pointer of the struct, find more quickly the values of each member of the struct?

Page 1 of 2 12 LastLast
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. reading files into memory
    By bobthebullet990 in forum C Programming
    Replies: 3
    Last Post: 11-30-2005, 02:39 PM
  2. CreateProcess and Memory mapped files
    By kevcri in forum C++ Programming
    Replies: 14
    Last Post: 12-10-2003, 02:14 AM
  3. Is it necessary to write a specific memory manager ?
    By Morglum in forum Game Programming
    Replies: 18
    Last Post: 07-01-2002, 01:41 PM
  4. Loading a file into memory ???
    By Unregistered in forum C++ Programming
    Replies: 2
    Last Post: 01-09-2002, 06:35 AM
  5. Loading all files in a directory
    By steve_i83 in forum Windows Programming
    Replies: 2
    Last Post: 08-26-2001, 02:18 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21