Thread: C++ using iconv

  1. #1
    Registered User Chris87's Avatar
    Join Date
    Dec 2007
    Posts
    131

    C++ using iconv

    I'm trying to parse a binary file where some of its strings are in CP437 encoding. I figure iconv would be suitable because it's cross platform and CP437 to UTF-8 isn't exactly widening characters, but rather decoding them. I haven't found a suitable solution on Google search anywhere, so I winged it. Sure enough, there were problems. The output got truncated to about 2 bytes. What am I doing wrong in this?
    Code:
    std::string ReadStr(IOStream *ios)
        {
          static iconv_t cnv = iconv_open("UTF-8", "CP437"); // might as well last the whole execution of the parser program
          std::vector<char> s, s2; // not sure if a reserved std::string would work here or not
          
          while (true) // strings in this format are null terminated, which by then we'll return a value
          {
            char c = 0;
            
            ios->Read(&c, 1, 1); // buffer, size, count
            if (c == 0)
            {
              auto insz = s.size();
              auto *inp = &s[0];
              auto outsz = insz * 2; // in case there is any extra data in the output
              auto *outp = &s2[0];
              
              s2.reserve(outsz);
              iconv(cnv, &inp, &insz, &outp, &outsz);
              s2.shrink_to_fit(); // don't need extra memory anymore
              return std::string(s2.begin(), s2.end()); // again, not sure if vector is needed here?
            }
            else
              s.push_back(c);
          }
        }
    Last edited by Chris87; 01-17-2018 at 02:53 PM.

  2. #2
    Registered User
    Join Date
    Dec 2017
    Posts
    870
    Do you maybe just need to "resize" s2 instead of "reserve" it?
    The world hangs on a thin thread, and that is the psyche of man. - Carl Jung

  3. #3
    Registered User Chris87's Avatar
    Join Date
    Dec 2007
    Posts
    131
    Well, I initially used a string instead of a vector, and I'm not sure if string's size() counts possible null characters interlaced.. Hmm... I guess vector might work, my bad.

  4. #4
    Registered User
    Join Date
    Apr 2006
    Posts
    2,149
    Firstly, you definitely need resize, not reserve. Resize will create an vector with that many elements, default initialized each element to '\0'. Reserve allocates memory, but you can't write to that memory through a pointer as you are doing.

    Secondly, you side to resize s2 before you set outp, since resize and reserve may both change where the vector's buffer is located, and because at size 0, you can't actually call s2[0].

    Third, shrink_to_fit still won't do what you want. Because you're writing through a pointer, s2 doesn't know how many characters are written. You need to use the values of outp and outsz to figure out the number of bytes written, then resize to trim the string.

    And yes, s2 can be a string.

    One more thing: you really should use longer variable names. It makes the code easier to read.
    It is too clear and so it is hard to see.
    A dunce once searched for fire with a lighted lantern.
    Had he known what fire was,
    He could have cooked his rice much sooner.

  5. #5
    Registered User Chris87's Avatar
    Join Date
    Dec 2007
    Posts
    131
    Sorry. I guess I just cringe over verbosity

  6. #6
    Registered User
    Join Date
    Apr 2006
    Posts
    2,149
    Quote Originally Posted by Chris87 View Post
    Sorry. I guess I just cringe over verbosity
    Code is written once, and read many times. Easy to understand variable names make code all the more easy to read. you don't want to be guessing what an abbreviation stands for when you look back at code.

    Verbosity is bad when it's boilerplate. When it actually says something, it's a good thing.
    It is too clear and so it is hard to see.
    A dunce once searched for fire with a lighted lantern.
    Had he known what fire was,
    He could have cooked his rice much sooner.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. iconv.dll
    By d1987 in forum Windows Programming
    Replies: 0
    Last Post: 08-16-2010, 09:28 AM

Tags for this Thread