If you can supply me a test '.gz' data file you are using, I can send back what the correct codeword tables should be for that file.
If you can supply me a test '.gz' data file you are using, I can send back what the correct codeword tables should be for that file.
Read your previous post, all stuff I already knew, problem is getting those repeat counts of the "literal+repeat" mentioned, because I know the filter for the "pngsuite/f00n0g08.png" works correctly I've been testing my zlib code against that. The "aba..." example provided in the link below comes out correctly but as soon as I try to use the same code for the bigger example in the greyscale image I end up with an offset error during the final loop, this can only mean I've either not set something correctly (I can only think of the repeat codes being the issue here) or I'm not reading bits I should be reading. Either way having the list for that f00n0g08.png image would be much appreciated, I forgot, is there an app that will list them for me or was that a part of your code?
Understanding gzip
With the help of the github document I previously link a few posts ago I finally understood where the repeat counts and bit retrieval counts came from, still doing something wrong though as still getting an offset error
I now know exactly where the repeat counts come from. (6b6866bd) * Commits * Lee Shallis / glEngine * GitLab
Found another document, I'll add the link here before I loose it, still struggling with the decompression
Deflate | File Formats Wiki | Fandom
Found another useful document, one of the links to source code helped me see I was reading the extra bits in the wrong order at the later half of the decompression, fixed that, still having issues, since I couldn't figure out the problem yet I decided to improve memory efficiency and speed by simply putting all the potentially required symbols into a single buffer in the ZLIB object then modified the ZLIB_SYMBOLS object to use an offseted address of that buffer instead of reallocating on the fly, by doing so I reduced the complexity of the setup function enough that it's possible for me to return nothing from it (chose to return an EINVAL on a NULL Buffers pointer though), removed the need for EmptyZlibObject() along with ExpandZlibSymbolsBuffer(), I also went ahead and moved the necessary code so that I could remove the get & cpy members of the ZLIB_SYMBOL object, also removed a couple of other unused members (with exception of the nxt member, that's planned to be used for compression). Now however I seem to be having an issue with the stream not stopping when it runs out of data to read, will need to fix that before I go back to focusing on the reason I'm getting corrupt data from the decompression attempt (yes I finally get to the PNG filters but the data it's receiving is incorrect)
Edit: Forgot to add the link:
Spent some time improving the memory efficiency and removing the need to (200a2d47) * Commits * Lee Shallis / glEngine * GitLab
Edit 2: And the other one too:
Deflate - Wikiwand
As requested
Note that "x:y" is value x, with repeat count y.
Image is properly decoded.
Code:Last block = true Compressed (dynamic dictionary) HLIT 273, HDIST 19, HCLEN 18 7 2 0 3 2 2 0 5 0 5 0 5 0 7 0 7 0 7 Codelen table 0: 3 110 1: 7 1111100 2: 7 1111101 3: 7 1111110 4: 5 11100 5: 5 11101 6: 5 11110 7: 2 00 8: 2 01 16: 7 1111111 17: 2 10 Reading in literal/length plus distance dictionary lengths 2 8 7 7 8 0 7 0 7 0 8 0 0 7 0 8 17:3 8 0 0 8 0 0 7 17:3 7 17:3 7 17:4 8 17:3 7 17:4 8 17:4 7 17:5 6 17:4 6 17:5 6 17:6 8 17:5 7 17:6 8 17:6 8 17:6 7 17:7 7 17:6 7 17:8 8 17:7 7 17:6 7 17:7 7 17:6 8 17:6 7 17:6 8 17:6 8 17:5 7 17:5 7 17:5 7 17:4 8 17:4 8 17:4 8 17:4 8 17:3 8 17:4 8 0 0 8 17:3 8 0 0 8 0 0 7 0 0 8 0 0 8 0 8 0 7 0 7 7 8 7 6 8 8 5 16:3 8 0 0 4 4 5 8 6 17:3 4 5 17:8 5 1 0 5 4 3 5 3 5 5 Literal/Repeat Length table 0: 2 00 1: 8 11100000 2: 7 1010110 3: 7 1010111 4: 8 11100001 6: 7 1011000 8: 7 1011001 10: 8 11100010 13: 7 1011010 15: 8 11100011 19: 8 11100100 22: 8 11100101 25: 7 1011011 29: 7 1011100 33: 7 1011101 38: 8 11100110 42: 7 1011110 47: 8 11100111 52: 7 1011111 58: 6 100110 63: 6 100111 69: 6 101000 76: 8 11101000 82: 7 1100000 89: 8 11101001 96: 8 11101010 103: 7 1100001 111: 7 1100010 118: 7 1100011 127: 8 11101011 135: 7 1100100 142: 7 1100101 150: 7 1100110 157: 8 11101100 164: 7 1100111 171: 8 11101101 178: 8 11101110 184: 7 1101000 190: 7 1101001 196: 7 1101010 201: 8 11101111 206: 8 11110000 211: 8 11110001 216: 8 11110010 220: 8 11110011 225: 8 11110100 228: 8 11110101 232: 8 11110110 235: 8 11110111 238: 7 1101011 241: 8 11111000 244: 8 11111001 246: 8 11111010 248: 7 1101100 250: 7 1101101 251: 7 1101110 252: 8 11111011 253: 7 1101111 254: 6 101001 255: 8 11111100 256: 8 11111101 257: 5 01110 258: 5 01111 259: 5 10000 260: 5 10001 261: 8 11111110 264: 4 0100 265: 4 0101 266: 5 10010 267: 8 11111111 268: 6 101010 272: 4 0110 Distance table 0: 5 11010 9: 5 11011 10: 1 0 12: 5 11100 13: 4 1100 14: 3 100 15: 5 11101 16: 3 101 17: 5 11110 18: 5 11111
Still struggling with loading the values, should be easier for outsiders to understand what's being loaded now though:
Made loading of implied values a bit easier to understand (884436ed) * Commits * Lee Shallis / glEngine * GitLab
Edit: This is the sorta output I'm getting at the moment:Code:#define ZLIB_COPY_LIT0 257 #define ZLIB_LOOP_LIT0 288 ZLIB_IMPLIED implied_type_data = { /* get extra bits */ { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 3, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 }, /* base cpy value */ { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 } }; ZLIB_IMPLIED implied_code_data = { /* get extra bits */ { 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 5, 5, 5, 5, 0, 0, 0, 0 }, /* base cpy value */ { 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 15, 17, 19, 23, 27, 31, 35, 43, 51, 59, 67, 83, 99, 115, 131, 163, 195, 227, 258, 0, 0, 0, } }; ZLIB_IMPLIED implied_loop_data = { /* get extra bits */ { 0, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 10, 11, 11, 12, 12, 13, 13, 0, 0 }, /* base cpy value */ { 1, 2, 3, 4, 5, 7, 9, 13, 17, 25, 33, 49, 65, 97, 129, 193, 257, 385, 513, 769, 1025, 1537, 2049, 3073, 4097, 6145, 8193, 12289, 16385, 24577, 0, 0 } }; typedef struct _ZLIB_FULL_SYMBOL { bool _tt, use; long src, lit; long get, cpy; long len, uid; long nxt[2]; } ZLIB_FULL_SYMBOL; void ZlibFullSymbol( ZLIB_FULL_SYMBOL *full, ZLIB_SYMBOL *part ) { int get = 0, cpy = 0, i = part->lit; if ( part->_tt ) { get = implied_type_data.get[i]; cpy = implied_type_data.cpy[i]; } else if ( i >= ZLIB_LOOP_LIT0 ) { i -= ZLIB_LOOP_LIT0; get = implied_loop_data.get[i]; cpy = implied_loop_data.cpy[i]; } else if ( i >= ZLIB_COPY_LIT0 && i < ZLIB_COPY_LIT0 + 32 ) { i -= ZLIB_COPY_LIT0; get = implied_code_data.get[i]; cpy = implied_code_data.cpy[i]; } memset( full, 0, sizeof(ZLIB_FULL_SYMBOL) ); full->_tt = part->_tt; full->use = part->use; full->src = part->src; full->lit = part->lit; full->len = part->len; full->uid = part->uid; full->get = get; full->cpy = cpy; }
I can see the current problem is corruption of get/cpy counts, not sure about UIDs yet, gonna compare after this editCode:make run ARGS="--detailed -f ./pngsuite/f00n0g08.png" ... ./a.out --detailed -f ./pngsuite/f00n0g08.png main.c:511: ProcessArgv( 4, 0x7fffa728c398 ) main.c:433: ArgHasIssue( -1, 1, 0x7fffa728c398 ) arg = './a.out', val = '(null)' main.c:433: ArgHasIssue( 1, 0, 0x7fffa728c398 ) arg = '--detailed', val = '(null)' main.c:433: ArgHasIssue( 2, 1, 0x7fffa728c398 ) arg = '-f', val = './pngsuite/f00n0g08.png' Attempting to open ./pngsuite/f00n0g08.png view/png.c:421: chunks[ 0] 13 56112528 'IHDR' view/png.c:421: chunks[ 1] 262 50A6CEB6 'IDAT' view/png.c:387: chunks[ 2] 0 00000000 'IEND' buff/zlib.c:1134: Zlib: method = 8, info = 7, type = 0, dictionary = false, check = 13 buff/zlib.c:1081: last = 1, type = 10, Stream->used = 19 buff/zlib.c:826: Deflation Type 2: buff/zlib.c:829: Code Symbols List Details: foresee: 273, longest = 0, highest = 0, have = 65536, used = 0 buff/zlib.c:832: Loop Symbols List Details: foresee: 19, longest = 0, highest = 0, have = 131072, used = 0 buff/zlib.c:835: Type Symbols List Details: foresee: 18, longest = 0, highest = 0, have = 256, used = 0 buff/zlib.c:856: Type Symbols List Details: foresee: 18, longest = 7, highest = 127, have = 256, used = 128 type_symbols[ 0]: _tt = true, src = 5, lit = 7, use = true, get = 0, cpy = 0, len = 2, uid = 00 type_symbols[ 1]: _tt = true, src = 4, lit = 8, use = true, get = 0, cpy = 0, len = 2, uid = 01 type_symbols[ 2]: _tt = true, src = 1, lit = 17, use = true, get = 3, cpy = 3, len = 2, uid = 10 type_symbols[ 6]: _tt = true, src = 3, lit = 0, use = true, get = 0, cpy = 0, len = 3, uid = 110 type_symbols[ 28]: _tt = true, src = 11, lit = 4, use = true, get = 0, cpy = 0, len = 5, uid = 11100 type_symbols[ 29]: _tt = true, src = 9, lit = 5, use = true, get = 0, cpy = 0, len = 5, uid = 11101 type_symbols[ 30]: _tt = true, src = 7, lit = 6, use = true, get = 0, cpy = 0, len = 5, uid = 11110 type_symbols[ 124]: _tt = true, src = 17, lit = 1, use = true, get = 0, cpy = 0, len = 7, uid = 1111100 type_symbols[ 125]: _tt = true, src = 15, lit = 2, use = true, get = 0, cpy = 0, len = 7, uid = 1111101 type_symbols[ 126]: _tt = true, src = 13, lit = 3, use = true, get = 0, cpy = 0, len = 7, uid = 1111110 type_symbols[ 127]: _tt = true, src = 0, lit = 16, use = true, get = 2, cpy = 3, len = 7, uid = 1111111 ... Code Symbols List Details: foresee: 273, longest = 9, highest = 1112, have = 65536, used = 512 code_symbols[ 0]: _tt = false, src = 138, lit = 283, use = true, get = 5, cpy = 195, len = 1, uid = 0 code_symbols[ 1]: _tt = false, src = 190, lit = 386, use = true, get = 1405354608, cpy = 0, len = 1, uid = 1 code_symbols[ 4]: _tt = false, src = 1, lit = 0, use = true, get = 0, cpy = 0, len = 2, uid = 00 code_symbols[ 5]: _tt = false, src = 186, lit = 382, use = true, get = 1405354448, cpy = 0, len = 2, uid = 01 code_symbols[ 6]: _tt = false, src = 198, lit = 404, use = true, get = 0, cpy = 0, len = 2, uid = 10 code_symbols[ 14]: _tt = false, src = 142, lit = 287, use = true, get = 0, cpy = 0, len = 3, uid = 110 code_symbols[ 15]: _tt = false, src = 144, lit = 289, use = true, get = 0, cpy = 2, len = 3, uid = 111 code_symbols[ 16]: _tt = false, src = 159, lit = 329, use = true, get = 25, cpy = 5, len = 3, uid = 000 code_symbols[ 17]: _tt = false, src = 179, lit = 358, use = true, get = 9, cpy = 0, len = 3, uid = 001 code_symbols[ 18]: _tt = false, src = 188, lit = 384, use = true, get = 1405354528, cpy = 0, len = 3, uid = 010 code_symbols[ 19]: _tt = false, src = 191, lit = 387, use = true, get = 22090, cpy = 0, len = 3, uid = 011 code_symbols[ 40]: _tt = false, src = 128, lit = 264, use = true, get = 0, cpy = 10, len = 4, uid = 1000 code_symbols[ 41]: _tt = false, src = 129, lit = 265, use = true, get = 1, cpy = 11, len = 4, uid = 1001 code_symbols[ 42]: _tt = false, src = 134, lit = 272, use = true, get = 2, cpy = 31, len = 4, uid = 1010 code_symbols[ 43]: _tt = false, src = 141, lit = 286, use = true, get = 0, cpy = 0, len = 4, uid = 1011 code_symbols[ 44]: _tt = false, src = 150, lit = 295, use = true, get = 2, cpy = 13, len = 4, uid = 1100 code_symbols[ 45]: _tt = false, src = 173, lit = 352, use = true, get = 16, cpy = 1405354528, len = 4, uid = 1101 code_symbols[ 46]: _tt = false, src = 178, lit = 357, use = true, get = 7, cpy = 22090, len = 4, uid = 1110 code_symbols[ 94]: _tt = false, src = 120, lit = 257, use = true, get = 0, cpy = 3, len = 5, uid = 11110 code_symbols[ 95]: _tt = false, src = 258, lit = 258, use = true, get = 0, cpy = 4, len = 5, uid = 11111 code_symbols[ 96]: _tt = false, src = 259, lit = 259, use = true, get = 0, cpy = 5, len = 5, uid = 00000 code_symbols[ 97]: _tt = false, src = 260, lit = 260, use = true, get = 0, cpy = 6, len = 5, uid = 00001 code_symbols[ 98]: _tt = false, src = 130, lit = 266, use = true, get = 1, cpy = 13, len = 5, uid = 00010 code_symbols[ 99]: _tt = false, src = 135, lit = 273, use = true, get = 3, cpy = 35, len = 5, uid = 00011 code_symbols[ 100]: _tt = false, src = 137, lit = 282, use = true, get = 5, cpy = 163, len = 5, uid = 00100 code_symbols[ 101]: _tt = false, src = 140, lit = 285, use = true, get = 0, cpy = 258, len = 5, uid = 00101 code_symbols[ 102]: _tt = false, src = 143, lit = 288, use = true, get = 0, cpy = 1, len = 5, uid = 00110 code_symbols[ 103]: _tt = false, src = 145, lit = 290, use = true, get = 0, cpy = 3, len = 5, uid = 00111 code_symbols[ 104]: _tt = false, src = 146, lit = 291, use = true, get = 0, cpy = 4, len = 5, uid = 01000 code_symbols[ 105]: _tt = false, src = 148, lit = 293, use = true, get = 1, cpy = 7, len = 5, uid = 01001 code_symbols[ 106]: _tt = false, src = 156, lit = 318, use = true, get = 0, cpy = 0, len = 5, uid = 01010 code_symbols[ 107]: _tt = false, src = 162, lit = 332, use = true, get = 65, cpy = 12, len = 5, uid = 01011 code_symbols[ 108]: _tt = false, src = 166, lit = 342, use = true, get = 2049, cpy = 1405336688, len = 5, uid = 01100 code_symbols[ 109]: _tt = false, src = 196, lit = 402, use = true, get = 0, cpy = 0, len = 5, uid = 01101 code_symbols[ 110]: _tt = false, src = 199, lit = 405, use = true, get = 0, cpy = 0, len = 5, uid = 01110 code_symbols[ 222]: _tt = false, src = 38, lit = 58, use = true, get = 0, cpy = 0, len = 6, uid = 011110 code_symbols[ 223]: _tt = false, src = 40, lit = 63, use = true, get = 0, cpy = 0, len = 6, uid = 011111 code_symbols[ 224]: _tt = false, src = 42, lit = 69, use = true, get = 0, cpy = 0, len = 6, uid = 100000 code_symbols[ 225]: _tt = false, src = 117, lit = 254, use = true, get = 0, cpy = 0, len = 6, uid = 100001 code_symbols[ 226]: _tt = false, src = 132, lit = 268, use = true, get = 1, cpy = 17, len = 6, uid = 100010 code_symbols[ 227]: _tt = false, src = 161, lit = 331, use = true, get = 49, cpy = 4, len = 6, uid = 100011 code_symbols[ 228]: _tt = false, src = 175, lit = 354, use = true, get = 18, cpy = 1405354608, len = 6, uid = 100100 code_symbols[ 229]: _tt = false, src = 185, lit = 381, use = true, get = 22090, cpy = 0, len = 6, uid = 100101 code_symbols[ 230]: _tt = false, src = 200, lit = 406, use = true, get = 0, cpy = 0, len = 6, uid = 100110 code_symbols[ 462]: _tt = false, src = 3, lit = 2, use = true, get = 0, cpy = 0, len = 7, uid = 1001110 code_symbols[ 463]: _tt = false, src = 4, lit = 3, use = true, get = 0, cpy = 0, len = 7, uid = 1001111 code_symbols[ 464]: _tt = false, src = 7, lit = 6, use = true, get = 0, cpy = 0, len = 7, uid = 1010000 code_symbols[ 465]: _tt = false, src = 9, lit = 8, use = true, get = 0, cpy = 0, len = 7, uid = 1010001 code_symbols[ 466]: _tt = false, src = 14, lit = 13, use = true, get = 0, cpy = 0, len = 7, uid = 1010010 code_symbols[ 467]: _tt = false, src = 24, lit = 25, use = true, get = 0, cpy = 0, len = 7, uid = 1010011 code_symbols[ 468]: _tt = false, src = 26, lit = 29, use = true, get = 0, cpy = 0, len = 7, uid = 1010100 code_symbols[ 469]: _tt = false, src = 28, lit = 33, use = true, get = 0, cpy = 0, len = 7, uid = 1010101 code_symbols[ 470]: _tt = false, src = 32, lit = 42, use = true, get = 0, cpy = 0, len = 7, uid = 1010110 code_symbols[ 471]: _tt = false, src = 36, lit = 52, use = true, get = 0, cpy = 0, len = 7, uid = 1010111 code_symbols[ 472]: _tt = false, src = 46, lit = 82, use = true, get = 0, cpy = 0, len = 7, uid = 1011000 code_symbols[ 473]: _tt = false, src = 52, lit = 103, use = true, get = 0, cpy = 0, len = 7, uid = 1011001 code_symbols[ 474]: _tt = false, src = 54, lit = 111, use = true, get = 0, cpy = 0, len = 7, uid = 1011010 code_symbols[ 475]: _tt = false, src = 56, lit = 118, use = true, get = 0, cpy = 0, len = 7, uid = 1011011 code_symbols[ 476]: _tt = false, src = 60, lit = 135, use = true, get = 0, cpy = 0, len = 7, uid = 1011100 code_symbols[ 477]: _tt = false, src = 62, lit = 142, use = true, get = 0, cpy = 0, len = 7, uid = 1011101 code_symbols[ 478]: _tt = false, src = 64, lit = 150, use = true, get = 0, cpy = 0, len = 7, uid = 1011110 code_symbols[ 479]: _tt = false, src = 68, lit = 164, use = true, get = 0, cpy = 0, len = 7, uid = 1011111 code_symbols[ 480]: _tt = false, src = 74, lit = 184, use = true, get = 0, cpy = 0, len = 7, uid = 1100000 code_symbols[ 481]: _tt = false, src = 76, lit = 190, use = true, get = 0, cpy = 0, len = 7, uid = 1100001 code_symbols[ 482]: _tt = false, src = 78, lit = 196, use = true, get = 0, cpy = 0, len = 7, uid = 1100010 code_symbols[ 483]: _tt = false, src = 101, lit = 238, use = true, get = 0, cpy = 0, len = 7, uid = 1100011 code_symbols[ 484]: _tt = false, src = 111, lit = 248, use = true, get = 0, cpy = 0, len = 7, uid = 1100100 code_symbols[ 485]: _tt = false, src = 113, lit = 250, use = true, get = 0, cpy = 0, len = 7, uid = 1100101 code_symbols[ 486]: _tt = false, src = 114, lit = 251, use = true, get = 0, cpy = 0, len = 7, uid = 1100110 code_symbols[ 487]: _tt = false, src = 116, lit = 253, use = true, get = 0, cpy = 0, len = 7, uid = 1100111 code_symbols[ 488]: _tt = false, src = 147, lit = 292, use = true, get = 1, cpy = 5, len = 7, uid = 1101000 code_symbols[ 489]: _tt = false, src = 152, lit = 305, use = true, get = 7, cpy = 385, len = 7, uid = 1101001 code_symbols[ 490]: _tt = false, src = 154, lit = 316, use = true, get = 13, cpy = 16385, len = 7, uid = 1101010 code_symbols[ 491]: _tt = false, src = 158, lit = 328, use = true, get = 17, cpy = 10, len = 7, uid = 1101011 code_symbols[ 492]: _tt = false, src = 171, lit = 350, use = true, get = 0, cpy = 1405354448, len = 7, uid = 1101100 code_symbols[ 493]: _tt = false, src = 189, lit = 385, use = true, get = 22090, cpy = 0, len = 7, uid = 1101101 code_symbols[ 494]: _tt = false, src = 203, lit = 417, use = true, get = 0, cpy = 0, len = 7, uid = 1101110 code_symbols[ 495]: _tt = false, src = 418, lit = 418, use = true, get = 0, cpy = 0, len = 7, uid = 1101111 code_symbols[ 496]: _tt = false, src = 419, lit = 419, use = true, get = 0, cpy = 0, len = 7, uid = 1110000 code_symbols[ 497]: _tt = false, src = 420, lit = 420, use = true, get = 0, cpy = 0, len = 7, uid = 1110001 code_symbols[ 498]: _tt = false, src = 208, lit = 421, use = true, get = 0, cpy = 0, len = 7, uid = 1110010 code_symbols[ 499]: _tt = false, src = 210, lit = 423, use = true, get = 0, cpy = 0, len = 7, uid = 1110011 code_symbols[ 500]: _tt = false, src = 213, lit = 426, use = true, get = 0, cpy = 0, len = 7, uid = 1110100 code_symbols[ 501]: _tt = false, src = 215, lit = 428, use = true, get = 0, cpy = 0, len = 7, uid = 1110101 code_symbols[ 502]: _tt = false, src = 216, lit = 429, use = true, get = 0, cpy = 0, len = 7, uid = 1110110 code_symbols[ 503]: _tt = false, src = 219, lit = 438, use = true, get = 0, cpy = 0, len = 7, uid = 1110111 code_symbols[ 504]: _tt = false, src = 221, lit = 440, use = true, get = 0, cpy = 0, len = 7, uid = 1111000 code_symbols[ 505]: _tt = false, src = 223, lit = 442, use = true, get = 0, cpy = 0, len = 7, uid = 1111001 code_symbols[ 506]: _tt = false, src = 224, lit = 443, use = true, get = 0, cpy = 0, len = 7, uid = 1111010 code_symbols[ 507]: _tt = false, src = 228, lit = 454, use = true, get = 0, cpy = 0, len = 7, uid = 1111011 code_symbols[ 508]: _tt = false, src = 230, lit = 456, use = true, get = 0, cpy = 0, len = 7, uid = 1111100 code_symbols[ 509]: _tt = false, src = 231, lit = 457, use = true, get = 0, cpy = 0, len = 7, uid = 1111101 code_symbols[ 510]: _tt = false, src = 236, lit = 465, use = true, get = 0, cpy = 0, len = 7, uid = 1111110 code_symbols[ 511]: _tt = false, src = 238, lit = 467, use = true, get = 0, cpy = 0, len = 7, uid = 1111111 Loop Symbols List Details: foresee: 19, longest = 9, highest = 285, have = 131072, used = 512 loop_symbols[ 0]: _tt = false, src = 0, lit = 288, use = true, get = 0, cpy = 1, len = 1, uid = 0 loop_symbols[ 32]: _tt = false, src = 7, lit = 304, use = true, get = 7, cpy = 257, len = 6, uid = 100000 loop_symbols[ 132]: _tt = false, src = 3, lit = 300, use = true, get = 5, cpy = 65, len = 8, uid = 10000100 loop_symbols[ 133]: _tt = false, src = 4, lit = 301, use = true, get = 5, cpy = 97, len = 8, uid = 10000101 loop_symbols[ 134]: _tt = false, src = 6, lit = 303, use = true, get = 6, cpy = 193, len = 8, uid = 10000110 loop_symbols[ 135]: _tt = false, src = 9, lit = 306, use = true, get = 8, cpy = 513, len = 8, uid = 10000111 loop_symbols[ 136]: _tt = false, src = 11, lit = 310, use = true, get = 10, cpy = 2049, len = 8, uid = 10001000 loop_symbols[ 137]: _tt = false, src = 17, lit = 316, use = true, get = 13, cpy = 16385, len = 8, uid = 10001001 loop_symbols[ 138]: _tt = false, src = 18, lit = 317, use = true, get = 13, cpy = 24577, len = 8, uid = 10001010 loop_symbols[ 278]: _tt = false, src = 5, lit = 302, use = true, get = 6, cpy = 129, len = 9, uid = 100010110 loop_symbols[ 279]: _tt = false, src = 8, lit = 305, use = true, get = 7, cpy = 385, len = 9, uid = 100010111 loop_symbols[ 280]: _tt = false, src = 12, lit = 311, use = true, get = 10, cpy = 3073, len = 9, uid = 100011000 loop_symbols[ 281]: _tt = false, src = 13, lit = 312, use = true, get = 11, cpy = 4097, len = 9, uid = 100011001 loop_symbols[ 282]: _tt = false, src = 14, lit = 313, use = true, get = 11, cpy = 6145, len = 9, uid = 100011010 loop_symbols[ 283]: _tt = false, src = 15, lit = 314, use = true, get = 12, cpy = 8193, len = 9, uid = 100011011 loop_symbols[ 284]: _tt = false, src = 16, lit = 315, use = true, get = 12, cpy = 12289, len = 9, uid = 100011100 loop_symbols[ 285]: _tt = false, src = 19, lit = 318, use = true, get = 0, cpy = 0, len = 9, uid = 100011101 ...
Last edited by awsdert; 08-27-2021 at 09:09 AM.
Having compared the UIDs of the code values vs what hamster_nz provided I can see that the fullType variable in LoadZlibHuffs() is not receiving the correct values somehow, in case someone has an idea from seeing the code here's the two functions:
Code:int LoadZlibTypes( ZLIB *zlib ) { ZLIB_SYMBOLS_ID id = ZLIB_SYMBOLS_ID_TYPE; STREAM *Stream = zlib->Stream; ZLIB_SYMBOLS *Types = &(zlib->Symbols[id]); int i = Types->foresee; ZLIB_SYMBOL *types = Types->symbols; for ( i = 0; i < Types->foresee && Stream->err == 0; ++i ) { static uint adjust[] = { 16, 17, 18, 0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15, 19 }; uint pos = (i < 20) ? adjust[i] : 31; ZLIB_SYMBOL *Type = types + pos; Type->_tt = true; Type->src = i; Type->lit = pos; Type->len = StreamBits( Stream, 3, true ); Type->use = !!(Type->len); if ( Type->len > Types->longest ) Types->longest = Type->len; } Types->used = 19; InitZlibSymbolUIDs( Types ); SortZlibSymbolsByUID( zlib, id ); return 0; }Code:int LoadZlibHuffs( ZLIB *zlib ) { ZLIB_SYMBOLS *Types = &(zlib->Symbols[ZLIB_SYMBOLS_ID_TYPE]); ZLIB_SYMBOLS *Codes = &(zlib->Symbols[ZLIB_SYMBOLS_ID_CODE]); ZLIB_SYMBOLS *Loops = &(zlib->Symbols[ZLIB_SYMBOLS_ID_LOOP]); ZLIB_SYMBOL *codes = Codes->symbols, *Code; ZLIB_FULL_SYMBOL fullType = {0}; STREAM *Stream = zlib->Stream; FILE *errors = zlib->Buffers->Alloc->errors; FILE *verbose = zlib->Buffers->Alloc->verbose; FILE *detailed = zlib->Buffers->Alloc->detailed; int done = 0, pos = ZLIB_MAX_TYPES, lit = 0, prv = 0, keep = Codes->foresee, count = keep + Loops->foresee; if ( !quiet_zlib ) { ECHO ( verbose, fprintf ( verbose, "LoadZlibHuffs( %p, %d )\n", (void*)zlib, count ) ); } while ( done < count && Stream->err == 0 ) { bool use; int i = 0, copy = 0, len = prv; ZLIB_SYMBOL *Type = SeekZlibSymbol( Stream, Types ); if ( !Type ) { ECHO ( errors, fputs( "Couldn't find code!\n", errors ); EchoZlibSymbolsListDetails( errors, zlib, ZLIB_SYMBOLS_ID_TYPE ); ECHO_ERR( errors, EINVAL ) ); return EINVAL; } ZlibFullSymbol( &fullType, Type ); ++done; if ( !quiet_zlib ) { ECHO ( detailed, EchoStreamDetails( detailed, Stream ); fprintf( detailed,"type_" ); EchoZlibSymbolDetails( detailed, Type, Type->src ) ); } if ( done == keep ) { lit = ZLIB_LOOP_LIT0; pos = ZLIB_MAX_TYPES + ZLIB_MAX_CODES; } if ( Type->lit >= 16 ) { copy = StreamBits( Stream, fullType.get, true ); copy += fullType.cpy; if ( Type->lit > 16 || !len ) { len = codes->len; if ( !len ) { lit += copy; pos += copy; copy = 0; len = -1; } } } else { Code = codes + pos; Code->lit = lit; if ( done < keep ) { Code->src = done; Code->len = Type->lit; Code->use = !!(Type->lit); } else { Code->use = true; Code->src = done - keep; Code->len = Type->lit + 1; } if ( !quiet_zlib && detailed && Code->len ) { fprintf(detailed,"%s_", (done >= keep) ? "loop" : "code" ); EchoZlibSymbolDetails( detailed, Code, Code->src ); } if ( Code->len > Codes->longest ) Codes->longest = Code->len; //if ( Type->lit || done >= keep ) prv = Code->len; ++lit; ++pos; } if ( copy ) { if ( lit < ZLIB_LOOP_LIT0 ) { if ( lit + copy > 286 ) { ECHO ( errors, ECHO( errors, EINVAL ); fprintf ( errors, "copy of a literal's length would have gone " "beyond the limit for literal/repeater codes (286), " "lit = %3u, copy = %3u\n", lit, copy ) ); return EINVAL; } } else if ( done + copy > count ) { ECHO ( errors, ECHO( errors, EINVAL ); fprintf ( errors, "copy of a loop code's length would have gone " "beyond the expected limit for loop codes (%u)\n", count - keep ) ); return EINVAL; } if ( !quiet_zlib ) { ECHO ( detailed, fprintf ( detailed, "Copying %d length %u times...\n", len, copy ) ); } } len += !(done < keep); use = !!len; for ( i = 0; i < copy; ++i, ++lit, ++done, ++pos ) { Code = codes + pos; Code->lit = lit; Code->use = use; Code->len = len; Code->src = (done < keep) ? lit : done - keep; } } Codes->used = ZLIB_MAX_CODES; Loops->used = ZLIB_MAX_LOOPS; Loops->longest = Codes->longest; InitZlibSymbolUIDs( Codes ); InitZlibSymbolUIDs( Loops ); SortZlibSymbolsByUID( zlib, ZLIB_SYMBOLS_ID_CODE ); SortZlibSymbolsByUID( zlib, ZLIB_SYMBOLS_ID_LOOP ); return 0; }
Looking at your output, the first table seems perfect.
The second one has literals that should not occur. 386? 404? 382?
This is what the codewords should be, sorted in order:Code:code_symbols[ 0]: _tt = false, src = 138, lit = 283, use = true, get = 5, cpy = 195, len = 1, uid = 0 code_symbols[ 1]: _tt = false, src = 190, lit = 386, use = true, get = 1405354608, cpy = 0, len = 1, uid = 1 code_symbols[ 4]: _tt = false, src = 1, lit = 0, use = true, get = 0, cpy = 0, len = 2, uid = 00 code_symbols[ 5]: _tt = false, src = 186, lit = 382, use = true, get = 1405354448, cpy = 0, len = 2, uid = 01 code_symbols[ 6]: _tt = false, src = 198, lit = 404, use = true, get = 0, cpy = 0, len = 2, uid = 10
Are you able to dump "codeword length table", before it is sorted?Code:Value Len Codeword 0: 2 00 264: 4 0100 265: 4 0101 272: 4 0110 257: 5 01110 258: 5 01111 259: 5 10000 260: 5 10001 266: 5 10010 58: 6 100110 63: 6 100111 69: 6 101000 254: 6 101001 268: 6 101010 2: 7 1010110 3: 7 1010111 6: 7 1011000 8: 7 1011001 13: 7 1011010 25: 7 1011011 29: 7 1011100 33: 7 1011101 42: 7 1011110 52: 7 1011111 82: 7 1100000 103: 7 1100001 111: 7 1100010 118: 7 1100011 135: 7 1100100 142: 7 1100101 150: 7 1100110 164: 7 1100111 184: 7 1101000 190: 7 1101001 196: 7 1101010 238: 7 1101011 248: 7 1101100 250: 7 1101101 251: 7 1101110 253: 7 1101111 1: 8 11100000 4: 8 11100001 10: 8 11100010 15: 8 11100011 19: 8 11100100 22: 8 11100101 38: 8 11100110 47: 8 11100111 76: 8 11101000 89: 8 11101001 96: 8 11101010 127: 8 11101011 157: 8 11101100 171: 8 11101101 178: 8 11101110 201: 8 11101111 206: 8 11110000 211: 8 11110001 216: 8 11110010 220: 8 11110011 225: 8 11110100 228: 8 11110101 232: 8 11110110 235: 8 11110111 241: 8 11111000 244: 8 11111001 246: 8 11111010 252: 8 11111011 255: 8 11111100 256: 8 11111101 261: 8 11111110 267: 8 11111111
Last edited by hamster_nz; 08-27-2021 at 02:40 PM.
I noticed that oddity too, wasn't until just now that I spotted why, I simply forgot to update the 'done' variable when adding the 'copy' variable to 'lit' & 'pos'
I've updated the gitlab project with the fixes, still got an offset issue with the data after the trees so I'm assuming I mis-read somethin or mis-assigned a variable in my vain attempts to resolve the "too many codes" issue, I need some sleep so I will just do a dump of my current output and add a link to the upload, if you feel like looking through it that's great, if not then no worries, wasn't expecting you too, I'll still have a look tomorrow.
Finally realised I failed to update the 'done' variable with 'lit' & (4c7b3adc) * Commits * Lee Shallis / glEngine * GitLab
Edit: Just compared the UIDs, it seems the code UIDs are fine but the loop UIDs are longer than they should be, I've left my editor focused on the assignment of the 'len' member in LoadZlibHuffs() ready for me to look at tomorrowCode:make run ARGS="--detailed -f ./pngsuite/f00n0g08.png" ... ./a.out --detailed -f ./pngsuite/f00n0g08.png main.c:511: ProcessArgv( 4, 0x7ffc58fca388 ) main.c:433: ArgHasIssue( -1, 1, 0x7ffc58fca388 ) arg = './a.out', val = '(null)' main.c:433: ArgHasIssue( 1, 0, 0x7ffc58fca388 ) arg = '--detailed', val = '(null)' main.c:433: ArgHasIssue( 2, 1, 0x7ffc58fca388 ) arg = '-f', val = './pngsuite/f00n0g08.png' Attempting to open ./pngsuite/f00n0g08.png view/png.c:421: chunks[ 0] 13 56112528 'IHDR' view/png.c:421: chunks[ 1] 262 50A6CEB6 'IDAT' view/png.c:387: chunks[ 2] 0 00000000 'IEND' buff/zlib.c:1134: Zlib: method = 8, info = 7, type = 0, dictionary = false, check = 13 buff/zlib.c:1081: last = 1, type = 10, Stream->used = 19 buff/zlib.c:826: Deflation Type 2: buff/zlib.c:829: Code Symbols List Details: foresee: 273, longest = 0, highest = 0, have = 65536, used = 0 buff/zlib.c:832: Loop Symbols List Details: foresee: 19, longest = 0, highest = 0, have = 131072, used = 0 buff/zlib.c:835: Type Symbols List Details: foresee: 18, longest = 0, highest = 0, have = 256, used = 0 buff/zlib.c:856: Type Symbols List Details: foresee: 18, longest = 7, highest = 127, have = 256, used = 128 type_symbols[ 0]: _tt = true, src = 5, lit = 7, use = true, get = 0, cpy = 0, len = 2, uid = 00 type_symbols[ 1]: _tt = true, src = 4, lit = 8, use = true, get = 0, cpy = 0, len = 2, uid = 01 type_symbols[ 2]: _tt = true, src = 1, lit = 17, use = true, get = 3, cpy = 3, len = 2, uid = 10 type_symbols[ 6]: _tt = true, src = 3, lit = 0, use = true, get = 0, cpy = 0, len = 3, uid = 110 type_symbols[ 28]: _tt = true, src = 11, lit = 4, use = true, get = 0, cpy = 0, len = 5, uid = 11100 type_symbols[ 29]: _tt = true, src = 9, lit = 5, use = true, get = 0, cpy = 0, len = 5, uid = 11101 type_symbols[ 30]: _tt = true, src = 7, lit = 6, use = true, get = 0, cpy = 0, len = 5, uid = 11110 type_symbols[ 124]: _tt = true, src = 17, lit = 1, use = true, get = 0, cpy = 0, len = 7, uid = 1111100 type_symbols[ 125]: _tt = true, src = 15, lit = 2, use = true, get = 0, cpy = 0, len = 7, uid = 1111101 type_symbols[ 126]: _tt = true, src = 13, lit = 3, use = true, get = 0, cpy = 0, len = 7, uid = 1111110 type_symbols[ 127]: _tt = true, src = 0, lit = 16, use = true, get = 2, cpy = 3, len = 7, uid = 1111111 buff/zlib.c:349: LoadZlibHuffs( 0x7ffc589c3da0, 292 ) ... Code Symbols List Details: foresee: 273, longest = 8, highest = 255, have = 65536, used = 256 code_symbols[ 0]: _tt = false, src = 0, lit = 0, use = true, get = 0, cpy = 0, len = 2, uid = 00 code_symbols[ 4]: _tt = false, src = 264, lit = 264, use = true, get = 0, cpy = 10, len = 4, uid = 0100 code_symbols[ 5]: _tt = false, src = 265, lit = 265, use = true, get = 1, cpy = 11, len = 4, uid = 0101 code_symbols[ 6]: _tt = false, src = 272, lit = 272, use = true, get = 2, cpy = 31, len = 4, uid = 0110 code_symbols[ 14]: _tt = false, src = 257, lit = 257, use = true, get = 0, cpy = 3, len = 5, uid = 01110 code_symbols[ 15]: _tt = false, src = 258, lit = 258, use = true, get = 0, cpy = 4, len = 5, uid = 01111 code_symbols[ 16]: _tt = false, src = 259, lit = 259, use = true, get = 0, cpy = 5, len = 5, uid = 10000 code_symbols[ 17]: _tt = false, src = 260, lit = 260, use = true, get = 0, cpy = 6, len = 5, uid = 10001 code_symbols[ 18]: _tt = false, src = 266, lit = 266, use = true, get = 1, cpy = 13, len = 5, uid = 10010 code_symbols[ 38]: _tt = false, src = 58, lit = 58, use = true, get = 0, cpy = 0, len = 6, uid = 100110 code_symbols[ 39]: _tt = false, src = 63, lit = 63, use = true, get = 0, cpy = 0, len = 6, uid = 100111 code_symbols[ 40]: _tt = false, src = 69, lit = 69, use = true, get = 0, cpy = 0, len = 6, uid = 101000 code_symbols[ 41]: _tt = false, src = 254, lit = 254, use = true, get = 0, cpy = 0, len = 6, uid = 101001 code_symbols[ 42]: _tt = false, src = 268, lit = 268, use = true, get = 1, cpy = 17, len = 6, uid = 101010 code_symbols[ 86]: _tt = false, src = 2, lit = 2, use = true, get = 0, cpy = 0, len = 7, uid = 1010110 code_symbols[ 87]: _tt = false, src = 3, lit = 3, use = true, get = 0, cpy = 0, len = 7, uid = 1010111 code_symbols[ 88]: _tt = false, src = 6, lit = 6, use = true, get = 0, cpy = 0, len = 7, uid = 1011000 code_symbols[ 89]: _tt = false, src = 8, lit = 8, use = true, get = 0, cpy = 0, len = 7, uid = 1011001 code_symbols[ 90]: _tt = false, src = 13, lit = 13, use = true, get = 0, cpy = 0, len = 7, uid = 1011010 code_symbols[ 91]: _tt = false, src = 25, lit = 25, use = true, get = 0, cpy = 0, len = 7, uid = 1011011 code_symbols[ 92]: _tt = false, src = 29, lit = 29, use = true, get = 0, cpy = 0, len = 7, uid = 1011100 code_symbols[ 93]: _tt = false, src = 33, lit = 33, use = true, get = 0, cpy = 0, len = 7, uid = 1011101 code_symbols[ 94]: _tt = false, src = 42, lit = 42, use = true, get = 0, cpy = 0, len = 7, uid = 1011110 code_symbols[ 95]: _tt = false, src = 52, lit = 52, use = true, get = 0, cpy = 0, len = 7, uid = 1011111 code_symbols[ 96]: _tt = false, src = 82, lit = 82, use = true, get = 0, cpy = 0, len = 7, uid = 1100000 code_symbols[ 97]: _tt = false, src = 103, lit = 103, use = true, get = 0, cpy = 0, len = 7, uid = 1100001 code_symbols[ 98]: _tt = false, src = 111, lit = 111, use = true, get = 0, cpy = 0, len = 7, uid = 1100010 code_symbols[ 99]: _tt = false, src = 118, lit = 118, use = true, get = 0, cpy = 0, len = 7, uid = 1100011 code_symbols[ 100]: _tt = false, src = 135, lit = 135, use = true, get = 0, cpy = 0, len = 7, uid = 1100100 code_symbols[ 101]: _tt = false, src = 142, lit = 142, use = true, get = 0, cpy = 0, len = 7, uid = 1100101 code_symbols[ 102]: _tt = false, src = 150, lit = 150, use = true, get = 0, cpy = 0, len = 7, uid = 1100110 code_symbols[ 103]: _tt = false, src = 164, lit = 164, use = true, get = 0, cpy = 0, len = 7, uid = 1100111 code_symbols[ 104]: _tt = false, src = 184, lit = 184, use = true, get = 0, cpy = 0, len = 7, uid = 1101000 code_symbols[ 105]: _tt = false, src = 190, lit = 190, use = true, get = 0, cpy = 0, len = 7, uid = 1101001 code_symbols[ 106]: _tt = false, src = 196, lit = 196, use = true, get = 0, cpy = 0, len = 7, uid = 1101010 code_symbols[ 107]: _tt = false, src = 238, lit = 238, use = true, get = 0, cpy = 0, len = 7, uid = 1101011 code_symbols[ 108]: _tt = false, src = 248, lit = 248, use = true, get = 0, cpy = 0, len = 7, uid = 1101100 code_symbols[ 109]: _tt = false, src = 250, lit = 250, use = true, get = 0, cpy = 0, len = 7, uid = 1101101 code_symbols[ 110]: _tt = false, src = 251, lit = 251, use = true, get = 0, cpy = 0, len = 7, uid = 1101110 code_symbols[ 111]: _tt = false, src = 253, lit = 253, use = true, get = 0, cpy = 0, len = 7, uid = 1101111 code_symbols[ 224]: _tt = false, src = 1, lit = 1, use = true, get = 0, cpy = 0, len = 8, uid = 11100000 code_symbols[ 225]: _tt = false, src = 4, lit = 4, use = true, get = 0, cpy = 0, len = 8, uid = 11100001 code_symbols[ 226]: _tt = false, src = 10, lit = 10, use = true, get = 0, cpy = 0, len = 8, uid = 11100010 code_symbols[ 227]: _tt = false, src = 15, lit = 15, use = true, get = 0, cpy = 0, len = 8, uid = 11100011 code_symbols[ 228]: _tt = false, src = 19, lit = 19, use = true, get = 0, cpy = 0, len = 8, uid = 11100100 code_symbols[ 229]: _tt = false, src = 22, lit = 22, use = true, get = 0, cpy = 0, len = 8, uid = 11100101 code_symbols[ 230]: _tt = false, src = 38, lit = 38, use = true, get = 0, cpy = 0, len = 8, uid = 11100110 code_symbols[ 231]: _tt = false, src = 47, lit = 47, use = true, get = 0, cpy = 0, len = 8, uid = 11100111 code_symbols[ 232]: _tt = false, src = 76, lit = 76, use = true, get = 0, cpy = 0, len = 8, uid = 11101000 code_symbols[ 233]: _tt = false, src = 89, lit = 89, use = true, get = 0, cpy = 0, len = 8, uid = 11101001 code_symbols[ 234]: _tt = false, src = 96, lit = 96, use = true, get = 0, cpy = 0, len = 8, uid = 11101010 code_symbols[ 235]: _tt = false, src = 127, lit = 127, use = true, get = 0, cpy = 0, len = 8, uid = 11101011 code_symbols[ 236]: _tt = false, src = 157, lit = 157, use = true, get = 0, cpy = 0, len = 8, uid = 11101100 code_symbols[ 237]: _tt = false, src = 171, lit = 171, use = true, get = 0, cpy = 0, len = 8, uid = 11101101 code_symbols[ 238]: _tt = false, src = 178, lit = 178, use = true, get = 0, cpy = 0, len = 8, uid = 11101110 code_symbols[ 239]: _tt = false, src = 201, lit = 201, use = true, get = 0, cpy = 0, len = 8, uid = 11101111 code_symbols[ 240]: _tt = false, src = 206, lit = 206, use = true, get = 0, cpy = 0, len = 8, uid = 11110000 code_symbols[ 241]: _tt = false, src = 211, lit = 211, use = true, get = 0, cpy = 0, len = 8, uid = 11110001 code_symbols[ 242]: _tt = false, src = 216, lit = 216, use = true, get = 0, cpy = 0, len = 8, uid = 11110010 code_symbols[ 243]: _tt = false, src = 220, lit = 220, use = true, get = 0, cpy = 0, len = 8, uid = 11110011 code_symbols[ 244]: _tt = false, src = 225, lit = 225, use = true, get = 0, cpy = 0, len = 8, uid = 11110100 code_symbols[ 245]: _tt = false, src = 228, lit = 228, use = true, get = 0, cpy = 0, len = 8, uid = 11110101 code_symbols[ 246]: _tt = false, src = 232, lit = 232, use = true, get = 0, cpy = 0, len = 8, uid = 11110110 code_symbols[ 247]: _tt = false, src = 235, lit = 235, use = true, get = 0, cpy = 0, len = 8, uid = 11110111 code_symbols[ 248]: _tt = false, src = 241, lit = 241, use = true, get = 0, cpy = 0, len = 8, uid = 11111000 code_symbols[ 249]: _tt = false, src = 244, lit = 244, use = true, get = 0, cpy = 0, len = 8, uid = 11111001 code_symbols[ 250]: _tt = false, src = 246, lit = 246, use = true, get = 0, cpy = 0, len = 8, uid = 11111010 code_symbols[ 251]: _tt = false, src = 252, lit = 252, use = true, get = 0, cpy = 0, len = 8, uid = 11111011 code_symbols[ 252]: _tt = false, src = 255, lit = 255, use = true, get = 0, cpy = 0, len = 8, uid = 11111100 code_symbols[ 253]: _tt = false, src = 256, lit = 256, use = true, get = 0, cpy = 0, len = 8, uid = 11111101 code_symbols[ 254]: _tt = false, src = 261, lit = 261, use = true, get = 0, cpy = 7, len = 8, uid = 11111110 code_symbols[ 255]: _tt = false, src = 267, lit = 267, use = true, get = 1, cpy = 15, len = 8, uid = 11111111 Loop Symbols List Details: foresee: 19, longest = 8, highest = 46, have = 131072, used = 256 loop_symbols[ 0]: _tt = false, src = 11, lit = 299, use = true, get = 4, cpy = 49, len = 4, uid = 0000 loop_symbols[ 4]: _tt = false, src = 0, lit = 288, use = true, get = 0, cpy = 1, len = 6, uid = 000100 loop_symbols[ 5]: _tt = false, src = 9, lit = 297, use = true, get = 3, cpy = 25, len = 6, uid = 000101 loop_symbols[ 6]: _tt = false, src = 12, lit = 300, use = true, get = 5, cpy = 65, len = 6, uid = 000110 loop_symbols[ 7]: _tt = false, src = 13, lit = 301, use = true, get = 5, cpy = 97, len = 6, uid = 000111 loop_symbols[ 8]: _tt = false, src = 15, lit = 303, use = true, get = 6, cpy = 193, len = 6, uid = 001000 loop_symbols[ 9]: _tt = false, src = 17, lit = 305, use = true, get = 7, cpy = 385, len = 6, uid = 001001 loop_symbols[ 10]: _tt = false, src = 18, lit = 306, use = true, get = 8, cpy = 513, len = 6, uid = 001010 loop_symbols[ 44]: _tt = false, src = 10, lit = 298, use = true, get = 4, cpy = 33, len = 8, uid = 00101100 loop_symbols[ 45]: _tt = false, src = 14, lit = 302, use = true, get = 6, cpy = 129, len = 8, uid = 00101101 loop_symbols[ 46]: _tt = false, src = 16, lit = 304, use = true, get = 7, cpy = 257, len = 8, uid = 00101110 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 588, used = 588 buff/zlib.c:913: code_symbols[ 0]: _tt = false, src = 0, lit = 0, use = true, get = 0, cpy = 0, len = 2, uid = 00 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 590, used = 590 buff/zlib.c:913: code_symbols[ 235]: _tt = false, src = 127, lit = 127, use = true, get = 0, cpy = 0, len = 8, uid = 11101011 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 598, used = 598 buff/zlib.c:913: code_symbols[ 100]: _tt = false, src = 135, lit = 135, use = true, get = 0, cpy = 0, len = 7, uid = 1100100 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 605, used = 605 buff/zlib.c:913: code_symbols[ 101]: _tt = false, src = 142, lit = 142, use = true, get = 0, cpy = 0, len = 7, uid = 1100101 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 612, used = 612 buff/zlib.c:913: code_symbols[ 102]: _tt = false, src = 150, lit = 150, use = true, get = 0, cpy = 0, len = 7, uid = 1100110 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 619, used = 619 buff/zlib.c:913: code_symbols[ 236]: _tt = false, src = 157, lit = 157, use = true, get = 0, cpy = 0, len = 8, uid = 11101100 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 627, used = 627 buff/zlib.c:913: code_symbols[ 103]: _tt = false, src = 164, lit = 164, use = true, get = 0, cpy = 0, len = 7, uid = 1100111 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 634, used = 634 buff/zlib.c:913: code_symbols[ 237]: _tt = false, src = 171, lit = 171, use = true, get = 0, cpy = 0, len = 8, uid = 11101101 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 642, used = 642 buff/zlib.c:913: code_symbols[ 238]: _tt = false, src = 178, lit = 178, use = true, get = 0, cpy = 0, len = 8, uid = 11101110 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 650, used = 650 buff/zlib.c:913: code_symbols[ 104]: _tt = false, src = 184, lit = 184, use = true, get = 0, cpy = 0, len = 7, uid = 1101000 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 657, used = 657 buff/zlib.c:913: code_symbols[ 105]: _tt = false, src = 190, lit = 190, use = true, get = 0, cpy = 0, len = 7, uid = 1101001 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 664, used = 664 buff/zlib.c:913: code_symbols[ 106]: _tt = false, src = 196, lit = 196, use = true, get = 0, cpy = 0, len = 7, uid = 1101010 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 671, used = 671 buff/zlib.c:913: code_symbols[ 239]: _tt = false, src = 201, lit = 201, use = true, get = 0, cpy = 0, len = 8, uid = 11101111 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 679, used = 679 buff/zlib.c:913: code_symbols[ 240]: _tt = false, src = 206, lit = 206, use = true, get = 0, cpy = 0, len = 8, uid = 11110000 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 687, used = 687 buff/zlib.c:913: code_symbols[ 241]: _tt = false, src = 211, lit = 211, use = true, get = 0, cpy = 0, len = 8, uid = 11110001 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 695, used = 695 buff/zlib.c:913: code_symbols[ 242]: _tt = false, src = 216, lit = 216, use = true, get = 0, cpy = 0, len = 8, uid = 11110010 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 703, used = 703 buff/zlib.c:913: code_symbols[ 243]: _tt = false, src = 220, lit = 220, use = true, get = 0, cpy = 0, len = 8, uid = 11110011 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 711, used = 711 buff/zlib.c:913: code_symbols[ 244]: _tt = false, src = 225, lit = 225, use = true, get = 0, cpy = 0, len = 8, uid = 11110100 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 719, used = 719 buff/zlib.c:913: code_symbols[ 245]: _tt = false, src = 228, lit = 228, use = true, get = 0, cpy = 0, len = 8, uid = 11110101 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 727, used = 727 buff/zlib.c:913: code_symbols[ 246]: _tt = false, src = 232, lit = 232, use = true, get = 0, cpy = 0, len = 8, uid = 11110110 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 735, used = 735 buff/zlib.c:913: code_symbols[ 247]: _tt = false, src = 235, lit = 235, use = true, get = 0, cpy = 0, len = 8, uid = 11110111 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 743, used = 743 buff/zlib.c:913: code_symbols[ 107]: _tt = false, src = 238, lit = 238, use = true, get = 0, cpy = 0, len = 7, uid = 1101011 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 750, used = 750 buff/zlib.c:913: code_symbols[ 248]: _tt = false, src = 241, lit = 241, use = true, get = 0, cpy = 0, len = 8, uid = 11111000 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 758, used = 758 buff/zlib.c:913: code_symbols[ 249]: _tt = false, src = 244, lit = 244, use = true, get = 0, cpy = 0, len = 8, uid = 11111001 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 766, used = 766 buff/zlib.c:913: code_symbols[ 250]: _tt = false, src = 246, lit = 246, use = true, get = 0, cpy = 0, len = 8, uid = 11111010 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 774, used = 774 buff/zlib.c:913: code_symbols[ 108]: _tt = false, src = 248, lit = 248, use = true, get = 0, cpy = 0, len = 7, uid = 1101100 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 781, used = 781 buff/zlib.c:913: code_symbols[ 109]: _tt = false, src = 250, lit = 250, use = true, get = 0, cpy = 0, len = 7, uid = 1101101 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 788, used = 788 buff/zlib.c:913: code_symbols[ 110]: _tt = false, src = 251, lit = 251, use = true, get = 0, cpy = 0, len = 7, uid = 1101110 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 795, used = 795 buff/zlib.c:913: code_symbols[ 251]: _tt = false, src = 252, lit = 252, use = true, get = 0, cpy = 0, len = 8, uid = 11111011 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 803, used = 803 buff/zlib.c:913: code_symbols[ 111]: _tt = false, src = 253, lit = 253, use = true, get = 0, cpy = 0, len = 7, uid = 1101111 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 810, used = 810 buff/zlib.c:913: code_symbols[ 41]: _tt = false, src = 254, lit = 254, use = true, get = 0, cpy = 0, len = 6, uid = 101001 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 816, used = 816 buff/zlib.c:913: code_symbols[ 41]: _tt = false, src = 254, lit = 254, use = true, get = 0, cpy = 0, len = 6, uid = 101001 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 822, used = 822 buff/zlib.c:913: code_symbols[ 252]: _tt = false, src = 255, lit = 255, use = true, get = 0, cpy = 0, len = 8, uid = 11111100 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 830, used = 830 buff/zlib.c:913: code_symbols[ 0]: _tt = false, src = 0, lit = 0, use = true, get = 0, cpy = 0, len = 2, uid = 00 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 832, used = 832 buff/zlib.c:913: code_symbols[ 99]: _tt = false, src = 118, lit = 118, use = true, get = 0, cpy = 0, len = 7, uid = 1100011 buff/zlib.c:890: Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 839, used = 839 buff/zlib.c:913: code_symbols[ 6]: _tt = false, src = 272, lit = 272, use = true, get = 2, cpy = 31, len = 4, uid = 0110 buff/zlib.c:253: Error 0x00000022 (34) 'Numerical result out of range' Unable to identify symbol, last UID sought: 001000001 , permitted upto: 100000000 buff/zlib.c:953: Error 0x00000016 (22) 'Invalid argument' Stream Details: End Of Data = true, End Of Stream = false, data = 0x7ffc589bfd10, byte = 262, took = 845, used = 845 ...
Last edited by awsdert; 08-27-2021 at 03:48 PM.
I think you might end up with cleaner code if you realized that
is better represented by:Code:ZLIB_IMPLIED implied_type_data = { /* get extra bits */ { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 3, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 }, /* base cpy value */ { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 } };
i.e. for values < 16 there is one copy of the value added to the table.Code:ZLIB_IMPLIED implied_type_data = { /* get extra bits */ { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 3, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 }, /* base cpy value */ { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 3, 3, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 } };
Just tried your suggestion, didn't see a change when I ran it but I'll give it a look tomorrow, anyways here's the upload for it:
Adding in a suggestion by hamster_nz, didn't see a change when I ran it (a317a3d4) * Commits * Lee Shallis / glEngine * GitLab
You still need to adjust the rest of your code accordingly too.
This change means that you will only need to add entries to your table in one place in your code, rather than handling those 16 and over differently to those under 16.
(It does however still leave 16 as the odd one out, where the value to be added to the table isn't explicitly known, but depends on the value that was most recently added).
Last edited by hamster_nz; 08-27-2021 at 04:42 PM.
? Are you referring to the type codes? They're supposed to be handled differently, remember, they're using either prev symbol or nothing at all, if I try to fuse the two then I'll have to check inside the loop whether I'm copying a length or using the type's original index as a length, that's less efficient in a subloop so I keep 'em separate from the subloop, I'm also considering doing a copy paste of the LoadZlibHuffs() once I confirm it's working and renaming both to reflect the list they'll be focusing on and removing the inner if statement or inner else statement