Thread: Assistance to program a compression algorithm

  1. #1
    Registered User
    Join Date
    Jun 2014

    Assistance to program a compression algorithm

    I have been researching compression in general and different techniques.
    I have noticed that general tools such as 7z/rar are relatively poor on files which are mostly repetition.
    There is an example of a BMP of 8 colours which repeat through most of the file where the PNG is 5KB and without the 56B header (and even with) RAR is 2KB and 7z is 1KB.
    A tool such as DEFLATE can get it to less than 100B, and can shrink it further for its patterns to fit in less bits to around 50B.

    I have written an overview for a versatile and adjustable compression tool where its elements can be specifically set for a customized result.
    The most effective one will require a lot of calculations and temporary space and the result should be generally just about as compact as a file should get without moving data in advance.
    General purpose settings for quicker encoding and much more effective results than common tools such as 7z/rar is possible.

    The idea is to be able to have an algorithm created from the text, as I am not familiar with programming languages to develop an algorithm/pseudocode to suit a computer.
    Once a working tool can be done, ideally the code can be used to make an ASM port and also a bootable scheduled form to directly process without video/sound etc for more thorough settings on more permanent data like archival/video/audio/asset etc.

    It is aimed to be more of a one-off type of tool where it elements can be adjusted for speed/time and be comprehensive on data in the data spectrum (without anything targetting specific data types per se) - it should work on files where compression is possible (which ideally will not be for strings resembling coded data).

    Encoding will take its time depending on settings selected. Decoding should generally be quick in general.

    If anyone is willing to have a look to assist in an algorithm and a tool I can post the writeup.

    Last edited by SoraK05; 06-15-2014 at 02:18 PM.

  2. #2
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    The edge of the known universe
    My first thought would be for you to read these links
    comp.compression Frequently Asked Questions (part 1/3)
    Data compression - Wikipedia, the free encyclopedia
    to find out whether you've just re-invented something which already exists.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.

  3. #3
    Registered User
    Join Date
    Jun 2014
    I have not
    The idea is to have a tweakable tool for its elements to be more specific to compression, and the more thorough compression option should generally be able to capture almost all the compression available looking at strings without moving data in advance. This one looks at variable bit strings which are detected and also includes a repeat of a specific process to layer parts within the result so the result can be coded+layered<coded>+raw and the layered part will decode temporarily and be appended for example.

    I have looked in general at capturing strings in a file for repetition to compress and general layouts, and it is to be comprehensive and have settings to adjust compression levels to suit a preference.

    I might add all current methods are lossless for retaining data in a compact manner. I have not yet researched lossy compression and string detection which may be able to be included
    Last edited by SoraK05; 06-16-2014 at 06:36 AM.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Compression algorithm help
    By bhargav_g in forum C Programming
    Replies: 2
    Last Post: 02-24-2013, 09:34 AM
  2. Working with LZW data compression algorithm
    By husslela2 in forum C Programming
    Replies: 3
    Last Post: 07-01-2010, 09:23 AM
  3. Simple compression algorithm?
    By cyberfish in forum Tech Board
    Replies: 8
    Last Post: 05-02-2008, 09:10 AM
  4. efficient compression algorithm pls
    By gooddevil in forum C Programming
    Replies: 7
    Last Post: 05-08-2004, 03:33 AM
  5. what's the best compression algorithm made so far?
    By toaster in forum A Brief History of
    Replies: 13
    Last Post: 05-15-2002, 03:20 PM