I am writing output to a text file. However these files are getting quite large. I am writing information line by line and finding files get to about 1MB. How can I minimise the size of these files. I have thought that changing the seperation may work but uses the same space. How can I reduce the size of these files? I dont want the files to be hard to read by user either. Any idea?
Files are something like this
Well, if these are text files you want to be read by an human, the only way to reduce the size of the file is to write less to it.
You may need to provide more details here. Are all those numbers needed?
"Position A" could probably be reduced to "Pos. A" or even further to "A". But without information on what those numbers mean, I don't think anyone can help you much.
Well whether you write it as
Isn't going to make that much of a difference.
Anything else is really going to impact on readability.
Well, since you tried changing how the file is represented, maybe you should consider whether or not everything in the file needs to be there. I mean, the file you showed us is meaningless to me, maybe you're logging too much.
To be honest, one megabyte is not that bad. I could easily produce a megabyte document in Word. If you really want smaller files, consider using a compression algorithm/tool, and decompressing the file when you read it. You can learn one such algorithm from the site: http://cprogramming.com/tutorial/com...y/huffman.html
but there are many of these.
I have tried minimising the characters in the file. I am away from my office machine with the code on it so could not post an actual example. Yes a tab character "\t" uses the same space as newline "\n". I dont think compression is the way forward...its actually scientific research and I am generating thousands upon thousands of these files so they add up really.
I guess I am stuck.
Then do as Citizen suggested. Store those files compressed and provide the user with a "reader" program that decompresses the file and displays the contents.
Our access_log and error_log generate way more than 1mb per hour depending on how many hits we get and if we are currently working on the code and have more debug output turned on. On days when a lot of work is going to PHP, the error_log file can grow to 50m+ easy in a couple hours. 1m is almost nothing by newer standards... You can pick up a 80 gig harddrive used for 20-30 bucks off craigslist. If you are worried about how much logging you are doing. Then get one of those. 80gig ~=80,000 of your log files (no one needs to go into a gig is 1024 meg.. just easier to round).
You could gzip the files. If the files are quite repetitive it can save a lot of space.