I am writing output to a text file. However these files are getting quite large. I am writing information line by line and finding files get to about 1MB. How can I minimise the size of these files. I have thought that changing the seperation may work but uses the same space. How can I reduce the size of these files? I dont want the files to be hard to read by user either. Any idea?
Files are something like this
Position A
44.44
434.34
343.434
43.43
4343.434
Position B
Thanks
Well, if these are text files you want to be read by an human, the only way to reduce the size of the file is to write less to it.
You may need to provide more details here. Are all those numbers needed?
"Position A" could probably be reduced to "Pos. A" or even further to "A". But without information on what those numbers mean, I don't think anyone can help you much.
Originally Posted by brewbuck:
Reimplementing a large system in another language to get a 25% performance boost is nonsense. It would be cheaper to just get a computer which is 25% faster.
Well whether you write it as
44.44
434.34
or
44.44 434.34
Isn't going to make that much of a difference.
Anything else is really going to impact on readability.
If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
If at first you don't succeed, try writing your phone number on the exam paper.
Well, since you tried changing how the file is represented, maybe you should consider whether or not everything in the file needs to be there. I mean, the file you showed us is meaningless to me, maybe you're logging too much.
To be honest, one megabyte is not that bad. I could easily produce a megabyte document in Word. If you really want smaller files, consider using a compression algorithm/tool, and decompressing the file when you read it. You can learn one such algorithm from the site: http://cprogramming.com/tutorial/com...y/huffman.html
but there are many of these.
Thanks guys
I have tried minimising the characters in the file. I am away from my office machine with the code on it so could not post an actual example. Yes a tab character "\t" uses the same space as newline "\n". I dont think compression is the way forward...its actually scientific research and I am generating thousands upon thousands of these files so they add up really.
I guess I am stuck.
Then do as Citizen suggested. Store those files compressed and provide the user with a "reader" program that decompresses the file and displays the contents.
Originally Posted by brewbuck:
Reimplementing a large system in another language to get a 25% performance boost is nonsense. It would be cheaper to just get a computer which is 25% faster.
Our access_log and error_log generate way more than 1mb per hour depending on how many hits we get and if we are currently working on the code and have more debug output turned on. On days when a lot of work is going to PHP, the error_log file can grow to 50m+ easy in a couple hours. 1m is almost nothing by newer standards... You can pick up a 80 gig harddrive used for 20-30 bucks off craigslist. If you are worried about how much logging you are doing. Then get one of those. 80gig ~=80,000 of your log files (no one needs to go into a gig is 1024 meg.. just easier to round).
You could gzip the files. If the files are quite repetitive it can save a lot of space.
dwk
Seek and ye shall find. quaere et invenies.
"Simplicity does not precede complexity, but follows it." -- Alan Perlis
"Testing can only prove the presence of bugs, not their absence." -- Edsger Dijkstra
"The only real mistake is the one from which we learn nothing." -- John Powell
Other boards: DaniWeb, TPS
Unofficial Wiki FAQ: cpwiki.sf.net
My website: http://dwks.theprogrammingsite.com/
Projects: codeform, xuni, atlantis, nort, etc.
