Thread: linux writing often on main disk

I had a little argue with my boss:
he thinks that for our puropuses (embedded Vision Systems using GigE cameras) buildroot crosscompiling together with uclibc on an embedded system with flash and no hard disk is fine
I think instead that an industrial PC without hard disk, with two flash and a standard linux distribution like Debian installed in minimum form (only those applications that are strictly necessary to us like ssh, ftp) is better.
His arguments are that standard linux distribution writes often on disk so can block our system for many msec (we have a required response time of about 25 msec that means we need not to be hanged by the OS for more than 25 msec) and can damage the compact flash
My arguments are :
(1) a standard linux distribution uses NPTL (while buildroot + uclibc uses linuxthreads) -> much more efficient multithreading and more easy to program as it is 99% POSIX compliant
(2) standard linux with glibc has commercial libraries available for opur cameras system

My questions are:
(1) Is his argument about linux writing often to main file system correct?
(2) Is my argument about NPTL vs. linuxthreads correct?

Or also after some study might be emdebian could the right agreement between the two opinions?

Thank you

(2) Is my argument about NPTL vs. linuxthreads correct?

Yes.

(1) Is his argument about linux writing often to main file system correct?

I don't think any standard linux distro writes to disk more often than any other linux.

re: linux writing to disk: I read a little while ago that because linux, like all *nix, maintains "access times" for the filesystem, this means every disk read must also be complimented by a disk WRITE in order to update the atime. Supposedly all this adds up, so in heavy use situations where the atime is not considered crucial (since the m(odification)time is seperate from the atime, you might wonder when it ever would really be important) you can start linux with the "noatime" kernel parameter which means the kernel no longer writes an atime to disk every time a file is accessed. And this means something like a 10-20% improvement in speed where the disk access is constant.

Since you are dealing with significance on the order of 25ms and are worried about time wasted with (basically irrelevent) writes to disk, I thot I'd mention that. Google "noatime" and some articles will appear.

Gentoo by default introduces all mount lines with noatime. I'm booting my laptop to see if Ubuntu does it too, but I'm convinced it does, just as I'm convinced every other distro out there does it by default. atime is an extremely specialized feature, and I'd be very surprised to find anything that enables it by default.

Edit: Ubuntu mounts with relatime, which means "update atime if it's older than mtime or ctime", or in other words, "update atime on write, or when accessing after someone accessed it with noatime". In other words, it's mostly a noop.

You may want to put something in place to log where the writes to disk are coming from - for example which process and what file (filehandle and PID should be enough).

I guess having a kernel-side buffer inside the write system call, that stores a count of each PID/FILE combination, and whenever you feel it's a good time, dump it to syslog, serial port or some such.

If it's a regular thing, then it would most likely happen quite often.

One solution may be to move that file to a tmpfs (ram-disk)...

--
Mats

Why would anything be writing to disk if nothing is running except your code?

Disk writes don't just appear from nowhere, some process has to request them.

And even if atime updating is turned on, the filesystem is cached, including inodes. I think your boss is inventing a problem where none exists.

I think a number of distros are using relatime now but certainly not all of them (eg, neither fedora nor debian do it by default), and most of them do not use noatime, since my original source for this was a recent article by Mr. Linus Torvalds (or rather interviewing him), where he says that most large commercial installations (which I guess is the opposite of what the OP is doing, so mebbe this all irrelevent) in fact use neither, tho he suggests they should.

Here's a link to a linux foundation article and tests on solid state disks where the author claims that "What the results show is that metadata-heavy workloads, such as make clean, do result in almost twice the amount data written to disk." That's the biggest difference scenario.

http://beta.linuxfoundation.org/news...oatimerelatime

This could even be the bird that half whispered in that bosses ear, starting all the fuss...

Originally Posted by mynickmynick

Thank you everybody for help

Does this mean that you have found a solution, and if so, can you share some of what you've found with us?

--
Mats

Originally Posted by mynickmynick

But I remember I made some tests on Debian with RAMdisk and I found out that the time for writing to ramdisk was only 20% less than writing on hard disk

That just means Linux's filesystem caching is about 80% efficient.

Yes, Linux is generally using any spare memory (memory that isn't currently used by applications or the kernel itself) as a cache for disk operations, so writing to disk will actually just write to memory, and later on write to disk proper.

--
Mats