linux writing often on main disk

I had a little argue with my boss:
he thinks that for our puropuses (embedded Vision Systems using GigE cameras) buildroot crosscompiling together with uclibc on an embedded system with flash and no hard disk is fine
I think instead that an industrial PC without hard disk, with two flash and a standard linux distribution like Debian installed in minimum form (only those applications that are strictly necessary to us like ssh, ftp) is better.
His arguments are that standard linux distribution writes often on disk so can block our system for many msec (we have a required response time of about 25 msec that means we need not to be hanged by the OS for more than 25 msec) and can damage the compact flash
My arguments are :
(1) a standard linux distribution uses NPTL (while buildroot + uclibc uses linuxthreads) -> much more efficient multithreading and more easy to program as it is 99% POSIX compliant
(2) standard linux with glibc has commercial libraries available for opur cameras system

My questions are:
(1) Is his argument about linux writing often to main file system correct?
(2) Is my argument about NPTL vs. linuxthreads correct?

Or also after some study might be emdebian could the right agreement between the two opinions?

Thank you

Quote:

---

(2) Is my argument about NPTL vs. linuxthreads correct?

---

Yes.

Quote:

---

(1) Is his argument about linux writing often to main file system correct?

---

I don't think any standard linux distro writes to disk more often than any other linux.

re: linux writing to disk: I read a little while ago that because linux, like all *nix, maintains "access times" for the filesystem, this means every disk read must also be complimented by a disk WRITE in order to update the atime. Supposedly all this adds up, so in heavy use situations where the atime is not considered crucial (since the m(odification)time is seperate from the atime, you might wonder when it ever would really be important) you can start linux with the "noatime" kernel parameter which means the kernel no longer writes an atime to disk every time a file is accessed. And this means something like a 10-20% improvement in speed where the disk access is constant.

Since you are dealing with significance on the order of 25ms and are worried about time wasted with (basically irrelevent) writes to disk, I thot I'd mention that. Google "noatime" and some articles will appear.

Gentoo by default introduces all mount lines with noatime. I'm booting my laptop to see if Ubuntu does it too, but I'm convinced it does, just as I'm convinced every other distro out there does it by default. atime is an extremely specialized feature, and I'd be very surprised to find anything that enables it by default.

Edit: Ubuntu mounts with relatime, which means "update atime if it's older than mtime or ctime", or in other words, "update atime on write, or when accessing after someone accessed it with noatime". In other words, it's mostly a noop.

You may want to put something in place to log where the writes to disk are coming from - for example which process and what file (filehandle and PID should be enough).

I guess having a kernel-side buffer inside the write system call, that stores a count of each PID/FILE combination, and whenever you feel it's a good time, dump it to syslog, serial port or some such.

If it's a regular thing, then it would most likely happen quite often.

One solution may be to move that file to a tmpfs (ram-disk)...

--
Mats

Why would anything be writing to disk if nothing is running except your code?

Disk writes don't just appear from nowhere, some process has to request them.

And even if atime updating is turned on, the filesystem is cached, including inodes. I think your boss is inventing a problem where none exists.

I think a number of distros are using relatime now but certainly not all of them (eg, neither fedora nor debian do it by default), and most of them do not use noatime, since my original source for this was a recent article by Mr. Linus Torvalds (or rather interviewing him), where he says that most large commercial installations (which I guess is the opposite of what the OP is doing, so mebbe this all irrelevent) in fact use neither, tho he suggests they should.

Here's a link to a linux foundation article and tests on solid state disks where the author claims that "What the results show is that metadata-heavy workloads, such as make clean, do result in almost twice the amount data written to disk." That's the biggest difference scenario.

http://beta.linuxfoundation.org/news...oatimerelatime

This could even be the bird that half whispered in that bosses ear, starting all the fuss...

Quote:

---

Originally Posted by brewbuck

Why would anything be writing to disk if nothing is running except your code?

Disk writes don't just appear from nowhere, some process has to request them.

And even if atime updating is turned on, the filesystem is cached, including inodes. I think your boss is inventing a problem where none exists.

---

He is sponsoring his platform (he had already developed and used for other purpouses, in this case with very little adaptaments)

Thank you everybody for help

Quote:

---

Originally Posted by CornedBee

Yes.


I don't think any standard linux distro writes to disk more often than any other linux.

---

I apologize I was not detailed and precise:
actually his complete argument is that his embedded linux has the main file system on ramdisk and only a subdir as flash
This would eliminate compact flash progressive corruption (still actually I guess reached in so many years it is not very relevant) and would reduce the time loss
But I remember I made some tests on Debian with RAMdisk and I found out that the time for writing to ramdisk was only 20% less than writing on hard disk
Plus as I said our applications would write so little that the compact flash would be corrupted in xxx years so..

Quote:

---

Originally Posted by mynickmynick

Thank you everybody for help

---

Does this mean that you have found a solution, and if so, can you share some of what you've found with us?

--
Mats

unfortunately not
I have to work now with this buildroot crosscompiling tools that I don't like
It even has only alpha C++ support
That means I will program only in C with no C++
I will in the future investigate on using emdebian instead and maybe without cross compiling.
May be compiling directly on the target or on a copy of the target

Quote:

---

Originally Posted by mynickmynick

But I remember I made some tests on Debian with RAMdisk and I found out that the time for writing to ramdisk was only 20% less than writing on hard disk

---

That just means Linux's filesystem caching is about 80% efficient.

Quote:

---

Originally Posted by brewbuck

That just means Linux's filesystem caching is about 80% efficient.

---

what you mean? I don't know chaching mechanism in the case of file system.. Is the writing on hard disk delayed after a wriitng on RAM?

Yes, Linux is generally using any spare memory (memory that isn't currently used by applications or the kernel itself) as a cache for disk operations, so writing to disk will actually just write to memory, and later on write to disk proper.

--
Mats

oh that's cool but it then prevents me from measuring the time for writing to the hard disk!!

Quote:

---

Originally Posted by mynickmynick

oh that's cool but it then prevents me from measuring the time for writing to the hard disk!!

---

No, but it may prevent you from getting an accurate measurement for a short sequence of writes.

--
Mats

You can get around this by making your test use direct IO. Open the file with the O_DIRECT option set, and it wont cache the writes. Keep in mind that you will need to make all your writes page aligned (usually 4096 bytes).