Debian on a system with Compact Flash, without Hard Disk

I have been told that Debian on an industrial PC with a Comapct flash and without hard disk is not a great idea cause the OS writes often on the main disk and this would hit with flash limited write amount
is that correct?
I am preparing a prototype for driving cameras and analyzing images. I do not need much stuff to run. Just my threads, a few servers like ssh or ftp and very probably also Eclipse to avoid cross compiling.
Cannot I just install the minimal needed so that probably the OS will not write much on flash and corrupt it?

Thank you

I'm sure there is a flash-filesystem you can use in Linux, but I wouldn't worry about it too much - modern flash devices are writable [guaranteed/typical] a million times or more - so even if you write every second to the same sector, it would last a few weeks (around 12 days).

CF cards cost near nothing, so for now it wouldn't be a problem, right?

--
Mats

Quote:

---

Originally Posted by matsp

I'm sure there is a flash-filesystem you can use in Linux, but I wouldn't worry about it too much - modern flash devices are writable [guaranteed/typical] a million times or more - so even if you write every second to the same sector, it would last a few weeks (around 12 days).

CF cards cost near nothing, so for now it wouldn't be a problem, right?

--
Mats

---

well so and so
I would like to better investigate this problem

The other point is of course that on such a system, I expect you'd be using a RAM disk to store temporary data, etc [and you probably don't want to "syslog" to your CF device either - at least not once you've got a working system - and even boot-debug etc, I'd prefer to send the console output to a serial port than to rely on what was stored by syslog in the filesystem - it's usually missing the very last dozen lines or so, which is the part where it crashes.].

So what do you expect to write to the CF device very much?

--
Mats

Quote:

---

Originally Posted by matsp

The other point is of course that on such a system, I expect you'd be using a RAM disk to store temporary data, etc [and you probably don't want to "syslog" to your CF device either - at least not once you've got a working system - and even boot-debug etc, I'd prefer to send the console output to a serial port than to rely on what was stored by syslog in the filesystem - it's usually missing the very last dozen lines or so, which is the part where it crashes.].

So what do you expect to write to the CF device very much?

--
Mats

---

My process would save some important data (a few files)say in average 100 times a day
For sure I would install a quite big RAM disk (I should know more or less how to do that, even if I did only on a disk with PC)
The problems are two:
(1) how to configure the OS so that its processes save the least (no logs etc.)
(2) Can Debian be installed on a disk less system or do I have to look for some not very common and tested minimal Linux distributions?

Another doubt. Maybe I can find some alternative to a flash, like some USB device?
My problem is that this prototype has to run with relatively high vibrations so an hard disk is not a great idea

Are there external solid-state disks? SSDs are writeable very often.

Quote:

---

Originally Posted by CornedBee

Are there external solid-state disks? SSDs are writeable very often.

---

there are INTERNAL solid state disks, made for IDE, SATA, SCSI, and SAS. the SATA, SCSI, and SAS types are usually intended for server applications, meaning it's likely that they can be written to a nearly infinite number of times, especially with todays servers requiring database inserts and updates at a high rate. I have seen SATA and IDE SSDs for as little as a few hundred dollars (US) for up to 64GB.

> Another doubt. Maybe I can find some alternative to a flash, like some USB device?

That's not a doubt, it's a question.

Flash is pretty much the way to go if you need non-volatile storeage. It's cheap and pretty fast (especially for reading).

I was reading an article on flash the other day. It said that most flash chips actually have about 20%-50% more space than they say they have. As soon as part of the flash goes bad, after the million-some writes are up or whatever, the chip will silently mark it as so and use some of the spare space instead. That's what it said, anyway.

So I really wouldn't worry about it too much. If you write in the same place over and over, it will just get replaced when it gets fried. If you write to the whole disk frequently, well, you're getting a lot of use out of the device. :)

For what it's worth -- how many times have you scanned your hard drive and discovered bad sectors on it? :)

Quote:

---

(2) Can Debian be installed on a disk less system or do I have to look for some not very common and tested minimal Linux distributions?

---

Absolutely. I myself have a bootable Debian system on a USB flash pendrive. ;) It's amazing what you can fit into a gigabyte.

ok great
And what about the corruption if somebody switches the system off abruptly (in my case (where the device will be installed) it happens very often)?
i have been told the flash can be corrupted if the system was writing in that moment? Is that correct? can ext3 system recover the file system?
What about solid state hard drives instead?

Quote:

---

Originally Posted by mynickmynick

ok great
And what about the corruption if somebody switches the system off abruptly (in my case (where the device will be installed) it happens very often)?
i have been told the flash can be corrupted if the system was writing in that moment? Is that correct? can ext3 system recover the file system?
What about solid state hard drives instead?

---

If this happens "often", then you probably need some sort of preventative measure - such as a "power-off that isn't immediate".

There is no real difference between flash and real hard-disks here - if you pull the power on a hard disk when data is being written, you end up with inconsistent data on the disk. Same with flash - you will have some random data in the flash. If that is some IMPORTANT data, then that could be fatal, but modern filesystems do have backup mechanisms to recover that USUALLY works.

--
Mats

Quote:

---

Originally Posted by matsp

If this happens "often", then you probably need some sort of preventative measure - such as a "power-off that isn't immediate".

There is no real difference between flash and real hard-disks here - if you pull the power on a hard disk when data is being written, you end up with inconsistent data on the disk. Same with flash - you will have some random data in the flash. If that is some IMPORTANT data, then that could be fatal, but modern filesystems do have backup mechanisms to recover that USUALLY works.

--
Mats

---

I will explain myself better. For the data being written in that moment of course they are very probably lost. But if one has implemented some smart double recording he can prevent that (he can retrieve data in the last old version).
What I am talking of is more general all-flash corruption (you do not retrieve also all other data on flash) : file system corruption!

Quote:

---

Originally Posted by mynickmynick

I will explain myself better. For the data being written in that moment of course they are very probably lost. But if one has implemented some smart double recording he can prevent that (he can retrieve data in the last old version).
What I am talking of is more general all-flash corruption (you do not retrieve also all other data on flash) : file system corruption!

---

Flash is no more (or less) vulnerable to this [assuming the filesystem is indeed duplicating crucial data - and most Linux/Unix filesystems do in one way or another - even FAT does that]. The flash device itself is not "destroyed" during a power-down during writing - it will just contain incomplete/incorrect data in that area. Don't you think you would have heard about it if this happened easily, considering the number of iPod's that are out there - must be some people who accidentally shut down the power on their iPod [or run out of power or whatever] whilst the device is writing to the flash-device.

--
Mats

>> And what about the corruption if somebody switches the system off abruptly
Journaling file systems are designed for this to make corruption much less likely...like JFFS2 for example.

gg

Quote:

---

Originally Posted by Codeplug

>> And what about the corruption if somebody switches the system off abruptly
Journaling file systems are designed for this to make corruption much less likely...like JFFS2 for example.

gg

---

thank you
I have been told only some linux partitioning have journaling (like ext3). Is that correct?

Quote:

---

Originally Posted by matsp

Flash is no more (or less) vulnerable to this [assuming the filesystem is indeed duplicating crucial data - and most Linux/Unix filesystems do in one way or another - even FAT does that]. The flash device itself is not "destroyed" during a power-down during writing - it will just contain incomplete/incorrect data in that area. Don't you think you would have heard about it if this happened easily, considering the number of iPod's that are out there - must be some people who accidentally shut down the power on their iPod [or run out of power or whatever] whilst the device is writing to the flash-device.

--
Mats

---

I quite agree. i will try to further discuss about it with my colleague who disagrees with you.

Quote:

---

Originally Posted by mynickmynick

I have been told that Debian on an industrial PC with a Comapct flash and without hard disk is not a great idea cause the OS writes often on the main disk and this would hit with flash limited write amount
is that correct?
I am preparing a prototype for driving cameras and analyzing images. I do not need much stuff to run. Just my threads, a few servers like ssh or ftp and very probably also Eclipse to avoid cross compiling.
Cannot I just install the minimal needed so that probably the OS will not write much on flash and corrupt it?

Thank you

---

moreover I guess i can disable swapping

All filesystems really used by Linux except ext2 have journalling.

Quote:

---

Originally Posted by CornedBee

All filesystems really used by Linux except ext2 have journalling.

---

great!
and you think that it protects fine the flash from sudden power off?

http://www.ibm.com/developerworks/li...ems/index.html

gg