FAT and Physical Disk Information

Hi, whats up there?

I went through something that I really didn't have enough information about it. It was about a flash drive that Windows displayed a wrong size for it. I had some research about that but what I've got is not well classified and I'm kinda confuzed. It seemed that the physical information is stored in MBR and each partition's FAT. I found some texts about it. I used a hex editor and found that bytes in their positions. There I could change them.

My problem is that if the FAT has wrong info then Windows doesn't see the memory correctly. For example windows may see only a portion of memory 60 MBs of a 256MB flash memory for instance. How can this be solved? Am I correct that Windows reads size info from FAT?

Reformat the drive.

Of course reformating the drive wont solve the problem. As Windows reads the size info wrongly, it formats the disk using wrong size. Or for some corrupted disks it can't format the drive. You can test it using a disk editor. Change some FAT data and try to format the disk. It wont format it.

[Disk == Flash Memory == Memory Device]

Quote:

---

Originally Posted by siavoshkc

Of course reformating the drive wont solve the problem. As Windows reads the size info wrongly, it formats the disk using wrong size. Or for some corrupted disks it can't format the drive. You can test it using a disk editor. Change some FAT data and try to format the disk. It wont format it.

[Disk == Flash Memory == Memory Device]

---

AFAIK Windows doesn't use the MBR or FAT system to determine the size of the drive. Drives have "direct" commands that are accessed by the driver to gather that information, so the problem is with the driver being used, most likely.

'Windows' does use the MBR, 'Format' however uses the actual disk specifications it retrieves from the BIOS. BIOS gets the data from the hard drives controller.


Partition Table - OSDev Wiki

What I mean is there is no ACTUAL specifications. Its only FAT. There is no special place in a flash memory to keep such data.

Quote:

---

Originally Posted by siavoshkc

What I mean is there is no ACTUAL specifications. Its only FAT. There is no special place in a flash memory to keep such data.

---

But flash drives do contain a driver controller with a ROM where this information could be stored, I think. On the other hand, to my knowledge standard flash drives all use 512 bytes of memory per sector. So the full drive size can be inferred from there.

Another option is for you to use ScanDisk which does a much better job at identifying and fixing these issues.

Windows cannot possibly use some FAT data in the way the OP claims since you can reformat and partition flash drives in non-FAT formats, but I am sure after you do that windows will still be willing to erase the whole thing and re-format it using FAT, altho originally there was no FAT partition at all.

Quote:

---

Originally Posted by siavoshkc

What I mean is there is no ACTUAL specifications. Its only FAT. There is no special place in a flash memory to keep such data.

---

Its not limited to FAT, I have thumb drives that are formatted in NTFS as well as several others. Sector 0 on flash devices is usually treated specially with more redundancy and hard coded locations than generic sectors. So I'm not sure exactly what you are trying to say/ask.

Quote:

---

Its not limited to FAT, I have thumb drives that are formatted in NTFS as well as several others. Sector 0 on flash devices is usually treated specially with more redundancy and hard coded locations than generic sectors. So I'm not sure exactly what you are trying to say/ask.

---

I know. But when I saw Windows read the wrong info from FAT in format dialog box. I concluded that Windows sees FAT as the most reliable disk info source. I used an Ontrack product for repairing disks. That software saw it wrong too. Windows can format the disk in whatever format possible. But before that it should know the physical info of drive like size, cylinder, sector count, etc.

You can test these things by yourself. Using DeviceIo() API and a disk editor you can do some actions.

Quote:

---

But flash drives do contain a driver controller with a ROM where this information could be stored.

---

How can that ROM be accessed programmatically?

Quote:

---

Originally Posted by siavoshkc

How can that ROM be accessed programmatically?

---

Maybe this will help?
Flash memory - low level access

This is borderline knowledge to me. I'm not comfortable speaking much more about it. I'm not even sure the flash controller ROM contains the information you are after. I'd assume so. Same as it containing the driver that gets installed to your system during first install.

I'm also a little confused... Do you want to fix a flash drive that somehow started to misbehave? Or do you want to programatically force a flash drive to display a different size than its real one?

Quote:

---

I'm also a little confused... Do you want to fix a flash drive that somehow started to misbehave? Or do you want to programatically force a flash drive to display a different size than its real one?

---

I already have the flash displaying different size. But I really get mad when I don't know something I want to know. I am really curious. I always had an idea that there is a place in the disk that keeps its info like that ROM, but never really thought about it.

Are you sure windows is misreading it or are you just getting confused with the difference between 1 megabyte and 1 million bytes (they aren't the same).

There are many scam USB drives from China (less than reputable vendors) that intentionally mislabel flash drives. It would say, for example, 4GB on the box and the drive, and Windows would detect it as a 4GB drive, and it will work fine until you try to write, say, more than 128MB of data to it. Then bad things start to happen.

Quote:

---

Originally Posted by cyberfish

There are many scam USB drives from China (less than reputable vendors) that intentionally mislabel flash drives. It would say, for example, 4GB on the box and the drive, and Windows would detect it as a 4GB drive, and it will work fine until you try to write, say, more than 128MB of data to it. Then bad things start to happen.

---

Really? Wow, that sucks!

Quote:

---

Are you sure windows is misreading it or are you just getting confused with the difference between 1 megabyte and 1 million bytes (they aren't the same).

---

Man, I'm a programmer.

Quote:

---

There are many scam USB drives from China (less than reputable vendors) that intentionally mislabel flash drives. It would say, for example, 4GB on the box and the drive, and Windows would detect it as a 4GB drive, and it will work fine until you try to write, say, more than 128MB of data to it. Then bad things start to happen.

---

You missed the point. How Windows or any other software can obtain disk size info? Its only FAT or there is some other place? There can be a readonly sector on disk to keep such info!
Imagine a fake Flash Mem like your example. It is 256MBs. But it has fake info in it(beside the label) that makes Windows think that its a 4GBs one. Can we fix this by knowing the exact disk specs and rewrite them on FAT? Can we obtain exact right specs from flash somehow?

Quote:

---

Originally Posted by siavoshkc

Man, I'm a programmer.


You missed the point. How Windows or any other software can obtain disk size info? Its only FAT or there is some other place? There can be a readonly sector on disk to keep such info!
Imagine a fake Flash Mem like your example. It is 256MBs. But it has fake info in it(beside the label) that makes Windows think that its a 4GBs one. Can we fix this by knowing the exact disk specs and rewrite them on FAT? Can we obtain exact right specs from flash somehow?

---

That's what I was originally trying to point out, actually. All hardware devices contain control registers that provide access to that information. The problem is you have to know how the OS maps to that device (used to be that I/O ports that could be accessed directly with assembly, but now I think the situation is more complicated) and the protocol required to communicate to that specific device (and USB is not simple in that respect). That said, there may be an existing library that can do all that, but I haven't personally seen one myself. Writing a device driver and USB software interface isn't exactly trivial, so I think your best bet is to find a flash-drive format tool (I've heard Hewlett-Packard has one that works for all kinds - just be careful where you download it from, obviously).

Quote:

---

You missed the point. How Windows or any other software can obtain disk size info? Its only FAT or there is some other place? There can be a readonly sector on disk to keep such info!
Imagine a fake Flash Mem like your example. It is 256MBs. But it has fake info in it(beside the label) that makes Windows think that its a 4GBs one. Can we fix this by knowing the exact disk specs and rewrite them on FAT? Can we obtain exact right specs from flash somehow?

---

I didn't post it as a solution to your problem. It's just a FYI thing :).

Quote:

---

Originally Posted by siavoshkc

Can we fix this by knowing the exact disk specs and rewrite them on FAT? Can we obtain exact right specs from flash somehow?

---

To my knowledge you get this information from the FAT. However it is not shown in itself. The first 32 bits of the FAT contain it somewhere. You need to find the offsets for the following information:

B. Bytes per Sector
S. Sectors per Cluster
C. Number of Clusters

The partition size is then a simple mathematical exercise.

Do note that there may be two offsets for C. One is 2 bytes which may read 0 in which case you'll need to find the other of 4 bytes for larger disk sizes. The FAT structure can be found a little all over the web.

Another place to get the disk size is the MBR. I'm unsure how to get to it, but just grab some documentation on its structure. I suspect this to be the best place since it contains information for all disk partitions and you won't have to be fishing around for FATs. However the structure is certainly more complex and deserves a little more study.

Quote:

---

Originally Posted by abachler

Are you sure windows is misreading it or are you just getting confused with the difference between 1 megabyte and 1 million bytes (they aren't the same).

---

Actually they are (now), you should be using the new terms "Mibibyte" if you mean 2^20 bytes :). Unless you disagree with the IEC 60027 standard.

The FAT is not the authoratative source for disk size information. Never was, never will be. How you get the actual size is probably hardware dependent, and certainly, from a program's point-of-view, OS dependent.

Quote:

---

Originally Posted by zacs7

Actually they are (now), you should be using the new terms "Mibibyte" if you mean 2^20 bytes . Unless you disagree with the IEC 60027 standard.

---

I can't stand that "standard". First, let me acknowledge by bias - these terms sound retarded as hell. "Pebibyte", I mean... really?

First, I cannot use the terms professionally. One, nobody would know what I'm talking about, and two, they would not stop laughing. "MiB" sorta works in writing, but the terms are not widespread enough.

Second, what's the point? 1,000 bytes is meaningless in most contexts. 1,024 is a round number (;-)) and it's meaningfully - generally 2 sectors, a power of 2, etc. Take the other perspective - if someone says "1 KB" or "1 kilobyte", they probably mean 1,024 bytes.

Kibibyte... right. Stupid disk manufacturers.

So, the real answer to the initial question is that nobody knows?

IOCTL_DISK_GET_DRIVE_GEOMETRY

IOCTL_DISK_GET_DRIVE_GEOMETRY

This takes the geometry of the physical disk, not whatever garbage is reported in the MBR

Quote:

---

Originally Posted by Cactus_Hugger

Kibibyte... right. Stupid disk manufacturers.

---

Yeah I kind of roll my eyes whenever i see the kibibit, mibibyte thing. Everyone knows that disk manufacturers call a megabyte 1 million bytes, not 1048576 bytes, which it actually is. They are the ONLY ones that do so, so there is no reason to rewrite all the technical documentation to accommodate their arrogance. It's really a joke that they even think they are the authority on such matters. It would be like the automotive industry thinking it is the authority on what a mile is, and deciding they would call a mile 5000 feet so they could boost their speeds and MPG ratings, then expecting everyone else in all other fields to call 5280 feet a 'myle'.

Quote:

---

It would be like the automotive industry thinking it is the authority on what a mile is, and deciding they would call a mile 5000 feet so they could boost their speeds and MPG ratings, then expecting everyone else in all other fields to call 5280 feet a 'myle'.

---

It's a little different than that. The prefixes kilo, mega, giga, existed long before computers came around. Someone can just as easily argue that it was computer engineers who modified the terms to mean something else than they originally did (kilo now means 1024 instead of 1000).

That being said, "kibibyte" just sounds stupid.

Sure "kibibyte" may sound stupid, but "kilo" is a SI-prefix for 1000 (since the year 1795 it seems), so I would say "the computer industry" got it wrong from the start. If anything the disk manufacturers have done the technically correct thing.

Excuse me for living in the 21st century with the metric system ;)

That being said, I won't be using the "kibi"-like prefixes since they're stupid, even if they are correct. :-)

Quote:

---

Originally Posted by bithub

It's a little different than that. The prefixes kilo, mega, giga, existed long before computers came around. Someone can just as easily argue that it was computer engineers who modified the terms to mean something else than they originally did (kilo now means 1024 instead of 1000).

That being said, "kibibyte" just sounds stupid.

---

except that computer engineers reassign the metric for kilo etc only within their specialized field.

Quote:

---

Originally Posted by brewbuck

IOCTL_DISK_GET_DRIVE_GEOMETRY

IOCTL_DISK_GET_DRIVE_GEOMETRY

This takes the geometry of the physical disk, not whatever garbage is reported in the MBR

---

Nice brewbuck, thanks. That definitely puts us in the right track.

Reading through it though, I feel the need to point out that the above IOCTL does use the MBR exclusively. It's also obsolete. You may be referring to IOCTL_DISK_GET_DRIVE_GEOMETRY_EX.

Quote:

---

The difference between IOCTL_DISK_GET_DRIVE_GEOMETRY_EX and the older IOCTL_DISK_GET_DRIVE_GEOMETRY request is that IOCTL_DISK_GET_DRIVE_GEOMETRY_EX can retrieve information from both Master Boot Record (MBR) and GUID Partition Table (GPT)-type partitioned media, whereas IOCTL_DISK_GET_DRIVE_GEOMETRY can only read MBR-style media.

---

In any case, this also offers the needed information on where to look in case one doesn't want to use the WinAPI.

Quote:

---

This takes the geometry of the physical disk, not whatever garbage is reported in the MBR

---

Quote:

---

The difference between IOCTL_DISK_GET_DRIVE_GEOMETRY_EX and the older IOCTL_DISK_GET_DRIVE_GEOMETRY request is that IOCTL_DISK_GET_DRIVE_GEOMETRY_EX can retrieve information from both Master Boot Record (MBR) and GUID Partition Table (GPT)-type partitioned media, whereas IOCTL_DISK_GET_DRIVE_GEOMETRY can only read MBR-style media.

---

It still doesn't say where that informaion are read from.

I don't feel like studying the MBR and GPT structures for you. Thank you very much.
I thought it would be clear by now all you need to do is study them yourself. You are bound to get this information from online references.

Start here. And after that get to the External References and Further Reading sections at the bottom.
GUID Partition Table - Wikipedia, the free encyclopedia
Master boot record - Wikipedia, the free encyclopedia

Its not what I mean. I mean does it read MBR or obtains info from another source.

I'm assuming it reads the GPT. If the first logical block (LBA 0, the Protected MBR) does not contain a partition type of 0xEE (obtained from reading the 1-byte offset at 0x04), it knows it is reading a legacy device and uses the MBR instead.

Quote:

---

Its not what I mean. I mean does it read MBR or obtains info from another source.

---

The source for the FAT filesystem driver comes with the WDK. Download it and be enlightened.

Consider this:

Quote:

---

The difference between IOCTL_DISK_GET_DRIVE_GEOMETRY_EX and the older IOCTL_DISK_GET_DRIVE_GEOMETRY request is that IOCTL_DISK_GET_DRIVE_GEOMETRY_EX can retrieve information from both Master Boot Record (MBR) and GUID Partition Table (GPT)-type partitioned media, whereas IOCTL_DISK_GET_DRIVE_GEOMETRY can only read MBR-style media.

---

It still does not clear if it reads from MBR(GUID) or somewhere else. If we persume it reads info from MBR, we should look for a more reliable source of disk info.