Half a croissant, on a plate, with a sign in front of it saying '50c'
h a l f b a k e r y
Superficial Intelligence

idea: add, search, annotate, link, view, overview, recent, by name, random

meta: news, help, about, links, report a problem

account: browse anonymously, or get an account and write.

user:
pass:
register,


                                                                   

Non-Fragging OS

Always keeps files contiguous, to maximize system performance
  (+1)
(+1)
  [vote for,
against]

I just spent 3 days dealing with a problem on my most- frequently-used computer. A message appeared saying that I was running dangerously low on disk space. It was true. So I erased a bunch of unnecessary files and moved some other things off that drive onto a different one. But, a few hours later, the same message popped up again!

Something more permanent was needed. The Operating System was willing to compress all the files on the drive, which would approximately halve the amount of space used. OK. But before I did that, I decided to "defrag" the drive first. Those who know what I'm about to describe can skip the next three paragraphs.

There is a rather standardized/generic method that computer Operating Systems use, with respect to organizing files on a storage device. The total available storage space is divided into equal-sized segments, and a special table is used to record which segments are being used to hold which files. Large files often need many storage segments, and this is where "fragmentation" enters the picture.

The process for saving a file starts by looking for the first available storage-segment, as recorded in the special table. If multiple adjacent segments are available, fine. But if not, the table is able to indicate that the file that is located in this storage-segment continues in another distant storage-segment. It works well, and efficiently uses the total storage-space on a drive, which is why most computer Operating Systems use that method.

However, just because storage-space is used efficiently, that doesn't mean it is ACCESSED efficiently. If the read/write "head" of a disk drive has to go here and then there and then 'way over there to access all the "fragments" of a total file, that is NOT efficient. "Defrag" programs exist to re-write the data on the disk, so that most files end up using contiguous storage-segments. It can take a while to do this (I just spent more than 2 days running defrag programs over and over again, because of limited available space for shuffling files about.)

So, what if the Operating System used a modified filing method? All it really needs is a second special table. This table keeps track of the sizes of all the available contiguous storage-segments (basically the same thing as the file sizes). If some file is modified such that it becomes larger than the space into which it originally fit, the OS does NOT put most of the file here, and the rest of it over there. It puts ALL of the file over there, into a space big enough to hold it, and marks the space where it used to sit as "available contiguous storage-segments", to be used for the next file that can fit in that space.

The short-term result will of course be a lot of "holes" in the first special table, and so one could argue that this is not very space-efficient in terms of overall disk usage, even if it is very efficient in terms of accessing the files. However, a modest modification to the preceding could help reduce the problem, involving some more special data....

Using the same second special table, it would also record the "distance" (in terms of storage-segments" each file is located from a "hole" (doesn't matter if hole precedes or follows the file; distances can be negative or positive). Upon moving the too-big-to-fit file to a place where it can fit, the OS now does a simple search for another file --the nearer a hole the better-- that can fit in the just-available space, and moves it to there.

The net effect of that is that holes tend to be consolidated. If the Operating System did the preceding thing just twice for every file that got too big and was moved 'way over there, In The Long Run the total number of holes in the first special table will, statistically, probably seldom exceed a certain number (to be determined). If the OS did the thing described in the previous paragraph three times instead of twice (moving a third file into the space opened up when the second file was moved), the statistical number of holes will go down. Move 4 files, and more-hole consolidation happens, and again the total number of holes tends to go down. Possibly, that first file, moved 'way over there, can be moved "back", into the space where one of those other files (plus- adjacent-hole!) used to be.

All we need to decide is what sort of trade-off do we want, between minimizing the numbers of holes in the first special table, and how many files we want to shuffle any time one file gets too big, when the goal is to keep files non-fragmented.

Vernon, Jan 27 2014

[link]






       1) It's not the OS that determines how files are stored on the drive. That's the job of the file system.   

       2) Fragmentation isn't a problem on most modern file systems. There are techniques used to effectively prevent it, at least until the drive becomes almost full. The only reason you have to defrag regularly on Windows is because NTFS is crap.   

       3) Fragmentation is irrelevant on an SSD, and such drives should never be defragmented because it only serves to put undue wear on the drive. Not everyone is using SSDs of course, but that's the way things are going—at a minimum for OS and application storage, which is what benefits most from defragging anyway.   

       4) Your proposed method—of consolidating data and filling holes—is just a fancy way of saying continuously defrag on the fly. This is what Windows currently does, and it's an acceptable Band-Aid fix for the problems with it's broken file system, but it's not that great an idea in general. Holes are good. You want as many holes as possible, so files can grow. The strategy then is to scatter files all around the drive, with the expectation that some will grow and this leaves them room to do so. Eventually the drive will fill up, of course, but the solution is to always leave a certain percentage of free space on the drive to ensure best performance.
ytk, Jan 27 2014
  

       [ytk], regarding your #1, note that "DOS" was an acronym for "Disk Operating System". The filing system was built into the operating system. I'm not aware that that fact is significantly different for other operating systems, even though all of them do have special "file manager" programs that INVOKE the stuff built into the operating system, to actually move files about...   

       Regarding it being a Good Thing to keep holes near files, this is ONLY true for files that are likely to grow. A huge percentage of files don't do that! You would be wasting disk space unnecessarily, for files that never grow.   

       Perhaps a disk could be "divided" into two sections, one reserved for files that change size, and one reserved for files that don't. All new files are installed in the second region. As files are accessed, the OS can mark them as fixed or growing, and eventually move the fixed files to the other section of the disk, where there are never any holes....   

       And yes, I am proposing a defrag-on-the-fly system. However, I am not proposing that it be TOTALLY THOROUGH --because that obviously means the system will spend lots of time doing defrag stuff every time any file grows too big for its existing storage space, which counts against overall system performance. I'm only proposing that the defrag stuff be done just enough to keep the holes down to a reasonable number.
Vernon, Jan 27 2014
  

       // I'm not aware that that fact is significantly different for other operating systems//   

       Nowadays, "other operating systems" basically means the UNIX-ish family, and they're *not* basically extensions of a file system. Typically, each one is able to handle a range of different file systems, (even NTFS, if they're feeling indulgent). So, now you *are* aware of it.
pertinax, Jan 27 2014
  

       //note that "DOS" was an acronym for "Disk Operating System". The filing system was built into the operating system//   

       The file system was called FAT. The drivers for the FS were built into the OS, but there was no reason why you couldn't write drivers for another FS for any operating system, including DOS. Think about network shares, for example. When you write to a networked drive, the OS treats it like any other file write, but it's the remote system that actually is responsible for writing the file to disk. The OS and the FS are completely different things, and it's not correct to think of them as inseparable by any means.   

       //Regarding it being a Good Thing to keep holes near files, this is ONLY true for files that are likely to grow. A huge percentage of files don't do that! You would be wasting disk space unnecessarily, for files that never grow.//   

       Please explain how you're “wasting” disk space by doing this. You're not losing any additional space over writing them contiguously. You have exactly the same number of bytes available either way. Nothing is wasted.   

       In fact, if you insist on writing files contiguously to the drive, I'd call that “wasting” space, because you have all this extra space on the drive that you're not putting to good use as breathing room! It'd be like having a huge warehouse, but nevertheless cramming your merchandise into a single corner as tightly as possible.   

       Let me describe a simple algorithm for storing files on a disk:   

       1) Find the biggest hole on the drive.
2) Put the file halfway into that hole.
3) If there's no place big enough for the file, split it into multiple chunks and restart at step 1 for each chunk.
  

       Even if most files don't grow, that's not a problem, because some files might and these files have room to. With this algorithm, you won't have to start fragmenting files until the disk is substantially filled up, even if some files do grow in size. But this way, the average amount of “growing room” for each file decreases as files are added to the disk. This works well unless you have a ton of files that are likely to grow from very small sizes to very large sizes. Fortunately, most files don't do that.
ytk, Jan 27 2014
  

       [pertinax], OK, thanks. It doesn't bother me if the filing system, with its own appropriate special tables, qualifies as an "application" in a *nix environment.   

       [ytk], FAT stands for "file allocation table", and is exactly the "first special table" mentioned in the main text. The overall filing system is more than just the FAT, because code has to be executed that makes use of the FAT (plus keep it updated). And that code was part of the Operating System.   

       Next, you may have misinterpreted something. I'm quite aware that the last "storage segment" associated with a file can be partially filled. This Idea is NOT about trying to fill it completely, using part of another file! The "holes" I'm talking about are "completely empty storage segments" (and are directly recorded in the ordinary FAT), and nothing else. So, if a region of a disk can be reserved for files that don't change in size, they CAN be packed into that region without any of THOSE kind of holes --and none of those files would NEED any "breathing room", since they DON'T change in size!   

       Finally, you have specifically detailed fragmentation as a solution to holes in the FAT. DUH! The main reason for my writing the main text here was to prevent fragmentation of files!   

       Yes, I realize that "fragmentation of available empty space" is the result of what I proposed -- but note that it is FILES that are read by the computer, not the empty space, so the existence of holes doesn't affect performance in the way that fragmentation affects performance.   

       Anyway, I also proposed a way of minimizing holes without greatly affecting ordinary performance (because most of the time, most files are not growing in size, and the things I proposed only "kick in" when one file has grown to use up the last of the space in the last storage-segment it occupies, and needs to be moved to a different place on the disk).   

       Sure, it could be helpful if there was a hole following the file, so it could grow into that hole without being moved. But even most files that grow don't do it every day. Some only grow when a software update is done, for example. Why should those files be associated with holes if updates only get done once a year?   

       The whole point of not having holes in the FAT is to be able to say that the disk is being efficiently used to store stuff. Nobody wants to buy a new hard disk if 10% of it is still available as scattered holes, none big enough for a single unfragmented file. Fragmentation specifically exists to finish using that space. I do NOT deny its usefulness.   

       I'm just saying that fragmentation isn't necessary UNTIL the disk is almost full! And if the quantity of holes can be kept to a reasonably small number, very few files will be fragmented when the "disk full" error happens.
Vernon, Jan 27 2014
  

       //minimizing holes//   

       This is the opposite of what you want to do. You want to maximize holes. Do a bit of research into file system design and you'll see why this is the case.   

       The main point, however, is this: You are trying to solve a problem that no modern file system has, with the exception of ones used by Microsoft operating systems. There is no such thing as maintenance defragging on Macs or Linux systems. The requirement to defrag is 100% because of bad design choices on the part of Microsoft, combined with their continued use of legacy technology for reasons of backwards compatibility.   

       This problem has been solved for *decades*. Your attempt to design a filesystem that requires no defragging is roughly equivalent to designing a way of interacting with your computer that doesn't require you to write programs in assembly.
ytk, Jan 27 2014
  

       What [ytk] said, with brass knobs on.   

       WKTE and Baked.   

       [suggested-for-deletion], not a new idea.
8th of 7, Jan 27 2014
  

       //The requirement to defrag is 100% because of bad design choices on the part of Microsoft, combined with their continued use of legacy technology for reasons of backwards compatibility// Do you know if this is still true in Windows 8.1?
DIYMatt, Jan 27 2014
  

       Yes.   

       Windows. Microsoft. Bad design choices. Legacy technology.   

       What part of this don't you understand ?
8th of 7, Jan 27 2014
  

       You forgot "Ironic names"
WcW, Jan 27 2014
  

       If he has to ask the question, best not to overload him with information; he's clearly only got a little brane.
8th of 7, Jan 27 2014
  

       //Do you know if this is still true in Windows 8.1?//   

       Yes. Windows 8.1 just automatically defrags your drive for you once every week or so.
ytk, Jan 27 2014
  

       Waste of time totally useless idea. Not needed and never will be, because:   

       1. Drives are super dirt cheap now. And you can get terabytes of cloud storage for nothing. There is no excuse for running out of space. Go buy another drive, cheapskate.   

       2. While you're at it, make it a solid state drive, which eliminates the fragmentation problem.
tatterdemalion, Jan 27 2014
  

       I was disappointed that frag grenades were not involved in some way.
DenholmRicshaw, Jan 27 2014
  

       What everybody said, as long as it was about how Unix/Linux is so much better than Microsmurf stuff....
not_morrison_rm, Jan 27 2014
  

       Same thing happened to me. It was an out of control virus checker gobbling space no matter how much was freed. You are not alone.
popbottle, Jan 27 2014
  

       //You are trying to solve a problem that no modern file system has, with the exception of ones used by Microsoft operating systems.//   

       You are trying to solve a problem that every modern file system has already solved, with the exception of ones used by Microsoft operating systems.   

       FTFY.
Loris, Jan 28 2014
  

       [ytk], I misstated what I meant in a couple places. I was talking about minimizing QUANTITIES of holes, by consolidating a small number of them each time some file outgrew its existing location. Also, I mentioned something about getting a new hard disk, but neglected to specify that it would be the OLD disk that still had 10% of its space available.   

       [8th of 7] and others, OK, so the TITLE of this Idea is WKTE. But if they way those existing systems do that thing is different from the proposal here, then there is no reason to delete this.
Vernon, Jan 28 2014
  

       //[ytk], I misstated what I meant in a couple places. I was talking about minimizing QUANTITIES of holes, by consolidating a small number of them each time some file outgrew its existing location.//   

       I got that part. I'm saying that it's exactly the wrong strategy. The ideal situation is to have a hole after each file. The notion that you're wasting disk space by doing so is incorrect. The amount of disk space is fixed. If you're not using the extra empty space on the disk as “breathing room”, you're wasting it.   

       Look at it this way. If we're trying to minimize the number of holes on the drive, the optimum quantity for holes is obviously 1. So let's say we have a bunch of files arranged contiguously that fill up the first 10% of a 1TB drive, leaving a single gap of 900GB at the end. Now, let's say a file in the first 5% of the drive grows beyond the capacity assigned to it. In order to “optimize” the drive according to your scheme, you need to:

1) Read the entire contents of the file
2) Write the modified file to a scratch area at the end of the disk, and mark the original file location as empty
3) Find a file (or group of files) that fits into the gap, and read its contents
4) Write those files to the empty hole, and mark the old location of the files as empty
5) Read the contents of the original file, which is now in the scratch space
6) Write the contents of that file to the end of the contiguous group of files, and mark the scratch space empty.

Okay, so far just growing a single file involves a MINIMUM of three read and three write operations. But wait! Unless the file that was moved into the gap left by the original file happened to be the last file on the disk, we now have a gap open where that file previously was. Okay, loop back to step 3, and repeat until all of the files are optimized. So a write operation to a single file at the head of the disk could potentially involve moving *every single file on the disk*, and on average will require moving roughly *half* of them. The only alternative is to ignore these small gaps, but then you're back to your original problem of increasing the likelihood of fragmentation as more files are added to the disk. Or you could just not write to these small gaps, but then you really are wasting space.

And what if a file shrinks, or is deleted? Then you have the same problem. You're left with a tiny gap, and you have to start juggling files around to fill it. If you delete the first file on the drive, hope you didn't have anything to do for the next 6 hours or so while we consolidate gaps for you.

Compare this to the process of growing a single file where files are spread out evenly on the drive, so every file has a maximum amount of space to grow:

1) Expand the file.
2) If the file needs to expand beyond the available gap, read the file that's in the way, and put it in the middle of the next largest gap. Repeat from step 1 as necessary.
  

       The minimum number of operations is a single write, and the average number of operations will depend on the size of the files you're generally writing, but for the vast majority of use cases will be fairly low until the drive is almost full.
ytk, Jan 28 2014
  

       permanent vs ephemeral (existence), static vs dynamic (size).   

       Basically you want 4 partitions/drives to roughly separate chaff from wheat.   

       C: OS, Program files.
D: Ephemerals: swap, spooler, internet cache, downloads, system restore points, logfiles, etc.
E: Personal: personal. You can move "My Documents" here.
F: Media: static files which aren't being changed or deleted. This could include the mp3 collection, Shakespeare's works in .pdf, etc. etc.
  

       M$ won't let you do it properly; there will still be loose dynamic files in C:, but it keeps fragmentation down to almost nothing.   

       D: doesn't need to be defragmented - all the files there are only going to be read a couple times then deleted anyways.   

       E: won't be too fragmented; depends on what you do.   

       F: will never be fragmented so it never needs to be defragmented.   

       If you have enough RAM you can ditch the swapfile and internet-cache completely and move all the per-session temp files to a ramdisk.
FlyingToaster, Jan 28 2014
  

       [ytk], the main text here only specified moving 3 or 4 different files around, when it happens that the first one outgrew its space. I also specified a special table to record file-sizes and such, so no need to scan the whole disk every time some file gets too big; all the needed data is already in that special table. I was relying on statistics to keep the total number of holes to a minimum. So, sure, I knew that some performance would be lost in moving the 3 or 4 files around. But ONLY when one file gets too big. Currently, with lots of fragmented files, you almost always have a performance hit. I think it is a fair trade, to accept an occasional performance hit in exchange for having no performance hit most of the time.   

       Also, you seem to have missed the point that some files almost never change their size. If you download a .PDF for reference purposes, you are not planning on editing it. Therefore its size should never change. What does it need a hole in the filing system for? Lots of files are in that category, of not needing any extra space (beyond the typical bit of space in a single storage- segment, only partly used by the last part of the file).   

       So, in the scheme I proposed, files that never need to grow will STAY contiguous and stay unassociated with special holes in the filing system. Files that do need to grow will probably be associated with holes, one way or another.
Vernon, Jan 28 2014
  

       interesting [+]   

       So what do you do when you're downloading a file that you've no clue what size it is.
FlyingToaster, Jan 28 2014
  

       The whole discussion is blinkered by the physical limitations of your current rotating magnetic media, which is not "direct access" but "pseudo-direct semi-sequential indexed access with variable access time".   

       And as such, is about as useful as a detailed dissertation on improved methods of flint-knapping*, and therefore of interest only to experimental archaeologists, survivalists, and owners of muzzle- loading flintlock firearms.   

       *Officially, flint-knapping is the world's oldest profession, which has led to some rather embarrassing misunderstandings.
8th of 7, Jan 28 2014
  

       // you can get terabytes of cloud storage for nothing //   

       If you live in a place with high speed internet access. Too often these days I run up against tech that has clearly been designed by people who assume the entire world has overlapping 4G wireless bubbles and fiberoptic landlines.
Alterother, Jan 28 2014
  

       I'm going back to storing the zeros and ones perpendicular to to the surface of whatever the storage medium is, it's the most logical solution.
not_morrison_rm, Jan 28 2014
  

       [8th of 7] I do think that if you look into zoology far enough, you can find a number of examples of males offering females tasty morsels in exchange for mating privileges. No flint-knapping needed.   

       [Flying Toaster], at least one browser appears to download a "temporary file" and doesn't save the file to the official download location until it is fully downloaded. That makes some sense, in case the connection breaks during the download. And of course if the download succeeds, then its full file- size would be known at that time....
Vernon, Jan 28 2014
  

       //[ytk], the main text here only specified moving 3 or 4 different files around, when it happens that the first one outgrew its space.//   

       That doesn't make any sense. You can't have it both ways—either you're moving all the files around to fill all the gaps, or you're leaving the gaps in place, leading eventually to fragmentation.   

       //I also specified a special table to record file-sizes and such, so no need to scan the whole disk every time some file gets too big; all the needed data is already in that special table.//   

       This doesn't mean anything. Of course there's a special table that specifies where the holes are—that's how the file system finds the data it's looking for. You don't need a special table just to list the holes—you look at the table of contents for where there's empty space. The notion of a special table for keeping track of where there *isn't* data is completely redundant.   

       I never said anything about scanning the whole disk. Scanning the disk isn't an issue. The problem has nothing to do with finding where the empty space is on the disk, and everything to do with consolidating the empty space. The only way to do that is by rewriting entire files.   

       //So, sure, I knew that some performance would be lost in moving the 3 or 4 files around. But ONLY when one file gets too big.//   

       Or when you delete a file. Guess what's left in that space? A gap. And how do you fill that gap? With another file. And what do you put in the gap left by THAT file? Yet another file. And so on until the disk is completely optimized. Repeat for the entire disk any time a gap opens up—i.e., whenever a file grows, shrinks, or is deleted.   

       //Currently, with lots of fragmented files, you almost always have a performance hit. I think it is a fair trade, to accept an occasional performance hit in exchange for having no performance hit most of the time.//   

       Except you'll have a *huge* performance hit every time you expand a file, or delete a file, or do anything that makes a gap. Your disk will be thrashing like crazy trying to move stuff around in an obsessive-compulsive attempt to stuff all of the data contiguously at the head of the disk.   

       //Also, you seem to have missed the point that some files almost never change their size.//   

       But some do, and the system has no real way of knowing in advance which ones will and which ones won't. Better to leave as much space as possible for every file, in case it does grow. Then you only need to start moving data around for files that grow larger than their allotted space. And as you add files to the disk, the allotted space for each file decreases.   

       //If you download a .PDF for reference purposes, you are not planning on editing it. Therefore its size should never change.//   

       Unless you delete it, then its size changes to zero.   

       //What does it need a hole in the filing system for?//   

       Because if you don't need the space, eventually you'll be able to stick another file in there. And if you delete the file, you don't need to move anything around to replace it with a larger file without fragmenting.   

       //Files that do need to grow will probably be associated with holes, one way or another.//   

       You never specified any way to determine which files will need to grow in the future. How is the system supposed to know that?   

       Anyway, why is it important to keep free space contiguous? If you have a 1TB disk that's 10% full, you're not going to suddenly write an 800 GB file to it. So there's no reason to have your free space contiguous. But you might well write a bunch of smaller files, and if your 100GB of data is spread around the drive, you'll have no trouble finding places to put these files. And as the drive fills up, if you have a bunch of gaps it's easy to move smaller files into the gaps between other files to clear up a contiguous zone of free space on an as-needed basis.   

       I'm sorry, but this is just a completely unworkable idea. The only reason it's not MFD WKTE is because it's so demonstrably flawed that nobody would ever implement this. Even Microsoft wouldn't do this, and that's saying something.
ytk, Jan 28 2014
  

       [V] Regards downloading, if both Temp (where the temp file is located) and the end-location (where you told it you want it to go) directories are on the same disk, the physical location remains in the same place: the filesystem simply jiggers the FAT so it appears in the new directory while disappearing from the temp.   

       By contrast your system would download the file into the biggest hole then move it to the best fit location after the download was finished.   

       I wonder if a pattern would appear on-disk after awhile of using your system.
FlyingToaster, Jan 29 2014
  

       [ytk], what I'm talking about DOES make sense. When I last looked at the details of a FAT, I noticed that it didn't care how long a file was. Each file was associated with a start-location in the FAT, and the table itself simply recorded the next "block" where the file continued, with a flag- bit indicating when the last block was reached.   

       So, that second special table I mentioned WOULD BE primarily a table of file-lengths and the amount of FAT associated with each file (including any extra unused data-segments, --holes, that is). It means that when a big hole opens up in the FAT, like when a file is deleted, it can be pretty easy to find a file that could fit into the hole, using the second table.   

       I specifically indicated that we want to seek a file- that-fits that happens to already be associated with a hole. That means when the file is moved, the available hole becomes bigger than before, when the original file was deleted. If you can do this 2 or 3 times, then you will have consolidated multiple holes, and consequently the total number of holes in the Fat will have gone down, after the file was deleted and those few files got moved around.   

       If this is done every time a file is deleted, or moved because of being too big to fit after the latest edit, then certainly the FAT will acquire some holes. But STATISTICALLY, in the long run, holes will be consolidated about as often as they get created. The net result is a fairly small and relatively constant number of holes. Should you ever decide to do a "full defrag", it shouldn't take a huge amount of time, certainly not when compared to what I wrote at the start of the main text here!   

       Remember the Title of this Idea. It is about keeping files non-fragmented. It is also about preventing lots of holes, NOT so much about ensuring their number is always Zero. I think that what I've described would work to accomplish what I set out to accomplish.   

       You keep talking about things like a "need" for a vast number of holes, or a "need" to waste vast amounts of system performance to prevent all holes, and I've been describing a compromise with regard to holes, while staying focused on the main goal of keeping each file contiguous (with respect to itself) on the disk, to maximize the performance associated with file-loads and saves.   

       [Flying Toaster], you may be right about the temp file, but it may depend on the browser and the computer. It might be saved in RAM, for example, until the download is finished.
Vernon, Jan 29 2014
  

       //[ytk], what I'm talking about DOES make sense.//   

       No. It makes no sense to keep a redundant table of where files *aren't*, when you have a perfectly good table of where files *are*.   

       You're clearly not a programmer, or you'd realize that what you're describing is a classic anti-pattern. You *never* want to have two different sources describing the same set of data, because then you face the problem of determining which one is authoritative. What if the file allocation table gets updated, but the free space allocation table doesn't for some reason? Say the power gets cut after updating one table but not the other. Well, then you're looking at potential data loss when the empty space table says “go ahead and write there” but the file allocation table says “there's a file there”. So if the solution is to verify each table against the other, then you're faced with another problem—which table is correct? Okay, for safety you could say the file allocation table must always be correct, in which case why bother with the free space table at all? If every lookup in the free space table has to be verified against the allocation table, then the free space table does you no good.   

       //When I last looked at the details of a FAT, I noticed that it didn't care how long a file was. Each file was associated with a start-location in the FAT, and the table itself simply recorded the next "block" where the file continued, with a flag- bit indicating when the last block was reached.//   

       FAT is an ancient technology. Even Microsoft doesn't use FAT anymore, except to maintain backwards compatibility. And it's for this reason that it's futile to attempt to improve on the FAT filesystem—any “improvement” would be incompatible. If you want to eliminate the problem of file fragmentation, you have to switch to another filesystem that's not compatible with FAT.   

       You still haven't explained *why* you want to get rid of holes. You can't just assert that holes lead to fragmentation, because they simply don't. The proof of this is that NO MODERN FILESYSTEM HAS A PROBLEM WITH FRAGMENTATION.
ytk, Jan 29 2014
  

       //You're clearly not a programmer, or you'd realize that what you're describing is a classic anti-pattern. You *never* want to have two different sources describing the same set of data, because then you face the problem of determining which one is authoritative.//   

       Um, never say never.
In fact, it's pretty common in various forms. How you cope with potential disagreements varies wildly depending on the case in hand.
"All programming is an exercise in caching."
-Terje Mathisen (apparently)
  

       As a related example, I remember reading a description of a memory allocation system which stored the allocated blocks (as it must), but also stored details about the empty space in the empty blocks themselves.   

       For what it's worth, I think you're arguing over a difference in terminology. ytk is saying there should be gaps between files. Vernon is saying that each file should be written as a contiguous block.
I think you're both mostly right.
Loris, Jan 29 2014
  

       [ytk], if what I've proposed counts as a fix for something that is obsolete and has been replaced by things that work differently from this proposal, then obviously the thing I proposed counts, perfectly, as Half-Baked!
Vernon, Jan 31 2014
  

       //counts as a fix//   

       Counts as a “fix”.
ytk, Jan 31 2014
  
      
[annotate]
  


 

back: main index

business  computer  culture  fashion  food  halfbakery  home  other  product  public  science  sport  vehicle