Talk:File system fragmentation

Computing Start‑class Mid‑importance

	This article is within the scope of WikiProject Computing, a collaborative effort to improve the coverage of computers, computing, and information technology on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.ComputingWikipedia:WikiProject ComputingTemplate:WikiProject ComputingComputing
Start	This article has been rated as Start-class on Wikipedia's content assessment scale.
Mid	This article has been rated as Mid-importance on the project's importance scale.

Bittorrent, etc. preallocation does not always prevent fragmentation

Consider copy on write filesystems, which use immutable allocated blocks (like ZFS) This allocation will just waste time, and will not change the characteristics of the filesystem —Preceding unsigned comment added by 129.65.102.1 (talk) 17:49, 27 February 2008 (UTC)[reply]

Specific File System

Wasn't there an attempt at a file system that attempted to avoid that by writing data in layers, from top to bottom, letting the old data drop? Could someone tell me its name please? Thanks! 84.129.170.44 (talk) —Preceding undated comment added 04:23, 5 April 2009 (UTC).[reply]

Reasons for duplicating "defragmentation" article

For the record, I created this article as I was dissatisfied with the current defragmentation article that:

Approaches the problem from the inverse perspective, talking about how to solve the problem without saying what exactly the problem is.
Uses imprecise terminology, occasionally saying "partition" instead of "file system", etc.
Includes the misconception that fragmentation only takes place on the level of individual files.
Spreads the common myth of Unix file systems not needing defragmentation, citing only unreliable sources (the majority of "reliable" research on file systems does indicate that fragmentation is a problem, and I will cite sources as necessary)

I have attempted to mitigate these somewhat, but ultimately decided to write this article. I don't know if I can get it into a good enough shape to be merged with "defragmentation" (if at all), but I will try, and I will cite genuine research in the progress. It may or may not be considered a "rewrite" at this point. Any criticisms and comments are very welcome. -- intgr 03:53, 14 December 2006 (UTC)[reply]

"Related file fragmentation"?

While I myself added this to the article, I'm not sure it is fair to consider "related file fragmentation" a type of fragmentation. While research dealing with fragmentation very often also touches the topic of keeping related files together (e.g, files in a single directory), I don't think I can recall any instances where it's actually referred to as "fragmentation" per se.

However, consider when an archive is unpacked. As all files are decompressed in order, they will likely be laid out sequentially. But when time goes on, and files are added and deleted, the directory tree in which the files were decompressed into, becomes less and less "contiguous", e.g., can be considered "fragmented". -- intgr 14:21, 19 December 2006 (UTC)[reply]

http://www.kernelthread.com/mac/apme/fragmentation/ talks about this as "User-level data fragmentation" -- intgr 10:40, 21 December 2006 (UTC)[reply]

Mac OS X

I struck the Mac OS X note, since it isn't what actually happens. Mac OS X/HFS+ do not defrag at idle. What happens is that when a fragmented file is opened, it is defragged (unless the system has been up less than some specific time, and I forget what that time is). Thus, there's no "at idle" about it. Now if there's a seperate "at idle" process, by all means put the claim back in (but please reference it). Thanks. :) --Steven Fisher 18:58, 19 December 2006 (UTC)[reply]

Thanks for correcting, I should have looked it up before mentioning; I've heard this myth from several people and it turns out not to be true indeed. :) -- intgr 19:32, 19 December 2006

Merge with defragmentation

As the article has been tagged with {{merge|Defragmentation}}, does anyone have ideas how to do that (e.g., what to merge, what not to merge, and what to throw away)? -- intgr 14:32, 30 January 2007 (UTC)[reply]

Don't Merge

Although it would seem logical to merge the "Defragmentation" and the "File system fragmentation" articles, the first will naturally focus on practical aspects of dealing with the problem, and the second on a more theoretical understanding of the root cause of the problem. Combined into one article, there is a danger that it will get overly complex -- or that important material will be deleted to keep the article simple.--69.87.193.53 19:22, 18 February 2007 (UTC)[reply]

I totally disagree...defragging is the way by which the natural order is kept...extra work that users have to do is good for their character. —The preceding unsigned comment was added by 68.249.171.240 (talk)

Don't merge. I agree with 69.87.193.53. File system fragmentation is an issue for OS/filesystem designers, while defragmentation is an issue for system administrators. They are as different as Dirt and Cleaning. --N Shar (talk • contribs) 04:18, 13 March 2007 (UTC)[reply]

I also disagree, merging the two articles would lead to one gigantic overly complex article. As stated earlier, information would probably be cut for the sake of simplicity, resulting in an incomplete article overall. Just link from the fragmentation article to the defragmentation article. --Rollerskatejamms 13:07, 13 March 2007 (UTC)[reply]

Merge Defragmentation#Causes of fragmentation with File system fragmentation

A much better method, only merges the needless copy of File system fragmentation#Cause.

Spitfire (talk) 01:35, 25 September 2008 (UTC)[reply]

posix_fallocate

There is POSIX fallocate API for allocating predermined size for file (or part of it). It have two motivations: make sure there is enaugh free space for file (so write operations will not file becuase of no space), and file can be preallocated on single extent on disk if possible in optimal manner.

Some tools already uses it to decress fratgmentation. —Preceding unsigned comment added by 149.156.67.102 (talk) 21:50, 7 December 2009 (UTC)[reply]

Infinite fragmentation

User:Codename Lisa recently changed some content in this article and introduced a concept of "infinite fragmentation", which I reverted. He/she started a discussion at User talk:Intgr#File system fragmentation, which I have moved here:

Hi.

I am a little shocked the you deleted a sentence that had a source associated with and called it original research!

Here is what the source says:

It’s also somewhat of a misconception that fragmentation is not a problem on SSDs. If an SSD gets too fragmented you can hit maximum file fragmentation (when the metadata can’t represent any more file fragments) which will result in errors when you try to write/extend a file. Furthermore, more file fragments means more metadata to process while reading/writing a file, which can lead to slower performance.

And...

Additionally, there is a maximum level of fragmentation that the file system can handle. [... Read on]

Also, as you are aware, statements in the lead need not have a footnote as long as the body has one. (WP:LEAD)

Best regards,
Codename Lisa (talk) 17:17, 30 November 2015 (UTC)[reply]

Sorry, I was perhaps too blunt. I was mainly ticked off by the weird term "infinite fragmentation" (that it appears you made up, hence "original research") and the claim that filesystems can't "sustain" some level of fragmentation.

The only source you're citing is a post on the personal blog of Scott Hanselman. Looking at his publications, he doesn't appear to be an expert on file systems. I don't claim that he's incompetent, maybe just makes oversimplifications, which I think are too misleading for an encyclopedia.

You (and/or this source) are confusing the fragmentation within the SSD device itself (in the flash translation layer, or FTL) and fragmentation of file systems. Those are two separate layers; fragmentation occurs at both layers, but FTL fragmentation is of no concern to the file system. The difference here is that SSDs must make guarantees about how much user data they're capable of storing. Perhaps it's possible for very fragmented SSDs to violate that guarantee. It sounds dubious to me, but that's not really relevant. It doesn't apply to file systems because FSes don't make such guarantees.

On most regular file systems, file data and overflow extent (fragment) mappings are allocated from the same pool of storage. What this means is, if your files are very fragmented, you'll run out of disk space slightly quicker. But the filesystems will "handle" and "sustain" it fine, nothing will break. Out of space is a well-defined state for file systems to be in. File systems don't make any guarantees about how much of the disk space will be available for file content. The "limits" of file system fragmentation are bound by disk space. -- intgr [talk] 09:07, 1 December 2015 (UTC)[reply]

Hi.

"he doesn't appear to be an expert on file systems". Skipping over the questionable "appear" part, I didn't appeal to his expertise (even though 16 published books goes long way to establish expertise). Scott Hanselman, in this case, is a secondary source in good standing for relying information on behalf of the storage team.

"You (and/or this source) are confusing the fragmentation [...]". The subject of discussion is file system fragmentation and the source is quite clear that it is Windows Defragmenter that is involved, that the operation is on volume level and that the fragment limit is an issue on both traditional and SSD storages. Furthermore, judging the source from a technical standpoint without another source is exactly original research. Finally, I have a purely unofficial FYI to add: Frankly, I think the last two paragraph are bits and peaces of random facts with highly technical words, sewn together like a Chimera, to frighten.

Best regards,

Codename Lisa (talk) 06:34, 2 December 2015 (UTC)[reply]

@Codename Lisa: Fair enough, you're welcome to ignore my explanation (original research) for how fragments are allocated on file systems. No, it wasn't meant to frighten.

Instead, let's start with you finding an actual reliable source.

Yes, he has published 16 books on the subject of programming languages -- not file systems. The reason why I pointed out that he's not an expert on file systems, is that this source fails WP:USERGENERATED. There's an exception for blogs published by experts, but does not apply to this source: "Self-published material may sometimes be acceptable when its author is an established expert whose work in the relevant field has been published by reliable third-party publications"

-- intgr [talk] 08:11, 2 December 2015 (UTC)[reply]