What is the best filesystem for storing thousands of files in one dictionary-like id-blob structure?

Posted by Ivan on Server Fault See other posts from Server Fault or by Ivan
Published on 2010-05-11T18:41:04Z Indexed on 2010/05/11 18:44 UTC
Read the original article Hit count: 254

Filed under:
|

What filesystem best suits my needs?

  1. Thousands or even millions of files in one directory.
  2. Good (ext4 & ntfs level or close) reliability (incl. fault tolerance) and access speed.
  3. No directories actually needed, as well as descriptive names, just a dictionary-like structure of id-blob pairs is all I need.
  4. No links, attributes, and access control features needed.

The purpose is a file storage where all the metadata (data describing all the facts about what the file actually contains and who can access it) is stored in a MySQL database.

As far as I know common filesystems like NTFS and ext3/4 can go dead-slow if there are too many files placed in one directory - that's why I ask.

© Server Fault or respective owner

Related posts about filesystems

Related posts about storage