S3 is files, but not a filesystem (calpaterson.com)

Article summary

The article discusses the differences between Amazon S3 and a traditional filesystem, highlighting that S3 is not a filesystem despite being used as one. S3 lacks features like partial file overwrites and rename operations, which can cause issues when used with certain software. The author argues that S3's simplicity and scalability come at the cost of functionality. The article also touches on the concept of 'deep' and 'shallow' modules in software design.

Main themes

• S3 vs traditional filesystems
• Cloud storage limitations
• Software design principles
• Filesystem semantics
• Cloud computing trade-offs

What commenters say

• S3's lack of features like partial file overwrites and rename operations makes it unsuitable for certain use cases, despite its scalability and simplicity.
• The concept of a 'filesystem' is not well-defined and has evolved over time, making it difficult to determine whether S3 qualifies as one.
• S3's design is optimized for certain use cases, such as high-bandwidth reads and writes, but is not a general-purpose filesystem.
• The distinction between S3 and a traditional filesystem is not significant, as users can still use S3 as a storage solution with the right tools and abstractions.
• The idea that S3 is not a filesystem is not a criticism, but rather a recognition of its unique design and trade-offs.
• The lack of a clear definition of a 'filesystem' leads to confusion and disagreements about whether S3 qualifies as one.
• S3's limitations, such as the need to copy and delete objects to simulate a rename operation, can be mitigated with the right software and design patterns.