S3 Files (allthingsdistributed.com)

Article summary

The article discusses the introduction of S3 Files, a new feature that integrates Amazon Elastic File System (EFS) into S3, allowing users to access S3 data as a network-attached file system. This feature aims to simplify the process of working with large amounts of data and reduce the friction between file systems and object storage. The article also explores the design and development of S3 Files, highlighting the technical challenges and compromises made to achieve this integration. The goal of S3 Files is to provide a seamless experience for users who want to work with S3 data using file system APIs.

Main themes

• S3 Files
• Object Storage
• File Systems
• Data Management
• Cloud Computing
• Storage Solutions

What commenters say

• The introduction of S3 Files is seen as a significant improvement for users who have been using S3 as a file system despite its original intent as an object store.
• Some commenters view S3 Files as a way for AWS to profit from users who want to use S3 as a file system, by charging for EFS as a cache layer.
• Others argue that S3 Files is a necessary development to support the growing number of users who want to use S3 as a file system, and that it provides a more convenient and supported way to do so.
• There are concerns about the pricing model for S3 Files, particularly the cost of writes and reads that hit the EFS cache, which could be a dealbreaker for write-heavy applications.
• Some commenters note that S3 Files is not a new concept, and that similar solutions have existed for decades, but the integration with EFS provides a more seamless experience.
• The use of EFS as a cache layer raises questions about the consistency model and the potential for data loss or corruption.
• The threshold for using the cache layer is configurable, but some commenters question whether this threshold is optimal for all use cases, particularly for frequent random access to large files.