S3/NFS multi-protocol access to directories and files

ECS supports writing objects using the S3 protocol and accessing them as files using NFS and, conversely, writing files using NFS and accessing the files as objects using the S3 protocol. You must understand how directories are managed when you use multi-protocol access.

The S3 protocol does not make provision for the creation of folders or directories.

To enable multi-protocol operation, ECS support for the S3 protocol formalizes the use of / and creates directory objects for all intermediate paths in an object name. An object named /a/b/c.txt results in the creation of a file object named c.txt and directory objects for a and b. The directory objects are not exposed to users through the S3 protocol, and are maintained only to provide multi-protocol access and compatibility with file system-based APIs. This means that ECS can display files within a directory structure when the bucket is viewed as an NFS or HDFS file system.

Limitations

  • An issue can arise where both a directory object and a file object are created with the same name. This can occur in the following ways:
    • A file path1/path2 is created from NFS, then an object path1/path2/path3 is created from S3. Because S3 allows creation of objects that have another object's name as the prefix, this operation is valid and is supported. A file and a directory called path2 will exist.
    • A directory path1/path2 is created from NFS, then an object path1/path2 is created from S3. This operation is a valid operation from S3 because directory path1/path2 is not visible through the S3 API. A file and a directory called path2 will exist.

    To resolve this issue, requests from S3 always return the file, and requests from NFS always return the directory. However, this means that in the first case the file created by NFS is hidden by the object created by S3.

  • NFS does not support filenames with a trailing / in them, but the S3 protocol does. NFS does not show these files.