NFS WORM (Write Once, Read Many)

NFS data become Write Once Read Many (WORM) compliant when autocommit is implemented on it.

In detail, creating files through NFS is a multi step process. To write to a new file, NFS client first sends the CREATE request with no payload to NFS server. Upon receiving a response, the server issues a WRITE request. It is a problem for FS enabled buckets under retention as the file created with 0 bytes blocks any writes to it. Due to this reason, until ECS 3.3, retention on FS enabled bucket makes the whole mounted file-system read-only. There is no End of File (EOF) concept in NFS. Therefore, setting a retention for files, on the FS enabled buckets, after writing to them does not work as expected.

In order to remove the constraints placed on NFS files in a retention enabled bucket, the autocommit period is implemented on NFS data. For this reason, it is decided to introduce the autocommit period during which certain types of updates (for now identified as writes, Acl updates and deletes that are required for rsync, and rename that is required for Vim editor) are allowed, which removes the retention constraints for that period alone.
  • The autocommit and the Atmos retention start delay are the same.
  • Autocommit period is a bucket property like retention period.
  • Autocommit period is,
    • Applicable only for the file system enabled buckets with retention period.
    • Applicable to the buckets in noncompliant namespace.
    • Applies to only requests from NFS and Atmos.

Seal file

The seal file functionality helps to commit the file to WORM state as soon as the file is completely written ignoring the remaining autocommit period. The seal function is performed through the command: chmod ugo-w <file> on the file.

The seal functionality does not have any effect outside the retention period.

High level overview

Table 1. Autocommit terms
Term Description
Autocommit period Time interval relative to the object's last modified time during which certain retention constraints (example: file modifications, file deletions and so on) are not applied. This does not have any effect outside of the retention period.
Retention Start Delay Atmos head uses the start delay to indicate the autocommit period.

The following diagram provides an overview of the autocommit period behavior.

Highlevel auto-commit overview

Autocommit configuration

The autocommit period can be set from the user interface or bucket REST API or S3 head or Atmos subtenant API.

User Interface

The user interface has the following support during bucket create and edit:
  • When the File System is not enabled, no autocommit option is displayed.
  • When the File System is enabled /no retention value specified, autocommit is displayed but disabled.
  • When the File System is enabled/retention value selected/autocommit is displayed and enabled for selection.
Maximum autocommit period is limited to the smaller of the Bucket Retention period or the default maximum period of one day.

REST API

Create bucket REST API is modified with the new header, x-emc-autocommit-period. 
lglou063:~ # curl -i -k -T /tmp/bucket -X POST https://10.247.99.11:4443/object/bucket -H "$token" -H "Content-Type: application/xml" -v

The contents of /tmp/bucket
<object_bucket_create>
	<name>bucket2</name>
	<namespace>s3</namespace>
	<filesystem_enabled>true</filesystem_enabled>
	<autocommit_period>300</autocommit_period>
	<retention>1500</retention>
</object_bucket_create>

S3 head

Bucket creation

Bucket creation flow through s3 head can make use of optional request header, x-emc-auto-commit-period:seconds to set the autocommit period. The following checks are made in this flow:
  • Allow only positive integers
  • Settable only for file system buckets
  • Settable only when the retention value is present 
./s3curl.pl --ord --id=naveen --key=+1Zh4YC2r2puuUaj3Lbnj3u0G9qgPRj0RIWJhPxH --createbucket -- -H 'x-emc-autocommit-period:600' -H 'x-emc-file-system-access-enabled:true' -H 'x-emc-namespace:ns1' http://10.249.245.187:9020/bucket5 -v

Atmos

Atmos creates a subtenant request header, x-emc-retention-start-delay, captures the autocommit interval. 

./atmoscurl.pl -user USER1 -action PUT -pmode TID -path / -header "x-emc-retention-period:300" -header "x-emc-retention-start-delay:120" -include

Behavior of file operations

File Operation Expected within autocommit period Expected within retention period (after autocommit period)
Change permission of file Allowed Denied
Change ownership of file Allowed Denied
Write to existing file Allowed Denied
Create empty file Allowed Allowed
Create non-empty file Allowed Denied
Remove file Allowed Denied
Move file Allowed Denied
Rename file Allowed Denied
Make dir Allowed Allowed
Remove directory Denied Denied
Move directory Denied Denied
Rename directory Denied Denied
Change permission on directory Denied Denied
list Allowed Allowed
Read file Allowed Allowed
Truncate file Allowed Denied
Copy of local read only files to NFS share Allowed Allowed
Copy of read-only files from NFS share to NFS share Allowed Allowed
Change atime/mtime of file/ directory Allowed Denied