Object Subsystem (ObjectNode)

The object subsystem provides S3-compatible object storage interface. It makes possible for ChubaoFS becomes a fusion storage that expose two type interface (POSIX and S3-compatible). So that user can operate files stored in ChubaoFS by using native Amazon S3 SDKs.

Structure

../_images/cfs-object-subsystem-structure.png

The ObjectNode is a functional subsystem node. It fetch volume view (volume topology) on demand from Resource Manager (Master). Each ObjectNode communicate with metadata subsystem (MetaNode) and data subsystem (DataNode) directly.

ObjectNode is stateless design with high scalability and it have ability to operate all files stored in the ChubaoFS cluster which it belong to directly without any volume-mount operation.

Features

  • Provides object storage interface compatible with native Amazon S3 SDKs.
  • Fusion storage expose two type interface (POSIX and S3-compatible).
  • Stateless and high scalability

Semantic Transform

Based on our POSIX-compatible design. Every file operate request comes from object storage interface need to be made semantic transform to POSIX.

POSIX Object Storage
Cluster Region
Volume Bucket
Path Key

Example:

../_images/cfs-object-subsystem-semantic.png

Put object ‘example/a/b.txt’ will be create and write data to file ‘/a/b.txt’ in volume ‘example’.

User

Before using the object storage service, you need to create a user through the Master. While creating users, AccessKey and SecretKey will be generated for each user.

ChubaoFS uses the field Owner of the volume as the user ID. There are two ways to create users:

  • When creating a volume through the API of the Master, if there is no user with the same name as the owner of the volume in the cluster, a user with the user ID of Owner will be automatically created.
  • Create a user by calling the user management API of the Master.

Authentication

The signature validation algorithm in object storage interface is fully compatible with native Amazon S3 service. The authentication consisting of AccessKey and SecretKey generated by Resource Manager (Master) with user creation, which can be obtained through the Master API. The AccessKey is a 16-character string unique in the entire ChubaoFS cluster.

The user has all access permissions to the volume owned by him. Users can grant other users specified permissions to access volumes under their own names. The permissions are divided into the following three categories:

  • Readonly or readwrite permission.
  • Permission for a single operation, such as GetObject, PutObject, etc.
  • Custom permission.

When a user uses the object storage service to execute a certain operation, ChubaoFS will identify whether the user has the corresponding permission.

Invisible Temporary Data

In order to make write operation in object storage interface atomically. Every write operation will create and write data to an invisible temporary. The volume operator in ObjectNode puts file data into temporary which only have ‘inode’ without ‘dentry’ in metadata. When all the file data stored successfully, the volume operator create or update ‘dentry’ in metadata makes it visible to users.

Object Mode Conflict (Important)

The POSIX and object storage are two different types of storage product, and the object storage is a Key-Value pair storage service. So that the object with key ‘a/b/c’ and the object with key ‘a/b’ are different object without any conflict.

But ChubaoFS is based on POSIX design. According to semantic transformation rule, the ‘b’ part in key ‘a/b/c’ transform to folder ‘b’ under the folder ‘a’ , and in key ‘a/b’ transform to file ‘b’ under the folder ‘a’.

The object key like this is conflict in ChubaoFS.

When the object being operated has conflicts with the existing object due to the above mode inconsistency, the ObjectNode will return a 409 status code to the client.

Supported S3 Features

  • File object operations.
  • Directory object operations.
  • Multipart upload.
  • Parallel download for high-level SDK APIs.
  • Tagging for bucket and object.
  • User-defined metadata for object.
  • IP address and network segment black and white list for bucket ACL.
  • Signature Algorithm V2 and V4.
  • Cross-Origin Resource Sharing (CORS).

Unsupported S3 Features

  • Version
  • Restore deleted objects
  • Locking objects
  • Lifecycle configuration for bucket and object.
  • Hosting Websites
  • Encryption
  • BitTorrent

Supported APIs

API Reference
AbortMultipartUpload https://docs.aws.amazon.com/AmazonS3/latest/API/API_AbortMultipartUpload.html
CompleteMultipartUpload https://docs.aws.amazon.com/AmazonS3/latest/API/API_CompleteMultipartUpload.html
CopyObject https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html
CreateBucket https://docs.aws.amazon.com/AmazonS3/latest/API/API_CreateBucket.html
CreateMultipartUpload https://docs.aws.amazon.com/AmazonS3/latest/API/API_CreateMultipartUpload.html
DeleteBucket https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucket.html
DeleteBucketCors https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketCors.html
DeleteBucketPolicy https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketPolicy.html
DeleteBucketTagging https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketTagging.html
DeleteObject https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObject.html
DeleteObjects https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObjects.html
DeleteObjectTagging https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObjectTagging.html
GetBucketAcl https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketAcl.html
GetBucketCors https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketCors.html
GetBucketLocation https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketLocation.html
GetBucketPolicy https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketPolicy.html
GetBucketTagging https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketTagging.html
GetObject https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html
GetObjectAcl https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectAcl.html
GetObjectTagging https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectTagging.html
HeadBucket https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadBucket.html
HeadObject https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadObject.html
ListBuckets https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListBuckets.html
ListMultipartUploads https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListMultipartUploads.html
ListObjects https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjects.html
ListObjectsV2 https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html
ListParts https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListParts.html
PutBucketAcl https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketAcl.html
PutBucketCors https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketCors.html
PutBucketPolicy https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketPolicy.html
PutBucketTagging https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketTagging.html
PutObject https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html
PutObjectAcl https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectAcl.html
PutObjectTagging https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectTagging.html
UploadPart https://docs.aws.amazon.com/AmazonS3/latest/API/API_UploadPart.html

Supported SDKs

Object Node provides S3-compatible object storage interface, so that you can operate files by using native Amazon S3 SDKs.

Name Language Link
AWS SDK for Java Java https://aws.amazon.com/sdk-for-java/
AWS SDK for JavaScript JavaScript https://aws.amazon.com/sdk-for-browser/
AWS SDK for JavaScript in Node.js JavaScript https://aws.amazon.com/sdk-for-node-js/
AWS SDK for Go Go https://docs.aws.amazon.com/sdk-for-go/
AWS SDK for PHP PHP https://aws.amazon.com/sdk-for-php/
AWS SDK for Ruby Ruby https://aws.amazon.com/sdk-for-ruby/
AWS SDK for .NET .NET https://aws.amazon.com/sdk-for-net/
AWS SDK for C++ C++ https://aws.amazon.com/sdk-for-cpp/
Boto3 Python http://boto.cloudhackers.com