AWS S3 Protocol Support Description

Overview

The S3 protocol is introduced by AWS and has become the de facto standard in the object storage industry. On the basis of its own standard, the US3 product has added compatibility support for the S3 v4 protocol standard.

Supported APIs

The current support status of the US3’s S3 protocol module for the standard S3 protocol is as follows:

Please note:

• PutObject currently only supports 1GB file size, if you need to upload files larger than 1GB, please use the multipart upload API

• PostObject currently only supports a maximum file upload of 32MB

• CopyObject currently only supports a maximum file copy of 100MB

• UploadPart currently only supports 8MB fixed-length part size (the last part is allowed to be less than 8MB). If there is a requirement for variable-length parts, please contact technical support

• US3 S3 storage types compatible with AWS S3 and their conversion rules refer to Storage Type Conversion Rules

• The ETag calculation method of US3 is slightly different from that of AWS S3, it is recommended not to rely on this ETag

• S3 API’s MD5 verification is currently not supported, it is recommended to turn off:

For example, AWS S3 Java SDK: System.setProperty(SkipMd5CheckStrategy.DISABLEGETOBJECTMD5VALIDATION_PROPERTY,"");

System.setProperty(SkipMd5CheckStrategy.DISABLEPUTOBJECTMD5VALIDATION_PROPERTY,"");

• The access control (ACL) definition of US3 is different from that of AWS S3, for details, please refer to Access Control Definition (ACL)

• Currently, the file access control API (GetObjectAcl, PutObjectAcl) is only supported in some regions

• Currently, the life cycle configuration rule control API (PutBucketLifecycleConfiguration, GetBucketLifecycleConfiguration, DeleteBucketLifecycle) is only supported in some regions

• Currently, UploadPartCopy is in the internal testing stage. If there is a usage requirement, please contact technical support

• The versioning feature (Versioning) is currently not supported

• The tagging feature (Tagging) is currently not supported

• The max-keys parameter in ListObjects request (the maximum number of objects returned by the request) has a maximum value of 5000

Access Control Definition（ACL）

Only Support Signature V4

Scenarios supporting V4 signatures:

1. Carry parameters in the URL (x-amz-credential field in the URL Query section);

2. POST (the x-amz-credential field in the form);

3. Carry parameters in the Header (Authorization field);

Explanation of S3 AccessKeyID and SecretAccessKey

The AccessKeyID (or AccessKey) and SecretAccessKey (or SecretKey) of S3 correspond to the public and private keys of the UCloud Global API, or the public and private keys of the Token provided by the US3 service;

Note: It is required that whether it is an API key or a Token key, the operation of the bucket must meet the following conditions:

• The account that created the bucket must match the owner of the API key;

• The account that created the bucket must match the account that created the Token;

S3 Part Size Description

1. To achieve better transmission performance, 8M size parts are supported by default.
2. Some regions have opened dynamic part function. If the fixed 8M part cannot meet the demand, customers can contact technical support to enable dynamic part function.

API Supports Path Style and Virtual Host Style

Path style format: http://\${Endpoint}/\${bucket name}/\${key name} , The bucket name is used as part of the path. For example, the AWS S3 Java SDK uses the US3 S3 service in the UCloud Global Beijing region via the public network as follows:

“AWSCredential credentials = new BasicAWSCredentials(ACCESS_KEY, SECRET_KEY); ClientConfiguration clientConfig = new ClientConfiguration(); … S3ClientOptions clientOptions = S3ClientOptions.builder().build(); clientOptions.setPathStyleAccess(true); // Indicate to use the path style API AmazonS3 conn = new AmazonS3Client(credential, clientConfig); conn.setS3ClientOptions(clientOptions); conn.setEndpoint(“s3-cn-bj.example.com”);“

Virtual Host style: http://${bucket name}.${Endpoint}/${key name}, similar to the current URL format used by US3.

Access Domain Name (Endpoint)

The general syntax of the access domain name is as follows:

protocol://s3-<region_code>.<custom_domain>

For example, http://s3-cn-sh2.example.com is the public network access domain name of the S3 service in the Shanghai region of China provided by example. In which, s3 is the <service_code> of the AWS S3 service, cn-sh2 is the <region_code>, and .example.com is the <custom_domain>.

For intranet access domain names, you can use the following format:

protocol://internal.s3-<region_code>.<custom_domain>

The service_code for intranet access needs to be preceded by the intranet access identifier internal. That is, the access domain name in the above example is changed to http://internal.s3-cn-sh2.example.com

Note: Currently North China 1, Hong Kong, Ho Chi Minh, Seoul, Sao Paulo, Los Angeles, and Washington regions already support https protocol, other regions can support path style https, and virtual host style https will be supported in the future (https is not supported in all regions)

About region_code.custom_domain specifications

Enter the object storage module of the console, find the following label under the single region space management page <Bucket Domain> The label is composed of <bucket name>.<region_code>.<custom_domain>

Callback Extension Function Support

√:Supported ×:Not Supported

Storage Type Conversion Rules