Docs
ufile
Common Tools
US3SYNC Migration Tool
Release History

Release Notes

v1.10.4

  • Fixed an issue where some files were not migrated when using delimit.

v1.10.1

  • Resolved a panic error in worker nodes during errors.

v1.10.0

  • Added an alert system that allows users to configure alerts via the web interface.

v1.9.1

  • Fixed a missing Content-Type: application/json header in fetch task callback requests.

v1.9.0

  • Added retry functionality for HTTP requests with incorrect body lengths.
  • Introduced error rate monitoring for nodes and real-time transfer speed display.
  • Added node disconnection alerts, with disconnected nodes marked in red on the web interface.

v1.8.0

  • Resolved an issue where S3-type endpoints could not use HTTPS.
  • Added timeout configuration for endpoint requests.

v1.7.0

  • Fixed a bug where only the top-level prefix was validated during prefix auto-discovery listing.
  • Ensured that empty directories in the source are synchronized to the destination when prefix auto-discovery is disabled.
  • Added support for Qiniu private cloud endpoints.

v1.6.1

  • Resolved an issue where endpoints could not be created on private clouds.
  • Fixed a potential hang issue during prefix auto-discovery.
  • Resolved an issue where URL list mode endpoints could not be created.

v1.6.0

  • Optimized data statistics collection.
  • Resolved a panic issue caused by empty nextmarker during S3 list fetching.
  • Fixed upload ID invalidation due to canceled multipart upload requests during retries.
  • Resolved an issue with list operations always using the same marker.
  • Added functionality to set usernames and passwords during the first login if not configured at startup.

v1.5.0

  • Introduced fetch task mode, allowing incremental file synchronization via POST requests to the /fetch endpoint.
  • Added support for specifying the --sentinel-addr parameter at startup for connecting nodes to the Pika database in sentinel mode.
  • Enabled HTTPS downloads and list fetching for AWS, US3, OSS, Qiniu, and URL list endpoint types.
  • Added automatic prefix discovery for endpoint list fetching, enabling parallel listing for each prefix.
  • Introduced the prefix-file-list API for US3 endpoints, useful in private cloud environments.
  • Enhanced multipart upload functionality with dynamic chunk size based on initialization responses.
  • Treated source 404 errors as task failures.
  • Implemented an exponential backoff strategy for worker heartbeats when disconnected from the master, with a maximum interval of 32 seconds.
  • Ensured proper abortion of incomplete multipart uploads by worker nodes.
  • Improved handling of HTTP status codes (404, 503, 500, 504) during downloads with exponential backoff retries (up to 5 times).
  • Optimized task allocation for quicker recovery after worker failures.
  • Enhanced the web interface to display file list operation progress during task listing and auditing.
  • Added regex-based filtering for source object keys during task creation.

v1.4.0

  • Introduced sync deletion, allowing users to remove files from the destination that do not exist in the source.
  • Improved the web interface by separating synchronization, deletion, and active metric displays.

v1.3.1

  • Ignored non-existent files in the source during migration.
  • Optimized URL list fetching for source file lists.
  • Fixed a panic issue caused by the “No Overwrite” option.

v1.3.0

  • Enhanced listing for multiple databases.
  • Optimized source listing for local and HTTP sources.
  • Resolved migration issues for OSS FileGroup types and empty files.
  • Added support for validating new files in non-scheduled tasks.

v1.2.0

  • Scheduled tasks no longer start immediately.
  • Added support for MD5 checksum verification during task creation.
  • Enabled prefix validation for endpoints with specific tokens.
  • Allowed skipping of 404 links in URL list migrations.
  • Optimized task distribution for improved migration speed.

v1.1.2

  • Fixed endpoint creation issues for NAS and URL list types.
  • Resolved MIME type validation failures for uarchive migrations.

v1.1.1

  • Fixed task blocking and incomplete task termination issues.
  • Enhanced startup script to validate pre-installed commands.
  • Added mandatory field validation in the front-end interface.

v1.1.0

  • Added source management capabilities.
  • Introduced scheduled tasks.
  • Supported exporting failed files.
  • Added file overwrite options.
  • Enabled rate limiting and QPS configuration.

v1.0.0

  • Supported migration sources: S3, OSS, Qiniu, Youpai, US3, NAS, and URL lists.
  • Provided web-based management for migration tasks and nodes.