Using Object Storage

Updated at: 2025-12-12 16:03:25

Object storage is a widely used cloud storage model designed for storing and managing large amount of unstructured data. In Virtual Kubernetes Services (VKS), object storage is commonly used for staging, persistence, and data transfer workflows. We recommend using Rclone to manage your object storage resources.

Introduction to Rclone

Rclone is a powerful command-line tool for managing cloud storage. It supports synchronization, transfer, caching, and file management across a wide range of storage providers, including Google Drive, Dropbox, OneDrive, and S3-compatible services. This guide shows how to use Rclone to manage your object storage on Alaya NeW Cloud.

Prerequisite

Object Storage has been provisioned under your account.

Procedure

Download and install

Linux

1. Manually Install Rclone

#Download and install Rclone
curl https://rclone.org/install.sh | sudo bash

#Verify installation
rclone --version

2. Install Using a Package Manager

2.1 Using apt (Debian/Ubuntu)

# Update package list
sudo apt update

# Install Rclone
sudo apt install rclone

# Verify installation
rclone --version

2.2 Using yum (CentOS/RHEL)

# Install the EPEL repository (if not already installed)
sudo yum install epel-release
#Install Rclone
sudo yum install rclone
#Verify installation
rclone --version

2.3 Using dnf (Fedora)

# Install Rclone
sudo dnf install rclone
# Verify installation
rclone --version

Windows

1. Download the Installer:

   • Visit the Rclone Official Website to download the latest version of the installation package.
   • Once the download is complete, double-click the installation package to begin the installation.

2. Verify Installation:

   • Open a command prompt and enter the following command：

rclone --version

macOS

Install Using Homebrew:

# Install Rclone
brew install rclone

#  Verify Installation
rclone --version

Configuration

You can configure Rclone either via the command-line interface or by creating a configuration file (using a config file is recommended).

Configuration File

Example for Linux: create the config file at /root/.config/rclone/rclone.conf, with the following content： Note: The access_key_id and secret_access_key are provided in your object storage provisioning SMS notification. The endpoint can be found on the Object Storage details page under Resource Center-Storage Management in the web console.

[s3_store]     #Custom name for this remote
type = s3  
provider = Ceph
access_key_id = ******* #Replace with your actual access_key_id
secret_access_key = ******* #Replace with your actual secret_access_key
endpoint = https://s3.hd-01.alayanew.com:8082
acl = public-read-write
no_check_bucket = true

When editing your configuration file, replace the following variables:

Variable	Description	Source	Example
s3_store	Custom remote name	User-defined	s3_store
access_key_id	access_key_id	Provisioning SMS	Asjnf****233s
secret_access_key	secret_access_key	Provisioning SMS	sder*******nfksu9j5lw
endpoint	Service endpoint URL	Object Storage details page	http://s3.hd-01.alayanew.com:8082

config command

Run the configuration wizard:

# Run the configuration wizard
rclone config

Follow the interactive prompts:

• Choose n to create a new remote.
• Enter a name for the remote (for example: s3_store).
• Select the storage provider (for example: s3).
• Follow the prompts to provide authentication details (access key, secret key, endpoint, and etc.) or to complete any provider-specific auth flow.
• When configuration is complete, enter q to quit the wizard.

Usage

Common Commands

Command	Description
rclone lsd	List all directories or buckets in a remote.
rclone ls	List objects along with size and path.
rclone copy	Copy files from source to destination.
rclone move	Move files from source to destination.
rclone delete	Delete specified files or directories.
rclone rmdir	Remove empty directories.
rclone sync	Sync files, updating only target files.
rclone help	Show rclone help documentation.
rclone help [command_name]	Show detailed help for a specific command.
rclone listremotes	List configured remotes.
rclone config show [s3_store]	Display configuration for a remote.

Advanced Commands

Command	Description
rclone find s3_store:/remote/path --name *.txt	Search files by name pattern.
rclone copy /local/path s3_store:/remote/path --bwlimit 1M	Limit transfer speed to 1 MB/s.
rclone copy /local/path s3_store:/remote/path --retries 3	Retry failed transfers up to 3 times.
rclone copy /local/path s3_store:/remote/path --log-file transfer.log	Log transfers to a file.
rclone copy /local/path s3_store:/remote/path --progress	Show transfer progress.
rclone copy /local/path s3_store:/remote/path --include ".txt" --exclude ".log"	Include/exclude files during transfer.
rclone copy /local/path s3_store:/remote/path --transfers 4	Set number of concurrent transfers.

Advanced Features

Rclone also supports advanced capabilities such as caching and encryption. Please visit the Rclone official website for more details and usage instructions.

Important Notes

• Security: Keep your Access Key ID and Secret Access Key confidential to prevent unauthorized access.
• Performance: For large-scale data operations, use Rclone in a stable, high-bandwidth network environment to ensure optimal performance.