AWS S3 (Simple Storage Service) is a highly scalable and durable object storage service offered by Amazon Web Services (AWS).
AWS S3 is one of the most used service offerings of AWS and it is built to provide high-performance and zero-latency object(file) storage with security standards and compliance like HIPAA, PCI-DSS, FedRAMP
We often use the AWS Management Console to browse S3 and complete simple tasks there are other programmatic ways such as CLI, SDKs (boto)
In addition to the AWS Management Console and SDKs, AWS S3 also offers two Command Line Interface (CLI) tool, to interact with S3 using commands in the terminal.
Both these commands are indispensable and have their own features and use cases to solve.
In this article, we will explore the AWS S3API CLI command and provide examples of how to use it effectively to interact with your S3 buckets and objects.
We will also see the difference of AWS S3 and S3API command
Before we move on, S3API CLI is part of the AWS CLI suite. You need to install AWS CLI to use the both S3 and S3API commands
Use the following instructions to set up your programmatic access and install and configure AWS CLI
Setup your Programmatic Access - Create Access Key
If you would like to create a new user in IAM along with the Access Key follow these steps.
Login to AWS Console
In the services go to IAM
Create a User and Click on map existing Policies
Choose Username and Select the Policy (Administrator Access Policy)
The final Stage would present the AccessKEY and Secret Access like given below.
If you would like to Choose the existing user and create an Access Key follow this
- Sign in to the AWS Management Console and open the IAM console at https://console.aws.amazon.com/iam/.
- In the navigation pane, choose Users.
- Choose the name of the user whose access keys you want to create, and then choose the Security credentials tab.
- In the Access Keys section, choose to Create an access key.
- To view the new access key pair, choose Show. You will not have access to the secret access key again after this dialogue box closes. ( Refer the image given above)
Install AWS CLI
Based on your base machine the AWS CLI installation and command might vary.
AWS has given clear instructions on how to install AWS CLI on each platform. Choose any of the following links and get your AWS CLI installed and ready
- Installing AWS CLI version 2 on Linux or Unix
- Installing AWS CLI version 2 on macOS
- Installing AWS CLI version 2 on Windows
Configure AWS CLI
I presume that you have installed the AWS CLI package and that everything went well.
You should be able to see the version of the AWS CLI installed when entering the following command in your terminal or command prompt
I am using the AWS CLI Version1 as CLI Version 2 is still on Beta.
Now it is time to configure the AWS CLI, Just enter the following command and you would be prompted a few questions about the Access Key and Passwords.
it would look like this as you are setting it up.
You enter your own
AWS Access Key ID and
Secret Access Key and the one is given below is not correct. Just a made-up.
➜ ~ aws configure AWS Access Key ID [None]: AKIAS790KQGK63WUK6T5 AWS Secret Access Key [None]: kkQEiBjJSKrDkWBLO9G/JJKQWIOKL/CpHjMGyoiJWW Default region name [None]: us-east-1 Default output format [None]:
Well done. You are ready with AWS CLI
Quick Syntax of AWS CLI Commands
Before we are going in further, let me quickly give you the syntax of AWS CLI commands
aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
aws CLI should be invoked with a command and a subcommand. AWS CLI provides us with a huge list of commands and their associated subcommands and their documentation is also awesome.
Before we move on to our objective s3API command let us see the difference between S3 and S3API
AWS CLI S3 vs S3API - What's the difference?
There is a high chance you might be already aware of similar AWS CLI command named S3.
We often get this question, what is the difference between S3 and S3API
To start with, S3API is a direct invocation command of low-level API if you look at the commands of s3API, All of them would have names like PUT, GET, and HEAD as close resemblance to HTTP REST API
The S3API command provides more options and features than the S3 command. It is fair advanced and covers almost all the S3 administration/management tasks
It has around 95 subcommands as of Jun 2023
I have put all of them in the following image for your glance
On the other hand, The S3 command is an abstraction for those who do not want to perform multiple lower API calls for a task
For example, If you want to SYNC two s3 buckets. aws s3 can easily do it with
aws s3 sync command while the s3api has to do it the hard way
AWS S3 CLI has a set of curated commands to address the frequent use cases like list, copy, move etc
These are the list of commands available with aws S3 while aws S3API
- cp - Copy objects
- mv - Move Objects
- ls - List Objects
- sync - to Sync Buckets
- mb - Create a Bucket
- rb - Remove a Bucket
- presign - Create a resigned URL for an object
- rm - remove an object
You can just do the match 95 vs 6 subcommands. As you see the s3API is fair advanced than aws s3 but it has curated workspaces for easy use which makes both indispensable
S3API CLI Command Examples
The S3API CLI command allows you to perform a wide range of operations on your S3 buckets and objects. It provides low-level access to S3 functionality and can be useful for advanced use cases. The basic syntax for the S3API CLI command is as follows:
aws s3api <operation> [options]
<operation> represents the specific S3 operation you want to perform, and
[options] are additional parameters or flags specific to that operation?
Now, let's explore some common S3API CLI commands with examples:
Creating an S3 Bucket using S3API with create-bucket
To create an S3 bucket using the S3API CLI command, you can use the
create-bucket operation. Specify the bucket name using the
--bucket parameter and the region using the
--region parameter. Here's an example:
aws s3api create-bucket --bucket my-bucket-name --region us-east-2
Uploading an Object to an S3 Bucket using S3API put-object
To upload a file or object to an S3 bucket, you can use the
put-object operation. Provide the bucket name using the
--bucket parameter and the file path using the
--body parameter. Optionally, you can specify the object key using the
--key parameter. Here's an example:
aws s3api put-object --bucket my-bucket-name --key path/to/my-object.txt --body /path/to/local-file.txt
Listing Objects in an S3 Bucket S3API list-objects
To list all objects in an S3 bucket, you can use the
list-objects operation. Specify the bucket name using the
--bucket parameter. This command will return a list of objects along with their metadata. Here's an example:
aws s3api list-objects --bucket my-bucket-name
Downloading an Object from an S3 Bucket with S3API get-object
To download an object from an S3 bucket to your local machine, you can use the
get-object operation. Provide the bucket name using the
--bucket parameter and the object key using the
--key parameter. Specify the local file path where the object will be saved using the
--output parameter. Here's an example:
aws s3api get-object --bucket my-bucket-name --key path/to/my-object.txt --output /path/to/local-file.txt
Get Object/file metadata with head-object
To fetch the document metadata you can use the
head-object command of s3api, this is to simply fetch the object's metadata
If the given object does not exist it would return 404 which is an indication that the requested object is not present in the bucket
Mostly we use this to validate if a certain object/file is present or not before we take any subsequent action like deleting or overwriting etc
Here is the sample command you can use to try.
aws s3api head-object --bucket my-bucket-name --key index.html
The preceding command fetches the metadata of the file index.html if the object is present otherwise it would return Object not found with 404
Deleting an Object from an S3 Bucket with delete-object
To delete an object from an S3 bucket, you can use the
delete-object operation. Specify the bucket name using the
--bucket parameter and the object key using the
--key parameter. Here's an example:
aws s3api delete-object --bucket my-bucket-name --key path/to/my-object.txt
These examples provide a glimpse of what you can achieve with the S3API CLI command. However, AWS S3API offers a wide range of operations, including managing bucket policies, setting access control lists (ACLs), and configuring bucket notifications, among others.
We have listed all the available subcommands of S3API earlier, Refer to the AWS CLI documentation for a complete list of S3API operations and their respective options.
The AWS S3API CLI command is a powerful tool that enables you to interact with your S3 buckets and objects using a command-line interface. In this article, we covered some common S3API operations, such as
- creating buckets,
- uploading and downloading objects,
- listing objects
- deleting objects.
Hope it helps
Signup for Exclusive "Subscriber-only" Content