Pre-signed URLs
Object Storage incorporates multiple mechanisms for managing access to resources. To learn how these mechanisms interact, see Access management methods in Object Storage: Overview.
With pre-signed URLs, any web user can perform various operations in Object Storage, such as:
- Download an object
- Upload an object
- Create a bucket
A pre-signed URL is a URL containing request authorization data in its parameters. Pre-signed URLs can be created by users with static access keys.
This section outlines the general principles for generating pre-signed URLs using AWS Signature V4
Note
SDKs for various programming languages and other tools for AWS S3 feature out-of-the-box methods for generating pre-signed URLs that you can also use for Object Storage.
General pre-signed URL format
https://<bucket_name>.storage.yandexcloud.net/<object_key>?
X-Amz-Algorithm=AWS4-HMAC-SHA256
&X-Amz-Credential=<access_key-id>%2F<YYYYMMDD>%2Fru-central1%2Fs3%2Faws4_request
&X-Amz-Date=<time_in_ISO_8601_format>
&X-Amz-Expires=<URL_lifetime>
&X-Amz-SignedHeaders=<list_of_signed_headers>
&X-Amz-Signature=<signature>
Pre-signed URL parameters:
Parameter | Description |
---|---|
X-Amz-Algorithm |
Identifies the signature version and algorithm for its calculation. Value: AWS4-HMAC-SHA256 . |
X-Amz-Credential |
Signature ID. This is a string in <access-key-id>/<YYYYMMDD>/ru-central1/s3/aws4_request format, where <YYYYMMDD> must match the date set in the X-Amz-Date header. |
x-amz-date |
Time in ISO860120180719T000000Z . The date specified must match the date in the X-Amz-Credential parameter (by value rather than by format). |
X-Amz-Expires |
Link validity time in seconds. The starting point is the time specified in X-Amz-Date . The maximum value is 2592000 seconds (30 days). |
X-Amz-SignedHeaders |
Headers of the request you want to sign, delimited by a semicolon (; ).Make sure to sign the Host header and all X-Amz-* headers used in the request. You do not have to sign other headers; however, the more headers you sign, the safer your request is going to be. |
X-Amz-Signature |
Request signature. |
Creating pre-signed URLs
Note
Generating pre-signed URLs is optional for public buckets. You can get files from a publicly available bucket via both HTTP and HTTPS even if the bucket has no website hosting configured.
To get a pre-signed URL:
- Create a canonical request.
- Compose a string to sign.
- Generate a signing key.
- Calculate the signature using the key.
- Generate a pre-signed URL.
To create a pre-signed URL, you must have static access keys.
Canonical request
The general canonical request format is as follows:
<HTTPVerb>\n
<CanonicalURL>\n
<CanonicalQueryString>\n
<CanonicalHeaders>\n
<SignedHeaders>\n
UNSIGNED-PAYLOAD
HTTPVerb
HTTPVerb stands for the HTTP method used to send a request: GET
, PUT
, HEAD
, or DELETE
.
CanonicalURL
URL-encoded object key. For example, /folder/object.ext
.
Note
Do not normalize the path. For example, if an object has a some//strange//key//example
key, normalizing the path to /<bucket-name>/some/strange/key/example
will invalidate it.
CanonicalQueryString
The canonical query string must include all query parameters of the destination URL, except X-Amz-Signature
. The parameters in the string must be URL-encoded and sorted alphabetically.
For example:
X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=JK38EXAMPLEAKDID8%2F20190801%2Fru-central1%2Fs3%2Faws4_request&X-Amz-Date=20190801T000000Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host
CanonicalHeaders
This section includes the list of the request headers and their values.
The requirements are as follows:
- Each header must be separated with the
\n
newline character. - Header names must be lowercase.
- Headers must be sorted alphabetically.
- There may not be any extra spaces.
- The list must contain the
host
header and allx-amz-*
headers used in the request.
You can also add any request header to the list. The more headers you sign, the safer your request is going to be.
For example:
host:sample-bucket.storage.yandexcloud.net
x-amz-date:20190801T000000Z
SignedHeaders
This is a list of lowercase request header names, sorted alphabetically and separated by semicolons.
For example:
host;x-amz-date
Canonical request ending
A canonical request must always end with an UNSIGNED-PAYLOAD
string.
String to sign
General format of a string to sign:
"AWS4-HMAC-SHA256" + "\n" +
<timestamp> + "\n" +
<scope> + "\n" +
Hex(Hash-SHA256(<CanonicalRequest>))
Where:
AWS4-HMAC-SHA256
: Hashing algorithm.timestamp
: Current time in ISO 8601 format, for example,20190801T000000Z
. The specified date must match the date inscope
(by value rather than by format).scope
:<YYYYMMDD>/ru-central1/s3/aws4_request
.CanonicalRequest
: Canonical request generated earlier. The signature string contains the SHA256 hash of the canonical request in hexadecimal representation.
Signing key
To generate a signing key, you need static access keys for Object Storage. To learn how to get them, see Before you start.
Generate a signing key
-
Use the secret key to encode the date:
DateKey = sign("AWS4" + "SecretKey", "yyyymmdd")
-
Encode the region using the
DateKey
obtained in the previous step:RegionKey = sign(DateKey, "ru-central1")
-
Encode the service using the
RegionKey
obtained in the previous step:ServiceKey = sign(RegionKey, "s3")
-
Get a signing key:
SigningKey = sign(ServiceKey, "aws4_request")
Sign a string with a key
To get a string signature, use HMAC
with the SHA256
hash function and convert the result to hexadecimal format.
signature = Hex(sign(SigningKey, StringToSign))
Pre-signed URLs
To compose a pre-signed URL, add the parameters required to authorize the request to the Object Storage resource URL, including the X-Amz-Signature
parameter containing the calculated signature.
The other parameter values must match their respective values specified earlier in the canonical request and the signature string.
Example of composing a pre-signed URL for downloading an object
Let's put together a pre-signed URL to download the object-for-share.txt
object from the bucket:
-
Static key:
access_key_id = 'JK38EXAMP********' secret_access_key = 'ExamP1eSecReTKeykdo********'
-
Canonical request:
GET /object-for-share.txt X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=YCAJEK0Iv6x********eLTAdg%2F20231208%2Fru-central1%2Fs3%2Faws4_request&X-Amz-Date=20231208T184504Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host host:<bucket_name>.storage.yandexcloud.net host UNSIGNED-PAYLOAD
-
String to sign:
AWS4-HMAC-SHA256 20231208T184504Z 20231208/ru-central1/s3/aws4_request e823d75aad02c1317589bd5373fe9e20d5ef44499237703ff23e5600********
-
Signing key:
sign(sign(sign(sign("AWS4" + "ExamP1eSecReTKeykdokKK38800","20190801"),"ru-central1"),"s3"),"aws4_request")
Here, we introduce the
sign
function to indicate the method of key calculation that uses the HMAC algorithm with SHA256 . -
Signature:
b10c16a1997bb524bf59974512f1a6561cf2953c29dc3efbdb920790********
-
Pre-signed URL:
https://<bucket_name>.storage.yandexcloud.net/object-for-share.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=YCAJEK0Iv6xqy-pEQcueLTAdg%2F20231208%2Fru-central1%2Fs3%2Faws4_request&X-Amz-Date=20231208T195434Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=b10c16a1997bb524bf59974512f1a6561cf2953c29dc3efbdb920790********
Code examples for generating pre-signed URLs
<?php
$keyid = "<static_key_ID>";
$secretkey = "static_key_contents";
$path = "<object_key>";
$objectname = implode("/", array_map("rawurlencode", explode("/", $path)));
$host = "<bucket_name>.storage.yandexcloud.net";
$region = "ru-central1";
$timestamp = time();
$dater = strval(date('Ymd', $timestamp));
$dateValue = strval(date('Ymd', $timestamp))."T".strval(date('His', $timestamp))."Z";
// Generate the canonical request
$canonical_request = "GET\n".$objectname."\nX-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=".$keyid."%2F".$dater."%2Fru-central1%2Fs3%2Faws4_request&X-Amz-Date=".$dateValue."&X-Amz-Expires=3600&X-Amz-SignedHeaders=host\nhost:".$host."\n\nhost\nUNSIGNED-PAYLOAD";
echo "<b>Canonical request: </b><br>".$canonical_request."<br><br>";
// Generate the string to be signed
$string_to_sign = "AWS4-HMAC-SHA256\n".$dateValue."\n".$dater."/".$region."/s3/aws4_request\n".openssl_digest($canonical_request, "sha256", $binary = false);
echo "<b>String to be signed: </b><br>".$string_to_sign."<br><br>";
// Generate the signing key
$signing_key = hash_hmac('sha256', 'aws4_request', hash_hmac('sha256', 's3', hash_hmac('sha256', 'ru-central1', hash_hmac('sha256', $dater, 'AWS4'.$secretkey, true), true), true), true);
echo "<b>Signing key: </b><br>".$signing_key."<br><br>";
// Generate the signature
$signature = hash_hmac('sha256', $string_to_sign, $signing_key);
echo "<b>Signature: </b><br>".$signature."<br><br>";
// Generate the pre-signed link
$signed_link = "https://".$host.$objectname."?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=".$keyid."%2F".$dater."%2Fru-central1%2Fs3%2Faws4_request&X-Amz-Date=".$dateValue."&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=".$signature."\n";
echo '<b>Signed link: </b><br>'.'<a href = "'.$signed_link.'" target = "_blank">'.$signed_link.'</a>';
?>
Examples of getting pre-signed links in Object Storage tools
- In the management console
, select the folder. - Select Object Storage.
- Click the name of the bucket you need.
- Click the object name.
- Click Get link in the top-right corner.
- If your bucket has restricted access, specify the link Lifetime in hours or days (the maximum time is thirty days).
- Click Get link.
- Copy the link.
You can also use AWS CLI to generate a link for downloading an object. To do this, run the following command:
aws s3 presign s3://<bucket-name>/<object-key> --endpoint-url "https://storage.yandexcloud.net/" [--expires-in <value>]
To generate the link properly, make sure to provide the --endpoint-url
parameter pointing to the Object Storage hostname. For detailed information, see this section covering AWS CLI specifics.
The example below generates a pre-signed URL for downloading the object-for-share
object from the bucket-with-objects
bucket. The URL is valid for 100 seconds.
# coding=utf-8
import boto3
from botocore.client import Config
ENDPOINT = "https://storage.yandexcloud.net"
ACCESS_KEY = "JK38EXAMP********"
SECRET_KEY = "ExamP1eSecReTKeykdo********"
session = boto3.Session(
aws_access_key_id=ACCESS_KEY,
aws_secret_access_key=SECRET_KEY,
region_name="ru-central1",
)
s3 = session.client(
"s3", endpoint_url=ENDPOINT, config=Config(signature_version="s3v4")
)
presigned_url = s3.generate_presigned_url(
"get_object",
Params={"Bucket": "bucket-with-objects", "Key": "object-for-share"},
ExpiresIn=100,
)
print(presigned_url)