Upload a file via an HTML form

Overview
HTML form
- Form fields
Security policy
- Security policy signature
Example of generating a form using boto3

This section describes how to upload files from the browser to Object Storage using an HTML form.

Note

Objects larger than 5 GB can't be uploaded using a form (see Quotas and limits).

Overview

If you want to let your service users upload files to your bucket directly from the browser:

You develop an HTML form with everything you need to send a request to Object Storage and put it on a page in your service.
The user opens your service page from the browser and uses the form to upload their file to storage.

To set rules and restrictions for file uploads, attach your security policy to the form. You don't need a policy when your bucket is publicly available for unrestricted writing.

To create a form, follow these steps:

Develop a security policy to describe the parameters of the request to Object Storage. For example, your policy may set a limit on the size of an uploaded object.
Generate a signature based on the security policy.
Create an HTML form with a signed security policy that you offer users when uploading files.

HTML form

Generic layout of an HTML page with an upload form:

<html>
    <head>
        ...
        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
        ...
    </head>
    <body>
        ...
        <form action="https://storage.yandexcloud.net/{bucket-name}" method="post" enctype="multipart/form-data">
            ...
            <input .../>
            ...
        </form>
        ...
    </body>
</html>

The HTML form is defined by the <form> tag and consists of a declaration and fields.

The form declaration contains the following attributes:

action: The URL of the bucket where the object is to be uploaded.
method: The HTTP method. Value: POST.
enctype: The content type of the request. Value: multipart/form-data.

The form fields contain a detailed description of the request to Object Storage and the restrictions that apply to it.

The form and its fields must be UTF-8 encoded. Set the charset attribute for the <meta> tag of your page to UTF-8.

<html>
    <head>
        ...
        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
        ...

Form fields

Object Storage supports form signing based on AWS Signature V2 and V4. The signature mechanism chosen determines the set of fields in the form and their names. AWS Signature V2 is only supported for compatibility reasons. Please avoid it if possible.

Generic form layout:

AWS Signature V4

<form action="https://storage.yandexcloud.net/{bucket-name}" method="post" enctype="multipart/form-data">

    Key in storage:

517ff823af39d7b98ad8cd4df635e4e8676170f7 />

    <!-- Request properties -->
    <input type="hidden" name="x-amz-credential" value="access_key_id/date/ru-central1/s3/aws4_request" />
    <input type="hidden" name="acl" value="predefined-acl-name" />
    <input type="hidden" name="x-amz-algorithm" value="AWS4-HMAC-SHA256" />
    <input type="hidden" name="x-amz-date" value="date" />
    <input type="hidden" name="success_action_redirect" value="some-URL" />
    <input type="hidden" name="policy" value="base64-encoded-policy-document" />
    <input type="hidden" name="x-amz-signature" value="signature-string" />
    <!-- Other required fields -->
    File to upload:
    <input type="file" name="file" /> <br />
    <!-- Fields after file are ignored -->
    <input type="submit" name="submit" value="Upload" />
 </form>
 ```

AWS Signature V2

 <form action="https://storage.yandexcloud.net/{bucket-name}" method="post" enctype="multipart/form-data">
      Key in storage:
      <input type="input" name="key" value="object_key" />
      <!-- Request properties -->
      <input type="hidden" name="AWSAccessKeyId" value="access_key_id" />
      <input type="hidden" name="acl" value="access_type" />
      <input type="hidden" name="success_action_redirect" value="url" />
      <input type="hidden" name="policy" value="base64-encoded-policy-document" />
      <input type="hidden" name="signature" value="signature_string" />
      <input type="hidden" name="Content-Type" value="content/type" />
      <!-- Other required fields -->
      File to upload:
      <input type="file" name="file" /> <br />
      <!-- Fields after file are ignored -->
      <br />
      <input type="submit" value="Upload file" />
</form>

Note

The following applies to AWS Signature V4 only.

Description of form fields:

Field	Description	Required
`acl`	ACL for the object. You can set one of the pre-defined ACLs. For example, if you want to make an object public, use `public-read`.	No
`Cache-Control`	A set of directives for caching data according to RFC 2616.	No
`Content-Disposition`	The name Object Storage suggests saving an object as a file under when it's downloaded. Compliant with RFC 2616.	No
`Content-Encoding`	Defines content encoding according to RFC 2616.	No
`Content-Type`	The MIME type of the uploaded file. If you don't specify `Content-Type`, Object Storage saves the object as an `application/octet-stream`. This can affect end user programs since they won't understand the file format (for example, a browser won't be able to render an image).	No
`Expires`	Response expiration date. Complaint with RFC 2616.	No
`key`	The object key. You can enter your key completely or as a template in `prefix/${filename}` format. So, if you upload `some_file.jpg`, the final object key is `prefix/some_file.jpg`.	Yes
`policy`	Security policy defining request permissions. Requests without a policy are treated as anonymous and are only processed for buckets with public write access.	Conditional
`x-amz-signature`	The policy signature that has to be generated using the secret key. It's required if the form has a security policy.	Conditional
`success_action_redirect`	The URL the user is redirected to when the file is successfully uploaded. If the value isn't set, Object Storage returns the response specified in the `success_action_status` field.	No
`success_action_status`	The response status after a successful upload. If `success_action_redirect` isn't specified, Object Storage returns `success_action_status`. The response body is empty. Acceptable values: 200, 204 (default).	No

<<<<<<< HEAD | `x-amz-algorithm` | The security policy signature algorithm. Value: `AWS4-HMAC-SHA256`.

Required if the form has a security policy. | Conditional | | `x-amz-credential` | Signature ID.

A string in `<access-key-id>/<date>/ru-central1/s3/aws4_request` format, where `<date>` must match the `x-amz-date` field value and the date used to sign the policy.

Required if the form has a security policy. | Conditional | | `x-amz-date` | Date in ISO8601 format, for example: `20180719T000000Z`. It must match the date in the `x-amz-credential` field (by the value rather than format) and the date used to sign the policy.

Required if the form has a security policy. | Conditional |

| x-amz-algorithm | The security policy signature algorithm. Value: AWS4-HMAC-SHA256.

It's required if the form has a security policy. | Conditional | | x-amz-credential | The ID for the signature.

A string in <access-key-id>/<date>/ru-central1/s3/aws4_request format, where must match the x-amz-date field value and date used to sign the policy.

It's required if the form has a security policy. | Conditional | | x-amz-date | Date in ISO8601 format, for example: 20180719T000000Z. It must match the date in the x-amz-credential field and the date used to sign the policy.

It's required if the form has a security policy. | Conditional |

517ff823af39d7b98ad8cd4df635e4e8676170f7 | x-amz-storage-class | The storage class for the object. With an HTML form, you can only put an object in Standard storage. | No | | x-amz-meta-* | User-defined object metadata.

Object Storage considers all headers starting with x-amz-meta- as user-defined. It doesn't process these headers. Instead, it saves them in their original format.

The total size of user-defined headers must not exceed 2 KB. The size of user-defined data is determined as the length of the UTF-8 encoded string. The header names and their values are included when calculating the size. | No | | x-amz-website- redirect-location | If the bucket is configured as a website, this field sets a redirect from the specified object to any other object in the bucket or any URL on the internet. The redirect is saved in the metadata of the object. | No | | file | An input field that lets the user select a file to upload. This field must be the last field in the form. All fields given after file are ignored. You can't upload more than one file in a single request. | Yes |

Security policy

The HTML form contains a security policy that puts restrictions on uploadable files.

The security policy is a JSON document and may look like the following:

{
    "expiration": "timestamp",
<<<<<<< HEAD
    "conditions": [
        {"bucket": "bucket-name"},
=======
    "conditions": [ 
        {"bucket": "bucket-name"}, 
>>>>>>> 517ff823af39d7b98ad8cd4df635e4e8676170f7
        ["starts-with", "$key", "users-uploads/"],
        {"acl": "public-read"},
        {"success_action_redirect": "http://localhost/"},
        ["starts-with", "$Content-Type", ""],
        ["content-length-range", 0, 1048576]
    ]
}

The expiration field contains the policy expiration date in ISO8601 format, for example, 20190722T153936Z. When the policy expires, Object Storage no longer accepts any files uploaded from the form.

The conditions field contains a set of rules for the form fields. At least one rule must be specified for each form field.

Security policy rules can be of the following types:

Rule type	Description
Exact match	The form field value must be exactly the same as in the policy. For example, `{"acl": "public-read"}`. An alternative format is also supported: `["eq", "$acl", "public-read" ]`.
Partial match	The form field value must start with the string specified in the policy. For example, `["starts-with", "$key", "key_prefix"]`. If an empty string is specified as a value, the field can take any value. For example, `["starts-with", "$Content-Type", ""]`.
`content-length-range`	The size limit of the object to upload. For example, `["content-length-range", 0, 1048576]`.

Possible restrictions:

Element	Restriction type	Restriction scope
`acl`	Exact and partial match.	The `acl` field in the form.
`bucket`	Exact and partial match.	Name of the bucket.
`content-length-range`	`content-length-range`	`content-length-range`
`key`	Exact and partial match.	The `key` field in the form. It lets you set the object key or prefix.
`success_action_redirect`	Exact and partial match.	The `success_action_redirect` field in the form.
`success_action_status`	Exact and partial match.	The `success_action_status` field in the form.
`x-amz-*`	Exact match.	The `x-amz-` fields in the form, except `x-amz-meta-`.
`x-amz-meta-*`	Exact and partial match.	`x-amz-meta-*` fields in the form.
`Cache-Control` `Content-Disposition` `Content-Encoding` `Content-Type` `Expires`	Exact and partial match.	Form fields with the same names.

If the key field has a template format, the policy is applied after the user-specified file name is inserted into the template.

Security policy signature

Common policy signature algorithm:

Encode the policy JSON document in base64.
Generate a signing key. <<<<<<< HEAD
Generate a policy signature. =======
Generate the policy signature.

517ff823af39d7b98ad8cd4df635e4e8676170f7

Example of generating a form using boto3

Input conditions:

Files must be saved in the user-data bucket with the /users/upload/ prefix.
Uploaded objects are publicly accessible for reading.
If the upload is successful, the user is redirected to https://cloud.yandex.ru/docs/storage/concepts/presigned-post-forms.

To generate form fields, we use boto3 from the Python SDK:

aws_access_key_id = 'JK38EXAMPLEAKDID8'
aws_secret_access_key = 'ExamP1eSecReTKeykdokKK38800'
endpoint = 'https://storage.yandexcloud.net'

<<<<<<< HEAD
s3 = boto3.client('s3',
=======
s3 = boto3.client('s3', 
>>>>>>> 517ff823af39d7b98ad8cd4df635e4e8676170f7
                  aws_access_key_id=aws_access_key_id,
                  aws_secret_access_key=aws_secret_access_key,
                  region_name='ru-central1',
                  endpoint_url=endpoint,
                  config=botocore.client.Config(signature_version='s3v4'),
                  )

key = 'users/uploads/${filename}'
bucket = 'user-data'
conditions = [{"acl":"public-read"}, ["starts-with", "$key", "users/uploads"], {'success_action_redirect': 'https://cloud.yandex.ru/docs/storage/concepts/presigned-post-forms'}]
fields = {'success_action_redirect': 'https://cloud.yandex.ru/docs/storage/concepts/presigned-post-forms'}

prepared_form_fields = s3.generate_presigned_post(Bucket=bucket,
                                                  Key=key,
                                                  Conditions=conditions,
                                                  Fields=fields,
                                                  ExpiresIn=60 * 60)


print(prepared_form_fields)

The script returns a JSON document in the following format:

{
    'url': u'https://storage.yandexcloud.net/user-data',
    'fields': {
        'x-amz-algorithm': 'AWS4-HMAC-SHA256',
        'x-amz-date': '20190722T153936Z',
        'success_action_redirect': 'https://cloud.yandex.ru/docs/storage/concepts/presigned-post-forms',
        'x-amz-signature': '4bdfb2209fc30744458be10bc3b99361f2f50add20f2ca2425587a2722859f96',
        'key': 'users/uploads/${filename}',
        'policy': u'eyJjb25kaXRpb25zIj...M5OjM2WiJ9',
        'x-amz-credential': u'JK38EXAMPLEAKDID8/20190722/ru-central1/s3/aws4_request'}
}

Using the values from the returned document, you can build an HTML page with a file upload form:

<html>
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    </head>
    <body>
        <form action="https://storage.yandexcloud.net/user-data" method="post" enctype="multipart/form-data">
<<<<<<< HEAD
            Key in storage:
=======
            Key in storage: 
>>>>>>> 517ff823af39d7b98ad8cd4df635e4e8676170f7
            <input type="input"    name="key" value="users/uploads/${filename}" /><br />
            <input type="hidden"   name="x-amz-credential" value="JK38EXAMPLEAKDID8/20190722/ru-central1/s3/aws4_request" />
            <input type="hidden"   name="acl" value="public-read" />
            <input type="hidden"   name="x-amz-algorithm" value="AWS4-HMAC-SHA256" />
            <input type="hidden"   name="x-amz-date" value="20190722T153936Z" />
            <input type="hidden"   name="success_action_redirect" value="https://cloud.yandex.ru/docs/storage/concepts/presigned-post-forms" />
            <input type="hidden"   name="policy" value="eyJjb25kaXRpb25zIj...M5OjM2WiJ9" />
            <input type="hidden" name="x-amz-signature" value="4bdfb2209fc30744458be10bc3b99361f2f50add20f2ca2425587a2722859f96" />
<<<<<<< HEAD
            File to upload:
=======
            File to upload: 
>>>>>>> 517ff823af39d7b98ad8cd4df635e4e8676170f7
            <input type="file"   name="file" /> <br />
            <input type="submit" name="submit" value="Upload" />
        </form>
    </body>
</html>