Get started
Language / Region
© 2022 Yandex.Cloud LLC
Practical guidelines

URL shortener

Written by

With this script, you'll create a URL shortening service using serverless technologies available in Yandex Cloud.

The service accepts user requests via a public API gateway. The hosting service sends the user an HTML page with a field for entering the URL. A function sends the entered URL for storage in a serverless database, shortens it, and returns it to the user. When the user enters the shortened URL, the function finds the full URL in the database and redirects the user's request to it.

To configure and test the service:

  1. Before you start.
  2. Set up hosting for the URL shortener page.
  3. Create a service account.
  4. Create a database in Managed Service for YDB.
  5. Set up a function in Cloud Functions.
  6. Publish the service via API Gateway.
  7. Test the URL shortener.

If you no longer need the created resources, delete them.

Before you start

Before deploying the service, sign up for Yandex Cloud and create a billing account:

  1. Go to the management console. Then log in to Yandex Cloud or sign up if don't already have an account.
  2. On the billing page, make sure you linked a billing account, and it has the ACTIVE or TRIAL_ACTIVE status. If you don't have a billing account, create one.

If you have an active billing account, you can create or select a folder to run your service components in using the Yandex Cloud page.

Learn more about clouds and folders.

The cost of resources for the script includes:

Set up hosting for the URL shortener page

To create a bucket to place the HTML page of your service in and configure it for hosting static websites:

  1. In the management console, select your working folder.

  2. Select Object Storage.

  3. Click Create bucket.

  4. On the bucket creation page:

    1. Enter a name for the bucket like for-serverless-shortener.

      Warning

      Bucket names are unique throughout Object Storage, so you can't create two buckets with the same name (even in different folders in different clouds).

    2. Set the maximum size to 1 GB.

    3. Choose Public access to read objects.

    4. Click Create bucket to complete the operation.

  5. Copy the HTML code and paste it into the index.html file:

    HTML code
    <!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <title>URL shortener</title>
    <!-- warns against sending unnecessary GET requests to /favicon.ico -->
    <link rel="icon" href="data:;base64,iVBORw0KGgo=">
</head>

<body>
    <h1>Welcome</h1>
    <form action="javascript:shorten()">
        <label for="url">Enter the URL:</label><br>
        <input id="url" name="url" type="text"><br>
        <input type="submit" value="Shorten">
    </form>
    <p id="shortened"></p>
</body>

<script>
    function shorten() {
        const link = document.getElementById("url").value
        fetch("/shorten", {
            method: 'POST',
            headers: {
                'Content-Type': 'application/json'
            },
            body: link
        })
            .then(response => response.json())
            .then(data => {
                const url = data.url
                document.getElementById("shortened").innerHTML = `<a href=${url}>${url}</a>`
            })
            .catch(error => {
                document.getElementById("shortened").innerHTML = `<p>The ${error} error occurred, please try again</p>`
            })
    }
</script>

</html>

  6. Click on the name of the created bucket.

  7. Click Upload objects.

  8. Specify the prepared index.html file.

  9. Click Upload.

  10. In the left pane, select Website.

  11. Select Hosting.

  12. Specify the website's homepage: index.html.

  13. Click Save.

Create a service account

To create a service account for the service components to interact:

  1. Go to your working folder.

  2. In the left pane, select Service accounts.

  3. Click Create service account.

  4. Enter the name of the service account: serverless-shortener.

  5. Click Add role and choose the editor role.

  6. Click Create.

  7. Click on the name of the created service account.

    Save the service account ID, you'll need it in the next steps.

Create a database in Managed Service for YDB

To create a YDB database and configure it to store URLs:

  1. Go to your working folder.

  2. In the list of services, select Managed Service for YDB.

  3. Click Create database.

  4. Enter the DB name: for-serverless-shortener.

  5. Select the Serverless database type.

  6. Click Create database.

  7. Wait until the database starts.

    When a database is being created, it has the Provisioning status. When it's ready for use, the status changes to Running.

  8. Click on the name of the created database.

    Save the values of the Endpoint, Database, and Protocol fields of the YDB endpoint section. You'll need them in the next steps.

  9. In the left pane, select Navigation.

  10. Click SQL query.

  11. Copy the SQL query and paste it into the Query field:

    CREATE TABLE links
(
  id Utf8,
  link Utf8,
  PRIMARY KEY (id)
);

  12. Click Run.

Set up a function in Cloud Functions

To create and set up a URL shortening function:

  1. Go to your working folder.

  2. In the list of services, select Cloud Functions.

  3. Click Create function.

  4. Enter the function name: for-serverless-shortener.

  5. Click Create.

  6. In the Python drop-down list, choose the python37 runtime environment.

  7. Click Next.

  8. Copy the function code and paste it into the index.py file under Function code.

    Function code
    from kikimr.public.sdk.python import client as ydb

import urllib.parse
import hashlib
import base64
import json
import os


def decode(event, body):
    # the request body can be encoded
    is_base64_encoded = event.get('isBase64Encoded')
    if is_base64_encoded:
        body = str(base64.b64decode(body), 'utf-8')
    return body


def response(statusCode, headers, isBase64Encoded, body):
    return {
        'statusCode': statusCode,
        'headers': headers,
        'isBase64Encoded': isBase64Encoded,
        'body': body,
    }


def get_config():
    endpoint = os.getenv("endpoint")
    database = os.getenv("database")
    if endpoint is None or database is None:
        raise AssertionError("You need to specify both environment variables")
    credentials = ydb.construct_credentials_from_environ()
    return ydb.DriverConfig(endpoint, database, credentials=credentials)


def execute(config, query, params):
    with ydb.Driver(config) as driver:
        try:
            driver.wait(timeout=5)
        except TimeoutError:
            print("Connect failed to YDB")
            print("Last reported errors by discovery:")
            print(driver.discovery_debug_details())
            return None

        session = driver.table_client.session().create()
        prepared_query = session.prepare(query)

        return session.transaction(ydb.SerializableReadWrite()).execute(
            prepared_query,
            params,
            commit_tx=True
        )


def insert_link(id, link):
    config = get_config()
    query = """
        DECLARE $id AS Utf8;
        DECLARE $link AS Utf8;

        UPSERT INTO links (id, link) VALUES ($id, $link);
        """
    params = {'$id': id, '$link': link}
    execute(config, query, params)


def find_link(id):
    print(id)
    config = get_config()
    query = """
        DECLARE $id AS Utf8;

        SELECT link FROM links where id=$id;
        """
    params = {'$id': id}
    result_set = execute(config, query, params)
    if not result_set or not result_set[0].rows:
        return None

    return result_set[0].rows[0].link


def shorten(event):
    body = event.get('body')

    if body:
        body = decode(event, body)
        original_host = event.get('headers').get('Origin')
        link_id = hashlib.sha256(body.encode('utf8')).hexdigest()[:6]
        # the URL may contain encoded characters like %, this will hinder the api-gateway to work properly when making a redirect,
        # therefore, you should get rid of these characters by invoking urllib.parse.unquote
        insert_link(link_id, urllib.parse.unquote(body))
        return response(200, {'Content-Type': 'application/json'}, False, json.dumps({'url': f'{original_host}/r/{link_id}'}))

    return response(400, {}, False, 'The url parameter is missing in the request body')


def redirect(event):
    link_id = event.get('pathParams').get('id')
    redirect_to = find_link(link_id)

    if redirect_to:
        return response(302, {'Location': redirect_to}, False, '')

    return response(404, {}, False, 'This link does not exist')


# these checks are necessary because we have only one function
# ideally, each path in the api-gw should have its own function
def get_result(url, event):
    if url == "/shorten":
        return shorten(event)
    if url.startswith("/r/"):
        return redirect(event)

    return response(404, {}, False, 'This path does not exist')


def handler(event, context):
    url = event.get('url')
    if url:
        # The URL may come from the API-gateway with a question mark at the end
        if url[-1] == '?':
            url = url[:-1]
        return get_result(url, event)

    return response(404, {}, False, 'This function should be called using the api-gateway')

  9. Copy the following text and paste it into the requirements.txt file under Function code.

    ydb==0.0.41

  10. Specify the entry point: index.handler.

  11. Set the timeout value to 5.

  12. Select the serverless-shortener service account.

  13. Add environment variables:

    • endpoint: Enter a string that is generated from the database protocol and endpoint.

      For example, if the protocol is grpcs and the endpoint is ydb.serverless.yandexcloud.net:2135, enter grpcs://ydb.serverless.yandexcloud.net:2135.

    • database: Enter the previously saved Database field value.

    • USE_METADATA_CREDENTIALS: Enter 1.

  14. In the upper-right part of the Editor section, click Create version.

  15. Under General information, enable Public function.

Save the function ID, you'll need it in the next steps.

Publish the service via API Gateway

To publish the service via API Gateway:

  1. Go to your working folder.

  2. In the list of services, select API Gateway.

  3. Click Create API gateway.

  4. In the Name field, enter for-serverless-shortener.

  5. Copy and paste the following code into the Specification section:

    Specification
    openapi: 3.0.0
info:
  title: for-serverless-shortener
  version: 1.0.0
paths:
  /:
    get:
      x-yc-apigateway-integration:
        type: object_storage
        bucket: for-serverless-shortener # <-- bucket name
        object: index.html # <-- HTML file name
        presigned_redirect: false
        service_account: <service_account_id> # <-- service account ID
      operationId: static
  /shorten:
    post:
      x-yc-apigateway-integration:
        type: cloud_functions
        function_id: <function_id> # <-- function ID
      operationId: shorten
  /r/{id}:
    get:
      x-yc-apigateway-integration:
        type: cloud_functions
        function_id: <function_id> # <-- function ID
      operationId: redirect
      parameters:
      - description: id of the url
        explode: false
        in: path
        name: id
        required: true
        schema:
          type: string
        style: simple

    Edit the specification code:

    • Replace <service_account_id> with the ID of the previously created service account.
    • Replace <function_id> with the ID of the previously created function.

  6. Click Create.

  7. Click on the name of the created API gateway.

  8. Copy the url value from the specification.

    Use this URL to work with the created service.

Test the URL shortener

To check that the service components interact properly:

  1. Open the copied URL in the browser.

  2. In the input field, enter the URL that you want to shorten.

  3. Click Shorten.

    You'll see the shortened URL below.

  4. Follow this link. As a result, the same page should open as when using the full URL.

Delete the service components

To delete all the created service components:

  1. Delete the API gateway:
    1. Go to your working folder.
    2. In the list of services, select API Gateway.
    3. To the right of the API gateway name, click and select Delete.
    4. Click Delete.
  2. Delete the function:
    1. Go to your working folder.
    2. In the list of services, select Cloud Functions.
    3. To the right of the function name, click and select Delete.
    4. Click Delete.
  3. Delete the database:
    1. Go to your working folder.
    2. In the list of services, select Managed Service for YDB.
    3. To the right of the database name, click and select Delete.
    4. Click Delete.
  4. Delete the service account:
    1. Go to your working folder.
    2. In the left pane, select Service accounts.
    3. To the right of the service account name, click and select Delete.
    4. Click Delete.
  5. Delete the bucket:
    1. Go to your working folder.
    2. Select Object Storage.
    3. Click on the name of the created bucket.
    4. To the right of the bucket name, click and select Delete.
    5. Click Delete.
    6. Return to the Buckets page.
    7. To the right of the bucket name, click and select Delete.
    8. Click Delete.
Language / Region
© 2022 Yandex.Cloud LLC
In this article: