Get started
Language / Region
© 2022 Yandex.Cloud LLC
Yandex Message Queue

Converting a video to a GIF in Python

Written by

In this scenario, you'll create a video converter using FFmpeg and Yandex Message Queue.

To create an application:

  1. Prepare your cloud
  2. Create resources
  3. Create an API function
  4. Create a converter function
  5. Create a trigger
  6. Test the application

Prepare your cloud

  1. Go to the management console. Then log in to Yandex Cloud or sign up if you don't have an account yet.

  2. On the billing page, make sure you have a billing account activated and its status is ACTIVE or TRIAL_ACTIVE. If you don't have a billing account, create one.

  3. On the cloud page, create or select a folder for a new service.

    Learn more about clouds and folders.

The infrastructure support cost for this scenario includes:

Create resources

  1. Download the archive with the files needed to create a video converter.

  2. Create a service account named ffmpeg-sa and assign it the following roles:

    • ymq.reader
    • ymq.writer
    • lockbox.payloadViewer
    • storage.viewer
    • storage.uploader
    • ydb.admin
    • serverless.functions.invoker

  3. Create a static key for the service account. Save the Key ID and Your secret key.

  4. Create a secret named ffmpeg-sa-secret in Yandex Lockbox. Under Version, specify:

    • ACCESS_KEY_ID as the key and Key ID from the previous step as the value.
    • SECRET_ACCESS_KEY as the key and Your secret key from the previous step as the value.

    Save the secret ID from the Information about secret section.

  5. Create a message queue named converter-queue in Yandex Message Queue. Save the queue URL from the General information section.

  6. Create a YDB database in Serverless mode. Save the Endpoint from the Document API endpoint section.

  7. Create a table in the database:

    • Table name: tasks.
    • Table type: Document table.
    • Columns: One column with the name task_id and the String type. Set the Partition key attribute.

  8. Create a bucket named converter-bucket in Yandex Object Storage.

Create an API function

The function implements an API which you can use to perform the following actions:

  • convert: transfer a video to convert. The function writes the task to the tasks table using the Document API.
  • get_task_status: get the task status. The function checks whether the task is completed and returns a link to a GIF file.

  1. Create a function named ffmpeg-api.

  2. Create a function version:

    1. Create a file named requirements.txt and specify the following library in it:

      boto3

    2. Create a file named index.py and paste the contents of ffmpeg-api.py from the archive into it.

    3. Indicate the following:

      • Runtime environment: python37.
      • Entry point: index.handle_api.
      • Timeout: 5 seconds.
      • Service account: ffmpeg-sa.

    4. Add environment variables:

      • DOCAPI_ENDPOINT: The endpoint from the database configuration.
      • SECRET_ID: The Yandex Lockbox secret ID.
      • YMQ_QUEUE_URL: The Message Queue queue URL.

Create a converter function

A converter function is run by a trigger. It performs video processing and registers the execution result in the tasks table.

Video conversion is done using the FFmpeg utility. The FFmpeg executable file is more than 70 MB in size. To upload it along with the function code, create a ZIP archive and upload it via Object Storage. Learn more about code upload formats.

  1. Create a function named ffmpeg-converter.

  2. Create an src.zip archive with the following files:

    • The requirements.txt file: 
      boto3
requests
    • The index.py file with the contents of ffmpeg-converter.py from the archive.
    • The FFmpeg executable file. To get it, go to the FFmpeg official website, the Linux Static Builds section, and download the archive with the 64-bit FFmpeg version.

  3. Upload src.zip to converter-bucket.

  4. Create a function version:

    1. Indicate the following:

      • Upload method: Object Storage.
      • Bucket: converter-bucket.
      • Object: src.zip.
      • Runtime environment: python37.
      • Entry point: index.handle_process_event.
      • Timeout: 600 seconds.
      • Amount of RAM: 2048 MB.
      • Service account: ffmpeg-sa.

    2. Add environment variables:

      • DOCAPI_ENDPOINT: The endpoint from the database configuration.
      • SECRET_ID: The Yandex Lockbox secret ID.
      • YMQ_QUEUE_URL: The Message Queue queue URL.
      • S3_BUCKET: converter-bucket.

Create a trigger

A message queue is handled using a trigger for Message Queue. It invokes the converter function when messages arrive in converter-queue.

  1. In the management console, go to the folder where you want to create a trigger.
  2. Open Cloud Functions.
  3. Go to the Triggers tab.
  4. Click Create trigger.
  5. Under Basic parameters:
    • Name the trigger ffmpeg-trigger.
    • In the Type field, select Message Queue.
  6. Under Message Queue settings, select converter-queue and ffmpeg-sa with rights to read messages from it.
  7. Under Function settings:
    • Select the function to be invoked by the trigger: ffmpeg-converter.
    • Specify the function version tag: $latest.
    • Specify the service account to be used to invoke the function: ffmpeg-sa.
  8. Click Create trigger.

Test the application

Create a task

  1. In the management console, select the folder with the ffmpeg-api function.
  2. Open Cloud Functions.
  3. Select ffmpeg-api.
  4. Go to the Testing tab.
  5. In the Input field, enter: 
    {"action":"convert", "src_url":"<link to the video>"}
  6. Click Run test.
  7. You'll see the task ID in the Function output field: 
    { "task_id": "c4269ceb-8d3a-40fe-95f0-84cf16e8c17f" }

View the queue statistics

After the task is created, the number of messages in the queue increases by one and a trigger fires. Make sure that messages arrive in the queue and are handled. To do this, view the queue statistics.

  1. In the management console, select the folder with converter-queue.
  2. Open Message Queue.
  3. Select converter-queue.
  4. Under General information, you can see the number of enqueued messages and those being handled.
  5. Go to Monitoring. View the Overall queue stats charts.

View the function logs

The trigger should invoke the converter function for each message in the queue. To make sure the function is invoked, check its logs.

  1. In the management console, select the folder with the ffmpeg-converter function.
  2. Open Cloud Functions.
  3. Select ffmpeg-converter.
  4. Go to the Logs tab and specify the period to view them for.

  1. In the management console, select the folder with the ffmpeg-api function.

  2. Open Cloud Functions.

  3. Select ffmpeg-api.

  4. Go to the Testing tab.

  5. In the Input field, enter the following request:

    {"action":"get_task_status", "task_id":"<task id>"}

  6. Click Run test.

  7. If video conversion to GIF is not completed, the Function output field returns:

    {
    "ready": false
}

    Otherwise, you'll get a link to the GIF file:

    {
    "ready": true,
    "gif_url": "https://storage.yandexcloud.net/converter-bucket/1b4db1a6-f2b2-4b1c-b662-37f7a62e6e2e.gif?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=qxLftbbZ91U695ysemyZ%2F20210831%2Fru-central1%2Fs3%2Faws4_request&X-Amz-Date=20210831T110351Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=f4a5fe7848274a09be5b221fbf8a9f6f2b385708cfa351861a4e69df4ee4183c"
}
Language / Region
© 2022 Yandex.Cloud LLC
In this article: