Yandex Cloud
  • Services
  • Solutions
  • Why Yandex Cloud
  • Pricing
  • Documentation
  • Contact us
Get started
Language / Region
© 2022 Yandex.Cloud LLC
Yandex Data Transfer
  • Getting started
  • Step-by-step instructions
    • All instructions
    • Preparing for the transfer
    • Managing a source endpoint
    • Managing a target endpoint
    • Managing the transfer process
    • Working with databases during the transfer
    • Monitoring the transfer status
  • Concepts
    • Relationship between service resources
    • What tasks is the service used for?
    • Transfer lifecycle
    • Quotas and limits
  • Access management
  • Pricing policy
  • API reference
    • Authentication in the API
    • gRPC
      • Overview
      • EndpointService
      • TransferService
      • OperationService
    • REST
      • Overview
      • Endpoint
        • Overview
        • create
        • delete
        • get
        • list
        • update
      • Transfer
        • Overview
        • activate
        • create
        • deactivate
        • delete
        • get
        • list
        • update
  • Questions and answers
  1. Getting started

Getting started with Data Transfer

Written by
Yandex.Cloud
  • Before you start
  • Configure the source and the target
  • Creating an endpoint for the source
  • Creating an endpoint for the target
  • Create a transfer
  • Activate the transfer
  • What's next

Copy or replicate data from a source to a target by creating endpoints and configuring transfers between them.

Before you start

  1. Go to the Yandex Cloud management console and select the folder where you want to perform the operations. If that folder doesn't exist, create it:

    Management console
    CLI
    API
    1. Click Create folder on the cloud page.

    2. Enter the folder name. Naming requirements:

      • The length can be from 3 to 63 characters.
      • It may contain lowercase Latin letters, numbers, and hyphens.
      • The first character must be a letter. The last character can't be a hyphen.
    3. (Optional) Enter a description of the folder.

    4. Select the Create a default network option. A network is created with subnets in each availability zone. A default security group, where all network traffic is allowed inside, will also be created in this network.

    5. Click Create.

    If you don't have the Yandex Cloud command line interface yet, install and initialize it.

    1. See a description of the create folder command:

      yc resource-manager folder create --help
      
    2. Create a new folder:

      • with a name and without a description:

        yc resource-manager folder create \
           --name new-folder
        
        • The length can be from 3 to 63 characters.
        • It may contain lowercase Latin letters, numbers, and hyphens.
        • The first character must be a letter. The last character can't be a hyphen.
      • with a name and description:

        yc resource-manager folder create \
           --name new-folder \
           --description "my first folder with description"
        

    Use the create method for the Folder resource of the Yandex Resource Manager service.

  2. On the billing page, make sure that a billing account is linked and that its status is ACTIVE or TRIAL_ACTIVE. If you don't have a billing account, create one.

  3. On the access management page, make sure you have the editor role or higher to the desired folder or the cloud that the folder belongs to.

Configure the source and the target

Prepare the source service for sending data:

  • Apache Kafka®
  • ClickHouse
  • MongoDB
  • MySQL
  • PostgreSQL
  • Yandex Data Streams

Prepare the target service for receiving the data:

  • ClickHouse
  • MongoDB
  • MySQL
  • Object Storage
  • PostgreSQL

To receive data in Managed Service for YDB, no setup is necessary.

Creating an endpoint for the source

  1. Go to the folder page and select Data Transfer.
  2. On the Endpoints tab, click Create endpoint.
  3. Make sure that the Direction field is set to Source.
  4. Enter a name for the endpoint.
  5. In the Database type field, select the type of the DBMS to transfer data from.
  6. Specify endpoint parameters in the corresponding settings section.
  7. Click Create.

For more information, see Managing a source endpoint.

Creating an endpoint for the target

  1. Go to the folder page and select Data Transfer.
  2. On the Endpoints tab, click Create endpoint.
  3. Make sure that the Direction field is set to Target.
  4. Enter a name for the endpoint.
  5. In the Database type field, select the type of DBMS that you want to transfer data to.
  6. Specify endpoint parameters in the corresponding settings section.
  7. Click Create.

For more information, see Managing a target endpoint.

Create a transfer

  1. Go to the folder page and select Data Transfer.
  2. On the Transfers tab, click Create transfer.
  3. Select the endpoint for the source and the endpoint for the target.
  4. Enter a name for the transfer.
  5. Select the type of transfer:
    • Copy and replicate: To create a full copy of the source data and keep it up-to-date.
    • Copy: To create a full copy of the data without receiving further updates from the source.
    • Replicate: To continuously receive data updates from the source and apply them to the target (without creating a full copy of the source data).
  6. (Optional) add a transfer description.
  7. Click Create.

For more information, see Types of transfers.

Activate the transfer

  1. Go to the folder page and select Data Transfer.
  2. On the Transfers tab, click next to the name of the desired transfer and select Activate.

The data transfer process will begin.

For more information, see Managing the transfer process.

What's next

  • Read about service concepts.
  • Learn more about preparing databases for the service and setting up transfers.
  • Read questions and answers.

Was the article helpful?

Language / Region
© 2022 Yandex.Cloud LLC
In this article:
  • Before you start
  • Configure the source and the target
  • Creating an endpoint for the source
  • Creating an endpoint for the target
  • Create a transfer
  • Activate the transfer
  • What's next