Migrating data to Yandex Managed Service for PostgreSQL

To migrate your database to Yandex Managed Service for PostgreSQL, you need to directly transfer the data, acquire a write lock for the old database, and switch over the load to the database cluster in Yandex Cloud.

Performing migration with Data Transfer lets you:

• Go without creating an intermediate VM or granting online access to your Managed Service for PostgreSQL target cluster.
• Minimize the DB downtime when migrating data.
• Migrate from older PostgreSQL versions to newer versions.

For more information, see What tasks is Yandex Data Transfer used for?.

To use this migration method, allow connecting to the source cluster from the internet.

Transferring data

1. Prepare the source cluster.

2. Prepare the infrastructure:

   Manually

   Using Terraform

   1. Create a Managed Service for PostgreSQL target cluster with any suitable configuration. In this case:

      • The PostgreSQL version must be the same or higher than the version in the source cluster. Migration with a PostgreSQL version downgrade is impossible.
      • When creating a cluster, specify the same database name as in the source cluster.
      • Enable the same PostgreSQL extensions as in the source cluster.

   2. Prepare the target cluster.

   3. Create a source endpoint with the following parameters:

      • Database type: PostgreSQL.
      • Endpoint parameters → Connection settings: Custom installation.

      Specify the parameters for connecting to the source cluster.

   4. Create a target endpoint with the following parameters:

      • Database type: PostgreSQL.
      • Endpoint parameters → Connection settings: MDB cluster.

      Specify the ID of the target cluster.

   5. Create a transfer of the Snapshot and increment type that will use the created endpoints.

   6. Activate the transfer.

      Warning

      Abstain from making any changes to the data schema in the source and target clusters when the data transfer is running. For more information, see Working with databases during the transfer.

   1. If you don't have Terraform, install and configure it.

   2. Download the file with provider settings. Place it in a separate working directory and specify the parameter values.

   3. Download the configuration file data-transfer-pgsql-mpg.tf to the same working directory.

      This file describes:

      • Network.
      • Subnet.
      • Security groups and the rule required to connect to a cluster.
      • Managed Service for PostgreSQL target cluster.
      • Source endpoint.
      • Target endpoint.
      • Transfer.

   4. Specify the following in data-transfer-pgsql-mpg.tf:

      • Source endpoint parameters.

      • pg-extensions: List of PostgreSQL extensions in the source cluster.

      • Target cluster parameters also used as target endpoint parameters:

        • target_mysql_version: PostgreSQL version, must be the same as or higher than the version in the source cluster.
        • target_user and target_password: Username and password of the database owner.

   5. Run the command terraform init in the directory with the configuration file. This command initializes the provider specified in the configuration files and enables you to use the provider resources and data sources.

   6. Make sure the Terraform configuration files are correct using the command:

      terraform validate
      

      If there are errors in the configuration files, Terraform will point to them.

   7. Create the required infrastructure:

      1. Run the command to view planned changes:

         terraform plan
         

         If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.

      2. If you are happy with the planned changes, apply them:

         1. Run the command:

            terraform apply
            

         2. Confirm the update of resources.

         3. Wait for the operation to complete.

      All the required resources will be created in the specified folder. You can check resource availability and their settings in the management console.

      Once created, a transfer is activated automatically.

3. Wait for the transfer status to change to Replicating.

4. Switch the source cluster to read-only mode.

5. On the transfer monitoring page, wait for the Maximum lag on delivery metric to decrease to zero. This means that all changes that occurred in the source cluster after data copying was completed are transferred to the target cluster.

6. Deactivate the transfer and wait for its status to change to Stopped.

   For more information about transfer statuses, see Transfer lifecycle.

7. Switch over the load to the target cluster.

8. Delete the resources you created:

   Manually created resources

   Resources created using Terraform

   • Delete a Managed Service for PostgreSQL cluster.
   • Delete the stopped transfer.
   • Delete endpoints for the source and target.

   1. In the terminal window, change to the directory containing the infrastructure plan.

   2. Delete the configuration file data-transfer-pgsql-mpg.tf.

   3. Make sure the Terraform configuration files are correct using the command:

      terraform validate
      

      If there are errors in the configuration files, Terraform will point to them.

   4. Confirm the update of resources.

      1. Run the command to view planned changes:

         terraform plan
         

         If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.

      2. If you are happy with the planned changes, apply them:

         1. Run the command:

            terraform apply
            

         2. Confirm the update of resources.

         3. Wait for the operation to complete.

      All the resources described in the configuration file data-transfer-pgsql-mpg.tf will be deleted.

Transferring tables with data types from PostgreSQL extensions

Data Transfer allows you to copy tables, the columns of which contain data types defined in PostgreSQL and tables with derived types, i.e. arrays of these types and composite types with fields of these types. Currently, there is a limitation: the data type must implement binary I/O functions. This means that, for the type of data in the pg_type system view, the typsend and typreceive column values must not be zero.

For example, for the PostGIS extension, columns of the GEOMETRY, GEOMETRY_DUMP, and GEOGRAPHY types can be transferred, while those of the BOX2D and BOX3D types cannot.

See also

For other migration methods, see the Yandex Managed Service for PostgreSQL documentation.