Configuring a Apache Kafka® target endpoint
When creating or editing an endpoint, you can define:
- Yandex Managed Service for Apache Kafka® cluster connection or custom installation settings, including those based on Yandex Compute Cloud VMs. These are required parameters.
- Apache Kafka topic settings.
Managed Service for Apache Kafka® cluster
Connecting to the database with the cluster ID specified in Yandex Cloud. Available only for clusters deployed in Yandex Managed Service for Apache Kafka®.
Managed Kafka: Select the cluster to connect to.
Username: Specify the username that Data Transfer will use to connect to the database.
Password: Enter the user's password to the database.
Connecting to the database with explicitly specified network addresses.
Broker URI: Specify broker host IPs or FQDNs.
If the Apache Kafka® port number differs from the standard one, specify it with a colon after the host name:
<broker host IP or FQDN>:<port number>
SSL: Use encryption to protect the connection.
PEM certificate: If transmitted data needs to be encrypted, for example, to meet the requirements of PCI DSS, upload the certificate file or add its contents as text.
Authentication: Select the
SASLis selected, specify the hashing mechanism and the password and username of the account that Data Transfer will connect to the topic under.
Topic settings Apache Kafka®
Apache Kafka topic settings:
Topic full name: Specify the name of the topic to send messages to.
Topic prefix: Specify the topic prefix, similar to the
Debezium database.server.namesetting. Messages will be sent to a topic named
Yandex Data Transfer supports CDC for transfers from PostgreSQL, MongoDB, and MySQL databases to Apache Kafka® and Yandex Data Streams. Data is sent to the target in Debezium format. For more information about CDC mode, see Change data capture.
- Save tx order: Do not split an event stream into independent queues by table. This setting is only compatible with the Topic full name setting.