Configuring an EventHub source endpoint

Namespace: Specify the namespace that the hub will relate to.

Hub: Specify the hub name.

Topic: Specify the name of the topic to connect to.

Key name: Specify the key name (policy) that will be used for authentication.

Password: Enter the user's password to the database.

Consumer group: Specify the hub view. It is used for independent reading of an event stream.

Initial offset: Specify the initial position of the event in the section.

Transformation rules

The rules used by Cloud Function to process an incoming stream:

• Processing function: Select one of the functions created in Yandex Cloud Functions.

  • Service account: Select or create a service account that the processing function will start under.

• Number of attempts: Set the number of attempts to invoke the processing function.

• Buffer size to send: Set the size of the buffer (in bytes) which when full data will be transferred to the processing function.

  The maximum buffer size is 3.5 MB. For more information about restrictions that apply when working with functions in Cloud Functions, see the corresponding section.

• Sending interval: Set the duration of the interval (in seconds) after the expiration of which the data from the stream should be transferred to the processing function.

  Note

  If the buffer becomes full or the sending interval expires, the data is transferred to the processing function.

• Call timeout: Set the allowed timeout of the response from the processing function (in seconds).

Conversion rules:

• Data format: Select one of the available formats:

  • JSON
  • CSV

• Data schema: Specify the schema as a list of fields or upload a file with a description of the schema in JSON format.

  Sample data schema

  [
      {
          "name": "request",
          "type": "string"
        }
  ]
  

• Add unmarked columns: Select this option to have the fields missing in the schema appear in the _rest column.

• Allow null in key columns: Select this option to allow the null value in key columns.