Data Transfer API, REST: Transfer.update
HTTP request
PATCH https://datatransfer.api.cloud.yandex.net/v1/transfer/{transferId}
Path parameters
Parameter | Description |
---|---|
transferId | Identifier of the transfer to be updated. |
Body parameters
{
"description": "string",
"runtime": {
"ycRuntime": {
"jobCount": "string",
"uploadShardParams": {
"jobCount": "string",
"processCount": "string"
}
}
},
"name": "string",
"updateMask": "string",
"labels": "object",
"transformation": {
"transformers": [
{
// `transformation.transformers[]` includes only one of the fields `maskField`, `filterColumns`, `renameTables`, `replacePrimaryKey`, `convertToString`, `sharderTransformer`, `tableSplitterTransformer`, `filterRows`
"maskField": {
"tables": {
"includeTables": [
"string"
],
"excludeTables": [
"string"
]
},
"columns": [
"string"
],
"function": {
"maskFunctionHash": {
"userDefinedSalt": "string"
}
}
},
"filterColumns": {
"tables": {
"includeTables": [
"string"
],
"excludeTables": [
"string"
]
},
"columns": {
"includeColumns": [
"string"
],
"excludeColumns": [
"string"
]
}
},
"renameTables": {
"renameTables": [
{
"originalName": {
"nameSpace": "string",
"name": "string"
},
"newName": {
"nameSpace": "string",
"name": "string"
}
}
]
},
"replacePrimaryKey": {
"tables": {
"includeTables": [
"string"
],
"excludeTables": [
"string"
]
},
"keys": [
"string"
]
},
"convertToString": {
"tables": {
"includeTables": [
"string"
],
"excludeTables": [
"string"
]
},
"columns": {
"includeColumns": [
"string"
],
"excludeColumns": [
"string"
]
}
},
"sharderTransformer": {
"tables": {
"includeTables": [
"string"
],
"excludeTables": [
"string"
]
},
"columns": {
"includeColumns": [
"string"
],
"excludeColumns": [
"string"
]
},
"shardsCount": "string"
},
"tableSplitterTransformer": {
"tables": {
"includeTables": [
"string"
],
"excludeTables": [
"string"
]
},
"columns": [
"string"
],
"splitter": "string"
},
"filterRows": {
"tables": {
"includeTables": [
"string"
],
"excludeTables": [
"string"
]
},
"filter": "string"
},
// end of the list of possible fields`transformation.transformers[]`
}
]
}
}
Field | Description |
---|---|
description | string The new description for the transfer. |
runtime | object |
runtime. ycRuntime |
object |
runtime. ycRuntime. jobCount |
string (int64) |
runtime. ycRuntime. uploadShardParams |
object |
runtime. ycRuntime. uploadShardParams. jobCount |
string (int64) |
runtime. ycRuntime. uploadShardParams. processCount |
string (int64) |
name | string The new transfer name. Must be unique within the folder. |
updateMask | string Field mask specifying transfer fields to be updated. Semantics for this field is described here: https://pkg.go.dev/google.golang.org/protobuf/types/known/fieldmaskpb#FieldMask The only exception: if the repeated field is specified in the mask, then the new value replaces the old one instead of being appended to the old one. A comma-separated names off ALL fields to be updated. Only the specified fields will be changed. The others will be left untouched. If the field is specified in If |
labels | object |
transformation | object Transformation is converting data using special transformer functions. These functions are executed on a data stream, applied to each data change item, and transform them. A transformer can be run at both the metadata and data levels. Data can only be transformed if the source and target are of different types. |
transformation. transformers[] |
object Transformers are set as a list. When activating a transfer, a transformation plan is made for the tables that match the specified criteria. Transformers are applied to the tables in the sequence specified in the list. |
transformation. transformers[]. maskField |
object transformation.transformers[] includes only one of the fields maskField , filterColumns , renameTables , replacePrimaryKey , convertToString , sharderTransformer , tableSplitterTransformer , filterRows Mask field transformer allows you to hash data |
transformation. transformers[]. maskField. tables |
object List of included and excluded tables Filter tables using lists of included and excluded tables. |
transformation. transformers[]. maskField. tables. includeTables[] |
string List of tables that will be included to transfer |
transformation. transformers[]. maskField. tables. excludeTables[] |
string List of tables that will be excluded to transfer |
transformation. transformers[]. maskField. columns[] |
string Specify the name of the column for data masking (a regular expression). |
transformation. transformers[]. maskField. function |
object Mask function Mask function |
transformation. transformers[]. maskField. function. maskFunctionHash |
object Hash mask function |
transformation. transformers[]. maskField. function. maskFunctionHash. userDefinedSalt |
string This string will be used in the HMAC(sha256, salt) function applied to the column data. |
transformation. transformers[]. filterColumns |
object transformation.transformers[] includes only one of the fields maskField , filterColumns , renameTables , replacePrimaryKey , convertToString , sharderTransformer , tableSplitterTransformer , filterRows Set up a list of table columns to transfer |
transformation. transformers[]. filterColumns. tables |
object List of the tables to filter using lists of included and excluded tables. Filter tables using lists of included and excluded tables. |
transformation. transformers[]. filterColumns. tables. includeTables[] |
string List of tables that will be included to transfer |
transformation. transformers[]. filterColumns. tables. excludeTables[] |
string List of tables that will be excluded to transfer |
transformation. transformers[]. filterColumns. columns |
object List of the columns to transfer to the target tables using lists of included and excluded columns. Filter columns using lists of included and excluded columns. |
transformation. transformers[]. filterColumns. columns. includeColumns[] |
string List of columns that will be included to transfer |
transformation. transformers[]. filterColumns. columns. excludeColumns[] |
string List of columns that will be excluded to transfer |
transformation. transformers[]. renameTables |
object transformation.transformers[] includes only one of the fields maskField , filterColumns , renameTables , replacePrimaryKey , convertToString , sharderTransformer , tableSplitterTransformer , filterRows Set rules for renaming tables by specifying the current names of the tables in the source and new names for these tables in the target. |
transformation. transformers[]. renameTables. renameTables[] |
object List of renaming rules |
transformation. transformers[]. renameTables. renameTables[]. originalName |
object Specify the current names of the table in the source |
transformation. transformers[]. renameTables. renameTables[]. originalName. nameSpace |
string |
transformation. transformers[]. renameTables. renameTables[]. originalName. name |
string |
transformation. transformers[]. renameTables. renameTables[]. newName |
object Specify the new names for this table in the target |
transformation. transformers[]. renameTables. renameTables[]. newName. nameSpace |
string |
transformation. transformers[]. renameTables. renameTables[]. newName. name |
string |
transformation. transformers[]. replacePrimaryKey |
object transformation.transformers[] includes only one of the fields maskField , filterColumns , renameTables , replacePrimaryKey , convertToString , sharderTransformer , tableSplitterTransformer , filterRows Override primary keys |
transformation. transformers[]. replacePrimaryKey. tables |
object List of included and excluded tables Filter tables using lists of included and excluded tables. |
transformation. transformers[]. replacePrimaryKey. tables. includeTables[] |
string List of tables that will be included to transfer |
transformation. transformers[]. replacePrimaryKey. tables. excludeTables[] |
string List of tables that will be excluded to transfer |
transformation. transformers[]. replacePrimaryKey. keys[] |
string List of columns to be used as primary keys |
transformation. transformers[]. convertToString |
object transformation.transformers[] includes only one of the fields maskField , filterColumns , renameTables , replacePrimaryKey , convertToString , sharderTransformer , tableSplitterTransformer , filterRows Convert column values to strings The values will be converted depending on the source type Conversion rules are described here: https://cloud.yandex.com/en/docs/data-transfer/concepts/data-transformation#convert-to-string |
transformation. transformers[]. convertToString. tables |
object List of included and excluded tables Filter tables using lists of included and excluded tables. |
transformation. transformers[]. convertToString. tables. includeTables[] |
string List of tables that will be included to transfer |
transformation. transformers[]. convertToString. tables. excludeTables[] |
string List of tables that will be excluded to transfer |
transformation. transformers[]. convertToString. columns |
object List of included and excluded columns Filter columns using lists of included and excluded columns. |
transformation. transformers[]. convertToString. columns. includeColumns[] |
string List of columns that will be included to transfer |
transformation. transformers[]. convertToString. columns. excludeColumns[] |
string List of columns that will be excluded to transfer |
transformation. transformers[]. sharderTransformer |
object transformation.transformers[] includes only one of the fields maskField , filterColumns , renameTables , replacePrimaryKey , convertToString , sharderTransformer , tableSplitterTransformer , filterRows Set the number of shards for particular tables and a list of columns whose values will be used for calculating a hash to determine a shard. |
transformation. transformers[]. sharderTransformer. tables |
object List of included and excluded tables Filter tables using lists of included and excluded tables. |
transformation. transformers[]. sharderTransformer. tables. includeTables[] |
string List of tables that will be included to transfer |
transformation. transformers[]. sharderTransformer. tables. excludeTables[] |
string List of tables that will be excluded to transfer |
transformation. transformers[]. sharderTransformer. columns |
object List of included and excluded columns Filter columns using lists of included and excluded columns. |
transformation. transformers[]. sharderTransformer. columns. includeColumns[] |
string List of columns that will be included to transfer |
transformation. transformers[]. sharderTransformer. columns. excludeColumns[] |
string List of columns that will be excluded to transfer |
transformation. transformers[]. sharderTransformer. shardsCount |
string (int64) Number of shards |
transformation. transformers[]. tableSplitterTransformer |
object transformation.transformers[] includes only one of the fields maskField , filterColumns , renameTables , replacePrimaryKey , convertToString , sharderTransformer , tableSplitterTransformer , filterRows A transfer splits the X table into multiple tables (X_1, X_2, ..., X_n) based on data. If a row was located in the X table before it was split, it is now in the X_i table, where i is determined by the column list and split string parameters. Example: If the column list has two columns, month of birth and gender, specified and the split string states @, information about an employee whose name is John and who was born on February 11, 1984, from the Employees table will get to a new table named Employees@February@male. |
transformation. transformers[]. tableSplitterTransformer. tables |
object List of included and excluded tables Filter tables using lists of included and excluded tables. |
transformation. transformers[]. tableSplitterTransformer. tables. includeTables[] |
string List of tables that will be included to transfer |
transformation. transformers[]. tableSplitterTransformer. tables. excludeTables[] |
string List of tables that will be excluded to transfer |
transformation. transformers[]. tableSplitterTransformer. columns[] |
string Specify the columns in the tables to be partitioned. |
transformation. transformers[]. tableSplitterTransformer. splitter |
string Specify the split string to be used for merging components in a new table name. |
transformation. transformers[]. filterRows |
object transformation.transformers[] includes only one of the fields maskField , filterColumns , renameTables , replacePrimaryKey , convertToString , sharderTransformer , tableSplitterTransformer , filterRows This filter only applies to transfers with queues (Logbroker or Apache Kafka®) as a data source. When running a transfer, only the strings meeting the specified criteria remain in a changefeed. |
transformation. transformers[]. filterRows. tables |
object List of included and excluded tables Filter tables using lists of included and excluded tables. |
transformation. transformers[]. filterRows. tables. includeTables[] |
string List of tables that will be included to transfer |
transformation. transformers[]. filterRows. tables. excludeTables[] |
string List of tables that will be excluded to transfer |
transformation. transformers[]. filterRows. filter |
string Filtering criterion. This can be comparison operators for numeric, string, and Boolean values, comparison to NULL, and checking whether a substring is part of a string. Details here: https://cloud.yandex.com/en/docs/data-transfer/concepts/data-transformation#append-only-sources |
Response
HTTP Code: 200 - OK
{
"id": "string",
"description": "string",
"createdAt": "string",
"createdBy": "string",
"modifiedAt": "string",
"done": true,
"metadata": "object",
// includes only one of the fields `error`, `response`
"error": {
"code": "integer",
"message": "string",
"details": [
"object"
]
},
"response": "object",
// end of the list of possible fields
}
An Operation resource. For more information, see Operation.
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
createdAt | string (date-time) Creation timestamp. String in RFC3339 text format. The range of possible values is from To work with values in this field, use the APIs described in the Protocol Buffers reference. In some languages, built-in datetime utilities do not support nanosecond precision (9 digits). |
createdBy | string ID of the user or service account who initiated the operation. |
modifiedAt | string (date-time) The time when the Operation resource was last modified. String in RFC3339 text format. The range of possible values is from To work with values in this field, use the APIs described in the Protocol Buffers reference. In some languages, built-in datetime utilities do not support nanosecond precision (9 digits). |
done | boolean (boolean) If the value is |
metadata | object Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
error | object The error result of the operation in case of failure or cancellation. includes only one of the fields error , response |
error. code |
integer (int32) Error code. An enum value of google.rpc.Code. |
error. message |
string An error message. |
error. details[] |
object A list of messages that carry the error details. |
response | object includes only one of the fields error , response The normal response of the operation in case of success. If the original method returns no data on success, such as Delete, the response is google.protobuf.Empty. If the original method is the standard Create/Update, the response should be the target resource of the operation. Any method that returns a long-running operation should document the response type, if any. |