Yandex Data Streams
Scalable data stream management in real time.
Data streaming bus
Read and write data in near real time. Adjust data throughput and storage times to meet your needs.
Elastic resource allocation
Scale resources for processing data streams: from small streams of 100 KB/s to streams of 10 MB/s at the Preview stage.
Data stream management
Deliver a single stream to multiple targets with different retention policies using Yandex Data Transfer.
Questions and answers
What does Data Streams do?
Yandex Data Streams can continuously collect data from such sources as browsing histories, application and system logs, and social media feeds.
Yandex Data Streams can continuously collect data from such sources as browsing histories, application and system logs, and social media feeds.
What kinds of data can be transmitted with Data Streams?
- Access logs. All user activities for the previous year are saved in Object Storage cold storage whereas data for the previous month is saved in the OLAP ClickHouse DB.
- Application logs. To meet compliance requirements, application logs are saved long-term in Object Storage cold storage.
- Telemetry data from devices (mobiles phones and IoT) are delivered to Data Streams over HTTP and from there, to different data storage and processing systems.
- Access logs. All user activities for the previous year are saved in Object Storage cold storage whereas data for the previous month is saved in the OLAP ClickHouse DB.
- Application logs. To meet compliance requirements, application logs are saved long-term in Object Storage cold storage.
- Telemetry data from devices (mobiles phones and IoT) are delivered to Data Streams over HTTP and from there, to different data storage and processing systems.
How do I send data?
Using standard log delivery solutions: fluentd, logstash, and others. Data Streams is also compatible with Amazon Kinesis Data Streams, so data can be sent by any other applications using the AWS Kinesis SDK or the standard HTTP protocol.
Using standard log delivery solutions: fluentd, logstash, and others. Data Streams is also compatible with Amazon Kinesis Data Streams, so data can be sent by any other applications using the AWS Kinesis SDK or the standard HTTP protocol.
What is the difference between Data Streams and Apache Kafka®?
Both Data Streams and Apache Kafka® input data streams and transmits them to consumers.
Unlike Apache Kafka, Data Streams does not require dedicated virtual machines, but operates with the concept of a data stream. All data streams in Data Streams are reserved for all availability zones.
Data Streams integrates with other Yandex.Cloud services, enabling you to use a single management console to configure data from Data Streams to be stored in target managed DBs.
Both Data Streams and Apache Kafka® input data streams and transmits them to consumers.
Unlike Apache Kafka, Data Streams does not require dedicated virtual machines, but operates with the concept of a data stream. All data streams in Data Streams are reserved for all availability zones.
Data Streams integrates with other Yandex.Cloud services, enabling you to use a single management console to configure data from Data Streams to be stored in target managed DBs.