Neoflex Datagram is a data integration platform that allows you to design, develop and execute data processing tasks from any source using an easy-to-use graphical editor.
This solution comes with a built-in Apache Spark cluster for task execution. To use Serverless Spark of Yandex DataProc services, use the Neoflex Datagram @ DataProc solution.
Neoflex Datagram provides an environment for developing graphical data transformation processes. A task developed as a set of diagrams is transformed into Scala code, transformed into executable modules and executed on Apache Oozie, Apache Livy. The product contains a large number of built-in connectors: Oracle, Microsoft SQL Server, PostgreSQL, Hadoop, HBase, XML, Excel, CSV, etc. and allows you to load data based on as-is source metadata, without going down to the level of source-to-target mapping development.
Neoflex Datagram allows you to develop data processing applications in both batch and streaming modes. Apache Kafka and Spark Streaming are used to support streaming. The product also features native BRMS integration, which allows real-time business rules to be applied to event streams.
Neoflex Datagram provides convenient debugging and optimization tools:
• View content, structure of sources and receivers;
• Tracking the origin of data flow objects to individual fields (lineage);
• Partial conversion with viewing intermediate results;
• Execution of individual steps and branches of transformations;
• View the generated application code;
• Automatic validation of transformation;
• Support for Spark Catalyst Optimizer.
Data ingestion into analytical systems
Databases migration to the Cloud
Data Pipelines visual design
Streaming data processing
BigData solutions development
Neoflex provides support for Yandex.Cloud Datagram customers. You may contact us by email firstname.lastname@example.org. Support engineers provide answers from 9 to 18 during working days.