Yandex Cloud
  • Services
  • Solutions
  • Why Yandex Cloud
  • Blog
  • Pricing
  • Documentation
  • Contact us
Get started
Language / Region
Yandex project
© 2023 Yandex.Cloud LLC
Yandex Data Proc
  • Getting started
  • Tutorials
    • All practical guidelines
    • Working with jobs
      • Overview
      • Working with Hive jobs
      • Working with MapReduce jobs
      • Working with PySpark jobs
      • Working with Spark jobs
      • Running Apache Hive jobs
      • Running Spark applications
      • Running jobs from a remote host
    • Configuring networks for Data Proc
    • Using Yandex Object Storage in Data Proc
    • Exchanging data with Yandex Managed Service for ClickHouse
    • Importing data from Yandex Managed Service for MySQL clusters using Sqoop
    • Importing data from Yandex Managed Service for PostgreSQL clusters using Sqoop
    • "Using initialization scripts to configure GeeseFS in Data Proc"
    • Integration with Yandex DataSphere
  • Step-by-step instructions
    • All instructions
    • Information about existing clusters
    • Creating clusters
    • Connecting to a cluster
    • Updating clusters
    • Managing subclusters
    • Updating subclusters
    • Connecting to component interfaces
    • How to use Sqoop
    • Managing jobs
      • All jobs
      • Spark jobs
      • PySpark jobs
      • Hive jobs
      • MapReduce jobs
    • Deleting clusters
    • Working with logs
    • Monitoring the state of clusters and hosts
  • Concepts
    • Relationships between service resources
    • Host classes
    • Runtime environment
    • Data Proc component interfaces and ports
    • Jobs in Data Proc
    • Automatic scaling
    • Decommissioning subclusters and hosts
    • Network in Data Proc
    • Maintenance
    • Quotas and limits
    • Storage in Data Proc
    • Component properties
    • Logs in Data Proc
    • Initialization scripts
  • Access management
  • Pricing policy
  • API reference
    • Authentication in the API
    • gRPC
      • Overview
      • ClusterService
      • JobService
      • ResourcePresetService
      • SubclusterService
      • OperationService
    • REST
      • Overview
      • Cluster
        • Overview
        • create
        • delete
        • get
        • list
        • listHosts
        • listOperations
        • listUILinks
        • start
        • stop
        • update
      • Job
        • Overview
        • cancel
        • create
        • get
        • list
        • listLog
      • ResourcePreset
        • Overview
        • get
        • list
      • Subcluster
        • Overview
        • create
        • delete
        • get
        • list
        • update
  • Revision history
    • Service updates
    • Images
  • Questions and answers
  1. Concepts
  2. Data Proc component interfaces and ports

Component interfaces and ports in Data Proc

Written by
Yandex Cloud
  • Components and ports

Custom web interfaces of some Data Proc components, such as Hadoop, Spark, Yarn, and Zeppelin, are available on the MASTER cluster node. These interfaces can be used:

  • To manage and monitor cluster resources: YARN Resource Manager and HDFS Name Node.
  • To view job statuses and debug jobs: Spark History and JobHistory.
  • For collaboration, experiments, or ad-hoc operations: Apache Zeppelin.

Data Proc enables you to create clusters accessible from the internet or only from a cloud network. However, we recommend making service component interfaces inaccessible from outside Yandex Cloud in any configuration. You can connect to Data Proc component interfaces either using UI Proxy or an intermediate virtual machine.

UI proxy is a mechanism that lets you proxy cluster component interfaces with HTTP traffic encryption and authentication via the Yandex Cloud IAM. To access the interfaces, the user must be logged into Yandex Cloud, have cluster view permissions and the dataproc.user role.

UI Proxy is disabled by default. To take advantage of UI Proxy, enable it when creating or configuring a cluster and view a list of web interfaces available for connection.

Warning

You may need to additionally set up security groups to use UI Proxy.

Components and ports

Service Port
HBase Master 16010
HBase REST 8085
HDFS Name Node 9870
Hive Server2 10002
MapReduce Application History 19888
Oozie 11000
Spark History 18080
YARN Application History 8188
YARN Resource Manager 8088
Zeppelin 8890

Was the article helpful?

Language / Region
Yandex project
© 2023 Yandex.Cloud LLC