Yandex Cloud
  • Services
  • Solutions
  • Why Yandex Cloud
  • Pricing
  • Documentation
  • Contact us
Get started
Language / Region
© 2022 Yandex.Cloud LLC
Yandex Data Proc
  • Practical guidelines
    • Working with jobs
      • Overview
      • Working with Hive jobs
      • Working with MapReduce jobs
      • Working with PySpark jobs
      • Working with Spark jobs
      • Using Apache Hive
      • Running Spark applications
      • Running applications from a remote host
    • Configuring networks for Data Proc clusters
    • Using Yandex Object Storage in Data Proc
  • Step-by-step instructions
    • All instructions
    • Information about existing clusters
    • Creating clusters
    • Connecting to clusters
    • Editing clusters
    • Updating subclusters
    • Managing subclusters
    • Sqoop usage
    • Managing jobs
      • All jobs
      • Spark jobs
      • PySpark jobs
      • Hive jobs
      • MapReduce jobs
    • Deleting clusters
    • Monitoring the state of a cluster and hosts
    • Working with logs
  • Concepts
    • Data Proc overview
    • Host classes
    • Hadoop and component versions
    • Component interfaces and ports
    • Component web interfaces
    • Jobs in Data Proc
    • Autoscaling
    • Decommissioning subclusters and hosts
    • Network in Data Proc
    • Quotas and limits
    • Storage in Data Proc
    • Component properties
    • Logs in Data Proc
    • Initialization scripts
  • Access management
  • Pricing policy
  • API reference
    • Authentication in the API
    • gRPC
      • Overview
      • ClusterService
      • JobService
      • ResourcePresetService
      • SubclusterService
      • OperationService
    • REST
      • Overview
      • Cluster
        • Overview
        • create
        • delete
        • get
        • list
        • listHosts
        • listOperations
        • listUILinks
        • start
        • stop
        • update
      • Job
        • Overview
        • cancel
        • create
        • get
        • list
        • listLog
      • ResourcePreset
        • Overview
        • get
        • list
      • Subcluster
        • Overview
        • create
        • delete
        • get
        • list
        • update
  • Releases
    • Images
  • Questions and answers
  1. Concepts
  2. Component web interfaces

Data Proc component web interfaces

Written by
Yandex Cloud

    Custom web interfaces of some Data Proc components, such as Hadoop, Spark, Yarn, and Zeppelin, are available on the MASTER cluster node. These interfaces can be used:

    • To manage and monitor cluster resources: YARN Resource Manager and HDFS Name Node.
    • To view job statuses and debug jobs: Spark History and JobHistory.
    • For collaboration, experiments, or ad-hoc operations: Apache Zeppelin.

    For more information about the interfaces, see Working with component network interfaces.

    You can connect to the web interfaces of Data Proc components using either an intermediate VM or UI proxy in Data Proc version 1.2 and higher.

    UI proxy is a mechanism that lets you proxy cluster component interfaces with HTTP traffic encryption and authentication via the Yandex Cloud IAM. To access the interfaces, the user must be logged into Yandex Cloud, have cluster view permissions and the dataproc.user role.

    UI Proxy is disabled by default. To use it, enable it when creating or configuring a cluster.

    Interfaces:

    Service URL
    HDFS Name Node https://cluster-{cluster_id}.dataproc-ui.yandexcloud.net/gateway/default-topology/hdfs/
    YARN Resource Manager https://cluster-{cluster_id}.dataproc-ui.yandexcloud.net/gateway/default-topology/yarn/
    Livy https://cluster-{cluster_id}.dataproc-ui.yandexcloud.net/gateway/default-topology/livy/
    YARN Application History https://cluster-{cluster_id}.dataproc-ui.yandexcloud.net/gateway/default-topology/jobhistory/
    Hive Server2 https://cluster-{cluster_id}.dataproc-ui.yandexcloud.net/gateway/default-topology/hive/
    Zeppelin https://cluster-{cluster_id}.dataproc-ui.yandexcloud.net/gateway/default-topology/zeppelin/
    Oozie https://cluster-{cluster_id}.dataproc-ui.yandexcloud.net/gateway/default-topology/oozieui/

    Warning

    You may need to additionally set up security groups to use UI Proxy.

    Was the article helpful?

    Language / Region
    © 2022 Yandex.Cloud LLC