Yandex Cloud
  • Services
  • Solutions
  • Why Yandex Cloud
  • Pricing
  • Documentation
  • Contact us
Get started
Language / Region
© 2022 Yandex.Cloud LLC
Yandex DataSphere
  • Getting started
  • Step-by-step instructions
    • All instructions
    • Project management
      • Creating a project
      • Installing dependencies
      • Managing computing resources
      • Setting up consumption limits for a project
      • Setting up consumption limits for a folder
      • Resizing project storage
      • Changing a name or description
      • Deleting a notebook or project
    • Sharing a notebook
      • Publishing a notebook
      • Exporting a project
    • Working with a notebook
      • Running sample code in a notebook
      • Versioning. Working with checkpoints
      • Clearing the interpreter state
      • Working with Git
    • Managing Docker images
      • Docker image for a project
      • Docker image in a cell
    • Connecting to data sources
      • Connecting to a ClickHouse database
      • Connecting to a PostgreSQL database
      • Connecting to S3 storage
    • Setting up integration with Data Proc
    • Working with confidential data
      • Creating a secret
      • Referencing a secret
      • Editing a secret
      • Copying a secret
      • Destroying a secret
    • Launching distributed training
    • Deploying models
      • Creating a node from a Python code cell
      • Configuring the node environment
      • Queries to nodes
  • Concepts
    • Overview
    • Project
    • List of pre-installed software
    • Available commands
    • #pragma service commands
    • Computing resource configurations
    • Integration with version and data control systems
    • Saving a state
    • Integration with Data Proc
    • Background operations
    • Datasets
    • Private data storage
    • Deploying models
    • Using TensorBoard in Yandex DataSphere
    • Distributed training
    • Cost management
    • Quotas and limits
  • Early access
    • Overview
    • Special background operations
  • Practical guidelines
    • All tutorials
    • Getting started with Yandex DataSphere
    • Voice biometrics
    • Evaluating the quality of STT models
    • Marking up audio files
    • Classification of images in video frames
  • API reference
    • Authentication in the API
    • gRPC
      • Overview
      • AppTokenService
      • FolderBudgetService
      • NodeService
      • ProjectDataService
      • ProjectService
      • OperationService
    • REST
      • Overview
      • AppToken
        • Overview
        • validate
      • FolderBudget
        • Overview
        • get
        • set
      • Node
        • Overview
        • execute
      • Project
        • Overview
        • create
        • delete
        • execute
        • get
        • getCellOutputs
        • getNotebookMetadata
        • getStateVariables
        • getUnitBalance
        • list
        • open
        • setUnitBalance
        • update
  • Access management
  • Pricing policy
  • Releases
  • Questions and answers
  1. Step-by-step instructions
  2. Managing Docker images
  3. Docker image for a project

Creating a custom Docker image for a project

Written by
Yandex.Cloud
  • Docker image requirements
  • Creating a Docker image

You can configure the environment to run your code using Docker images.

Yandex DataSphere lets you create repositories of Docker images in a project and select an image for the project. The selected image will be used when running code in all project notebooks.

Docker image requirements

For a Docker image to run and operate correctly in DataSphere, it must include:

  • A Python 3.7 or Python 3.8 installation.
  • A pip installation.
  • A Jupiter user.

Note

The Docker image templates presented in DataSphere already meet these requirements. If you are downloading an image from a third-party source or writing your own, include the required commands.

Creating a Docker image

To create a Docker image:

  1. Go to the Docker images tab.
  2. Click the icon.
  3. On the tab that opens:
    1. Enter a name for the repository, such as cuda.

    2. Set a tag for the image, such as 1.0.0.

    3. Select a Python installation script template: python_3_7 or python_3_8.

    4. Edit the Dockerfile section. For example, to install CUDA® packages, choose the python 3.8 template and add the following code:

      FROM ubuntu:18.04
      ENV DEBIAN_FRONTEND noninteractive
      RUN useradd -ms /bin/bash --uid 1000 jupyter\
       && apt update\
       && apt install -y python3.8-dev python3.8-distutils gnupg wget software-properties-common curl\
       && ln -s /usr/bin/python3.8 /usr/local/bin/python3\
       && curl https://bootstrap.pypa.io/get-pip.py | python3
      ENV LD_LIBRARY_PATH /usr/local/cuda-11.2/lib64:/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64
      RUN apt-get update &&\
       apt-get install -y -q xserver-xorg-core wget &&\
       wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin -O /etc/apt/preferences.d/cuda-repository-pin-600 &&\
       apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub &&\
       add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/ /" &&\
       apt-get update &&\
       apt-get -y install cuda libcudnn8 nvidia-cuda-toolkit &&\
       exit
      RUN pip install tensorflow-gpu==2.4.1
      
    5. Click Build.

      This creates a Docker image with CUDA® packages to use the GPU in computations.

    6. To use the image for your project, click Set image to project and then click Ok.

Make sure that the custom image environment is available in your project. For example, for the image with CUDA® packages, create and run a cell with the following code:

#!g1.1
import tensorflow as tf
tf.config.list_physical_devices('GPU')

Execution output:

...
[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]

Note

To return to the default environment, click Reset docker image in project on the Docker images tab.

Was the article helpful?

Language / Region
© 2022 Yandex.Cloud LLC
In this article:
  • Docker image requirements
  • Creating a Docker image