Visualizing geodata from a CSV file

This scenario analyzes public data on road accidents in Russia. In addition to identifying the facts of accident statistics in Russia, during the analysis you'll learn how to:

• Work with the DataLens key entities: Connections, Datasets, Charts, and Dashboards.
• Combine multiple sources at the level of a single dataset.
• Use the Date and time data type and edit chart-level groupings.
• Work with geodata: geopoints and geopolygons.
• Create public dashboards that will be available to everyone from any device without authentication.

To visualize and analyze the data, make sure you have a ready-to-use cloud and follow these steps:

1. Create a connection to a CSV file.
2. Create a dataset and configure its fields.
3. Create your first chart: a heat map.
4. Create your second chart: a bar chart.
5. Create line charts.
6. Create a dashboard.
7. Add charts to the dashboard.
8. Add selectors and create a dashboard.
9. Connect an additional source with region geolayers.
10. Add new fields to the dataset.
11. Create a chart using the new fields.
12. Add the new chart to the dashboard.
13. Publish the dashboard.

Before you start

To use DataLens, you have to sign up for Yandex.Cloud and create a billing account (without a billing account, DataLens is only available in demo mode):

1. Go to the management console. Then log in to Yandex.Cloud or sign up if you don't have an account yet.

2. On the billing page make sure that a billing account is linked and that its status is ACTIVE or TRIAL_ACTIVE. If you don't have a billing account, create one.

When you activate your billing account, you automatically switch to the Free DataLens plan. By default, you can create a DataLens instance for no more than 1 folder. If you haven't done this yet:

1. Select a folder on the cloud page.
2. Select the DataLens service.
3. Click Activate DataLens.

An instance of the service is created in the selected folder and you can go directly to the DataLens interface.

Learn more about clouds and folders.

Step 1. Create a connection to a CSV file

1. Download public accident statistics in CSV format. As of writing these instructions in February 2020, the most current dataset was for 2018. Unpack the downloaded archive. You can also download a ready-to-use file with data for 2018.

   

2. Open the DataLens homepage and click Create connection.

   

3. Select the CSV connection type.

   

4. Click Select CSV file and select the downloaded file. It may take up to several minutes to load it, depending on the speed of your internet connection.

   

   When loaded, you'll see a preview of the file data. To view it, use the scroll bars on the right and at the bottom.

   

5. In the upper-right corner, click Create.

   

The connection to the CSV file is created. The data is presented in the same form as in the file. To use it, you need to create a dataset.

Step 2. Create a dataset and configure its fields

The dataset will consist of a single source: the CSV file.

1. Click Create dataset.

   

2. Go to the Fields tab.

   

   Here you can add data fields and calculated fields, change field data types and aggregation rules, as well as rename fields.

   

3. Change the field names to Russian ones in the following order:

   • Region code
   • Region name
   • Road code
   • Road name
   • Road type
   • OKTMO code
   • Address
   • Accident type

   

4. The accident date and time are shown in the following fields:

   • crash_date: The date in numeric format like 20190218.
   • crash_time: The time written like 19:34.

   They won't be used in the dataset in this form. Click the visibility icon to hide them.

   

5. To display information about the accident date and time, add a new field.

   1. Click Add field.

      

   2. Specify the following data for it:

      • Name: Date and time
      • Formula: datetime_parse(str([crash_date])+'-'+str([crash_time]))

      

      Tip

      To review the documentation for these functions yourself, click Reference.

   3. Click Create.

   The preview section now correctly displays the accident date and time as a value of the Date and time type. The new Date and time field is at the top of the table. Click to edit the field formula.

   

6. Continue renaming the fields in the following order:

   • Cause of accident
   • Number of victims
   • Death toll
   • Number of vehicles
   • Number of participants

   Set the Sum aggregation type for the following fields:

   • Number of victims
   • Death toll
   • Number of vehicles
   • Number of participants

   

7. Add a field to display a geopoint.

   1. Click Add field at the top right.

      

   2. Enter data for the new field:

      • Name: Geopoint
      • Formula: GEOPOINT([latitude],[longitude])

      where [latitude] and [longitude] are dataset fields.

      

   3. Click Create.

      Note

      The example is based on ready-to-use data in coordinate format. In other cases, to convert data to coordinate format, you can use the geocoding function.

   4. Make sure that the table and preview contain the new Geopoint type field.

      

8. Add a field to display the number of accidents.

   1. Click Add field.

      

   2. Enter data for the new field:

      • Name: Number of accidents
      • Formula: SUM(1)

      

   3. Click Create.

   You don't need to set the aggregation type for this field: when you add it to the chart, DataLens applies the aggregation function automatically.

9. Add a field for data on deaths in road accidents.

   1. Click Add field.

      

   2. Enter data for the new field:

      • Name: Accident death rate
      • Formula: [Death toll]/[Number of accidents]

      

   3. Click Create

   Note

   You can create calculated fields directly in charts. However, it's more convenient to do this at the dataset level. This makes them available in all charts.

10. Save the dataset by clicking Save at the top right.

    

11. Specify the dataset name: Accidents. Click Create.

    

The dataset is created.

Step 3. Create your first chart: a heat map

Proceed to creating the first chart.

1. Click Create chart.

   

2. In the wizard window that opens, click on the field that defaults to Column chart and select the Map chart type.

   

3. Under Points, select the Points (heat map) layer type.

   

4. Drag the Geopoint field to the Geopoints section.

   

   That's your first chart on the map.

5. To save it, click the arrow icon next to the Save button and select Save as.

   

6. In the dialog box, name the chart Heat map and click Save.

   

Step 4. Create your second chart: a bar chart

1. Analyze the number of accidents by region.

   1. In the same window, select Bar chart as the chart type.

      

   2. Drag:

      • The Region name field to the Y section.
      • The Number of accidents field to the X section.
      • The Number of accidents field to the Sorting section.

      

      The chart shows the number of accidents by region. Most accidents occurred in the Moscow region.

      Note

      The legend may be missing for some of the chart lines. If this is the case, hover over the line and you'll see a tooltip with the region name.

   3. Click Save as to save the chart.

      

      Name the chart Number of accidents by region and save it.

      

2. Analyze the death rate in accidents by region.

   1. Replace the Number of accidents measure with Accident death rate by dragging and dropping.

      

      This time, Ingushetia is far ahead in the number of deaths, followed by Tyva and Kalmykia.

   2. Save the chart by clicking Save as

      

      and name it Death rate by region.

      

Step 5. Create line charts

Let's see how the number of accidents and related deaths are distributed by week, day of the week, and time of day.

1. Analyze the number of accidents and death rate by week.

   1. In the same window, select Line chart as the chart type.

      

   2. The chart management sections contain the previous values. Delete them by clicking .

      

   3. Drag the Date and time field to the X section and click the green calendar icon.

      

   4. In the Grouping field, choose Date part ⟶ Week and click Apply.

      

   5. Drag:

      • The Number of accidents field to the Y section.
      • The Accident death rate field to the Y2 section.

      

      Now you can see a chart with two graphs: the number of accidents and death rate. If you hover over a point on the chart, a tooltip appears with specific values.

   6. Save the chart by clicking Save as

      

      and name it Number of accidents and death rate by week.

      

2. Now analyze the number of accidents and death rate by day of the week.

   1. Change grouping by date and time to Date part ⟶ Day of week.

      

      This chart shows more obvious trends.

      

   2. Save the chart by clicking Save as

      

      and name it Number of accidents and death rate by day of week.

      

3. Analyze the number of accidents and death rate by hour of day.

   1. Change grouping by date and time to Date part ⟶ Hour.

      

      The chart shows that less accidents occur at night, but they result in more casualties.

      

   2. Save the chart by clicking Save as

      

      and name it Number of accidents and death rate by hour of day.

      

Step 6. Create a dashboard

1. Open the menu by clicking the icon to the left of the logo and select Dashboards.

   

2. Click Create

   

   and select Dashboard.

   

3. Name the dashboard Accidents in Russia and click Create.

   

Step 7. Add charts to the dashboard

1. Add the first chart.

   1. Click Add and select Chart from the drop-down list.

      

   2. In the Chart field, select the previously created Heat Map from the list of charts.

      

   3. Click Add.

      

      The chart is displayed on the dashboard.

2. Repeat the previous three steps for the Number of accidents by region, Number of accidents and death rate by week, Number of accidents and death rate by day of week, and Number of accidents and death rate by hour of day charts by adding them to the dashboard.

3. Create a switch for the Number of accidents by region and Death rate by region charts on the dashboard.

   1. Click the Number of accidents by region chart settings icon to open them.

      

   2. Click Add to the left of the screen.

      

   3. In the Chart field, click Select and choose the Death rate by region chart.

      

   4. Click Save.

      

Step 8. Add selectors and create a dashboard

Selectors let you filter data by value.

1. Add a selector to show statistics by region name.

   1. Click Add and choose Selector.

      

   2. Select Based on dataset, click Select, and choose the previously created Accidents dataset.

      

   3. In the Field property, choose Region name and select Show next to the selector name. Click Add.

      

   The selector is displayed on the dashboard as a rectangle.

2. Repeat the previous three steps for the fields:

   • Cause of accident
   • Accident type
   • Road type
   • Road name

3. You can drag and resize dashboard elements. Place the elements according to the screenshot or however is convenient for you and click Save to save the dashboard.

   

If you choose any values in the selectors, the charts will display data for these values.

You can switch the Number of accidents by region chart to Death rate by region.

Step 9. Connect an additional source with region geolayers

Heat maps aren't always the most informative. Upload the polygon reference and add a chart with a color fill for Russian regions.

1. Open the menu by clicking the button to the left of the logo and select Connections.

   

2. At the top right, click Create

   

   and select Connection.

   

3. Select the CSV file type.

   

   Download the Regions.csv file. Then, in the create connection window, upload it by clicking Select CSV file.

4. Take a look at a preview of the file data. For the Table header parameter, set the Available value. At the top right, click Create.

   

Step 10. Add new fields to the dataset

You need to add new fields to the previously created Accidents dataset.

1. In the left menu, select Datasets.

   

2. Choose the Accidents dataset.

   

3. Go to the Sources tab.

   

4. Click Add.

   

5. Choose Connections and then Regions.

   

   Since the connection data has not yet been merged, an error message may appear. Follow these steps.

6. Click the merge data icon.

   

7. Click Add link.

   

8. Select the fields to link: reg_name and Accident region. Then click Save.

   

   Make sure the data preview is displayed correctly.

   

9. Go to the Fields tab.

   

10. New fields are displayed at the bottom of the list. For the Polygon field, specify the Geopolygon data type.

    

11. Click Save to save the dataset.

    

Step 11. Create a chart using the new fields

1. Click Create chart.

   

2. Select the Map chart type.

   

3. Add polygons to the map. Drag the Polygon field from the Dimensions section to the Polygons section.

4. Change colors of the polygons based on the number of accidents. Drag the Number of accidents field from the Measures section to the Colors section.

5. Drag the following fields to the Tooltips section:

   • Region name
   • Number of accidents
   • Death toll
   • Number of victims
   • Number of vehicles
   • Number of participants
   • Accident death rate

   The chart shows a map with a color fill for regions. If you hover over a region, information for that region appears.

   

6. Click Save at the top right to save the chart.

   

7. Name the chart Map of regions and click Save once again.

   

Step 12. Add a new chart to the dashboard

1. Select Dashboards in the top-left menu.

   

2. Select the previously created dashboard with accident data.

   

3. Click Edit at the top right.

   

4. Click the Heat map settings icon to open them.

   

5. Click Add to the left of the screen.

   

6. In the Chart field, click Select and choose the last chart named Map of regions.

   

7. Click Save.

   

You can now switch the type of map data visualization: heat map or region fill.

Step 13. Publish the dashboard

1. To set up public access to the dashboard, click .

   

2. Enable access via link. You can copy and share the link that appears. Everyone can use the link to access the dashboard from any device and without authentication. Click Apply.

   

   Note

   You can enable or disable public access separately for each chart and dataset associated with the dashboard.