Status monitoring of geographically distributed devices
In this tutorial, you will configure status monitoring for devices, such as vending machines, that are connected to Yandex IoT Core and located at various sites in a city. Sensors are emulated using the Yandex Cloud Functions service. If your sensors are connected, use them. You can monitor the status of vending machines on the Yandex DataLens map and charts.
The source code of this scenario is available on GitHub
To configure monitoring of sensor readings in the server room:
- Prepare your cloud.
- Create the required resourcesYandex IoT Core.
- Create a device emulator in Yandex Cloud Functions.
- Create a cluster in Yandex Managed Service for PostgreSQL.
- Create a function for processing data in Yandex Cloud Functions.
- Configure monitoring in Yandex DataLens.
If you no longer need the resources you created, delete them.
Prepare your cloud
Sign up for Yandex Cloud and create a billing account:
- Go to the management console
and log in to Yandex Cloud or create an account if you do not have one yet. - On the Billing
page, make sure you have a billing account linked and it has theACTIVE
orTRIAL_ACTIVE
status. If you do not have a billing account, create one.
If you have an active billing account, you can go to the cloud page
Learn more about clouds and folders.
If you do not have the command line interface Yandex Cloud yet, install and initialize it.
Required paid resources
The cost includes:
- Fee for the number of messages Yandex IoT Core (see pricing );
- Fee for the number of function calls Yandex Cloud Functions (see pricing );
- Fee for computing resources and cluster storage in Yandex Managed Service for PostgreSQL (see pricing).
Create the required Yandex IoT Core resources
To use Yandex IoT Core, you need to create a registry and devices that will exchange data and commands.
Create a registry and configure authorization using your login and password
To create a registry:
-
In the management console
, select the folder where you are running the scenario. -
Select Yandex IoT Core.
-
Click Create registry.
-
In the Name field, enter a name for the registry, e.g.,
my-registry
. -
In the Password field, set the password to access the registry.
The password must be at least 14 characters long and contain lowercase letters, capital letters, and numbers.
To create a password, you can use the password generator . -
Save the password locally or remember it. The service does not show passwords after the registry is created.
-
(optional) In the Description field, add additional information about the registry.
-
Click Create.
You can also complete authorization using certificates. Learn more about authorization in Yandex IoT Core.
Create a device and configure authorization using your login and password
Create three devices: my-device-1
, my-device-2
, and my-device-3
.
To create a device:
-
In the management console
, select the folder where you are running the scenario. -
Select Yandex IoT Core.
-
Select the registry you created in the previous step.
-
In the left part of the window, select Devices.
-
Click Add device.
-
In the Name field, enter the device name, e.g.,
my-device-1
. -
In the Password field, set the password to access the device.
To create a password, you can use the password generator
.
The password must be at least 14 characters long and contain lowercase letters, capital letters, and numbers. -
Save the password locally or remember it. The service does not show passwords after the registry is created.
-
(optional) In the Description field, add additional information about the device.
-
(optional) Add aliases:
-
Click Add alias.
-
Click Edit.
-
Fill out the fields by providing an alias, e.g.,
events
, and the topic type after$devices/<deviceID>
, e.g.,events
.You can use the
events
alias instead of the$devices/<deviceID>/events
topic. -
Repeat the steps for each alias you add.
-
-
Click Create.
-
Repeat the steps for each device you want to create.
You can also complete authorization using certificates. Learn more about authorization in Yandex IoT Core.
Create a device emulator in Cloud Functions
The emulator will send data from the devices to the Yandex Managed Service for PostgreSQL cluster.
You will need to:
- Create and test a function that emulates transmitting data from each device sensor.
- Create a trigger to call the emulation function once per minute.
Create a function that emulates transmitting data from the device
To create a function:
- In the management console
, select the folder where you are running the scenario. - Select Cloud Functions.
- In the left part of the window, select Functions.
- Click Create function.
- In the Name field, enter the name of the function, e.g.,
my-device-emulator-function
. - (optional) In the Description field, add additional information about the function.
- Click Create.
- In the Editor window that opens, select
nodejs12
from the Runtime environment list. - Select Method: Code editor.
- In the left part of the Code editor window, click Create file.
- In the New file window that opens, enter the name
device-emulator.js
. - Click Create.
- Select the created file in the left part of the Code editor window.
- In the right part of the Code editor window, insert the function code from GitHub
. - In the Editor window, enter
device-emulator.handler
in the Entry point field. - In the Timeout, sec field, enter
10
. - In the Memory field, leave the value
128 MB
. - Create a service account that the function will use to send data to Yandex IoT Core:
- Click Create an account.
- In the Create service account window that opens, enter the account name in the Name field, e.g.,
my-emulator-function-service-account
. - Add the
viewer
andiot.devices.writer
roles for retrieving the device list and writing to resources:- Click the
icon. - Select a role from the list.
- Click Create.
- Click the
- Set Environment variables for each sensor in the server room:
-
Click Add environment variable.
-
Fill in the Key and Value fields for the environment variables:
Key Description Value CASH_DRAWER_SENSOR_VALUE
Fullness percentage of the cash drawer. 67.89
COIN_DRAWER_SENSOR_VALUE
Fullness percentage of the coin drawer. 67.89
TEMPERATURE_SENSOR_VALUE
Baseline temperature in the pickup box. 10.34
POWER_SENSOR_VALUE
Baseline mains voltage. 24.12
SERVICE_DOOR_SENSOR_VALUE
State of the open service door sensor. False
ITEM1_SENSOR_VALUE
Stock of product 1. 50.65
ITEM2_SENSOR_VALUE
Stock of product 2. 80.97
ITEM3_SENSOR_VALUE
Stock of product 3. 30.33
ITEM4_SENSOR_VALUE
Stock of product 4. 15.15
REGISTRY_ID
ID of the registry you created. For details, see the service
management console Yandex IoT Core
-
- In the top-right part of the window, click Create version.
Test the emulation function
To test the function:
-
(optional) To get detailed information from the sensors, subscribe the registry to the Yandex IoT Core device topic, where
$devices/<deviceID>/events
is the device topic,<deviceID>
is the device ID in the service:CLIRun this command:
yc iot mqtt subscribe \ --username <registry ID> \ --password <registry password> \ --topic '$devices/<device ID>/events' \ --qos 1
Where:
--username
and--password
: Parameters for authorization using a username and password.--topic
: Device topic for sending data.--message
: Message text.--qos
: Quality of service (QoS).
Learn more about subscription to device topics inYandex IoT Core .
-
In the management console
, select the folder where you are running the scenario. -
Select Cloud Functions.
-
In the left part of the window, select Testing.
-
In the Version tag field, select
$latest
(most recent function). -
Click Run test.
If the function is successful, the Function status field shows Done and the Function output field shows the following:
{
"statusCode" : 200
}
If you subscribe to the Yandex IoT Core device topic, you get the following JSON:
{
"DeviceId":"arealt9f3jh445it1laq",
"TimeStamp":"2020-07-21T22:38:12Z",
"Values":[
{"Type":"Bool","Name":"Service door sensor","Value":"false"},
{"Type":"Float","Name":"Power Voltage","Value":"24.12"},
{"Type":"Float","Name":"Temperature","Value":"10.34"},
{"Type":"Float","Name":"Cash drawer fullness","Value":"67.89"},
{"Type":"Float","Name":"Coin drawer fullness","Value":"67.89"},
{"Items":[
{"Type":"Float", "Id":"1","Name":"Item 1","Fullness":"50.65"},
{"Type":"Float", "Id":"2","Name":"Item 2","Fullness":"80.97"},
{"Type":"Float", "Id":"3","Name":"Item 3","Fullness":"30.33"},
{"Type":"Float", "Id":"4","Name":"Item 4","Fullness":"15.15"},
]}
]
}
Learn more about MQTT topics in Yandex IoT Core.
Create a trigger that invokes the function once per minute
To create a trigger:
- In the management console
, select the folder where you are running the scenario. - Select Cloud Functions.
- Select Triggers.
- Click Create trigger.
- In the Name field, enter a name for the trigger, e.g.,
my-emulator-function-trigger
. - (optional) In the Description field, add additional information about the trigger.
- Select Type: Timer.
- In the Cron expression field, enter
* * * * ? *
(invoke once per minute). - Under Function settings, enter the parameters previously set for the function:
- Function:
my-device-emulator-function
. - Function version tag:
$latest
. - Service account:
my-emulator-function-service-account
.
- Function:
- (optional) Set parameters under Repeat request settings and Dead Letter Queue settings. They provide data security.
- Repeat request settings are useful if you want to call the function again when the current function request fails.
- Dead Letter Queue settings are required to forward messages that consumers could not process in standard queues.
You can configure a standard message queue as a DLQ. If you do not have a message queue yet, create one in Yandex Message Queue.
- Click Create trigger.
Create a cluster in Yandex Managed Service for PostgreSQL
The example is based on the minimum values of host parameters. For real tasks, it's better to choose hosts with a guaranteed 100% vCPU share.
To create a cluster:
- In the management console
, select the folder where you are running the scenario. - Select Yandex Managed Service for PostgreSQL.
- Select Clusters.
- Click Create cluster.
- Under Basic parameters, give your cluster a name in the Cluster name field, e.g.,
my-pg-database
. - (optional) In the Description field, add additional information about the cluster.
- In the Environment list, select
PRODUCTION
. - In the Version list, select
12
. - Under Host class, select
Intel Ice Lake
in the Platform list. - Select the VM type: on the burstable tab, select b2.medium.
- Under Storage size, go to the network-hdd tab.
- Set the storage size to 10 GB.
- Under Database, enter:
-
In the DB name field, enter
db1
. -
In the User name field, enter
user1
. -
In the Password field, set the password to access the database.
Make sure to save your password, as you will need it later.
-
- Do not change the values in the Locale for sorting and Charset locale fields. The default value is
C
. - Under Network, select
default
in the list. - Under Hosts, set up a connection to your database via a public IP address, which is required to access the database from Yandex Cloud Functions.
- On the right side of the availability zone line, click
. - In the window that opens, select an Availability zone, e.g.,
ru-central1-a
. - Select a Subnet, e.g.,
default-ru-central1-a
. - Enable Public access.
- Click Save.
- On the right side of the availability zone line, click
- Enable DataLens access.
- Click Create cluster.
It may take a few minutes to create a cluster. As a result, a window with cluster details will open.
Create a function for processing data in Yandex Cloud Functions
The function processes data from devices.
Create a function for processing received data
To create a function:
- In the management console
, select the folder where you are running the scenario. - Select Cloud Functions.
- In the left part of the window, select Functions.
- Click Create function.
- In the Name field, enter the name of the function, e.g.,
my-db-function
. - (optional) In the Description field, add additional information about the function.
- Click Create.
- In the Editor window that opens, select
python37
in the Runtime environment list. - Select Method: go to the Code editor tab.
- In the left part of the Code editor window, click Create file.
- In the New file window that opens, enter the name
myfunction.py
. - Click Create.
- In the left part of the Code editor window, select the created file.
- In the right part of the window, insert the function code from GitHub
. - In the Entry point field of the Editor window, specify
myfunction.msgHandler
. - In the Timeout, sec field, enter
10
. - In the Memory field, leave the value
128 MB
. - Create a service account to be used for the function to process data from the device:
- Click Create an account.
- In the Create service account window that opens, enter the account name in the Name field, e.g.,
my-db-function-service-account
. - Add the
serverless.functions.invoker
andeditor
roles for invoking functions and editing resources:- Click the
icon. - Select a role from the list.
- Click Create.
- Click the
- Configure Environment variables. You can view database connection parameters in Yandex Managed Service for PostgreSQL.
-
Click Add environment variable.
-
Fill in the Key and Value fields for the environment variables:
Key Description Value VERBOSE_LOG
Turning data logging on and off. True
DB_HOSTNAME
Hostname in Yandex Managed Service for PostgreSQL. For details, see the
management console for Yandex Managed Service for PostgreSQLDB_PORT
Port to connect to the cluster in Yandex Managed Service for PostgreSQL. 6432
DB_NAME
Cluster name in Yandex Managed Service for PostgreSQL. db1
DB_USER
Username to connect to the cluster in Yandex Managed Service for PostgreSQL. user1
DB_PASSWORD
Password to connect to the cluster in Yandex Managed Service for PostgreSQL. The password you set in Yandex Managed Service for PostgreSQL.
-
- In the top-right part of the window, click Create version.
Test the data processing function
To test the function:
-
In the management console
, select the folder where you are running the scenario. -
Select Cloud Functions.
-
In the left part of the window, select Testing.
-
In the Version tag field, select
$latest
(most recent function). -
In the Payload field, insert the following data:
{ "messages": [ { "event_metadata": { "event_id": "160d239876d9714800", "event_type": "yandex.cloud.events.iot.IoTMessage", "created_at": "2020-05-08T19:16:21.267616072Z", "folder_id": "b112345678910" }, "details": { "registry_id": "are1234567890", "device_id": "are0987654321", "mqtt_topic": "$devices/are0987654321/events", "payload": "ewogICAgICAgICAgICAiRGV2aWNlSWQiOiJhcmU1NzBrZTA1N29pcjg1bDlmciIsCiAgICAgICAgICAgICJUaW1lU3RhbXAiOiIyMDIwLTA2LTExVDExOjA3OjIwWiIsCiAgICAgICAgICAgICJWYWx1ZXMiOlsKICAgICAgICAgICAgICAgIHsiVHlwZSI6IkJvb2wiLCJOYW1lIjoiU2VydmljZSBkb29yIHNlbnNvciIsIlZhbHVlIjoiRmFsc2UifSwKICAgICAgICAgICAgICAgIHsiVHlwZSI6IkZsb2F0IiwiTmFtZSI6IlBvd2VyIFZvbHRhZ2UiLCJWYWx1ZSI6IjI1LjA2In0sCiAgICAgICAgICAgICAgICB7IlR5cGUiOiJGbG9hdCIsIk5hbWUiOiJUZW1wZXJhdHVyZSIsIlZhbHVlIjoiMTEuMjEifSwKICAgICAgICAgICAgICAgIHsiVHlwZSI6IkZsb2F0IiwiTmFtZSI6IkNhc2ggZHJhd2VyIGZ1bGxuZXNzIiwiVmFsdWUiOiI2Ny44OSJ9LAogICAgICAgICAgICAgICAgeyJJdGVtcyI6WwogICAgICAgICAgICAgICAgICAgIHsiVHlwZSI6IkZsb2F0IiwgIklkIjoiMSIsIk5hbWUiOiJJdGVtIDEiLCJGdWxsbmVzcyI6IjUwLjY1In0sCiAgICAgICAgICAgICAgICAgICAgeyJUeXBlIjoiRmxvYXQiLCAiSWQiOiIyIiwiTmFtZSI6Ikl0ZW0gMiIsIkZ1bGxuZXNzIjoiODAuOTcifSwKICAgICAgICAgICAgICAgICAgICB7IlR5cGUiOiJGbG9hdCIsICJJZCI6IjMiLCJOYW1lIjoiSXRlbSAzIiwiRnVsbG5lc3MiOiIzMC4zMyJ9LAogICAgICAgICAgICAgICAgICAgIHsiVHlwZSI6IkZsb2F0IiwgIklkIjoiNCIsIk5hbWUiOiJJdGVtIDQiLCJGdWxsbmVzcyI6IjE1LjE1In0KICAgICAgICAgICAgICAgIF19CiAgICAgICAgICAgICAgICBdCiAgICAgICAgICAgIH0=" } } ] }
-
Click Run test.
If the function is successful, the Function status field shows Done and the Function output field shows the following:
{
"statusCode" : 200 ,
"headers" : {
"Content-Type" : "text/plain"
},
"isBase64Encoded" : false
}
View the data processing result in Managed Service for PostgreSQL
To view the data processing result:
- In the management console
, select the folder where you are running the scenario. - Select Yandex Managed Service for PostgreSQL.
- Go to the SQL section.
A table with data processing results is displayed in the right part of the window.
Create a trigger to invoke the data processing function
The trigger will call the function when messages appear in the device topic.
To create a trigger:
- In the management console
, select the folder where you are running the scenario. - Select Cloud Functions.
- Select Triggers.
- Click Create trigger.
- In the Name field, enter a name for the trigger, e.g.,
my-db-func-trigger
. - (optional) In the Description field, add additional information about the trigger.
- Select a Type: Yandex IoT Core.
- Under Message settingsYandex IoT Core, enter the parameters previously set for the registry and device:
-
Registry:
my-registry
. -
Device:
Any device
. -
MQTT topic:
$devices/#
.Learn more about MQTT topics in Yandex IoT Core.
-
- Under Function settings, enter the parameters previously set for the function:
- Function:
my-database-function
. - Function version tag:
$latest
. - Service account:
my-db-func-trigger-service-account
.
- Function:
- (optional) Set parameters under Repeat request settings and Dead Letter Queue settings. They provide data security.
- Repeat request settings are useful if you want to call the function again when the current function request fails.
- Dead Letter Queue settings are required to forward messages that consumers could not process in standard queues.
You can configure a standard message queue as a DLQ. If you do not have a message queue yet, create one in Yandex Message Queue.
- Click Create trigger.
View the trigger result in Managed Service for PostgreSQL
Shortly after creating the trigger, you can check how it works.
To view the trigger result:
- In the management console
, select the folder where you are running the scenario. - Select Yandex Managed Service for PostgreSQL.
- Go to the SQL section.
In the right part of the window, you'll see an updated table with a large amount of data.
Configure monitoring in Yandex DataLens
Set up a connection to Managed Service for PostgreSQL
To set up a Yandex DataLens connection to Managed Service for PostgreSQL:
-
In the management console
, select the folder where you are running the scenario. -
Select Yandex DataLens.
-
Click Create connection.
-
Select the PostgreSQL connector.
When connecting to an external data source (which is not a Yandex Cloud resource), grant access to the source for the IP address rangesDataLens.
-
Enter a name for the connection:
MyPGConnection
. -
Select a Connection: Select in cloud.
-
Under Cluster, select
my-pg-database
. -
Under Hostname, select the host you created in Yandex Managed Service for PostgreSQL.
-
In the Port field, enter
6432
. -
Under Database name, select
db1
. -
Under Username, select
user1
. -
In the Password field, enter the password you set to access the cluster in Yandex Managed Service for PostgreSQL.
-
Click Create. The connection will appear in the list.
Create a dataset
To create a dataset:
- In the management console
, select the folder where you are running the scenario. - Select Yandex DataLens.
- Click Create dataset.
- On the left side of the screen, click
Add. - Select
MyPGConnection
. - In the left part of the window, select the
public.iot_events
andpublic.iot_position
tables, and drag them to the right. - Click Save.
- In the window that opens, set the name of the dataset to
My-pg-dataset
and click Create.
The dataset appears in the list.
Create a chart for temperature and mains voltage
To create a chart for temperature and mains voltage:
- In the management console
, select the folder where you are running the scenario. - Select Yandex DataLens.
- Click Create chart.
- Under Dataset, select
My-pg-dataset
, which you created earlier. - Select Line chart as your chart type.
- From the Dimensions section in the left part of the window, drag the dimension to the Line chart section:
-
event_datetime
to the X section.A timeline will appear in the lower part of the chart on the X-axis.
-
temperature
andpower_voltage
to the Y section.The temperature and mains voltage values will appear on the Y-axis of the chart.
-
- Click Save.
- In the window that opens, set the chart name or use the generated
My-pg-dataset — Line chart
name and click Save.
Create a chart with a map
To create a chart with a map:
- In the management console
, select the folder where you are running the scenario. - Select Yandex DataLens.
- In the left part of the window, select Datasets.
- In the list, select
My-pg-dataset
. - Go to the Fields tab.
- In the right part of the window, click Add field.
- In the window that opens, in the Field name field, enter
Position
. - In the Formula field, insert
GEOPOINT([latitude],[longitude])
. - Click Create.
- Click Save.
- In the top-right corner, click Create chart.
- Select the Map chart type.
- From the Dimensions section in the left part of the window, drag the measurements to the Map section:
-
Position
, to the Geopoints section. -
item1_fullness
,item2_fullness
,item3_fullness
,item4_fullness
,cash_drawer
, andcoin_drawer
, to the Tooltips section.In the right part of the window, you will see a scalable map with vending machines marked with points on the map and tooltips marked with legend lines.
-
- Click Save.
- In the window that opens, set the chart name or use the generated
My-pg-dataset — Map
name and click Save.
Create a dashboard
To create a dashboard:
- In the management console
, select the folder where you are running the scenario. - Select Yandex DataLens.
- Click Create dashboard.
- In the window that opens, enter the dashboard name
MyDash
. - Add the following charts to the dashboard:
My-pg-dataset — Map
andMy-pg-dataset — Line chart
, which you created at the previous steps:-
In the Add dropdown list at the top of the window, select Chart.
-
In the dropdown Chart list, select the
My-pg-dataset — Map
chart.The chart name will appear in the Name field.
-
(optional) In the Description field, enter a description for the chart.
-
Click Add.
-
Repeat the steps to add the
My-pg-dataset — Line chart
chart.
-
- Configure the selector:
- In the Add dropdown list at the top of the window, select Selector.
- In the Name field, enter Device.
- In the Dataset list, select
My-pg-dataset
. - In the Field list, select
device_id
. - In the Default value list, select the ID of the device you created in Yandex IoT Core.
- Click Add.
- Configure links:
-
At the top of the window, click Links.
-
In the list at the top of the window that opens, select the
My-pg-dataset — Point map
chart. -
In the dropdown In.link list, select
Ignore
.Maps do not support the Links parameter.
-
Click Save.
-
From the list at the top of the window, select the
My-pg-dataset — Line chart
chart. -
In the In.link dropdown list, select
In.link
. -
Click Save.
-
- Click Save.
Learn more about dashboards Yandex DataLens.