Time to Live (TTL)
This section describes how the TTL mechanism works and what its limits are. It also gives examples of commands and code snippets that can be used to enable, configure, and disable TTL.
How it works
YDB lets you specify a table column (TTL column), whose values set the lifetime of items. TTL automatically deletes the item from your table once the specified number of seconds passes after the time set in the TTL column.
An item with the
NULL value in the TTL column is never deleted.
The timestamp for deleting a table item is determined by the formula:
expiration_time = valueof(ttl_column) + expire_after_seconds
TTL doesn't guarantee that the item will be deleted exactly at
expiration_time, it might happen later. If it's important to exclude logically obsolete but not yet physically deleted items from the selection, use request-level filtering.
Data is deleted by the Background Removal Operation (BRO), consisting of two stages:
- Checking the values in the TTL column.
- Deleting expired data.
The BRO has the following properties:
- The concurrency unit is a table shard.
- For tables without secondary indexes, blind deletes are performed. This means that if the TTL column value is updated in-between stages 1 and 2 (for example, with an
UPDATErequest), the updated value is not rechecked.
- For tables with secondary indexes, the delete stage is a distributed transaction.
- At every point in time, a BRO is run in no more than 1 instance per table.
- BROs are run no more than once per hour for the same shard.
- For tables with secondary indexes, data consistency is guaranteed.
- The TTL column must be of one of the following types:
- You can't specify multiple TTL columns.
- You can't delete the TTL column. However, if this is required, you should first disable TTL for the table.
Currently, you can manage TTL settings using:
Enabling TTL for an existing table
In the example below, the items of the
mytable table will be deleted an hour after the time set in the
$ ydb -e <endpoint> -d <database> table ttl set --column created_at --expire-after 3600 mytable
session.alter_table('mytable', set_ttl_settings=ydb.TtlSettings().with_date_type_column('created_at', 3600))
Enabling TTL for a newly created table
For a newly created table, you can pass TTL settings along with the table description:
session.create_table( 'mytable', ydb.TableDescription() .with_column(ydb.Column('id', ydb.OptionalType(ydb.DataType.Uint64))) .with_column(ydb.Column('expire_at', ydb.OptionalType(ydb.DataType.Timestamp))) .with_primary_key('id') .with_ttl(ydb.TtlSettings().with_date_type_column('expire_at')) )
$ ydb -e <endpoint> -d <database> table ttl drop mytable