description of the "ydb workload transfer topic-to-table" command

Описание команды "ydb workload transfer topic-to-table"

4 files changed, 147 insertions, 2 deletions

diff --git a/ydb/docs/en/core/reference/ydb-cli/commands/workload/_includes/index.md b/ydb/docs/en/core/reference/ydb-cli/commands/workload/_includes/index.md
index bdb564dd5d..239b6251a4 100644
--- a/ydb/docs/en/core/reference/ydb-cli/commands/workload/_includes/index.md
+++ b/ydb/docs/en/core/reference/ydb-cli/commands/workload/_includes/index.md
@@ -25,4 +25,5 @@ The following types of load tests are supported at the moment:
 * [Key-value](../../../workload-kv.md): Key-Value load.
 * [ClickBench](../../../workload-click-bench.md): ClickBench analytical benchmark (https://github.com/ClickHouse/ClickBench).
 * [TPC-H](../../../workload-tpch.md): TPC-H benchmark (https://www.tpc.org/tpch/).
-* [Topic](../../../workload-topic.md): Topic load.
\ No newline at end of file
+* [Topic](../../../workload-topic.md): Topic load.
+* [Transfer](../../../workload-transfer.md): Transfer load.
diff --git a/ydb/docs/en/core/reference/ydb-cli/toc_i.yaml b/ydb/docs/en/core/reference/ydb-cli/toc_i.yaml
index cc97e600b7..0b974a0fe5 100644
--- a/ydb/docs/en/core/reference/ydb-cli/toc_i.yaml
+++ b/ydb/docs/en/core/reference/ydb-cli/toc_i.yaml
@@ -121,5 +121,7 @@ items:
       href: workload-kv.md
     - name: Topic load
       href: workload-topic.md
+    - name: Transfer load
+      href: workload-transfer.md
     - name: TPC-H load
       href: workload-tpch.md
diff --git a/ydb/docs/en/core/reference/ydb-cli/workload-transfer.md b/ydb/docs/en/core/reference/ydb-cli/workload-transfer.md
new file mode 100644
index 0000000000..15b8b8fcce
--- /dev/null
+++ b/ydb/docs/en/core/reference/ydb-cli/workload-transfer.md
@@ -0,0 +1,140 @@
+# Transfer load
+
+Starts the load in the form of transactions {{ ydb-short-name }} involving topics and tables simultaneously. The data is read from the topic and written to the table. To simulate a real load, you can set various input parameters: the number of messages, the size of messages, the target write speed, the number of consumers and producers, the number of partitions. During operation, the console displays the results: the number of written messages, the speed of writing messages, etc.
+
+{% include [ydb-cli-profile.md](../../_includes/ydb-cli-profile.md) %}
+
+## Initializing the test environment {#init}
+
+Before starting the load, it is necessary to initialize the test environment. You can use the command `{{ ydb-cli }} workload transfer topic-to-table init` to do this. It will create a topic and a table with the necessary parameters.
+
+Command syntax:
+```bash
+{{ ydb-cli }} [global options...] workload transfer topic-to-table init [options...]
+```
+* `global options` — [global parameters](commands/global-options.md).
+* `options` - parameters of the subcommand.
+
+View the command description:
+
+```bash
+{{ ydb-cli }} workload transfer topic-to-table init --help
+```
+
+Parameters of the subcommand:
+Parameter name | Parameter description | Default value
+---|---|---
+`--topic` | Topic name | `transfer-topic`
+`--consumer-prefix` | Prefix of the consumers name | `workload-consumer`
+`--table` | Table name | `transfer-table`
+`--consumers` | Number of topic consumers | `1`
+`--topic-partitions` | Number of topic partitions | `128`
+`--table-partitions` | Number of table partitions | `128`
+
+After executing the `init` subcommand, a table, topic and consumers will be created. Reader names are created by the rule `${CONSUMER_PREFIX}-${INDEX}`. The value of `${INDEX}` is an integer from 0 to the value of the parameter `--consumers` minus 1.
+
+For example, the command `{{ ydb-cli }} --profile quickstart workload transfer topic-to-table init --consumers 2 --topic-partitions 143 --table-partitions 237` will create a topic `transfer-topic` with 2 consumers, 143 partitions, and a table `transfer-table` with 237 partitions. The consumer names are `workload-consumer-0` and `workload-consumer-1`.
+
+## Running a load test {#run}
+
+The test simulates the load from an application that receives messages from a topic, processes them and writes the processing results to a database table.
+
+During the operation of the program, two types of work streams are simulated:
+* Input stream: messages are written to the topic in the non-transaction mode. The user can control the writing speed, the message size, the number of producers.
+* Processing flow: messages are read from the topic and written to the table using the {{ ydb-short-name }} transaction.
+
+The following actions are performed in the processing flow within a single transaction: 
+* messages from the topic are being read until the `--commit-period` period has expired;
+* one `UPSERT` command and a `COMMIT` command are executed on the table to commit the transaction after the period expires.
+
+Command syntax:
+```bash
+{{ ydb-cli }} [global options...] workload transfer topic-to-table run [options...]
+```
+* `global options` — [global parameters](commands/global-options.md).
+* `options` - parameters of the subcommand.
+
+View the command description:
+
+```bash
+{{ ydb-cli }} workload transfer topic-to-table run --help
+```
+
+Parameters of the subcommand:
+Parameter name | Parameter Description | Default value
+---|---|---
+`--seconds`, `-s` | Duration of the test in seconds | `60`
+`--window`, `-w` | Duration of the statistics collection window in seconds | `1`
+`--quiet`, `-q` | Output only the final test result | `0`
+`--print-timestamp` | Print the time together with the statistics of each time window | `0`
+`--percentile` | Percentile in statistics output | `50`
+`--warmup` | The warm-up time of the test in seconds. No statistics are calculated during this time | `5`
+`--topic` | Topic name | `transfer-topic`
+`--consumer-prefix` | Prefix of the consumers name | `workload-consumer`
+`--table` | Table name | `transfer-table`
+`--producer-threads`, `-p` | Number of producer threads | `1`
+`--consumer-threads`, `-t` | Number of consumer threads | `1`
+`--consumers`, `-c` | Number of consumers | `1`
+`--message-size`, `-m` | Message size in bytes. It is possible to specify in KB, MB, GB by adding suffixes `K`, `M`, `G` respectively | `10240`
+`--message-rate` | Target total write speed. In messages per second. Excludes the use of the `--byte-rate` | `0` parameter
+`--byte-rate` | Target total write speed. In bytes per second. Excludes the use of the --message-rate parameter. It is possible to specify in KB/s, MB/s, GB/s by adding suffixes `K`, `M`, `G` respectively | `0`
+`--commit-period` | The period between `COMMIT` calls. In seconds | `10`
+
+For example, the command `{{ ydb-cli }} --profile quickstart workload transfer topic-to-table run` will run a test lasting 60 seconds. The data for the first 5 seconds will not be taken into account in the work statistics. Example of console output:
+
+```text
+Window  Write speed     Write time      Inflight        Lag             Lag time        Read speed      Full time
+#       msg/s   MB/s    percentile,ms   percentile,msg  percentile,msg  percentile,ms   msg/s   MB/s    percentile,ms
+1       0       0       0               0               0               0               0       0       0
+2       0       0       0               0               0               0               0       0       0
+3       0       0       0               0               0               0               0       0       0
+4       0       0       0               0               0               0               0       0       0
+5       0       0       0               0               0               0               0       0       0
+6       103     1       911             78              0               0               103     1       914
+7       103     1       983             78              0               0               103     1       984
+8       103     1       1103            80              0               0               103     1       1106
+9       103     1       1003            85              0               0               103     1       1004
+10      103     1       1003            86              0               0               103     1       1006
+11      103     1       1015            85              0               0               103     1       1019
+12      103     1       1047            91              0               0               103     1       1043
+13      103     1       999             88              0               0               103     1       1003
+14      103     1       1063            85              0               0               103     1       1064
+15      103     1       1003            89              0               0               103     1       1008
+16      103     1       999             88              0               0               103     1       1003
+17      103     1       1071            78              0               0               103     1       1077
+...
+```
+
+* `Window` — the serial number of the time window for collecting statistics.
+* `Write speed` — the speed of writing messages by producers. In messages per second and in megabytes per second.
+* `Write time` — the specified percentile of the message writing time in ms.
+* `Inflight` — the maximum number of messages waiting for confirmation for all batches.
+* `Lag` — the specified percentile of maximum number of messages waiting to be read in the statistics collection window. Messages for all batches are taken into account.
+* `Lag time` — the specified percentile of message delay time in ms.
+* `Read speed` — the speed of reading messages by consumers. In messages per second and in megabytes per second.
+* `Full time` — the specified percentile of the time of complete processing of the message, from writing by the producer to reading by the consumer in ms.
+
+## Removing the test environment {#clean}
+
+After the test is completed, you can delete the test environment.
+
+Command syntax:
+```bash
+{{ ydb-cli }} [global options...] workload transfer topic-to-table clean [options...]
+```
+* `global options` — [global parameters](commands/global-options.md).
+* `options` - parameters of the subcommand.
+
+View the command description:
+
+```bash
+{{ ydb-cli }} workload transfer topic-to-table clean --help
+```
+
+Parameters of the subcommand:
+Parameter Name | Parameter Description | Default value
+---|---|---
+`--topic` | Topic name | `transfer-topic`
+`--table` | Table name | `transfer-table`
+
+For example, the command `{{ ydb-cli }} --profile quickstart workload transfer topic-to-table clean` will delete the topic `transfer-topic`, its consumers and the table `transfer-table`.
diff --git a/ydb/docs/ru/core/reference/ydb-cli/workload-transfer.md b/ydb/docs/ru/core/reference/ydb-cli/workload-transfer.md
index 9154d38f9f..96ef4210f0 100644
--- a/ydb/docs/ru/core/reference/ydb-cli/workload-transfer.md
+++ b/ydb/docs/ru/core/reference/ydb-cli/workload-transfer.md
@@ -133,6 +133,8 @@ Window  Write speed     Write time      Inflight        Lag             Lag time
 * `global options` — [глобальные параметры](commands/global-options.md).
 * `options` - параметры подкоманды.
 
+Посмотреть описание команды:
+
 ```bash
 {{ ydb-cli }} workload transfer topic-to-table clean --help
 ```
@@ -147,4 +149,4 @@ Window  Write speed     Write time      Inflight        Lag             Lag time
 ```bash
 {{ ydb-cli }} --profile quickstart workload transfer topic-to-table clean
 ```
-удалит топик `transfer-topic`, его читателей и таблицу `transfer-table`
+удалит топик `transfer-topic`, его читателей и таблицу `transfer-table`.