aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authoryuryalekseev <yuryalekseev@yandex-team.com>2022-12-31 14:41:55 +0300
committeryuryalekseev <yuryalekseev@yandex-team.com>2022-12-31 14:41:55 +0300
commitebf3c8be8039a35cd2b98541e22279af290e3f55 (patch)
tree07dc02314bc5d1413f28d4e6017c8ddb2984ff66
parentc8099767cc762ca726d0c98f6858a51e477acd19 (diff)
downloadydb-ebf3c8be8039a35cd2b98541e22279af290e3f55.tar.gz
Modify executable name in README.md.
-rw-r--r--contrib/restricted/boost/locale/include/boost/locale/util/string.hpp23
-rw-r--r--ydb/apps/dstool/README.md113
2 files changed, 80 insertions, 56 deletions
diff --git a/contrib/restricted/boost/locale/include/boost/locale/util/string.hpp b/contrib/restricted/boost/locale/include/boost/locale/util/string.hpp
new file mode 100644
index 0000000000..3dc465f157
--- /dev/null
+++ b/contrib/restricted/boost/locale/include/boost/locale/util/string.hpp
@@ -0,0 +1,23 @@
+//
+// Copyright (c) 2022 Alexander Grund
+//
+// Distributed under the Boost Software License, Version 1.0.
+// https://www.boost.org/LICENSE_1_0.txt
+
+#ifndef BOOST_LOCALE_UTIL_STRING_HPP
+#define BOOST_LOCALE_UTIL_STRING_HPP
+
+#include <boost/locale/config.hpp>
+
+namespace boost { namespace locale { namespace util {
+ /// Return the end of a C-string, i.e. the pointer to the trailing NULL byte
+ template<typename Char>
+ Char* str_end(Char* str)
+ {
+ while(*str)
+ ++str;
+ return str;
+ }
+}}} // namespace boost::locale::util
+
+#endif \ No newline at end of file
diff --git a/ydb/apps/dstool/README.md b/ydb/apps/dstool/README.md
index e596a4732f..ca555175e8 100644
--- a/ydb/apps/dstool/README.md
+++ b/ydb/apps/dstool/README.md
@@ -25,9 +25,10 @@ user@host:~/github/ydb$ ./ydb/apps/dstool/compile_protos.py --ydb-root ${ydb_roo
```bash
user@host:~$ cd ~/github/ydb
+user@host:~/github/ydb$ export PATH=$PATH:${ydb_root}/ydb/apps/dstool
user@host:~/github/ydb$ export PYTHONPATH=$PYTHONPATH:${ydb_root}
user@host:~/github/ydb$ chmod +x ./ydb/apps/dstool/ydb-dstool.py
-user@host:~/github/ydb$ ./ydb/apps/dstool/ydb-dstool.py -e ydb.endpoint cluster list
+user@host:~/github/ydb$ ydb-dstool.py -e ydb.endpoint cluster list
```
# How to do things with ydb-dstool
@@ -37,19 +38,19 @@ user@host:~/github/ydb$ ./ydb/apps/dstool/ydb-dstool.py -e ydb.endpoint cluster
In order to list all available commands along with their descriptions in a nicely printed tree run
```bash
-ydb-dstool --help
+user@host:~$ ydb-dstool.py --help
```
### Get help for a particular subset of commands or a command
```bash
-ydb-dstool pdisk --help
+user@host:~$ ydb-dstool.py pdisk --help
```
The above command prints help for the ```pdisk``` commands.
```bash
-ydb-dstool pdisk list --help
+user@host:~$ ydb-dstool.py pdisk list --help
```
The above command prints help for the ```pdisk list``` command.
@@ -59,7 +60,7 @@ The above command prints help for the ```pdisk list``` command.
To make operation of a command verbose add ```--verbose``` to global options:
```bash
-ydb-dstool --verbose -e ydbd.endpoint vdisk evict --vdisk-ids ${vdisk_id}
+user@host:~$ ydb-dstool.py --verbose -e ydbd.endpoint vdisk evict --vdisk-ids ${vdisk_id}
```
### Don't show non-vital messages
@@ -67,7 +68,7 @@ ydb-dstool --verbose -e ydbd.endpoint vdisk evict --vdisk-ids ${vdisk_id}
To dismiss non-vital messages of a command add ```--quiet``` to global options:
```bash
-ydb-dstool --quiet -e ydbd.endpoint pool balance
+user@host:~$ ydb-dstool.py --quiet -e ydbd.endpoint pool balance
```
### Run command without side effects
@@ -75,23 +76,23 @@ ydb-dstool --quiet -e ydbd.endpoint pool balance
To run command without side effect add ```--dry-run``` to global options:
```bash
-ydb-dstool --dry-run -e ydbd.endpoint vdisk evict --vdisk-ids ${vdisk_id}
+user@host:~$ ydb-dstool.py --dry-run -e ydbd.endpoint vdisk evict --vdisk-ids ${vdisk_id}
```
### Handle errors
-By convention ```ydb-dstool``` returns 0 on success, and non-zero on failure. You can check exit status
+By convention ```user@host:~$ ydb-dstool.py``` returns 0 on success, and non-zero on failure. You can check exit status
as follows:
```bash
-~$ ydb-dstool -e ydbd.endpoint vdisk evict --vdisk-ids ${vdisk_id}
+~$ user@host:~$ ydb-dstool.py -e ydbd.endpoint vdisk evict --vdisk-ids ${vdisk_id}
~$ if [ $? -eq 0 ]; then echo "success"; else echo "failure"; fi
```
-```ydb-dstool``` outputs errors to ```stderr``` so to redirect errors to ```errors.txt``` one could run:
+```user@host:~$ ydb-dstool.py``` outputs errors to ```stderr``` so to redirect errors to ```errors.txt``` one could run:
```bash
-~$ ydb-dstool -e ydbd.endpoint vdisk evict --vdisk-ids ${vdisk_id} 2> ~/errors.txt
+~$ user@host:~$ ydb-dstool.py -e ydbd.endpoint vdisk evict --vdisk-ids ${vdisk_id} 2> ~/errors.txt
```
### Set endpoint
@@ -100,7 +101,7 @@ as follows:
```[PROTOCOL://]HOST[:PORT]```. To set endpoint use ```--endpoint``` global option:
```bash
-ydb-dstool --endpoint https://ydbd.endpoint:8765 pdisk list
+user@host:~$ ydb-dstool.py --endpoint https://ydbd.endpoint:8765 pdisk list
```
The endpoint's protocol from the above command is ```https```, host is ```ydbd.endpoint```, port is ```8765```.
@@ -112,7 +113,7 @@ There is support for authentication with access token. When authentication is re
token with ```--token-file``` global option:
```bash
-ydb-dstool -e ydbd.endpoint --token-file ~/access_token
+user@host:~$ ydb-dstool.py -e ydbd.endpoint --token-file ~/access_token
```
The above command reads ```~/access_token``` and uses it's contents as an access token for authentication.
@@ -130,7 +131,7 @@ are available:
To set output format to ```tsv``` add ```--format tsv``` to command options:
```bash
-ydb-dstool -e ydbd.endpoint pdisk list --format tsv
+user@host:~$ ydb-dstool.py -e ydbd.endpoint pdisk list --format tsv
```
### Exclude header from the output
@@ -138,7 +139,7 @@ ydb-dstool -e ydbd.endpoint pdisk list --format tsv
To exclude header with the column names from the output add ```--no-header``` to command options:
```bash
-ydb-dstool -e ydbd.endpoint pdisk list --format tsv --no-header
+user@host:~$ ydb-dstool.py -e ydbd.endpoint pdisk list --format tsv --no-header
```
### Output all available columns
@@ -147,7 +148,7 @@ By default a listing like command outputs only certain columns. The default colu
To output all available columns add ```--all-columns``` to command options:
```bash
-ydb-dstool -e ydbd.endpoint pdisk list --all-columns
+user@host:~$ ydb-dstool.py -e ydbd.endpoint pdisk list --all-columns
```
### Output only certain columns
@@ -156,7 +157,7 @@ To output only certain columns add ```--columns``` along with a space separated
options:
```bash
-ydb-dstool -e ydbd.endpoint pdisk list --columns NodeId:PDiskId Path
+user@host:~$ ydb-dstool.py -e ydbd.endpoint pdisk list --columns NodeId:PDiskId Path
```
The above command lists only the ```NodeId:PDiskId```, ```Path``` columns while listing pdisks.
@@ -167,7 +168,7 @@ To sort output by certain columns add ```--sort-by``` along with a space separat
options:
```bash
-ydb-dstool -e ydbd.endpoint pdisk list --sort-by FQDN
+user@host:~$ ydb-dstool.py -e ydbd.endpoint pdisk list --sort-by FQDN
```
The above command lists pdisks sorted by the ```FQDN``` column.
@@ -178,7 +179,7 @@ To output sizes in terms of kilobytes, megabytes, etc. and fractions in terms of
or ```-H``` to command options:
```bash
-ydb-dstool -e ydbd.endpoint pdisk list --show-pdisk-usage -H
+user@host:~$ ydb-dstool.py -e ydbd.endpoint pdisk list --show-pdisk-usage -H
```
## Do things with pdisks
@@ -186,7 +187,7 @@ ydb-dstool -e ydbd.endpoint pdisk list --show-pdisk-usage -H
### List pdisks
```bash
-ydb-dstool -e ydbd.endpoint pdisk list
+user@host:~$ ydb-dstool.py -e ydbd.endpoint pdisk list
```
The above command lists all pdisks of a cluster along with their state.
@@ -194,7 +195,7 @@ The above command lists all pdisks of a cluster along with their state.
### Show space usage of every pdisk
```bash
-ydb-dstool -e ydbd.endpoint pdisk list --show-pdisk-usage --human-readable
+user@host:~$ ydb-dstool.py -e ydbd.endpoint pdisk list --show-pdisk-usage --human-readable
```
The above command lists usage of all pdisks of a cluster in a human-readable way.
@@ -202,7 +203,7 @@ The above command lists usage of all pdisks of a cluster in a human-readable way
### Prevent new groups from using certain pdisks
```bash
-ydb-dstool -e ydbd.endpoint pdisk set --decommit-status DECOMMIT_PENDING --pdisk-ids "[NODE_ID:PDISK_ID]"
+user@host:~$ ydb-dstool.py -e ydbd.endpoint pdisk set --decommit-status DECOMMIT_PENDING --pdisk-ids "[NODE_ID:PDISK_ID]"
```
The above command prevents new groups from using pdisk ```"[NODE_ID:PDISK_ID]"```.
@@ -210,7 +211,7 @@ The above command prevents new groups from using pdisk ```"[NODE_ID:PDISK_ID]"``
### Move data out from certain pdisks
```bash
-ydb-dstool -e ydbd.endpoint pdisk set --decommit-status DECOMMIT_IMMINENT --pdisk-ids "[NODE_ID:PDISK_ID]"
+user@host:~$ ydb-dstool.py -e ydbd.endpoint pdisk set --decommit-status DECOMMIT_IMMINENT --pdisk-ids "[NODE_ID:PDISK_ID]"
```
The above command initiates a background process that is going to move all of the data from
@@ -222,7 +223,7 @@ this step prior to unplugging either certain disks or complete host from a clust
### List vdisks
```bash
-ydb-dstool -e ydbd.endpoint vdisk list
+user@host:~$ ydb-dstool.py -e ydbd.endpoint vdisk list
```
The above command lists all vdisks of a cluster along with the corresponding pdisks.
@@ -230,7 +231,7 @@ The above command lists all vdisks of a cluster along with the corresponding pdi
### Show status of pdisks were vdisks reside
```bash
-ydb-dstool -e ydbd.endpoint vdisk list --show-pdisk-status
+user@host:~$ ydb-dstool.py -e ydbd.endpoint vdisk list --show-pdisk-status
```
The above command lists all vdisks of a cluster along with the corresponding pdisks. On top of that, for every
@@ -239,7 +240,7 @@ vdisk it lists the status of the corresponding pdisk where vdisk resides.
### Show space usage every vdisk
```bash
-ydb-dstool -e ydbd.endpoint vdisk list --show-vdisk-usage --human-readable
+user@host:~$ ydb-dstool.py -e ydbd.endpoint vdisk list --show-vdisk-usage --human-readable
```
The above command lists usage of all vdisks of a cluster in a human-readable way.
@@ -247,7 +248,7 @@ The above command lists usage of all vdisks of a cluster in a human-readable way
### Unload certain pdisks by moving some vdisks from them
```bash
-ydb-dstool -e ydbd.endpoint vdisk evict --vdisk-ids "[8200001b:3:0:7:0] [8200001c:1:0:1:0]"
+user@host:~$ ydb-dstool.py -e ydbd.endpoint vdisk evict --vdisk-ids "[8200001b:3:0:7:0] [8200001c:1:0:1:0]"
```
The above command evicts vdisks ```[8200001b:3:0:7:0]```, ```[8200001c:1:0:1:0]``` from their current pdisks to
@@ -257,7 +258,7 @@ are running out of space. This might happen because of usage sckew of certain gr
### Wipe certain vdisks
```bash
-ydb-dstool -e ydbd.endpoint vdisk wipe --vdisk-ids "[8200001b:3:0:7:0] [8200001c:1:0:1:0]" --run
+user@host:~$ ydb-dstool.py -e ydbd.endpoint vdisk wipe --vdisk-ids "[8200001b:3:0:7:0] [8200001c:1:0:1:0]" --run
```
The above command wipes out vdisks ```[8200001b:3:0:7:0]```, ```[8200001c:1:0:1:0]```. This command is useful when
@@ -266,7 +267,7 @@ vdisk becomes unhealable.
### Remove no longer needed donor vdisks
```bash
-ydb-dstool -e ydbd.endpoint vdisk remove-donor --vdisk-ids "[8200001b:3:0:7:0] [8200001c:1:0:1:0]"
+user@host:~$ ydb-dstool.py -e ydbd.endpoint vdisk remove-donor --vdisk-ids "[8200001b:3:0:7:0] [8200001c:1:0:1:0]"
```
The above command removes donor vdisks ```[8200001b:3:0:7:0]```, ```[8200001c:1:0:1:0]```. The provided vdisks
@@ -288,7 +289,7 @@ Group can be thought of as a RAID of vdisks. Any vdisk belongs to a single group
### List groups
```bash
-ydb-dstool -e ydbd.endpoint group list
+user@host:~$ ydb-dstool.py -e ydbd.endpoint group list
```
The above command lists all groups of a cluster.
@@ -299,7 +300,7 @@ To show aggregated statuses of vdisks within a group (i.e. how many vdisks withi
add ```--show-vdisk-status``` to command options:
```bash
-ydb-dstool -e ydbd.endpoint group list --show-vdisk-status
+user@host:~$ ydb-dstool.py -e ydbd.endpoint group list --show-vdisk-status
```
### Show space usage of every group
@@ -307,7 +308,7 @@ ydb-dstool -e ydbd.endpoint group list --show-vdisk-status
To show space usage of groups, add ```--show-vdisk-usage``` to command options:
```bash
-ydb-dstool -e ydbd.endpoint group list --show-vdisk-usage -H
+user@host:~$ ydb-dstool.py -e ydbd.endpoint group list --show-vdisk-usage -H
```
The above command lists all groups of a cluster along with their space usage in a human-readable way.
@@ -315,7 +316,7 @@ The above command lists all groups of a cluster along with their space usage in
### Check certain groups for compliance with failure model
```bash
-ydb-dstool -e ydbd.endpoint group check --group-ids 2181038097 2181038105 --failure-model
+user@host:~$ ydb-dstool.py -e ydbd.endpoint group check --group-ids 2181038097 2181038105 --failure-model
```
The above command checks groups ```2181038097```, ```2181038105``` for compliance with their failure model.
@@ -323,7 +324,7 @@ The above command checks groups ```2181038097```, ```2181038105``` for complianc
### Show space usage of groups by tablets
```bash
-ydb-dstool -e ydbd.endpoint group show usage-by-tablets
+user@host:~$ ydb-dstool.py -e ydbd.endpoint group show usage-by-tablets
```
The above command shows which tablets are using which groups and what the space usage is.
@@ -331,7 +332,7 @@ The above command shows which tablets are using which groups and what the space
### Show info about certain blob from a certain group
```bash
-ydb-dstool -e ydbd.endpoint group show blob-info --group-id 2181038081 --blob-id "[72075186224037892:1:2:1:8192:410:0]"
+user@host:~$ ydb-dstool.py -e ydbd.endpoint group show blob-info --group-id 2181038081 --blob-id "[72075186224037892:1:2:1:8192:410:0]"
```
The above command shows information about blob ```[72075186224037892:1:2:1:8192:410:0]``` that is stored in
@@ -340,7 +341,7 @@ group ```2181038081```. This command might be useful in certain debug scenarios.
### Add new groups to certain pool
```bash
-ydb-dstool -e ydbd.endpoint group add --pool-name /Root:nvme --groups 1
+user@host:~$ ydb-dstool.py -e ydbd.endpoint group add --pool-name /Root:nvme --groups 1
```
The above command adds one group to the pool ```/Root:nvme```
@@ -348,7 +349,7 @@ The above command adds one group to the pool ```/Root:nvme```
### Figure out whether certain number of groups can be added
```bash
-ydb-dstool --dry-run -e ydbd.endpoint group add --pool-name /Root:nvme --groups 10
+user@host:~$ ydb-dstool.py --dry-run -e ydbd.endpoint group add --pool-name /Root:nvme --groups 10
```
The above command adds ten groups to the pool ```/Root:nvme``` without actually adding them. It might be useful
@@ -361,7 +362,7 @@ Pool is a collection of groups.
### List pools
```bash
-ydb-dstool -e ydbd.endpoint pool list
+user@host:~$ ydb-dstool.py -e ydbd.endpoint pool list
```
The above command lists all pools of a cluster.
@@ -372,7 +373,7 @@ To show aggregated statuses of groups within a pool (i.e. how many groups within
add ```--show-group-status``` to command options:
```bash
-ydb-dstool -e ydbd.endpoint pool list --show-group-status
+user@host:~$ ydb-dstool.py -e ydbd.endpoint pool list --show-group-status
```
### Show aggregated statuses of vdisks within pool
@@ -381,7 +382,7 @@ To show aggregated statuses of vdisks within a pool (i.e. how many vdisks within
add ```--show-vdisk-status``` to command options:
```bash
-ydb-dstool -e ydbd.endpoint pool list --show-vdisk-status
+user@host:~$ ydb-dstool.py -e ydbd.endpoint pool list --show-vdisk-status
```
### Show space usage of pools
@@ -389,7 +390,7 @@ ydb-dstool -e ydbd.endpoint pool list --show-vdisk-status
To show space usage of pools, add ```--show-vdisk-usage``` to command options:
```bash
-ydb-dstool -e ydbd.endpoint pool list --show-vdisk-usage -H
+user@host:~$ ydb-dstool.py -e ydbd.endpoint pool list --show-vdisk-usage -H
```
The above command lists all pools of a cluster along with their space usage in a human-readable way.
@@ -397,7 +398,7 @@ The above command lists all pools of a cluster along with their space usage in a
### Show estimated space usage of pools
```bash
-ydb-dstool -e ydbd.endpoint pool list --show-vdisk-estimated-usage
+user@host:~$ ydb-dstool.py -e ydbd.endpoint pool list --show-vdisk-estimated-usage
```
The above command shows:
@@ -412,7 +413,7 @@ Box is a collection of pdisks.
### List boxes
```bash
-ydb-dstool -e ydbd.endpoint box list
+user@host:~$ ydb-dstool.py -e ydbd.endpoint box list
```
The above command lists all boxes of a cluster.
@@ -423,7 +424,7 @@ Tow show aggregated statuses of pdisks within a box (i.e. how many pdisks within
add ```--show-pdisk-status``` to command options:
```bash
-ydb-dstool -e ydbd.endpoint box list --show-pdisk-status
+user@host:~$ ydb-dstool.py -e ydbd.endpoint box list --show-pdisk-status
```
### Show space usage of boxes
@@ -431,7 +432,7 @@ ydb-dstool -e ydbd.endpoint box list --show-pdisk-status
To show space usage of boxes, add ```--show-pdisk-usage``` to command options:
```bash
-ydb-dstool -e ydbd.endpoint box list --show-pdisk-usage -H
+user@host:~$ ydb-dstool.py -e ydbd.endpoint box list --show-pdisk-usage -H
```
The above command lists all boxes of a cluster along with their space usage in a human-readable way.
@@ -444,7 +445,7 @@ In terms of implementation, a node is a a YDB process running on one of cluster'
### List nodes
```bash
-ydb-dstool -e ydbd.endpoint node list
+user@host:~$ ydb-dstool.py -e ydbd.endpoint node list
```
The above command lists all nodes of a cluster.
@@ -454,7 +455,7 @@ The above command lists all nodes of a cluster.
### Show how many cluster entities there are
```bash
-ydb-dstool -e ydbd.endpoint cluster list
+user@host:~$ ydb-dstool.py -e ydbd.endpoint cluster list
```
The above command shows how many
@@ -475,7 +476,7 @@ In rare cases some pdisks can become overpopulated (i.e. they host too many vdis
balancing of vdisks over pdisks. To accomplish this, run the following command:
```bash
-ydb-dstool -e ydbd.endpoint cluster balance
+user@host:~$ ydb-dstool.py -e ydbd.endpoint cluster balance
```
The above command moves out vdisks from overpopulated pdisks. A single vdisk is moved at a time so that the failure model of
@@ -496,13 +497,13 @@ failed vdisk within a group.
To enable self-healing on a cluster, run the following command:
```bash
-ydb-dstool -e ydbd.endpoint cluster set --enable-self-heal
+user@host:~$ ydb-dstool.py -e ydbd.endpoint cluster set --enable-self-heal
```
To disable self-healing on a cluster, run the following command:
```bash
-ydb-dstool -e ydbd.endpoint cluster set --disable-self-heal
+user@host:~$ ydb-dstool.py -e ydbd.endpoint cluster set --disable-self-heal
```
### Enable/Disable donors
@@ -514,13 +515,13 @@ old vdisk could be used as a donor, so that the new vdisk would copy all of the
To enable support for donor vdisk mode on a cluster, run the following command:
```bash
-ydb-dstool -e ydbd.endpoint cluster set --enable-donor-mode
+user@host:~$ ydb-dstool.py -e ydbd.endpoint cluster set --enable-donor-mode
```
To disable support for donor vdisk mode on a cluster, run the following command:
```bash
-ydb-dstool -e ydbd.endpoint cluster set --disable-donor-mode
+user@host:~$ ydb-dstool.py -e ydbd.endpoint cluster set --disable-donor-mode
```
### Adjust scrubbing intervals
@@ -529,19 +530,19 @@ Scrubbing is a background process that checks data integrity and performs data r
a cluster enter the following command:
```bash
-ydb-dstool -e ydbd.endpoint cluster set --scrub-periodicity disable
+user@host:~$ ydb-dstool.py -e ydbd.endpoint cluster set --scrub-periodicity disable
```
To set scrubbing interval to two days run the following command:
```bash
-ydb-dstool -e ydbd.endpoint cluster set --scrub-periodicity 2d
+user@host:~$ ydb-dstool.py -e ydbd.endpoint cluster set --scrub-periodicity 2d
```
### Set maximum number of simultaneously scrubbed pdisks
```bash
-ydb-dstool -e ydbd.endpoint cluster set --max-scrubbed-disks-at-once 2
+user@host:~$ ydb-dstool.py -e ydbd.endpoint cluster set --max-scrubbed-disks-at-once 2
```
The above command sets maximum number of simultaneously scrubbed pdisk to two.
@@ -551,7 +552,7 @@ The above command sets maximum number of simultaneously scrubbed pdisk to two.
To run workload that allows to stress test failure model of groups, run the following command:
```bash
-ydb-dstool -e ydbd.endpoint cluster workload run
+user@host:~$ ydb-dstool.py -e ydbd.endpoint cluster workload run
```
The above command performs various