ydb/build/plugins/ytest.py, branch CLI_2.21.0Mirror of YDB github reposhttps://code.mastervirt.ru/ydb/atom?h=CLI_2.21.02025-05-16T06:46:52Zget rid of six in plugins2025-05-16T06:46:52Zsnermolaevsnermolaev@yandex-team.com2025-05-16T06:31:22Zurn:sha1:bd2e3ff66d6b4398194fe32537a0156cfd95310a
commit_hash:d85f21d6b4179697b8d5f8ab39c7d5216d35d1a8
fix junit5-runner ForkSubtests: should distribute evenly tests in chunks2025-04-21T17:56:34Zalmishevalmishev@yandex-team.com2025-04-21T17:00:50Zurn:sha1:6b17ad8c52e16d0088291781e1b5a9e28d028b9b
commit_hash:478736ce9fbf76b0b269d4ac39055a3989b92e7f
Drop custom config support in STYLE_PYTHON2025-04-02T06:47:24Zalevitskiialevitskii@yandex-team.com2025-04-02T06:30:41Zurn:sha1:5cbf4c4aeaeccd21e788ccbe02494639c98717cb
Drop custom config support in STYLE_PYTHON
commit_hash:47f5ddb6a0d62c66074a89276b77c6b34db33a6c
Drop ruff_config_paths.json support in linter tests2025-04-02T04:43:06Zalevitskiialevitskii@yandex-team.com2025-04-02T04:26:41Zurn:sha1:dd91da01678cb17b1d61d7dcea7c037ce47a8207
Drop ruff_config_paths.json support in linter tests
commit_hash:45e008aa1b336014dc657f460ffca2abfd0649df
Delete STYLE_CPP_ARC and _STYLE_CPP2025-03-25T06:21:55Zalevitskiialevitskii@yandex-team.com2025-03-25T06:07:10Zurn:sha1:4fe6ab0fdb1d112409176be736ba6e586b16eb7f
Delete STYLE_CPP_ARC and _STYLE_CPP
commit_hash:cf9019ef25cbc6f6ac7571926096006f99af4b13
Send porto_layers requirements to SB task2025-03-03T18:37:52Zalevitskiialevitskii@yandex-team.com2025-03-03T17:16:31Zurn:sha1:44d24c35433de6b811c408108e71dbd4fd74663d
Send porto_layers requirements to SB task
commit_hash:13f2b498c203c43770c7c4dcfb8cbea59f12569b
change build policy for mobile/geo/maps ya.make build2025-02-28T17:37:35Zr3zchr3zch@yandex-team.com2025-02-28T17:00:51Zurn:sha1:a00c5f24963aa5b3e9fc08a197dce8399b705c89
commit_hash:4875b5ef44f2f87a84bf374989d1c70b1fd4afff
[devtools/ya] Added LANG_AGNOSTIC type to lang statistics2025-01-17T19:03:58Zprettyboyprettyboy@yandex-team.com2025-01-17T17:01:01Zurn:sha1:c89ef5ac36522b186f4372a6c3ec8b4e96a08666
commit_hash:8dadcd633f444b7d9b0783f1ccc5395baa2a2e38
Prototype - external python files mode for tests2025-01-16T19:26:51Zprettyboyprettyboy@yandex-team.com2025-01-16T18:01:45Zurn:sha1:2f3fd95aac24e27a3b0aa2badda49db82bf36cc5
Режим позволяет перезапускать python тесты или интеграционные тесты на питон программы при внесении изменений в python файлы без перекомпиляции.
Сценарий:
- прогнали тесты
- внесли изменение в `library/python/func/__init__.py`
- перезапустили, смотрим на времена
На примере интеграционного devtools/ya/handlers/analyze_make/tests в codenv (base конфигурация).
#|
||
ya m -rA
|
ya m -rA --ext-py
||
||
real 10m25,928s
user 0m53,465s
sys 1m14,791s
```
Total time by type:
[935171 ms] [LD] [count: 3, ave time 311723.67 msec]
[ 13657 ms] [prepare:AC] [count: 4, ave time 3414.25 msec]
[ 10436 ms] [TM] [count: 1, ave time 10436.00 msec]
[ 10387 ms] [prepare:get from local cache] [count: 1521, ave time 6.83 msec]
[ 6480 ms] [TS] [count: 1, ave time 6480.00 msec]
[ 3823 ms] [prepare:put into local cache, clean build dir] [count: 7, ave time 546.14 msec]
[ 2279 ms] [CC] [count: 1, ave time 2279.00 msec]
[ 1251 ms] [PY] [count: 1, ave time 1251.00 msec]
[ 372 ms] [prepare:yt-store] [count: 2, ave time 186.00 msec]
[ 154 ms] [AR] [count: 1, ave time 154.00 msec]
...
```
|
real 0m22,094s
user 0m26,372s
sys 0m3,363s
```
Total time by type:
[13140 ms] [TM] [count: 1, ave time 13140.00 msec]
[ 351 ms] [prepare:yt-store] [count: 1, ave time 351.00 msec]
[ 81 ms] [prepare:tools] [count: 8, ave time 10.12 msec]
[ 32 ms] [prepare:AC] [count: 2, ave time 16.00 msec]
[ 31 ms] [prepare:get from local cache] [count: 7, ave time 4.43 msec]
[ 16 ms] [prepare:clean] [count: 3, ave time 5.33 msec]
```
||
|#
В codenv в base конфигуарции (4cpu) невероятно долго идёт линковка. Тот же самый сценарий, но на mous (54cpu)
#|
||
ya m -rA
|
ya m -rA --ext-py
||
||
real 0m25,435s
user 0m40,242s
sys 0m22,434s
```
Total time by type:
[30958 ms] [prepare:get from local cache] [count: 1521, ave time 20.35 msec]
[16293 ms] [prepare:AC] [count: 4, ave time 4073.25 msec]
[11161 ms] [LD] [count: 3, ave time 3720.33 msec]
[ 4256 ms] [TM] [count: 1, ave time 4256.00 msec]
[ 3529 ms] [TS] [count: 1, ave time 3529.00 msec]
[ 1542 ms] [CC] [count: 1, ave time 1542.00 msec]
[ 635 ms] [PY] [count: 1, ave time 635.00 msec]
...
```
|
real 0m18,321s
user 0m21,673s
sys 0m3,204s
```
Total time by type:
[10786 ms] [TM] [count: 1, ave time 10786.00 msec]
[ 1568 ms] [prepare:yt-store] [count: 1, ave time 1568.00 msec]
[ 151 ms] [prepare:tools] [count: 8, ave time 18.88 msec]
[ 61 ms] [prepare:get from local cache] [count: 7, ave time 8.71 msec]
[ 48 ms] [prepare:AC] [count: 2, ave time 24.00 msec]
[ 14 ms] [prepare:clean] [count: 3, ave time 4.67 msec]
```
||
|#
В новом режиме тест ожидаемо замедляется, так как все питон программы при каждом старте вынуждены читать файлы с фс (ещё через арк), а потом ещё и компилировать байткод, т.е. холодный запуск
commit_hash:24c5a46a8385d3c065abfb6fc5b40f7ad24bb1cb
Never compress python build resources to speedup python configuration and build2024-12-13T06:07:25Zspreisspreis@yandex-team.com2024-12-13T05:25:28Zurn:sha1:e90b83df32a421a93c021eaa91710c1193585f9a
There is no runtime perf impact, but
- The change reduces build graph by removing 2 nodes out of 5 for python which accelerates both configuration and build
- surprisingly in all experiments there were positive impact on binary size about 2MiB
commit_hash:909fdeeb91b7f20c749177d638e97d5a469e422d