aboutsummaryrefslogtreecommitdiffstats
path: root/library/python
Commit message (Collapse)AuthorAgeFilesLines
* [library/python/runtime_py3] Made Y_PYTHON_SOURCE_ROOT bytes againprettyboy2025-01-231-1/+1
| | | | commit_hash:3433c11c4d386c785422b1e149e9a3a8de7f4f0f
* [devtools/ya] Dont' write __pycache__ to the arcadia in external-py-files modeprettyboy2025-01-231-1/+29
| | | | | | | | | В репозитории больше не будет `__pycache__` при запусках через ya make / test На примере devtools/ya/handlers/analyze_make/tests Время конфигурации в горячем сценарии в режиме --ext-py при изменении одного питонфайлика: `Configure time - 6.1 s` Время конфигурации в горячем сценарии в режиме --ext-py при изменении одного питонфайлика с текущими правками: `Configure time - 4.6 s` commit_hash:adf5219cd4f347bb572393d4a6debd1436dc0815
* [library/python/runtime_py3] Added bytecode support for external-py-files modeprettyboy2025-01-182-9/+72
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Добавляется поддержка стандартной работы с байткодом для бинарей собранных с --ext-py (HIDDEN_URL На примере импорт теста для `devtools/ya/bin` `time Y_PYTHON_ENTRY_POINT=library.python.testing.import_test.import_test:main YA_TEST_CONTEXT_FILE=/tmp/test.context ./ya-bin >/dev/null` #| || Без патча ya m -r | real 0m2,934s user 0m2,772s sys 0m0,157s || || Без патча ya m -r --ext-py | real 0m10,916s user 0m9,927s sys 0m0,368s || || C патчем ya m -r | real 0m2,931s user 0m2,774s sys 0m0,156s || || C патчем ya m -r --ext-py первый запуск с записью байткода | real 0m7,482s user 0m5,174s sys 0m0,620s || || C патчем ya m -r --ext-py последующие запуски с использованием байткода с фс | real 0m3,550s user 0m2,801s sys 0m0,262s || |# commit_hash:84f7bb273b09d51b88eb5c5b6dfcd3bb7c108307
* Prototype - external python files mode for testsprettyboy2025-01-163-9/+51
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Режим позволяет перезапускать python тесты или интеграционные тесты на питон программы при внесении изменений в python файлы без перекомпиляции. Сценарий: - прогнали тесты - внесли изменение в `library/python/func/__init__.py` - перезапустили, смотрим на времена На примере интеграционного devtools/ya/handlers/analyze_make/tests в codenv (base конфигурация). #| || ya m -rA | ya m -rA --ext-py || || real 10m25,928s user 0m53,465s sys 1m14,791s ``` Total time by type: [935171 ms] [LD] [count: 3, ave time 311723.67 msec] [ 13657 ms] [prepare:AC] [count: 4, ave time 3414.25 msec] [ 10436 ms] [TM] [count: 1, ave time 10436.00 msec] [ 10387 ms] [prepare:get from local cache] [count: 1521, ave time 6.83 msec] [ 6480 ms] [TS] [count: 1, ave time 6480.00 msec] [ 3823 ms] [prepare:put into local cache, clean build dir] [count: 7, ave time 546.14 msec] [ 2279 ms] [CC] [count: 1, ave time 2279.00 msec] [ 1251 ms] [PY] [count: 1, ave time 1251.00 msec] [ 372 ms] [prepare:yt-store] [count: 2, ave time 186.00 msec] [ 154 ms] [AR] [count: 1, ave time 154.00 msec] ... ``` | real 0m22,094s user 0m26,372s sys 0m3,363s ``` Total time by type: [13140 ms] [TM] [count: 1, ave time 13140.00 msec] [ 351 ms] [prepare:yt-store] [count: 1, ave time 351.00 msec] [ 81 ms] [prepare:tools] [count: 8, ave time 10.12 msec] [ 32 ms] [prepare:AC] [count: 2, ave time 16.00 msec] [ 31 ms] [prepare:get from local cache] [count: 7, ave time 4.43 msec] [ 16 ms] [prepare:clean] [count: 3, ave time 5.33 msec] ``` || |# В codenv в base конфигуарции (4cpu) невероятно долго идёт линковка. Тот же самый сценарий, но на mous (54cpu) #| || ya m -rA | ya m -rA --ext-py || || real 0m25,435s user 0m40,242s sys 0m22,434s ``` Total time by type: [30958 ms] [prepare:get from local cache] [count: 1521, ave time 20.35 msec] [16293 ms] [prepare:AC] [count: 4, ave time 4073.25 msec] [11161 ms] [LD] [count: 3, ave time 3720.33 msec] [ 4256 ms] [TM] [count: 1, ave time 4256.00 msec] [ 3529 ms] [TS] [count: 1, ave time 3529.00 msec] [ 1542 ms] [CC] [count: 1, ave time 1542.00 msec] [ 635 ms] [PY] [count: 1, ave time 635.00 msec] ... ``` | real 0m18,321s user 0m21,673s sys 0m3,204s ``` Total time by type: [10786 ms] [TM] [count: 1, ave time 10786.00 msec] [ 1568 ms] [prepare:yt-store] [count: 1, ave time 1568.00 msec] [ 151 ms] [prepare:tools] [count: 8, ave time 18.88 msec] [ 61 ms] [prepare:get from local cache] [count: 7, ave time 8.71 msec] [ 48 ms] [prepare:AC] [count: 2, ave time 24.00 msec] [ 14 ms] [prepare:clean] [count: 3, ave time 4.67 msec] ``` || |# В новом режиме тест ожидаемо замедляется, так как все питон программы при каждом старте вынуждены читать файлы с фс (ещё через арк), а потом ещё и компилировать байткод, т.е. холодный запуск commit_hash:24c5a46a8385d3c065abfb6fc5b40f7ad24bb1cb
* Fix for Python 3.13shadchin2025-01-161-6/+0
| | | | commit_hash:2a9f4a61b44a7e34d430d8615b5c9d2d120fa7d9
* YDB Import 657robot-ydb-importer2024-12-263-24/+0
| | | | commit_hash:a80065301df8a3de8b4d2ad596dfd995d3988485
* Intermediate changesrobot-piglet2024-12-234-0/+186
| | | | commit_hash:e95937c5f2c37075ed358ebb01e274fc9ddfcbb6
* Intermediate changesrobot-piglet2024-12-131-0/+2
| | | | commit_hash:e4f538de906ec2dcfa4e6907edad0b20916b93ca
* Intermediate changesrobot-piglet2024-12-131-2/+0
| | | | commit_hash:27af5f8a0a7836d78ffdc205213f07f79dd9f7ad
* Never compress python build resources to speedup python configuration and buildspreis2024-12-131-0/+1
| | | | | | | There is no runtime perf impact, but - The change reduces build graph by removing 2 nodes out of 5 for python which accelerates both configuration and build - surprisingly in all experiments there were positive impact on binary size about 2MiB commit_hash:909fdeeb91b7f20c749177d638e97d5a469e422d
* Switch all tests to atd from snapshotthegeorg2024-12-121-3/+3
| | | | commit_hash:a9a7c2df09742a540851227c4fcb7d925edd9793
* Intermediate changesrobot-piglet2024-12-111-5/+17
| | | | commit_hash:a633306b66f89adbf188bf9c6d521bdf7b01f82f
* Check custom error in xfail tests. Remove canonization from dq/hybrid testsudovichenko-r2024-11-231-4/+0
| | | | | | | | | | | | | | | 1. xfail тесты теперь должны содержать спец комментарий `custom error` с ожидаемым текстом ошибки 2. Все suites для xfail размечены спец. коментариями, некоторые тесты разбиты на несколько 3. Включены тесты с Python и Javascript udf для yt провайдера и переканонизированы (были отключены в github) 4. Для тяжелых xfail тестов аналогично убрана канонизация и включена проверка по спец. коментарию 5. Из dq_file/hybrid тестов убрана канонизация. Осталась только кросс-проверка результатов с yqlrun и блочным режимом 6. В dqrun теперь используется exception_policy вместо fail_policy (для корректной выдачи ошибок в тестах) commit_hash:a92b3835f86b0c01225e81e3f28bb6d11d8d67a3
* YDB Import 625robot-ydb-importer2024-11-226-70/+163
| | | | commit_hash:bbf24592c367fa158fbc53e041fe367374d0750e
* Intermediate changesrobot-piglet2024-11-151-0/+2
| | | | commit_hash:2ad7a4bc9a9ae76fb2a70feb2681becd0de5c5c7
* Intermediate changesrobot-piglet2024-11-141-3/+4
| | | | commit_hash:bbe5982909469ab84284cb16547e69a096d992ea
* Add external autoincludes.jsonalevitskii2024-11-143-0/+132
| | | | | Add external autoincludes.json commit_hash:1b3de1732fee590582ae5620570facc2c0987390
* Intermediate changesrobot-piglet2024-11-121-24/+26
| | | | commit_hash:63f0ba341b35ad9ea632bfc2971f028cebbec800
* Move yql/udfs/common/ to /yql/essentials YQL-19206imunkin2024-11-081-0/+4
| | | | | | | | | | Except the following directories: * clickhouse/client * datetime * knn * roaring commit_hash:c7da95636144d28db109d6b17ddc762e9bacb59f
* Intermediate changesrobot-piglet2024-11-071-1/+1
| | | | commit_hash:8dfe46ba3b02673d0e6a35fdf824ab2c98255393
* Intermediate changesrobot-piglet2024-11-061-1/+1
| | | | commit_hash:d313ff8d16f964d796106f71b9e2d65b0ea39f82
* Intermediate changesrobot-piglet2024-11-014-25/+151
| | | | commit_hash:afe2bc6dcfc79e6d34dabab9a0c92d3fa18bc87d
* Intermediate changesrobot-piglet2024-10-231-1/+1
| | | | commit_hash:3e55f833d0efbd9feb7cb3f9ff320ce0faa9ec37
* Fix python-iptablesshadchin2024-10-061-0/+2
| | | | commit_hash:c1f1219d64d21ec34a6f2724029ef2252f1fdb7e
* Intermediate changesrobot-piglet2024-09-301-4/+2
| | | | commit_hash:99ac877e6e05737455ad173d5ce45946fde7d623
* Intermediate changesrobot-piglet2024-09-121-4/+1
| | | | commit_hash:ecafab280d4dfb52401ab81932e10a8d434b1999
* Intermediate changesrobot-piglet2024-09-041-0/+4
|
* Customize bt colorizationiaz16072024-08-191-7/+31
| | | | 221e27d03ecf65a066e77e452326e20c49e89df5
* Intermediate changesrobot-piglet2024-08-083-2/+26
|
* Intermediate changesrobot-piglet2024-07-311-30/+37
|
* [yt/cpp/mapreduce]Fix build with system protoc, std::string and system stlnadya732024-07-302-1/+14
| | | | | Fix build with system protoc, std::string and system stl 8b5b81b2d7855f6b0dcb4bb0583a6408e5b7d3f7
* Intermediate changesrobot-piglet2024-07-271-1/+1
|
* Intermediate changesrobot-piglet2024-07-251-1/+5
|
* Intermediate changesrobot-piglet2024-07-221-2/+9
|
* Intermediate changesrobot-piglet2024-07-211-0/+3
|
* Intermediate changesrobot-piglet2024-07-196-0/+6
|
* Intermediate changesrobot-piglet2024-07-073-3/+3
|
* Enable UTF-8 mode for allshadchin2024-06-201-1/+16
| | | | 0003a9ac951e380a13106a8f180014e0aed57f3a
* Intermediate changesrobot-piglet2024-06-131-6/+6
|
* Intermediate changesrobot-piglet2024-06-061-1/+4
|
* Intermediate changesrobot-piglet2024-06-042-4/+44
|
* Intermediate changesrobot-piglet2024-05-211-1/+1
|
* Intermediate changesrobot-piglet2024-05-181-1/+18
|
* YDB Import 597robot-ydb-importer2024-05-141-4/+0
| | | | 2221b5c9d4887055279d8e5e336b944948a706cc
* Extend support importlib.resources in Arcadiashadchin2024-05-082-25/+48
| | | | 5d391b8f31717f85fcd88c4ae5ce3b57a723e42e
* YT-21446: Add python udfs dependency in pigletmpereskokova2024-04-164-0/+40
| | | | a960b2510dc0ed8b11807d92f1826459a8c9a5c8
* Simplify support `importlib.metadata`shadchin2024-04-132-11/+16
| | | | 996025a2ee725b626c3b77aac016d8f8c0ac4e76
* Use same configs in ya style and style-testsmikhnenko2024-04-041-8/+4
| | | | 853c2293128f6f534d97c1a5c97204e1f72868cf
* Intermediate changesrobot-piglet2024-03-281-2/+2
|
* Intermediate changesrobot-piglet2024-03-281-0/+2
|
generated by cgit v1.2.3 (git 2.25.1) at 2025-08-03 00:33:15 +0000