aboutsummaryrefslogtreecommitdiffstats
path: root/library/python/runtime_py3
Commit message (Collapse)AuthorAgeFilesLines
* Subinterpretor compatible __res module (2nd attempt)snermolaev2025-04-2915-83/+617
| | | | | | | Cython is not yet subinterpreter compatible. There are no ETA when cython is going to support subinterpreters. This PR removes cython from hermetic python imoprt hooks in order to make them subinterpretr-compatible. commit_hash:427b6f9db6afa6695659ee147621e1ccb391d3cb
* Revert "Subinterpretor compatible __res module" rXXXXXXsnermolaev2025-04-1515-587/+83
| | | | commit_hash:1d9f5675b9d3ddaa44db4472376a06ad3e811e2d
* Subinterpretor compatible __res modulesnermolaev2025-04-1415-83/+587
| | | | | | | Cython is not yet subinterpreter compatible. There are no ETA when cython is going to support subinterpreters. This PR removes cython from hermetic python imoprt hooks in order to make them subinterpretr-compatible. commit_hash:1b067c37f55a4f1d9a6172df7009c75231cc1e25
* [library/python/runtime_py3/importer.pxi] Support trailing newline in ↵prettyboy2025-03-191-3/+6
| | | | | | | | | | root.path files Для того чтобы работал сценарий ``` echo `arc root` > /tmp/.root.path ``` commit_hash:255b16f66c1a610cea5322a3842edc492634faaf
* fix arcadia py3 runtime for win due to encodingkuzmich3212025-02-281-1/+30
| | | | | | | | | [Вот так](HIDDEN_URL делает нативный питон [Репродьюсер](HIDDEN_URL Как оно выглядит [ДО](HIDDEN_URL и [ПОСЛЕ](HIDDEN_URL, а вот так [нативный](HIDDEN_URL commit_hash:cdce69c28e699d42f25e43f5d6efb214c8227024
* Add `stem` propertyshadchin2025-02-021-0/+3
| | | | commit_hash:ee709cc1095985919972e31b34d0127eb322959f
* [library/python/runtime_py3] Made Y_PYTHON_SOURCE_ROOT bytes againprettyboy2025-01-231-1/+1
| | | | commit_hash:3433c11c4d386c785422b1e149e9a3a8de7f4f0f
* [devtools/ya] Dont' write __pycache__ to the arcadia in external-py-files modeprettyboy2025-01-231-1/+29
| | | | | | | | | В репозитории больше не будет `__pycache__` при запусках через ya make / test На примере devtools/ya/handlers/analyze_make/tests Время конфигурации в горячем сценарии в режиме --ext-py при изменении одного питонфайлика: `Configure time - 6.1 s` Время конфигурации в горячем сценарии в режиме --ext-py при изменении одного питонфайлика с текущими правками: `Configure time - 4.6 s` commit_hash:adf5219cd4f347bb572393d4a6debd1436dc0815
* [library/python/runtime_py3] Added bytecode support for external-py-files modeprettyboy2025-01-182-9/+72
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Добавляется поддержка стандартной работы с байткодом для бинарей собранных с --ext-py (HIDDEN_URL На примере импорт теста для `devtools/ya/bin` `time Y_PYTHON_ENTRY_POINT=library.python.testing.import_test.import_test:main YA_TEST_CONTEXT_FILE=/tmp/test.context ./ya-bin >/dev/null` #| || Без патча ya m -r | real 0m2,934s user 0m2,772s sys 0m0,157s || || Без патча ya m -r --ext-py | real 0m10,916s user 0m9,927s sys 0m0,368s || || C патчем ya m -r | real 0m2,931s user 0m2,774s sys 0m0,156s || || C патчем ya m -r --ext-py первый запуск с записью байткода | real 0m7,482s user 0m5,174s sys 0m0,620s || || C патчем ya m -r --ext-py последующие запуски с использованием байткода с фс | real 0m3,550s user 0m2,801s sys 0m0,262s || |# commit_hash:84f7bb273b09d51b88eb5c5b6dfcd3bb7c108307
* Prototype - external python files mode for testsprettyboy2025-01-163-9/+51
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Режим позволяет перезапускать python тесты или интеграционные тесты на питон программы при внесении изменений в python файлы без перекомпиляции. Сценарий: - прогнали тесты - внесли изменение в `library/python/func/__init__.py` - перезапустили, смотрим на времена На примере интеграционного devtools/ya/handlers/analyze_make/tests в codenv (base конфигурация). #| || ya m -rA | ya m -rA --ext-py || || real 10m25,928s user 0m53,465s sys 1m14,791s ``` Total time by type: [935171 ms] [LD] [count: 3, ave time 311723.67 msec] [ 13657 ms] [prepare:AC] [count: 4, ave time 3414.25 msec] [ 10436 ms] [TM] [count: 1, ave time 10436.00 msec] [ 10387 ms] [prepare:get from local cache] [count: 1521, ave time 6.83 msec] [ 6480 ms] [TS] [count: 1, ave time 6480.00 msec] [ 3823 ms] [prepare:put into local cache, clean build dir] [count: 7, ave time 546.14 msec] [ 2279 ms] [CC] [count: 1, ave time 2279.00 msec] [ 1251 ms] [PY] [count: 1, ave time 1251.00 msec] [ 372 ms] [prepare:yt-store] [count: 2, ave time 186.00 msec] [ 154 ms] [AR] [count: 1, ave time 154.00 msec] ... ``` | real 0m22,094s user 0m26,372s sys 0m3,363s ``` Total time by type: [13140 ms] [TM] [count: 1, ave time 13140.00 msec] [ 351 ms] [prepare:yt-store] [count: 1, ave time 351.00 msec] [ 81 ms] [prepare:tools] [count: 8, ave time 10.12 msec] [ 32 ms] [prepare:AC] [count: 2, ave time 16.00 msec] [ 31 ms] [prepare:get from local cache] [count: 7, ave time 4.43 msec] [ 16 ms] [prepare:clean] [count: 3, ave time 5.33 msec] ``` || |# В codenv в base конфигуарции (4cpu) невероятно долго идёт линковка. Тот же самый сценарий, но на mous (54cpu) #| || ya m -rA | ya m -rA --ext-py || || real 0m25,435s user 0m40,242s sys 0m22,434s ``` Total time by type: [30958 ms] [prepare:get from local cache] [count: 1521, ave time 20.35 msec] [16293 ms] [prepare:AC] [count: 4, ave time 4073.25 msec] [11161 ms] [LD] [count: 3, ave time 3720.33 msec] [ 4256 ms] [TM] [count: 1, ave time 4256.00 msec] [ 3529 ms] [TS] [count: 1, ave time 3529.00 msec] [ 1542 ms] [CC] [count: 1, ave time 1542.00 msec] [ 635 ms] [PY] [count: 1, ave time 635.00 msec] ... ``` | real 0m18,321s user 0m21,673s sys 0m3,204s ``` Total time by type: [10786 ms] [TM] [count: 1, ave time 10786.00 msec] [ 1568 ms] [prepare:yt-store] [count: 1, ave time 1568.00 msec] [ 151 ms] [prepare:tools] [count: 8, ave time 18.88 msec] [ 61 ms] [prepare:get from local cache] [count: 7, ave time 8.71 msec] [ 48 ms] [prepare:AC] [count: 2, ave time 24.00 msec] [ 14 ms] [prepare:clean] [count: 3, ave time 4.67 msec] ``` || |# В новом режиме тест ожидаемо замедляется, так как все питон программы при каждом старте вынуждены читать файлы с фс (ещё через арк), а потом ещё и компилировать байткод, т.е. холодный запуск commit_hash:24c5a46a8385d3c065abfb6fc5b40f7ad24bb1cb
* Fix for Python 3.13shadchin2025-01-161-6/+0
| | | | commit_hash:2a9f4a61b44a7e34d430d8615b5c9d2d120fa7d9
* Never compress python build resources to speedup python configuration and buildspreis2024-12-131-0/+1
| | | | | | | There is no runtime perf impact, but - The change reduces build graph by removing 2 nodes out of 5 for python which accelerates both configuration and build - surprisingly in all experiments there were positive impact on binary size about 2MiB commit_hash:909fdeeb91b7f20c749177d638e97d5a469e422d
* Enable UTF-8 mode for allshadchin2024-06-201-1/+16
| | | | 0003a9ac951e380a13106a8f180014e0aed57f3a
* Extend support importlib.resources in Arcadiashadchin2024-05-082-25/+48
| | | | 5d391b8f31717f85fcd88c4ae5ce3b57a723e42e
* Simplify support `importlib.metadata`shadchin2024-04-132-11/+16
| | | | 996025a2ee725b626c3b77aac016d8f8c0ac4e76
* Probe bytes and str environment keysv-korovin2024-02-291-3/+18
| | | | 58bed0cef82a2a691f8f949d16c3d3b03d94f627
* Sync contrib/tools/python3 layout with upstreamthegeorg2024-02-194-6/+6
| | | | | | | * Move src/ subdir contents to the top of the layout * Rename self-written lib -> lib2 to avoid CaseFolding warning from the VCS * Regenerate contrib/libs/python proxy-headers accordingly 4ccc62ac1511abcf0fed14ccade38e984e088f1e
* Update Python from 3.11.8 to 3.12.2shadchin2024-02-122-13/+13
|
* Rework joinpath for importlib.resourcesshadchin2024-02-093-30/+98
|
* Library import 5, delete go dependencies (#832)AlexSm2024-01-041-3/+6
| | | | | * Library import 5, delete go dependencies * Fix yt client
* Import libs 3 (#679)AlexSm2023-12-221-1/+2
|
* Support override `distutils` from `setuptools`shadchin2023-12-121-0/+26
|
* First implementation is `importlib.resources.files`shadchin2023-11-262-15/+82
|
* Python Import Tracingkuzmich3212023-11-241-2/+7
|
* Add import callbacks for import hookkuzmich3212023-11-222-2/+21
| | | | | | * typo-fix * add before and after import callbacks
* add ydb depsnkozlovskiy2023-09-2939-0/+1929
|
* fix ya.makemonster2022-07-0735-1738/+0
|
* Update contrib/python/ipython/py3 to 8.3.0robot-contrib2022-05-185-71/+31
| | | | ref:e84342d4d30476f9148137f37fd0c6405fd36f55
* YMAKE-148: Don't follow symlinkssay2022-05-172-5/+17
| | | | ref:b200b70556036c6a735ab81b1414c4b334ba5958
* YMAKE-144: Fix RuntimeError: dictionary changed size during iterationsay2022-05-142-2/+30
| | | | ref:919db9913468e91d031bbd511beeacdcfdd74489
* DEVTOOLSSUPPORT-18626: In venv directly call Py_BytesMain()say2022-05-042-0/+10
| | | | | | | | | | The arcadia main function (library/python/runtime_py3/main/main.c::pymain) calls Py_InitializeFromConfig() before going to the vanilla python Py_Main(). Py_InitializeFromConfig() overrides sys.prefix and sys.exec_prefix by default values ('/var/empty') and then import __res module (and importer.pxi as a part of it). importer.pxi::_init_venv() set a proper value to sys.prefix and sys.exec_prefix attributes. Py_Main() in turn also calls Py_InitializeFromConfig() and reset the sys attributes to their default values again. I don't know why it was not a problem in python 3.9 but in 3.10 it is. ref:0466de641f92d66c4f992ae12c02d8d58db397f6
* intermediate changesarcadia-devtools2022-03-154-134/+0
| | | | ref:ca7a95e8c9a9d780f96497136a152091d54e61b5
* Restoring authorship annotation for <roboslone@yandex-team.ru>. Commit 2 of 2.roboslone2022-02-101-1/+1
|
* Restoring authorship annotation for <roboslone@yandex-team.ru>. Commit 1 of 2.roboslone2022-02-101-1/+1
|
* Restoring authorship annotation for <viknet@yandex-team.ru>. Commit 2 of 2.viknet2022-02-101-20/+20
|
* Restoring authorship annotation for <viknet@yandex-team.ru>. Commit 1 of 2.viknet2022-02-101-20/+20
|
* Restoring authorship annotation for <say@yandex-team.ru>. Commit 2 of 2.say2022-02-103-539/+539
|
* Restoring authorship annotation for <say@yandex-team.ru>. Commit 1 of 2.say2022-02-103-539/+539
|
* Restoring authorship annotation for Aleksandr <ivansduck@gmail.com>. Commit ↵Aleksandr2022-02-102-23/+23
| | | | 2 of 2.
* Restoring authorship annotation for Aleksandr <ivansduck@gmail.com>. Commit ↵Aleksandr2022-02-102-23/+23
| | | | 1 of 2.
* Restoring authorship annotation for <abodrov@yandex-team.ru>. Commit 2 of 2.abodrov2022-02-1021-335/+335
|
* Restoring authorship annotation for <abodrov@yandex-team.ru>. Commit 1 of 2.abodrov2022-02-1021-335/+335
|
* Restoring authorship annotation for <dldmitry@yandex-team.ru>. Commit 2 of 2.dldmitry2022-02-101-3/+3
|
* Restoring authorship annotation for <dldmitry@yandex-team.ru>. Commit 1 of 2.dldmitry2022-02-101-3/+3
|
* Restoring authorship annotation for <spreis@yandex-team.ru>. Commit 2 of 2.spreis2022-02-102-12/+12
|
* Restoring authorship annotation for <spreis@yandex-team.ru>. Commit 1 of 2.spreis2022-02-102-12/+12
|
* Restoring authorship annotation for <deshevoy@yandex-team.ru>. Commit 2 of 2.deshevoy2022-02-103-159/+159
|
* Restoring authorship annotation for <deshevoy@yandex-team.ru>. Commit 1 of 2.deshevoy2022-02-103-159/+159
|
* Restoring authorship annotation for Nikita Slyusarev ↵Nikita Slyusarev2022-02-104-21/+21
| | | | <nslus@yandex-team.com>. Commit 2 of 2.
* Restoring authorship annotation for Nikita Slyusarev ↵Nikita Slyusarev2022-02-104-21/+21
| | | | <nslus@yandex-team.com>. Commit 1 of 2.
generated by cgit v1.2.3 (git 2.25.1) at 2025-06-14 01:58:38 +0000