summaryrefslogtreecommitdiffstats
path: root/build/scripts
Commit message (Collapse)AuthorAgeFilesLines
* Up compileSdk -> 36aosipenko2026-07-092-6/+6
| | | | | | | Поднимаю compileSdk, targetSdk: 35 -> 36 Понадобилось для подключения kotlin-native модулей камеры в тест-апп. В камере используется sdk 36. commit_hash:7cf3f51e915ad22b867f5eae1b4c4d146a296bcf
* [clang-tidy] Allow clang-tidy on generated sources.born-in-void2026-07-081-2/+14
| | | | | | | Adds an opt-in mode for clang-tidy to process generated sources. By default, clang-tidy keeps the current behavior and skips sources from the build root. With `-DTIDY_ALLOW_GENERATED=yes`, the generated `clang_tidy.py` command receives `--allow-generated-sources` and does not skip those files. commit_hash:7d7939f1fd226722c2a6a07a48ea025194897c73
* conf var feature flag (__NO_UID__) + shallow root-based cache diriniklyaev2026-07-022-7/+12
| | | | | | | | | | **Feature-flag the persistent javac/kotlinc daemon and relocate its runtime dir to SHALLOW_ROOT** The javac/kotlinc daemon is now opt-in via `ya make -DYA_JAVAC_DAEMON` (default off). Toggling it does not change node UIDs; enabling/disabling the daemon never invalidates the build cache. The daemon's runtime dir (jar \+ logs) moves from `/tmp` to ya's `$(SHALLOW_ROOT)` via `YA_JAVAC_DAEMON_HOME`, so it works in environments where `/tmp` is unwanted. The unix socket stays in `/tmp` because of the AF_UNIX path limit; the socket name gains the jar hash to distinguish daemon versions across protocol changes. This code is LLM-generated. commit_hash:179a98b265f6458d04a6bf6c7d02721c90cc355b
* Javac/kotlinc daemon [AI generated]iniklyaev2026-06-304-18/+1009
| | | | | | | | | | | | | **Persistent javac/kotlinc compilation daemon for `ya make` (opt-in)** **Note:** the code in this PR was generated mostly by an LLM. This PR adds an opt-in persistent compilation daemon that runs `javac` and `kotlinc` in-process inside a long-lived JVM, instead of spawning a fresh subprocess per build node. Each Java/Kotlin module today pays the full JVM \+ compiler startup cost (\~150–340 ms of startup, plus \~55 MB of kotlinc class-loading) on every node, which dominates the wall time of short compiles; amortizing that startup across all nodes in a build session yields measurable speedups (1.3x–3.7x CPU economy on the benchmarked targets, with zero correctness fallbacks or failed nodes). The daemon (`devtools/buildstep_tools/javac_daemon`, a `JAVA_PROGRAM` built to a committed `JavacDaemon.jar`) listens on a Unix domain socket and runs both compilers concurrently; a thin Python client in the existing build scripts (`javac_daemon_client.py`, wired into `run_javac.py` / `compile_java.py` via `build/conf/java.conf`) decides per-node eligibility, routes launcher flags, selects/launches the right daemon instance, and **always falls back to a plain subprocess** on any doubt (old JDK, resource JVM flag, denied annotation processor, daemon error), so build correctness never depends on the daemon. The feature is **off by default** and activated only via `YA_JAVAC_DAEMON=1` passed to `ya make`; without it the build uses the existing subprocess path unchanged. See `junk/iniklyaev/javac-daemon/DESIGN_javac_daemon.md` for the full design, flag-routing rules, fault-isolation model, and benchmark methodology. commit_hash:f4e07c5bb51064e0bc24539348e01a85ff63a835
* use modern threading api .daemon=Trueya-shishqa2026-06-251-1/+1
| | | | | | | | | | fix warning: ``` build/scripts/yndexer.py:31: DeprecationWarning: setDaemon() is deprecated, set the daemon attribute instead thread.setDaemon(True) ``` commit_hash:e0f678398f698f93b319d3f864ba4ef495176df9
* Remove Go 1.25 coverage feature flag from conf, remove old Go coverage code ↵dimdim112026-06-231-73/+0
| | | | | | from conf commit_hash:6e76745385a337c7399c1d643db30025b47f7bd5
* [clang-tidy] Aggregate build volume metrics.born-in-void2026-06-201-3/+28
| | | | | | Aggregates build_volume from .global.tidyjson inputs into clang-tidy suite metrics. This keeps dependency statistics out of per-file test cases while preserving them as target-level metrics in the test trace. commit_hash:56b9580abe19b8a9f336d6378cdec988bea4d778
* Fixes for Go coverings empty source setsdimdim112026-06-202-2/+24
| | | | | Fixes for Go coverings empty source sets commit_hash:8529d3bff57df23db59b0c15e97f33c25213e156
* Fix merge issuespreis2026-06-171-2/+2
| | | | commit_hash:ff7a1cf6cf69a636f8a1ecd31af363331eb168be
* [clang-tidy] Add build volume counting.born-in-void2026-06-171-5/+39
| | | | | When TIDY=yes, clang-tidy now calculates the total source volume of the translation unit (in bytes) and passes it to profile.build_volume. It is enabled by default commit_hash:1a2608000828b09165d431c72cab0778d7a06355
* Report usage of banned clang-tidy checksalevitskii2026-06-161-3/+17
| | | | commit_hash:a4643a5df8002e7f8387908a1dbc89448935125a
* Aggregate clang-tidy metricsspreis2026-06-151-1/+22
| | | | commit_hash:a8994274757635fd3a37114db47a7cf2db9c3cb8
* move classpath related args into response filesvidyuk2026-06-071-6/+3
| | | | | | classpath length is proportional to a number of a project transitive dependencies and is not limited by any reasonable upper bound. commit_hash:d9f3eb9016298ce371b2c632e70e78d60f7b3ee7
* Disable Java Annotation Processing by defaultdimdim112026-06-012-6/+7
| | | | commit_hash:b1d2bd3f39ea9039fb7bd70d18f20ff7cad737d2
* Use Local=False mode for make coverage datadimdim112026-05-291-4/+21
| | | | | Use Local=False mode for make coverage data commit_hash:9ec3fd45232a4960081116b45a4e9bd8667ef5e1
* Implement PARALLEL_TESTS_WITHIN_NODE for javaayevttukh2026-05-291-0/+6
| | | | commit_hash:68b519ca14bc9c0656b7e38db569a04aca127e65
* remap -Xfriend-paths to interface jar when ijar is activeiniklyaev2026-05-261-0/+19
| | | | | | | | | | | | | When ijar compilation is active, the classpath contains X-interface.jar instead of X.jar. Kotlin only grants 'internal' visibility access via -Xfriend-paths when the specified path matches a classpath entry. Fix: in run_javac.py, for Kotlin compilations, rewrite any -Xfriend-paths=X.jar argument to -Xfriend-paths=X-interface.jar when X-interface.jar exists on disk. This restores access to 'internal' members for test modules that use -Xfriend-paths without requiring changes to user ya.make files. commit_hash:a11a6acd8994258a32090a71de9a55d5aeccc5dd
* Fix Windows go_tool.py linking + smoke testzienag2026-05-251-2/+15
| | | | | | | | | | | | | | | | | | Фикс go_tool.py + тест. go_tool.py пишет аргументы линкера в response file и зовёт `go link @file`. Линкер парсит файл через cmd/internal/objabi.DecodeArg, который трактует `\` как начало эскейпа и падает ("badly formatted input") на всём кроме `\\` и `\n`. На Windows-хосте пути вида `C:\place\...\main.a` валят сборку любой GO_PROGRAM. Фикс: писать аргументы через аналог Go EncodeArg (`\` → `\\`, `\n` → `\n`) и открывать tempfile с newline='\n', чтобы текстовый режим Python не подменял переводы строк на CRLF. Добавлен build/tests/scripts/go_win_smoke — крошечный GO_PROGRAM и PY3TEST, которые гоняются под common/arcadia/ya_make_windows на нативном Windows. commit_hash:ef8abe298a65d1cf1c1463a16f937a31db2b6fb2
* Tests for Go SSO script and minor fixes in go SSOdimdim112026-05-151-3/+12
| | | | | | Refactor tests and some fixes in Go SSO Tests for Go SSO script commit_hash:31a766a749be2a9ca05bbca314a2f159c2b6507d
* Fix clang analyzer after enabling debug section zstd compressionsvidyuk2026-05-141-0/+2
| | | | commit_hash:10a3174bd909631074d85243caff1e54ec53b646
* Support any one package name for Go 1.25 coveragedimdim112026-05-141-16/+32
| | | | | Support any one package name for Go 1.25 coverage commit_hash:c2c5ba2c3b4e18dddf4518abaf6dec045ef8c3cf
* feat all: use zstd compression of debug symbols on Linuxantoshkka2026-05-141-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | File sizes: ``` -g 910385468 Jan 1 1970 libyt-yt-client.a -g -gz 589148980 May 6 12:16 libyt-yt-client.a -g -gz=zstd 585019668 May 6 12:34 libyt-yt-client.a -g -gz=zstd 1674173480 May 6 13:27 drive-billing-exports -g -gz=zstd -Wl,--compress-debug-sections=zstd 899287344 May 6 14:00 drive-billing-exports ``` Mesured compile+link times with the following script: ``` arc checkout . sed -i "s/${SERVICE_NAME}/${SERVICE_NAME}_${I}_${RANDOM}_${RANDOM}_${RANDOM}_${RANDOM}_${RANDOM}_${RANDOM}/g" src/custom/dependencies.cpp OUTPUT=$(~/workspace/arcadia/ya make --stat 2>&1) echo "$OUTPUT" | grep -om1 "^\[.* \[LD\] .*${SERVICE_NAME}/${SERVICE_NAME}" >>ld.txt echo "$OUTPUT" | grep -om1 "^\[.* \[CC\] .*${SERVICE_NAME}/src/custom/dependencies.cpp" >>cc.txt echo "$OUTPUT" >>full.txt ``` Trunk times in milliseconds: CC: 8944 8405 8117 11203 19236 11660 9149 24319 21236 14753 25508 28099 9226 31387 9555 11268 9088 10640 13751 8855 11266 9141 8168 8712 11111 10235 28576 8996 9362 8973 9055 9822 LD: 5956 9394 15380 9055 7089 7308 15433 12948 14123 5524 4808 10559 6837 4906 18253 5208 16335 7493 5180 47220 12491 21603 34733 4432 5353 8633 4732 25937 19158 24316 5825 7357 ZSTD times in milliseconds: CC: 10180 8017 13135 8569 16033 17698 12524 8442 12902 7797 20236 7857 14715 8096 8541 9232 10839 8989 8988 17173 8211 8737 22424 8139 23770 8622 13765 12744 8204 23573 8358 8123 9091 18869 LD: 6125 5914 6975 21408 15227 7309 6401 13417 8862 24954 5509 33813 6207 5571 15423 5716 5807 6352 16953 6064 22793 14156 7203 10811 6287 15363 6259 5897 25905 5400 20217 5624 17351 6593 ``` | CC min | CC median | CC max | CC avg | LD min | LD median | LD max | LD avg ------|--------|-----------|--------|--------|--------|-----------|--------|------- Trunk | 8117 | 9822 | 31387 | 13369 | 4432 | 8633 | 47220 | 12612 ZSTD | 7797 | 9091 | 23770 | 12135 | 5400 | 6975 | 33813 | 11584 ``` **Compilation time and linking time drop down on ~10%** **Conclusion**: archive and binary sizes drop down on 33%-45%. As a result: * OS FS cache works better, less files required for compilation are flushed out from memory * linking touches smaller files and works faster * compilation produces smaller files and does less slow disk IO commit_hash:6053e4fe15a64966bd96c99dbc7a068c3a2cc0e7
* Fix unused _cover_xtest importborinskikh2026-05-121-1/+5
| | | | commit_hash:5fb76d5479c4990d9c9a2876f9e5753820f32ed1
* Revert "feat all: use zstd compression"antoshkka2026-05-101-2/+0
| | | | | | This reverts commit bebcd6cedb271b14b7d771d50924c005ab9375bf, reversing changes made to 700c24e95953f7be1a6c425a35dbf7d41492469a. commit_hash:e14a9ef12e284a3ba034063c3c03b0d57323e574
* feat all: use zstd compressionantoshkka2026-05-091-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | File sizes: ``` -g 910385468 Jan 1 1970 libyt-yt-client.a -g -gz 589148980 May 6 12:16 libyt-yt-client.a -g -gz=zstd 585019668 May 6 12:34 libyt-yt-client.a -g -gz=zstd 1674173480 May 6 13:27 drive-billing-exports -g -gz=zstd -Wl,--compress-debug-sections=zstd 899287344 May 6 14:00 drive-billing-exports ``` Mesured compile+link times with the following script: ``` arc checkout . sed -i "s/${SERVICE_NAME}/${SERVICE_NAME}_${I}_${RANDOM}_${RANDOM}_${RANDOM}_${RANDOM}_${RANDOM}_${RANDOM}/g" src/custom/dependencies.cpp OUTPUT=$(~/workspace/arcadia/ya make --stat 2>&1) echo "$OUTPUT" | grep -om1 "^\[.* \[LD\] .*${SERVICE_NAME}/${SERVICE_NAME}" >>ld.txt echo "$OUTPUT" | grep -om1 "^\[.* \[CC\] .*${SERVICE_NAME}/src/custom/dependencies.cpp" >>cc.txt echo "$OUTPUT" >>full.txt ``` Trunk times in milliseconds: CC: 8944 8405 8117 11203 19236 11660 9149 24319 21236 14753 25508 28099 9226 31387 9555 11268 9088 10640 13751 8855 11266 9141 8168 8712 11111 10235 28576 8996 9362 8973 9055 9822 LD: 5956 9394 15380 9055 7089 7308 15433 12948 14123 5524 4808 10559 6837 4906 18253 5208 16335 7493 5180 47220 12491 21603 34733 4432 5353 8633 4732 25937 19158 24316 5825 7357 ZSTD times in milliseconds: CC: 10180 8017 13135 8569 16033 17698 12524 8442 12902 7797 20236 7857 14715 8096 8541 9232 10839 8989 8988 17173 8211 8737 22424 8139 23770 8622 13765 12744 8204 23573 8358 8123 9091 18869 LD: 6125 5914 6975 21408 15227 7309 6401 13417 8862 24954 5509 33813 6207 5571 15423 5716 5807 6352 16953 6064 22793 14156 7203 10811 6287 15363 6259 5897 25905 5400 20217 5624 17351 6593 ``` | CC min | CC median | CC max | CC avg | LD min | LD median | LD max | LD avg ------|--------|-----------|--------|--------|--------|-----------|--------|------- Trunk | 8117 | 9822 | 31387 | 13369 | 4432 | 8633 | 47220 | 12612 ZSTD | 7797 | 9091 | 23770 | 12135 | 5400 | 6975 | 33813 | 11584 ``` **Compilation time and linking time drop down on ~10%** **Conclusion**: archive and binary sizes drop down on 33%-45%. As a result: * OS FS cache works better, less files required for compilation are flushed out from memory * linking touches smaller files and works faster * compilation produces smaller files and does less slow disk IO commit_hash:bebcd6cedb271b14b7d771d50924c005ab9375bf
* поправить скрипт распаковки output.tarbaymer2026-05-061-10/+20
| | | | | | | | | | | | | | | | | <section id="quibbler-autodescription"> #### Исправление скрипта распаковки tar-архива 📝 - 🛠️ Обновлён механизм распаковки tar-архива с использованием встроенного модуля `tarfile` в качестве резервного варианта, если системная команда `tar` недоступна. - 🧪 Добавлена проверка исполняемости команды `tar` перед её использованием, обеспечивая более надёжную работу скрипта. - 📦 Изменена логика работы с архивами: теперь поддерживается распаковка как через системную утилиту, так и через Python-модуль, что повышает совместимость. - 📁 Упрощён код основной функции, вынесена логика распаковки в отдельную функцию `unpack_dir` для лучшей читаемости и поддержки. - 📦 Обновлены параметры сборки: изменён формат архива с `.tar.gz` на `.tar.zst` и добавлен параметр сжатия `zstd`. - 🔄 Исправлен путь к скрипту распаковки в конфигурации сборки, теперь он указывает на актуальное расположение файла. <a href="https://nda.ya.ru/t/qa0kX64r7DqvtN"><font size="2">Autodescription by Yandex Code Assistant</font></a> </section> commit_hash:b2b7d3aa3bb8543aae5627c43536c1bfca5a9cfe
* Добавить скрипт для распаковки output.tar под ↵baymer2026-05-051-0/+48
| | | | | | ya package commit_hash:61c6668829d1d4a00cae6e046cacdc31496010e3
* do not try to run compiler without sources.svidyuk2026-04-241-0/+3
| | | | | Print human readable description instead of misleading compiler help message. commit_hash:78aa6c12f5a3b88535805f3dc5f39565c1295493
* Support Go 1.25 coverage in configure under disabled feature flagdimdim112026-04-133-63/+395
| | | | commit_hash:8c93eea4565075c47db3532653529ae3b7a76e40
* response file for go program linkingvpozdyayev2026-04-092-3/+12
| | | | commit_hash:9d59413e4935e2c5b33c1a84bf6244c151fe16e7
* Use clang18 for BPFmikhnenko2026-03-261-2/+12
| | | | | | | | | | | | | #### Обновление clang до версии 18 и оптимизация фильтрации флагов для BPF 📝 - 🛠️ Обновлен инструмент компиляции с clang14 на clang18 для генерации eBPF-кода - 🧹 Удалены дублирующиеся флаги целевой архитектуры при сборке BPF-модулей - 📦 Оптимизирован фильтр флагов компилятора для исключения STL-заголовков в BPF-сборке - 📝 Удалены устаревшие флаги `-march=bpf`, так как они больше не требуются при использовании clang18 - 🧪 Уточнена конфигурация оптимизаций для BPF-модулей, включая флаги `-mcpu=v3` и `-O2` <a href="https://nda.ya.ru/t/qa0kX64r7DqvtN"><font size="2">Autodescription by Yandex Code Assistant</font></a> commit_hash:e4e0178f359b52d5961185edf15ec8a8648765e4
* Build bpf with clang14mikhnenko2026-03-251-3/+0
| | | | commit_hash:35714ea1cb62649a0482502ac3bf0912803e1ad6
* Use clang20 for bpfmikhnenko2026-03-241-2/+5
| | | | commit_hash:3b3a73c72b12bf59c5ff009a535b93abadb1f96a
* Fix after rXXXXXXshadchin2026-03-071-1/+0
| | | | commit_hash:ef7b604b707e213dbd9fbaf1cc1e7c4cbd7c4f4c
* ya tool black release 8robot-contrib2026-03-0733-75/+37
| | | | | #| || **<a href="https://nda.ya.ru/t/NmXL8DSH7W8pjd" target="_blank">![](https://nda.ya.ru/t/mAiQIjHx7Mm3JC =30x) Echo tests</a>** || |# commit_hash:595f1e5ce2f0b1215088b7c8d2b79f07955f9a4b
* Support building go_std for wasip1+wasmmikhnenko2026-02-201-2/+2
| | | | | | | | | | Тестировалось так: ``` mikhnenko@jakku:~/arcadia$ ya m devtools/dummy_arcadia/hello_go --target-platform=default-emscripten-wasm32 --sandboxing mikhnenko@jakku:~/arcadia$ wasmtime devtools/dummy_arcadia/hello_go/hello_go Hello, World! ``` commit_hash:95b3a3c437b7d0a64994c6b9aed97a1cc5ee8561
* create sanitizer toolchains with CUDAabityugov2026-02-161-3/+7
| | | | | Добавляю тулчейны asan и msan, собранные с CUDA commit_hash:3731067c580fc1343521a3a08cccbfb363f12ce2
* Enable lrodata resource insertionnechda2026-02-131-1/+1
| | | | commit_hash:e50a07bf54f78e9b26e88619aacf9ae76f68575b
* Insert large resources into lrodata (objcopy part)nechda2026-02-101-4/+25
| | | | commit_hash:24bf692e9b505b74d0b3cc7257a8ccb77c84ca64
* [build] cuda: Add .module_id sanity check to CUDA_SRCSdeshevoy2026-02-021-0/+11
| | | | | | | IDs generated by different cicc invocations should match ISSUE: commit_hash:7cd593cee44b31875e7166709d7614dcfa3f1f14
* [build] cuda: Introduce CUDA_SRCS macro utilizing parallelized device code ↵deshevoy2026-02-012-6/+30
| | | | | | | | | | | | | | | | compilation Instead of a single graph node launching NVCC to compile .cu for both host and all device architectures CUDA_SRCS generates multiple nodes: - node per each device architecture producing PTX and CUBIN - node merging all PTX and CUBIN files into a single FATBIN blob - node producing .cpp with host code - node compiling host .cpp with embedded FATBIN blob CUDA_ARCHITECTURES variable is used to determine the list of architectures to compile device code for. ISSUE: commit_hash:0a4c2797dd238ae062482af30694df6978301278
* Fix 'Argument list too long'say2026-01-152-2/+7
| | | | commit_hash:42cbe679894a95a825348d1af0e50bac0c87a7eb
* Split py3cc into slow & fastnechda2025-12-011-1/+2
| | | | commit_hash:d23fab7a0d429fff427da7d877e63fc4eee2ed4f
* DOCSBOOK is final target again; additional output preprocessed.tar.gz for ↵snermolaev2025-11-201-14/+2
| | | | | | DOCSBOOK commit_hash:5a3ad292b315b6435b4929ce2a5dfe5f9ffed7cf
* Mach-o objcopy resource insertion support (preparation only)nechda2025-11-181-35/+76
| | | | commit_hash:c8efa31c9cc0e080a87437782707506a75fbf48c
* docs.manifestsnermolaev2025-11-122-0/+114
| | | | commit_hash:650c7d56fe7044ca181cce3e6feac7b7f28b4e41
* Migrate to py3 java related scriptsnechda2025-11-082-3/+16
| | | | commit_hash:8bb7796dc00308eff1ba80875b36a09d2c23b366
* Migrate to py3 clang-tidy scriptsnechda2025-11-051-7/+17
| | | | commit_hash:a56e54a5e36856311957f70412a32eaab769532e
* Migrate wine & dsymutil to py3nechda2025-11-042-54/+55
| | | | commit_hash:6cec762a041602a74dce5a4c6c3828f032b4344f
* betterpg2025-10-291-1/+0
| | | | commit_hash:044bb76cacea25d38a1939236f98aa8e3d6b4def
generated by cgit v1.3 (git 2.53.0) at 2026-07-25 01:47:24 +0000