summaryrefslogtreecommitdiffstats
path: root/library
Commit message (Collapse)AuthorAgeFilesLines
* Sync branches 260727-0105YDBot28 hours7-33/+382
|\
| * YT-28451: Add TLoggingTagList, WithFormat and fix -Wunused-value in fluent ↵babenko32 hours7-33/+382
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | tagged logging Three additions to the fluent tagged logging API in `library/cpp/yt/logging`. **1. `.WithFormat(key, format, args...)`** composes one tag out of several values, for tags that are not a single formattable value: ```cpp .WithFormat("Method", "%v.%v", service, method) ``` **2. `TLoggingTagList`** (new `tag.h` / `tag-inl.h`) — an opaque, pre-serialized list of keyed tags holding the tag section of the `TTaggedPayloadWriter` wire format. A component builds its diagnostic tags once, with the same fluent API, and splices them into many events via `.With(tagList)`, so the tags stay structured key/value pairs instead of collapsing into a single string: ```cpp auto tags = TLoggingTagList() .With("ConnectionId", id) .With("Endpoint", endpoint); ... YT_TLOG_DEBUG("Pollable registered") .With(tags); ``` **3. A `-Wunused-value` fix.** The fluent macros now end their expansion in `loggingGuard__.Self()` rather than naming the guard. A tag-less `YT_TLOG_INFO("Message");` otherwise expands to a discarded id-expression, and `-Wunused-value` fires on those whenever the call is spelled inside a macro *argument* (e.g. within `BIND(...)`) rather than a macro *body*. commit_hash:ef3213619e4c47808519906b5a16673bf5e94c45
* | Sync branches 260726-0104YDBot2 days2-17/+15
|\|
| * YT-28451: Migrate library/cpp/yt consumers to YT_TLOG_* APIbabenko3 days2-17/+15
| | | | | | | | commit_hash:6ab9c84d85a668a8047593f61b5c9e1a7be60377
* | Sync branches 260725-0103YDBot3 days2-13/+14
|\|
| * New version of the tld SKIP_CHECK SKIP_REVIEWrobot-ratatosk3 days1-1/+2
| | | | | | | | commit_hash:0a10662cb824dede898d600662723224e893c38b
| * YT-28451: Fix -Wdangling-else in YT_TLOG_*_IF / _UNLESS macrosbabenko4 days1-12/+12
| | | | | | | | commit_hash:ebf99d345c17fbfedf78ef06424c615c900689f9
* | Sync branches 260724-0100YDBot4 days14-0/+706
|\|
| * Intermediate changesrobot-piglet5 days7-0/+392
| | | | | | | | commit_hash:c6efe325f5bf64dbd849147c48c7b8678d0ae5cd
| * Intermediate changesrobot-piglet6 days7-0/+314
| | | | | | | | commit_hash:8521bae955ae15bf7265f8bb58ebe126a62841ea
* | Sync branches 260722-0101YDBot6 days5-4/+313
|\|
| * add methods ForEach() and ForEachReverse() to TPagedVector for fast ↵tobo7 days3-0/+298
| | | | | | | | | | | | iteration over elements commit_hash:3cde1a78c5f1af49a50168bbc85c6f9a8f28ad73
| * Make THROW_ERROR_EXCEPTION_UNLESS/IF chainable with error attributespechatnov7 days2-4/+15
| | | | | | | | commit_hash:4e76ece0b75ab787ed04b6be5f183cecbf9b8167
* | Sync branches 260721-0101YDBot7 days1-7/+16
|\|
| * fix fork deadlockmixx2398 days1-7/+16
| | | | | | | | | | | | | | Python-клиент (обертка над С\+\+ клиентом) UA с fork support виснет в `fork()` (\>50%): `TClientSession::CheckGrpcCallInactivity` (inactivity-watchdog на потоке gRPC completion-queue) брал сессионный лок `with_lock` не проверив что не идет форк. Пока pthread_atfork-обработчик `TClient::PreFork` держит этот лок в `WaitAll()`, поллер застревает на CheckGrpcCallInactivity -\> сессия не закрывается -\> `fork()` не завершается. Фикс: `CheckGrpcCallInactivity` захватывает лок `TryAcquire` \+ выход по `ForkInProgress` - как уже делает `Poll()` commit_hash:79b1f727d8ecc9b54e386b586f4f49bf1d6512d8
* | Sync branches 260718-0100YDBot10 days16-546/+60
|\|
| * Intermediate changesrobot-piglet10 days3-0/+20
| | | | | | | | commit_hash:7466cacb2971d86743b7541ddb2fbb6c8850085a
| * Make Ascii-related functions and StripString() constexprthegeorg10 days1-0/+18
| | | | | | | | commit_hash:ff7ca9a2428930638288f8c0e92a303b8f620063
| * move dqrun to yt/uzhas11 days9-540/+0
| | | | | | | | | | | | | | | | | | | | | | - Move `contrib/ydb/library/yql/providers/dq/{config,local_gateway,service,stats_collector}` and gateway/control implementation to `yt/yql/providers/dq/` - Move `contrib/ydb/library/yql/tools/dqrun` to `yt/yql/tools/dqrun` - Clean up `contrib/ydb` provider (slim `yql_dq_gateway.h`, PEERDIR updates) and fix consumers (`scheduler`, `global_worker_manager`, `pq/provider/ut`, benchmarks, solomon tests) ## Test plan - [x] `./ya make --build debug contrib/ydb/library/yql/providers/dq/provider contrib/ydb/library/yql/providers/dq/provider/exec yt/yql/providers/dq/gateway yt/yql/providers/dq/local_gateway yt/yql/providers/dq/service yt/yql/tools/dqrun` - [x] `./ya make --build debug -tA yt/yql/providers/dq/provider/ut` commit_hash:18a321f318f5903167749b88541a79b34ab1c8d7
| * Unpoison use-after-dtor access to ref countermakxenov11 days3-6/+22
| | | | | | | | | | | | | | | | Two scenarios addressed: - Access to ref counters after explicit destructor of an object - Access to vptr for upcast of an already destroyed object commit_hash:a8f27500111817f325832d006feb3ce6c3c830f1
* | Sync branches 260717-1016YDBot11 days23-274/+1966
|\|
| * YT-28451: Add tagged logging API and opaque event payloadbabenko11 days23-274/+1966
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Replace `TLogEvent::MessageRef` with an opaque `Payload` that carries the message together with structured key/value tags, deferring tag rendering to the consumer (logging thread). - Add the `TTaggedPayloadWriter`/`TTaggedPayloadReader` codec (`NDetail`), with a per-thread chunk-cached builder so tagged logging does no per-message heap allocation. - Model the payload as a typed `TLogEventPayload` — a `std::variant` of the opaque strong typedefs `TTaggedLogEventPayload` and `TStructuredLogEventPayload` (each over `TSharedRef`). The active alternative identifies the event kind, so the separate `ELogMessageKind` enum and `TLogEvent::MessageKind` field are removed; consumers dispatch via `std::holds_alternative`/`std::get` instead of a tag. - Add the fluent `YT_TLOG_*` API: `YT_TLOG_INFO("Message").With(key, value)` and `.With(key, value, "%spec")`; disabled levels skip argument evaluation. - Add well-known tags: `.With(value)` attaches a value under a statically known key resolved by ADL (e.g. `.With(error)` for the `Error` tag). - Teach the plain-text and structured formatters to render tags; add the `enable_native_tags` knob to emit tags into a nested structured attribute. - Add producer-side benchmarks. The tagged API is cheaper than `YT_LOG_*` for tagged calls and on par for tag-free ones: #| || **Producer call** | **`YT_LOG_*`** | **`YT_TLOG_*`** | **Δ** || || no tags | 81 ns | 82 ns | +2% || || 1 tag | 124 ns | 88 ns | -29% || || 2 tags | 140 ns | 101 ns | -28% || || 3 tags | 372 ns | 283 ns | -24% || |# -- #| || **<a href="https://nda.ya.ru/t/p0sVNSOC7ijzFF" target="_blank">![](https://nda.ya.ru/t/mAiQIjHx7Mm3JC =30x) Echo tests</a>** || |# commit_hash:70efc90e5c2b71e5311415a4e4508db42ff28971
* | Sync branches 260714-0059YDBot2026-07-141-3/+23
|\|
| * Fix yt logging use-after-dtormakxenov2026-07-131-3/+23
| | | | | | | | | | | | | | Similar to `CacheDestroyed` above in this file. It is possible that `ThreadMessageTag` is read by the logger during thread shutdown after this thread-local's destruction (e.g. when another thread-local's destructor logs). Added a trivially destructible flag to guard the tag content. commit_hash:852afe8245de23e9abe69a08bf5d9bfe4b17496b
* | Sync branches 260713-0103YDBot2026-07-1322-28/+689
|\|
| * Fix use-after-free in TError during program shutdownperst202026-07-122-16/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | --- Type: fix Component: library/cpp/yt/error Problem: TError::Enricher_ and TError::FromExceptionEnricher_ are static class members whose destructors are registered when error.cpp is loaded. However, if any code calls Singleton<>() during static initialization before error.cpp loads, the OnExit handler gets registered in standard atexit() first. Due to LIFO ordering, at program exit the enrichers are destroyed before OnExit() runs, but OnExit() then destroys Singletons whose destructors may create TError objects (e.g., to cancel futures), which invokes Enrich() on the already-destroyed std::function, causing use-after-free. This can lead to intermittent segfaults depending on the link order of translation units. Solution: Store enrichers in a LeakySingleton<TEnricherStorage> so they are never destroyed, as TError can be created anywhere including during program shutdown. --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1582 commit_hash:a9607f0094b4c60414d00ebca844db6a2ceafeb9
| * YT-18571: Drop the redundant `#undef FOO_INL_H_` from `-inl.h` guard blocks.babenko2026-07-1212-12/+0
| | | | | | | | commit_hash:8ccfa9ed373c83b84c21d12078e06befb05f026c
| * Add bit I/O and binary interpolative coding to library/cpp/yt/codingbabenko2026-07-128-0/+660
| | | | | | | | | | | | | | | | | | | | ### `bit_io.h` MSB-first bit-stream writer/reader (`TBitWriter` / `TBitReader`) over a caller-owned buffer. The writer flushes whole 32-bit words via the unaligned-store API; the reader assumes a few bytes of over-read slack. ### `interpolative.h` - **Truncated-binary (minimal) code** — the entropy-optimal integer code for a uniform value in `[0, rangeSize)`. - **Binary interpolative coding** — `InterpolativeEncode` / `InterpolativeDecode` for sorted, strictly increasing integer sequences over a known range `[lo, hi]`. It recursively codes the median of each subrange, compressing clustered sequences well below a flat `log2` per element with no per-element headers. Length is conveyed out of band (e.g. via the existing `varint`). commit_hash:8baf84444b8cf8e8a6e32776b4ff48582187ac2b
* | Sync branches 260712-0103YDBot2026-07-125-12/+4
|\|
| * Avoid copy in TYsonString::ToSharedRef for TCowString payloadbabenko2026-07-111-2/+2
| | | | | | | | | | Use the payload itself as the shared range holder instead of copying the string via TSharedRef::FromString. commit_hash:b427dbe6e8a8eaf3aa3c46b57f0f9965d57b3a95
| * YT-18571: Drop TSharedRef::FromString(const char* str)babenko2026-07-114-10/+2
| | | | | | | | | | | | | | | | | | Rationale: all overloads of `TSharedRef::FromString` must remain cheap and don't copy any payload. Remove the zero-terminated C string overload of `TSharedRef::FromString`. Callers passing a string literal should use `std::string/TString` explicitly. #| || **<a href="https://nda.ya.ru/t/-VG7qyBv7iD8vm" target="_blank">![](https://nda.ya.ru/t/mAiQIjHx7Mm3JC =30x) Echo tests</a>** || |# commit_hash:1a6718abcbe4e6a8f58592f55de8c37ceb2b73d3
* | Sync branches 260711-0101YDBot2026-07-112-8/+8
|\|
| * add Y_LIFETIME_BOUND to TParsedHttp* classesswarmer2026-07-112-8/+8
| | | | | | | | commit_hash:5158390863c48b4ceb69eb344ea1996573220c4b
* | Sync branches 260710-0105YDBot2026-07-109-20/+226
|\|
| * Add VNNI int8 dot product fast pathmirali7772026-07-095-3/+176
| | | | | | | | | | | | | | | | | | | | | | | | | | Adds AVX512 VNNI runtime dispatch for signed int8 dot products used by Link HNSW distance calculations, with safe fallback to the existing AVX2/SSE implementations. The signed int8 VNNI path corrects the u8*i8 bias per 64-byte chunk so accumulator semantics stay aligned with the existing int8 DotProduct implementation. Verification: - `ya make -tt library/cpp/dot_product/ut` - `ya make library/cpp/dot_product/bench` - `ya make -r library/cpp/dot_product/bench` - `ya make -tt library/cpp/dot_product/ut --sanitize=address` - `ya make -tt library/cpp/dot_product/ut --sanitize=undefined -F 'TDocProductTestSuite::TestDotProduct8' -F 'TDocProductTestSuite::TestDotProduct8VnniEdges' -F 'TDocProductTestSuite::TestDotProductCharStability'` - `ya make -tt util/system/ut -F TestCpuId` commit_hash:5624243c26df2a9e067e631477f6c3603ef45626
| * Intermediate changesrobot-piglet2026-07-091-2/+2
| | | | | | | | commit_hash:117a004505c534f300d1314155387f3e582b31c9
| * rename TPagedVectorIterator::GetOffset() to GetIndex()tobo2026-07-093-15/+48
| | | | | | | | | | | | | | | | The returned value is the element's absolute index in the vector, not an offset within a page, so GetIndex() describes it accurately. This also aligns with LLVM's PagedVector::MaterializedIterator, which exposes getIndex() for the same concept. - README.md updated to document GetIndex() - new test for iterator index added commit_hash:08a8f5f70faf0e7cef26429446886177433afa0f
* | Sync branches 260709-0106YDBot2026-07-093-62/+65
|\|
| * Intermediate changesrobot-piglet2026-07-081-0/+65
| | | | | | | | commit_hash:b287a7802acecdc3d13442505be5815167cf4f71
| * remove commended codetobo2026-07-082-62/+0
| | | | | | | | commit_hash:07da9e448456fc129126a9b96f2fef0c40492aec
* | Sync branches 260708-0102YDBot2026-07-085-185/+403
|\|
| * Intermediate changesrobot-piglet2026-07-081-35/+0
| | | | | | | | commit_hash:991ef644d607ef39801bae86415ae0190465084d
| * Make the yt and flow sources buildable in the ytsaurus-cpp-sdk exportmikari2026-07-071-0/+1
| | | | | | | | | | Make the yt and flow sources buildable in the ytsaurus-cpp-sdk export commit_hash:a733ad0d534a3717117cd80d162f6d527843decd
| * use more effective page implementationtobo2026-07-072-161/+389
| | | | | | | | | | | | - use more effective page implementation that avoid extra level of memory indirection - it also allows emplace_back() for non-movavable non-copyable types commit_hash:ef25b1f123742dea29a7d4fdf59a607e18a9850c
| * label size extensionivantsvetkov2026-07-072-1/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | **Было:** Prometheus decoder ограничивает длину значений меток метрик максимум 256 символами. Попытка передать метрику с более длинным значением метки приводит к исключению `TPrometheusDecodeException`. **Стало:** Максимальная длина значения метки расширена до 1024 символов. Системы, которые генерируют метрики с более длинными значениями (например, с ID источников данных, URL, или сложных идентификаторов), теперь могут успешно передавать такие метрики в Unified Agent без ошибок. **Практический эффект:** Улучшение совместимости с внешними системами мониторинга и приложениями, которые требуют передачи длинных меток. Существующие системы с метками < 256 символов продолжают работать без изменений. --- ## С точки зрения разработчика ### Структурные изменения Расширение требовало изменения способа хранения длины строки в пуле коротких строк: 1. **Prometheus decoder** (library/cpp/monlib): - `MAX_LABEL_VALUE_LEN`: 256 → 1024 2. **Short string pool** (logbroker/unified_agent/common): - `MaxLabelSize`: 255 → 1024 - **Ключевое изменение:** Размер строки теперь хранится как `ui16` (2 байта) вместо `unsigned char` (1 байт) - Было: `*slot->Payload() = static_cast<unsigned char>(s.size())` - Стало: `memcpy(slot->Payload(), &size, sizeof(ui16))` - Это позволяет хранить строки до 65535 символов (но логически ограничено на уровне валидации) ### Изменённые файлы | Файл | Основные изменения | |------|------------------| | `prometheus_decoder.cpp` | `MAX_LABEL_VALUE_LEN: 256 → 1024` | | `prometheus_decoder_ut.cpp` | +2 юнит-теста для новых граничных значений | | `short_string_pool.h` | `MaxLabelSize: 255 → 1024`, operator[] использует ui16 для чтения размера | | `short_string_pool.cpp` | Allocation и copy логика адаптирована для ui16 | ### Тестирование Добавлены два новых юнит-теста: - **LabelValueAtNewLimitIsAccepted:** Метрика с меткой ровно 1023 символа успешно парсится ``` "m{l=\"" + string(1023, 'a') + "\"} 1\n" ``` - **LabelValueOverNewLimitStillThrows:** Метрика с меткой ≥1024 символов выбрасывает исключение ``` "m{l=\"" + string(1024, 'a') + "\"} 1\n" // → TPrometheusDecodeException: "trying to parse too long label value, size >= 1024" ``` ### Безопасность и совместимость - ✅ Обратно совместимо: строки < 256 символов работают так же - ✅ Валидация: лимит 1024 проверяется на уровне Ref() в пуле - ✅ Граничные случаи: добавлены тесты для 1023 и 1024 символов - ✅ Выделение памяти: правильно рассчитано с учётом новых 2 байт для размера - ✅ Я не нашел ситуации когда закодированная строка (длина + строка) улетала куда-то из пула и потом пыталась раскодироваться. Так что считаю, что миграции никакой не надо и все обратно совместимо. НАДЕЮСЬ Я НЕ ОШИБАЮСЬ commit_hash:b30852954951bd8e65d69777a59ffeb58da76b47
* | Sync branches 260707-0109YDBot2026-07-0710-0/+413
|\|
| * Extract TSlotMap and use it in TFuturebabenko2026-07-065-0/+382
| | | | | | | | commit_hash:624d9e33790566421f01090a99e63155da5027f8
| * Intermediate changesrobot-piglet2026-07-065-0/+31
| | | | | | | | commit_hash:95796ec02100ee579a0afedeec45a837f553ae73
| * Revert commit rXXXXXX, Revert TCowString in TYsonString (library/cpp/yt part)babenko2026-07-0619-18/+3094
| | | | | | | | commit_hash:e6e226cb571efc3d4045f3154e7028c6cbbdef3d
| * Revert TCowString in TYsonString (library/cpp/yt part)babenko2026-07-0519-3094/+18
| | | | | | | | | | Reverts the library/cpp/yt portion of rXXXXXX. commit_hash:61c578eb480ea5a760364158808fd1f304773ae7
generated by cgit v1.3 (git 2.53.0) at 2026-07-28 04:37:52 +0000