| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
|
|
|
| |
iteration over elements
commit_hash:3cde1a78c5f1af49a50168bbc85c6f9a8f28ad73
|
| |
|
|
| |
commit_hash:4e76ece0b75ab787ed04b6be5f183cecbf9b8167
|
| |
|
|
|
|
|
| |
Python-клиент (обертка над С\+\+ клиентом) UA с fork support виснет в `fork()` (\>50%): `TClientSession::CheckGrpcCallInactivity` (inactivity-watchdog на потоке gRPC completion-queue) брал сессионный лок `with_lock` не проверив что не идет форк. Пока pthread_atfork-обработчик `TClient::PreFork` держит этот лок в `WaitAll()`, поллер застревает на CheckGrpcCallInactivity -\> сессия не закрывается -\> `fork()` не завершается.
Фикс: `CheckGrpcCallInactivity` захватывает лок `TryAcquire` \+ выход по `ForkInProgress` - как уже делает `Poll()`
commit_hash:79b1f727d8ecc9b54e386b586f4f49bf1d6512d8
|
| |
|
|
| |
commit_hash:ff7ca9a2428930638288f8c0e92a303b8f620063
|
| |
|
|
|
|
|
|
| |
Two scenarios addressed:
- Access to ref counters after explicit destructor of an object
- Access to vptr for upcast of an already destroyed object
commit_hash:a8f27500111817f325832d006feb3ce6c3c830f1
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Replace `TLogEvent::MessageRef` with an opaque `Payload` that carries the message
together with structured key/value tags, deferring tag rendering to the consumer
(logging thread).
- Add the `TTaggedPayloadWriter`/`TTaggedPayloadReader` codec (`NDetail`), with a
per-thread chunk-cached builder so tagged logging does no per-message heap
allocation.
- Model the payload as a typed `TLogEventPayload` — a `std::variant` of the opaque
strong typedefs `TTaggedLogEventPayload` and `TStructuredLogEventPayload` (each
over `TSharedRef`). The active alternative identifies the event kind, so the
separate `ELogMessageKind` enum and `TLogEvent::MessageKind` field are removed;
consumers dispatch via `std::holds_alternative`/`std::get` instead of a tag.
- Add the fluent `YT_TLOG_*` API: `YT_TLOG_INFO("Message").With(key, value)` and
`.With(key, value, "%spec")`; disabled levels skip argument evaluation.
- Add well-known tags: `.With(value)` attaches a value under a statically known
key resolved by ADL (e.g. `.With(error)` for the `Error` tag).
- Teach the plain-text and structured formatters to render tags; add the
`enable_native_tags` knob to emit tags into a nested structured attribute.
- Add producer-side benchmarks. The tagged API is cheaper than `YT_LOG_*` for
tagged calls and on par for tag-free ones:
#|
|| **Producer call** | **`YT_LOG_*`** | **`YT_TLOG_*`** | **Δ** ||
|| no tags | 81 ns | 82 ns | +2% ||
|| 1 tag | 124 ns | 88 ns | -29% ||
|| 2 tags | 140 ns | 101 ns | -28% ||
|| 3 tags | 372 ns | 283 ns | -24% ||
|#
--
#| || **<a href="https://nda.ya.ru/t/p0sVNSOC7ijzFF" target="_blank"> Echo tests</a>** || |#
commit_hash:70efc90e5c2b71e5311415a4e4508db42ff28971
|
| |
|
|
|
|
|
| |
Similar to `CacheDestroyed` above in this file.
It is possible that `ThreadMessageTag` is read by the logger during thread shutdown after this thread-local's destruction (e.g. when another thread-local's destructor logs). Added a trivially destructible flag to guard the tag content.
commit_hash:852afe8245de23e9abe69a08bf5d9bfe4b17496b
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
---
Type: fix
Component: library/cpp/yt/error
Problem: TError::Enricher_ and TError::FromExceptionEnricher_ are static class members whose destructors are registered when error.cpp is loaded. However, if any code calls Singleton<>() during static initialization before error.cpp loads, the OnExit handler gets registered in standard atexit() first. Due to LIFO ordering, at program exit the enrichers are destroyed before OnExit() runs, but OnExit() then destroys Singletons whose destructors may create TError objects (e.g., to cancel futures), which invokes Enrich() on the already-destroyed std::function, causing use-after-free. This can lead to intermittent segfaults depending on the link order of translation units.
Solution: Store enrichers in a LeakySingleton<TEnricherStorage> so they are never destroyed, as TError can be created anywhere including during program shutdown.
---
Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1582
commit_hash:a9607f0094b4c60414d00ebca844db6a2ceafeb9
|
| |
|
|
| |
commit_hash:8ccfa9ed373c83b84c21d12078e06befb05f026c
|
| |
|
|
|
|
|
|
|
|
| |
### `bit_io.h`
MSB-first bit-stream writer/reader (`TBitWriter` / `TBitReader`) over a caller-owned buffer. The writer flushes whole 32-bit words via the unaligned-store API; the reader assumes a few bytes of over-read slack.
### `interpolative.h`
- **Truncated-binary (minimal) code** — the entropy-optimal integer code for a uniform value in `[0, rangeSize)`.
- **Binary interpolative coding** — `InterpolativeEncode` / `InterpolativeDecode` for sorted, strictly increasing integer sequences over a known range `[lo, hi]`. It recursively codes the median of each subrange, compressing clustered sequences well below a flat `log2` per element with no per-element headers. Length is conveyed out of band (e.g. via the existing `varint`).
commit_hash:8baf84444b8cf8e8a6e32776b4ff48582187ac2b
|
| |
|
|
|
| |
Use the payload itself as the shared range holder instead of copying the string via TSharedRef::FromString.
commit_hash:b427dbe6e8a8eaf3aa3c46b57f0f9965d57b3a95
|
| |
|
|
|
|
|
|
|
| |
Rationale: all overloads of `TSharedRef::FromString` must remain cheap and don't copy any payload.
Remove the zero-terminated C string overload of `TSharedRef::FromString`. Callers passing a string literal should use `std::string/TString` explicitly.
#| || **<a href="https://nda.ya.ru/t/-VG7qyBv7iD8vm" target="_blank"> Echo tests</a>** || |#
commit_hash:1a6718abcbe4e6a8f58592f55de8c37ceb2b73d3
|
| |
|
|
| |
commit_hash:5158390863c48b4ceb69eb344ea1996573220c4b
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Adds AVX512 VNNI runtime dispatch for signed int8 dot products used by Link HNSW distance calculations, with safe fallback to the existing AVX2/SSE implementations. The signed int8 VNNI path corrects the u8*i8 bias per 64-byte chunk so accumulator semantics stay aligned with the existing int8 DotProduct implementation.
Verification:
- `ya make -tt library/cpp/dot_product/ut`
- `ya make library/cpp/dot_product/bench`
- `ya make -r library/cpp/dot_product/bench`
- `ya make -tt library/cpp/dot_product/ut --sanitize=address`
- `ya make -tt library/cpp/dot_product/ut --sanitize=undefined -F 'TDocProductTestSuite::TestDotProduct8' -F 'TDocProductTestSuite::TestDotProduct8VnniEdges' -F 'TDocProductTestSuite::TestDotProductCharStability'`
- `ya make -tt util/system/ut -F TestCpuId`
commit_hash:5624243c26df2a9e067e631477f6c3603ef45626
|
| |
|
|
| |
commit_hash:117a004505c534f300d1314155387f3e582b31c9
|
| |
|
|
|
|
|
|
| |
The returned value is the element's absolute index in the vector, not an offset within a page, so GetIndex() describes it accurately. This also aligns with LLVM's PagedVector::MaterializedIterator, which exposes getIndex() for the same concept.
- README.md updated to document GetIndex()
- new test for iterator index added
commit_hash:08a8f5f70faf0e7cef26429446886177433afa0f
|
| |
|
|
| |
commit_hash:b287a7802acecdc3d13442505be5815167cf4f71
|
| |
|
|
| |
commit_hash:07da9e448456fc129126a9b96f2fef0c40492aec
|
| |
|
|
| |
commit_hash:991ef644d607ef39801bae86415ae0190465084d
|
| |
|
|
|
| |
Make the yt and flow sources buildable in the ytsaurus-cpp-sdk export
commit_hash:a733ad0d534a3717117cd80d162f6d527843decd
|
| |
|
|
|
|
| |
- use more effective page implementation that avoid extra level of memory indirection
- it also allows emplace_back() for non-movavable non-copyable types
commit_hash:ef25b1f123742dea29a7d4fdf59a607e18a9850c
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
**Было:** Prometheus decoder ограничивает длину значений меток метрик максимум 256 символами. Попытка передать метрику с более длинным значением метки приводит к исключению `TPrometheusDecodeException`.
**Стало:** Максимальная длина значения метки расширена до 1024 символов. Системы, которые генерируют метрики с более длинными значениями (например, с ID источников данных, URL, или сложных идентификаторов), теперь могут успешно передавать такие метрики в Unified Agent без ошибок.
**Практический эффект:** Улучшение совместимости с внешними системами мониторинга и приложениями, которые требуют передачи длинных меток. Существующие системы с метками < 256 символов продолжают работать без изменений.
---
## С точки зрения разработчика
### Структурные изменения
Расширение требовало изменения способа хранения длины строки в пуле коротких строк:
1. **Prometheus decoder** (library/cpp/monlib):
- `MAX_LABEL_VALUE_LEN`: 256 → 1024
2. **Short string pool** (logbroker/unified_agent/common):
- `MaxLabelSize`: 255 → 1024
- **Ключевое изменение:** Размер строки теперь хранится как `ui16` (2 байта) вместо `unsigned char` (1 байт)
- Было: `*slot->Payload() = static_cast<unsigned char>(s.size())`
- Стало: `memcpy(slot->Payload(), &size, sizeof(ui16))`
- Это позволяет хранить строки до 65535 символов (но логически ограничено на уровне валидации)
### Изменённые файлы
| Файл | Основные изменения |
|------|------------------|
| `prometheus_decoder.cpp` | `MAX_LABEL_VALUE_LEN: 256 → 1024` |
| `prometheus_decoder_ut.cpp` | +2 юнит-теста для новых граничных значений |
| `short_string_pool.h` | `MaxLabelSize: 255 → 1024`, operator[] использует ui16 для чтения размера |
| `short_string_pool.cpp` | Allocation и copy логика адаптирована для ui16 |
### Тестирование
Добавлены два новых юнит-теста:
- **LabelValueAtNewLimitIsAccepted:** Метрика с меткой ровно 1023 символа успешно парсится
```
"m{l=\"" + string(1023, 'a') + "\"} 1\n"
```
- **LabelValueOverNewLimitStillThrows:** Метрика с меткой ≥1024 символов выбрасывает исключение
```
"m{l=\"" + string(1024, 'a') + "\"} 1\n"
// → TPrometheusDecodeException: "trying to parse too long label value, size >= 1024"
```
### Безопасность и совместимость
- ✅ Обратно совместимо: строки < 256 символов работают так же
- ✅ Валидация: лимит 1024 проверяется на уровне Ref() в пуле
- ✅ Граничные случаи: добавлены тесты для 1023 и 1024 символов
- ✅ Выделение памяти: правильно рассчитано с учётом новых 2 байт для размера
- ✅ Я не нашел ситуации когда закодированная строка (длина + строка) улетала куда-то из пула и потом пыталась раскодироваться. Так что считаю, что миграции никакой не надо и все обратно совместимо. НАДЕЮСЬ Я НЕ ОШИБАЮСЬ
commit_hash:b30852954951bd8e65d69777a59ffeb58da76b47
|
| |
|
|
| |
commit_hash:624d9e33790566421f01090a99e63155da5027f8
|
| |
|
|
| |
commit_hash:95796ec02100ee579a0afedeec45a837f553ae73
|
| |
|
|
| |
commit_hash:e6e226cb571efc3d4045f3154e7028c6cbbdef3d
|
| |
|
|
|
| |
Reverts the library/cpp/yt portion of rXXXXXX.
commit_hash:61c578eb480ea5a760364158808fd1f304773ae7
|
| |
|
|
|
| |
Migrate absl_flat_hash users to library/cpp/containers/absl
commit_hash:0db509df0f012089a56c5f500e0da1dfe9035c23
|
| |
|
|
|
|
| |
- erase() method erases element from pages first, then pushes back a new one
- make size() a bit faster
commit_hash:2dd667908f10f03c976c53002e1a9c51cb3be8f6
|
| |
|
|
| |
commit_hash:5f0c248f2c24b501b04777fb0300693b0e942c89
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Original fix merged to the nbs repo a while ago: https://github.com/ydb-platform/nbs/pull/4479
Can you bring to the upstream?
---
Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1771
Co-authored-by: Alexander Smirnov <[email protected]>
commit_hash:5bf628be8a9c7b591f290ed05754e206b23efc34
|
| |
|
|
|
| |
No behaviuor changes, just style fixes
commit_hash:0d251b393424c70c7c26c563b08d2746c756c7ad
|
| |
|
|
|
| |
Make TLogger::WithTag use a compile-time format string
commit_hash:0a119b8eb278bceb411b7d071bbefe0b41e6233d
|
| |
|
|
|
|
|
|
|
| |
refactor and fix paged vector:
- use more effective THolder instead of TSimpleSharedPtr
- fixed copy constructor and copy assignment (was shallow copy instead of deep copy)
- no more private inheritance from TVector
- add tests for copy and move constructors, copy and move assignments
commit_hash:a32c0247dd4cc5f8e29a9046c43627f4ede29044
|
| |
|
|
| |
commit_hash:f46c06c29696f6cfd01e306372ca9b430bd6b9e3
|
| |
|
|
| |
commit_hash:39506fb1534ae2d5c5dfe509c9d95a7f74c2aa18
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
**Проблема**
TCronExpression::CronNext/CronPrev бросал `library/cpp/cron_expression/cron_expression.cpp: Requested date does not exist`
для валидного выражения вроде `59 59 23 * *` (UTC) в определённые даты (например, когда now = 30 июня → следующее срабатывание должно быть 30 июля 23:59:59).
**Причина**
В DoNextPrev шаг по дню месяца определяет, изменилась ли дата, сравнивая только номер дня (value == updateValue). Однако FindDay может перевести календарь через границу месяца/года, попав при этом на тот же номер дня (30 июня → 30 июля, в
обоих случаях день 30, потому что `` — это фиксированное смещение от конца месяца). Цикл тогда считает, что ничего не изменилось, пропускает continue, который пере-резолвит секунды/минуты/часы, и оставляет поля времени, которые FindDay уже
сбросил в 00:00:00. Финальная проверка консистентности отклоняет 00:00:00 относительно second=\{59\} и бросает исключение.
Это было замаскировано, потому что существующие тесты используют \* для полей времени, где 00:00:00 — валидное совпадение.
**Фикс**
Сохраняем месяц/год до вызова FindDay и требуем, чтобы они остались неизменными (в дополнение к номеру дня), прежде чем считать день окончательным; иначе — перезапускаем цикл. Затрагивает оба направления.
commit_hash:62055eed169ba210e81c89aab860301f3e445f46
|
| |
|
|
| |
commit_hash:a5ad51e30c00aeacf1073d685849e929e3113e9a
|
| |
|
|
|
|
|
|
|
|
| |
Prepare to use external coroutine/fiber/etc pool with non-blocking read/writes instead of plain blocking system threads. We already can replace system thread pool with something else, allow to inject different streams around connection socket:
- move common part of http connection (http streams and output buffer) directly into it;
- replace connection Impl with socket streams provider;
- add TClientRequest::CreateHttpConnection factory;
- check that we are able to override socket streams in unittest.
commit_hash:afe39ce57ee1d10673f4c36a12e01b467d9f77b0
|
| |
|
|
| |
commit_hash:106e53bffa668818abf8e4003d694e1eb0a0316f
|
| |
|
|
| |
commit_hash:46268aab3c7952cc83f661c4c1d1e3c858f20a2d
|
| |
|
|
|
| |
Self-validating signature word at the head of every `TRefCounter` (`YT_ENABLE_REF_COUNTED_SIGNATURE`, on by default in debug) so a coredump walker can distinguish a live ref-counted object from freed-but-unreclaimed memory and locate the counter without virtual-base casts. Consumed by gdb_plugin (separate PR).
commit_hash:6696d8c26a8298be6543b1e3456de617d06417e9
|
| |
|
|
| |
commit_hash:8ba59395f9d9a69e5d6ebc82ab5ed0765f142f39
|
| |
|
|
| |
commit_hash:1f26f20bb31b7321a5f24277d10d6bcd814ea442
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Rename `percpu.{cpp,h}` to `per_cpu.{cpp,h}` and update all includers.
Add an rseq-backed implementation of the hot per-CPU counter, time counter and
gauge (`rseq_sensor_impl.{h,cpp}`): each update commits to the calling CPU's shard
lock-free via an rseq critical section (`library/cpp/yt/rseq`
`AddPerCpu`/`StorePerCpu`) — no atomic and no lock on the fast path. The shard
array is sized to `NRseq::GetCpuCount()` and folded into the sensor allocation
(`NewWithExtraSpace`); reads aggregate with `LoadPerCpu`. The gauge keeps
last-writer-wins semantics. Linux-only.
The existing `TPerCpu{Counter,TimeCounter,Gauge}` stay (`per_cpu_sensor_impl.{h,cpp}`)
as the atomic sharded fallback. The two are interchangeable and chosen per sensor
at construction in `TSolomonRegistry`, so the hot Increment/Update path carries no
dispatch:
* The rseq fast path is **off by default**; opt in via
`singletons/solomon_registry/enable_rseq`. Even when on, a hot sensor uses it
only in a process where the kernel rseq area sits at a fixed thread-pointer
offset (tcmalloc/glibc-owned), per the rseq library's runtime safety probe
(`NRseq::IsPerCpuFastPathSafe`). Everything else — notably a `dlopen`'d YQL UDF
whose `__rseq_abi` lands in dynamically allocated TLS — uses the atomic sharded
sensors.
* `TSolomonRegistry` is a reconfigurable singleton (`solomon_registry`) with an
`enable_rseq` knob (default false), settable in static config and updatable via
dynamic config.
Off Linux the rseq sensors do not exist and the registry uses the atomic sharded
sensors for hot requests. The per-CPU summary is unchanged (TTscp + spinlock).
Unit tests cover the atomic sensors, the rseq sensors (Linux-only), and the simple
sensors. The controller-agent memory-watchdog integration tests are made resilient
to the (core-count-dependent) per-sensor footprint.
Benchmark (hot per-CPU path, 64-core host), atomic sharded vs rseq:
| Benchmark | atomic | rseq | speedup |
| --- | --- | --- | --- |
| BM_PerCpuCounter, threads:1 | 30.7 ns | 3.6 ns | ~8.5x |
| BM_PerCpuCounter, threads:16 | 30.9 ns | 4.4 ns | ~7x |
| BM_PerCpuGauge, threads:1 | 32.8 ns | 12.4 ns | ~2.6x |
| BM_PerCpuGauge, threads:16 | 32.7 ns | 12.5 ns | ~2.5x |
commit_hash:8c633d31f03b2cc862ed2217ae08342bf42adc52
|
| |
|
|
|
|
|
|
|
| |
Some handy helpers.
`ExactRefCountedCast<T>(p)`: exact-type downcast for `New<T>()`-allocated objects. `New<T>(`) builds a final `TRefCountedWrapper<T>`, so this casts to the wrapper and upcasts back to `T*`; being final, the `dynamic_cast` lowers to a single `type_info` compare (~2ns vs ~20ns for the is-a path).
`TRef::Contains(other)`: true iff other's range lies within this range.
commit_hash:138e8719b8ecbd953437b81380e54f736db029ef
|
| |
|
|
|
| |
Добавлена поддержка HTTP-сжатия zstd в apphost http_adapter и включен эксперимент для WEB@hamster. Для этого зарегистрирован кодек zstd в общей HTTP-библиотеке, добавлен отдельный hamster-конфиг с приоритетом zstd.
commit_hash:b959243e6a6508f93bd9920c2cc445fa012c8247
|
| |
|
|
| |
commit_hash:54a07f0d411d0ece7af812a9fdf91509f25ee5cb
|
| |
|
|
|
|
|
| |
fix weak ptr
add swap test
reinterpret\_cast, precise memory\_order
commit_hash:a1b73fa28d9314d4cd21a473f421cfedcaf19330
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Hardens `library/cpp/yt/rseq` for the case where it is linked into a dlopen'd,
position-independent module (e.g. a YQL UDF `.so`). Extracted from the profiling
work that enables the rseq fast path by default.
**TLS model.** The weak `__rseq_abi` gets `global-dynamic` linkage under
`__PIC__/__PIE__` (`initial-exec` otherwise), mirroring `contrib/libs/tcmalloc`.
`initial-exec` needs a slot in the static TLS block reserved at startup, which
the loader cannot grant a module dlopen'd later — the module would fail to load
with "cannot allocate memory in static TLS block". This only changes the cold
`&__rseq_abi` accesses; the hot path still reads `*(thread_pointer + CpuIdFieldOffset)`.
**Runtime safety probe `IsPerCpuFastPathSafe()`.** The cached thread-pointer
offset is valid only when `__rseq_abi` sits at a fixed offset from the thread
pointer — a glibc-owned area or the static TLS block (incl. tcmalloc), the common
case. When our `__rseq_abi` instead lands in a dlopen'd module's *dynamically
allocated* TLS, the offset is valid only on the thread that computed it; on other
threads the hot path's first store (`area->rseq_cs`) would corrupt unrelated
memory. The probe spawns one thread and checks — by pointer comparison, never
dereferencing the suspect offset — that the offset names that thread's rseq area;
if not, callers use the atomic fallback. Decided once and cached (one thread spawn
at first use).=
commit_hash:633f58f500d9d097800da81f526c56283445ffc7
|
| |
|
|
| |
commit_hash:bfab0d0115b50949f66878004cf718b988575734
|