| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
TTscp::GetApproximate takes the processor id from the rseq fast path
(GetCurrentCpuId) and the instant from a non-serializing rdtsc, instead of the
single serializing rdtscp of TTscp::Get.
TPerCpuGauge::Update switches to it: the per-shard timestamp only orders writes
across shards to pick the freshest value, so the lower precision is fine. Update
is virtual and now YT_PREVENT_TLS_CACHING -- the fiber-TLS boundary the inlined
rseq read needs.
#### Benchmark
sas2-2769 (glibc 2.31 + tcmalloc, rseq fast path), median of 5:
| primitive | time |
|---|---|
| TTscp::Get() (rdtscp) | 14.1 ns |
| TTscp::GetApproximate() (rseq + rdtsc) | 10.6 ns (-25%) |
commit_hash:b277b6551accd6d0b879f8ffb168bcbe8d9fbb74
|
| |
|
|
|
| |
Make library/cpp/yt/rseq a Linux-only dependency of library/cpp/yt/system
commit_hash:7d6f5e738658447529440425b55b2891f6664d81
|
| |
|
|
| |
commit_hash:33721d8fd9919cec2c217db529145c881baf144b
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Self-contained current-CPU-id reader backed by Linux **rseq** (restartable
sequences), with **no third-party dependency** (no librseq):
* The rseq ABI is hand-defined; the calling thread is registered lazily via the
rseq syscall.
* Fast path is a single inlined, **branch-free** thread-local read. The offset
always points at a readable `cpu_id` -- the glibc-owned area when glibc registers
rseq (>= 2.35, via the weak `__rseq_offset`/`__rseq_size`), otherwise our own
area -- so an unregistered thread reads `-1` and routes to the slow path.
* Falls back to `sched_getcpu()` (Linux) or `0` (darwin/windows). Works on glibc
**and musl** alike (librseq does not build on musl).
Fiber-TLS contract: the inlined read must be reached only via a non-inlinable,
fiber-switch-free frame (a virtual call or `YT_PREVENT_TLS_CACHING`).
#### Benchmark -- cost of one cpu-id read
| source | time / call |
|---|---|
| `GetCurrentCpuId()` (rseq) | **0.34 ns** |
| `sched_getcpu()` (vDSO) | 3.5 ns |
| `rdtscp` (what `TTscp::Get()` does) | 23 ns |
This is an alternative to the librseq-based review/13886037 -- same speed, but no
contrib dependency and it also covers musl. The unit test pins to each allowed CPU
and asserts the reported id matches.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
commit_hash:09d282c2f48755836b1cd68cedbffc3c6a662eed
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
## Motivation
Profiling the YT master Automaton thread showed TOriginAttributes::Capture (run on every non-OK TError) spending ~60% of its time in a getpid() syscall — uncached on glibc >= 2.25. NYT::GetCurrentThreadId() (gettid) feeds hot thread-affinity / log-manager checks on the same thread.
## Changes
- New library/cpp/yt/system/process_id.* with cached GetProcessId(); GetSystemThreadId() now caches the kernel tid in TLS. Both caches reset in the child after fork.
- Moved thread_name.{h,cpp} from misc to system.
- Removed GetCurrentProcessId/GetCurrentThreadId shims from yt/yt/core/misc/proc.{h,cpp}; migrated all callers to NYT::GetProcessId / NYT::GetSystemThreadId.
- TOriginAttributes::Capture uses the cached getters; recorded Tid is now the real kernel tid (matches perf/ps).
- Added microbenchmarks (library/cpp/yt/system/benchmarks, yt/yt/core/benchmarks/error.cpp).
## Microbenchmarks (release)
| | before | after |
|---|---|---|
| getpid | 101 ns | 0.33 ns |
| gettid | 102 ns | 1.64 ns |
| Capture | 161 ns | 50 ns |
| failed TError | 221 ns | 74 ns |
commit_hash:ee37ae57d61a5a2dd33daee935270f4bb93b7ff9
|
| |
|
|
|
| |
Add a templated GetEnvValueOrThrow<T> that parses the environment variable value via FromString, next to the existing std::string overload.
commit_hash:0421b0463c473c8c7f88c0b1619484e52b002897
|
| |
|
|
| |
commit_hash:5a11a7288f325d6fa0a9bec9a1ce0b7afa1f4984
|
| |
|
|
| |
commit_hash:72977e13820ded4a38d19076b1d336c9bb910dbf
|
| |
|
|
| |
commit_hash:a4ecfb1e06e82ba192646d3ab5417cb3f0c6991f
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
becoming CPO, move TErrorCode and TError to library/cpp/yt/error
\[nodiff:caesar\]
List of changes:
1) Make `ConvertTo` a normal function again (basically, a revert of previous change). It turned out to be very inconvenient entry point
2) Introduce concept for "Primitive types" which are handled without yt/core module. Please note that inclusion of core **does not** affect work with primitive types (otherwise we would be facing and ODR violation). These are numerics, duration, instant, string-like objects and guid
3) Introduce `FromErrorAttributeValue` which is a substitute to `ConvertTo` with special behavior (see below)
4) Scheme for (de-)serialization has changed:
1) Primitive types are handled via `ConvertToTextYsonString` and `ConvertFromTextYsonString`
2) Other types are not supported without yt/core. Otherwise `ConvertToYsonString(value, EYsonFormat::Text)` used for serialization and `ConvertTo<T>(value)` for deserialization.
5) New format of attribute value storage allows to not care about concrete type of the attribute (text yson strings can be detected since they start with `\"` and deserialized so are text yson bools, everything else is already human readable). This drops dependency on `TTokenizer`
6) Some small parts (e.g. enums or constants or aliases) of yt/core/misc files were moved to library/cpp/yt/\* locations and re-included to drop dependencies of stripped\_error on them
7) `TErrorAttributes` is now a simple hash map instead of a handle to `IAttributeDictionary`
8) `ExtractFromAttributes` weak symbol is finally moved to library/cpp/yt/error due to point 9 allowing to fully drop dependency on `IAttributeDictionary`. Combined with point 7 this drops dep on yt/core/ytree altogether
9) Moved `TErrorCode` to library/cpp/yt/error. There is a logger there which forces dep on library/cpp/yt/logging. It is not required really, and can be later removed
10) Moved `TError` with format and macroes to library/cpp/yt/error. It still (and probably forever will) depend on yson.
What can be done next: Switch TYsonString to TString and move ConvertTo/FromTextYsonString to library/cpp/yt/error. This would remove dep on library/cpp/yt/yson\_string
commit_hash:6f11dc478ab782a1f98a5aedcd45a4800d3f4e7b
|
| |
|
|
| |
commit_hash:7bbc13afbd569ca9bb064aed656c12644fd2de96
|
| |
|
|
| |
ec1d0c473e709e8aad76b4e190a75ad4125a9ce2
|
| |
|
|
| |
970c33b44a7bd166b2716d86d3d2053dcaf05d7d
|
| |
|
|
| |
Relates: https://st.yandex-team.ru/, https://st.yandex-team.ru/
|
| | |
|
| |
|
|
| |
Update tools: yexport, os-yexport
|
| | |
|
| |
|
|
| |
After this, a new target libyqlplugin.so appears. in open-source cmake build.
Diff in open-source YDB repo looks like the following: https://paste.yandex-team.ru/f302bdb4-7ef2-4362-91c7-6ca45f329264
|
| |
|
|
| |
friendly way
|
| |
|
|
|
|
|
|
|
| |
In case of heavy load and high rps current thread pool implementation seems to have problems at least with contention on lock inside condvar (long futex wait calls from http server listener thread), so try to implement something more efficient:
- replace condvar with TEventCounter implementation without internal lock (pthread condvar maintains waiters wakeup order, thread pool doesn't need it);
- introduce well-known bounded mpmc queue over ring buffer;
- get rid of TDecrementingWrapper;
- add options to turn on new pool in library/cpp/http/server and search/daemons (will remove after adoption);
- make elastic queue ut check both versions;
- workaround problems with android/arm build targets.
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
