| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
| |
commit_hash:d932ec0ffc0aaea61db064d99b47b64653cd406e
|
| |
|
|
| |
commit_hash:daa3d76bfd4283d7cda9ffd741c8ed10457a5c84
|
| |
|
|
|
| |
`TForkAwareSpinLock` implementation takes `ForkLock_` reader, which can (and does) violate the non-reentrancy.
commit_hash:6eb1092777ac21dd8303b938f855d0cd61276641
|
| |
|
|
| |
commit_hash:2476930524e6179b49fbf312f907bd03413cd8b8
|
| |
|
|
| |
commit_hash:ca1f7319b759adc9a144adc5d01ae005c364e806
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
TWriterStarvingRWLock
Previously, it was possible that `TReaderWriterSpinLock` wouldn't let the writer through if there's a steady flow of readers.
This change addresses that by:
1. Prioritizing writers inside the spinlock by adding an additional `WriterReady` flag that writers set on arrival. This flag doesn't allow any readers to come through.
2. Adding the proper tests to verify this functionality, as well as spinlock's behaviour under forks.
3. Clarifying the documentation about spinlock guarantees
4. Adding a TLA+ model, formally specifying and verifying the guarantees of the new spinlock.
5. Renaming the old lock to `TWriterStarvingRWSpinLock`, and replacing all usages inside YT with the new version (renaming all usages outside of YT to the WriterStarving version).
This is a second attempt of REVIEW: 8233768, the first one was rolled back as it lead to deadlocks in user code with reentrant reader locks:
the case of `AcquireReader(thread0) -> AcquireWriter(thread1) -> AcquireReader(thread0)` is a deadlock, as `thread0` will not be able to acquire the lock (for the second time) before `thread1` frees writer lock, and `thread1` will not be able to acquire writer lock before the reader lock will be released by `thread0`, which won't happen until `thread0` acquires the lock for the second time. See/for more context and a real example of such situation. Analogous problem can happen with fibers: this is why you shouldn't allow context switches under the lock.
Wondering why this ugly name `WriterStarvingRWSpinLock` appeared in your beautiful code? No worries, if you are **sure** that you don't use reentrant locks or fiber switches under the lock, you can freely replace your usage with the new `ReaderWriterSpinLock`. The replacement is drop-in.
[nodiff:caesar]
commit_hash:97683f854defca00cc283f5a2a10a1730b3c9174
|
| |
|
|
|
| |
It turns out, RWSpinLock previously accidentally supported recursive AcquireReaders, and this property was broken with the change.
commit_hash:f996e7b52ef8b3d37118034530a094af0efbe435
|
| |
|
|
| |
commit_hash:94fee5363799655628bd7e2c144a7869a9d89002
|
| |
|
|
|
|
|
|
|
| |
Приедет в ytsaurus: <HIDDEN_URL>
Приедет в `/contrib/ydb/` <HIDDEN_URL>
Проверка, что в github ydb ничего не сломается: <https://github.com/ydb-platform/ydb/pull/13286>
commit_hash:73ab1b4a3245322afc9fc6e9d71424ad07106477
|
| |
|
|
| |
commit_hash:afeeeea3c2cbdec0ddeed0fea5f88df9d8575f8b
|
| |
|
|
|
| |
[nodiff:caesar]
commit_hash:93eef00a7132e68e727aa1f4028616db6522d03c
|
| |
|
|
|
| |
[nodiff:caesar]
commit_hash:446e45e0378f6b2cb31d85bcc3e4516efbdfe5a7
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
deps on global variables
What happened:
1. error contents has been split into stripped_error and error. stripped_error contains the error itself (with attributes for now) and macros; error contains stripped_error and some extensions, namely, functions to get fiberId, hostname and traceid/spanid and all functions used to (de-)serialize error. This means that you cannot print error if you only include stripped_error, therefore you are likely to still require the entire error.h at the moment.
2. Mechanic for gathering origin attributes has been moved to newly created library/cpp/yt/error thus having no dependency on fibers, net or tracing. stripped_error uses these attributes as extendable semi-erased (meaning, you still would have to add a field and recompile the entire thing, but you don't have to introduce an extra dependency) storage for a bunch of attributes
3. Parsing of said attributes is done in error file (and not stripped_error).
P.S. So far the plan is to eventually move stripped_error (once dependency on core/ytree/attributes is eliminated) without any actual change to dependency graph of anything outside of core (e.g. you would still have to include misc/error.h to use it). Next step would be re-teaching the error how to print, which would move some more methods from core to the standalone module. After that one could finally depend on the error itself and not the entire core.
Annotations: [nodiff:caesar]
66615172181355821241d2e5f8e4a0f15e0ea791
|
| |
|
|
| |
4356fc70f6d9ceec9f94b1879f49e27f3fee4342
|
| |
|
|
| |
4792c7769dba84e38c0d981714741114add571d3
|
| |
|
|
| |
970c33b44a7bd166b2716d86d3d2053dcaf05d7d
|
| |
|
|
| |
Relates: https://st.yandex-team.ru/, https://st.yandex-team.ru/
|
| |
|
|
|
| |
* Library import 8
* Add contrib/libs/cxxsupp/libcxx/include/__verbose_abort
|
| |
|
|
| |
Update tools: yexport, os-yexport
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
mentioned tracking
1) Добавлена возможность инструментировать ваш любимый мьютекс/спинлок/что-либо с методами Acquire/Release со схожим смыслом. Делается это засчет методов ```NThreading::NDetail::RecordSpinlockAcquired```/```NThreading::NDetail::RecordSpinlockReleased```. Они отслеживают число захваченных спинлоков.
2) Добавлен макрос ```REGISTER_TRACKED_SPIN_LOCK_CLASS```, который позволяет номинально проверить на этапе компиляции, что данный спинлок отслеживается.
3) Добавлен метод ```NThreading::VerifyNoSpinlockAffinity```, который коркается, если число захваченных спинлоков больше нуля.
4) Добавлены Traceless версии (почти) всех гардов, чтобы можно было не отслеживать конкретную критическую секцию, если очень нужно.
5) Внутри файберного ```WaitUntilSet``` вызывается ```VerifyNoSpinlockAffinity``` -- делать ```WaitFor``` или ```Yield``` внутри отслеживаемых критических секций нельзя. Если очень хочется -- используем TracelessGuard нужного вида.
6) Теперь отслеживаются такие спинлоки и их наследники: ```TSpinLock```, ```TRecursiveSpinLock```, ```TReaderWriterSpinLock```.
7) Зарегистрированы как отслеживаемые все вышеперечисленные спинлоки и их Padded версии.
8) Все эти действия имеют эффект только в дебаг сборке. В релизе вся эта диагностика стирается.
|
| |
|
|
| |
https://clubs.at.yandex-team.ru/arcadia/29404
|
| |
|
|
| |
After this, a new target libyqlplugin.so appears. in open-source cmake build.
Diff in open-source YDB repo looks like the following: https://paste.yandex-team.ru/f302bdb4-7ef2-4362-91c7-6ca45f329264
|
| |
|
|
| |
friendly way
|
| |
|
|
|
|
|
|
|
| |
In case of heavy load and high rps current thread pool implementation seems to have problems at least with contention on lock inside condvar (long futex wait calls from http server listener thread), so try to implement something more efficient:
- replace condvar with TEventCounter implementation without internal lock (pthread condvar maintains waiters wakeup order, thread pool doesn't need it);
- introduce well-known bounded mpmc queue over ring buffer;
- get rid of TDecrementingWrapper;
- add options to turn on new pool in library/cpp/http/server and search/daemons (will remove after adoption);
- make elastic queue ut check both versions;
- workaround problems with android/arm build targets.
|
| |
|
|
| |
This reverts commit ca272f12fdd0e8d5c3e957fc87939148f1caaf72, reversing
changes made to 49f8acfc8b0b5c0071b804423bcf53fda26c7c12.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit is formed by the following script: https://paste.yandex-team.ru/6f92e4b8-efc5-4d34-948b-15ee2accd7e7/text.
This commit has zero effect on all projects that depend on YQL.
The summary of changes:
- `yql/providers/yt -> ydb/library/yql/providers/yt `- the whole implementation of YT provider is moved into YDB code base for further export as a part of YT YQL plugin shared library;
- `yql/providers/stat/{expr_nodes,uploader} -> ydb/library/yql/providers/stat/{expr_nodes,uploader}` - a small interface without implementation and the description of stat expr nodes;
- `yql/core/extract_predicate/ut -> ydb/library/yql/core/extract_predicate/ut`;
- `yql/core/{ut,ut_common} -> ydb/library/yql/core/{ut,ut_common}`;
- `yql/core` is gone;
- `yql/library/url_preprocessing -> ydb/library/yql/core/url_preprocessing`.
**NB**: all new targets inside `ydb/` are under `IF (NOT CMAKE_EXPORT)` clause which disables them from open-source cmake generation and ya make build. They will be enabled in the subsequent commits.
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
| |
Iteration no. 2. First one reverted due to YT-18997
|
| |
|
|
| |
This reverts commit 617a1d07971366c19cdf278579ee9b1cbfa53db8, reversing
changes made to 27e0312d3842c4e5e3ea6b09611c8f6ff6938dd6.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
| |
rebalancing
|
| | |
|
| |
|
| |
Нашёл локальный для Маркета фикс и вынес в код самого контриба.
|
| |
|
