summaryrefslogtreecommitdiffstats
path: root/yt/yt_proto
Commit message (Collapse)AuthorAgeFilesLines
* YT-27780: Make push-based shuffle configurable via start_shuffleapollo132135 hours1-0/+3
| | | | commit_hash:109d7b77afce712c68e57058c5f38e52eade292e
* YT-27780: Use TTableSchema instead of TTableSchemaExt in proxy APIapollo13217 days1-1/+1
| | | | commit_hash:2a15f45c98aa40190e39522a2155d9ddcf83263b
* YT-27780: Add push based shuffle fields in shuffle service APIapollo13218 days1-0/+5
| | | | commit_hash:bbd5ab6a8913a7d713da2082152a643532831810
* YT-27508: Implement uniform prepare transaction signature generator for ↵atalmenev2026-05-221-0/+2
| | | | | | | | | | | | tablet writes * Changelog entry Type: feature Component: dynamic-tables All prepare signatures are now equal to 1. This enables re-assembling commits across cell commit sessions on write retry, e.g. when a tablet moves. commit_hash:40ce0e6541a09888417f5b84b25012b796c4bfdd
* YT-27807: Parallel GROUP BY on coordinatorsabdenovch2026-05-191-0/+1
| | | | | | | | | | | Also affects all schemaful pipes - backpressure and stream fragmentation are implemented. * Changelog entry Type: feature Component: dynamic-tables Implemented parallel group by on proxy, significantly improving latency of high-cardinality grouping queries. commit_hash:29af1e557205b21e22268a1fa333172a9d1fc681
* YT-25951: Introduce the OffshoreDataGateway componentPavel Bashkirov2026-05-151-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Changelog entry Type: feature Component: offshore data gateway This PR adds the `OffshoreDataGateway` component which is used when reading data from S3. This component is also integrated into the replication reader to actually perform the reads. One of the most important topics to note in this work is the introduction of the new proto message `TChunkReplicaSpec` and its usage in the `OffshoreDataGateway`'s RPC requests. This structure allows us to pass the medium index which is required to find the specific offshore medium where the chunk is located. Maybe there are other solutions, I'll be happy to discuss that. Another topic is the approach to handling different media in the replication reader. Right now the replication reader uses only addresses to uniquely identify different peers to read from. This works because we read from data nodes only, and we do not care about the medium there. Now the reads may also happen from `OffshoreDataGateway`-s, and the medium starts to matter. Imagine a scenario when a chunk has two offshore replicas on two different mediums, one pointing to Google S3 storage, another to AWS S3. We must have a way to differentiate those two replicas, even though the address is the same - the sentinel `OffshoreNodeAddress`. This is why I introduce a structure called `TPeerId` which includes both the address and the medium index, and now replication reader works with it to differentiate different replicas. Last topic is the testing. It's impossible to implement an "honest" integration test at the moment as writes to offshore replicas are not implemented, and also masters know nothing of them. I have implemented a C\+\+ unit test to check the behaviour of replication reader only - `test_s3_data.cpp`. --- Pull Request resolved: <https://github.com/ytsaurus/ytsaurus/pull/1688> Co-authored-by: cherepashka <[email protected]> Co-authored-by: cherepashka <[email protected]> Co-authored-by: cherepashka <[email protected]> Co-authored-by: cherepashka <[email protected]> Co-authored-by: cherepashka <[email protected]> Co-authored-by: cherepashka <[email protected]> Co-authored-by: cherepashka <[email protected]> Co-authored-by: cherepashka <[email protected]> Co-authored-by: cherepashka <[email protected]> Co-authored-by: cherepashka <[email protected]> Co-authored-by: cherepashka <[email protected]> Co-authored-by: cherepashka <[email protected]> Co-authored-by: cherepashka <[email protected]> Co-authored-by: cherepashka <[email protected]> commit_hash:7941b82f5735c5788fefec1ccf5175ddd86528a5
* YT-27651: Fetch actual node tags and dynamic config on bundle controller requestnavasardianna2026-05-051-0/+1
| | | | commit_hash:8d9c3b0bbddd5c1c725cc3c1eefddcc6498f936a
* YT-27848: fetch node descriptors in partitionsachains2026-04-221-0/+1
| | | | | | | | | * Changelog entry Type: feature Component: proxy Introduce "FetchCookieNodeDescriptors" flag to serialize node directory for partition inside cookie. It should prevent unresolved node id errors during ReadTablePartition call. commit_hash:fbdda26266e4efec8f1202c5ccdcd98b905db928
* Revert commit YT: Make NYT.NProto.TError.code field optionaldgolear2026-04-141-1/+1
| | | | | <https://nda.ya.ru/t/lADBX2ru7ZPHnh> commit_hash:3c379194a4882d7ed93f922d5bf5ea25e6ee0ddd
* [dynamic] fix YT-27866: write, delete, lookup rows with allowMissingKeyColumnspanesher2026-04-091-0/+6
| | | | | | | | * Changelog entry Type: fix Component: dynamic-tables Write, Delete, Lookup Rows allowed missing keys if allowMissingKeyColumns flag is set commit_hash:4f90a0f906fab5ad43914a34c4737f1f78986a3e
* YT-27651: Speed up offline node detection with out of band applying dynamic ↵navasardianna2026-04-021-0/+1
| | | | | | node config commit_hash:9fb73ada8d5eb337a1f9fd0962b589e7ff21fe76
* YT-27348: Support compressed data size in PartitionTablespavook2026-04-011-1/+2
| | | | | | | | | * Changelog entry Type: feature Component: api Support compressed data size in PartitionTables commit_hash:ff707e5628ee3096ac17f0a3f04cbc1a92916996
* YT-9652: Fix naming: rename read_size_estimation to read_size_estimate in ↵apollo13212026-03-281-1/+1
| | | | | | | | columnar statistics 'estimation' refers to the process; 'estimate' is the result value. Rename the proto field and output key accordingly. commit_hash:9654e9efe2b2d73f49b4ce22b0aa60dbff5eeece
* YT-26329: Drop some cosmetics todos: move several functions from ↵cherepashka2026-03-181-3/+2
| | | | | | cell_directory.cpp into helpers.cpp commit_hash:ae257311dc7104cf31e35dd26c1c41ddfda4353d
* YT-27525: Add heartbeats from nodes to bundle controllernavasardianna2026-03-111-0/+12
| | | | commit_hash:8ed3cffde90496b2de8440d5b59674301761de28
* YT: Make NYT.NProto.TError.code field optionaldgolear2026-03-011-1/+1
| | | | | | | | | | | * Changelog entry Type: fix Component: misc-api Switch code field from required to optional. <Message for release notes> commit_hash:e86cf4ef3843f8f0bbd36267dd1a0298f8b203b7
* YT-26489: Worker pool in rpc proxy is fair-sharesabdenovch2026-02-021-0/+17
| | | | | | | | | * Changelog entry Type: feature Component: proxy Worker pool in rpc proxies is made fair-share. commit_hash:9ce485656813b6821badbf1bc855ce79809b7206
* YT-27097: Added location index into confirm chunkcherepashka2026-01-311-0/+2
| | | | | | | | | * Changelog entry Type: feature Component: master & node Added location index into chunk confirmation commit_hash:1f0ba59fdbe8f25bf80086b312d8df17e6a3f464
* YT-27085: Rename shadowed data member Timeout to SessionTimeoutachains2026-01-301-2/+2
| | | | | | | | | | | * Changelog entry Type: fix Component: proxy Rename shadowed data member NYT::TStartDistributedWriteTable(File)Options::Timeout -\> SessionTimeout <Message for release notes> commit_hash:1b5b967b5136a7105627854044de6eee634dde03
* YT-26877: Allow unfolding columns in table and index to have different namessabdenovch2026-01-281-0/+7
| | | | | | | | | * Changelog entry Type: feature Component: dynamic-tables Unfolded columns in unfolding indices are allowed to have different names in table and in index. commit_hash:e4761712a3b4cd69f215f69ea496118b9eff6990
* YT-26468: Make masters reliable explicitly on cell additionaleksandra-zh2026-01-271-0/+10
| | | | commit_hash:e8801b81ac480cb7b5c883c912d4ec2d6f3c74d7
* YT-26685: Made masters to return secondary masters configs as a response to ↵cherepashka2026-01-271-0/+1
| | | | | | | | | | | cluster node registration and heartbeat * Changelog entry Type: feature Component: master & node Made masters to return secondary masters configs as a response to cluster node registration and heartbeat commit_hash:5136cda005902249cf2feafcf205e289f0ee1235
* Revert commit rXXXXXX, YT-26489: Worker pool in rpc proxy is fair-sharesabdenovch2026-01-201-17/+0
| | | | commit_hash:8cfcaa94f1147884e6d5c2a91e2728889fc906cd
* Fix babenko-issuesdann2392026-01-191-3/+3
| | | | commit_hash:14d1302c7ee0b288f4f5f1625abcb0e2da0b59d1
* YT-26489: Worker pool in rpc proxy is fair-sharesabdenovch2026-01-161-0/+17
| | | | | | | | | * Changelog entry Type: feature Component: proxy Worker pool in rpc proxies is made fair-share. commit_hash:7e3b72360ee72775a46b7812ea54cfa1ca3c703e
* YT-18763: Add validation of dynamic table constraints for row and value sizesatalmenev2026-01-131-0/+3
| | | | commit_hash:217465a241bdd9018ca30954e645ffa9917ffd6c
* YT-26675: Suppress strongly ordered transaction barrier in ObjectServicedanilalexeev2026-01-131-0/+1
| | | | | | * Do not wait for the prepared transaction barrier on `yt <command> --params '{suppress_strongly_ordered_transaction_barrier=%true}'`. * Make HTTP & Cypress Proxies registration commands ignore the barrier. commit_hash:52ffad5a6418fd9d2cb1af763b33fc1b19f2a648
* YT-27002: Unify override_* semantics in chunk spec descriptorsapollo13212025-12-291-0/+3
| | | | commit_hash:aa3a56ee71eec7893d8e7986d35eed908a083f5f
* Rename distributed job options againpogorelov2025-12-221-3/+3
| | | | commit_hash:0646bf50b5be7e93f78eb08ed151859116a8a7c3
* YT-26525: Introduce run-job-shell-commandbystrovserg2025-12-221-0/+13
| | | | | | | | | * Changelog entry Type: feature Component: scheduler Introduce run-job-shell-command: API to run command in job shell and receive its output. commit_hash:1c0204a7548195d945b0915e5ff734ea48438532
* YT-24491, YT-26945: Allow user-defined precision in HLLdtorilov2025-12-191-0/+2
| | | | commit_hash:71a0cdcaf571a4ed247dfe29e9a5f0850a2a9c5a
* YT-9652: Support read size estimation in get table columnar statistics RPC ↵apollo13212025-12-171-0/+3
| | | | | | | | | | | | | | | method <Message NOT for release notes> * Changelog entry Type: feature Component: proxy This change adds read size estimation support to the GetColumnarStatistics RPC method. Read size estimation provides a good approximation of compressed data size for column groups when chunks have the same column groups as the table schema. <Message for release notes> commit_hash:43c2dfda01b2b274f24beb43f1bef7576c87f276
* YT-17749: Introduce transfer_bundle_resources commandifsmirnov2025-12-121-0/+15
| | | | | | | | | | * Changelog entry Type: feature Component: dynamic-tables Introduce transfer_bundle_resources command. It allows to transfer resources between two tablet cell bundle atomically. commit_hash:14ac71963fe8594c4c4d7f200812ca06fcb2a0b6
* YT-21148 Master-side support for struct field renaming and removals-berdnikov2025-12-111-0/+8
| | | | | | | | | | | | | | * Changelog entry Type: feature Component: master cpp-sdk 1. Optionally allow renaming and removing fields of structs and variant structs. 2. Provide separate options for static and dynamic tables. 3. Add unit- and integration tests. 4. Support in chunk readers will be implemented separately. commit_hash:3d5d4412381b5b35f90ba81488836f9f40a04ff7
* YT-25450: Do not use full text search by defaultmpereskokova2025-12-051-1/+1
| | | | commit_hash:c1cf6352dd53436db248717a38c82f686a3f600c
* YT-24418: Add constrained table schemacherepashka2025-12-043-1/+19
| | | | | | | | | | | | | | | | | * Changelog entry Type: feature Component: master Introduce constraints for columns in table node schema on master side API: Create and Alter Table commands will have 2 more parameters: `constrained_schema` and `constraints`. The first parameter represents usual schema with columns marked by constraints, for example: `[{"constraint": "BETWEEN 0 AND 60", "name": "minute", "stable_name": "_minute", "type": "int64", "sort_order": "ascending"}, {"constraint": "BETWEEN 0 AND 23", "name": "hour", "type": "int64"}]`. The second parameter represents mapping between name of column into constraint, for example: `{"minute": "BETWEEN 0 AND 60", "hour": "BETWEEN 0 AND 23"}`. Addition of `constrained_schema` allows us to save the old behavior with altering tables and avoid inadvertent removals of constraints which would inevitably arise from mixing up new (constained) and old (unconstrained) schemas. Addition of `constraints` parameter will allow users to add constraints even if they didn't want to change schema of table. Tables will have 2 more attributes: `@constrained_schema` and `@constraints`. They are just similar to new parameters in create and alter-tables commands. Deduplication of schemas will work as before: schemas with no constraints will be deduplicated by `@schema_id`. Constraints from schemas will not be deduplicated yet (by `@constraints_id`), this feature will be supported in future PRs. commit_hash:eda85e01afc0e89398196327c5b5de7f334e96c0
* Fix http->rpc sessguard passthroughse4min2025-12-031-0/+1
| | | | commit_hash:cfd8cb99e2ef5ffe758f397dbb2180cfbb795222
* YT-26321: Introduce check-operation-permissionbystrovserg2025-11-271-0/+22
| | | | | | | | | * Changelog entry Type: feature Component: scheduler Introduce check\_operation\_permission: API to verify user's permission to operation. commit_hash:5dd48763fd15e5a912351f959f8d120e6b3f140e
* YT-26594 Introduce a minimal gRPC service in job_proxydann2392025-11-252-0/+13
| | | | commit_hash:e0f1278fe0e37eca9ebbd11456f617c3665b4b32
* YT-26363: Introduce list-job-tracesbystrovserg2025-11-231-0/+57
| | | | | | | | * Changelog entry Type: feature Component: scheduler Introduce list-job-traces: API to receive info about job's GPU traces commit_hash:0587ef97d31238f84bb3b1f69dc17e1f202610a0
* Rename distributed job optionpogorelov2025-11-191-3/+3
| | | | commit_hash:c2fc4993b3226aacc10fee87fef584f8a5360a88
* Intermediate changesrobot-piglet2025-11-181-0/+2
| | | | commit_hash:d5a8edf11a5436650f288d4cd884480e778a6387
* Compaction timestamp digest fix for min_data_versions=1tem-shett2025-10-221-0/+1
| | | | | | | | | * Changelog entry Type: fix Component: dynamic-tables Compaction timestamp digest fix for min\_data\_versions=1. commit_hash:9bd762943f60d9e82be30cb698b574fd45e79049
* YT-26108: Distributed file writeachains2025-10-211-0/+48
| | | | | | | | | * Changelog entry Type: feature Component: proxy Introduce distributed write protocol for files commit_hash:6e90fa9dc228b4b1b9fecc19473152f851c39ce1
* YT-24624: Rework get_job_trace apibystrovserg2025-10-171-21/+18
| | | | | | | | | | | <Message NOT for release notes> * Changelog entry Type: fix Component: scheduler Rework get\_job\_trace api: enable streaming and return perfetto-friendly format commit_hash:082e255afb9022eb137dac3d0c3be8ac49d3b5e2
* YT-26389: Support RLS in read_table_partitioncoteeq2025-10-151-0/+2
| | | | commit_hash:7993a8aa42588e32d89c7620cfdbce19bb5e4cde
* Wait for strongly ordered transactions when entering read-only modealeksandra-zh2025-10-131-0/+1
| | | | commit_hash:0a2dda0b5dc87a8fccae9081a099d1fc2f1d06e1
* YT-26380: Add option to set clip timestamp during alteralexelexa2025-10-081-0/+1
| | | | commit_hash:0c4facfeb34aaab4f733073b297329fa923702a4
* YT-26180: Add optimization level optiondtorilov2025-10-071-0/+1
| | | | commit_hash:db18c78c543df7c113c96252143bfd877f57e215
* Support distributed jobsNikita Sokolov2025-10-071-0/+4
| | | | | | | | | | | | | | | | | | | | | | Makes it possible to run map operation job groups consisting of several containers, distinguishing one from another using `YT_JOB_COOKIE_GROUP_INDEX` env-variable. Only the main job (`YT_JOB_COOKIE_GROUP_INDEX == 0`) receives the input and output streams. Group elements can be queried using the `main_job_id` filter. `YT_JOB_COOKIE_MAIN_JOB_ID` lets the jobs find the main job. https://docs.google.com/document/d/13eXf7i2ylDybhasRoocZlgdpOk0ttCXjOKIC5xGHGdc/edit?tab=t.0#heading=h.8kt1zjdp11cm --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1149 Co-authored-by: pogorelov <[email protected]> Co-authored-by: pogorelov <[email protected]> Co-authored-by: pogorelov <[email protected]> Co-authored-by: pogorelov <[email protected]> Co-authored-by: pogorelov <[email protected]> Co-authored-by: pogorelov <[email protected]> Co-authored-by: pogorelov <[email protected]> Co-authored-by: pogorelov <[email protected]> commit_hash:edf43e210f9db0974047bba3084777cc07d51e89
generated by cgit v1.3 (git 2.53.0) at 2026-06-09 01:50:16 +0000