diff options
author | h0pless <h0pless@yandex-team.com> | 2024-12-18 06:25:39 +0300 |
---|---|---|
committer | h0pless <h0pless@yandex-team.com> | 2024-12-18 06:39:36 +0300 |
commit | ea1ee8bc4725f339434fd1d48cf0d6f56129f711 (patch) | |
tree | a1f280a500991ec3da52c7b4529f5f5c619f060b /contrib/restricted/boost/math | |
parent | 5b49bf7290defc0b7dd86d96043b3d6f67b57b05 (diff) | |
download | ydb-ea1ee8bc4725f339434fd1d48cf0d6f56129f711.tar.gz |
Introduce new pipeline for cross-cell copy
New cross-cell copy protocol:
1. LockCopyDestination - Early check that copy won’t fail because of conflicts, bad flag usage, etc. On this stage we learn to which cell the subtree is being copied to, inherited attributes of the destination and some other minor things.
2. LockCopySource - Lock every node with an appropriate lock on source cell, and return a map, describing topology of a subtree. (Amounts to a hash map parent_id -\> child_ids). During this step we also can find all schemas and cell tags that are used by the source.
3. SerializeNodes - Get binary representation of each node and send it to the client side. Additionally send information about schema id and external cell tags. The former is used to deduplicate schemas and the latter will be used during sequoia copy.
4. CalculateInheritedAttributes - Unfortunately for us, we have inheritable attributes. They behave differently for copy and create right now, because copy always preserves the current value and create calculates it anew. We plan to unify said behavior later by always re-evaluating the value of an attribute on destination cell. For the compatibility.
5. MaterializeCopyPrerequisites - This step is responsible for schema materialization. Later this should also include, for example, secondary indices.
6. MaterializeNodes - Create nodes from binary representation, but keep them unattached.
7. AssembleTreeCopy - Assemble nodes, created in step 6 into a tree on destination cell and attach the resulting tree to the destination node (or it's parent, in case of replace).
List of notable changes:
1. Deprecated “BeginCopy” and “EndCopy” verbs.
2. Deprecated “Internalize” command.
3. Added a code that should support new inherited attributes addition. Currently mostly inactive (only chunk_merger_mode is supported).
4. Opaqueness does not interfere with cross-cell copy now.
5. Now max subtree that can be copied with a single command is limited to 100’000 by default (this limit can be changed in master dynamic config)
6. Inplace copy only used for externalization of a map node, never during copy. With this change a decent chunk of dead code has been removed, hopefully making the overall algorithm more easily understandable.
7. Removed “inheritedAttributes“ and “factory“ from DoEndCopy.
List of notable bugfixes:
1. When attempting to cross-cell copy under a transaction and traversing a source subtree, most metadata was taken from the trunk version of the node instead of the branch. This results in some attributes being missing on destination if they were set under a transaction.
2. Fixed an issue where the problem from the above could be replicated for inheritable attributes.
3. Fixed a few tests that amounted to “assert True“.
### Changelog entry
type: feature
component: master
Introduced new pipeline for cross-cell copy
commit_hash:99cb45639f5ae55f431f5711269badb000e814a1
Diffstat (limited to 'contrib/restricted/boost/math')
0 files changed, 0 insertions, 0 deletions