diff options
| author | Grigory Reznikov <[email protected]> | 2024-03-19 13:38:52 +0300 |
|---|---|---|
| committer | robot-piglet <[email protected]> | 2024-03-19 14:16:23 +0300 |
| commit | 51c750cf25c79121b833e2a7882b10e8cd371bef (patch) | |
| tree | c4e54afd7028d5f1b9859ecb54b287994cc6eee7 /yt/cpp/mapreduce/io/proto_table_reader.cpp | |
| parent | 7bfb3ae386d49f6392b1e4fcc8b6f1c43940f27c (diff) | |
Support building yt/cpp and yt/yt/core with vanilla protobuf
After this PR yt/cpp and yt/yt/core are possible to be built both with Arcadia protobuf (that uses TString as a string) and vanilla protobuf (that uses std::string as a string). To achieve so, a couple of interoperability primitives are introduced.
* `TProtobufString` is an alias to protobuf string type, i.e. it can be `TString` or `std::string` depending on the protobuf implementation.
* `IsVanillaProtobuf` and `IsArcadiaProtobuf` are the constexpr boolean values that allow to check protobuf implementation both in the compile time and runtime.
The most challenging interoperability issue solved here is a string copy between protobuf message and C++ code that has a form of `TString str = msg.str()`. This code works perfect with Arcadia protobuf but does not work with vanilla protobuf. To solve it, a previously introduced primitive `FromProto<TString>` is used. This expression makes the most efficient cast possible between protobuf string and C++ string. Internally, it is just a copy in both cases. Since TString is CoW by default, this expression is almost zero-cost (actually it's just one atomic operation), so no degradation is expected for YTsaurus server builds. The most hot code is handled differently to avoid even atomic operations (see `GetRequestTargetYPath`). In case of vanilla protobuf string is copied, however there are no places in C++ SDK where it might be a problem. If such issues would appear, performance-critial code can be rewritten in `GetRequestTargetYPath`-style.
---
1a6f3e02cb6e83915102c24b73bc8734f6a48e74
Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/466
Diffstat (limited to 'yt/cpp/mapreduce/io/proto_table_reader.cpp')
| -rw-r--r-- | yt/cpp/mapreduce/io/proto_table_reader.cpp | 17 |
1 files changed, 11 insertions, 6 deletions
diff --git a/yt/cpp/mapreduce/io/proto_table_reader.cpp b/yt/cpp/mapreduce/io/proto_table_reader.cpp index 555c5345a7d..c0bf90d2536 100644 --- a/yt/cpp/mapreduce/io/proto_table_reader.cpp +++ b/yt/cpp/mapreduce/io/proto_table_reader.cpp @@ -4,6 +4,8 @@ #include "proto_helpers.h" +#include <yt/yt/core/misc/protobuf_helpers.h> + #include <yt/yt_proto/yt/formats/extension.pb.h> #include <util/string/escape.h> @@ -15,16 +17,19 @@ using ::google::protobuf::Descriptor; using ::google::protobuf::FieldDescriptor; using ::google::protobuf::EnumValueDescriptor; -const TString& GetFieldColumnName(const FieldDescriptor* fieldDesc) { - const auto& columnName = fieldDesc->options().GetExtension(column_name); +using NYT::FromProto; + +TString GetFieldColumnName(const FieldDescriptor* fieldDesc) +{ + auto columnName = FromProto<TString>(fieldDesc->options().GetExtension(column_name)); if (!columnName.empty()) { return columnName; } - const auto& keyColumnName = fieldDesc->options().GetExtension(key_column_name); + auto keyColumnName = FromProto<TString>(fieldDesc->options().GetExtension(key_column_name)); if (!keyColumnName.empty()) { return keyColumnName; } - return fieldDesc->name(); + return FromProto<TString>(fieldDesc->name()); } void ReadMessageFromNode(const TNode& node, Message* row) @@ -48,10 +53,10 @@ void ReadMessageFromNode(const TNode& node, Message* row) continue; // null field } - auto checkType = [&columnName] (TNode::EType expected, TNode::EType actual) { + auto checkType = [fieldDesc] (TNode::EType expected, TNode::EType actual) { if (expected != actual) { ythrow TNode::TTypeError() << "expected node type " << expected - << ", actual " << actual << " for node " << columnName.data(); + << ", actual " << actual << " for node " << GetFieldColumnName(fieldDesc); } }; |
