diff options
author | Sergey Polovko <sergey@polovko.me> | 2022-02-10 16:47:03 +0300 |
---|---|---|
committer | Daniil Cherednik <dcherednik@yandex-team.ru> | 2022-02-10 16:47:03 +0300 |
commit | 2e714b5ebd40a1f4cc31c27f1ad6e49ca6d895f5 (patch) | |
tree | b83306b6e37edeea782e9eed673d89286c4fef35 | |
parent | 3e0b762a82514bac89c1dd6ea7211e381d8aa248 (diff) | |
download | ydb-2e714b5ebd40a1f4cc31c27f1ad6e49ca6d895f5.tar.gz |
Restoring authorship annotation for Sergey Polovko <sergey@polovko.me>. Commit 2 of 2.
1721 files changed, 34232 insertions, 34232 deletions
diff --git a/contrib/libs/hdr_histogram/COPYING.txt b/contrib/libs/hdr_histogram/COPYING.txt index 32ae9cf3ff3..0e259d42c99 100644 --- a/contrib/libs/hdr_histogram/COPYING.txt +++ b/contrib/libs/hdr_histogram/COPYING.txt @@ -1,121 +1,121 @@ -Creative Commons Legal Code - -CC0 1.0 Universal - - CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE - LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN - ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS - INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES - REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS - PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM - THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED - HEREUNDER. - -Statement of Purpose - -The laws of most jurisdictions throughout the world automatically confer -exclusive Copyright and Related Rights (defined below) upon the creator -and subsequent owner(s) (each and all, an "owner") of an original work of -authorship and/or a database (each, a "Work"). - -Certain owners wish to permanently relinquish those rights to a Work for -the purpose of contributing to a commons of creative, cultural and -scientific works ("Commons") that the public can reliably and without fear -of later claims of infringement build upon, modify, incorporate in other -works, reuse and redistribute as freely as possible in any form whatsoever -and for any purposes, including without limitation commercial purposes. -These owners may contribute to the Commons to promote the ideal of a free -culture and the further production of creative, cultural and scientific -works, or to gain reputation or greater distribution for their Work in -part through the use and efforts of others. - -For these and/or other purposes and motivations, and without any -expectation of additional consideration or compensation, the person -associating CC0 with a Work (the "Affirmer"), to the extent that he or she -is an owner of Copyright and Related Rights in the Work, voluntarily -elects to apply CC0 to the Work and publicly distribute the Work under its -terms, with knowledge of his or her Copyright and Related Rights in the -Work and the meaning and intended legal effect of CC0 on those rights. - -1. Copyright and Related Rights. A Work made available under CC0 may be -protected by copyright and related or neighboring rights ("Copyright and -Related Rights"). Copyright and Related Rights include, but are not -limited to, the following: - - i. the right to reproduce, adapt, distribute, perform, display, - communicate, and translate a Work; - ii. moral rights retained by the original author(s) and/or performer(s); -iii. publicity and privacy rights pertaining to a person's image or - likeness depicted in a Work; - iv. rights protecting against unfair competition in regards to a Work, - subject to the limitations in paragraph 4(a), below; - v. rights protecting the extraction, dissemination, use and reuse of data - in a Work; - vi. database rights (such as those arising under Directive 96/9/EC of the - European Parliament and of the Council of 11 March 1996 on the legal - protection of databases, and under any national implementation - thereof, including any amended or successor version of such - directive); and -vii. other similar, equivalent or corresponding rights throughout the - world based on applicable law or treaty, and any national - implementations thereof. - -2. Waiver. To the greatest extent permitted by, but not in contravention -of, applicable law, Affirmer hereby overtly, fully, permanently, -irrevocably and unconditionally waives, abandons, and surrenders all of -Affirmer's Copyright and Related Rights and associated claims and causes -of action, whether now known or unknown (including existing as well as -future claims and causes of action), in the Work (i) in all territories -worldwide, (ii) for the maximum duration provided by applicable law or -treaty (including future time extensions), (iii) in any current or future -medium and for any number of copies, and (iv) for any purpose whatsoever, -including without limitation commercial, advertising or promotional -purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each -member of the public at large and to the detriment of Affirmer's heirs and -successors, fully intending that such Waiver shall not be subject to -revocation, rescission, cancellation, termination, or any other legal or -equitable action to disrupt the quiet enjoyment of the Work by the public -as contemplated by Affirmer's express Statement of Purpose. - -3. Public License Fallback. Should any part of the Waiver for any reason -be judged legally invalid or ineffective under applicable law, then the -Waiver shall be preserved to the maximum extent permitted taking into -account Affirmer's express Statement of Purpose. In addition, to the -extent the Waiver is so judged Affirmer hereby grants to each affected -person a royalty-free, non transferable, non sublicensable, non exclusive, -irrevocable and unconditional license to exercise Affirmer's Copyright and -Related Rights in the Work (i) in all territories worldwide, (ii) for the -maximum duration provided by applicable law or treaty (including future -time extensions), (iii) in any current or future medium and for any number -of copies, and (iv) for any purpose whatsoever, including without -limitation commercial, advertising or promotional purposes (the -"License"). The License shall be deemed effective as of the date CC0 was -applied by Affirmer to the Work. Should any part of the License for any -reason be judged legally invalid or ineffective under applicable law, such -partial invalidity or ineffectiveness shall not invalidate the remainder -of the License, and in such case Affirmer hereby affirms that he or she -will not (i) exercise any of his or her remaining Copyright and Related -Rights in the Work or (ii) assert any associated claims and causes of -action with respect to the Work, in either case contrary to Affirmer's -express Statement of Purpose. - -4. Limitations and Disclaimers. - - a. No trademark or patent rights held by Affirmer are waived, abandoned, - surrendered, licensed or otherwise affected by this document. - b. Affirmer offers the Work as-is and makes no representations or - warranties of any kind concerning the Work, express, implied, - statutory or otherwise, including without limitation warranties of - title, merchantability, fitness for a particular purpose, non - infringement, or the absence of latent or other defects, accuracy, or - the present or absence of errors, whether or not discoverable, all to - the greatest extent permissible under applicable law. - c. Affirmer disclaims responsibility for clearing rights of other persons - that may apply to the Work or any use thereof, including without - limitation any person's Copyright and Related Rights in the Work. - Further, Affirmer disclaims responsibility for obtaining any necessary - consents, permissions or other rights required for any use of the - Work. - d. Affirmer understands and acknowledges that Creative Commons is not a - party to this document and has no duty or obligation with respect to - this CC0 or use of the Work. +Creative Commons Legal Code + +CC0 1.0 Universal + + CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE + LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN + ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS + INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES + REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS + PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM + THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED + HEREUNDER. + +Statement of Purpose + +The laws of most jurisdictions throughout the world automatically confer +exclusive Copyright and Related Rights (defined below) upon the creator +and subsequent owner(s) (each and all, an "owner") of an original work of +authorship and/or a database (each, a "Work"). + +Certain owners wish to permanently relinquish those rights to a Work for +the purpose of contributing to a commons of creative, cultural and +scientific works ("Commons") that the public can reliably and without fear +of later claims of infringement build upon, modify, incorporate in other +works, reuse and redistribute as freely as possible in any form whatsoever +and for any purposes, including without limitation commercial purposes. +These owners may contribute to the Commons to promote the ideal of a free +culture and the further production of creative, cultural and scientific +works, or to gain reputation or greater distribution for their Work in +part through the use and efforts of others. + +For these and/or other purposes and motivations, and without any +expectation of additional consideration or compensation, the person +associating CC0 with a Work (the "Affirmer"), to the extent that he or she +is an owner of Copyright and Related Rights in the Work, voluntarily +elects to apply CC0 to the Work and publicly distribute the Work under its +terms, with knowledge of his or her Copyright and Related Rights in the +Work and the meaning and intended legal effect of CC0 on those rights. + +1. Copyright and Related Rights. A Work made available under CC0 may be +protected by copyright and related or neighboring rights ("Copyright and +Related Rights"). Copyright and Related Rights include, but are not +limited to, the following: + + i. the right to reproduce, adapt, distribute, perform, display, + communicate, and translate a Work; + ii. moral rights retained by the original author(s) and/or performer(s); +iii. publicity and privacy rights pertaining to a person's image or + likeness depicted in a Work; + iv. rights protecting against unfair competition in regards to a Work, + subject to the limitations in paragraph 4(a), below; + v. rights protecting the extraction, dissemination, use and reuse of data + in a Work; + vi. database rights (such as those arising under Directive 96/9/EC of the + European Parliament and of the Council of 11 March 1996 on the legal + protection of databases, and under any national implementation + thereof, including any amended or successor version of such + directive); and +vii. other similar, equivalent or corresponding rights throughout the + world based on applicable law or treaty, and any national + implementations thereof. + +2. Waiver. To the greatest extent permitted by, but not in contravention +of, applicable law, Affirmer hereby overtly, fully, permanently, +irrevocably and unconditionally waives, abandons, and surrenders all of +Affirmer's Copyright and Related Rights and associated claims and causes +of action, whether now known or unknown (including existing as well as +future claims and causes of action), in the Work (i) in all territories +worldwide, (ii) for the maximum duration provided by applicable law or +treaty (including future time extensions), (iii) in any current or future +medium and for any number of copies, and (iv) for any purpose whatsoever, +including without limitation commercial, advertising or promotional +purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each +member of the public at large and to the detriment of Affirmer's heirs and +successors, fully intending that such Waiver shall not be subject to +revocation, rescission, cancellation, termination, or any other legal or +equitable action to disrupt the quiet enjoyment of the Work by the public +as contemplated by Affirmer's express Statement of Purpose. + +3. Public License Fallback. Should any part of the Waiver for any reason +be judged legally invalid or ineffective under applicable law, then the +Waiver shall be preserved to the maximum extent permitted taking into +account Affirmer's express Statement of Purpose. In addition, to the +extent the Waiver is so judged Affirmer hereby grants to each affected +person a royalty-free, non transferable, non sublicensable, non exclusive, +irrevocable and unconditional license to exercise Affirmer's Copyright and +Related Rights in the Work (i) in all territories worldwide, (ii) for the +maximum duration provided by applicable law or treaty (including future +time extensions), (iii) in any current or future medium and for any number +of copies, and (iv) for any purpose whatsoever, including without +limitation commercial, advertising or promotional purposes (the +"License"). The License shall be deemed effective as of the date CC0 was +applied by Affirmer to the Work. Should any part of the License for any +reason be judged legally invalid or ineffective under applicable law, such +partial invalidity or ineffectiveness shall not invalidate the remainder +of the License, and in such case Affirmer hereby affirms that he or she +will not (i) exercise any of his or her remaining Copyright and Related +Rights in the Work or (ii) assert any associated claims and causes of +action with respect to the Work, in either case contrary to Affirmer's +express Statement of Purpose. + +4. Limitations and Disclaimers. + + a. No trademark or patent rights held by Affirmer are waived, abandoned, + surrendered, licensed or otherwise affected by this document. + b. Affirmer offers the Work as-is and makes no representations or + warranties of any kind concerning the Work, express, implied, + statutory or otherwise, including without limitation warranties of + title, merchantability, fitness for a particular purpose, non + infringement, or the absence of latent or other defects, accuracy, or + the present or absence of errors, whether or not discoverable, all to + the greatest extent permissible under applicable law. + c. Affirmer disclaims responsibility for clearing rights of other persons + that may apply to the Work or any use thereof, including without + limitation any person's Copyright and Related Rights in the Work. + Further, Affirmer disclaims responsibility for obtaining any necessary + consents, permissions or other rights required for any use of the + Work. + d. Affirmer understands and acknowledges that Creative Commons is not a + party to this document and has no duty or obligation with respect to + this CC0 or use of the Work. diff --git a/contrib/libs/hdr_histogram/LICENSE.txt b/contrib/libs/hdr_histogram/LICENSE.txt index bda854f6833..9b4e66ed7de 100644 --- a/contrib/libs/hdr_histogram/LICENSE.txt +++ b/contrib/libs/hdr_histogram/LICENSE.txt @@ -1,41 +1,41 @@ -The code in this repository code was Written by Gil Tene, Michael Barker, -and Matt Warren, and released to the public domain, as explained at -http://creativecommons.org/publicdomain/zero/1.0/ - -For users of this code who wish to consume it under the "BSD" license -rather than under the public domain or CC0 contribution text mentioned -above, the code found under this directory is *also* provided under the -following license (commonly referred to as the BSD 2-Clause License). This -license does not detract from the above stated release of the code into -the public domain, and simply represents an additional license granted by -the Author. - ------------------------------------------------------------------------------ -** Beginning of "BSD 2-Clause License" text. ** - - Copyright (c) 2012, 2013, 2014 Gil Tene - Copyright (c) 2014 Michael Barker - Copyright (c) 2014 Matt Warren - All rights reserved. - - Redistribution and use in source and binary forms, with or without - modification, are permitted provided that the following conditions are met: - - 1. Redistributions of source code must retain the above copyright notice, - this list of conditions and the following disclaimer. - - 2. Redistributions in binary form must reproduce the above copyright notice, - this list of conditions and the following disclaimer in the documentation - and/or other materials provided with the distribution. - - THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" - AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE - LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR - CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS - INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN - CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF - THE POSSIBILITY OF SUCH DAMAGE. +The code in this repository code was Written by Gil Tene, Michael Barker, +and Matt Warren, and released to the public domain, as explained at +http://creativecommons.org/publicdomain/zero/1.0/ + +For users of this code who wish to consume it under the "BSD" license +rather than under the public domain or CC0 contribution text mentioned +above, the code found under this directory is *also* provided under the +following license (commonly referred to as the BSD 2-Clause License). This +license does not detract from the above stated release of the code into +the public domain, and simply represents an additional license granted by +the Author. + +----------------------------------------------------------------------------- +** Beginning of "BSD 2-Clause License" text. ** + + Copyright (c) 2012, 2013, 2014 Gil Tene + Copyright (c) 2014 Michael Barker + Copyright (c) 2014 Matt Warren + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions are met: + + 1. Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimer. + + 2. Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" + AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE + LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF + SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN + CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF + THE POSSIBILITY OF SUCH DAMAGE. diff --git a/contrib/libs/hdr_histogram/README.md b/contrib/libs/hdr_histogram/README.md index dbb56a7ae00..b669c3fe95a 100644 --- a/contrib/libs/hdr_histogram/README.md +++ b/contrib/libs/hdr_histogram/README.md @@ -1,79 +1,79 @@ -HdrHistogram_c: 'C' port of High Dynamic Range (HDR) Histogram - -HdrHistogram ----------------------------------------------- -[](https://gitter.im/HdrHistogram/HdrHistogram?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) - -Windows Build:  - -This port contains a subset of the functionality supported by the Java -implementation. The current supported features are: - -* Standard histogram with 64 bit counts (32/16 bit counts not supported) -* All iterator types (all values, recorded, percentiles, linear, logarithmic) -* Histogram serialisation (encoding version 1.2, decoding 1.0-1.2) -* Reader/writer phaser and interval recorder - -Features not supported, but planned - -* Auto-resizing of histograms - -Features unlikely to be implemented - -* Double histograms -* Atomic/Concurrent histograms -* 16/32 bit histograms - -# Simple Tutorial - -## Recording values - -```C -#include <hdr_histogram.h> - -struct hdr_histogram* histogram; - -// Initialise the histogram -hdr_init( - 1, // Minimum value - INT64_C(3600000000), // Maximum value - 3, // Number of significant figures - &histogram) // Pointer to initialise - -// Record value -hdr_record_value( - histogram, // Histogram to record to - value) // Value to record - -// Record value n times -hdr_record_values( - histogram, // Histogram to record to - value, // Value to record - 10) // Record value 10 times - -// Record value with correction for co-ordinated omission. -hdr_record_corrected_value( - histogram, // Histogram to record to - value, // Value to record - 1000) // Record with expected interval of 1000. - -// Print out the values of the histogram -hdr_percentiles_print( - histogram, - stdout, // File to write to - 5, // Granularity of printed values - 1.0, // Multiplier for results - CLASSIC); // Format CLASSIC/CSV supported. -``` - -## More examples - -For more detailed examples of recording and logging results look at the -[hdr_decoder](examples/hdr_decoder.c) -and [hiccup](examples/hiccup.c) -examples. You can run hiccup and decoder -and pipe the results of one into the other. - -``` -$ ./examples/hiccup | ./examples/hdr_decoder -``` +HdrHistogram_c: 'C' port of High Dynamic Range (HDR) Histogram + +HdrHistogram +---------------------------------------------- +[](https://gitter.im/HdrHistogram/HdrHistogram?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) + +Windows Build:  + +This port contains a subset of the functionality supported by the Java +implementation. The current supported features are: + +* Standard histogram with 64 bit counts (32/16 bit counts not supported) +* All iterator types (all values, recorded, percentiles, linear, logarithmic) +* Histogram serialisation (encoding version 1.2, decoding 1.0-1.2) +* Reader/writer phaser and interval recorder + +Features not supported, but planned + +* Auto-resizing of histograms + +Features unlikely to be implemented + +* Double histograms +* Atomic/Concurrent histograms +* 16/32 bit histograms + +# Simple Tutorial + +## Recording values + +```C +#include <hdr_histogram.h> + +struct hdr_histogram* histogram; + +// Initialise the histogram +hdr_init( + 1, // Minimum value + INT64_C(3600000000), // Maximum value + 3, // Number of significant figures + &histogram) // Pointer to initialise + +// Record value +hdr_record_value( + histogram, // Histogram to record to + value) // Value to record + +// Record value n times +hdr_record_values( + histogram, // Histogram to record to + value, // Value to record + 10) // Record value 10 times + +// Record value with correction for co-ordinated omission. +hdr_record_corrected_value( + histogram, // Histogram to record to + value, // Value to record + 1000) // Record with expected interval of 1000. + +// Print out the values of the histogram +hdr_percentiles_print( + histogram, + stdout, // File to write to + 5, // Granularity of printed values + 1.0, // Multiplier for results + CLASSIC); // Format CLASSIC/CSV supported. +``` + +## More examples + +For more detailed examples of recording and logging results look at the +[hdr_decoder](examples/hdr_decoder.c) +and [hiccup](examples/hiccup.c) +examples. You can run hiccup and decoder +and pipe the results of one into the other. + +``` +$ ./examples/hiccup | ./examples/hdr_decoder +``` diff --git a/contrib/libs/hdr_histogram/src/hdr_atomic.h b/contrib/libs/hdr_histogram/src/hdr_atomic.h index 4c44aa350ac..7015d985da3 100644 --- a/contrib/libs/hdr_histogram/src/hdr_atomic.h +++ b/contrib/libs/hdr_histogram/src/hdr_atomic.h @@ -1,59 +1,59 @@ -/** - * hdr_atomic.h - * Written by Philip Orwig and released to the public domain, - * as explained at http://creativecommons.org/publicdomain/zero/1.0/ - */ - -#ifndef HDR_ATOMIC_H__ -#define HDR_ATOMIC_H__ - - -#if defined(_MSC_VER) - -#include <stdint.h> -#include <intrin.h> - -static void __inline * hdr_atomic_load_pointer(void** pointer) +/** + * hdr_atomic.h + * Written by Philip Orwig and released to the public domain, + * as explained at http://creativecommons.org/publicdomain/zero/1.0/ + */ + +#ifndef HDR_ATOMIC_H__ +#define HDR_ATOMIC_H__ + + +#if defined(_MSC_VER) + +#include <stdint.h> +#include <intrin.h> + +static void __inline * hdr_atomic_load_pointer(void** pointer) +{ + _ReadBarrier(); + return *pointer; +} + +static void hdr_atomic_store_pointer(void** pointer, void* value) +{ + _WriteBarrier(); + *pointer = value; +} + +static int64_t __inline hdr_atomic_load_64(int64_t* field) { - _ReadBarrier(); - return *pointer; -} - -static void hdr_atomic_store_pointer(void** pointer, void* value) -{ - _WriteBarrier(); - *pointer = value; -} - -static int64_t __inline hdr_atomic_load_64(int64_t* field) -{ - _ReadBarrier(); - return *field; -} - -static void __inline hdr_atomic_store_64(int64_t* field, int64_t value) -{ - _WriteBarrier(); - *field = value; -} - -static int64_t __inline hdr_atomic_exchange_64(volatile int64_t* field, int64_t initial) -{ - return _InterlockedExchange64(field, initial); -} - -static int64_t __inline hdr_atomic_add_fetch_64(volatile int64_t* field, int64_t value) -{ - return _InterlockedExchangeAdd64(field, value) + value; -} - -#else -#define hdr_atomic_load_pointer(x) __atomic_load_n(x, __ATOMIC_SEQ_CST) -#define hdr_atomic_store_pointer(f,v) __atomic_store_n(f,v, __ATOMIC_SEQ_CST) -#define hdr_atomic_load_64(x) __atomic_load_n(x, __ATOMIC_SEQ_CST) -#define hdr_atomic_store_64(f,v) __atomic_store_n(f,v, __ATOMIC_SEQ_CST) -#define hdr_atomic_exchange_64(f,i) __atomic_exchange_n(f,i, __ATOMIC_SEQ_CST) -#define hdr_atomic_add_fetch_64(field, value) __atomic_add_fetch(field, value, __ATOMIC_SEQ_CST) -#endif - -#endif /* HDR_ATOMIC_H__ */ + _ReadBarrier(); + return *field; +} + +static void __inline hdr_atomic_store_64(int64_t* field, int64_t value) +{ + _WriteBarrier(); + *field = value; +} + +static int64_t __inline hdr_atomic_exchange_64(volatile int64_t* field, int64_t initial) +{ + return _InterlockedExchange64(field, initial); +} + +static int64_t __inline hdr_atomic_add_fetch_64(volatile int64_t* field, int64_t value) +{ + return _InterlockedExchangeAdd64(field, value) + value; +} + +#else +#define hdr_atomic_load_pointer(x) __atomic_load_n(x, __ATOMIC_SEQ_CST) +#define hdr_atomic_store_pointer(f,v) __atomic_store_n(f,v, __ATOMIC_SEQ_CST) +#define hdr_atomic_load_64(x) __atomic_load_n(x, __ATOMIC_SEQ_CST) +#define hdr_atomic_store_64(f,v) __atomic_store_n(f,v, __ATOMIC_SEQ_CST) +#define hdr_atomic_exchange_64(f,i) __atomic_exchange_n(f,i, __ATOMIC_SEQ_CST) +#define hdr_atomic_add_fetch_64(field, value) __atomic_add_fetch(field, value, __ATOMIC_SEQ_CST) +#endif + +#endif /* HDR_ATOMIC_H__ */ diff --git a/contrib/libs/hdr_histogram/src/hdr_encoding.c b/contrib/libs/hdr_histogram/src/hdr_encoding.c index 43db22a3fa2..758cf68ebbe 100644 --- a/contrib/libs/hdr_histogram/src/hdr_encoding.c +++ b/contrib/libs/hdr_histogram/src/hdr_encoding.c @@ -1,313 +1,313 @@ -// -// Created by barkerm on 9/09/15. -// - -#include <errno.h> -#include <stddef.h> -#include <math.h> - -#include "hdr_encoding.h" -#include "hdr_tests.h" - -int zig_zag_encode_i64(uint8_t* buffer, int64_t signed_value) -{ - int64_t value = signed_value; - - value = (value << 1) ^ (value >> 63); - int bytesWritten = 0; - if (value >> 7 == 0) - { - buffer[0] = (uint8_t) value; - bytesWritten = 1; - } - else - { - buffer[0] = (uint8_t) ((value & 0x7F) | 0x80); - if (value >> 14 == 0) - { - buffer[1] = (uint8_t) (value >> 7); - bytesWritten = 2; - } - else - { - buffer[1] = (uint8_t) ((value >> 7 | 0x80)); - if (value >> 21 == 0) - { - buffer[2] = (uint8_t) (value >> 14); - bytesWritten = 3; - } - else - { - buffer[2] = (uint8_t) (value >> 14 | 0x80); - if (value >> 28 == 0) - { - buffer[3] = (uint8_t) (value >> 21); - bytesWritten = 4; - } - else - { - buffer[3] = (uint8_t) (value >> 21 | 0x80); - if (value >> 35 == 0) - { - buffer[4] = (uint8_t) (value >> 28); - bytesWritten = 5; - } - else - { - buffer[4] = (uint8_t) (value >> 28 | 0x80); - if (value >> 42 == 0) - { - buffer[5] = (uint8_t) (value >> 35); - bytesWritten = 6; - } - else - { - buffer[5] = (uint8_t) (value >> 35 | 0x80); - if (value >> 49 == 0) - { - buffer[6] = (uint8_t) (value >> 42); - bytesWritten = 7; - } - else - { - buffer[6] = (uint8_t) (value >> 42 | 0x80); - if (value >> 56 == 0) - { - buffer[7] = (uint8_t) (value >> 49); - bytesWritten = 8; - } - else - { - buffer[7] = (uint8_t) (value >> 49 | 0x80); - buffer[8] = (uint8_t) (value >> 56); - bytesWritten = 9; - } - } - } - } - } - } - } - } - - return bytesWritten; -} - -int zig_zag_decode_i64(const uint8_t* buffer, int64_t* retVal) -{ - uint64_t v = buffer[0]; - uint64_t value = v & 0x7F; - int bytesRead = 1; - if ((v & 0x80) != 0) - { - bytesRead = 2; - v = buffer[1]; - value |= (v & 0x7F) << 7; - if ((v & 0x80) != 0) - { - bytesRead = 3; - v = buffer[2]; - value |= (v & 0x7F) << 14; - if ((v & 0x80) != 0) - { - bytesRead = 4; - v = buffer[3]; - value |= (v & 0x7F) << 21; - if ((v & 0x80) != 0) - { - bytesRead = 5; - v = buffer[4]; - value |= (v & 0x7F) << 28; - if ((v & 0x80) != 0) - { - bytesRead = 6; - v = buffer[5]; - value |= (v & 0x7F) << 35; - if ((v & 0x80) != 0) - { - bytesRead = 7; - v = buffer[6]; - value |= (v & 0x7F) << 42; - if ((v & 0x80) != 0) - { - bytesRead = 8; - v = buffer[7]; - value |= (v & 0x7F) << 49; - if ((v & 0x80) != 0) - { - bytesRead = 9; - v = buffer[8]; - value |= v << 56; - } - } - } - } - } - } - } - } - - value = (value >> 1) ^ (-(value & 1)); - *retVal = (int64_t) value; - - return bytesRead; -} - -static const char base64_table[] = - { - 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', - 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z', - 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', - 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', - '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '+', '/', '\0' - }; - -static char get_base_64(uint32_t _24_bit_value, int shift) -{ - uint32_t _6_bit_value = 0x3F & (_24_bit_value >> shift); - return base64_table[_6_bit_value]; -} - -static int from_base_64(int c) -{ - if ('A' <= c && c <= 'Z') - { - return c - 'A'; - } - else if ('a' <= c && c <= 'z') - { - return (c - 'a') + 26; - } - else if ('0' <= c && c <= '9') - { - return (c - '0') + 52; - } - else if ('+' == c) - { - return 62; - } - else if ('/' == c) - { - return 63; - } - else if ('=' == c) - { - return 0; - } - - return EINVAL; -} - -size_t hdr_base64_encoded_len(size_t decoded_size) -{ - return (size_t) (ceil(decoded_size / 3.0) * 4.0); -} - -size_t hdr_base64_decoded_len(size_t encoded_size) -{ - return (encoded_size / 4) * 3; -} - -static void hdr_base64_encode_block_pad(const uint8_t* input, char* output, size_t pad) -{ - uint32_t _24_bit_value = 0; - - switch (pad) - { - case 2: - _24_bit_value = (input[0] << 16) + (input[1] << 8); - - output[0] = get_base_64(_24_bit_value, 18); - output[1] = get_base_64(_24_bit_value, 12); - output[2] = get_base_64(_24_bit_value, 6); - output[3] = '='; - - break; - - case 1: - _24_bit_value = (input[0] << 16); - - output[0] = get_base_64(_24_bit_value, 18); - output[1] = get_base_64(_24_bit_value, 12); - output[2] = '='; - output[3] = '='; - - break; - - default: - // No-op - break; - } -} - -/** - * Assumes that there is 3 input bytes and 4 output chars. - */ -void hdr_base64_encode_block(const uint8_t* input, char* output) -{ - uint32_t _24_bit_value = (input[0] << 16) + (input[1] << 8) + (input[2]); - - output[0] = get_base_64(_24_bit_value, 18); - output[1] = get_base_64(_24_bit_value, 12); - output[2] = get_base_64(_24_bit_value, 6); - output[3] = get_base_64(_24_bit_value, 0); -} - -int hdr_base64_encode( - const uint8_t* input, size_t input_len, char* output, size_t output_len) -{ - if (hdr_base64_encoded_len(input_len) != output_len) - { - return EINVAL; - } - - size_t i = 0; - size_t j = 0; - for (; input_len - i >= 3 && j < output_len; i += 3, j += 4) - { - hdr_base64_encode_block(&input[i], &output[j]); - } - - size_t remaining = input_len - i; - - hdr_base64_encode_block_pad(&input[i], &output[j], remaining); - - return 0; -} - -/** - * Assumes that there is 4 input chars available and 3 output chars. - */ -void hdr_base64_decode_block(const char* input, uint8_t* output) -{ - uint32_t _24_bit_value = 0; - - _24_bit_value |= from_base_64(input[0]) << 18; - _24_bit_value |= from_base_64(input[1]) << 12; - _24_bit_value |= from_base_64(input[2]) << 6; - _24_bit_value |= from_base_64(input[3]); - - output[0] = (uint8_t) ((_24_bit_value >> 16) & 0xFF); - output[1] = (uint8_t) ((_24_bit_value >> 8) & 0xFF); - output[2] = (uint8_t) ((_24_bit_value) & 0xFF); -} - -int hdr_base64_decode( - const char* input, size_t input_len, uint8_t* output, size_t output_len) -{ - size_t i, j; - - if (input_len < 4 || - (input_len & 3) != 0 || - (input_len / 4) * 3 != output_len) - { - return EINVAL; - } - - for (i = 0, j = 0; i < input_len; i += 4, j += 3) - { - hdr_base64_decode_block(&input[i], &output[j]); - } - - return 0; -} +// +// Created by barkerm on 9/09/15. +// + +#include <errno.h> +#include <stddef.h> +#include <math.h> + +#include "hdr_encoding.h" +#include "hdr_tests.h" + +int zig_zag_encode_i64(uint8_t* buffer, int64_t signed_value) +{ + int64_t value = signed_value; + + value = (value << 1) ^ (value >> 63); + int bytesWritten = 0; + if (value >> 7 == 0) + { + buffer[0] = (uint8_t) value; + bytesWritten = 1; + } + else + { + buffer[0] = (uint8_t) ((value & 0x7F) | 0x80); + if (value >> 14 == 0) + { + buffer[1] = (uint8_t) (value >> 7); + bytesWritten = 2; + } + else + { + buffer[1] = (uint8_t) ((value >> 7 | 0x80)); + if (value >> 21 == 0) + { + buffer[2] = (uint8_t) (value >> 14); + bytesWritten = 3; + } + else + { + buffer[2] = (uint8_t) (value >> 14 | 0x80); + if (value >> 28 == 0) + { + buffer[3] = (uint8_t) (value >> 21); + bytesWritten = 4; + } + else + { + buffer[3] = (uint8_t) (value >> 21 | 0x80); + if (value >> 35 == 0) + { + buffer[4] = (uint8_t) (value >> 28); + bytesWritten = 5; + } + else + { + buffer[4] = (uint8_t) (value >> 28 | 0x80); + if (value >> 42 == 0) + { + buffer[5] = (uint8_t) (value >> 35); + bytesWritten = 6; + } + else + { + buffer[5] = (uint8_t) (value >> 35 | 0x80); + if (value >> 49 == 0) + { + buffer[6] = (uint8_t) (value >> 42); + bytesWritten = 7; + } + else + { + buffer[6] = (uint8_t) (value >> 42 | 0x80); + if (value >> 56 == 0) + { + buffer[7] = (uint8_t) (value >> 49); + bytesWritten = 8; + } + else + { + buffer[7] = (uint8_t) (value >> 49 | 0x80); + buffer[8] = (uint8_t) (value >> 56); + bytesWritten = 9; + } + } + } + } + } + } + } + } + + return bytesWritten; +} + +int zig_zag_decode_i64(const uint8_t* buffer, int64_t* retVal) +{ + uint64_t v = buffer[0]; + uint64_t value = v & 0x7F; + int bytesRead = 1; + if ((v & 0x80) != 0) + { + bytesRead = 2; + v = buffer[1]; + value |= (v & 0x7F) << 7; + if ((v & 0x80) != 0) + { + bytesRead = 3; + v = buffer[2]; + value |= (v & 0x7F) << 14; + if ((v & 0x80) != 0) + { + bytesRead = 4; + v = buffer[3]; + value |= (v & 0x7F) << 21; + if ((v & 0x80) != 0) + { + bytesRead = 5; + v = buffer[4]; + value |= (v & 0x7F) << 28; + if ((v & 0x80) != 0) + { + bytesRead = 6; + v = buffer[5]; + value |= (v & 0x7F) << 35; + if ((v & 0x80) != 0) + { + bytesRead = 7; + v = buffer[6]; + value |= (v & 0x7F) << 42; + if ((v & 0x80) != 0) + { + bytesRead = 8; + v = buffer[7]; + value |= (v & 0x7F) << 49; + if ((v & 0x80) != 0) + { + bytesRead = 9; + v = buffer[8]; + value |= v << 56; + } + } + } + } + } + } + } + } + + value = (value >> 1) ^ (-(value & 1)); + *retVal = (int64_t) value; + + return bytesRead; +} + +static const char base64_table[] = + { + 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', + 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z', + 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', + 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', + '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '+', '/', '\0' + }; + +static char get_base_64(uint32_t _24_bit_value, int shift) +{ + uint32_t _6_bit_value = 0x3F & (_24_bit_value >> shift); + return base64_table[_6_bit_value]; +} + +static int from_base_64(int c) +{ + if ('A' <= c && c <= 'Z') + { + return c - 'A'; + } + else if ('a' <= c && c <= 'z') + { + return (c - 'a') + 26; + } + else if ('0' <= c && c <= '9') + { + return (c - '0') + 52; + } + else if ('+' == c) + { + return 62; + } + else if ('/' == c) + { + return 63; + } + else if ('=' == c) + { + return 0; + } + + return EINVAL; +} + +size_t hdr_base64_encoded_len(size_t decoded_size) +{ + return (size_t) (ceil(decoded_size / 3.0) * 4.0); +} + +size_t hdr_base64_decoded_len(size_t encoded_size) +{ + return (encoded_size / 4) * 3; +} + +static void hdr_base64_encode_block_pad(const uint8_t* input, char* output, size_t pad) +{ + uint32_t _24_bit_value = 0; + + switch (pad) + { + case 2: + _24_bit_value = (input[0] << 16) + (input[1] << 8); + + output[0] = get_base_64(_24_bit_value, 18); + output[1] = get_base_64(_24_bit_value, 12); + output[2] = get_base_64(_24_bit_value, 6); + output[3] = '='; + + break; + + case 1: + _24_bit_value = (input[0] << 16); + + output[0] = get_base_64(_24_bit_value, 18); + output[1] = get_base_64(_24_bit_value, 12); + output[2] = '='; + output[3] = '='; + + break; + + default: + // No-op + break; + } +} + +/** + * Assumes that there is 3 input bytes and 4 output chars. + */ +void hdr_base64_encode_block(const uint8_t* input, char* output) +{ + uint32_t _24_bit_value = (input[0] << 16) + (input[1] << 8) + (input[2]); + + output[0] = get_base_64(_24_bit_value, 18); + output[1] = get_base_64(_24_bit_value, 12); + output[2] = get_base_64(_24_bit_value, 6); + output[3] = get_base_64(_24_bit_value, 0); +} + +int hdr_base64_encode( + const uint8_t* input, size_t input_len, char* output, size_t output_len) +{ + if (hdr_base64_encoded_len(input_len) != output_len) + { + return EINVAL; + } + + size_t i = 0; + size_t j = 0; + for (; input_len - i >= 3 && j < output_len; i += 3, j += 4) + { + hdr_base64_encode_block(&input[i], &output[j]); + } + + size_t remaining = input_len - i; + + hdr_base64_encode_block_pad(&input[i], &output[j], remaining); + + return 0; +} + +/** + * Assumes that there is 4 input chars available and 3 output chars. + */ +void hdr_base64_decode_block(const char* input, uint8_t* output) +{ + uint32_t _24_bit_value = 0; + + _24_bit_value |= from_base_64(input[0]) << 18; + _24_bit_value |= from_base_64(input[1]) << 12; + _24_bit_value |= from_base_64(input[2]) << 6; + _24_bit_value |= from_base_64(input[3]); + + output[0] = (uint8_t) ((_24_bit_value >> 16) & 0xFF); + output[1] = (uint8_t) ((_24_bit_value >> 8) & 0xFF); + output[2] = (uint8_t) ((_24_bit_value) & 0xFF); +} + +int hdr_base64_decode( + const char* input, size_t input_len, uint8_t* output, size_t output_len) +{ + size_t i, j; + + if (input_len < 4 || + (input_len & 3) != 0 || + (input_len / 4) * 3 != output_len) + { + return EINVAL; + } + + for (i = 0, j = 0; i < input_len; i += 4, j += 3) + { + hdr_base64_decode_block(&input[i], &output[j]); + } + + return 0; +} diff --git a/contrib/libs/hdr_histogram/src/hdr_encoding.h b/contrib/libs/hdr_histogram/src/hdr_encoding.h index dc9076d7bb8..141093ba3be 100644 --- a/contrib/libs/hdr_histogram/src/hdr_encoding.h +++ b/contrib/libs/hdr_histogram/src/hdr_encoding.h @@ -1,77 +1,77 @@ -// -// Created by barkerm on 9/09/15. -// - -#ifndef HDR_ENCODING_H -#define HDR_ENCODING_H - -#include <stdint.h> - -#define MAX_BYTES_LEB128 9 - -#ifdef __cplusplus -extern "C" { -#endif - -/** - * Writes a int64_t value to the given buffer in LEB128 ZigZag encoded format - * - * @param buffer the buffer to write to - * @param signed_value the value to write to the buffer - * @return the number of bytes written to the buffer - */ -int zig_zag_encode_i64(uint8_t* buffer, int64_t signed_value); - -/** - * Read an LEB128 ZigZag encoded long value from the given buffer - * - * @param buffer the buffer to read from - * @param retVal out value to capture the read value - * @return the number of bytes read from the buffer - */ -int zig_zag_decode_i64(const uint8_t* buffer, int64_t* signed_value); - -/** - * Gets the length in bytes of base64 data, given the input size. - * - * @param decoded_size the size of the unencoded values. - * @return the encoded size - */ -size_t hdr_base64_encoded_len(size_t decoded_size); - -/** - * Encode into base64. - * - * @param input the data to encode - * @param input_len the length of the data to encode - * @param output the buffer to write the output to - * @param output_len the number of bytes to write to the output - */ -int hdr_base64_encode( - const uint8_t* input, size_t input_len, char* output, size_t output_len); - -/** - * Gets the length in bytes of decoded base64 data, given the size of the base64 encoded - * data. - * - * @param encoded_size the size of the encoded value. - * @return the decoded size - */ -size_t hdr_base64_decoded_len(size_t encoded_size); - -/** - * Decode from base64. - * - * @param input the base64 encoded data - * @param input_len the size in bytes of the endcoded data - * @param output the buffer to write the decoded data to - * @param output_len the number of bytes to write to the output data - */ -int hdr_base64_decode( - const char* input, size_t input_len, uint8_t* output, size_t output_len); - -#ifdef __cplusplus -} -#endif - -#endif //HDR_HISTOGRAM_HDR_ENCODING_H +// +// Created by barkerm on 9/09/15. +// + +#ifndef HDR_ENCODING_H +#define HDR_ENCODING_H + +#include <stdint.h> + +#define MAX_BYTES_LEB128 9 + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * Writes a int64_t value to the given buffer in LEB128 ZigZag encoded format + * + * @param buffer the buffer to write to + * @param signed_value the value to write to the buffer + * @return the number of bytes written to the buffer + */ +int zig_zag_encode_i64(uint8_t* buffer, int64_t signed_value); + +/** + * Read an LEB128 ZigZag encoded long value from the given buffer + * + * @param buffer the buffer to read from + * @param retVal out value to capture the read value + * @return the number of bytes read from the buffer + */ +int zig_zag_decode_i64(const uint8_t* buffer, int64_t* signed_value); + +/** + * Gets the length in bytes of base64 data, given the input size. + * + * @param decoded_size the size of the unencoded values. + * @return the encoded size + */ +size_t hdr_base64_encoded_len(size_t decoded_size); + +/** + * Encode into base64. + * + * @param input the data to encode + * @param input_len the length of the data to encode + * @param output the buffer to write the output to + * @param output_len the number of bytes to write to the output + */ +int hdr_base64_encode( + const uint8_t* input, size_t input_len, char* output, size_t output_len); + +/** + * Gets the length in bytes of decoded base64 data, given the size of the base64 encoded + * data. + * + * @param encoded_size the size of the encoded value. + * @return the decoded size + */ +size_t hdr_base64_decoded_len(size_t encoded_size); + +/** + * Decode from base64. + * + * @param input the base64 encoded data + * @param input_len the size in bytes of the endcoded data + * @param output the buffer to write the decoded data to + * @param output_len the number of bytes to write to the output data + */ +int hdr_base64_decode( + const char* input, size_t input_len, uint8_t* output, size_t output_len); + +#ifdef __cplusplus +} +#endif + +#endif //HDR_HISTOGRAM_HDR_ENCODING_H diff --git a/contrib/libs/hdr_histogram/src/hdr_endian.h b/contrib/libs/hdr_histogram/src/hdr_endian.h index a677e5a44ad..839fdb16b1d 100644 --- a/contrib/libs/hdr_histogram/src/hdr_endian.h +++ b/contrib/libs/hdr_histogram/src/hdr_endian.h @@ -1,116 +1,116 @@ -/** -* hdr_time.h -* Released to the public domain, as explained at http://creativecommons.org/publicdomain/zero/1.0/ -*/ - -#ifndef HDR_ENDIAN_H__ -#define HDR_ENDIAN_H__ - -#if (defined(_WIN16) || defined(_WIN32) || defined(_WIN64)) && !defined(__WINDOWS__) - -# define __WINDOWS__ - -#endif - -#if defined(__linux__) || defined(__CYGWIN__) - -# include <endian.h> - -#elif defined(__APPLE__) - -# include <libkern/OSByteOrder.h> - -# define htobe16(x) OSSwapHostToBigInt16(x) -# define htole16(x) OSSwapHostToLittleInt16(x) -# define be16toh(x) OSSwapBigToHostInt16(x) -# define le16toh(x) OSSwapLittleToHostInt16(x) - -# define htobe32(x) OSSwapHostToBigInt32(x) -# define htole32(x) OSSwapHostToLittleInt32(x) -# define be32toh(x) OSSwapBigToHostInt32(x) -# define le32toh(x) OSSwapLittleToHostInt32(x) - -# define htobe64(x) OSSwapHostToBigInt64(x) -# define htole64(x) OSSwapHostToLittleInt64(x) -# define be64toh(x) OSSwapBigToHostInt64(x) -# define le64toh(x) OSSwapLittleToHostInt64(x) - -# define __BYTE_ORDER BYTE_ORDER -# define __BIG_ENDIAN BIG_ENDIAN -# define __LITTLE_ENDIAN LITTLE_ENDIAN -# define __PDP_ENDIAN PDP_ENDIAN - -#elif defined(__OpenBSD__) - -# include <sys/endian.h> - -#elif defined(__NetBSD__) || defined(__FreeBSD__) || defined(__DragonFly__) - -# include <sys/endian.h> - -# define be16toh(x) betoh16(x) -# define le16toh(x) letoh16(x) - -# define be32toh(x) betoh32(x) -# define le32toh(x) letoh32(x) - -# define be64toh(x) betoh64(x) -# define le64toh(x) letoh64(x) - -#elif defined(__WINDOWS__) - -# include <winsock2.h> - -# if BYTE_ORDER == LITTLE_ENDIAN - -# define htobe16(x) htons(x) -# define htole16(x) (x) -# define be16toh(x) ntohs(x) -# define le16toh(x) (x) - -# define htobe32(x) htonl(x) -# define htole32(x) (x) -# define be32toh(x) ntohl(x) -# define le32toh(x) (x) - -# define htobe64(x) htonll(x) -# define htole64(x) (x) -# define be64toh(x) ntohll(x) -# define le64toh(x) (x) - -# elif BYTE_ORDER == BIG_ENDIAN - - /* that would be xbox 360 */ -# define htobe16(x) (x) -# define htole16(x) __builtin_bswap16(x) -# define be16toh(x) (x) -# define le16toh(x) __builtin_bswap16(x) - -# define htobe32(x) (x) -# define htole32(x) __builtin_bswap32(x) -# define be32toh(x) (x) -# define le32toh(x) __builtin_bswap32(x) - -# define htobe64(x) (x) -# define htole64(x) __builtin_bswap64(x) -# define be64toh(x) (x) -# define le64toh(x) __builtin_bswap64(x) - -# else - -# error byte order not supported - -# endif - -# define __BYTE_ORDER BYTE_ORDER -# define __BIG_ENDIAN BIG_ENDIAN -# define __LITTLE_ENDIAN LITTLE_ENDIAN -# define __PDP_ENDIAN PDP_ENDIAN - -#else - -# error platform not supported - -#endif - -#endif +/** +* hdr_time.h +* Released to the public domain, as explained at http://creativecommons.org/publicdomain/zero/1.0/ +*/ + +#ifndef HDR_ENDIAN_H__ +#define HDR_ENDIAN_H__ + +#if (defined(_WIN16) || defined(_WIN32) || defined(_WIN64)) && !defined(__WINDOWS__) + +# define __WINDOWS__ + +#endif + +#if defined(__linux__) || defined(__CYGWIN__) + +# include <endian.h> + +#elif defined(__APPLE__) + +# include <libkern/OSByteOrder.h> + +# define htobe16(x) OSSwapHostToBigInt16(x) +# define htole16(x) OSSwapHostToLittleInt16(x) +# define be16toh(x) OSSwapBigToHostInt16(x) +# define le16toh(x) OSSwapLittleToHostInt16(x) + +# define htobe32(x) OSSwapHostToBigInt32(x) +# define htole32(x) OSSwapHostToLittleInt32(x) +# define be32toh(x) OSSwapBigToHostInt32(x) +# define le32toh(x) OSSwapLittleToHostInt32(x) + +# define htobe64(x) OSSwapHostToBigInt64(x) +# define htole64(x) OSSwapHostToLittleInt64(x) +# define be64toh(x) OSSwapBigToHostInt64(x) +# define le64toh(x) OSSwapLittleToHostInt64(x) + +# define __BYTE_ORDER BYTE_ORDER +# define __BIG_ENDIAN BIG_ENDIAN +# define __LITTLE_ENDIAN LITTLE_ENDIAN +# define __PDP_ENDIAN PDP_ENDIAN + +#elif defined(__OpenBSD__) + +# include <sys/endian.h> + +#elif defined(__NetBSD__) || defined(__FreeBSD__) || defined(__DragonFly__) + +# include <sys/endian.h> + +# define be16toh(x) betoh16(x) +# define le16toh(x) letoh16(x) + +# define be32toh(x) betoh32(x) +# define le32toh(x) letoh32(x) + +# define be64toh(x) betoh64(x) +# define le64toh(x) letoh64(x) + +#elif defined(__WINDOWS__) + +# include <winsock2.h> + +# if BYTE_ORDER == LITTLE_ENDIAN + +# define htobe16(x) htons(x) +# define htole16(x) (x) +# define be16toh(x) ntohs(x) +# define le16toh(x) (x) + +# define htobe32(x) htonl(x) +# define htole32(x) (x) +# define be32toh(x) ntohl(x) +# define le32toh(x) (x) + +# define htobe64(x) htonll(x) +# define htole64(x) (x) +# define be64toh(x) ntohll(x) +# define le64toh(x) (x) + +# elif BYTE_ORDER == BIG_ENDIAN + + /* that would be xbox 360 */ +# define htobe16(x) (x) +# define htole16(x) __builtin_bswap16(x) +# define be16toh(x) (x) +# define le16toh(x) __builtin_bswap16(x) + +# define htobe32(x) (x) +# define htole32(x) __builtin_bswap32(x) +# define be32toh(x) (x) +# define le32toh(x) __builtin_bswap32(x) + +# define htobe64(x) (x) +# define htole64(x) __builtin_bswap64(x) +# define be64toh(x) (x) +# define le64toh(x) __builtin_bswap64(x) + +# else + +# error byte order not supported + +# endif + +# define __BYTE_ORDER BYTE_ORDER +# define __BIG_ENDIAN BIG_ENDIAN +# define __LITTLE_ENDIAN LITTLE_ENDIAN +# define __PDP_ENDIAN PDP_ENDIAN + +#else + +# error platform not supported + +#endif + +#endif diff --git a/contrib/libs/hdr_histogram/src/hdr_histogram.c b/contrib/libs/hdr_histogram/src/hdr_histogram.c index 044afe2a221..8cad5a0d617 100644 --- a/contrib/libs/hdr_histogram/src/hdr_histogram.c +++ b/contrib/libs/hdr_histogram/src/hdr_histogram.c @@ -1,1010 +1,1010 @@ -/** - * hdr_histogram.c - * Written by Michael Barker and released to the public domain, - * as explained at http://creativecommons.org/publicdomain/zero/1.0/ - */ - -#include <stdlib.h> -#include <stdbool.h> -#include <math.h> -#include <assert.h> -#include <stdio.h> -#include <string.h> -#include <stdint.h> -#include <errno.h> -#include <inttypes.h> - -#include "hdr_histogram.h" -#include "hdr_tests.h" - -// ###### ####### ## ## ## ## ######## ###### -// ## ## ## ## ## ## ### ## ## ## ## -// ## ## ## ## ## #### ## ## ## -// ## ## ## ## ## ## ## ## ## ###### -// ## ## ## ## ## ## #### ## ## -// ## ## ## ## ## ## ## ### ## ## ## -// ###### ####### ####### ## ## ## ###### - -static int32_t normalize_index(const struct hdr_histogram* h, int32_t index) -{ - if (h->normalizing_index_offset == 0) - { - return index; - } - - int32_t normalized_index = index - h->normalizing_index_offset; - int32_t adjustment = 0; - - if (normalized_index < 0) - { - adjustment = h->counts_len; - } - else if (normalized_index >= h->counts_len) - { - adjustment = -h->counts_len; - } - - return normalized_index + adjustment; -} - -static int64_t counts_get_direct(const struct hdr_histogram* h, int32_t index) -{ - return h->counts[index]; -} - -static int64_t counts_get_normalised(const struct hdr_histogram* h, int32_t index) -{ - return counts_get_direct(h, normalize_index(h, index)); -} - -static void counts_inc_normalised( - struct hdr_histogram* h, int32_t index, int64_t value) -{ - int32_t normalised_index = normalize_index(h, index); - h->counts[normalised_index] += value; - h->total_count += value; -} - -static void update_min_max(struct hdr_histogram* h, int64_t value) -{ - h->min_value = (value < h->min_value && value != 0) ? value : h->min_value; - h->max_value = (value > h->max_value) ? value : h->max_value; -} - -// ## ## ######## #### ## #### ######## ## ## -// ## ## ## ## ## ## ## ## ## -// ## ## ## ## ## ## ## #### -// ## ## ## ## ## ## ## ## -// ## ## ## ## ## ## ## ## -// ## ## ## ## ## ## ## ## -// ####### ## #### ######## #### ## ## - -static int64_t power(int64_t base, int64_t exp) -{ - int64_t result = 1; - while(exp) - { - result *= base; exp--; - } - return result; -} - -#if defined(_MSC_VER) -#pragma intrinsic(_BitScanReverse64) -#endif - -static int32_t get_bucket_index(const struct hdr_histogram* h, int64_t value) -{ -#if defined(_MSC_VER) - uint32_t leading_zero = 0; - _BitScanReverse64(&leading_zero, value | h->sub_bucket_mask); - int32_t pow2ceiling = 64 - (63 - leading_zero); // smallest power of 2 containing value -#else - int32_t pow2ceiling = 64 - __builtin_clzll(value | h->sub_bucket_mask); // smallest power of 2 containing value -#endif - return pow2ceiling - h->unit_magnitude - (h->sub_bucket_half_count_magnitude + 1); -} - -static int32_t get_sub_bucket_index(int64_t value, int32_t bucket_index, int32_t unit_magnitude) -{ - return (int32_t)(value >> (bucket_index + unit_magnitude)); -} - -static int32_t counts_index(const struct hdr_histogram* h, int32_t bucket_index, int32_t sub_bucket_index) -{ - // Calculate the index for the first entry in the bucket: - // (The following is the equivalent of ((bucket_index + 1) * subBucketHalfCount) ): - int32_t bucket_base_index = (bucket_index + 1) << h->sub_bucket_half_count_magnitude; - // Calculate the offset in the bucket: - int32_t offset_in_bucket = sub_bucket_index - h->sub_bucket_half_count; - // The following is the equivalent of ((sub_bucket_index - subBucketHalfCount) + bucketBaseIndex; - return bucket_base_index + offset_in_bucket; -} - -static int64_t value_from_index(int32_t bucket_index, int32_t sub_bucket_index, int32_t unit_magnitude) -{ - return ((int64_t) sub_bucket_index) << (bucket_index + unit_magnitude); -} - -int32_t counts_index_for(const struct hdr_histogram* h, int64_t value) -{ - int32_t bucket_index = get_bucket_index(h, value); - int32_t sub_bucket_index = get_sub_bucket_index(value, bucket_index, h->unit_magnitude); - - return counts_index(h, bucket_index, sub_bucket_index); -} - -int64_t hdr_value_at_index(const struct hdr_histogram *h, int32_t index) -{ - int32_t bucket_index = (index >> h->sub_bucket_half_count_magnitude) - 1; - int32_t sub_bucket_index = (index & (h->sub_bucket_half_count - 1)) + h->sub_bucket_half_count; - - if (bucket_index < 0) - { - sub_bucket_index -= h->sub_bucket_half_count; - bucket_index = 0; - } - - return value_from_index(bucket_index, sub_bucket_index, h->unit_magnitude); -} - -int64_t hdr_size_of_equivalent_value_range(const struct hdr_histogram* h, int64_t value) -{ - int32_t bucket_index = get_bucket_index(h, value); - int32_t sub_bucket_index = get_sub_bucket_index(value, bucket_index, h->unit_magnitude); - int32_t adjusted_bucket = (sub_bucket_index >= h->sub_bucket_count) ? (bucket_index + 1) : bucket_index; - return INT64_C(1) << (h->unit_magnitude + adjusted_bucket); -} - -static int64_t lowest_equivalent_value(const struct hdr_histogram* h, int64_t value) -{ - int32_t bucket_index = get_bucket_index(h, value); - int32_t sub_bucket_index = get_sub_bucket_index(value, bucket_index, h->unit_magnitude); - return value_from_index(bucket_index, sub_bucket_index, h->unit_magnitude); -} - -int64_t hdr_next_non_equivalent_value(const struct hdr_histogram *h, int64_t value) -{ - return lowest_equivalent_value(h, value) + hdr_size_of_equivalent_value_range(h, value); -} - -static int64_t highest_equivalent_value(const struct hdr_histogram* h, int64_t value) -{ - return hdr_next_non_equivalent_value(h, value) - 1; -} - -int64_t hdr_median_equivalent_value(const struct hdr_histogram *h, int64_t value) -{ - return lowest_equivalent_value(h, value) + (hdr_size_of_equivalent_value_range(h, value) >> 1); -} - -static int64_t non_zero_min(const struct hdr_histogram* h) -{ - if (INT64_MAX == h->min_value) - { - return INT64_MAX; - } - - return lowest_equivalent_value(h, h->min_value); -} - -void hdr_reset_internal_counters(struct hdr_histogram* h) -{ - int min_non_zero_index = -1; - int max_index = -1; - int64_t observed_total_count = 0; - int i; - - for (i = 0; i < h->counts_len; i++) - { - int64_t count_at_index; - - if ((count_at_index = counts_get_direct(h, i)) > 0) - { - observed_total_count += count_at_index; - max_index = i; - if (min_non_zero_index == -1 && i != 0) - { - min_non_zero_index = i; - } - } - } - - if (max_index == -1) - { - h->max_value = 0; - } - else - { - int64_t max_value = hdr_value_at_index(h, max_index); - h->max_value = highest_equivalent_value(h, max_value); - } - - if (min_non_zero_index == -1) - { - h->min_value = INT64_MAX; - } - else - { - h->min_value = hdr_value_at_index(h, min_non_zero_index); - } - - h->total_count = observed_total_count; -} - -static int32_t buckets_needed_to_cover_value(int64_t value, int32_t sub_bucket_count, int32_t unit_magnitude) -{ - int64_t smallest_untrackable_value = ((int64_t) sub_bucket_count) << unit_magnitude; - int32_t buckets_needed = 1; - while (smallest_untrackable_value <= value) - { - if (smallest_untrackable_value > INT64_MAX / 2) - { - return buckets_needed + 1; - } - smallest_untrackable_value <<= 1; - buckets_needed++; - } - - return buckets_needed; -} - -// ## ## ######## ## ## ####### ######## ## ## -// ### ### ## ### ### ## ## ## ## ## ## -// #### #### ## #### #### ## ## ## ## #### -// ## ### ## ###### ## ### ## ## ## ######## ## -// ## ## ## ## ## ## ## ## ## ## -// ## ## ## ## ## ## ## ## ## ## -// ## ## ######## ## ## ####### ## ## ## - -int hdr_calculate_bucket_config( - int64_t lowest_trackable_value, - int64_t highest_trackable_value, - int significant_figures, - struct hdr_histogram_bucket_config* cfg) -{ - if (lowest_trackable_value < 1 || - significant_figures < 1 || 5 < significant_figures) - { - return EINVAL; - } - else if (lowest_trackable_value * 2 > highest_trackable_value) - { - return EINVAL; - } - - cfg->lowest_trackable_value = lowest_trackable_value; - cfg->significant_figures = significant_figures; - cfg->highest_trackable_value = highest_trackable_value; - - int64_t largest_value_with_single_unit_resolution = 2 * power(10, significant_figures); - int32_t sub_bucket_count_magnitude = (int32_t) ceil(log((double)largest_value_with_single_unit_resolution) / log(2)); - cfg->sub_bucket_half_count_magnitude = ((sub_bucket_count_magnitude > 1) ? sub_bucket_count_magnitude : 1) - 1; - - cfg->unit_magnitude = (int32_t) floor(log((double)lowest_trackable_value) / log(2)); - - cfg->sub_bucket_count = (int32_t) pow(2, (cfg->sub_bucket_half_count_magnitude + 1)); - cfg->sub_bucket_half_count = cfg->sub_bucket_count / 2; - cfg->sub_bucket_mask = ((int64_t) cfg->sub_bucket_count - 1) << cfg->unit_magnitude; - - // determine exponent range needed to support the trackable value with no overflow: - cfg->bucket_count = buckets_needed_to_cover_value(highest_trackable_value, cfg->sub_bucket_count, (int32_t)cfg->unit_magnitude); - cfg->counts_len = (cfg->bucket_count + 1) * (cfg->sub_bucket_count / 2); - - return 0; -} - -void hdr_init_preallocated(struct hdr_histogram* h, struct hdr_histogram_bucket_config* cfg) -{ - h->lowest_trackable_value = cfg->lowest_trackable_value; - h->highest_trackable_value = cfg->highest_trackable_value; - h->unit_magnitude = (int32_t)cfg->unit_magnitude; - h->significant_figures = (int32_t)cfg->significant_figures; - h->sub_bucket_half_count_magnitude = cfg->sub_bucket_half_count_magnitude; - h->sub_bucket_half_count = cfg->sub_bucket_half_count; - h->sub_bucket_mask = cfg->sub_bucket_mask; - h->sub_bucket_count = cfg->sub_bucket_count; - h->min_value = INT64_MAX; - h->max_value = 0; - h->normalizing_index_offset = 0; - h->conversion_ratio = 1.0; - h->bucket_count = cfg->bucket_count; - h->counts_len = cfg->counts_len; - h->total_count = 0; -} - -int hdr_init( - int64_t lowest_trackable_value, - int64_t highest_trackable_value, - int significant_figures, - struct hdr_histogram** result) -{ - struct hdr_histogram_bucket_config cfg; - - int r = hdr_calculate_bucket_config(lowest_trackable_value, highest_trackable_value, significant_figures, &cfg); - if (r) - { - return r; - } - - size_t histogram_size = sizeof(struct hdr_histogram) + cfg.counts_len * sizeof(int64_t); - struct hdr_histogram* histogram = malloc(histogram_size); - - if (!histogram) - { - return ENOMEM; - } - - // memset will ensure that all of the function pointers are null. - memset((void*) histogram, 0, histogram_size); - - hdr_init_preallocated(histogram, &cfg); - *result = histogram; - - return 0; -} - - -int hdr_alloc(int64_t highest_trackable_value, int significant_figures, struct hdr_histogram** result) -{ - return hdr_init(1, highest_trackable_value, significant_figures, result); -} - -// reset a histogram to zero. -void hdr_reset(struct hdr_histogram *h) -{ - h->total_count=0; - h->min_value = INT64_MAX; - h->max_value = 0; - memset((void *) &h->counts, 0, (sizeof(int64_t) * h->counts_len)); - return; -} - -size_t hdr_get_memory_size(struct hdr_histogram *h) -{ - return sizeof(struct hdr_histogram) + h->counts_len * sizeof(int64_t); -} - -// ## ## ######## ######## ### ######## ######## ###### -// ## ## ## ## ## ## ## ## ## ## ## ## -// ## ## ## ## ## ## ## ## ## ## ## -// ## ## ######## ## ## ## ## ## ###### ###### -// ## ## ## ## ## ######### ## ## ## -// ## ## ## ## ## ## ## ## ## ## ## -// ####### ## ######## ## ## ## ######## ###### - - -bool hdr_record_value(struct hdr_histogram* h, int64_t value) -{ - return hdr_record_values(h, value, 1); -} - -bool hdr_record_values(struct hdr_histogram* h, int64_t value, int64_t count) -{ - if (value < 0) - { - return false; - } - - int32_t counts_index = counts_index_for(h, value); - - if (counts_index < 0 || h->counts_len <= counts_index) - { - return false; - } - - counts_inc_normalised(h, counts_index, count); - update_min_max(h, value); - - return true; -} - -bool hdr_record_corrected_value(struct hdr_histogram* h, int64_t value, int64_t expected_interval) -{ - return hdr_record_corrected_values(h, value, 1, expected_interval); -} - - -bool hdr_record_corrected_values(struct hdr_histogram* h, int64_t value, int64_t count, int64_t expected_interval) -{ - if (!hdr_record_values(h, value, count)) - { - return false; - } - - if (expected_interval <= 0 || value <= expected_interval) - { - return true; - } - - int64_t missing_value = value - expected_interval; - for (; missing_value >= expected_interval; missing_value -= expected_interval) - { - if (!hdr_record_values(h, missing_value, count)) - { - return false; - } - } - - return true; -} - -int64_t hdr_add(struct hdr_histogram* h, const struct hdr_histogram* from) -{ - struct hdr_iter iter; - hdr_iter_recorded_init(&iter, from); - int64_t dropped = 0; - - while (hdr_iter_next(&iter)) - { - int64_t value = iter.value; - int64_t count = iter.count; - - if (!hdr_record_values(h, value, count)) - { - dropped += count; - } - } - - return dropped; -} - -int64_t hdr_add_while_correcting_for_coordinated_omission( - struct hdr_histogram* h, struct hdr_histogram* from, int64_t expected_interval) -{ - struct hdr_iter iter; - hdr_iter_recorded_init(&iter, from); - int64_t dropped = 0; - - while (hdr_iter_next(&iter)) - { - int64_t value = iter.value; - int64_t count = iter.count; - - if (!hdr_record_corrected_values(h, value, count, expected_interval)) - { - dropped += count; - } - } - - return dropped; -} - - - -// ## ## ### ## ## ## ######## ###### -// ## ## ## ## ## ## ## ## ## ## -// ## ## ## ## ## ## ## ## ## -// ## ## ## ## ## ## ## ###### ###### -// ## ## ######### ## ## ## ## ## -// ## ## ## ## ## ## ## ## ## ## -// ### ## ## ######## ####### ######## ###### - - -int64_t hdr_max(const struct hdr_histogram* h) -{ - if (0 == h->max_value) - { - return 0; - } - - return highest_equivalent_value(h, h->max_value); -} - -int64_t hdr_min(const struct hdr_histogram* h) -{ - if (0 < hdr_count_at_index(h, 0)) - { - return 0; - } - - return non_zero_min(h); -} - -int64_t hdr_value_at_percentile(const struct hdr_histogram* h, double percentile) -{ - struct hdr_iter iter; - hdr_iter_init(&iter, h); - - double requested_percentile = percentile < 100.0 ? percentile : 100.0; - int64_t count_at_percentile = - (int64_t) (((requested_percentile / 100) * h->total_count) + 0.5); - count_at_percentile = count_at_percentile > 1 ? count_at_percentile : 1; - int64_t total = 0; - - while (hdr_iter_next(&iter)) - { - total += iter.count; - - if (total >= count_at_percentile) - { - int64_t value_from_index = iter.value; - return highest_equivalent_value(h, value_from_index); - } - } - - return 0; -} - -double hdr_mean(const struct hdr_histogram* h) -{ - struct hdr_iter iter; - int64_t total = 0; - - hdr_iter_init(&iter, h); - - while (hdr_iter_next(&iter)) - { - if (0 != iter.count) - { - total += iter.count * hdr_median_equivalent_value(h, iter.value); - } - } - - return (total * 1.0) / h->total_count; -} - -double hdr_stddev(const struct hdr_histogram* h) -{ - double mean = hdr_mean(h); - double geometric_dev_total = 0.0; - - struct hdr_iter iter; - hdr_iter_init(&iter, h); - - while (hdr_iter_next(&iter)) - { - if (0 != iter.count) - { - double dev = (hdr_median_equivalent_value(h, iter.value) * 1.0) - mean; - geometric_dev_total += (dev * dev) * iter.count; - } - } - - return sqrt(geometric_dev_total / h->total_count); -} - -bool hdr_values_are_equivalent(const struct hdr_histogram* h, int64_t a, int64_t b) -{ - return lowest_equivalent_value(h, a) == lowest_equivalent_value(h, b); -} - -int64_t hdr_lowest_equivalent_value(const struct hdr_histogram* h, int64_t value) -{ - return lowest_equivalent_value(h, value); -} - -int64_t hdr_count_at_value(const struct hdr_histogram* h, int64_t value) -{ - return counts_get_normalised(h, counts_index_for(h, value)); -} - -int64_t hdr_count_at_index(const struct hdr_histogram* h, int32_t index) -{ - return counts_get_normalised(h, index); -} - - -// #### ######## ######## ######## ### ######## ####### ######## ###### -// ## ## ## ## ## ## ## ## ## ## ## ## ## ## -// ## ## ## ## ## ## ## ## ## ## ## ## ## -// ## ## ###### ######## ## ## ## ## ## ######## ###### -// ## ## ## ## ## ######### ## ## ## ## ## ## -// ## ## ## ## ## ## ## ## ## ## ## ## ## ## -// #### ## ######## ## ## ## ## ## ####### ## ## ###### - - -static bool has_buckets(struct hdr_iter* iter) -{ - return iter->counts_index < iter->h->counts_len; -} - -static bool has_next(struct hdr_iter* iter) -{ - return iter->cumulative_count < iter->total_count; -} - -static bool move_next(struct hdr_iter* iter) -{ - iter->counts_index++; - - if (!has_buckets(iter)) - { - return false; - } - - iter->count = counts_get_normalised(iter->h, iter->counts_index); - iter->cumulative_count += iter->count; - - iter->value = hdr_value_at_index(iter->h, iter->counts_index); - iter->highest_equivalent_value = highest_equivalent_value(iter->h, iter->value); - iter->lowest_equivalent_value = lowest_equivalent_value(iter->h, iter->value); - iter->median_equivalent_value = hdr_median_equivalent_value(iter->h, iter->value); - - return true; -} - -static int64_t peek_next_value_from_index(struct hdr_iter* iter) -{ - return hdr_value_at_index(iter->h, iter->counts_index + 1); -} - -static bool next_value_greater_than_reporting_level_upper_bound( - struct hdr_iter *iter, int64_t reporting_level_upper_bound) -{ - if (iter->counts_index >= iter->h->counts_len) - { - return false; - } - - return peek_next_value_from_index(iter) > reporting_level_upper_bound; -} - -static bool _basic_iter_next(struct hdr_iter *iter) -{ - if (!has_next(iter)) - { - return false; - } - - move_next(iter); - - return true; -} - -static void _update_iterated_values(struct hdr_iter* iter, int64_t new_value_iterated_to) -{ - iter->value_iterated_from = iter->value_iterated_to; - iter->value_iterated_to = new_value_iterated_to; -} - -static bool _all_values_iter_next(struct hdr_iter* iter) -{ - bool result = move_next(iter); - - if (result) - { - _update_iterated_values(iter, iter->value); - } - - return result; -} - -void hdr_iter_init(struct hdr_iter* iter, const struct hdr_histogram* h) -{ - iter->h = h; - - iter->counts_index = -1; - iter->total_count = h->total_count; - iter->count = 0; - iter->cumulative_count = 0; - iter->value = 0; - iter->highest_equivalent_value = 0; - iter->value_iterated_from = 0; - iter->value_iterated_to = 0; - - iter->_next_fp = _all_values_iter_next; -} - -bool hdr_iter_next(struct hdr_iter* iter) -{ - return iter->_next_fp(iter); -} - -// ######## ######## ######## ###### ######## ## ## ######## #### ## ######## ###### -// ## ## ## ## ## ## ## ## ### ## ## ## ## ## ## ## -// ## ## ## ## ## ## ## #### ## ## ## ## ## ## -// ######## ###### ######## ## ###### ## ## ## ## ## ## ###### ###### -// ## ## ## ## ## ## ## #### ## ## ## ## ## -// ## ## ## ## ## ## ## ## ### ## ## ## ## ## ## -// ## ######## ## ## ###### ######## ## ## ## #### ######## ######## ###### - -static bool _percentile_iter_next(struct hdr_iter* iter) -{ - struct hdr_iter_percentiles* percentiles = &iter->specifics.percentiles; - - if (!has_next(iter)) - { - if (percentiles->seen_last_value) - { - return false; - } - - percentiles->seen_last_value = true; - percentiles->percentile = 100.0; - - return true; - } - - if (iter->counts_index == -1 && !_basic_iter_next(iter)) - { - return false; - } - - do - { - double current_percentile = (100.0 * (double) iter->cumulative_count) / iter->h->total_count; - if (iter->count != 0 && - percentiles->percentile_to_iterate_to <= current_percentile) - { - _update_iterated_values(iter, highest_equivalent_value(iter->h, iter->value)); - - percentiles->percentile = percentiles->percentile_to_iterate_to; - int64_t temp = (int64_t)(log(100 / (100.0 - (percentiles->percentile_to_iterate_to))) / log(2)) + 1; - int64_t half_distance = (int64_t) pow(2, (double) temp); - int64_t percentile_reporting_ticks = percentiles->ticks_per_half_distance * half_distance; - percentiles->percentile_to_iterate_to += 100.0 / percentile_reporting_ticks; - - return true; - } - } - while (_basic_iter_next(iter)); - - return true; -} - -void hdr_iter_percentile_init(struct hdr_iter* iter, const struct hdr_histogram* h, int32_t ticks_per_half_distance) -{ - iter->h = h; - - hdr_iter_init(iter, h); - - iter->specifics.percentiles.seen_last_value = false; - iter->specifics.percentiles.ticks_per_half_distance = ticks_per_half_distance; - iter->specifics.percentiles.percentile_to_iterate_to = 0.0; - iter->specifics.percentiles.percentile = 0.0; - - iter->_next_fp = _percentile_iter_next; -} - -static void format_line_string(char* str, size_t len, int significant_figures, format_type format) -{ -#if defined(_MSC_VER) -#define snprintf _snprintf -#pragma warning(push) -#pragma warning(disable: 4996) -#endif - const char* format_str = "%s%d%s"; - - switch (format) - { - case CSV: - snprintf(str, len, format_str, "%.", significant_figures, "f,%f,%d,%.2f\n"); - break; - case CLASSIC: - snprintf(str, len, format_str, "%12.", significant_figures, "f %12f %12d %12.2f\n"); - break; - default: - snprintf(str, len, format_str, "%12.", significant_figures, "f %12f %12d %12.2f\n"); - } -#if defined(_MSC_VER) -#undef snprintf -#pragma warning(pop) -#endif -} - - -// ######## ######## ###### ####### ######## ######## ######## ######## -// ## ## ## ## ## ## ## ## ## ## ## ## ## ## -// ## ## ## ## ## ## ## ## ## ## ## ## ## -// ######## ###### ## ## ## ######## ## ## ###### ## ## -// ## ## ## ## ## ## ## ## ## ## ## ## ## -// ## ## ## ## ## ## ## ## ## ## ## ## ## ## -// ## ## ######## ###### ####### ## ## ######## ######## ######## - - -static bool _recorded_iter_next(struct hdr_iter* iter) -{ - while (_basic_iter_next(iter)) - { - if (iter->count != 0) - { - _update_iterated_values(iter, iter->value); - - iter->specifics.recorded.count_added_in_this_iteration_step = iter->count; - return true; - } - } - - return false; -} - -void hdr_iter_recorded_init(struct hdr_iter* iter, const struct hdr_histogram* h) -{ - hdr_iter_init(iter, h); - - iter->specifics.recorded.count_added_in_this_iteration_step = 0; - - iter->_next_fp = _recorded_iter_next; -} - -// ## #### ## ## ######## ### ######## -// ## ## ### ## ## ## ## ## ## -// ## ## #### ## ## ## ## ## ## -// ## ## ## ## ## ###### ## ## ######## -// ## ## ## #### ## ######### ## ## -// ## ## ## ### ## ## ## ## ## -// ######## #### ## ## ######## ## ## ## ## - - -static bool _iter_linear_next(struct hdr_iter* iter) -{ - struct hdr_iter_linear* linear = &iter->specifics.linear; - - linear->count_added_in_this_iteration_step = 0; - - if (has_next(iter) || - next_value_greater_than_reporting_level_upper_bound( - iter, linear->next_value_reporting_level_lowest_equivalent)) - { - do - { - if (iter->value >= linear->next_value_reporting_level_lowest_equivalent) - { - _update_iterated_values(iter, linear->next_value_reporting_level); - - linear->next_value_reporting_level += linear->value_units_per_bucket; - linear->next_value_reporting_level_lowest_equivalent = - lowest_equivalent_value(iter->h, linear->next_value_reporting_level); - - return true; - } - - if (!move_next(iter)) - { - return true; - } - - linear->count_added_in_this_iteration_step += iter->count; - } - while (true); - } - - return false; -} - - -void hdr_iter_linear_init(struct hdr_iter* iter, const struct hdr_histogram* h, int64_t value_units_per_bucket) -{ - hdr_iter_init(iter, h); - - iter->specifics.linear.count_added_in_this_iteration_step = 0; - iter->specifics.linear.value_units_per_bucket = value_units_per_bucket; - iter->specifics.linear.next_value_reporting_level = value_units_per_bucket; - iter->specifics.linear.next_value_reporting_level_lowest_equivalent = lowest_equivalent_value(h, value_units_per_bucket); - - iter->_next_fp = _iter_linear_next; -} - -// ## ####### ###### ### ######## #### ######## ## ## ## ## #### ###### -// ## ## ## ## ## ## ## ## ## ## ## ## ## ### ### ## ## ## -// ## ## ## ## ## ## ## ## ## ## ## ## #### #### ## ## -// ## ## ## ## #### ## ## ######## ## ## ######### ## ### ## ## ## -// ## ## ## ## ## ######### ## ## ## ## ## ## ## ## ## ## -// ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## -// ######## ####### ###### ## ## ## ## #### ## ## ## ## ## #### ###### - -static bool _log_iter_next(struct hdr_iter *iter) -{ - struct hdr_iter_log* logarithmic = &iter->specifics.log; - - logarithmic->count_added_in_this_iteration_step = 0; - - if (has_next(iter) || - next_value_greater_than_reporting_level_upper_bound( - iter, logarithmic->next_value_reporting_level_lowest_equivalent)) - { - do - { - if (iter->value >= logarithmic->next_value_reporting_level_lowest_equivalent) - { - _update_iterated_values(iter, logarithmic->next_value_reporting_level); - - logarithmic->next_value_reporting_level *= (int64_t)logarithmic->log_base; - logarithmic->next_value_reporting_level_lowest_equivalent = lowest_equivalent_value(iter->h, logarithmic->next_value_reporting_level); - - return true; - } - - if (!move_next(iter)) - { - return true; - } - - logarithmic->count_added_in_this_iteration_step += iter->count; - } - while (true); - } - - return false; -} - -void hdr_iter_log_init( - struct hdr_iter* iter, - const struct hdr_histogram* h, - int64_t value_units_first_bucket, - double log_base) -{ - hdr_iter_init(iter, h); - iter->specifics.log.count_added_in_this_iteration_step = 0; - iter->specifics.log.log_base = log_base; - iter->specifics.log.next_value_reporting_level = value_units_first_bucket; - iter->specifics.log.next_value_reporting_level_lowest_equivalent = lowest_equivalent_value(h, value_units_first_bucket); - - iter->_next_fp = _log_iter_next; -} - -// Printing. - -static const char* format_head_string(format_type format) -{ - switch (format) - { - case CSV: - return "%s,%s,%s,%s\n"; - case CLASSIC: - return "%12s %12s %12s %12s\n\n"; - default: - return "%12s %12s %12s %12s\n\n"; - } -} - -static const char CLASSIC_FOOTER[] = - "#[Mean = %12.3f, StdDeviation = %12.3f]\n" - "#[Max = %12.3f, Total count = %12" PRIu64 "]\n" - "#[Buckets = %12d, SubBuckets = %12d]\n"; - -int hdr_percentiles_print( - struct hdr_histogram* h, FILE* stream, int32_t ticks_per_half_distance, - double value_scale, format_type format) -{ - char line_format[25]; - format_line_string(line_format, 25, h->significant_figures, format); - const char* head_format = format_head_string(format); - int rc = 0; - - struct hdr_iter iter; - hdr_iter_percentile_init(&iter, h, ticks_per_half_distance); - - if (fprintf( - stream, head_format, - "Value", "Percentile", "TotalCount", "1/(1-Percentile)") < 0) - { - rc = EIO; - goto cleanup; - } - - struct hdr_iter_percentiles * percentiles = &iter.specifics.percentiles; - while (hdr_iter_next(&iter)) - { - double value = iter.highest_equivalent_value / value_scale; - double percentile = percentiles->percentile / 100.0; - int64_t total_count = iter.cumulative_count; - double inverted_percentile = (1.0 / (1.0 - percentile)); - - if (fprintf( - stream, line_format, value, percentile, total_count, inverted_percentile) < 0) - { - rc = EIO; - goto cleanup; - } - } - - if (CLASSIC == format) - { - double mean = hdr_mean(h) / value_scale; - double stddev = hdr_stddev(h) / value_scale; - double max = hdr_max(h) / value_scale; - - if (fprintf( - stream, CLASSIC_FOOTER, mean, stddev, max, - h->total_count, h->bucket_count, h->sub_bucket_count) < 0) - { - rc = EIO; - goto cleanup; - } - } - - cleanup: - return rc; -} +/** + * hdr_histogram.c + * Written by Michael Barker and released to the public domain, + * as explained at http://creativecommons.org/publicdomain/zero/1.0/ + */ + +#include <stdlib.h> +#include <stdbool.h> +#include <math.h> +#include <assert.h> +#include <stdio.h> +#include <string.h> +#include <stdint.h> +#include <errno.h> +#include <inttypes.h> + +#include "hdr_histogram.h" +#include "hdr_tests.h" + +// ###### ####### ## ## ## ## ######## ###### +// ## ## ## ## ## ## ### ## ## ## ## +// ## ## ## ## ## #### ## ## ## +// ## ## ## ## ## ## ## ## ## ###### +// ## ## ## ## ## ## #### ## ## +// ## ## ## ## ## ## ## ### ## ## ## +// ###### ####### ####### ## ## ## ###### + +static int32_t normalize_index(const struct hdr_histogram* h, int32_t index) +{ + if (h->normalizing_index_offset == 0) + { + return index; + } + + int32_t normalized_index = index - h->normalizing_index_offset; + int32_t adjustment = 0; + + if (normalized_index < 0) + { + adjustment = h->counts_len; + } + else if (normalized_index >= h->counts_len) + { + adjustment = -h->counts_len; + } + + return normalized_index + adjustment; +} + +static int64_t counts_get_direct(const struct hdr_histogram* h, int32_t index) +{ + return h->counts[index]; +} + +static int64_t counts_get_normalised(const struct hdr_histogram* h, int32_t index) +{ + return counts_get_direct(h, normalize_index(h, index)); +} + +static void counts_inc_normalised( + struct hdr_histogram* h, int32_t index, int64_t value) +{ + int32_t normalised_index = normalize_index(h, index); + h->counts[normalised_index] += value; + h->total_count += value; +} + +static void update_min_max(struct hdr_histogram* h, int64_t value) +{ + h->min_value = (value < h->min_value && value != 0) ? value : h->min_value; + h->max_value = (value > h->max_value) ? value : h->max_value; +} + +// ## ## ######## #### ## #### ######## ## ## +// ## ## ## ## ## ## ## ## ## +// ## ## ## ## ## ## ## #### +// ## ## ## ## ## ## ## ## +// ## ## ## ## ## ## ## ## +// ## ## ## ## ## ## ## ## +// ####### ## #### ######## #### ## ## + +static int64_t power(int64_t base, int64_t exp) +{ + int64_t result = 1; + while(exp) + { + result *= base; exp--; + } + return result; +} + +#if defined(_MSC_VER) +#pragma intrinsic(_BitScanReverse64) +#endif + +static int32_t get_bucket_index(const struct hdr_histogram* h, int64_t value) +{ +#if defined(_MSC_VER) + uint32_t leading_zero = 0; + _BitScanReverse64(&leading_zero, value | h->sub_bucket_mask); + int32_t pow2ceiling = 64 - (63 - leading_zero); // smallest power of 2 containing value +#else + int32_t pow2ceiling = 64 - __builtin_clzll(value | h->sub_bucket_mask); // smallest power of 2 containing value +#endif + return pow2ceiling - h->unit_magnitude - (h->sub_bucket_half_count_magnitude + 1); +} + +static int32_t get_sub_bucket_index(int64_t value, int32_t bucket_index, int32_t unit_magnitude) +{ + return (int32_t)(value >> (bucket_index + unit_magnitude)); +} + +static int32_t counts_index(const struct hdr_histogram* h, int32_t bucket_index, int32_t sub_bucket_index) +{ + // Calculate the index for the first entry in the bucket: + // (The following is the equivalent of ((bucket_index + 1) * subBucketHalfCount) ): + int32_t bucket_base_index = (bucket_index + 1) << h->sub_bucket_half_count_magnitude; + // Calculate the offset in the bucket: + int32_t offset_in_bucket = sub_bucket_index - h->sub_bucket_half_count; + // The following is the equivalent of ((sub_bucket_index - subBucketHalfCount) + bucketBaseIndex; + return bucket_base_index + offset_in_bucket; +} + +static int64_t value_from_index(int32_t bucket_index, int32_t sub_bucket_index, int32_t unit_magnitude) +{ + return ((int64_t) sub_bucket_index) << (bucket_index + unit_magnitude); +} + +int32_t counts_index_for(const struct hdr_histogram* h, int64_t value) +{ + int32_t bucket_index = get_bucket_index(h, value); + int32_t sub_bucket_index = get_sub_bucket_index(value, bucket_index, h->unit_magnitude); + + return counts_index(h, bucket_index, sub_bucket_index); +} + +int64_t hdr_value_at_index(const struct hdr_histogram *h, int32_t index) +{ + int32_t bucket_index = (index >> h->sub_bucket_half_count_magnitude) - 1; + int32_t sub_bucket_index = (index & (h->sub_bucket_half_count - 1)) + h->sub_bucket_half_count; + + if (bucket_index < 0) + { + sub_bucket_index -= h->sub_bucket_half_count; + bucket_index = 0; + } + + return value_from_index(bucket_index, sub_bucket_index, h->unit_magnitude); +} + +int64_t hdr_size_of_equivalent_value_range(const struct hdr_histogram* h, int64_t value) +{ + int32_t bucket_index = get_bucket_index(h, value); + int32_t sub_bucket_index = get_sub_bucket_index(value, bucket_index, h->unit_magnitude); + int32_t adjusted_bucket = (sub_bucket_index >= h->sub_bucket_count) ? (bucket_index + 1) : bucket_index; + return INT64_C(1) << (h->unit_magnitude + adjusted_bucket); +} + +static int64_t lowest_equivalent_value(const struct hdr_histogram* h, int64_t value) +{ + int32_t bucket_index = get_bucket_index(h, value); + int32_t sub_bucket_index = get_sub_bucket_index(value, bucket_index, h->unit_magnitude); + return value_from_index(bucket_index, sub_bucket_index, h->unit_magnitude); +} + +int64_t hdr_next_non_equivalent_value(const struct hdr_histogram *h, int64_t value) +{ + return lowest_equivalent_value(h, value) + hdr_size_of_equivalent_value_range(h, value); +} + +static int64_t highest_equivalent_value(const struct hdr_histogram* h, int64_t value) +{ + return hdr_next_non_equivalent_value(h, value) - 1; +} + +int64_t hdr_median_equivalent_value(const struct hdr_histogram *h, int64_t value) +{ + return lowest_equivalent_value(h, value) + (hdr_size_of_equivalent_value_range(h, value) >> 1); +} + +static int64_t non_zero_min(const struct hdr_histogram* h) +{ + if (INT64_MAX == h->min_value) + { + return INT64_MAX; + } + + return lowest_equivalent_value(h, h->min_value); +} + +void hdr_reset_internal_counters(struct hdr_histogram* h) +{ + int min_non_zero_index = -1; + int max_index = -1; + int64_t observed_total_count = 0; + int i; + + for (i = 0; i < h->counts_len; i++) + { + int64_t count_at_index; + + if ((count_at_index = counts_get_direct(h, i)) > 0) + { + observed_total_count += count_at_index; + max_index = i; + if (min_non_zero_index == -1 && i != 0) + { + min_non_zero_index = i; + } + } + } + + if (max_index == -1) + { + h->max_value = 0; + } + else + { + int64_t max_value = hdr_value_at_index(h, max_index); + h->max_value = highest_equivalent_value(h, max_value); + } + + if (min_non_zero_index == -1) + { + h->min_value = INT64_MAX; + } + else + { + h->min_value = hdr_value_at_index(h, min_non_zero_index); + } + + h->total_count = observed_total_count; +} + +static int32_t buckets_needed_to_cover_value(int64_t value, int32_t sub_bucket_count, int32_t unit_magnitude) +{ + int64_t smallest_untrackable_value = ((int64_t) sub_bucket_count) << unit_magnitude; + int32_t buckets_needed = 1; + while (smallest_untrackable_value <= value) + { + if (smallest_untrackable_value > INT64_MAX / 2) + { + return buckets_needed + 1; + } + smallest_untrackable_value <<= 1; + buckets_needed++; + } + + return buckets_needed; +} + +// ## ## ######## ## ## ####### ######## ## ## +// ### ### ## ### ### ## ## ## ## ## ## +// #### #### ## #### #### ## ## ## ## #### +// ## ### ## ###### ## ### ## ## ## ######## ## +// ## ## ## ## ## ## ## ## ## ## +// ## ## ## ## ## ## ## ## ## ## +// ## ## ######## ## ## ####### ## ## ## + +int hdr_calculate_bucket_config( + int64_t lowest_trackable_value, + int64_t highest_trackable_value, + int significant_figures, + struct hdr_histogram_bucket_config* cfg) +{ + if (lowest_trackable_value < 1 || + significant_figures < 1 || 5 < significant_figures) + { + return EINVAL; + } + else if (lowest_trackable_value * 2 > highest_trackable_value) + { + return EINVAL; + } + + cfg->lowest_trackable_value = lowest_trackable_value; + cfg->significant_figures = significant_figures; + cfg->highest_trackable_value = highest_trackable_value; + + int64_t largest_value_with_single_unit_resolution = 2 * power(10, significant_figures); + int32_t sub_bucket_count_magnitude = (int32_t) ceil(log((double)largest_value_with_single_unit_resolution) / log(2)); + cfg->sub_bucket_half_count_magnitude = ((sub_bucket_count_magnitude > 1) ? sub_bucket_count_magnitude : 1) - 1; + + cfg->unit_magnitude = (int32_t) floor(log((double)lowest_trackable_value) / log(2)); + + cfg->sub_bucket_count = (int32_t) pow(2, (cfg->sub_bucket_half_count_magnitude + 1)); + cfg->sub_bucket_half_count = cfg->sub_bucket_count / 2; + cfg->sub_bucket_mask = ((int64_t) cfg->sub_bucket_count - 1) << cfg->unit_magnitude; + + // determine exponent range needed to support the trackable value with no overflow: + cfg->bucket_count = buckets_needed_to_cover_value(highest_trackable_value, cfg->sub_bucket_count, (int32_t)cfg->unit_magnitude); + cfg->counts_len = (cfg->bucket_count + 1) * (cfg->sub_bucket_count / 2); + + return 0; +} + +void hdr_init_preallocated(struct hdr_histogram* h, struct hdr_histogram_bucket_config* cfg) +{ + h->lowest_trackable_value = cfg->lowest_trackable_value; + h->highest_trackable_value = cfg->highest_trackable_value; + h->unit_magnitude = (int32_t)cfg->unit_magnitude; + h->significant_figures = (int32_t)cfg->significant_figures; + h->sub_bucket_half_count_magnitude = cfg->sub_bucket_half_count_magnitude; + h->sub_bucket_half_count = cfg->sub_bucket_half_count; + h->sub_bucket_mask = cfg->sub_bucket_mask; + h->sub_bucket_count = cfg->sub_bucket_count; + h->min_value = INT64_MAX; + h->max_value = 0; + h->normalizing_index_offset = 0; + h->conversion_ratio = 1.0; + h->bucket_count = cfg->bucket_count; + h->counts_len = cfg->counts_len; + h->total_count = 0; +} + +int hdr_init( + int64_t lowest_trackable_value, + int64_t highest_trackable_value, + int significant_figures, + struct hdr_histogram** result) +{ + struct hdr_histogram_bucket_config cfg; + + int r = hdr_calculate_bucket_config(lowest_trackable_value, highest_trackable_value, significant_figures, &cfg); + if (r) + { + return r; + } + + size_t histogram_size = sizeof(struct hdr_histogram) + cfg.counts_len * sizeof(int64_t); + struct hdr_histogram* histogram = malloc(histogram_size); + + if (!histogram) + { + return ENOMEM; + } + + // memset will ensure that all of the function pointers are null. + memset((void*) histogram, 0, histogram_size); + + hdr_init_preallocated(histogram, &cfg); + *result = histogram; + + return 0; +} + + +int hdr_alloc(int64_t highest_trackable_value, int significant_figures, struct hdr_histogram** result) +{ + return hdr_init(1, highest_trackable_value, significant_figures, result); +} + +// reset a histogram to zero. +void hdr_reset(struct hdr_histogram *h) +{ + h->total_count=0; + h->min_value = INT64_MAX; + h->max_value = 0; + memset((void *) &h->counts, 0, (sizeof(int64_t) * h->counts_len)); + return; +} + +size_t hdr_get_memory_size(struct hdr_histogram *h) +{ + return sizeof(struct hdr_histogram) + h->counts_len * sizeof(int64_t); +} + +// ## ## ######## ######## ### ######## ######## ###### +// ## ## ## ## ## ## ## ## ## ## ## ## +// ## ## ## ## ## ## ## ## ## ## ## +// ## ## ######## ## ## ## ## ## ###### ###### +// ## ## ## ## ## ######### ## ## ## +// ## ## ## ## ## ## ## ## ## ## ## +// ####### ## ######## ## ## ## ######## ###### + + +bool hdr_record_value(struct hdr_histogram* h, int64_t value) +{ + return hdr_record_values(h, value, 1); +} + +bool hdr_record_values(struct hdr_histogram* h, int64_t value, int64_t count) +{ + if (value < 0) + { + return false; + } + + int32_t counts_index = counts_index_for(h, value); + + if (counts_index < 0 || h->counts_len <= counts_index) + { + return false; + } + + counts_inc_normalised(h, counts_index, count); + update_min_max(h, value); + + return true; +} + +bool hdr_record_corrected_value(struct hdr_histogram* h, int64_t value, int64_t expected_interval) +{ + return hdr_record_corrected_values(h, value, 1, expected_interval); +} + + +bool hdr_record_corrected_values(struct hdr_histogram* h, int64_t value, int64_t count, int64_t expected_interval) +{ + if (!hdr_record_values(h, value, count)) + { + return false; + } + + if (expected_interval <= 0 || value <= expected_interval) + { + return true; + } + + int64_t missing_value = value - expected_interval; + for (; missing_value >= expected_interval; missing_value -= expected_interval) + { + if (!hdr_record_values(h, missing_value, count)) + { + return false; + } + } + + return true; +} + +int64_t hdr_add(struct hdr_histogram* h, const struct hdr_histogram* from) +{ + struct hdr_iter iter; + hdr_iter_recorded_init(&iter, from); + int64_t dropped = 0; + + while (hdr_iter_next(&iter)) + { + int64_t value = iter.value; + int64_t count = iter.count; + + if (!hdr_record_values(h, value, count)) + { + dropped += count; + } + } + + return dropped; +} + +int64_t hdr_add_while_correcting_for_coordinated_omission( + struct hdr_histogram* h, struct hdr_histogram* from, int64_t expected_interval) +{ + struct hdr_iter iter; + hdr_iter_recorded_init(&iter, from); + int64_t dropped = 0; + + while (hdr_iter_next(&iter)) + { + int64_t value = iter.value; + int64_t count = iter.count; + + if (!hdr_record_corrected_values(h, value, count, expected_interval)) + { + dropped += count; + } + } + + return dropped; +} + + + +// ## ## ### ## ## ## ######## ###### +// ## ## ## ## ## ## ## ## ## ## +// ## ## ## ## ## ## ## ## ## +// ## ## ## ## ## ## ## ###### ###### +// ## ## ######### ## ## ## ## ## +// ## ## ## ## ## ## ## ## ## ## +// ### ## ## ######## ####### ######## ###### + + +int64_t hdr_max(const struct hdr_histogram* h) +{ + if (0 == h->max_value) + { + return 0; + } + + return highest_equivalent_value(h, h->max_value); +} + +int64_t hdr_min(const struct hdr_histogram* h) +{ + if (0 < hdr_count_at_index(h, 0)) + { + return 0; + } + + return non_zero_min(h); +} + +int64_t hdr_value_at_percentile(const struct hdr_histogram* h, double percentile) +{ + struct hdr_iter iter; + hdr_iter_init(&iter, h); + + double requested_percentile = percentile < 100.0 ? percentile : 100.0; + int64_t count_at_percentile = + (int64_t) (((requested_percentile / 100) * h->total_count) + 0.5); + count_at_percentile = count_at_percentile > 1 ? count_at_percentile : 1; + int64_t total = 0; + + while (hdr_iter_next(&iter)) + { + total += iter.count; + + if (total >= count_at_percentile) + { + int64_t value_from_index = iter.value; + return highest_equivalent_value(h, value_from_index); + } + } + + return 0; +} + +double hdr_mean(const struct hdr_histogram* h) +{ + struct hdr_iter iter; + int64_t total = 0; + + hdr_iter_init(&iter, h); + + while (hdr_iter_next(&iter)) + { + if (0 != iter.count) + { + total += iter.count * hdr_median_equivalent_value(h, iter.value); + } + } + + return (total * 1.0) / h->total_count; +} + +double hdr_stddev(const struct hdr_histogram* h) +{ + double mean = hdr_mean(h); + double geometric_dev_total = 0.0; + + struct hdr_iter iter; + hdr_iter_init(&iter, h); + + while (hdr_iter_next(&iter)) + { + if (0 != iter.count) + { + double dev = (hdr_median_equivalent_value(h, iter.value) * 1.0) - mean; + geometric_dev_total += (dev * dev) * iter.count; + } + } + + return sqrt(geometric_dev_total / h->total_count); +} + +bool hdr_values_are_equivalent(const struct hdr_histogram* h, int64_t a, int64_t b) +{ + return lowest_equivalent_value(h, a) == lowest_equivalent_value(h, b); +} + +int64_t hdr_lowest_equivalent_value(const struct hdr_histogram* h, int64_t value) +{ + return lowest_equivalent_value(h, value); +} + +int64_t hdr_count_at_value(const struct hdr_histogram* h, int64_t value) +{ + return counts_get_normalised(h, counts_index_for(h, value)); +} + +int64_t hdr_count_at_index(const struct hdr_histogram* h, int32_t index) +{ + return counts_get_normalised(h, index); +} + + +// #### ######## ######## ######## ### ######## ####### ######## ###### +// ## ## ## ## ## ## ## ## ## ## ## ## ## ## +// ## ## ## ## ## ## ## ## ## ## ## ## ## +// ## ## ###### ######## ## ## ## ## ## ######## ###### +// ## ## ## ## ## ######### ## ## ## ## ## ## +// ## ## ## ## ## ## ## ## ## ## ## ## ## ## +// #### ## ######## ## ## ## ## ## ####### ## ## ###### + + +static bool has_buckets(struct hdr_iter* iter) +{ + return iter->counts_index < iter->h->counts_len; +} + +static bool has_next(struct hdr_iter* iter) +{ + return iter->cumulative_count < iter->total_count; +} + +static bool move_next(struct hdr_iter* iter) +{ + iter->counts_index++; + + if (!has_buckets(iter)) + { + return false; + } + + iter->count = counts_get_normalised(iter->h, iter->counts_index); + iter->cumulative_count += iter->count; + + iter->value = hdr_value_at_index(iter->h, iter->counts_index); + iter->highest_equivalent_value = highest_equivalent_value(iter->h, iter->value); + iter->lowest_equivalent_value = lowest_equivalent_value(iter->h, iter->value); + iter->median_equivalent_value = hdr_median_equivalent_value(iter->h, iter->value); + + return true; +} + +static int64_t peek_next_value_from_index(struct hdr_iter* iter) +{ + return hdr_value_at_index(iter->h, iter->counts_index + 1); +} + +static bool next_value_greater_than_reporting_level_upper_bound( + struct hdr_iter *iter, int64_t reporting_level_upper_bound) +{ + if (iter->counts_index >= iter->h->counts_len) + { + return false; + } + + return peek_next_value_from_index(iter) > reporting_level_upper_bound; +} + +static bool _basic_iter_next(struct hdr_iter *iter) +{ + if (!has_next(iter)) + { + return false; + } + + move_next(iter); + + return true; +} + +static void _update_iterated_values(struct hdr_iter* iter, int64_t new_value_iterated_to) +{ + iter->value_iterated_from = iter->value_iterated_to; + iter->value_iterated_to = new_value_iterated_to; +} + +static bool _all_values_iter_next(struct hdr_iter* iter) +{ + bool result = move_next(iter); + + if (result) + { + _update_iterated_values(iter, iter->value); + } + + return result; +} + +void hdr_iter_init(struct hdr_iter* iter, const struct hdr_histogram* h) +{ + iter->h = h; + + iter->counts_index = -1; + iter->total_count = h->total_count; + iter->count = 0; + iter->cumulative_count = 0; + iter->value = 0; + iter->highest_equivalent_value = 0; + iter->value_iterated_from = 0; + iter->value_iterated_to = 0; + + iter->_next_fp = _all_values_iter_next; +} + +bool hdr_iter_next(struct hdr_iter* iter) +{ + return iter->_next_fp(iter); +} + +// ######## ######## ######## ###### ######## ## ## ######## #### ## ######## ###### +// ## ## ## ## ## ## ## ## ### ## ## ## ## ## ## ## +// ## ## ## ## ## ## ## #### ## ## ## ## ## ## +// ######## ###### ######## ## ###### ## ## ## ## ## ## ###### ###### +// ## ## ## ## ## ## ## #### ## ## ## ## ## +// ## ## ## ## ## ## ## ## ### ## ## ## ## ## ## +// ## ######## ## ## ###### ######## ## ## ## #### ######## ######## ###### + +static bool _percentile_iter_next(struct hdr_iter* iter) +{ + struct hdr_iter_percentiles* percentiles = &iter->specifics.percentiles; + + if (!has_next(iter)) + { + if (percentiles->seen_last_value) + { + return false; + } + + percentiles->seen_last_value = true; + percentiles->percentile = 100.0; + + return true; + } + + if (iter->counts_index == -1 && !_basic_iter_next(iter)) + { + return false; + } + + do + { + double current_percentile = (100.0 * (double) iter->cumulative_count) / iter->h->total_count; + if (iter->count != 0 && + percentiles->percentile_to_iterate_to <= current_percentile) + { + _update_iterated_values(iter, highest_equivalent_value(iter->h, iter->value)); + + percentiles->percentile = percentiles->percentile_to_iterate_to; + int64_t temp = (int64_t)(log(100 / (100.0 - (percentiles->percentile_to_iterate_to))) / log(2)) + 1; + int64_t half_distance = (int64_t) pow(2, (double) temp); + int64_t percentile_reporting_ticks = percentiles->ticks_per_half_distance * half_distance; + percentiles->percentile_to_iterate_to += 100.0 / percentile_reporting_ticks; + + return true; + } + } + while (_basic_iter_next(iter)); + + return true; +} + +void hdr_iter_percentile_init(struct hdr_iter* iter, const struct hdr_histogram* h, int32_t ticks_per_half_distance) +{ + iter->h = h; + + hdr_iter_init(iter, h); + + iter->specifics.percentiles.seen_last_value = false; + iter->specifics.percentiles.ticks_per_half_distance = ticks_per_half_distance; + iter->specifics.percentiles.percentile_to_iterate_to = 0.0; + iter->specifics.percentiles.percentile = 0.0; + + iter->_next_fp = _percentile_iter_next; +} + +static void format_line_string(char* str, size_t len, int significant_figures, format_type format) +{ +#if defined(_MSC_VER) +#define snprintf _snprintf +#pragma warning(push) +#pragma warning(disable: 4996) +#endif + const char* format_str = "%s%d%s"; + + switch (format) + { + case CSV: + snprintf(str, len, format_str, "%.", significant_figures, "f,%f,%d,%.2f\n"); + break; + case CLASSIC: + snprintf(str, len, format_str, "%12.", significant_figures, "f %12f %12d %12.2f\n"); + break; + default: + snprintf(str, len, format_str, "%12.", significant_figures, "f %12f %12d %12.2f\n"); + } +#if defined(_MSC_VER) +#undef snprintf +#pragma warning(pop) +#endif +} + + +// ######## ######## ###### ####### ######## ######## ######## ######## +// ## ## ## ## ## ## ## ## ## ## ## ## ## ## +// ## ## ## ## ## ## ## ## ## ## ## ## ## +// ######## ###### ## ## ## ######## ## ## ###### ## ## +// ## ## ## ## ## ## ## ## ## ## ## ## ## +// ## ## ## ## ## ## ## ## ## ## ## ## ## ## +// ## ## ######## ###### ####### ## ## ######## ######## ######## + + +static bool _recorded_iter_next(struct hdr_iter* iter) +{ + while (_basic_iter_next(iter)) + { + if (iter->count != 0) + { + _update_iterated_values(iter, iter->value); + + iter->specifics.recorded.count_added_in_this_iteration_step = iter->count; + return true; + } + } + + return false; +} + +void hdr_iter_recorded_init(struct hdr_iter* iter, const struct hdr_histogram* h) +{ + hdr_iter_init(iter, h); + + iter->specifics.recorded.count_added_in_this_iteration_step = 0; + + iter->_next_fp = _recorded_iter_next; +} + +// ## #### ## ## ######## ### ######## +// ## ## ### ## ## ## ## ## ## +// ## ## #### ## ## ## ## ## ## +// ## ## ## ## ## ###### ## ## ######## +// ## ## ## #### ## ######### ## ## +// ## ## ## ### ## ## ## ## ## +// ######## #### ## ## ######## ## ## ## ## + + +static bool _iter_linear_next(struct hdr_iter* iter) +{ + struct hdr_iter_linear* linear = &iter->specifics.linear; + + linear->count_added_in_this_iteration_step = 0; + + if (has_next(iter) || + next_value_greater_than_reporting_level_upper_bound( + iter, linear->next_value_reporting_level_lowest_equivalent)) + { + do + { + if (iter->value >= linear->next_value_reporting_level_lowest_equivalent) + { + _update_iterated_values(iter, linear->next_value_reporting_level); + + linear->next_value_reporting_level += linear->value_units_per_bucket; + linear->next_value_reporting_level_lowest_equivalent = + lowest_equivalent_value(iter->h, linear->next_value_reporting_level); + + return true; + } + + if (!move_next(iter)) + { + return true; + } + + linear->count_added_in_this_iteration_step += iter->count; + } + while (true); + } + + return false; +} + + +void hdr_iter_linear_init(struct hdr_iter* iter, const struct hdr_histogram* h, int64_t value_units_per_bucket) +{ + hdr_iter_init(iter, h); + + iter->specifics.linear.count_added_in_this_iteration_step = 0; + iter->specifics.linear.value_units_per_bucket = value_units_per_bucket; + iter->specifics.linear.next_value_reporting_level = value_units_per_bucket; + iter->specifics.linear.next_value_reporting_level_lowest_equivalent = lowest_equivalent_value(h, value_units_per_bucket); + + iter->_next_fp = _iter_linear_next; +} + +// ## ####### ###### ### ######## #### ######## ## ## ## ## #### ###### +// ## ## ## ## ## ## ## ## ## ## ## ## ## ### ### ## ## ## +// ## ## ## ## ## ## ## ## ## ## ## ## #### #### ## ## +// ## ## ## ## #### ## ## ######## ## ## ######### ## ### ## ## ## +// ## ## ## ## ## ######### ## ## ## ## ## ## ## ## ## ## +// ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## +// ######## ####### ###### ## ## ## ## #### ## ## ## ## ## #### ###### + +static bool _log_iter_next(struct hdr_iter *iter) +{ + struct hdr_iter_log* logarithmic = &iter->specifics.log; + + logarithmic->count_added_in_this_iteration_step = 0; + + if (has_next(iter) || + next_value_greater_than_reporting_level_upper_bound( + iter, logarithmic->next_value_reporting_level_lowest_equivalent)) + { + do + { + if (iter->value >= logarithmic->next_value_reporting_level_lowest_equivalent) + { + _update_iterated_values(iter, logarithmic->next_value_reporting_level); + + logarithmic->next_value_reporting_level *= (int64_t)logarithmic->log_base; + logarithmic->next_value_reporting_level_lowest_equivalent = lowest_equivalent_value(iter->h, logarithmic->next_value_reporting_level); + + return true; + } + + if (!move_next(iter)) + { + return true; + } + + logarithmic->count_added_in_this_iteration_step += iter->count; + } + while (true); + } + + return false; +} + +void hdr_iter_log_init( + struct hdr_iter* iter, + const struct hdr_histogram* h, + int64_t value_units_first_bucket, + double log_base) +{ + hdr_iter_init(iter, h); + iter->specifics.log.count_added_in_this_iteration_step = 0; + iter->specifics.log.log_base = log_base; + iter->specifics.log.next_value_reporting_level = value_units_first_bucket; + iter->specifics.log.next_value_reporting_level_lowest_equivalent = lowest_equivalent_value(h, value_units_first_bucket); + + iter->_next_fp = _log_iter_next; +} + +// Printing. + +static const char* format_head_string(format_type format) +{ + switch (format) + { + case CSV: + return "%s,%s,%s,%s\n"; + case CLASSIC: + return "%12s %12s %12s %12s\n\n"; + default: + return "%12s %12s %12s %12s\n\n"; + } +} + +static const char CLASSIC_FOOTER[] = + "#[Mean = %12.3f, StdDeviation = %12.3f]\n" + "#[Max = %12.3f, Total count = %12" PRIu64 "]\n" + "#[Buckets = %12d, SubBuckets = %12d]\n"; + +int hdr_percentiles_print( + struct hdr_histogram* h, FILE* stream, int32_t ticks_per_half_distance, + double value_scale, format_type format) +{ + char line_format[25]; + format_line_string(line_format, 25, h->significant_figures, format); + const char* head_format = format_head_string(format); + int rc = 0; + + struct hdr_iter iter; + hdr_iter_percentile_init(&iter, h, ticks_per_half_distance); + + if (fprintf( + stream, head_format, + "Value", "Percentile", "TotalCount", "1/(1-Percentile)") < 0) + { + rc = EIO; + goto cleanup; + } + + struct hdr_iter_percentiles * percentiles = &iter.specifics.percentiles; + while (hdr_iter_next(&iter)) + { + double value = iter.highest_equivalent_value / value_scale; + double percentile = percentiles->percentile / 100.0; + int64_t total_count = iter.cumulative_count; + double inverted_percentile = (1.0 / (1.0 - percentile)); + + if (fprintf( + stream, line_format, value, percentile, total_count, inverted_percentile) < 0) + { + rc = EIO; + goto cleanup; + } + } + + if (CLASSIC == format) + { + double mean = hdr_mean(h) / value_scale; + double stddev = hdr_stddev(h) / value_scale; + double max = hdr_max(h) / value_scale; + + if (fprintf( + stream, CLASSIC_FOOTER, mean, stddev, max, + h->total_count, h->bucket_count, h->sub_bucket_count) < 0) + { + rc = EIO; + goto cleanup; + } + } + + cleanup: + return rc; +} diff --git a/contrib/libs/hdr_histogram/src/hdr_histogram.h b/contrib/libs/hdr_histogram/src/hdr_histogram.h index 3d124e8fe90..c4ba1a43f9b 100644 --- a/contrib/libs/hdr_histogram/src/hdr_histogram.h +++ b/contrib/libs/hdr_histogram/src/hdr_histogram.h @@ -1,428 +1,428 @@ -/** - * hdr_histogram.h - * Written by Michael Barker and released to the public domain, - * as explained at http://creativecommons.org/publicdomain/zero/1.0/ - * - * The source for the hdr_histogram utilises a few C99 constructs, specifically - * the use of stdint/stdbool and inline variable declaration. - */ - -#ifndef HDR_HISTOGRAM_H -#define HDR_HISTOGRAM_H 1 - -#include <stdint.h> -#include <stdbool.h> -#include <stdio.h> - -struct hdr_histogram -{ - int64_t lowest_trackable_value; - int64_t highest_trackable_value; - int32_t unit_magnitude; - int32_t significant_figures; - int32_t sub_bucket_half_count_magnitude; - int32_t sub_bucket_half_count; - int64_t sub_bucket_mask; - int32_t sub_bucket_count; - int32_t bucket_count; - int64_t min_value; - int64_t max_value; - int32_t normalizing_index_offset; - double conversion_ratio; - int32_t counts_len; - int64_t total_count; - int64_t counts[0]; -}; - -#ifdef __cplusplus -extern "C" { -#endif - -/** - * Allocate the memory and initialise the hdr_histogram. - * - * Due to the size of the histogram being the result of some reasonably - * involved math on the input parameters this function it is tricky to stack allocate. - * The histogram is allocated in a single contigious block so can be delete via free, - * without any structure specific destructor. - * - * @param lowest_trackable_value The smallest possible value to be put into the - * histogram. - * @param highest_trackable_value The largest possible value to be put into the - * histogram. - * @param significant_figures The level of precision for this histogram, i.e. the number - * of figures in a decimal number that will be maintained. E.g. a value of 3 will mean - * the results from the histogram will be accurate up to the first three digits. Must - * be a value between 1 and 5 (inclusive). - * @param result Output parameter to capture allocated histogram. - * @return 0 on success, EINVAL if lowest_trackable_value is < 1 or the - * significant_figure value is outside of the allowed range, ENOMEM if malloc - * failed. - */ -int hdr_init( - int64_t lowest_trackable_value, - int64_t highest_trackable_value, - int significant_figures, - struct hdr_histogram** result); - -/** - * Allocate the memory and initialise the hdr_histogram. This is the equivalent of calling - * hdr_init(1, highest_trackable_value, significant_figures, result); - * - * @deprecated use hdr_init. - */ -int hdr_alloc(int64_t highest_trackable_value, int significant_figures, struct hdr_histogram** result); - - -/** - * Reset a histogram to zero - empty out a histogram and re-initialise it - * - * If you want to re-use an existing histogram, but reset everything back to zero, this - * is the routine to use. - * - * @param h The histogram you want to reset to empty. - * - */ -void hdr_reset(struct hdr_histogram* h); - -/** - * Get the memory size of the hdr_histogram. - * - * @param h "This" pointer - * @return The amount of memory used by the hdr_histogram in bytes - */ -size_t hdr_get_memory_size(struct hdr_histogram* h); - -/** - * Records a value in the histogram, will round this value of to a precision at or better - * than the significant_figure specified at construction time. - * - * @param h "This" pointer - * @param value Value to add to the histogram - * @return false if the value is larger than the highest_trackable_value and can't be recorded, - * true otherwise. - */ -bool hdr_record_value(struct hdr_histogram* h, int64_t value); - -/** - * Records count values in the histogram, will round this value of to a - * precision at or better than the significant_figure specified at construction - * time. - * - * @param h "This" pointer - * @param value Value to add to the histogram - * @param count Number of 'value's to add to the histogram - * @return false if any value is larger than the highest_trackable_value and can't be recorded, - * true otherwise. - */ -bool hdr_record_values(struct hdr_histogram* h, int64_t value, int64_t count); - - -/** - * Record a value in the histogram and backfill based on an expected interval. - * - * Records a value in the histogram, will round this value of to a precision at or better - * than the significant_figure specified at contruction time. This is specifically used - * for recording latency. If the value is larger than the expected_interval then the - * latency recording system has experienced co-ordinated omission. This method fills in the - * values that would have occured had the client providing the load not been blocked. - - * @param h "This" pointer - * @param value Value to add to the histogram - * @param expected_interval The delay between recording values. - * @return false if the value is larger than the highest_trackable_value and can't be recorded, - * true otherwise. - */ -bool hdr_record_corrected_value(struct hdr_histogram* h, int64_t value, int64_t expexcted_interval); -/** - * Record a value in the histogram 'count' times. Applies the same correcting logic - * as 'hdr_record_corrected_value'. - * - * @param h "This" pointer - * @param value Value to add to the histogram - * @param count Number of 'value's to add to the histogram - * @param expected_interval The delay between recording values. - * @return false if the value is larger than the highest_trackable_value and can't be recorded, - * true otherwise. - */ -bool hdr_record_corrected_values(struct hdr_histogram* h, int64_t value, int64_t count, int64_t expected_interval); - -/** - * Adds all of the values from 'from' to 'this' histogram. Will return the - * number of values that are dropped when copying. Values will be dropped - * if they around outside of h.lowest_trackable_value and - * h.highest_trackable_value. - * - * @param h "This" pointer - * @param from Histogram to copy values from. - * @return The number of values dropped when copying. - */ -int64_t hdr_add(struct hdr_histogram* h, const struct hdr_histogram* from); - -/** - * Adds all of the values from 'from' to 'this' histogram. Will return the - * number of values that are dropped when copying. Values will be dropped - * if they around outside of h.lowest_trackable_value and - * h.highest_trackable_value. - * - * @param h "This" pointer - * @param from Histogram to copy values from. - * @return The number of values dropped when copying. - */ -int64_t hdr_add_while_correcting_for_coordinated_omission( - struct hdr_histogram* h, struct hdr_histogram* from, int64_t expected_interval); - -/** - * Get minimum value from the histogram. Will return 2^63-1 if the histogram - * is empty. - * - * @param h "This" pointer - */ -int64_t hdr_min(const struct hdr_histogram* h); - -/** - * Get maximum value from the histogram. Will return 0 if the histogram - * is empty. - * - * @param h "This" pointer - */ -int64_t hdr_max(const struct hdr_histogram* h); - -/** - * Get the value at a specific percentile. - * - * @param h "This" pointer. - * @param percentile The percentile to get the value for - */ -int64_t hdr_value_at_percentile(const struct hdr_histogram* h, double percentile); - -/** - * Gets the standard deviation for the values in the histogram. - * - * @param h "This" pointer - * @return The standard deviation - */ -double hdr_stddev(const struct hdr_histogram* h); - -/** - * Gets the mean for the values in the histogram. - * - * @param h "This" pointer - * @return The mean - */ -double hdr_mean(const struct hdr_histogram* h); - -/** - * Determine if two values are equivalent with the histogram's resolution. - * Where "equivalent" means that value samples recorded for any two - * equivalent values are counted in a common total count. - * - * @param h "This" pointer - * @param a first value to compare - * @param b second value to compare - * @return 'true' if values are equivalent with the histogram's resolution. - */ -bool hdr_values_are_equivalent(const struct hdr_histogram* h, int64_t a, int64_t b); - -/** - * Get the lowest value that is equivalent to the given value within the histogram's resolution. - * Where "equivalent" means that value samples recorded for any two - * equivalent values are counted in a common total count. - * - * @param h "This" pointer - * @param value The given value - * @return The lowest value that is equivalent to the given value within the histogram's resolution. - */ -int64_t hdr_lowest_equivalent_value(const struct hdr_histogram* h, int64_t value); - -/** - * Get the count of recorded values at a specific value - * (to within the histogram resolution at the value level). - * - * @param h "This" pointer - * @param value The value for which to provide the recorded count - * @return The total count of values recorded in the histogram within the value range that is - * {@literal >=} lowestEquivalentValue(<i>value</i>) and {@literal <=} highestEquivalentValue(<i>value</i>) - */ -int64_t hdr_count_at_value(const struct hdr_histogram* h, int64_t value); - -int64_t hdr_count_at_index(const struct hdr_histogram* h, int32_t index); - -int64_t hdr_value_at_index(const struct hdr_histogram* h, int32_t index); - -struct hdr_iter_percentiles -{ - bool seen_last_value; - int32_t ticks_per_half_distance; - double percentile_to_iterate_to; - double percentile; -}; - -struct hdr_iter_recorded -{ - int64_t count_added_in_this_iteration_step; -}; - -struct hdr_iter_linear -{ - int64_t value_units_per_bucket; - int64_t count_added_in_this_iteration_step; - int64_t next_value_reporting_level; - int64_t next_value_reporting_level_lowest_equivalent; -}; - -struct hdr_iter_log -{ - double log_base; - int64_t count_added_in_this_iteration_step; - int64_t next_value_reporting_level; - int64_t next_value_reporting_level_lowest_equivalent; -}; - -/** - * The basic iterator. This is a generic structure - * that supports all of the types of iteration. Use - * the appropriate initialiser to get the desired - * iteration. - * - * @ - */ -struct hdr_iter -{ - const struct hdr_histogram* h; - /** raw index into the counts array */ - int32_t counts_index; - /** snapshot of the length at the time the iterator is created */ - int32_t total_count; - /** value directly from array for the current counts_index */ - int64_t count; - /** sum of all of the counts up to and including the count at this index */ - int64_t cumulative_count; - /** The current value based on counts_index */ - int64_t value; - int64_t highest_equivalent_value; - int64_t lowest_equivalent_value; - int64_t median_equivalent_value; - int64_t value_iterated_from; - int64_t value_iterated_to; - - union - { - struct hdr_iter_percentiles percentiles; - struct hdr_iter_recorded recorded; - struct hdr_iter_linear linear; - struct hdr_iter_log log; - } specifics; - - bool (* _next_fp)(struct hdr_iter* iter); - -}; - -/** - * Initalises the basic iterator. - * - * @param itr 'This' pointer - * @param h The histogram to iterate over - */ -void hdr_iter_init(struct hdr_iter* iter, const struct hdr_histogram* h); - -/** - * Initialise the iterator for use with percentiles. - */ -void hdr_iter_percentile_init(struct hdr_iter* iter, const struct hdr_histogram* h, int32_t ticks_per_half_distance); - -/** - * Initialise the iterator for use with recorded values. - */ -void hdr_iter_recorded_init(struct hdr_iter* iter, const struct hdr_histogram* h); - -/** - * Initialise the iterator for use with linear values. - */ -void hdr_iter_linear_init( - struct hdr_iter* iter, - const struct hdr_histogram* h, - int64_t value_units_per_bucket); - -/** - * Initialise the iterator for use with logarithmic values - */ -void hdr_iter_log_init( - struct hdr_iter* iter, - const struct hdr_histogram* h, - int64_t value_units_first_bucket, - double log_base); - -/** - * Iterate to the next value for the iterator. If there are no more values - * available return faluse. - * - * @param itr 'This' pointer - * @return 'false' if there are no values remaining for this iterator. - */ -bool hdr_iter_next(struct hdr_iter* iter); - -typedef enum -{ - CLASSIC, - CSV -} format_type; - -/** - * Print out a percentile based histogram to the supplied stream. Note that - * this call will not flush the FILE, this is left up to the user. - * - * @param h 'This' pointer - * @param stream The FILE to write the output to - * @param ticks_per_half_distance The number of iteration steps per half-distance to 100% - * @param value_scale Scale the output values by this amount - * @param format_type Format to use, e.g. CSV. - * @return 0 on success, error code on failure. EIO if an error occurs writing - * the output. - */ -int hdr_percentiles_print( - struct hdr_histogram* h, FILE* stream, int32_t ticks_per_half_distance, - double value_scale, format_type format); - -/** -* Internal allocation methods, used by hdr_dbl_histogram. -*/ -struct hdr_histogram_bucket_config -{ - int64_t lowest_trackable_value; - int64_t highest_trackable_value; - int64_t unit_magnitude; - int64_t significant_figures; - int32_t sub_bucket_half_count_magnitude; - int32_t sub_bucket_half_count; - int64_t sub_bucket_mask; - int32_t sub_bucket_count; - int32_t bucket_count; - int32_t counts_len; -}; - -int hdr_calculate_bucket_config( - int64_t lowest_trackable_value, - int64_t highest_trackable_value, - int significant_figures, - struct hdr_histogram_bucket_config* cfg); - -void hdr_init_preallocated(struct hdr_histogram* h, struct hdr_histogram_bucket_config* cfg); - -int64_t hdr_size_of_equivalent_value_range(const struct hdr_histogram* h, int64_t value); - -int64_t hdr_next_non_equivalent_value(const struct hdr_histogram* h, int64_t value); - -int64_t hdr_median_equivalent_value(const struct hdr_histogram* h, int64_t value); - -/** - * Used to reset counters after importing data manuallying into the histogram, used by the logging code - * and other custom serialisation tools. - */ -void hdr_reset_internal_counters(struct hdr_histogram* h); - -#ifdef __cplusplus -} -#endif - -#endif +/** + * hdr_histogram.h + * Written by Michael Barker and released to the public domain, + * as explained at http://creativecommons.org/publicdomain/zero/1.0/ + * + * The source for the hdr_histogram utilises a few C99 constructs, specifically + * the use of stdint/stdbool and inline variable declaration. + */ + +#ifndef HDR_HISTOGRAM_H +#define HDR_HISTOGRAM_H 1 + +#include <stdint.h> +#include <stdbool.h> +#include <stdio.h> + +struct hdr_histogram +{ + int64_t lowest_trackable_value; + int64_t highest_trackable_value; + int32_t unit_magnitude; + int32_t significant_figures; + int32_t sub_bucket_half_count_magnitude; + int32_t sub_bucket_half_count; + int64_t sub_bucket_mask; + int32_t sub_bucket_count; + int32_t bucket_count; + int64_t min_value; + int64_t max_value; + int32_t normalizing_index_offset; + double conversion_ratio; + int32_t counts_len; + int64_t total_count; + int64_t counts[0]; +}; + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * Allocate the memory and initialise the hdr_histogram. + * + * Due to the size of the histogram being the result of some reasonably + * involved math on the input parameters this function it is tricky to stack allocate. + * The histogram is allocated in a single contigious block so can be delete via free, + * without any structure specific destructor. + * + * @param lowest_trackable_value The smallest possible value to be put into the + * histogram. + * @param highest_trackable_value The largest possible value to be put into the + * histogram. + * @param significant_figures The level of precision for this histogram, i.e. the number + * of figures in a decimal number that will be maintained. E.g. a value of 3 will mean + * the results from the histogram will be accurate up to the first three digits. Must + * be a value between 1 and 5 (inclusive). + * @param result Output parameter to capture allocated histogram. + * @return 0 on success, EINVAL if lowest_trackable_value is < 1 or the + * significant_figure value is outside of the allowed range, ENOMEM if malloc + * failed. + */ +int hdr_init( + int64_t lowest_trackable_value, + int64_t highest_trackable_value, + int significant_figures, + struct hdr_histogram** result); + +/** + * Allocate the memory and initialise the hdr_histogram. This is the equivalent of calling + * hdr_init(1, highest_trackable_value, significant_figures, result); + * + * @deprecated use hdr_init. + */ +int hdr_alloc(int64_t highest_trackable_value, int significant_figures, struct hdr_histogram** result); + + +/** + * Reset a histogram to zero - empty out a histogram and re-initialise it + * + * If you want to re-use an existing histogram, but reset everything back to zero, this + * is the routine to use. + * + * @param h The histogram you want to reset to empty. + * + */ +void hdr_reset(struct hdr_histogram* h); + +/** + * Get the memory size of the hdr_histogram. + * + * @param h "This" pointer + * @return The amount of memory used by the hdr_histogram in bytes + */ +size_t hdr_get_memory_size(struct hdr_histogram* h); + +/** + * Records a value in the histogram, will round this value of to a precision at or better + * than the significant_figure specified at construction time. + * + * @param h "This" pointer + * @param value Value to add to the histogram + * @return false if the value is larger than the highest_trackable_value and can't be recorded, + * true otherwise. + */ +bool hdr_record_value(struct hdr_histogram* h, int64_t value); + +/** + * Records count values in the histogram, will round this value of to a + * precision at or better than the significant_figure specified at construction + * time. + * + * @param h "This" pointer + * @param value Value to add to the histogram + * @param count Number of 'value's to add to the histogram + * @return false if any value is larger than the highest_trackable_value and can't be recorded, + * true otherwise. + */ +bool hdr_record_values(struct hdr_histogram* h, int64_t value, int64_t count); + + +/** + * Record a value in the histogram and backfill based on an expected interval. + * + * Records a value in the histogram, will round this value of to a precision at or better + * than the significant_figure specified at contruction time. This is specifically used + * for recording latency. If the value is larger than the expected_interval then the + * latency recording system has experienced co-ordinated omission. This method fills in the + * values that would have occured had the client providing the load not been blocked. + + * @param h "This" pointer + * @param value Value to add to the histogram + * @param expected_interval The delay between recording values. + * @return false if the value is larger than the highest_trackable_value and can't be recorded, + * true otherwise. + */ +bool hdr_record_corrected_value(struct hdr_histogram* h, int64_t value, int64_t expexcted_interval); +/** + * Record a value in the histogram 'count' times. Applies the same correcting logic + * as 'hdr_record_corrected_value'. + * + * @param h "This" pointer + * @param value Value to add to the histogram + * @param count Number of 'value's to add to the histogram + * @param expected_interval The delay between recording values. + * @return false if the value is larger than the highest_trackable_value and can't be recorded, + * true otherwise. + */ +bool hdr_record_corrected_values(struct hdr_histogram* h, int64_t value, int64_t count, int64_t expected_interval); + +/** + * Adds all of the values from 'from' to 'this' histogram. Will return the + * number of values that are dropped when copying. Values will be dropped + * if they around outside of h.lowest_trackable_value and + * h.highest_trackable_value. + * + * @param h "This" pointer + * @param from Histogram to copy values from. + * @return The number of values dropped when copying. + */ +int64_t hdr_add(struct hdr_histogram* h, const struct hdr_histogram* from); + +/** + * Adds all of the values from 'from' to 'this' histogram. Will return the + * number of values that are dropped when copying. Values will be dropped + * if they around outside of h.lowest_trackable_value and + * h.highest_trackable_value. + * + * @param h "This" pointer + * @param from Histogram to copy values from. + * @return The number of values dropped when copying. + */ +int64_t hdr_add_while_correcting_for_coordinated_omission( + struct hdr_histogram* h, struct hdr_histogram* from, int64_t expected_interval); + +/** + * Get minimum value from the histogram. Will return 2^63-1 if the histogram + * is empty. + * + * @param h "This" pointer + */ +int64_t hdr_min(const struct hdr_histogram* h); + +/** + * Get maximum value from the histogram. Will return 0 if the histogram + * is empty. + * + * @param h "This" pointer + */ +int64_t hdr_max(const struct hdr_histogram* h); + +/** + * Get the value at a specific percentile. + * + * @param h "This" pointer. + * @param percentile The percentile to get the value for + */ +int64_t hdr_value_at_percentile(const struct hdr_histogram* h, double percentile); + +/** + * Gets the standard deviation for the values in the histogram. + * + * @param h "This" pointer + * @return The standard deviation + */ +double hdr_stddev(const struct hdr_histogram* h); + +/** + * Gets the mean for the values in the histogram. + * + * @param h "This" pointer + * @return The mean + */ +double hdr_mean(const struct hdr_histogram* h); + +/** + * Determine if two values are equivalent with the histogram's resolution. + * Where "equivalent" means that value samples recorded for any two + * equivalent values are counted in a common total count. + * + * @param h "This" pointer + * @param a first value to compare + * @param b second value to compare + * @return 'true' if values are equivalent with the histogram's resolution. + */ +bool hdr_values_are_equivalent(const struct hdr_histogram* h, int64_t a, int64_t b); + +/** + * Get the lowest value that is equivalent to the given value within the histogram's resolution. + * Where "equivalent" means that value samples recorded for any two + * equivalent values are counted in a common total count. + * + * @param h "This" pointer + * @param value The given value + * @return The lowest value that is equivalent to the given value within the histogram's resolution. + */ +int64_t hdr_lowest_equivalent_value(const struct hdr_histogram* h, int64_t value); + +/** + * Get the count of recorded values at a specific value + * (to within the histogram resolution at the value level). + * + * @param h "This" pointer + * @param value The value for which to provide the recorded count + * @return The total count of values recorded in the histogram within the value range that is + * {@literal >=} lowestEquivalentValue(<i>value</i>) and {@literal <=} highestEquivalentValue(<i>value</i>) + */ +int64_t hdr_count_at_value(const struct hdr_histogram* h, int64_t value); + +int64_t hdr_count_at_index(const struct hdr_histogram* h, int32_t index); + +int64_t hdr_value_at_index(const struct hdr_histogram* h, int32_t index); + +struct hdr_iter_percentiles +{ + bool seen_last_value; + int32_t ticks_per_half_distance; + double percentile_to_iterate_to; + double percentile; +}; + +struct hdr_iter_recorded +{ + int64_t count_added_in_this_iteration_step; +}; + +struct hdr_iter_linear +{ + int64_t value_units_per_bucket; + int64_t count_added_in_this_iteration_step; + int64_t next_value_reporting_level; + int64_t next_value_reporting_level_lowest_equivalent; +}; + +struct hdr_iter_log +{ + double log_base; + int64_t count_added_in_this_iteration_step; + int64_t next_value_reporting_level; + int64_t next_value_reporting_level_lowest_equivalent; +}; + +/** + * The basic iterator. This is a generic structure + * that supports all of the types of iteration. Use + * the appropriate initialiser to get the desired + * iteration. + * + * @ + */ +struct hdr_iter +{ + const struct hdr_histogram* h; + /** raw index into the counts array */ + int32_t counts_index; + /** snapshot of the length at the time the iterator is created */ + int32_t total_count; + /** value directly from array for the current counts_index */ + int64_t count; + /** sum of all of the counts up to and including the count at this index */ + int64_t cumulative_count; + /** The current value based on counts_index */ + int64_t value; + int64_t highest_equivalent_value; + int64_t lowest_equivalent_value; + int64_t median_equivalent_value; + int64_t value_iterated_from; + int64_t value_iterated_to; + + union + { + struct hdr_iter_percentiles percentiles; + struct hdr_iter_recorded recorded; + struct hdr_iter_linear linear; + struct hdr_iter_log log; + } specifics; + + bool (* _next_fp)(struct hdr_iter* iter); + +}; + +/** + * Initalises the basic iterator. + * + * @param itr 'This' pointer + * @param h The histogram to iterate over + */ +void hdr_iter_init(struct hdr_iter* iter, const struct hdr_histogram* h); + +/** + * Initialise the iterator for use with percentiles. + */ +void hdr_iter_percentile_init(struct hdr_iter* iter, const struct hdr_histogram* h, int32_t ticks_per_half_distance); + +/** + * Initialise the iterator for use with recorded values. + */ +void hdr_iter_recorded_init(struct hdr_iter* iter, const struct hdr_histogram* h); + +/** + * Initialise the iterator for use with linear values. + */ +void hdr_iter_linear_init( + struct hdr_iter* iter, + const struct hdr_histogram* h, + int64_t value_units_per_bucket); + +/** + * Initialise the iterator for use with logarithmic values + */ +void hdr_iter_log_init( + struct hdr_iter* iter, + const struct hdr_histogram* h, + int64_t value_units_first_bucket, + double log_base); + +/** + * Iterate to the next value for the iterator. If there are no more values + * available return faluse. + * + * @param itr 'This' pointer + * @return 'false' if there are no values remaining for this iterator. + */ +bool hdr_iter_next(struct hdr_iter* iter); + +typedef enum +{ + CLASSIC, + CSV +} format_type; + +/** + * Print out a percentile based histogram to the supplied stream. Note that + * this call will not flush the FILE, this is left up to the user. + * + * @param h 'This' pointer + * @param stream The FILE to write the output to + * @param ticks_per_half_distance The number of iteration steps per half-distance to 100% + * @param value_scale Scale the output values by this amount + * @param format_type Format to use, e.g. CSV. + * @return 0 on success, error code on failure. EIO if an error occurs writing + * the output. + */ +int hdr_percentiles_print( + struct hdr_histogram* h, FILE* stream, int32_t ticks_per_half_distance, + double value_scale, format_type format); + +/** +* Internal allocation methods, used by hdr_dbl_histogram. +*/ +struct hdr_histogram_bucket_config +{ + int64_t lowest_trackable_value; + int64_t highest_trackable_value; + int64_t unit_magnitude; + int64_t significant_figures; + int32_t sub_bucket_half_count_magnitude; + int32_t sub_bucket_half_count; + int64_t sub_bucket_mask; + int32_t sub_bucket_count; + int32_t bucket_count; + int32_t counts_len; +}; + +int hdr_calculate_bucket_config( + int64_t lowest_trackable_value, + int64_t highest_trackable_value, + int significant_figures, + struct hdr_histogram_bucket_config* cfg); + +void hdr_init_preallocated(struct hdr_histogram* h, struct hdr_histogram_bucket_config* cfg); + +int64_t hdr_size_of_equivalent_value_range(const struct hdr_histogram* h, int64_t value); + +int64_t hdr_next_non_equivalent_value(const struct hdr_histogram* h, int64_t value); + +int64_t hdr_median_equivalent_value(const struct hdr_histogram* h, int64_t value); + +/** + * Used to reset counters after importing data manuallying into the histogram, used by the logging code + * and other custom serialisation tools. + */ +void hdr_reset_internal_counters(struct hdr_histogram* h); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/contrib/libs/hdr_histogram/src/hdr_interval_recorder.c b/contrib/libs/hdr_histogram/src/hdr_interval_recorder.c index 2c4f9551936..2b0ba87cae5 100644 --- a/contrib/libs/hdr_histogram/src/hdr_interval_recorder.c +++ b/contrib/libs/hdr_histogram/src/hdr_interval_recorder.c @@ -1,53 +1,53 @@ -/** - * hdr_interval_recorder.h - * Written by Michael Barker and released to the public domain, - * as explained at http://creativecommons.org/publicdomain/zero/1.0/ - */ - -#include "hdr_atomic.h" -#include "hdr_interval_recorder.h" - -int hdr_interval_recorder_init(struct hdr_interval_recorder* r) -{ - return hdr_writer_reader_phaser_init(&r->phaser); -} - -void hdr_interval_recorder_destroy(struct hdr_interval_recorder* r) -{ - hdr_writer_reader_phaser_destory(&r->phaser); -} - -void hdr_interval_recorder_update( - struct hdr_interval_recorder* r, - void(*update_action)(void*, void*), - void* arg) -{ - int64_t val = hdr_phaser_writer_enter(&r->phaser); - - void* active = hdr_atomic_load_pointer(&r->active); - - update_action(active, arg); - - hdr_phaser_writer_exit(&r->phaser, val); -} - -void* hdr_interval_recorder_sample(struct hdr_interval_recorder* r) -{ - void* temp; - - hdr_phaser_reader_lock(&r->phaser); - - temp = r->inactive; - - // volatile read - r->inactive = hdr_atomic_load_pointer(&r->active); - - // volatile write - hdr_atomic_store_pointer(&r->active, temp); - - hdr_phaser_flip_phase(&r->phaser, 0); - - hdr_phaser_reader_unlock(&r->phaser); - - return r->inactive; -} +/** + * hdr_interval_recorder.h + * Written by Michael Barker and released to the public domain, + * as explained at http://creativecommons.org/publicdomain/zero/1.0/ + */ + +#include "hdr_atomic.h" +#include "hdr_interval_recorder.h" + +int hdr_interval_recorder_init(struct hdr_interval_recorder* r) +{ + return hdr_writer_reader_phaser_init(&r->phaser); +} + +void hdr_interval_recorder_destroy(struct hdr_interval_recorder* r) +{ + hdr_writer_reader_phaser_destory(&r->phaser); +} + +void hdr_interval_recorder_update( + struct hdr_interval_recorder* r, + void(*update_action)(void*, void*), + void* arg) +{ + int64_t val = hdr_phaser_writer_enter(&r->phaser); + + void* active = hdr_atomic_load_pointer(&r->active); + + update_action(active, arg); + + hdr_phaser_writer_exit(&r->phaser, val); +} + +void* hdr_interval_recorder_sample(struct hdr_interval_recorder* r) +{ + void* temp; + + hdr_phaser_reader_lock(&r->phaser); + + temp = r->inactive; + + // volatile read + r->inactive = hdr_atomic_load_pointer(&r->active); + + // volatile write + hdr_atomic_store_pointer(&r->active, temp); + + hdr_phaser_flip_phase(&r->phaser, 0); + + hdr_phaser_reader_unlock(&r->phaser); + + return r->inactive; +} diff --git a/contrib/libs/hdr_histogram/src/hdr_interval_recorder.h b/contrib/libs/hdr_histogram/src/hdr_interval_recorder.h index 3774e4e4272..e363c23c669 100644 --- a/contrib/libs/hdr_histogram/src/hdr_interval_recorder.h +++ b/contrib/libs/hdr_histogram/src/hdr_interval_recorder.h @@ -1,40 +1,40 @@ -/** - * hdr_interval_recorder.h - * Written by Michael Barker and released to the public domain, - * as explained at http://creativecommons.org/publicdomain/zero/1.0/ - */ - -#ifndef HDR_INTERVAL_RECORDER_H -#define HDR_INTERVAL_RECORDER_H 1 - -#include "hdr_writer_reader_phaser.h" - -HDR_ALIGN_PREFIX(8) -struct hdr_interval_recorder -{ - void* active; - void* inactive; - struct hdr_writer_reader_phaser phaser; -} -HDR_ALIGN_SUFFIX(8); - -#ifdef __cplusplus -extern "C" { -#endif - -int hdr_interval_recorder_init(struct hdr_interval_recorder* r); - -void hdr_interval_recorder_destroy(struct hdr_interval_recorder* r); - -void hdr_interval_recorder_update( - struct hdr_interval_recorder* r, - void(*update_action)(void*, void*), - void* arg); - -void* hdr_interval_recorder_sample(struct hdr_interval_recorder* r); - -#ifdef __cplusplus +/** + * hdr_interval_recorder.h + * Written by Michael Barker and released to the public domain, + * as explained at http://creativecommons.org/publicdomain/zero/1.0/ + */ + +#ifndef HDR_INTERVAL_RECORDER_H +#define HDR_INTERVAL_RECORDER_H 1 + +#include "hdr_writer_reader_phaser.h" + +HDR_ALIGN_PREFIX(8) +struct hdr_interval_recorder +{ + void* active; + void* inactive; + struct hdr_writer_reader_phaser phaser; } -#endif - -#endif +HDR_ALIGN_SUFFIX(8); + +#ifdef __cplusplus +extern "C" { +#endif + +int hdr_interval_recorder_init(struct hdr_interval_recorder* r); + +void hdr_interval_recorder_destroy(struct hdr_interval_recorder* r); + +void hdr_interval_recorder_update( + struct hdr_interval_recorder* r, + void(*update_action)(void*, void*), + void* arg); + +void* hdr_interval_recorder_sample(struct hdr_interval_recorder* r); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/contrib/libs/hdr_histogram/src/hdr_tests.h b/contrib/libs/hdr_histogram/src/hdr_tests.h index 34f67c9a64f..c016d3a6de0 100644 --- a/contrib/libs/hdr_histogram/src/hdr_tests.h +++ b/contrib/libs/hdr_histogram/src/hdr_tests.h @@ -1,22 +1,22 @@ -#ifndef HDR_TESTS_H -#define HDR_TESTS_H - -/* These are functions used in tests and are not intended for normal usage. */ - -#include "hdr_histogram.h" - -#ifdef __cplusplus -extern "C" { -#endif - -int32_t counts_index_for(const struct hdr_histogram* h, int64_t value); -int hdr_encode_compressed(struct hdr_histogram* h, uint8_t** compressed_histogram, size_t* compressed_len); -int hdr_decode_compressed(uint8_t* buffer, size_t length, struct hdr_histogram** histogram); -void hdr_base64_decode_block(const char* input, uint8_t* output); -void hdr_base64_encode_block(const uint8_t* input, char* output); - -#ifdef __cplusplus -} -#endif - -#endif +#ifndef HDR_TESTS_H +#define HDR_TESTS_H + +/* These are functions used in tests and are not intended for normal usage. */ + +#include "hdr_histogram.h" + +#ifdef __cplusplus +extern "C" { +#endif + +int32_t counts_index_for(const struct hdr_histogram* h, int64_t value); +int hdr_encode_compressed(struct hdr_histogram* h, uint8_t** compressed_histogram, size_t* compressed_len); +int hdr_decode_compressed(uint8_t* buffer, size_t length, struct hdr_histogram** histogram); +void hdr_base64_decode_block(const char* input, uint8_t* output); +void hdr_base64_encode_block(const uint8_t* input, char* output); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/contrib/libs/hdr_histogram/src/hdr_thread.c b/contrib/libs/hdr_histogram/src/hdr_thread.c index a38e64f2ad3..721780edb9e 100644 --- a/contrib/libs/hdr_histogram/src/hdr_thread.c +++ b/contrib/libs/hdr_histogram/src/hdr_thread.c @@ -1,102 +1,102 @@ -/** -* hdr_thread.c -* Written by Philip Orwig and released to the public domain, -* as explained at http://creativecommons.org/publicdomain/zero/1.0/ -*/ - -#include <stdlib.h> -#include "hdr_thread.h" - -struct hdr_mutex* hdr_mutex_alloc(void) -{ - return malloc(sizeof(hdr_mutex)); -} - -void hdr_mutex_free(struct hdr_mutex* mutex) -{ - free(mutex); -} - -#if defined(_WIN32) || defined(_WIN64) || defined(__CYGWIN__) - -#if !defined(WIN32_LEAN_AND_MEAN) -#define WIN32_LEAN_AND_MEAN -#endif - -#include <windows.h> -#include <WinSock2.h> - -int hdr_mutex_init(struct hdr_mutex* mutex) -{ - InitializeCriticalSection((CRITICAL_SECTION*)(mutex->_critical_section)); - return 0; -} - -void hdr_mutex_destroy(struct hdr_mutex* mutex) -{ - DeleteCriticalSection((CRITICAL_SECTION*)(mutex->_critical_section)); -} - -void hdr_mutex_lock(struct hdr_mutex* mutex) -{ - EnterCriticalSection((CRITICAL_SECTION*)(mutex->_critical_section)); -} - -void hdr_mutex_unlock(struct hdr_mutex* mutex) -{ - LeaveCriticalSection((CRITICAL_SECTION*)(mutex->_critical_section)); -} - -void hdr_yield() -{ - Sleep(0); -} - -int hdr_usleep(unsigned int useconds) -{ - struct timeval tv; - - tv.tv_sec = (long)useconds / 1000000; - tv.tv_usec = useconds % 1000000; - select(0, NULL, NULL, NULL, &tv); - - return 0; -} - - -#else -#include <pthread.h> -#include <unistd.h> - -int hdr_mutex_init(struct hdr_mutex* mutex) -{ - return pthread_mutex_init(&mutex->_mutex, NULL); -} - -void hdr_mutex_destroy(struct hdr_mutex* mutex) -{ - pthread_mutex_destroy(&mutex->_mutex); -} - -void hdr_mutex_lock(struct hdr_mutex* mutex) -{ - pthread_mutex_lock(&mutex->_mutex); -} - -void hdr_mutex_unlock(struct hdr_mutex* mutex) -{ - pthread_mutex_unlock(&mutex->_mutex); -} - -void hdr_yield() -{ - sched_yield(); -} - -int hdr_usleep(unsigned int useconds) -{ - return usleep(useconds); -} - - -#endif +/** +* hdr_thread.c +* Written by Philip Orwig and released to the public domain, +* as explained at http://creativecommons.org/publicdomain/zero/1.0/ +*/ + +#include <stdlib.h> +#include "hdr_thread.h" + +struct hdr_mutex* hdr_mutex_alloc(void) +{ + return malloc(sizeof(hdr_mutex)); +} + +void hdr_mutex_free(struct hdr_mutex* mutex) +{ + free(mutex); +} + +#if defined(_WIN32) || defined(_WIN64) || defined(__CYGWIN__) + +#if !defined(WIN32_LEAN_AND_MEAN) +#define WIN32_LEAN_AND_MEAN +#endif + +#include <windows.h> +#include <WinSock2.h> + +int hdr_mutex_init(struct hdr_mutex* mutex) +{ + InitializeCriticalSection((CRITICAL_SECTION*)(mutex->_critical_section)); + return 0; +} + +void hdr_mutex_destroy(struct hdr_mutex* mutex) +{ + DeleteCriticalSection((CRITICAL_SECTION*)(mutex->_critical_section)); +} + +void hdr_mutex_lock(struct hdr_mutex* mutex) +{ + EnterCriticalSection((CRITICAL_SECTION*)(mutex->_critical_section)); +} + +void hdr_mutex_unlock(struct hdr_mutex* mutex) +{ + LeaveCriticalSection((CRITICAL_SECTION*)(mutex->_critical_section)); +} + +void hdr_yield() +{ + Sleep(0); +} + +int hdr_usleep(unsigned int useconds) +{ + struct timeval tv; + + tv.tv_sec = (long)useconds / 1000000; + tv.tv_usec = useconds % 1000000; + select(0, NULL, NULL, NULL, &tv); + + return 0; +} + + +#else +#include <pthread.h> +#include <unistd.h> + +int hdr_mutex_init(struct hdr_mutex* mutex) +{ + return pthread_mutex_init(&mutex->_mutex, NULL); +} + +void hdr_mutex_destroy(struct hdr_mutex* mutex) +{ + pthread_mutex_destroy(&mutex->_mutex); +} + +void hdr_mutex_lock(struct hdr_mutex* mutex) +{ + pthread_mutex_lock(&mutex->_mutex); +} + +void hdr_mutex_unlock(struct hdr_mutex* mutex) +{ + pthread_mutex_unlock(&mutex->_mutex); +} + +void hdr_yield() +{ + sched_yield(); +} + +int hdr_usleep(unsigned int useconds) +{ + return usleep(useconds); +} + + +#endif diff --git a/contrib/libs/hdr_histogram/src/hdr_thread.h b/contrib/libs/hdr_histogram/src/hdr_thread.h index da51b705073..a7d77c02b7d 100644 --- a/contrib/libs/hdr_histogram/src/hdr_thread.h +++ b/contrib/libs/hdr_histogram/src/hdr_thread.h @@ -1,55 +1,55 @@ -/** - * hdr_thread.h - * Written by Philip Orwig and released to the public domain, - * as explained at http://creativecommons.org/publicdomain/zero/1.0/ - */ - -#ifndef HDR_THREAD_H__ -#define HDR_THREAD_H__ - -#include <stdint.h> - -#if defined(_WIN32) || defined(_WIN64) || defined(__CYGWIN__) - - -#define HDR_ALIGN_PREFIX(alignment) __declspec( align(alignment) ) -#define HDR_ALIGN_SUFFIX(alignment) - -typedef struct hdr_mutex -{ - uint8_t _critical_section[40]; -} hdr_mutex; - -#else - -#include <pthread.h> - -#define HDR_ALIGN_PREFIX(alignment) -#define HDR_ALIGN_SUFFIX(alignment) __attribute__((aligned(alignment))) - -typedef struct hdr_mutex -{ - pthread_mutex_t _mutex; -} hdr_mutex; -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -struct hdr_mutex* hdr_mutex_alloc(void); -void hdr_mutex_free(struct hdr_mutex*); - -int hdr_mutex_init(struct hdr_mutex* mutex); -void hdr_mutex_destroy(struct hdr_mutex* mutex); - -void hdr_mutex_lock(struct hdr_mutex* mutex); -void hdr_mutex_unlock(struct hdr_mutex* mutex); - -void hdr_yield(); -int hdr_usleep(unsigned int useconds); - -#ifdef __cplusplus -} -#endif -#endif +/** + * hdr_thread.h + * Written by Philip Orwig and released to the public domain, + * as explained at http://creativecommons.org/publicdomain/zero/1.0/ + */ + +#ifndef HDR_THREAD_H__ +#define HDR_THREAD_H__ + +#include <stdint.h> + +#if defined(_WIN32) || defined(_WIN64) || defined(__CYGWIN__) + + +#define HDR_ALIGN_PREFIX(alignment) __declspec( align(alignment) ) +#define HDR_ALIGN_SUFFIX(alignment) + +typedef struct hdr_mutex +{ + uint8_t _critical_section[40]; +} hdr_mutex; + +#else + +#include <pthread.h> + +#define HDR_ALIGN_PREFIX(alignment) +#define HDR_ALIGN_SUFFIX(alignment) __attribute__((aligned(alignment))) + +typedef struct hdr_mutex +{ + pthread_mutex_t _mutex; +} hdr_mutex; +#endif + +#ifdef __cplusplus +extern "C" { +#endif + +struct hdr_mutex* hdr_mutex_alloc(void); +void hdr_mutex_free(struct hdr_mutex*); + +int hdr_mutex_init(struct hdr_mutex* mutex); +void hdr_mutex_destroy(struct hdr_mutex* mutex); + +void hdr_mutex_lock(struct hdr_mutex* mutex); +void hdr_mutex_unlock(struct hdr_mutex* mutex); + +void hdr_yield(); +int hdr_usleep(unsigned int useconds); + +#ifdef __cplusplus +} +#endif +#endif diff --git a/contrib/libs/hdr_histogram/src/hdr_time.c b/contrib/libs/hdr_histogram/src/hdr_time.c index 584e1142a01..44742076ad9 100644 --- a/contrib/libs/hdr_histogram/src/hdr_time.c +++ b/contrib/libs/hdr_histogram/src/hdr_time.c @@ -1,70 +1,70 @@ -/** -* hdr_time.h -* Written by Michael Barker and Philip Orwig and released to the public domain, -* as explained at http://creativecommons.org/publicdomain/zero/1.0/ -*/ - -#include "hdr_time.h" - -#if defined(_WIN32) || defined(_WIN64) - -#if !defined(WIN32_LEAN_AND_MEAN) -#define WIN32_LEAN_AND_MEAN -#endif - -#include <windows.h> - -static int s_clockPeriodSet = 0; -static double s_clockPeriod = 1.0; - -void hdr_gettime(hdr_timespec* t) -{ - LARGE_INTEGER num; - /* if this is distasteful, we can add in an hdr_time_init() */ - if (!s_clockPeriodSet) - { - QueryPerformanceFrequency(&num); - s_clockPeriod = 1.0 / (double) num.QuadPart; - s_clockPeriodSet = 1; - } - - QueryPerformanceCounter(&num); - double seconds = num.QuadPart * s_clockPeriod; - double integral; - double remainder = modf(seconds, &integral); - - t->tv_sec = (long) integral; - t->tv_nsec = (long) (remainder * 1000000000); -} - -#elif defined(__APPLE__) - -#include <mach/clock.h> -#include <mach/mach.h> - - -void hdr_gettime(hdr_timespec* ts) -{ - clock_serv_t cclock; - mach_timespec_t mts; - host_get_clock_service(mach_host_self(), CALENDAR_CLOCK, &cclock); - clock_get_time(cclock, &mts); - mach_port_deallocate(mach_task_self(), cclock); - ts->tv_sec = mts.tv_sec; - ts->tv_nsec = mts.tv_nsec; -} - -#elif defined(__linux__) || defined(__CYGWIN__) - - -void hdr_gettime(hdr_timespec* t) -{ - clock_gettime(CLOCK_MONOTONIC, (struct timespec*)t); -} - -#else - -#warning "Platform not supported\n" - -#endif - +/** +* hdr_time.h +* Written by Michael Barker and Philip Orwig and released to the public domain, +* as explained at http://creativecommons.org/publicdomain/zero/1.0/ +*/ + +#include "hdr_time.h" + +#if defined(_WIN32) || defined(_WIN64) + +#if !defined(WIN32_LEAN_AND_MEAN) +#define WIN32_LEAN_AND_MEAN +#endif + +#include <windows.h> + +static int s_clockPeriodSet = 0; +static double s_clockPeriod = 1.0; + +void hdr_gettime(hdr_timespec* t) +{ + LARGE_INTEGER num; + /* if this is distasteful, we can add in an hdr_time_init() */ + if (!s_clockPeriodSet) + { + QueryPerformanceFrequency(&num); + s_clockPeriod = 1.0 / (double) num.QuadPart; + s_clockPeriodSet = 1; + } + + QueryPerformanceCounter(&num); + double seconds = num.QuadPart * s_clockPeriod; + double integral; + double remainder = modf(seconds, &integral); + + t->tv_sec = (long) integral; + t->tv_nsec = (long) (remainder * 1000000000); +} + +#elif defined(__APPLE__) + +#include <mach/clock.h> +#include <mach/mach.h> + + +void hdr_gettime(hdr_timespec* ts) +{ + clock_serv_t cclock; + mach_timespec_t mts; + host_get_clock_service(mach_host_self(), CALENDAR_CLOCK, &cclock); + clock_get_time(cclock, &mts); + mach_port_deallocate(mach_task_self(), cclock); + ts->tv_sec = mts.tv_sec; + ts->tv_nsec = mts.tv_nsec; +} + +#elif defined(__linux__) || defined(__CYGWIN__) + + +void hdr_gettime(hdr_timespec* t) +{ + clock_gettime(CLOCK_MONOTONIC, (struct timespec*)t); +} + +#else + +#warning "Platform not supported\n" + +#endif + diff --git a/contrib/libs/hdr_histogram/src/hdr_time.h b/contrib/libs/hdr_histogram/src/hdr_time.h index 829599c3940..8b1a71ec8aa 100644 --- a/contrib/libs/hdr_histogram/src/hdr_time.h +++ b/contrib/libs/hdr_histogram/src/hdr_time.h @@ -1,42 +1,42 @@ -/** - * hdr_time.h - * Written by Michael Barker and released to the public domain, - * as explained at http://creativecommons.org/publicdomain/zero/1.0/ - */ - -#ifndef HDR_TIME_H__ -#define HDR_TIME_H__ - -#include <math.h> -#include <time.h> - -#if defined(_WIN32) || defined(_WIN64) || defined(__CYGWIN__) - -typedef struct hdr_timespec -{ - long tv_sec; - long tv_nsec; -} hdr_timespec; - -#else - -typedef struct timespec hdr_timespec; - -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -#if defined(_MSC_VER) -void hdr_gettime(hdr_timespec* t); -#else -void hdr_gettime(hdr_timespec* t); -#endif - -#ifdef __cplusplus -} -#endif - -#endif - +/** + * hdr_time.h + * Written by Michael Barker and released to the public domain, + * as explained at http://creativecommons.org/publicdomain/zero/1.0/ + */ + +#ifndef HDR_TIME_H__ +#define HDR_TIME_H__ + +#include <math.h> +#include <time.h> + +#if defined(_WIN32) || defined(_WIN64) || defined(__CYGWIN__) + +typedef struct hdr_timespec +{ + long tv_sec; + long tv_nsec; +} hdr_timespec; + +#else + +typedef struct timespec hdr_timespec; + +#endif + +#ifdef __cplusplus +extern "C" { +#endif + +#if defined(_MSC_VER) +void hdr_gettime(hdr_timespec* t); +#else +void hdr_gettime(hdr_timespec* t); +#endif + +#ifdef __cplusplus +} +#endif + +#endif + diff --git a/contrib/libs/hdr_histogram/src/hdr_writer_reader_phaser.c b/contrib/libs/hdr_histogram/src/hdr_writer_reader_phaser.c index 1d3eda8f545..21d3adeeaa2 100644 --- a/contrib/libs/hdr_histogram/src/hdr_writer_reader_phaser.c +++ b/contrib/libs/hdr_histogram/src/hdr_writer_reader_phaser.c @@ -1,135 +1,135 @@ -/** - * hdr_writer_reader_phaser.h - * Written by Michael Barker and released to the public domain, - * as explained at http://creativecommons.org/publicdomain/zero/1.0/ - */ - -#include <stdint.h> -#include <stdbool.h> -#include <errno.h> - -#include "hdr_atomic.h" -#include "hdr_thread.h" - -#include "hdr_writer_reader_phaser.h" - -static int64_t _hdr_phaser_get_epoch(int64_t* field) -{ - return hdr_atomic_load_64(field); -} - -static void _hdr_phaser_set_epoch(int64_t* field, int64_t val) -{ - hdr_atomic_store_64(field, val); -} - -static int64_t _hdr_phaser_reset_epoch(int64_t* field, int64_t initial_value) -{ - return hdr_atomic_exchange_64(field, initial_value); -} - -int hdr_writer_reader_phaser_init(struct hdr_writer_reader_phaser* p) -{ - if (NULL == p) - { - return EINVAL; - } - - p->start_epoch = 0; - p->even_end_epoch = 0; - p->odd_end_epoch = INT64_MIN; - p->reader_mutex = hdr_mutex_alloc(); - - if (!p->reader_mutex) - { - return ENOMEM; - } - - int rc = hdr_mutex_init(p->reader_mutex); - if (0 != rc) - { - return rc; - } - - // TODO: Should I fence here. - - return 0; -} - -void hdr_writer_reader_phaser_destory(struct hdr_writer_reader_phaser* p) -{ - hdr_mutex_destroy(p->reader_mutex); -} - -int64_t hdr_phaser_writer_enter(struct hdr_writer_reader_phaser* p) -{ - return hdr_atomic_add_fetch_64(&p->start_epoch, 1); -} - -void hdr_phaser_writer_exit( - struct hdr_writer_reader_phaser* p, int64_t critical_value_at_enter) -{ - int64_t* end_epoch = - (critical_value_at_enter < 0) ? &p->odd_end_epoch : &p->even_end_epoch; - hdr_atomic_add_fetch_64(end_epoch, 1); -} - -void hdr_phaser_reader_lock(struct hdr_writer_reader_phaser* p) -{ - hdr_mutex_lock(p->reader_mutex); -} - -void hdr_phaser_reader_unlock(struct hdr_writer_reader_phaser* p) -{ - hdr_mutex_unlock(p->reader_mutex); -} - -void hdr_phaser_flip_phase( - struct hdr_writer_reader_phaser* p, int64_t sleep_time_ns) -{ - // TODO: is_held_by_current_thread - unsigned int sleep_time_us = sleep_time_ns < 1000000000 ? (unsigned int) (sleep_time_ns / 1000) : 1000000; - - int64_t start_epoch = _hdr_phaser_get_epoch(&p->start_epoch); - - bool next_phase_is_even = (start_epoch < 0); - - // Clear currently used phase end epoch. - int64_t initial_start_value; - if (next_phase_is_even) - { - initial_start_value = 0; - _hdr_phaser_set_epoch(&p->even_end_epoch, initial_start_value); - } - else - { - initial_start_value = INT64_MIN; - _hdr_phaser_set_epoch(&p->odd_end_epoch, initial_start_value); - } - - // Reset start value, indicating new phase. - int64_t start_value_at_flip = - _hdr_phaser_reset_epoch(&p->start_epoch, initial_start_value); - - bool caught_up = false; - do - { - int64_t* end_epoch = - next_phase_is_even ? &p->odd_end_epoch : &p->even_end_epoch; - - caught_up = _hdr_phaser_get_epoch(end_epoch) == start_value_at_flip; - - if (!caught_up) - { - if (sleep_time_us <= 0) - { - hdr_yield(); - } - else - { - hdr_usleep(sleep_time_us); - } - } - } - while (!caught_up); -} +/** + * hdr_writer_reader_phaser.h + * Written by Michael Barker and released to the public domain, + * as explained at http://creativecommons.org/publicdomain/zero/1.0/ + */ + +#include <stdint.h> +#include <stdbool.h> +#include <errno.h> + +#include "hdr_atomic.h" +#include "hdr_thread.h" + +#include "hdr_writer_reader_phaser.h" + +static int64_t _hdr_phaser_get_epoch(int64_t* field) +{ + return hdr_atomic_load_64(field); +} + +static void _hdr_phaser_set_epoch(int64_t* field, int64_t val) +{ + hdr_atomic_store_64(field, val); +} + +static int64_t _hdr_phaser_reset_epoch(int64_t* field, int64_t initial_value) +{ + return hdr_atomic_exchange_64(field, initial_value); +} + +int hdr_writer_reader_phaser_init(struct hdr_writer_reader_phaser* p) +{ + if (NULL == p) + { + return EINVAL; + } + + p->start_epoch = 0; + p->even_end_epoch = 0; + p->odd_end_epoch = INT64_MIN; + p->reader_mutex = hdr_mutex_alloc(); + + if (!p->reader_mutex) + { + return ENOMEM; + } + + int rc = hdr_mutex_init(p->reader_mutex); + if (0 != rc) + { + return rc; + } + + // TODO: Should I fence here. + + return 0; +} + +void hdr_writer_reader_phaser_destory(struct hdr_writer_reader_phaser* p) +{ + hdr_mutex_destroy(p->reader_mutex); +} + +int64_t hdr_phaser_writer_enter(struct hdr_writer_reader_phaser* p) +{ + return hdr_atomic_add_fetch_64(&p->start_epoch, 1); +} + +void hdr_phaser_writer_exit( + struct hdr_writer_reader_phaser* p, int64_t critical_value_at_enter) +{ + int64_t* end_epoch = + (critical_value_at_enter < 0) ? &p->odd_end_epoch : &p->even_end_epoch; + hdr_atomic_add_fetch_64(end_epoch, 1); +} + +void hdr_phaser_reader_lock(struct hdr_writer_reader_phaser* p) +{ + hdr_mutex_lock(p->reader_mutex); +} + +void hdr_phaser_reader_unlock(struct hdr_writer_reader_phaser* p) +{ + hdr_mutex_unlock(p->reader_mutex); +} + +void hdr_phaser_flip_phase( + struct hdr_writer_reader_phaser* p, int64_t sleep_time_ns) +{ + // TODO: is_held_by_current_thread + unsigned int sleep_time_us = sleep_time_ns < 1000000000 ? (unsigned int) (sleep_time_ns / 1000) : 1000000; + + int64_t start_epoch = _hdr_phaser_get_epoch(&p->start_epoch); + + bool next_phase_is_even = (start_epoch < 0); + + // Clear currently used phase end epoch. + int64_t initial_start_value; + if (next_phase_is_even) + { + initial_start_value = 0; + _hdr_phaser_set_epoch(&p->even_end_epoch, initial_start_value); + } + else + { + initial_start_value = INT64_MIN; + _hdr_phaser_set_epoch(&p->odd_end_epoch, initial_start_value); + } + + // Reset start value, indicating new phase. + int64_t start_value_at_flip = + _hdr_phaser_reset_epoch(&p->start_epoch, initial_start_value); + + bool caught_up = false; + do + { + int64_t* end_epoch = + next_phase_is_even ? &p->odd_end_epoch : &p->even_end_epoch; + + caught_up = _hdr_phaser_get_epoch(end_epoch) == start_value_at_flip; + + if (!caught_up) + { + if (sleep_time_us <= 0) + { + hdr_yield(); + } + else + { + hdr_usleep(sleep_time_us); + } + } + } + while (!caught_up); +} diff --git a/contrib/libs/hdr_histogram/src/hdr_writer_reader_phaser.h b/contrib/libs/hdr_histogram/src/hdr_writer_reader_phaser.h index 3ee0a57ba4a..f14a7db6e7d 100644 --- a/contrib/libs/hdr_histogram/src/hdr_writer_reader_phaser.h +++ b/contrib/libs/hdr_histogram/src/hdr_writer_reader_phaser.h @@ -1,51 +1,51 @@ -/** - * hdr_writer_reader_phaser.h - * Written by Michael Barker and released to the public domain, - * as explained at http://creativecommons.org/publicdomain/zero/1.0/ - */ - -#ifndef HDR_WRITER_READER_PHASER_H -#define HDR_WRITER_READER_PHASER_H 1 - -#include <stdlib.h> -#include <stdbool.h> -#include <stdlib.h> -#include <errno.h> - -#include "hdr_thread.h" - -HDR_ALIGN_PREFIX(8) -struct hdr_writer_reader_phaser -{ - int64_t start_epoch; - int64_t even_end_epoch; - int64_t odd_end_epoch; - hdr_mutex* reader_mutex; -} -HDR_ALIGN_SUFFIX(8); - -#ifdef __cplusplus -extern "C" { -#endif - - int hdr_writer_reader_phaser_init(struct hdr_writer_reader_phaser* p); - - void hdr_writer_reader_phaser_destory(struct hdr_writer_reader_phaser* p); - - int64_t hdr_phaser_writer_enter(struct hdr_writer_reader_phaser* p); - - void hdr_phaser_writer_exit( - struct hdr_writer_reader_phaser* p, int64_t critical_value_at_enter); - - void hdr_phaser_reader_lock(struct hdr_writer_reader_phaser* p); - - void hdr_phaser_reader_unlock(struct hdr_writer_reader_phaser* p); - - void hdr_phaser_flip_phase( - struct hdr_writer_reader_phaser* p, int64_t sleep_time_ns); - -#ifdef __cplusplus +/** + * hdr_writer_reader_phaser.h + * Written by Michael Barker and released to the public domain, + * as explained at http://creativecommons.org/publicdomain/zero/1.0/ + */ + +#ifndef HDR_WRITER_READER_PHASER_H +#define HDR_WRITER_READER_PHASER_H 1 + +#include <stdlib.h> +#include <stdbool.h> +#include <stdlib.h> +#include <errno.h> + +#include "hdr_thread.h" + +HDR_ALIGN_PREFIX(8) +struct hdr_writer_reader_phaser +{ + int64_t start_epoch; + int64_t even_end_epoch; + int64_t odd_end_epoch; + hdr_mutex* reader_mutex; } -#endif - -#endif +HDR_ALIGN_SUFFIX(8); + +#ifdef __cplusplus +extern "C" { +#endif + + int hdr_writer_reader_phaser_init(struct hdr_writer_reader_phaser* p); + + void hdr_writer_reader_phaser_destory(struct hdr_writer_reader_phaser* p); + + int64_t hdr_phaser_writer_enter(struct hdr_writer_reader_phaser* p); + + void hdr_phaser_writer_exit( + struct hdr_writer_reader_phaser* p, int64_t critical_value_at_enter); + + void hdr_phaser_reader_lock(struct hdr_writer_reader_phaser* p); + + void hdr_phaser_reader_unlock(struct hdr_writer_reader_phaser* p); + + void hdr_phaser_flip_phase( + struct hdr_writer_reader_phaser* p, int64_t sleep_time_ns); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/contrib/libs/hdr_histogram/ya.make b/contrib/libs/hdr_histogram/ya.make index c2d1cc92c9d..b20d977b573 100644 --- a/contrib/libs/hdr_histogram/ya.make +++ b/contrib/libs/hdr_histogram/ya.make @@ -1,5 +1,5 @@ -LIBRARY() - +LIBRARY() + LICENSE( BSD-2-Clause AND CC0-1.0 @@ -9,31 +9,31 @@ LICENSE_TEXTS(.yandex_meta/licenses.list.txt) VERSION(0.9.5) -OWNER( - jamel +OWNER( + jamel g:contrib g:cpp-contrib -) - -NO_UTIL() +) + +NO_UTIL() + +NO_COMPILER_WARNINGS() -NO_COMPILER_WARNINGS() - ADDINCL( contrib/libs/hdr_histogram/src ) - -SRCS( - src/hdr_encoding.c - src/hdr_interval_recorder.c - src/hdr_histogram.c - src/hdr_writer_reader_phaser.c - src/hdr_time.c - src/hdr_thread.c -) - -PEERDIR( - contrib/libs/zlib -) - -END() + +SRCS( + src/hdr_encoding.c + src/hdr_interval_recorder.c + src/hdr_histogram.c + src/hdr_writer_reader_phaser.c + src/hdr_time.c + src/hdr_thread.c +) + +PEERDIR( + contrib/libs/zlib +) + +END() diff --git a/contrib/libs/ya.make b/contrib/libs/ya.make index f5c7c53f299..9c4640fdcf2 100644 --- a/contrib/libs/ya.make +++ b/contrib/libs/ya.make @@ -100,7 +100,7 @@ RECURSE( h3 harfbuzz hdf5 - hdr_histogram + hdr_histogram highwayhash hiredis httpd24 diff --git a/library/cpp/actors/core/actor.cpp b/library/cpp/actors/core/actor.cpp index c84b8d7e2f4..6f9ba6a42bb 100644 --- a/library/cpp/actors/core/actor.cpp +++ b/library/cpp/actors/core/actor.cpp @@ -7,7 +7,7 @@ namespace NActors { Y_POD_THREAD(TActivationContext*) TlsActivationContext((TActivationContext*)nullptr); - bool TActorContext::Send(const TActorId& recipient, IEventBase* ev, ui32 flags, ui64 cookie, NWilson::TTraceId traceId) const { + bool TActorContext::Send(const TActorId& recipient, IEventBase* ev, ui32 flags, ui64 cookie, NWilson::TTraceId traceId) const { return Send(new IEventHandle(recipient, SelfID, ev, flags, cookie, nullptr, std::move(traceId))); } @@ -15,7 +15,7 @@ namespace NActors { return ExecutorThread.Send(ev); } - void IActor::Registered(TActorSystem* sys, const TActorId& owner) { + void IActor::Registered(TActorSystem* sys, const TActorId& owner) { // fallback to legacy method, do not use it anymore if (auto eh = AfterRegister(SelfId(), owner)) sys->Send(eh); @@ -25,7 +25,7 @@ namespace NActors { SelfActorId.Out(out); } - bool IActor::Send(const TActorId& recipient, IEventBase* ev, ui32 flags, ui64 cookie, NWilson::TTraceId traceId) const noexcept { + bool IActor::Send(const TActorId& recipient, IEventBase* ev, ui32 flags, ui64 cookie, NWilson::TTraceId traceId) const noexcept { return SelfActorId.Send(recipient, ev, flags, cookie, std::move(traceId)); } @@ -33,19 +33,19 @@ namespace NActors { return TlsActivationContext->ExecutorThread.Send(ev); } - void TActivationContext::Schedule(TInstant deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie) { - TlsActivationContext->ExecutorThread.Schedule(deadline, ev, cookie); - } - + void TActivationContext::Schedule(TInstant deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie) { + TlsActivationContext->ExecutorThread.Schedule(deadline, ev, cookie); + } + void TActivationContext::Schedule(TMonotonic deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie) { TlsActivationContext->ExecutorThread.Schedule(deadline, ev, cookie); } void TActivationContext::Schedule(TDuration delta, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie) { - TlsActivationContext->ExecutorThread.Schedule(delta, ev, cookie); + TlsActivationContext->ExecutorThread.Schedule(delta, ev, cookie); } - bool TActorIdentity::Send(const TActorId& recipient, IEventBase* ev, ui32 flags, ui64 cookie, NWilson::TTraceId traceId) const { + bool TActorIdentity::Send(const TActorId& recipient, IEventBase* ev, ui32 flags, ui64 cookie, NWilson::TTraceId traceId) const { return TActivationContext::Send(new IEventHandle(recipient, *this, ev, flags, cookie, nullptr, std::move(traceId))); } @@ -75,11 +75,11 @@ namespace NActors { return TlsActivationContext->ExecutorThread.RegisterActor(actor, &TlsActivationContext->Mailbox, SelfActorId.Hint(), SelfActorId); } - TActorId TActivationContext::Register(IActor* actor, TActorId parentId, TMailboxType::EType mailboxType, ui32 poolId) { + TActorId TActivationContext::Register(IActor* actor, TActorId parentId, TMailboxType::EType mailboxType, ui32 poolId) { return TlsActivationContext->ExecutorThread.RegisterActor(actor, mailboxType, poolId, parentId); } - TActorId TActivationContext::InterconnectProxy(ui32 destinationNodeId) { + TActorId TActivationContext::InterconnectProxy(ui32 destinationNodeId) { return TlsActivationContext->ExecutorThread.ActorSystem->InterconnectProxy(destinationNodeId); } @@ -95,36 +95,36 @@ namespace NActors { return NHPTimer::GetSeconds(GetCurrentEventTicks()); } - TActorId TActorContext::Register(IActor* actor, TMailboxType::EType mailboxType, ui32 poolId) const { + TActorId TActorContext::Register(IActor* actor, TMailboxType::EType mailboxType, ui32 poolId) const { return ExecutorThread.RegisterActor(actor, mailboxType, poolId, SelfID); } - TActorId IActor::Register(IActor* actor, TMailboxType::EType mailboxType, ui32 poolId) const noexcept { + TActorId IActor::Register(IActor* actor, TMailboxType::EType mailboxType, ui32 poolId) const noexcept { return TlsActivationContext->ExecutorThread.RegisterActor(actor, mailboxType, poolId, SelfActorId); } - void TActorContext::Schedule(TInstant deadline, IEventBase* ev, ISchedulerCookie* cookie) const { - ExecutorThread.Schedule(deadline, new IEventHandle(SelfID, TActorId(), ev), cookie); - } - + void TActorContext::Schedule(TInstant deadline, IEventBase* ev, ISchedulerCookie* cookie) const { + ExecutorThread.Schedule(deadline, new IEventHandle(SelfID, TActorId(), ev), cookie); + } + void TActorContext::Schedule(TMonotonic deadline, IEventBase* ev, ISchedulerCookie* cookie) const { ExecutorThread.Schedule(deadline, new IEventHandle(SelfID, TActorId(), ev), cookie); } void TActorContext::Schedule(TDuration delta, IEventBase* ev, ISchedulerCookie* cookie) const { - ExecutorThread.Schedule(delta, new IEventHandle(SelfID, TActorId(), ev), cookie); + ExecutorThread.Schedule(delta, new IEventHandle(SelfID, TActorId(), ev), cookie); + } + + void IActor::Schedule(TInstant deadline, IEventBase* ev, ISchedulerCookie* cookie) const noexcept { + TlsActivationContext->ExecutorThread.Schedule(deadline, new IEventHandle(SelfActorId, TActorId(), ev), cookie); } - void IActor::Schedule(TInstant deadline, IEventBase* ev, ISchedulerCookie* cookie) const noexcept { - TlsActivationContext->ExecutorThread.Schedule(deadline, new IEventHandle(SelfActorId, TActorId(), ev), cookie); - } - void IActor::Schedule(TMonotonic deadline, IEventBase* ev, ISchedulerCookie* cookie) const noexcept { TlsActivationContext->ExecutorThread.Schedule(deadline, new IEventHandle(SelfActorId, TActorId(), ev), cookie); } void IActor::Schedule(TDuration delta, IEventBase* ev, ISchedulerCookie* cookie) const noexcept { - TlsActivationContext->ExecutorThread.Schedule(delta, new IEventHandle(SelfActorId, TActorId(), ev), cookie); + TlsActivationContext->ExecutorThread.Schedule(delta, new IEventHandle(SelfActorId, TActorId(), ev), cookie); } TInstant TActivationContext::Now() { diff --git a/library/cpp/actors/core/actor.h b/library/cpp/actors/core/actor.h index 8bfef6b5bd4..ed29bd14b9e 100644 --- a/library/cpp/actors/core/actor.h +++ b/library/cpp/actors/core/actor.h @@ -36,17 +36,17 @@ namespace NActors { public: static bool Send(TAutoPtr<IEventHandle> ev); - - /** - * Schedule one-shot event that will be send at given time point in the future. - * + + /** + * Schedule one-shot event that will be send at given time point in the future. + * * @param deadline the wallclock time point in future when event must be send - * @param ev the event to send - * @param cookie cookie that will be piggybacked with event - */ - static void Schedule(TInstant deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie = nullptr); - - /** + * @param ev the event to send + * @param cookie cookie that will be piggybacked with event + */ + static void Schedule(TInstant deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie = nullptr); + + /** * Schedule one-shot event that will be send at given time point in the future. * * @param deadline the monotonic time point in future when event must be send @@ -56,12 +56,12 @@ namespace NActors { static void Schedule(TMonotonic deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie = nullptr); /** - * Schedule one-shot event that will be send after given delay. - * - * @param delta the time from now to delay event sending - * @param ev the event to send - * @param cookie cookie that will be piggybacked with event - */ + * Schedule one-shot event that will be send after given delay. + * + * @param delta the time from now to delay event sending + * @param ev the event to send + * @param cookie cookie that will be piggybacked with event + */ static void Schedule(TDuration delta, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie = nullptr); static TInstant Now(); @@ -79,9 +79,9 @@ namespace NActors { static TActorId RegisterWithSameMailbox(IActor* actor, TActorId parentId); static const TActorContext& AsActorContext(); - static TActorContext ActorContextFor(TActorId id); + static TActorContext ActorContextFor(TActorId id); - static TActorId InterconnectProxy(ui32 nodeid); + static TActorId InterconnectProxy(ui32 nodeid); static TActorSystem* ActorSystem(); static i64 GetCurrentEventTicks(); @@ -89,34 +89,34 @@ namespace NActors { }; struct TActorContext: public TActivationContext { - const TActorId SelfID; + const TActorId SelfID; - explicit TActorContext(TMailboxHeader& mailbox, TExecutorThread& executorThread, NHPTimer::STime eventStart, const TActorId& selfID) + explicit TActorContext(TMailboxHeader& mailbox, TExecutorThread& executorThread, NHPTimer::STime eventStart, const TActorId& selfID) : TActivationContext(mailbox, executorThread, eventStart) , SelfID(selfID) { } - bool Send(const TActorId& recipient, IEventBase* ev, ui32 flags = 0, ui64 cookie = 0, NWilson::TTraceId traceId = {}) const; + bool Send(const TActorId& recipient, IEventBase* ev, ui32 flags = 0, ui64 cookie = 0, NWilson::TTraceId traceId = {}) const; template <typename TEvent> - bool Send(const TActorId& recipient, THolder<TEvent> ev, ui32 flags = 0, ui64 cookie = 0, NWilson::TTraceId traceId = {}) const { + bool Send(const TActorId& recipient, THolder<TEvent> ev, ui32 flags = 0, ui64 cookie = 0, NWilson::TTraceId traceId = {}) const { return Send(recipient, static_cast<IEventBase*>(ev.Release()), flags, cookie, std::move(traceId)); } bool Send(TAutoPtr<IEventHandle> ev) const; - + TInstant Now() const; TMonotonic Monotonic() const; - /** - * Schedule one-shot event that will be send at given time point in the future. - * + /** + * Schedule one-shot event that will be send at given time point in the future. + * * @param deadline the wallclock time point in future when event must be send - * @param ev the event to send - * @param cookie cookie that will be piggybacked with event - */ - void Schedule(TInstant deadline, IEventBase* ev, ISchedulerCookie* cookie = nullptr) const; - - /** + * @param ev the event to send + * @param cookie cookie that will be piggybacked with event + */ + void Schedule(TInstant deadline, IEventBase* ev, ISchedulerCookie* cookie = nullptr) const; + + /** * Schedule one-shot event that will be send at given time point in the future. * * @param deadline the monotonic time point in future when event must be send @@ -126,20 +126,20 @@ namespace NActors { void Schedule(TMonotonic deadline, IEventBase* ev, ISchedulerCookie* cookie = nullptr) const; /** - * Schedule one-shot event that will be send after given delay. - * - * @param delta the time from now to delay event sending - * @param ev the event to send - * @param cookie cookie that will be piggybacked with event - */ + * Schedule one-shot event that will be send after given delay. + * + * @param delta the time from now to delay event sending + * @param ev the event to send + * @param cookie cookie that will be piggybacked with event + */ void Schedule(TDuration delta, IEventBase* ev, ISchedulerCookie* cookie = nullptr) const; - - TActorContext MakeFor(const TActorId& otherId) const { + + TActorContext MakeFor(const TActorId& otherId) const { return TActorContext(Mailbox, ExecutorThread, EventStart, otherId); } // register new actor in ActorSystem on new fresh mailbox. - TActorId Register(IActor* actor, TMailboxType::EType mailboxType = TMailboxType::HTSwap, ui32 poolId = Max<ui32>()) const; + TActorId Register(IActor* actor, TMailboxType::EType mailboxType = TMailboxType::HTSwap, ui32 poolId = Max<ui32>()) const; // Register new actor in ActorSystem on same _mailbox_ as current actor. // There is one thread per mailbox to execute actor, which mean @@ -153,17 +153,17 @@ namespace NActors { extern Y_POD_THREAD(TActivationContext*) TlsActivationContext; - struct TActorIdentity: public TActorId { - explicit TActorIdentity(TActorId actorId) - : TActorId(actorId) + struct TActorIdentity: public TActorId { + explicit TActorIdentity(TActorId actorId) + : TActorId(actorId) { } - void operator=(TActorId actorId) { + void operator=(TActorId actorId) { *this = TActorIdentity(actorId); } - bool Send(const TActorId& recipient, IEventBase* ev, ui32 flags = 0, ui64 cookie = 0, NWilson::TTraceId traceId = {}) const; + bool Send(const TActorId& recipient, IEventBase* ev, ui32 flags = 0, ui64 cookie = 0, NWilson::TTraceId traceId = {}) const; void Schedule(TInstant deadline, IEventBase* ev, ISchedulerCookie* cookie = nullptr) const; void Schedule(TMonotonic deadline, IEventBase* ev, ISchedulerCookie* cookie = nullptr) const; void Schedule(TDuration delta, IEventBase* ev, ISchedulerCookie* cookie = nullptr) const; @@ -174,18 +174,18 @@ namespace NActors { class IActorOps : TNonCopyable { public: virtual void Describe(IOutputStream&) const noexcept = 0; - virtual bool Send(const TActorId& recipient, IEventBase*, ui32 flags = 0, ui64 cookie = 0, NWilson::TTraceId traceId = {}) const noexcept = 0; - - /** - * Schedule one-shot event that will be send at given time point in the future. - * + virtual bool Send(const TActorId& recipient, IEventBase*, ui32 flags = 0, ui64 cookie = 0, NWilson::TTraceId traceId = {}) const noexcept = 0; + + /** + * Schedule one-shot event that will be send at given time point in the future. + * * @param deadline the wallclock time point in future when event must be send - * @param ev the event to send - * @param cookie cookie that will be piggybacked with event - */ - virtual void Schedule(TInstant deadline, IEventBase* ev, ISchedulerCookie* cookie = nullptr) const noexcept = 0; - - /** + * @param ev the event to send + * @param cookie cookie that will be piggybacked with event + */ + virtual void Schedule(TInstant deadline, IEventBase* ev, ISchedulerCookie* cookie = nullptr) const noexcept = 0; + + /** * Schedule one-shot event that will be send at given time point in the future. * * @param deadline the monotonic time point in future when event must be send @@ -195,15 +195,15 @@ namespace NActors { virtual void Schedule(TMonotonic deadline, IEventBase* ev, ISchedulerCookie* cookie = nullptr) const noexcept = 0; /** - * Schedule one-shot event that will be send after given delay. - * - * @param delta the time from now to delay event sending - * @param ev the event to send - * @param cookie cookie that will be piggybacked with event - */ - virtual void Schedule(TDuration delta, IEventBase* ev, ISchedulerCookie* cookie = nullptr) const noexcept = 0; - - virtual TActorId Register(IActor*, TMailboxType::EType mailboxType = TMailboxType::HTSwap, ui32 poolId = Max<ui32>()) const noexcept = 0; + * Schedule one-shot event that will be send after given delay. + * + * @param delta the time from now to delay event sending + * @param ev the event to send + * @param cookie cookie that will be piggybacked with event + */ + virtual void Schedule(TDuration delta, IEventBase* ev, ISchedulerCookie* cookie = nullptr) const noexcept = 0; + + virtual TActorId Register(IActor*, TMailboxType::EType mailboxType = TMailboxType::HTSwap, ui32 poolId = Max<ui32>()) const noexcept = 0; virtual TActorId RegisterWithSameMailbox(IActor*) const noexcept = 0; }; @@ -219,7 +219,7 @@ namespace NActors { i64 ElapsedTicks; ui64 HandledEvents; - friend void DoActorInit(TActorSystem*, IActor*, const TActorId&, const TActorId&); + friend void DoActorInit(TActorSystem*, IActor*, const TActorId&, const TActorId&); friend class TDecorator; public: @@ -254,7 +254,7 @@ namespace NActors { protected: IActor(TReceiveFunc stateFunc, ui32 activityType = OTHER) : StateFunc(stateFunc) - , SelfActorId(TActorId()) + , SelfActorId(TActorId()) , ElapsedTicks(0) , HandledEvents(0) , ActivityType(activityType) @@ -310,7 +310,7 @@ namespace NActors { InvokeOtherActor(TActor& actor, TMethod&& method, TArgs&&... args) { struct TRecurseContext : TActorContext { TActivationContext *Prev; - TRecurseContext(const TActorId& actorId) + TRecurseContext(const TActorId& actorId) : TActorContext(TActivationContext::ActorContextFor(actorId)) , Prev(TlsActivationContext) { @@ -323,9 +323,9 @@ namespace NActors { return (actor.*method)(std::forward<TArgs>(args)...); } - virtual void Registered(TActorSystem* sys, const TActorId& owner); + virtual void Registered(TActorSystem* sys, const TActorId& owner); - virtual TAutoPtr<IEventHandle> AfterRegister(const TActorId& self, const TActorId& parentId) { + virtual TAutoPtr<IEventHandle> AfterRegister(const TActorId& self, const TActorId& parentId) { Y_UNUSED(self); Y_UNUSED(parentId); return TAutoPtr<IEventHandle>(); @@ -350,23 +350,23 @@ namespace NActors { protected: void Describe(IOutputStream&) const noexcept override; - bool Send(const TActorId& recipient, IEventBase* ev, ui32 flags = 0, ui64 cookie = 0, NWilson::TTraceId traceId = {}) const noexcept final; + bool Send(const TActorId& recipient, IEventBase* ev, ui32 flags = 0, ui64 cookie = 0, NWilson::TTraceId traceId = {}) const noexcept final; template <typename TEvent> - bool Send(const TActorId& recipient, THolder<TEvent> ev, ui32 flags = 0, ui64 cookie = 0, NWilson::TTraceId traceId = {}) const{ + bool Send(const TActorId& recipient, THolder<TEvent> ev, ui32 flags = 0, ui64 cookie = 0, NWilson::TTraceId traceId = {}) const{ return Send(recipient, static_cast<IEventBase*>(ev.Release()), flags, cookie, std::move(traceId)); } - + template <class TEvent, class ... TEventArgs> bool Send(TActorId recipient, TEventArgs&& ... args) const { return Send(recipient, MakeHolder<TEvent>(std::forward<TEventArgs>(args)...)); } - void Schedule(TInstant deadline, IEventBase* ev, ISchedulerCookie* cookie = nullptr) const noexcept final; + void Schedule(TInstant deadline, IEventBase* ev, ISchedulerCookie* cookie = nullptr) const noexcept final; void Schedule(TMonotonic deadline, IEventBase* ev, ISchedulerCookie* cookie = nullptr) const noexcept final; void Schedule(TDuration delta, IEventBase* ev, ISchedulerCookie* cookie = nullptr) const noexcept final; // register new actor in ActorSystem on new fresh mailbox. - TActorId Register(IActor* actor, TMailboxType::EType mailboxType = TMailboxType::HTSwap, ui32 poolId = Max<ui32>()) const noexcept final; + TActorId Register(IActor* actor, TMailboxType::EType mailboxType = TMailboxType::HTSwap, ui32 poolId = Max<ui32>()) const noexcept final; // Register new actor in ActorSystem on same _mailbox_ as current actor. // There is one thread per mailbox to execute actor, which mean @@ -454,7 +454,7 @@ namespace NActors { return *static_cast<TActorContext*>(tls); } - inline TActorContext TActivationContext::ActorContextFor(TActorId id) { + inline TActorContext TActivationContext::ActorContextFor(TActorId id) { auto& tls = *TlsActivationContext; return TActorContext(tls.Mailbox, tls.ExecutorThread, tls.EventStart, id); } diff --git a/library/cpp/actors/core/actor_bootstrapped.h b/library/cpp/actors/core/actor_bootstrapped.h index 25e6b3fbe20..a37887c9398 100644 --- a/library/cpp/actors/core/actor_bootstrapped.h +++ b/library/cpp/actors/core/actor_bootstrapped.h @@ -9,7 +9,7 @@ namespace NActors { template<typename TDerived> class TActorBootstrapped : public TActor<TDerived> { protected: - TAutoPtr<IEventHandle> AfterRegister(const TActorId& self, const TActorId& parentId) override { + TAutoPtr<IEventHandle> AfterRegister(const TActorId& self, const TActorId& parentId) override { return new IEventHandle(TEvents::TSystem::Bootstrap, 0, self, parentId, {}, 0); } @@ -19,11 +19,11 @@ namespace NActors { TDerived& self = static_cast<TDerived&>(*this); if constexpr (std::is_invocable_v<T, TDerived, const TActorContext&>) { self.Bootstrap(ctx); - } else if constexpr (std::is_invocable_v<T, TDerived, const TActorId&, const TActorContext&>) { + } else if constexpr (std::is_invocable_v<T, TDerived, const TActorId&, const TActorContext&>) { self.Bootstrap(ev->Sender, ctx); } else if constexpr (std::is_invocable_v<T, TDerived>) { self.Bootstrap(); - } else if constexpr (std::is_invocable_v<T, TDerived, const TActorId&>) { + } else if constexpr (std::is_invocable_v<T, TDerived, const TActorId&>) { self.Bootstrap(ev->Sender); } else { static_assert(dependent_false<TDerived>::value, "No correct Bootstrap() signature"); diff --git a/library/cpp/actors/core/actor_coroutine.h b/library/cpp/actors/core/actor_coroutine.h index af7328d3a1f..6bcb768eafc 100644 --- a/library/cpp/actors/core/actor_coroutine.h +++ b/library/cpp/actors/core/actor_coroutine.h @@ -24,8 +24,8 @@ namespace NActors { TActorContext *ActorContext = nullptr; protected: - TActorIdentity SelfActorId = TActorIdentity(TActorId()); - TActorId ParentActorId; + TActorIdentity SelfActorId = TActorIdentity(TActorId()); + TActorId ParentActorId; private: template <typename TFirstEvent, typename... TOtherEvents> @@ -107,12 +107,12 @@ namespace NActors { TActorSystem *GetActorSystem() const { return GetActorContext().ExecutorThread.ActorSystem; } TInstant Now() const { return GetActorContext().Now(); } - bool Send(const TActorId& recipient, IEventBase* ev, ui32 flags = 0, ui64 cookie = 0, NWilson::TTraceId traceId = {}) { + bool Send(const TActorId& recipient, IEventBase* ev, ui32 flags = 0, ui64 cookie = 0, NWilson::TTraceId traceId = {}) { return GetActorContext().Send(recipient, ev, flags, cookie, std::move(traceId)); } template <typename TEvent> - bool Send(const TActorId& recipient, THolder<TEvent> ev, ui32 flags = 0, ui64 cookie = 0, NWilson::TTraceId traceId = {}) { + bool Send(const TActorId& recipient, THolder<TEvent> ev, ui32 flags = 0, ui64 cookie = 0, NWilson::TTraceId traceId = {}) { return GetActorContext().Send(recipient, ev.Release(), flags, cookie, std::move(traceId)); } @@ -130,7 +130,7 @@ namespace NActors { return GetActorContext().Schedule(deadline, ev, cookie); } - TActorId Register(IActor* actor, TMailboxType::EType mailboxType = TMailboxType::HTSwap, ui32 poolId = Max<ui32>()) { + TActorId Register(IActor* actor, TMailboxType::EType mailboxType = TMailboxType::HTSwap, ui32 poolId = Max<ui32>()) { return GetActorContext().Register(actor, mailboxType, poolId); } @@ -159,7 +159,7 @@ namespace NActors { , Impl(std::move(impl)) {} - TAutoPtr<IEventHandle> AfterRegister(const TActorId& self, const TActorId& parent) override { + TAutoPtr<IEventHandle> AfterRegister(const TActorId& self, const TActorId& parent) override { return new IEventHandle(TEvents::TSystem::Bootstrap, 0, self, parent, {}, 0); } diff --git a/library/cpp/actors/core/actor_coroutine_ut.cpp b/library/cpp/actors/core/actor_coroutine_ut.cpp index 13bfdcc2bfc..951512b8772 100644 --- a/library/cpp/actors/core/actor_coroutine_ut.cpp +++ b/library/cpp/actors/core/actor_coroutine_ut.cpp @@ -29,7 +29,7 @@ Y_UNIT_TEST_SUITE(ActorCoro) { }; class TBasicResponderActor: public TActorBootstrapped<TBasicResponderActor> { - TDeque<TActorId> RespondTo; + TDeque<TActorId> RespondTo; public: TBasicResponderActor() { @@ -77,7 +77,7 @@ Y_UNIT_TEST_SUITE(ActorCoro) { } void Run() override { - TActorId child = GetActorContext().Register(new TBasicResponderActor); + TActorId child = GetActorContext().Register(new TBasicResponderActor); ui32 itemsProcessed = 0; try { while (!Finish) { diff --git a/library/cpp/actors/core/actorid.cpp b/library/cpp/actors/core/actorid.cpp index 3330cd0ab3d..ccda035eac0 100644 --- a/library/cpp/actors/core/actorid.cpp +++ b/library/cpp/actors/core/actorid.cpp @@ -3,18 +3,18 @@ #include <util/string/cast.h> namespace NActors { - void TActorId::Out(IOutputStream& o) const { + void TActorId::Out(IOutputStream& o) const { o << "[" << NodeId() << ":" << LocalId() << ":" << Hint() << "]"; } - TString TActorId::ToString() const { + TString TActorId::ToString() const { TString x; TStringOutput o(x); Out(o); return x; } - bool TActorId::Parse(const char* buf, ui32 sz) { + bool TActorId::Parse(const char* buf, ui32 sz) { if (sz < 4 || buf[0] != '[' || buf[sz - 1] != ']') return false; diff --git a/library/cpp/actors/core/actorid.h b/library/cpp/actors/core/actorid.h index c631ef3a721..d972b1a0ff1 100644 --- a/library/cpp/actors/core/actorid.h +++ b/library/cpp/actors/core/actorid.h @@ -11,7 +11,7 @@ namespace NActors { // next 11 bits of node-id - pool id // next 20 bits - node id itself - struct TActorId { + struct TActorId { static constexpr ui32 MaxServiceIDLength = 12; static constexpr ui32 MaxPoolID = 0x000007FF; static constexpr ui32 MaxNodeId = 0x000FFFFF; @@ -37,19 +37,19 @@ namespace NActors { } Raw; public: - TActorId() noexcept { + TActorId() noexcept { Raw.X.X1 = 0; Raw.X.X2 = 0; } - explicit TActorId(ui32 nodeId, ui32 poolId, ui64 localId, ui32 hint) noexcept { + explicit TActorId(ui32 nodeId, ui32 poolId, ui64 localId, ui32 hint) noexcept { Y_VERIFY_DEBUG(poolId <= MaxPoolID); Raw.N.LocalId = localId; Raw.N.Hint = hint; Raw.N.NodeId = nodeId | (poolId << PoolIndexShift); } - explicit TActorId(ui32 nodeId, const TStringBuf& x) noexcept { + explicit TActorId(ui32 nodeId, const TStringBuf& x) noexcept { Y_VERIFY(x.size() <= MaxServiceIDLength, "service id is too long"); Raw.N.LocalId = 0; Raw.N.Hint = 0; @@ -57,7 +57,7 @@ namespace NActors { memcpy(Raw.Buf, x.data(), x.size()); } - explicit TActorId(ui64 x1, ui64 x2) noexcept { + explicit TActorId(ui64 x1, ui64 x2) noexcept { Raw.X.X1 = x1; Raw.X.X2 = x2; } @@ -103,7 +103,7 @@ namespace NActors { return Raw.X.X2; } - bool operator<(const TActorId& x) const noexcept { + bool operator<(const TActorId& x) const noexcept { const ui64 s1 = Raw.X.X1; const ui64 s2 = Raw.X.X2; const ui64 x1 = x.Raw.X.X1; @@ -112,11 +112,11 @@ namespace NActors { return (s1 != x1) ? (s1 < x1) : (s2 < x2); } - bool operator!=(const TActorId& x) const noexcept { + bool operator!=(const TActorId& x) const noexcept { return Raw.X.X1 != x.Raw.X.X1 || Raw.X.X2 != x.Raw.X.X2; } - bool operator==(const TActorId& x) const noexcept { + bool operator==(const TActorId& x) const noexcept { return !(x != *this); } @@ -153,19 +153,19 @@ namespace NActors { } struct THash { - ui64 operator()(const TActorId& actorId) const noexcept { - return actorId.Hash(); + ui64 operator()(const TActorId& actorId) const noexcept { + return actorId.Hash(); } }; struct THash32 { - ui64 operator()(const TActorId& actorId) const noexcept { - return actorId.Hash(); + ui64 operator()(const TActorId& actorId) const noexcept { + return actorId.Hash(); } }; struct TOrderedCmp { - bool operator()(const TActorId &left, const TActorId &right) const noexcept { + bool operator()(const TActorId &left, const TActorId &right) const noexcept { Y_VERIFY_DEBUG(!left.IsService() && !right.IsService(), "ordered compare works for plain actorids only"); const ui32 n1 = left.NodeId(); const ui32 n2 = right.NodeId(); @@ -179,18 +179,18 @@ namespace NActors { bool Parse(const char* buf, ui32 sz); }; - static_assert(sizeof(TActorId) == 16, "expect sizeof(TActorId) == 16"); + static_assert(sizeof(TActorId) == 16, "expect sizeof(TActorId) == 16"); static_assert(MaxPools < TActorId::MaxPoolID); // current implementation of united pool has limit MaxPools on pool id } template <> -inline void Out<NActors::TActorId>(IOutputStream& o, const NActors::TActorId& x) { +inline void Out<NActors::TActorId>(IOutputStream& o, const NActors::TActorId& x) { return x.Out(o); } template <> -struct THash<NActors::TActorId> { - inline ui64 operator()(const NActors::TActorId& x) const { +struct THash<NActors::TActorId> { + inline ui64 operator()(const NActors::TActorId& x) const { return x.Hash(); } }; diff --git a/library/cpp/actors/core/actorsystem.cpp b/library/cpp/actors/core/actorsystem.cpp index 488abd29638..c58698a2061 100644 --- a/library/cpp/actors/core/actorsystem.cpp +++ b/library/cpp/actors/core/actorsystem.cpp @@ -21,16 +21,16 @@ namespace NActors { LWTRACE_USING(ACTORLIB_PROVIDER); struct TActorSystem::TServiceMap : TNonCopyable { - NActors::TServiceMap<TActorId, TActorId, TActorId::THash> LocalMap; + NActors::TServiceMap<TActorId, TActorId, TActorId::THash> LocalMap; TTicketLock Lock; - TActorId RegisterLocalService(const TActorId& serviceId, const TActorId& actorId) { + TActorId RegisterLocalService(const TActorId& serviceId, const TActorId& actorId) { TTicketLock::TGuard guard(&Lock); - const TActorId old = LocalMap.Update(serviceId, actorId); + const TActorId old = LocalMap.Update(serviceId, actorId); return old; } - TActorId LookupLocal(const TActorId& x) { + TActorId LookupLocal(const TActorId& x) { return LocalMap.Find(x); } }; @@ -68,7 +68,7 @@ namespace NActors { ev->Callstack.TraceIfEmpty(); #endif - TActorId recipient = ev->GetRecipientRewrite(); + TActorId recipient = ev->GetRecipientRewrite(); const ui32 recpNodeId = recipient.NodeId(); if (recpNodeId != NodeId && recpNodeId != 0) { @@ -114,13 +114,13 @@ namespace NActors { return false; } - bool TActorSystem::Send(const TActorId& recipient, IEventBase* ev, ui32 flags) const { + bool TActorSystem::Send(const TActorId& recipient, IEventBase* ev, ui32 flags) const { return this->Send(new IEventHandle(recipient, DefSelfID, ev, flags)); } - void TActorSystem::Schedule(TInstant deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie) const { + void TActorSystem::Schedule(TInstant deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie) const { Schedule(deadline - Timestamp(), ev, cookie); - } + } void TActorSystem::Schedule(TMonotonic deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie) const { const auto current = Monotonic(); @@ -131,22 +131,22 @@ namespace NActors { ScheduleQueue->Writer.Push(deadline.MicroSeconds(), ev.Release(), cookie); } - void TActorSystem::Schedule(TDuration delta, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie) const { + void TActorSystem::Schedule(TDuration delta, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie) const { const auto deadline = Monotonic() + delta; - - TTicketLock::TGuard guard(&ScheduleLock); - ScheduleQueue->Writer.Push(deadline.MicroSeconds(), ev.Release(), cookie); + + TTicketLock::TGuard guard(&ScheduleLock); + ScheduleQueue->Writer.Push(deadline.MicroSeconds(), ev.Release(), cookie); } - TActorId TActorSystem::Register(IActor* actor, TMailboxType::EType mailboxType, ui32 executorPool, ui64 revolvingCounter, - const TActorId& parentId) { + TActorId TActorSystem::Register(IActor* actor, TMailboxType::EType mailboxType, ui32 executorPool, ui64 revolvingCounter, + const TActorId& parentId) { Y_VERIFY(executorPool < ExecutorPoolCount, "executorPool# %" PRIu32 ", ExecutorPoolCount# %" PRIu32, (ui32)executorPool, (ui32)ExecutorPoolCount); return CpuManager->GetExecutorPool(executorPool)->Register(actor, mailboxType, revolvingCounter, parentId); } NThreading::TFuture<THolder<IEventBase>> TActorSystem::AskGeneric(TMaybe<ui32> expectedEventType, - TActorId recipient, THolder<IEventBase> event, + TActorId recipient, THolder<IEventBase> event, TDuration timeout) { auto promise = NThreading::NewPromise<THolder<IEventBase>>(); Register(MakeAskActor(expectedEventType, recipient, std::move(event), timeout, promise).Release()); @@ -173,16 +173,16 @@ namespace NActors { return ret; } - TActorId TActorSystem::InterconnectProxy(ui32 destinationNode) const { + TActorId TActorSystem::InterconnectProxy(ui32 destinationNode) const { if (destinationNode < InterconnectCount) return Interconnect[destinationNode]; else if (destinationNode != NodeId) return MakeInterconnectProxyId(destinationNode); else - return TActorId(); + return TActorId(); } - ui32 TActorSystem::BroadcastToProxies(const std::function<IEventHandle*(const TActorId&)>& eventFabric) { + ui32 TActorSystem::BroadcastToProxies(const std::function<IEventHandle*(const TActorId&)>& eventFabric) { // TODO: get rid of this method for (ui32 i = 0; i < InterconnectCount; ++i) { Send(eventFabric(Interconnect[i])); @@ -194,9 +194,9 @@ namespace NActors { return ServiceMap->LookupLocal(x); } - TActorId TActorSystem::RegisterLocalService(const TActorId& serviceId, const TActorId& actorId) { - // TODO: notify old actor about demotion - return ServiceMap->RegisterLocalService(serviceId, actorId); + TActorId TActorSystem::RegisterLocalService(const TActorId& serviceId, const TActorId& actorId) { + // TODO: notify old actor about demotion + return ServiceMap->RegisterLocalService(serviceId, actorId); } void TActorSystem::GetPoolStats(ui32 poolId, TExecutorPoolStats& poolStats, TVector<TExecutorThreadStats>& statsCopy) const { @@ -217,7 +217,7 @@ namespace NActors { // setup interconnect proxies { const TInterconnectSetup& setup = SystemSetup->Interconnect; - Interconnect.Reset(new TActorId[InterconnectCount + 1]); + Interconnect.Reset(new TActorId[InterconnectCount + 1]); for (ui32 i = 0, e = InterconnectCount; i != e; ++i) { const TActorSetupCmd& x = setup.ProxyActors[i]; if (x.Actor) { @@ -231,8 +231,8 @@ namespace NActors { // setup local services { for (ui32 i = 0, e = (ui32)SystemSetup->LocalServices.size(); i != e; ++i) { - const std::pair<TActorId, TActorSetupCmd>& x = SystemSetup->LocalServices[i]; - const TActorId xid = Register(x.second.Actor, x.second.MailboxType, x.second.PoolId, i); + const std::pair<TActorId, TActorSetupCmd>& x = SystemSetup->LocalServices[i]; + const TActorId xid = Register(x.second.Actor, x.second.MailboxType, x.second.PoolId, i); Y_VERIFY(!!xid); if (!!x.first) RegisterLocalService(x.first, xid); diff --git a/library/cpp/actors/core/actorsystem.h b/library/cpp/actors/core/actorsystem.h index c2c88ed2ec2..40499d7586f 100644 --- a/library/cpp/actors/core/actorsystem.h +++ b/library/cpp/actors/core/actorsystem.h @@ -66,17 +66,17 @@ namespace NActors { virtual ui32 GetReadyActivation(TWorkerContext& wctx, ui64 revolvingCounter) = 0; virtual void ReclaimMailbox(TMailboxType::EType mailboxType, ui32 hint, TWorkerId workerId, ui64 revolvingCounter) = 0; - /** - * Schedule one-shot event that will be send at given time point in the future. - * + /** + * Schedule one-shot event that will be send at given time point in the future. + * * @param deadline the wallclock time point in future when event must be send - * @param ev the event to send - * @param cookie cookie that will be piggybacked with event + * @param ev the event to send + * @param cookie cookie that will be piggybacked with event * @param workerId index of thread which will perform event dispatching - */ + */ virtual void Schedule(TInstant deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie, TWorkerId workerId) = 0; - - /** + + /** * Schedule one-shot event that will be send at given time point in the future. * * @param deadline the monotonic time point in future when event must be send @@ -87,21 +87,21 @@ namespace NActors { virtual void Schedule(TMonotonic deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie, TWorkerId workerId) = 0; /** - * Schedule one-shot event that will be send after given delay. - * - * @param delta the time from now to delay event sending - * @param ev the event to send - * @param cookie cookie that will be piggybacked with event + * Schedule one-shot event that will be send after given delay. + * + * @param delta the time from now to delay event sending + * @param ev the event to send + * @param cookie cookie that will be piggybacked with event * @param workerId index of thread which will perform event dispatching - */ + */ virtual void Schedule(TDuration delta, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie, TWorkerId workerId) = 0; - + // for actorsystem virtual bool Send(TAutoPtr<IEventHandle>& ev) = 0; virtual void ScheduleActivation(ui32 activation) = 0; virtual void ScheduleActivationEx(ui32 activation, ui64 revolvingCounter) = 0; - virtual TActorId Register(IActor* actor, TMailboxType::EType mailboxType, ui64 revolvingCounter, const TActorId& parentId) = 0; - virtual TActorId Register(IActor* actor, TMailboxHeader* mailbox, ui32 hint, const TActorId& parentId) = 0; + virtual TActorId Register(IActor* actor, TMailboxType::EType mailboxType, ui64 revolvingCounter, const TActorId& parentId) = 0; + virtual TActorId Register(IActor* actor, TMailboxHeader* mailbox, ui32 hint, const TActorId& parentId) = 0; // lifecycle stuff virtual void Prepare(TActorSystem* actorSystem, NSchedulerQueue::TReader** scheduleReaders, ui32* scheduleSz) = 0; @@ -223,7 +223,7 @@ namespace NActors { THolder<TServiceMap> ServiceMap; const ui32 InterconnectCount; - TArrayHolder<TActorId> Interconnect; + TArrayHolder<TActorId> Interconnect; volatile ui64 CurrentTimestamp; volatile ui64 CurrentMonotonic; @@ -235,7 +235,7 @@ namespace NActors { friend class TExecutorThread; THolder<TActorSystemSetup> SystemSetup; - TActorId DefSelfID; + TActorId DefSelfID; void* AppData0; TIntrusivePtr<NLog::TSettings> LoggerSettings0; TProxyWrapperFactory ProxyWrapperFactory; @@ -255,22 +255,22 @@ namespace NActors { void Stop(); void Cleanup(); - TActorId Register(IActor* actor, TMailboxType::EType mailboxType = TMailboxType::HTSwap, ui32 executorPool = 0, - ui64 revolvingCounter = 0, const TActorId& parentId = TActorId()); + TActorId Register(IActor* actor, TMailboxType::EType mailboxType = TMailboxType::HTSwap, ui32 executorPool = 0, + ui64 revolvingCounter = 0, const TActorId& parentId = TActorId()); bool Send(TAutoPtr<IEventHandle> ev) const; - bool Send(const TActorId& recipient, IEventBase* ev, ui32 flags = 0) const; + bool Send(const TActorId& recipient, IEventBase* ev, ui32 flags = 0) const; /** - * Schedule one-shot event that will be send at given time point in the future. - * + * Schedule one-shot event that will be send at given time point in the future. + * * @param deadline the wallclock time point in future when event must be send - * @param ev the event to send - * @param cookie cookie that will be piggybacked with event - */ - void Schedule(TInstant deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie = nullptr) const; - - /** + * @param ev the event to send + * @param cookie cookie that will be piggybacked with event + */ + void Schedule(TInstant deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie = nullptr) const; + + /** * Schedule one-shot event that will be send at given time point in the future. * * @param deadline the monotonic time point in future when event must be send @@ -280,15 +280,15 @@ namespace NActors { void Schedule(TMonotonic deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie = nullptr) const; /** - * Schedule one-shot event that will be send after given delay. - * - * @param delta the time from now to delay event sending - * @param ev the event to send - * @param cookie cookie that will be piggybacked with event - */ - void Schedule(TDuration delta, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie = nullptr) const; - - /** + * Schedule one-shot event that will be send after given delay. + * + * @param delta the time from now to delay event sending + * @param ev the event to send + * @param cookie cookie that will be piggybacked with event + */ + void Schedule(TDuration delta, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie = nullptr) const; + + /** * A way to interact with actors from non-actor context. * * This method will send the `event` to the `recipient` and then will wait for a response. When response arrives, @@ -303,7 +303,7 @@ namespace NActors { */ template <typename T> [[nodiscard]] - NThreading::TFuture<THolder<T>> Ask(TActorId recipient, THolder<IEventBase> event, TDuration timeout = TDuration::Max()) { + NThreading::TFuture<THolder<T>> Ask(TActorId recipient, THolder<IEventBase> event, TDuration timeout = TDuration::Max()) { if constexpr (std::is_same_v<T, IEventBase>) { return AskGeneric(Nothing(), recipient, std::move(event), timeout); } else { @@ -317,20 +317,20 @@ namespace NActors { [[nodiscard]] NThreading::TFuture<THolder<IEventBase>> AskGeneric( TMaybe<ui32> expectedEventType, - TActorId recipient, + TActorId recipient, THolder<IEventBase> event, TDuration timeout); ui64 AllocateIDSpace(ui64 count); - TActorId InterconnectProxy(ui32 destinationNode) const; - ui32 BroadcastToProxies(const std::function<IEventHandle*(const TActorId&)>&); + TActorId InterconnectProxy(ui32 destinationNode) const; + ui32 BroadcastToProxies(const std::function<IEventHandle*(const TActorId&)>&); void UpdateLinkStatus(ui8 status, ui32 destinationNode); ui8 LinkStatus(ui32 destinationNode); TActorId LookupLocalService(const TActorId& x) const; - TActorId RegisterLocalService(const TActorId& serviceId, const TActorId& actorId); + TActorId RegisterLocalService(const TActorId& serviceId, const TActorId& actorId); ui32 GetMaxActivityType() const { return SystemSetup ? SystemSetup->MaxActivityType : 1; diff --git a/library/cpp/actors/core/actorsystem_ut.cpp b/library/cpp/actors/core/actorsystem_ut.cpp index 2c93b12dcd7..231d6f0ca1d 100644 --- a/library/cpp/actors/core/actorsystem_ut.cpp +++ b/library/cpp/actors/core/actorsystem_ut.cpp @@ -1,45 +1,45 @@ -#include "actorsystem.h" - -#include <library/cpp/actors/testlib/test_runtime.h> -#include <library/cpp/testing/unittest/registar.h> - -using namespace NActors; - -Y_UNIT_TEST_SUITE(TActorSystemTest) { - - class TTestActor: public TActor<TTestActor> { - public: - TTestActor() - : TActor{&TThis::Main} - { - } - - STATEFN(Main) { - Y_UNUSED(ev); - } - }; - - THolder<TTestActorRuntimeBase> CreateRuntime() { - auto runtime = MakeHolder<TTestActorRuntimeBase>(); - runtime->SetScheduledEventFilter([](auto&&, auto&&, auto&&, auto&&) { return false; }); - runtime->Initialize(); - return runtime; - } - - Y_UNIT_TEST(LocalService) { - THolder<TTestActorRuntimeBase> runtime = CreateRuntime(); - auto actorA = runtime->Register(new TTestActor); - auto actorB = runtime->Register(new TTestActor); - - TActorId myServiceId{0, TStringBuf{"my-service"}}; - - auto prevActorId = runtime->RegisterService(myServiceId, actorA); - UNIT_ASSERT(!prevActorId); - UNIT_ASSERT_EQUAL(runtime->GetLocalServiceId(myServiceId), actorA); - - prevActorId = runtime->RegisterService(myServiceId, actorB); - UNIT_ASSERT(prevActorId); - UNIT_ASSERT_EQUAL(prevActorId, actorA); - UNIT_ASSERT_EQUAL(runtime->GetLocalServiceId(myServiceId), actorB); - } -} +#include "actorsystem.h" + +#include <library/cpp/actors/testlib/test_runtime.h> +#include <library/cpp/testing/unittest/registar.h> + +using namespace NActors; + +Y_UNIT_TEST_SUITE(TActorSystemTest) { + + class TTestActor: public TActor<TTestActor> { + public: + TTestActor() + : TActor{&TThis::Main} + { + } + + STATEFN(Main) { + Y_UNUSED(ev); + } + }; + + THolder<TTestActorRuntimeBase> CreateRuntime() { + auto runtime = MakeHolder<TTestActorRuntimeBase>(); + runtime->SetScheduledEventFilter([](auto&&, auto&&, auto&&, auto&&) { return false; }); + runtime->Initialize(); + return runtime; + } + + Y_UNIT_TEST(LocalService) { + THolder<TTestActorRuntimeBase> runtime = CreateRuntime(); + auto actorA = runtime->Register(new TTestActor); + auto actorB = runtime->Register(new TTestActor); + + TActorId myServiceId{0, TStringBuf{"my-service"}}; + + auto prevActorId = runtime->RegisterService(myServiceId, actorA); + UNIT_ASSERT(!prevActorId); + UNIT_ASSERT_EQUAL(runtime->GetLocalServiceId(myServiceId), actorA); + + prevActorId = runtime->RegisterService(myServiceId, actorB); + UNIT_ASSERT(prevActorId); + UNIT_ASSERT_EQUAL(prevActorId, actorA); + UNIT_ASSERT_EQUAL(runtime->GetLocalServiceId(myServiceId), actorB); + } +} diff --git a/library/cpp/actors/core/ask.cpp b/library/cpp/actors/core/ask.cpp index 4e4d6a1a2b3..0054c9a906c 100644 --- a/library/cpp/actors/core/ask.cpp +++ b/library/cpp/actors/core/ask.cpp @@ -19,7 +19,7 @@ namespace NActors { public: TAskActor( TMaybe<ui32> expectedEventType, - TActorId recipient, + TActorId recipient, THolder<IEventBase> event, TDuration timeout, const NThreading::TPromise<THolder<IEventBase>>& promise) @@ -55,7 +55,7 @@ namespace NActors { public: TMaybe<ui32> ExpectedEventType_; - TActorId Recipient_; + TActorId Recipient_; THolder<IEventBase> Event_; TDuration Timeout_; NThreading::TPromise<THolder<IEventBase>> Promise_; @@ -64,7 +64,7 @@ namespace NActors { THolder<IActor> MakeAskActor( TMaybe<ui32> expectedEventType, - TActorId recipient, + TActorId recipient, THolder<IEventBase> event, TDuration timeout, const NThreading::TPromise<THolder<IEventBase>>& promise) diff --git a/library/cpp/actors/core/ask.h b/library/cpp/actors/core/ask.h index b935fac5643..036f1833a4f 100644 --- a/library/cpp/actors/core/ask.h +++ b/library/cpp/actors/core/ask.h @@ -11,7 +11,7 @@ namespace NActors { */ THolder<IActor> MakeAskActor( TMaybe<ui32> expectedEventType, - TActorId recipient, + TActorId recipient, THolder<IEventBase> event, TDuration timeout, const NThreading::TPromise<THolder<IEventBase>>& promise); diff --git a/library/cpp/actors/core/event.h b/library/cpp/actors/core/event.h index 9519978bc1b..6ff02aaf943 100644 --- a/library/cpp/actors/core/event.h +++ b/library/cpp/actors/core/event.h @@ -48,9 +48,9 @@ namespace NActors { // fat handle class IEventHandle : TNonCopyable { struct TOnNondelivery { - TActorId Recipient; + TActorId Recipient; - TOnNondelivery(const TActorId& recipient) + TOnNondelivery(const TActorId& recipient) : Recipient(recipient) { } @@ -99,8 +99,8 @@ namespace NActors { const ui32 Type; const ui32 Flags; - const TActorId Recipient; - const TActorId Sender; + const TActorId Recipient; + const TActorId Sender; const ui64 Cookie; const TScopeId OriginScopeId = TScopeId::LocallyGenerated; // filled in when the message is received from Interconnect @@ -108,7 +108,7 @@ namespace NActors { NWilson::TTraceId TraceId; // filled if feeded by interconnect session - const TActorId InterconnectSession; + const TActorId InterconnectSession; #ifdef ACTORSLIB_COLLECT_EXEC_STATS ::NHPTimer::STime SendTime; @@ -138,13 +138,13 @@ namespace NActors { THolder<IEventBase> Event; TIntrusivePtr<TEventSerializedData> Buffer; - TActorId RewriteRecipient; + TActorId RewriteRecipient; ui32 RewriteType; THolder<TOnNondelivery> OnNondeliveryHolder; // only for local events public: - void Rewrite(ui32 typeRewrite, TActorId recipientRewrite) { + void Rewrite(ui32 typeRewrite, TActorId recipientRewrite) { RewriteRecipient = recipientRewrite; RewriteType = typeRewrite; } @@ -154,7 +154,7 @@ namespace NActors { RewriteType = Type; } - const TActorId& GetRecipientRewrite() const { + const TActorId& GetRecipientRewrite() const { return RewriteRecipient; } @@ -162,12 +162,12 @@ namespace NActors { return RewriteType; } - TActorId GetForwardOnNondeliveryRecipient() const { - return OnNondeliveryHolder.Get() ? OnNondeliveryHolder->Recipient : TActorId(); + TActorId GetForwardOnNondeliveryRecipient() const { + return OnNondeliveryHolder.Get() ? OnNondeliveryHolder->Recipient : TActorId(); } - IEventHandle(const TActorId& recipient, const TActorId& sender, IEventBase* ev, ui32 flags = 0, ui64 cookie = 0, - const TActorId* forwardOnNondelivery = nullptr, NWilson::TTraceId traceId = {}) + IEventHandle(const TActorId& recipient, const TActorId& sender, IEventBase* ev, ui32 flags = 0, ui64 cookie = 0, + const TActorId* forwardOnNondelivery = nullptr, NWilson::TTraceId traceId = {}) : Type(ev->Type()) , Flags(flags) , Recipient(recipient) @@ -187,11 +187,11 @@ namespace NActors { IEventHandle(ui32 type, ui32 flags, - const TActorId& recipient, - const TActorId& sender, + const TActorId& recipient, + const TActorId& sender, TIntrusivePtr<TEventSerializedData> buffer, ui64 cookie, - const TActorId* forwardOnNondelivery = nullptr, + const TActorId* forwardOnNondelivery = nullptr, NWilson::TTraceId traceId = {}) : Type(type) , Flags(flags) @@ -211,11 +211,11 @@ namespace NActors { } // Special ctor for events from interconnect. - IEventHandle(const TActorId& session, + IEventHandle(const TActorId& session, ui32 type, ui32 flags, - const TActorId& recipient, - const TActorId& sender, + const TActorId& recipient, + const TActorId& sender, TIntrusivePtr<TEventSerializedData> buffer, ui64 cookie, TScopeId originScopeId, @@ -276,7 +276,7 @@ namespace NActors { return x; } - TAutoPtr<IEventHandle> Forward(const TActorId& dest) { + TAutoPtr<IEventHandle> Forward(const TActorId& dest) { if (Event) return new IEventHandle(dest, Sender, Event.Release(), Flags, Cookie, nullptr, std::move(TraceId)); else diff --git a/library/cpp/actors/core/event_pb.h b/library/cpp/actors/core/event_pb.h index baaf333ca9c..d7546b901a0 100644 --- a/library/cpp/actors/core/event_pb.h +++ b/library/cpp/actors/core/event_pb.h @@ -488,11 +488,11 @@ namespace NActors { } }; - inline TActorId ActorIdFromProto(const NActorsProto::TActorId& actorId) { - return TActorId(actorId.GetRawX1(), actorId.GetRawX2()); + inline TActorId ActorIdFromProto(const NActorsProto::TActorId& actorId) { + return TActorId(actorId.GetRawX1(), actorId.GetRawX2()); } - inline void ActorIdToProto(const TActorId& src, NActorsProto::TActorId* dest) { + inline void ActorIdToProto(const TActorId& src, NActorsProto::TActorId* dest) { Y_VERIFY_DEBUG(dest); dest->SetRawX1(src.RawX1()); dest->SetRawX2(src.RawX2()); diff --git a/library/cpp/actors/core/events.h b/library/cpp/actors/core/events.h index b5b9d7c9fa3..702cf50fadf 100644 --- a/library/cpp/actors/core/events.h +++ b/library/cpp/actors/core/events.h @@ -20,7 +20,7 @@ namespace NActors { ES_INTERCONNECT_TCP = 8, ES_PROFILER = 9, ES_YF = 10, - ES_HTTP = 11, + ES_HTTP = 11, ES_USERSPACE = 4096, @@ -99,7 +99,7 @@ namespace NActors { InvokeQuery, End, - // Compatibility section + // Compatibility section PoisonPill = Poison, ActorDied = Gone, }; @@ -191,17 +191,17 @@ namespace NActors { struct TEvCallbackException: public TEventPB<TEvCallbackException, NActorsProto::TCallbackException, TSystem::CallbackException> { - TEvCallbackException(const TActorId& id, const TString& msg) { - ActorIdToProto(id, Record.MutableActorId()); + TEvCallbackException(const TActorId& id, const TString& msg) { + ActorIdToProto(id, Record.MutableActorId()); Record.SetExceptionMessage(msg); } }; struct TEvCallbackCompletion: public TEventPB<TEvCallbackCompletion, - NActorsProto::TActorId, + NActorsProto::TActorId, TSystem::CallbackCompletion> { - TEvCallbackCompletion(const TActorId& id) { - ActorIdToProto(id, &Record); + TEvCallbackCompletion(const TActorId& id) { + ActorIdToProto(id, &Record); } }; diff --git a/library/cpp/actors/core/events_undelivered.cpp b/library/cpp/actors/core/events_undelivered.cpp index 2a5a0b1cc64..23deaffd106 100644 --- a/library/cpp/actors/core/events_undelivered.cpp +++ b/library/cpp/actors/core/events_undelivered.cpp @@ -41,7 +41,7 @@ namespace NActors { TAutoPtr<IEventHandle> IEventHandle::ForwardOnNondelivery(ui32 reason, bool unsure) { if (Flags & FlagForwardOnNondelivery) { const ui32 updatedFlags = Flags & ~(FlagForwardOnNondelivery | FlagSubscribeOnSession); - const TActorId recp = OnNondeliveryHolder ? OnNondeliveryHolder->Recipient : TActorId(); + const TActorId recp = OnNondeliveryHolder ? OnNondeliveryHolder->Recipient : TActorId(); if (Event) return new IEventHandle(recp, Sender, Event.Release(), updatedFlags, Cookie, &Recipient, TraceId.Clone()); diff --git a/library/cpp/actors/core/executelater.h b/library/cpp/actors/core/executelater.h index ec55c43b40f..e7a13c10053 100644 --- a/library/cpp/actors/core/executelater.h +++ b/library/cpp/actors/core/executelater.h @@ -17,8 +17,8 @@ namespace NActors { IActor::EActivityType activityType, ui32 channel = 0, ui64 cookie = 0, - const TActorId& reportCompletionTo = TActorId(), - const TActorId& reportExceptionTo = TActorId()) noexcept + const TActorId& reportCompletionTo = TActorId(), + const TActorId& reportExceptionTo = TActorId()) noexcept : Callback(std::move(callback)) , Channel(channel) , Cookie(cookie) @@ -65,8 +65,8 @@ namespace NActors { TCallback Callback; const ui32 Channel; const ui64 Cookie; - const TActorId ReportCompletionTo; - const TActorId ReportExceptionTo; + const TActorId ReportCompletionTo; + const TActorId ReportExceptionTo; }; template <typename T> @@ -75,8 +75,8 @@ namespace NActors { IActor::EActivityType activityType, ui32 channel = 0, ui64 cookie = 0, - const TActorId& reportCompletionTo = TActorId(), - const TActorId& reportExceptionTo = TActorId()) noexcept { + const TActorId& reportCompletionTo = TActorId(), + const TActorId& reportExceptionTo = TActorId()) noexcept { return new TExecuteLater<T>(std::forward<T>(func), activityType, channel, diff --git a/library/cpp/actors/core/executor_pool_base.cpp b/library/cpp/actors/core/executor_pool_base.cpp index 860496f1087..c3b99991680 100644 --- a/library/cpp/actors/core/executor_pool_base.cpp +++ b/library/cpp/actors/core/executor_pool_base.cpp @@ -7,7 +7,7 @@ namespace NActors { LWTRACE_USING(ACTORLIB_PROVIDER); - void DoActorInit(TActorSystem* sys, IActor* actor, const TActorId& self, const TActorId& owner) { + void DoActorInit(TActorSystem* sys, IActor* actor, const TActorId& self, const TActorId& owner) { actor->SelfActorId = self; actor->Registered(sys, owner); } @@ -97,7 +97,7 @@ namespace NActors { mailbox->AttachActor(localActorId, actor); // do init - const TActorId actorId(ActorSystem->NodeId, PoolId, localActorId, hint); + const TActorId actorId(ActorSystem->NodeId, PoolId, localActorId, hint); DoActorInit(ActorSystem, actor, actorId, parentId); // Once we unlock the mailbox the actor starts running and we cannot use the pointer any more @@ -144,7 +144,7 @@ namespace NActors { const ui64 localActorId = AllocateID(); mailbox->AttachActor(localActorId, actor); - const TActorId actorId(ActorSystem->NodeId, PoolId, localActorId, hint); + const TActorId actorId(ActorSystem->NodeId, PoolId, localActorId, hint); DoActorInit(ActorSystem, actor, actorId, parentId); NHPTimer::STime elapsed = GetCycleCountFast() - hpstart; if (elapsed > 1000000) { diff --git a/library/cpp/actors/core/executor_pool_base.h b/library/cpp/actors/core/executor_pool_base.h index d52a242fc6e..c84ce1af779 100644 --- a/library/cpp/actors/core/executor_pool_base.h +++ b/library/cpp/actors/core/executor_pool_base.h @@ -24,8 +24,8 @@ namespace NActors { ~TExecutorPoolBaseMailboxed(); void ReclaimMailbox(TMailboxType::EType mailboxType, ui32 hint, TWorkerId workerId, ui64 revolvingWriteCounter) override; bool Send(TAutoPtr<IEventHandle>& ev) override; - TActorId Register(IActor* actor, TMailboxType::EType mailboxType, ui64 revolvingWriteCounter, const TActorId& parentId) override; - TActorId Register(IActor* actor, TMailboxHeader* mailbox, ui32 hint, const TActorId& parentId) override; + TActorId Register(IActor* actor, TMailboxType::EType mailboxType, ui64 revolvingWriteCounter, const TActorId& parentId) override; + TActorId Register(IActor* actor, TMailboxHeader* mailbox, ui32 hint, const TActorId& parentId) override; bool Cleanup() override; }; diff --git a/library/cpp/actors/core/executor_pool_basic.cpp b/library/cpp/actors/core/executor_pool_basic.cpp index 936d2e94a71..4dce16939ae 100644 --- a/library/cpp/actors/core/executor_pool_basic.cpp +++ b/library/cpp/actors/core/executor_pool_basic.cpp @@ -315,11 +315,11 @@ namespace NActors { void TBasicExecutorPool::Schedule(TDuration delta, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie, TWorkerId workerId) { Y_VERIFY_DEBUG(workerId < PoolThreads); - + const auto deadline = ActorSystem->Monotonic() + delta; ScheduleWriters[workerId].Push(deadline.MicroSeconds(), ev.Release(), cookie); - } - + } + void TBasicExecutorPool::SetRealTimeMode() const { // TODO: musl-libc version of `sched_param` struct is for some reason different from pthread // version in Ubuntu 12.04 diff --git a/library/cpp/actors/core/executor_pool_basic.h b/library/cpp/actors/core/executor_pool_basic.h index dd83c85c747..023190f7fe3 100644 --- a/library/cpp/actors/core/executor_pool_basic.h +++ b/library/cpp/actors/core/executor_pool_basic.h @@ -6,7 +6,7 @@ #include "executor_pool_base.h" #include <library/cpp/actors/util/unordered_cache.h> #include <library/cpp/actors/util/threadparkpad.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <util/system/mutex.h> @@ -87,7 +87,7 @@ namespace NActors { void Schedule(TInstant deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie, TWorkerId workerId) override; void Schedule(TMonotonic deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie, TWorkerId workerId) override; void Schedule(TDuration delta, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie, TWorkerId workerId) override; - + void ScheduleActivationEx(ui32 activation, ui64 revolvingWriteCounter) override; void Prepare(TActorSystem* actorSystem, NSchedulerQueue::TReader** scheduleReaders, ui32* scheduleSz) override; diff --git a/library/cpp/actors/core/executor_pool_basic_ut.cpp b/library/cpp/actors/core/executor_pool_basic_ut.cpp index 8c170c2d84f..76dff693af5 100644 --- a/library/cpp/actors/core/executor_pool_basic_ut.cpp +++ b/library/cpp/actors/core/executor_pool_basic_ut.cpp @@ -25,7 +25,7 @@ private: private: TAtomic Counter; - TActorId Receiver; + TActorId Receiver; std::function<void(void)> Action; @@ -36,7 +36,7 @@ public: , Action(action) {} - void Start(TActorId receiver, size_t count) + void Start(TActorId receiver, size_t count) { AtomicSet(Counter, count); Receiver = receiver; @@ -102,7 +102,7 @@ Y_UNIT_TEST_SUITE(BasicExecutorPool) { executorPool->SetThreadCount(halfSize); TTestSenderActor* actors[size]; - TActorId actorIds[size]; + TActorId actorIds[size]; for (size_t i = 0; i < size; ++i) { actors[i] = new TTestSenderActor(); actorIds[i] = actorSystem.Register(actors[i]); @@ -176,7 +176,7 @@ Y_UNIT_TEST_SUITE(BasicExecutorPool) { executorPool->SetThreadCount(halfSize); TTestSenderActor* actors[size]; - TActorId actorIds[size]; + TActorId actorIds[size]; for (size_t i = 0; i < size; ++i) { actors[i] = new TTestSenderActor(); actorIds[i] = actorSystem.Register(actors[i]); @@ -201,7 +201,7 @@ Y_UNIT_TEST_SUITE(BasicExecutorPool) { counter = 0; } }); - TActorId changerActorId = actorSystem.Register(changerActor); + TActorId changerActorId = actorSystem.Register(changerActor); changerActor->Start(changerActorId, msgCount); actorSystem.Send(changerActorId, new TEvMsg()); @@ -260,7 +260,7 @@ Y_UNIT_TEST_SUITE(BasicExecutorPool) { auto begin = TInstant::Now(); TTestSenderActor* actors[size]; - TActorId actorIds[size]; + TActorId actorIds[size]; for (size_t i = 0; i < size; ++i) { actors[i] = new TTestSenderActor(); @@ -304,7 +304,7 @@ Y_UNIT_TEST_SUITE(BasicExecutorPool) { auto begin = TInstant::Now(); TTestSenderActor* actors[actorsCount]; - TActorId actorIds[actorsCount]; + TActorId actorIds[actorsCount]; for (size_t i = 0; i < actorsCount; ++i) { actors[i] = new TTestSenderActor(); @@ -348,7 +348,7 @@ Y_UNIT_TEST_SUITE(BasicExecutorPool) { auto begin = TInstant::Now(); TTestSenderActor* actors[actorsCount]; - TActorId actorIds[actorsCount]; + TActorId actorIds[actorsCount]; for (size_t i = 0; i < actorsCount; ++i) { actors[i] = new TTestSenderActor(); diff --git a/library/cpp/actors/core/executor_pool_io.cpp b/library/cpp/actors/core/executor_pool_io.cpp index 025b5a22c27..fb557ae6b00 100644 --- a/library/cpp/actors/core/executor_pool_io.cpp +++ b/library/cpp/actors/core/executor_pool_io.cpp @@ -81,11 +81,11 @@ namespace NActors { void TIOExecutorPool::Schedule(TDuration delta, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie, TWorkerId workerId) { Y_UNUSED(workerId); const auto deadline = ActorSystem->Monotonic() + delta; - - TTicketLock::TGuard guard(&ScheduleLock); - ScheduleQueue->Writer.Push(deadline.MicroSeconds(), ev.Release(), cookie); - } - + + TTicketLock::TGuard guard(&ScheduleLock); + ScheduleQueue->Writer.Push(deadline.MicroSeconds(), ev.Release(), cookie); + } + void TIOExecutorPool::ScheduleActivationEx(ui32 activation, ui64 revolvingWriteCounter) { Activations.Push(activation, revolvingWriteCounter); const TAtomic x = AtomicIncrement(Semaphore); diff --git a/library/cpp/actors/core/executor_pool_io.h b/library/cpp/actors/core/executor_pool_io.h index a1359ba4abf..e576d642a1d 100644 --- a/library/cpp/actors/core/executor_pool_io.h +++ b/library/cpp/actors/core/executor_pool_io.h @@ -35,7 +35,7 @@ namespace NActors { void Schedule(TInstant deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie, TWorkerId workerId) override; void Schedule(TMonotonic deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie, TWorkerId workerId) override; void Schedule(TDuration delta, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie, TWorkerId workerId) override; - + void ScheduleActivationEx(ui32 activation, ui64 revolvingWriteCounter) override; void Prepare(TActorSystem* actorSystem, NSchedulerQueue::TReader** scheduleReaders, ui32* scheduleSz) override; diff --git a/library/cpp/actors/core/executor_pool_united.cpp b/library/cpp/actors/core/executor_pool_united.cpp index da4934eccd3..dac6245635d 100644 --- a/library/cpp/actors/core/executor_pool_united.cpp +++ b/library/cpp/actors/core/executor_pool_united.cpp @@ -1255,13 +1255,13 @@ namespace NActors { inline bool TUnitedWorkers::NextExecution(TPoolId pool, ui32& activation, ui64 revolvingCounter) { return Pools[pool].NextExecution(activation, revolvingCounter); } - + inline void TUnitedWorkers::StopExecution(TPoolId pool) { if (Pools[pool].StopExecution()) { // pending token TryWake(pool); } - } - + } + inline void TUnitedWorkers::Balance() { ui64 ts = GetCycleCountFast(); if (Balancer->TryLock(ts)) { diff --git a/library/cpp/actors/core/executor_pool_united.h b/library/cpp/actors/core/executor_pool_united.h index b1af8503127..a090ba24665 100644 --- a/library/cpp/actors/core/executor_pool_united.h +++ b/library/cpp/actors/core/executor_pool_united.h @@ -7,7 +7,7 @@ #include <library/cpp/actors/util/unordered_cache.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <library/cpp/actors/util/unordered_cache.h> #include <library/cpp/containers/stack_vector/stack_vec.h> @@ -79,7 +79,7 @@ namespace NActors { // Stop currently active execution and start new one if token is available // NOTE: Reuses token if it's not destroyed bool NextExecution(TPoolId pool, ui32& activation, ui64 revolvingCounter); - + // Stop active execution void StopExecution(TPoolId pool); diff --git a/library/cpp/actors/core/executor_pool_united_ut.cpp b/library/cpp/actors/core/executor_pool_united_ut.cpp index be92b9352a0..d4df17f1b8c 100644 --- a/library/cpp/actors/core/executor_pool_united_ut.cpp +++ b/library/cpp/actors/core/executor_pool_united_ut.cpp @@ -37,7 +37,7 @@ private: private: TAtomic Counter; - TActorId Receiver; + TActorId Receiver; std::function<void(void)> Action; diff --git a/library/cpp/actors/core/executor_thread.cpp b/library/cpp/actors/core/executor_thread.cpp index 03ef88ea511..446b651efd2 100644 --- a/library/cpp/actors/core/executor_thread.cpp +++ b/library/cpp/actors/core/executor_thread.cpp @@ -50,14 +50,14 @@ namespace NActors { &Ctx.WorkerStats); } - TActorId TExecutorThread::RegisterActor(IActor* actor, TMailboxType::EType mailboxType, ui32 poolId, const TActorId& parentId) { + TActorId TExecutorThread::RegisterActor(IActor* actor, TMailboxType::EType mailboxType, ui32 poolId, const TActorId& parentId) { if (poolId == Max<ui32>()) return Ctx.Executor->Register(actor, mailboxType, ++RevolvingWriteCounter, parentId ? parentId : CurrentRecipient); else return ActorSystem->Register(actor, mailboxType, poolId, ++RevolvingWriteCounter, parentId ? parentId : CurrentRecipient); } - TActorId TExecutorThread::RegisterActor(IActor* actor, TMailboxHeader* mailbox, ui32 hint, const TActorId& parentId) { + TActorId TExecutorThread::RegisterActor(IActor* actor, TMailboxHeader* mailbox, ui32 hint, const TActorId& parentId) { return Ctx.Executor->Register(actor, mailbox, hint, parentId ? parentId : CurrentRecipient); } @@ -71,7 +71,7 @@ namespace NActors { DyingActors.clear(); // here is actual destruction of actors } - void TExecutorThread::Schedule(TInstant deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie) { + void TExecutorThread::Schedule(TInstant deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie) { ++CurrentActorScheduledEventsCounter; Ctx.Executor->Schedule(deadline, ev, cookie, Ctx.WorkerId); } @@ -81,11 +81,11 @@ namespace NActors { Ctx.Executor->Schedule(deadline, ev, cookie, Ctx.WorkerId); } - void TExecutorThread::Schedule(TDuration delta, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie) { - ++CurrentActorScheduledEventsCounter; + void TExecutorThread::Schedule(TDuration delta, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie) { + ++CurrentActorScheduledEventsCounter; Ctx.Executor->Schedule(delta, ev, cookie, Ctx.WorkerId); - } - + } + template <class T> inline TString SafeTypeName(T* t) { if (t == nullptr) { @@ -102,7 +102,7 @@ namespace NActors { return actor ? SafeTypeName(actor) : ("activityType_" + ToString(activityType) + " (destroyed)"); } - inline void LwTraceSlowDelivery(IEventHandle* ev, const IActor* actor, ui32 poolId, const TActorId& currentRecipient, + inline void LwTraceSlowDelivery(IEventHandle* ev, const IActor* actor, ui32 poolId, const TActorId& currentRecipient, double delivMs, double sinceActivationMs, ui32 eventsExecutedBefore) { const auto baseEv = (ev && ev->HasEvent()) ? ev->GetBase() : nullptr; LWPROBE(EventSlowDelivery, @@ -116,7 +116,7 @@ namespace NActors { } inline void LwTraceSlowEvent(IEventHandle* ev, ui32 evTypeForTracing, const IActor* actor, ui32 poolId, ui32 activityType, - const TActorId& currentRecipient, double eventMs) { + const TActorId& currentRecipient, double eventMs) { // Event could have been destroyed by actor->Receive(); const auto baseEv = (ev && ev->HasEvent()) ? ev->GetBase() : nullptr; LWPROBE(SlowEvent, @@ -198,7 +198,7 @@ namespace NActors { if (actor) actor->AddElapsedTicks(elapsed); - CurrentRecipient = TActorId(); + CurrentRecipient = TActorId(); } else { TAutoPtr<IEventHandle> nonDelivered = ev->ForwardOnNondelivery(TEvents::TEvUndelivered::ReasonActorUnknown); if (nonDelivered.Get()) { diff --git a/library/cpp/actors/core/executor_thread.h b/library/cpp/actors/core/executor_thread.h index f3f1d527d63..9d3c573f0d6 100644 --- a/library/cpp/actors/core/executor_thread.h +++ b/library/cpp/actors/core/executor_thread.h @@ -39,17 +39,17 @@ namespace NActors { : TExecutorThread(workerId, 0, actorSystem, executorPool, mailboxTable, threadName, timePerMailbox, eventsPerMailbox) {} - TActorId RegisterActor(IActor* actor, TMailboxType::EType mailboxType = TMailboxType::HTSwap, ui32 poolId = Max<ui32>(), - const TActorId& parentId = TActorId()); - TActorId RegisterActor(IActor* actor, TMailboxHeader* mailbox, ui32 hint, const TActorId& parentId = TActorId()); + TActorId RegisterActor(IActor* actor, TMailboxType::EType mailboxType = TMailboxType::HTSwap, ui32 poolId = Max<ui32>(), + const TActorId& parentId = TActorId()); + TActorId RegisterActor(IActor* actor, TMailboxHeader* mailbox, ui32 hint, const TActorId& parentId = TActorId()); void UnregisterActor(TMailboxHeader* mailbox, ui64 localActorId); void DropUnregistered(); const std::vector<THolder<IActor>>& GetUnregistered() const { return DyingActors; } - void Schedule(TInstant deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie = nullptr); + void Schedule(TInstant deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie = nullptr); void Schedule(TMonotonic deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie = nullptr); - void Schedule(TDuration delta, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie = nullptr); - + void Schedule(TDuration delta, TAutoPtr<IEventHandle> ev, ISchedulerCookie* cookie = nullptr); + bool Send(TAutoPtr<IEventHandle> ev) { #ifdef USE_ACTOR_CALLSTACK ev->Callstack = TCallstack::GetTlsCallstack(); @@ -81,7 +81,7 @@ namespace NActors { // Event-specific (currently executing) TVector<THolder<IActor>> DyingActors; - TActorId CurrentRecipient; + TActorId CurrentRecipient; ui64 CurrentActorScheduledEventsCounter = 0; // Thread-specific diff --git a/library/cpp/actors/core/invoke.h b/library/cpp/actors/core/invoke.h index 26de350a955..931a9767ddd 100644 --- a/library/cpp/actors/core/invoke.h +++ b/library/cpp/actors/core/invoke.h @@ -92,7 +92,7 @@ namespace NActors { , Complete(std::move(complete)) {} - void Bootstrap(const TActorId& parentId, const TActorContext& ctx) { + void Bootstrap(const TActorId& parentId, const TActorContext& ctx) { auto process = [complete = std::move(Complete)](TEvents::TEvInvokeResult& res, const TActorContext& ctx) { complete([&] { return res.GetResult<TCallback>(); }, ctx); }; diff --git a/library/cpp/actors/core/log.cpp b/library/cpp/actors/core/log.cpp index 88e24f4c01f..5f63b5af580 100644 --- a/library/cpp/actors/core/log.cpp +++ b/library/cpp/actors/core/log.cpp @@ -1,7 +1,7 @@ #include "log.h" #include "log_settings.h" -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> static_assert(int(NActors::NLog::PRI_EMERG) == int(::TLOG_EMERG), "expect int(NActors::NLog::PRI_EMERG) == int(::TLOG_EMERG)"); static_assert(int(NActors::NLog::PRI_ALERT) == int(::TLOG_ALERT), "expect int(NActors::NLog::PRI_ALERT) == int(::TLOG_ALERT)"); diff --git a/library/cpp/actors/core/log.h b/library/cpp/actors/core/log.h index c7b6e85bef4..c11a7cf3c19 100644 --- a/library/cpp/actors/core/log.h +++ b/library/cpp/actors/core/log.h @@ -14,7 +14,7 @@ #include <util/string/printf.h> #include <util/string/builder.h> #include <library/cpp/logger/all.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <library/cpp/monlib/metrics/metric_registry.h> #include <library/cpp/json/writer/json.h> #include <library/cpp/svnversion/svnversion.h> @@ -323,7 +323,7 @@ namespace NActors { { const NLog::TSettings *mSettings = ctx.LoggerSettings(); TLoggerActor::Throttle(*mSettings); - ctx.Send(new IEventHandle(mSettings->LoggerActorId, TActorId(), new NLog::TEvLog(mPriority, mComponent, std::move(str)))); + ctx.Send(new IEventHandle(mSettings->LoggerActorId, TActorId(), new NLog::TEvLog(mPriority, mComponent, std::move(str)))); } template <typename TCtx, typename... TArgs> diff --git a/library/cpp/actors/core/log_settings.cpp b/library/cpp/actors/core/log_settings.cpp index de2a3a9a688..f52f2fc5d22 100644 --- a/library/cpp/actors/core/log_settings.cpp +++ b/library/cpp/actors/core/log_settings.cpp @@ -4,7 +4,7 @@ namespace NActors { namespace NLog { - TSettings::TSettings(const TActorId& loggerActorId, const EComponent loggerComponent, + TSettings::TSettings(const TActorId& loggerActorId, const EComponent loggerComponent, EComponent minVal, EComponent maxVal, EComponentToStringFunc func, EPriority defPriority, EPriority defSamplingPriority, ui32 defSamplingRate, ui64 timeThresholdMs) @@ -27,7 +27,7 @@ namespace NActors { Append(minVal, maxVal, func); } - TSettings::TSettings(const TActorId& loggerActorId, const EComponent loggerComponent, + TSettings::TSettings(const TActorId& loggerActorId, const EComponent loggerComponent, EPriority defPriority, EPriority defSamplingPriority, ui32 defSamplingRate, ui64 timeThresholdMs) : LoggerActorId(loggerActorId) diff --git a/library/cpp/actors/core/log_settings.h b/library/cpp/actors/core/log_settings.h index 564d2db73e6..7fe4504edd9 100644 --- a/library/cpp/actors/core/log_settings.h +++ b/library/cpp/actors/core/log_settings.h @@ -69,7 +69,7 @@ namespace NActors { struct TSettings: public TThrRefBase { public: - TActorId LoggerActorId; + TActorId LoggerActorId; EComponent LoggerComponent; ui64 TimeThresholdMs; bool AllowDrop; @@ -98,12 +98,12 @@ namespace NActors { // protobuf enumeration of components. In this case protoc // automatically generates YOURTYPE_MIN, YOURTYPE_MAX and // YOURTYPE_Name for you. - TSettings(const TActorId& loggerActorId, const EComponent loggerComponent, + TSettings(const TActorId& loggerActorId, const EComponent loggerComponent, EComponent minVal, EComponent maxVal, EComponentToStringFunc func, EPriority defPriority, EPriority defSamplingPriority = PRI_DEBUG, ui32 defSamplingRate = 0, ui64 timeThresholdMs = 1000); - TSettings(const TActorId& loggerActorId, const EComponent loggerComponent, + TSettings(const TActorId& loggerActorId, const EComponent loggerComponent, EPriority defPriority, EPriority defSamplingPriority = PRI_DEBUG, ui32 defSamplingRate = 0, ui64 timeThresholdMs = 1000); diff --git a/library/cpp/actors/core/log_ut.cpp b/library/cpp/actors/core/log_ut.cpp index 2a65e270de0..09b5f88ea2c 100644 --- a/library/cpp/actors/core/log_ut.cpp +++ b/library/cpp/actors/core/log_ut.cpp @@ -15,7 +15,7 @@ namespace { } TIntrusivePtr<TSettings> DefaultSettings() { - auto loggerId = TActorId{0, "Logger"}; + auto loggerId = TActorId{0, "Logger"}; auto s = MakeIntrusive<TSettings>(loggerId, 0, EPriority::PRI_TRACE); s->SetAllowDrop(false); s->Append(0, 1, ServiceToString); @@ -98,7 +98,7 @@ namespace { TIntrusivePtr<TDynamicCounters> Counters{MakeIntrusive<TDynamicCounters>()}; std::shared_ptr<TMockBackend> LogBackend; - TActorId LoggerActor; + TActorId LoggerActor; TTestActorRuntimeBase Runtime; }; } diff --git a/library/cpp/actors/core/mailbox.cpp b/library/cpp/actors/core/mailbox.cpp index 40fcebaa723..d84b4f9e466 100644 --- a/library/cpp/actors/core/mailbox.cpp +++ b/library/cpp/actors/core/mailbox.cpp @@ -163,7 +163,7 @@ namespace NActors { } bool TMailboxTable::SendTo(TAutoPtr<IEventHandle>& ev, IExecutorPool* executorPool) { - const TActorId& recipient = ev->GetRecipientRewrite(); + const TActorId& recipient = ev->GetRecipientRewrite(); const ui32 hint = recipient.Hint(); // copy-paste from Get to avoid duplicated type-switches diff --git a/library/cpp/actors/core/mailbox.h b/library/cpp/actors/core/mailbox.h index 38a03af42d8..0bd9c4d314e 100644 --- a/library/cpp/actors/core/mailbox.h +++ b/library/cpp/actors/core/mailbox.h @@ -305,14 +305,14 @@ namespace NActors { static const ui32 LineIndexShift = 12; static const ui32 LineIndexMask = 0x1FFFFu << LineIndexShift; static const ui32 LineHintMask = 0xFFFu; - static const ui32 PoolIndexShift = TActorId::PoolIndexShift; - static const ui32 PoolIndexMask = TActorId::PoolIndexMask; + static const ui32 PoolIndexShift = TActorId::PoolIndexShift; + static const ui32 PoolIndexMask = TActorId::PoolIndexMask; static ui32 LineIndex(ui32 hint) { return ((hint & LineIndexMask) >> LineIndexShift); } static ui32 PoolIndex(ui32 hint) { - return TActorId::PoolIndex(hint); + return TActorId::PoolIndex(hint); } TMailboxHeader* Get(ui32 hint); diff --git a/library/cpp/actors/core/mon.h b/library/cpp/actors/core/mon.h index 45e0e7ff659..c450f2338eb 100644 --- a/library/cpp/actors/core/mon.h +++ b/library/cpp/actors/core/mon.h @@ -2,8 +2,8 @@ #include "events.h" #include "event_local.h" -#include <library/cpp/monlib/service/monservice.h> -#include <library/cpp/monlib/service/pages/mon_page.h> +#include <library/cpp/monlib/service/monservice.h> +#include <library/cpp/monlib/service/pages/mon_page.h> namespace NActors { namespace NMon { diff --git a/library/cpp/actors/core/process_stats.cpp b/library/cpp/actors/core/process_stats.cpp index 61bf7452a71..0e1dbd00314 100644 --- a/library/cpp/actors/core/process_stats.cpp +++ b/library/cpp/actors/core/process_stats.cpp @@ -3,8 +3,8 @@ #include "hfunc.h" #include "process_stats.h" -#include <library/cpp/monlib/dynamic_counters/counters.h> -#include <library/cpp/monlib/metrics/metric_registry.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/metrics/metric_registry.h> #include <util/datetime/uptime.h> #include <util/system/defaults.h> @@ -197,7 +197,7 @@ namespace { MinorPageFaults = ProcStatGroup->GetCounter("Process/MinorPageFaults", true); MajorPageFaults = ProcStatGroup->GetCounter("Process/MajorPageFaults", true); UptimeSeconds = ProcStatGroup->GetCounter("Process/UptimeSeconds", false); - NumThreads = ProcStatGroup->GetCounter("Process/NumThreads", false); + NumThreads = ProcStatGroup->GetCounter("Process/NumThreads", false); SystemUptimeSeconds = ProcStatGroup->GetCounter("System/UptimeSeconds", false); } @@ -213,7 +213,7 @@ namespace { *MinorPageFaults = procStat.MinFlt; *MajorPageFaults = procStat.MajFlt; *UptimeSeconds = procStat.Uptime.Seconds(); - *NumThreads = procStat.NumThreads; + *NumThreads = procStat.NumThreads; *SystemUptimeSeconds = procStat.Uptime.Seconds(); } @@ -228,7 +228,7 @@ namespace { NMonitoring::TDynamicCounters::TCounterPtr MinorPageFaults; NMonitoring::TDynamicCounters::TCounterPtr MajorPageFaults; NMonitoring::TDynamicCounters::TCounterPtr UptimeSeconds; - NMonitoring::TDynamicCounters::TCounterPtr NumThreads; + NMonitoring::TDynamicCounters::TCounterPtr NumThreads; NMonitoring::TDynamicCounters::TCounterPtr SystemUptimeSeconds; }; @@ -236,7 +236,7 @@ namespace { class TRegistryCollector: public TProcStatCollectingActor<TRegistryCollector> { using TBase = TProcStatCollectingActor<TRegistryCollector>; public: - TRegistryCollector(TDuration interval, NMonitoring::TMetricRegistry& registry) + TRegistryCollector(TDuration interval, NMonitoring::TMetricRegistry& registry) : TBase{interval} { VmSize = registry.IntGauge({{"sensor", "process.VmSize"}}); @@ -244,13 +244,13 @@ namespace { FileRssSize = registry.IntGauge({{"sensor", "process.FileRssSize"}}); CGroupMemLimit = registry.IntGauge({{"sensor", "process.CGroupMemLimit"}}); UptimeSeconds = registry.IntGauge({{"sensor", "process.UptimeSeconds"}}); - NumThreads = registry.IntGauge({{"sensor", "process.NumThreads"}}); + NumThreads = registry.IntGauge({{"sensor", "process.NumThreads"}}); SystemUptimeSeconds = registry.IntGauge({{"sensor", "system.UptimeSeconds"}}); - - UserTime = registry.Rate({{"sensor", "process.UserTime"}}); - SysTime = registry.Rate({{"sensor", "process.SystemTime"}}); - MinorPageFaults = registry.Rate({{"sensor", "process.MinorPageFaults"}}); - MajorPageFaults = registry.Rate({{"sensor", "process.MajorPageFaults"}}); + + UserTime = registry.Rate({{"sensor", "process.UserTime"}}); + SysTime = registry.Rate({{"sensor", "process.SystemTime"}}); + MinorPageFaults = registry.Rate({{"sensor", "process.MinorPageFaults"}}); + MajorPageFaults = registry.Rate({{"sensor", "process.MajorPageFaults"}}); } void UpdateCounters(const TProcStat& procStat) { @@ -259,23 +259,23 @@ namespace { FileRssSize->Set(procStat.FileRss); CGroupMemLimit->Set(procStat.CGroupMemLim); UptimeSeconds->Set(procStat.Uptime.Seconds()); - NumThreads->Set(procStat.NumThreads); + NumThreads->Set(procStat.NumThreads); SystemUptimeSeconds->Set(procStat.SystemUptime.Seconds()); - - // it is ok here to reset and add metric value, because mutation - // is performed in siglethreaded context - - UserTime->Reset(); - UserTime->Add(procStat.Utime); - - SysTime->Reset(); - SysTime->Add(procStat.Stime); - - MinorPageFaults->Reset(); - MinorPageFaults->Add(procStat.MinFlt); - - MajorPageFaults->Reset(); - MajorPageFaults->Add(procStat.MajFlt); + + // it is ok here to reset and add metric value, because mutation + // is performed in siglethreaded context + + UserTime->Reset(); + UserTime->Add(procStat.Utime); + + SysTime->Reset(); + SysTime->Add(procStat.Stime); + + MinorPageFaults->Reset(); + MinorPageFaults->Add(procStat.MinFlt); + + MajorPageFaults->Reset(); + MajorPageFaults->Add(procStat.MajFlt); } private: @@ -283,12 +283,12 @@ namespace { NMonitoring::TIntGauge* AnonRssSize; NMonitoring::TIntGauge* FileRssSize; NMonitoring::TIntGauge* CGroupMemLimit; - NMonitoring::TRate* UserTime; - NMonitoring::TRate* SysTime; - NMonitoring::TRate* MinorPageFaults; - NMonitoring::TRate* MajorPageFaults; + NMonitoring::TRate* UserTime; + NMonitoring::TRate* SysTime; + NMonitoring::TRate* MinorPageFaults; + NMonitoring::TRate* MajorPageFaults; NMonitoring::TIntGauge* UptimeSeconds; - NMonitoring::TIntGauge* NumThreads; + NMonitoring::TIntGauge* NumThreads; NMonitoring::TIntGauge* SystemUptimeSeconds; }; } // namespace @@ -297,7 +297,7 @@ namespace { return new TDynamicCounterCollector(intervalSec, counters); } - IActor* CreateProcStatCollector(TDuration interval, NMonitoring::TMetricRegistry& registry) { + IActor* CreateProcStatCollector(TDuration interval, NMonitoring::TMetricRegistry& registry) { return new TRegistryCollector(interval, registry); } } diff --git a/library/cpp/actors/core/process_stats.h b/library/cpp/actors/core/process_stats.h index 4e6bb310904..66346d0b5aa 100644 --- a/library/cpp/actors/core/process_stats.h +++ b/library/cpp/actors/core/process_stats.h @@ -3,10 +3,10 @@ #include "defs.h" #include "actor.h" -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> namespace NMonitoring { - class TMetricRegistry; + class TMetricRegistry; } namespace NActors { @@ -62,5 +62,5 @@ namespace NActors { }; IActor* CreateProcStatCollector(ui32 intervalSec, NMonitoring::TDynamicCounterPtr counters); - IActor* CreateProcStatCollector(TDuration interval, NMonitoring::TMetricRegistry& registry); + IActor* CreateProcStatCollector(TDuration interval, NMonitoring::TMetricRegistry& registry); } diff --git a/library/cpp/actors/core/scheduler_actor.cpp b/library/cpp/actors/core/scheduler_actor.cpp index d1e6501fd11..febc5e40dd2 100644 --- a/library/cpp/actors/core/scheduler_actor.cpp +++ b/library/cpp/actors/core/scheduler_actor.cpp @@ -39,7 +39,7 @@ namespace NActors { TVector<NSchedulerQueue::TReader*> Readers; - TActorId PollerActor; + TActorId PollerActor; TPollerToken::TPtr PollerToken; ui64 RealTime; @@ -68,7 +68,7 @@ namespace NActors { : TActor(&TSchedulerActor::StateFunc) , Cfg(cfg) , TimerDescriptor(new TTimerDescriptor()) - , PollerActor(MakePollerActorId()) + , PollerActor(MakePollerActorId()) { Y_ASSERT(Cfg.ResolutionMicroseconds != 0); Y_ASSERT(Cfg.ProgressThreshold != 0); diff --git a/library/cpp/actors/core/scheduler_actor.h b/library/cpp/actors/core/scheduler_actor.h index 600f8d98fff..c2c561b43d2 100644 --- a/library/cpp/actors/core/scheduler_actor.h +++ b/library/cpp/actors/core/scheduler_actor.h @@ -21,9 +21,9 @@ namespace NActors { IActor* CreateSchedulerActor(const TSchedulerConfig& cfg); - inline TActorId MakeSchedulerActorId() { + inline TActorId MakeSchedulerActorId() { char x[12] = {'s', 'c', 'h', 'e', 'd', 'u', 'l', 'e', 'r', 's', 'e', 'r'}; - return TActorId(0, TStringBuf(x, 12)); + return TActorId(0, TStringBuf(x, 12)); } } diff --git a/library/cpp/actors/core/scheduler_actor_ut.cpp b/library/cpp/actors/core/scheduler_actor_ut.cpp index dae14cbe674..09b7369d360 100644 --- a/library/cpp/actors/core/scheduler_actor_ut.cpp +++ b/library/cpp/actors/core/scheduler_actor_ut.cpp @@ -60,14 +60,14 @@ Y_UNIT_TEST_SUITE(SchedulerActor) { setup->Executors[i] = new TBasicExecutorPool(i, 5, 10, "basic"); } // create poller actor (whether platform supports it) - TActorId pollerActorId; + TActorId pollerActorId; if (IActor* poller = CreatePollerActor()) { - pollerActorId = MakePollerActorId(); + pollerActorId = MakePollerActorId(); setup->LocalServices.emplace_back(pollerActorId, TActorSetupCmd(poller, TMailboxType::ReadAsFilled, 0)); } - TActorId schedulerActorId; + TActorId schedulerActorId; if (IActor* schedulerActor = CreateSchedulerActor(TSchedulerConfig())) { - schedulerActorId = MakeSchedulerActorId(); + schedulerActorId = MakeSchedulerActorId(); setup->LocalServices.emplace_back(schedulerActorId, TActorSetupCmd(schedulerActor, TMailboxType::ReadAsFilled, 0)); } setup->Scheduler = CreateSchedulerThread(TSchedulerConfig()); diff --git a/library/cpp/actors/core/ut/ya.make b/library/cpp/actors/core/ut/ya.make index 11b2ea3eb7a..3ee28d58503 100644 --- a/library/cpp/actors/core/ut/ya.make +++ b/library/cpp/actors/core/ut/ya.make @@ -29,18 +29,18 @@ PEERDIR( ) SRCS( - actor_coroutine_ut.cpp - actor_ut.cpp - actorsystem_ut.cpp + actor_coroutine_ut.cpp + actor_ut.cpp + actorsystem_ut.cpp ask_ut.cpp balancer_ut.cpp - event_pb_payload_ut.cpp + event_pb_payload_ut.cpp event_pb_ut.cpp executor_pool_basic_ut.cpp executor_pool_united_ut.cpp log_ut.cpp memory_tracker_ut.cpp - scheduler_actor_ut.cpp + scheduler_actor_ut.cpp ) END() diff --git a/library/cpp/actors/core/ya.make b/library/cpp/actors/core/ya.make index 40f27456c8c..880a9d00dba 100644 --- a/library/cpp/actors/core/ya.make +++ b/library/cpp/actors/core/ya.make @@ -111,7 +111,7 @@ PEERDIR( library/cpp/json/writer library/cpp/logger library/cpp/lwtrace - library/cpp/monlib/dynamic_counters + library/cpp/monlib/dynamic_counters library/cpp/svnversion library/cpp/threading/future ) diff --git a/library/cpp/actors/helpers/activeactors.h b/library/cpp/actors/helpers/activeactors.h index b0e4f5cc994..0fdb0fab108 100644 --- a/library/cpp/actors/helpers/activeactors.h +++ b/library/cpp/actors/helpers/activeactors.h @@ -10,9 +10,9 @@ namespace NActors { // TActiveActors // This class helps manage created actors and kill them all on PoisonPill. //////////////////////////////////////////////////////////////////////////// - class TActiveActors : public THashSet<TActorId> { + class TActiveActors : public THashSet<TActorId> { public: - void Insert(const TActorId &aid) { + void Insert(const TActorId &aid) { bool inserted = insert(aid).second; Y_VERIFY(inserted); } @@ -23,7 +23,7 @@ namespace NActors { } } - void Erase(const TActorId &aid) { + void Erase(const TActorId &aid) { auto num = erase(aid); Y_VERIFY(num == 1); } diff --git a/library/cpp/actors/helpers/flow_controlled_queue.cpp b/library/cpp/actors/helpers/flow_controlled_queue.cpp index 61610ec3d36..d75cc540236 100644 --- a/library/cpp/actors/helpers/flow_controlled_queue.cpp +++ b/library/cpp/actors/helpers/flow_controlled_queue.cpp @@ -18,12 +18,12 @@ class TFlowControlledRequestActor : public IActor { void HandleReply(TAutoPtr<IEventHandle> &ev); void HandleUndelivered(TEvents::TEvUndelivered::TPtr &ev); public: - const TActorId Source; + const TActorId Source; const ui64 Cookie; const ui32 Flags; const ui64 StartCounter; - TFlowControlledRequestActor(ui32 activity, TFlowControlledRequestQueue *queue, TActorId source, ui64 cookie, ui32 flags) + TFlowControlledRequestActor(ui32 activity, TFlowControlledRequestQueue *queue, TActorId source, ui64 cookie, ui32 flags) : IActor(static_cast<TReceiveFunc>(&TFlowControlledRequestActor::StateWait), activity) , QueueActor(queue) , Source(source) @@ -49,7 +49,7 @@ public: }; class TFlowControlledRequestQueue : public IActor { - const TActorId Target; + const TActorId Target; const TFlowControlledQueueConfig Config; TDeque<THolder<IEventHandle>> UnhandledRequests; @@ -123,7 +123,7 @@ class TFlowControlledRequestQueue : public IActor { if (reqActor) { if (reqActor->Flags & IEventHandle::FlagSubscribeOnSession) { TActivationContext::Send( - new IEventHandle(reqActor->Source, TActorId(), new TEvInterconnect::TEvNodeDisconnected(nodeid), 0, reqActor->Cookie) + new IEventHandle(reqActor->Source, TActorId(), new TEvInterconnect::TEvNodeDisconnected(nodeid), 0, reqActor->Cookie) ); } reqActor->PassAway(); @@ -153,7 +153,7 @@ class TFlowControlledRequestQueue : public IActor { PassAway(); } public: - TFlowControlledRequestQueue(TActorId target, ui32 activity, const TFlowControlledQueueConfig &config) + TFlowControlledRequestQueue(TActorId target, ui32 activity, const TFlowControlledQueueConfig &config) : IActor(static_cast<TReceiveFunc>(&TFlowControlledRequestQueue::StateWork), activity) , Target(target) , Config(config) @@ -208,7 +208,7 @@ void TFlowControlledRequestActor::HandleUndelivered(TEvents::TEvUndelivered::TPt } -IActor* CreateFlowControlledRequestQueue(TActorId targetId, ui32 activity, const TFlowControlledQueueConfig &config) { +IActor* CreateFlowControlledRequestQueue(TActorId targetId, ui32 activity, const TFlowControlledQueueConfig &config) { return new TFlowControlledRequestQueue(targetId, activity, config); } diff --git a/library/cpp/actors/helpers/flow_controlled_queue.h b/library/cpp/actors/helpers/flow_controlled_queue.h index 1d03226103c..d2504053047 100644 --- a/library/cpp/actors/helpers/flow_controlled_queue.h +++ b/library/cpp/actors/helpers/flow_controlled_queue.h @@ -13,6 +13,6 @@ namespace NActors { ui32 LatencyFactor = 4; }; - IActor* CreateFlowControlledRequestQueue(TActorId targetId, ui32 activity = IActor::ACTORLIB_COMMON, const TFlowControlledQueueConfig &config = TFlowControlledQueueConfig()); + IActor* CreateFlowControlledRequestQueue(TActorId targetId, ui32 activity = IActor::ACTORLIB_COMMON, const TFlowControlledQueueConfig &config = TFlowControlledQueueConfig()); } diff --git a/library/cpp/actors/helpers/mon_histogram_helper.h b/library/cpp/actors/helpers/mon_histogram_helper.h index 80b9690a757..a9a57e38238 100644 --- a/library/cpp/actors/helpers/mon_histogram_helper.h +++ b/library/cpp/actors/helpers/mon_histogram_helper.h @@ -1,9 +1,9 @@ #pragma once -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> + +#include <util/string/cast.h> -#include <util/string/cast.h> - namespace NActors { namespace NMon { class THistogramCounterHelper { @@ -79,7 +79,7 @@ namespace NActors { ui64 FirstBucketVal; ui64 BucketCount; TVector<NMonitoring::TDynamicCounters::TCounterPtr> BucketsHolder; - TVector<NMonitoring::TDeprecatedCounter*> Buckets; + TVector<NMonitoring::TDeprecatedCounter*> Buckets; }; } diff --git a/library/cpp/actors/helpers/selfping_actor.h b/library/cpp/actors/helpers/selfping_actor.h index d1f320509ed..d7d07f9fa8b 100644 --- a/library/cpp/actors/helpers/selfping_actor.h +++ b/library/cpp/actors/helpers/selfping_actor.h @@ -1,7 +1,7 @@ #pragma once #include <library/cpp/actors/core/actor.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> namespace NActors { diff --git a/library/cpp/actors/helpers/ya.make b/library/cpp/actors/helpers/ya.make index 0169a2c7276..d8771179de8 100644 --- a/library/cpp/actors/helpers/ya.make +++ b/library/cpp/actors/helpers/ya.make @@ -14,7 +14,7 @@ SRCS( PEERDIR( library/cpp/actors/core - library/cpp/monlib/dynamic_counters + library/cpp/monlib/dynamic_counters ) END() diff --git a/library/cpp/actors/http/http_cache.cpp b/library/cpp/actors/http/http_cache.cpp index 834fe47b738..27c4eeb6f32 100644 --- a/library/cpp/actors/http/http_cache.cpp +++ b/library/cpp/actors/http/http_cache.cpp @@ -16,7 +16,7 @@ namespace NHttp { class THttpOutgoingCacheActor : public NActors::TActorBootstrapped<THttpOutgoingCacheActor>, THttpConfig { public: using TBase = NActors::TActorBootstrapped<THttpOutgoingCacheActor>; - NActors::TActorId HttpProxyId; + NActors::TActorId HttpProxyId; TGetCachePolicy GetCachePolicy; static constexpr TDuration RefreshTimeout = TDuration::Seconds(1); @@ -584,7 +584,7 @@ TCachePolicy GetDefaultCachePolicy(const THttpRequest* request, const TCachePoli return policy; } -NActors::IActor* CreateHttpCache(const NActors::TActorId& httpProxyId, TGetCachePolicy cachePolicy) { +NActors::IActor* CreateHttpCache(const NActors::TActorId& httpProxyId, TGetCachePolicy cachePolicy) { return new THttpOutgoingCacheActor(httpProxyId, std::move(cachePolicy)); } diff --git a/library/cpp/actors/http/http_cache.h b/library/cpp/actors/http/http_cache.h index 313c7bd2665..ac38bdcac89 100644 --- a/library/cpp/actors/http/http_cache.h +++ b/library/cpp/actors/http/http_cache.h @@ -19,7 +19,7 @@ struct TCachePolicy { using TGetCachePolicy = std::function<TCachePolicy(const THttpRequest*)>; -NActors::IActor* CreateHttpCache(const NActors::TActorId& httpProxyId, TGetCachePolicy cachePolicy); +NActors::IActor* CreateHttpCache(const NActors::TActorId& httpProxyId, TGetCachePolicy cachePolicy); NActors::IActor* CreateOutgoingHttpCache(const NActors::TActorId& httpProxyId, TGetCachePolicy cachePolicy); NActors::IActor* CreateIncomingHttpCache(const NActors::TActorId& httpProxyId, TGetCachePolicy cachePolicy); TCachePolicy GetDefaultCachePolicy(const THttpRequest* request, const TCachePolicy& policy = TCachePolicy()); diff --git a/library/cpp/actors/http/http_proxy.cpp b/library/cpp/actors/http/http_proxy.cpp index 3a466006cde..36c6855d93f 100644 --- a/library/cpp/actors/http/http_proxy.cpp +++ b/library/cpp/actors/http/http_proxy.cpp @@ -1,5 +1,5 @@ #include <library/cpp/actors/core/events.h> -#include <library/cpp/monlib/metrics/metric_registry.h> +#include <library/cpp/monlib/metrics/metric_registry.h> #include "http_proxy.h" namespace NHttp { @@ -8,7 +8,7 @@ class THttpProxy : public NActors::TActorBootstrapped<THttpProxy>, public THttpC public: IActor* AddListeningPort(TEvHttpProxy::TEvAddListeningPort::TPtr event, const NActors::TActorContext& ctx) { IActor* listeningSocket = CreateHttpAcceptorActor(ctx.SelfID, Poller); - TActorId acceptorId = ctx.Register(listeningSocket); + TActorId acceptorId = ctx.Register(listeningSocket); ctx.Send(event->Forward(acceptorId)); Acceptors.emplace_back(acceptorId); return listeningSocket; @@ -16,7 +16,7 @@ public: IActor* AddOutgoingConnection(const TString& address, bool secure, const NActors::TActorContext& ctx) { IActor* connectionSocket = CreateOutgoingConnectionActor(ctx.SelfID, address, secure, Poller); - TActorId connectionId = ctx.Register(connectionSocket); + TActorId connectionId = ctx.Register(connectionSocket); Connections.emplace(connectionId); return connectionSocket; } @@ -26,7 +26,7 @@ public: Become(&THttpProxy::StateWork); } - THttpProxy(NMonitoring::TMetricRegistry& sensors) + THttpProxy(NMonitoring::TMetricRegistry& sensors) : Sensors(sensors) {} @@ -49,10 +49,10 @@ protected: void PassAway() override { Send(Poller, new NActors::TEvents::TEvPoisonPill()); - for (const NActors::TActorId& connection : Connections) { + for (const NActors::TActorId& connection : Connections) { Send(connection, new NActors::TEvents::TEvPoisonPill()); } - for (const NActors::TActorId& acceptor : Acceptors) { + for (const NActors::TActorId& acceptor : Acceptors) { Send(acceptor, new NActors::TEvents::TEvPoisonPill()); } NActors::TActorBootstrapped<THttpProxy>::PassAway(); @@ -60,7 +60,7 @@ protected: void Handle(TEvHttpProxy::TEvHttpIncomingRequest::TPtr event, const NActors::TActorContext& ctx) { TStringBuf url = event->Get()->Request->URL.Before('?'); - THashMap<TString, TActorId>::iterator it; + THashMap<TString, TActorId>::iterator it; while (!url.empty()) { it = Handlers.find(url); if (it != Handlers.end()) { @@ -204,8 +204,8 @@ protected: PassAway(); } - NActors::TActorId Poller; - TVector<TActorId> Acceptors; + NActors::TActorId Poller; + TVector<TActorId> Acceptors; struct THostEntry { TSockAddrInet6 Address; @@ -215,9 +215,9 @@ protected: static constexpr TDuration HostsTimeToLive = TDuration::Seconds(60); THashMap<TString, THostEntry> Hosts; - THashMap<TString, TActorId> Handlers; - THashSet<TActorId> Connections; // outgoing - NMonitoring::TMetricRegistry& Sensors; + THashMap<TString, TActorId> Handlers; + THashSet<TActorId> Connections; // outgoing + NMonitoring::TMetricRegistry& Sensors; }; TEvHttpProxy::TEvReportSensors* BuildOutgoingRequestSensors(const THttpOutgoingRequestPtr& request, const THttpIncomingResponsePtr& response) { @@ -240,7 +240,7 @@ TEvHttpProxy::TEvReportSensors* BuildIncomingRequestSensors(const THttpIncomingR ); } -NActors::IActor* CreateHttpProxy(NMonitoring::TMetricRegistry& sensors) { +NActors::IActor* CreateHttpProxy(NMonitoring::TMetricRegistry& sensors) { return new THttpProxy(sensors); } diff --git a/library/cpp/actors/http/http_proxy.h b/library/cpp/actors/http/http_proxy.h index 97ea6fbd442..afd0170997e 100644 --- a/library/cpp/actors/http/http_proxy.h +++ b/library/cpp/actors/http/http_proxy.h @@ -8,7 +8,7 @@ #include <library/cpp/actors/core/log.h> #include <library/cpp/actors/interconnect/poller_actor.h> #include <library/cpp/dns/cache.h> -#include <library/cpp/monlib/metrics/metric_registry.h> +#include <library/cpp/monlib/metrics/metric_registry.h> #include <util/generic/variant.h> #include "http.h" #include "http_proxy_ssl.h" @@ -25,7 +25,7 @@ struct TSocketDescriptor : NActors::TSharedDescriptor, THttpConfig { struct TEvHttpProxy { enum EEv { - EvAddListeningPort = EventSpaceBegin(NActors::TEvents::ES_HTTP), + EvAddListeningPort = EventSpaceBegin(NActors::TEvents::ES_HTTP), EvConfirmListen, EvRegisterHandler, EvHttpIncomingRequest, @@ -41,7 +41,7 @@ struct TEvHttpProxy { EvEnd }; - static_assert(EvEnd < EventSpaceEnd(NActors::TEvents::ES_HTTP), "ES_HTTP event space is too small."); + static_assert(EvEnd < EventSpaceEnd(NActors::TEvents::ES_HTTP), "ES_HTTP event space is too small."); struct TEvAddListeningPort : NActors::TEventLocal<TEvAddListeningPort, EvAddListeningPort> { TIpPort Port; @@ -71,9 +71,9 @@ struct TEvHttpProxy { struct TEvRegisterHandler : NActors::TEventLocal<TEvRegisterHandler, EvRegisterHandler> { TString Path; - TActorId Handler; + TActorId Handler; - TEvRegisterHandler(const TString& path, const TActorId& handler) + TEvRegisterHandler(const TString& path, const TActorId& handler) : Path(path) , Handler(handler) {} @@ -142,32 +142,32 @@ struct TEvHttpProxy { struct TEvHttpConnectionOpened : NActors::TEventLocal<TEvHttpConnectionOpened, EvHttpConnectionOpened> { TString PeerAddress; - TActorId ConnectionID; + TActorId ConnectionID; - TEvHttpConnectionOpened(const TString& peerAddress, const TActorId& connectionID) + TEvHttpConnectionOpened(const TString& peerAddress, const TActorId& connectionID) : PeerAddress(peerAddress) , ConnectionID(connectionID) {} }; struct TEvHttpConnectionClosed : NActors::TEventLocal<TEvHttpConnectionClosed, EvHttpConnectionClosed> { - TActorId ConnectionID; + TActorId ConnectionID; TDeque<THttpIncomingRequestPtr> RecycledRequests; - TEvHttpConnectionClosed(const TActorId& connectionID) + TEvHttpConnectionClosed(const TActorId& connectionID) : ConnectionID(connectionID) {} - TEvHttpConnectionClosed(const TActorId& connectionID, TDeque<THttpIncomingRequestPtr> recycledRequests) + TEvHttpConnectionClosed(const TActorId& connectionID, TDeque<THttpIncomingRequestPtr> recycledRequests) : ConnectionID(connectionID) , RecycledRequests(std::move(recycledRequests)) {} }; struct TEvHttpAcceptorClosed : NActors::TEventLocal<TEvHttpAcceptorClosed, EvHttpAcceptorClosed> { - TActorId ConnectionID; + TActorId ConnectionID; - TEvHttpAcceptorClosed(const TActorId& connectionID) + TEvHttpAcceptorClosed(const TActorId& connectionID) : ConnectionID(connectionID) {} }; @@ -218,16 +218,16 @@ struct TEvHttpProxy { }; struct TEndpointInfo { - TActorId Proxy; - TActorId Owner; + TActorId Proxy; + TActorId Owner; TString WorkerName; bool Secure; TSslHelpers::TSslHolder<SSL_CTX> SecureContext; }; -NActors::IActor* CreateHttpProxy(NMonitoring::TMetricRegistry& sensors); -NActors::IActor* CreateHttpAcceptorActor(const TActorId& owner, const TActorId& poller); -NActors::IActor* CreateOutgoingConnectionActor(const TActorId& owner, const TString& host, bool secure, const TActorId& poller); +NActors::IActor* CreateHttpProxy(NMonitoring::TMetricRegistry& sensors); +NActors::IActor* CreateHttpAcceptorActor(const TActorId& owner, const TActorId& poller); +NActors::IActor* CreateOutgoingConnectionActor(const TActorId& owner, const TString& host, bool secure, const TActorId& poller); NActors::IActor* CreateIncomingConnectionActor( const TEndpointInfo& endpoint, TIntrusivePtr<TSocketDescriptor> socket, diff --git a/library/cpp/actors/http/http_proxy_acceptor.cpp b/library/cpp/actors/http/http_proxy_acceptor.cpp index 95b07ffa843..9780541b71f 100644 --- a/library/cpp/actors/http/http_proxy_acceptor.cpp +++ b/library/cpp/actors/http/http_proxy_acceptor.cpp @@ -7,15 +7,15 @@ namespace NHttp { class TAcceptorActor : public NActors::TActor<TAcceptorActor>, public THttpConfig { public: using TBase = NActors::TActor<TAcceptorActor>; - const TActorId Owner; - const TActorId Poller; + const TActorId Owner; + const TActorId Poller; TIntrusivePtr<TSocketDescriptor> Socket; NActors::TPollerToken::TPtr PollerToken; - THashSet<TActorId> Connections; + THashSet<TActorId> Connections; TDeque<THttpIncomingRequestPtr> RecycledRequests; TEndpointInfo Endpoint; - TAcceptorActor(const TActorId& owner, const TActorId& poller) + TAcceptorActor(const TActorId& owner, const TActorId& poller) : NActors::TActor<TAcceptorActor>(&TAcceptorActor::StateInit) , Owner(owner) , Poller(poller) @@ -77,12 +77,12 @@ protected: } } LOG_WARN_S(ctx, HttpLog, "Failed to listen on " << bindAddress.ToString() << " - retrying..."); - ctx.ExecutorThread.Schedule(TDuration::Seconds(1), event.Release()); + ctx.ExecutorThread.Schedule(TDuration::Seconds(1), event.Release()); } void Die(const NActors::TActorContext& ctx) override { ctx.Send(Owner, new TEvHttpProxy::TEvHttpAcceptorClosed(ctx.SelfID)); - for (const NActors::TActorId& connection : Connections) { + for (const NActors::TActorId& connection : Connections) { ctx.Send(connection, new NActors::TEvents::TEvPoisonPill()); } } @@ -104,7 +104,7 @@ protected: connectionSocket = CreateIncomingConnectionActor(Endpoint, socket, addr, std::move(RecycledRequests.front())); RecycledRequests.pop_front(); } - NActors::TActorId connectionId = ctx.Register(connectionSocket); + NActors::TActorId connectionId = ctx.Register(connectionSocket); ctx.Send(Poller, new NActors::TEvPollerRegister(socket, connectionId, connectionId)); Connections.emplace(connectionId); socket = new TSocketDescriptor(); @@ -128,7 +128,7 @@ protected: } }; -NActors::IActor* CreateHttpAcceptorActor(const TActorId& owner, const TActorId& poller) { +NActors::IActor* CreateHttpAcceptorActor(const TActorId& owner, const TActorId& poller) { return new TAcceptorActor(owner, poller); } diff --git a/library/cpp/actors/http/http_proxy_outgoing.cpp b/library/cpp/actors/http/http_proxy_outgoing.cpp index 5bd4dd74b02..d9189dba8ab 100644 --- a/library/cpp/actors/http/http_proxy_outgoing.cpp +++ b/library/cpp/actors/http/http_proxy_outgoing.cpp @@ -8,18 +8,18 @@ class TOutgoingConnectionActor : public NActors::TActor<TOutgoingConnectionActor public: using TBase = NActors::TActor<TOutgoingConnectionActor<TSocketImpl>>; using TSelf = TOutgoingConnectionActor<TSocketImpl>; - const TActorId Owner; - const TActorId Poller; + const TActorId Owner; + const TActorId Poller; SocketAddressType Address; TString Host; - TActorId RequestOwner; + TActorId RequestOwner; THttpOutgoingRequestPtr Request; THttpIncomingResponsePtr Response; TInstant LastActivity; TDuration ConnectionTimeout = CONNECTION_TIMEOUT; NActors::TPollerToken::TPtr PollerToken; - TOutgoingConnectionActor(const TActorId& owner, const TString& host, const TActorId& poller) + TOutgoingConnectionActor(const TActorId& owner, const TString& host, const TActorId& poller) : TBase(&TSelf::StateWaiting) , Owner(owner) , Poller(poller) @@ -38,7 +38,7 @@ public: void ReplyAndDie(const NActors::TActorContext& ctx) { LOG_DEBUG_S(ctx, HttpLog, "(#" << TSocketImpl::GetRawSocket() << "," << Address << ") -> (" << Response->Status << " " << Response->Message << ")"); ctx.Send(RequestOwner, new TEvHttpProxy::TEvHttpIncomingResponse(Request, Response)); - RequestOwner = TActorId(); + RequestOwner = TActorId(); THolder<TEvHttpProxy::TEvReportSensors> sensors(BuildOutgoingRequestSensors(Request, Response)); ctx.Send(Owner, sensors.Release()); LOG_DEBUG_S(ctx, HttpLog, "(#" << TSocketImpl::GetRawSocket() << "," << Address << ") connection closed"); @@ -49,7 +49,7 @@ public: LOG_ERROR_S(ctx, HttpLog, "(#" << TSocketImpl::GetRawSocket() << "," << Address << ") connection closed with error: " << error); if (RequestOwner) { ctx.Send(RequestOwner, new TEvHttpProxy::TEvHttpIncomingResponse(Request, Response, error)); - RequestOwner = TActorId(); + RequestOwner = TActorId(); THolder<TEvHttpProxy::TEvReportSensors> sensors(BuildOutgoingRequestSensors(Request, Response)); ctx.Send(Owner, sensors.Release()); Die(ctx); @@ -287,7 +287,7 @@ protected: } }; -NActors::IActor* CreateOutgoingConnectionActor(const TActorId& owner, const TString& host, bool secure, const TActorId& poller) { +NActors::IActor* CreateOutgoingConnectionActor(const TActorId& owner, const TString& host, bool secure, const TActorId& poller) { if (secure) { return new TOutgoingConnectionActor<TSecureSocketImpl>(owner, host, poller); } else { diff --git a/library/cpp/actors/http/http_ut.cpp b/library/cpp/actors/http/http_ut.cpp index b21ceb550f9..4c922f8d0fc 100644 --- a/library/cpp/actors/http/http_ut.cpp +++ b/library/cpp/actors/http/http_ut.cpp @@ -180,17 +180,17 @@ Y_UNIT_TEST_SUITE(HttpProxy) { TIpPort port = portManager.GetTcpPort(); TAutoPtr<NActors::IEventHandle> handle; actorSystem.Initialize(); - NMonitoring::TMetricRegistry sensors; + NMonitoring::TMetricRegistry sensors; NActors::IActor* proxy = NHttp::CreateHttpProxy(sensors); - NActors::TActorId proxyId = actorSystem.Register(proxy); - actorSystem.Send(new NActors::IEventHandle(proxyId, TActorId(), new NHttp::TEvHttpProxy::TEvAddListeningPort(port)), 0, true); + NActors::TActorId proxyId = actorSystem.Register(proxy); + actorSystem.Send(new NActors::IEventHandle(proxyId, TActorId(), new NHttp::TEvHttpProxy::TEvAddListeningPort(port)), 0, true); actorSystem.DispatchEvents(); - NActors::TActorId serverId = actorSystem.AllocateEdgeActor(); + NActors::TActorId serverId = actorSystem.AllocateEdgeActor(); actorSystem.Send(new NActors::IEventHandle(proxyId, serverId, new NHttp::TEvHttpProxy::TEvRegisterHandler("/test", serverId)), 0, true); - NActors::TActorId clientId = actorSystem.AllocateEdgeActor(); + NActors::TActorId clientId = actorSystem.AllocateEdgeActor(); NHttp::THttpOutgoingRequestPtr httpRequest = NHttp::THttpOutgoingRequest::CreateRequestGet("http://[::1]:" + ToString(port) + "/test"); actorSystem.Send(new NActors::IEventHandle(proxyId, clientId, new NHttp::TEvHttpProxy::TEvHttpOutgoingRequest(httpRequest)), 0, true); @@ -213,7 +213,7 @@ Y_UNIT_TEST_SUITE(HttpProxy) { TIpPort port = portManager.GetTcpPort(); TAutoPtr<NActors::IEventHandle> handle; actorSystem.Initialize(); - NMonitoring::TMetricRegistry sensors; + NMonitoring::TMetricRegistry sensors; TString certificateContent = R"___(-----BEGIN PRIVATE KEY----- MIIEvwIBADANBgkqhkiG9w0BAQEFAASCBKkwggSlAgEAAoIBAQCzRZjodO7Aqe1w @@ -273,7 +273,7 @@ CRA/5XcX13GJwHHj6LCoc3sL7mt8qV9HKY2AOZ88mpObzISZxgPpdKCfjsrdm63V certificateFile.Write(certificateContent.data(), certificateContent.size()); NActors::IActor* proxy = NHttp::CreateHttpProxy(sensors); - NActors::TActorId proxyId = actorSystem.Register(proxy); + NActors::TActorId proxyId = actorSystem.Register(proxy); THolder<NHttp::TEvHttpProxy::TEvAddListeningPort> add = MakeHolder<NHttp::TEvHttpProxy::TEvAddListeningPort>(port); ///////// https configuration @@ -281,13 +281,13 @@ CRA/5XcX13GJwHHj6LCoc3sL7mt8qV9HKY2AOZ88mpObzISZxgPpdKCfjsrdm63V add->CertificateFile = certificateFile.Name(); add->PrivateKeyFile = certificateFile.Name(); ///////// - actorSystem.Send(new NActors::IEventHandle(proxyId, TActorId(), add.Release()), 0, true); + actorSystem.Send(new NActors::IEventHandle(proxyId, TActorId(), add.Release()), 0, true); actorSystem.DispatchEvents(); - NActors::TActorId serverId = actorSystem.AllocateEdgeActor(); + NActors::TActorId serverId = actorSystem.AllocateEdgeActor(); actorSystem.Send(new NActors::IEventHandle(proxyId, serverId, new NHttp::TEvHttpProxy::TEvRegisterHandler("/test", serverId)), 0, true); - NActors::TActorId clientId = actorSystem.AllocateEdgeActor(); + NActors::TActorId clientId = actorSystem.AllocateEdgeActor(); NHttp::THttpOutgoingRequestPtr httpRequest = NHttp::THttpOutgoingRequest::CreateRequestGet("https://[::1]:" + ToString(port) + "/test"); actorSystem.Send(new NActors::IEventHandle(proxyId, clientId, new NHttp::TEvHttpProxy::TEvHttpOutgoingRequest(httpRequest)), 0, true); @@ -314,11 +314,11 @@ CRA/5XcX13GJwHHj6LCoc3sL7mt8qV9HKY2AOZ88mpObzISZxgPpdKCfjsrdm63V NActors::TActorSystem actorSystem(setup); actorSystem.Start(); NHttp::THttpProxy* incomingProxy = new NHttp::THttpProxy(); - NActors::TActorId incomingProxyId = actorSystem.Register(incomingProxy); + NActors::TActorId incomingProxyId = actorSystem.Register(incomingProxy); actorSystem.Send(incomingProxyId, new NHttp::TEvHttpProxy::TEvAddListeningPort(13337)); NHttp::THttpProxy* outgoingProxy = new NHttp::THttpProxy(); - NActors::TActorId outgoingProxyId = actorSystem.Register(outgoingProxy); + NActors::TActorId outgoingProxyId = actorSystem.Register(outgoingProxy); THolder<NHttp::THttpStaticStringRequest> httpRequest = MakeHolder<NHttp::THttpStaticStringRequest>("GET /test HTTP/1.1\r\n\r\n"); actorSystem.Send(outgoingProxyId, new NHttp::TEvHttpProxy::TEvHttpOutgoingRequest("[::]:13337", std::move(httpRequest))); diff --git a/library/cpp/actors/http/ya.make b/library/cpp/actors/http/ya.make index ade447be3fb..7ce68b7a757 100644 --- a/library/cpp/actors/http/ya.make +++ b/library/cpp/actors/http/ya.make @@ -26,7 +26,7 @@ PEERDIR( library/cpp/actors/core library/cpp/actors/interconnect library/cpp/dns - library/cpp/monlib/metrics + library/cpp/monlib/metrics library/cpp/string_utils/quote ) diff --git a/library/cpp/actors/interconnect/events_local.h b/library/cpp/actors/interconnect/events_local.h index a7da62c3d77..8a46ffd535f 100644 --- a/library/cpp/actors/interconnect/events_local.h +++ b/library/cpp/actors/interconnect/events_local.h @@ -107,29 +107,29 @@ namespace NActors { struct TEvHandshakeAsk: public TEventLocal<TEvHandshakeAsk, ui32(ENetwork::HandshakeAsk)> { DEFINE_SIMPLE_LOCAL_EVENT(TEvHandshakeAsk, "Network: TEvHandshakeAsk") - TEvHandshakeAsk(const TActorId& self, - const TActorId& peer, + TEvHandshakeAsk(const TActorId& self, + const TActorId& peer, ui64 counter) : Self(self) , Peer(peer) , Counter(counter) { } - const TActorId Self; - const TActorId Peer; + const TActorId Self; + const TActorId Peer; const ui64 Counter; }; struct TEvHandshakeAck: public TEventLocal<TEvHandshakeAck, ui32(ENetwork::HandshakeAck)> { DEFINE_SIMPLE_LOCAL_EVENT(TEvHandshakeAck, "Network: TEvHandshakeAck") - TEvHandshakeAck(const TActorId& self, ui64 nextPacket, TSessionParams params) + TEvHandshakeAck(const TActorId& self, ui64 nextPacket, TSessionParams params) : Self(self) , NextPacket(nextPacket) , Params(std::move(params)) {} - const TActorId Self; + const TActorId Self; const ui64 NextPacket; const TSessionParams Params; }; @@ -185,8 +185,8 @@ namespace NActors { TEvHandshakeDone( TIntrusivePtr<NInterconnect::TStreamSocket> socket, - const TActorId& peer, - const TActorId& self, + const TActorId& peer, + const TActorId& self, ui64 nextPacket, TAutoPtr<TProgramInfo>&& programInfo, TSessionParams params) @@ -200,8 +200,8 @@ namespace NActors { } TIntrusivePtr<NInterconnect::TStreamSocket> Socket; - const TActorId Peer; - const TActorId Self; + const TActorId Peer; + const TActorId Self; const ui64 NextPacket; TAutoPtr<TProgramInfo> ProgramInfo; const TSessionParams Params; @@ -319,10 +319,10 @@ namespace NActors { template <typename TContainer> TEvLoadMessage(const TContainer& route, const TString& id, const TString* payload) { - for (const TActorId& actorId : route) { + for (const TActorId& actorId : route) { auto* hop = Record.AddHops(); if (actorId) { - ActorIdToProto(actorId, hop->MutableNextHop()); + ActorIdToProto(actorId, hop->MutableNextHop()); } } Record.SetId(id); @@ -366,13 +366,13 @@ namespace NActors { }; struct TEvSessionBufferSizeResponse : TEventLocal<TEvSessionBufferSizeResponse, static_cast<ui32>(ENetwork::EvSessionBufferSizeResponse)> { - TEvSessionBufferSizeResponse(const TActorId& sessionId, ui64 outputBufferSize) + TEvSessionBufferSizeResponse(const TActorId& sessionId, ui64 outputBufferSize) : SessionID(sessionId) , BufferSize(outputBufferSize) { } - TActorId SessionID; + TActorId SessionID; ui64 BufferSize; }; diff --git a/library/cpp/actors/interconnect/interconnect.h b/library/cpp/actors/interconnect/interconnect.h index f052a6e92ee..225a5243fd9 100644 --- a/library/cpp/actors/interconnect/interconnect.h +++ b/library/cpp/actors/interconnect/interconnect.h @@ -10,7 +10,7 @@ namespace NActors { TString SelfAddress; ui32 SelfPort; - TVector<TActorId> GlobalNameservers; // todo: add some info about (like expected reply time) + TVector<TActorId> GlobalNameservers; // todo: add some info about (like expected reply time) }; struct TInterconnectProxySetup: public TThrRefBase { @@ -41,12 +41,12 @@ namespace NActors { TIntrusivePtr<TInterconnectGlobalState> GlobalState; - virtual IActor* CreateSession(const TActorId& ownerId, IProxy* owner) = 0; // returned actor is session and would be attached to same mailbox as proxy to allow sync calls + virtual IActor* CreateSession(const TActorId& ownerId, IProxy* owner) = 0; // returned actor is session and would be attached to same mailbox as proxy to allow sync calls virtual TActorSetupCmd CreateAcceptor() = 0; }; struct TNameserverSetup { - TActorId ServiceID; + TActorId ServiceID; TIntrusivePtr<TInterconnectGlobalState> GlobalState; }; @@ -118,12 +118,12 @@ namespace NActors { }; struct TNodeRegistrarSetup { - TActorId ServiceID; + TActorId ServiceID; TIntrusivePtr<TInterconnectGlobalState> GlobalState; }; - TActorId GetNameserviceActorId(); + TActorId GetNameserviceActorId(); /** * Const table-lookup based name service diff --git a/library/cpp/actors/interconnect/interconnect_channel.h b/library/cpp/actors/interconnect/interconnect_channel.h index 659a6a9e5c8..e4a0ae3cdab 100644 --- a/library/cpp/actors/interconnect/interconnect_channel.h +++ b/library/cpp/actors/interconnect/interconnect_channel.h @@ -1,6 +1,6 @@ #pragma once -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <library/cpp/actors/core/actorsystem.h> #include <library/cpp/actors/core/event_load.h> #include <library/cpp/actors/util/rope.h> diff --git a/library/cpp/actors/interconnect/interconnect_common.h b/library/cpp/actors/interconnect/interconnect_common.h index 81e0694da16..285709a00cf 100644 --- a/library/cpp/actors/interconnect/interconnect_common.h +++ b/library/cpp/actors/interconnect/interconnect_common.h @@ -3,7 +3,7 @@ #include <library/cpp/actors/core/actorid.h> #include <library/cpp/actors/core/actorsystem.h> #include <library/cpp/actors/util/datetime.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <library/cpp/monlib/metrics/metric_registry.h> #include <util/generic/map.h> #include <util/generic/set.h> @@ -63,7 +63,7 @@ namespace NActors { typedef TMap<ui16, TChannelSettings> TChannelsConfig; using TRegisterMonPageCallback = std::function<void(const TString& path, const TString& title, - TActorSystem* actorSystem, const TActorId& actorId)>; + TActorSystem* actorSystem, const TActorId& actorId)>; using TInitWhiteboardCallback = std::function<void(ui16 icPort, TActorSystem* actorSystem)>; @@ -71,13 +71,13 @@ namespace NActors { bool orange, bool red, TActorSystem* actorSystem)>; struct TInterconnectProxyCommon : TAtomicRefCount<TInterconnectProxyCommon> { - TActorId NameserviceId; + TActorId NameserviceId; NMonitoring::TDynamicCounterPtr MonCounters; std::shared_ptr<NMonitoring::IMetricRegistry> Metrics; TChannelsConfig ChannelsConfig; TInterconnectSettings Settings; TRegisterMonPageCallback RegisterMonPage; - TActorId DestructorId; + TActorId DestructorId; std::shared_ptr<std::atomic<TAtomicBase>> DestructorQueueSize; TAtomicBase MaxDestructorQueueSize = 1024 * 1024 * 1024; TString ClusterUUID; diff --git a/library/cpp/actors/interconnect/interconnect_counters.cpp b/library/cpp/actors/interconnect/interconnect_counters.cpp index e389e936885..224160d4b4f 100644 --- a/library/cpp/actors/interconnect/interconnect_counters.cpp +++ b/library/cpp/actors/interconnect/interconnect_counters.cpp @@ -619,11 +619,11 @@ namespace { TotalBytesRead_ = createRate(Metrics_, "interconnect.total_bytes_read"); for (const char *reason : TDisconnectReason::Reasons) { - DisconnectByReason_[reason] = Metrics_->Rate( - NMonitoring::MakeLabels({ - {"sensor", "interconnect.disconnect_reason"}, - {"reason", reason}, - })); + DisconnectByReason_[reason] = Metrics_->Rate( + NMonitoring::MakeLabels({ + {"sensor", "interconnect.disconnect_reason"}, + {"reason", reason}, + })); } } diff --git a/library/cpp/actors/interconnect/interconnect_handshake.cpp b/library/cpp/actors/interconnect/interconnect_handshake.cpp index 51d1e607bce..9ede998d8e7 100644 --- a/library/cpp/actors/interconnect/interconnect_handshake.cpp +++ b/library/cpp/actors/interconnect/interconnect_handshake.cpp @@ -25,8 +25,8 @@ namespace NActors { struct TInitialPacket { struct { - TActorId SelfVirtualId; - TActorId PeerVirtualId; + TActorId SelfVirtualId; + TActorId PeerVirtualId; ui64 NextPacket; ui64 Version; } Header; @@ -34,7 +34,7 @@ namespace NActors { TInitialPacket() = default; - TInitialPacket(const TActorId& self, const TActorId& peer, ui64 nextPacket, ui64 version) { + TInitialPacket(const TActorId& self, const TActorId& peer, ui64 nextPacket, ui64 version) { Header.SelfVirtualId = self; Header.PeerVirtualId = peer; Header.NextPacket = nextPacket; @@ -79,8 +79,8 @@ namespace NActors { private: TInterconnectProxyCommon::TPtr Common; - TActorId SelfVirtualId; - TActorId PeerVirtualId; + TActorId SelfVirtualId; + TActorId PeerVirtualId; ui32 PeerNodeId = 0; ui64 NextPacketToPeer = 0; TMaybe<ui64> NextPacketFromPeer; // will be obtained from incoming initial packet @@ -102,7 +102,7 @@ namespace NActors { return IActor::INTERCONNECT_HANDSHAKE; } - THandshakeActor(TInterconnectProxyCommon::TPtr common, const TActorId& self, const TActorId& peer, + THandshakeActor(TInterconnectProxyCommon::TPtr common, const TActorId& self, const TActorId& peer, ui32 nodeId, ui64 nextPacket, TString peerHostName, TSessionParams params) : TActorCoroImpl(StackSize, true, true) // allow unhandled poison pills and dtors , Common(std::move(common)) @@ -377,7 +377,7 @@ namespace NActors { // set up virtual self id to ensure peer will not drop our connection char buf[12] = {'c', 'o', 'o', 'k', 'i', 'e', ' ', 'c', 'h', 'e', 'c', 'k'}; - SelfVirtualId = TActorId(SelfActorId.NodeId(), TStringBuf(buf, 12)); + SelfVirtualId = TActorId(SelfActorId.NodeId(), TStringBuf(buf, 12)); bool success = true; try { @@ -401,7 +401,7 @@ namespace NActors { request.SetProgramStartTime(0); request.SetSerial(0); request.SetReceiverNodeId(0); - request.SetSenderActorId(TString()); + request.SetSenderActorId(TString()); request.SetCookie(cookie); request.SetDoCheckCookie(true); SendExBlock(request, "SendExBlockDoCheckCookie"); @@ -419,7 +419,7 @@ namespace NActors { } // restore state - SelfVirtualId = TActorId(); + SelfVirtualId = TActorId(); std::swap(tempSocket, Socket); std::swap(tempPollerToken, PollerToken); return success; @@ -455,7 +455,7 @@ namespace NActors { request.SetProgramStartTime(Common->StartTime); request.SetSerial(SelfVirtualId.LocalId()); request.SetReceiverNodeId(PeerNodeId); - request.SetSenderActorId(SelfVirtualId.ToString()); + request.SetSenderActorId(SelfVirtualId.ToString()); request.SetSenderHostName(Common->TechnicalSelfHostName); request.SetReceiverHostName(PeerHostName); @@ -519,7 +519,7 @@ namespace NActors { ValidateClusterUUID(success, generateError); ValidateVersionTag(success, generateError); - const auto& s = success.GetSenderActorId(); + const auto& s = success.GetSenderActorId(); PeerVirtualId.Parse(s.data(), s.size()); // recover flags @@ -599,8 +599,8 @@ namespace NActors { SendInitialPacket(); } else { // peer wants a new session, clear fields and send initial packet - SelfVirtualId = TActorId(); - PeerVirtualId = TActorId(); + SelfVirtualId = TActorId(); + PeerVirtualId = TActorId(); NextPacketToPeer = 0; SendInitialPacket(); @@ -637,7 +637,7 @@ namespace NActors { PeerHostName = request.GetSenderHostName(); // parse peer virtual id - const auto& str = request.GetSenderActorId(); + const auto& str = request.GetSenderActorId(); PeerVirtualId.Parse(str.data(), str.size()); // validate request @@ -709,7 +709,7 @@ namespace NActors { SendExBlock(record, "ExReply"); // extract sender actor id (self virtual id) - const auto& str = success.GetSenderActorId(); + const auto& str = success.GetSenderActorId(); SelfVirtualId.Parse(str.data(), str.size()); } else if (auto ev = reply->CastAsLocal<TEvHandshakeReplyError>()) { // in case of error just send reply to the peer and terminate handshake @@ -981,8 +981,8 @@ namespace NActors { } }; - IActor* CreateOutgoingHandshakeActor(TInterconnectProxyCommon::TPtr common, const TActorId& self, - const TActorId& peer, ui32 nodeId, ui64 nextPacket, TString peerHostName, + IActor* CreateOutgoingHandshakeActor(TInterconnectProxyCommon::TPtr common, const TActorId& self, + const TActorId& peer, ui32 nodeId, ui64 nextPacket, TString peerHostName, TSessionParams params) { return new TActorCoro(MakeHolder<THandshakeActor>(std::move(common), self, peer, nodeId, nextPacket, std::move(peerHostName), std::move(params))); diff --git a/library/cpp/actors/interconnect/interconnect_handshake.h b/library/cpp/actors/interconnect/interconnect_handshake.h index 7c5c25c3b85..b3c0db6c5db 100644 --- a/library/cpp/actors/interconnect/interconnect_handshake.h +++ b/library/cpp/actors/interconnect/interconnect_handshake.h @@ -15,8 +15,8 @@ namespace NActors { using TSocketPtr = TIntrusivePtr<NInterconnect::TStreamSocket>; - IActor* CreateOutgoingHandshakeActor(TInterconnectProxyCommon::TPtr common, const TActorId& self, - const TActorId& peer, ui32 nodeId, ui64 nextPacket, TString peerHostName, + IActor* CreateOutgoingHandshakeActor(TInterconnectProxyCommon::TPtr common, const TActorId& self, + const TActorId& peer, ui32 nodeId, ui64 nextPacket, TString peerHostName, TSessionParams params); IActor* CreateIncomingHandshakeActor(TInterconnectProxyCommon::TPtr common, TSocketPtr socket); diff --git a/library/cpp/actors/interconnect/interconnect_impl.h b/library/cpp/actors/interconnect/interconnect_impl.h index 2ca0db87636..ee29e4d3971 100644 --- a/library/cpp/actors/interconnect/interconnect_impl.h +++ b/library/cpp/actors/interconnect/interconnect_impl.h @@ -4,7 +4,7 @@ #include <library/cpp/actors/protos/interconnect.pb.h> #include <library/cpp/actors/core/event_pb.h> #include <library/cpp/actors/helpers/mon_histogram_helper.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> namespace NActors { // resolve node info diff --git a/library/cpp/actors/interconnect/interconnect_mon.cpp b/library/cpp/actors/interconnect/interconnect_mon.cpp index 48823c5b0ec..cf924ccbf9d 100644 --- a/library/cpp/actors/interconnect/interconnect_mon.cpp +++ b/library/cpp/actors/interconnect/interconnect_mon.cpp @@ -1,9 +1,9 @@ #include "interconnect_mon.h" #include "interconnect_tcp_proxy.h" - -#include <library/cpp/json/json_value.h> -#include <library/cpp/json/json_writer.h> -#include <library/cpp/monlib/service/pages/templates.h> + +#include <library/cpp/json/json_value.h> +#include <library/cpp/json/json_writer.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <openssl/ssl.h> #include <openssl/pem.h> @@ -14,7 +14,7 @@ namespace NInterconnect { class TInterconnectMonActor : public TActor<TInterconnectMonActor> { class TQueryProcessor : public TActorBootstrapped<TQueryProcessor> { - const TActorId Sender; + const TActorId Sender; const bool Json; TMap<ui32, TInterconnectProxyTCP::TProxyStats> Stats; ui32 PendingReplies = 0; @@ -24,7 +24,7 @@ namespace NInterconnect { return INTERCONNECT_MONACTOR; } - TQueryProcessor(const TActorId& sender, bool json) + TQueryProcessor(const TActorId& sender, bool json) : Sender(sender) , Json(json) {} diff --git a/library/cpp/actors/interconnect/interconnect_mon.h b/library/cpp/actors/interconnect/interconnect_mon.h index e78229a2c42..3fb26053fbe 100644 --- a/library/cpp/actors/interconnect/interconnect_mon.h +++ b/library/cpp/actors/interconnect/interconnect_mon.h @@ -7,9 +7,9 @@ namespace NInterconnect { NActors::IActor *CreateInterconnectMonActor(TIntrusivePtr<NActors::TInterconnectProxyCommon> common = nullptr); - static inline NActors::TActorId MakeInterconnectMonActorId(ui32 nodeId) { + static inline NActors::TActorId MakeInterconnectMonActorId(ui32 nodeId) { char s[12] = {'I', 'C', 'O', 'v', 'e', 'r', 'v', 'i', 'e', 'w', 0, 0}; - return NActors::TActorId(nodeId, TStringBuf(s, 12)); + return NActors::TActorId(nodeId, TStringBuf(s, 12)); } } // NInterconnect diff --git a/library/cpp/actors/interconnect/interconnect_nameserver_table.cpp b/library/cpp/actors/interconnect/interconnect_nameserver_table.cpp index c9f6f8b5dcb..43419bf70d3 100644 --- a/library/cpp/actors/interconnect/interconnect_nameserver_table.cpp +++ b/library/cpp/actors/interconnect/interconnect_nameserver_table.cpp @@ -79,8 +79,8 @@ namespace NActors { return true; } - TActorId GetNameserviceActorId() { - return TActorId(0, "namesvc"); + TActorId GetNameserviceActorId() { + return TActorId(0, "namesvc"); } } diff --git a/library/cpp/actors/interconnect/interconnect_tcp_input_session.cpp b/library/cpp/actors/interconnect/interconnect_tcp_input_session.cpp index b42ae8dffd2..0abe9fe6598 100644 --- a/library/cpp/actors/interconnect/interconnect_tcp_input_session.cpp +++ b/library/cpp/actors/interconnect/interconnect_tcp_input_session.cpp @@ -6,7 +6,7 @@ namespace NActors { LWTRACE_USING(ACTORLIB_PROVIDER); - TInputSessionTCP::TInputSessionTCP(const TActorId& sessionId, TIntrusivePtr<NInterconnect::TStreamSocket> socket, + TInputSessionTCP::TInputSessionTCP(const TActorId& sessionId, TIntrusivePtr<NInterconnect::TStreamSocket> socket, TIntrusivePtr<TReceiveContext> context, TInterconnectProxyCommon::TPtr common, std::shared_ptr<IInterconnectMetrics> metrics, ui32 nodeId, ui64 lastConfirmed, TDuration deadPeerTimeout, TSessionParams params) diff --git a/library/cpp/actors/interconnect/interconnect_tcp_proxy.cpp b/library/cpp/actors/interconnect/interconnect_tcp_proxy.cpp index 4191951abdb..7e2d8ccb948 100644 --- a/library/cpp/actors/interconnect/interconnect_tcp_proxy.cpp +++ b/library/cpp/actors/interconnect/interconnect_tcp_proxy.cpp @@ -3,7 +3,7 @@ #include "interconnect_tcp_session.h" #include <library/cpp/actors/core/log.h> #include <library/cpp/actors/protos/services_common.pb.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/system/getpid.h> namespace NActors { @@ -45,7 +45,7 @@ namespace NActors { LOG_INFO_IC("ICP01", "ready to work"); } - void TInterconnectProxyTCP::Registered(TActorSystem* sys, const TActorId& owner) { + void TInterconnectProxyTCP::Registered(TActorSystem* sys, const TActorId& owner) { if (!DynamicPtr) { // perform usual bootstrap for static nodes sys->Send(new IEventHandle(TEvents::TSystem::Bootstrap, 0, SelfId(), owner, nullptr, 0)); @@ -311,9 +311,9 @@ namespace NActors { auto event = MakeHolder<TEvHandshakeReplyOK>(); auto* pb = event->Record.MutableSuccess(); - const TActorId virtualId = GenerateSessionVirtualId(); + const TActorId virtualId = GenerateSessionVirtualId(); pb->SetProtocol(INTERCONNECT_PROTOCOL_VERSION); - pb->SetSenderActorId(virtualId.ToString()); + pb->SetSenderActorId(virtualId.ToString()); pb->SetProgramPID(GetPID()); pb->SetProgramStartTime(Common->StartTime); pb->SetSerial(virtualId.LocalId()); @@ -536,14 +536,14 @@ namespace NActors { SessionVirtualId.ToString().data()); Session = nullptr; - SessionID = TActorId(); + SessionID = TActorId(); // drop all pending events as we are closed ProcessPendingSessionEvents(); // reset virtual ids as this session is terminated - SessionVirtualId = TActorId(); - RemoteSessionVirtualId = TActorId(); + SessionVirtualId = TActorId(); + RemoteSessionVirtualId = TActorId(); if (Metrics) { Metrics->IncSessionDeaths(); diff --git a/library/cpp/actors/interconnect/interconnect_tcp_proxy.h b/library/cpp/actors/interconnect/interconnect_tcp_proxy.h index e5921134eda..023e5bd1eee 100644 --- a/library/cpp/actors/interconnect/interconnect_tcp_proxy.h +++ b/library/cpp/actors/interconnect/interconnect_tcp_proxy.h @@ -4,7 +4,7 @@ #include <library/cpp/actors/core/hfunc.h> #include <library/cpp/actors/core/event_pb.h> #include <library/cpp/actors/core/events.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include "interconnect_common.h" #include "interconnect_counters.h" @@ -70,7 +70,7 @@ namespace NActors { } void Bootstrap(); - void Registered(TActorSystem* sys, const TActorId& owner) override; + void Registered(TActorSystem* sys, const TActorId& owner) override; private: friend class TInterconnectSessionTCP; @@ -366,7 +366,7 @@ namespace NActors { // read only TInterconnectProxyCommon::TPtr const Common; - const TActorId& GetNameserviceId() const { + const TActorId& GetNameserviceId() const { return Common->NameserviceId; } @@ -403,24 +403,24 @@ namespace NActors { void DropSessionEvent(STATEFN_SIG); TInterconnectSessionTCP* Session = nullptr; - TActorId SessionID; + TActorId SessionID; // virtual ids used during handshake to check if it is the connection // for the same session or to find out the latest shandshake // it's virtual because session actor apears after successfull handshake - TActorId SessionVirtualId; - TActorId RemoteSessionVirtualId; + TActorId SessionVirtualId; + TActorId RemoteSessionVirtualId; - TActorId GenerateSessionVirtualId() { + TActorId GenerateSessionVirtualId() { ICPROXY_PROFILED; const ui64 localId = TlsActivationContext->ExecutorThread.ActorSystem->AllocateIDSpace(1); - return NActors::TActorId(SelfId().NodeId(), 0, localId, 0); + return NActors::TActorId(SelfId().NodeId(), 0, localId, 0); } //////////////////////////////////////////////////////////////////////////////////////////////////////////////////// - TActorId IncomingHandshakeActor; + TActorId IncomingHandshakeActor; TInstant IncomingHandshakeActorFilledIn; TInstant IncomingHandshakeActorReset; TMaybe<ui64> LastSerialFromIncomingHandshake; @@ -429,7 +429,7 @@ namespace NActors { void DropIncomingHandshake(bool poison = true) { ICPROXY_PROFILED; - if (const TActorId& actorId = std::exchange(IncomingHandshakeActor, TActorId())) { + if (const TActorId& actorId = std::exchange(IncomingHandshakeActor, TActorId())) { LOG_DEBUG_IC("ICP111", "dropped incoming handshake: %s poison: %s", actorId.ToString().data(), poison ? "true" : "false"); if (poison) { @@ -444,7 +444,7 @@ namespace NActors { void DropOutgoingHandshake(bool poison = true) { ICPROXY_PROFILED; - if (const TActorId& actorId = std::exchange(OutgoingHandshakeActor, TActorId())) { + if (const TActorId& actorId = std::exchange(OutgoingHandshakeActor, TActorId())) { LOG_DEBUG_IC("ICP112", "dropped outgoing handshake: %s poison: %s", actorId.ToString().data(), poison ? "true" : "false"); if (poison) { @@ -477,12 +477,12 @@ namespace NActors { SwitchToState(__LINE__, "PendingConnection", &TThis::PendingConnection); } - void IssueIncomingHandshakeReply(const TActorId& handshakeId, ui64 peerLocalId, + void IssueIncomingHandshakeReply(const TActorId& handshakeId, ui64 peerLocalId, THolder<IEventBase> event); //////////////////////////////////////////////////////////////////////////////////////////////////////////////////// - TActorId OutgoingHandshakeActor; + TActorId OutgoingHandshakeActor; TInstant OutgoingHandshakeActorCreated; TInstant OutgoingHandshakeActorReset; diff --git a/library/cpp/actors/interconnect/interconnect_tcp_server.cpp b/library/cpp/actors/interconnect/interconnect_tcp_server.cpp index 2c025dc3893..b95c994598d 100644 --- a/library/cpp/actors/interconnect/interconnect_tcp_server.cpp +++ b/library/cpp/actors/interconnect/interconnect_tcp_server.cpp @@ -23,7 +23,7 @@ namespace NActors { } } - TAutoPtr<IEventHandle> TInterconnectListenerTCP::AfterRegister(const TActorId& self, const TActorId& parentId) { + TAutoPtr<IEventHandle> TInterconnectListenerTCP::AfterRegister(const TActorId& self, const TActorId& parentId) { return new IEventHandle(self, parentId, new TEvents::TEvBootstrap, 0); } diff --git a/library/cpp/actors/interconnect/interconnect_tcp_server.h b/library/cpp/actors/interconnect/interconnect_tcp_server.h index 086fe26ab3e..fc71073c2df 100644 --- a/library/cpp/actors/interconnect/interconnect_tcp_server.h +++ b/library/cpp/actors/interconnect/interconnect_tcp_server.h @@ -34,7 +34,7 @@ namespace NActors { } } - TAutoPtr<IEventHandle> AfterRegister(const TActorId& self, const TActorId& parentId) override; + TAutoPtr<IEventHandle> AfterRegister(const TActorId& self, const TActorId& parentId) override; void Die(const TActorContext& ctx) override; @@ -50,8 +50,8 @@ namespace NActors { TInterconnectProxyCommon::TPtr const ProxyCommonCtx; }; - static inline TActorId MakeInterconnectListenerActorId(bool dynamic) { + static inline TActorId MakeInterconnectListenerActorId(bool dynamic) { char x[12] = {'I', 'C', 'L', 'i', 's', 't', 'e', 'n', 'e', 'r', '/', dynamic ? 'D' : 'S'}; - return TActorId(0, TStringBuf(x, 12)); + return TActorId(0, TStringBuf(x, 12)); } } diff --git a/library/cpp/actors/interconnect/interconnect_tcp_session.cpp b/library/cpp/actors/interconnect/interconnect_tcp_session.cpp index 468e8bdd644..2ded7f9f537 100644 --- a/library/cpp/actors/interconnect/interconnect_tcp_session.cpp +++ b/library/cpp/actors/interconnect/interconnect_tcp_session.cpp @@ -7,7 +7,7 @@ #include <library/cpp/actors/core/interconnect.h> #include <library/cpp/actors/util/datetime.h> #include <library/cpp/actors/protos/services_common.pb.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> namespace NActors { LWTRACE_USING(ACTORLIB_PROVIDER); @@ -474,7 +474,7 @@ namespace NActors { if (ev->Sender == ReceiverId) { const bool wasConnected(Socket); LOG_INFO_IC_SESSION("ICS07", "socket disconnect %" PRIi64 " reason# %s", Socket ? i64(*Socket) : -1, ev->Get()->Reason.ToString().data()); - ReceiverId = TActorId(); // reset receiver actor id as we have no more receiver yet + ReceiverId = TActorId(); // reset receiver actor id as we have no more receiver yet if (wasConnected) { // we were sucessfully connected and did not expect failure, so it arrived from the input side; we should // restart handshake process, closing our part of socket first diff --git a/library/cpp/actors/interconnect/interconnect_tcp_session.h b/library/cpp/actors/interconnect/interconnect_tcp_session.h index dfab4065c02..7fc00dbcc5a 100644 --- a/library/cpp/actors/interconnect/interconnect_tcp_session.h +++ b/library/cpp/actors/interconnect/interconnect_tcp_session.h @@ -10,7 +10,7 @@ #include <library/cpp/actors/util/rope.h> #include <library/cpp/actors/util/funnel_queue.h> #include <library/cpp/actors/util/recentwnd.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <library/cpp/actors/core/actor_bootstrapped.h> #include <util/generic/queue.h> @@ -179,7 +179,7 @@ namespace NActors { return INTERCONNECT_SESSION_TCP; } - TInputSessionTCP(const TActorId& sessionId, + TInputSessionTCP(const TActorId& sessionId, TIntrusivePtr<NInterconnect::TStreamSocket> socket, TIntrusivePtr<TReceiveContext> context, TInterconnectProxyCommon::TPtr common, @@ -495,7 +495,7 @@ namespace NActors { void GenerateHttpInfo(TStringStream& str); TIntrusivePtr<TReceiveContext> ReceiveContext; - TActorId ReceiverId; + TActorId ReceiverId; TDuration Ping; ui64 ConfirmPacketsForcedBySize = 0; @@ -513,7 +513,7 @@ namespace NActors { : public TActorBootstrapped<TInterconnectSessionKiller> { ui32 RepliesReceived = 0; ui32 RepliesNumber = 0; - TActorId LargestSession = TActorId(); + TActorId LargestSession = TActorId(); ui64 MaxBufferSize = 0; TInterconnectProxyCommon::TPtr Common; @@ -529,7 +529,7 @@ namespace NActors { void Bootstrap() { auto sender = SelfId(); - const auto eventFabric = [&sender](const TActorId& recp) -> IEventHandle* { + const auto eventFabric = [&sender](const TActorId& recp) -> IEventHandle* { auto ev = new TEvSessionBufferSizeRequest(); return new IEventHandle(recp, sender, ev, IEventHandle::FlagTrackDelivery); }; diff --git a/library/cpp/actors/interconnect/load.cpp b/library/cpp/actors/interconnect/load.cpp index 22850b31268..2a8443da71f 100644 --- a/library/cpp/actors/interconnect/load.cpp +++ b/library/cpp/actors/interconnect/load.cpp @@ -72,7 +72,7 @@ namespace NInterconnect { }; class TLoadResponderMasterActor : public TActorBootstrapped<TLoadResponderMasterActor> { - TVector<TActorId> Slaves; + TVector<TActorId> Slaves; ui32 SlaveIndex = 0; STRICT_STFUNC(StateFunc, @@ -93,7 +93,7 @@ namespace NInterconnect { } void Die(const TActorContext& ctx) override { - for (const TActorId& actorId : Slaves) { + for (const TActorId& actorId : Slaves) { ctx.Send(actorId, new TEvents::TEvPoisonPill); } TActorBootstrapped::Die(ctx); @@ -122,9 +122,9 @@ namespace NInterconnect { return new TLoadResponderMasterActor(); } - TActorId MakeLoadResponderActorId(ui32 nodeId) { + TActorId MakeLoadResponderActorId(ui32 nodeId) { char x[12] = {'I', 'C', 'L', 'o', 'a', 'd', 'R', 'e', 's', 'p', 'A', 'c'}; - return TActorId(nodeId, TStringBuf(x, 12)); + return TActorId(nodeId, TStringBuf(x, 12)); } class TLoadActor: public TActorBootstrapped<TLoadActor> { @@ -144,8 +144,8 @@ namespace NInterconnect { TInstant NextMessageTimestamp; THashMap<TString, TMessageInfo> InFly; ui64 NextId = 1; - TVector<TActorId> Hops; - TActorId FirstHop; + TVector<TActorId> Hops; + TActorId FirstHop; ui64 NumDropped = 0; std::shared_ptr<std::atomic_uint64_t> Traffic; @@ -167,7 +167,7 @@ namespace NInterconnect { Traffic = std::move(ev->Get()->Traffic); for (const ui32 nodeId : Params.NodeHops) { - const TActorId& actorId = nodeId ? MakeLoadResponderActorId(nodeId) : TActorId(); + const TActorId& actorId = nodeId ? MakeLoadResponderActorId(nodeId) : TActorId(); if (!FirstHop) { FirstHop = actorId; } else { diff --git a/library/cpp/actors/interconnect/load.h b/library/cpp/actors/interconnect/load.h index 060fa7641be..0a01a0dc048 100644 --- a/library/cpp/actors/interconnect/load.h +++ b/library/cpp/actors/interconnect/load.h @@ -5,7 +5,7 @@ namespace NInterconnect { // load responder -- lives on every node as a service actor NActors::IActor* CreateLoadResponderActor(); - NActors::TActorId MakeLoadResponderActorId(ui32 node); + NActors::TActorId MakeLoadResponderActorId(ui32 node); // load actor -- generates load with specific parameters struct TLoadParams { diff --git a/library/cpp/actors/interconnect/mock/ic_mock.cpp b/library/cpp/actors/interconnect/mock/ic_mock.cpp index 1267920559d..884503e6022 100644 --- a/library/cpp/actors/interconnect/mock/ic_mock.cpp +++ b/library/cpp/actors/interconnect/mock/ic_mock.cpp @@ -42,7 +42,7 @@ namespace NActors { : Key(key) {} - void Attach(ui32 nodeId, TActorSystem *as, const TActorId& actorId) { + void Attach(ui32 nodeId, TActorSystem *as, const TActorId& actorId) { TPeerInfo *peer = GetPeer(nodeId); auto guard = TWriteGuard(peer->Mutex); Y_VERIFY(!peer->ActorSystem); @@ -188,7 +188,7 @@ namespace NActors { , Common(std::move(common)) {} - void Registered(TActorSystem *as, const TActorId& parent) override { + void Registered(TActorSystem *as, const TActorId& parent) override { TActor::Registered(as, parent); State.Attach(NodeId, as, SelfId()); } diff --git a/library/cpp/actors/interconnect/packet.h b/library/cpp/actors/interconnect/packet.h index 187d0b6bdfb..4ba50a2b5f4 100644 --- a/library/cpp/actors/interconnect/packet.h +++ b/library/cpp/actors/interconnect/packet.h @@ -18,7 +18,7 @@ using NActors::IEventBase; using NActors::IEventHandle; -using NActors::TActorId; +using NActors::TActorId; using NActors::TConstIoVec; using NActors::TEventSerializedData; @@ -91,8 +91,8 @@ union TTcpPacketBuf { struct TEventDescr { ui32 Type; ui32 Flags; - TActorId Recipient; - TActorId Sender; + TActorId Recipient; + TActorId Sender; ui64 Cookie; // wilson trace id is stored as a serialized entity to avoid using complex object with prohibited copy ctor NWilson::TTraceId::TSerializedTraceId TraceId; @@ -102,7 +102,7 @@ struct TEventDescr { struct TEventHolder : TNonCopyable { TEventDescr Descr; - TActorId ForwardRecipient; + TActorId ForwardRecipient; THolder<IEventBase> Event; TIntrusivePtr<TEventSerializedData> Buffer; ui64 Serial; diff --git a/library/cpp/actors/interconnect/poller_actor.cpp b/library/cpp/actors/interconnect/poller_actor.cpp index 8c7b61a7a7f..e75cbcaef43 100644 --- a/library/cpp/actors/interconnect/poller_actor.cpp +++ b/library/cpp/actors/interconnect/poller_actor.cpp @@ -1,35 +1,35 @@ #include "poller_actor.h" #include "interconnect_common.h" -#include <library/cpp/actors/core/actor_bootstrapped.h> +#include <library/cpp/actors/core/actor_bootstrapped.h> #include <library/cpp/actors/core/actorsystem.h> #include <library/cpp/actors/core/hfunc.h> #include <library/cpp/actors/core/log.h> #include <library/cpp/actors/core/probes.h> -#include <library/cpp/actors/protos/services_common.pb.h> +#include <library/cpp/actors/protos/services_common.pb.h> #include <library/cpp/actors/util/funnel_queue.h> - + #include <util/generic/intrlist.h> #include <util/system/thread.h> #include <util/system/event.h> #include <util/system/pipe.h> #include <variant> - + namespace NActors { LWTRACE_USING(ACTORLIB_PROVIDER); - namespace { + namespace { int LastSocketError() { #if defined(_win_) return WSAGetLastError(); #else return errno; #endif - } - } - + } + } + struct TSocketRecord : TThrRefBase { const TIntrusivePtr<TSharedDescriptor> Socket; const TActorId ReadActorId; @@ -57,7 +57,7 @@ namespace NActors { : Socket(std::move(socket)) {} }; - + using TPollerSyncOperation = std::variant<TPollerExitThread, TPollerWakeup, TPollerUnregisterSocket>; struct TPollerSyncOperationWrapper { @@ -149,7 +149,7 @@ namespace NActors { bool DrainReadEnd() { size_t totalRead = 0; char buffer[4096]; - for (;;) { + for (;;) { ssize_t n = ReadEnd.Read(buffer, sizeof(buffer)); if (n < 0) { const int error = LastSocketError(); @@ -157,17 +157,17 @@ namespace NActors { continue; } else if (error == EAGAIN || error == EWOULDBLOCK) { break; - } else { + } else { Y_FAIL("read() failed with %s", strerror(errno)); - } + } } else { Y_VERIFY(n); totalRead += n; - } + } } return totalRead; } - + bool ProcessSyncOpQueue() { if (DrainReadEnd()) { Y_VERIFY(!SyncOperationsQ.IsEmpty()); @@ -181,25 +181,25 @@ namespace NActors { return false; // terminate the thread } else if (std::get_if<TPollerWakeup>(&op->Operation)) { op->SignalDone(); - } else { + } else { Y_FAIL(); - } + } } while (SyncOperationsQ.Pop()); - } + } return true; - } - + } + void *ThreadProc() override { SetCurrentThreadName("network poller"); while (ProcessSyncOpQueue()) { static_cast<TDerived&>(*this).ProcessEventsInLoop(); - } + } return nullptr; - } + } }; - + } // namespace NActors - + #if defined(_linux_) # include "poller_actor_linux.h" #elif defined(_darwin_) @@ -209,38 +209,38 @@ namespace NActors { #else # error "Unsupported platform" #endif - + namespace NActors { - + class TPollerToken::TImpl { std::weak_ptr<TPollerThread> Thread; TIntrusivePtr<TSocketRecord> Record; // valid only when Thread is held locked - - public: + + public: TImpl(std::shared_ptr<TPollerThread> thread, TIntrusivePtr<TSocketRecord> record) : Thread(thread) , Record(std::move(record)) - { + { thread->RegisterSocket(Record); } - + ~TImpl() { if (auto thread = Thread.lock()) { thread->UnregisterSocket(Record); - } - } - + } + } + void Request(bool read, bool write) { if (auto thread = Thread.lock()) { thread->Request(Record, read, write); - } - } + } + } const TIntrusivePtr<TSharedDescriptor>& Socket() const { return Record->Socket; } - }; - + }; + class TPollerActor: public TActorBootstrapped<TPollerActor> { // poller thread std::shared_ptr<TPollerThread> PollerThread; diff --git a/library/cpp/actors/interconnect/poller_actor.h b/library/cpp/actors/interconnect/poller_actor.h index 5bd4f507049..f927b820893 100644 --- a/library/cpp/actors/interconnect/poller_actor.h +++ b/library/cpp/actors/interconnect/poller_actor.h @@ -55,9 +55,9 @@ namespace NActors { IActor* CreatePollerActor(); - inline TActorId MakePollerActorId() { + inline TActorId MakePollerActorId() { char x[12] = {'I', 'C', 'P', 'o', 'l', 'l', 'e', 'r', '\xDE', '\xAD', '\xBE', '\xEF'}; - return TActorId(0, TStringBuf(std::begin(x), std::end(x))); + return TActorId(0, TStringBuf(std::begin(x), std::end(x))); } } diff --git a/library/cpp/actors/interconnect/ut/channel_scheduler_ut.cpp b/library/cpp/actors/interconnect/ut/channel_scheduler_ut.cpp index bbdabbd3392..565a511859e 100644 --- a/library/cpp/actors/interconnect/ut/channel_scheduler_ut.cpp +++ b/library/cpp/actors/interconnect/ut/channel_scheduler_ut.cpp @@ -20,7 +20,7 @@ Y_UNIT_TEST_SUITE(ChannelScheduler) { auto pushEvent = [&](size_t size, int channel) { TString payload(size, 'X'); - auto ev = MakeHolder<IEventHandle>(1, 0, TActorId(), TActorId(), MakeIntrusive<TEventSerializedData>(payload, false), 0); + auto ev = MakeHolder<IEventHandle>(1, 0, TActorId(), TActorId(), MakeIntrusive<TEventSerializedData>(payload, false), 0); auto& ch = scheduler.GetOutputChannel(channel); const bool wasWorking = ch.IsWorking(); ch.Push(*ev); diff --git a/library/cpp/actors/interconnect/ut/event_holder_pool_ut.cpp b/library/cpp/actors/interconnect/ut/event_holder_pool_ut.cpp index 334859882f3..e6b2bd4e4c4 100644 --- a/library/cpp/actors/interconnect/ut/event_holder_pool_ut.cpp +++ b/library/cpp/actors/interconnect/ut/event_holder_pool_ut.cpp @@ -2,7 +2,7 @@ #include <library/cpp/actors/core/events.h> #include <library/cpp/actors/core/event_local.h> #include <library/cpp/actors/interconnect/interconnect_common.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <library/cpp/actors/interconnect/event_holder_pool.h> #include <atomic> diff --git a/library/cpp/actors/interconnect/ut/large.cpp b/library/cpp/actors/interconnect/ut/large.cpp index d67509f0589..ba2a50c6f6f 100644 --- a/library/cpp/actors/interconnect/ut/large.cpp +++ b/library/cpp/actors/interconnect/ut/large.cpp @@ -14,10 +14,10 @@ Y_UNIT_TEST_SUITE(LargeMessage) { using namespace NActors; class TProducer: public TActorBootstrapped<TProducer> { - const TActorId RecipientActorId; + const TActorId RecipientActorId; public: - TProducer(const TActorId& recipientActorId) + TProducer(const TActorId& recipientActorId) : RecipientActorId(recipientActorId) {} @@ -41,7 +41,7 @@ Y_UNIT_TEST_SUITE(LargeMessage) { class TConsumer : public TActorBootstrapped<TConsumer> { TManualEvent& Done; - TActorId SessionId; + TActorId SessionId; public: TConsumer(TManualEvent& done) @@ -77,7 +77,7 @@ Y_UNIT_TEST_SUITE(LargeMessage) { TManualEvent done; TConsumer* consumer = new TConsumer(done); - const TActorId recp = testCluster.RegisterActor(consumer, 1); + const TActorId recp = testCluster.RegisterActor(consumer, 1); testCluster.RegisterActor(new TProducer(recp), 2); done.WaitI(); } diff --git a/library/cpp/actors/interconnect/ut/lib/ic_test_cluster.h b/library/cpp/actors/interconnect/ut/lib/ic_test_cluster.h index ac46180804d..2b6d27cd3f3 100644 --- a/library/cpp/actors/interconnect/ut/lib/ic_test_cluster.h +++ b/library/cpp/actors/interconnect/ut/lib/ic_test_cluster.h @@ -70,7 +70,7 @@ public: ~TTestICCluster() { } - TActorId RegisterActor(NActors::IActor* actor, ui32 nodeId) { + TActorId RegisterActor(NActors::IActor* actor, ui32 nodeId) { return Nodes[nodeId]->RegisterActor(actor); } @@ -78,7 +78,7 @@ public: return Nodes[nodeId]->InterconnectProxy(peerNodeId); } - void KillActor(ui32 nodeId, const TActorId& id) { + void KillActor(ui32 nodeId, const TActorId& id) { Nodes[nodeId]->Send(id, new NActors::TEvents::TEvPoisonPill); } }; diff --git a/library/cpp/actors/interconnect/ut/lib/node.h b/library/cpp/actors/interconnect/ut/lib/node.h index 59dd2554c8b..ff30b1445e8 100644 --- a/library/cpp/actors/interconnect/ut/lib/node.h +++ b/library/cpp/actors/interconnect/ut/lib/node.h @@ -62,7 +62,7 @@ public: setup.LocalServices.emplace_back(MakePollerActorId(), TActorSetupCmd(CreatePollerActor(), TMailboxType::ReadAsFilled, 0)); - const TActorId loggerActorId(0, "logger"); + const TActorId loggerActorId(0, "logger"); constexpr ui32 LoggerComponentId = 410; // NKikimrServices::LOGGER auto loggerSettings = MakeIntrusive<NLog::TSettings>( @@ -114,7 +114,7 @@ public: ActorSystem->Stop(); } - bool Send(const TActorId& recipient, IEventBase* ev) { + bool Send(const TActorId& recipient, IEventBase* ev) { return ActorSystem->Send(recipient, ev); } @@ -127,7 +127,7 @@ public: } void RegisterServiceActor(const TActorId& serviceId, IActor* actor) { - const TActorId actorId = ActorSystem->Register(actor); + const TActorId actorId = ActorSystem->Register(actor); ActorSystem->RegisterLocalService(serviceId, actorId); } diff --git a/library/cpp/actors/interconnect/ut/lib/test_actors.h b/library/cpp/actors/interconnect/ut/lib/test_actors.h index 07fe10d93ad..7591200471b 100644 --- a/library/cpp/actors/interconnect/ut/lib/test_actors.h +++ b/library/cpp/actors/interconnect/ut/lib/test_actors.h @@ -3,13 +3,13 @@ namespace NActors { class TSenderBaseActor: public TActorBootstrapped<TSenderBaseActor> { protected: - const TActorId RecipientActorId; + const TActorId RecipientActorId; const ui32 Preload; ui64 SequenceNumber = 0; ui32 InFlySize = 0; public: - TSenderBaseActor(const TActorId& recipientActorId, ui32 preload = 1) + TSenderBaseActor(const TActorId& recipientActorId, ui32 preload = 1) : RecipientActorId(recipientActorId) , Preload(preload) { diff --git a/library/cpp/actors/interconnect/ut/poller_actor_ut.cpp b/library/cpp/actors/interconnect/ut/poller_actor_ut.cpp index dbd05ce746a..23d846a2fd8 100644 --- a/library/cpp/actors/interconnect/ut/poller_actor_ut.cpp +++ b/library/cpp/actors/interconnect/ut/poller_actor_ut.cpp @@ -1,38 +1,38 @@ -#include <library/cpp/actors/interconnect/poller_actor.h> -#include <library/cpp/actors/testlib/test_runtime.h> - +#include <library/cpp/actors/interconnect/poller_actor.h> +#include <library/cpp/actors/testlib/test_runtime.h> + #include <library/cpp/testing/unittest/registar.h> - -#include <util/network/pair.h> -#include <util/network/socket.h> - -using namespace NActors; - -class TTestSocket: public TSharedDescriptor { -public: - explicit TTestSocket(SOCKET fd) - : Fd_(fd) - { - } - - int GetDescriptor() override { - return Fd_; - } - -private: - SOCKET Fd_; -}; -using TTestSocketPtr = TIntrusivePtr<TTestSocket>; - -// create pair of connected, non-blocking sockets -std::pair<TTestSocketPtr, TTestSocketPtr> NonBlockSockets() { - SOCKET fds[2]; - SocketPair(fds); - SetNonBlock(fds[0]); - SetNonBlock(fds[1]); - return {MakeIntrusive<TTestSocket>(fds[0]), MakeIntrusive<TTestSocket>(fds[1])}; -} - + +#include <util/network/pair.h> +#include <util/network/socket.h> + +using namespace NActors; + +class TTestSocket: public TSharedDescriptor { +public: + explicit TTestSocket(SOCKET fd) + : Fd_(fd) + { + } + + int GetDescriptor() override { + return Fd_; + } + +private: + SOCKET Fd_; +}; +using TTestSocketPtr = TIntrusivePtr<TTestSocket>; + +// create pair of connected, non-blocking sockets +std::pair<TTestSocketPtr, TTestSocketPtr> NonBlockSockets() { + SOCKET fds[2]; + SocketPair(fds); + SetNonBlock(fds[0]); + SetNonBlock(fds[1]); + return {MakeIntrusive<TTestSocket>(fds[0]), MakeIntrusive<TTestSocket>(fds[1])}; +} + std::pair<TTestSocketPtr, TTestSocketPtr> TcpSockets() { // create server (listening) socket SOCKET server = socket(AF_INET, SOCK_STREAM, 0); @@ -74,101 +74,101 @@ std::pair<TTestSocketPtr, TTestSocketPtr> TcpSockets() { return std::make_pair(MakeIntrusive<TTestSocket>(client), MakeIntrusive<TTestSocket>(accepted)); } -class TPollerActorTest: public TTestBase { - UNIT_TEST_SUITE(TPollerActorTest); - UNIT_TEST(Registration) - UNIT_TEST(ReadNotification) - UNIT_TEST(WriteNotification) - UNIT_TEST(HangupNotification) - UNIT_TEST_SUITE_END(); - -public: - void SetUp() override { - ActorSystem_ = MakeHolder<TTestActorRuntimeBase>(); - ActorSystem_->Initialize(); - - PollerId_ = ActorSystem_->Register(CreatePollerActor()); - - TDispatchOptions opts; - opts.FinalEvents.emplace_back(TEvents::TSystem::Bootstrap, 1); - ActorSystem_->DispatchEvents(opts); - } - - void Registration() { - auto [s1, s2] = NonBlockSockets(); - auto readerId = ActorSystem_->AllocateEdgeActor(); - auto writerId = ActorSystem_->AllocateEdgeActor(); - +class TPollerActorTest: public TTestBase { + UNIT_TEST_SUITE(TPollerActorTest); + UNIT_TEST(Registration) + UNIT_TEST(ReadNotification) + UNIT_TEST(WriteNotification) + UNIT_TEST(HangupNotification) + UNIT_TEST_SUITE_END(); + +public: + void SetUp() override { + ActorSystem_ = MakeHolder<TTestActorRuntimeBase>(); + ActorSystem_->Initialize(); + + PollerId_ = ActorSystem_->Register(CreatePollerActor()); + + TDispatchOptions opts; + opts.FinalEvents.emplace_back(TEvents::TSystem::Bootstrap, 1); + ActorSystem_->DispatchEvents(opts); + } + + void Registration() { + auto [s1, s2] = NonBlockSockets(); + auto readerId = ActorSystem_->AllocateEdgeActor(); + auto writerId = ActorSystem_->AllocateEdgeActor(); + RegisterSocket(s1, readerId, writerId); - - // reader should receive event after socket registration + + // reader should receive event after socket registration TPollerToken::TPtr token; - { + { auto ev = ActorSystem_->GrabEdgeEvent<TEvPollerRegisterResult>(readerId); token = ev->Get()->PollerToken; - } - - // writer should receive event after socket registration - { + } + + // writer should receive event after socket registration + { auto ev = ActorSystem_->GrabEdgeEvent<TEvPollerRegisterResult>(writerId); UNIT_ASSERT_EQUAL(token, ev->Get()->PollerToken); - } - } - - void ReadNotification() { - auto [r, w] = NonBlockSockets(); - auto clientId = ActorSystem_->AllocateEdgeActor(); + } + } + + void ReadNotification() { + auto [r, w] = NonBlockSockets(); + auto clientId = ActorSystem_->AllocateEdgeActor(); RegisterSocket(r, clientId, {}); - - // notification after registration + + // notification after registration TPollerToken::TPtr token; - { + { auto ev = ActorSystem_->GrabEdgeEvent<TEvPollerRegisterResult>(clientId); token = ev->Get()->PollerToken; - } - - char buf; - - // data not ready yet for read - UNIT_ASSERT(read(r->GetDescriptor(), &buf, sizeof(buf)) == -1); - UNIT_ASSERT(errno == EWOULDBLOCK); - + } + + char buf; + + // data not ready yet for read + UNIT_ASSERT(read(r->GetDescriptor(), &buf, sizeof(buf)) == -1); + UNIT_ASSERT(errno == EWOULDBLOCK); + // request read poll token->Request(true, false); - // write data - UNIT_ASSERT(write(w->GetDescriptor(), "x", 1) == 1); - - // notification after socket become readable - { + // write data + UNIT_ASSERT(write(w->GetDescriptor(), "x", 1) == 1); + + // notification after socket become readable + { auto ev = ActorSystem_->GrabEdgeEvent<TEvPollerReady>(clientId); UNIT_ASSERT_EQUAL(ev->Get()->Socket, r); UNIT_ASSERT(ev->Get()->Read); UNIT_ASSERT(!ev->Get()->Write); - } - - // read data - UNIT_ASSERT(read(r->GetDescriptor(), &buf, sizeof(buf)) == 1); - UNIT_ASSERT_EQUAL('x', buf); - - // no more data to read - UNIT_ASSERT(read(r->GetDescriptor(), &buf, sizeof(buf)) == -1); - UNIT_ASSERT(errno == EWOULDBLOCK); - } - - void WriteNotification() { + } + + // read data + UNIT_ASSERT(read(r->GetDescriptor(), &buf, sizeof(buf)) == 1); + UNIT_ASSERT_EQUAL('x', buf); + + // no more data to read + UNIT_ASSERT(read(r->GetDescriptor(), &buf, sizeof(buf)) == -1); + UNIT_ASSERT(errno == EWOULDBLOCK); + } + + void WriteNotification() { auto [r, w] = TcpSockets(); - auto clientId = ActorSystem_->AllocateEdgeActor(); + auto clientId = ActorSystem_->AllocateEdgeActor(); SetNonBlock(w->GetDescriptor()); RegisterSocket(w, TActorId{}, clientId); - - // notification after registration + + // notification after registration TPollerToken::TPtr token; { auto ev = ActorSystem_->GrabEdgeEvent<TEvPollerRegisterResult>(clientId); token = ev->Get()->PollerToken; - } - + } + char buffer[4096]; memset(buffer, 'x', sizeof(buffer)); @@ -181,7 +181,7 @@ public: written += res; } else if (res == 0) { UNIT_FAIL("unexpected zero return from send()"); - } else { + } else { UNIT_ASSERT(res == -1); if (errno == EINTR) { continue; @@ -191,10 +191,10 @@ public: } else { UNIT_FAIL("unexpected error from send()"); } - } - } + } + } Cerr << "written " << written << " bytes" << Endl; - + // read all written data from the read end for (;;) { char buffer[4096]; @@ -216,7 +216,7 @@ public: } } } - + // wait for notification after socket becomes writable again { auto ev = ActorSystem_->GrabEdgeEvent<TEvPollerReady>(clientId); @@ -224,41 +224,41 @@ public: UNIT_ASSERT(!ev->Get()->Read); UNIT_ASSERT(ev->Get()->Write); } - } - } - - void HangupNotification() { - auto [r, w] = NonBlockSockets(); - auto clientId = ActorSystem_->AllocateEdgeActor(); + } + } + + void HangupNotification() { + auto [r, w] = NonBlockSockets(); + auto clientId = ActorSystem_->AllocateEdgeActor(); RegisterSocket(r, clientId, TActorId{}); - - // notification after registration + + // notification after registration TPollerToken::TPtr token; - { + { auto ev = ActorSystem_->GrabEdgeEvent<TEvPollerRegisterResult>(clientId); token = ev->Get()->PollerToken; - } - + } + token->Request(true, false); ShutDown(w->GetDescriptor(), SHUT_RDWR); - + // notification after peer shuts down its socket - { + { auto ev = ActorSystem_->GrabEdgeEvent<TEvPollerReady>(clientId); UNIT_ASSERT_EQUAL(ev->Get()->Socket, r); UNIT_ASSERT(ev->Get()->Read); - } - } - -private: + } + } + +private: void RegisterSocket(TTestSocketPtr socket, TActorId readActorId, TActorId writeActorId) { auto ev = new TEvPollerRegister{socket, readActorId, writeActorId}; - ActorSystem_->Send(new IEventHandle(PollerId_, TActorId{}, ev)); - } - -private: - THolder<TTestActorRuntimeBase> ActorSystem_; - TActorId PollerId_; -}; - -UNIT_TEST_SUITE_REGISTRATION(TPollerActorTest); + ActorSystem_->Send(new IEventHandle(PollerId_, TActorId{}, ev)); + } + +private: + THolder<TTestActorRuntimeBase> ActorSystem_; + TActorId PollerId_; +}; + +UNIT_TEST_SUITE_REGISTRATION(TPollerActorTest); diff --git a/library/cpp/actors/interconnect/ut/ya.make b/library/cpp/actors/interconnect/ut/ya.make index ec19f1a64a7..2f5b13352ef 100644 --- a/library/cpp/actors/interconnect/ut/ya.make +++ b/library/cpp/actors/interconnect/ut/ya.make @@ -15,11 +15,11 @@ ELSE() ENDIF() SRCS( - channel_scheduler_ut.cpp + channel_scheduler_ut.cpp event_holder_pool_ut.cpp interconnect_ut.cpp large.cpp - poller_actor_ut.cpp + poller_actor_ut.cpp dynamic_proxy_ut.cpp ) @@ -28,7 +28,7 @@ PEERDIR( library/cpp/actors/interconnect library/cpp/actors/interconnect/ut/lib library/cpp/actors/interconnect/ut/protos - library/cpp/actors/testlib + library/cpp/actors/testlib library/cpp/digest/md5 library/cpp/testing/unittest ) diff --git a/library/cpp/actors/interconnect/ut_fat/main.cpp b/library/cpp/actors/interconnect/ut_fat/main.cpp index 69374cd080d..5d19bc30030 100644 --- a/library/cpp/actors/interconnect/ut_fat/main.cpp +++ b/library/cpp/actors/interconnect/ut_fat/main.cpp @@ -23,7 +23,7 @@ Y_UNIT_TEST_SUITE(InterconnectUnstableConnection) { ui16 SendFlags; public: - TSenderActor(const TActorId& recipientActorId, ui16 sendFlags) + TSenderActor(const TActorId& recipientActorId, ui16 sendFlags) : TSenderBaseActor(recipientActorId, 32) , SendFlags(sendFlags) { @@ -108,7 +108,7 @@ Y_UNIT_TEST_SUITE(InterconnectUnstableConnection) { TTestICCluster testCluster(numNodes, TChannelsConfig(), &interrupterSettings); TReceiverActor* receiverActor = new TReceiverActor(testCluster.GetNode(1)); - const TActorId recipient = testCluster.RegisterActor(receiverActor, 2); + const TActorId recipient = testCluster.RegisterActor(receiverActor, 2); TSenderActor* senderActor = new TSenderActor(recipient, flags); testCluster.RegisterActor(senderActor, 1); @@ -124,7 +124,7 @@ Y_UNIT_TEST_SUITE(InterconnectUnstableConnection) { TTestICCluster testCluster(numNodes, TChannelsConfig(), &interrupterSettings); TReceiverActor* receiverActor = new TReceiverActor(testCluster.GetNode(1)); - const TActorId recipient = testCluster.RegisterActor(receiverActor, 2); + const TActorId recipient = testCluster.RegisterActor(receiverActor, 2); TSenderActor* senderActor = new TSenderActor(recipient, flags); testCluster.RegisterActor(senderActor, 1); diff --git a/library/cpp/actors/interconnect/ya.make b/library/cpp/actors/interconnect/ya.make index 9e4fb46fdbf..60d29b0fc0c 100644 --- a/library/cpp/actors/interconnect/ya.make +++ b/library/cpp/actors/interconnect/ya.make @@ -75,18 +75,18 @@ PEERDIR( contrib/libs/libc_compat contrib/libs/openssl library/cpp/actors/core - library/cpp/actors/dnscachelib + library/cpp/actors/dnscachelib library/cpp/actors/dnsresolver library/cpp/actors/helpers library/cpp/actors/prof library/cpp/actors/protos library/cpp/actors/util library/cpp/digest/crc32c - library/cpp/json + library/cpp/json library/cpp/lwtrace - library/cpp/monlib/dynamic_counters + library/cpp/monlib/dynamic_counters library/cpp/monlib/metrics - library/cpp/monlib/service/pages/tablesorter + library/cpp/monlib/service/pages/tablesorter library/cpp/openssl/init library/cpp/packedtypes ) diff --git a/library/cpp/actors/protos/actors.proto b/library/cpp/actors/protos/actors.proto index 8155535f1fb..5fbd6d44ee5 100644 --- a/library/cpp/actors/protos/actors.proto +++ b/library/cpp/actors/protos/actors.proto @@ -2,12 +2,12 @@ package NActorsProto; option java_package = "ru.yandex.kikimr.proto"; option java_outer_classname = "NActorsBaseProto"; -message TActorId { +message TActorId { required fixed64 RawX1 = 1; required fixed64 RawX2 = 2; } message TCallbackException { - required TActorId ActorId = 1; + required TActorId ActorId = 1; required string ExceptionMessage = 2; } diff --git a/library/cpp/actors/protos/interconnect.proto b/library/cpp/actors/protos/interconnect.proto index 8656c8ea1f3..2e3b0d0d15d 100644 --- a/library/cpp/actors/protos/interconnect.proto +++ b/library/cpp/actors/protos/interconnect.proto @@ -51,7 +51,7 @@ message THandshakeRequest { required uint64 Serial = 4; required uint32 ReceiverNodeId = 5; - required string SenderActorId = 6; + required string SenderActorId = 6; optional string SenderHostName = 7; optional string ReceiverHostName = 8; @@ -81,7 +81,7 @@ message THandshakeSuccess { required uint64 ProgramStartTime = 3; required uint64 Serial = 4; - required string SenderActorId = 5; + required string SenderActorId = 5; optional string VersionTag = 6; repeated string AcceptedVersionTags = 7; @@ -104,7 +104,7 @@ message THandshakeReply { message TEvLoadMessage { message THop { - optional NActorsProto.TActorId NextHop = 1; // if zero, then the payload is trimmed out of the message + optional NActorsProto.TActorId NextHop = 1; // if zero, then the payload is trimmed out of the message } repeated THop Hops = 1; // the route for the message diff --git a/library/cpp/actors/testlib/test_runtime.cpp b/library/cpp/actors/testlib/test_runtime.cpp index cda6980b1e2..6fa25b99656 100644 --- a/library/cpp/actors/testlib/test_runtime.cpp +++ b/library/cpp/actors/testlib/test_runtime.cpp @@ -295,17 +295,17 @@ namespace NActors { void Schedule(TInstant deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie *cookie, TWorkerId workerId) override { DoSchedule(deadline, ev, cookie, workerId); - } - + } + void Schedule(TMonotonic deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie *cookie, TWorkerId workerId) override { DoSchedule(TInstant::FromValue(deadline.GetValue()), ev, cookie, workerId); } void Schedule(TDuration delay, TAutoPtr<IEventHandle> ev, ISchedulerCookie *cookie, TWorkerId workerId) override { - TInstant deadline = Runtime->GetTimeProvider()->Now() + delay; + TInstant deadline = Runtime->GetTimeProvider()->Now() + delay; DoSchedule(deadline, ev, cookie, workerId); - } - + } + void DoSchedule(TInstant deadline, TAutoPtr<IEventHandle> ev, ISchedulerCookie *cookie, TWorkerId workerId) { Y_UNUSED(workerId); @@ -319,13 +319,13 @@ namespace NActors { Cerr << "Got scheduled event at " << TInstant::MicroSeconds(Runtime->CurrentTimestamp) << ", "; PrintEvent(ev, Runtime); } - - auto now = Runtime->GetTimeProvider()->Now(); - if (deadline < now) { - deadline = now; // avoid going backwards in time - } - TDuration delay = (deadline - now); - + + auto now = Runtime->GetTimeProvider()->Now(); + if (deadline < now) { + deadline = now; // avoid going backwards in time + } + TDuration delay = (deadline - now); + if (Runtime->SingleSysEnv || !Runtime->ScheduledEventFilterFunc(*Runtime, ev, delay, deadline)) { ui32 mailboxHint = ev->GetRecipientRewrite().Hint(); Runtime->GetMailbox(Runtime->FirstNodeId + NodeIndex, mailboxHint).Schedule(TScheduledEventQueueItem(deadline, ev, cookie)); @@ -336,9 +336,9 @@ namespace NActors { if (cookie) { cookie->Detach(); } - if (verbose) { + if (verbose) { Cerr << "Scheduled event for " << ev->GetRecipientRewrite().ToString() << " was dropped\n"; - } + } } } @@ -366,8 +366,8 @@ namespace NActors { ui32 mailboxHint = ev->GetRecipientRewrite().Hint(); if (ev->GetTypeRewrite() == ui32(NActors::NLog::EEv::Log)) { - const NActors::TActorId loggerActorId = NActors::TActorId(nodeId, "logger"); - TActorId logger = node->ActorSystem->LookupLocalService(loggerActorId); + const NActors::TActorId loggerActorId = NActors::TActorId(nodeId, "logger"); + TActorId logger = node->ActorSystem->LookupLocalService(loggerActorId); if (ev->GetRecipientRewrite() == logger) { TMailboxHeader* mailbox = node->MailboxTable->Get(mailboxHint); IActor* recipientActor = mailbox->FindActor(ev->GetRecipientRewrite().LocalId()); @@ -403,7 +403,7 @@ namespace NActors { } TActorId Register(IActor *actor, TMailboxType::EType mailboxType, ui64 revolvingCounter, - const TActorId& parentId) override { + const TActorId& parentId) override { return Runtime->Register(actor, NodeIndex, PoolId, mailboxType, revolvingCounter, parentId); } @@ -486,7 +486,7 @@ namespace NActors { } void TTestActorRuntimeBase::InitNode(TNodeDataBase* node, size_t nodeIndex) { - const NActors::TActorId loggerActorId = NActors::TActorId(FirstNodeId + nodeIndex, "logger"); + const NActors::TActorId loggerActorId = NActors::TActorId(FirstNodeId + nodeIndex, "logger"); node->LogSettings = new NActors::NLog::TSettings(loggerActorId, 410 /* NKikimrServices::LOGGER */, NActors::NLog::PRI_WARN, NActors::NLog::PRI_WARN, 0); node->LogSettings->SetAllowDrop(false); @@ -579,7 +579,7 @@ namespace NActors { } - void TTestActorRuntimeBase::DefaultRegistrationObserver(TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId) { + void TTestActorRuntimeBase::DefaultRegistrationObserver(TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId) { if (runtime.ScheduleWhiteList.find(parentId) != runtime.ScheduleWhiteList.end()) { runtime.ScheduleWhiteList.insert(actorId); runtime.ScheduleWhiteListParent[actorId] = parentId; @@ -640,7 +640,7 @@ namespace NActors { TInstant time = scheduledEvents.begin()->Deadline; while (!scheduledEvents.empty() && scheduledEvents.begin()->Deadline == time) { - static THashMap<std::pair<TActorId, TString>, ui64> eventTypes; + static THashMap<std::pair<TActorId, TString>, ui64> eventTypes; auto& item = *scheduledEvents.begin(); TString name = item.Event->GetBase() ? TypeName(*item.Event->GetBase()) : Sprintf("%08" PRIx32, item.Event->Type); eventTypes[std::make_pair(item.Event->Recipient, name)]++; @@ -725,7 +725,7 @@ namespace NActors { VERBOSE = verbose; } - void TTestActorRuntimeBase::AddLocalService(const TActorId& actorId, const TActorSetupCmd& cmd, ui32 nodeIndex) { + void TTestActorRuntimeBase::AddLocalService(const TActorId& actorId, const TActorSetupCmd& cmd, ui32 nodeIndex) { Y_VERIFY(!IsInitialized); Y_VERIFY(nodeIndex < NodeCount); auto node = Nodes[nodeIndex + FirstNodeId]; @@ -857,8 +857,8 @@ namespace NActors { return (*TmpDir)(); } - TActorId TTestActorRuntimeBase::Register(IActor* actor, ui32 nodeIndex, ui32 poolId, TMailboxType::EType mailboxType, - ui64 revolvingCounter, const TActorId& parentId) { + TActorId TTestActorRuntimeBase::Register(IActor* actor, ui32 nodeIndex, ui32 poolId, TMailboxType::EType mailboxType, + ui64 revolvingCounter, const TActorId& parentId) { Y_VERIFY(nodeIndex < NodeCount); TGuard<TMutex> guard(Mutex); TNodeDataBase* node = Nodes[FirstNodeId + nodeIndex].Get(); @@ -897,7 +897,7 @@ namespace NActors { mailbox->AttachActor(localActorId, actor); // do init - const TActorId actorId(FirstNodeId + nodeIndex, poolId, localActorId, hint); + const TActorId actorId(FirstNodeId + nodeIndex, poolId, localActorId, hint); ActorNames[actorId] = TypeName(*actor); RegistrationObserver(*this, parentId ? parentId : CurrentRecipient, actorId); DoActorInit(node->ActorSystem.Get(), actor, actorId, parentId ? parentId : CurrentRecipient); @@ -925,8 +925,8 @@ namespace NActors { return actorId; } - TActorId TTestActorRuntimeBase::Register(IActor *actor, ui32 nodeIndex, ui32 poolId, TMailboxHeader *mailbox, ui32 hint, - const TActorId& parentId) { + TActorId TTestActorRuntimeBase::Register(IActor *actor, ui32 nodeIndex, ui32 poolId, TMailboxHeader *mailbox, ui32 hint, + const TActorId& parentId) { Y_VERIFY(nodeIndex < NodeCount); TGuard<TMutex> guard(Mutex); TNodeDataBase* node = Nodes[FirstNodeId + nodeIndex].Get(); @@ -941,7 +941,7 @@ namespace NActors { } mailbox->AttachActor(localActorId, actor); - const TActorId actorId(FirstNodeId + nodeIndex, poolId, localActorId, hint); + const TActorId actorId(FirstNodeId + nodeIndex, poolId, localActorId, hint); ActorNames[actorId] = TypeName(*actor); RegistrationObserver(*this, parentId ? parentId : CurrentRecipient, actorId); DoActorInit(node->ActorSystem.Get(), actor, actorId, parentId ? parentId : CurrentRecipient); @@ -949,7 +949,7 @@ namespace NActors { return actorId; } - TActorId TTestActorRuntimeBase::RegisterService(const TActorId& serviceId, const TActorId& actorId, ui32 nodeIndex) { + TActorId TTestActorRuntimeBase::RegisterService(const TActorId& serviceId, const TActorId& actorId, ui32 nodeIndex) { TGuard<TMutex> guard(Mutex); Y_VERIFY(nodeIndex < NodeCount); TNodeDataBase* node = Nodes[FirstNodeId + nodeIndex].Get(); @@ -959,13 +959,13 @@ namespace NActors { node->ActorToActorId[actor] = actorId; } - return node->ActorSystem->RegisterLocalService(serviceId, actorId); + return node->ActorSystem->RegisterLocalService(serviceId, actorId); } - TActorId TTestActorRuntimeBase::AllocateEdgeActor(ui32 nodeIndex) { + TActorId TTestActorRuntimeBase::AllocateEdgeActor(ui32 nodeIndex) { TGuard<TMutex> guard(Mutex); Y_VERIFY(nodeIndex < NodeCount); - TActorId edgeActor = Register(new TEdgeActor(this), nodeIndex); + TActorId edgeActor = Register(new TEdgeActor(this), nodeIndex); EdgeActors.insert(edgeActor); EdgeActorByMailbox[TEventMailboxId(edgeActor.NodeId(), edgeActor.Hint())] = edgeActor; return edgeActor; @@ -1414,14 +1414,14 @@ namespace NActors { return it->second; } - TActorId TTestActorRuntimeBase::GetLocalServiceId(const TActorId& serviceId, ui32 nodeIndex) { + TActorId TTestActorRuntimeBase::GetLocalServiceId(const TActorId& serviceId, ui32 nodeIndex) { TGuard<TMutex> guard(Mutex); Y_VERIFY(nodeIndex < NodeCount); TNodeDataBase* node = Nodes[FirstNodeId + nodeIndex].Get(); return node->ActorSystem->LookupLocalService(serviceId); } - void TTestActorRuntimeBase::WaitForEdgeEvents(TEventFilter filter, const TSet<TActorId>& edgeFilter, TDuration simTimeout) { + void TTestActorRuntimeBase::WaitForEdgeEvents(TEventFilter filter, const TSet<TActorId>& edgeFilter, TDuration simTimeout) { TGuard<TMutex> guard(Mutex); ui32 dispatchCount = 0; if (!edgeFilter.empty()) { @@ -1429,7 +1429,7 @@ namespace NActors { Y_VERIFY(EdgeActors.contains(edgeActor), "%s is not an edge actor", ToString(edgeActor).data()); } } - const TSet<TActorId>& edgeActors = edgeFilter.empty() ? EdgeActors : edgeFilter; + const TSet<TActorId>& edgeActors = edgeFilter.empty() ? EdgeActors : edgeFilter; TInstant deadline = TInstant::MicroSeconds(CurrentTimestamp) + simTimeout; for (;;) { for (auto edgeActor : edgeActors) { @@ -1460,7 +1460,7 @@ namespace NActors { } } - TActorId TTestActorRuntimeBase::GetInterconnectProxy(ui32 nodeIndexFrom, ui32 nodeIndexTo) { + TActorId TTestActorRuntimeBase::GetInterconnectProxy(ui32 nodeIndexFrom, ui32 nodeIndexTo) { TGuard<TMutex> guard(Mutex); Y_VERIFY(nodeIndexFrom < NodeCount); Y_VERIFY(nodeIndexTo < NodeCount); @@ -1469,7 +1469,7 @@ namespace NActors { return node->ActorSystem->InterconnectProxy(FirstNodeId + nodeIndexTo); } - void TTestActorRuntimeBase::BlockOutputForActor(const TActorId& actorId) { + void TTestActorRuntimeBase::BlockOutputForActor(const TActorId& actorId) { TGuard<TMutex> guard(Mutex); BlockedOutput.insert(actorId); } @@ -1480,7 +1480,7 @@ namespace NActors { DispatcherRandomProvider = CreateDeterministicRandomProvider(DispatcherRandomSeed); } - IActor* TTestActorRuntimeBase::FindActor(const TActorId& actorId, ui32 nodeIndex) const { + IActor* TTestActorRuntimeBase::FindActor(const TActorId& actorId, ui32 nodeIndex) const { TGuard<TMutex> guard(Mutex); if (nodeIndex == Max<ui32>()) { Y_VERIFY(actorId.NodeId()); @@ -1494,7 +1494,7 @@ namespace NActors { return FindActor(actorId, node); } - void TTestActorRuntimeBase::EnableScheduleForActor(const TActorId& actorId, bool allow) { + void TTestActorRuntimeBase::EnableScheduleForActor(const TActorId& actorId, bool allow) { TGuard<TMutex> guard(Mutex); if (allow) { if (VERBOSE) { @@ -1509,7 +1509,7 @@ namespace NActors { } } - bool TTestActorRuntimeBase::IsScheduleForActorEnabled(const TActorId& actorId) const { + bool TTestActorRuntimeBase::IsScheduleForActorEnabled(const TActorId& actorId) const { TGuard<TMutex> guard(Mutex); return ScheduleWhiteList.find(actorId) != ScheduleWhiteList.end(); } @@ -1570,7 +1570,7 @@ namespace NActors { IActor* recipientActor = mailbox->FindActor(recipientLocalId); if (recipientActor) { // Save actorId by value in order to prevent ctx from being invalidated during another Send call. - TActorId actorId = ev->GetRecipientRewrite(); + TActorId actorId = ev->GetRecipientRewrite(); node->ActorToActorId[recipientActor] = ev->GetRecipientRewrite(); TActorContext ctx(*mailbox, *node->ExecutorThread, GetCycleCountFast(), actorId); TActivationContext *prevTlsActivationContext = TlsActivationContext; @@ -1585,7 +1585,7 @@ namespace NActors { recipientActor->Receive(evHolder, ctx); node->ExecutorThread->DropUnregistered(); } - CurrentRecipient = TActorId(); + CurrentRecipient = TActorId(); TlsActivationContext = prevTlsActivationContext; } else { if (VERBOSE) { @@ -1599,7 +1599,7 @@ namespace NActors { } } - IActor* TTestActorRuntimeBase::FindActor(const TActorId& actorId, TNodeDataBase* node) const { + IActor* TTestActorRuntimeBase::FindActor(const TActorId& actorId, TNodeDataBase* node) const { ui32 mailboxHint = actorId.Hint(); ui64 localId = actorId.LocalId(); TMailboxHeader* mailbox = node->MailboxTable->Get(mailboxHint); @@ -1644,7 +1644,7 @@ namespace NActors { setup->LocalServices = node->LocalServices; setup->Interconnect.ProxyActors.resize(FirstNodeId + NodeCount); - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); TIntrusivePtr<TInterconnectProxyCommon> common; common.Reset(new TInterconnectProxyCommon); @@ -1688,7 +1688,7 @@ namespace NActors { NActors::TLoggerActor *loggerActor = new NActors::TLoggerActor(node->LogSettings, logBackend, GetCountersForComponent(node->DynamicCounters, "utils")); NActors::TActorSetupCmd loggerActorCmd(loggerActor, NActors::TMailboxType::Simple, node->GetLoggerPoolId()); - std::pair<NActors::TActorId, NActors::TActorSetupCmd> loggerActorPair(node->LogSettings->LoggerActorId, loggerActorCmd); + std::pair<NActors::TActorId, NActors::TActorSetupCmd> loggerActorPair(node->LogSettings->LoggerActorId, loggerActorCmd); setup->LocalServices.push_back(loggerActorPair); } @@ -1732,7 +1732,7 @@ namespace NActors { Mailboxes.erase(mboxId); } - TString TTestActorRuntimeBase::GetActorName(const TActorId& actorId) const { + TString TTestActorRuntimeBase::GetActorName(const TActorId& actorId) const { auto it = ActorNames.find(actorId); if (it != ActorNames.end()) return it->second; @@ -1773,7 +1773,7 @@ namespace NActors { return TEST_ACTOR_RUNTIME; } - TStrandingActorDecorator(const TActorId& delegatee, bool isSync, const TVector<TActorId>& additionalActors, + TStrandingActorDecorator(const TActorId& delegatee, bool isSync, const TVector<TActorId>& additionalActors, TSimpleSharedPtr<TStrandingActorDecoratorContext> context, TTestActorRuntimeBase* runtime, TReplyCheckerCreator createReplyChecker) : Delegatee(delegatee) @@ -1813,7 +1813,7 @@ namespace NActors { STFUNC(Reply) { Y_VERIFY(!HasReply); IEventHandle *requestEv = Context->Queue->Head(); - TActorId originalSender = requestEv->Sender; + TActorId originalSender = requestEv->Sender; HasReply = !ReplyChecker->IsWaitingForMoreResponses(ev.Get()); if (HasReply) { delete Context->Queue->Pop(); @@ -1857,11 +1857,11 @@ namespace NActors { return forwardedEv; } private: - const TActorId Delegatee; + const TActorId Delegatee; const bool IsSync; - const TVector<TActorId> AdditionalActors; + const TVector<TActorId> AdditionalActors; TSimpleSharedPtr<TStrandingActorDecoratorContext> Context; - TActorId ReplyId; + TActorId ReplyId; bool HasReply; TDispatchOptions DelegateeOptions; TTestActorRuntimeBase* Runtime; @@ -1882,7 +1882,7 @@ namespace NActors { { } - IActor* Wrap(const TActorId& delegatee, bool isSync, const TVector<TActorId>& additionalActors) override { + IActor* Wrap(const TActorId& delegatee, bool isSync, const TVector<TActorId>& additionalActors) override { return new TStrandingActorDecorator(delegatee, isSync, additionalActors, Context, Runtime, CreateReplyChecker); } diff --git a/library/cpp/actors/testlib/test_runtime.h b/library/cpp/actors/testlib/test_runtime.h index b7e8edd1c52..26e3b45c984 100644 --- a/library/cpp/actors/testlib/test_runtime.h +++ b/library/cpp/actors/testlib/test_runtime.h @@ -199,7 +199,7 @@ namespace NActors { typedef std::function<void(TTestActorRuntimeBase& runtime, TScheduledEventsList& scheduledEvents, TEventsList& queue)> TScheduledEventsSelector; typedef std::function<bool(TTestActorRuntimeBase& runtime, TAutoPtr<IEventHandle>& event)> TEventFilter; typedef std::function<bool(TTestActorRuntimeBase& runtime, TAutoPtr<IEventHandle>& event, TDuration delay, TInstant& deadline)> TScheduledEventFilter; - typedef std::function<void(TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId)> TRegistrationObserver; + typedef std::function<void(TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId)> TRegistrationObserver; TTestActorRuntimeBase(THeSingleSystemEnv); @@ -213,7 +213,7 @@ namespace NActors { static void CollapsedTimeScheduledEventsSelector(TTestActorRuntimeBase& runtime, TScheduledEventsList& scheduledEvents, TEventsList& queue); static bool DefaultFilterFunc(TTestActorRuntimeBase& runtime, TAutoPtr<IEventHandle>& event); static bool NopFilterFunc(TTestActorRuntimeBase& runtime, TAutoPtr<IEventHandle>& event, TDuration delay, TInstant& deadline); - static void DefaultRegistrationObserver(TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId); + static void DefaultRegistrationObserver(TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId); TEventObserver SetObserverFunc(TEventObserver observerFunc); TScheduledEventsSelector SetScheduledEventsSelectorFunc(TScheduledEventsSelector scheduledEventsSelectorFunc); TEventFilter SetEventFilter(TEventFilter filterFunc); @@ -232,20 +232,20 @@ namespace NActors { TInstant GetCurrentTime() const; void UpdateCurrentTime(TInstant newTime); void AdvanceCurrentTime(TDuration duration); - void AddLocalService(const TActorId& actorId, const TActorSetupCmd& cmd, ui32 nodeIndex = 0); + void AddLocalService(const TActorId& actorId, const TActorSetupCmd& cmd, ui32 nodeIndex = 0); virtual void Initialize(); ui32 GetNodeId(ui32 index = 0) const; ui32 GetNodeCount() const; ui64 AllocateLocalId(); ui32 InterconnectPoolId() const; TString GetTempDir(); - TActorId Register(IActor* actor, ui32 nodeIndex = 0, ui32 poolId = 0, + TActorId Register(IActor* actor, ui32 nodeIndex = 0, ui32 poolId = 0, TMailboxType::EType mailboxType = TMailboxType::Simple, ui64 revolvingCounter = 0, - const TActorId& parentid = TActorId()); - TActorId Register(IActor *actor, ui32 nodeIndex, ui32 poolId, TMailboxHeader *mailbox, ui32 hint, - const TActorId& parentid = TActorId()); - TActorId RegisterService(const TActorId& serviceId, const TActorId& actorId, ui32 nodeIndex = 0); - TActorId AllocateEdgeActor(ui32 nodeIndex = 0); + const TActorId& parentid = TActorId()); + TActorId Register(IActor *actor, ui32 nodeIndex, ui32 poolId, TMailboxHeader *mailbox, ui32 hint, + const TActorId& parentid = TActorId()); + TActorId RegisterService(const TActorId& serviceId, const TActorId& actorId, ui32 nodeIndex = 0); + TActorId AllocateEdgeActor(ui32 nodeIndex = 0); TEventsList CaptureEvents(); TEventsList CaptureMailboxEvents(ui32 hint, ui32 nodeId); TScheduledEventsList CaptureScheduledEvents(); @@ -260,13 +260,13 @@ namespace NActors { void Schedule(IEventHandle* ev, const TDuration& duration, ui32 nodeIndex = 0); void ClearCounters(); ui64 GetCounter(ui32 evType) const; - TActorId GetLocalServiceId(const TActorId& serviceId, ui32 nodeIndex = 0); - void WaitForEdgeEvents(TEventFilter filter, const TSet<TActorId>& edgeFilter = {}, TDuration simTimeout = TDuration::Max()); - TActorId GetInterconnectProxy(ui32 nodeIndexFrom, ui32 nodeIndexTo); - void BlockOutputForActor(const TActorId& actorId); - IActor* FindActor(const TActorId& actorId, ui32 nodeIndex = Max<ui32>()) const; - void EnableScheduleForActor(const TActorId& actorId, bool allow = true); - bool IsScheduleForActorEnabled(const TActorId& actorId) const; + TActorId GetLocalServiceId(const TActorId& serviceId, ui32 nodeIndex = 0); + void WaitForEdgeEvents(TEventFilter filter, const TSet<TActorId>& edgeFilter = {}, TDuration simTimeout = TDuration::Max()); + TActorId GetInterconnectProxy(ui32 nodeIndexFrom, ui32 nodeIndexTo); + void BlockOutputForActor(const TActorId& actorId); + IActor* FindActor(const TActorId& actorId, ui32 nodeIndex = Max<ui32>()) const; + void EnableScheduleForActor(const TActorId& actorId, bool allow = true); + bool IsScheduleForActorEnabled(const TActorId& actorId) const; TIntrusivePtr<NMonitoring::TDynamicCounters> GetDynamicCounters(ui32 nodeIndex = 0); void SetupMonitoring(); @@ -317,7 +317,7 @@ namespace NActors { template<class TEvent> typename TEvent::TPtr GrabEdgeEventIf( - const TSet<TActorId>& edgeFilter, + const TSet<TActorId>& edgeFilter, const std::function<bool(const typename TEvent::TPtr&)>& predicate, TDuration simTimeout = TDuration::Max()) { @@ -345,11 +345,11 @@ namespace NActors { template<class TEvent> typename TEvent::TPtr GrabEdgeEventIf( - const TActorId& edgeActor, + const TActorId& edgeActor, const std::function<bool(const typename TEvent::TPtr&)>& predicate, TDuration simTimeout = TDuration::Max()) { - TSet<TActorId> edgeFilter{edgeActor}; + TSet<TActorId> edgeFilter{edgeActor}; return GrabEdgeEventIf<TEvent>(edgeFilter, predicate, simTimeout); } @@ -368,13 +368,13 @@ namespace NActors { } template<class TEvent> - typename TEvent::TPtr GrabEdgeEvent(const TSet<TActorId>& edgeFilter, TDuration simTimeout = TDuration::Max()) { + typename TEvent::TPtr GrabEdgeEvent(const TSet<TActorId>& edgeFilter, TDuration simTimeout = TDuration::Max()) { return GrabEdgeEventIf<TEvent>(edgeFilter, [](const typename TEvent::TPtr&) { return true; }, simTimeout); } template<class TEvent> - typename TEvent::TPtr GrabEdgeEvent(const TActorId& edgeActor, TDuration simTimeout = TDuration::Max()) { - TSet<TActorId> edgeFilter{edgeActor}; + typename TEvent::TPtr GrabEdgeEvent(const TActorId& edgeActor, TDuration simTimeout = TDuration::Max()) { + TSet<TActorId> edgeFilter{edgeActor}; return GrabEdgeEvent<TEvent>(edgeFilter, simTimeout); } @@ -409,7 +409,7 @@ namespace NActors { } template<class TEvent> - typename TEvent::TPtr GrabEdgeEventRethrow(const TSet<TActorId>& edgeFilter, TDuration simTimeout = TDuration::Max()) { + typename TEvent::TPtr GrabEdgeEventRethrow(const TSet<TActorId>& edgeFilter, TDuration simTimeout = TDuration::Max()) { try { return GrabEdgeEvent<TEvent>(edgeFilter, simTimeout); } catch (...) { @@ -418,7 +418,7 @@ namespace NActors { } template<class TEvent> - typename TEvent::TPtr GrabEdgeEventRethrow(const TActorId& edgeActor, TDuration simTimeout = TDuration::Max()) { + typename TEvent::TPtr GrabEdgeEventRethrow(const TActorId& edgeActor, TDuration simTimeout = TDuration::Max()) { try { return GrabEdgeEvent<TEvent>(edgeActor, simTimeout); } catch (...) { @@ -462,7 +462,7 @@ namespace NActors { } void SetDispatcherRandomSeed(TInstant time, ui64 iteration); - TString GetActorName(const TActorId& actorId) const; + TString GetActorName(const TActorId& actorId) const; const TVector<ui64>& GetTxAllocatorTabletIds() const { return TxAllocatorTabletIds; } void SetTxAllocatorTabletIds(const TVector<ui64>& ids) { TxAllocatorTabletIds = ids; } @@ -493,7 +493,7 @@ namespace NActors { } private: - IActor* FindActor(const TActorId& actorId, TNodeDataBase* node) const; + IActor* FindActor(const TActorId& actorId, TNodeDataBase* node) const; void SendInternal(IEventHandle* ev, ui32 nodeIndex, bool viaActorSystem); TEventMailBox& GetMailbox(ui32 nodeId, ui32 hint); void ClearMailbox(ui32 nodeId, ui32 hint); @@ -526,7 +526,7 @@ namespace NActors { ui64 DispatchCyclesCount; ui64 DispatchedEventsCount; ui64 DispatchedEventsLimit = 2'500'000; - TActorId CurrentRecipient; + TActorId CurrentRecipient; ui64 DispatcherRandomSeed; TIntrusivePtr<IRandomProvider> DispatcherRandomProvider; TAutoPtr<TLogBackend> LogBackend; @@ -559,9 +559,9 @@ namespace NActors { TIntrusivePtr<NInterconnect::TPollerThreads> Poller; volatile ui64* ActorSystemTimestamp; volatile ui64* ActorSystemMonotonic; - TVector<std::pair<TActorId, TActorSetupCmd> > LocalServices; - TMap<TActorId, IActor*> LocalServicesActors; - TMap<IActor*, TActorId> ActorToActorId; + TVector<std::pair<TActorId, TActorSetupCmd> > LocalServices; + TMap<TActorId, IActor*> LocalServicesActors; + TMap<IActor*, TActorId> ActorToActorId; THolder<TMailboxTable> MailboxTable; std::shared_ptr<void> AppData0; THolder<TActorSystem> ActorSystem; @@ -613,8 +613,8 @@ namespace NActors { TProgramShouldContinue ShouldContinue; TMap<ui32, TIntrusivePtr<TNodeDataBase>> Nodes; ui64 CurrentTimestamp; - TSet<TActorId> EdgeActors; - THashMap<TEventMailboxId, TActorId, TEventMailboxId::THash> EdgeActorByMailbox; + TSet<TActorId> EdgeActors; + THashMap<TEventMailboxId, TActorId, TEventMailboxId::THash> EdgeActorByMailbox; TDuration DispatchTimeout; TDuration ReschedulingDelay; TEventObserver ObserverFunc; @@ -622,10 +622,10 @@ namespace NActors { TEventFilter EventFilterFunc; TScheduledEventFilter ScheduledEventFilterFunc; TRegistrationObserver RegistrationObserver; - TSet<TActorId> BlockedOutput; - TSet<TActorId> ScheduleWhiteList; - THashMap<TActorId, TActorId> ScheduleWhiteListParent; - THashMap<TActorId, TString> ActorNames; + TSet<TActorId> BlockedOutput; + TSet<TActorId> ScheduleWhiteList; + THashMap<TActorId, TActorId> ScheduleWhiteListParent; + THashMap<TActorId, TString> ActorNames; TDispatchContext* CurrentDispatchContext; TVector<ui64> TxAllocatorTabletIds; @@ -686,7 +686,7 @@ namespace NActors { class IStrandingDecoratorFactory { public: virtual ~IStrandingDecoratorFactory() {} - virtual IActor* Wrap(const TActorId& delegatee, bool isSync, const TVector<TActorId>& additionalActors) = 0; + virtual IActor* Wrap(const TActorId& delegatee, bool isSync, const TVector<TActorId>& additionalActors) = 0; }; struct IReplyChecker { diff --git a/library/cpp/codecs/ut/codecs_ut.cpp b/library/cpp/codecs/ut/codecs_ut.cpp index 0f24c8158fa..caf6089aef7 100644 --- a/library/cpp/codecs/ut/codecs_ut.cpp +++ b/library/cpp/codecs/ut/codecs_ut.cpp @@ -914,7 +914,7 @@ private: { TBufferInput bin(buff); c = ICodec::Restore(&bin); - UNIT_ASSERT(c->AlreadyTrained()); + UNIT_ASSERT(c->AlreadyTrained()); } } @@ -937,7 +937,7 @@ private: UNIT_ASSERT_EQUAL_C(AsStrBuf(inlearn[i]), AsStrBuf(vecl), PrintError(TStringBuf(inlearn[i].data(), inlearn[i].size()), - TStringBuf(vecl.data(), vecl.size()), c->GetName(), i)); + TStringBuf(vecl.data(), vecl.size()), c->GetName(), i)); } } } @@ -961,7 +961,7 @@ private: c->Decode(AsStrBuf(out), in1); UNIT_ASSERT_EQUAL_C(AsStrBuf(in[i]), AsStrBuf(in1), PrintError(TStringBuf(in[i].data(), in[i].size()), - TStringBuf(in1.data(), in1.size()), c->GetName(), i)); + TStringBuf(in1.data(), in1.size()), c->GetName(), i)); } } } diff --git a/library/cpp/containers/comptrie/loader/loader.cpp b/library/cpp/containers/comptrie/loader/loader.cpp index 22f282d4069..4811e9d5216 100644 --- a/library/cpp/containers/comptrie/loader/loader.cpp +++ b/library/cpp/containers/comptrie/loader/loader.cpp @@ -1 +1 @@ -#include "loader.h" +#include "loader.h" diff --git a/library/cpp/containers/comptrie/loader/ya.make b/library/cpp/containers/comptrie/loader/ya.make index e62c8a76fa0..1e23e442a01 100644 --- a/library/cpp/containers/comptrie/loader/ya.make +++ b/library/cpp/containers/comptrie/loader/ya.make @@ -4,7 +4,7 @@ OWNER(my34) SRCS( loader.h - loader.cpp + loader.cpp ) PEERDIR( diff --git a/library/cpp/grpc/client/grpc_client_low.h b/library/cpp/grpc/client/grpc_client_low.h index 356d1e1f0bc..ab0a0627be0 100644 --- a/library/cpp/grpc/client/grpc_client_low.h +++ b/library/cpp/grpc/client/grpc_client_low.h @@ -10,7 +10,7 @@ #include <deque> #include <typeindex> #include <typeinfo> -#include <variant> +#include <variant> #include <vector> #include <unordered_map> #include <unordered_set> @@ -185,7 +185,7 @@ using TAdvancedResponseCallback = std::function<void (const grpc::ClientContext& struct TCallMeta { std::shared_ptr<grpc::CallCredentials> CallCredentials; std::vector<std::pair<TString, TString>> Aux; - std::variant<TDuration, TInstant> Timeout; // timeout as duration from now or time point in future + std::variant<TDuration, TInstant> Timeout; // timeout as duration from now or time point in future }; class TGRpcRequestProcessorCommon { @@ -197,17 +197,17 @@ protected: if (meta.CallCredentials) { Context.set_credentials(meta.CallCredentials); } - if (const TDuration* timeout = std::get_if<TDuration>(&meta.Timeout)) { - if (*timeout) { - auto deadline = gpr_time_add( - gpr_now(GPR_CLOCK_MONOTONIC), - gpr_time_from_micros(timeout->MicroSeconds(), GPR_TIMESPAN)); - Context.set_deadline(deadline); - } - } else if (const TInstant* deadline = std::get_if<TInstant>(&meta.Timeout)) { - if (*deadline) { - Context.set_deadline(gpr_time_from_micros(deadline->MicroSeconds(), GPR_CLOCK_MONOTONIC)); - } + if (const TDuration* timeout = std::get_if<TDuration>(&meta.Timeout)) { + if (*timeout) { + auto deadline = gpr_time_add( + gpr_now(GPR_CLOCK_MONOTONIC), + gpr_time_from_micros(timeout->MicroSeconds(), GPR_TIMESPAN)); + Context.set_deadline(deadline); + } + } else if (const TInstant* deadline = std::get_if<TInstant>(&meta.Timeout)) { + if (*deadline) { + Context.set_deadline(gpr_time_from_micros(deadline->MicroSeconds(), GPR_CLOCK_MONOTONIC)); + } } } diff --git a/library/cpp/grpc/server/actors/logger.cpp b/library/cpp/grpc/server/actors/logger.cpp index 176675366a5..d8b20425764 100644 --- a/library/cpp/grpc/server/actors/logger.cpp +++ b/library/cpp/grpc/server/actors/logger.cpp @@ -1,45 +1,45 @@ -#include "logger.h" - -namespace NGrpc { -namespace { - -static_assert( - ui16(TLOG_EMERG) == ui16(NActors::NLog::PRI_EMERG) && - ui16(TLOG_DEBUG) == ui16(NActors::NLog::PRI_DEBUG), - "log levels in the library/log and library/cpp/actors don't match"); - -class TActorSystemLogger final: public TLogger { -public: - TActorSystemLogger(NActors::TActorSystem& as, NActors::NLog::EComponent component) noexcept - : ActorSystem_{as} - , Component_{component} - { - } - -protected: - bool DoIsEnabled(ELogPriority p) const noexcept override { - const auto* settings = static_cast<::NActors::NLog::TSettings*>(ActorSystem_.LoggerSettings()); - const auto priority = static_cast<::NActors::NLog::EPriority>(p); - - return settings && settings->Satisfies(priority, Component_, 0); - } - - void DoWrite(ELogPriority p, const char* format, va_list args) noexcept override { - Y_VERIFY_DEBUG(DoIsEnabled(p)); - - const auto priority = static_cast<::NActors::NLog::EPriority>(p); - ::NActors::MemLogAdapter(ActorSystem_, priority, Component_, format, args); - } - -private: - NActors::TActorSystem& ActorSystem_; - NActors::NLog::EComponent Component_; -}; - -} // namespace - -TLoggerPtr CreateActorSystemLogger(NActors::TActorSystem& as, NActors::NLog::EComponent component) { - return MakeIntrusive<TActorSystemLogger>(as, component); -} - -} // namespace NGrpc +#include "logger.h" + +namespace NGrpc { +namespace { + +static_assert( + ui16(TLOG_EMERG) == ui16(NActors::NLog::PRI_EMERG) && + ui16(TLOG_DEBUG) == ui16(NActors::NLog::PRI_DEBUG), + "log levels in the library/log and library/cpp/actors don't match"); + +class TActorSystemLogger final: public TLogger { +public: + TActorSystemLogger(NActors::TActorSystem& as, NActors::NLog::EComponent component) noexcept + : ActorSystem_{as} + , Component_{component} + { + } + +protected: + bool DoIsEnabled(ELogPriority p) const noexcept override { + const auto* settings = static_cast<::NActors::NLog::TSettings*>(ActorSystem_.LoggerSettings()); + const auto priority = static_cast<::NActors::NLog::EPriority>(p); + + return settings && settings->Satisfies(priority, Component_, 0); + } + + void DoWrite(ELogPriority p, const char* format, va_list args) noexcept override { + Y_VERIFY_DEBUG(DoIsEnabled(p)); + + const auto priority = static_cast<::NActors::NLog::EPriority>(p); + ::NActors::MemLogAdapter(ActorSystem_, priority, Component_, format, args); + } + +private: + NActors::TActorSystem& ActorSystem_; + NActors::NLog::EComponent Component_; +}; + +} // namespace + +TLoggerPtr CreateActorSystemLogger(NActors::TActorSystem& as, NActors::NLog::EComponent component) { + return MakeIntrusive<TActorSystemLogger>(as, component); +} + +} // namespace NGrpc diff --git a/library/cpp/grpc/server/actors/logger.h b/library/cpp/grpc/server/actors/logger.h index c066a40add1..abf9270f7b0 100644 --- a/library/cpp/grpc/server/actors/logger.h +++ b/library/cpp/grpc/server/actors/logger.h @@ -1,11 +1,11 @@ -#pragma once - -#include <library/cpp/actors/core/actorsystem.h> -#include <library/cpp/actors/core/log.h> -#include <library/cpp/grpc/server/logger.h> - -namespace NGrpc { - -TLoggerPtr CreateActorSystemLogger(NActors::TActorSystem& as, NActors::NLog::EComponent component); - -} // namespace NGrpc +#pragma once + +#include <library/cpp/actors/core/actorsystem.h> +#include <library/cpp/actors/core/log.h> +#include <library/cpp/grpc/server/logger.h> + +namespace NGrpc { + +TLoggerPtr CreateActorSystemLogger(NActors::TActorSystem& as, NActors::NLog::EComponent component); + +} // namespace NGrpc diff --git a/library/cpp/grpc/server/actors/ya.make b/library/cpp/grpc/server/actors/ya.make index 072db841424..6c9d80aa458 100644 --- a/library/cpp/grpc/server/actors/ya.make +++ b/library/cpp/grpc/server/actors/ya.make @@ -1,13 +1,13 @@ -LIBRARY() - -OWNER(g:kikimr g:solomon) - -SRCS( - logger.cpp -) - -PEERDIR( - library/cpp/actors/core -) - -END() +LIBRARY() + +OWNER(g:kikimr g:solomon) + +SRCS( + logger.cpp +) + +PEERDIR( + library/cpp/actors/core +) + +END() diff --git a/library/cpp/grpc/server/event_callback.h b/library/cpp/grpc/server/event_callback.h index 13d9bb46b24..d0b700b3c92 100644 --- a/library/cpp/grpc/server/event_callback.h +++ b/library/cpp/grpc/server/event_callback.h @@ -2,7 +2,7 @@ #include "grpc_server.h" -namespace NGrpc { +namespace NGrpc { enum class EQueueEventStatus { OK, @@ -10,7 +10,7 @@ enum class EQueueEventStatus { }; template<class TCallback> -class TQueueEventCallback: public IQueueEvent { +class TQueueEventCallback: public IQueueEvent { public: TQueueEventCallback(const TCallback& callback) : Callback(callback) @@ -33,9 +33,9 @@ private: TCallback Callback; }; -// Implementation of IQueueEvent that reduces allocations +// Implementation of IQueueEvent that reduces allocations template<class TSelf> -class TQueueFixedEvent: private IQueueEvent { +class TQueueFixedEvent: private IQueueEvent { using TCallback = void (TSelf::*)(EQueueEventStatus); public: @@ -44,7 +44,7 @@ public: , Callback(callback) { } - IQueueEvent* Prepare() { + IQueueEvent* Prepare() { Self->Ref(); return this; } @@ -65,16 +65,16 @@ private: }; template<class TCallback> -inline IQueueEvent* MakeQueueEventCallback(TCallback&& callback) { +inline IQueueEvent* MakeQueueEventCallback(TCallback&& callback) { return new TQueueEventCallback<TCallback>(std::forward<TCallback>(callback)); } template<class T> -inline IQueueEvent* MakeQueueEventCallback(T* self, void (T::*method)(EQueueEventStatus)) { +inline IQueueEvent* MakeQueueEventCallback(T* self, void (T::*method)(EQueueEventStatus)) { using TPtr = TIntrusivePtr<T>; return MakeQueueEventCallback([self = TPtr(self), method] (EQueueEventStatus status) { ((*self).*method)(status); }); } -} // namespace NGrpc +} // namespace NGrpc diff --git a/library/cpp/grpc/server/grpc_async_ctx_base.h b/library/cpp/grpc/server/grpc_async_ctx_base.h index 65341fa1ad1..51356d4ce5a 100644 --- a/library/cpp/grpc/server/grpc_async_ctx_base.h +++ b/library/cpp/grpc/server/grpc_async_ctx_base.h @@ -5,17 +5,17 @@ #include <util/generic/vector.h> #include <util/generic/string.h> #include <util/system/yassert.h> -#include <util/generic/set.h> +#include <util/generic/set.h> #include <grpc++/server.h> #include <grpc++/server_context.h> #include <chrono> -namespace NGrpc { +namespace NGrpc { template<typename TService> -class TBaseAsyncContext: public ICancelableContext { +class TBaseAsyncContext: public ICancelableContext { public: TBaseAsyncContext(typename TService::TCurrentGRpcService::AsyncService* service, grpc::ServerCompletionQueue* cq) : Service(service) @@ -29,44 +29,44 @@ public: TInstant Deadline() const { // The timeout transferred in "grpc-timeout" header [1] and calculated from the deadline - // right before the request is getting to be send. - // 1. https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md - // + // right before the request is getting to be send. + // 1. https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md + // // After this timeout calculated back to the deadline on the server side // using server grpc GPR_CLOCK_MONOTONIC time (raw_deadline() method). // deadline() method convert this to epoch related deadline GPR_CLOCK_REALTIME // - + std::chrono::system_clock::time_point t = Context.deadline(); if (t == std::chrono::system_clock::time_point::max()) { return TInstant::Max(); - } + } auto us = std::chrono::time_point_cast<std::chrono::microseconds>(t); return TInstant::MicroSeconds(us.time_since_epoch().count()); - } - - TSet<TStringBuf> GetPeerMetaKeys() const { - TSet<TStringBuf> keys; - for (const auto& [key, _]: Context.client_metadata()) { - keys.emplace(key.data(), key.size()); - } - return keys; - } - - TVector<TStringBuf> GetPeerMetaValues(TStringBuf key) const { + } + + TSet<TStringBuf> GetPeerMetaKeys() const { + TSet<TStringBuf> keys; + for (const auto& [key, _]: Context.client_metadata()) { + keys.emplace(key.data(), key.size()); + } + return keys; + } + + TVector<TStringBuf> GetPeerMetaValues(TStringBuf key) const { const auto& clientMetadata = Context.client_metadata(); - const auto range = clientMetadata.equal_range(grpc::string_ref{key.data(), key.size()}); - if (range.first == range.second) { - return {}; - } - - TVector<TStringBuf> values; - values.reserve(std::distance(range.first, range.second)); - + const auto range = clientMetadata.equal_range(grpc::string_ref{key.data(), key.size()}); + if (range.first == range.second) { + return {}; + } + + TVector<TStringBuf> values; + values.reserve(std::distance(range.first, range.second)); + for (auto it = range.first; it != range.second; ++it) { - values.emplace_back(it->second.data(), it->second.size()); + values.emplace_back(it->second.data(), it->second.size()); } - return values; + return values; } grpc_compression_level GetCompressionLevel() const { @@ -91,4 +91,4 @@ protected: grpc::ServerContext Context; }; -} // namespace NGrpc +} // namespace NGrpc diff --git a/library/cpp/grpc/server/grpc_counters.cpp b/library/cpp/grpc/server/grpc_counters.cpp index bdd72b32928..fa96e0100b9 100644 --- a/library/cpp/grpc/server/grpc_counters.cpp +++ b/library/cpp/grpc/server/grpc_counters.cpp @@ -1,45 +1,45 @@ #include "grpc_counters.h" - -namespace NGrpc { -namespace { - -class TFakeCounterBlock final: public ICounterBlock { -private: - void CountNotOkRequest() override { - } - - void CountNotOkResponse() override { - } - - void CountNotAuthenticated() override { - } - - void CountResourceExhausted() override { - } - - void CountRequestBytes(ui32 /*requestSize*/) override { - } - - void CountResponseBytes(ui32 /*responseSize*/) override { - } - - void StartProcessing(ui32 /*requestSize*/) override { - } - - void FinishProcessing( - ui32 /*requestSize*/, - ui32 /*responseSize*/, - bool /*ok*/, - ui32 /*status*/, - TDuration /*requestDuration*/) override - { - } -}; - -} // namespace - -ICounterBlockPtr FakeCounterBlock() { - return MakeIntrusive<TFakeCounterBlock>(); -} - -} // namespace NGrpc + +namespace NGrpc { +namespace { + +class TFakeCounterBlock final: public ICounterBlock { +private: + void CountNotOkRequest() override { + } + + void CountNotOkResponse() override { + } + + void CountNotAuthenticated() override { + } + + void CountResourceExhausted() override { + } + + void CountRequestBytes(ui32 /*requestSize*/) override { + } + + void CountResponseBytes(ui32 /*responseSize*/) override { + } + + void StartProcessing(ui32 /*requestSize*/) override { + } + + void FinishProcessing( + ui32 /*requestSize*/, + ui32 /*responseSize*/, + bool /*ok*/, + ui32 /*status*/, + TDuration /*requestDuration*/) override + { + } +}; + +} // namespace + +ICounterBlockPtr FakeCounterBlock() { + return MakeIntrusive<TFakeCounterBlock>(); +} + +} // namespace NGrpc diff --git a/library/cpp/grpc/server/grpc_counters.h b/library/cpp/grpc/server/grpc_counters.h index a591beb84e0..0b6c36c84cc 100644 --- a/library/cpp/grpc/server/grpc_counters.h +++ b/library/cpp/grpc/server/grpc_counters.h @@ -1,10 +1,10 @@ #pragma once -#include <library/cpp/monlib/dynamic_counters/percentile/percentile.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/percentile/percentile.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <util/generic/ptr.h> -namespace NGrpc { +namespace NGrpc { struct ICounterBlock : public TThrRefBase { virtual void CountNotOkRequest() = 0; @@ -14,7 +14,7 @@ struct ICounterBlock : public TThrRefBase { virtual void CountRequestBytes(ui32 requestSize) = 0; virtual void CountResponseBytes(ui32 responseSize) = 0; virtual void StartProcessing(ui32 requestSize) = 0; - virtual void FinishProcessing(ui32 requestSize, ui32 responseSize, bool ok, ui32 status, TDuration requestDuration) = 0; + virtual void FinishProcessing(ui32 requestSize, ui32 responseSize, bool ok, ui32 status, TDuration requestDuration) = 0; virtual void CountRequestsWithoutDatabase() {} virtual void CountRequestsWithoutToken() {} virtual void CountRequestWithoutTls() {} @@ -126,11 +126,11 @@ public: using TCounterBlockPtr = TIntrusivePtr<TCounterBlock>; -/** - * Creates new instance of ICounterBlock implementation which does nothing. - * - * @return new instance - */ -ICounterBlockPtr FakeCounterBlock(); - -} // namespace NGrpc +/** + * Creates new instance of ICounterBlock implementation which does nothing. + * + * @return new instance + */ +ICounterBlockPtr FakeCounterBlock(); + +} // namespace NGrpc diff --git a/library/cpp/grpc/server/grpc_request.cpp b/library/cpp/grpc/server/grpc_request.cpp index 60db2f230d3..d18a32776f2 100644 --- a/library/cpp/grpc/server/grpc_request.cpp +++ b/library/cpp/grpc/server/grpc_request.cpp @@ -1,10 +1,10 @@ #include "grpc_request.h" -namespace NGrpc { +namespace NGrpc { const char* GRPC_USER_AGENT_HEADER = "user-agent"; -class TStreamAdaptor: public IStreamAdaptor { +class TStreamAdaptor: public IStreamAdaptor { public: TStreamAdaptor() : StreamIsReady_(true) @@ -56,4 +56,4 @@ IStreamAdaptor::TPtr CreateStreamAdaptor() { return std::make_unique<TStreamAdaptor>(); } -} // namespace NGrpc +} // namespace NGrpc diff --git a/library/cpp/grpc/server/grpc_request.h b/library/cpp/grpc/server/grpc_request.h index dd9041eec7f..5bd8d3902b5 100644 --- a/library/cpp/grpc/server/grpc_request.h +++ b/library/cpp/grpc/server/grpc_request.h @@ -4,19 +4,19 @@ #include <google/protobuf/arena.h> #include <google/protobuf/message.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <library/cpp/logger/priority.h> -#include "grpc_response.h" +#include "grpc_response.h" #include "event_callback.h" -#include "grpc_async_ctx_base.h" -#include "grpc_counters.h" +#include "grpc_async_ctx_base.h" +#include "grpc_counters.h" #include "grpc_request_base.h" #include "grpc_server.h" -#include "logger.h" +#include "logger.h" + +#include <util/system/hp_timer.h> -#include <util/system/hp_timer.h> - #include <grpc++/server.h> #include <grpc++/server_context.h> #include <grpc++/support/async_stream.h> @@ -24,7 +24,7 @@ #include <grpc++/support/byte_buffer.h> #include <grpc++/impl/codegen/async_stream.h> -namespace NGrpc { +namespace NGrpc { class IStreamAdaptor { public: @@ -57,7 +57,7 @@ public: grpc::ServerCompletionQueue* cq, TOnRequest cb, TRequestCallback requestCallback, - const char* name, + const char* name, TLoggerPtr logger, ICounterBlockPtr counters, IGRpcRequestLimiterPtr limiter) @@ -67,10 +67,10 @@ public: , RequestCallback_(requestCallback) , StreamRequestCallback_(nullptr) , Name_(name) - , Logger_(std::move(logger)) + , Logger_(std::move(logger)) , Counters_(std::move(counters)) , RequestLimiter_(std::move(limiter)) - , Writer_(new grpc::ServerAsyncResponseWriter<TUniversalResponseRef<TOut>>(&this->Context)) + , Writer_(new grpc::ServerAsyncResponseWriter<TUniversalResponseRef<TOut>>(&this->Context)) , StateFunc_(&TThis::SetRequestDone) { AuthState_ = Server_->NeedAuth() ? TAuthState(true) : TAuthState(false); @@ -85,7 +85,7 @@ public: grpc::ServerCompletionQueue* cq, TOnRequest cb, TStreamRequestCallback requestCallback, - const char* name, + const char* name, TLoggerPtr logger, ICounterBlockPtr counters, IGRpcRequestLimiterPtr limiter) @@ -95,7 +95,7 @@ public: , RequestCallback_(nullptr) , StreamRequestCallback_(requestCallback) , Name_(name) - , Logger_(std::move(logger)) + , Logger_(std::move(logger)) , Counters_(std::move(counters)) , RequestLimiter_(std::move(limiter)) , StreamWriter_(new grpc::ServerAsyncWriter<TUniversalResponse<TOut>>(&this->Context)) @@ -157,13 +157,13 @@ public: TInstant Deadline() const override { return TBaseAsyncContext<TService>::Deadline(); - } - - TSet<TStringBuf> GetPeerMetaKeys() const override { - return TBaseAsyncContext<TService>::GetPeerMetaKeys(); - } - - TVector<TStringBuf> GetPeerMetaValues(TStringBuf key) const override { + } + + TSet<TStringBuf> GetPeerMetaKeys() const override { + return TBaseAsyncContext<TService>::GetPeerMetaKeys(); + } + + TVector<TStringBuf> GetPeerMetaValues(TStringBuf key) const override { return TBaseAsyncContext<TService>::GetPeerMetaValues(key); } @@ -233,10 +233,10 @@ private: if (!Server_->IsShuttingDown()) { if (RequestCallback_) { MakeIntrusive<TThis>( - Server_, this->Service, this->CQ, Cb_, RequestCallback_, Name_, Logger_, Counters_->Clone(), RequestLimiter_)->Run(); + Server_, this->Service, this->CQ, Cb_, RequestCallback_, Name_, Logger_, Counters_->Clone(), RequestLimiter_)->Run(); } else { MakeIntrusive<TThis>( - Server_, this->Service, this->CQ, Cb_, StreamRequestCallback_, Name_, Logger_, Counters_->Clone(), RequestLimiter_)->Run(); + Server_, this->Service, this->CQ, Cb_, StreamRequestCallback_, Name_, Logger_, Counters_->Clone(), RequestLimiter_)->Run(); } } } @@ -257,20 +257,20 @@ private: StateFunc_ = &TThis::SetFinishDone; ResponseSize = sz; Y_VERIFY(this->Context.c_call()); - Writer_->Finish(TUniversalResponseRef<TOut>(resp), grpc::Status::OK, GetGRpcTag()); + Writer_->Finish(TUniversalResponseRef<TOut>(resp), grpc::Status::OK, GetGRpcTag()); } else { GRPC_LOG_DEBUG(Logger_, "[%p] issuing response Name# %s data# %s peer# %s (enqueued)", this, Name_, makeResponseString().data(), this->Context.peer().c_str()); - - // because of std::function cannot hold move-only captured object - // we allocate shared object on heap to avoid message copy - auto uResp = MakeIntrusive<TUniversalResponse<TOut>>(resp); - auto cb = [this, uResp = std::move(uResp), sz, &makeResponseString]() { + + // because of std::function cannot hold move-only captured object + // we allocate shared object on heap to avoid message copy + auto uResp = MakeIntrusive<TUniversalResponse<TOut>>(resp); + auto cb = [this, uResp = std::move(uResp), sz, &makeResponseString]() { GRPC_LOG_DEBUG(Logger_, "[%p] issuing response Name# %s data# %s peer# %s (pushed to grpc)", this, Name_, makeResponseString().data(), this->Context.peer().c_str()); StateFunc_ = &TThis::NextReply; ResponseSize += sz; - StreamWriter_->Write(*uResp, GetGRpcTag()); + StreamWriter_->Write(*uResp, GetGRpcTag()); }; StreamAdaptor_->Enqueue(std::move(cb), false); } @@ -283,20 +283,20 @@ private: this->Context.peer().c_str()); StateFunc_ = &TThis::SetFinishDone; ResponseSize = sz; - Writer_->Finish(TUniversalResponseRef<TOut>(resp), grpc::Status::OK, GetGRpcTag()); + Writer_->Finish(TUniversalResponseRef<TOut>(resp), grpc::Status::OK, GetGRpcTag()); } else { GRPC_LOG_DEBUG(Logger_, "[%p] issuing response Name# %s data# byteString peer# %s (enqueued)", this, Name_, this->Context.peer().c_str()); - - // because of std::function cannot hold move-only captured object - // we allocate shared object on heap to avoid buffer copy - auto uResp = MakeIntrusive<TUniversalResponse<TOut>>(resp); - auto cb = [this, uResp = std::move(uResp), sz]() { + + // because of std::function cannot hold move-only captured object + // we allocate shared object on heap to avoid buffer copy + auto uResp = MakeIntrusive<TUniversalResponse<TOut>>(resp); + auto cb = [this, uResp = std::move(uResp), sz]() { GRPC_LOG_DEBUG(Logger_, "[%p] issuing response Name# %s data# byteString peer# %s (pushed to grpc)", this, Name_, this->Context.peer().c_str()); StateFunc_ = &TThis::NextReply; ResponseSize += sz; - StreamWriter_->Write(*uResp, GetGRpcTag()); + StreamWriter_->Write(*uResp, GetGRpcTag()); }; StreamAdaptor_->Enqueue(std::move(cb), false); } @@ -314,8 +314,8 @@ private: GRPC_LOG_DEBUG(Logger_, "[%p] issuing response Name# %s nodata (%s) peer# %s, grpc status# (%d)", this, Name_, msg.c_str(), this->Context.peer().c_str(), (int)code); StateFunc_ = &TThis::SetFinishError; - TOut resp; - Writer_->Finish(TUniversalResponseRef<TOut>(&resp), grpc::Status(code, msg), GetGRpcTag()); + TOut resp; + Writer_->Finish(TUniversalResponseRef<TOut>(&resp), grpc::Status(code, msg), GetGRpcTag()); } else { GRPC_LOG_DEBUG(Logger_, "[%p] issuing response Name# %s nodata (%s) peer# %s, grpc status# (%d)" " (enqueued)", this, Name_, msg.c_str(), this->Context.peer().c_str(), (int)code); @@ -380,7 +380,7 @@ private: } auto maybeToken = GetPeerMetaValues(TStringBuf("x-ydb-auth-ticket")); if (maybeToken.empty() || maybeToken[0].empty()) { - TString db{maybeDatabase ? maybeDatabase[0] : TStringBuf{}}; + TString db{maybeDatabase ? maybeDatabase[0] : TStringBuf{}}; Counters_->CountRequestsWithoutToken(); GRPC_LOG_DEBUG(Logger_, "[%p] received request without user token " "Name# %s data# %s peer# %s database# %s", this, Name_, @@ -484,12 +484,12 @@ private: TOnRequest Cb_; TRequestCallback RequestCallback_; TStreamRequestCallback StreamRequestCallback_; - const char* const Name_; + const char* const Name_; TLoggerPtr Logger_; ICounterBlockPtr Counters_; IGRpcRequestLimiterPtr RequestLimiter_; - THolder<grpc::ServerAsyncResponseWriter<TUniversalResponseRef<TOut>>> Writer_; + THolder<grpc::ServerAsyncResponseWriter<TUniversalResponseRef<TOut>>> Writer_; THolder<grpc::ServerAsyncWriterInterface<TUniversalResponse<TOut>>> StreamWriter_; TStateFunc StateFunc_; TIn* Request_; @@ -520,10 +520,10 @@ public: typename TBase::TOnRequest cb, typename TBase::TRequestCallback requestCallback, const char* name, - TLoggerPtr logger, + TLoggerPtr logger, ICounterBlockPtr counters, IGRpcRequestLimiterPtr limiter = nullptr) - : TBase{server, service, cq, std::move(cb), std::move(requestCallback), name, std::move(logger), std::move(counters), std::move(limiter)} + : TBase{server, service, cq, std::move(cb), std::move(requestCallback), name, std::move(logger), std::move(counters), std::move(limiter)} { } @@ -533,11 +533,11 @@ public: typename TBase::TOnRequest cb, typename TBase::TStreamRequestCallback requestCallback, const char* name, - TLoggerPtr logger, + TLoggerPtr logger, ICounterBlockPtr counters) - : TBase{server, service, cq, std::move(cb), std::move(requestCallback), name, std::move(logger), std::move(counters), nullptr} + : TBase{server, service, cq, std::move(cb), std::move(requestCallback), name, std::move(logger), std::move(counters), nullptr} { } }; -} // namespace NGrpc +} // namespace NGrpc diff --git a/library/cpp/grpc/server/grpc_request_base.h b/library/cpp/grpc/server/grpc_request_base.h index b61cf553aa7..fcfce1c181a 100644 --- a/library/cpp/grpc/server/grpc_request_base.h +++ b/library/cpp/grpc/server/grpc_request_base.h @@ -9,7 +9,7 @@ namespace grpc { class ByteBuffer; } -namespace NGrpc { +namespace NGrpc { extern const char* GRPC_USER_AGENT_HEADER; @@ -30,7 +30,7 @@ struct TAuthState { //! An interface that may be used to limit concurrency of requests -class IGRpcRequestLimiter: public TThrRefBase { +class IGRpcRequestLimiter: public TThrRefBase { public: virtual bool IncRequest() = 0; virtual void DecRequest() = 0; @@ -39,7 +39,7 @@ public: using IGRpcRequestLimiterPtr = TIntrusivePtr<IGRpcRequestLimiter>; //! State of current request -class IRequestContextBase: public TThrRefBase { +class IRequestContextBase: public TThrRefBase { public: enum class EFinishStatus { OK, @@ -72,12 +72,12 @@ public: //! Returns deadline (server epoch related) if peer set it on its side, or Instanse::Max() otherwise virtual TInstant Deadline() const = 0; - - //! Returns available peer metadata keys - virtual TSet<TStringBuf> GetPeerMetaKeys() const = 0; - + + //! Returns available peer metadata keys + virtual TSet<TStringBuf> GetPeerMetaKeys() const = 0; + //! Returns peer optional metavalue - virtual TVector<TStringBuf> GetPeerMetaValues(TStringBuf key) const = 0; + virtual TVector<TStringBuf> GetPeerMetaValues(TStringBuf key) const = 0; //! Returns request compression level virtual grpc_compression_level GetCompressionLevel() const = 0; @@ -113,4 +113,4 @@ public: virtual bool SslServer() const = 0; }; -} // namespace NGrpc +} // namespace NGrpc diff --git a/library/cpp/grpc/server/grpc_response.h b/library/cpp/grpc/server/grpc_response.h index 47b22c28d09..8e9afe44d53 100644 --- a/library/cpp/grpc/server/grpc_response.h +++ b/library/cpp/grpc/server/grpc_response.h @@ -1,90 +1,90 @@ -#pragma once - -#include <grpc++/impl/codegen/byte_buffer.h> -#include <grpc++/impl/codegen/proto_utils.h> - -#include <variant> - -namespace NGrpc { - -/** - * Universal response that owns underlying message or buffer. - */ -template <typename TMsg> +#pragma once + +#include <grpc++/impl/codegen/byte_buffer.h> +#include <grpc++/impl/codegen/proto_utils.h> + +#include <variant> + +namespace NGrpc { + +/** + * Universal response that owns underlying message or buffer. + */ +template <typename TMsg> class TUniversalResponse: public TAtomicRefCount<TUniversalResponse<TMsg>>, public TMoveOnly { - friend class grpc::SerializationTraits<TUniversalResponse<TMsg>>; - -public: - explicit TUniversalResponse(NProtoBuf::Message* msg) noexcept - : Data_{TMsg{}} - { - std::get<TMsg>(Data_).Swap(static_cast<TMsg*>(msg)); - } - - explicit TUniversalResponse(grpc::ByteBuffer* buffer) noexcept - : Data_{grpc::ByteBuffer{}} - { - std::get<grpc::ByteBuffer>(Data_).Swap(buffer); - } - -private: - std::variant<TMsg, grpc::ByteBuffer> Data_; -}; - -/** - * Universal response that only keeps reference to underlying message or buffer. - */ -template <typename TMsg> -class TUniversalResponseRef: private TMoveOnly { - friend class grpc::SerializationTraits<TUniversalResponseRef<TMsg>>; - -public: - explicit TUniversalResponseRef(const NProtoBuf::Message* msg) - : Data_{msg} - { - } - - explicit TUniversalResponseRef(const grpc::ByteBuffer* buffer) - : Data_{buffer} - { - } - -private: - std::variant<const NProtoBuf::Message*, const grpc::ByteBuffer*> Data_; -}; - -} // namespace NGrpc - -namespace grpc { - -template <typename TMsg> -class SerializationTraits<NGrpc::TUniversalResponse<TMsg>> { -public: - static Status Serialize( - const NGrpc::TUniversalResponse<TMsg>& resp, - ByteBuffer* buffer, - bool* ownBuffer) - { - return std::visit([&](const auto& data) { - using T = std::decay_t<decltype(data)>; - return SerializationTraits<T>::Serialize(data, buffer, ownBuffer); - }, resp.Data_); - } -}; - -template <typename TMsg> -class SerializationTraits<NGrpc::TUniversalResponseRef<TMsg>> { -public: - static Status Serialize( - const NGrpc::TUniversalResponseRef<TMsg>& resp, - ByteBuffer* buffer, - bool* ownBuffer) - { - return std::visit([&](const auto* data) { - using T = std::decay_t<std::remove_pointer_t<decltype(data)>>; - return SerializationTraits<T>::Serialize(*data, buffer, ownBuffer); - }, resp.Data_); - } -}; - -} // namespace grpc + friend class grpc::SerializationTraits<TUniversalResponse<TMsg>>; + +public: + explicit TUniversalResponse(NProtoBuf::Message* msg) noexcept + : Data_{TMsg{}} + { + std::get<TMsg>(Data_).Swap(static_cast<TMsg*>(msg)); + } + + explicit TUniversalResponse(grpc::ByteBuffer* buffer) noexcept + : Data_{grpc::ByteBuffer{}} + { + std::get<grpc::ByteBuffer>(Data_).Swap(buffer); + } + +private: + std::variant<TMsg, grpc::ByteBuffer> Data_; +}; + +/** + * Universal response that only keeps reference to underlying message or buffer. + */ +template <typename TMsg> +class TUniversalResponseRef: private TMoveOnly { + friend class grpc::SerializationTraits<TUniversalResponseRef<TMsg>>; + +public: + explicit TUniversalResponseRef(const NProtoBuf::Message* msg) + : Data_{msg} + { + } + + explicit TUniversalResponseRef(const grpc::ByteBuffer* buffer) + : Data_{buffer} + { + } + +private: + std::variant<const NProtoBuf::Message*, const grpc::ByteBuffer*> Data_; +}; + +} // namespace NGrpc + +namespace grpc { + +template <typename TMsg> +class SerializationTraits<NGrpc::TUniversalResponse<TMsg>> { +public: + static Status Serialize( + const NGrpc::TUniversalResponse<TMsg>& resp, + ByteBuffer* buffer, + bool* ownBuffer) + { + return std::visit([&](const auto& data) { + using T = std::decay_t<decltype(data)>; + return SerializationTraits<T>::Serialize(data, buffer, ownBuffer); + }, resp.Data_); + } +}; + +template <typename TMsg> +class SerializationTraits<NGrpc::TUniversalResponseRef<TMsg>> { +public: + static Status Serialize( + const NGrpc::TUniversalResponseRef<TMsg>& resp, + ByteBuffer* buffer, + bool* ownBuffer) + { + return std::visit([&](const auto* data) { + using T = std::decay_t<std::remove_pointer_t<decltype(data)>>; + return SerializationTraits<T>::Serialize(*data, buffer, ownBuffer); + }, resp.Data_); + } +}; + +} // namespace grpc diff --git a/library/cpp/grpc/server/grpc_server.cpp b/library/cpp/grpc/server/grpc_server.cpp index 4f4c7412fcd..7437b7a8f5e 100644 --- a/library/cpp/grpc/server/grpc_server.cpp +++ b/library/cpp/grpc/server/grpc_server.cpp @@ -15,7 +15,7 @@ #endif -namespace NGrpc { +namespace NGrpc { using NThreading::TFuture; @@ -82,7 +82,7 @@ void TGRpcServer::Start() { service->SetGlobalLimiterHandle(&Limiter_); } - class TKeepAliveOption: public grpc::ServerBuilderOption { + class TKeepAliveOption: public grpc::ServerBuilderOption { public: TKeepAliveOption(int idle, int interval) : Idle(idle) @@ -153,7 +153,7 @@ void TGRpcServer::Start() { size_t index = 0; for (IGRpcServicePtr service : Services_) { // TODO: provide something else for services instead of ServerCompletionQueue - service->InitService(CQS_[index++ % CQS_.size()].get(), Options_.Logger); + service->InitService(CQS_[index++ % CQS_.size()].get(), Options_.Logger); } if (Options_.UseCompletionQueuePerThread) { @@ -237,4 +237,4 @@ TString TGRpcServer::GetHost() const { return Options_.Host; } -} // namespace NGrpc +} // namespace NGrpc diff --git a/library/cpp/grpc/server/grpc_server.h b/library/cpp/grpc/server/grpc_server.h index 24c8caa78d1..d6814a90a0d 100644 --- a/library/cpp/grpc/server/grpc_server.h +++ b/library/cpp/grpc/server/grpc_server.h @@ -1,7 +1,7 @@ #pragma once #include "grpc_request_base.h" -#include "logger.h" +#include "logger.h" #include <library/cpp/threading/future/future.h> @@ -17,7 +17,7 @@ #include <grpc++/grpc++.h> -namespace NGrpc { +namespace NGrpc { constexpr ui64 DEFAULT_GRPC_MESSAGE_SIZE_LIMIT = 64000000; @@ -95,9 +95,9 @@ struct TServerOptions { DECLARE_FIELD(ExternalListener, IExternalListener::TPtr, nullptr); - //! Logger which will be used to write logs about requests handling (iff appropriate log level is enabled). - DECLARE_FIELD(Logger, TLoggerPtr, nullptr); - + //! Logger which will be used to write logs about requests handling (iff appropriate log level is enabled). + DECLARE_FIELD(Logger, TLoggerPtr, nullptr); + #undef DECLARE_FIELD }; @@ -161,11 +161,11 @@ private: using TGlobalLimiter = TInFlightLimiterImpl<i64>; -class IGRpcService: public TThrRefBase { +class IGRpcService: public TThrRefBase { public: virtual grpc::Service* GetService() = 0; virtual void StopService() noexcept = 0; - virtual void InitService(grpc::ServerCompletionQueue* cq, TLoggerPtr logger) = 0; + virtual void InitService(grpc::ServerCompletionQueue* cq, TLoggerPtr logger) = 0; virtual void SetGlobalLimiterHandle(TGlobalLimiter* limiter) = 0; virtual bool IsUnsafeToShutdown() const = 0; virtual size_t RequestsInProgress() const = 0; @@ -178,7 +178,7 @@ public: }; template<typename T> -class TGrpcServiceBase: public IGRpcService { +class TGrpcServiceBase: public IGRpcService { public: class TShutdownGuard { using TOwner = TGrpcServiceBase<T>; @@ -353,4 +353,4 @@ private: TGlobalLimiter Limiter_; }; -} // namespace NGrpc +} // namespace NGrpc diff --git a/library/cpp/grpc/server/logger.h b/library/cpp/grpc/server/logger.h index 5e44d83d67b..53af26be9c5 100644 --- a/library/cpp/grpc/server/logger.h +++ b/library/cpp/grpc/server/logger.h @@ -1,43 +1,43 @@ -#pragma once - -#include <library/cpp/logger/priority.h> - -#include <util/generic/ptr.h> - -namespace NGrpc { - -class TLogger: public TThrRefBase { -protected: - TLogger() = default; - -public: - [[nodiscard]] - bool IsEnabled(ELogPriority priority) const noexcept { - return DoIsEnabled(priority); - } - - void Y_PRINTF_FORMAT(3, 4) Write(ELogPriority priority, const char* format, ...) noexcept { - va_list args; - va_start(args, format); - DoWrite(priority, format, args); - va_end(args); - } - -protected: - virtual bool DoIsEnabled(ELogPriority priority) const noexcept = 0; - virtual void DoWrite(ELogPriority p, const char* format, va_list args) noexcept = 0; -}; - -using TLoggerPtr = TIntrusivePtr<TLogger>; - -#define GRPC_LOG_DEBUG(logger, format, ...) \ - if (logger && logger->IsEnabled(ELogPriority::TLOG_DEBUG)) { \ - logger->Write(ELogPriority::TLOG_DEBUG, format, __VA_ARGS__); \ - } else { } - -#define GRPC_LOG_INFO(logger, format, ...) \ - if (logger && logger->IsEnabled(ELogPriority::TLOG_INFO)) { \ - logger->Write(ELogPriority::TLOG_INFO, format, __VA_ARGS__); \ - } else { } - -} // namespace NGrpc +#pragma once + +#include <library/cpp/logger/priority.h> + +#include <util/generic/ptr.h> + +namespace NGrpc { + +class TLogger: public TThrRefBase { +protected: + TLogger() = default; + +public: + [[nodiscard]] + bool IsEnabled(ELogPriority priority) const noexcept { + return DoIsEnabled(priority); + } + + void Y_PRINTF_FORMAT(3, 4) Write(ELogPriority priority, const char* format, ...) noexcept { + va_list args; + va_start(args, format); + DoWrite(priority, format, args); + va_end(args); + } + +protected: + virtual bool DoIsEnabled(ELogPriority priority) const noexcept = 0; + virtual void DoWrite(ELogPriority p, const char* format, va_list args) noexcept = 0; +}; + +using TLoggerPtr = TIntrusivePtr<TLogger>; + +#define GRPC_LOG_DEBUG(logger, format, ...) \ + if (logger && logger->IsEnabled(ELogPriority::TLOG_DEBUG)) { \ + logger->Write(ELogPriority::TLOG_DEBUG, format, __VA_ARGS__); \ + } else { } + +#define GRPC_LOG_INFO(logger, format, ...) \ + if (logger && logger->IsEnabled(ELogPriority::TLOG_INFO)) { \ + logger->Write(ELogPriority::TLOG_INFO, format, __VA_ARGS__); \ + } else { } + +} // namespace NGrpc diff --git a/library/cpp/grpc/server/ut/grpc_response_ut.cpp b/library/cpp/grpc/server/ut/grpc_response_ut.cpp index cb66478e940..8abc4e4e0ec 100644 --- a/library/cpp/grpc/server/ut/grpc_response_ut.cpp +++ b/library/cpp/grpc/server/ut/grpc_response_ut.cpp @@ -1,88 +1,88 @@ -#include <library/cpp/grpc/server/grpc_response.h> -#include <library/cpp/testing/unittest/registar.h> - -#include <google/protobuf/duration.pb.h> -#include <grpc++/impl/codegen/proto_utils.h> -#include <grpc++/impl/grpc_library.h> - -static ::grpc::internal::GrpcLibraryInitializer grpcInitializer; - -using namespace NGrpc; - -using google::protobuf::Duration; - -Y_UNIT_TEST_SUITE(ResponseTest) { - - template <typename T> - grpc::ByteBuffer Serialize(T resp) { - grpc::ByteBuffer buf; - bool ownBuf = false; - grpc::Status status = grpc::SerializationTraits<T>::Serialize(resp, &buf, &ownBuf); - UNIT_ASSERT(status.ok()); - return buf; - } - - template <typename T> - T Deserialize(grpc::ByteBuffer* buf) { - T message; - auto status = grpc::SerializationTraits<T>::Deserialize(buf, &message); - UNIT_ASSERT(status.ok()); - return message; - } - - Y_UNIT_TEST(UniversalResponseMsg) { - Duration d1; - d1.set_seconds(12345); - d1.set_nanos(67890); - - auto buf = Serialize(TUniversalResponse<Duration>(&d1)); - Duration d2 = Deserialize<Duration>(&buf); - - UNIT_ASSERT_VALUES_EQUAL(d2.seconds(), 12345); - UNIT_ASSERT_VALUES_EQUAL(d2.nanos(), 67890); - } - - Y_UNIT_TEST(UniversalResponseBuf) { - Duration d1; - d1.set_seconds(123); - d1.set_nanos(456); - - TString data = d1.SerializeAsString(); - grpc::Slice dataSlice{data.data(), data.size()}; - grpc::ByteBuffer dataBuf{&dataSlice, 1}; - - auto buf = Serialize(TUniversalResponse<Duration>(&dataBuf)); - Duration d2 = Deserialize<Duration>(&buf); - - UNIT_ASSERT_VALUES_EQUAL(d2.seconds(), 123); - UNIT_ASSERT_VALUES_EQUAL(d2.nanos(), 456); - } - - Y_UNIT_TEST(UniversalResponseRefMsg) { - Duration d1; - d1.set_seconds(12345); - d1.set_nanos(67890); - - auto buf = Serialize(TUniversalResponseRef<Duration>(&d1)); - Duration d2 = Deserialize<Duration>(&buf); - - UNIT_ASSERT_VALUES_EQUAL(d2.seconds(), 12345); - UNIT_ASSERT_VALUES_EQUAL(d2.nanos(), 67890); - } - - Y_UNIT_TEST(UniversalResponseRefBuf) { - Duration d1; - d1.set_seconds(123); - d1.set_nanos(456); - - TString data = d1.SerializeAsString(); - grpc::Slice dataSlice{data.data(), data.size()}; - grpc::ByteBuffer dataBuf{&dataSlice, 1}; - - auto buf = Serialize(TUniversalResponseRef<Duration>(&dataBuf)); - Duration d2 = Deserialize<Duration>(&buf); - - UNIT_ASSERT_VALUES_EQUAL(d2.seconds(), 123); - UNIT_ASSERT_VALUES_EQUAL(d2.nanos(), 456); - } -} +#include <library/cpp/grpc/server/grpc_response.h> +#include <library/cpp/testing/unittest/registar.h> + +#include <google/protobuf/duration.pb.h> +#include <grpc++/impl/codegen/proto_utils.h> +#include <grpc++/impl/grpc_library.h> + +static ::grpc::internal::GrpcLibraryInitializer grpcInitializer; + +using namespace NGrpc; + +using google::protobuf::Duration; + +Y_UNIT_TEST_SUITE(ResponseTest) { + + template <typename T> + grpc::ByteBuffer Serialize(T resp) { + grpc::ByteBuffer buf; + bool ownBuf = false; + grpc::Status status = grpc::SerializationTraits<T>::Serialize(resp, &buf, &ownBuf); + UNIT_ASSERT(status.ok()); + return buf; + } + + template <typename T> + T Deserialize(grpc::ByteBuffer* buf) { + T message; + auto status = grpc::SerializationTraits<T>::Deserialize(buf, &message); + UNIT_ASSERT(status.ok()); + return message; + } + + Y_UNIT_TEST(UniversalResponseMsg) { + Duration d1; + d1.set_seconds(12345); + d1.set_nanos(67890); + + auto buf = Serialize(TUniversalResponse<Duration>(&d1)); + Duration d2 = Deserialize<Duration>(&buf); + + UNIT_ASSERT_VALUES_EQUAL(d2.seconds(), 12345); + UNIT_ASSERT_VALUES_EQUAL(d2.nanos(), 67890); + } + + Y_UNIT_TEST(UniversalResponseBuf) { + Duration d1; + d1.set_seconds(123); + d1.set_nanos(456); + + TString data = d1.SerializeAsString(); + grpc::Slice dataSlice{data.data(), data.size()}; + grpc::ByteBuffer dataBuf{&dataSlice, 1}; + + auto buf = Serialize(TUniversalResponse<Duration>(&dataBuf)); + Duration d2 = Deserialize<Duration>(&buf); + + UNIT_ASSERT_VALUES_EQUAL(d2.seconds(), 123); + UNIT_ASSERT_VALUES_EQUAL(d2.nanos(), 456); + } + + Y_UNIT_TEST(UniversalResponseRefMsg) { + Duration d1; + d1.set_seconds(12345); + d1.set_nanos(67890); + + auto buf = Serialize(TUniversalResponseRef<Duration>(&d1)); + Duration d2 = Deserialize<Duration>(&buf); + + UNIT_ASSERT_VALUES_EQUAL(d2.seconds(), 12345); + UNIT_ASSERT_VALUES_EQUAL(d2.nanos(), 67890); + } + + Y_UNIT_TEST(UniversalResponseRefBuf) { + Duration d1; + d1.set_seconds(123); + d1.set_nanos(456); + + TString data = d1.SerializeAsString(); + grpc::Slice dataSlice{data.data(), data.size()}; + grpc::ByteBuffer dataBuf{&dataSlice, 1}; + + auto buf = Serialize(TUniversalResponseRef<Duration>(&dataBuf)); + Duration d2 = Deserialize<Duration>(&buf); + + UNIT_ASSERT_VALUES_EQUAL(d2.seconds(), 123); + UNIT_ASSERT_VALUES_EQUAL(d2.nanos(), 456); + } +} diff --git a/library/cpp/grpc/server/ut/stream_adaptor_ut.cpp b/library/cpp/grpc/server/ut/stream_adaptor_ut.cpp index 3457e98bf18..c34d3b8c2bf 100644 --- a/library/cpp/grpc/server/ut/stream_adaptor_ut.cpp +++ b/library/cpp/grpc/server/ut/stream_adaptor_ut.cpp @@ -1,14 +1,14 @@ -#include <library/cpp/grpc/server/grpc_request.h> +#include <library/cpp/grpc/server/grpc_request.h> #include <library/cpp/testing/unittest/registar.h> #include <library/cpp/testing/unittest/tests_data.h> #include <util/system/thread.h> #include <util/thread/pool.h> -using namespace NGrpc; +using namespace NGrpc; // Here we emulate stream data producer -class TOrderedProducer: public TThread { +class TOrderedProducer: public TThread { public: TOrderedProducer(IStreamAdaptor* adaptor, ui64 max, bool withSleep, std::function<void(ui64)>&& consumerOp) : TThread(&ThreadProc, this) diff --git a/library/cpp/grpc/server/ut/ya.make b/library/cpp/grpc/server/ut/ya.make index ff6c8fdb7b7..feb3291af92 100644 --- a/library/cpp/grpc/server/ut/ya.make +++ b/library/cpp/grpc/server/ut/ya.make @@ -1,19 +1,19 @@ -UNITTEST_FOR(library/cpp/grpc/server) +UNITTEST_FOR(library/cpp/grpc/server) OWNER( dcherednik g:kikimr ) -TIMEOUT(600) -SIZE(MEDIUM) +TIMEOUT(600) +SIZE(MEDIUM) PEERDIR( - library/cpp/grpc/server + library/cpp/grpc/server ) SRCS( - grpc_response_ut.cpp + grpc_response_ut.cpp stream_adaptor_ut.cpp ) diff --git a/library/cpp/grpc/server/ya.make b/library/cpp/grpc/server/ya.make index b0f262e5dc6..356a1b6793d 100644 --- a/library/cpp/grpc/server/ya.make +++ b/library/cpp/grpc/server/ya.make @@ -16,10 +16,10 @@ GENERATE_ENUM_SERIALIZATION(grpc_request_base.h) PEERDIR( contrib/libs/grpc - library/cpp/monlib/dynamic_counters/percentile + library/cpp/monlib/dynamic_counters/percentile ) END() - -RECURSE_FOR_TESTS(ut) - + +RECURSE_FOR_TESTS(ut) + diff --git a/library/cpp/grpc/ya.make b/library/cpp/grpc/ya.make index 7d96e067d13..36351241159 100644 --- a/library/cpp/grpc/ya.make +++ b/library/cpp/grpc/ya.make @@ -1,5 +1,5 @@ RECURSE( client - common - server + common + server ) diff --git a/library/cpp/histogram/hdr/histogram.cpp b/library/cpp/histogram/hdr/histogram.cpp index 4e799ce61f4..a213d5d8fd3 100644 --- a/library/cpp/histogram/hdr/histogram.cpp +++ b/library/cpp/histogram/hdr/histogram.cpp @@ -1,155 +1,155 @@ -#include "histogram.h" - +#include "histogram.h" + #include <util/generic/cast.h> -#include <util/generic/yexception.h> - -#include <contrib/libs/hdr_histogram/src/hdr_histogram.h> - -namespace NHdr { - namespace { - struct hdr_histogram* CreateHistogram( - i64 lowestDiscernibleValue, i64 highestTrackableValue, - i32 numberOfSignificantValueDigits, IAllocator* allocator) { - struct hdr_histogram_bucket_config cfg; - - int r = hdr_calculate_bucket_config( - lowestDiscernibleValue, highestTrackableValue, - numberOfSignificantValueDigits, &cfg); - if (r) { - ythrow yexception() << "illegal arguments values"; - } - - size_t histogramSize = sizeof(struct hdr_histogram) + - cfg.counts_len * sizeof(i64); - - IAllocator::TBlock mem = allocator->Allocate(histogramSize); - struct hdr_histogram* histogram = +#include <util/generic/yexception.h> + +#include <contrib/libs/hdr_histogram/src/hdr_histogram.h> + +namespace NHdr { + namespace { + struct hdr_histogram* CreateHistogram( + i64 lowestDiscernibleValue, i64 highestTrackableValue, + i32 numberOfSignificantValueDigits, IAllocator* allocator) { + struct hdr_histogram_bucket_config cfg; + + int r = hdr_calculate_bucket_config( + lowestDiscernibleValue, highestTrackableValue, + numberOfSignificantValueDigits, &cfg); + if (r) { + ythrow yexception() << "illegal arguments values"; + } + + size_t histogramSize = sizeof(struct hdr_histogram) + + cfg.counts_len * sizeof(i64); + + IAllocator::TBlock mem = allocator->Allocate(histogramSize); + struct hdr_histogram* histogram = reinterpret_cast<struct hdr_histogram*>(mem.Data); - - // memset will ensure that all of the function pointers are null - memset(histogram, 0, histogramSize); - - hdr_init_preallocated(histogram, &cfg); - return histogram; - } - - } - - THistogram::THistogram(i64 lowestDiscernibleValue, i64 highestTrackableValue, - i32 numberOfSignificantValueDigits, IAllocator* allocator) - : Data_(CreateHistogram( - lowestDiscernibleValue, highestTrackableValue, - numberOfSignificantValueDigits, allocator)) - , Allocator_(allocator) - { - } - - THistogram::~THistogram() { - if (Data_) { - size_t size = GetMemorySize(); - Allocator_->Release({Data_.Release(), size}); - } - } - - // Histogram structure querying support ----------------------------------- - - i64 THistogram::GetLowestDiscernibleValue() const { - return Data_->lowest_trackable_value; - } - - i64 THistogram::GetHighestTrackableValue() const { - return Data_->highest_trackable_value; - } - - i32 THistogram::GetNumberOfSignificantValueDigits() const { - return Data_->significant_figures; - } - - size_t THistogram::GetMemorySize() const { - return hdr_get_memory_size(Data_.Get()); - } - - i32 THistogram::GetCountsLen() const { - return Data_->counts_len; - } - - i64 THistogram::GetTotalCount() const { - return Data_->total_count; - } - - // Value recording support ------------------------------------------------ - - bool THistogram::RecordValue(i64 value) { - return hdr_record_value(Data_.Get(), value); - } - - bool THistogram::RecordValues(i64 value, i64 count) { - return hdr_record_values(Data_.Get(), value, count); - } - - bool THistogram::RecordValueWithExpectedInterval(i64 value, i64 expectedInterval) { - return hdr_record_corrected_value(Data_.Get(), value, expectedInterval); - } - - bool THistogram::RecordValuesWithExpectedInterval( - i64 value, i64 count, i64 expectedInterval) { - return hdr_record_corrected_values( - Data_.Get(), value, count, expectedInterval); - } - - i64 THistogram::Add(const THistogram& rhs) { - return hdr_add(Data_.Get(), rhs.Data_.Get()); - } - - i64 THistogram::AddWithExpectedInterval(const THistogram& rhs, i64 expectedInterval) { - return hdr_add_while_correcting_for_coordinated_omission( - Data_.Get(), rhs.Data_.Get(), expectedInterval); - } - - // Histogram Data access support ------------------------------------------ - - i64 THistogram::GetMin() const { - return hdr_min(Data_.Get()); - } - - i64 THistogram::GetMax() const { - return hdr_max(Data_.Get()); - } - - double THistogram::GetMean() const { - return hdr_mean(Data_.Get()); - } - - double THistogram::GetStdDeviation() const { - return hdr_stddev(Data_.Get()); - } - - i64 THistogram::GetValueAtPercentile(double percentile) const { - return hdr_value_at_percentile(Data_.Get(), percentile); - } - - i64 THistogram::GetCountAtValue(i64 value) const { - return hdr_count_at_value(Data_.Get(), value); - } - - bool THistogram::ValuesAreEqual(i64 v1, i64 v2) const { - return hdr_values_are_equivalent(Data_.Get(), v1, v2); - } - - i64 THistogram::GetLowestEquivalentValue(i64 value) const { - return hdr_lowest_equivalent_value(Data_.Get(), value); - } - - i64 THistogram::GetHighestEquivalentValue(i64 value) const { - return hdr_next_non_equivalent_value(Data_.Get(), value) - 1; - } - - i64 THistogram::GetMedianEquivalentValue(i64 value) const { - return hdr_median_equivalent_value(Data_.Get(), value); - } - - void THistogram::Reset() { - hdr_reset(Data_.Get()); - } - -} + + // memset will ensure that all of the function pointers are null + memset(histogram, 0, histogramSize); + + hdr_init_preallocated(histogram, &cfg); + return histogram; + } + + } + + THistogram::THistogram(i64 lowestDiscernibleValue, i64 highestTrackableValue, + i32 numberOfSignificantValueDigits, IAllocator* allocator) + : Data_(CreateHistogram( + lowestDiscernibleValue, highestTrackableValue, + numberOfSignificantValueDigits, allocator)) + , Allocator_(allocator) + { + } + + THistogram::~THistogram() { + if (Data_) { + size_t size = GetMemorySize(); + Allocator_->Release({Data_.Release(), size}); + } + } + + // Histogram structure querying support ----------------------------------- + + i64 THistogram::GetLowestDiscernibleValue() const { + return Data_->lowest_trackable_value; + } + + i64 THistogram::GetHighestTrackableValue() const { + return Data_->highest_trackable_value; + } + + i32 THistogram::GetNumberOfSignificantValueDigits() const { + return Data_->significant_figures; + } + + size_t THistogram::GetMemorySize() const { + return hdr_get_memory_size(Data_.Get()); + } + + i32 THistogram::GetCountsLen() const { + return Data_->counts_len; + } + + i64 THistogram::GetTotalCount() const { + return Data_->total_count; + } + + // Value recording support ------------------------------------------------ + + bool THistogram::RecordValue(i64 value) { + return hdr_record_value(Data_.Get(), value); + } + + bool THistogram::RecordValues(i64 value, i64 count) { + return hdr_record_values(Data_.Get(), value, count); + } + + bool THistogram::RecordValueWithExpectedInterval(i64 value, i64 expectedInterval) { + return hdr_record_corrected_value(Data_.Get(), value, expectedInterval); + } + + bool THistogram::RecordValuesWithExpectedInterval( + i64 value, i64 count, i64 expectedInterval) { + return hdr_record_corrected_values( + Data_.Get(), value, count, expectedInterval); + } + + i64 THistogram::Add(const THistogram& rhs) { + return hdr_add(Data_.Get(), rhs.Data_.Get()); + } + + i64 THistogram::AddWithExpectedInterval(const THistogram& rhs, i64 expectedInterval) { + return hdr_add_while_correcting_for_coordinated_omission( + Data_.Get(), rhs.Data_.Get(), expectedInterval); + } + + // Histogram Data access support ------------------------------------------ + + i64 THistogram::GetMin() const { + return hdr_min(Data_.Get()); + } + + i64 THistogram::GetMax() const { + return hdr_max(Data_.Get()); + } + + double THistogram::GetMean() const { + return hdr_mean(Data_.Get()); + } + + double THistogram::GetStdDeviation() const { + return hdr_stddev(Data_.Get()); + } + + i64 THistogram::GetValueAtPercentile(double percentile) const { + return hdr_value_at_percentile(Data_.Get(), percentile); + } + + i64 THistogram::GetCountAtValue(i64 value) const { + return hdr_count_at_value(Data_.Get(), value); + } + + bool THistogram::ValuesAreEqual(i64 v1, i64 v2) const { + return hdr_values_are_equivalent(Data_.Get(), v1, v2); + } + + i64 THistogram::GetLowestEquivalentValue(i64 value) const { + return hdr_lowest_equivalent_value(Data_.Get(), value); + } + + i64 THistogram::GetHighestEquivalentValue(i64 value) const { + return hdr_next_non_equivalent_value(Data_.Get(), value) - 1; + } + + i64 THistogram::GetMedianEquivalentValue(i64 value) const { + return hdr_median_equivalent_value(Data_.Get(), value); + } + + void THistogram::Reset() { + hdr_reset(Data_.Get()); + } + +} diff --git a/library/cpp/histogram/hdr/histogram.h b/library/cpp/histogram/hdr/histogram.h index b1b27b4c4ff..5f1cebbd9f3 100644 --- a/library/cpp/histogram/hdr/histogram.h +++ b/library/cpp/histogram/hdr/histogram.h @@ -1,303 +1,303 @@ -#pragma once - -#include <util/generic/ptr.h> -#include <util/generic/noncopyable.h> -#include <util/memory/alloc.h> - -struct hdr_histogram; - -namespace NHdr { - /** - * A High Dynamic Range (HDR) Histogram - * - * THdrHistogram supports the recording and analyzing sampled data value counts - * across a configurable integer value range with configurable value precision - * within the range. Value precision is expressed as the number of significant - * digits in the value recording, and provides control over value quantization - * behavior across the value range and the subsequent value resolution at any - * given level. - */ - class THistogram: public TMoveOnly { - public: - /** - * Construct a histogram given the Highest value to be tracked and a number - * of significant decimal digits. The histogram will be constructed to - * implicitly track (distinguish from 0) values as low as 1. Default - * allocator will be used to allocate underlying memory. - * - * @param highestTrackableValue The highest value to be tracked by the - * histogram. Must be a positive integer that is literal >= 2. - * - * @param numberOfSignificantValueDigits Specifies the precision to use. - * This is the number of significant decimal digits to which the - * histogram will maintain value resolution and separation. Must be - * a non-negative integer between 0 and 5. - */ - THistogram(i64 highestTrackableValue, i32 numberOfSignificantValueDigits) - : THistogram(1, highestTrackableValue, numberOfSignificantValueDigits) - { - } - - /** - * Construct a histogram given the Lowest and Highest values to be tracked - * and a number of significant decimal digits. Providing a - * lowestDiscernibleValue is useful in situations where the units used for - * the histogram's values are much smaller that the minimal accuracy - * required. E.g. when tracking time values stated in nanosecond units, - * where the minimal accuracy required is a microsecond, the proper value - * for lowestDiscernibleValue would be 1000. - * - * @param lowestDiscernibleValue The lowest value that can be discerned - * (distinguished from 0) by the histogram. Must be a positive - * integer that is >= 1. May be internally rounded down to nearest - * power of 2. - * - * @param highestTrackableValue The highest value to be tracked by the - * histogram. Must be a positive integer that is - * >= (2 * lowestDiscernibleValue). - * - * @param numberOfSignificantValueDigits Specifies the precision to use. - * This is the number of significant decimal digits to which the - * histogram will maintain value resolution and separation. Must be - * a non-negative integer between 0 and 5. - * - * @param allocator Specifies allocator which will be used to allocate - * memory for histogram. - */ - THistogram(i64 lowestDiscernibleValue, i64 highestTrackableValue, - i32 numberOfSignificantValueDigits, - IAllocator* allocator = TDefaultAllocator::Instance()); - - ~THistogram(); - - // Histogram structure querying support ----------------------------------- - - /** - * @return The configured lowestDiscernibleValue - */ - i64 GetLowestDiscernibleValue() const; - - /** - * @return The configured highestTrackableValue - */ - i64 GetHighestTrackableValue() const; - - /** - * @return The configured numberOfSignificantValueDigits - */ - i32 GetNumberOfSignificantValueDigits() const; - - /** - * @return The size of allocated memory for histogram - */ - size_t GetMemorySize() const; - - /** - * @return The number of created counters - */ - i32 GetCountsLen() const; - - /** - * @return The total count of all recorded values in the histogram - */ - i64 GetTotalCount() const; - - // Value recording support ------------------------------------------------ - - /** - * Records a value in the histogram, will round this value of to a - * precision at or better than the NumberOfSignificantValueDigits specified - * at construction time. - * - * @param value Value to add to the histogram - * @return false if the value is larger than the HighestTrackableValue - * and can't be recorded, true otherwise. - */ - bool RecordValue(i64 value); - - /** - * Records count values in the histogram, will round this value of to a - * precision at or better than the NumberOfSignificantValueDigits specified - * at construction time. - * - * @param value Value to add to the histogram - * @param count Number of values to add to the histogram - * @return false if the value is larger than the HighestTrackableValue - * and can't be recorded, true otherwise. - */ - bool RecordValues(i64 value, i64 count); - - /** - * Records a value in the histogram and backfill based on an expected - * interval. Value will be rounded this to a precision at or better - * than the NumberOfSignificantValueDigits specified at contruction time. - * This is specifically used for recording latency. If the value is larger - * than the expectedInterval then the latency recording system has - * experienced co-ordinated omission. This method fills in the values that - * would have occured had the client providing the load not been blocked. - * - * @param value Value to add to the histogram - * @param expectedInterval The delay between recording values - * @return false if the value is larger than the HighestTrackableValue - * and can't be recorded, true otherwise. - */ - bool RecordValueWithExpectedInterval(i64 value, i64 expectedInterval); - - /** - * Record a value in the histogram count times. Applies the same correcting - * logic as {@link THistogram::RecordValueWithExpectedInterval}. - * - * @param value Value to add to the histogram - * @param count Number of values to add to the histogram - * @param expectedInterval The delay between recording values. - * @return false if the value is larger than the HighestTrackableValue - * and can't be recorded, true otherwise. - */ - bool RecordValuesWithExpectedInterval( - i64 value, i64 count, i64 expectedInterval); - - /** - * Adds all of the values from rhs to this histogram. Will return the - * number of values that are dropped when copying. Values will be dropped - * if they around outside of [LowestDiscernibleValue, GetHighestTrackableValue]. - * - * @param rhs Histogram to copy values from. - * @return The number of values dropped when copying. - */ - i64 Add(const THistogram& rhs); - - /** - * Adds all of the values from rhs to this histogram. Will return the - * number of values that are dropped when copying. Values will be dropped - * if they around outside of [LowestDiscernibleValue, GetHighestTrackableValue]. - * Applies the same correcting logic as - * {@link THistogram::RecordValueWithExpectedInterval}. - * - * @param rhs Histogram to copy values from. - * @return The number of values dropped when copying. - */ - i64 AddWithExpectedInterval(const THistogram& rhs, i64 expectedInterval); - - // Histogram Data access support ------------------------------------------ - - /** - * Get the lowest recorded value level in the histogram. If the histogram - * has no recorded values, the value returned is undefined. - * - * @return the Min value recorded in the histogram - */ - i64 GetMin() const; - - /** - * Get the highest recorded value level in the histogram. If the histogram - * has no recorded values, the value returned is undefined. - * - * @return the Max value recorded in the histogram - */ - i64 GetMax() const; - - /** - * Get the computed mean value of all recorded values in the histogram - * - * @return the mean value (in value units) of the histogram data - */ - double GetMean() const; - - /** - * Get the computed standard deviation of all recorded values in the histogram - * - * @return the standard deviation (in value units) of the histogram data - */ - double GetStdDeviation() const; - - /** - * Get the value at a given percentile. - * Note that two values are "equivalent" in this statement if - * {@link THistogram::ValuesAreEquivalent} would return true. - * - * @param percentile The percentile for which to return the associated - * value - * @return The value that the given percentage of the overall recorded - * value entries in the histogram are either smaller than or - * equivalent to. When the percentile is 0.0, returns the value - * that all value entries in the histogram are either larger than - * or equivalent to. - */ - i64 GetValueAtPercentile(double percentile) const; - - /** - * Get the count of recorded values at a specific value (to within the - * histogram resolution at the value level). - * - * @param value The value for which to provide the recorded count - * @return The total count of values recorded in the histogram within the - * value range that is >= GetLowestEquivalentValue(value) and - * <= GetHighestEquivalentValue(value) - */ - i64 GetCountAtValue(i64 value) const; - - /** - * Determine if two values are equivalent with the histogram's resolution. - * Where "equivalent" means that value samples recorded for any two - * equivalent values are counted in a common total count. - * - * @param v1 first value to compare - * @param v2 second value to compare - * @return True if values are equivalent with the histogram's resolution. - */ - bool ValuesAreEqual(i64 v1, i64 v2) const; - - /** - * Get the lowest value that is equivalent to the given value within the - * histogram's resolution. Where "equivalent" means that value samples - * recorded for any two equivalent values are counted in a common total - * count. - * - * @param value The given value - * @return The lowest value that is equivalent to the given value within - * the histogram's resolution. - */ - i64 GetLowestEquivalentValue(i64 value) const; - - /** - * Get the highest value that is equivalent to the given value within the - * histogram's resolution. Where "equivalent" means that value samples - * recorded for any two equivalent values are counted in a common total - * count. - * - * @param value The given value - * @return The highest value that is equivalent to the given value within - * the histogram's resolution. - */ - i64 GetHighestEquivalentValue(i64 value) const; - - /** - * Get a value that lies in the middle (rounded up) of the range of values - * equivalent the given value. Where "equivalent" means that value samples - * recorded for any two equivalent values are counted in a common total - * count. - * - * @param value The given value - * @return The value lies in the middle (rounded up) of the range of values - * equivalent the given value. - */ - i64 GetMedianEquivalentValue(i64 value) const; - - // misc functions --------------------------------------------------------- - - /** - * Reset a histogram to zero - empty out a histogram and re-initialise it. - * If you want to re-use an existing histogram, but reset everything back - * to zero, this is the routine to use. - */ - void Reset(); - - const hdr_histogram* GetHdrHistogramImpl() const { - return Data_.Get(); - } - - private: - THolder<hdr_histogram> Data_; - IAllocator* Allocator_; - }; -} +#pragma once + +#include <util/generic/ptr.h> +#include <util/generic/noncopyable.h> +#include <util/memory/alloc.h> + +struct hdr_histogram; + +namespace NHdr { + /** + * A High Dynamic Range (HDR) Histogram + * + * THdrHistogram supports the recording and analyzing sampled data value counts + * across a configurable integer value range with configurable value precision + * within the range. Value precision is expressed as the number of significant + * digits in the value recording, and provides control over value quantization + * behavior across the value range and the subsequent value resolution at any + * given level. + */ + class THistogram: public TMoveOnly { + public: + /** + * Construct a histogram given the Highest value to be tracked and a number + * of significant decimal digits. The histogram will be constructed to + * implicitly track (distinguish from 0) values as low as 1. Default + * allocator will be used to allocate underlying memory. + * + * @param highestTrackableValue The highest value to be tracked by the + * histogram. Must be a positive integer that is literal >= 2. + * + * @param numberOfSignificantValueDigits Specifies the precision to use. + * This is the number of significant decimal digits to which the + * histogram will maintain value resolution and separation. Must be + * a non-negative integer between 0 and 5. + */ + THistogram(i64 highestTrackableValue, i32 numberOfSignificantValueDigits) + : THistogram(1, highestTrackableValue, numberOfSignificantValueDigits) + { + } + + /** + * Construct a histogram given the Lowest and Highest values to be tracked + * and a number of significant decimal digits. Providing a + * lowestDiscernibleValue is useful in situations where the units used for + * the histogram's values are much smaller that the minimal accuracy + * required. E.g. when tracking time values stated in nanosecond units, + * where the minimal accuracy required is a microsecond, the proper value + * for lowestDiscernibleValue would be 1000. + * + * @param lowestDiscernibleValue The lowest value that can be discerned + * (distinguished from 0) by the histogram. Must be a positive + * integer that is >= 1. May be internally rounded down to nearest + * power of 2. + * + * @param highestTrackableValue The highest value to be tracked by the + * histogram. Must be a positive integer that is + * >= (2 * lowestDiscernibleValue). + * + * @param numberOfSignificantValueDigits Specifies the precision to use. + * This is the number of significant decimal digits to which the + * histogram will maintain value resolution and separation. Must be + * a non-negative integer between 0 and 5. + * + * @param allocator Specifies allocator which will be used to allocate + * memory for histogram. + */ + THistogram(i64 lowestDiscernibleValue, i64 highestTrackableValue, + i32 numberOfSignificantValueDigits, + IAllocator* allocator = TDefaultAllocator::Instance()); + + ~THistogram(); + + // Histogram structure querying support ----------------------------------- + + /** + * @return The configured lowestDiscernibleValue + */ + i64 GetLowestDiscernibleValue() const; + + /** + * @return The configured highestTrackableValue + */ + i64 GetHighestTrackableValue() const; + + /** + * @return The configured numberOfSignificantValueDigits + */ + i32 GetNumberOfSignificantValueDigits() const; + + /** + * @return The size of allocated memory for histogram + */ + size_t GetMemorySize() const; + + /** + * @return The number of created counters + */ + i32 GetCountsLen() const; + + /** + * @return The total count of all recorded values in the histogram + */ + i64 GetTotalCount() const; + + // Value recording support ------------------------------------------------ + + /** + * Records a value in the histogram, will round this value of to a + * precision at or better than the NumberOfSignificantValueDigits specified + * at construction time. + * + * @param value Value to add to the histogram + * @return false if the value is larger than the HighestTrackableValue + * and can't be recorded, true otherwise. + */ + bool RecordValue(i64 value); + + /** + * Records count values in the histogram, will round this value of to a + * precision at or better than the NumberOfSignificantValueDigits specified + * at construction time. + * + * @param value Value to add to the histogram + * @param count Number of values to add to the histogram + * @return false if the value is larger than the HighestTrackableValue + * and can't be recorded, true otherwise. + */ + bool RecordValues(i64 value, i64 count); + + /** + * Records a value in the histogram and backfill based on an expected + * interval. Value will be rounded this to a precision at or better + * than the NumberOfSignificantValueDigits specified at contruction time. + * This is specifically used for recording latency. If the value is larger + * than the expectedInterval then the latency recording system has + * experienced co-ordinated omission. This method fills in the values that + * would have occured had the client providing the load not been blocked. + * + * @param value Value to add to the histogram + * @param expectedInterval The delay between recording values + * @return false if the value is larger than the HighestTrackableValue + * and can't be recorded, true otherwise. + */ + bool RecordValueWithExpectedInterval(i64 value, i64 expectedInterval); + + /** + * Record a value in the histogram count times. Applies the same correcting + * logic as {@link THistogram::RecordValueWithExpectedInterval}. + * + * @param value Value to add to the histogram + * @param count Number of values to add to the histogram + * @param expectedInterval The delay between recording values. + * @return false if the value is larger than the HighestTrackableValue + * and can't be recorded, true otherwise. + */ + bool RecordValuesWithExpectedInterval( + i64 value, i64 count, i64 expectedInterval); + + /** + * Adds all of the values from rhs to this histogram. Will return the + * number of values that are dropped when copying. Values will be dropped + * if they around outside of [LowestDiscernibleValue, GetHighestTrackableValue]. + * + * @param rhs Histogram to copy values from. + * @return The number of values dropped when copying. + */ + i64 Add(const THistogram& rhs); + + /** + * Adds all of the values from rhs to this histogram. Will return the + * number of values that are dropped when copying. Values will be dropped + * if they around outside of [LowestDiscernibleValue, GetHighestTrackableValue]. + * Applies the same correcting logic as + * {@link THistogram::RecordValueWithExpectedInterval}. + * + * @param rhs Histogram to copy values from. + * @return The number of values dropped when copying. + */ + i64 AddWithExpectedInterval(const THistogram& rhs, i64 expectedInterval); + + // Histogram Data access support ------------------------------------------ + + /** + * Get the lowest recorded value level in the histogram. If the histogram + * has no recorded values, the value returned is undefined. + * + * @return the Min value recorded in the histogram + */ + i64 GetMin() const; + + /** + * Get the highest recorded value level in the histogram. If the histogram + * has no recorded values, the value returned is undefined. + * + * @return the Max value recorded in the histogram + */ + i64 GetMax() const; + + /** + * Get the computed mean value of all recorded values in the histogram + * + * @return the mean value (in value units) of the histogram data + */ + double GetMean() const; + + /** + * Get the computed standard deviation of all recorded values in the histogram + * + * @return the standard deviation (in value units) of the histogram data + */ + double GetStdDeviation() const; + + /** + * Get the value at a given percentile. + * Note that two values are "equivalent" in this statement if + * {@link THistogram::ValuesAreEquivalent} would return true. + * + * @param percentile The percentile for which to return the associated + * value + * @return The value that the given percentage of the overall recorded + * value entries in the histogram are either smaller than or + * equivalent to. When the percentile is 0.0, returns the value + * that all value entries in the histogram are either larger than + * or equivalent to. + */ + i64 GetValueAtPercentile(double percentile) const; + + /** + * Get the count of recorded values at a specific value (to within the + * histogram resolution at the value level). + * + * @param value The value for which to provide the recorded count + * @return The total count of values recorded in the histogram within the + * value range that is >= GetLowestEquivalentValue(value) and + * <= GetHighestEquivalentValue(value) + */ + i64 GetCountAtValue(i64 value) const; + + /** + * Determine if two values are equivalent with the histogram's resolution. + * Where "equivalent" means that value samples recorded for any two + * equivalent values are counted in a common total count. + * + * @param v1 first value to compare + * @param v2 second value to compare + * @return True if values are equivalent with the histogram's resolution. + */ + bool ValuesAreEqual(i64 v1, i64 v2) const; + + /** + * Get the lowest value that is equivalent to the given value within the + * histogram's resolution. Where "equivalent" means that value samples + * recorded for any two equivalent values are counted in a common total + * count. + * + * @param value The given value + * @return The lowest value that is equivalent to the given value within + * the histogram's resolution. + */ + i64 GetLowestEquivalentValue(i64 value) const; + + /** + * Get the highest value that is equivalent to the given value within the + * histogram's resolution. Where "equivalent" means that value samples + * recorded for any two equivalent values are counted in a common total + * count. + * + * @param value The given value + * @return The highest value that is equivalent to the given value within + * the histogram's resolution. + */ + i64 GetHighestEquivalentValue(i64 value) const; + + /** + * Get a value that lies in the middle (rounded up) of the range of values + * equivalent the given value. Where "equivalent" means that value samples + * recorded for any two equivalent values are counted in a common total + * count. + * + * @param value The given value + * @return The value lies in the middle (rounded up) of the range of values + * equivalent the given value. + */ + i64 GetMedianEquivalentValue(i64 value) const; + + // misc functions --------------------------------------------------------- + + /** + * Reset a histogram to zero - empty out a histogram and re-initialise it. + * If you want to re-use an existing histogram, but reset everything back + * to zero, this is the routine to use. + */ + void Reset(); + + const hdr_histogram* GetHdrHistogramImpl() const { + return Data_.Get(); + } + + private: + THolder<hdr_histogram> Data_; + IAllocator* Allocator_; + }; +} diff --git a/library/cpp/histogram/hdr/histogram_iter.cpp b/library/cpp/histogram/hdr/histogram_iter.cpp index 4975348e225..d251fd5dd9b 100644 --- a/library/cpp/histogram/hdr/histogram_iter.cpp +++ b/library/cpp/histogram/hdr/histogram_iter.cpp @@ -1,146 +1,146 @@ -#include "histogram_iter.h" - -#include <contrib/libs/hdr_histogram/src/hdr_histogram.h> - -namespace NHdr { - // TBaseHistogramIterator ----------------------------------------------------- - TBaseHistogramIterator::TBaseHistogramIterator() - : Iter_(new hdr_iter) - { - } - - TBaseHistogramIterator::~TBaseHistogramIterator() { - } - - bool TBaseHistogramIterator::Next() { - return hdr_iter_next(Iter_.Get()); - } - - i32 TBaseHistogramIterator::GetCountsIndex() const { - return Iter_->counts_index; - } - - i32 TBaseHistogramIterator::GetTotalCount() const { - return Iter_->total_count; - } - - i64 TBaseHistogramIterator::GetCount() const { - return Iter_->count; - } - - i64 TBaseHistogramIterator::GetCumulativeCount() const { - return Iter_->cumulative_count; - } - - i64 TBaseHistogramIterator::GetValue() const { - return Iter_->value; - } - - i64 TBaseHistogramIterator::GetHighestEquivalentValue() const { - return Iter_->highest_equivalent_value; - } - - i64 TBaseHistogramIterator::GetLowestEquivalentValue() const { - return Iter_->lowest_equivalent_value; - } - - i64 TBaseHistogramIterator::GetMedianEquivalentValue() const { - return Iter_->median_equivalent_value; - } - - i64 TBaseHistogramIterator::GetValueIteratedFrom() const { - return Iter_->value_iterated_from; - } - - i64 TBaseHistogramIterator::GetValueIteratedTo() const { - return Iter_->value_iterated_to; - } - - // TAllValuesIterator --------------------------------------------------------- - - TAllValuesIterator::TAllValuesIterator(const THistogram& histogram) { - hdr_iter_init(Iter_.Get(), histogram.GetHdrHistogramImpl()); - } - - // TRecordedValuesIterator ---------------------------------------------------- - - TRecordedValuesIterator::TRecordedValuesIterator(const THistogram& histogram) { - hdr_iter_recorded_init(Iter_.Get(), histogram.GetHdrHistogramImpl()); - } - - i64 TRecordedValuesIterator::GetCountAddedInThisIterationStep() const { - return Iter_->specifics.recorded.count_added_in_this_iteration_step; - } - - // TPercentileIterator -------------------------------------------------------- - - TPercentileIterator::TPercentileIterator( - const THistogram& histogram, ui32 ticksPerHalfDistance) { - hdr_iter_percentile_init( - Iter_.Get(), histogram.GetHdrHistogramImpl(), - ticksPerHalfDistance); - } - - i32 TPercentileIterator::GetTicketsPerHalfDistance() const { - return Iter_->specifics.percentiles.ticks_per_half_distance; - } - - double TPercentileIterator::GetPercentileToIterateTo() const { - return Iter_->specifics.percentiles.percentile_to_iterate_to; - } - - double TPercentileIterator::GetPercentile() const { - return Iter_->specifics.percentiles.percentile; - } - - // TLinearIterator ------------------------------------------------------------ - - TLinearIterator::TLinearIterator( - const THistogram& histogram, i64 valueUnitsPerBucket) { - hdr_iter_linear_init( - Iter_.Get(), histogram.GetHdrHistogramImpl(), valueUnitsPerBucket); - } - - i64 TLinearIterator::GetValueUnitsPerBucket() const { - return Iter_->specifics.linear.value_units_per_bucket; - } - - i64 TLinearIterator::GetCountAddedInThisIterationStep() const { - return Iter_->specifics.linear.count_added_in_this_iteration_step; - } - - i64 TLinearIterator::GetNextValueReportingLevel() const { - return Iter_->specifics.linear.next_value_reporting_level; - } - - i64 TLinearIterator::GetNextValueReportingLevelLowestEquivalent() const { - return Iter_->specifics.linear.next_value_reporting_level_lowest_equivalent; - } - - // TLogarithmicIterator ------------------------------------------------------- - - TLogarithmicIterator::TLogarithmicIterator( - const THistogram& histogram, i64 valueUnitsInFirstBucket, - double logBase) { - hdr_iter_log_init( - Iter_.Get(), histogram.GetHdrHistogramImpl(), - valueUnitsInFirstBucket, logBase); - } - - double TLogarithmicIterator::GetLogBase() const { - return Iter_->specifics.log.log_base; - } - - i64 TLogarithmicIterator::GetCountAddedInThisIterationStep() const { - return Iter_->specifics.log.count_added_in_this_iteration_step; - } - - i64 TLogarithmicIterator::GetNextValueReportingLevel() const { - return Iter_->specifics.log.next_value_reporting_level; - } - - i64 TLogarithmicIterator::GetNextValueReportingLevelLowestEquivalent() const { - return Iter_->specifics.log.next_value_reporting_level_lowest_equivalent; - } - -} +#include "histogram_iter.h" + +#include <contrib/libs/hdr_histogram/src/hdr_histogram.h> + +namespace NHdr { + // TBaseHistogramIterator ----------------------------------------------------- + TBaseHistogramIterator::TBaseHistogramIterator() + : Iter_(new hdr_iter) + { + } + + TBaseHistogramIterator::~TBaseHistogramIterator() { + } + + bool TBaseHistogramIterator::Next() { + return hdr_iter_next(Iter_.Get()); + } + + i32 TBaseHistogramIterator::GetCountsIndex() const { + return Iter_->counts_index; + } + + i32 TBaseHistogramIterator::GetTotalCount() const { + return Iter_->total_count; + } + + i64 TBaseHistogramIterator::GetCount() const { + return Iter_->count; + } + + i64 TBaseHistogramIterator::GetCumulativeCount() const { + return Iter_->cumulative_count; + } + + i64 TBaseHistogramIterator::GetValue() const { + return Iter_->value; + } + + i64 TBaseHistogramIterator::GetHighestEquivalentValue() const { + return Iter_->highest_equivalent_value; + } + + i64 TBaseHistogramIterator::GetLowestEquivalentValue() const { + return Iter_->lowest_equivalent_value; + } + + i64 TBaseHistogramIterator::GetMedianEquivalentValue() const { + return Iter_->median_equivalent_value; + } + + i64 TBaseHistogramIterator::GetValueIteratedFrom() const { + return Iter_->value_iterated_from; + } + + i64 TBaseHistogramIterator::GetValueIteratedTo() const { + return Iter_->value_iterated_to; + } + + // TAllValuesIterator --------------------------------------------------------- + + TAllValuesIterator::TAllValuesIterator(const THistogram& histogram) { + hdr_iter_init(Iter_.Get(), histogram.GetHdrHistogramImpl()); + } + + // TRecordedValuesIterator ---------------------------------------------------- + + TRecordedValuesIterator::TRecordedValuesIterator(const THistogram& histogram) { + hdr_iter_recorded_init(Iter_.Get(), histogram.GetHdrHistogramImpl()); + } + + i64 TRecordedValuesIterator::GetCountAddedInThisIterationStep() const { + return Iter_->specifics.recorded.count_added_in_this_iteration_step; + } + + // TPercentileIterator -------------------------------------------------------- + + TPercentileIterator::TPercentileIterator( + const THistogram& histogram, ui32 ticksPerHalfDistance) { + hdr_iter_percentile_init( + Iter_.Get(), histogram.GetHdrHistogramImpl(), + ticksPerHalfDistance); + } + + i32 TPercentileIterator::GetTicketsPerHalfDistance() const { + return Iter_->specifics.percentiles.ticks_per_half_distance; + } + + double TPercentileIterator::GetPercentileToIterateTo() const { + return Iter_->specifics.percentiles.percentile_to_iterate_to; + } + + double TPercentileIterator::GetPercentile() const { + return Iter_->specifics.percentiles.percentile; + } + + // TLinearIterator ------------------------------------------------------------ + + TLinearIterator::TLinearIterator( + const THistogram& histogram, i64 valueUnitsPerBucket) { + hdr_iter_linear_init( + Iter_.Get(), histogram.GetHdrHistogramImpl(), valueUnitsPerBucket); + } + + i64 TLinearIterator::GetValueUnitsPerBucket() const { + return Iter_->specifics.linear.value_units_per_bucket; + } + + i64 TLinearIterator::GetCountAddedInThisIterationStep() const { + return Iter_->specifics.linear.count_added_in_this_iteration_step; + } + + i64 TLinearIterator::GetNextValueReportingLevel() const { + return Iter_->specifics.linear.next_value_reporting_level; + } + + i64 TLinearIterator::GetNextValueReportingLevelLowestEquivalent() const { + return Iter_->specifics.linear.next_value_reporting_level_lowest_equivalent; + } + + // TLogarithmicIterator ------------------------------------------------------- + + TLogarithmicIterator::TLogarithmicIterator( + const THistogram& histogram, i64 valueUnitsInFirstBucket, + double logBase) { + hdr_iter_log_init( + Iter_.Get(), histogram.GetHdrHistogramImpl(), + valueUnitsInFirstBucket, logBase); + } + + double TLogarithmicIterator::GetLogBase() const { + return Iter_->specifics.log.log_base; + } + + i64 TLogarithmicIterator::GetCountAddedInThisIterationStep() const { + return Iter_->specifics.log.count_added_in_this_iteration_step; + } + + i64 TLogarithmicIterator::GetNextValueReportingLevel() const { + return Iter_->specifics.log.next_value_reporting_level; + } + + i64 TLogarithmicIterator::GetNextValueReportingLevelLowestEquivalent() const { + return Iter_->specifics.log.next_value_reporting_level_lowest_equivalent; + } + +} diff --git a/library/cpp/histogram/hdr/histogram_iter.h b/library/cpp/histogram/hdr/histogram_iter.h index 05aa869e812..adfc1616e3c 100644 --- a/library/cpp/histogram/hdr/histogram_iter.h +++ b/library/cpp/histogram/hdr/histogram_iter.h @@ -1,231 +1,231 @@ -#pragma once - -#include "histogram.h" - -struct hdr_iter; - -namespace NHdr { - /** - * Used for iterating through histogram values. - */ - class TBaseHistogramIterator { - public: - /** - * Iterate to the next value for the iterator. If there are no more values - * available return false. - * - * @return 'false' if there are no values remaining for this iterator. - */ - bool Next(); - - /** - * @return Raw index into the counts array. - */ - i32 GetCountsIndex() const; - - /** - * @return Snapshot of the length at the time the iterator is created. - */ - i32 GetTotalCount() const; - - /** - * @return Value directly from array for the current countsIndex. - */ - i64 GetCount() const; - - /** - * @return Sum of all of the counts up to and including the count at - * this index. - */ - i64 GetCumulativeCount() const; - - /** - * @return The current value based on countsIndex. - */ - i64 GetValue() const; - - /** - * @return The highest value that is equivalent to the current value - * within the histogram's resolution. - */ - i64 GetHighestEquivalentValue() const; - - /** - * @return The lowest value that is equivalent to the current value - * within the histogram's resolution. - */ - i64 GetLowestEquivalentValue() const; - - /** - * @return The value lies in the middle (rounded up) of the range of - * values equivalent the current value. - */ - i64 GetMedianEquivalentValue() const; - - /** - * @return The actual value level that was iterated from by the iterator. - */ - i64 GetValueIteratedFrom() const; - - /** - * @return The actual value level that was iterated to by the iterator. - */ - i64 GetValueIteratedTo() const; - - protected: - // must not be instantiated directly - TBaseHistogramIterator(); - ~TBaseHistogramIterator(); - - protected: - THolder<hdr_iter> Iter_; - }; - - /** - * Used for iterating through histogram values using the finest granularity - * steps supported by the underlying representation. The iteration steps - * through all possible unit value levels, regardless of whether or not there - * were recorded values for that value level, and terminates when all recorded - * histogram values are exhausted. - */ - class TAllValuesIterator: public TBaseHistogramIterator { - public: - /** - * @param histogram The histogram this iterator will operate on - */ - explicit TAllValuesIterator(const THistogram& histogram); - }; - - /** - * Used for iterating through all recorded histogram values using the finest - * granularity steps supported by the underlying representation. The iteration - * steps through all non-zero recorded value counts, and terminates when all - * recorded histogram values are exhausted. - */ - class TRecordedValuesIterator: public TBaseHistogramIterator { - public: - /** - * @param histogram The histogram this iterator will operate on - */ - explicit TRecordedValuesIterator(const THistogram& histogram); - - /** - * @return The count of recorded values in the histogram that were added - * to the totalCount as a result on this iteration step. Since - * multiple iteration steps may occur with overlapping equivalent - * value ranges, the count may be lower than the count found at - * the value (e.g. multiple linear steps or percentile levels can - * occur within a single equivalent value range). - */ - i64 GetCountAddedInThisIterationStep() const; - }; - - /** - * Used for iterating through histogram values according to percentile levels. - * The iteration is performed in steps that start at 0% and reduce their - * distance to 100% according to the <i>percentileTicksPerHalfDistance</i> - * parameter, ultimately reaching 100% when all recorded histogram - * values are exhausted. - */ - class TPercentileIterator: public TBaseHistogramIterator { - public: - /** - * @param histogram The histogram this iterator will operate on - * @param ticksPerHalfDistance The number of equal-sized iteration steps - * per half-distance to 100%. - */ - TPercentileIterator(const THistogram& histogram, ui32 ticksPerHalfDistance); - - /** - * @return The number of equal-sized iteration steps per half-distance - * to 100%. - */ - i32 GetTicketsPerHalfDistance() const; - - double GetPercentileToIterateTo() const; - - /** - * @return The percentile of recorded values in the histogram at values - * equal or smaller than valueIteratedTo. - * - */ - double GetPercentile() const; - }; - - /** - * Used for iterating through histogram values in linear steps. The iteration - * is performed in steps of <i>valueUnitsPerBucket</i> in size, terminating - * when all recorded histogram values are exhausted. Note that each iteration - * "bucket" includes values up to and including the next bucket boundary value. - */ - class TLinearIterator: public TBaseHistogramIterator { - public: - /** - * @param histogram The histogram this iterator will operate on - * @param valueUnitsPerBucket The size (in value units) of each bucket - * iteration. - */ - TLinearIterator(const THistogram& histogram, i64 valueUnitsPerBucket); - - /** - * @return The size (in value units) of each bucket iteration. - */ - i64 GetValueUnitsPerBucket() const; - - /** - * @return The count of recorded values in the histogram that were added - * to the totalCount as a result on this iteration step. Since - * multiple iteration steps may occur with overlapping equivalent - * value ranges, the count may be lower than the count found at - * the value (e.g. multiple linear steps or percentile levels can - * occur within a single equivalent value range). - */ - i64 GetCountAddedInThisIterationStep() const; - - i64 GetNextValueReportingLevel() const; - - i64 GetNextValueReportingLevelLowestEquivalent() const; - }; - - /** - * Used for iterating through histogram values in logarithmically increasing - * levels. The iteration is performed in steps that start at - * <i>valueUnitsInFirstBucket</i> and increase exponentially according to - * <i>logBase</i>, terminating when all recorded histogram values are - * exhausted. Note that each iteration "bucket" includes values up to and - * including the next bucket boundary value. - */ - class TLogarithmicIterator: public TBaseHistogramIterator { - public: - /** - * @param histogram The histogram this iterator will operate on - * @param valueUnitsInFirstBucket the size (in value units) of the first - * value bucket step - * @param logBase the multiplier by which the bucket size is expanded in - * each iteration step. - */ - TLogarithmicIterator( - const THistogram& histogram, i64 valueUnitsInFirstBucket, - double logBase); - - /** - * @return The multiplier by which the bucket size is expanded in each - * iteration step. - */ - double GetLogBase() const; - - /** - * @return The count of recorded values in the histogram that were added - * to the totalCount as a result on this iteration step. Since - * multiple iteration steps may occur with overlapping equivalent - * value ranges, the count may be lower than the count found at - * the value (e.g. multiple linear steps or percentile levels can - * occur within a single equivalent value range). - */ - i64 GetCountAddedInThisIterationStep() const; - - i64 GetNextValueReportingLevel() const; - - i64 GetNextValueReportingLevelLowestEquivalent() const; - }; -} +#pragma once + +#include "histogram.h" + +struct hdr_iter; + +namespace NHdr { + /** + * Used for iterating through histogram values. + */ + class TBaseHistogramIterator { + public: + /** + * Iterate to the next value for the iterator. If there are no more values + * available return false. + * + * @return 'false' if there are no values remaining for this iterator. + */ + bool Next(); + + /** + * @return Raw index into the counts array. + */ + i32 GetCountsIndex() const; + + /** + * @return Snapshot of the length at the time the iterator is created. + */ + i32 GetTotalCount() const; + + /** + * @return Value directly from array for the current countsIndex. + */ + i64 GetCount() const; + + /** + * @return Sum of all of the counts up to and including the count at + * this index. + */ + i64 GetCumulativeCount() const; + + /** + * @return The current value based on countsIndex. + */ + i64 GetValue() const; + + /** + * @return The highest value that is equivalent to the current value + * within the histogram's resolution. + */ + i64 GetHighestEquivalentValue() const; + + /** + * @return The lowest value that is equivalent to the current value + * within the histogram's resolution. + */ + i64 GetLowestEquivalentValue() const; + + /** + * @return The value lies in the middle (rounded up) of the range of + * values equivalent the current value. + */ + i64 GetMedianEquivalentValue() const; + + /** + * @return The actual value level that was iterated from by the iterator. + */ + i64 GetValueIteratedFrom() const; + + /** + * @return The actual value level that was iterated to by the iterator. + */ + i64 GetValueIteratedTo() const; + + protected: + // must not be instantiated directly + TBaseHistogramIterator(); + ~TBaseHistogramIterator(); + + protected: + THolder<hdr_iter> Iter_; + }; + + /** + * Used for iterating through histogram values using the finest granularity + * steps supported by the underlying representation. The iteration steps + * through all possible unit value levels, regardless of whether or not there + * were recorded values for that value level, and terminates when all recorded + * histogram values are exhausted. + */ + class TAllValuesIterator: public TBaseHistogramIterator { + public: + /** + * @param histogram The histogram this iterator will operate on + */ + explicit TAllValuesIterator(const THistogram& histogram); + }; + + /** + * Used for iterating through all recorded histogram values using the finest + * granularity steps supported by the underlying representation. The iteration + * steps through all non-zero recorded value counts, and terminates when all + * recorded histogram values are exhausted. + */ + class TRecordedValuesIterator: public TBaseHistogramIterator { + public: + /** + * @param histogram The histogram this iterator will operate on + */ + explicit TRecordedValuesIterator(const THistogram& histogram); + + /** + * @return The count of recorded values in the histogram that were added + * to the totalCount as a result on this iteration step. Since + * multiple iteration steps may occur with overlapping equivalent + * value ranges, the count may be lower than the count found at + * the value (e.g. multiple linear steps or percentile levels can + * occur within a single equivalent value range). + */ + i64 GetCountAddedInThisIterationStep() const; + }; + + /** + * Used for iterating through histogram values according to percentile levels. + * The iteration is performed in steps that start at 0% and reduce their + * distance to 100% according to the <i>percentileTicksPerHalfDistance</i> + * parameter, ultimately reaching 100% when all recorded histogram + * values are exhausted. + */ + class TPercentileIterator: public TBaseHistogramIterator { + public: + /** + * @param histogram The histogram this iterator will operate on + * @param ticksPerHalfDistance The number of equal-sized iteration steps + * per half-distance to 100%. + */ + TPercentileIterator(const THistogram& histogram, ui32 ticksPerHalfDistance); + + /** + * @return The number of equal-sized iteration steps per half-distance + * to 100%. + */ + i32 GetTicketsPerHalfDistance() const; + + double GetPercentileToIterateTo() const; + + /** + * @return The percentile of recorded values in the histogram at values + * equal or smaller than valueIteratedTo. + * + */ + double GetPercentile() const; + }; + + /** + * Used for iterating through histogram values in linear steps. The iteration + * is performed in steps of <i>valueUnitsPerBucket</i> in size, terminating + * when all recorded histogram values are exhausted. Note that each iteration + * "bucket" includes values up to and including the next bucket boundary value. + */ + class TLinearIterator: public TBaseHistogramIterator { + public: + /** + * @param histogram The histogram this iterator will operate on + * @param valueUnitsPerBucket The size (in value units) of each bucket + * iteration. + */ + TLinearIterator(const THistogram& histogram, i64 valueUnitsPerBucket); + + /** + * @return The size (in value units) of each bucket iteration. + */ + i64 GetValueUnitsPerBucket() const; + + /** + * @return The count of recorded values in the histogram that were added + * to the totalCount as a result on this iteration step. Since + * multiple iteration steps may occur with overlapping equivalent + * value ranges, the count may be lower than the count found at + * the value (e.g. multiple linear steps or percentile levels can + * occur within a single equivalent value range). + */ + i64 GetCountAddedInThisIterationStep() const; + + i64 GetNextValueReportingLevel() const; + + i64 GetNextValueReportingLevelLowestEquivalent() const; + }; + + /** + * Used for iterating through histogram values in logarithmically increasing + * levels. The iteration is performed in steps that start at + * <i>valueUnitsInFirstBucket</i> and increase exponentially according to + * <i>logBase</i>, terminating when all recorded histogram values are + * exhausted. Note that each iteration "bucket" includes values up to and + * including the next bucket boundary value. + */ + class TLogarithmicIterator: public TBaseHistogramIterator { + public: + /** + * @param histogram The histogram this iterator will operate on + * @param valueUnitsInFirstBucket the size (in value units) of the first + * value bucket step + * @param logBase the multiplier by which the bucket size is expanded in + * each iteration step. + */ + TLogarithmicIterator( + const THistogram& histogram, i64 valueUnitsInFirstBucket, + double logBase); + + /** + * @return The multiplier by which the bucket size is expanded in each + * iteration step. + */ + double GetLogBase() const; + + /** + * @return The count of recorded values in the histogram that were added + * to the totalCount as a result on this iteration step. Since + * multiple iteration steps may occur with overlapping equivalent + * value ranges, the count may be lower than the count found at + * the value (e.g. multiple linear steps or percentile levels can + * occur within a single equivalent value range). + */ + i64 GetCountAddedInThisIterationStep() const; + + i64 GetNextValueReportingLevel() const; + + i64 GetNextValueReportingLevelLowestEquivalent() const; + }; +} diff --git a/library/cpp/histogram/hdr/histogram_iter_ut.cpp b/library/cpp/histogram/hdr/histogram_iter_ut.cpp index 7a843dec65e..9c291a2547b 100644 --- a/library/cpp/histogram/hdr/histogram_iter_ut.cpp +++ b/library/cpp/histogram/hdr/histogram_iter_ut.cpp @@ -1,210 +1,210 @@ -#include "histogram_iter.h" - +#include "histogram_iter.h" + #include <library/cpp/testing/unittest/registar.h> - -using namespace NHdr; - + +using namespace NHdr; + Y_UNIT_TEST_SUITE(THistogramIterTest) { Y_UNIT_TEST(RecordedValues) { - THistogram h(TDuration::Hours(1).MicroSeconds(), 3); - UNIT_ASSERT(h.RecordValues(1000, 1000)); - UNIT_ASSERT(h.RecordValue(1000 * 1000)); - - int index = 0; - TRecordedValuesIterator it(h); - - while (it.Next()) { - i64 countInBucket = it.GetCount(); - i64 countInStep = it.GetCountAddedInThisIterationStep(); - if (index == 0) { - UNIT_ASSERT_EQUAL(countInBucket, 1000); - UNIT_ASSERT_EQUAL(countInStep, 1000); - } else if (index == 1) { - UNIT_ASSERT_EQUAL(countInBucket, 1); - UNIT_ASSERT_EQUAL(countInStep, 1); - } else { - UNIT_FAIL("unexpected index value: " << index); - } - - index++; - } - - UNIT_ASSERT_EQUAL(index, 2); - } - + THistogram h(TDuration::Hours(1).MicroSeconds(), 3); + UNIT_ASSERT(h.RecordValues(1000, 1000)); + UNIT_ASSERT(h.RecordValue(1000 * 1000)); + + int index = 0; + TRecordedValuesIterator it(h); + + while (it.Next()) { + i64 countInBucket = it.GetCount(); + i64 countInStep = it.GetCountAddedInThisIterationStep(); + if (index == 0) { + UNIT_ASSERT_EQUAL(countInBucket, 1000); + UNIT_ASSERT_EQUAL(countInStep, 1000); + } else if (index == 1) { + UNIT_ASSERT_EQUAL(countInBucket, 1); + UNIT_ASSERT_EQUAL(countInStep, 1); + } else { + UNIT_FAIL("unexpected index value: " << index); + } + + index++; + } + + UNIT_ASSERT_EQUAL(index, 2); + } + Y_UNIT_TEST(CorrectedRecordedValues) { - THistogram h(TDuration::Hours(1).MicroSeconds(), 3); - UNIT_ASSERT(h.RecordValuesWithExpectedInterval(1000, 1000, 1000)); - UNIT_ASSERT(h.RecordValueWithExpectedInterval(1000 * 1000, 1000)); - - int index = 0; - i64 totalCount = 0; - TRecordedValuesIterator it(h); - - while (it.Next()) { - i64 countInBucket = it.GetCount(); - i64 countInStep = it.GetCountAddedInThisIterationStep(); - if (index == 0) { - UNIT_ASSERT_EQUAL(countInBucket, 1001); - UNIT_ASSERT_EQUAL(countInStep, 1001); - } else { - UNIT_ASSERT(countInBucket >= 1); - UNIT_ASSERT(countInStep >= 1); - } - index++; - totalCount += countInStep; - } - - UNIT_ASSERT_EQUAL(index, 1000); - UNIT_ASSERT_EQUAL(totalCount, 2000); - } - + THistogram h(TDuration::Hours(1).MicroSeconds(), 3); + UNIT_ASSERT(h.RecordValuesWithExpectedInterval(1000, 1000, 1000)); + UNIT_ASSERT(h.RecordValueWithExpectedInterval(1000 * 1000, 1000)); + + int index = 0; + i64 totalCount = 0; + TRecordedValuesIterator it(h); + + while (it.Next()) { + i64 countInBucket = it.GetCount(); + i64 countInStep = it.GetCountAddedInThisIterationStep(); + if (index == 0) { + UNIT_ASSERT_EQUAL(countInBucket, 1001); + UNIT_ASSERT_EQUAL(countInStep, 1001); + } else { + UNIT_ASSERT(countInBucket >= 1); + UNIT_ASSERT(countInStep >= 1); + } + index++; + totalCount += countInStep; + } + + UNIT_ASSERT_EQUAL(index, 1000); + UNIT_ASSERT_EQUAL(totalCount, 2000); + } + Y_UNIT_TEST(LinearValues) { - THistogram h(TDuration::Hours(1).MicroSeconds(), 3); - UNIT_ASSERT(h.RecordValues(1000, 1000)); - UNIT_ASSERT(h.RecordValue(1000 * 1000)); - - int index = 0; - TLinearIterator it(h, 1000); - - while (it.Next()) { - i64 countInBucket = it.GetCount(); - i64 countInStep = it.GetCountAddedInThisIterationStep(); - if (index == 0) { - UNIT_ASSERT_EQUAL(countInBucket, 1000); - UNIT_ASSERT_EQUAL(countInStep, 1000); - } else if (index == 999) { - UNIT_ASSERT_EQUAL(countInBucket, 1); - UNIT_ASSERT_EQUAL(countInStep, 1); - } else { - UNIT_ASSERT_EQUAL(countInBucket, 0); - UNIT_ASSERT_EQUAL(countInStep, 0); - } - - index++; - } - - UNIT_ASSERT_EQUAL(index, 1000); - } - + THistogram h(TDuration::Hours(1).MicroSeconds(), 3); + UNIT_ASSERT(h.RecordValues(1000, 1000)); + UNIT_ASSERT(h.RecordValue(1000 * 1000)); + + int index = 0; + TLinearIterator it(h, 1000); + + while (it.Next()) { + i64 countInBucket = it.GetCount(); + i64 countInStep = it.GetCountAddedInThisIterationStep(); + if (index == 0) { + UNIT_ASSERT_EQUAL(countInBucket, 1000); + UNIT_ASSERT_EQUAL(countInStep, 1000); + } else if (index == 999) { + UNIT_ASSERT_EQUAL(countInBucket, 1); + UNIT_ASSERT_EQUAL(countInStep, 1); + } else { + UNIT_ASSERT_EQUAL(countInBucket, 0); + UNIT_ASSERT_EQUAL(countInStep, 0); + } + + index++; + } + + UNIT_ASSERT_EQUAL(index, 1000); + } + Y_UNIT_TEST(CorrectLinearValues) { - THistogram h(TDuration::Hours(1).MicroSeconds(), 3); - UNIT_ASSERT(h.RecordValuesWithExpectedInterval(1000, 1000, 1000)); - UNIT_ASSERT(h.RecordValueWithExpectedInterval(1000 * 1000, 1000)); - - int index = 0; - i64 totalCount = 0; - TLinearIterator it(h, 1000); - - while (it.Next()) { - i64 countInBucket = it.GetCount(); - i64 countInStep = it.GetCountAddedInThisIterationStep(); - - if (index == 0) { - UNIT_ASSERT_EQUAL(countInBucket, 1001); - UNIT_ASSERT_EQUAL(countInStep, 1001); - } else { - UNIT_ASSERT_EQUAL(countInBucket, 1); - UNIT_ASSERT_EQUAL(countInStep, 1); - } - - index++; - totalCount += countInStep; - } - - UNIT_ASSERT_EQUAL(index, 1000); - UNIT_ASSERT_EQUAL(totalCount, 2000); - } - + THistogram h(TDuration::Hours(1).MicroSeconds(), 3); + UNIT_ASSERT(h.RecordValuesWithExpectedInterval(1000, 1000, 1000)); + UNIT_ASSERT(h.RecordValueWithExpectedInterval(1000 * 1000, 1000)); + + int index = 0; + i64 totalCount = 0; + TLinearIterator it(h, 1000); + + while (it.Next()) { + i64 countInBucket = it.GetCount(); + i64 countInStep = it.GetCountAddedInThisIterationStep(); + + if (index == 0) { + UNIT_ASSERT_EQUAL(countInBucket, 1001); + UNIT_ASSERT_EQUAL(countInStep, 1001); + } else { + UNIT_ASSERT_EQUAL(countInBucket, 1); + UNIT_ASSERT_EQUAL(countInStep, 1); + } + + index++; + totalCount += countInStep; + } + + UNIT_ASSERT_EQUAL(index, 1000); + UNIT_ASSERT_EQUAL(totalCount, 2000); + } + Y_UNIT_TEST(LogarithmicValues) { - THistogram h(TDuration::Hours(1).MicroSeconds(), 3); - UNIT_ASSERT(h.RecordValues(1000, 1000)); - UNIT_ASSERT(h.RecordValue(1000 * 1000)); - - int index = 0; - i64 expectedValue = 1000; - TLogarithmicIterator it(h, 1000, 2.0); - - while (it.Next()) { - i64 value = it.GetValue(); - i64 countInBucket = it.GetCount(); - i64 countInStep = it.GetCountAddedInThisIterationStep(); - - UNIT_ASSERT_EQUAL(value, expectedValue); - - if (index == 0) { - UNIT_ASSERT_EQUAL(countInBucket, 1000); - UNIT_ASSERT_EQUAL(countInStep, 1000); - } else if (index == 10) { - UNIT_ASSERT_EQUAL(countInBucket, 0); - UNIT_ASSERT_EQUAL(countInStep, 1); - } else { - UNIT_ASSERT_EQUAL(countInBucket, 0); - UNIT_ASSERT_EQUAL(countInStep, 0); - } - - index++; - expectedValue *= 2; - } - - UNIT_ASSERT_EQUAL(index, 11); - } - + THistogram h(TDuration::Hours(1).MicroSeconds(), 3); + UNIT_ASSERT(h.RecordValues(1000, 1000)); + UNIT_ASSERT(h.RecordValue(1000 * 1000)); + + int index = 0; + i64 expectedValue = 1000; + TLogarithmicIterator it(h, 1000, 2.0); + + while (it.Next()) { + i64 value = it.GetValue(); + i64 countInBucket = it.GetCount(); + i64 countInStep = it.GetCountAddedInThisIterationStep(); + + UNIT_ASSERT_EQUAL(value, expectedValue); + + if (index == 0) { + UNIT_ASSERT_EQUAL(countInBucket, 1000); + UNIT_ASSERT_EQUAL(countInStep, 1000); + } else if (index == 10) { + UNIT_ASSERT_EQUAL(countInBucket, 0); + UNIT_ASSERT_EQUAL(countInStep, 1); + } else { + UNIT_ASSERT_EQUAL(countInBucket, 0); + UNIT_ASSERT_EQUAL(countInStep, 0); + } + + index++; + expectedValue *= 2; + } + + UNIT_ASSERT_EQUAL(index, 11); + } + Y_UNIT_TEST(CorrectedLogarithmicValues) { - THistogram h(TDuration::Hours(1).MicroSeconds(), 3); - UNIT_ASSERT(h.RecordValuesWithExpectedInterval(1000, 1000, 1000)); - UNIT_ASSERT(h.RecordValueWithExpectedInterval(1000 * 1000, 1000)); - - int index = 0; - i64 totalCount = 0; - i64 expectedValue = 1000; - TLogarithmicIterator it(h, 1000, 2.0); - - while (it.Next()) { - i64 value = it.GetValue(); - i64 countInBucket = it.GetCount(); - i64 countInStep = it.GetCountAddedInThisIterationStep(); - - UNIT_ASSERT_EQUAL(value, expectedValue); - - if (index == 0) { - UNIT_ASSERT_EQUAL(countInBucket, 1001); - UNIT_ASSERT_EQUAL(countInStep, 1001); - } - - index++; - totalCount += countInStep; - expectedValue *= 2; - } - - UNIT_ASSERT_EQUAL(index, 11); - UNIT_ASSERT_EQUAL(totalCount, 2000); - } - + THistogram h(TDuration::Hours(1).MicroSeconds(), 3); + UNIT_ASSERT(h.RecordValuesWithExpectedInterval(1000, 1000, 1000)); + UNIT_ASSERT(h.RecordValueWithExpectedInterval(1000 * 1000, 1000)); + + int index = 0; + i64 totalCount = 0; + i64 expectedValue = 1000; + TLogarithmicIterator it(h, 1000, 2.0); + + while (it.Next()) { + i64 value = it.GetValue(); + i64 countInBucket = it.GetCount(); + i64 countInStep = it.GetCountAddedInThisIterationStep(); + + UNIT_ASSERT_EQUAL(value, expectedValue); + + if (index == 0) { + UNIT_ASSERT_EQUAL(countInBucket, 1001); + UNIT_ASSERT_EQUAL(countInStep, 1001); + } + + index++; + totalCount += countInStep; + expectedValue *= 2; + } + + UNIT_ASSERT_EQUAL(index, 11); + UNIT_ASSERT_EQUAL(totalCount, 2000); + } + Y_UNIT_TEST(LinearIterBucketsCorrectly) { - THistogram h(255, 2); - UNIT_ASSERT(h.RecordValue(193)); - UNIT_ASSERT(h.RecordValue(255)); - UNIT_ASSERT(h.RecordValue(0)); - UNIT_ASSERT(h.RecordValue(1)); - UNIT_ASSERT(h.RecordValue(64)); - UNIT_ASSERT(h.RecordValue(128)); - - int index = 0; - i64 totalCount = 0; - TLinearIterator it(h, 64); - - while (it.Next()) { - if (index == 0) { - // change after iterator was created - UNIT_ASSERT(h.RecordValue(2)); - } - - index++; - totalCount += it.GetCountAddedInThisIterationStep(); - } - - UNIT_ASSERT_EQUAL(index, 4); - UNIT_ASSERT_EQUAL(totalCount, 6); - } -} + THistogram h(255, 2); + UNIT_ASSERT(h.RecordValue(193)); + UNIT_ASSERT(h.RecordValue(255)); + UNIT_ASSERT(h.RecordValue(0)); + UNIT_ASSERT(h.RecordValue(1)); + UNIT_ASSERT(h.RecordValue(64)); + UNIT_ASSERT(h.RecordValue(128)); + + int index = 0; + i64 totalCount = 0; + TLinearIterator it(h, 64); + + while (it.Next()) { + if (index == 0) { + // change after iterator was created + UNIT_ASSERT(h.RecordValue(2)); + } + + index++; + totalCount += it.GetCountAddedInThisIterationStep(); + } + + UNIT_ASSERT_EQUAL(index, 4); + UNIT_ASSERT_EQUAL(totalCount, 6); + } +} diff --git a/library/cpp/histogram/hdr/histogram_ut.cpp b/library/cpp/histogram/hdr/histogram_ut.cpp index 58855753136..4841b76e71e 100644 --- a/library/cpp/histogram/hdr/histogram_ut.cpp +++ b/library/cpp/histogram/hdr/histogram_ut.cpp @@ -1,178 +1,178 @@ -#include "histogram.h" - +#include "histogram.h" + #include <library/cpp/testing/unittest/registar.h> - -#include <util/generic/cast.h> - -using namespace NHdr; - -void LoadData(THistogram* h1, THistogram* h2) { - for (int i = 0; i < 1000; i++) { - UNIT_ASSERT(h1->RecordValue(1000)); - UNIT_ASSERT(h2->RecordValueWithExpectedInterval(1000, 1000)); - } - - UNIT_ASSERT(h1->RecordValue(1000 * 1000)); - UNIT_ASSERT(h2->RecordValueWithExpectedInterval(1000 * 1000, 1000)); -} - + +#include <util/generic/cast.h> + +using namespace NHdr; + +void LoadData(THistogram* h1, THistogram* h2) { + for (int i = 0; i < 1000; i++) { + UNIT_ASSERT(h1->RecordValue(1000)); + UNIT_ASSERT(h2->RecordValueWithExpectedInterval(1000, 1000)); + } + + UNIT_ASSERT(h1->RecordValue(1000 * 1000)); + UNIT_ASSERT(h2->RecordValueWithExpectedInterval(1000 * 1000, 1000)); +} + Y_UNIT_TEST_SUITE(THistogramTest) { Y_UNIT_TEST(Creation) { - THistogram h(TDuration::Hours(1).MicroSeconds(), 3); - UNIT_ASSERT_EQUAL(h.GetMemorySize(), 188512); - UNIT_ASSERT_EQUAL(h.GetCountsLen(), 23552); - } - + THistogram h(TDuration::Hours(1).MicroSeconds(), 3); + UNIT_ASSERT_EQUAL(h.GetMemorySize(), 188512); + UNIT_ASSERT_EQUAL(h.GetCountsLen(), 23552); + } + Y_UNIT_TEST(CreateWithLargeValues) { - THistogram h(20L * 1000 * 1000, 100L * 1000 * 1000, 5); - UNIT_ASSERT(h.RecordValue(100L * 1000 * 1000)); - UNIT_ASSERT(h.RecordValue(20L * 1000 * 1000)); - UNIT_ASSERT(h.RecordValue(30L * 1000 * 1000)); - - i64 v50 = h.GetValueAtPercentile(50.0); - i64 v8333 = h.GetValueAtPercentile(83.33); - i64 v8334 = h.GetValueAtPercentile(83.34); - i64 v99 = h.GetValueAtPercentile(99.0); - - UNIT_ASSERT_EQUAL(v50, 33554431); - UNIT_ASSERT_EQUAL(v8333, 33554431); - UNIT_ASSERT_EQUAL(v8334, 100663295); - UNIT_ASSERT_EQUAL(v99, 100663295); - - UNIT_ASSERT(h.ValuesAreEqual(v50, 20L * 1000 * 1000)); - UNIT_ASSERT(h.ValuesAreEqual(v8333, 30L * 1000 * 1000)); - UNIT_ASSERT(h.ValuesAreEqual(v8334, 100L * 1000 * 1000)); - UNIT_ASSERT(h.ValuesAreEqual(v99, 100L * 1000 * 1000)); - } - + THistogram h(20L * 1000 * 1000, 100L * 1000 * 1000, 5); + UNIT_ASSERT(h.RecordValue(100L * 1000 * 1000)); + UNIT_ASSERT(h.RecordValue(20L * 1000 * 1000)); + UNIT_ASSERT(h.RecordValue(30L * 1000 * 1000)); + + i64 v50 = h.GetValueAtPercentile(50.0); + i64 v8333 = h.GetValueAtPercentile(83.33); + i64 v8334 = h.GetValueAtPercentile(83.34); + i64 v99 = h.GetValueAtPercentile(99.0); + + UNIT_ASSERT_EQUAL(v50, 33554431); + UNIT_ASSERT_EQUAL(v8333, 33554431); + UNIT_ASSERT_EQUAL(v8334, 100663295); + UNIT_ASSERT_EQUAL(v99, 100663295); + + UNIT_ASSERT(h.ValuesAreEqual(v50, 20L * 1000 * 1000)); + UNIT_ASSERT(h.ValuesAreEqual(v8333, 30L * 1000 * 1000)); + UNIT_ASSERT(h.ValuesAreEqual(v8334, 100L * 1000 * 1000)); + UNIT_ASSERT(h.ValuesAreEqual(v99, 100L * 1000 * 1000)); + } + Y_UNIT_TEST(InvalidSignificantValueDigits) { - UNIT_ASSERT_EXCEPTION(THistogram(1000, -1), yexception); - UNIT_ASSERT_EXCEPTION(THistogram(1000, 0), yexception); - UNIT_ASSERT_EXCEPTION(THistogram(1000, 6), yexception); - } - + UNIT_ASSERT_EXCEPTION(THistogram(1000, -1), yexception); + UNIT_ASSERT_EXCEPTION(THistogram(1000, 0), yexception); + UNIT_ASSERT_EXCEPTION(THistogram(1000, 6), yexception); + } + Y_UNIT_TEST(InvalidLowestDiscernibleValue) { - UNIT_ASSERT_EXCEPTION(THistogram(0, 100, 3), yexception); - UNIT_ASSERT_EXCEPTION(THistogram(110, 100, 3), yexception); - } - + UNIT_ASSERT_EXCEPTION(THistogram(0, 100, 3), yexception); + UNIT_ASSERT_EXCEPTION(THistogram(110, 100, 3), yexception); + } + Y_UNIT_TEST(TotalCount) { - i64 oneHour = SafeIntegerCast<i64>(TDuration::Hours(1).MicroSeconds()); - THistogram h1(oneHour, 3); - THistogram h2(oneHour, 3); - LoadData(&h1, &h2); - - UNIT_ASSERT_EQUAL(h1.GetTotalCount(), 1001); - UNIT_ASSERT_EQUAL(h2.GetTotalCount(), 2000); - } - + i64 oneHour = SafeIntegerCast<i64>(TDuration::Hours(1).MicroSeconds()); + THistogram h1(oneHour, 3); + THistogram h2(oneHour, 3); + LoadData(&h1, &h2); + + UNIT_ASSERT_EQUAL(h1.GetTotalCount(), 1001); + UNIT_ASSERT_EQUAL(h2.GetTotalCount(), 2000); + } + Y_UNIT_TEST(StatsValues) { - i64 oneHour = SafeIntegerCast<i64>(TDuration::Hours(1).MicroSeconds()); - THistogram h1(oneHour, 3); - THistogram h2(oneHour, 3); - LoadData(&h1, &h2); - - // h1 - histogram without correction - { - UNIT_ASSERT_EQUAL(h1.GetMin(), 1000); - UNIT_ASSERT_EQUAL(h1.GetMax(), 1000447); - UNIT_ASSERT(h1.ValuesAreEqual(h1.GetMax(), 1000 * 1000)); - - // >>> numpy.mean([1000 for i in range(1000)] + [1000000]) - // 1998.0019980019979 - UNIT_ASSERT_DOUBLES_EQUAL(h1.GetMean(), 1998, 0.5); - - // >>> numpy.std([1000 for i in range(1000)] + [1000000]) - // 31559.59423085126 - UNIT_ASSERT_DOUBLES_EQUAL(h1.GetStdDeviation(), 31559, 10); - } - - // h2 - histogram with correction of co-ordinated omission - { - UNIT_ASSERT_EQUAL(h2.GetMin(), 1000); - UNIT_ASSERT_EQUAL(h2.GetMax(), 1000447); - UNIT_ASSERT(h2.ValuesAreEqual(h1.GetMax(), 1000 * 1000)); - UNIT_ASSERT_DOUBLES_EQUAL(h2.GetMean(), 250752, 0.5); - UNIT_ASSERT_DOUBLES_EQUAL(h2.GetStdDeviation(), 322557, 0.5); - } - } - + i64 oneHour = SafeIntegerCast<i64>(TDuration::Hours(1).MicroSeconds()); + THistogram h1(oneHour, 3); + THistogram h2(oneHour, 3); + LoadData(&h1, &h2); + + // h1 - histogram without correction + { + UNIT_ASSERT_EQUAL(h1.GetMin(), 1000); + UNIT_ASSERT_EQUAL(h1.GetMax(), 1000447); + UNIT_ASSERT(h1.ValuesAreEqual(h1.GetMax(), 1000 * 1000)); + + // >>> numpy.mean([1000 for i in range(1000)] + [1000000]) + // 1998.0019980019979 + UNIT_ASSERT_DOUBLES_EQUAL(h1.GetMean(), 1998, 0.5); + + // >>> numpy.std([1000 for i in range(1000)] + [1000000]) + // 31559.59423085126 + UNIT_ASSERT_DOUBLES_EQUAL(h1.GetStdDeviation(), 31559, 10); + } + + // h2 - histogram with correction of co-ordinated omission + { + UNIT_ASSERT_EQUAL(h2.GetMin(), 1000); + UNIT_ASSERT_EQUAL(h2.GetMax(), 1000447); + UNIT_ASSERT(h2.ValuesAreEqual(h1.GetMax(), 1000 * 1000)); + UNIT_ASSERT_DOUBLES_EQUAL(h2.GetMean(), 250752, 0.5); + UNIT_ASSERT_DOUBLES_EQUAL(h2.GetStdDeviation(), 322557, 0.5); + } + } + Y_UNIT_TEST(Percentiles) { - i64 oneHour = SafeIntegerCast<i64>(TDuration::Hours(1).MicroSeconds()); - THistogram h1(oneHour, 3); - THistogram h2(oneHour, 3); - LoadData(&h1, &h2); - - // >>> a = [1000 for i in range(1000)] + [1000000] - // >>> [(p, numpy.percentile(a, p)) for p in [50, 90, 99, 99.99, 99.999, 100]] - // [(50, 1000.0), (90, 1000.0), (99, 1000.0), (99.99, 900099.99999986368), (99.999, 990009.99999989558), (100, 1000000.0)] - - // h1 - histogram without correction - { - i64 v50 = h1.GetValueAtPercentile(50); - i64 v90 = h1.GetValueAtPercentile(90); - i64 v99 = h1.GetValueAtPercentile(99); - i64 v9999 = h1.GetValueAtPercentile(99.99); - i64 v99999 = h1.GetValueAtPercentile(99.999); - i64 v100 = h1.GetValueAtPercentile(100); - - UNIT_ASSERT_EQUAL(v50, 1000); - UNIT_ASSERT_EQUAL(v90, 1000); - UNIT_ASSERT_EQUAL(v99, 1000); - UNIT_ASSERT_EQUAL(v9999, 1000447); - UNIT_ASSERT_EQUAL(v99999, 1000447); - UNIT_ASSERT_EQUAL(v100, 1000447); - - UNIT_ASSERT(h1.ValuesAreEqual(v50, 1000)); - UNIT_ASSERT(h1.ValuesAreEqual(v90, 1000)); - UNIT_ASSERT(h1.ValuesAreEqual(v99, 1000)); - UNIT_ASSERT(h1.ValuesAreEqual(v9999, 1000 * 1000)); - UNIT_ASSERT(h1.ValuesAreEqual(v99999, 1000 * 1000)); - UNIT_ASSERT(h1.ValuesAreEqual(v100, 1000 * 1000)); - } - - // h2 - histogram with correction of co-ordinated omission - { - i64 v50 = h2.GetValueAtPercentile(50); - i64 v90 = h2.GetValueAtPercentile(90); - i64 v99 = h2.GetValueAtPercentile(99); - i64 v9999 = h2.GetValueAtPercentile(99.99); - i64 v99999 = h2.GetValueAtPercentile(99.999); - i64 v100 = h2.GetValueAtPercentile(100); - - UNIT_ASSERT_EQUAL(v50, 1000); - UNIT_ASSERT_EQUAL(v90, 800255); - UNIT_ASSERT_EQUAL(v99, 980479); - UNIT_ASSERT_EQUAL(v9999, 1000447); - UNIT_ASSERT_EQUAL(v99999, 1000447); - UNIT_ASSERT_EQUAL(v100, 1000447); - - UNIT_ASSERT(h2.ValuesAreEqual(v50, 1000)); - UNIT_ASSERT(h2.ValuesAreEqual(v90, 800 * 1000)); - UNIT_ASSERT(h2.ValuesAreEqual(v99, 980 * 1000)); - UNIT_ASSERT(h2.ValuesAreEqual(v9999, 1000 * 1000)); - UNIT_ASSERT(h2.ValuesAreEqual(v99999, 1000 * 1000)); - UNIT_ASSERT(h2.ValuesAreEqual(v100, 1000 * 1000)); - } - } - + i64 oneHour = SafeIntegerCast<i64>(TDuration::Hours(1).MicroSeconds()); + THistogram h1(oneHour, 3); + THistogram h2(oneHour, 3); + LoadData(&h1, &h2); + + // >>> a = [1000 for i in range(1000)] + [1000000] + // >>> [(p, numpy.percentile(a, p)) for p in [50, 90, 99, 99.99, 99.999, 100]] + // [(50, 1000.0), (90, 1000.0), (99, 1000.0), (99.99, 900099.99999986368), (99.999, 990009.99999989558), (100, 1000000.0)] + + // h1 - histogram without correction + { + i64 v50 = h1.GetValueAtPercentile(50); + i64 v90 = h1.GetValueAtPercentile(90); + i64 v99 = h1.GetValueAtPercentile(99); + i64 v9999 = h1.GetValueAtPercentile(99.99); + i64 v99999 = h1.GetValueAtPercentile(99.999); + i64 v100 = h1.GetValueAtPercentile(100); + + UNIT_ASSERT_EQUAL(v50, 1000); + UNIT_ASSERT_EQUAL(v90, 1000); + UNIT_ASSERT_EQUAL(v99, 1000); + UNIT_ASSERT_EQUAL(v9999, 1000447); + UNIT_ASSERT_EQUAL(v99999, 1000447); + UNIT_ASSERT_EQUAL(v100, 1000447); + + UNIT_ASSERT(h1.ValuesAreEqual(v50, 1000)); + UNIT_ASSERT(h1.ValuesAreEqual(v90, 1000)); + UNIT_ASSERT(h1.ValuesAreEqual(v99, 1000)); + UNIT_ASSERT(h1.ValuesAreEqual(v9999, 1000 * 1000)); + UNIT_ASSERT(h1.ValuesAreEqual(v99999, 1000 * 1000)); + UNIT_ASSERT(h1.ValuesAreEqual(v100, 1000 * 1000)); + } + + // h2 - histogram with correction of co-ordinated omission + { + i64 v50 = h2.GetValueAtPercentile(50); + i64 v90 = h2.GetValueAtPercentile(90); + i64 v99 = h2.GetValueAtPercentile(99); + i64 v9999 = h2.GetValueAtPercentile(99.99); + i64 v99999 = h2.GetValueAtPercentile(99.999); + i64 v100 = h2.GetValueAtPercentile(100); + + UNIT_ASSERT_EQUAL(v50, 1000); + UNIT_ASSERT_EQUAL(v90, 800255); + UNIT_ASSERT_EQUAL(v99, 980479); + UNIT_ASSERT_EQUAL(v9999, 1000447); + UNIT_ASSERT_EQUAL(v99999, 1000447); + UNIT_ASSERT_EQUAL(v100, 1000447); + + UNIT_ASSERT(h2.ValuesAreEqual(v50, 1000)); + UNIT_ASSERT(h2.ValuesAreEqual(v90, 800 * 1000)); + UNIT_ASSERT(h2.ValuesAreEqual(v99, 980 * 1000)); + UNIT_ASSERT(h2.ValuesAreEqual(v9999, 1000 * 1000)); + UNIT_ASSERT(h2.ValuesAreEqual(v99999, 1000 * 1000)); + UNIT_ASSERT(h2.ValuesAreEqual(v100, 1000 * 1000)); + } + } + Y_UNIT_TEST(OutOfRangeValues) { - THistogram h(1000, 4); - UNIT_ASSERT(h.RecordValue(32767)); - UNIT_ASSERT(!h.RecordValue(32768)); - } - + THistogram h(1000, 4); + UNIT_ASSERT(h.RecordValue(32767)); + UNIT_ASSERT(!h.RecordValue(32768)); + } + Y_UNIT_TEST(Reset) { - THistogram h(TDuration::Hours(1).MicroSeconds(), 3); - UNIT_ASSERT(h.RecordValues(1000, 1000)); - UNIT_ASSERT(h.RecordValue(1000 * 1000)); - UNIT_ASSERT_EQUAL(h.GetTotalCount(), 1001); - - h.Reset(); - - UNIT_ASSERT_EQUAL(h.GetTotalCount(), 0); - UNIT_ASSERT_EQUAL(h.GetMin(), Max<i64>()); - UNIT_ASSERT_EQUAL(h.GetMax(), 0); - UNIT_ASSERT_EQUAL(h.GetValueAtPercentile(99.0), 0); - } -} + THistogram h(TDuration::Hours(1).MicroSeconds(), 3); + UNIT_ASSERT(h.RecordValues(1000, 1000)); + UNIT_ASSERT(h.RecordValue(1000 * 1000)); + UNIT_ASSERT_EQUAL(h.GetTotalCount(), 1001); + + h.Reset(); + + UNIT_ASSERT_EQUAL(h.GetTotalCount(), 0); + UNIT_ASSERT_EQUAL(h.GetMin(), Max<i64>()); + UNIT_ASSERT_EQUAL(h.GetMax(), 0); + UNIT_ASSERT_EQUAL(h.GetValueAtPercentile(99.0), 0); + } +} diff --git a/library/cpp/histogram/hdr/ut/ya.make b/library/cpp/histogram/hdr/ut/ya.make index cce72456126..13ceb143c87 100644 --- a/library/cpp/histogram/hdr/ut/ya.make +++ b/library/cpp/histogram/hdr/ut/ya.make @@ -2,12 +2,12 @@ OWNER( g:util jamel ) - + UNITTEST_FOR(library/cpp/histogram/hdr) - -SRCS( - histogram_ut.cpp - histogram_iter_ut.cpp -) - -END() + +SRCS( + histogram_ut.cpp + histogram_iter_ut.cpp +) + +END() diff --git a/library/cpp/histogram/hdr/ya.make b/library/cpp/histogram/hdr/ya.make index f5732e8a1aa..885099608d6 100644 --- a/library/cpp/histogram/hdr/ya.make +++ b/library/cpp/histogram/hdr/ya.make @@ -1,17 +1,17 @@ -LIBRARY() - +LIBRARY() + OWNER( g:util jamel ) - -SRCS( - histogram.cpp - histogram_iter.cpp -) - -PEERDIR( - contrib/libs/hdr_histogram -) - -END() + +SRCS( + histogram.cpp + histogram_iter.cpp +) + +PEERDIR( + contrib/libs/hdr_histogram +) + +END() diff --git a/library/cpp/histogram/ya.make b/library/cpp/histogram/ya.make index 218aba76ad7..77cfd96902f 100644 --- a/library/cpp/histogram/ya.make +++ b/library/cpp/histogram/ya.make @@ -1,9 +1,9 @@ -RECURSE( +RECURSE( adaptive - hdr - hdr/ut + hdr + hdr/ut simple simple/ut rt rt/ut -) +) diff --git a/library/cpp/http/server/options.h b/library/cpp/http/server/options.h index f837ef0b087..38eda0e5e78 100644 --- a/library/cpp/http/server/options.h +++ b/library/cpp/http/server/options.h @@ -18,18 +18,18 @@ public: using TBindAddresses = TVector<TNetworkAddress>; void BindAddresses(TBindAddresses& ret) const; - inline THttpServerOptions& AddBindAddress(const TString& address, ui16 port) { + inline THttpServerOptions& AddBindAddress(const TString& address, ui16 port) { const TAddr addr = { address, port, }; BindSockaddr.push_back(addr); - return *this; + return *this; } - inline THttpServerOptions& AddBindAddress(const TString& address) { - return AddBindAddress(address, 0); + inline THttpServerOptions& AddBindAddress(const TString& address) { + return AddBindAddress(address, 0); } inline THttpServerOptions& EnableKeepAlive(bool enable) noexcept { diff --git a/library/cpp/http/server/response.h b/library/cpp/http/server/response.h index ff03ddebd59..a75cb85605f 100644 --- a/library/cpp/http/server/response.h +++ b/library/cpp/http/server/response.h @@ -68,13 +68,13 @@ public: return Code; } - THttpResponse& SetHttpCode(HttpCodes code) { - Code = code; - return *this; - } - + THttpResponse& SetHttpCode(HttpCodes code) { + Code = code; + return *this; + } + void OutTo(IOutputStream& out) const; - + private: HttpCodes Code; THttpHeaders Headers; diff --git a/library/cpp/ipv6_address/ut/ipv6_address_ut.cpp b/library/cpp/ipv6_address/ut/ipv6_address_ut.cpp index d6ab4034492..73bb7cffddd 100644 --- a/library/cpp/ipv6_address/ut/ipv6_address_ut.cpp +++ b/library/cpp/ipv6_address/ut/ipv6_address_ut.cpp @@ -89,8 +89,8 @@ void TIpv6AddressTest::CheckAddressValidity() { static_assert(partsV4 == intV4); const auto parsedV4 = TIpv6Address::FromString("12.34.56.78", Ok); - UNIT_ASSERT(Ok); - UNIT_ASSERT_EQUAL(parsedV4, partsV4); + UNIT_ASSERT(Ok); + UNIT_ASSERT_EQUAL(parsedV4, partsV4); constexpr TIpv6Address partsV6 {0xFB, 0x1634, 0x19, 0xABED, 0, 0x8001, 0x1670, 0x742}; static_assert(partsV6.Type() == TIpv6Address::Ipv6); @@ -99,8 +99,8 @@ void TIpv6AddressTest::CheckAddressValidity() { static_assert(partsV6 == intV6); const auto parsedV6 = TIpv6Address::FromString("FB:1634:19:ABED:0:8001:1670:742", Ok); - UNIT_ASSERT(Ok); - UNIT_ASSERT_EQUAL(parsedV6, partsV6); + UNIT_ASSERT(Ok); + UNIT_ASSERT_EQUAL(parsedV6, partsV6); static_assert(Get127001() == TIpv6Address(0x7F000001, TIpv6Address::Ipv4)); static_assert(Get1() == TIpv6Address(0, 0, 0, 0, 0, 0, 0, 1)); diff --git a/library/cpp/lwtrace/mon/mon_lwtrace.cpp b/library/cpp/lwtrace/mon/mon_lwtrace.cpp index caad0e81a26..a61ee9ce22d 100644 --- a/library/cpp/lwtrace/mon/mon_lwtrace.cpp +++ b/library/cpp/lwtrace/mon/mon_lwtrace.cpp @@ -6,9 +6,9 @@ #include <google/protobuf/text_format.h> #include <library/cpp/lwtrace/mon/analytics/all.h> #include <library/cpp/lwtrace/all.h> -#include <library/cpp/monlib/service/pages/mon_page.h> -#include <library/cpp/monlib/service/pages/resource_mon_page.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/mon_page.h> +#include <library/cpp/monlib/service/pages/resource_mon_page.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <library/cpp/resource/resource.h> #include <library/cpp/string_utils/base64/base64.h> #include <library/cpp/html/pcdata/pcdata.h> diff --git a/library/cpp/lwtrace/mon/mon_lwtrace.h b/library/cpp/lwtrace/mon/mon_lwtrace.h index 4cfd24ad9db..8030f6ea616 100644 --- a/library/cpp/lwtrace/mon/mon_lwtrace.h +++ b/library/cpp/lwtrace/mon/mon_lwtrace.h @@ -1,7 +1,7 @@ #pragma once #include <library/cpp/lwtrace/protos/lwtrace.pb.h> -#include <library/cpp/monlib/service/monservice.h> +#include <library/cpp/monlib/service/monservice.h> #include <library/cpp/lwtrace/control.h> #include <util/generic/vector.h> diff --git a/library/cpp/lwtrace/mon/ya.make b/library/cpp/lwtrace/mon/ya.make index fea1ad31509..bdcb315754b 100644 --- a/library/cpp/lwtrace/mon/ya.make +++ b/library/cpp/lwtrace/mon/ya.make @@ -47,7 +47,7 @@ PEERDIR( library/cpp/html/pcdata library/cpp/lwtrace library/cpp/lwtrace/mon/analytics - library/cpp/monlib/dynamic_counters + library/cpp/monlib/dynamic_counters library/cpp/resource library/cpp/string_utils/base64 ) diff --git a/library/cpp/messagebus/messqueue.cpp b/library/cpp/messagebus/messqueue.cpp index 6197b6bbf3f..3474d62705f 100644 --- a/library/cpp/messagebus/messqueue.cpp +++ b/library/cpp/messagebus/messqueue.cpp @@ -159,8 +159,8 @@ TBusServerSessionPtr TBusMessageQueue::CreateDestination(TBusProtocol* proto, IB } catch (...) { Y_FAIL("create destination failure: %s", CurrentExceptionMessage().c_str()); } -} - +} + void TBusMessageQueue::Add(TIntrusivePtr<TBusSessionImpl> session) { TGuard<TMutex> scope(Lock); Sessions.push_back(session); diff --git a/library/cpp/messagebus/monitoring/mon_proto.proto b/library/cpp/messagebus/monitoring/mon_proto.proto index eda77de7a52..73b6614481e 100644 --- a/library/cpp/messagebus/monitoring/mon_proto.proto +++ b/library/cpp/messagebus/monitoring/mon_proto.proto @@ -1,4 +1,4 @@ -import "library/cpp/monlib/encode/legacy_protobuf/protos/metric_meta.proto"; +import "library/cpp/monlib/encode/legacy_protobuf/protos/metric_meta.proto"; package NBus; @@ -29,19 +29,19 @@ message TMessageStatusRecord { } message TConnectionStatusMonRecord { - optional uint32 SendQueueSize = 1 [ (NMonProto.Metric).Type = GAUGE ]; + optional uint32 SendQueueSize = 1 [ (NMonProto.Metric).Type = GAUGE ]; // client only - optional uint32 AckMessagesSize = 2 [ (NMonProto.Metric).Type = GAUGE ]; - optional uint32 ErrorCount = 3 [ (NMonProto.Metric).Type = RATE ]; + optional uint32 AckMessagesSize = 2 [ (NMonProto.Metric).Type = GAUGE ]; + optional uint32 ErrorCount = 3 [ (NMonProto.Metric).Type = RATE ]; - optional uint64 WriteBytes = 10 [ (NMonProto.Metric).Type = RATE ]; + optional uint64 WriteBytes = 10 [ (NMonProto.Metric).Type = RATE ]; optional uint64 WriteBytesCompressed = 11; - optional uint64 WriteMessages = 12 [ (NMonProto.Metric).Type = RATE ]; + optional uint64 WriteMessages = 12 [ (NMonProto.Metric).Type = RATE ]; optional uint64 WriteSyscalls = 13; optional uint64 WriteActs = 14; - optional uint64 ReadBytes = 20 [ (NMonProto.Metric).Type = RATE ]; + optional uint64 ReadBytes = 20 [ (NMonProto.Metric).Type = RATE ]; optional uint64 ReadBytesCompressed = 21; - optional uint64 ReadMessages = 22 [ (NMonProto.Metric).Type = RATE ]; + optional uint64 ReadMessages = 22 [ (NMonProto.Metric).Type = RATE ]; optional uint64 ReadSyscalls = 23; optional uint64 ReadActs = 24; @@ -49,7 +49,7 @@ message TConnectionStatusMonRecord { } message TSessionStatusMonRecord { - optional uint32 InFlight = 1 [ (NMonProto.Metric).Type = GAUGE ]; - optional uint32 ConnectionCount = 2 [ (NMonProto.Metric).Type = GAUGE ]; - optional uint32 ConnectCount = 3 [ (NMonProto.Metric).Type = RATE ]; + optional uint32 InFlight = 1 [ (NMonProto.Metric).Type = GAUGE ]; + optional uint32 ConnectionCount = 2 [ (NMonProto.Metric).Type = GAUGE ]; + optional uint32 ConnectCount = 3 [ (NMonProto.Metric).Type = RATE ]; } diff --git a/library/cpp/messagebus/monitoring/ya.make b/library/cpp/messagebus/monitoring/ya.make index 7eec26150fb..25782492b11 100644 --- a/library/cpp/messagebus/monitoring/ya.make +++ b/library/cpp/messagebus/monitoring/ya.make @@ -3,7 +3,7 @@ PROTO_LIBRARY() OWNER(g:messagebus) PEERDIR( - library/cpp/monlib/encode/legacy_protobuf/protos + library/cpp/monlib/encode/legacy_protobuf/protos ) SRCS( diff --git a/library/cpp/messagebus/network.h b/library/cpp/messagebus/network.h index 6f589bd8d81..cc4bd76ea31 100644 --- a/library/cpp/messagebus/network.h +++ b/library/cpp/messagebus/network.h @@ -8,7 +8,7 @@ #include <utility> -namespace NBus { +namespace NBus { namespace NPrivate { void SetSockOptTcpCork(SOCKET s, bool value); @@ -24,5 +24,5 @@ namespace NBus { }; std::pair<unsigned, TVector<TBindResult>> BindOnPort(int port, bool reusePort); - + } diff --git a/library/cpp/messagebus/session.cpp b/library/cpp/messagebus/session.cpp index 7da9a931c13..46a7ece6a8f 100644 --- a/library/cpp/messagebus/session.cpp +++ b/library/cpp/messagebus/session.cpp @@ -124,7 +124,7 @@ TBusClientSessionPtr TBusClientSession::Create(TBusProtocol* proto, IBusClientHa TBusServerSessionPtr TBusServerSession::Create(TBusProtocol* proto, IBusServerHandler* handler, const TBusServerSessionConfig& config, TBusMessageQueuePtr queue) { return queue->CreateDestination(proto, handler, config); } - + TBusServerSessionPtr TBusServerSession::Create(TBusProtocol* proto, IBusServerHandler* handler, const TBusServerSessionConfig& config, TBusMessageQueuePtr queue, const TVector<TBindResult>& bindTo) { - return queue->CreateDestination(proto, handler, config, bindTo); -} + return queue->CreateDestination(proto, handler, config, bindTo); +} diff --git a/library/cpp/messagebus/session.h b/library/cpp/messagebus/session.h index b1734a89e82..fb12ab7c229 100644 --- a/library/cpp/messagebus/session.h +++ b/library/cpp/messagebus/session.h @@ -124,10 +124,10 @@ namespace NBus { static TBusServerSessionPtr Create( TBusProtocol* proto, IBusServerHandler* handler, - const TBusServerSessionConfig& config, - TBusMessageQueuePtr queue, + const TBusServerSessionConfig& config, + TBusMessageQueuePtr queue, const TVector<TBindResult>& bindTo); - + // TODO: make parameter non-const virtual EMessageStatus SendReply(const TBusIdentity& ident, TBusMessage* pRep) = 0; diff --git a/library/cpp/messagebus/session_impl.cpp b/library/cpp/messagebus/session_impl.cpp index 5e11c70d891..ddf9f360c43 100644 --- a/library/cpp/messagebus/session_impl.cpp +++ b/library/cpp/messagebus/session_impl.cpp @@ -485,23 +485,23 @@ void TBusSessionImpl::Act(TConnectionTag) { void TBusSessionImpl::Listen(int port, TBusMessageQueue* q) { Listen(BindOnPort(port, Config.ReusePort).second, q); -} - +} + void TBusSessionImpl::Listen(const TVector<TBindResult>& bindTo, TBusMessageQueue* q) { Y_ASSERT(q == Queue); - int actualPort = -1; + int actualPort = -1; for (const TBindResult& br : bindTo) { - if (actualPort == -1) { - actualPort = br.Addr.GetPort(); + if (actualPort == -1) { + actualPort = br.Addr.GetPort(); } else { - Y_VERIFY(actualPort == br.Addr.GetPort(), "state check"); + Y_VERIFY(actualPort == br.Addr.GetPort(), "state check"); } if (Config.SocketToS >= 0) { - SetSocketToS(*br.Socket, &(br.Addr), Config.SocketToS); + SetSocketToS(*br.Socket, &(br.Addr), Config.SocketToS); } - TAcceptorPtr acceptor(new TAcceptor(this, ++LastAcceptorId, br.Socket->Release(), br.Addr)); + TAcceptorPtr acceptor(new TAcceptor(this, ++LastAcceptorId, br.Socket->Release(), br.Addr)); TConnectionsGuard guard(ConnectionsLock); InsertAcceptorLockAcquired(acceptor.Get()); diff --git a/library/cpp/messagebus/www/www.cpp b/library/cpp/messagebus/www/www.cpp index 222195e70bb..62ec241d852 100644 --- a/library/cpp/messagebus/www/www.cpp +++ b/library/cpp/messagebus/www/www.cpp @@ -4,7 +4,7 @@ #include "html_output.h" #include <library/cpp/messagebus/remote_connection_status.h> -#include <library/cpp/monlib/deprecated/json/writer.h> +#include <library/cpp/monlib/deprecated/json/writer.h> #include <library/cpp/archive/yarchive.h> #include <library/cpp/http/fetch/httpfsm.h> @@ -621,57 +621,57 @@ struct TBusWww::TImpl { } } - void WriteQueueSensors(NMonitoring::TDeprecatedJsonWriter& sj, TStringBuf queueName, TBusMessageQueue* queue) { + void WriteQueueSensors(NMonitoring::TDeprecatedJsonWriter& sj, TStringBuf queueName, TBusMessageQueue* queue) { auto status = queue->GetStatusRecordInternal(); - sj.OpenMetric(); + sj.OpenMetric(); sj.WriteLabels("mb_queue", queueName, "sensor", "WorkQueueSize"); sj.WriteValue(status.ExecutorStatus.WorkQueueSize); - sj.CloseMetric(); + sj.CloseMetric(); } - void WriteMessageCounterSensors(NMonitoring::TDeprecatedJsonWriter& sj, + void WriteMessageCounterSensors(NMonitoring::TDeprecatedJsonWriter& sj, TStringBuf labelName, TStringBuf sessionName, bool read, const TMessageCounter& counter) { TStringBuf readOrWrite = read ? "read" : "write"; - sj.OpenMetric(); + sj.OpenMetric(); sj.WriteLabels(labelName, sessionName, "mb_dir", readOrWrite, "sensor", "MessageBytes"); sj.WriteValue(counter.BytesData); sj.WriteModeDeriv(); - sj.CloseMetric(); + sj.CloseMetric(); - sj.OpenMetric(); + sj.OpenMetric(); sj.WriteLabels(labelName, sessionName, "mb_dir", readOrWrite, "sensor", "MessageCount"); sj.WriteValue(counter.Count); sj.WriteModeDeriv(); - sj.CloseMetric(); + sj.CloseMetric(); } - void WriteSessionStatus(NMonitoring::TDeprecatedJsonWriter& sj, TStringBuf sessionName, bool client, + void WriteSessionStatus(NMonitoring::TDeprecatedJsonWriter& sj, TStringBuf sessionName, bool client, TBusSession* session) { TStringBuf labelName = client ? "mb_client_session" : "mb_server_session"; auto status = session->GetStatusRecordInternal(); - sj.OpenMetric(); + sj.OpenMetric(); sj.WriteLabels(labelName, sessionName, "sensor", "InFlightCount"); sj.WriteValue(status.Status.InFlightCount); - sj.CloseMetric(); + sj.CloseMetric(); - sj.OpenMetric(); + sj.OpenMetric(); sj.WriteLabels(labelName, sessionName, "sensor", "InFlightSize"); sj.WriteValue(status.Status.InFlightSize); - sj.CloseMetric(); + sj.CloseMetric(); - sj.OpenMetric(); + sj.OpenMetric(); sj.WriteLabels(labelName, sessionName, "sensor", "SendQueueSize"); sj.WriteValue(status.ConnectionStatusSummary.WriterStatus.SendQueueSize); - sj.CloseMetric(); + sj.CloseMetric(); if (client) { - sj.OpenMetric(); + sj.OpenMetric(); sj.WriteLabels(labelName, sessionName, "sensor", "AckMessagesSize"); sj.WriteValue(status.ConnectionStatusSummary.WriterStatus.AckMessagesSize); - sj.CloseMetric(); + sj.CloseMetric(); } WriteMessageCounterSensors(sj, labelName, sessionName, false, @@ -686,10 +686,10 @@ struct TBusWww::TImpl { Y_UNUSED(ss); bool all = q == "" && cs == "" && ss == ""; - NMonitoring::TDeprecatedJsonWriter sj(&Os); + NMonitoring::TDeprecatedJsonWriter sj(&Os); sj.OpenDocument(); - sj.OpenMetrics(); + sj.OpenMetrics(); for (unsigned i = 0; i < Outer->Queues.size(); ++i) { TString queueName = Outer->Queues.Entries[i].first; @@ -716,7 +716,7 @@ struct TBusWww::TImpl { } } - sj.CloseMetrics(); + sj.CloseMetrics(); sj.CloseDocument(); } diff --git a/library/cpp/messagebus/www/ya.make b/library/cpp/messagebus/www/ya.make index de2fc0becad..972390cea3f 100644 --- a/library/cpp/messagebus/www/ya.make +++ b/library/cpp/messagebus/www/ya.make @@ -22,7 +22,7 @@ PEERDIR( library/cpp/json/writer library/cpp/messagebus library/cpp/messagebus/oldmodule - library/cpp/monlib/deprecated/json + library/cpp/monlib/deprecated/json library/cpp/uri ) diff --git a/library/cpp/monlib/counters/counters.cpp b/library/cpp/monlib/counters/counters.cpp index f49589fac27..50dca4c577a 100644 --- a/library/cpp/monlib/counters/counters.cpp +++ b/library/cpp/monlib/counters/counters.cpp @@ -2,7 +2,7 @@ #include "counters.h" namespace NMonitoring { - char* PrettyNumShort(i64 val, char* buf, size_t size) { + char* PrettyNumShort(i64 val, char* buf, size_t size) { static const char shorts[] = {' ', 'K', 'M', 'G', 'T', 'P', 'E'}; unsigned i = 0; i64 major = val; @@ -26,7 +26,7 @@ namespace NMonitoring { return buf; } - char* PrettyNum(i64 val, char* buf, size_t size) { + char* PrettyNum(i64 val, char* buf, size_t size) { Y_ASSERT(buf); if (size < 4) { buf[0] = 0; @@ -46,4 +46,4 @@ namespace NMonitoring { return buf; } -} +} diff --git a/library/cpp/monlib/counters/counters.h b/library/cpp/monlib/counters/counters.h index 71ee2b45279..038b55f0c87 100644 --- a/library/cpp/monlib/counters/counters.h +++ b/library/cpp/monlib/counters/counters.h @@ -19,330 +19,330 @@ #include <array> namespace NMonitoring { -#define BEGIN_OUTPUT_COUNTERS \ - void OutputImpl(IOutputStream& out) { \ +#define BEGIN_OUTPUT_COUNTERS \ + void OutputImpl(IOutputStream& out) { \ char prettyBuf[32]; -#define END_OUTPUT_COUNTERS \ - out.Flush(); \ - } +#define END_OUTPUT_COUNTERS \ + out.Flush(); \ + } #define OUTPUT_NAMED_COUNTER(var, name) out << name << ": \t" << var << NMonitoring::PrettyNum(var, prettyBuf, 32) << '\n' #define OUTPUT_COUNTER(var) OUTPUT_NAMED_COUNTER(var, #var); - char* PrettyNumShort(i64 val, char* buf, size_t size); - char* PrettyNum(i64 val, char* buf, size_t size); - - // This class is deprecated. Please consider to use - // library/cpp/monlib/metrics instead. See more info at - // https://wiki.yandex-team.ru/solomon/libs/monlib_cpp/ - class TDeprecatedCounter { - public: - using TValue = TAtomic; - using TValueBase = TAtomicBase; - - TDeprecatedCounter() - : Value() - , Derivative(false) - { - } - - TDeprecatedCounter(TValue value, bool derivative = false) - : Value(value) - , Derivative(derivative) - { - } - - bool ForDerivative() const { - return Derivative; - } - - operator TValueBase() const { - return AtomicGet(Value); - } - TValueBase Val() const { - return AtomicGet(Value); - } - - void Set(TValue val) { - AtomicSet(Value, val); - } - - TValueBase Inc() { - return AtomicIncrement(Value); - } - TValueBase Dec() { - return AtomicDecrement(Value); - } - - TValueBase Add(const TValue val) { - return AtomicAdd(Value, val); - } - TValueBase Sub(const TValue val) { - return AtomicAdd(Value, -val); - } - - // operator overloads convinient - void operator++() { - Inc(); - } - void operator++(int) { - Inc(); - } - - void operator--() { - Dec(); - } - void operator--(int) { - Dec(); - } - - void operator+=(TValue rhs) { - Add(rhs); - } - void operator-=(TValue rhs) { - Sub(rhs); - } - - TValueBase operator=(TValue rhs) { - AtomicSwap(&Value, rhs); - return rhs; - } - - bool operator!() const { - return AtomicGet(Value) == 0; - } + char* PrettyNumShort(i64 val, char* buf, size_t size); + char* PrettyNum(i64 val, char* buf, size_t size); + + // This class is deprecated. Please consider to use + // library/cpp/monlib/metrics instead. See more info at + // https://wiki.yandex-team.ru/solomon/libs/monlib_cpp/ + class TDeprecatedCounter { + public: + using TValue = TAtomic; + using TValueBase = TAtomicBase; + + TDeprecatedCounter() + : Value() + , Derivative(false) + { + } + + TDeprecatedCounter(TValue value, bool derivative = false) + : Value(value) + , Derivative(derivative) + { + } + + bool ForDerivative() const { + return Derivative; + } + + operator TValueBase() const { + return AtomicGet(Value); + } + TValueBase Val() const { + return AtomicGet(Value); + } + + void Set(TValue val) { + AtomicSet(Value, val); + } + + TValueBase Inc() { + return AtomicIncrement(Value); + } + TValueBase Dec() { + return AtomicDecrement(Value); + } + + TValueBase Add(const TValue val) { + return AtomicAdd(Value, val); + } + TValueBase Sub(const TValue val) { + return AtomicAdd(Value, -val); + } + + // operator overloads convinient + void operator++() { + Inc(); + } + void operator++(int) { + Inc(); + } + + void operator--() { + Dec(); + } + void operator--(int) { + Dec(); + } + + void operator+=(TValue rhs) { + Add(rhs); + } + void operator-=(TValue rhs) { + Sub(rhs); + } + + TValueBase operator=(TValue rhs) { + AtomicSwap(&Value, rhs); + return rhs; + } + + bool operator!() const { + return AtomicGet(Value) == 0; + } TAtomic& GetAtomic() { return Value; } - private: - TAtomic Value; - bool Derivative; - }; - - template <typename T> - struct TDeprecatedCountersBase { - virtual ~TDeprecatedCountersBase() { - } - - virtual void OutputImpl(IOutputStream&) = 0; - - static T& Instance() { - return *Singleton<T>(); - } - - static void Output(IOutputStream& out) { - Instance().OutputImpl(out); - } - }; - - // This class is deprecated. Please consider to use - // library/cpp/monlib/metrics instead. See more info at - // https://wiki.yandex-team.ru/solomon/libs/monlib_cpp/ - // - // Groups of G counters, defined by T type. - // Less(a,b) returns true, if a < b. - // It's threadsafe. - template <typename T, typename G, typename TL = TLess<T>> - class TDeprecatedCounterGroups { - public: - typedef TMap<T, G*> TGroups; - typedef TVector<T> TGroupsNames; - typedef THolder<TGroupsNames> TGroupsNamesPtr; - - private: - class TCollection { - struct TElement { - T* Name; - G* Counters; - - public: - static bool Compare(const TElement& a, const TElement& b) { - return Less(*(a.Name), *(b.Name)); - } - }; // TElement - private: - TArrayHolder<TElement> Elements; - size_t Size; - + private: + TAtomic Value; + bool Derivative; + }; + + template <typename T> + struct TDeprecatedCountersBase { + virtual ~TDeprecatedCountersBase() { + } + + virtual void OutputImpl(IOutputStream&) = 0; + + static T& Instance() { + return *Singleton<T>(); + } + + static void Output(IOutputStream& out) { + Instance().OutputImpl(out); + } + }; + + // This class is deprecated. Please consider to use + // library/cpp/monlib/metrics instead. See more info at + // https://wiki.yandex-team.ru/solomon/libs/monlib_cpp/ + // + // Groups of G counters, defined by T type. + // Less(a,b) returns true, if a < b. + // It's threadsafe. + template <typename T, typename G, typename TL = TLess<T>> + class TDeprecatedCounterGroups { + public: + typedef TMap<T, G*> TGroups; + typedef TVector<T> TGroupsNames; + typedef THolder<TGroupsNames> TGroupsNamesPtr; + + private: + class TCollection { + struct TElement { + T* Name; + G* Counters; + + public: + static bool Compare(const TElement& a, const TElement& b) { + return Less(*(a.Name), *(b.Name)); + } + }; // TElement + private: + TArrayHolder<TElement> Elements; + size_t Size; + public: - TCollection() - : Size(0) - { + TCollection() + : Size(0) + { } - TCollection(const TCollection& collection) + TCollection(const TCollection& collection) : Elements(new TElement[collection.Size]) - , Size(collection.Size) - { - for (int i = 0; i < Size; ++i) { - Elements[i] = collection.Elements[i]; - } + , Size(collection.Size) + { + for (int i = 0; i < Size; ++i) { + Elements[i] = collection.Elements[i]; + } } - TCollection(const TCollection& collection, T* name, G* counters) + TCollection(const TCollection& collection, T* name, G* counters) : Elements(new TElement[collection.Size + 1]) - , Size(collection.Size + 1) - { - for (size_t i = 0; i < Size - 1; ++i) { - Elements[i] = collection.Elements[i]; - } - Elements[Size - 1].Name = name; - Elements[Size - 1].Counters = counters; - for (size_t i = 1; i < Size; ++i) { - size_t j = i; - while (j > 0 && - TElement::Compare(Elements[j], Elements[j - 1])) { - std::swap(Elements[j], Elements[j - 1]); - --j; - } - } + , Size(collection.Size + 1) + { + for (size_t i = 0; i < Size - 1; ++i) { + Elements[i] = collection.Elements[i]; + } + Elements[Size - 1].Name = name; + Elements[Size - 1].Counters = counters; + for (size_t i = 1; i < Size; ++i) { + size_t j = i; + while (j > 0 && + TElement::Compare(Elements[j], Elements[j - 1])) { + std::swap(Elements[j], Elements[j - 1]); + --j; + } + } } - - G* Find(const T& name) const { - G* result = nullptr; - if (Size == 0) { - return nullptr; + + G* Find(const T& name) const { + G* result = nullptr; + if (Size == 0) { + return nullptr; } - size_t l = 0; - size_t r = Size - 1; - while (l < r) { - size_t m = (l + r) / 2; - if (Less(*(Elements[m].Name), name)) { - l = m + 1; - } else { - r = m; - } - } - if (!Less(*(Elements[l].Name), name) && !Less(name, *(Elements[l].Name))) { - result = Elements[l].Counters; - } - return result; + size_t l = 0; + size_t r = Size - 1; + while (l < r) { + size_t m = (l + r) / 2; + if (Less(*(Elements[m].Name), name)) { + l = m + 1; + } else { + r = m; + } + } + if (!Less(*(Elements[l].Name), name) && !Less(name, *(Elements[l].Name))) { + result = Elements[l].Counters; + } + return result; } - void Free() { - for (size_t i = 0; i < Size; ++i) { - T* name = Elements[i].Name; - G* counters = Elements[i].Counters; - Elements[i].Name = nullptr; - Elements[i].Counters = nullptr; - delete name; - delete counters; - } - Size = 0; + void Free() { + for (size_t i = 0; i < Size; ++i) { + T* name = Elements[i].Name; + G* counters = Elements[i].Counters; + Elements[i].Name = nullptr; + Elements[i].Counters = nullptr; + delete name; + delete counters; + } + Size = 0; } - - TGroupsNamesPtr GetNames() const { - TGroupsNamesPtr result(new TGroupsNames()); - for (size_t i = 0; i < Size; ++i) { - result->push_back(*(Elements[i].Name)); + + TGroupsNamesPtr GetNames() const { + TGroupsNamesPtr result(new TGroupsNames()); + for (size_t i = 0; i < Size; ++i) { + result->push_back(*(Elements[i].Name)); } - return result; + return result; } - }; // TCollection - struct TOldGroup { - TCollection* Collection; - ui64 Time; - }; - - private: - TCollection* Groups; - TList<TOldGroup> OldGroups; - TSpinLock AddMutex; - - ui64 Timeout; - - static TL Less; - - private: - G* Add(const T& name) { - TGuard<TSpinLock> guard(AddMutex); - G* result = Groups->Find(name); - if (result == nullptr) { - T* newName = new T(name); - G* newCounters = new G(); - TCollection* newGroups = - new TCollection(*Groups, newName, newCounters); - ui64 now = ::Now().MicroSeconds(); - TOldGroup group; - group.Collection = Groups; - group.Time = now; - OldGroups.push_back(group); - for (ui32 i = 0; i < 5; ++i) { - if (OldGroups.front().Time + Timeout < now) { - delete OldGroups.front().Collection; - OldGroups.front().Collection = nullptr; - OldGroups.pop_front(); - } else { - break; - } - } - Groups = newGroups; - result = Groups->Find(name); + }; // TCollection + struct TOldGroup { + TCollection* Collection; + ui64 Time; + }; + + private: + TCollection* Groups; + TList<TOldGroup> OldGroups; + TSpinLock AddMutex; + + ui64 Timeout; + + static TL Less; + + private: + G* Add(const T& name) { + TGuard<TSpinLock> guard(AddMutex); + G* result = Groups->Find(name); + if (result == nullptr) { + T* newName = new T(name); + G* newCounters = new G(); + TCollection* newGroups = + new TCollection(*Groups, newName, newCounters); + ui64 now = ::Now().MicroSeconds(); + TOldGroup group; + group.Collection = Groups; + group.Time = now; + OldGroups.push_back(group); + for (ui32 i = 0; i < 5; ++i) { + if (OldGroups.front().Time + Timeout < now) { + delete OldGroups.front().Collection; + OldGroups.front().Collection = nullptr; + OldGroups.pop_front(); + } else { + break; + } + } + Groups = newGroups; + result = Groups->Find(name); } return result; } - public: - TDeprecatedCounterGroups(ui64 timeout = 5 * 1000000L) { - Groups = new TCollection(); - Timeout = timeout; + public: + TDeprecatedCounterGroups(ui64 timeout = 5 * 1000000L) { + Groups = new TCollection(); + Timeout = timeout; } - virtual ~TDeprecatedCounterGroups() { - TGuard<TSpinLock> guard(AddMutex); - Groups->Free(); - delete Groups; - Groups = nullptr; - typename TList<TOldGroup>::iterator i; - for (i = OldGroups.begin(); i != OldGroups.end(); ++i) { - delete i->Collection; - i->Collection = nullptr; + virtual ~TDeprecatedCounterGroups() { + TGuard<TSpinLock> guard(AddMutex); + Groups->Free(); + delete Groups; + Groups = nullptr; + typename TList<TOldGroup>::iterator i; + for (i = OldGroups.begin(); i != OldGroups.end(); ++i) { + delete i->Collection; + i->Collection = nullptr; } - OldGroups.clear(); + OldGroups.clear(); } - bool Has(const T& name) const { - TCollection* groups = Groups; - return groups->Find(name) != nullptr; - } + bool Has(const T& name) const { + TCollection* groups = Groups; + return groups->Find(name) != nullptr; + } G* Find(const T& name) const { - TCollection* groups = Groups; - return groups->Find(name); + TCollection* groups = Groups; + return groups->Find(name); } - // Get group with the name, if it exists. - // If there is no group with the name, add new group. - G& Get(const T& name) { - G* result = Find(name); - if (result == nullptr) { - result = Add(name); - Y_ASSERT(result != nullptr); - } - return *result; + // Get group with the name, if it exists. + // If there is no group with the name, add new group. + G& Get(const T& name) { + G* result = Find(name); + if (result == nullptr) { + result = Add(name); + Y_ASSERT(result != nullptr); + } + return *result; } - // Get copy of groups names array. - TGroupsNamesPtr GetGroupsNames() const { - TCollection* groups = Groups; - TGroupsNamesPtr result = groups->GetNames(); - return result; + // Get copy of groups names array. + TGroupsNamesPtr GetGroupsNames() const { + TCollection* groups = Groups; + TGroupsNamesPtr result = groups->GetNames(); + return result; } - }; // TDeprecatedCounterGroups + }; // TDeprecatedCounterGroups - template <typename T, typename G, typename TL> - TL TDeprecatedCounterGroups<T, G, TL>::Less; -} + template <typename T, typename G, typename TL> + TL TDeprecatedCounterGroups<T, G, TL>::Less; +} -static inline IOutputStream& operator<<(IOutputStream& o, const NMonitoring::TDeprecatedCounter& rhs) { +static inline IOutputStream& operator<<(IOutputStream& o, const NMonitoring::TDeprecatedCounter& rhs) { return o << rhs.Val(); } template <size_t N> -static inline IOutputStream& operator<<(IOutputStream& o, const std::array<NMonitoring::TDeprecatedCounter, N>& rhs) { - for (typename std::array<NMonitoring::TDeprecatedCounter, N>::const_iterator it = rhs.begin(); it != rhs.end(); ++it) { +static inline IOutputStream& operator<<(IOutputStream& o, const std::array<NMonitoring::TDeprecatedCounter, N>& rhs) { + for (typename std::array<NMonitoring::TDeprecatedCounter, N>::const_iterator it = rhs.begin(); it != rhs.end(); ++it) { if (!!*it) o << *it << Endl; } diff --git a/library/cpp/monlib/counters/counters_ut.cpp b/library/cpp/monlib/counters/counters_ut.cpp index cd006fab2f2..2845efb97ba 100644 --- a/library/cpp/monlib/counters/counters_ut.cpp +++ b/library/cpp/monlib/counters/counters_ut.cpp @@ -1,49 +1,49 @@ -#include "counters.h" - +#include "counters.h" + #include <library/cpp/testing/unittest/registar.h> - -#include <util/generic/set.h> + +#include <util/generic/set.h> #include <util/thread/pool.h> - -using namespace NMonitoring; - -Y_UNIT_TEST_SUITE(TDeprecatedCountersTest) { + +using namespace NMonitoring; + +Y_UNIT_TEST_SUITE(TDeprecatedCountersTest) { Y_UNIT_TEST(CounterGroupsAreThreadSafe) { - const static ui32 GROUPS_COUNT = 1000; - const static ui32 THREADS_COUNT = 10; - - TDeprecatedCounterGroups<ui32, ui32> groups; - - auto adder = [&groups]() { - for (ui32 id = 0; id < GROUPS_COUNT; id++) { - groups.Get(id); - - // adds contention - ::NanoSleep(42); - } - }; - + const static ui32 GROUPS_COUNT = 1000; + const static ui32 THREADS_COUNT = 10; + + TDeprecatedCounterGroups<ui32, ui32> groups; + + auto adder = [&groups]() { + for (ui32 id = 0; id < GROUPS_COUNT; id++) { + groups.Get(id); + + // adds contention + ::NanoSleep(42); + } + }; + TThreadPool q; - q.Start(THREADS_COUNT); - for (ui32 i = 0; i < THREADS_COUNT; i++) { - q.SafeAddFunc(adder); - } - q.Stop(); - - // each group id is present - for (ui32 id = 0; id < GROUPS_COUNT; id++) { - UNIT_ASSERT(groups.Has(id)); - } - - // group names contains only appropriate ids - auto ids = groups.GetGroupsNames(); - for (ui32 id : *ids) { - UNIT_ASSERT(id < GROUPS_COUNT); - } - - // no duplication in group names + q.Start(THREADS_COUNT); + for (ui32 i = 0; i < THREADS_COUNT; i++) { + q.SafeAddFunc(adder); + } + q.Stop(); + + // each group id is present + for (ui32 id = 0; id < GROUPS_COUNT; id++) { + UNIT_ASSERT(groups.Has(id)); + } + + // group names contains only appropriate ids + auto ids = groups.GetGroupsNames(); + for (ui32 id : *ids) { + UNIT_ASSERT(id < GROUPS_COUNT); + } + + // no duplication in group names TSet<ui32> uniqueIds(ids->begin(), ids->end()); - UNIT_ASSERT_EQUAL(ids->size(), uniqueIds.size()); - UNIT_ASSERT_EQUAL(ids->size(), GROUPS_COUNT); - } -} + UNIT_ASSERT_EQUAL(ids->size(), uniqueIds.size()); + UNIT_ASSERT_EQUAL(ids->size(), GROUPS_COUNT); + } +} diff --git a/library/cpp/monlib/counters/histogram.cpp b/library/cpp/monlib/counters/histogram.cpp index 6a9d7a1f0b3..46cf4e6ec8d 100644 --- a/library/cpp/monlib/counters/histogram.cpp +++ b/library/cpp/monlib/counters/histogram.cpp @@ -1,40 +1,40 @@ -#include "histogram.h" - -namespace NMonitoring { - void THistogramSnapshot::Print(IOutputStream* out) const { - (*out) << "mean: " << Mean - << ", stddev: " << StdDeviation - << ", min: " << Min - << ", max: " << Max - << ", 50%: " << Percentile50 - << ", 75%: " << Percentile75 - << ", 90%: " << Percentile90 - << ", 95%: " << Percentile95 - << ", 98%: " << Percentile98 - << ", 99%: " << Percentile99 +#include "histogram.h" + +namespace NMonitoring { + void THistogramSnapshot::Print(IOutputStream* out) const { + (*out) << "mean: " << Mean + << ", stddev: " << StdDeviation + << ", min: " << Min + << ", max: " << Max + << ", 50%: " << Percentile50 + << ", 75%: " << Percentile75 + << ", 90%: " << Percentile90 + << ", 95%: " << Percentile95 + << ", 98%: " << Percentile98 + << ", 99%: " << Percentile99 << ", 99.9%: " << Percentile999 << ", count: " << TotalCount; - } - - void THdrHistogram::TakeSnaphot(THistogramSnapshot* snapshot) { - with_lock (Lock_) { - // TODO: get data with single traverse - snapshot->Mean = Data_.GetMean(); - snapshot->StdDeviation = Data_.GetStdDeviation(); - snapshot->Min = Data_.GetMin(); - snapshot->Max = Data_.GetMax(); - snapshot->Percentile50 = Data_.GetValueAtPercentile(50.0); - snapshot->Percentile75 = Data_.GetValueAtPercentile(75.0); - snapshot->Percentile90 = Data_.GetValueAtPercentile(90.0); - snapshot->Percentile95 = Data_.GetValueAtPercentile(95.0); - snapshot->Percentile98 = Data_.GetValueAtPercentile(98.0); - snapshot->Percentile99 = Data_.GetValueAtPercentile(99.0); - snapshot->Percentile999 = Data_.GetValueAtPercentile(99.9); + } + + void THdrHistogram::TakeSnaphot(THistogramSnapshot* snapshot) { + with_lock (Lock_) { + // TODO: get data with single traverse + snapshot->Mean = Data_.GetMean(); + snapshot->StdDeviation = Data_.GetStdDeviation(); + snapshot->Min = Data_.GetMin(); + snapshot->Max = Data_.GetMax(); + snapshot->Percentile50 = Data_.GetValueAtPercentile(50.0); + snapshot->Percentile75 = Data_.GetValueAtPercentile(75.0); + snapshot->Percentile90 = Data_.GetValueAtPercentile(90.0); + snapshot->Percentile95 = Data_.GetValueAtPercentile(95.0); + snapshot->Percentile98 = Data_.GetValueAtPercentile(98.0); + snapshot->Percentile99 = Data_.GetValueAtPercentile(99.0); + snapshot->Percentile999 = Data_.GetValueAtPercentile(99.9); snapshot->TotalCount = Data_.GetTotalCount(); - - // cleanup histogram data - Data_.Reset(); - } - } - -} + + // cleanup histogram data + Data_.Reset(); + } + } + +} diff --git a/library/cpp/monlib/counters/histogram.h b/library/cpp/monlib/counters/histogram.h index e6bc625daf2..96361b00238 100644 --- a/library/cpp/monlib/counters/histogram.h +++ b/library/cpp/monlib/counters/histogram.h @@ -1,189 +1,189 @@ -#pragma once - +#pragma once + #include <library/cpp/histogram/hdr/histogram.h> - -#include <util/system/spinlock.h> -#include <util/stream/output.h> - -namespace NMonitoring { - /** - * A statistical snapshot of values recorded in histogram. - */ - struct THistogramSnapshot { - double Mean; - double StdDeviation; - i64 Min; - i64 Max; - i64 Percentile50; - i64 Percentile75; - i64 Percentile90; - i64 Percentile95; - i64 Percentile98; - i64 Percentile99; - i64 Percentile999; + +#include <util/system/spinlock.h> +#include <util/stream/output.h> + +namespace NMonitoring { + /** + * A statistical snapshot of values recorded in histogram. + */ + struct THistogramSnapshot { + double Mean; + double StdDeviation; + i64 Min; + i64 Max; + i64 Percentile50; + i64 Percentile75; + i64 Percentile90; + i64 Percentile95; + i64 Percentile98; + i64 Percentile99; + i64 Percentile999; i64 TotalCount; - - void Print(IOutputStream* out) const; - }; - - /** - * Special counter which calculates the distribution of a value. - */ - class THdrHistogram { - public: - /** - * Construct a histogram given the Lowest and Highest values to be tracked - * and a number of significant decimal digits. Providing a - * lowestDiscernibleValue is useful in situations where the units used for - * the histogram's values are much smaller that the minimal accuracy - * required. E.g. when tracking time values stated in nanosecond units, - * where the minimal accuracy required is a microsecond, the proper value - * for lowestDiscernibleValue would be 1000. - * - * @param lowestDiscernibleValue The lowest value that can be discerned - * (distinguished from 0) by the histogram. Must be a positive - * integer that is >= 1. May be internally rounded down to nearest - * power of 2. - * - * @param highestTrackableValue The highest value to be tracked by the - * histogram. Must be a positive integer that is - * >= (2 * lowestDiscernibleValue). - * - * @param numberOfSignificantValueDigits Specifies the precision to use. - * This is the number of significant decimal digits to which the - * histogram will maintain value resolution and separation. Must be - * a non-negative integer between 0 and 5. - */ - THdrHistogram(i64 lowestDiscernibleValue, i64 highestTrackableValue, - i32 numberOfSignificantValueDigits) - : Data_(lowestDiscernibleValue, highestTrackableValue, - numberOfSignificantValueDigits) { - } - - /** - * Records a value in the histogram, will round this value of to a - * precision at or better than the NumberOfSignificantValueDigits specified - * at construction time. - * - * @param value Value to add to the histogram - * @return false if the value is larger than the HighestTrackableValue - * and can't be recorded, true otherwise. - */ - bool RecordValue(i64 value) { - with_lock (Lock_) { - return Data_.RecordValue(value); - } - } - - /** - * Records count values in the histogram, will round this value of to a - * precision at or better than the NumberOfSignificantValueDigits specified - * at construction time. - * - * @param value Value to add to the histogram - * @param count Number of values to add to the histogram - * @return false if the value is larger than the HighestTrackableValue - * and can't be recorded, true otherwise. - */ - bool RecordValues(i64 value, i64 count) { - with_lock (Lock_) { - return Data_.RecordValues(value, count); - } - } - - /** - * Records a value in the histogram and backfill based on an expected - * interval. Value will be rounded this to a precision at or better - * than the NumberOfSignificantValueDigits specified at contruction time. - * This is specifically used for recording latency. If the value is larger - * than the expectedInterval then the latency recording system has - * experienced co-ordinated omission. This method fills in the values that - * would have occured had the client providing the load not been blocked. - * - * @param value Value to add to the histogram - * @param expectedInterval The delay between recording values - * @return false if the value is larger than the HighestTrackableValue - * and can't be recorded, true otherwise. - */ - bool RecordValueWithExpectedInterval(i64 value, i64 expectedInterval) { - with_lock (Lock_) { - return Data_.RecordValueWithExpectedInterval(value, expectedInterval); - } - } - - /** - * Record a value in the histogram count times. Applies the same correcting - * logic as {@link THdrHistogram::RecordValueWithExpectedInterval}. - * - * @param value Value to add to the histogram - * @param count Number of values to add to the histogram - * @param expectedInterval The delay between recording values. - * @return false if the value is larger than the HighestTrackableValue - * and can't be recorded, true otherwise. - */ - bool RecordValuesWithExpectedInterval( - i64 value, i64 count, i64 expectedInterval) { - with_lock (Lock_) { - return Data_.RecordValuesWithExpectedInterval( - value, count, expectedInterval); - } - } - - /** - * @return The configured lowestDiscernibleValue - */ - i64 GetLowestDiscernibleValue() const { - with_lock (Lock_) { - return Data_.GetLowestDiscernibleValue(); - } - } - - /** - * @return The configured highestTrackableValue - */ - i64 GetHighestTrackableValue() const { - with_lock (Lock_) { - return Data_.GetHighestTrackableValue(); - } - } - - /** - * @return The configured numberOfSignificantValueDigits - */ - i32 GetNumberOfSignificantValueDigits() const { - with_lock (Lock_) { - return Data_.GetNumberOfSignificantValueDigits(); - } - } - - /** - * @return The total count of all recorded values in the histogram - */ - i64 GetTotalCount() const { - with_lock (Lock_) { - return Data_.GetTotalCount(); - } - } - - /** - * Place a copy of the value counts accumulated since the last snapshot - * was taken into {@code snapshot}. Calling this member-function will - * reset the value counts, and start accumulating value counts for the - * next interval. - * - * @param snapshot the structure into which the values should be copied. - */ - void TakeSnaphot(THistogramSnapshot* snapshot); - - private: - mutable TSpinLock Lock_; - NHdr::THistogram Data_; - }; - -} - -template <> -inline void Out<NMonitoring::THistogramSnapshot>( - IOutputStream& out, const NMonitoring::THistogramSnapshot& snapshot) { - snapshot.Print(&out); -} + + void Print(IOutputStream* out) const; + }; + + /** + * Special counter which calculates the distribution of a value. + */ + class THdrHistogram { + public: + /** + * Construct a histogram given the Lowest and Highest values to be tracked + * and a number of significant decimal digits. Providing a + * lowestDiscernibleValue is useful in situations where the units used for + * the histogram's values are much smaller that the minimal accuracy + * required. E.g. when tracking time values stated in nanosecond units, + * where the minimal accuracy required is a microsecond, the proper value + * for lowestDiscernibleValue would be 1000. + * + * @param lowestDiscernibleValue The lowest value that can be discerned + * (distinguished from 0) by the histogram. Must be a positive + * integer that is >= 1. May be internally rounded down to nearest + * power of 2. + * + * @param highestTrackableValue The highest value to be tracked by the + * histogram. Must be a positive integer that is + * >= (2 * lowestDiscernibleValue). + * + * @param numberOfSignificantValueDigits Specifies the precision to use. + * This is the number of significant decimal digits to which the + * histogram will maintain value resolution and separation. Must be + * a non-negative integer between 0 and 5. + */ + THdrHistogram(i64 lowestDiscernibleValue, i64 highestTrackableValue, + i32 numberOfSignificantValueDigits) + : Data_(lowestDiscernibleValue, highestTrackableValue, + numberOfSignificantValueDigits) { + } + + /** + * Records a value in the histogram, will round this value of to a + * precision at or better than the NumberOfSignificantValueDigits specified + * at construction time. + * + * @param value Value to add to the histogram + * @return false if the value is larger than the HighestTrackableValue + * and can't be recorded, true otherwise. + */ + bool RecordValue(i64 value) { + with_lock (Lock_) { + return Data_.RecordValue(value); + } + } + + /** + * Records count values in the histogram, will round this value of to a + * precision at or better than the NumberOfSignificantValueDigits specified + * at construction time. + * + * @param value Value to add to the histogram + * @param count Number of values to add to the histogram + * @return false if the value is larger than the HighestTrackableValue + * and can't be recorded, true otherwise. + */ + bool RecordValues(i64 value, i64 count) { + with_lock (Lock_) { + return Data_.RecordValues(value, count); + } + } + + /** + * Records a value in the histogram and backfill based on an expected + * interval. Value will be rounded this to a precision at or better + * than the NumberOfSignificantValueDigits specified at contruction time. + * This is specifically used for recording latency. If the value is larger + * than the expectedInterval then the latency recording system has + * experienced co-ordinated omission. This method fills in the values that + * would have occured had the client providing the load not been blocked. + * + * @param value Value to add to the histogram + * @param expectedInterval The delay between recording values + * @return false if the value is larger than the HighestTrackableValue + * and can't be recorded, true otherwise. + */ + bool RecordValueWithExpectedInterval(i64 value, i64 expectedInterval) { + with_lock (Lock_) { + return Data_.RecordValueWithExpectedInterval(value, expectedInterval); + } + } + + /** + * Record a value in the histogram count times. Applies the same correcting + * logic as {@link THdrHistogram::RecordValueWithExpectedInterval}. + * + * @param value Value to add to the histogram + * @param count Number of values to add to the histogram + * @param expectedInterval The delay between recording values. + * @return false if the value is larger than the HighestTrackableValue + * and can't be recorded, true otherwise. + */ + bool RecordValuesWithExpectedInterval( + i64 value, i64 count, i64 expectedInterval) { + with_lock (Lock_) { + return Data_.RecordValuesWithExpectedInterval( + value, count, expectedInterval); + } + } + + /** + * @return The configured lowestDiscernibleValue + */ + i64 GetLowestDiscernibleValue() const { + with_lock (Lock_) { + return Data_.GetLowestDiscernibleValue(); + } + } + + /** + * @return The configured highestTrackableValue + */ + i64 GetHighestTrackableValue() const { + with_lock (Lock_) { + return Data_.GetHighestTrackableValue(); + } + } + + /** + * @return The configured numberOfSignificantValueDigits + */ + i32 GetNumberOfSignificantValueDigits() const { + with_lock (Lock_) { + return Data_.GetNumberOfSignificantValueDigits(); + } + } + + /** + * @return The total count of all recorded values in the histogram + */ + i64 GetTotalCount() const { + with_lock (Lock_) { + return Data_.GetTotalCount(); + } + } + + /** + * Place a copy of the value counts accumulated since the last snapshot + * was taken into {@code snapshot}. Calling this member-function will + * reset the value counts, and start accumulating value counts for the + * next interval. + * + * @param snapshot the structure into which the values should be copied. + */ + void TakeSnaphot(THistogramSnapshot* snapshot); + + private: + mutable TSpinLock Lock_; + NHdr::THistogram Data_; + }; + +} + +template <> +inline void Out<NMonitoring::THistogramSnapshot>( + IOutputStream& out, const NMonitoring::THistogramSnapshot& snapshot) { + snapshot.Print(&out); +} diff --git a/library/cpp/monlib/counters/histogram_ut.cpp b/library/cpp/monlib/counters/histogram_ut.cpp index 654994d12f3..5a0800505ab 100644 --- a/library/cpp/monlib/counters/histogram_ut.cpp +++ b/library/cpp/monlib/counters/histogram_ut.cpp @@ -1,47 +1,47 @@ -#include "histogram.h" - +#include "histogram.h" + #include <library/cpp/testing/unittest/registar.h> - -using namespace NMonitoring; - + +using namespace NMonitoring; + Y_UNIT_TEST_SUITE(THistorgamTest) { Y_UNIT_TEST(TakeSnapshot) { - THdrHistogram h(1, 10, 3); - UNIT_ASSERT(h.RecordValue(1)); - UNIT_ASSERT(h.RecordValue(2)); - UNIT_ASSERT(h.RecordValues(3, 10)); - UNIT_ASSERT(h.RecordValue(4)); - UNIT_ASSERT(h.RecordValue(5)); - - UNIT_ASSERT_EQUAL(h.GetTotalCount(), 14); - - THistogramSnapshot snapshot; - h.TakeSnaphot(&snapshot); - - UNIT_ASSERT_EQUAL(h.GetTotalCount(), 0); - - UNIT_ASSERT_EQUAL(snapshot.Min, 1); - UNIT_ASSERT_EQUAL(snapshot.Max, 5); - - // >>> a = [1, 2] + [3 for i in range(10)] + [4, 5] - // >>> numpy.mean(a) - // 3.0 - UNIT_ASSERT_DOUBLES_EQUAL(snapshot.Mean, 3.0, 1e-6); - - // >>> numpy.std(a) - // 0.84515425472851657 - UNIT_ASSERT_DOUBLES_EQUAL(snapshot.StdDeviation, 0.84515425472851657, 1e-6); - - // >>> [(p, round(numpy.percentile(a, p))) for p in [50, 75, 90, 95, 98, 99, 99.9, 100]] - // [(50, 3.0), (75, 3.0), (90, 4.0), (95, 4.0), (98, 5.0), (99, 5.0), (99.9, 5.0), (100, 5.0)] - UNIT_ASSERT_EQUAL(snapshot.Percentile50, 3); - UNIT_ASSERT_EQUAL(snapshot.Percentile75, 3); - UNIT_ASSERT_EQUAL(snapshot.Percentile90, 4); - UNIT_ASSERT_EQUAL(snapshot.Percentile95, 4); - UNIT_ASSERT_EQUAL(snapshot.Percentile98, 5); - UNIT_ASSERT_EQUAL(snapshot.Percentile99, 5); - UNIT_ASSERT_EQUAL(snapshot.Percentile999, 5); + THdrHistogram h(1, 10, 3); + UNIT_ASSERT(h.RecordValue(1)); + UNIT_ASSERT(h.RecordValue(2)); + UNIT_ASSERT(h.RecordValues(3, 10)); + UNIT_ASSERT(h.RecordValue(4)); + UNIT_ASSERT(h.RecordValue(5)); + + UNIT_ASSERT_EQUAL(h.GetTotalCount(), 14); + + THistogramSnapshot snapshot; + h.TakeSnaphot(&snapshot); + + UNIT_ASSERT_EQUAL(h.GetTotalCount(), 0); + + UNIT_ASSERT_EQUAL(snapshot.Min, 1); + UNIT_ASSERT_EQUAL(snapshot.Max, 5); + + // >>> a = [1, 2] + [3 for i in range(10)] + [4, 5] + // >>> numpy.mean(a) + // 3.0 + UNIT_ASSERT_DOUBLES_EQUAL(snapshot.Mean, 3.0, 1e-6); + + // >>> numpy.std(a) + // 0.84515425472851657 + UNIT_ASSERT_DOUBLES_EQUAL(snapshot.StdDeviation, 0.84515425472851657, 1e-6); + + // >>> [(p, round(numpy.percentile(a, p))) for p in [50, 75, 90, 95, 98, 99, 99.9, 100]] + // [(50, 3.0), (75, 3.0), (90, 4.0), (95, 4.0), (98, 5.0), (99, 5.0), (99.9, 5.0), (100, 5.0)] + UNIT_ASSERT_EQUAL(snapshot.Percentile50, 3); + UNIT_ASSERT_EQUAL(snapshot.Percentile75, 3); + UNIT_ASSERT_EQUAL(snapshot.Percentile90, 4); + UNIT_ASSERT_EQUAL(snapshot.Percentile95, 4); + UNIT_ASSERT_EQUAL(snapshot.Percentile98, 5); + UNIT_ASSERT_EQUAL(snapshot.Percentile99, 5); + UNIT_ASSERT_EQUAL(snapshot.Percentile999, 5); UNIT_ASSERT_EQUAL(snapshot.TotalCount, 14); - } -} + } +} diff --git a/library/cpp/monlib/counters/meter.cpp b/library/cpp/monlib/counters/meter.cpp index 2ecdb97666a..6f15f173d11 100644 --- a/library/cpp/monlib/counters/meter.cpp +++ b/library/cpp/monlib/counters/meter.cpp @@ -1,4 +1,4 @@ -#include "meter.h" - -namespace NMonitoring { -} +#include "meter.h" + +namespace NMonitoring { +} diff --git a/library/cpp/monlib/counters/meter.h b/library/cpp/monlib/counters/meter.h index ea0bdad11f3..1219f95c4d8 100644 --- a/library/cpp/monlib/counters/meter.h +++ b/library/cpp/monlib/counters/meter.h @@ -1,285 +1,285 @@ -#pragma once - -#include <util/system/types.h> -#include <util/generic/noncopyable.h> -#include <util/system/atomic.h> - -#include <chrono> -#include <cstdlib> -#include <cmath> - -namespace NMonitoring { - /** - * An exponentially-weighted moving average. - * - * @see <a href="http://www.teamquest.com/pdfs/whitepaper/ldavg1.pdf"> - * UNIX Load Average Part 1: How It Works</a> - * @see <a href="http://www.teamquest.com/pdfs/whitepaper/ldavg2.pdf"> - * UNIX Load Average Part 2: Not Your Average Average</a> - * @see <a href="http://en.wikipedia.org/wiki/Moving_average#Exponential_moving_average">EMA</a> - */ - class TMovingAverage { - public: - enum { - INTERVAL = 5 // in seconds - }; - - public: - /** - * Creates a new EWMA which is equivalent to the UNIX one minute load - * average and which expects to be ticked every 5 seconds. - * - * @return a one-minute EWMA - */ - static TMovingAverage OneMinute() { - static const double M1_ALPHA = 1 - std::exp(-INTERVAL / 60.0 / 1); - return {M1_ALPHA, std::chrono::seconds(INTERVAL)}; - } - - /** - * Creates a new EWMA which is equivalent to the UNIX five minute load - * average and which expects to be ticked every 5 seconds. - * - * @return a five-minute EWMA - */ - static TMovingAverage FiveMinutes() { - static const double M5_ALPHA = 1 - std::exp(-INTERVAL / 60.0 / 5); - return {M5_ALPHA, std::chrono::seconds(INTERVAL)}; - } - - /** - * Creates a new EWMA which is equivalent to the UNIX fifteen minute load - * average and which expects to be ticked every 5 seconds. - * - * @return a fifteen-minute EWMA - */ - static TMovingAverage FifteenMinutes() { - static const double M15_ALPHA = 1 - std::exp(-INTERVAL / 60.0 / 15); - return {M15_ALPHA, std::chrono::seconds(INTERVAL)}; - } - - /** - * Create a new EWMA with a specific smoothing constant. - * - * @param alpha the smoothing constant - * @param interval the expected tick interval - */ - TMovingAverage(double alpha, std::chrono::seconds interval) - : Initialized_(0) - , Rate_(0) - , Uncounted_(0) - , Alpha_(alpha) - , Interval_(std::chrono::nanoseconds(interval).count()) - { - } - - TMovingAverage(const TMovingAverage& rhs) - : Initialized_(AtomicGet(rhs.Initialized_)) - , Rate_(AtomicGet(rhs.Rate_)) - , Uncounted_(AtomicGet(rhs.Uncounted_)) - , Alpha_(rhs.Alpha_) - , Interval_(rhs.Interval_) - { - } - - TMovingAverage& operator=(const TMovingAverage& rhs) { - AtomicSet(Initialized_, AtomicGet(Initialized_)); - AtomicSet(Rate_, AtomicGet(rhs.Rate_)); - AtomicSet(Uncounted_, AtomicGet(rhs.Uncounted_)); - Alpha_ = rhs.Alpha_; - Interval_ = rhs.Interval_; - return *this; - } - - /** - * Update the moving average with a new value. - * - * @param n the new value - */ - void Update(ui64 n = 1) { - AtomicAdd(Uncounted_, n); - } - - /** - * Mark the passage of time and decay the current rate accordingly. - */ - void Tick() { - double instantRate = AtomicSwap(&Uncounted_, 0) / Interval_; - if (AtomicGet(Initialized_)) { - double rate = AsDouble(AtomicGet(Rate_)); - rate += (Alpha_ * (instantRate - rate)); - AtomicSet(Rate_, AsAtomic(rate)); - } else { - AtomicSet(Rate_, AsAtomic(instantRate)); - AtomicSet(Initialized_, 1); - } - } - - /** - * @return the rate in the seconds - */ - double GetRate() const { - double rate = AsDouble(AtomicGet(Rate_)); - return rate * std::nano::den; - } - - private: - static double AsDouble(TAtomicBase val) { - union { - double D; - TAtomicBase A; - } doubleAtomic; - doubleAtomic.A = val; - return doubleAtomic.D; - } - - static TAtomicBase AsAtomic(double val) { - union { - double D; - TAtomicBase A; - } doubleAtomic; - doubleAtomic.D = val; - return doubleAtomic.A; - } - - private: - TAtomic Initialized_; - TAtomic Rate_; - TAtomic Uncounted_; - double Alpha_; - double Interval_; - }; - - /** - * A meter metric which measures mean throughput and one-, five-, and - * fifteen-minute exponentially-weighted moving average throughputs. - */ - template <typename TClock> - class TMeterImpl: private TNonCopyable { - public: - TMeterImpl() - : StartTime_(TClock::now()) - , LastTick_(StartTime_.time_since_epoch().count()) - , Count_(0) - , OneMinuteRate_(TMovingAverage::OneMinute()) - , FiveMinutesRate_(TMovingAverage::FiveMinutes()) - , FifteenMinutesRate_(TMovingAverage::FifteenMinutes()) - { - } - - /** - * Mark the occurrence of events. - * - * @param n the number of events - */ - void Mark(ui64 n = 1) { - TickIfNecessary(); - AtomicAdd(Count_, n); - OneMinuteRate_.Update(n); - FiveMinutesRate_.Update(n); - FifteenMinutesRate_.Update(n); - } - - /** - * Returns the one-minute exponentially-weighted moving average rate at - * which events have occurred since the meter was created. - * - * This rate has the same exponential decay factor as the one-minute load - * average in the top Unix command. - * - * @return the one-minute exponentially-weighted moving average rate at - * which events have occurred since the meter was created - */ - double GetOneMinuteRate() const { - return OneMinuteRate_.GetRate(); - } - - /** - * Returns the five-minute exponentially-weighted moving average rate at - * which events have occurred since the meter was created. - * - * This rate has the same exponential decay factor as the five-minute load - * average in the top Unix command. - * - * @return the five-minute exponentially-weighted moving average rate at - * which events have occurred since the meter was created - */ - double GetFiveMinutesRate() const { - return FiveMinutesRate_.GetRate(); - } - - /** - * Returns the fifteen-minute exponentially-weighted moving average rate - * at which events have occurred since the meter was created. - * - * This rate has the same exponential decay factor as the fifteen-minute - * load average in the top Unix command. - * - * @return the fifteen-minute exponentially-weighted moving average rate - * at which events have occurred since the meter was created - */ - double GetFifteenMinutesRate() const { - return FifteenMinutesRate_.GetRate(); - } - - /** - * @return the mean rate at which events have occurred since the meter - * was created - */ - double GetMeanRate() const { - if (GetCount() == 0) { - return 0.0; - } - - auto now = TClock::now(); - std::chrono::duration<double> elapsedSeconds = now - StartTime_; - return GetCount() / elapsedSeconds.count(); - } - - /** - * @return the number of events which have been marked - */ - ui64 GetCount() const { - return AtomicGet(Count_); - } - - private: - void TickIfNecessary() { - static ui64 TICK_INTERVAL_NS = - std::chrono::nanoseconds( - std::chrono::seconds(TMovingAverage::INTERVAL)) - .count(); - - auto oldTickNs = AtomicGet(LastTick_); - auto newTickNs = TClock::now().time_since_epoch().count(); - ui64 elapsedNs = std::abs(newTickNs - oldTickNs); - - if (elapsedNs > TICK_INTERVAL_NS) { - // adjust to interval begining - newTickNs -= elapsedNs % TICK_INTERVAL_NS; - if (AtomicCas(&LastTick_, newTickNs, oldTickNs)) { - ui64 requiredTicks = elapsedNs / TICK_INTERVAL_NS; - for (ui64 i = 0; i < requiredTicks; ++i) { - OneMinuteRate_.Tick(); - FiveMinutesRate_.Tick(); - FifteenMinutesRate_.Tick(); - } - } - } - } - - private: - const typename TClock::time_point StartTime_; - TAtomic LastTick_; - TAtomic Count_; - TMovingAverage OneMinuteRate_; - TMovingAverage FiveMinutesRate_; - TMovingAverage FifteenMinutesRate_; - }; - - using TSystemMeter = TMeterImpl<std::chrono::system_clock>; - using TSteadyMeter = TMeterImpl<std::chrono::steady_clock>; - using THighResMeter = TMeterImpl<std::chrono::high_resolution_clock>; - using TMeter = THighResMeter; - -} +#pragma once + +#include <util/system/types.h> +#include <util/generic/noncopyable.h> +#include <util/system/atomic.h> + +#include <chrono> +#include <cstdlib> +#include <cmath> + +namespace NMonitoring { + /** + * An exponentially-weighted moving average. + * + * @see <a href="http://www.teamquest.com/pdfs/whitepaper/ldavg1.pdf"> + * UNIX Load Average Part 1: How It Works</a> + * @see <a href="http://www.teamquest.com/pdfs/whitepaper/ldavg2.pdf"> + * UNIX Load Average Part 2: Not Your Average Average</a> + * @see <a href="http://en.wikipedia.org/wiki/Moving_average#Exponential_moving_average">EMA</a> + */ + class TMovingAverage { + public: + enum { + INTERVAL = 5 // in seconds + }; + + public: + /** + * Creates a new EWMA which is equivalent to the UNIX one minute load + * average and which expects to be ticked every 5 seconds. + * + * @return a one-minute EWMA + */ + static TMovingAverage OneMinute() { + static const double M1_ALPHA = 1 - std::exp(-INTERVAL / 60.0 / 1); + return {M1_ALPHA, std::chrono::seconds(INTERVAL)}; + } + + /** + * Creates a new EWMA which is equivalent to the UNIX five minute load + * average and which expects to be ticked every 5 seconds. + * + * @return a five-minute EWMA + */ + static TMovingAverage FiveMinutes() { + static const double M5_ALPHA = 1 - std::exp(-INTERVAL / 60.0 / 5); + return {M5_ALPHA, std::chrono::seconds(INTERVAL)}; + } + + /** + * Creates a new EWMA which is equivalent to the UNIX fifteen minute load + * average and which expects to be ticked every 5 seconds. + * + * @return a fifteen-minute EWMA + */ + static TMovingAverage FifteenMinutes() { + static const double M15_ALPHA = 1 - std::exp(-INTERVAL / 60.0 / 15); + return {M15_ALPHA, std::chrono::seconds(INTERVAL)}; + } + + /** + * Create a new EWMA with a specific smoothing constant. + * + * @param alpha the smoothing constant + * @param interval the expected tick interval + */ + TMovingAverage(double alpha, std::chrono::seconds interval) + : Initialized_(0) + , Rate_(0) + , Uncounted_(0) + , Alpha_(alpha) + , Interval_(std::chrono::nanoseconds(interval).count()) + { + } + + TMovingAverage(const TMovingAverage& rhs) + : Initialized_(AtomicGet(rhs.Initialized_)) + , Rate_(AtomicGet(rhs.Rate_)) + , Uncounted_(AtomicGet(rhs.Uncounted_)) + , Alpha_(rhs.Alpha_) + , Interval_(rhs.Interval_) + { + } + + TMovingAverage& operator=(const TMovingAverage& rhs) { + AtomicSet(Initialized_, AtomicGet(Initialized_)); + AtomicSet(Rate_, AtomicGet(rhs.Rate_)); + AtomicSet(Uncounted_, AtomicGet(rhs.Uncounted_)); + Alpha_ = rhs.Alpha_; + Interval_ = rhs.Interval_; + return *this; + } + + /** + * Update the moving average with a new value. + * + * @param n the new value + */ + void Update(ui64 n = 1) { + AtomicAdd(Uncounted_, n); + } + + /** + * Mark the passage of time and decay the current rate accordingly. + */ + void Tick() { + double instantRate = AtomicSwap(&Uncounted_, 0) / Interval_; + if (AtomicGet(Initialized_)) { + double rate = AsDouble(AtomicGet(Rate_)); + rate += (Alpha_ * (instantRate - rate)); + AtomicSet(Rate_, AsAtomic(rate)); + } else { + AtomicSet(Rate_, AsAtomic(instantRate)); + AtomicSet(Initialized_, 1); + } + } + + /** + * @return the rate in the seconds + */ + double GetRate() const { + double rate = AsDouble(AtomicGet(Rate_)); + return rate * std::nano::den; + } + + private: + static double AsDouble(TAtomicBase val) { + union { + double D; + TAtomicBase A; + } doubleAtomic; + doubleAtomic.A = val; + return doubleAtomic.D; + } + + static TAtomicBase AsAtomic(double val) { + union { + double D; + TAtomicBase A; + } doubleAtomic; + doubleAtomic.D = val; + return doubleAtomic.A; + } + + private: + TAtomic Initialized_; + TAtomic Rate_; + TAtomic Uncounted_; + double Alpha_; + double Interval_; + }; + + /** + * A meter metric which measures mean throughput and one-, five-, and + * fifteen-minute exponentially-weighted moving average throughputs. + */ + template <typename TClock> + class TMeterImpl: private TNonCopyable { + public: + TMeterImpl() + : StartTime_(TClock::now()) + , LastTick_(StartTime_.time_since_epoch().count()) + , Count_(0) + , OneMinuteRate_(TMovingAverage::OneMinute()) + , FiveMinutesRate_(TMovingAverage::FiveMinutes()) + , FifteenMinutesRate_(TMovingAverage::FifteenMinutes()) + { + } + + /** + * Mark the occurrence of events. + * + * @param n the number of events + */ + void Mark(ui64 n = 1) { + TickIfNecessary(); + AtomicAdd(Count_, n); + OneMinuteRate_.Update(n); + FiveMinutesRate_.Update(n); + FifteenMinutesRate_.Update(n); + } + + /** + * Returns the one-minute exponentially-weighted moving average rate at + * which events have occurred since the meter was created. + * + * This rate has the same exponential decay factor as the one-minute load + * average in the top Unix command. + * + * @return the one-minute exponentially-weighted moving average rate at + * which events have occurred since the meter was created + */ + double GetOneMinuteRate() const { + return OneMinuteRate_.GetRate(); + } + + /** + * Returns the five-minute exponentially-weighted moving average rate at + * which events have occurred since the meter was created. + * + * This rate has the same exponential decay factor as the five-minute load + * average in the top Unix command. + * + * @return the five-minute exponentially-weighted moving average rate at + * which events have occurred since the meter was created + */ + double GetFiveMinutesRate() const { + return FiveMinutesRate_.GetRate(); + } + + /** + * Returns the fifteen-minute exponentially-weighted moving average rate + * at which events have occurred since the meter was created. + * + * This rate has the same exponential decay factor as the fifteen-minute + * load average in the top Unix command. + * + * @return the fifteen-minute exponentially-weighted moving average rate + * at which events have occurred since the meter was created + */ + double GetFifteenMinutesRate() const { + return FifteenMinutesRate_.GetRate(); + } + + /** + * @return the mean rate at which events have occurred since the meter + * was created + */ + double GetMeanRate() const { + if (GetCount() == 0) { + return 0.0; + } + + auto now = TClock::now(); + std::chrono::duration<double> elapsedSeconds = now - StartTime_; + return GetCount() / elapsedSeconds.count(); + } + + /** + * @return the number of events which have been marked + */ + ui64 GetCount() const { + return AtomicGet(Count_); + } + + private: + void TickIfNecessary() { + static ui64 TICK_INTERVAL_NS = + std::chrono::nanoseconds( + std::chrono::seconds(TMovingAverage::INTERVAL)) + .count(); + + auto oldTickNs = AtomicGet(LastTick_); + auto newTickNs = TClock::now().time_since_epoch().count(); + ui64 elapsedNs = std::abs(newTickNs - oldTickNs); + + if (elapsedNs > TICK_INTERVAL_NS) { + // adjust to interval begining + newTickNs -= elapsedNs % TICK_INTERVAL_NS; + if (AtomicCas(&LastTick_, newTickNs, oldTickNs)) { + ui64 requiredTicks = elapsedNs / TICK_INTERVAL_NS; + for (ui64 i = 0; i < requiredTicks; ++i) { + OneMinuteRate_.Tick(); + FiveMinutesRate_.Tick(); + FifteenMinutesRate_.Tick(); + } + } + } + } + + private: + const typename TClock::time_point StartTime_; + TAtomic LastTick_; + TAtomic Count_; + TMovingAverage OneMinuteRate_; + TMovingAverage FiveMinutesRate_; + TMovingAverage FifteenMinutesRate_; + }; + + using TSystemMeter = TMeterImpl<std::chrono::system_clock>; + using TSteadyMeter = TMeterImpl<std::chrono::steady_clock>; + using THighResMeter = TMeterImpl<std::chrono::high_resolution_clock>; + using TMeter = THighResMeter; + +} diff --git a/library/cpp/monlib/counters/meter_ut.cpp b/library/cpp/monlib/counters/meter_ut.cpp index 39b1f0440a6..b507d16fbdd 100644 --- a/library/cpp/monlib/counters/meter_ut.cpp +++ b/library/cpp/monlib/counters/meter_ut.cpp @@ -1,41 +1,41 @@ -#include "meter.h" - +#include "meter.h" + #include <library/cpp/testing/unittest/registar.h> - -using namespace NMonitoring; - -struct TMockClock { - using duration = std::chrono::nanoseconds; - using rep = duration::rep; - using period = duration::period; - using time_point = std::chrono::time_point<TMockClock, duration>; - - static time_point now() noexcept { - static int index = 0; - return index++ < 2 ? time_point() : time_point(std::chrono::seconds(10)); - } -}; - -using TMockMeter = TMeterImpl<TMockClock>; - + +using namespace NMonitoring; + +struct TMockClock { + using duration = std::chrono::nanoseconds; + using rep = duration::rep; + using period = duration::period; + using time_point = std::chrono::time_point<TMockClock, duration>; + + static time_point now() noexcept { + static int index = 0; + return index++ < 2 ? time_point() : time_point(std::chrono::seconds(10)); + } +}; + +using TMockMeter = TMeterImpl<TMockClock>; + Y_UNIT_TEST_SUITE(TMeterTest) { Y_UNIT_TEST(StartsOutWithNoRatesOrCount) { - TMeter meter; - UNIT_ASSERT_EQUAL(meter.GetCount(), 0L); - UNIT_ASSERT_DOUBLES_EQUAL(meter.GetMeanRate(), 0.0, 0.0001); - UNIT_ASSERT_DOUBLES_EQUAL(meter.GetOneMinuteRate(), 0.0, 0.0001); - UNIT_ASSERT_DOUBLES_EQUAL(meter.GetFiveMinutesRate(), 0.0, 0.0001); - UNIT_ASSERT_DOUBLES_EQUAL(meter.GetFifteenMinutesRate(), 0.0, 0.0001); - } - + TMeter meter; + UNIT_ASSERT_EQUAL(meter.GetCount(), 0L); + UNIT_ASSERT_DOUBLES_EQUAL(meter.GetMeanRate(), 0.0, 0.0001); + UNIT_ASSERT_DOUBLES_EQUAL(meter.GetOneMinuteRate(), 0.0, 0.0001); + UNIT_ASSERT_DOUBLES_EQUAL(meter.GetFiveMinutesRate(), 0.0, 0.0001); + UNIT_ASSERT_DOUBLES_EQUAL(meter.GetFifteenMinutesRate(), 0.0, 0.0001); + } + Y_UNIT_TEST(MarksEventsAndUpdatesRatesAndCount) { - TMockMeter meter; - meter.Mark(); - meter.Mark(2); - UNIT_ASSERT_EQUAL(meter.GetCount(), 3L); - UNIT_ASSERT_DOUBLES_EQUAL(meter.GetMeanRate(), 0.3, 0.001); - UNIT_ASSERT_DOUBLES_EQUAL(meter.GetOneMinuteRate(), 0.1840, 0.0001); - UNIT_ASSERT_DOUBLES_EQUAL(meter.GetFiveMinutesRate(), 0.1966, 0.0001); - UNIT_ASSERT_DOUBLES_EQUAL(meter.GetFifteenMinutesRate(), 0.1988, 0.0001); - } -} + TMockMeter meter; + meter.Mark(); + meter.Mark(2); + UNIT_ASSERT_EQUAL(meter.GetCount(), 3L); + UNIT_ASSERT_DOUBLES_EQUAL(meter.GetMeanRate(), 0.3, 0.001); + UNIT_ASSERT_DOUBLES_EQUAL(meter.GetOneMinuteRate(), 0.1840, 0.0001); + UNIT_ASSERT_DOUBLES_EQUAL(meter.GetFiveMinutesRate(), 0.1966, 0.0001); + UNIT_ASSERT_DOUBLES_EQUAL(meter.GetFifteenMinutesRate(), 0.1988, 0.0001); + } +} diff --git a/library/cpp/monlib/counters/timer.h b/library/cpp/monlib/counters/timer.h index 5b234e09078..03dfb35337f 100644 --- a/library/cpp/monlib/counters/timer.h +++ b/library/cpp/monlib/counters/timer.h @@ -1,176 +1,176 @@ -#pragma once - -#include "histogram.h" - +#pragma once + +#include "histogram.h" + #include <util/generic/scope.h> -#include <chrono> - -namespace NMonitoring { - /** - * A timer counter which aggregates timing durations and provides duration - * statistics in selected time resolution. - */ - template <typename TResolution> - class TTimerImpl { - public: - /** - * Construct a timer given the Lowest and Highest values to be tracked - * and a number of significant decimal digits. Providing a - * lowestDiscernibleValue is useful in situations where the units used for - * the timer's values are much smaller that the minimal accuracy - * required. E.g. when tracking time values stated in nanosecond units, - * where the minimal accuracy required is a microsecond, the proper value - * for lowestDiscernibleValue would be 1000. - * - * @param min The lowest value that can be discerned (distinguished from - * 0) by the timer. Must be a positive integer that is >= 1. - * May be internally rounded down to nearest power of 2. - * - * @param max The highest value to be tracked by the timer. Must be a - * positive integer that is >= (2 * min). - * - * @param numberOfSignificantValueDigits Specifies the precision to use. - * This is the number of significant decimal digits to which the - * timer will maintain value resolution and separation. Must be - * a non-negative integer between 0 and 5. - */ - TTimerImpl(ui64 min, ui64 max, i32 numberOfSignificantValueDigits = 3) - : TTimerImpl(TResolution(min), TResolution(max), - numberOfSignificantValueDigits) { - } - - /** - * Construct a timer given the Lowest and Highest values to be tracked - * and a number of significant decimal digits. - * - * @param min The lowest value that can be discerned (distinguished from - * 0) by the timer. - * - * @param max The highest value to be tracked by the histogram. Must be a - * positive integer that is >= (2 * min). - * - * @param numberOfSignificantValueDigits Specifies the precision to use. - */ - template <typename TDurationMin, typename TDurationMax> - TTimerImpl(TDurationMin min, TDurationMax max, - i32 numberOfSignificantValueDigits = 3) - : Histogram_(std::chrono::duration_cast<TResolution>(min).count(), - std::chrono::duration_cast<TResolution>(max).count(), - numberOfSignificantValueDigits) { - } - - /** - * Records a value in the timer with timer resulution. Recorded value will - * be rounded to a precision at or better than the - * NumberOfSignificantValueDigits specified at construction time. - * - * @param duration duration to add to the timer - * @return false if the value is larger than the max and can't be recorded, - * true otherwise. - */ - bool RecordValue(ui64 duration) { - return Histogram_.RecordValue(duration); - } - - /** - * Records a duration in the timer. Recorded value will be converted to - * the timer resulution and rounded to a precision at or better than the - * NumberOfSignificantValueDigits specified at construction time. - * - * @param duration duration to add to the timer - * @return false if the value is larger than the max and can't be recorded, - * true otherwise. - */ - template <typename TDuration> - bool RecordValue(TDuration duration) { - auto count = static_cast<ui64>( - std::chrono::duration_cast<TResolution>(duration).count()); - return RecordValue(count); - } - - /** - * Records count values in the timer with timer resulution. Recorded value will - * be rounded to a precision at or better than the - * NumberOfSignificantValueDigits specified at construction time. - * - * @param duration duration to add to the timer - * @param count number of values to add to the histogram - * @return false if the value is larger than the max and can't be recorded, - * true otherwise. - */ - bool RecordValues(ui64 duration, ui64 count) { - return Histogram_.RecordValues(duration, count); - } - - /** - * Measures a time of functor execution. - * - * @param fn functor whose duration should be timed - */ - template <typename TFunc> - void Measure(TFunc&& fn) { - using TClock = std::chrono::high_resolution_clock; - - auto start = TClock::now(); +#include <chrono> + +namespace NMonitoring { + /** + * A timer counter which aggregates timing durations and provides duration + * statistics in selected time resolution. + */ + template <typename TResolution> + class TTimerImpl { + public: + /** + * Construct a timer given the Lowest and Highest values to be tracked + * and a number of significant decimal digits. Providing a + * lowestDiscernibleValue is useful in situations where the units used for + * the timer's values are much smaller that the minimal accuracy + * required. E.g. when tracking time values stated in nanosecond units, + * where the minimal accuracy required is a microsecond, the proper value + * for lowestDiscernibleValue would be 1000. + * + * @param min The lowest value that can be discerned (distinguished from + * 0) by the timer. Must be a positive integer that is >= 1. + * May be internally rounded down to nearest power of 2. + * + * @param max The highest value to be tracked by the timer. Must be a + * positive integer that is >= (2 * min). + * + * @param numberOfSignificantValueDigits Specifies the precision to use. + * This is the number of significant decimal digits to which the + * timer will maintain value resolution and separation. Must be + * a non-negative integer between 0 and 5. + */ + TTimerImpl(ui64 min, ui64 max, i32 numberOfSignificantValueDigits = 3) + : TTimerImpl(TResolution(min), TResolution(max), + numberOfSignificantValueDigits) { + } + + /** + * Construct a timer given the Lowest and Highest values to be tracked + * and a number of significant decimal digits. + * + * @param min The lowest value that can be discerned (distinguished from + * 0) by the timer. + * + * @param max The highest value to be tracked by the histogram. Must be a + * positive integer that is >= (2 * min). + * + * @param numberOfSignificantValueDigits Specifies the precision to use. + */ + template <typename TDurationMin, typename TDurationMax> + TTimerImpl(TDurationMin min, TDurationMax max, + i32 numberOfSignificantValueDigits = 3) + : Histogram_(std::chrono::duration_cast<TResolution>(min).count(), + std::chrono::duration_cast<TResolution>(max).count(), + numberOfSignificantValueDigits) { + } + + /** + * Records a value in the timer with timer resulution. Recorded value will + * be rounded to a precision at or better than the + * NumberOfSignificantValueDigits specified at construction time. + * + * @param duration duration to add to the timer + * @return false if the value is larger than the max and can't be recorded, + * true otherwise. + */ + bool RecordValue(ui64 duration) { + return Histogram_.RecordValue(duration); + } + + /** + * Records a duration in the timer. Recorded value will be converted to + * the timer resulution and rounded to a precision at or better than the + * NumberOfSignificantValueDigits specified at construction time. + * + * @param duration duration to add to the timer + * @return false if the value is larger than the max and can't be recorded, + * true otherwise. + */ + template <typename TDuration> + bool RecordValue(TDuration duration) { + auto count = static_cast<ui64>( + std::chrono::duration_cast<TResolution>(duration).count()); + return RecordValue(count); + } + + /** + * Records count values in the timer with timer resulution. Recorded value will + * be rounded to a precision at or better than the + * NumberOfSignificantValueDigits specified at construction time. + * + * @param duration duration to add to the timer + * @param count number of values to add to the histogram + * @return false if the value is larger than the max and can't be recorded, + * true otherwise. + */ + bool RecordValues(ui64 duration, ui64 count) { + return Histogram_.RecordValues(duration, count); + } + + /** + * Measures a time of functor execution. + * + * @param fn functor whose duration should be timed + */ + template <typename TFunc> + void Measure(TFunc&& fn) { + using TClock = std::chrono::high_resolution_clock; + + auto start = TClock::now(); Y_SCOPE_EXIT(this, start) { - RecordValue(TClock::now() - start); - }; - - fn(); - } - - /** - * Place a copy of the value counts accumulated since the last snapshot - * was taken into {@code snapshot}. Calling this member-function will - * reset the value counts, and start accumulating value counts for the - * next interval. - * - * @param snapshot the structure into which the values should be copied. - */ - void TakeSnapshot(THistogramSnapshot* snapshot) { - Histogram_.TakeSnaphot(snapshot); - } - - private: - THdrHistogram Histogram_; - }; - - /** - * Timer template instantiations for certain time resolutions. - */ - using TTimerNs = TTimerImpl<std::chrono::nanoseconds>; - using TTimerUs = TTimerImpl<std::chrono::microseconds>; - using TTimerMs = TTimerImpl<std::chrono::milliseconds>; - using TTimerS = TTimerImpl<std::chrono::seconds>; - - /** - * A timing scope to record elapsed time since creation. - */ - template <typename TTimer, typename TFunc = std::function<void(std::chrono::high_resolution_clock::duration)>> - class TTimerScope { - using TClock = std::chrono::high_resolution_clock; - - public: - explicit TTimerScope(TTimer* timer, TFunc* callback = nullptr) - : Timer_(timer) - , StartTime_(TClock::now()) - , Callback_(callback) - { - } - - ~TTimerScope() { - TClock::duration duration = TClock::now() - StartTime_; - if (Callback_) { - (*Callback_)(duration); - } - Timer_->RecordValue(duration); + RecordValue(TClock::now() - start); + }; + + fn(); + } + + /** + * Place a copy of the value counts accumulated since the last snapshot + * was taken into {@code snapshot}. Calling this member-function will + * reset the value counts, and start accumulating value counts for the + * next interval. + * + * @param snapshot the structure into which the values should be copied. + */ + void TakeSnapshot(THistogramSnapshot* snapshot) { + Histogram_.TakeSnaphot(snapshot); + } + + private: + THdrHistogram Histogram_; + }; + + /** + * Timer template instantiations for certain time resolutions. + */ + using TTimerNs = TTimerImpl<std::chrono::nanoseconds>; + using TTimerUs = TTimerImpl<std::chrono::microseconds>; + using TTimerMs = TTimerImpl<std::chrono::milliseconds>; + using TTimerS = TTimerImpl<std::chrono::seconds>; + + /** + * A timing scope to record elapsed time since creation. + */ + template <typename TTimer, typename TFunc = std::function<void(std::chrono::high_resolution_clock::duration)>> + class TTimerScope { + using TClock = std::chrono::high_resolution_clock; + + public: + explicit TTimerScope(TTimer* timer, TFunc* callback = nullptr) + : Timer_(timer) + , StartTime_(TClock::now()) + , Callback_(callback) + { } - - private: - TTimer* Timer_; - TClock::time_point StartTime_; - TFunc* Callback_; - }; -} + + ~TTimerScope() { + TClock::duration duration = TClock::now() - StartTime_; + if (Callback_) { + (*Callback_)(duration); + } + Timer_->RecordValue(duration); + } + + private: + TTimer* Timer_; + TClock::time_point StartTime_; + TFunc* Callback_; + }; +} diff --git a/library/cpp/monlib/counters/timer_ut.cpp b/library/cpp/monlib/counters/timer_ut.cpp index f92d38a445b..c5cd07e89d5 100644 --- a/library/cpp/monlib/counters/timer_ut.cpp +++ b/library/cpp/monlib/counters/timer_ut.cpp @@ -1,14 +1,14 @@ -#include "timer.h" - +#include "timer.h" + #include <library/cpp/testing/unittest/registar.h> - -using namespace NMonitoring; -using namespace std::literals::chrono_literals; - + +using namespace NMonitoring; +using namespace std::literals::chrono_literals; + class TCallback { public: - explicit TCallback(int value) - : Value_(value){}; + explicit TCallback(int value) + : Value_(value){}; void operator()(std::chrono::high_resolution_clock::duration duration) { Value_ = duration.count(); }; @@ -18,50 +18,50 @@ public: Y_UNIT_TEST_SUITE(TTimerTest) { Y_UNIT_TEST(RecordValue) { - TTimerNs timerNs(1ns, 1s); - UNIT_ASSERT(timerNs.RecordValue(10us)); - - TTimerUs timerUs(1us, 1s); - UNIT_ASSERT(timerUs.RecordValue(10us)); - - THistogramSnapshot snapshot; - timerNs.TakeSnapshot(&snapshot); - UNIT_ASSERT_EQUAL(snapshot.Min, 10000); - UNIT_ASSERT_EQUAL(snapshot.Max, 10007); - UNIT_ASSERT_DOUBLES_EQUAL(snapshot.StdDeviation, 0.0, 1e-6); - - timerUs.TakeSnapshot(&snapshot); - UNIT_ASSERT_EQUAL(snapshot.Min, 10); - UNIT_ASSERT_EQUAL(snapshot.Max, 10); - UNIT_ASSERT_DOUBLES_EQUAL(snapshot.StdDeviation, 0.0, 1e-6); - } - + TTimerNs timerNs(1ns, 1s); + UNIT_ASSERT(timerNs.RecordValue(10us)); + + TTimerUs timerUs(1us, 1s); + UNIT_ASSERT(timerUs.RecordValue(10us)); + + THistogramSnapshot snapshot; + timerNs.TakeSnapshot(&snapshot); + UNIT_ASSERT_EQUAL(snapshot.Min, 10000); + UNIT_ASSERT_EQUAL(snapshot.Max, 10007); + UNIT_ASSERT_DOUBLES_EQUAL(snapshot.StdDeviation, 0.0, 1e-6); + + timerUs.TakeSnapshot(&snapshot); + UNIT_ASSERT_EQUAL(snapshot.Min, 10); + UNIT_ASSERT_EQUAL(snapshot.Max, 10); + UNIT_ASSERT_DOUBLES_EQUAL(snapshot.StdDeviation, 0.0, 1e-6); + } + Y_UNIT_TEST(Measure) { - TTimerNs timer(1ns, 1s); - timer.Measure([]() { - Sleep(TDuration::MilliSeconds(1)); - }); - THistogramSnapshot snapshot; - timer.TakeSnapshot(&snapshot); - - UNIT_ASSERT(snapshot.Min > std::chrono::nanoseconds(1ms).count()); - UNIT_ASSERT(snapshot.Max > std::chrono::nanoseconds(1ms).count()); - UNIT_ASSERT_DOUBLES_EQUAL(snapshot.StdDeviation, 0.0, 1e-6); - } - + TTimerNs timer(1ns, 1s); + timer.Measure([]() { + Sleep(TDuration::MilliSeconds(1)); + }); + THistogramSnapshot snapshot; + timer.TakeSnapshot(&snapshot); + + UNIT_ASSERT(snapshot.Min > std::chrono::nanoseconds(1ms).count()); + UNIT_ASSERT(snapshot.Max > std::chrono::nanoseconds(1ms).count()); + UNIT_ASSERT_DOUBLES_EQUAL(snapshot.StdDeviation, 0.0, 1e-6); + } + Y_UNIT_TEST(TimerScope) { - TTimerUs timer(1us, 1000s); - { - TTimerScope<TTimerUs> scope(&timer); - Sleep(TDuration::MilliSeconds(10)); - } - THistogramSnapshot snapshot; - timer.TakeSnapshot(&snapshot); - - UNIT_ASSERT(snapshot.Min > std::chrono::microseconds(10ms).count()); - UNIT_ASSERT(snapshot.Max > std::chrono::microseconds(10ms).count()); - UNIT_ASSERT_DOUBLES_EQUAL(snapshot.StdDeviation, 0.0, 1e-6); - } + TTimerUs timer(1us, 1000s); + { + TTimerScope<TTimerUs> scope(&timer); + Sleep(TDuration::MilliSeconds(10)); + } + THistogramSnapshot snapshot; + timer.TakeSnapshot(&snapshot); + + UNIT_ASSERT(snapshot.Min > std::chrono::microseconds(10ms).count()); + UNIT_ASSERT(snapshot.Max > std::chrono::microseconds(10ms).count()); + UNIT_ASSERT_DOUBLES_EQUAL(snapshot.StdDeviation, 0.0, 1e-6); + } Y_UNIT_TEST(TimerScopeWithCallback) { TCallback callback(0); @@ -78,4 +78,4 @@ Y_UNIT_TEST_SUITE(TTimerTest) { UNIT_ASSERT_DOUBLES_EQUAL(snapshot.StdDeviation, 0.0, 1e-6); UNIT_ASSERT(callback.Value_ > std::chrono::microseconds(10ms).count()); } -} +} diff --git a/library/cpp/monlib/counters/ut/ya.make b/library/cpp/monlib/counters/ut/ya.make index 3c8540c67d5..999dadb199d 100644 --- a/library/cpp/monlib/counters/ut/ya.make +++ b/library/cpp/monlib/counters/ut/ya.make @@ -1,12 +1,12 @@ -UNITTEST_FOR(library/cpp/monlib/counters) - -OWNER(jamel) - -SRCS( - counters_ut.cpp - histogram_ut.cpp - meter_ut.cpp - timer_ut.cpp -) - -END() +UNITTEST_FOR(library/cpp/monlib/counters) + +OWNER(jamel) + +SRCS( + counters_ut.cpp + histogram_ut.cpp + meter_ut.cpp + timer_ut.cpp +) + +END() diff --git a/library/cpp/monlib/counters/ya.make b/library/cpp/monlib/counters/ya.make index d5696080351..aa1a671bf80 100644 --- a/library/cpp/monlib/counters/ya.make +++ b/library/cpp/monlib/counters/ya.make @@ -1,15 +1,15 @@ -LIBRARY() - -OWNER(jamel) - -SRCS( - counters.cpp - histogram.cpp - meter.cpp -) - -PEERDIR( +LIBRARY() + +OWNER(jamel) + +SRCS( + counters.cpp + histogram.cpp + meter.cpp +) + +PEERDIR( library/cpp/histogram/hdr -) - -END() +) + +END() diff --git a/library/cpp/monlib/deprecated/json/ut/ya.make b/library/cpp/monlib/deprecated/json/ut/ya.make index 3c4c338882a..18315993b51 100644 --- a/library/cpp/monlib/deprecated/json/ut/ya.make +++ b/library/cpp/monlib/deprecated/json/ut/ya.make @@ -1,4 +1,4 @@ -UNITTEST_FOR(library/cpp/monlib/deprecated/json) +UNITTEST_FOR(library/cpp/monlib/deprecated/json) OWNER( jamel @@ -6,7 +6,7 @@ OWNER( ) SRCS( - writer_ut.cpp + writer_ut.cpp ) END() diff --git a/library/cpp/monlib/deprecated/json/writer.cpp b/library/cpp/monlib/deprecated/json/writer.cpp index fca3834ebc3..a581f2e07ac 100644 --- a/library/cpp/monlib/deprecated/json/writer.cpp +++ b/library/cpp/monlib/deprecated/json/writer.cpp @@ -1,100 +1,100 @@ -#include "writer.h" - -namespace NMonitoring { - TDeprecatedJsonWriter::TDeprecatedJsonWriter(IOutputStream* out) - : JsonWriter(out, false) - , State(STATE_ROOT) - { - } - - void TDeprecatedJsonWriter::TransitionState(EState current, EState next) { - if (State != current) { - ythrow yexception() << "wrong state"; - } - State = next; - } - - void TDeprecatedJsonWriter::OpenDocument() { - TransitionState(STATE_ROOT, STATE_DOCUMENT); - JsonWriter.OpenMap(); - } - - void TDeprecatedJsonWriter::CloseDocument() { - TransitionState(STATE_DOCUMENT, STATE_ROOT); - JsonWriter.CloseMap(); - JsonWriter.Flush(); - } - - void TDeprecatedJsonWriter::OpenCommonLabels() { - TransitionState(STATE_DOCUMENT, STATE_COMMON_LABELS); - JsonWriter.Write("commonLabels"); - JsonWriter.OpenMap(); - } - - void TDeprecatedJsonWriter::CloseCommonLabels() { - TransitionState(STATE_COMMON_LABELS, STATE_DOCUMENT); - JsonWriter.CloseMap(); - } - - void TDeprecatedJsonWriter::WriteCommonLabel(TStringBuf name, TStringBuf value) { - TransitionState(STATE_COMMON_LABELS, STATE_COMMON_LABELS); - JsonWriter.Write(name, value); - } - - void TDeprecatedJsonWriter::OpenMetrics() { - TransitionState(STATE_DOCUMENT, STATE_METRICS); - JsonWriter.Write("sensors"); - JsonWriter.OpenArray(); - } - - void TDeprecatedJsonWriter::CloseMetrics() { - TransitionState(STATE_METRICS, STATE_DOCUMENT); - JsonWriter.CloseArray(); - } - - void TDeprecatedJsonWriter::OpenMetric() { - TransitionState(STATE_METRICS, STATE_METRIC); - JsonWriter.OpenMap(); - } - - void TDeprecatedJsonWriter::CloseMetric() { - TransitionState(STATE_METRIC, STATE_METRICS); - JsonWriter.CloseMap(); - } - - void TDeprecatedJsonWriter::OpenLabels() { - TransitionState(STATE_METRIC, STATE_LABELS); - JsonWriter.Write("labels"); - JsonWriter.OpenMap(); - } - - void TDeprecatedJsonWriter::CloseLabels() { - TransitionState(STATE_LABELS, STATE_METRIC); - JsonWriter.CloseMap(); - } - - void TDeprecatedJsonWriter::WriteLabel(TStringBuf name, TStringBuf value) { - TransitionState(STATE_LABELS, STATE_LABELS); - JsonWriter.Write(name, value); - } - - void TDeprecatedJsonWriter::WriteModeDeriv() { - TransitionState(STATE_METRIC, STATE_METRIC); - JsonWriter.Write("mode", "deriv"); - } - - void TDeprecatedJsonWriter::WriteValue(long long value) { - TransitionState(STATE_METRIC, STATE_METRIC); - JsonWriter.Write("value", value); - } - - void TDeprecatedJsonWriter::WriteDoubleValue(double value) { - TransitionState(STATE_METRIC, STATE_METRIC); - JsonWriter.Write("value", value); - } - - void TDeprecatedJsonWriter::WriteTs(ui64 ts) { - TransitionState(STATE_METRIC, STATE_METRIC); - JsonWriter.Write("ts", ts); - } +#include "writer.h" + +namespace NMonitoring { + TDeprecatedJsonWriter::TDeprecatedJsonWriter(IOutputStream* out) + : JsonWriter(out, false) + , State(STATE_ROOT) + { + } + + void TDeprecatedJsonWriter::TransitionState(EState current, EState next) { + if (State != current) { + ythrow yexception() << "wrong state"; + } + State = next; + } + + void TDeprecatedJsonWriter::OpenDocument() { + TransitionState(STATE_ROOT, STATE_DOCUMENT); + JsonWriter.OpenMap(); + } + + void TDeprecatedJsonWriter::CloseDocument() { + TransitionState(STATE_DOCUMENT, STATE_ROOT); + JsonWriter.CloseMap(); + JsonWriter.Flush(); + } + + void TDeprecatedJsonWriter::OpenCommonLabels() { + TransitionState(STATE_DOCUMENT, STATE_COMMON_LABELS); + JsonWriter.Write("commonLabels"); + JsonWriter.OpenMap(); + } + + void TDeprecatedJsonWriter::CloseCommonLabels() { + TransitionState(STATE_COMMON_LABELS, STATE_DOCUMENT); + JsonWriter.CloseMap(); + } + + void TDeprecatedJsonWriter::WriteCommonLabel(TStringBuf name, TStringBuf value) { + TransitionState(STATE_COMMON_LABELS, STATE_COMMON_LABELS); + JsonWriter.Write(name, value); + } + + void TDeprecatedJsonWriter::OpenMetrics() { + TransitionState(STATE_DOCUMENT, STATE_METRICS); + JsonWriter.Write("sensors"); + JsonWriter.OpenArray(); + } + + void TDeprecatedJsonWriter::CloseMetrics() { + TransitionState(STATE_METRICS, STATE_DOCUMENT); + JsonWriter.CloseArray(); + } + + void TDeprecatedJsonWriter::OpenMetric() { + TransitionState(STATE_METRICS, STATE_METRIC); + JsonWriter.OpenMap(); + } + + void TDeprecatedJsonWriter::CloseMetric() { + TransitionState(STATE_METRIC, STATE_METRICS); + JsonWriter.CloseMap(); + } + + void TDeprecatedJsonWriter::OpenLabels() { + TransitionState(STATE_METRIC, STATE_LABELS); + JsonWriter.Write("labels"); + JsonWriter.OpenMap(); + } + + void TDeprecatedJsonWriter::CloseLabels() { + TransitionState(STATE_LABELS, STATE_METRIC); + JsonWriter.CloseMap(); + } + + void TDeprecatedJsonWriter::WriteLabel(TStringBuf name, TStringBuf value) { + TransitionState(STATE_LABELS, STATE_LABELS); + JsonWriter.Write(name, value); + } + + void TDeprecatedJsonWriter::WriteModeDeriv() { + TransitionState(STATE_METRIC, STATE_METRIC); + JsonWriter.Write("mode", "deriv"); + } + + void TDeprecatedJsonWriter::WriteValue(long long value) { + TransitionState(STATE_METRIC, STATE_METRIC); + JsonWriter.Write("value", value); + } + + void TDeprecatedJsonWriter::WriteDoubleValue(double value) { + TransitionState(STATE_METRIC, STATE_METRIC); + JsonWriter.Write("value", value); + } + + void TDeprecatedJsonWriter::WriteTs(ui64 ts) { + TransitionState(STATE_METRIC, STATE_METRIC); + JsonWriter.Write("ts", ts); + } } diff --git a/library/cpp/monlib/deprecated/json/writer.h b/library/cpp/monlib/deprecated/json/writer.h index 003992dd9f6..183288143c1 100644 --- a/library/cpp/monlib/deprecated/json/writer.h +++ b/library/cpp/monlib/deprecated/json/writer.h @@ -2,30 +2,30 @@ #include <library/cpp/json/json_writer.h> -namespace NMonitoring { - /** - * Deprecated writer of Solomon JSON format - * https://wiki.yandex-team.ru/solomon/api/dataformat/json - * - * This writer will be deleted soon, so please consider to use - * high level library library/cpp/monlib/encode which is decoupled from the - * particular format. - */ - class TDeprecatedJsonWriter { +namespace NMonitoring { + /** + * Deprecated writer of Solomon JSON format + * https://wiki.yandex-team.ru/solomon/api/dataformat/json + * + * This writer will be deleted soon, so please consider to use + * high level library library/cpp/monlib/encode which is decoupled from the + * particular format. + */ + class TDeprecatedJsonWriter { private: NJson::TJsonWriter JsonWriter; enum EState { STATE_ROOT, STATE_DOCUMENT, STATE_COMMON_LABELS, - STATE_METRICS, - STATE_METRIC, + STATE_METRICS, + STATE_METRIC, STATE_LABELS, }; EState State; - + public: - explicit TDeprecatedJsonWriter(IOutputStream* out); + explicit TDeprecatedJsonWriter(IOutputStream* out); void OpenDocument(); void CloseDocument(); @@ -35,11 +35,11 @@ namespace NMonitoring { void WriteCommonLabel(TStringBuf name, TStringBuf value); - void OpenMetrics(); - void CloseMetrics(); + void OpenMetrics(); + void CloseMetrics(); - void OpenMetric(); - void CloseMetric(); + void OpenMetric(); + void CloseMetric(); void OpenLabels(); void CloseLabels(); @@ -59,13 +59,13 @@ namespace NMonitoring { void WriteDoubleValue(double d); void WriteTs(ui64 ts); - + private: void WriteLabelsInner(TStringBuf name, TStringBuf value) { WriteLabel(name, value); } - template <typename... T> + template <typename... T> void WriteLabelsInner(TStringBuf name, TStringBuf value, T... pairs) { WriteLabel(name, value); WriteLabelsInner(pairs...); @@ -73,4 +73,4 @@ namespace NMonitoring { inline void TransitionState(EState current, EState next); }; -} +} diff --git a/library/cpp/monlib/deprecated/json/writer_ut.cpp b/library/cpp/monlib/deprecated/json/writer_ut.cpp index f39fe85f1c4..1f9fc8f3933 100644 --- a/library/cpp/monlib/deprecated/json/writer_ut.cpp +++ b/library/cpp/monlib/deprecated/json/writer_ut.cpp @@ -1,30 +1,30 @@ -#include "writer.h" - +#include "writer.h" + #include <library/cpp/testing/unittest/registar.h> -using namespace NMonitoring; +using namespace NMonitoring; -Y_UNIT_TEST_SUITE(JsonWriterTests) { +Y_UNIT_TEST_SUITE(JsonWriterTests) { Y_UNIT_TEST(One) { TStringStream ss; - TDeprecatedJsonWriter w(&ss); + TDeprecatedJsonWriter w(&ss); w.OpenDocument(); - w.OpenMetrics(); + w.OpenMetrics(); for (int i = 0; i < 5; ++i) { - w.OpenMetric(); + w.OpenMetric(); w.OpenLabels(); - w.WriteLabel("user", TString("") + (char)('a' + i)); - w.WriteLabel("name", "NWrites"); + w.WriteLabel("user", TString("") + (char)('a' + i)); + w.WriteLabel("name", "NWrites"); w.CloseLabels(); if (i % 2 == 0) { w.WriteModeDeriv(); } w.WriteValue(10l); - w.CloseMetric(); + w.CloseMetric(); } - w.CloseMetrics(); + w.CloseMetrics(); w.CloseDocument(); //Cout << ss.Str() << "\n"; diff --git a/library/cpp/monlib/deprecated/json/ya.make b/library/cpp/monlib/deprecated/json/ya.make index b20030dd31a..0ca903ee62b 100644 --- a/library/cpp/monlib/deprecated/json/ya.make +++ b/library/cpp/monlib/deprecated/json/ya.make @@ -1,26 +1,26 @@ LIBRARY() -# Deprecated writer of Solomon JSON format +# Deprecated writer of Solomon JSON format # https://wiki.yandex-team.ru/solomon/api/dataformat/json -# -# This writer will be deleted soon, so please consider to use -# high level library library/cpp/monlib/encode which is decoupled from the -# particular format. +# +# This writer will be deleted soon, so please consider to use +# high level library library/cpp/monlib/encode which is decoupled from the +# particular format. OWNER( jamel g:solomon ) -SRCS( - writer.h - writer.cpp -) - +SRCS( + writer.h + writer.cpp +) + PEERDIR( library/cpp/json ) END() - -RECURSE_FOR_TESTS(ut) + +RECURSE_FOR_TESTS(ut) diff --git a/library/cpp/monlib/deprecated/ya.make b/library/cpp/monlib/deprecated/ya.make index 7d541ab2a8f..9345139aee8 100644 --- a/library/cpp/monlib/deprecated/ya.make +++ b/library/cpp/monlib/deprecated/ya.make @@ -1,8 +1,8 @@ -OWNER( - g:solomon - jamel -) - -RECURSE( - json -) +OWNER( + g:solomon + jamel +) + +RECURSE( + json +) diff --git a/library/cpp/monlib/dynamic_counters/contention_ut.cpp b/library/cpp/monlib/dynamic_counters/contention_ut.cpp index 9b6f8af254d..8798044ee3c 100644 --- a/library/cpp/monlib/dynamic_counters/contention_ut.cpp +++ b/library/cpp/monlib/dynamic_counters/contention_ut.cpp @@ -28,7 +28,7 @@ Y_UNIT_TEST_SUITE(TDynamicCountersContentionTest) { Thread.Join(); } - void OnCounter(const TString& /*labelName*/, const TString& /*labelValue*/, const TCounterForPtr* /*counter*/) override { + void OnCounter(const TString& /*labelName*/, const TString& /*labelValue*/, const TCounterForPtr* /*counter*/) override { Ev.Signal(); Response.Wait(); } @@ -36,10 +36,10 @@ Y_UNIT_TEST_SUITE(TDynamicCountersContentionTest) { void OnHistogram(const TString& /*labelName*/, const TString& /*labelValue*/, IHistogramSnapshotPtr /*snapshot*/, bool /*derivative*/) override { } - void OnGroupBegin(const TString& /*labelName*/, const TString& /*labelValue*/, const TDynamicCounters* /*group*/) override { + void OnGroupBegin(const TString& /*labelName*/, const TString& /*labelValue*/, const TDynamicCounters* /*group*/) override { } - void OnGroupEnd(const TString& /*labelName*/, const TString& /*labelValue*/, const TDynamicCounters* /*group*/) override { + void OnGroupEnd(const TString& /*labelName*/, const TString& /*labelValue*/, const TDynamicCounters* /*group*/) override { } private: diff --git a/library/cpp/monlib/dynamic_counters/counters.cpp b/library/cpp/monlib/dynamic_counters/counters.cpp index 96de74b3c94..3635d87d0d2 100644 --- a/library/cpp/monlib/dynamic_counters/counters.cpp +++ b/library/cpp/monlib/dynamic_counters/counters.cpp @@ -1,7 +1,7 @@ -#include "counters.h" - -#include <library/cpp/monlib/service/pages/templates.h> - +#include "counters.h" + +#include <library/cpp/monlib/service/pages/templates.h> + #include <util/generic/cast.h> using namespace NMonitoring; @@ -45,14 +45,14 @@ namespace { } static constexpr TStringBuf INDENT = " "; - + TDynamicCounters::TDynamicCounters(EVisibility vis) { Visibility_ = vis; -} +} -TDynamicCounters::~TDynamicCounters() { -} +TDynamicCounters::~TDynamicCounters() { +} TDynamicCounters::TCounterPtr TDynamicCounters::GetExpiringCounter(const TString& value, bool derivative, EVisibility vis) { return GetExpiringNamedCounter("sensor", value, derivative, vis); @@ -72,20 +72,20 @@ TDynamicCounters::TCounterPtr TDynamicCounters::GetNamedCounter(const TString& n THistogramPtr TDynamicCounters::GetHistogram(const TString& value, IHistogramCollectorPtr collector, bool derivative, EVisibility vis) { return GetNamedHistogram("sensor", value, std::move(collector), derivative, vis); -} - +} + THistogramPtr TDynamicCounters::GetNamedHistogram(const TString& name, const TString& value, IHistogramCollectorPtr collector, bool derivative, EVisibility vis) { return AsHistogramRef(GetNamedCounterImpl<false, THistogramCounter>(name, value, std::move(collector), derivative, vis)); } - + THistogramPtr TDynamicCounters::GetExpiringHistogram(const TString& value, IHistogramCollectorPtr collector, bool derivative, EVisibility vis) { return GetExpiringNamedHistogram("sensor", value, std::move(collector), derivative, vis); } - + THistogramPtr TDynamicCounters::GetExpiringNamedHistogram(const TString& name, const TString& value, IHistogramCollectorPtr collector, bool derivative, EVisibility vis) { return AsHistogramRef(GetNamedCounterImpl<true, TExpiringHistogramCounter>(name, value, std::move(collector), derivative, vis)); -} - +} + TDynamicCounters::TCounterPtr TDynamicCounters::FindCounter(const TString& value) const { return FindNamedCounter("sensor", value); } @@ -113,7 +113,7 @@ void TDynamicCounters::RemoveNamedCounter(const TString& name, const TString &va } } -TIntrusivePtr<TDynamicCounters> TDynamicCounters::GetSubgroup(const TString& name, const TString& value) { +TIntrusivePtr<TDynamicCounters> TDynamicCounters::GetSubgroup(const TString& name, const TString& value) { auto res = FindSubgroup(name, value); if (!res) { auto g = LockForUpdate("GetSubgroup", name, value); @@ -128,7 +128,7 @@ TIntrusivePtr<TDynamicCounters> TDynamicCounters::GetSubgroup(const TString& nam return res; } -TIntrusivePtr<TDynamicCounters> TDynamicCounters::FindSubgroup(const TString& name, const TString& value) const { +TIntrusivePtr<TDynamicCounters> TDynamicCounters::FindSubgroup(const TString& name, const TString& value) const { TReadGuard g(Lock); const auto it = Counters.find({name, value}); return it != Counters.end() ? AsDynamicCounters(it->second) : nullptr; @@ -183,7 +183,7 @@ void TDynamicCounters::RegisterSubgroup(const TString& name, const TString& valu RegisterCountable(name, value, subgroup); } -void TDynamicCounters::OutputHtml(IOutputStream& os) const { +void TDynamicCounters::OutputHtml(IOutputStream& os) const { HTML(os) { PRE() { OutputPlainText(os); @@ -191,7 +191,7 @@ void TDynamicCounters::OutputHtml(IOutputStream& os) const { } } -void TDynamicCounters::EnumerateSubgroups(const std::function<void(const TString& name, const TString& value)>& output) const { +void TDynamicCounters::EnumerateSubgroups(const std::function<void(const TString& name, const TString& value)>& output) const { TReadGuard g(Lock); for (const auto& [key, value] : Counters) { if (AsDynamicCounters(value)) { @@ -209,29 +209,29 @@ void TDynamicCounters::OutputPlainText(IOutputStream& os, const TString& indent) for (const auto& [key, value] : snap) { if (const auto counter = AsCounter(value)) { - os << indent + os << indent << key.LabelName << '=' << key.LabelValue - << ": " << counter->Val() + << ": " << counter->Val() << outputVisibilityMarker(counter->Visibility()) - << '\n'; + << '\n'; } else if (const auto histogram = AsHistogram(value)) { - os << indent + os << indent << key.LabelName << '=' << key.LabelValue << ":" << outputVisibilityMarker(histogram->Visibility()) << "\n"; - - auto snapshot = histogram->Snapshot(); - for (ui32 i = 0, count = snapshot->Count(); i < count; i++) { + + auto snapshot = histogram->Snapshot(); + for (ui32 i = 0, count = snapshot->Count(); i < count; i++) { os << indent << INDENT << TStringBuf("bin="); - TBucketBound bound = snapshot->UpperBound(i); - if (bound == Max<TBucketBound>()) { + TBucketBound bound = snapshot->UpperBound(i); + if (bound == Max<TBucketBound>()) { os << TStringBuf("inf"); - } else { - os << bound; - } - os << ": " << snapshot->Value(i) << '\n'; - } + } else { + os << bound; + } + os << ": " << snapshot->Value(i) << '\n'; + } } } @@ -239,22 +239,22 @@ void TDynamicCounters::OutputPlainText(IOutputStream& os, const TString& indent) if (const auto subgroup = AsDynamicCounters(value)) { os << "\n"; os << indent << key.LabelName << "=" << key.LabelValue << ":\n"; - subgroup->OutputPlainText(os, indent + INDENT); + subgroup->OutputPlainText(os, indent + INDENT); } } } -void TDynamicCounters::Accept(const TString& labelName, const TString& labelValue, ICountableConsumer& consumer) const { +void TDynamicCounters::Accept(const TString& labelName, const TString& labelValue, ICountableConsumer& consumer) const { if (!IsVisible(Visibility(), consumer.Visibility())) { return; } - consumer.OnGroupBegin(labelName, labelValue, this); + consumer.OnGroupBegin(labelName, labelValue, this); for (auto& [key, value] : ReadSnapshot()) { value->Accept(key.LabelName, key.LabelValue, consumer); - } - consumer.OnGroupEnd(labelName, labelValue, this); -} + } + consumer.OnGroupEnd(labelName, labelValue, this); +} void TDynamicCounters::RemoveExpired() const { if (AtomicGet(ExpiringCount) == 0) { diff --git a/library/cpp/monlib/dynamic_counters/counters.h b/library/cpp/monlib/dynamic_counters/counters.h index bc78538ef84..dc178cfbe01 100644 --- a/library/cpp/monlib/dynamic_counters/counters.h +++ b/library/cpp/monlib/dynamic_counters/counters.h @@ -1,24 +1,24 @@ #pragma once -#include <library/cpp/monlib/counters/counters.h> -#include <library/cpp/monlib/metrics/histogram_collector.h> - +#include <library/cpp/monlib/counters/counters.h> +#include <library/cpp/monlib/metrics/histogram_collector.h> + #include <library/cpp/threading/light_rw_lock/lightrwlock.h> #include <library/cpp/containers/stack_vector/stack_vec.h> #include <util/generic/cast.h> #include <util/generic/map.h> #include <util/generic/ptr.h> -#include <util/string/cast.h> +#include <util/string/cast.h> #include <util/system/rwlock.h> - + #include <functional> namespace NMonitoring { - struct TCounterForPtr; - struct TDynamicCounters; - struct ICountableConsumer; - + struct TCounterForPtr; + struct TDynamicCounters; + struct ICountableConsumer; + struct TCountableBase: public TAtomicRefCount<TCountableBase> { // Private means that the object must not be serialized unless the consumer @@ -31,12 +31,12 @@ namespace NMonitoring { Private, }; - virtual ~TCountableBase() { - } - - virtual void Accept( - const TString& labelName, const TString& labelValue, - ICountableConsumer& consumer) const = 0; + virtual ~TCountableBase() { + } + + virtual void Accept( + const TString& labelName, const TString& labelValue, + ICountableConsumer& consumer) const = 0; virtual EVisibility Visibility() const { return Visibility_; @@ -56,64 +56,64 @@ namespace NMonitoring { return true; } - struct ICountableConsumer { - virtual ~ICountableConsumer() { - } - - virtual void OnCounter( - const TString& labelName, const TString& labelValue, - const TCounterForPtr* counter) = 0; - - virtual void OnHistogram( - const TString& labelName, const TString& labelValue, + struct ICountableConsumer { + virtual ~ICountableConsumer() { + } + + virtual void OnCounter( + const TString& labelName, const TString& labelValue, + const TCounterForPtr* counter) = 0; + + virtual void OnHistogram( + const TString& labelName, const TString& labelValue, IHistogramSnapshotPtr snapshot, bool derivative) = 0; - - virtual void OnGroupBegin( - const TString& labelName, const TString& labelValue, - const TDynamicCounters* group) = 0; - - virtual void OnGroupEnd( - const TString& labelName, const TString& labelValue, - const TDynamicCounters* group) = 0; + + virtual void OnGroupBegin( + const TString& labelName, const TString& labelValue, + const TDynamicCounters* group) = 0; + + virtual void OnGroupEnd( + const TString& labelName, const TString& labelValue, + const TDynamicCounters* group) = 0; virtual TCountableBase::EVisibility Visibility() const { return TCountableBase::EVisibility::Unspecified; } - }; - + }; + #ifdef _MSC_VER #pragma warning(push) #pragma warning(disable : 4522) // multiple assignment operators specified #endif // _MSC_VER - struct TCounterForPtr: public TDeprecatedCounter, public TCountableBase { + struct TCounterForPtr: public TDeprecatedCounter, public TCountableBase { TCounterForPtr(bool derivative = false, EVisibility vis = EVisibility::Public) - : TDeprecatedCounter(0ULL, derivative) - { + : TDeprecatedCounter(0ULL, derivative) + { Visibility_ = vis; - } - + } + TCounterForPtr(const TCounterForPtr&) = delete; TCounterForPtr& operator=(const TCounterForPtr& other) = delete; - void Accept( - const TString& labelName, const TString& labelValue, - ICountableConsumer& consumer) const override { + void Accept( + const TString& labelName, const TString& labelValue, + ICountableConsumer& consumer) const override { if (IsVisible(Visibility(), consumer.Visibility())) { consumer.OnCounter(labelName, labelValue, this); } - } - + } + TCountableBase::EVisibility Visibility() const override { return Visibility_; } - using TDeprecatedCounter::operator++; - using TDeprecatedCounter::operator--; - using TDeprecatedCounter::operator+=; - using TDeprecatedCounter::operator-=; - using TDeprecatedCounter::operator=; - using TDeprecatedCounter::operator!; + using TDeprecatedCounter::operator++; + using TDeprecatedCounter::operator--; + using TDeprecatedCounter::operator+=; + using TDeprecatedCounter::operator-=; + using TDeprecatedCounter::operator=; + using TDeprecatedCounter::operator!; }; struct TExpiringCounter: public TCounterForPtr { @@ -124,27 +124,27 @@ namespace NMonitoring { } void Reset() { - TDeprecatedCounter::operator=(0); + TDeprecatedCounter::operator=(0); } }; - struct THistogramCounter: public TCountableBase { + struct THistogramCounter: public TCountableBase { explicit THistogramCounter( IHistogramCollectorPtr collector, bool derivative = true, EVisibility vis = EVisibility::Public) - : Collector_(std::move(collector)) + : Collector_(std::move(collector)) , Derivative_(derivative) - { + { Visibility_ = vis; - } - - void Collect(i64 value) { - Collector_->Collect(value); - } - - void Collect(i64 value, ui32 count) { - Collector_->Collect(value, count); - } - + } + + void Collect(i64 value) { + Collector_->Collect(value); + } + + void Collect(i64 value, ui32 count) { + Collector_->Collect(value, count); + } + void Collect(double value, ui32 count) { Collector_->Collect(value, count); } @@ -153,34 +153,34 @@ namespace NMonitoring { Collector_->Collect(snapshot); } - void Accept( - const TString& labelName, const TString& labelValue, - ICountableConsumer& consumer) const override - { + void Accept( + const TString& labelName, const TString& labelValue, + ICountableConsumer& consumer) const override + { if (IsVisible(Visibility(), consumer.Visibility())) { consumer.OnHistogram(labelName, labelValue, Collector_->Snapshot(), Derivative_); } - } - + } + void Reset() { Collector_->Reset(); } - IHistogramSnapshotPtr Snapshot() const { - return Collector_->Snapshot(); - } - - private: - IHistogramCollectorPtr Collector_; + IHistogramSnapshotPtr Snapshot() const { + return Collector_->Snapshot(); + } + + private: + IHistogramCollectorPtr Collector_; bool Derivative_; - }; - + }; + struct TExpiringHistogramCounter: public THistogramCounter { using THistogramCounter::THistogramCounter; }; - using THistogramPtr = TIntrusivePtr<THistogramCounter>; - + using THistogramPtr = TIntrusivePtr<THistogramCounter>; + #ifdef _MSC_VER #pragma warning(pop) #endif @@ -203,8 +203,8 @@ namespace NMonitoring { struct TChildId { TString LabelName; TString LabelValue; - TChildId() { - } + TChildId() { + } TChildId(const TString& labelName, const TString& labelValue) : LabelName(labelName) , LabelValue(labelValue) @@ -230,7 +230,7 @@ namespace NMonitoring { /// XXX: hack for deferred removal of expired counters. Remove once Output* functions are not used for serialization mutable TCounters Counters; mutable TAtomic ExpiringCount = 0; - + public: TDynamicCounters(TCountableBase::EVisibility visibility = TCountableBase::EVisibility::Public); @@ -306,7 +306,7 @@ namespace NMonitoring { const TString& value, bool derivative = false, TCountableBase::EVisibility visibility = TCountableBase::EVisibility::Public); - + THistogramPtr GetExpiringHistogram( const TString& value, IHistogramCollectorPtr collector, @@ -344,15 +344,15 @@ namespace NMonitoring { TIntrusivePtr<TDynamicCounters> subgroup); void OutputHtml(IOutputStream& os) const; - void EnumerateSubgroups(const std::function<void(const TString& name, const TString& value)>& output) const; + void EnumerateSubgroups(const std::function<void(const TString& name, const TString& value)>& output) const; // mostly for debugging purposes -- use accept with encoder instead void OutputPlainText(IOutputStream& os, const TString& indent = "") const; - void Accept( - const TString& labelName, const TString& labelValue, - ICountableConsumer& consumer) const override; - + void Accept( + const TString& labelName, const TString& labelValue, + ICountableConsumer& consumer) const override; + private: TCounters Resign() { TCounters counters; diff --git a/library/cpp/monlib/dynamic_counters/counters_ut.cpp b/library/cpp/monlib/dynamic_counters/counters_ut.cpp index 512728be1d9..3591037e0a7 100644 --- a/library/cpp/monlib/dynamic_counters/counters_ut.cpp +++ b/library/cpp/monlib/dynamic_counters/counters_ut.cpp @@ -1,101 +1,101 @@ -#include "counters.h" - +#include "counters.h" + #include <library/cpp/testing/unittest/registar.h> - -using namespace NMonitoring; - -class TCountersPrinter: public ICountableConsumer { -public: + +using namespace NMonitoring; + +class TCountersPrinter: public ICountableConsumer { +public: TCountersPrinter(IOutputStream* out) - : Out_(out) - , Level_(0) - { - } - -private: - void OnCounter( - const TString& labelName, const TString& labelValue, - const TCounterForPtr* counter) override { - Indent(Out_, Level_) - << labelName << ':' << labelValue - << " = " << counter->Val() << '\n'; - } - - void OnHistogram( - const TString& labelName, const TString& labelValue, + : Out_(out) + , Level_(0) + { + } + +private: + void OnCounter( + const TString& labelName, const TString& labelValue, + const TCounterForPtr* counter) override { + Indent(Out_, Level_) + << labelName << ':' << labelValue + << " = " << counter->Val() << '\n'; + } + + void OnHistogram( + const TString& labelName, const TString& labelValue, IHistogramSnapshotPtr snapshot, bool /*derivative*/) override { - Indent(Out_, Level_) - << labelName << ':' << labelValue - << " = " << *snapshot << '\n'; - } - - void OnGroupBegin( - const TString& labelName, const TString& labelValue, - const TDynamicCounters*) override { - Indent(Out_, Level_++) << labelName << ':' << labelValue << " {\n"; - } - - void OnGroupEnd( - const TString&, const TString&, - const TDynamicCounters*) override { - Indent(Out_, --Level_) << "}\n"; - } - + Indent(Out_, Level_) + << labelName << ':' << labelValue + << " = " << *snapshot << '\n'; + } + + void OnGroupBegin( + const TString& labelName, const TString& labelValue, + const TDynamicCounters*) override { + Indent(Out_, Level_++) << labelName << ':' << labelValue << " {\n"; + } + + void OnGroupEnd( + const TString&, const TString&, + const TDynamicCounters*) override { + Indent(Out_, --Level_) << "}\n"; + } + static IOutputStream& Indent(IOutputStream* out, int level) { - for (int i = 0; i < level; i++) { - out->Write(" "); - } - return *out; - } - -private: + for (int i = 0; i < level; i++) { + out->Write(" "); + } + return *out; + } + +private: IOutputStream* Out_; - int Level_ = 0; -}; - + int Level_ = 0; +}; + Y_UNIT_TEST_SUITE(TDynamicCountersTest) { Y_UNIT_TEST(CountersConsumer) { - TDynamicCounterPtr rootGroup(new TDynamicCounters()); - - auto usersCounter = rootGroup->GetNamedCounter("users", "count"); - *usersCounter = 7; - - auto hostGroup = rootGroup->GetSubgroup("counters", "resources"); - auto cpuCounter = hostGroup->GetNamedCounter("resource", "cpu"); - *cpuCounter = 30; - - auto memGroup = hostGroup->GetSubgroup("resource", "mem"); - auto usedCounter = memGroup->GetCounter("used"); - auto freeCounter = memGroup->GetCounter("free"); - *usedCounter = 100; - *freeCounter = 28; - - auto netGroup = hostGroup->GetSubgroup("resource", "net"); - auto rxCounter = netGroup->GetCounter("rx", true); - auto txCounter = netGroup->GetCounter("tx", true); - *rxCounter = 8; - *txCounter = 9; - - TStringStream ss; - TCountersPrinter printer(&ss); - rootGroup->Accept("root", "counters", printer); - - UNIT_ASSERT_STRINGS_EQUAL(ss.Str(), - "root:counters {\n" - " counters:resources {\n" - " resource:cpu = 30\n" - " resource:mem {\n" - " sensor:free = 28\n" - " sensor:used = 100\n" - " }\n" - " resource:net {\n" - " sensor:rx = 8\n" - " sensor:tx = 9\n" - " }\n" - " }\n" - " users:count = 7\n" - "}\n"); - } + TDynamicCounterPtr rootGroup(new TDynamicCounters()); + + auto usersCounter = rootGroup->GetNamedCounter("users", "count"); + *usersCounter = 7; + + auto hostGroup = rootGroup->GetSubgroup("counters", "resources"); + auto cpuCounter = hostGroup->GetNamedCounter("resource", "cpu"); + *cpuCounter = 30; + + auto memGroup = hostGroup->GetSubgroup("resource", "mem"); + auto usedCounter = memGroup->GetCounter("used"); + auto freeCounter = memGroup->GetCounter("free"); + *usedCounter = 100; + *freeCounter = 28; + + auto netGroup = hostGroup->GetSubgroup("resource", "net"); + auto rxCounter = netGroup->GetCounter("rx", true); + auto txCounter = netGroup->GetCounter("tx", true); + *rxCounter = 8; + *txCounter = 9; + + TStringStream ss; + TCountersPrinter printer(&ss); + rootGroup->Accept("root", "counters", printer); + + UNIT_ASSERT_STRINGS_EQUAL(ss.Str(), + "root:counters {\n" + " counters:resources {\n" + " resource:cpu = 30\n" + " resource:mem {\n" + " sensor:free = 28\n" + " sensor:used = 100\n" + " }\n" + " resource:net {\n" + " sensor:rx = 8\n" + " sensor:tx = 9\n" + " }\n" + " }\n" + " users:count = 7\n" + "}\n"); + } Y_UNIT_TEST(MergeSubgroup) { TDynamicCounterPtr rootGroup(new TDynamicCounters()); @@ -249,7 +249,7 @@ Y_UNIT_TEST_SUITE(TDynamicCountersTest) { UNIT_ASSERT_STRINGS_EQUAL(ss.Str(), "root:counters {\n" "}\n"); - } + } Y_UNIT_TEST(ExpiringCountersDiesAfterRegistry) { TDynamicCounters::TCounterPtr ptr; @@ -266,24 +266,24 @@ Y_UNIT_TEST_SUITE(TDynamicCountersTest) { " sensor:foo = 0\n" "}\n"); } - } - - Y_UNIT_TEST(HistogramCounter) { - TDynamicCounterPtr rootGroup(new TDynamicCounters()); - - auto h = rootGroup->GetHistogram("timeMillis", ExponentialHistogram(4, 2)); - for (i64 i = 1; i < 100; i++) { - h->Collect(i); - } - - TStringStream ss; - TCountersPrinter printer(&ss); - rootGroup->Accept("root", "counters", printer); - UNIT_ASSERT_STRINGS_EQUAL(ss.Str(), - "root:counters {\n" - " sensor:timeMillis = {1: 1, 2: 1, 4: 2, inf: 95}\n" - "}\n"); - } + } + + Y_UNIT_TEST(HistogramCounter) { + TDynamicCounterPtr rootGroup(new TDynamicCounters()); + + auto h = rootGroup->GetHistogram("timeMillis", ExponentialHistogram(4, 2)); + for (i64 i = 1; i < 100; i++) { + h->Collect(i); + } + + TStringStream ss; + TCountersPrinter printer(&ss); + rootGroup->Accept("root", "counters", printer); + UNIT_ASSERT_STRINGS_EQUAL(ss.Str(), + "root:counters {\n" + " sensor:timeMillis = {1: 1, 2: 1, 4: 2, inf: 95}\n" + "}\n"); + } Y_UNIT_TEST(CounterLookupCounter) { TDynamicCounterPtr rootGroup(new TDynamicCounters()); @@ -339,4 +339,4 @@ Y_UNIT_TEST_SUITE(TDynamicCountersTest) { histogram = rootGroup->FindNamedHistogram("name", "histogram2"); UNIT_ASSERT(histogram); } -} +} diff --git a/library/cpp/monlib/dynamic_counters/encode.cpp b/library/cpp/monlib/dynamic_counters/encode.cpp index 8a8b31314bc..ffa48d276e8 100644 --- a/library/cpp/monlib/dynamic_counters/encode.cpp +++ b/library/cpp/monlib/dynamic_counters/encode.cpp @@ -1,131 +1,131 @@ -#include "encode.h" - -#include <library/cpp/monlib/encode/encoder.h> -#include <library/cpp/monlib/encode/json/json.h> -#include <library/cpp/monlib/encode/spack/spack_v1.h> +#include "encode.h" + +#include <library/cpp/monlib/encode/encoder.h> +#include <library/cpp/monlib/encode/json/json.h> +#include <library/cpp/monlib/encode/spack/spack_v1.h> #include <library/cpp/monlib/encode/prometheus/prometheus.h> - -#include <util/stream/str.h> - -namespace NMonitoring { - namespace { - constexpr TInstant ZERO_TIME = TInstant::Zero(); - - class TConsumer final: public ICountableConsumer { - using TLabel = std::pair<TString, TString>; // name, value - - public: - explicit TConsumer(NMonitoring::IMetricEncoderPtr encoderImpl, TCountableBase::EVisibility vis) - : EncoderImpl_(std::move(encoderImpl)) + +#include <util/stream/str.h> + +namespace NMonitoring { + namespace { + constexpr TInstant ZERO_TIME = TInstant::Zero(); + + class TConsumer final: public ICountableConsumer { + using TLabel = std::pair<TString, TString>; // name, value + + public: + explicit TConsumer(NMonitoring::IMetricEncoderPtr encoderImpl, TCountableBase::EVisibility vis) + : EncoderImpl_(std::move(encoderImpl)) , Visibility_{vis} - { - } - - void OnCounter( - const TString& labelName, const TString& labelValue, - const TCounterForPtr* counter) override { - NMonitoring::EMetricType metricType = counter->ForDerivative() - ? NMonitoring::EMetricType::RATE - : NMonitoring::EMetricType::GAUGE; - EncoderImpl_->OnMetricBegin(metricType); - EncodeLabels(labelName, labelValue); - - if (metricType == NMonitoring::EMetricType::GAUGE) { - EncoderImpl_->OnDouble(ZERO_TIME, static_cast<double>(counter->Val())); - } else { - EncoderImpl_->OnUint64(ZERO_TIME, counter->Val()); - } - - EncoderImpl_->OnMetricEnd(); - } - - void OnHistogram( - const TString& labelName, const TString& labelValue, + { + } + + void OnCounter( + const TString& labelName, const TString& labelValue, + const TCounterForPtr* counter) override { + NMonitoring::EMetricType metricType = counter->ForDerivative() + ? NMonitoring::EMetricType::RATE + : NMonitoring::EMetricType::GAUGE; + EncoderImpl_->OnMetricBegin(metricType); + EncodeLabels(labelName, labelValue); + + if (metricType == NMonitoring::EMetricType::GAUGE) { + EncoderImpl_->OnDouble(ZERO_TIME, static_cast<double>(counter->Val())); + } else { + EncoderImpl_->OnUint64(ZERO_TIME, counter->Val()); + } + + EncoderImpl_->OnMetricEnd(); + } + + void OnHistogram( + const TString& labelName, const TString& labelValue, IHistogramSnapshotPtr snapshot, bool derivative) override { NMonitoring::EMetricType metricType = derivative ? EMetricType::HIST_RATE : EMetricType::HIST; EncoderImpl_->OnMetricBegin(metricType); - EncodeLabels(labelName, labelValue); - EncoderImpl_->OnHistogram(ZERO_TIME, snapshot); - EncoderImpl_->OnMetricEnd(); - } - - void OnGroupBegin( - const TString& labelName, const TString& labelValue, - const TDynamicCounters*) override { + EncodeLabels(labelName, labelValue); + EncoderImpl_->OnHistogram(ZERO_TIME, snapshot); + EncoderImpl_->OnMetricEnd(); + } + + void OnGroupBegin( + const TString& labelName, const TString& labelValue, + const TDynamicCounters*) override { if (labelName.empty() && labelValue.empty()) { - // root group has empty label name and value - EncoderImpl_->OnStreamBegin(); - } else { - ParentLabels_.emplace_back(labelName, labelValue); - } - } - - void OnGroupEnd( - const TString& labelName, const TString& labelValue, - const TDynamicCounters*) override { + // root group has empty label name and value + EncoderImpl_->OnStreamBegin(); + } else { + ParentLabels_.emplace_back(labelName, labelValue); + } + } + + void OnGroupEnd( + const TString& labelName, const TString& labelValue, + const TDynamicCounters*) override { if (labelName.empty() && labelValue.empty()) { - // root group has empty label name and value - EncoderImpl_->OnStreamEnd(); - EncoderImpl_->Close(); - } else { - ParentLabels_.pop_back(); - } - } - + // root group has empty label name and value + EncoderImpl_->OnStreamEnd(); + EncoderImpl_->Close(); + } else { + ParentLabels_.pop_back(); + } + } + TCountableBase::EVisibility Visibility() const override { return Visibility_; } - private: - void EncodeLabels(const TString& labelName, const TString& labelValue) { - EncoderImpl_->OnLabelsBegin(); - for (const auto& label : ParentLabels_) { - EncoderImpl_->OnLabel(label.first, label.second); - } - EncoderImpl_->OnLabel(labelName, labelValue); - EncoderImpl_->OnLabelsEnd(); - } - - private: - NMonitoring::IMetricEncoderPtr EncoderImpl_; - TVector<TLabel> ParentLabels_; + private: + void EncodeLabels(const TString& labelName, const TString& labelValue) { + EncoderImpl_->OnLabelsBegin(); + for (const auto& label : ParentLabels_) { + EncoderImpl_->OnLabel(label.first, label.second); + } + EncoderImpl_->OnLabel(labelName, labelValue); + EncoderImpl_->OnLabelsEnd(); + } + + private: + NMonitoring::IMetricEncoderPtr EncoderImpl_; + TVector<TLabel> ParentLabels_; TCountableBase::EVisibility Visibility_; - }; - - } - + }; + + } + THolder<ICountableConsumer> CreateEncoder(IOutputStream* out, EFormat format, TCountableBase::EVisibility vis) { - switch (format) { - case EFormat::JSON: + switch (format) { + case EFormat::JSON: return MakeHolder<TConsumer>(NMonitoring::EncoderJson(out), vis); - case EFormat::SPACK: + case EFormat::SPACK: return MakeHolder<TConsumer>(NMonitoring::EncoderSpackV1( - out, - NMonitoring::ETimePrecision::SECONDS, + out, + NMonitoring::ETimePrecision::SECONDS, NMonitoring::ECompression::ZSTD), vis); case EFormat::PROMETHEUS: return MakeHolder<TConsumer>(NMonitoring::EncoderPrometheus( out), vis); - default: - ythrow yexception() << "unsupported metric encoding format: " << format; - break; - } - } - + default: + ythrow yexception() << "unsupported metric encoding format: " << format; + break; + } + } + THolder<ICountableConsumer> AsCountableConsumer(IMetricEncoderPtr encoder, TCountableBase::EVisibility visibility) { return MakeHolder<TConsumer>(std::move(encoder), visibility); } - void ToJson(const TDynamicCounters& counters, IOutputStream* out) { - TConsumer consumer{EncoderJson(out), TCountableBase::EVisibility::Public}; - counters.Accept(TString{}, TString{}, consumer); - } - - TString ToJson(const TDynamicCounters& counters) { - TStringStream ss; - ToJson(counters, &ss); - return ss.Str(); - } - -} + void ToJson(const TDynamicCounters& counters, IOutputStream* out) { + TConsumer consumer{EncoderJson(out), TCountableBase::EVisibility::Public}; + counters.Accept(TString{}, TString{}, consumer); + } + + TString ToJson(const TDynamicCounters& counters) { + TStringStream ss; + ToJson(counters, &ss); + return ss.Str(); + } + +} diff --git a/library/cpp/monlib/dynamic_counters/encode.h b/library/cpp/monlib/dynamic_counters/encode.h index 81e924a0344..c79964d7cb7 100644 --- a/library/cpp/monlib/dynamic_counters/encode.h +++ b/library/cpp/monlib/dynamic_counters/encode.h @@ -1,23 +1,23 @@ -#pragma once - -#include "counters.h" - +#pragma once + +#include "counters.h" + #include <library/cpp/monlib/encode/encoder.h> -#include <library/cpp/monlib/encode/format.h> - -namespace NMonitoring { - +#include <library/cpp/monlib/encode/format.h> + +namespace NMonitoring { + THolder<ICountableConsumer> CreateEncoder( IOutputStream* out, EFormat format, TCountableBase::EVisibility visibility = TCountableBase::EVisibility::Public ); - + THolder<ICountableConsumer> AsCountableConsumer( NMonitoring::IMetricEncoderPtr encoder, TCountableBase::EVisibility visibility = TCountableBase::EVisibility::Public); - void ToJson(const TDynamicCounters& counters, IOutputStream* out); - - TString ToJson(const TDynamicCounters& counters); -} + void ToJson(const TDynamicCounters& counters, IOutputStream* out); + + TString ToJson(const TDynamicCounters& counters); +} diff --git a/library/cpp/monlib/dynamic_counters/encode_ut.cpp b/library/cpp/monlib/dynamic_counters/encode_ut.cpp index 9e5bd74f919..52d77b6b413 100644 --- a/library/cpp/monlib/dynamic_counters/encode_ut.cpp +++ b/library/cpp/monlib/dynamic_counters/encode_ut.cpp @@ -1,172 +1,172 @@ -#include "encode.h" - -#include <library/cpp/monlib/encode/json/json.h> -#include <library/cpp/monlib/encode/spack/spack_v1.h> -#include <library/cpp/monlib/encode/protobuf/protobuf.h> - -#include <library/cpp/monlib/encode/protobuf/protos/samples.pb.h> +#include "encode.h" + +#include <library/cpp/monlib/encode/json/json.h> +#include <library/cpp/monlib/encode/spack/spack_v1.h> +#include <library/cpp/monlib/encode/protobuf/protobuf.h> + +#include <library/cpp/monlib/encode/protobuf/protos/samples.pb.h> #include <library/cpp/testing/unittest/registar.h> - -#include <util/generic/buffer.h> -#include <util/stream/buffer.h> - -namespace NMonitoring { - struct TTestData: public TDynamicCounters { - TTestData() { - auto hostGroup = GetSubgroup("counters", "resources"); - { - auto cpuCounter = hostGroup->GetNamedCounter("resource", "cpu"); - *cpuCounter = 30; - - auto memGroup = hostGroup->GetSubgroup("resource", "mem"); - auto usedCounter = memGroup->GetCounter("used"); - auto freeCounter = memGroup->GetCounter("free"); - *usedCounter = 100; - *freeCounter = 28; - - auto netGroup = hostGroup->GetSubgroup("resource", "net"); - auto rxCounter = netGroup->GetCounter("rx", true); - auto txCounter = netGroup->GetCounter("tx", true); - *rxCounter = 8; - *txCounter = 9; - } - - auto usersCounter = GetNamedCounter("users", "count"); - *usersCounter = 7; - - auto responseTimeMillis = GetHistogram("responseTimeMillis", ExplicitHistogram({1, 5, 10, 15, 20, 100, 200})); - for (i64 i = 0; i < 400; i++) { - responseTimeMillis->Collect(i); - } - } - }; - - void AssertLabelsEqual(const NProto::TLabel& l, TStringBuf name, TStringBuf value) { - UNIT_ASSERT_STRINGS_EQUAL(l.GetName(), name); - UNIT_ASSERT_STRINGS_EQUAL(l.GetValue(), value); - } - - void AssertResult(const NProto::TSingleSamplesList& samples) { - UNIT_ASSERT_VALUES_EQUAL(samples.SamplesSize(), 7); - - { - auto s = samples.GetSamples(0); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 2); - AssertLabelsEqual(s.GetLabels(0), "counters", "resources"); - AssertLabelsEqual(s.GetLabels(1), "resource", "cpu"); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); - UNIT_ASSERT_DOUBLES_EQUAL(s.GetFloat64(), 30.0, Min<double>()); - } - { - auto s = samples.GetSamples(1); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 3); - AssertLabelsEqual(s.GetLabels(0), "counters", "resources"); - AssertLabelsEqual(s.GetLabels(1), "resource", "mem"); - AssertLabelsEqual(s.GetLabels(2), "sensor", "free"); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); - UNIT_ASSERT_DOUBLES_EQUAL(s.GetFloat64(), 28.0, Min<double>()); - } - { - auto s = samples.GetSamples(2); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 3); - AssertLabelsEqual(s.GetLabels(0), "counters", "resources"); - AssertLabelsEqual(s.GetLabels(1), "resource", "mem"); - AssertLabelsEqual(s.GetLabels(2), "sensor", "used"); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); - UNIT_ASSERT_DOUBLES_EQUAL(s.GetFloat64(), 100.0, Min<double>()); - } - { - auto s = samples.GetSamples(3); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 3); - AssertLabelsEqual(s.GetLabels(0), "counters", "resources"); - AssertLabelsEqual(s.GetLabels(1), "resource", "net"); - AssertLabelsEqual(s.GetLabels(2), "sensor", "rx"); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::RATE); - UNIT_ASSERT_VALUES_EQUAL(s.GetUint64(), 8); - } - { - auto s = samples.GetSamples(4); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 3); - AssertLabelsEqual(s.GetLabels(0), "counters", "resources"); - AssertLabelsEqual(s.GetLabels(1), "resource", "net"); - AssertLabelsEqual(s.GetLabels(2), "sensor", "tx"); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::RATE); - UNIT_ASSERT_VALUES_EQUAL(s.GetUint64(), 9); - } - { - auto s = samples.GetSamples(5); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); - AssertLabelsEqual(s.GetLabels(0), "sensor", "responseTimeMillis"); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::HIST_RATE); - - const NProto::THistogram& h = s.GetHistogram(); - - UNIT_ASSERT_EQUAL(h.BoundsSize(), 8); - UNIT_ASSERT_DOUBLES_EQUAL(h.GetBounds(0), 1, Min<double>()); - UNIT_ASSERT_DOUBLES_EQUAL(h.GetBounds(1), 5, Min<double>()); - UNIT_ASSERT_DOUBLES_EQUAL(h.GetBounds(2), 10, Min<double>()); - UNIT_ASSERT_DOUBLES_EQUAL(h.GetBounds(3), 15, Min<double>()); - UNIT_ASSERT_DOUBLES_EQUAL(h.GetBounds(4), 20, Min<double>()); - UNIT_ASSERT_DOUBLES_EQUAL(h.GetBounds(5), 100, Min<double>()); - UNIT_ASSERT_DOUBLES_EQUAL(h.GetBounds(6), 200, Min<double>()); - UNIT_ASSERT_DOUBLES_EQUAL(h.GetBounds(7), Max<double>(), Min<double>()); - - UNIT_ASSERT_EQUAL(h.ValuesSize(), 8); - UNIT_ASSERT_EQUAL(h.GetValues(0), 2); - UNIT_ASSERT_EQUAL(h.GetValues(1), 4); - UNIT_ASSERT_EQUAL(h.GetValues(2), 5); - UNIT_ASSERT_EQUAL(h.GetValues(3), 5); - UNIT_ASSERT_EQUAL(h.GetValues(4), 5); - UNIT_ASSERT_EQUAL(h.GetValues(5), 80); - UNIT_ASSERT_EQUAL(h.GetValues(6), 100); - UNIT_ASSERT_EQUAL(h.GetValues(7), 199); - } - { - auto s = samples.GetSamples(6); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); - AssertLabelsEqual(s.GetLabels(0), "users", "count"); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); - UNIT_ASSERT_DOUBLES_EQUAL(s.GetFloat64(), 7, Min<double>()); - } - } - + +#include <util/generic/buffer.h> +#include <util/stream/buffer.h> + +namespace NMonitoring { + struct TTestData: public TDynamicCounters { + TTestData() { + auto hostGroup = GetSubgroup("counters", "resources"); + { + auto cpuCounter = hostGroup->GetNamedCounter("resource", "cpu"); + *cpuCounter = 30; + + auto memGroup = hostGroup->GetSubgroup("resource", "mem"); + auto usedCounter = memGroup->GetCounter("used"); + auto freeCounter = memGroup->GetCounter("free"); + *usedCounter = 100; + *freeCounter = 28; + + auto netGroup = hostGroup->GetSubgroup("resource", "net"); + auto rxCounter = netGroup->GetCounter("rx", true); + auto txCounter = netGroup->GetCounter("tx", true); + *rxCounter = 8; + *txCounter = 9; + } + + auto usersCounter = GetNamedCounter("users", "count"); + *usersCounter = 7; + + auto responseTimeMillis = GetHistogram("responseTimeMillis", ExplicitHistogram({1, 5, 10, 15, 20, 100, 200})); + for (i64 i = 0; i < 400; i++) { + responseTimeMillis->Collect(i); + } + } + }; + + void AssertLabelsEqual(const NProto::TLabel& l, TStringBuf name, TStringBuf value) { + UNIT_ASSERT_STRINGS_EQUAL(l.GetName(), name); + UNIT_ASSERT_STRINGS_EQUAL(l.GetValue(), value); + } + + void AssertResult(const NProto::TSingleSamplesList& samples) { + UNIT_ASSERT_VALUES_EQUAL(samples.SamplesSize(), 7); + + { + auto s = samples.GetSamples(0); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 2); + AssertLabelsEqual(s.GetLabels(0), "counters", "resources"); + AssertLabelsEqual(s.GetLabels(1), "resource", "cpu"); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); + UNIT_ASSERT_DOUBLES_EQUAL(s.GetFloat64(), 30.0, Min<double>()); + } + { + auto s = samples.GetSamples(1); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 3); + AssertLabelsEqual(s.GetLabels(0), "counters", "resources"); + AssertLabelsEqual(s.GetLabels(1), "resource", "mem"); + AssertLabelsEqual(s.GetLabels(2), "sensor", "free"); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); + UNIT_ASSERT_DOUBLES_EQUAL(s.GetFloat64(), 28.0, Min<double>()); + } + { + auto s = samples.GetSamples(2); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 3); + AssertLabelsEqual(s.GetLabels(0), "counters", "resources"); + AssertLabelsEqual(s.GetLabels(1), "resource", "mem"); + AssertLabelsEqual(s.GetLabels(2), "sensor", "used"); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); + UNIT_ASSERT_DOUBLES_EQUAL(s.GetFloat64(), 100.0, Min<double>()); + } + { + auto s = samples.GetSamples(3); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 3); + AssertLabelsEqual(s.GetLabels(0), "counters", "resources"); + AssertLabelsEqual(s.GetLabels(1), "resource", "net"); + AssertLabelsEqual(s.GetLabels(2), "sensor", "rx"); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::RATE); + UNIT_ASSERT_VALUES_EQUAL(s.GetUint64(), 8); + } + { + auto s = samples.GetSamples(4); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 3); + AssertLabelsEqual(s.GetLabels(0), "counters", "resources"); + AssertLabelsEqual(s.GetLabels(1), "resource", "net"); + AssertLabelsEqual(s.GetLabels(2), "sensor", "tx"); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::RATE); + UNIT_ASSERT_VALUES_EQUAL(s.GetUint64(), 9); + } + { + auto s = samples.GetSamples(5); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); + AssertLabelsEqual(s.GetLabels(0), "sensor", "responseTimeMillis"); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::HIST_RATE); + + const NProto::THistogram& h = s.GetHistogram(); + + UNIT_ASSERT_EQUAL(h.BoundsSize(), 8); + UNIT_ASSERT_DOUBLES_EQUAL(h.GetBounds(0), 1, Min<double>()); + UNIT_ASSERT_DOUBLES_EQUAL(h.GetBounds(1), 5, Min<double>()); + UNIT_ASSERT_DOUBLES_EQUAL(h.GetBounds(2), 10, Min<double>()); + UNIT_ASSERT_DOUBLES_EQUAL(h.GetBounds(3), 15, Min<double>()); + UNIT_ASSERT_DOUBLES_EQUAL(h.GetBounds(4), 20, Min<double>()); + UNIT_ASSERT_DOUBLES_EQUAL(h.GetBounds(5), 100, Min<double>()); + UNIT_ASSERT_DOUBLES_EQUAL(h.GetBounds(6), 200, Min<double>()); + UNIT_ASSERT_DOUBLES_EQUAL(h.GetBounds(7), Max<double>(), Min<double>()); + + UNIT_ASSERT_EQUAL(h.ValuesSize(), 8); + UNIT_ASSERT_EQUAL(h.GetValues(0), 2); + UNIT_ASSERT_EQUAL(h.GetValues(1), 4); + UNIT_ASSERT_EQUAL(h.GetValues(2), 5); + UNIT_ASSERT_EQUAL(h.GetValues(3), 5); + UNIT_ASSERT_EQUAL(h.GetValues(4), 5); + UNIT_ASSERT_EQUAL(h.GetValues(5), 80); + UNIT_ASSERT_EQUAL(h.GetValues(6), 100); + UNIT_ASSERT_EQUAL(h.GetValues(7), 199); + } + { + auto s = samples.GetSamples(6); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); + AssertLabelsEqual(s.GetLabels(0), "users", "count"); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); + UNIT_ASSERT_DOUBLES_EQUAL(s.GetFloat64(), 7, Min<double>()); + } + } + Y_UNIT_TEST_SUITE(TDynamicCountersEncodeTest) { - TTestData Data; - + TTestData Data; + Y_UNIT_TEST(Json) { - TString result; - { - TStringOutput out(result); - auto encoder = CreateEncoder(&out, EFormat::JSON); - Data.Accept(TString(), TString(), *encoder); - } - - NProto::TSingleSamplesList samples; - { - auto e = EncoderProtobuf(&samples); - DecodeJson(result, e.Get()); - } - - AssertResult(samples); - } - + TString result; + { + TStringOutput out(result); + auto encoder = CreateEncoder(&out, EFormat::JSON); + Data.Accept(TString(), TString(), *encoder); + } + + NProto::TSingleSamplesList samples; + { + auto e = EncoderProtobuf(&samples); + DecodeJson(result, e.Get()); + } + + AssertResult(samples); + } + Y_UNIT_TEST(Spack) { - TBuffer result; - { - TBufferOutput out(result); - auto encoder = CreateEncoder(&out, EFormat::SPACK); - Data.Accept(TString(), TString(), *encoder); - } - - NProto::TSingleSamplesList samples; - { - auto e = EncoderProtobuf(&samples); - TBufferInput in(result); - DecodeSpackV1(&in, e.Get()); - } - - AssertResult(samples); - } + TBuffer result; + { + TBufferOutput out(result); + auto encoder = CreateEncoder(&out, EFormat::SPACK); + Data.Accept(TString(), TString(), *encoder); + } + + NProto::TSingleSamplesList samples; + { + auto e = EncoderProtobuf(&samples); + TBufferInput in(result); + DecodeSpackV1(&in, e.Get()); + } + + AssertResult(samples); + } Y_UNIT_TEST(PrivateSubgroupIsNotSerialized) { TBuffer result; @@ -209,18 +209,18 @@ namespace NMonitoring { AssertResult(samples); } - - Y_UNIT_TEST(ToJson) { - TString result = ToJson(Data); - - NProto::TSingleSamplesList samples; - { - auto e = EncoderProtobuf(&samples); - DecodeJson(result, e.Get()); - } - - AssertResult(samples); - } - } - -} + + Y_UNIT_TEST(ToJson) { + TString result = ToJson(Data); + + NProto::TSingleSamplesList samples; + { + auto e = EncoderProtobuf(&samples); + DecodeJson(result, e.Get()); + } + + AssertResult(samples); + } + } + +} diff --git a/library/cpp/monlib/dynamic_counters/golovan_page.cpp b/library/cpp/monlib/dynamic_counters/golovan_page.cpp index 03a5b1a399f..49cf2d39bb6 100644 --- a/library/cpp/monlib/dynamic_counters/golovan_page.cpp +++ b/library/cpp/monlib/dynamic_counters/golovan_page.cpp @@ -1,6 +1,6 @@ -#include "golovan_page.h" +#include "golovan_page.h" -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/string/split.h> #include <util/system/tls.h> @@ -40,8 +40,8 @@ public: } void OnHistogram(const TString&, const TString&, IHistogramSnapshotPtr, bool) override { - } - + } + void OnGroupBegin(const TString&, const TString& value, const TDynamicCounters*) override { prefix += value; if (!value.empty()) { diff --git a/library/cpp/monlib/dynamic_counters/golovan_page.h b/library/cpp/monlib/dynamic_counters/golovan_page.h index 855b4e9eabc..e1772c77345 100644 --- a/library/cpp/monlib/dynamic_counters/golovan_page.h +++ b/library/cpp/monlib/dynamic_counters/golovan_page.h @@ -1,8 +1,8 @@ #pragma once -#include "counters.h" +#include "counters.h" -#include <library/cpp/monlib/service/pages/mon_page.h> +#include <library/cpp/monlib/service/pages/mon_page.h> #include <util/generic/ptr.h> @@ -19,7 +19,7 @@ public: TOutputCallback outputCallback = nullptr); void Output(NMonitoring::IMonHttpRequest& request) override; - + private: TOutputCallback OutputCallback; }; diff --git a/library/cpp/monlib/dynamic_counters/page.cpp b/library/cpp/monlib/dynamic_counters/page.cpp index e554d330ad0..5124a47bb33 100644 --- a/library/cpp/monlib/dynamic_counters/page.cpp +++ b/library/cpp/monlib/dynamic_counters/page.cpp @@ -1,31 +1,31 @@ -#include "page.h" -#include "encode.h" +#include "page.h" +#include "encode.h" + +#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/string_utils/quote/quote.h> -#include <library/cpp/monlib/service/pages/templates.h> -#include <library/cpp/string_utils/quote/quote.h> - #include <util/string/split.h> #include <util/system/tls.h> using namespace NMonitoring; -namespace { - Y_POD_STATIC_THREAD(TDynamicCounters*) - currentCounters(nullptr); +namespace { + Y_POD_STATIC_THREAD(TDynamicCounters*) + currentCounters(nullptr); } TMaybe<EFormat> ParseFormat(TStringBuf str) { if (str == TStringBuf("json")) { - return EFormat::JSON; + return EFormat::JSON; } else if (str == TStringBuf("spack")) { - return EFormat::SPACK; + return EFormat::SPACK; } else if (str == TStringBuf("prometheus")) { return EFormat::PROMETHEUS; - } else { - return Nothing(); - } -} - + } else { + return Nothing(); + } +} + void TDynamicCountersPage::Output(NMonitoring::IMonHttpRequest& request) { if (OutputCallback) { OutputCallback(); @@ -41,10 +41,10 @@ void TDynamicCountersPage::Output(NMonitoring::IMonHttpRequest& request) { .SkipEmpty() .Collect(&parts); - TMaybe<EFormat> format = !parts.empty() ? ParseFormat(parts.back()) : Nothing(); - if (format) { + TMaybe<EFormat> format = !parts.empty() ? ParseFormat(parts.back()) : Nothing(); + if (format) { parts.pop_back(); - } + } if (!parts.empty() && parts.back() == TStringBuf("private")) { visibility = TCountableBase::EVisibility::Private; @@ -72,27 +72,27 @@ void TDynamicCountersPage::Output(NMonitoring::IMonHttpRequest& request) { } } - if (!format) { - currentCounters = counters.Get(); + if (!format) { + currentCounters = counters.Get(); THtmlMonPage::Output(request); - currentCounters = nullptr; - return; + currentCounters = nullptr; + return; } - - IOutputStream& out = request.Output(); - if (*format == EFormat::JSON) { - out << HTTPOKJSON; - } else if (*format == EFormat::SPACK) { - out << HTTPOKSPACK; + + IOutputStream& out = request.Output(); + if (*format == EFormat::JSON) { + out << HTTPOKJSON; + } else if (*format == EFormat::SPACK) { + out << HTTPOKSPACK; } else if (*format == EFormat::PROMETHEUS) { out << HTTPOKPROMETHEUS; - } else { - ythrow yexception() << "unsupported metric encoding format: " << *format; - } - - auto encoder = CreateEncoder(&out, *format, visibility); - counters->Accept(TString(), TString(), *encoder); - out.Flush(); + } else { + ythrow yexception() << "unsupported metric encoding format: " << *format; + } + + auto encoder = CreateEncoder(&out, *format, visibility); + counters->Accept(TString(), TString(), *encoder); + out.Flush(); } void TDynamicCountersPage::HandleAbsentSubgroup(IMonHttpRequest& request) { @@ -113,10 +113,10 @@ void TDynamicCountersPage::BeforePre(IMonHttpRequest& request) { out << " for <a href='https://wiki.yandex-team.ru/solomon/'>Solomon</a>"; } - H5() { - out << "Counters subgroups"; + H5() { + out << "Counters subgroups"; } - UL() { + UL() { currentCounters->EnumerateSubgroups([&](const TString& name, const TString& value) { LI() { TString pathPart = name + "=" + value; @@ -124,11 +124,11 @@ void TDynamicCountersPage::BeforePre(IMonHttpRequest& request) { out << "\n<a href='" << request.GetPath() << "/" << pathPart << "'>" << name << " " << value << "</a>"; } }); - } + } - H4() { - out << "Counters as text"; - } + H4() { + out << "Counters as text"; + } } } diff --git a/library/cpp/monlib/dynamic_counters/page.h b/library/cpp/monlib/dynamic_counters/page.h index 159ddaf578b..1f0ef6a5eaf 100644 --- a/library/cpp/monlib/dynamic_counters/page.h +++ b/library/cpp/monlib/dynamic_counters/page.h @@ -1,9 +1,9 @@ #pragma once -#include "counters.h" - -#include <library/cpp/monlib/service/pages/pre_mon_page.h> - +#include "counters.h" + +#include <library/cpp/monlib/service/pages/pre_mon_page.h> + #include <util/generic/ptr.h> #include <functional> @@ -28,7 +28,7 @@ namespace NMonitoring { public: TDynamicCountersPage(const TString& path, - const TString& title, + const TString& title, TIntrusivePtr<TDynamicCounters> counters, TOutputCallback outputCallback = nullptr) : TPreMonPage(path, title) diff --git a/library/cpp/monlib/dynamic_counters/percentile/percentile_base.h b/library/cpp/monlib/dynamic_counters/percentile/percentile_base.h index ee242b1cdbb..d3c825c43d8 100644 --- a/library/cpp/monlib/dynamic_counters/percentile/percentile_base.h +++ b/library/cpp/monlib/dynamic_counters/percentile/percentile_base.h @@ -1,6 +1,6 @@ #pragma once -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <util/string/printf.h> diff --git a/library/cpp/monlib/dynamic_counters/percentile/ut/ya.make b/library/cpp/monlib/dynamic_counters/percentile/ut/ya.make index 9859d87de21..f9f35641017 100644 --- a/library/cpp/monlib/dynamic_counters/percentile/ut/ya.make +++ b/library/cpp/monlib/dynamic_counters/percentile/ut/ya.make @@ -1,4 +1,4 @@ -UNITTEST_FOR(library/cpp/monlib/dynamic_counters/percentile) +UNITTEST_FOR(library/cpp/monlib/dynamic_counters/percentile) OWNER(alexvru g:kikimr g:solomon) diff --git a/library/cpp/monlib/dynamic_counters/percentile/ya.make b/library/cpp/monlib/dynamic_counters/percentile/ya.make index 3480330470b..cb52cdd9adb 100644 --- a/library/cpp/monlib/dynamic_counters/percentile/ya.make +++ b/library/cpp/monlib/dynamic_counters/percentile/ya.make @@ -9,7 +9,7 @@ LIBRARY() PEERDIR( library/cpp/containers/stack_vector - library/cpp/monlib/dynamic_counters + library/cpp/monlib/dynamic_counters ) END() diff --git a/library/cpp/monlib/dynamic_counters/ut/ya.make b/library/cpp/monlib/dynamic_counters/ut/ya.make index b0a7c7aa41a..8242f2fe307 100644 --- a/library/cpp/monlib/dynamic_counters/ut/ya.make +++ b/library/cpp/monlib/dynamic_counters/ut/ya.make @@ -1,16 +1,16 @@ -UNITTEST_FOR(library/cpp/monlib/dynamic_counters) - -OWNER(jamel) - -SRCS( +UNITTEST_FOR(library/cpp/monlib/dynamic_counters) + +OWNER(jamel) + +SRCS( contention_ut.cpp - counters_ut.cpp - encode_ut.cpp -) - -PEERDIR( - library/cpp/monlib/encode/protobuf - library/cpp/monlib/encode/json -) - -END() + counters_ut.cpp + encode_ut.cpp +) + +PEERDIR( + library/cpp/monlib/encode/protobuf + library/cpp/monlib/encode/json +) + +END() diff --git a/library/cpp/monlib/dynamic_counters/ya.make b/library/cpp/monlib/dynamic_counters/ya.make index cd65c4a4c9f..aafe1c34bed 100644 --- a/library/cpp/monlib/dynamic_counters/ya.make +++ b/library/cpp/monlib/dynamic_counters/ya.make @@ -1,27 +1,27 @@ -LIBRARY() - +LIBRARY() + OWNER( g:solomon jamel ) - -NO_WSHADOW() - -SRCS( - counters.cpp - encode.cpp - golovan_page.cpp - page.cpp -) - -PEERDIR( + +NO_WSHADOW() + +SRCS( + counters.cpp + encode.cpp + golovan_page.cpp + page.cpp +) + +PEERDIR( library/cpp/containers/stack_vector - library/cpp/monlib/encode/json - library/cpp/monlib/encode/spack + library/cpp/monlib/encode/json + library/cpp/monlib/encode/spack library/cpp/monlib/encode/prometheus - library/cpp/monlib/service/pages + library/cpp/monlib/service/pages library/cpp/string_utils/quote library/cpp/threading/light_rw_lock -) - -END() +) + +END() diff --git a/library/cpp/monlib/encode/buffered/buffered_encoder_base.cpp b/library/cpp/monlib/encode/buffered/buffered_encoder_base.cpp index b3d86b57ead..87c832d642b 100644 --- a/library/cpp/monlib/encode/buffered/buffered_encoder_base.cpp +++ b/library/cpp/monlib/encode/buffered/buffered_encoder_base.cpp @@ -18,57 +18,57 @@ void TBufferedEncoderBase::OnCommonTime(TInstant time) { CommonTime_ = time; } -void TBufferedEncoderBase::OnMetricBegin(EMetricType type) { - State_.Switch(TEncoderState::EState::ROOT, TEncoderState::EState::METRIC); - Metrics_.emplace_back(); - Metrics_.back().MetricType = type; +void TBufferedEncoderBase::OnMetricBegin(EMetricType type) { + State_.Switch(TEncoderState::EState::ROOT, TEncoderState::EState::METRIC); + Metrics_.emplace_back(); + Metrics_.back().MetricType = type; } -void TBufferedEncoderBase::OnMetricEnd() { - State_.Switch(TEncoderState::EState::METRIC, TEncoderState::EState::ROOT); +void TBufferedEncoderBase::OnMetricEnd() { + State_.Switch(TEncoderState::EState::METRIC, TEncoderState::EState::ROOT); - switch (MetricsMergingMode_) { - case EMetricsMergingMode::MERGE_METRICS: { - auto& metric = Metrics_.back(); - Sort(metric.Labels, [] (const TPooledLabel& lhs, const TPooledLabel& rhs) { + switch (MetricsMergingMode_) { + case EMetricsMergingMode::MERGE_METRICS: { + auto& metric = Metrics_.back(); + Sort(metric.Labels, [] (const TPooledLabel& lhs, const TPooledLabel& rhs) { return std::tie(lhs.Key, lhs.Value) < std::tie(rhs.Key, rhs.Value); }); - auto it = MetricMap_.find(metric.Labels); - if (it == std::end(MetricMap_)) { - MetricMap_.emplace(metric.Labels, Metrics_.size() - 1); + auto it = MetricMap_.find(metric.Labels); + if (it == std::end(MetricMap_)) { + MetricMap_.emplace(metric.Labels, Metrics_.size() - 1); } else { - auto& existing = Metrics_[it->second].TimeSeries; + auto& existing = Metrics_[it->second].TimeSeries; - Y_ENSURE(existing.GetValueType() == metric.TimeSeries.GetValueType(), + Y_ENSURE(existing.GetValueType() == metric.TimeSeries.GetValueType(), "Time series point type mismatch: expected " << existing.GetValueType() << " but found " << metric.TimeSeries.GetValueType() << ", labels '" << FormatLabels(metric.Labels) << "'"); - existing.CopyFrom(metric.TimeSeries); - Metrics_.pop_back(); + existing.CopyFrom(metric.TimeSeries); + Metrics_.pop_back(); } break; } - case EMetricsMergingMode::DEFAULT: + case EMetricsMergingMode::DEFAULT: break; } } void TBufferedEncoderBase::OnLabelsBegin() { - if (State_ == TEncoderState::EState::METRIC) { - State_ = TEncoderState::EState::METRIC_LABELS; + if (State_ == TEncoderState::EState::METRIC) { + State_ = TEncoderState::EState::METRIC_LABELS; } else if (State_ == TEncoderState::EState::ROOT) { State_ = TEncoderState::EState::COMMON_LABELS; } else { - State_.ThrowInvalid("expected METRIC or ROOT"); + State_.ThrowInvalid("expected METRIC or ROOT"); } } void TBufferedEncoderBase::OnLabelsEnd() { - if (State_ == TEncoderState::EState::METRIC_LABELS) { - State_ = TEncoderState::EState::METRIC; + if (State_ == TEncoderState::EState::METRIC_LABELS) { + State_ = TEncoderState::EState::METRIC; } else if (State_ == TEncoderState::EState::COMMON_LABELS) { State_ = TEncoderState::EState::ROOT; } else { @@ -76,10 +76,10 @@ void TBufferedEncoderBase::OnLabelsEnd() { } } -void TBufferedEncoderBase::OnLabel(TStringBuf name, TStringBuf value) { +void TBufferedEncoderBase::OnLabel(TStringBuf name, TStringBuf value) { TPooledLabels* labels; - if (State_ == TEncoderState::EState::METRIC_LABELS) { - labels = &Metrics_.back().Labels; + if (State_ == TEncoderState::EState::METRIC_LABELS) { + labels = &Metrics_.back().Labels; } else if (State_ == TEncoderState::EState::COMMON_LABELS) { labels = &CommonLabels_; } else { @@ -102,46 +102,46 @@ void TBufferedEncoderBase::OnLabel(ui32 name, ui32 value) { labels->emplace_back(LabelNamesPool_.GetByIndex(name), LabelValuesPool_.GetByIndex(value)); } -std::pair<ui32, ui32> TBufferedEncoderBase::PrepareLabel(TStringBuf name, TStringBuf value) { +std::pair<ui32, ui32> TBufferedEncoderBase::PrepareLabel(TStringBuf name, TStringBuf value) { auto nameLabel = LabelNamesPool_.PutIfAbsent(name); auto valueLabel = LabelValuesPool_.PutIfAbsent(value); return std::make_pair(nameLabel->Index, valueLabel->Index); } void TBufferedEncoderBase::OnDouble(TInstant time, double value) { - State_.Expect(TEncoderState::EState::METRIC); - TMetric& metric = Metrics_.back(); - metric.TimeSeries.Add(time, value); + State_.Expect(TEncoderState::EState::METRIC); + TMetric& metric = Metrics_.back(); + metric.TimeSeries.Add(time, value); +} + +void TBufferedEncoderBase::OnInt64(TInstant time, i64 value) { + State_.Expect(TEncoderState::EState::METRIC); + TMetric& metric = Metrics_.back(); + metric.TimeSeries.Add(time, value); } -void TBufferedEncoderBase::OnInt64(TInstant time, i64 value) { - State_.Expect(TEncoderState::EState::METRIC); - TMetric& metric = Metrics_.back(); - metric.TimeSeries.Add(time, value); -} - void TBufferedEncoderBase::OnUint64(TInstant time, ui64 value) { - State_.Expect(TEncoderState::EState::METRIC); - TMetric& metric = Metrics_.back(); - metric.TimeSeries.Add(time, value); + State_.Expect(TEncoderState::EState::METRIC); + TMetric& metric = Metrics_.back(); + metric.TimeSeries.Add(time, value); +} + +void TBufferedEncoderBase::OnHistogram(TInstant time, IHistogramSnapshotPtr s) { + State_.Expect(TEncoderState::EState::METRIC); + TMetric& metric = Metrics_.back(); + metric.TimeSeries.Add(time, s.Get()); } -void TBufferedEncoderBase::OnHistogram(TInstant time, IHistogramSnapshotPtr s) { - State_.Expect(TEncoderState::EState::METRIC); - TMetric& metric = Metrics_.back(); - metric.TimeSeries.Add(time, s.Get()); -} - void TBufferedEncoderBase::OnSummaryDouble(TInstant time, ISummaryDoubleSnapshotPtr s) { - State_.Expect(TEncoderState::EState::METRIC); - TMetric& metric = Metrics_.back(); - metric.TimeSeries.Add(time, s.Get()); + State_.Expect(TEncoderState::EState::METRIC); + TMetric& metric = Metrics_.back(); + metric.TimeSeries.Add(time, s.Get()); } void TBufferedEncoderBase::OnLogHistogram(TInstant time, TLogHistogramSnapshotPtr s) { - State_.Expect(TEncoderState::EState::METRIC); - TMetric& metric = Metrics_.back(); - metric.TimeSeries.Add(time, s.Get()); + State_.Expect(TEncoderState::EState::METRIC); + TMetric& metric = Metrics_.back(); + metric.TimeSeries.Add(time, s.Get()); } TString TBufferedEncoderBase::FormatLabels(const TPooledLabels& labels) const { diff --git a/library/cpp/monlib/encode/buffered/buffered_encoder_base.h b/library/cpp/monlib/encode/buffered/buffered_encoder_base.h index d57ed5cd14f..fe3714e58f0 100644 --- a/library/cpp/monlib/encode/buffered/buffered_encoder_base.h +++ b/library/cpp/monlib/encode/buffered/buffered_encoder_base.h @@ -2,10 +2,10 @@ #include "string_pool.h" -#include <library/cpp/monlib/encode/encoder.h> -#include <library/cpp/monlib/encode/encoder_state.h> -#include <library/cpp/monlib/encode/format.h> -#include <library/cpp/monlib/metrics/metric_value.h> +#include <library/cpp/monlib/encode/encoder.h> +#include <library/cpp/monlib/encode/encoder_state.h> +#include <library/cpp/monlib/encode/format.h> +#include <library/cpp/monlib/metrics/metric_value.h> #include <util/datetime/base.h> #include <util/digest/numeric.h> @@ -13,30 +13,30 @@ namespace NMonitoring { -class TBufferedEncoderBase : public IMetricEncoder { +class TBufferedEncoderBase : public IMetricEncoder { public: void OnStreamBegin() override; void OnStreamEnd() override; void OnCommonTime(TInstant time) override; - void OnMetricBegin(EMetricType type) override; - void OnMetricEnd() override; + void OnMetricBegin(EMetricType type) override; + void OnMetricEnd() override; void OnLabelsBegin() override; void OnLabelsEnd() override; - void OnLabel(TStringBuf name, TStringBuf value) override; + void OnLabel(TStringBuf name, TStringBuf value) override; void OnLabel(ui32 name, ui32 value) override; - std::pair<ui32, ui32> PrepareLabel(TStringBuf name, TStringBuf value) override; + std::pair<ui32, ui32> PrepareLabel(TStringBuf name, TStringBuf value) override; void OnDouble(TInstant time, double value) override; - void OnInt64(TInstant time, i64 value) override; + void OnInt64(TInstant time, i64 value) override; void OnUint64(TInstant time, ui64 value) override; - void OnHistogram(TInstant time, IHistogramSnapshotPtr snapshot) override; + void OnHistogram(TInstant time, IHistogramSnapshotPtr snapshot) override; void OnSummaryDouble(TInstant time, ISummaryDoubleSnapshotPtr snapshot) override; void OnLogHistogram(TInstant, TLogHistogramSnapshotPtr) override; - + protected: using TPooledStr = TStringPoolBuilder::TValue; @@ -74,12 +74,12 @@ protected: } }; - using TMetricMap = THashMap<TPooledLabels, size_t, TPooledLabelsHash>; + using TMetricMap = THashMap<TPooledLabels, size_t, TPooledLabelsHash>; - struct TMetric { - EMetricType MetricType = EMetricType::UNKNOWN; + struct TMetric { + EMetricType MetricType = EMetricType::UNKNOWN; TPooledLabels Labels; - TMetricTimeSeries TimeSeries; + TMetricTimeSeries TimeSeries; }; protected: @@ -92,9 +92,9 @@ protected: TStringPoolBuilder LabelValuesPool_; TInstant CommonTime_ = TInstant::Zero(); TPooledLabels CommonLabels_; - TVector<TMetric> Metrics_; - TMetricMap MetricMap_; - EMetricsMergingMode MetricsMergingMode_ = EMetricsMergingMode::DEFAULT; + TVector<TMetric> Metrics_; + TMetricMap MetricMap_; + EMetricsMergingMode MetricsMergingMode_ = EMetricsMergingMode::DEFAULT; }; } diff --git a/library/cpp/monlib/encode/buffered/string_pool.cpp b/library/cpp/monlib/encode/buffered/string_pool.cpp index 2015ae98600..b4c7988ba33 100644 --- a/library/cpp/monlib/encode/buffered/string_pool.cpp +++ b/library/cpp/monlib/encode/buffered/string_pool.cpp @@ -1,24 +1,24 @@ -#include "string_pool.h" - -namespace NMonitoring { - //////////////////////////////////////////////////////////////////////////////// - // TStringPoolBuilder - //////////////////////////////////////////////////////////////////////////////// - const TStringPoolBuilder::TValue* TStringPoolBuilder::PutIfAbsent(TStringBuf str) { +#include "string_pool.h" + +namespace NMonitoring { + //////////////////////////////////////////////////////////////////////////////// + // TStringPoolBuilder + //////////////////////////////////////////////////////////////////////////////// + const TStringPoolBuilder::TValue* TStringPoolBuilder::PutIfAbsent(TStringBuf str) { Y_ENSURE(!IsBuilt_, "Cannot add more values after string has been built"); - + auto [it, isInserted] = StrMap_.try_emplace(str, Max<ui32>(), 0); if (isInserted) { BytesSize_ += str.size(); it->second.Index = StrVector_.size(); StrVector_.emplace_back(it->first, &it->second); - } + } TValue* value = &it->second; - ++value->Frequency; - return value; - } - + ++value->Frequency; + return value; + } + const TStringPoolBuilder::TValue* TStringPoolBuilder::GetByIndex(ui32 index) const { return StrVector_.at(index).second; } @@ -29,30 +29,30 @@ namespace NMonitoring { std::sort(StrVector_.begin(), StrVector_.end(), [](auto& a, auto& b) { return a.second->Frequency > b.second->Frequency; }); - + ui32 i = 0; for (auto& value : StrVector_) { value.second->Index = i++; } - } - + } + IsBuilt_ = true; return *this; - } - - //////////////////////////////////////////////////////////////////////////////// - // TStringPool - //////////////////////////////////////////////////////////////////////////////// - void TStringPool::InitIndex(const char* data, ui32 size) { - const char* begin = data; - const char* end = begin + size; - for (const char* p = begin; p != end; ++p) { - if (*p == '\0') { - Index_.push_back(TStringBuf(begin, p)); - begin = p + 1; - } - } - } - -} + } + + //////////////////////////////////////////////////////////////////////////////// + // TStringPool + //////////////////////////////////////////////////////////////////////////////// + void TStringPool::InitIndex(const char* data, ui32 size) { + const char* begin = data; + const char* end = begin + size; + for (const char* p = begin; p != end; ++p) { + if (*p == '\0') { + Index_.push_back(TStringBuf(begin, p)); + begin = p + 1; + } + } + } + +} diff --git a/library/cpp/monlib/encode/buffered/string_pool.h b/library/cpp/monlib/encode/buffered/string_pool.h index 2d67fd37a35..00e5644608a 100644 --- a/library/cpp/monlib/encode/buffered/string_pool.h +++ b/library/cpp/monlib/encode/buffered/string_pool.h @@ -1,14 +1,14 @@ -#pragma once - -#include <util/generic/hash.h> -#include <util/generic/vector.h> - -namespace NMonitoring { - //////////////////////////////////////////////////////////////////////////////// - // TStringPoolBuilder - //////////////////////////////////////////////////////////////////////////////// - class TStringPoolBuilder { - public: +#pragma once + +#include <util/generic/hash.h> +#include <util/generic/vector.h> + +namespace NMonitoring { + //////////////////////////////////////////////////////////////////////////////// + // TStringPoolBuilder + //////////////////////////////////////////////////////////////////////////////// + class TStringPoolBuilder { + public: struct TValue: TNonCopyable { TValue(ui32 idx, ui32 freq) : Index{idx} @@ -16,77 +16,77 @@ namespace NMonitoring { { } - ui32 Index; - ui32 Frequency; - }; - - public: - const TValue* PutIfAbsent(TStringBuf str); + ui32 Index; + ui32 Frequency; + }; + + public: + const TValue* PutIfAbsent(TStringBuf str); const TValue* GetByIndex(ui32 index) const; - + /// Determines whether pool must be sorted by value frequencies TStringPoolBuilder& SetSorted(bool sorted) { RequiresSorting_ = sorted; return *this; } - + TStringPoolBuilder& Build(); - TStringBuf Get(ui32 index) const { + TStringBuf Get(ui32 index) const { Y_ENSURE(IsBuilt_, "Pool must be sorted first"); - return StrVector_.at(index).first; - } - + return StrVector_.at(index).first; + } + TStringBuf Get(const TValue* value) const { return StrVector_.at(value->Index).first; } - template <typename TConsumer> - void ForEach(TConsumer&& c) { + template <typename TConsumer> + void ForEach(TConsumer&& c) { Y_ENSURE(IsBuilt_, "Pool must be sorted first"); - for (const auto& value : StrVector_) { - c(value.first, value.second->Index, value.second->Frequency); - } - } - - size_t BytesSize() const noexcept { - return BytesSize_; - } - - size_t Count() const noexcept { - return StrMap_.size(); - } - - private: - THashMap<TString, TValue> StrMap_; - TVector<std::pair<TStringBuf, TValue*>> StrVector_; + for (const auto& value : StrVector_) { + c(value.first, value.second->Index, value.second->Frequency); + } + } + + size_t BytesSize() const noexcept { + return BytesSize_; + } + + size_t Count() const noexcept { + return StrMap_.size(); + } + + private: + THashMap<TString, TValue> StrMap_; + TVector<std::pair<TStringBuf, TValue*>> StrVector_; bool RequiresSorting_ = false; bool IsBuilt_ = false; - size_t BytesSize_ = 0; - }; - - //////////////////////////////////////////////////////////////////////////////// - // TStringPool - //////////////////////////////////////////////////////////////////////////////// - class TStringPool { - public: - TStringPool(const char* data, ui32 size) { - InitIndex(data, size); - } - - TStringBuf Get(ui32 i) const { - return Index_.at(i); - } - + size_t BytesSize_ = 0; + }; + + //////////////////////////////////////////////////////////////////////////////// + // TStringPool + //////////////////////////////////////////////////////////////////////////////// + class TStringPool { + public: + TStringPool(const char* data, ui32 size) { + InitIndex(data, size); + } + + TStringBuf Get(ui32 i) const { + return Index_.at(i); + } + size_t Size() const { return Index_.size(); } - private: - void InitIndex(const char* data, ui32 size); - - private: - TVector<TStringBuf> Index_; - }; - -} + private: + void InitIndex(const char* data, ui32 size); + + private: + TVector<TStringBuf> Index_; + }; + +} diff --git a/library/cpp/monlib/encode/buffered/string_pool_ut.cpp b/library/cpp/monlib/encode/buffered/string_pool_ut.cpp index 7f8b4c67240..9fc3421d0bb 100644 --- a/library/cpp/monlib/encode/buffered/string_pool_ut.cpp +++ b/library/cpp/monlib/encode/buffered/string_pool_ut.cpp @@ -1,84 +1,84 @@ -#include "string_pool.h" - +#include "string_pool.h" + #include <library/cpp/testing/unittest/registar.h> - -using namespace NMonitoring; - + +using namespace NMonitoring; + Y_UNIT_TEST_SUITE(TStringPoolTest) { Y_UNIT_TEST(PutIfAbsent) { - TStringPoolBuilder strPool; + TStringPoolBuilder strPool; strPool.SetSorted(true); - auto* h1 = strPool.PutIfAbsent("one"); - auto* h2 = strPool.PutIfAbsent("two"); - auto* h3 = strPool.PutIfAbsent("two"); - UNIT_ASSERT(h1 != h2); - UNIT_ASSERT(h2 == h3); - - UNIT_ASSERT_VALUES_EQUAL(h1->Frequency, 1); + auto* h1 = strPool.PutIfAbsent("one"); + auto* h2 = strPool.PutIfAbsent("two"); + auto* h3 = strPool.PutIfAbsent("two"); + UNIT_ASSERT(h1 != h2); + UNIT_ASSERT(h2 == h3); + + UNIT_ASSERT_VALUES_EQUAL(h1->Frequency, 1); UNIT_ASSERT_VALUES_EQUAL(h1->Index, 0); - - UNIT_ASSERT_VALUES_EQUAL(h2->Frequency, 2); + + UNIT_ASSERT_VALUES_EQUAL(h2->Frequency, 2); UNIT_ASSERT_VALUES_EQUAL(h2->Index, 1); - - UNIT_ASSERT_VALUES_EQUAL(strPool.BytesSize(), 6); - UNIT_ASSERT_VALUES_EQUAL(strPool.Count(), 2); - } - + + UNIT_ASSERT_VALUES_EQUAL(strPool.BytesSize(), 6); + UNIT_ASSERT_VALUES_EQUAL(strPool.Count(), 2); + } + Y_UNIT_TEST(SortByFrequency) { - TStringPoolBuilder strPool; + TStringPoolBuilder strPool; strPool.SetSorted(true); - auto* h1 = strPool.PutIfAbsent("one"); - auto* h2 = strPool.PutIfAbsent("two"); - auto* h3 = strPool.PutIfAbsent("two"); - UNIT_ASSERT(h1 != h2); - UNIT_ASSERT(h2 == h3); - + auto* h1 = strPool.PutIfAbsent("one"); + auto* h2 = strPool.PutIfAbsent("two"); + auto* h3 = strPool.PutIfAbsent("two"); + UNIT_ASSERT(h1 != h2); + UNIT_ASSERT(h2 == h3); + strPool.Build(); - - UNIT_ASSERT_VALUES_EQUAL(h1->Frequency, 1); - UNIT_ASSERT_VALUES_EQUAL(h1->Index, 1); - - UNIT_ASSERT_VALUES_EQUAL(h2->Frequency, 2); - UNIT_ASSERT_VALUES_EQUAL(h2->Index, 0); - - UNIT_ASSERT_VALUES_EQUAL(strPool.BytesSize(), 6); - UNIT_ASSERT_VALUES_EQUAL(strPool.Count(), 2); - } - + + UNIT_ASSERT_VALUES_EQUAL(h1->Frequency, 1); + UNIT_ASSERT_VALUES_EQUAL(h1->Index, 1); + + UNIT_ASSERT_VALUES_EQUAL(h2->Frequency, 2); + UNIT_ASSERT_VALUES_EQUAL(h2->Index, 0); + + UNIT_ASSERT_VALUES_EQUAL(strPool.BytesSize(), 6); + UNIT_ASSERT_VALUES_EQUAL(strPool.Count(), 2); + } + Y_UNIT_TEST(ForEach) { - TStringPoolBuilder strPool; + TStringPoolBuilder strPool; strPool.SetSorted(true); - strPool.PutIfAbsent("one"); - strPool.PutIfAbsent("two"); - strPool.PutIfAbsent("two"); - strPool.PutIfAbsent("three"); - strPool.PutIfAbsent("three"); - strPool.PutIfAbsent("three"); - - UNIT_ASSERT_VALUES_EQUAL(strPool.BytesSize(), 11); - UNIT_ASSERT_VALUES_EQUAL(strPool.Count(), 3); - + strPool.PutIfAbsent("one"); + strPool.PutIfAbsent("two"); + strPool.PutIfAbsent("two"); + strPool.PutIfAbsent("three"); + strPool.PutIfAbsent("three"); + strPool.PutIfAbsent("three"); + + UNIT_ASSERT_VALUES_EQUAL(strPool.BytesSize(), 11); + UNIT_ASSERT_VALUES_EQUAL(strPool.Count(), 3); + strPool.Build(); - + TVector<TString> strings; TVector<ui32> indexes; TVector<ui32> frequences; - strPool.ForEach([&](TStringBuf str, ui32 index, ui32 freq) { - strings.emplace_back(str); - indexes.push_back(index); - frequences.push_back(freq); - }); - + strPool.ForEach([&](TStringBuf str, ui32 index, ui32 freq) { + strings.emplace_back(str); + indexes.push_back(index); + frequences.push_back(freq); + }); + TVector<TString> expectedStrings = {"three", "two", "one"}; - UNIT_ASSERT_EQUAL(strings, expectedStrings); - + UNIT_ASSERT_EQUAL(strings, expectedStrings); + TVector<ui32> expectedIndexes = {0, 1, 2}; - UNIT_ASSERT_EQUAL(indexes, expectedIndexes); - + UNIT_ASSERT_EQUAL(indexes, expectedIndexes); + TVector<ui32> expectedFrequences = {3, 2, 1}; - UNIT_ASSERT_EQUAL(frequences, expectedFrequences); - } -} + UNIT_ASSERT_EQUAL(frequences, expectedFrequences); + } +} diff --git a/library/cpp/monlib/encode/buffered/ut/ya.make b/library/cpp/monlib/encode/buffered/ut/ya.make index af2e09ac67c..2157ac14906 100644 --- a/library/cpp/monlib/encode/buffered/ut/ya.make +++ b/library/cpp/monlib/encode/buffered/ut/ya.make @@ -1,4 +1,4 @@ -UNITTEST_FOR(library/cpp/monlib/encode/buffered) +UNITTEST_FOR(library/cpp/monlib/encode/buffered) OWNER( g:solomon diff --git a/library/cpp/monlib/encode/buffered/ya.make b/library/cpp/monlib/encode/buffered/ya.make index edaf29aa93a..81b6a78b93b 100644 --- a/library/cpp/monlib/encode/buffered/ya.make +++ b/library/cpp/monlib/encode/buffered/ya.make @@ -12,8 +12,8 @@ SRCS( ) PEERDIR( - library/cpp/monlib/encode - library/cpp/monlib/metrics + library/cpp/monlib/encode + library/cpp/monlib/metrics ) END() diff --git a/library/cpp/monlib/encode/encoder.cpp b/library/cpp/monlib/encode/encoder.cpp index 08b3fc4fd5f..becf932689b 100644 --- a/library/cpp/monlib/encode/encoder.cpp +++ b/library/cpp/monlib/encode/encoder.cpp @@ -1,6 +1,6 @@ -#include "encoder.h" - -namespace NMonitoring { - IMetricEncoder::~IMetricEncoder() { - } -} +#include "encoder.h" + +namespace NMonitoring { + IMetricEncoder::~IMetricEncoder() { + } +} diff --git a/library/cpp/monlib/encode/encoder.h b/library/cpp/monlib/encode/encoder.h index 5b307b4d45c..a26a133d16b 100644 --- a/library/cpp/monlib/encode/encoder.h +++ b/library/cpp/monlib/encode/encoder.h @@ -1,17 +1,17 @@ -#pragma once - -#include <util/generic/ptr.h> - -#include <library/cpp/monlib/metrics/metric_consumer.h> - -namespace NMonitoring { - class IMetricEncoder: public IMetricConsumer { - public: - virtual ~IMetricEncoder(); - - virtual void Close() = 0; - }; - - using IMetricEncoderPtr = THolder<IMetricEncoder>; - -} +#pragma once + +#include <util/generic/ptr.h> + +#include <library/cpp/monlib/metrics/metric_consumer.h> + +namespace NMonitoring { + class IMetricEncoder: public IMetricConsumer { + public: + virtual ~IMetricEncoder(); + + virtual void Close() = 0; + }; + + using IMetricEncoderPtr = THolder<IMetricEncoder>; + +} diff --git a/library/cpp/monlib/encode/encoder_state.cpp b/library/cpp/monlib/encode/encoder_state.cpp index 166369f8762..0ece696b1a6 100644 --- a/library/cpp/monlib/encode/encoder_state.cpp +++ b/library/cpp/monlib/encode/encoder_state.cpp @@ -1 +1 @@ -#include "encoder_state.h" +#include "encoder_state.h" diff --git a/library/cpp/monlib/encode/encoder_state.h b/library/cpp/monlib/encode/encoder_state.h index 51562a3d5f0..e6a098f4044 100644 --- a/library/cpp/monlib/encode/encoder_state.h +++ b/library/cpp/monlib/encode/encoder_state.h @@ -1,62 +1,62 @@ -#pragma once - +#pragma once + #include "encoder_state_enum.h" #include <util/generic/serialized_enum.h> -#include <util/generic/yexception.h> - +#include <util/generic/yexception.h> + -namespace NMonitoring { +namespace NMonitoring { template <typename EEncoderState> class TEncoderStateImpl { - public: + public: using EState = EEncoderState; - + explicit TEncoderStateImpl(EEncoderState state = EEncoderState::ROOT) - : State_(state) - { - } - + : State_(state) + { + } + TEncoderStateImpl& operator=(EEncoderState rhs) noexcept { - State_ = rhs; - return *this; - } - - inline bool operator==(EEncoderState rhs) const noexcept { - return State_ == rhs; - } - - inline bool operator!=(EEncoderState rhs) const noexcept { - return !operator==(rhs); - } - + State_ = rhs; + return *this; + } + + inline bool operator==(EEncoderState rhs) const noexcept { + return State_ == rhs; + } + + inline bool operator!=(EEncoderState rhs) const noexcept { + return !operator==(rhs); + } + [[noreturn]] inline void ThrowInvalid(TStringBuf message) const { - ythrow yexception() << "invalid encoder state: " - << ToStr() << ", " << message; - } - - inline void Expect(EEncoderState expected) const { - if (Y_UNLIKELY(State_ != expected)) { - ythrow yexception() - << "invalid encoder state: " << ToStr() + ythrow yexception() << "invalid encoder state: " + << ToStr() << ", " << message; + } + + inline void Expect(EEncoderState expected) const { + if (Y_UNLIKELY(State_ != expected)) { + ythrow yexception() + << "invalid encoder state: " << ToStr() << ", expected: " << TEncoderStateImpl(expected).ToStr(); - } - } - - inline void Switch(EEncoderState from, EEncoderState to) { - Expect(from); - State_ = to; - } - + } + } + + inline void Switch(EEncoderState from, EEncoderState to) { + Expect(from); + State_ = to; + } + TStringBuf ToStr() const noexcept { return NEnumSerializationRuntime::GetEnumNamesImpl<EEncoderState>().at(State_); } - - private: - EEncoderState State_; - }; - + + private: + EEncoderState State_; + }; + using TEncoderState = TEncoderStateImpl<EEncoderState>; } // namespace NMonitoring diff --git a/library/cpp/monlib/encode/encoder_state_enum.h b/library/cpp/monlib/encode/encoder_state_enum.h index 54f48a25e5e..471604f91db 100644 --- a/library/cpp/monlib/encode/encoder_state_enum.h +++ b/library/cpp/monlib/encode/encoder_state_enum.h @@ -5,8 +5,8 @@ namespace NMonitoring { enum class EEncoderState { ROOT, COMMON_LABELS, - METRIC, - METRIC_LABELS, + METRIC, + METRIC_LABELS, }; } // namespace NMonitoring diff --git a/library/cpp/monlib/encode/fake/fake.cpp b/library/cpp/monlib/encode/fake/fake.cpp index 3bd3be89947..69d691361ae 100644 --- a/library/cpp/monlib/encode/fake/fake.cpp +++ b/library/cpp/monlib/encode/fake/fake.cpp @@ -3,7 +3,7 @@ #include <util/datetime/base.h> namespace NMonitoring { - class TFakeEncoder: public IMetricEncoder { + class TFakeEncoder: public IMetricEncoder { public: void OnStreamBegin() override { } @@ -13,9 +13,9 @@ namespace NMonitoring { void OnCommonTime(TInstant) override { } - void OnMetricBegin(EMetricType) override { + void OnMetricBegin(EMetricType) override { } - void OnMetricEnd() override { + void OnMetricEnd() override { } void OnLabelsBegin() override { @@ -45,7 +45,7 @@ namespace NMonitoring { } }; - IMetricEncoderPtr EncoderFake() { + IMetricEncoderPtr EncoderFake() { return MakeHolder<TFakeEncoder>(); } } diff --git a/library/cpp/monlib/encode/fake/fake.h b/library/cpp/monlib/encode/fake/fake.h index 9b3e681c9b2..81093269878 100644 --- a/library/cpp/monlib/encode/fake/fake.h +++ b/library/cpp/monlib/encode/fake/fake.h @@ -1,10 +1,10 @@ #pragma once -#include <library/cpp/monlib/encode/encoder.h> +#include <library/cpp/monlib/encode/encoder.h> class IOutputStream; namespace NMonitoring { - // Does nothing: just implements IMetricEncoder interface with stubs - IMetricEncoderPtr EncoderFake(); + // Does nothing: just implements IMetricEncoder interface with stubs + IMetricEncoderPtr EncoderFake(); } diff --git a/library/cpp/monlib/encode/format.cpp b/library/cpp/monlib/encode/format.cpp index d9bb4fcd043..400ce5a6438 100644 --- a/library/cpp/monlib/encode/format.cpp +++ b/library/cpp/monlib/encode/format.cpp @@ -1,12 +1,12 @@ -#include "format.h" - -#include <util/string/ascii.h> +#include "format.h" + +#include <util/string/ascii.h> #include <util/string/split.h> -#include <util/string/strip.h> -#include <util/stream/output.h> -#include <util/string/cast.h> - -namespace NMonitoring { +#include <util/string/strip.h> +#include <util/stream/output.h> +#include <util/string/cast.h> + +namespace NMonitoring { static ECompression CompressionFromHeader(TStringBuf value) { if (value.empty()) { return ECompression::UNKNOWN; @@ -38,165 +38,165 @@ namespace NMonitoring { return EFormat::PROTOBUF; } else if (AsciiEqualsIgnoreCase(value, NFormatContenType::TEXT)) { return EFormat::TEXT; - } else if (AsciiEqualsIgnoreCase(value, NFormatContenType::PROMETHEUS)) { - return EFormat::PROMETHEUS; - } - + } else if (AsciiEqualsIgnoreCase(value, NFormatContenType::PROMETHEUS)) { + return EFormat::PROMETHEUS; + } + return EFormat::UNKNOWN; } EFormat FormatFromAcceptHeader(TStringBuf value) { EFormat result{EFormat::UNKNOWN}; - for (const auto& it : StringSplitter(value).Split(',').SkipEmpty()) { - TStringBuf token = StripString(it.Token()); - + for (const auto& it : StringSplitter(value).Split(',').SkipEmpty()) { + TStringBuf token = StripString(it.Token()); + result = FormatFromHttpMedia(token); if (result != EFormat::UNKNOWN) { break; - } - } - + } + } + return result; - } - + } + EFormat FormatFromContentType(TStringBuf value) { value = value.NextTok(';'); return FormatFromHttpMedia(value); } - TStringBuf ContentTypeByFormat(EFormat format) { - switch (format) { - case EFormat::SPACK: - return NFormatContenType::SPACK; - case EFormat::JSON: - return NFormatContenType::JSON; - case EFormat::PROTOBUF: - return NFormatContenType::PROTOBUF; - case EFormat::TEXT: - return NFormatContenType::TEXT; - case EFormat::PROMETHEUS: - return NFormatContenType::PROMETHEUS; - case EFormat::UNKNOWN: - return TStringBuf(); - } - - Y_FAIL(); // for GCC - } - - ECompression CompressionFromAcceptEncodingHeader(TStringBuf value) { + TStringBuf ContentTypeByFormat(EFormat format) { + switch (format) { + case EFormat::SPACK: + return NFormatContenType::SPACK; + case EFormat::JSON: + return NFormatContenType::JSON; + case EFormat::PROTOBUF: + return NFormatContenType::PROTOBUF; + case EFormat::TEXT: + return NFormatContenType::TEXT; + case EFormat::PROMETHEUS: + return NFormatContenType::PROMETHEUS; + case EFormat::UNKNOWN: + return TStringBuf(); + } + + Y_FAIL(); // for GCC + } + + ECompression CompressionFromAcceptEncodingHeader(TStringBuf value) { return CompressionFromHeader(value); } - + ECompression CompressionFromContentEncodingHeader(TStringBuf value) { return CompressionFromHeader(value); - } - - TStringBuf ContentEncodingByCompression(ECompression compression) { - switch (compression) { - case ECompression::IDENTITY: - return NFormatContentEncoding::IDENTITY; - case ECompression::ZLIB: - return NFormatContentEncoding::ZLIB; - case ECompression::LZ4: - return NFormatContentEncoding::LZ4; - case ECompression::ZSTD: - return NFormatContentEncoding::ZSTD; - case ECompression::UNKNOWN: - return TStringBuf(); - } - - Y_FAIL(); // for GCC - } - -} - -template <> -NMonitoring::EFormat FromStringImpl<NMonitoring::EFormat>(const char* str, size_t len) { - using NMonitoring::EFormat; - TStringBuf value(str, len); + } + + TStringBuf ContentEncodingByCompression(ECompression compression) { + switch (compression) { + case ECompression::IDENTITY: + return NFormatContentEncoding::IDENTITY; + case ECompression::ZLIB: + return NFormatContentEncoding::ZLIB; + case ECompression::LZ4: + return NFormatContentEncoding::LZ4; + case ECompression::ZSTD: + return NFormatContentEncoding::ZSTD; + case ECompression::UNKNOWN: + return TStringBuf(); + } + + Y_FAIL(); // for GCC + } + +} + +template <> +NMonitoring::EFormat FromStringImpl<NMonitoring::EFormat>(const char* str, size_t len) { + using NMonitoring::EFormat; + TStringBuf value(str, len); if (value == TStringBuf("SPACK")) { - return EFormat::SPACK; + return EFormat::SPACK; } else if (value == TStringBuf("JSON")) { - return EFormat::JSON; + return EFormat::JSON; } else if (value == TStringBuf("PROTOBUF")) { - return EFormat::PROTOBUF; + return EFormat::PROTOBUF; } else if (value == TStringBuf("TEXT")) { - return EFormat::TEXT; + return EFormat::TEXT; } else if (value == TStringBuf("PROMETHEUS")) { - return EFormat::PROMETHEUS; + return EFormat::PROMETHEUS; } else if (value == TStringBuf("UNKNOWN")) { - return EFormat::UNKNOWN; - } - ythrow yexception() << "unknown format: " << value; -} - -template <> -void Out<NMonitoring::EFormat>(IOutputStream& o, NMonitoring::EFormat f) { - using NMonitoring::EFormat; - switch (f) { - case EFormat::SPACK: + return EFormat::UNKNOWN; + } + ythrow yexception() << "unknown format: " << value; +} + +template <> +void Out<NMonitoring::EFormat>(IOutputStream& o, NMonitoring::EFormat f) { + using NMonitoring::EFormat; + switch (f) { + case EFormat::SPACK: o << TStringBuf("SPACK"); - return; - case EFormat::JSON: + return; + case EFormat::JSON: o << TStringBuf("JSON"); - return; - case EFormat::PROTOBUF: + return; + case EFormat::PROTOBUF: o << TStringBuf("PROTOBUF"); - return; - case EFormat::TEXT: + return; + case EFormat::TEXT: o << TStringBuf("TEXT"); - return; - case EFormat::PROMETHEUS: + return; + case EFormat::PROMETHEUS: o << TStringBuf("PROMETHEUS"); - return; - case EFormat::UNKNOWN: + return; + case EFormat::UNKNOWN: o << TStringBuf("UNKNOWN"); - return; - } - - Y_FAIL(); // for GCC -} - -template <> -NMonitoring::ECompression FromStringImpl<NMonitoring::ECompression>(const char* str, size_t len) { - using NMonitoring::ECompression; - TStringBuf value(str, len); + return; + } + + Y_FAIL(); // for GCC +} + +template <> +NMonitoring::ECompression FromStringImpl<NMonitoring::ECompression>(const char* str, size_t len) { + using NMonitoring::ECompression; + TStringBuf value(str, len); if (value == TStringBuf("IDENTITY")) { - return ECompression::IDENTITY; + return ECompression::IDENTITY; } else if (value == TStringBuf("ZLIB")) { - return ECompression::ZLIB; + return ECompression::ZLIB; } else if (value == TStringBuf("LZ4")) { - return ECompression::LZ4; + return ECompression::LZ4; } else if (value == TStringBuf("ZSTD")) { - return ECompression::ZSTD; + return ECompression::ZSTD; } else if (value == TStringBuf("UNKNOWN")) { - return ECompression::UNKNOWN; - } - ythrow yexception() << "unknown compression: " << value; -} - -template <> -void Out<NMonitoring::ECompression>(IOutputStream& o, NMonitoring::ECompression c) { - using NMonitoring::ECompression; - switch (c) { - case ECompression::IDENTITY: + return ECompression::UNKNOWN; + } + ythrow yexception() << "unknown compression: " << value; +} + +template <> +void Out<NMonitoring::ECompression>(IOutputStream& o, NMonitoring::ECompression c) { + using NMonitoring::ECompression; + switch (c) { + case ECompression::IDENTITY: o << TStringBuf("IDENTITY"); - return; - case ECompression::ZLIB: + return; + case ECompression::ZLIB: o << TStringBuf("ZLIB"); - return; - case ECompression::LZ4: + return; + case ECompression::LZ4: o << TStringBuf("LZ4"); - return; - case ECompression::ZSTD: + return; + case ECompression::ZSTD: o << TStringBuf("ZSTD"); - return; - case ECompression::UNKNOWN: + return; + case ECompression::UNKNOWN: o << TStringBuf("UNKNOWN"); - return; - } - - Y_FAIL(); // for GCC -} + return; + } + + Y_FAIL(); // for GCC +} diff --git a/library/cpp/monlib/encode/format.h b/library/cpp/monlib/encode/format.h index 8815a32a265..495d42d7867 100644 --- a/library/cpp/monlib/encode/format.h +++ b/library/cpp/monlib/encode/format.h @@ -1,96 +1,96 @@ -#pragma once - -#include <util/generic/strbuf.h> - -namespace NMonitoring { - namespace NFormatContenType { +#pragma once + +#include <util/generic/strbuf.h> + +namespace NMonitoring { + namespace NFormatContenType { constexpr TStringBuf TEXT = "application/x-solomon-txt"; constexpr TStringBuf JSON = "application/json"; constexpr TStringBuf PROTOBUF = "application/x-solomon-pb"; constexpr TStringBuf SPACK = "application/x-solomon-spack"; constexpr TStringBuf PROMETHEUS = "text/plain"; - } - - namespace NFormatContentEncoding { + } + + namespace NFormatContentEncoding { constexpr TStringBuf IDENTITY = "identity"; constexpr TStringBuf ZLIB = "zlib"; constexpr TStringBuf LZ4 = "lz4"; constexpr TStringBuf ZSTD = "zstd"; - } - - /** - * Defines format types for metric encoders. - */ - enum class EFormat { - /** - * Special case when it was not possible to determine format. - */ - UNKNOWN, - - /** - * Read more https://wiki.yandex-team.ru/solomon/api/dataformat/spackv1 - */ - SPACK, - - /** - * Read more https://wiki.yandex-team.ru/solomon/api/dataformat/json - */ - JSON, - - /** - * Simple protobuf format, only for testing purposes. - */ - PROTOBUF, - - /** - * Simple text representation, only for debug purposes. - */ - TEXT, - - /** - * Prometheus text-based format - */ - PROMETHEUS, - }; - - /** - * Defines compression algorithms for metric encoders. - */ - enum class ECompression { - /** - * Special case when it was not possible to determine compression. - */ - UNKNOWN, - - /** - * Means no compression. - */ - IDENTITY, - - /** - * Using the zlib structure (defined in RFC 1950), with the - * deflate compression algorithm and Adler32 checkums. - */ - ZLIB, - - /** - * Using LZ4 compression algorithm (read http://lz4.org for more info) - * with XxHash32 checksums. - */ - LZ4, - - /** - * Using Zstandard compression algorithm (read http://zstd.net for more - * info) with XxHash32 checksums. - */ - ZSTD, - }; - - enum class EMetricsMergingMode { + } + + /** + * Defines format types for metric encoders. + */ + enum class EFormat { + /** + * Special case when it was not possible to determine format. + */ + UNKNOWN, + + /** + * Read more https://wiki.yandex-team.ru/solomon/api/dataformat/spackv1 + */ + SPACK, + + /** + * Read more https://wiki.yandex-team.ru/solomon/api/dataformat/json + */ + JSON, + + /** + * Simple protobuf format, only for testing purposes. + */ + PROTOBUF, + + /** + * Simple text representation, only for debug purposes. + */ + TEXT, + + /** + * Prometheus text-based format + */ + PROMETHEUS, + }; + + /** + * Defines compression algorithms for metric encoders. + */ + enum class ECompression { + /** + * Special case when it was not possible to determine compression. + */ + UNKNOWN, + + /** + * Means no compression. + */ + IDENTITY, + + /** + * Using the zlib structure (defined in RFC 1950), with the + * deflate compression algorithm and Adler32 checkums. + */ + ZLIB, + + /** + * Using LZ4 compression algorithm (read http://lz4.org for more info) + * with XxHash32 checksums. + */ + LZ4, + /** - * Do not merge metric batches. If several points of the same metric were + * Using Zstandard compression algorithm (read http://zstd.net for more + * info) with XxHash32 checksums. + */ + ZSTD, + }; + + enum class EMetricsMergingMode { + /** + * Do not merge metric batches. If several points of the same metric were * added multiple times accross different writes, paste them as - * separate metrics. + * separate metrics. * * Example: * COUNTER [(ts1, val1)] | COUNTER [(ts1, val1)] @@ -98,9 +98,9 @@ namespace NMonitoring { * COUNTER [(ts3, val3)] | COUNTER [(ts3, val3)] */ DEFAULT, - + /** - * If several points of the same metric were added multiple times across + * If several points of the same metric were added multiple times across * different writes, merge all values to one timeseries. * * Example: @@ -108,18 +108,18 @@ namespace NMonitoring { * COUNTER [(ts2, val2)] | --> COUNTER [(ts1, val1), (ts2, val2), (ts3, val3)] * COUNTER [(ts3, val3)] | */ - MERGE_METRICS, + MERGE_METRICS, }; - /** - * Matches serialization format by the given "Accept" header value. - * - * @param value value of the "Accept" header. - * @return most preffered serialization format type - */ + /** + * Matches serialization format by the given "Accept" header value. + * + * @param value value of the "Accept" header. + * @return most preffered serialization format type + */ EFormat FormatFromAcceptHeader(TStringBuf value); - - /** + + /** * Matches serialization format by the given "Content-Type" header value * * @param value value of the "Content-Type" header @@ -128,24 +128,24 @@ namespace NMonitoring { EFormat FormatFromContentType(TStringBuf value); /** - * Returns value for "Content-Type" header determined by the given - * format type. - * - * @param format serialization format type - * @return mime-type indentificator - * or empty string if format is UNKNOWN - */ - TStringBuf ContentTypeByFormat(EFormat format); - - /** - * Matches compression algorithm by the given "Accept-Encoding" header value. - * - * @param value value of the "Accept-Encoding" header. - * @return most preffered compression algorithm - */ - ECompression CompressionFromAcceptEncodingHeader(TStringBuf value); - - /** + * Returns value for "Content-Type" header determined by the given + * format type. + * + * @param format serialization format type + * @return mime-type indentificator + * or empty string if format is UNKNOWN + */ + TStringBuf ContentTypeByFormat(EFormat format); + + /** + * Matches compression algorithm by the given "Accept-Encoding" header value. + * + * @param value value of the "Accept-Encoding" header. + * @return most preffered compression algorithm + */ + ECompression CompressionFromAcceptEncodingHeader(TStringBuf value); + + /** * Matches compression algorithm by the given "Content-Encoding" header value. * * @param value value of the "Accept-Encoding" header. @@ -154,13 +154,13 @@ namespace NMonitoring { ECompression CompressionFromContentEncodingHeader(TStringBuf value); /** - * Returns value for "Content-Encoding" header determined by the given - * compression algorithm. - * - * @param compression encoding compression alg - * @return media-type compresion algorithm - * or empty string if compression is UNKNOWN - */ - TStringBuf ContentEncodingByCompression(ECompression compression); - -} + * Returns value for "Content-Encoding" header determined by the given + * compression algorithm. + * + * @param compression encoding compression alg + * @return media-type compresion algorithm + * or empty string if compression is UNKNOWN + */ + TStringBuf ContentEncodingByCompression(ECompression compression); + +} diff --git a/library/cpp/monlib/encode/format_ut.cpp b/library/cpp/monlib/encode/format_ut.cpp index 63475674d24..22a0e30c037 100644 --- a/library/cpp/monlib/encode/format_ut.cpp +++ b/library/cpp/monlib/encode/format_ut.cpp @@ -1,14 +1,14 @@ -#include "format.h" - +#include "format.h" + #include <library/cpp/testing/unittest/registar.h> - -#include <util/generic/string.h> -#include <util/string/builder.h> - -#include <array> - -using namespace NMonitoring; - + +#include <util/generic/string.h> +#include <util/string/builder.h> + +#include <array> + +using namespace NMonitoring; + Y_UNIT_TEST_SUITE(TFormatTest) { Y_UNIT_TEST(ContentTypeHeader) { UNIT_ASSERT_EQUAL(FormatFromContentType(""), EFormat::UNKNOWN); @@ -21,116 +21,116 @@ Y_UNIT_TEST_SUITE(TFormatTest) { Y_UNIT_TEST(AcceptHeader) { UNIT_ASSERT_EQUAL(FormatFromAcceptHeader(""), EFormat::UNKNOWN); UNIT_ASSERT_EQUAL(FormatFromAcceptHeader("*/*"), EFormat::UNKNOWN); - - UNIT_ASSERT_EQUAL( + + UNIT_ASSERT_EQUAL( FormatFromAcceptHeader("application/xml"), - EFormat::UNKNOWN); - - UNIT_ASSERT_EQUAL( + EFormat::UNKNOWN); + + UNIT_ASSERT_EQUAL( FormatFromAcceptHeader("application/json"), - EFormat::JSON); - - UNIT_ASSERT_EQUAL( + EFormat::JSON); + + UNIT_ASSERT_EQUAL( FormatFromAcceptHeader("application/x-solomon-spack"), - EFormat::SPACK); - - UNIT_ASSERT_EQUAL( + EFormat::SPACK); + + UNIT_ASSERT_EQUAL( FormatFromAcceptHeader("application/x-solomon-pb"), - EFormat::PROTOBUF); - - UNIT_ASSERT_EQUAL( - FormatFromAcceptHeader("application/x-solomon-txt"), - EFormat::TEXT); - - UNIT_ASSERT_EQUAL( + EFormat::PROTOBUF); + + UNIT_ASSERT_EQUAL( + FormatFromAcceptHeader("application/x-solomon-txt"), + EFormat::TEXT); + + UNIT_ASSERT_EQUAL( FormatFromAcceptHeader("application/json, text/plain"), - EFormat::JSON); - - UNIT_ASSERT_EQUAL( + EFormat::JSON); + + UNIT_ASSERT_EQUAL( FormatFromAcceptHeader("application/x-solomon-spack, application/json, text/plain"), - EFormat::SPACK); - - UNIT_ASSERT_EQUAL( + EFormat::SPACK); + + UNIT_ASSERT_EQUAL( FormatFromAcceptHeader(" , application/x-solomon-spack ,, application/json , text/plain"), - EFormat::SPACK); - - UNIT_ASSERT_EQUAL( + EFormat::SPACK); + + UNIT_ASSERT_EQUAL( FormatFromAcceptHeader("application/xml, application/x-solomon-spack, text/plain"), - EFormat::SPACK); - - UNIT_ASSERT_EQUAL( - FormatFromAcceptHeader("text/plain"), - EFormat::PROMETHEUS); - } - + EFormat::SPACK); + + UNIT_ASSERT_EQUAL( + FormatFromAcceptHeader("text/plain"), + EFormat::PROMETHEUS); + } + Y_UNIT_TEST(FormatToStrFromStr) { - const std::array<EFormat, 6> formats = {{ - EFormat::UNKNOWN, - EFormat::SPACK, - EFormat::JSON, - EFormat::PROTOBUF, - EFormat::TEXT, - EFormat::PROMETHEUS, - }}; - - for (EFormat f : formats) { - TString str = (TStringBuilder() << f); - EFormat g = FromString<EFormat>(str); - UNIT_ASSERT_EQUAL(f, g); - } - } - + const std::array<EFormat, 6> formats = {{ + EFormat::UNKNOWN, + EFormat::SPACK, + EFormat::JSON, + EFormat::PROTOBUF, + EFormat::TEXT, + EFormat::PROMETHEUS, + }}; + + for (EFormat f : formats) { + TString str = (TStringBuilder() << f); + EFormat g = FromString<EFormat>(str); + UNIT_ASSERT_EQUAL(f, g); + } + } + Y_UNIT_TEST(AcceptEncodingHeader) { - UNIT_ASSERT_EQUAL( - CompressionFromAcceptEncodingHeader(""), - ECompression::UNKNOWN); - - UNIT_ASSERT_EQUAL( - CompressionFromAcceptEncodingHeader("br"), - ECompression::UNKNOWN); - - UNIT_ASSERT_EQUAL( - CompressionFromAcceptEncodingHeader("identity"), - ECompression::IDENTITY); - - UNIT_ASSERT_EQUAL( - CompressionFromAcceptEncodingHeader("zlib"), - ECompression::ZLIB); - - UNIT_ASSERT_EQUAL( - CompressionFromAcceptEncodingHeader("lz4"), - ECompression::LZ4); - - UNIT_ASSERT_EQUAL( - CompressionFromAcceptEncodingHeader("zstd"), - ECompression::ZSTD); - - UNIT_ASSERT_EQUAL( - CompressionFromAcceptEncodingHeader("zstd, zlib"), - ECompression::ZSTD); - - UNIT_ASSERT_EQUAL( - CompressionFromAcceptEncodingHeader(" ,, , zstd , zlib"), - ECompression::ZSTD); - - UNIT_ASSERT_EQUAL( - CompressionFromAcceptEncodingHeader("br, deflate,lz4, zlib"), - ECompression::LZ4); - } - + UNIT_ASSERT_EQUAL( + CompressionFromAcceptEncodingHeader(""), + ECompression::UNKNOWN); + + UNIT_ASSERT_EQUAL( + CompressionFromAcceptEncodingHeader("br"), + ECompression::UNKNOWN); + + UNIT_ASSERT_EQUAL( + CompressionFromAcceptEncodingHeader("identity"), + ECompression::IDENTITY); + + UNIT_ASSERT_EQUAL( + CompressionFromAcceptEncodingHeader("zlib"), + ECompression::ZLIB); + + UNIT_ASSERT_EQUAL( + CompressionFromAcceptEncodingHeader("lz4"), + ECompression::LZ4); + + UNIT_ASSERT_EQUAL( + CompressionFromAcceptEncodingHeader("zstd"), + ECompression::ZSTD); + + UNIT_ASSERT_EQUAL( + CompressionFromAcceptEncodingHeader("zstd, zlib"), + ECompression::ZSTD); + + UNIT_ASSERT_EQUAL( + CompressionFromAcceptEncodingHeader(" ,, , zstd , zlib"), + ECompression::ZSTD); + + UNIT_ASSERT_EQUAL( + CompressionFromAcceptEncodingHeader("br, deflate,lz4, zlib"), + ECompression::LZ4); + } + Y_UNIT_TEST(CompressionToStrFromStr) { - const std::array<ECompression, 5> algs = {{ - ECompression::UNKNOWN, - ECompression::IDENTITY, - ECompression::ZLIB, - ECompression::LZ4, - ECompression::ZSTD, - }}; - - for (ECompression a : algs) { - TString str = (TStringBuilder() << a); - ECompression b = FromString<ECompression>(str); - UNIT_ASSERT_EQUAL(a, b); - } - } -} + const std::array<ECompression, 5> algs = {{ + ECompression::UNKNOWN, + ECompression::IDENTITY, + ECompression::ZLIB, + ECompression::LZ4, + ECompression::ZSTD, + }}; + + for (ECompression a : algs) { + TString str = (TStringBuilder() << a); + ECompression b = FromString<ECompression>(str); + UNIT_ASSERT_EQUAL(a, b); + } + } +} diff --git a/library/cpp/monlib/encode/fuzz/ya.make b/library/cpp/monlib/encode/fuzz/ya.make index 44b76eee49a..d9ca172bae4 100644 --- a/library/cpp/monlib/encode/fuzz/ya.make +++ b/library/cpp/monlib/encode/fuzz/ya.make @@ -1,5 +1,5 @@ RECURSE_ROOT_RELATIVE( - library/cpp/monlib/encode/json/fuzz - library/cpp/monlib/encode/prometheus/fuzz - library/cpp/monlib/encode/spack/fuzz + library/cpp/monlib/encode/json/fuzz + library/cpp/monlib/encode/prometheus/fuzz + library/cpp/monlib/encode/spack/fuzz ) diff --git a/library/cpp/monlib/encode/json/fuzz/main.cpp b/library/cpp/monlib/encode/json/fuzz/main.cpp index 65f98d8a8b4..4f40310e068 100644 --- a/library/cpp/monlib/encode/json/fuzz/main.cpp +++ b/library/cpp/monlib/encode/json/fuzz/main.cpp @@ -1,5 +1,5 @@ -#include <library/cpp/monlib/encode/json/json.h> -#include <library/cpp/monlib/encode/fake/fake.h> +#include <library/cpp/monlib/encode/json/json.h> +#include <library/cpp/monlib/encode/fake/fake.h> #include <util/generic/strbuf.h> diff --git a/library/cpp/monlib/encode/json/fuzz/ya.make b/library/cpp/monlib/encode/json/fuzz/ya.make index 8ff5c6f9f97..75baa777168 100644 --- a/library/cpp/monlib/encode/json/fuzz/ya.make +++ b/library/cpp/monlib/encode/json/fuzz/ya.make @@ -6,8 +6,8 @@ OWNER( ) PEERDIR( - library/cpp/monlib/encode/json - library/cpp/monlib/encode/fake + library/cpp/monlib/encode/json + library/cpp/monlib/encode/fake ) SIZE(MEDIUM) diff --git a/library/cpp/monlib/encode/json/json.h b/library/cpp/monlib/encode/json/json.h index 2f0c8b0e1d2..21530f20c37 100644 --- a/library/cpp/monlib/encode/json/json.h +++ b/library/cpp/monlib/encode/json/json.h @@ -1,20 +1,20 @@ -#pragma once - -#include <library/cpp/monlib/encode/encoder.h> -#include <library/cpp/monlib/encode/format.h> - +#pragma once -class IOutputStream; - -namespace NMonitoring { +#include <library/cpp/monlib/encode/encoder.h> +#include <library/cpp/monlib/encode/format.h> + + +class IOutputStream; + +namespace NMonitoring { class TJsonDecodeError: public yexception { }; - IMetricEncoderPtr EncoderJson(IOutputStream* out, int indentation = 0); - + IMetricEncoderPtr EncoderJson(IOutputStream* out, int indentation = 0); + /// Buffered encoder will merge series with same labels into one. - IMetricEncoderPtr BufferedEncoderJson(IOutputStream* out, int indentation = 0); + IMetricEncoderPtr BufferedEncoderJson(IOutputStream* out, int indentation = 0); IMetricEncoderPtr EncoderCloudJson(IOutputStream* out, int indentation = 0, @@ -25,5 +25,5 @@ namespace NMonitoring { TStringBuf metricNameLabel = "name"); void DecodeJson(TStringBuf data, IMetricConsumer* c, TStringBuf metricNameLabel = "name"); - -} + +} diff --git a/library/cpp/monlib/encode/json/json_decoder.cpp b/library/cpp/monlib/encode/json/json_decoder.cpp index c8824e1ccc0..d44ff5fd286 100644 --- a/library/cpp/monlib/encode/json/json_decoder.cpp +++ b/library/cpp/monlib/encode/json/json_decoder.cpp @@ -1,24 +1,24 @@ -#include "json.h" -#include "typed_point.h" - +#include "json.h" +#include "typed_point.h" + #include <library/cpp/monlib/exception/exception.h> -#include <library/cpp/monlib/metrics/labels.h> -#include <library/cpp/monlib/metrics/metric_value.h> - +#include <library/cpp/monlib/metrics/labels.h> +#include <library/cpp/monlib/metrics/metric_value.h> + #include <library/cpp/json/json_reader.h> - -#include <util/datetime/base.h> -#include <util/string/cast.h> - -#include <limits> - -namespace NMonitoring { + +#include <util/datetime/base.h> +#include <util/string/cast.h> + +#include <limits> + +namespace NMonitoring { #define DECODE_ENSURE(COND, ...) MONLIB_ENSURE_EX(COND, TJsonDecodeError() << __VA_ARGS__) namespace { - + /////////////////////////////////////////////////////////////////////// // THistogramBuilder /////////////////////////////////////////////////////////////////////// @@ -32,11 +32,11 @@ public: } Bounds_.push_back(bound); } - + void AddValue(TBucketValue value) { Values_.push_back(value); } - + void AddInf(TBucketValue value) { InfPresented_ = true; InfValue_ = value; @@ -53,14 +53,14 @@ public: Bounds_.clear(); Values_.clear(); InfPresented_ = false; - + return snapshot; } - + bool Empty() const noexcept { return Bounds_.empty() && Values_.empty(); } - + void Clear() { Bounds_.clear(); Values_.clear(); @@ -69,7 +69,7 @@ public: private: TBucketBounds Bounds_; TBucketValues Values_; - + bool InfPresented_ = false; TBucketValue InfValue_; }; @@ -182,7 +182,7 @@ std::pair<double, bool> ParseSpecDouble(TStringBuf string) { return {0, false}; } } - + /////////////////////////////////////////////////////////////////////// // TMetricCollector /////////////////////////////////////////////////////////////////////// @@ -194,10 +194,10 @@ struct TMetricCollector { TLogHistogramBuilder LogHistBuilder; TTypedPoint LastPoint; TVector<TTypedPoint> TimeSeries; - + bool SeenTsOrValue = false; bool SeenTimeseries = false; - + void Clear() { Type = EMetricType::UNKNOWN; Labels.Clear(); @@ -209,20 +209,20 @@ struct TMetricCollector { SummaryBuilder.Clear(); LogHistBuilder.Clear(); } - + void AddLabel(const TLabel& label) { Labels.Add(label.Name(), label.Value()); } - + void SetLastTime(TInstant time) { LastPoint.SetTime(time); } - + template <typename T> void SetLastValue(T value) { LastPoint.SetValue(value); } - + void SaveLastPoint() { DECODE_ENSURE(LastPoint.GetTime() != TInstant::Zero(), "cannot add point without or zero timestamp"); @@ -248,21 +248,21 @@ struct TMetricCollector { } else { for (const auto& p: TimeSeries) { consumer(p.GetTime(), p.GetValueType(), p.GetValue()); - } + } } } }; - + struct TCommonParts { TInstant CommonTime; TLabels CommonLabels; }; - + class IHaltableMetricConsumer: public IMetricConsumer { public: virtual bool NeedToStop() const = 0; }; - + // TODO(ivanzhukov@): check all states for cases when a json document is invalid // e.g. "metrics" or "commonLabels" keys are specified multiple times class TCommonPartsCollector: public IHaltableMetricConsumer { @@ -433,11 +433,11 @@ class TDecoderJson final: public NJson::TJsonCallbacks { struct TState { enum EState { ROOT_OBJECT = 0x01, - + COMMON_LABELS, COMMON_TS, METRICS_ARRAY, - + METRIC_OBJECT, METRIC_NAME, METRIC_LABELS, @@ -462,21 +462,21 @@ class TDecoderJson final: public NJson::TJsonCallbacks { METRIC_LOG_HIST_START_POWER, METRIC_LOG_HIST_BUCKETS, }; - + constexpr EState Current() const noexcept { return static_cast<EState>(State_ & 0xFF); } - + void ToNext(EState state) noexcept { constexpr auto bitSize = 8 * sizeof(ui8); State_ = (State_ << bitSize) | static_cast<ui8>(state); } - + void ToPrev() noexcept { constexpr auto bitSize = 8 * sizeof(ui8); State_ = State_ >> bitSize; } - + private: ui64 State_ = static_cast<ui64>(ROOT_OBJECT); }; @@ -522,16 +522,16 @@ if (Y_UNLIKELY(!(CONDITION))) { \ LastMetric_.SetLastValue(static_cast<i64>(value)); State_.ToPrev(); break; - + case TState::METRIC_HIST_BOUNDS: LastMetric_.HistogramBuilder.AddBound(static_cast<double>(value)); break; - + case TState::METRIC_HIST_BUCKETS: PARSE_ENSURE(value >= 0 && static_cast<ui64>(value) <= Max<TBucketValues::value_type>(), "value is out of bounds " << value); LastMetric_.HistogramBuilder.AddValue(value); break; - + case TState::METRIC_HIST_INF: PARSE_ENSURE(value >= 0, "unexpected negative number in histogram inf: " << value); LastMetric_.HistogramBuilder.AddInf(value); @@ -584,7 +584,7 @@ if (Y_UNLIKELY(!(CONDITION))) { \ } return true; } - + bool OnUInteger(unsigned long long value) override { switch (State_.Current()) { case TState::COMMON_TS: @@ -597,32 +597,32 @@ if (Y_UNLIKELY(!(CONDITION))) { \ } break; - + case TState::METRIC_TS: LastMetric_.SetLastTime(TInstant::Seconds(value)); State_.ToPrev(); break; - + case TState::METRIC_VALUE: PARSE_ENSURE(value <= Max<ui64>(), "Metric value is out of bounds: " << value); LastMetric_.SetLastValue(static_cast<ui64>(value)); State_.ToPrev(); break; - + case TState::METRIC_HIST_BOUNDS: LastMetric_.HistogramBuilder.AddBound(static_cast<double>(value)); break; - + case TState::METRIC_HIST_BUCKETS: PARSE_ENSURE(value <= Max<TBucketValues::value_type>(), "Histogram bucket value is out of bounds: " << value); LastMetric_.HistogramBuilder.AddValue(value); break; - + case TState::METRIC_HIST_INF: LastMetric_.HistogramBuilder.AddInf(value); State_.ToPrev(); break; - + case TState::METRIC_DSUMMARY_COUNT: LastMetric_.SummaryBuilder.SetCount(value); State_.ToPrev(); @@ -669,18 +669,18 @@ if (Y_UNLIKELY(!(CONDITION))) { \ } return true; } - + bool OnDouble(double value) override { switch (State_.Current()) { case TState::METRIC_VALUE: LastMetric_.SetLastValue(value); State_.ToPrev(); break; - + case TState::METRIC_HIST_BOUNDS: LastMetric_.HistogramBuilder.AddBound(value); break; - + case TState::METRIC_DSUMMARY_SUM: LastMetric_.SummaryBuilder.SetSum(value); State_.ToPrev(); @@ -752,22 +752,22 @@ if (Y_UNLIKELY(!(CONDITION))) { \ LastMetric_.SetLastValue(doubleValue); } else { return false; - } + } State_.ToPrev(); break; - + case TState::METRIC_TYPE: LastMetric_.Type = MetricTypeFromStr(value); State_.ToPrev(); break; - + case TState::METRIC_MODE: if (value == TStringBuf("deriv")) { LastMetric_.Type = EMetricType::RATE; } State_.ToPrev(); break; - + case TState::METRIC_DSUMMARY_SUM: if (auto [doubleValue, ok] = ParseSpecDouble(value); ok) { LastMetric_.SummaryBuilder.SetSum(doubleValue); @@ -776,7 +776,7 @@ if (Y_UNLIKELY(!(CONDITION))) { \ } State_.ToPrev(); break; - + case TState::METRIC_DSUMMARY_MIN: if (auto [doubleValue, ok] = ParseSpecDouble(value); ok) { LastMetric_.SummaryBuilder.SetMin(doubleValue); @@ -785,7 +785,7 @@ if (Y_UNLIKELY(!(CONDITION))) { \ } State_.ToPrev(); break; - + case TState::METRIC_DSUMMARY_MAX: if (auto [doubleValue, ok] = ParseSpecDouble(value); ok) { LastMetric_.SummaryBuilder.SetMax(doubleValue); @@ -794,7 +794,7 @@ if (Y_UNLIKELY(!(CONDITION))) { \ } State_.ToPrev(); break; - + case TState::METRIC_DSUMMARY_LAST: if (auto [doubleValue, ok] = ParseSpecDouble(value); ok) { LastMetric_.SummaryBuilder.SetLast(doubleValue); @@ -803,11 +803,11 @@ if (Y_UNLIKELY(!(CONDITION))) { \ } State_.ToPrev(); break; - + default: return false; } - + return true; } @@ -877,7 +877,7 @@ if (Y_UNLIKELY(!(CONDITION))) { \ State_.ToNext(TState::METRIC_DSUMMARY); } else if (key == TStringBuf("log_hist")) { State_.ToNext(TState::METRIC_LOG_HIST); - } + } break; case TState::METRIC_HIST: @@ -889,7 +889,7 @@ if (Y_UNLIKELY(!(CONDITION))) { \ State_.ToNext(TState::METRIC_HIST_INF); } break; - + case TState::METRIC_LOG_HIST: if (key == TStringBuf("base")) { State_.ToNext(TState::METRIC_LOG_HIST_BASE); @@ -901,7 +901,7 @@ if (Y_UNLIKELY(!(CONDITION))) { \ State_.ToNext(TState::METRIC_LOG_HIST_BUCKETS); } break; - + case TState::METRIC_DSUMMARY: if (key == TStringBuf("sum")) { State_.ToNext(TState::METRIC_DSUMMARY_SUM); @@ -914,14 +914,14 @@ if (Y_UNLIKELY(!(CONDITION))) { \ } else if (key == TStringBuf("count")) { State_.ToNext(TState::METRIC_DSUMMARY_COUNT); } - + break; - - + + default: return false; } - + return true; } @@ -945,17 +945,17 @@ if (Y_UNLIKELY(!(CONDITION))) { \ } return true; } - + bool OnCloseMap() override { switch (State_.Current()) { case TState::ROOT_OBJECT: MetricConsumer_->OnStreamEnd(); break; - + case TState::METRIC_LABELS: State_.ToPrev(); break; - + case TState::COMMON_LABELS: MetricConsumer_->OnLabelsEnd(); State_.ToPrev(); @@ -966,22 +966,22 @@ if (Y_UNLIKELY(!(CONDITION))) { \ } break; - + case TState::METRIC_OBJECT: ConsumeMetric(); State_.ToPrev(); break; - + case TState::METRIC_TIMESERIES: LastMetric_.SaveLastPoint(); break; - + case TState::METRIC_HIST: case TState::METRIC_DSUMMARY: case TState::METRIC_LOG_HIST: State_.ToPrev(); break; - + default: break; } @@ -999,7 +999,7 @@ if (Y_UNLIKELY(!(CONDITION))) { \ "unexpected array begin"); return true; } - + bool OnCloseArray() override { switch (State_.Current()) { case TState::METRICS_ARRAY: @@ -1009,13 +1009,13 @@ if (Y_UNLIKELY(!(CONDITION))) { \ case TState::METRIC_LOG_HIST_BUCKETS: State_.ToPrev(); break; - + default: return false; } return true; } - + void OnError(size_t off, TStringBuf reason) override { if (IsIntentionallyHalted_) { return; @@ -1023,13 +1023,13 @@ if (Y_UNLIKELY(!(CONDITION))) { \ size_t snippetBeg = (off < 20) ? 0 : (off - 20); TStringBuf snippet = Data_.SubStr(snippetBeg, 40); - + throw TJsonDecodeError() << "cannot parse JSON, error at: " << off << ", reason: " << (ErrorMsg_.empty() ? reason : TStringBuf{ErrorMsg_}) << "\nsnippet: ..." << snippet << "..."; } - + bool OnEnd() override { return true; } @@ -1041,21 +1041,21 @@ if (Y_UNLIKELY(!(CONDITION))) { \ LastMetric_.Type = EMetricType::GAUGE; } else { LastMetric_.Type = EMetricType::HIST; - } + } } - + // (1) begin metric MetricConsumer_->OnMetricBegin(LastMetric_.Type); - + // (2) labels if (!LastMetric_.Labels.empty()) { MetricConsumer_->OnLabelsBegin(); for (auto&& label : LastMetric_.Labels) { MetricConsumer_->OnLabel(label.Name(), label.Value()); - } + } MetricConsumer_->OnLabelsEnd(); } - + // (3) values switch (LastMetric_.Type) { case EMetricType::GAUGE: @@ -1069,14 +1069,14 @@ if (Y_UNLIKELY(!(CONDITION))) { \ MetricConsumer_->OnInt64(time, value.AsInt64(valueType)); }); break; - + case EMetricType::COUNTER: case EMetricType::RATE: LastMetric_.Consume([this](TInstant time, EMetricValueType valueType, TMetricValue value) { MetricConsumer_->OnUint64(time, value.AsUint64(valueType)); }); break; - + case EMetricType::HIST: case EMetricType::HIST_RATE: if (LastMetric_.TimeSeries.empty()) { @@ -1087,10 +1087,10 @@ if (Y_UNLIKELY(!(CONDITION))) { \ for (const auto& p : LastMetric_.TimeSeries) { DECODE_ENSURE(p.GetValueType() == EMetricValueType::HISTOGRAM, "Value is not a histogram"); MetricConsumer_->OnHistogram(p.GetTime(), p.GetValue().AsHistogram()); - } - } + } + } break; - + case EMetricType::DSUMMARY: if (LastMetric_.TimeSeries.empty()) { auto time = LastMetric_.LastPoint.GetTime(); @@ -1103,7 +1103,7 @@ if (Y_UNLIKELY(!(CONDITION))) { \ } } break; - + case EMetricType::LOGHIST: if (LastMetric_.TimeSeries.empty()) { auto time = LastMetric_.LastPoint.GetTime(); @@ -1113,19 +1113,19 @@ if (Y_UNLIKELY(!(CONDITION))) { \ for (const auto& p : LastMetric_.TimeSeries) { DECODE_ENSURE(p.GetValueType() == EMetricValueType::LOGHISTOGRAM, "Value is not a log_histogram"); MetricConsumer_->OnLogHistogram(p.GetTime(), p.GetValue().AsLogHistogram()); - } - } + } + } break; - + case EMetricType::UNKNOWN: // TODO: output metric labels ythrow yexception() << "unknown metric type"; } - + // (4) end metric MetricConsumer_->OnMetricEnd(); } - + private: TStringBuf Data_; IHaltableMetricConsumer* MetricConsumer_; @@ -1136,9 +1136,9 @@ private: TString ErrorMsg_; bool IsIntentionallyHalted_{false}; }; - + } // namespace - + void DecodeJson(TStringBuf data, IMetricConsumer* c, TStringBuf metricNameLabel) { TCommonPartsCollector commonPartsCollector; { @@ -1159,4 +1159,4 @@ void DecodeJson(TStringBuf data, IMetricConsumer* c, TStringBuf metricNameLabel) #undef DECODE_ENSURE -} +} diff --git a/library/cpp/monlib/encode/json/json_encoder.cpp b/library/cpp/monlib/encode/json/json_encoder.cpp index 639177b5d11..20d2bb6283f 100644 --- a/library/cpp/monlib/encode/json/json_encoder.cpp +++ b/library/cpp/monlib/encode/json/json_encoder.cpp @@ -1,39 +1,39 @@ -#include "json.h" -#include "typed_point.h" - -#include <library/cpp/monlib/encode/buffered/buffered_encoder_base.h> -#include <library/cpp/monlib/encode/encoder_state.h> -#include <library/cpp/monlib/metrics/metric.h> -#include <library/cpp/monlib/metrics/metric_value.h> -#include <library/cpp/monlib/metrics/labels.h> - +#include "json.h" +#include "typed_point.h" + +#include <library/cpp/monlib/encode/buffered/buffered_encoder_base.h> +#include <library/cpp/monlib/encode/encoder_state.h> +#include <library/cpp/monlib/metrics/metric.h> +#include <library/cpp/monlib/metrics/metric_value.h> +#include <library/cpp/monlib/metrics/labels.h> + #include <library/cpp/json/writer/json.h> - + #include <util/charset/utf8.h> #include <util/generic/algorithm.h> -namespace NMonitoring { - namespace { +namespace NMonitoring { + namespace { enum class EJsonStyle { Solomon, Cloud }; - /////////////////////////////////////////////////////////////////////// - // TJsonWriter - /////////////////////////////////////////////////////////////////////// + /////////////////////////////////////////////////////////////////////// + // TJsonWriter + /////////////////////////////////////////////////////////////////////// class TJsonWriter { - public: + public: TJsonWriter(IOutputStream* out, int indentation, EJsonStyle style, TStringBuf metricNameLabel) - : Buf_(NJsonWriter::HEM_UNSAFE, out) + : Buf_(NJsonWriter::HEM_UNSAFE, out) , Style_(style) , MetricNameLabel_(metricNameLabel) , CurrentMetricName_() - { - Buf_.SetIndentSpaces(indentation); - Buf_.SetWriteNanAsString(); - } - + { + Buf_.SetIndentSpaces(indentation); + Buf_.SetWriteNanAsString(); + } + void WriteTime(TInstant time) { if (time != TInstant::Zero()) { Buf_.WriteKey(TStringBuf("ts")); @@ -50,49 +50,49 @@ namespace NMonitoring { Buf_.WriteDouble(value); } - void WriteValue(i64 value) { + void WriteValue(i64 value) { Buf_.WriteKey(TStringBuf("value")); - Buf_.WriteLongLong(value); - } - + Buf_.WriteLongLong(value); + } + void WriteValue(ui64 value) { Buf_.WriteKey(TStringBuf("value")); Buf_.WriteULongLong(value); } - void WriteValue(IHistogramSnapshot* s) { + void WriteValue(IHistogramSnapshot* s) { Y_ENSURE(Style_ == EJsonStyle::Solomon); Buf_.WriteKey(TStringBuf("hist")); - Buf_.BeginObject(); - if (ui32 count = s->Count()) { - bool hasInf = (s->UpperBound(count - 1) == Max<double>()); - if (hasInf) { - count--; - } - + Buf_.BeginObject(); + if (ui32 count = s->Count()) { + bool hasInf = (s->UpperBound(count - 1) == Max<double>()); + if (hasInf) { + count--; + } + Buf_.WriteKey(TStringBuf("bounds")); - Buf_.BeginList(); - for (ui32 i = 0; i < count; i++) { - Buf_.WriteDouble(s->UpperBound(i)); - } - Buf_.EndList(); - + Buf_.BeginList(); + for (ui32 i = 0; i < count; i++) { + Buf_.WriteDouble(s->UpperBound(i)); + } + Buf_.EndList(); + Buf_.WriteKey(TStringBuf("buckets")); - Buf_.BeginList(); - for (ui32 i = 0; i < count; i++) { - Buf_.WriteULongLong(s->Value(i)); - } - Buf_.EndList(); - - if (hasInf) { + Buf_.BeginList(); + for (ui32 i = 0; i < count; i++) { + Buf_.WriteULongLong(s->Value(i)); + } + Buf_.EndList(); + + if (hasInf) { Buf_.WriteKey(TStringBuf("inf")); - Buf_.WriteULongLong(s->Value(count)); - } - } - Buf_.EndObject(); - } - + Buf_.WriteULongLong(s->Value(count)); + } + } + Buf_.EndObject(); + } + void WriteValue(ISummaryDoubleSnapshot* s) { Y_ENSURE(Style_ == EJsonStyle::Solomon); @@ -142,25 +142,25 @@ namespace NMonitoring { Buf_.EndObject(); } - void WriteValue(EMetricValueType type, TMetricValue value) { + void WriteValue(EMetricValueType type, TMetricValue value) { switch (type) { - case EMetricValueType::DOUBLE: - WriteValue(value.AsDouble()); + case EMetricValueType::DOUBLE: + WriteValue(value.AsDouble()); + break; + + case EMetricValueType::INT64: + WriteValue(value.AsInt64()); + break; + + case EMetricValueType::UINT64: + WriteValue(value.AsUint64()); break; - case EMetricValueType::INT64: - WriteValue(value.AsInt64()); - break; - - case EMetricValueType::UINT64: - WriteValue(value.AsUint64()); + case EMetricValueType::HISTOGRAM: + WriteValue(value.AsHistogram()); break; - case EMetricValueType::HISTOGRAM: - WriteValue(value.AsHistogram()); - break; - - case EMetricValueType::SUMMARY: + case EMetricValueType::SUMMARY: WriteValue(value.AsSummaryDouble()); break; @@ -168,7 +168,7 @@ namespace NMonitoring { WriteValue(value.AsLogHistogram()); break; - case EMetricValueType::UNKNOWN: + case EMetricValueType::UNKNOWN: ythrow yexception() << "unknown metric value type"; } } @@ -229,10 +229,10 @@ namespace NMonitoring { TString CurrentMetricName_; }; - /////////////////////////////////////////////////////////////////////// - // TEncoderJson - /////////////////////////////////////////////////////////////////////// - class TEncoderJson final: public IMetricEncoder, public TJsonWriter { + /////////////////////////////////////////////////////////////////////// + // TEncoderJson + /////////////////////////////////////////////////////////////////////// + class TEncoderJson final: public IMetricEncoder, public TJsonWriter { public: TEncoderJson(IOutputStream* out, int indentation, EJsonStyle style, TStringBuf metricNameLabel) : TJsonWriter{out, indentation, style, metricNameLabel} @@ -240,202 +240,202 @@ namespace NMonitoring { } ~TEncoderJson() override { - Close(); - } - - private: - void OnStreamBegin() override { + Close(); + } + + private: + void OnStreamBegin() override { State_.Expect(TEncoderState::EState::ROOT); - Buf_.BeginObject(); - } - - void OnStreamEnd() override { + Buf_.BeginObject(); + } + + void OnStreamEnd() override { State_.Expect(TEncoderState::EState::ROOT); - if (!Buf_.KeyExpected()) { - // not closed metrics array - Buf_.EndList(); - } - Buf_.EndObject(); - } - - void OnCommonTime(TInstant time) override { + if (!Buf_.KeyExpected()) { + // not closed metrics array + Buf_.EndList(); + } + Buf_.EndObject(); + } + + void OnCommonTime(TInstant time) override { State_.Expect(TEncoderState::EState::ROOT); - WriteTime(time); - } - - void OnMetricBegin(EMetricType type) override { - State_.Switch(TEncoderState::EState::ROOT, TEncoderState::EState::METRIC); - if (Buf_.KeyExpected()) { - // first metric, so open metrics array + WriteTime(time); + } + + void OnMetricBegin(EMetricType type) override { + State_.Switch(TEncoderState::EState::ROOT, TEncoderState::EState::METRIC); + if (Buf_.KeyExpected()) { + // first metric, so open metrics array Buf_.WriteKey(TStringBuf(Style_ == EJsonStyle::Solomon ? "sensors" : "metrics")); - Buf_.BeginList(); - } - Buf_.BeginObject(); + Buf_.BeginList(); + } + Buf_.BeginObject(); WriteMetricType(type); - } - - void OnMetricEnd() override { - State_.Switch(TEncoderState::EState::METRIC, TEncoderState::EState::ROOT); - if (!Buf_.KeyExpected()) { - // not closed timeseries array - Buf_.EndList(); - } - - if (!TimeSeries_ && LastPoint_.HasValue()) { - // we have seen only one point between OnMetricBegin() and - // OnMetricEnd() calls - WriteTime(LastPoint_.GetTime()); - WriteValue(LastPoint_.GetValueType(), LastPoint_.GetValue()); - } - Buf_.EndObject(); - - LastPoint_ = {}; - TimeSeries_ = false; - } - - void OnLabelsBegin() override { - if (!Buf_.KeyExpected()) { - // not closed metrics or timeseries array if labels go after values - Buf_.EndList(); - } + } + + void OnMetricEnd() override { + State_.Switch(TEncoderState::EState::METRIC, TEncoderState::EState::ROOT); + if (!Buf_.KeyExpected()) { + // not closed timeseries array + Buf_.EndList(); + } + + if (!TimeSeries_ && LastPoint_.HasValue()) { + // we have seen only one point between OnMetricBegin() and + // OnMetricEnd() calls + WriteTime(LastPoint_.GetTime()); + WriteValue(LastPoint_.GetValueType(), LastPoint_.GetValue()); + } + Buf_.EndObject(); + + LastPoint_ = {}; + TimeSeries_ = false; + } + + void OnLabelsBegin() override { + if (!Buf_.KeyExpected()) { + // not closed metrics or timeseries array if labels go after values + Buf_.EndList(); + } if (State_ == TEncoderState::EState::ROOT) { State_ = TEncoderState::EState::COMMON_LABELS; Buf_.WriteKey(TStringBuf(Style_ == EJsonStyle::Solomon ? "commonLabels" : "labels")); - } else if (State_ == TEncoderState::EState::METRIC) { - State_ = TEncoderState::EState::METRIC_LABELS; + } else if (State_ == TEncoderState::EState::METRIC) { + State_ = TEncoderState::EState::METRIC_LABELS; Buf_.WriteKey(TStringBuf("labels")); - } else { - State_.ThrowInvalid("expected METRIC or ROOT"); - } - Buf_.BeginObject(); + } else { + State_.ThrowInvalid("expected METRIC or ROOT"); + } + Buf_.BeginObject(); EmptyLabels_ = true; - } - - void OnLabelsEnd() override { - if (State_ == TEncoderState::EState::METRIC_LABELS) { - State_ = TEncoderState::EState::METRIC; + } + + void OnLabelsEnd() override { + if (State_ == TEncoderState::EState::METRIC_LABELS) { + State_ = TEncoderState::EState::METRIC; } else if (State_ == TEncoderState::EState::COMMON_LABELS) { State_ = TEncoderState::EState::ROOT; - } else { - State_.ThrowInvalid("expected LABELS or COMMON_LABELS"); - } + } else { + State_.ThrowInvalid("expected LABELS or COMMON_LABELS"); + } Y_ENSURE(!EmptyLabels_, "Labels cannot be empty"); - Buf_.EndObject(); + Buf_.EndObject(); if (State_ == TEncoderState::EState::METRIC) { WriteName(); } - } - - void OnLabel(TStringBuf name, TStringBuf value) override { - if (State_ == TEncoderState::EState::METRIC_LABELS || State_ == TEncoderState::EState::COMMON_LABELS) { + } + + void OnLabel(TStringBuf name, TStringBuf value) override { + if (State_ == TEncoderState::EState::METRIC_LABELS || State_ == TEncoderState::EState::COMMON_LABELS) { WriteLabel(name, value); - } else { - State_.ThrowInvalid("expected LABELS or COMMON_LABELS"); - } + } else { + State_.ThrowInvalid("expected LABELS or COMMON_LABELS"); + } EmptyLabels_ = false; - } - - void OnDouble(TInstant time, double value) override { - State_.Expect(TEncoderState::EState::METRIC); - Write<double>(time, value); - } - - void OnInt64(TInstant time, i64 value) override { - State_.Expect(TEncoderState::EState::METRIC); - Write<i64>(time, value); - } - - void OnUint64(TInstant time, ui64 value) override { - State_.Expect(TEncoderState::EState::METRIC); - Write<ui64>(time, value); - } - - void OnHistogram(TInstant time, IHistogramSnapshotPtr snapshot) override { - State_.Expect(TEncoderState::EState::METRIC); - Write<IHistogramSnapshot*>(time, snapshot.Get()); - } - + } + + void OnDouble(TInstant time, double value) override { + State_.Expect(TEncoderState::EState::METRIC); + Write<double>(time, value); + } + + void OnInt64(TInstant time, i64 value) override { + State_.Expect(TEncoderState::EState::METRIC); + Write<i64>(time, value); + } + + void OnUint64(TInstant time, ui64 value) override { + State_.Expect(TEncoderState::EState::METRIC); + Write<ui64>(time, value); + } + + void OnHistogram(TInstant time, IHistogramSnapshotPtr snapshot) override { + State_.Expect(TEncoderState::EState::METRIC); + Write<IHistogramSnapshot*>(time, snapshot.Get()); + } + void OnSummaryDouble(TInstant time, ISummaryDoubleSnapshotPtr snapshot) override { - State_.Expect(TEncoderState::EState::METRIC); + State_.Expect(TEncoderState::EState::METRIC); Write<ISummaryDoubleSnapshot*>(time, snapshot.Get()); } void OnLogHistogram(TInstant time, TLogHistogramSnapshotPtr snapshot) override { - State_.Expect(TEncoderState::EState::METRIC); + State_.Expect(TEncoderState::EState::METRIC); Write<TLogHistogramSnapshot*>(time, snapshot.Get()); } - template <typename T> - void Write(TInstant time, T value) { - State_.Expect(TEncoderState::EState::METRIC); - - if (!LastPoint_.HasValue()) { - LastPoint_ = {time, value}; - } else { - // second point - // TODO: output types - Y_ENSURE(LastPoint_.GetValueType() == TValueType<T>::Type, - "mixed metric value types in one metric"); - - if (!TimeSeries_) { + template <typename T> + void Write(TInstant time, T value) { + State_.Expect(TEncoderState::EState::METRIC); + + if (!LastPoint_.HasValue()) { + LastPoint_ = {time, value}; + } else { + // second point + // TODO: output types + Y_ENSURE(LastPoint_.GetValueType() == TValueType<T>::Type, + "mixed metric value types in one metric"); + + if (!TimeSeries_) { Buf_.WriteKey(TStringBuf("timeseries")); - Buf_.BeginList(); - Buf_.BeginObject(); - Y_ENSURE(LastPoint_.GetTime() != TInstant::Zero(), + Buf_.BeginList(); + Buf_.BeginObject(); + Y_ENSURE(LastPoint_.GetTime() != TInstant::Zero(), "time cannot be empty or zero in a timeseries point"); - WriteTime(LastPoint_.GetTime()); - WriteValue(LastPoint_.GetValueType(), LastPoint_.GetValue()); - Buf_.EndObject(); - TimeSeries_ = true; - } - - if (TimeSeries_) { - Buf_.BeginObject(); + WriteTime(LastPoint_.GetTime()); + WriteValue(LastPoint_.GetValueType(), LastPoint_.GetValue()); + Buf_.EndObject(); + TimeSeries_ = true; + } + + if (TimeSeries_) { + Buf_.BeginObject(); Y_ENSURE(time != TInstant::Zero(), "time cannot be empty or zero in a timeseries point"); - - WriteTime(time); - WriteValue(value); - Buf_.EndObject(); - } - } - } - - void Close() override { - LastPoint_ = {}; - } - + + WriteTime(time); + WriteValue(value); + Buf_.EndObject(); + } + } + } + + void Close() override { + LastPoint_ = {}; + } + private: TEncoderState State_; - TTypedPoint LastPoint_; + TTypedPoint LastPoint_; bool TimeSeries_ = false; bool EmptyLabels_ = false; }; - /////////////////////////////////////////////////////////////////////// - // TBufferedJsonEncoder - /////////////////////////////////////////////////////////////////////// + /////////////////////////////////////////////////////////////////////// + // TBufferedJsonEncoder + /////////////////////////////////////////////////////////////////////// class TBufferedJsonEncoder : public TBufferedEncoderBase, public TJsonWriter { public: TBufferedJsonEncoder(IOutputStream* out, int indentation, EJsonStyle style, TStringBuf metricNameLabel) : TJsonWriter{out, indentation, style, metricNameLabel} { - MetricsMergingMode_ = EMetricsMergingMode::MERGE_METRICS; - } - + MetricsMergingMode_ = EMetricsMergingMode::MERGE_METRICS; + } + ~TBufferedJsonEncoder() override { Close(); - } - + } + void OnLabelsBegin() override { TBufferedEncoderBase::OnLabelsBegin(); EmptyLabels_ = true; } - void OnLabel(TStringBuf name, TStringBuf value) override { + void OnLabel(TStringBuf name, TStringBuf value) override { TBufferedEncoderBase::OnLabel(name, value); EmptyLabels_ = false; } @@ -456,36 +456,36 @@ namespace NMonitoring { } Closed_ = true; - + LabelValuesPool_.Build(); LabelNamesPool_.Build(); - + Buf_.BeginObject(); - + WriteTime(CommonTime_); if (CommonLabels_.size() > 0) { Buf_.WriteKey(TStringBuf(Style_ == EJsonStyle::Solomon ? "commonLabels": "labels")); WriteLabels(CommonLabels_, true); - } + } - if (Metrics_.size() > 0) { + if (Metrics_.size() > 0) { Buf_.WriteKey(TStringBuf(Style_ == EJsonStyle::Solomon ? "sensors" : "metrics")); - WriteMetrics(); + WriteMetrics(); } Buf_.EndObject(); - } - - private: - void WriteMetrics() { + } + + private: + void WriteMetrics() { Buf_.BeginList(); - for (auto&& metric : Metrics_) { - WriteMetric(metric); + for (auto&& metric : Metrics_) { + WriteMetric(metric); } Buf_.EndList(); } - void WriteMetric(TMetric& metric) { + void WriteMetric(TMetric& metric) { Buf_.BeginObject(); WriteMetricType(metric.MetricType); @@ -493,19 +493,19 @@ namespace NMonitoring { Buf_.WriteKey(TStringBuf("labels")); WriteLabels(metric.Labels, false); - metric.TimeSeries.SortByTs(); - if (metric.TimeSeries.Size() == 1) { - const auto& point = metric.TimeSeries[0]; - WriteTime(point.GetTime()); - WriteValue(metric.TimeSeries.GetValueType(), point.GetValue()); - } else if (metric.TimeSeries.Size() > 1) { + metric.TimeSeries.SortByTs(); + if (metric.TimeSeries.Size() == 1) { + const auto& point = metric.TimeSeries[0]; + WriteTime(point.GetTime()); + WriteValue(metric.TimeSeries.GetValueType(), point.GetValue()); + } else if (metric.TimeSeries.Size() > 1) { Buf_.WriteKey(TStringBuf("timeseries")); Buf_.BeginList(); - metric.TimeSeries.ForEach([this](TInstant time, EMetricValueType type, TMetricValue value) { + metric.TimeSeries.ForEach([this](TInstant time, EMetricValueType type, TMetricValue value) { Buf_.BeginObject(); // make gcc 6.1 happy https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61636 - this->WriteTime(time); - this->WriteValue(type, value); + this->WriteTime(time); + this->WriteValue(type, value); Buf_.EndObject(); }); @@ -535,14 +535,14 @@ namespace NMonitoring { private: bool Closed_{false}; bool EmptyLabels_ = false; - }; - } - - IMetricEncoderPtr EncoderJson(IOutputStream* out, int indentation) { + }; + } + + IMetricEncoderPtr EncoderJson(IOutputStream* out, int indentation) { return MakeHolder<TEncoderJson>(out, indentation, EJsonStyle::Solomon, ""); - } - - IMetricEncoderPtr BufferedEncoderJson(IOutputStream* out, int indentation) { + } + + IMetricEncoderPtr BufferedEncoderJson(IOutputStream* out, int indentation) { return MakeHolder<TBufferedJsonEncoder>(out, indentation, EJsonStyle::Solomon, ""); } @@ -553,4 +553,4 @@ namespace NMonitoring { IMetricEncoderPtr BufferedEncoderCloudJson(IOutputStream* out, int indentation, TStringBuf metricNameLabel) { return MakeHolder<TBufferedJsonEncoder>(out, indentation, EJsonStyle::Cloud, metricNameLabel); } -} +} diff --git a/library/cpp/monlib/encode/json/json_ut.cpp b/library/cpp/monlib/encode/json/json_ut.cpp index f80105795b8..09e79092890 100644 --- a/library/cpp/monlib/encode/json/json_ut.cpp +++ b/library/cpp/monlib/encode/json/json_ut.cpp @@ -1,19 +1,19 @@ -#include "json.h" - -#include <library/cpp/monlib/encode/protobuf/protobuf.h> -#include <library/cpp/monlib/metrics/labels.h> - +#include "json.h" + +#include <library/cpp/monlib/encode/protobuf/protobuf.h> +#include <library/cpp/monlib/metrics/labels.h> + #include <library/cpp/json/json_reader.h> #include <library/cpp/resource/resource.h> #include <library/cpp/testing/unittest/registar.h> - -#include <util/stream/str.h> + +#include <util/stream/str.h> #include <util/string/builder.h> - -#include <limits> - -using namespace NMonitoring; - + +#include <limits> + +using namespace NMonitoring; + namespace NMonitoring { bool operator<(const TLabel& lhs, const TLabel& rhs) { return lhs.Name() < rhs.Name() || @@ -134,7 +134,7 @@ namespace { Y_UNIT_TEST_SUITE(TJsonTest) { const TInstant now = TInstant::ParseIso8601Deprecated("2017-11-05T01:02:03Z"); - + Y_UNIT_TEST(Encode) { auto check = [](bool cloud, bool buffered, TStringBuf expectedResourceKey) { TString json; @@ -145,12 +145,12 @@ Y_UNIT_TEST_SUITE(TJsonTest) { e->OnStreamBegin(); { // common time e->OnCommonTime(TInstant::Seconds(1500000000)); - } + } { // common labels - e->OnLabelsBegin(); + e->OnLabelsBegin(); e->OnLabel("project", "solomon"); - e->OnLabelsEnd(); - } + e->OnLabelsEnd(); + } { // metric #1 e->OnMetricBegin(EMetricType::COUNTER); { @@ -161,7 +161,7 @@ Y_UNIT_TEST_SUITE(TJsonTest) { } e->OnUint64(now, 17); e->OnMetricEnd(); - } + } { // metric #2 e->OnMetricBegin(EMetricType::RATE); { @@ -172,7 +172,7 @@ Y_UNIT_TEST_SUITE(TJsonTest) { } e->OnUint64(now, 17); e->OnMetricEnd(); - } + } { // metric #3 e->OnMetricBegin(EMetricType::GAUGE); e->OnDouble(now, 3.14); @@ -251,7 +251,7 @@ Y_UNIT_TEST_SUITE(TJsonTest) { auto emit = [&](IMetricEncoder* encoder, EMetricType metricType) { encoder->OnStreamBegin(); - { + { encoder->OnMetricBegin(metricType); { encoder->OnLabelsBegin(); @@ -280,11 +280,11 @@ Y_UNIT_TEST_SUITE(TJsonTest) { } encoder->OnMetricEnd(); - } + } encoder->OnStreamEnd(); encoder->Close(); }; - + auto doTest = [&](bool buffered, EMetricType metricType) { TString json; TStringOutput out(json); @@ -301,107 +301,107 @@ Y_UNIT_TEST_SUITE(TJsonTest) { doTest(true, EMetricType::HIST); doTest(true, EMetricType::LOGHIST); doTest(true, EMetricType::DSUMMARY); - } - - Y_UNIT_TEST(MetricsWithDifferentLabelOrderGetMerged) { + } + + Y_UNIT_TEST(MetricsWithDifferentLabelOrderGetMerged) { TString json; TStringOutput out(json); auto e = BufferedEncoderJson(&out, 2); - + e->OnStreamBegin(); { - e->OnMetricBegin(EMetricType::RATE); + e->OnMetricBegin(EMetricType::RATE); { e->OnLabelsBegin(); - e->OnLabel("metric", "hello"); + e->OnLabel("metric", "hello"); e->OnLabel("label", "world"); e->OnLabelsEnd(); } e->OnUint64(TInstant::Zero(), 0); - e->OnMetricEnd(); - } + e->OnMetricEnd(); + } { - e->OnMetricBegin(EMetricType::RATE); + e->OnMetricBegin(EMetricType::RATE); { e->OnLabelsBegin(); e->OnLabel("label", "world"); - e->OnLabel("metric", "hello"); + e->OnLabel("metric", "hello"); e->OnLabelsEnd(); } e->OnUint64(TInstant::Zero(), 1); - e->OnMetricEnd(); + e->OnMetricEnd(); } e->OnStreamEnd(); e->Close(); json += "\n"; - + TString expectedJson = NResource::Find("/merged.json"); // we cannot be sure regarding the label order in the result, // so we'll have to parse the expected value and then compare it with actual - + NProto::TMultiSamplesList samples; - IMetricEncoderPtr d = EncoderProtobuf(&samples); + IMetricEncoderPtr d = EncoderProtobuf(&samples); DecodeJson(expectedJson, d.Get()); - + UNIT_ASSERT_VALUES_EQUAL(samples.SamplesSize(), 1); { const NProto::TMultiSample& s = samples.GetSamples(0); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::RATE); - AssertLabels(s, TLabels{{"metric", "hello"}, {"label", "world"}}); - + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::RATE); + AssertLabels(s, TLabels{{"metric", "hello"}, {"label", "world"}}); + UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); AssertPointEqual(s.GetPoints(0), TInstant::Zero(), ui64(1)); - } - } + } + } Y_UNIT_TEST(Decode1) { - NProto::TMultiSamplesList samples; - { - IMetricEncoderPtr e = EncoderProtobuf(&samples); - - TString testJson = NResource::Find("/expected.json"); - DecodeJson(testJson, e.Get()); - } - - UNIT_ASSERT_VALUES_EQUAL( - TInstant::MilliSeconds(samples.GetCommonTime()), - TInstant::Seconds(1500000000)); - - UNIT_ASSERT_VALUES_EQUAL(samples.CommonLabelsSize(), 1); - AssertLabelEqual(samples.GetCommonLabels(0), "project", "solomon"); - + NProto::TMultiSamplesList samples; + { + IMetricEncoderPtr e = EncoderProtobuf(&samples); + + TString testJson = NResource::Find("/expected.json"); + DecodeJson(testJson, e.Get()); + } + + UNIT_ASSERT_VALUES_EQUAL( + TInstant::MilliSeconds(samples.GetCommonTime()), + TInstant::Seconds(1500000000)); + + UNIT_ASSERT_VALUES_EQUAL(samples.CommonLabelsSize(), 1); + AssertLabelEqual(samples.GetCommonLabels(0), "project", "solomon"); + UNIT_ASSERT_VALUES_EQUAL(samples.SamplesSize(), 6); - { - const NProto::TMultiSample& s = samples.GetSamples(0); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::COUNTER); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 2); - AssertLabelEqual(s.GetLabels(0), "metric", "single"); + { + const NProto::TMultiSample& s = samples.GetSamples(0); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::COUNTER); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 2); + AssertLabelEqual(s.GetLabels(0), "metric", "single"); AssertLabelEqual(s.GetLabels(1), "labels", "l1"); - - UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); - AssertPointEqual(s.GetPoints(0), now, ui64(17)); - } - { - const NProto::TMultiSample& s = samples.GetSamples(1); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::RATE); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 2); - AssertLabelEqual(s.GetLabels(0), "metric", "single"); + + UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); + AssertPointEqual(s.GetPoints(0), now, ui64(17)); + } + { + const NProto::TMultiSample& s = samples.GetSamples(1); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::RATE); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 2); + AssertLabelEqual(s.GetLabels(0), "metric", "single"); AssertLabelEqual(s.GetLabels(1), "labels", "l2"); - - UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); - AssertPointEqual(s.GetPoints(0), now, ui64(17)); - } - { - const NProto::TMultiSample& s = samples.GetSamples(2); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 2); - AssertLabelEqual(s.GetLabels(0), "metric", "single"); + + UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); + AssertPointEqual(s.GetPoints(0), now, ui64(17)); + } + { + const NProto::TMultiSample& s = samples.GetSamples(2); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 2); + AssertLabelEqual(s.GetLabels(0), "metric", "single"); AssertLabelEqual(s.GetLabels(1), "labels", "l3"); - - UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); - AssertPointEqual(s.GetPoints(0), now, 3.14); - } - { - const NProto::TMultiSample& s = samples.GetSamples(3); + + UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); + AssertPointEqual(s.GetPoints(0), now, 3.14); + } + { + const NProto::TMultiSample& s = samples.GetSamples(3); UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::IGAUGE); UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 2); AssertLabelEqual(s.GetLabels(0), "metric", "single_igauge"); @@ -412,152 +412,152 @@ Y_UNIT_TEST_SUITE(TJsonTest) { } { const NProto::TMultiSample& s = samples.GetSamples(4); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 2); - AssertLabelEqual(s.GetLabels(0), "metric", "multiple"); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 2); + AssertLabelEqual(s.GetLabels(0), "metric", "multiple"); AssertLabelEqual(s.GetLabels(1), "labels", "l5"); - - UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 3); - AssertPointEqualNan(s.GetPoints(0), now); - AssertPointEqualInf(s.GetPoints(1), now + TDuration::Seconds(15), 1); - AssertPointEqualInf(s.GetPoints(2), now + TDuration::Seconds(30), -11); - } - { + + UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 3); + AssertPointEqualNan(s.GetPoints(0), now); + AssertPointEqualInf(s.GetPoints(1), now + TDuration::Seconds(15), 1); + AssertPointEqualInf(s.GetPoints(2), now + TDuration::Seconds(30), -11); + } + { const NProto::TMultiSample& s = samples.GetSamples(5); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::COUNTER); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 2); - AssertLabelEqual(s.GetLabels(0), "metric", "multiple"); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::COUNTER); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 2); + AssertLabelEqual(s.GetLabels(0), "metric", "multiple"); AssertLabelEqual(s.GetLabels(1), "labels", "l6"); - - UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 2); - AssertPointEqual(s.GetPoints(0), now, ui64(1337)); - AssertPointEqual(s.GetPoints(1), now + TDuration::Seconds(15), ui64(1338)); - } - } - - Y_UNIT_TEST(DecodeMetrics) { - NProto::TMultiSamplesList samples; - { - IMetricEncoderPtr e = EncoderProtobuf(&samples); - - TString metricsJson = NResource::Find("/metrics.json"); - DecodeJson(metricsJson, e.Get()); - } - - UNIT_ASSERT_VALUES_EQUAL( - TInstant::MilliSeconds(samples.GetCommonTime()), + + UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 2); + AssertPointEqual(s.GetPoints(0), now, ui64(1337)); + AssertPointEqual(s.GetPoints(1), now + TDuration::Seconds(15), ui64(1338)); + } + } + + Y_UNIT_TEST(DecodeMetrics) { + NProto::TMultiSamplesList samples; + { + IMetricEncoderPtr e = EncoderProtobuf(&samples); + + TString metricsJson = NResource::Find("/metrics.json"); + DecodeJson(metricsJson, e.Get()); + } + + UNIT_ASSERT_VALUES_EQUAL( + TInstant::MilliSeconds(samples.GetCommonTime()), TInstant::ParseIso8601Deprecated("2017-08-27T12:34:56Z")); - - UNIT_ASSERT_VALUES_EQUAL(samples.CommonLabelsSize(), 3); - AssertLabelEqual(samples.GetCommonLabels(0), "project", "solomon"); - AssertLabelEqual(samples.GetCommonLabels(1), "cluster", "man"); - AssertLabelEqual(samples.GetCommonLabels(2), "service", "stockpile"); - + + UNIT_ASSERT_VALUES_EQUAL(samples.CommonLabelsSize(), 3); + AssertLabelEqual(samples.GetCommonLabels(0), "project", "solomon"); + AssertLabelEqual(samples.GetCommonLabels(1), "cluster", "man"); + AssertLabelEqual(samples.GetCommonLabels(2), "service", "stockpile"); + UNIT_ASSERT_VALUES_EQUAL(samples.SamplesSize(), 4); - { - const NProto::TMultiSample& s = samples.GetSamples(0); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); - AssertLabelEqual(s.GetLabels(0), "metric", "Memory"); - - UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); - AssertPointEqual(s.GetPoints(0), TInstant::Zero(), 10.0); - } - { - const NProto::TMultiSample& s = samples.GetSamples(1); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::RATE); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); - AssertLabelEqual(s.GetLabels(0), "metric", "UserTime"); - - UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); - AssertPointEqual(s.GetPoints(0), TInstant::Zero(), ui64(1)); - } - { - const NProto::TMultiSample& s = samples.GetSamples(2); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 2); - AssertLabelEqual(s.GetLabels(0), "export", "Oxygen"); - AssertLabelEqual(s.GetLabels(1), "metric", "QueueSize"); - - UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); + { + const NProto::TMultiSample& s = samples.GetSamples(0); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); + AssertLabelEqual(s.GetLabels(0), "metric", "Memory"); + + UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); + AssertPointEqual(s.GetPoints(0), TInstant::Zero(), 10.0); + } + { + const NProto::TMultiSample& s = samples.GetSamples(1); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::RATE); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); + AssertLabelEqual(s.GetLabels(0), "metric", "UserTime"); + + UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); + AssertPointEqual(s.GetPoints(0), TInstant::Zero(), ui64(1)); + } + { + const NProto::TMultiSample& s = samples.GetSamples(2); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 2); + AssertLabelEqual(s.GetLabels(0), "export", "Oxygen"); + AssertLabelEqual(s.GetLabels(1), "metric", "QueueSize"); + + UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); auto ts = TInstant::ParseIso8601Deprecated("2017-11-05T12:34:56.000Z"); - AssertPointEqual(s.GetPoints(0), ts, 3.14159); - } - { + AssertPointEqual(s.GetPoints(0), ts, 3.14159); + } + { const NProto::TMultiSample& s = samples.GetSamples(3); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); - AssertLabelEqual(s.GetLabels(0), "metric", "Writes"); - - UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 2); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); + AssertLabelEqual(s.GetLabels(0), "metric", "Writes"); + + UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 2); auto ts1 = TInstant::ParseIso8601Deprecated("2017-08-28T12:32:11Z"); - AssertPointEqual(s.GetPoints(0), ts1, -10.0); - auto ts2 = TInstant::Seconds(1503923187); - AssertPointEqual(s.GetPoints(1), ts2, 20.0); - } - } - - Y_UNIT_TEST(DecodeSensors) { - NProto::TMultiSamplesList samples; - { - IMetricEncoderPtr e = EncoderProtobuf(&samples); - - TString sensorsJson = NResource::Find("/sensors.json"); - DecodeJson(sensorsJson, e.Get()); - } - - UNIT_ASSERT_VALUES_EQUAL( - TInstant::MilliSeconds(samples.GetCommonTime()), - TInstant::ParseIso8601Deprecated("2017-08-27T12:34:56Z")); - - UNIT_ASSERT_VALUES_EQUAL(samples.CommonLabelsSize(), 3); - AssertLabelEqual(samples.GetCommonLabels(0), "project", "solomon"); - AssertLabelEqual(samples.GetCommonLabels(1), "cluster", "man"); - AssertLabelEqual(samples.GetCommonLabels(2), "service", "stockpile"); - - UNIT_ASSERT_VALUES_EQUAL(samples.SamplesSize(), 4); - { - const NProto::TMultiSample& s = samples.GetSamples(0); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); - AssertLabelEqual(s.GetLabels(0), "metric", "Memory"); - - UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); - AssertPointEqual(s.GetPoints(0), TInstant::Zero(), 10.0); - } - { - const NProto::TMultiSample& s = samples.GetSamples(1); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::RATE); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); - AssertLabelEqual(s.GetLabels(0), "metric", "UserTime"); - - UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); - AssertPointEqual(s.GetPoints(0), TInstant::Zero(), ui64(1)); - } - { - const NProto::TMultiSample& s = samples.GetSamples(2); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 2); - AssertLabelEqual(s.GetLabels(0), "export", "Oxygen"); - AssertLabelEqual(s.GetLabels(1), "metric", "QueueSize"); - - UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); - auto ts = TInstant::ParseIso8601Deprecated("2017-11-05T12:34:56.000Z"); - AssertPointEqual(s.GetPoints(0), ts, 3.14159); - } - { - const NProto::TMultiSample& s = samples.GetSamples(3); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); - AssertLabelEqual(s.GetLabels(0), "metric", "Writes"); - - UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 2); - auto ts1 = TInstant::ParseIso8601Deprecated("2017-08-28T12:32:11Z"); - AssertPointEqual(s.GetPoints(0), ts1, -10.0); - auto ts2 = TInstant::Seconds(1503923187); - AssertPointEqual(s.GetPoints(1), ts2, 20.0); - } - } + AssertPointEqual(s.GetPoints(0), ts1, -10.0); + auto ts2 = TInstant::Seconds(1503923187); + AssertPointEqual(s.GetPoints(1), ts2, 20.0); + } + } + + Y_UNIT_TEST(DecodeSensors) { + NProto::TMultiSamplesList samples; + { + IMetricEncoderPtr e = EncoderProtobuf(&samples); + + TString sensorsJson = NResource::Find("/sensors.json"); + DecodeJson(sensorsJson, e.Get()); + } + + UNIT_ASSERT_VALUES_EQUAL( + TInstant::MilliSeconds(samples.GetCommonTime()), + TInstant::ParseIso8601Deprecated("2017-08-27T12:34:56Z")); + + UNIT_ASSERT_VALUES_EQUAL(samples.CommonLabelsSize(), 3); + AssertLabelEqual(samples.GetCommonLabels(0), "project", "solomon"); + AssertLabelEqual(samples.GetCommonLabels(1), "cluster", "man"); + AssertLabelEqual(samples.GetCommonLabels(2), "service", "stockpile"); + + UNIT_ASSERT_VALUES_EQUAL(samples.SamplesSize(), 4); + { + const NProto::TMultiSample& s = samples.GetSamples(0); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); + AssertLabelEqual(s.GetLabels(0), "metric", "Memory"); + + UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); + AssertPointEqual(s.GetPoints(0), TInstant::Zero(), 10.0); + } + { + const NProto::TMultiSample& s = samples.GetSamples(1); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::RATE); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); + AssertLabelEqual(s.GetLabels(0), "metric", "UserTime"); + + UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); + AssertPointEqual(s.GetPoints(0), TInstant::Zero(), ui64(1)); + } + { + const NProto::TMultiSample& s = samples.GetSamples(2); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 2); + AssertLabelEqual(s.GetLabels(0), "export", "Oxygen"); + AssertLabelEqual(s.GetLabels(1), "metric", "QueueSize"); + + UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); + auto ts = TInstant::ParseIso8601Deprecated("2017-11-05T12:34:56.000Z"); + AssertPointEqual(s.GetPoints(0), ts, 3.14159); + } + { + const NProto::TMultiSample& s = samples.GetSamples(3); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); + AssertLabelEqual(s.GetLabels(0), "metric", "Writes"); + + UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 2); + auto ts1 = TInstant::ParseIso8601Deprecated("2017-08-28T12:32:11Z"); + AssertPointEqual(s.GetPoints(0), ts1, -10.0); + auto ts2 = TInstant::Seconds(1503923187); + AssertPointEqual(s.GetPoints(1), ts2, 20.0); + } + } Y_UNIT_TEST(DecodeToEncoder) { auto testJson = NResource::Find("/test_decode_to_encode.json"); @@ -574,16 +574,16 @@ Y_UNIT_TEST_SUITE(TJsonTest) { UNIT_ASSERT_VALUES_EQUAL(val1, val2); } - void WriteEmptySeries(const IMetricEncoderPtr& e) { + void WriteEmptySeries(const IMetricEncoderPtr& e) { e->OnStreamBegin(); { - e->OnMetricBegin(EMetricType::COUNTER); + e->OnMetricBegin(EMetricType::COUNTER); { e->OnLabelsBegin(); e->OnLabel("foo", "bar"); e->OnLabelsEnd(); } - e->OnMetricEnd(); + e->OnMetricEnd(); } e->OnStreamEnd(); @@ -602,9 +602,9 @@ Y_UNIT_TEST_SUITE(TJsonTest) { UNIT_ASSERT_NO_DIFF(json, expectedJson); } - void WriteEmptyLabels(IMetricEncoderPtr& e) { + void WriteEmptyLabels(IMetricEncoderPtr& e) { e->OnStreamBegin(); - e->OnMetricBegin(EMetricType::COUNTER); + e->OnMetricBegin(EMetricType::COUNTER); e->OnLabelsBegin(); UNIT_ASSERT_EXCEPTION(e->OnLabelsEnd(), yexception); @@ -638,29 +638,29 @@ Y_UNIT_TEST_SUITE(TJsonTest) { UNIT_ASSERT_NO_DIFF(json, expectedJson); } - Y_UNIT_TEST(BufferedEncoderMergesMetrics) { + Y_UNIT_TEST(BufferedEncoderMergesMetrics) { TString json; TStringOutput out(json); auto e = BufferedEncoderJson(&out, 2); auto ts = 1; - auto writeMetric = [&] (const TString& val) { - e->OnMetricBegin(EMetricType::COUNTER); + auto writeMetric = [&] (const TString& val) { + e->OnMetricBegin(EMetricType::COUNTER); e->OnLabelsBegin(); e->OnLabel("foo", val); e->OnLabelsEnd(); e->OnUint64(TInstant::Seconds(ts++), 42); - e->OnMetricEnd(); + e->OnMetricEnd(); }; e->OnStreamBegin(); - writeMetric("bar"); - writeMetric("bar"); - writeMetric("baz"); - writeMetric("bar"); + writeMetric("bar"); + writeMetric("bar"); + writeMetric("baz"); + writeMetric("bar"); e->OnStreamEnd(); e->Close(); @@ -676,13 +676,13 @@ Y_UNIT_TEST_SUITE(TJsonTest) { auto e = EncoderJson(&out); auto writePreamble = [&] { e->OnStreamBegin(); - e->OnMetricBegin(EMetricType::COUNTER); + e->OnMetricBegin(EMetricType::COUNTER); e->OnLabelsBegin(); e->OnLabel("foo", "bar"); e->OnLabelsEnd(); }; - // writing two values for a metric in a row will trigger + // writing two values for a metric in a row will trigger // timeseries object construction writePreamble(); e->OnUint64(TInstant::Zero(), 42); @@ -705,7 +705,7 @@ Y_UNIT_TEST_SUITE(TJsonTest) { { auto e = BufferedEncoderJson(&out, 2); e->OnStreamBegin(); - e->OnMetricBegin(EMetricType::COUNTER); + e->OnMetricBegin(EMetricType::COUNTER); e->OnLabelsBegin(); e->OnLabel("foo", "bar"); e->OnLabelsEnd(); @@ -715,38 +715,38 @@ Y_UNIT_TEST_SUITE(TJsonTest) { e->OnUint64(TInstant::Zero(), 43); e->OnUint64(TInstant::Zero(), 44); e->OnUint64(TInstant::Zero(), 45); - e->OnMetricEnd(); + e->OnMetricEnd(); e->OnStreamEnd(); } out << "\n"; UNIT_ASSERT_NO_DIFF(out.Str(), NResource::Find("/buffered_ts_merge.json")); - } - - template <typename TFactory, typename TConsumer> - TString EncodeToString(TFactory factory, TConsumer consumer) { - TStringStream out; - { - IMetricEncoderPtr e = factory(&out, 2); - consumer(e.Get()); - } - out << '\n'; - return out.Str(); } - + + template <typename TFactory, typename TConsumer> + TString EncodeToString(TFactory factory, TConsumer consumer) { + TStringStream out; + { + IMetricEncoderPtr e = factory(&out, 2); + consumer(e.Get()); + } + out << '\n'; + return out.Str(); + } + Y_UNIT_TEST(SummaryValueEncode) { - auto writeDocument = [](IMetricEncoder* e) { + auto writeDocument = [](IMetricEncoder* e) { e->OnStreamBegin(); { - e->OnMetricBegin(EMetricType::DSUMMARY); + e->OnMetricBegin(EMetricType::DSUMMARY); { e->OnLabelsBegin(); - e->OnLabel("metric", "temperature"); + e->OnLabel("metric", "temperature"); e->OnLabelsEnd(); } e->OnSummaryDouble(now, MakeIntrusive<TSummaryDoubleSnapshot>(10., -0.5, 0.5, 0.3, 30u)); - e->OnMetricEnd(); + e->OnMetricEnd(); } e->OnStreamEnd(); }; @@ -768,18 +768,18 @@ Y_UNIT_TEST_SUITE(TJsonTest) { } Y_UNIT_TEST(SummaryInfEncode) { - auto writeDocument = [](IMetricEncoder* e) { + auto writeDocument = [](IMetricEncoder* e) { e->OnStreamBegin(); { - e->OnMetricBegin(EMetricType::DSUMMARY); + e->OnMetricBegin(EMetricType::DSUMMARY); { e->OnLabelsBegin(); - e->OnLabel("metric", "temperature"); + e->OnLabel("metric", "temperature"); e->OnLabelsEnd(); } e->OnSummaryDouble(now, TestInfSummary()); - e->OnMetricEnd(); + e->OnMetricEnd(); } e->OnStreamEnd(); }; @@ -794,7 +794,7 @@ Y_UNIT_TEST_SUITE(TJsonTest) { Y_UNIT_TEST(SummaryInfDecode) { NProto::TMultiSamplesList samples; { - IMetricEncoderPtr e = EncoderProtobuf(&samples); + IMetricEncoderPtr e = EncoderProtobuf(&samples); TString testJson = NResource::Find("/summary_inf.json"); DecodeJson(testJson, e.Get()); @@ -803,9 +803,9 @@ Y_UNIT_TEST_SUITE(TJsonTest) { UNIT_ASSERT_VALUES_EQUAL(1, samples.SamplesSize()); const NProto::TMultiSample& s = samples.GetSamples(0); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::DSUMMARY); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::DSUMMARY); UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); - AssertLabelEqual(s.GetLabels(0), "metric", "temperature"); + AssertLabelEqual(s.GetLabels(0), "metric", "temperature"); UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); @@ -820,7 +820,7 @@ Y_UNIT_TEST_SUITE(TJsonTest) { Y_UNIT_TEST(SummaryValueDecode) { NProto::TMultiSamplesList samples; { - IMetricEncoderPtr e = EncoderProtobuf(&samples); + IMetricEncoderPtr e = EncoderProtobuf(&samples); TString testJson = NResource::Find("/summary_value.json"); DecodeJson(testJson, e.Get()); @@ -829,9 +829,9 @@ Y_UNIT_TEST_SUITE(TJsonTest) { UNIT_ASSERT_VALUES_EQUAL(1, samples.SamplesSize()); const NProto::TMultiSample& s = samples.GetSamples(0); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::DSUMMARY); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::DSUMMARY); UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); - AssertLabelEqual(s.GetLabels(0), "metric", "temperature"); + AssertLabelEqual(s.GetLabels(0), "metric", "temperature"); UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); @@ -840,13 +840,13 @@ Y_UNIT_TEST_SUITE(TJsonTest) { } Y_UNIT_TEST(SummaryTimeSeriesEncode) { - auto writeDocument = [](IMetricEncoder* e) { + auto writeDocument = [](IMetricEncoder* e) { e->OnStreamBegin(); { - e->OnMetricBegin(EMetricType::DSUMMARY); + e->OnMetricBegin(EMetricType::DSUMMARY); { e->OnLabelsBegin(); - e->OnLabel("metric", "temperature"); + e->OnLabel("metric", "temperature"); e->OnLabelsEnd(); } @@ -862,7 +862,7 @@ Y_UNIT_TEST_SUITE(TJsonTest) { e->OnSummaryDouble(now + TDuration::Seconds(15), summary.Snapshot()); - e->OnMetricEnd(); + e->OnMetricEnd(); } e->OnStreamEnd(); }; @@ -877,7 +877,7 @@ Y_UNIT_TEST_SUITE(TJsonTest) { Y_UNIT_TEST(SummaryTimeSeriesDecode) { NProto::TMultiSamplesList samples; { - IMetricEncoderPtr e = EncoderProtobuf(&samples); + IMetricEncoderPtr e = EncoderProtobuf(&samples); TString testJson = NResource::Find("/summary_timeseries.json"); DecodeJson(testJson, e.Get()); @@ -886,9 +886,9 @@ Y_UNIT_TEST_SUITE(TJsonTest) { UNIT_ASSERT_VALUES_EQUAL(1, samples.SamplesSize()); const NProto::TMultiSample& s = samples.GetSamples(0); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::DSUMMARY); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::DSUMMARY); UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); - AssertLabelEqual(s.GetLabels(0), "metric", "temperature"); + AssertLabelEqual(s.GetLabels(0), "metric", "temperature"); UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 2); @@ -912,7 +912,7 @@ Y_UNIT_TEST_SUITE(TJsonTest) { e->OnMetricBegin(EMetricType::LOGHIST); { e->OnLabelsBegin(); - e->OnLabel("metric", "ms"); + e->OnLabel("metric", "ms"); e->OnLabelsEnd(); } @@ -943,7 +943,7 @@ Y_UNIT_TEST_SUITE(TJsonTest) { UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::LOGHISTOGRAM); UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); - AssertLabelEqual(s.GetLabels(0), "metric", "ms"); + AssertLabelEqual(s.GetLabels(0), "metric", "ms"); UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); @@ -951,36 +951,36 @@ Y_UNIT_TEST_SUITE(TJsonTest) { AssertPointEqual(s.GetPoints(0), now, *snapshot); } - Y_UNIT_TEST(HistogramValueEncode) { - auto writeDocument = [](IMetricEncoder* e) { - e->OnStreamBegin(); - { - e->OnMetricBegin(EMetricType::HIST); - { - e->OnLabelsBegin(); - e->OnLabel("metric", "responseTimeMillis"); - e->OnLabelsEnd(); - } - - // {1: 1, 2: 1, 4: 2, 8: 4, 16: 8, inf: 83} - auto h = ExponentialHistogram(6, 2); - for (i64 i = 1; i < 100; i++) { - h->Collect(i); - } - - e->OnHistogram(now, h->Snapshot()); - e->OnMetricEnd(); - } - e->OnStreamEnd(); - }; - - TString result1 = EncodeToString(EncoderJson, writeDocument); - UNIT_ASSERT_NO_DIFF(result1, NResource::Find("/histogram_value.json")); - - TString result2 = EncodeToString(BufferedEncoderJson, writeDocument); - UNIT_ASSERT_NO_DIFF(result2, NResource::Find("/histogram_value.json")); - } - + Y_UNIT_TEST(HistogramValueEncode) { + auto writeDocument = [](IMetricEncoder* e) { + e->OnStreamBegin(); + { + e->OnMetricBegin(EMetricType::HIST); + { + e->OnLabelsBegin(); + e->OnLabel("metric", "responseTimeMillis"); + e->OnLabelsEnd(); + } + + // {1: 1, 2: 1, 4: 2, 8: 4, 16: 8, inf: 83} + auto h = ExponentialHistogram(6, 2); + for (i64 i = 1; i < 100; i++) { + h->Collect(i); + } + + e->OnHistogram(now, h->Snapshot()); + e->OnMetricEnd(); + } + e->OnStreamEnd(); + }; + + TString result1 = EncodeToString(EncoderJson, writeDocument); + UNIT_ASSERT_NO_DIFF(result1, NResource::Find("/histogram_value.json")); + + TString result2 = EncodeToString(BufferedEncoderJson, writeDocument); + UNIT_ASSERT_NO_DIFF(result2, NResource::Find("/histogram_value.json")); + } + Y_UNIT_TEST(LogHistogramTimeSeriesEncode) { auto writeDocument = [](IMetricEncoder* e) { e->OnStreamBegin(); @@ -988,7 +988,7 @@ Y_UNIT_TEST_SUITE(TJsonTest) { e->OnMetricBegin(EMetricType::LOGHIST); { e->OnLabelsBegin(); - e->OnLabel("metric", "ms"); + e->OnLabel("metric", "ms"); e->OnLabelsEnd(); } @@ -1022,7 +1022,7 @@ Y_UNIT_TEST_SUITE(TJsonTest) { UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::LOGHISTOGRAM); UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); - AssertLabelEqual(s.GetLabels(0), "metric", "ms"); + AssertLabelEqual(s.GetLabels(0), "metric", "ms"); UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 2); @@ -1034,130 +1034,130 @@ Y_UNIT_TEST_SUITE(TJsonTest) { } void HistogramValueDecode(const TString& filePath) { - NProto::TMultiSamplesList samples; - { - IMetricEncoderPtr e = EncoderProtobuf(&samples); - + NProto::TMultiSamplesList samples; + { + IMetricEncoderPtr e = EncoderProtobuf(&samples); + TString testJson = NResource::Find(filePath); - DecodeJson(testJson, e.Get()); - } - - UNIT_ASSERT_VALUES_EQUAL(1, samples.SamplesSize()); - const NProto::TMultiSample& s = samples.GetSamples(0); - - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::HISTOGRAM); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); - AssertLabelEqual(s.GetLabels(0), "metric", "responseTimeMillis"); - - UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); - - auto h = ExponentialHistogram(6, 2); - for (i64 i = 1; i < 100; i++) { - h->Collect(i); - } - - AssertPointEqual(s.GetPoints(0), now, *h->Snapshot()); - } - + DecodeJson(testJson, e.Get()); + } + + UNIT_ASSERT_VALUES_EQUAL(1, samples.SamplesSize()); + const NProto::TMultiSample& s = samples.GetSamples(0); + + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::HISTOGRAM); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); + AssertLabelEqual(s.GetLabels(0), "metric", "responseTimeMillis"); + + UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); + + auto h = ExponentialHistogram(6, 2); + for (i64 i = 1; i < 100; i++) { + h->Collect(i); + } + + AssertPointEqual(s.GetPoints(0), now, *h->Snapshot()); + } + Y_UNIT_TEST(HistogramValueDecode) { HistogramValueDecode("/histogram_value.json"); HistogramValueDecode("/histogram_value_inf_before_bounds.json"); } - Y_UNIT_TEST(HistogramTimeSeriesEncode) { - auto writeDocument = [](IMetricEncoder* e) { - e->OnStreamBegin(); - { - e->OnMetricBegin(EMetricType::HIST_RATE); - { - e->OnLabelsBegin(); - e->OnLabel("metric", "responseTimeMillis"); - e->OnLabelsEnd(); - } - - // {1: 1, 2: 1, 4: 2, 8: 4, 16: 8, inf: 83} - auto h = ExponentialHistogram(6, 2); - for (i64 i = 1; i < 100; i++) { - h->Collect(i); - } - e->OnHistogram(now, h->Snapshot()); - - // {1: 2, 2: 2, 4: 4, 8: 8, 16: 16, inf: 166} - for (i64 i = 1; i < 100; i++) { - h->Collect(i); - } - e->OnHistogram(now + TDuration::Seconds(15), h->Snapshot()); - - e->OnMetricEnd(); - } - e->OnStreamEnd(); - }; - - TString result1 = EncodeToString(EncoderJson, writeDocument); - UNIT_ASSERT_NO_DIFF(result1, NResource::Find("/histogram_timeseries.json")); - - TString result2 = EncodeToString(BufferedEncoderJson, writeDocument); - UNIT_ASSERT_NO_DIFF(result2, NResource::Find("/histogram_timeseries.json")); - } - - Y_UNIT_TEST(HistogramTimeSeriesDecode) { - NProto::TMultiSamplesList samples; - { - IMetricEncoderPtr e = EncoderProtobuf(&samples); - - TString testJson = NResource::Find("/histogram_timeseries.json"); - DecodeJson(testJson, e.Get()); - } - - UNIT_ASSERT_VALUES_EQUAL(1, samples.SamplesSize()); - const NProto::TMultiSample& s = samples.GetSamples(0); - - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::HIST_RATE); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); - AssertLabelEqual(s.GetLabels(0), "metric", "responseTimeMillis"); - - UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 2); - - auto h = ExponentialHistogram(6, 2); - for (i64 i = 1; i < 100; i++) { - h->Collect(i); - } - - AssertPointEqual(s.GetPoints(0), now, *h->Snapshot()); - - for (i64 i = 1; i < 100; i++) { - h->Collect(i); - } - - AssertPointEqual(s.GetPoints(1), now + TDuration::Seconds(15), *h->Snapshot()); - } - - Y_UNIT_TEST(IntGaugeEncode) { - auto writeDocument = [](IMetricEncoder* e) { - e->OnStreamBegin(); - { - e->OnMetricBegin(EMetricType::IGAUGE); - { - e->OnLabelsBegin(); - e->OnLabel("metric", "a"); - e->OnLabelsEnd(); - } - e->OnInt64(now, Min<i64>()); - e->OnInt64(now + TDuration::Seconds(1), -1); - e->OnInt64(now + TDuration::Seconds(2), 0); - e->OnInt64(now + TDuration::Seconds(3), Max<i64>()); - e->OnMetricEnd(); - } - e->OnStreamEnd(); - }; - - TString result1 = EncodeToString(EncoderJson, writeDocument); - UNIT_ASSERT_NO_DIFF(result1, NResource::Find("/int_gauge.json")); - - TString result2 = EncodeToString(BufferedEncoderJson, writeDocument); - UNIT_ASSERT_NO_DIFF(result2, NResource::Find("/int_gauge.json")); - } - + Y_UNIT_TEST(HistogramTimeSeriesEncode) { + auto writeDocument = [](IMetricEncoder* e) { + e->OnStreamBegin(); + { + e->OnMetricBegin(EMetricType::HIST_RATE); + { + e->OnLabelsBegin(); + e->OnLabel("metric", "responseTimeMillis"); + e->OnLabelsEnd(); + } + + // {1: 1, 2: 1, 4: 2, 8: 4, 16: 8, inf: 83} + auto h = ExponentialHistogram(6, 2); + for (i64 i = 1; i < 100; i++) { + h->Collect(i); + } + e->OnHistogram(now, h->Snapshot()); + + // {1: 2, 2: 2, 4: 4, 8: 8, 16: 16, inf: 166} + for (i64 i = 1; i < 100; i++) { + h->Collect(i); + } + e->OnHistogram(now + TDuration::Seconds(15), h->Snapshot()); + + e->OnMetricEnd(); + } + e->OnStreamEnd(); + }; + + TString result1 = EncodeToString(EncoderJson, writeDocument); + UNIT_ASSERT_NO_DIFF(result1, NResource::Find("/histogram_timeseries.json")); + + TString result2 = EncodeToString(BufferedEncoderJson, writeDocument); + UNIT_ASSERT_NO_DIFF(result2, NResource::Find("/histogram_timeseries.json")); + } + + Y_UNIT_TEST(HistogramTimeSeriesDecode) { + NProto::TMultiSamplesList samples; + { + IMetricEncoderPtr e = EncoderProtobuf(&samples); + + TString testJson = NResource::Find("/histogram_timeseries.json"); + DecodeJson(testJson, e.Get()); + } + + UNIT_ASSERT_VALUES_EQUAL(1, samples.SamplesSize()); + const NProto::TMultiSample& s = samples.GetSamples(0); + + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::HIST_RATE); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); + AssertLabelEqual(s.GetLabels(0), "metric", "responseTimeMillis"); + + UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 2); + + auto h = ExponentialHistogram(6, 2); + for (i64 i = 1; i < 100; i++) { + h->Collect(i); + } + + AssertPointEqual(s.GetPoints(0), now, *h->Snapshot()); + + for (i64 i = 1; i < 100; i++) { + h->Collect(i); + } + + AssertPointEqual(s.GetPoints(1), now + TDuration::Seconds(15), *h->Snapshot()); + } + + Y_UNIT_TEST(IntGaugeEncode) { + auto writeDocument = [](IMetricEncoder* e) { + e->OnStreamBegin(); + { + e->OnMetricBegin(EMetricType::IGAUGE); + { + e->OnLabelsBegin(); + e->OnLabel("metric", "a"); + e->OnLabelsEnd(); + } + e->OnInt64(now, Min<i64>()); + e->OnInt64(now + TDuration::Seconds(1), -1); + e->OnInt64(now + TDuration::Seconds(2), 0); + e->OnInt64(now + TDuration::Seconds(3), Max<i64>()); + e->OnMetricEnd(); + } + e->OnStreamEnd(); + }; + + TString result1 = EncodeToString(EncoderJson, writeDocument); + UNIT_ASSERT_NO_DIFF(result1, NResource::Find("/int_gauge.json")); + + TString result2 = EncodeToString(BufferedEncoderJson, writeDocument); + UNIT_ASSERT_NO_DIFF(result2, NResource::Find("/int_gauge.json")); + } + Y_UNIT_TEST(InconsistentMetricTypes) { auto emitMetrics = [](IMetricEncoder& encoder, const TString& expectedError) { encoder.OnMetricBegin(EMetricType::GAUGE); @@ -1213,33 +1213,33 @@ Y_UNIT_TEST_SUITE(TJsonTest) { } } - Y_UNIT_TEST(IntGaugeDecode) { - NProto::TMultiSamplesList samples; - { - IMetricEncoderPtr e = EncoderProtobuf(&samples); - - TString testJson = NResource::Find("/int_gauge.json"); - DecodeJson(testJson, e.Get()); - } - - UNIT_ASSERT_VALUES_EQUAL(1, samples.SamplesSize()); - const NProto::TMultiSample& s = samples.GetSamples(0); - - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::IGAUGE); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); - AssertLabelEqual(s.GetLabels(0), "metric", "a"); - - UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 4); - AssertPointEqual(s.GetPoints(0), now, Min<i64>()); - AssertPointEqual(s.GetPoints(1), now + TDuration::Seconds(1), i64(-1)); - AssertPointEqual(s.GetPoints(2), now + TDuration::Seconds(2), i64(0)); - AssertPointEqual(s.GetPoints(3), now + TDuration::Seconds(3), Max<i64>()); - } + Y_UNIT_TEST(IntGaugeDecode) { + NProto::TMultiSamplesList samples; + { + IMetricEncoderPtr e = EncoderProtobuf(&samples); + + TString testJson = NResource::Find("/int_gauge.json"); + DecodeJson(testJson, e.Get()); + } + + UNIT_ASSERT_VALUES_EQUAL(1, samples.SamplesSize()); + const NProto::TMultiSample& s = samples.GetSamples(0); + + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::IGAUGE); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); + AssertLabelEqual(s.GetLabels(0), "metric", "a"); + + UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 4); + AssertPointEqual(s.GetPoints(0), now, Min<i64>()); + AssertPointEqual(s.GetPoints(1), now + TDuration::Seconds(1), i64(-1)); + AssertPointEqual(s.GetPoints(2), now + TDuration::Seconds(2), i64(0)); + AssertPointEqual(s.GetPoints(3), now + TDuration::Seconds(3), Max<i64>()); + } Y_UNIT_TEST(FuzzerRegression) { NProto::TMultiSamplesList samples; { - IMetricEncoderPtr e = EncoderProtobuf(&samples); + IMetricEncoderPtr e = EncoderProtobuf(&samples); for (auto f : { "/hist_crash.json", "/crash.json" }) { TString testJson = NResource::Find(f); @@ -1254,13 +1254,13 @@ Y_UNIT_TEST_SUITE(TJsonTest) { { "mode": "deriv", "value": -1, - "labels": { "metric": "SystemTime" } + "labels": { "metric": "SystemTime" } }, } ]}")"; NProto::TMultiSamplesList samples; - IMetricEncoderPtr e = EncoderProtobuf(&samples); + IMetricEncoderPtr e = EncoderProtobuf(&samples); UNIT_ASSERT_EXCEPTION(DecodeJson(input, e.Get()), yexception); } @@ -1287,4 +1287,4 @@ Y_UNIT_TEST_SUITE(TJsonTest) { } } -} +} diff --git a/library/cpp/monlib/encode/json/typed_point.h b/library/cpp/monlib/encode/json/typed_point.h index b606c82924e..fbaa840c4bf 100644 --- a/library/cpp/monlib/encode/json/typed_point.h +++ b/library/cpp/monlib/encode/json/typed_point.h @@ -1,123 +1,123 @@ -#pragma once - -#include <library/cpp/monlib/metrics/metric_value.h> - - -namespace NMonitoring { - - class TTypedPoint { - public: - TTypedPoint() - : Time_(TInstant::Zero()) - , ValueType_(EMetricValueType::UNKNOWN) - { - } - - template <typename T> - TTypedPoint(TInstant time, T value) - : Time_(time) - , ValueType_(TValueType<T>::Type) - , Value_(value) - { - Ref(); - } - - ~TTypedPoint() { - UnRef(); - } - - TTypedPoint(const TTypedPoint& rhs) - : Time_(rhs.Time_) - , ValueType_(rhs.ValueType_) - , Value_(rhs.Value_) - { - Ref(); - } - - TTypedPoint& operator=(const TTypedPoint& rhs) { - UnRef(); - - Time_ = rhs.Time_; - ValueType_ = rhs.ValueType_; - Value_ = rhs.Value_; - - Ref(); - return *this; - } - +#pragma once + +#include <library/cpp/monlib/metrics/metric_value.h> + + +namespace NMonitoring { + + class TTypedPoint { + public: + TTypedPoint() + : Time_(TInstant::Zero()) + , ValueType_(EMetricValueType::UNKNOWN) + { + } + + template <typename T> + TTypedPoint(TInstant time, T value) + : Time_(time) + , ValueType_(TValueType<T>::Type) + , Value_(value) + { + Ref(); + } + + ~TTypedPoint() { + UnRef(); + } + + TTypedPoint(const TTypedPoint& rhs) + : Time_(rhs.Time_) + , ValueType_(rhs.ValueType_) + , Value_(rhs.Value_) + { + Ref(); + } + + TTypedPoint& operator=(const TTypedPoint& rhs) { + UnRef(); + + Time_ = rhs.Time_; + ValueType_ = rhs.ValueType_; + Value_ = rhs.Value_; + + Ref(); + return *this; + } + TTypedPoint(TTypedPoint&& rhs) noexcept - : Time_(rhs.Time_) - , ValueType_(rhs.ValueType_) - , Value_(rhs.Value_) - { - rhs.ValueType_ = EMetricValueType::UNKNOWN; - rhs.Value_ = {}; - } - + : Time_(rhs.Time_) + , ValueType_(rhs.ValueType_) + , Value_(rhs.Value_) + { + rhs.ValueType_ = EMetricValueType::UNKNOWN; + rhs.Value_ = {}; + } + TTypedPoint& operator=(TTypedPoint&& rhs) noexcept { - UnRef(); - - Time_ = rhs.Time_; - ValueType_ = rhs.ValueType_; - Value_ = rhs.Value_; - - rhs.ValueType_ = EMetricValueType::UNKNOWN; - rhs.Value_ = {}; - - return *this; - } - - TInstant GetTime() const noexcept { - return Time_; - } - - void SetTime(TInstant time) noexcept { - Time_ = time; - } - - TMetricValue GetValue() const noexcept { - return Value_; - } - - EMetricValueType GetValueType() const noexcept { - return ValueType_; - } - - template <typename T> - void SetValue(T value) noexcept { - ValueType_ = TValueType<T>::Type; - Value_ = TMetricValue{value}; - } - - bool HasValue() { - return ValueType_ != EMetricValueType::UNKNOWN; - } - - private: - void Ref() { - if (ValueType_ == EMetricValueType::HISTOGRAM) { - Value_.AsHistogram()->Ref(); - } else if (ValueType_ == EMetricValueType::SUMMARY) { + UnRef(); + + Time_ = rhs.Time_; + ValueType_ = rhs.ValueType_; + Value_ = rhs.Value_; + + rhs.ValueType_ = EMetricValueType::UNKNOWN; + rhs.Value_ = {}; + + return *this; + } + + TInstant GetTime() const noexcept { + return Time_; + } + + void SetTime(TInstant time) noexcept { + Time_ = time; + } + + TMetricValue GetValue() const noexcept { + return Value_; + } + + EMetricValueType GetValueType() const noexcept { + return ValueType_; + } + + template <typename T> + void SetValue(T value) noexcept { + ValueType_ = TValueType<T>::Type; + Value_ = TMetricValue{value}; + } + + bool HasValue() { + return ValueType_ != EMetricValueType::UNKNOWN; + } + + private: + void Ref() { + if (ValueType_ == EMetricValueType::HISTOGRAM) { + Value_.AsHistogram()->Ref(); + } else if (ValueType_ == EMetricValueType::SUMMARY) { Value_.AsSummaryDouble()->Ref(); } else if (ValueType_ == EMetricValueType::LOGHISTOGRAM) { Value_.AsLogHistogram()->Ref(); } - } - - void UnRef() { - if (ValueType_ == EMetricValueType::HISTOGRAM) { - Value_.AsHistogram()->UnRef(); - } else if (ValueType_ == EMetricValueType::SUMMARY) { + } + + void UnRef() { + if (ValueType_ == EMetricValueType::HISTOGRAM) { + Value_.AsHistogram()->UnRef(); + } else if (ValueType_ == EMetricValueType::SUMMARY) { Value_.AsSummaryDouble()->UnRef(); } else if (ValueType_ == EMetricValueType::LOGHISTOGRAM) { Value_.AsLogHistogram()->UnRef(); } - } - - private: - TInstant Time_; - EMetricValueType ValueType_; - TMetricValue Value_; - }; - -} + } + + private: + TInstant Time_; + EMetricValueType ValueType_; + TMetricValue Value_; + }; + +} diff --git a/library/cpp/monlib/encode/json/ut/expected.json b/library/cpp/monlib/encode/json/ut/expected.json index 8541479be6c..ead853455ba 100644 --- a/library/cpp/monlib/encode/json/ut/expected.json +++ b/library/cpp/monlib/encode/json/ut/expected.json @@ -1,42 +1,42 @@ -{ - "ts":1500000000, - "commonLabels": - { - "project":"solomon" - }, - "sensors": - [ - { - "kind":"COUNTER", - "labels": - { - "metric":"single", +{ + "ts":1500000000, + "commonLabels": + { + "project":"solomon" + }, + "sensors": + [ + { + "kind":"COUNTER", + "labels": + { + "metric":"single", "labels":"l1" - }, - "ts":1509843723, - "value":17 - }, - { - "kind":"RATE", - "labels": - { - "metric":"single", + }, + "ts":1509843723, + "value":17 + }, + { + "kind":"RATE", + "labels": + { + "metric":"single", "labels":"l2" - }, - "ts":1509843723, - "value":17 - }, - { - "kind":"GAUGE", - "labels": - { - "metric":"single", + }, + "ts":1509843723, + "value":17 + }, + { + "kind":"GAUGE", + "labels": + { + "metric":"single", "labels":"l3" - }, - "ts":1509843723, - "value":3.14 - }, - { + }, + "ts":1509843723, + "value":3.14 + }, + { "kind":"IGAUGE", "labels": { @@ -47,46 +47,46 @@ "value":42 }, { - "kind":"GAUGE", - "labels": - { - "metric":"multiple", + "kind":"GAUGE", + "labels": + { + "metric":"multiple", "labels":"l5" - }, - "timeseries": - [ - { - "ts":1509843723, - "value":"nan" - }, - { - "ts":1509843738, - "value":"inf" - }, - { - "ts":1509843753, - "value":"-inf" - } - ] - }, - { - "kind":"COUNTER", - "timeseries": - [ - { - "ts":1509843723, - "value":1337 - }, - { - "ts":1509843738, - "value":1338 - } - ], - "labels": - { - "metric":"multiple", + }, + "timeseries": + [ + { + "ts":1509843723, + "value":"nan" + }, + { + "ts":1509843738, + "value":"inf" + }, + { + "ts":1509843753, + "value":"-inf" + } + ] + }, + { + "kind":"COUNTER", + "timeseries": + [ + { + "ts":1509843723, + "value":1337 + }, + { + "ts":1509843738, + "value":1338 + } + ], + "labels": + { + "metric":"multiple", "labels":"l6" - } - } - ] -} + } + } + ] +} diff --git a/library/cpp/monlib/encode/json/ut/expected_buffered.json b/library/cpp/monlib/encode/json/ut/expected_buffered.json index 17c0f063236..9a6a1d6201d 100644 --- a/library/cpp/monlib/encode/json/ut/expected_buffered.json +++ b/library/cpp/monlib/encode/json/ut/expected_buffered.json @@ -1,42 +1,42 @@ -{ - "ts":1500000000, - "commonLabels": - { - "project":"solomon" - }, - "sensors": - [ - { - "kind":"COUNTER", - "labels": - { +{ + "ts":1500000000, + "commonLabels": + { + "project":"solomon" + }, + "sensors": + [ + { + "kind":"COUNTER", + "labels": + { "labels":"l1", "metric":"single" - }, - "ts":1509843723, - "value":17 - }, - { - "kind":"RATE", - "labels": - { + }, + "ts":1509843723, + "value":17 + }, + { + "kind":"RATE", + "labels": + { "labels":"l2", "metric":"single" - }, - "ts":1509843723, - "value":17 - }, - { - "kind":"GAUGE", - "labels": - { + }, + "ts":1509843723, + "value":17 + }, + { + "kind":"GAUGE", + "labels": + { "labels":"l3", "metric":"single" - }, - "ts":1509843723, - "value":3.14 - }, - { + }, + "ts":1509843723, + "value":3.14 + }, + { "kind":"IGAUGE", "labels": { @@ -47,46 +47,46 @@ "value":42 }, { - "kind":"GAUGE", - "labels": - { + "kind":"GAUGE", + "labels": + { "labels":"l5", "metric":"multiple" - }, - "timeseries": - [ - { - "ts":1509843723, - "value":"nan" - }, - { - "ts":1509843738, - "value":"inf" - }, - { - "ts":1509843753, - "value":"-inf" - } - ] - }, - { - "kind":"COUNTER", + }, + "timeseries": + [ + { + "ts":1509843723, + "value":"nan" + }, + { + "ts":1509843738, + "value":"inf" + }, + { + "ts":1509843753, + "value":"-inf" + } + ] + }, + { + "kind":"COUNTER", "labels": { "labels":"l6", "metric":"multiple" }, - "timeseries": - [ - { - "ts":1509843723, - "value":1337 - }, - { - "ts":1509843738, - "value":1338 - } + "timeseries": + [ + { + "ts":1509843723, + "value":1337 + }, + { + "ts":1509843738, + "value":1338 + } ] - } - ] -} + } + ] +} diff --git a/library/cpp/monlib/encode/json/ut/expected_cloud.json b/library/cpp/monlib/encode/json/ut/expected_cloud.json index 5c4dc65ffc7..61848115790 100644 --- a/library/cpp/monlib/encode/json/ut/expected_cloud.json +++ b/library/cpp/monlib/encode/json/ut/expected_cloud.json @@ -1,47 +1,47 @@ -{ +{ "ts":"2017-07-14T02:40:00.000000Z", "labels": - { - "project":"solomon" - }, + { + "project":"solomon" + }, "metrics": - [ - { + [ + { "type":"COUNTER", - "labels": - { + "labels": + { "labels":"l1" - }, + }, "name":"single", "ts":"2017-11-05T01:02:03.000000Z", - "value":17 - }, - { + "value":17 + }, + { "type":"RATE", - "labels": - { + "labels": + { "labels":"l2" - }, + }, "name":"single", "ts":"2017-11-05T01:02:03.000000Z", - "value":17 - }, - { + "value":17 + }, + { "type":"DGAUGE", - "labels": - { + "labels": + { "labels":"l3" - }, + }, "name":"single", "ts":"2017-11-05T01:02:03.000000Z", - "value":3.14 - }, - { + "value":3.14 + }, + { "type":"IGAUGE", - "labels": - { + "labels": + { "labels":"l4" - }, + }, "name":"single_igauge", "ts":"2017-11-05T01:02:03.000000Z", "value":42 @@ -53,40 +53,40 @@ "labels":"l5" }, "name":"multiple", - "timeseries": - [ - { + "timeseries": + [ + { "ts":"2017-11-05T01:02:03.000000Z", - "value":"nan" - }, - { + "value":"nan" + }, + { "ts":"2017-11-05T01:02:18.000000Z", - "value":"inf" - }, - { + "value":"inf" + }, + { "ts":"2017-11-05T01:02:33.000000Z", - "value":"-inf" - } - ] - }, - { + "value":"-inf" + } + ] + }, + { "type":"COUNTER", - "timeseries": - [ - { + "timeseries": + [ + { "ts":"2017-11-05T01:02:03.000000Z", - "value":1337 - }, - { + "value":1337 + }, + { "ts":"2017-11-05T01:02:18.000000Z", - "value":1338 - } - ], - "labels": - { + "value":1338 + } + ], + "labels": + { "labels":"l6" }, "name":"multiple" - } - ] -} + } + ] +} diff --git a/library/cpp/monlib/encode/json/ut/histogram_timeseries.json b/library/cpp/monlib/encode/json/ut/histogram_timeseries.json index c3a6f8e8b09..f6131ffded4 100644 --- a/library/cpp/monlib/encode/json/ut/histogram_timeseries.json +++ b/library/cpp/monlib/encode/json/ut/histogram_timeseries.json @@ -1,61 +1,61 @@ -{ - "sensors": - [ - { - "kind":"HIST_RATE", - "labels": - { - "metric":"responseTimeMillis" - }, - "timeseries": - [ - { - "ts":1509843723, - "hist": - { - "bounds": - [ - 1, - 2, - 4, - 8, - 16 - ], - "buckets": - [ - 1, - 1, - 2, - 4, - 8 - ], - "inf":83 - } - }, - { - "ts":1509843738, - "hist": - { - "bounds": - [ - 1, - 2, - 4, - 8, - 16 - ], - "buckets": - [ - 2, - 2, - 4, - 8, - 16 - ], - "inf":166 - } - } - ] - } - ] -} +{ + "sensors": + [ + { + "kind":"HIST_RATE", + "labels": + { + "metric":"responseTimeMillis" + }, + "timeseries": + [ + { + "ts":1509843723, + "hist": + { + "bounds": + [ + 1, + 2, + 4, + 8, + 16 + ], + "buckets": + [ + 1, + 1, + 2, + 4, + 8 + ], + "inf":83 + } + }, + { + "ts":1509843738, + "hist": + { + "bounds": + [ + 1, + 2, + 4, + 8, + 16 + ], + "buckets": + [ + 2, + 2, + 4, + 8, + 16 + ], + "inf":166 + } + } + ] + } + ] +} diff --git a/library/cpp/monlib/encode/json/ut/histogram_value.json b/library/cpp/monlib/encode/json/ut/histogram_value.json index 91b4ffa3dd4..ec1ae5cdec4 100644 --- a/library/cpp/monlib/encode/json/ut/histogram_value.json +++ b/library/cpp/monlib/encode/json/ut/histogram_value.json @@ -1,33 +1,33 @@ -{ - "sensors": - [ - { - "kind":"HIST", - "labels": - { - "metric":"responseTimeMillis" - }, - "ts":1509843723, - "hist": - { - "bounds": - [ - 1, - 2, - 4, - 8, - 16 - ], - "buckets": - [ - 1, - 1, - 2, - 4, - 8 - ], - "inf":83 - } - } - ] -} +{ + "sensors": + [ + { + "kind":"HIST", + "labels": + { + "metric":"responseTimeMillis" + }, + "ts":1509843723, + "hist": + { + "bounds": + [ + 1, + 2, + 4, + 8, + 16 + ], + "buckets": + [ + 1, + 1, + 2, + 4, + 8 + ], + "inf":83 + } + } + ] +} diff --git a/library/cpp/monlib/encode/json/ut/histogram_value_inf_before_bounds.json b/library/cpp/monlib/encode/json/ut/histogram_value_inf_before_bounds.json index 87dc5279851..f8a17c8831c 100644 --- a/library/cpp/monlib/encode/json/ut/histogram_value_inf_before_bounds.json +++ b/library/cpp/monlib/encode/json/ut/histogram_value_inf_before_bounds.json @@ -5,7 +5,7 @@ "kind":"HIST", "labels": { - "metric":"responseTimeMillis" + "metric":"responseTimeMillis" }, "ts":1509843723, "hist": diff --git a/library/cpp/monlib/encode/json/ut/int_gauge.json b/library/cpp/monlib/encode/json/ut/int_gauge.json index 95aa12aa52b..fbe57f873cc 100644 --- a/library/cpp/monlib/encode/json/ut/int_gauge.json +++ b/library/cpp/monlib/encode/json/ut/int_gauge.json @@ -1,31 +1,31 @@ -{ - "sensors": - [ - { - "kind":"IGAUGE", - "labels": - { - "metric":"a" - }, - "timeseries": - [ - { - "ts":1509843723, - "value":-9223372036854775808 - }, - { - "ts":1509843724, - "value":-1 - }, - { - "ts":1509843725, - "value":0 - }, - { - "ts":1509843726, - "value":9223372036854775807 - } - ] - } - ] -} +{ + "sensors": + [ + { + "kind":"IGAUGE", + "labels": + { + "metric":"a" + }, + "timeseries": + [ + { + "ts":1509843723, + "value":-9223372036854775808 + }, + { + "ts":1509843724, + "value":-1 + }, + { + "ts":1509843725, + "value":0 + }, + { + "ts":1509843726, + "value":9223372036854775807 + } + ] + } + ] +} diff --git a/library/cpp/monlib/encode/json/ut/log_histogram_timeseries.json b/library/cpp/monlib/encode/json/ut/log_histogram_timeseries.json index 1e2a4a113c9..e811a2cc579 100644 --- a/library/cpp/monlib/encode/json/ut/log_histogram_timeseries.json +++ b/library/cpp/monlib/encode/json/ut/log_histogram_timeseries.json @@ -5,7 +5,7 @@ "kind":"LOGHIST", "labels": { - "metric":"ms" + "metric":"ms" }, "timeseries": [ diff --git a/library/cpp/monlib/encode/json/ut/log_histogram_value.json b/library/cpp/monlib/encode/json/ut/log_histogram_value.json index 98352211163..002478293b2 100644 --- a/library/cpp/monlib/encode/json/ut/log_histogram_value.json +++ b/library/cpp/monlib/encode/json/ut/log_histogram_value.json @@ -5,7 +5,7 @@ "kind":"LOGHIST", "labels": { - "metric":"ms" + "metric":"ms" }, "ts":1509843723, "log_hist": diff --git a/library/cpp/monlib/encode/json/ut/merged.json b/library/cpp/monlib/encode/json/ut/merged.json index 15711471f01..ea2c99a33c0 100644 --- a/library/cpp/monlib/encode/json/ut/merged.json +++ b/library/cpp/monlib/encode/json/ut/merged.json @@ -5,7 +5,7 @@ "kind":"RATE", "labels": { - "metric":"hello", + "metric":"hello", "label":"world" }, "value":1 diff --git a/library/cpp/monlib/encode/json/ut/metrics.json b/library/cpp/monlib/encode/json/ut/metrics.json index 0dc423d5e97..2be4617d515 100644 --- a/library/cpp/monlib/encode/json/ut/metrics.json +++ b/library/cpp/monlib/encode/json/ut/metrics.json @@ -1,43 +1,43 @@ -{ - "labels": { - "project": "solomon", - "cluster": "man", - "service": "stockpile" - }, - "metrics": [ - { - "type": "DGAUGE", - "labels": { - "metric": "Memory" - }, - "value": 10 - }, - { - "type": "RATE", - "value": 1, - "labels": { "metric": "UserTime" } - }, - { - "type": "GAUGE", - "value": 3.14159, - "labels": { "export": "Oxygen", "metric": "QueueSize" }, - "ts": "2017-11-05T12:34:56.000Z", - "memOnly": true - }, - { - "type": "GAUGE", - "labels": { "metric": "Writes" }, - "timeseries": [ - { - "ts": "2017-08-28T12:32:11Z", - "value": -10 - }, - { - "value": 20, - "ts": 1503923187 - } - ] - } - ], - "ts": "2017-08-27T12:34:56Z" -} +{ + "labels": { + "project": "solomon", + "cluster": "man", + "service": "stockpile" + }, + "metrics": [ + { + "type": "DGAUGE", + "labels": { + "metric": "Memory" + }, + "value": 10 + }, + { + "type": "RATE", + "value": 1, + "labels": { "metric": "UserTime" } + }, + { + "type": "GAUGE", + "value": 3.14159, + "labels": { "export": "Oxygen", "metric": "QueueSize" }, + "ts": "2017-11-05T12:34:56.000Z", + "memOnly": true + }, + { + "type": "GAUGE", + "labels": { "metric": "Writes" }, + "timeseries": [ + { + "ts": "2017-08-28T12:32:11Z", + "value": -10 + }, + { + "value": 20, + "ts": 1503923187 + } + ] + } + ], + "ts": "2017-08-27T12:34:56Z" +} diff --git a/library/cpp/monlib/encode/json/ut/sensors.json b/library/cpp/monlib/encode/json/ut/sensors.json index e1285934920..4d979a3c1ed 100644 --- a/library/cpp/monlib/encode/json/ut/sensors.json +++ b/library/cpp/monlib/encode/json/ut/sensors.json @@ -1,40 +1,40 @@ -{ - "commonLabels": { - "project": "solomon", - "cluster": "man", - "service": "stockpile" - }, - "sensors": [ - { - "labels": { - "metric": "Memory" - }, - "value": 10 - }, - { - "mode": "deriv", - "value": 1, - "labels": { "metric": "UserTime" } - }, - { - "value": 3.14159, - "labels": { "export": "Oxygen", "metric": "QueueSize" }, - "ts": "2017-11-05T12:34:56.000Z", - "memOnly": true - }, - { - "labels": { "metric": "Writes" }, - "timeseries": [ - { - "ts": "2017-08-28T12:32:11Z", - "value": -10 - }, - { - "value": 20, - "ts": 1503923187 - } - ] - } - ], - "ts": "2017-08-27T12:34:56Z" -} +{ + "commonLabels": { + "project": "solomon", + "cluster": "man", + "service": "stockpile" + }, + "sensors": [ + { + "labels": { + "metric": "Memory" + }, + "value": 10 + }, + { + "mode": "deriv", + "value": 1, + "labels": { "metric": "UserTime" } + }, + { + "value": 3.14159, + "labels": { "export": "Oxygen", "metric": "QueueSize" }, + "ts": "2017-11-05T12:34:56.000Z", + "memOnly": true + }, + { + "labels": { "metric": "Writes" }, + "timeseries": [ + { + "ts": "2017-08-28T12:32:11Z", + "value": -10 + }, + { + "value": 20, + "ts": 1503923187 + } + ] + } + ], + "ts": "2017-08-27T12:34:56Z" +} diff --git a/library/cpp/monlib/encode/json/ut/summary_inf.json b/library/cpp/monlib/encode/json/ut/summary_inf.json index 067102b7dad..625a6cd8ad8 100644 --- a/library/cpp/monlib/encode/json/ut/summary_inf.json +++ b/library/cpp/monlib/encode/json/ut/summary_inf.json @@ -5,7 +5,7 @@ "kind":"DSUMMARY", "labels": { - "metric":"temperature" + "metric":"temperature" }, "ts":1509843723, "summary": diff --git a/library/cpp/monlib/encode/json/ut/summary_timeseries.json b/library/cpp/monlib/encode/json/ut/summary_timeseries.json index dd06ec85449..92007af3e66 100644 --- a/library/cpp/monlib/encode/json/ut/summary_timeseries.json +++ b/library/cpp/monlib/encode/json/ut/summary_timeseries.json @@ -5,7 +5,7 @@ "kind":"DSUMMARY", "labels": { - "metric":"temperature" + "metric":"temperature" }, "timeseries": [ diff --git a/library/cpp/monlib/encode/json/ut/summary_value.json b/library/cpp/monlib/encode/json/ut/summary_value.json index 2ad9e9ad39e..366394c5e1f 100644 --- a/library/cpp/monlib/encode/json/ut/summary_value.json +++ b/library/cpp/monlib/encode/json/ut/summary_value.json @@ -5,7 +5,7 @@ "kind":"DSUMMARY", "labels": { - "metric":"temperature" + "metric":"temperature" }, "ts":1509843723, "summary": diff --git a/library/cpp/monlib/encode/json/ut/test_decode_to_encode.json b/library/cpp/monlib/encode/json/ut/test_decode_to_encode.json index db1e00c6e90..65f0c5c6e21 100644 --- a/library/cpp/monlib/encode/json/ut/test_decode_to_encode.json +++ b/library/cpp/monlib/encode/json/ut/test_decode_to_encode.json @@ -7,7 +7,7 @@ "sensors": [ { "kind": "GAUGE", - "labels": { "export": "Oxygen", "metric": "QueueSize" }, + "labels": { "export": "Oxygen", "metric": "QueueSize" }, "ts": 1509885296, "value": 3.14159 } diff --git a/library/cpp/monlib/encode/json/ut/ya.make b/library/cpp/monlib/encode/json/ut/ya.make index 4a24f76650a..e50c4f4903b 100644 --- a/library/cpp/monlib/encode/json/ut/ya.make +++ b/library/cpp/monlib/encode/json/ut/ya.make @@ -1,30 +1,30 @@ -UNITTEST_FOR(library/cpp/monlib/encode/json) - +UNITTEST_FOR(library/cpp/monlib/encode/json) + OWNER( g:solomon jamel ) - -SRCS( + +SRCS( json_decoder_ut.cpp - json_ut.cpp -) - -RESOURCE( + json_ut.cpp +) + +RESOURCE( buffered_test.json /buffered_test.json buffered_ts_merge.json /buffered_ts_merge.json - empty_series.json /empty_series.json - expected.json /expected.json + empty_series.json /empty_series.json + expected.json /expected.json expected_buffered.json /expected_buffered.json expected_cloud.json /expected_cloud.json expected_cloud_buffered.json /expected_cloud_buffered.json merged.json /merged.json - histogram_timeseries.json /histogram_timeseries.json - histogram_value.json /histogram_value.json + histogram_timeseries.json /histogram_timeseries.json + histogram_value.json /histogram_value.json histogram_value_inf_before_bounds.json /histogram_value_inf_before_bounds.json - int_gauge.json /int_gauge.json - sensors.json /sensors.json - metrics.json /metrics.json + int_gauge.json /int_gauge.json + sensors.json /sensors.json + metrics.json /metrics.json named_metrics.json /named_metrics.json test_decode_to_encode.json /test_decode_to_encode.json crash.json /crash.json @@ -34,13 +34,13 @@ RESOURCE( summary_timeseries.json /summary_timeseries.json log_histogram_value.json /log_histogram_value.json log_histogram_timeseries.json /log_histogram_timeseries.json -) - -PEERDIR( +) + +PEERDIR( library/cpp/json library/cpp/monlib/consumers - library/cpp/monlib/encode/protobuf + library/cpp/monlib/encode/protobuf library/cpp/resource -) - -END() +) + +END() diff --git a/library/cpp/monlib/encode/json/ya.make b/library/cpp/monlib/encode/json/ya.make index 221fbc2a6ab..a50fc412a99 100644 --- a/library/cpp/monlib/encode/json/ya.make +++ b/library/cpp/monlib/encode/json/ya.make @@ -1,21 +1,21 @@ -LIBRARY() - +LIBRARY() + OWNER( g:solomon jamel ) - -SRCS( - json_decoder.cpp - json_encoder.cpp -) - -PEERDIR( + +SRCS( + json_decoder.cpp + json_encoder.cpp +) + +PEERDIR( library/cpp/monlib/encode library/cpp/monlib/encode/buffered library/cpp/monlib/exception library/cpp/json library/cpp/json/writer -) - -END() +) + +END() diff --git a/library/cpp/monlib/encode/legacy_protobuf/legacy_proto_decoder.cpp b/library/cpp/monlib/encode/legacy_protobuf/legacy_proto_decoder.cpp index 4a305152e3f..f87a2d7e8f4 100644 --- a/library/cpp/monlib/encode/legacy_protobuf/legacy_proto_decoder.cpp +++ b/library/cpp/monlib/encode/legacy_protobuf/legacy_proto_decoder.cpp @@ -1,8 +1,8 @@ #include "legacy_protobuf.h" -#include <library/cpp/monlib/encode/legacy_protobuf/protos/metric_meta.pb.h> -#include <library/cpp/monlib/metrics/metric_consumer.h> -#include <library/cpp/monlib/metrics/labels.h> +#include <library/cpp/monlib/encode/legacy_protobuf/protos/metric_meta.pb.h> +#include <library/cpp/monlib/metrics/metric_consumer.h> +#include <library/cpp/monlib/metrics/labels.h> #include <util/generic/yexception.h> #include <util/generic/maybe.h> @@ -22,7 +22,7 @@ namespace NMonitoring { namespace { - using TMaybeMeta = TMaybe<NMonProto::TMetricMeta>; + using TMaybeMeta = TMaybe<NMonProto::TMetricMeta>; TString ReadLabelValue(const NProtoBuf::Message& msg, const NProtoBuf::FieldDescriptor* d, const NProtoBuf::Reflection& r) { using namespace NProtoBuf; @@ -137,8 +137,8 @@ namespace NMonitoring { } TMaybeMeta MaybeGetMeta(const NProtoBuf::FieldOptions& opts) { - if (opts.HasExtension(NMonProto::Metric)) { - return opts.GetExtension(NMonProto::Metric); + if (opts.HasExtension(NMonProto::Metric)) { + return opts.GetExtension(NMonProto::Metric); } return Nothing(); @@ -218,37 +218,37 @@ namespace NMonitoring { Reflection_ = nullptr; } - TDecoderContext CreateChildFromMeta(const NMonProto::TMetricMeta& metricMeta, const TString& name, i64 repeatedIdx = -1) { + TDecoderContext CreateChildFromMeta(const NMonProto::TMetricMeta& metricMeta, const TString& name, i64 repeatedIdx = -1) { TDecoderContext child{*this}; child.Clear(); - if (metricMeta.HasCustomPath()) { - if (const auto& nodePath = metricMeta.GetCustomPath()) { + if (metricMeta.HasCustomPath()) { + if (const auto& nodePath = metricMeta.GetCustomPath()) { child.AppendPath(nodePath); } - } else if (metricMeta.GetPath()) { + } else if (metricMeta.GetPath()) { child.AppendPath(name); } - if (metricMeta.HasKeys()) { - child.ParseKeys(metricMeta.GetKeys(), repeatedIdx); + if (metricMeta.HasKeys()) { + child.ParseKeys(metricMeta.GetKeys(), repeatedIdx); } return child; } - TDecoderContext CreateChildFromRepeatedScalar(const NMonProto::TMetricMeta& metricMeta, i64 repeatedIdx = -1) { + TDecoderContext CreateChildFromRepeatedScalar(const NMonProto::TMetricMeta& metricMeta, i64 repeatedIdx = -1) { TDecoderContext child{*this}; child.Clear(); - if (metricMeta.HasKeys()) { - child.ParseKeys(metricMeta.GetKeys(), repeatedIdx); + if (metricMeta.HasKeys()) { + child.ParseKeys(metricMeta.GetKeys(), repeatedIdx); } return child; } - TDecoderContext CreateChildFromEls(const TString& name, const NMonProto::TExtraLabelMetrics& metrics, size_t idx, TMaybeMeta maybeMeta) { + TDecoderContext CreateChildFromEls(const TString& name, const NMonProto::TExtraLabelMetrics& metrics, size_t idx, TMaybeMeta maybeMeta) { TDecoderContext child{*this}; child.Clear(); @@ -261,8 +261,8 @@ namespace NMonitoring { } child.Labels_.push_back(::MakeIntrusive<TLazyLabel>( - [ labelName = metrics.GetlabelName(), idx, &metrics ](const auto&) { - const auto& val = metrics.Getvalues(idx); + [ labelName = metrics.GetlabelName(), idx, &metrics ](const auto&) { + const auto& val = metrics.Getvalues(idx); TString labelVal; const auto uintLabel = val.GetlabelValueUint(); @@ -356,7 +356,7 @@ namespace NMonitoring { class TDecoder { public: - TDecoder(IMetricConsumer* consumer, const NProtoBuf::Message& message, TInstant timestamp) + TDecoder(IMetricConsumer* consumer, const NProtoBuf::Message& message, TInstant timestamp) : Consumer_{consumer} , Message_{message} , Timestamp_{timestamp} @@ -374,12 +374,12 @@ namespace NMonitoring { } private: - static const NMonProto::TExtraLabelMetrics& ExtractExtraMetrics(TDecoderContext& ctx, const NProtoBuf::FieldDescriptor& f) { + static const NMonProto::TExtraLabelMetrics& ExtractExtraMetrics(TDecoderContext& ctx, const NProtoBuf::FieldDescriptor& f) { const auto& parent = ctx.Message(); const auto& reflection = ctx.Reflection(); auto& subMessage = reflection.GetMessage(parent, &f); - return dynamic_cast<const NMonProto::TExtraLabelMetrics&>(subMessage); + return dynamic_cast<const NMonProto::TExtraLabelMetrics&>(subMessage); } void DecodeImpl(const NProtoBuf::Message& msg, TDecoderContext ctx) const { @@ -394,68 +394,68 @@ namespace NMonitoring { const auto& opts = f->options(); const auto isMessage = f->type() == NProtoBuf::FieldDescriptor::TYPE_MESSAGE; - const auto isExtraLabelMetrics = isMessage && f->message_type()->full_name() == "NMonProto.TExtraLabelMetrics"; + const auto isExtraLabelMetrics = isMessage && f->message_type()->full_name() == "NMonProto.TExtraLabelMetrics"; const auto maybeMeta = MaybeGetMeta(opts); - if (!(maybeMeta || isExtraLabelMetrics)) { + if (!(maybeMeta || isExtraLabelMetrics)) { continue; } - if (isExtraLabelMetrics) { - const auto& extra = ExtractExtraMetrics(ctx, *f); - RecurseExtraLabelMetrics(ctx, extra, f->name(), maybeMeta); + if (isExtraLabelMetrics) { + const auto& extra = ExtractExtraMetrics(ctx, *f); + RecurseExtraLabelMetrics(ctx, extra, f->name(), maybeMeta); } else if (isMessage) { RecurseMessage(ctx, *maybeMeta, *f); } else if (f->is_repeated()) { RecurseRepeatedScalar(ctx, *maybeMeta, *f); - } else if (maybeMeta->HasType()) { + } else if (maybeMeta->HasType()) { const auto val = ReadFieldAsDouble(msg, f, ctx.Reflection()); - const bool isRate = maybeMeta->GetType() == NMonProto::EMetricType::RATE; - WriteMetric(val, ctx, f->name(), isRate); + const bool isRate = maybeMeta->GetType() == NMonProto::EMetricType::RATE; + WriteMetric(val, ctx, f->name(), isRate); } } } - void RecurseRepeatedScalar(TDecoderContext ctx, const NMonProto::TMetricMeta& meta, const NProtoBuf::FieldDescriptor& f) const { + void RecurseRepeatedScalar(TDecoderContext ctx, const NMonProto::TMetricMeta& meta, const NProtoBuf::FieldDescriptor& f) const { auto&& msg = ctx.Message(); auto&& reflection = ctx.Reflection(); - const bool isRate = meta.GetType() == NMonProto::EMetricType::RATE; + const bool isRate = meta.GetType() == NMonProto::EMetricType::RATE; - // this is a repeated scalar field, which makes metric only if it's indexing + // this is a repeated scalar field, which makes metric only if it's indexing for (auto i = 0; i < reflection.FieldSize(msg, &f); ++i) { auto subCtx = ctx.CreateChildFromRepeatedScalar(meta, i); subCtx.Init(&msg); auto val = ReadRepeatedAsDouble(msg, &f, reflection, i); - WriteMetric(val, subCtx, f.name(), isRate); + WriteMetric(val, subCtx, f.name(), isRate); } } - void RecurseExtraLabelMetrics(TDecoderContext ctx, const NMonProto::TExtraLabelMetrics& msg, const TString& name, const TMaybeMeta& meta) const { + void RecurseExtraLabelMetrics(TDecoderContext ctx, const NMonProto::TExtraLabelMetrics& msg, const TString& name, const TMaybeMeta& meta) const { auto i = 0; for (const auto& val : msg.Getvalues()) { auto subCtx = ctx.CreateChildFromEls(name, msg, i++, meta); subCtx.Init(&val); - const bool isRate = val.Hastype() - ? val.Gettype() == NMonProto::EMetricType::RATE - : meta->GetType() == NMonProto::EMetricType::RATE; + const bool isRate = val.Hastype() + ? val.Gettype() == NMonProto::EMetricType::RATE + : meta->GetType() == NMonProto::EMetricType::RATE; - double metricVal{0}; - if (isRate) { - metricVal = val.GetlongValue(); + double metricVal{0}; + if (isRate) { + metricVal = val.GetlongValue(); } else { - metricVal = val.GetdoubleValue(); + metricVal = val.GetdoubleValue(); } - WriteMetric(metricVal, subCtx, "", isRate); + WriteMetric(metricVal, subCtx, "", isRate); for (const auto& child : val.Getchildren()) { - RecurseExtraLabelMetrics(subCtx, child, "", meta); + RecurseExtraLabelMetrics(subCtx, child, "", meta); } } } - void RecurseMessage(TDecoderContext ctx, const NMonProto::TMetricMeta& metricMeta, const NProtoBuf::FieldDescriptor& f) const { + void RecurseMessage(TDecoderContext ctx, const NMonProto::TMetricMeta& metricMeta, const NProtoBuf::FieldDescriptor& f) const { const auto& msg = ctx.Message(); const auto& reflection = ctx.Reflection(); @@ -463,12 +463,12 @@ namespace NMonitoring { TRACE("recurse into repeated message " << f.name()); for (auto i = 0; i < reflection.FieldSize(msg, &f); ++i) { auto& subMessage = reflection.GetRepeatedMessage(msg, &f, i); - DecodeImpl(subMessage, ctx.CreateChildFromMeta(metricMeta, f.name(), i)); + DecodeImpl(subMessage, ctx.CreateChildFromMeta(metricMeta, f.name(), i)); } } else { TRACE("recurse into message " << f.name()); auto& subMessage = reflection.GetMessage(msg, &f); - DecodeImpl(subMessage, ctx.CreateChildFromMeta(metricMeta, f.name())); + DecodeImpl(subMessage, ctx.CreateChildFromMeta(metricMeta, f.name())); } } @@ -480,12 +480,12 @@ namespace NMonitoring { Consumer_->OnDouble(Timestamp_, value); } - void WriteMetric(double value, const TDecoderContext& ctx, const TString& name, bool isRate) const { - if (isRate) { - Consumer_->OnMetricBegin(EMetricType::RATE); + void WriteMetric(double value, const TDecoderContext& ctx, const TString& name, bool isRate) const { + if (isRate) { + Consumer_->OnMetricBegin(EMetricType::RATE); WriteValue(static_cast<ui64>(value)); } else { - Consumer_->OnMetricBegin(EMetricType::GAUGE); + Consumer_->OnMetricBegin(EMetricType::GAUGE); WriteValue(static_cast<double>(value)); } @@ -504,23 +504,23 @@ namespace NMonitoring { } Consumer_->OnLabelsEnd(); - Consumer_->OnMetricEnd(); + Consumer_->OnMetricEnd(); } private: - IMetricConsumer* Consumer_{nullptr}; + IMetricConsumer* Consumer_{nullptr}; const NProtoBuf::Message& Message_; TInstant Timestamp_; }; } - void DecodeLegacyProto(const NProtoBuf::Message& data, IMetricConsumer* consumer, TInstant ts) { + void DecodeLegacyProto(const NProtoBuf::Message& data, IMetricConsumer* consumer, TInstant ts) { Y_ENSURE(consumer); TDecoder(consumer, data, ts).Decode(); } - void DecodeLegacyProtoToStream(const NProtoBuf::Message& data, IMetricConsumer* consumer, TInstant ts) { + void DecodeLegacyProtoToStream(const NProtoBuf::Message& data, IMetricConsumer* consumer, TInstant ts) { Y_ENSURE(consumer); TDecoder(consumer, data, ts).DecodeToStream(); } diff --git a/library/cpp/monlib/encode/legacy_protobuf/legacy_protobuf.h b/library/cpp/monlib/encode/legacy_protobuf/legacy_protobuf.h index 50f65bf2c88..7cf8985d656 100644 --- a/library/cpp/monlib/encode/legacy_protobuf/legacy_protobuf.h +++ b/library/cpp/monlib/encode/legacy_protobuf/legacy_protobuf.h @@ -9,8 +9,8 @@ namespace NMonitoring { // - memOnly; // - dropHost/ignorePath - void DecodeLegacyProto(const NProtoBuf::Message& data, class IMetricConsumer* c, TInstant ts = TInstant::Zero()); + void DecodeLegacyProto(const NProtoBuf::Message& data, class IMetricConsumer* c, TInstant ts = TInstant::Zero()); /// Does not open/close consumer stream unlike the above function. - void DecodeLegacyProtoToStream(const NProtoBuf::Message& data, class IMetricConsumer* c, TInstant ts = TInstant::Zero()); + void DecodeLegacyProtoToStream(const NProtoBuf::Message& data, class IMetricConsumer* c, TInstant ts = TInstant::Zero()); } diff --git a/library/cpp/monlib/encode/legacy_protobuf/legacy_protobuf_ut.cpp b/library/cpp/monlib/encode/legacy_protobuf/legacy_protobuf_ut.cpp index a751f8fb140..53683cb39c6 100644 --- a/library/cpp/monlib/encode/legacy_protobuf/legacy_protobuf_ut.cpp +++ b/library/cpp/monlib/encode/legacy_protobuf/legacy_protobuf_ut.cpp @@ -2,12 +2,12 @@ #include <library/cpp/testing/unittest/registar.h> -#include <library/cpp/monlib/encode/legacy_protobuf/ut/test_cases.pb.h> -#include <library/cpp/monlib/encode/legacy_protobuf/protos/metric_meta.pb.h> +#include <library/cpp/monlib/encode/legacy_protobuf/ut/test_cases.pb.h> +#include <library/cpp/monlib/encode/legacy_protobuf/protos/metric_meta.pb.h> -#include <library/cpp/monlib/encode/protobuf/protobuf.h> -#include <library/cpp/monlib/encode/text/text.h> -#include <library/cpp/monlib/metrics/labels.h> +#include <library/cpp/monlib/encode/protobuf/protobuf.h> +#include <library/cpp/monlib/encode/text/text.h> +#include <library/cpp/monlib/metrics/labels.h> #include <util/generic/algorithm.h> #include <util/generic/hash_set.h> @@ -24,7 +24,7 @@ TSimple MakeSimpleMessage() { return msg; } -IMetricEncoderPtr debugPrinter = EncoderText(&Cerr); +IMetricEncoderPtr debugPrinter = EncoderText(&Cerr); namespace NMonitoring { inline bool operator<(const TLabel& lhs, const TLabel& rhs) { @@ -34,20 +34,20 @@ namespace NMonitoring { } -void SetLabelValue(NMonProto::TExtraLabelMetrics::TValue& val, TString s) { +void SetLabelValue(NMonProto::TExtraLabelMetrics::TValue& val, TString s) { val.SetlabelValue(s); } -void SetLabelValue(NMonProto::TExtraLabelMetrics::TValue& val, ui64 u) { +void SetLabelValue(NMonProto::TExtraLabelMetrics::TValue& val, ui64 u) { val.SetlabelValueUint(u); } template <typename T, typename V> -NMonProto::TExtraLabelMetrics MakeExtra(TString labelName, V labelValue, T value, bool isDeriv) { - NMonProto::TExtraLabelMetrics metric; - auto* val = metric.Addvalues(); +NMonProto::TExtraLabelMetrics MakeExtra(TString labelName, V labelValue, T value, bool isDeriv) { + NMonProto::TExtraLabelMetrics metric; + auto* val = metric.Addvalues(); - metric.SetlabelName(labelName); + metric.SetlabelName(labelName); SetLabelValue(*val, labelValue); if (isDeriv) { @@ -56,7 +56,7 @@ NMonProto::TExtraLabelMetrics MakeExtra(TString labelName, V labelValue, T value val->SetdoubleValue(value); } - return metric; + return metric; } void AssertLabels(const TLabels& expected, const NProto::TMultiSample& actual) { @@ -100,13 +100,13 @@ void AssertSimpleMessage(const NProto::TMultiSamplesList& samples, TString pathP UNIT_ASSERT(expectedValues.contains(labelVal)); if (labelVal == pathPrefix + "Foo") { - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); UNIT_ASSERT_DOUBLES_EQUAL(s.GetPoints(0).GetFloat64(), 1, 1e-6); } else if (labelVal == pathPrefix + "Bar") { - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); UNIT_ASSERT_DOUBLES_EQUAL(s.GetPoints(0).GetFloat64(), 2, 1e-6); } else if (labelVal == pathPrefix + "Baz") { - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::RATE); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::RATE); UNIT_ASSERT_EQUAL(s.GetPoints(0).GetUint64(), 42); } } @@ -115,7 +115,7 @@ void AssertSimpleMessage(const NProto::TMultiSamplesList& samples, TString pathP Y_UNIT_TEST_SUITE(TLegacyProtoDecoderTest) { Y_UNIT_TEST(SimpleProto) { NProto::TMultiSamplesList samples; - IMetricEncoderPtr e = EncoderProtobuf(&samples); + IMetricEncoderPtr e = EncoderProtobuf(&samples); auto msg = MakeSimpleMessage(); DecodeLegacyProto(msg, e.Get()); @@ -125,7 +125,7 @@ Y_UNIT_TEST_SUITE(TLegacyProtoDecoderTest) { Y_UNIT_TEST(RepeatedProto) { NProto::TMultiSamplesList samples; - IMetricEncoderPtr e = EncoderProtobuf(&samples); + IMetricEncoderPtr e = EncoderProtobuf(&samples); auto simple = MakeSimpleMessage(); TRepeated msg; @@ -138,7 +138,7 @@ Y_UNIT_TEST_SUITE(TLegacyProtoDecoderTest) { Y_UNIT_TEST(RepeatedProtoWithPath) { NProto::TMultiSamplesList samples; - IMetricEncoderPtr e = EncoderProtobuf(&samples); + IMetricEncoderPtr e = EncoderProtobuf(&samples); auto simple = MakeSimpleMessage(); TRepeatedWithPath msg; @@ -151,7 +151,7 @@ Y_UNIT_TEST_SUITE(TLegacyProtoDecoderTest) { Y_UNIT_TEST(DeepNesting) { NProto::TMultiSamplesList samples; - IMetricEncoderPtr e = EncoderProtobuf(&samples); + IMetricEncoderPtr e = EncoderProtobuf(&samples); auto simple = MakeSimpleMessage(); TRepeatedWithPath internal; @@ -167,7 +167,7 @@ Y_UNIT_TEST_SUITE(TLegacyProtoDecoderTest) { Y_UNIT_TEST(Keys) { NProto::TMultiSamplesList samples; - IMetricEncoderPtr e = EncoderProtobuf(&samples); + IMetricEncoderPtr e = EncoderProtobuf(&samples); auto simple = MakeSimpleMessage(); simple.SetLabel("my_label_value"); @@ -206,7 +206,7 @@ Y_UNIT_TEST_SUITE(TLegacyProtoDecoderTest) { Y_UNIT_TEST(NonStringKeys) { NProto::TMultiSamplesList samples; - IMetricEncoderPtr e = EncoderProtobuf(&samples); + IMetricEncoderPtr e = EncoderProtobuf(&samples); TNonStringKeys msg; msg.SetFoo(42); @@ -241,7 +241,7 @@ Y_UNIT_TEST_SUITE(TLegacyProtoDecoderTest) { Y_UNIT_TEST(KeysFromNonLeafNodes) { NProto::TMultiSamplesList samples; - IMetricEncoderPtr e = EncoderProtobuf(&samples); + IMetricEncoderPtr e = EncoderProtobuf(&samples); auto simple = MakeSimpleMessage(); simple.SetLabel("label_value"); @@ -260,7 +260,7 @@ Y_UNIT_TEST_SUITE(TLegacyProtoDecoderTest) { Y_UNIT_TEST(SpacesAreGetReplaced) { NProto::TMultiSamplesList samples; - IMetricEncoderPtr e = EncoderProtobuf(&samples); + IMetricEncoderPtr e = EncoderProtobuf(&samples); auto simple = MakeSimpleMessage(); simple.SetLabel("my label_value"); @@ -288,7 +288,7 @@ Y_UNIT_TEST_SUITE(TLegacyProtoDecoderTest) { Y_UNIT_TEST(ExtraLabels) { NProto::TMultiSamplesList samples; - IMetricEncoderPtr e = EncoderProtobuf(&samples); + IMetricEncoderPtr e = EncoderProtobuf(&samples); TExtraLabels msg; msg.MutableExtraAsIs()->CopyFrom(MakeExtra("label", "foo", 42, false)); @@ -318,14 +318,14 @@ Y_UNIT_TEST_SUITE(TLegacyProtoDecoderTest) { Y_UNIT_TEST(NestedExtraLabels) { NProto::TMultiSamplesList samples; - IMetricEncoderPtr e = EncoderProtobuf(&samples); + IMetricEncoderPtr e = EncoderProtobuf(&samples); TExtraLabels msg; auto extra = MakeExtra("label", "foo", 42, false); auto* val = extra.Mutablevalues(0); { auto child = MakeExtra("child1", "label1", 24, true); - child.Mutablevalues(0)->Settype(NMonProto::EMetricType::RATE); + child.Mutablevalues(0)->Settype(NMonProto::EMetricType::RATE); val->Addchildren()->CopyFrom(child); } @@ -368,7 +368,7 @@ Y_UNIT_TEST_SUITE(TLegacyProtoDecoderTest) { Y_UNIT_TEST(RobotLabels) { NProto::TMultiSamplesList samples; - IMetricEncoderPtr e = EncoderProtobuf(&samples); + IMetricEncoderPtr e = EncoderProtobuf(&samples); TNamedCounter responses; responses.SetName("responses"); @@ -397,7 +397,7 @@ Y_UNIT_TEST_SUITE(TLegacyProtoDecoderTest) { Y_UNIT_TEST(ZoraLabels) { NProto::TMultiSamplesList samples; - IMetricEncoderPtr e = EncoderProtobuf(&samples); + IMetricEncoderPtr e = EncoderProtobuf(&samples); TTimeLogHist hist; hist.AddBuckets(42); diff --git a/library/cpp/monlib/encode/legacy_protobuf/protos/metric_meta.proto b/library/cpp/monlib/encode/legacy_protobuf/protos/metric_meta.proto index a8c5a2b93d9..fd23eb372bc 100644 --- a/library/cpp/monlib/encode/legacy_protobuf/protos/metric_meta.proto +++ b/library/cpp/monlib/encode/legacy_protobuf/protos/metric_meta.proto @@ -1,23 +1,23 @@ import "google/protobuf/descriptor.proto"; -package NMonProto; +package NMonProto; -option java_package = "ru.yandex.monlib.proto"; -option java_outer_classname = "MetricMetaProto"; +option java_package = "ru.yandex.monlib.proto"; +option java_outer_classname = "MetricMetaProto"; -enum EMetricType { - GAUGE = 1; - RATE = 2; +enum EMetricType { + GAUGE = 1; + RATE = 2; } -enum EMemOnly { +enum EMemOnly { DEFAULT = 0; STORE = 1; MEM_ONLY = 2; } -message TMetricMeta { - optional EMetricType Type = 1; +message TMetricMeta { + optional EMetricType Type = 1; optional bool Path = 2; optional string Keys = 3; optional bool MemOnly = 4; @@ -45,7 +45,7 @@ message THistogram { } // field of this type is recognized by Solomon -message TExtraLabelMetrics { +message TExtraLabelMetrics { optional string labelName = 1; message TValue { @@ -57,17 +57,17 @@ message TExtraLabelMetrics { optional double doubleValue = 3; optional THistogram histogramValue = 4; - optional EMetricType type = 7; - optional EMemOnly memOnly = 8; + optional EMetricType type = 7; + optional EMemOnly memOnly = 8; optional bool dropHost = 9; - repeated TExtraLabelMetrics children = 17; + repeated TExtraLabelMetrics children = 17; } repeated TValue values = 2; } extend google.protobuf.FieldOptions { - optional TMetricMeta Metric = 1719; + optional TMetricMeta Metric = 1719; } diff --git a/library/cpp/monlib/encode/legacy_protobuf/protos/python/ya.make b/library/cpp/monlib/encode/legacy_protobuf/protos/python/ya.make index ea96eaa9501..095b307b014 100644 --- a/library/cpp/monlib/encode/legacy_protobuf/protos/python/ya.make +++ b/library/cpp/monlib/encode/legacy_protobuf/protos/python/ya.make @@ -1,3 +1,3 @@ OWNER(g:solomon) -PY_PROTOS_FOR(library/cpp/monlib/encode/legacy_protobuf/protos) +PY_PROTOS_FOR(library/cpp/monlib/encode/legacy_protobuf/protos) diff --git a/library/cpp/monlib/encode/legacy_protobuf/protos/ya.make b/library/cpp/monlib/encode/legacy_protobuf/protos/ya.make index b5f1098a48d..489f361ab1b 100644 --- a/library/cpp/monlib/encode/legacy_protobuf/protos/ya.make +++ b/library/cpp/monlib/encode/legacy_protobuf/protos/ya.make @@ -3,7 +3,7 @@ PROTO_LIBRARY() OWNER(g:solomon) SRCS( - metric_meta.proto + metric_meta.proto ) IF (NOT PY_PROTOS_FOR) diff --git a/library/cpp/monlib/encode/legacy_protobuf/ut/test_cases.proto b/library/cpp/monlib/encode/legacy_protobuf/ut/test_cases.proto index c51d870e074..37e901de48d 100644 --- a/library/cpp/monlib/encode/legacy_protobuf/ut/test_cases.proto +++ b/library/cpp/monlib/encode/legacy_protobuf/ut/test_cases.proto @@ -1,26 +1,26 @@ -import "library/cpp/monlib/encode/legacy_protobuf/protos/metric_meta.proto"; +import "library/cpp/monlib/encode/legacy_protobuf/protos/metric_meta.proto"; message TSimple { - optional uint64 Foo = 1 [ (NMonProto.Metric).Type = GAUGE ]; - optional double Bar = 2 [ (NMonProto.Metric).Type = GAUGE ]; - optional double Baz = 3 [ (NMonProto.Metric).Type = RATE ]; + optional uint64 Foo = 1 [ (NMonProto.Metric).Type = GAUGE ]; + optional double Bar = 2 [ (NMonProto.Metric).Type = GAUGE ]; + optional double Baz = 3 [ (NMonProto.Metric).Type = RATE ]; optional string Label = 4; } message TRepeated { - repeated TSimple Messages = 1 [ (NMonProto.Metric).Path = false ]; + repeated TSimple Messages = 1 [ (NMonProto.Metric).Path = false ]; }; message TRepeatedWithPath { - repeated TSimple Namespace = 1 [ (NMonProto.Metric).Path = true ]; + repeated TSimple Namespace = 1 [ (NMonProto.Metric).Path = true ]; }; message TNestedWithKeys { - repeated TSimple Namespace = 1 [ (NMonProto.Metric).Path = true, (NMonProto.Metric).Keys = "my_label:Label fixed_label=fixed_value numbered:#" ]; + repeated TSimple Namespace = 1 [ (NMonProto.Metric).Path = true, (NMonProto.Metric).Keys = "my_label:Label fixed_label=fixed_value numbered:#" ]; }; message TDeepNesting { - optional TRepeatedWithPath Nested = 1 [ (NMonProto.Metric).Path = false ]; + optional TRepeatedWithPath Nested = 1 [ (NMonProto.Metric).Path = false ]; }; enum EEnum { @@ -29,43 +29,43 @@ enum EEnum { }; message TNonStringKeys { - optional uint32 Foo = 1 [ (NMonProto.Metric).Type = GAUGE ]; + optional uint32 Foo = 1 [ (NMonProto.Metric).Type = GAUGE ]; optional EEnum Enum = 2; optional uint32 Int = 3; }; message TRepeatedNonStringKeys { - repeated TNonStringKeys Nested = 1 [ (NMonProto.Metric).Path = true, (NMonProto.Metric).Keys = "enum:Enum int:Int" ]; + repeated TNonStringKeys Nested = 1 [ (NMonProto.Metric).Path = true, (NMonProto.Metric).Keys = "enum:Enum int:Int" ]; }; message TExtraLabels { - optional NMonProto.TExtraLabelMetrics ExtraAsIs = 1 [ (NMonProto.Metric).Type = GAUGE ]; - optional NMonProto.TExtraLabelMetrics ExtraDeriv = 2 [ (NMonProto.Metric).Type = RATE ]; + optional NMonProto.TExtraLabelMetrics ExtraAsIs = 1 [ (NMonProto.Metric).Type = GAUGE ]; + optional NMonProto.TExtraLabelMetrics ExtraDeriv = 2 [ (NMonProto.Metric).Type = RATE ]; }; message TRepeatedWithName { optional string Name = 1; - repeated TSimple Nested = 2 [ (NMonProto.Metric).Path = true, (NMonProto.Metric).Keys = "my_label:Label" ]; + repeated TSimple Nested = 2 [ (NMonProto.Metric).Path = true, (NMonProto.Metric).Keys = "my_label:Label" ]; }; message TKeysFromNonLeaf { - repeated TRepeatedWithName Nested = 1 [ (NMonProto.Metric).Path = true, (NMonProto.Metric).Keys = "name:Name" ]; + repeated TRepeatedWithName Nested = 1 [ (NMonProto.Metric).Path = true, (NMonProto.Metric).Keys = "name:Name" ]; }; message TNamedCounter { optional string Name = 1; - optional uint64 Count = 2 [ (NMonProto.Metric).Type = RATE ]; + optional uint64 Count = 2 [ (NMonProto.Metric).Type = RATE ]; } message TCrawlerCounters { message TStatusCounters { - repeated TNamedCounter ZoraResponses = 3 [ (NMonProto.Metric).Path = true, (NMonProto.Metric).Keys = "ZoraResponse:Name" ]; - repeated TNamedCounter FetcherResponses = 4 [ (NMonProto.Metric).Path = true, (NMonProto.Metric).Keys = "SpiderResponse:Name" ]; - repeated TNamedCounter RotorResponses = 5 [ (NMonProto.Metric).Path = true, (NMonProto.Metric).Keys = "SpiderResponse:Name" ]; - repeated TNamedCounter PDFetchResponses = 6 [ (NMonProto.Metric).Path = true, (NMonProto.Metric).Keys = "PDFetchResponse:Name" ]; - repeated TNamedCounter CalcResponses = 7 [ (NMonProto.Metric).Path = true, (NMonProto.Metric).Keys = "CalcResponse:Name" ]; - repeated TNamedCounter Responses = 8 [ (NMonProto.Metric).Path = true, (NMonProto.Metric).Keys = "AggregatedResponse:Name" ]; + repeated TNamedCounter ZoraResponses = 3 [ (NMonProto.Metric).Path = true, (NMonProto.Metric).Keys = "ZoraResponse:Name" ]; + repeated TNamedCounter FetcherResponses = 4 [ (NMonProto.Metric).Path = true, (NMonProto.Metric).Keys = "SpiderResponse:Name" ]; + repeated TNamedCounter RotorResponses = 5 [ (NMonProto.Metric).Path = true, (NMonProto.Metric).Keys = "SpiderResponse:Name" ]; + repeated TNamedCounter PDFetchResponses = 6 [ (NMonProto.Metric).Path = true, (NMonProto.Metric).Keys = "PDFetchResponse:Name" ]; + repeated TNamedCounter CalcResponses = 7 [ (NMonProto.Metric).Path = true, (NMonProto.Metric).Keys = "CalcResponse:Name" ]; + repeated TNamedCounter Responses = 8 [ (NMonProto.Metric).Path = true, (NMonProto.Metric).Keys = "AggregatedResponse:Name" ]; } message TPolicyCounters { @@ -73,18 +73,18 @@ message TCrawlerCounters { optional string Name = 2; optional string Zone = 3; - optional TStatusCounters StatusCounters = 4 [ (NMonProto.Metric).Path = true ]; + optional TStatusCounters StatusCounters = 4 [ (NMonProto.Metric).Path = true ]; } optional string Component = 1; - repeated TPolicyCounters PoliciesCounters = 3 [ (NMonProto.Metric).Path = true, (NMonProto.Metric).Keys = "SubComponent:SubComponent Policy:Name Zone:Zone" ]; + repeated TPolicyCounters PoliciesCounters = 3 [ (NMonProto.Metric).Path = true, (NMonProto.Metric).Keys = "SubComponent:SubComponent Policy:Name Zone:Zone" ]; } message TTimeLogHist { optional uint32 MinBucketMillisec = 1; - repeated uint64 Buckets = 2 [ (NMonProto.Metric).Type = RATE, (NMonProto.Metric).Keys = "slot:#" ]; + repeated uint64 Buckets = 2 [ (NMonProto.Metric).Type = RATE, (NMonProto.Metric).Keys = "slot:#" ]; } message TKiwiCounters { - optional TTimeLogHist Times = 22 [ (NMonProto.Metric).Path = true ]; + optional TTimeLogHist Times = 22 [ (NMonProto.Metric).Path = true ]; } diff --git a/library/cpp/monlib/encode/legacy_protobuf/ut/ya.make b/library/cpp/monlib/encode/legacy_protobuf/ut/ya.make index fa9b4ebb1a5..479a0c46c91 100644 --- a/library/cpp/monlib/encode/legacy_protobuf/ut/ya.make +++ b/library/cpp/monlib/encode/legacy_protobuf/ut/ya.make @@ -1,4 +1,4 @@ -UNITTEST_FOR(library/cpp/monlib/encode/legacy_protobuf) +UNITTEST_FOR(library/cpp/monlib/encode/legacy_protobuf) OWNER( g:solomon @@ -11,8 +11,8 @@ SRCS( ) PEERDIR( - library/cpp/monlib/encode/protobuf - library/cpp/monlib/encode/text + library/cpp/monlib/encode/protobuf + library/cpp/monlib/encode/text ) END() diff --git a/library/cpp/monlib/encode/legacy_protobuf/ya.make b/library/cpp/monlib/encode/legacy_protobuf/ya.make index 57500325377..74c82aac933 100644 --- a/library/cpp/monlib/encode/legacy_protobuf/ya.make +++ b/library/cpp/monlib/encode/legacy_protobuf/ya.make @@ -10,7 +10,7 @@ SRCS( ) PEERDIR( - library/cpp/monlib/encode/legacy_protobuf/protos + library/cpp/monlib/encode/legacy_protobuf/protos ) END() diff --git a/library/cpp/monlib/encode/prometheus/fuzz/main.cpp b/library/cpp/monlib/encode/prometheus/fuzz/main.cpp index a9ed0afc45d..24bda2d32e3 100644 --- a/library/cpp/monlib/encode/prometheus/fuzz/main.cpp +++ b/library/cpp/monlib/encode/prometheus/fuzz/main.cpp @@ -1,18 +1,18 @@ -#include <library/cpp/monlib/encode/prometheus/prometheus.h> -#include <library/cpp/monlib/encode/fake/fake.h> - -#include <util/stream/mem.h> - - -extern "C" int LLVMFuzzerTestOneInput(const ui8* buf, size_t size) { - using namespace NMonitoring; - - try { - TStringBuf data(reinterpret_cast<const char*>(buf), size); - auto encoder = EncoderFake(); - DecodePrometheus(data, encoder.Get()); - } catch (...) { - } - - return 0; -} +#include <library/cpp/monlib/encode/prometheus/prometheus.h> +#include <library/cpp/monlib/encode/fake/fake.h> + +#include <util/stream/mem.h> + + +extern "C" int LLVMFuzzerTestOneInput(const ui8* buf, size_t size) { + using namespace NMonitoring; + + try { + TStringBuf data(reinterpret_cast<const char*>(buf), size); + auto encoder = EncoderFake(); + DecodePrometheus(data, encoder.Get()); + } catch (...) { + } + + return 0; +} diff --git a/library/cpp/monlib/encode/prometheus/fuzz/ya.make b/library/cpp/monlib/encode/prometheus/fuzz/ya.make index 0fc38457189..4a6c796ed53 100644 --- a/library/cpp/monlib/encode/prometheus/fuzz/ya.make +++ b/library/cpp/monlib/encode/prometheus/fuzz/ya.make @@ -1,16 +1,16 @@ -FUZZ() - -OWNER(g:solomon jamel) - -PEERDIR( - library/cpp/monlib/encode/prometheus - library/cpp/monlib/encode/fake -) - -SIZE(MEDIUM) - -SRCS( - main.cpp -) - -END() +FUZZ() + +OWNER(g:solomon jamel) + +PEERDIR( + library/cpp/monlib/encode/prometheus + library/cpp/monlib/encode/fake +) + +SIZE(MEDIUM) + +SRCS( + main.cpp +) + +END() diff --git a/library/cpp/monlib/encode/prometheus/prometheus.h b/library/cpp/monlib/encode/prometheus/prometheus.h index f031f5e9338..2e7fa31c28c 100644 --- a/library/cpp/monlib/encode/prometheus/prometheus.h +++ b/library/cpp/monlib/encode/prometheus/prometheus.h @@ -1,18 +1,18 @@ -#pragma once - -#include <library/cpp/monlib/encode/encoder.h> -#include <library/cpp/monlib/encode/format.h> - -#include <util/generic/yexception.h> - - -namespace NMonitoring { - - class TPrometheusDecodeException: public yexception { - }; - +#pragma once + +#include <library/cpp/monlib/encode/encoder.h> +#include <library/cpp/monlib/encode/format.h> + +#include <util/generic/yexception.h> + + +namespace NMonitoring { + + class TPrometheusDecodeException: public yexception { + }; + IMetricEncoderPtr EncoderPrometheus(IOutputStream* out, TStringBuf metricNameLabel = "sensor"); - + void DecodePrometheus(TStringBuf data, IMetricConsumer* c, TStringBuf metricNameLabel = "sensor"); - -} + +} diff --git a/library/cpp/monlib/encode/prometheus/prometheus_decoder.cpp b/library/cpp/monlib/encode/prometheus/prometheus_decoder.cpp index f11dd33e31b..7e81357dbda 100644 --- a/library/cpp/monlib/encode/prometheus/prometheus_decoder.cpp +++ b/library/cpp/monlib/encode/prometheus/prometheus_decoder.cpp @@ -1,51 +1,51 @@ -#include "prometheus.h" -#include "prometheus_model.h" - -#include <library/cpp/monlib/metrics/histogram_snapshot.h> -#include <library/cpp/monlib/metrics/metric.h> - -#include <util/datetime/base.h> -#include <util/generic/hash.h> -#include <util/string/cast.h> -#include <util/string/builder.h> -#include <util/generic/maybe.h> -#include <util/string/ascii.h> - -#include <cmath> - -#define Y_PARSER_FAIL(message) \ - ythrow ::NMonitoring::TPrometheusDecodeException() << message << " at line #" << CurrentLine_ - -#define Y_PARSER_ENSURE(cond, message) \ - Y_ENSURE_EX(cond, ::NMonitoring::TPrometheusDecodeException() << message << " at line #" << CurrentLine_) - - -namespace NMonitoring { - namespace { +#include "prometheus.h" +#include "prometheus_model.h" + +#include <library/cpp/monlib/metrics/histogram_snapshot.h> +#include <library/cpp/monlib/metrics/metric.h> + +#include <util/datetime/base.h> +#include <util/generic/hash.h> +#include <util/string/cast.h> +#include <util/string/builder.h> +#include <util/generic/maybe.h> +#include <util/string/ascii.h> + +#include <cmath> + +#define Y_PARSER_FAIL(message) \ + ythrow ::NMonitoring::TPrometheusDecodeException() << message << " at line #" << CurrentLine_ + +#define Y_PARSER_ENSURE(cond, message) \ + Y_ENSURE_EX(cond, ::NMonitoring::TPrometheusDecodeException() << message << " at line #" << CurrentLine_) + + +namespace NMonitoring { + namespace { constexpr ui32 MAX_LABEL_VALUE_LEN = 256; - - using TLabelsMap = THashMap<TString, TString>; - - TString LabelsToStr(const TLabelsMap& labels) { - TStringBuilder sb; - auto it = labels.begin(); - auto end = labels.end(); - - sb << '{'; - while (it != end) { - sb << it->first; - sb << '='; - sb << '"' << it->second << '"'; - - ++it; - if (it != end) { - sb << ", "; - } - } - sb << '}'; - return sb; - } - + + using TLabelsMap = THashMap<TString, TString>; + + TString LabelsToStr(const TLabelsMap& labels) { + TStringBuilder sb; + auto it = labels.begin(); + auto end = labels.end(); + + sb << '{'; + while (it != end) { + sb << it->first; + sb << '='; + sb << '"' << it->second << '"'; + + ++it; + if (it != end) { + sb << ", "; + } + } + sb << '}'; + return sb; + } + template <typename T, typename U> bool TryStaticCast(U val, T& out) { static_assert(std::is_arithmetic_v<U>); @@ -64,534 +64,534 @@ namespace NMonitoring { return true; } - /////////////////////////////////////////////////////////////////////// - // THistogramBuilder - /////////////////////////////////////////////////////////////////////// - class THistogramBuilder { - using TBucketData = std::pair<TBucketBound, TBucketValue>; - constexpr static TBucketData ZERO_BUCKET = { -std::numeric_limits<TBucketBound>::max(), 0 }; - public: - TStringBuf GetName() const noexcept { - return Name_; - } - - void SetName(TStringBuf name) noexcept { - Name_ = name; - } - - const TLabelsMap& GetLabels() const noexcept { - return *Labels_; - } - - void SetLabels(TLabelsMap&& labels) { - if (Labels_.Defined()) { - Y_ENSURE(Labels_ == labels, - "mixed labels in one histogram, prev: " << LabelsToStr(*Labels_) << - ", current: " << LabelsToStr(labels)); - } else { - Labels_.ConstructInPlace(std::move(labels)); - } - } - - TInstant GetTime() const noexcept { - return Time_; - } - - void SetTime(TInstant time) noexcept { - Time_ = time; - } - - bool Empty() const noexcept { - return Bounds_.empty(); - } - - bool Same(TStringBuf name, const TLabelsMap& labels) const noexcept { - return Name_ == name && Labels_ == labels; - } - - void AddBucket(TBucketBound bound, TBucketValue value) { - Y_ENSURE_EX(PrevBucket_.first < bound, TPrometheusDecodeException() << - "invalid order of histogram bounds " << PrevBucket_.first << - " >= " << bound); - - Y_ENSURE_EX(PrevBucket_.second <= value, TPrometheusDecodeException() << - "invalid order of histogram bucket values " << PrevBucket_.second << - " > " << value); - - // convert infinite bound value - if (bound == std::numeric_limits<TBucketBound>::infinity()) { - bound = HISTOGRAM_INF_BOUND; - } - - Bounds_.push_back(bound); - Values_.push_back(value - PrevBucket_.second); // keep only delta between buckets - - PrevBucket_ = { bound, value }; - } - - // will clear builder state - IHistogramSnapshotPtr ToSnapshot() { - Y_ENSURE_EX(!Empty(), TPrometheusDecodeException() << "histogram cannot be empty"); - Time_ = TInstant::Zero(); - PrevBucket_ = ZERO_BUCKET; - Labels_.Clear(); + /////////////////////////////////////////////////////////////////////// + // THistogramBuilder + /////////////////////////////////////////////////////////////////////// + class THistogramBuilder { + using TBucketData = std::pair<TBucketBound, TBucketValue>; + constexpr static TBucketData ZERO_BUCKET = { -std::numeric_limits<TBucketBound>::max(), 0 }; + public: + TStringBuf GetName() const noexcept { + return Name_; + } + + void SetName(TStringBuf name) noexcept { + Name_ = name; + } + + const TLabelsMap& GetLabels() const noexcept { + return *Labels_; + } + + void SetLabels(TLabelsMap&& labels) { + if (Labels_.Defined()) { + Y_ENSURE(Labels_ == labels, + "mixed labels in one histogram, prev: " << LabelsToStr(*Labels_) << + ", current: " << LabelsToStr(labels)); + } else { + Labels_.ConstructInPlace(std::move(labels)); + } + } + + TInstant GetTime() const noexcept { + return Time_; + } + + void SetTime(TInstant time) noexcept { + Time_ = time; + } + + bool Empty() const noexcept { + return Bounds_.empty(); + } + + bool Same(TStringBuf name, const TLabelsMap& labels) const noexcept { + return Name_ == name && Labels_ == labels; + } + + void AddBucket(TBucketBound bound, TBucketValue value) { + Y_ENSURE_EX(PrevBucket_.first < bound, TPrometheusDecodeException() << + "invalid order of histogram bounds " << PrevBucket_.first << + " >= " << bound); + + Y_ENSURE_EX(PrevBucket_.second <= value, TPrometheusDecodeException() << + "invalid order of histogram bucket values " << PrevBucket_.second << + " > " << value); + + // convert infinite bound value + if (bound == std::numeric_limits<TBucketBound>::infinity()) { + bound = HISTOGRAM_INF_BOUND; + } + + Bounds_.push_back(bound); + Values_.push_back(value - PrevBucket_.second); // keep only delta between buckets + + PrevBucket_ = { bound, value }; + } + + // will clear builder state + IHistogramSnapshotPtr ToSnapshot() { + Y_ENSURE_EX(!Empty(), TPrometheusDecodeException() << "histogram cannot be empty"); + Time_ = TInstant::Zero(); + PrevBucket_ = ZERO_BUCKET; + Labels_.Clear(); auto snapshot = ExplicitHistogramSnapshot(Bounds_, Values_); Bounds_.clear(); Values_.clear(); return snapshot; - } - - private: - TStringBuf Name_; - TMaybe<TLabelsMap> Labels_; - TInstant Time_; - TBucketBounds Bounds_; - TBucketValues Values_; - TBucketData PrevBucket_ = ZERO_BUCKET; - }; - - /////////////////////////////////////////////////////////////////////// - // EPrometheusMetricType - /////////////////////////////////////////////////////////////////////// - enum class EPrometheusMetricType { - GAUGE, - COUNTER, - SUMMARY, - UNTYPED, - HISTOGRAM, - }; - - /////////////////////////////////////////////////////////////////////// - // TPrometheusReader - /////////////////////////////////////////////////////////////////////// - class TPrometheusReader { - public: + } + + private: + TStringBuf Name_; + TMaybe<TLabelsMap> Labels_; + TInstant Time_; + TBucketBounds Bounds_; + TBucketValues Values_; + TBucketData PrevBucket_ = ZERO_BUCKET; + }; + + /////////////////////////////////////////////////////////////////////// + // EPrometheusMetricType + /////////////////////////////////////////////////////////////////////// + enum class EPrometheusMetricType { + GAUGE, + COUNTER, + SUMMARY, + UNTYPED, + HISTOGRAM, + }; + + /////////////////////////////////////////////////////////////////////// + // TPrometheusReader + /////////////////////////////////////////////////////////////////////// + class TPrometheusReader { + public: TPrometheusReader(TStringBuf data, IMetricConsumer* c, TStringBuf metricNameLabel) - : Data_(data) - , Consumer_(c) + : Data_(data) + , Consumer_(c) , MetricNameLabel_(metricNameLabel) - { - } - - void Read() { - Consumer_->OnStreamBegin(); - - if (HasRemaining()) { - ReadNextByte(); - SkipSpaces(); - - try { - while (HasRemaining()) { - switch (CurrentByte_) { - case '\n': - ReadNextByte(); // skip '\n' - CurrentLine_++; - SkipSpaces(); - break; - case '#': - ParseComment(); - break; - default: - ParseMetric(); - break; - } - } - - if (!HistogramBuilder_.Empty()) { - ConsumeHistogram(); - } - } catch (const TPrometheusDecodeException& e) { - throw e; - } catch (...) { - Y_PARSER_FAIL("unexpected error " << CurrentExceptionMessage()); - } - } - - Consumer_->OnStreamEnd(); - } - - private: - bool HasRemaining() const noexcept { - return CurrentPos_ < Data_.Size(); - } - - // # 'TYPE' metric_name {counter|gauge|histogram|summary|untyped} - // # 'HELP' metric_name some help info - // # general comment message - void ParseComment() { - SkipExpectedChar('#'); - SkipSpaces(); - - TStringBuf keyword = ReadToken(); + { + } + + void Read() { + Consumer_->OnStreamBegin(); + + if (HasRemaining()) { + ReadNextByte(); + SkipSpaces(); + + try { + while (HasRemaining()) { + switch (CurrentByte_) { + case '\n': + ReadNextByte(); // skip '\n' + CurrentLine_++; + SkipSpaces(); + break; + case '#': + ParseComment(); + break; + default: + ParseMetric(); + break; + } + } + + if (!HistogramBuilder_.Empty()) { + ConsumeHistogram(); + } + } catch (const TPrometheusDecodeException& e) { + throw e; + } catch (...) { + Y_PARSER_FAIL("unexpected error " << CurrentExceptionMessage()); + } + } + + Consumer_->OnStreamEnd(); + } + + private: + bool HasRemaining() const noexcept { + return CurrentPos_ < Data_.Size(); + } + + // # 'TYPE' metric_name {counter|gauge|histogram|summary|untyped} + // # 'HELP' metric_name some help info + // # general comment message + void ParseComment() { + SkipExpectedChar('#'); + SkipSpaces(); + + TStringBuf keyword = ReadToken(); if (keyword == TStringBuf("TYPE")) { - SkipSpaces(); - - TStringBuf nextName = ReadTokenAsMetricName(); - Y_PARSER_ENSURE(!nextName.Empty(), "invalid metric name"); - - SkipSpaces(); - EPrometheusMetricType nextType = ReadType(); - - bool inserted = SeenTypes_.emplace(nextName, nextType).second; - Y_PARSER_ENSURE(inserted, "second TYPE line for metric " << nextName); - - if (nextType == EPrometheusMetricType::HISTOGRAM) { - if (!HistogramBuilder_.Empty()) { - ConsumeHistogram(); - } - HistogramBuilder_.SetName(nextName); - } - } else { - // skip HELP and general comments - SkipUntilEol(); - } - - Y_PARSER_ENSURE(CurrentByte_ == '\n', "expected '\\n', found '" << CurrentByte_ << '\''); - } - - // metric_name [labels] value [timestamp] - void ParseMetric() { - TStringBuf name = ReadTokenAsMetricName(); - SkipSpaces(); - - TLabelsMap labels = ReadLabels(); - SkipSpaces(); - - double value = ParseGoDouble(ReadToken()); - SkipSpaces(); - - TInstant time = TInstant::Zero(); - if (CurrentByte_ != '\n') { - time = TInstant::MilliSeconds(FromString<ui64>(ReadToken())); - } - - TStringBuf baseName = name; - EPrometheusMetricType type = EPrometheusMetricType::UNTYPED; - - if (auto* seenType = SeenTypes_.FindPtr(name)) { - type = *seenType; - } else { - baseName = NPrometheus::ToBaseName(name); - if (auto* baseType = SeenTypes_.FindPtr(baseName)) { - type = *baseType; - } - } - - switch (type) { - case EPrometheusMetricType::HISTOGRAM: - if (NPrometheus::IsBucket(name)) { - double bound = 0.0; - auto it = labels.find(NPrometheus::BUCKET_LABEL); - if (it != labels.end()) { - bound = ParseGoDouble(it->second); - labels.erase(it); - } else { - Y_PARSER_FAIL( - "metric " << name << "has no " << NPrometheus::BUCKET_LABEL << - " label at line #" << CurrentLine_); - } - - if (!HistogramBuilder_.Empty() && !HistogramBuilder_.Same(baseName, labels)) { - ConsumeHistogram(); - HistogramBuilder_.SetName(baseName); - } - + SkipSpaces(); + + TStringBuf nextName = ReadTokenAsMetricName(); + Y_PARSER_ENSURE(!nextName.Empty(), "invalid metric name"); + + SkipSpaces(); + EPrometheusMetricType nextType = ReadType(); + + bool inserted = SeenTypes_.emplace(nextName, nextType).second; + Y_PARSER_ENSURE(inserted, "second TYPE line for metric " << nextName); + + if (nextType == EPrometheusMetricType::HISTOGRAM) { + if (!HistogramBuilder_.Empty()) { + ConsumeHistogram(); + } + HistogramBuilder_.SetName(nextName); + } + } else { + // skip HELP and general comments + SkipUntilEol(); + } + + Y_PARSER_ENSURE(CurrentByte_ == '\n', "expected '\\n', found '" << CurrentByte_ << '\''); + } + + // metric_name [labels] value [timestamp] + void ParseMetric() { + TStringBuf name = ReadTokenAsMetricName(); + SkipSpaces(); + + TLabelsMap labels = ReadLabels(); + SkipSpaces(); + + double value = ParseGoDouble(ReadToken()); + SkipSpaces(); + + TInstant time = TInstant::Zero(); + if (CurrentByte_ != '\n') { + time = TInstant::MilliSeconds(FromString<ui64>(ReadToken())); + } + + TStringBuf baseName = name; + EPrometheusMetricType type = EPrometheusMetricType::UNTYPED; + + if (auto* seenType = SeenTypes_.FindPtr(name)) { + type = *seenType; + } else { + baseName = NPrometheus::ToBaseName(name); + if (auto* baseType = SeenTypes_.FindPtr(baseName)) { + type = *baseType; + } + } + + switch (type) { + case EPrometheusMetricType::HISTOGRAM: + if (NPrometheus::IsBucket(name)) { + double bound = 0.0; + auto it = labels.find(NPrometheus::BUCKET_LABEL); + if (it != labels.end()) { + bound = ParseGoDouble(it->second); + labels.erase(it); + } else { + Y_PARSER_FAIL( + "metric " << name << "has no " << NPrometheus::BUCKET_LABEL << + " label at line #" << CurrentLine_); + } + + if (!HistogramBuilder_.Empty() && !HistogramBuilder_.Same(baseName, labels)) { + ConsumeHistogram(); + HistogramBuilder_.SetName(baseName); + } + TBucketValue bucketVal; Y_PARSER_ENSURE(TryStaticCast(value, bucketVal), "Cannot convert " << value << " to bucket value type"); HistogramBuilder_.AddBucket(bound, bucketVal); - HistogramBuilder_.SetTime(time); - HistogramBuilder_.SetLabels(std::move(labels)); - } else if (NPrometheus::IsCount(name)) { - // translate x_count metric as COUNTER metric - ConsumeCounter(name, labels, time, value); - } else if (NPrometheus::IsSum(name)) { - // translate x_sum metric as GAUGE metric - ConsumeGauge(name, labels, time, value); - } else { - Y_PARSER_FAIL( - "metric " << name << - " should be part of HISTOGRAM " << baseName); - } - break; - - case EPrometheusMetricType::SUMMARY: - if (NPrometheus::IsCount(name)) { - // translate x_count metric as COUNTER metric - ConsumeCounter(name, labels, time, value); - } else if (NPrometheus::IsSum(name)) { - // translate x_sum metric as GAUGE metric - ConsumeGauge(name, labels, time, value); - } else { - ConsumeGauge(name, labels, time, value); - } - break; - - case EPrometheusMetricType::COUNTER: - ConsumeCounter(name, labels, time, value); - break; - - case EPrometheusMetricType::GAUGE: - ConsumeGauge(name, labels, time, value); - break; - - case EPrometheusMetricType::UNTYPED: - ConsumeGauge(name, labels, time, value); - break; - } - - Y_PARSER_ENSURE(CurrentByte_ == '\n', "expected '\\n', found '" << CurrentByte_ << '\''); - } - - // { name = "value", name2 = "value2", } - TLabelsMap ReadLabels() { - TLabelsMap labels; - if (CurrentByte_ != '{') { - return labels; - } - - SkipExpectedChar('{'); - SkipSpaces(); - - while (CurrentByte_ != '}') { - TStringBuf name = ReadTokenAsLabelName(); - SkipSpaces(); - - SkipExpectedChar('='); - SkipSpaces(); - - TString value = ReadTokenAsLabelValue(); - SkipSpaces(); - labels.emplace(name, value); - - if (CurrentByte_ == ',') { - SkipExpectedChar(','); - SkipSpaces(); - } - } - - SkipExpectedChar('}'); - return labels; - } - - EPrometheusMetricType ReadType() { - TStringBuf keyword = ReadToken(); - if (AsciiEqualsIgnoreCase(keyword, "GAUGE")) { - return EPrometheusMetricType::GAUGE; - } else if (AsciiEqualsIgnoreCase(keyword, "COUNTER")) { - return EPrometheusMetricType::COUNTER; - } else if (AsciiEqualsIgnoreCase(keyword, "SUMMARY")) { - return EPrometheusMetricType::SUMMARY; - } else if (AsciiEqualsIgnoreCase(keyword, "HISTOGRAM")) { - return EPrometheusMetricType::HISTOGRAM; - } else if (AsciiEqualsIgnoreCase(keyword, "UNTYPED")) { - return EPrometheusMetricType::UNTYPED; - } - - Y_PARSER_FAIL( - "unknown metric type: " << keyword << - " at line #" << CurrentLine_); - } - - Y_FORCE_INLINE void ReadNextByteUnsafe() { - CurrentByte_ = Data_[CurrentPos_++]; - } - + HistogramBuilder_.SetTime(time); + HistogramBuilder_.SetLabels(std::move(labels)); + } else if (NPrometheus::IsCount(name)) { + // translate x_count metric as COUNTER metric + ConsumeCounter(name, labels, time, value); + } else if (NPrometheus::IsSum(name)) { + // translate x_sum metric as GAUGE metric + ConsumeGauge(name, labels, time, value); + } else { + Y_PARSER_FAIL( + "metric " << name << + " should be part of HISTOGRAM " << baseName); + } + break; + + case EPrometheusMetricType::SUMMARY: + if (NPrometheus::IsCount(name)) { + // translate x_count metric as COUNTER metric + ConsumeCounter(name, labels, time, value); + } else if (NPrometheus::IsSum(name)) { + // translate x_sum metric as GAUGE metric + ConsumeGauge(name, labels, time, value); + } else { + ConsumeGauge(name, labels, time, value); + } + break; + + case EPrometheusMetricType::COUNTER: + ConsumeCounter(name, labels, time, value); + break; + + case EPrometheusMetricType::GAUGE: + ConsumeGauge(name, labels, time, value); + break; + + case EPrometheusMetricType::UNTYPED: + ConsumeGauge(name, labels, time, value); + break; + } + + Y_PARSER_ENSURE(CurrentByte_ == '\n', "expected '\\n', found '" << CurrentByte_ << '\''); + } + + // { name = "value", name2 = "value2", } + TLabelsMap ReadLabels() { + TLabelsMap labels; + if (CurrentByte_ != '{') { + return labels; + } + + SkipExpectedChar('{'); + SkipSpaces(); + + while (CurrentByte_ != '}') { + TStringBuf name = ReadTokenAsLabelName(); + SkipSpaces(); + + SkipExpectedChar('='); + SkipSpaces(); + + TString value = ReadTokenAsLabelValue(); + SkipSpaces(); + labels.emplace(name, value); + + if (CurrentByte_ == ',') { + SkipExpectedChar(','); + SkipSpaces(); + } + } + + SkipExpectedChar('}'); + return labels; + } + + EPrometheusMetricType ReadType() { + TStringBuf keyword = ReadToken(); + if (AsciiEqualsIgnoreCase(keyword, "GAUGE")) { + return EPrometheusMetricType::GAUGE; + } else if (AsciiEqualsIgnoreCase(keyword, "COUNTER")) { + return EPrometheusMetricType::COUNTER; + } else if (AsciiEqualsIgnoreCase(keyword, "SUMMARY")) { + return EPrometheusMetricType::SUMMARY; + } else if (AsciiEqualsIgnoreCase(keyword, "HISTOGRAM")) { + return EPrometheusMetricType::HISTOGRAM; + } else if (AsciiEqualsIgnoreCase(keyword, "UNTYPED")) { + return EPrometheusMetricType::UNTYPED; + } + + Y_PARSER_FAIL( + "unknown metric type: " << keyword << + " at line #" << CurrentLine_); + } + + Y_FORCE_INLINE void ReadNextByteUnsafe() { + CurrentByte_ = Data_[CurrentPos_++]; + } + Y_FORCE_INLINE bool IsSpace(char ch) { - return ch == ' ' || ch == '\t'; - } - - void ReadNextByte() { - Y_PARSER_ENSURE(HasRemaining(), "unexpected end of file"); - ReadNextByteUnsafe(); - } - - void SkipExpectedChar(char ch) { - Y_PARSER_ENSURE(CurrentByte_ == ch, - "expected '" << CurrentByte_ << "', found '" << ch << '\''); - ReadNextByte(); - } - - void SkipSpaces() { + return ch == ' ' || ch == '\t'; + } + + void ReadNextByte() { + Y_PARSER_ENSURE(HasRemaining(), "unexpected end of file"); + ReadNextByteUnsafe(); + } + + void SkipExpectedChar(char ch) { + Y_PARSER_ENSURE(CurrentByte_ == ch, + "expected '" << CurrentByte_ << "', found '" << ch << '\''); + ReadNextByte(); + } + + void SkipSpaces() { while (HasRemaining() && IsSpace(CurrentByte_)) { - ReadNextByteUnsafe(); - } - } - - void SkipUntilEol() { - while (HasRemaining() && CurrentByte_ != '\n') { - ReadNextByteUnsafe(); - } - } - - TStringBuf ReadToken() { - Y_VERIFY_DEBUG(CurrentPos_ > 0); - size_t begin = CurrentPos_ - 1; // read first byte again + ReadNextByteUnsafe(); + } + } + + void SkipUntilEol() { + while (HasRemaining() && CurrentByte_ != '\n') { + ReadNextByteUnsafe(); + } + } + + TStringBuf ReadToken() { + Y_VERIFY_DEBUG(CurrentPos_ > 0); + size_t begin = CurrentPos_ - 1; // read first byte again while (HasRemaining() && !IsSpace(CurrentByte_) && CurrentByte_ != '\n') { - ReadNextByteUnsafe(); - } - return TokenFromPos(begin); - } - - TStringBuf ReadTokenAsMetricName() { - if (!NPrometheus::IsValidMetricNameStart(CurrentByte_)) { - return ""; - } - - Y_VERIFY_DEBUG(CurrentPos_ > 0); - size_t begin = CurrentPos_ - 1; // read first byte again - while (HasRemaining()) { - ReadNextByteUnsafe(); - if (!NPrometheus::IsValidMetricNameContinuation(CurrentByte_)) { - break; - } - } - return TokenFromPos(begin); - } - - TStringBuf ReadTokenAsLabelName() { - if (!NPrometheus::IsValidLabelNameStart(CurrentByte_)) { - return ""; - } - - Y_VERIFY_DEBUG(CurrentPos_ > 0); - size_t begin = CurrentPos_ - 1; // read first byte again - while (HasRemaining()) { - ReadNextByteUnsafe(); - if (!NPrometheus::IsValidLabelNameContinuation(CurrentByte_)) { - break; - } - } - return TokenFromPos(begin); - } - - TString ReadTokenAsLabelValue() { - TString labelValue; - - SkipExpectedChar('"'); - for (ui32 i = 0; i < MAX_LABEL_VALUE_LEN; i++) { - switch (CurrentByte_) { - case '"': - SkipExpectedChar('"'); - return labelValue; - - case '\n': - Y_PARSER_FAIL("label value contains unescaped new-line"); - - case '\\': - ReadNextByte(); - switch (CurrentByte_) { - case '"': - case '\\': - labelValue.append(CurrentByte_); - break; - case 'n': - labelValue.append('\n'); - break; - default: - Y_PARSER_FAIL("invalid escape sequence '" << CurrentByte_ << '\''); - } - break; - - default: - labelValue.append(CurrentByte_); - break; - } - - ReadNextByte(); - } - - Y_PARSER_FAIL("trying to parse too long label value, size >= " << MAX_LABEL_VALUE_LEN); - } - - TStringBuf TokenFromPos(size_t begin) { - Y_VERIFY_DEBUG(CurrentPos_ > begin); - size_t len = CurrentPos_ - begin - 1; - if (len == 0) { - return {}; - } - - return Data_.SubString(begin, len); - } - - void ConsumeLabels(TStringBuf name, const TLabelsMap& labels) { + ReadNextByteUnsafe(); + } + return TokenFromPos(begin); + } + + TStringBuf ReadTokenAsMetricName() { + if (!NPrometheus::IsValidMetricNameStart(CurrentByte_)) { + return ""; + } + + Y_VERIFY_DEBUG(CurrentPos_ > 0); + size_t begin = CurrentPos_ - 1; // read first byte again + while (HasRemaining()) { + ReadNextByteUnsafe(); + if (!NPrometheus::IsValidMetricNameContinuation(CurrentByte_)) { + break; + } + } + return TokenFromPos(begin); + } + + TStringBuf ReadTokenAsLabelName() { + if (!NPrometheus::IsValidLabelNameStart(CurrentByte_)) { + return ""; + } + + Y_VERIFY_DEBUG(CurrentPos_ > 0); + size_t begin = CurrentPos_ - 1; // read first byte again + while (HasRemaining()) { + ReadNextByteUnsafe(); + if (!NPrometheus::IsValidLabelNameContinuation(CurrentByte_)) { + break; + } + } + return TokenFromPos(begin); + } + + TString ReadTokenAsLabelValue() { + TString labelValue; + + SkipExpectedChar('"'); + for (ui32 i = 0; i < MAX_LABEL_VALUE_LEN; i++) { + switch (CurrentByte_) { + case '"': + SkipExpectedChar('"'); + return labelValue; + + case '\n': + Y_PARSER_FAIL("label value contains unescaped new-line"); + + case '\\': + ReadNextByte(); + switch (CurrentByte_) { + case '"': + case '\\': + labelValue.append(CurrentByte_); + break; + case 'n': + labelValue.append('\n'); + break; + default: + Y_PARSER_FAIL("invalid escape sequence '" << CurrentByte_ << '\''); + } + break; + + default: + labelValue.append(CurrentByte_); + break; + } + + ReadNextByte(); + } + + Y_PARSER_FAIL("trying to parse too long label value, size >= " << MAX_LABEL_VALUE_LEN); + } + + TStringBuf TokenFromPos(size_t begin) { + Y_VERIFY_DEBUG(CurrentPos_ > begin); + size_t len = CurrentPos_ - begin - 1; + if (len == 0) { + return {}; + } + + return Data_.SubString(begin, len); + } + + void ConsumeLabels(TStringBuf name, const TLabelsMap& labels) { Y_PARSER_ENSURE(labels.count(MetricNameLabel_) == 0, "label name '" << MetricNameLabel_ << "' is reserved, but is used with metric: " << name << LabelsToStr(labels)); - - Consumer_->OnLabelsBegin(); + + Consumer_->OnLabelsBegin(); Consumer_->OnLabel(MetricNameLabel_, TString(name)); // TODO: remove this string allocation - for (const auto& it: labels) { - Consumer_->OnLabel(it.first, it.second); - } - Consumer_->OnLabelsEnd(); - } - - void ConsumeCounter(TStringBuf name, const TLabelsMap& labels, TInstant time, double value) { + for (const auto& it: labels) { + Consumer_->OnLabel(it.first, it.second); + } + Consumer_->OnLabelsEnd(); + } + + void ConsumeCounter(TStringBuf name, const TLabelsMap& labels, TInstant time, double value) { i64 intValue{0}; // not nan if (value == value) { Y_PARSER_ENSURE(TryStaticCast(value, intValue), "value " << value << " is out of range"); } - - // see https://st.yandex-team.ru/SOLOMON-4142 for more details - // why we convert Prometheus COUNTER into Solomon RATE - // TODO: need to fix after server-side aggregation become correct for COUNTERs - Consumer_->OnMetricBegin(EMetricType::RATE); - ConsumeLabels(name, labels); - Consumer_->OnUint64(time, intValue); - Consumer_->OnMetricEnd(); - } - - void ConsumeGauge(TStringBuf name, const TLabelsMap& labels, TInstant time, double value) { - Consumer_->OnMetricBegin(EMetricType::GAUGE); - ConsumeLabels(name, labels); - Consumer_->OnDouble(time, value); - Consumer_->OnMetricEnd(); - } - - void ConsumeHistogram() { - Consumer_->OnMetricBegin(EMetricType::HIST_RATE); - ConsumeLabels(HistogramBuilder_.GetName(), HistogramBuilder_.GetLabels()); - auto time = HistogramBuilder_.GetTime(); - auto hist = HistogramBuilder_.ToSnapshot(); - Consumer_->OnHistogram(time, std::move(hist)); - Consumer_->OnMetricEnd(); - } - - double ParseGoDouble(TStringBuf str) { + + // see https://st.yandex-team.ru/SOLOMON-4142 for more details + // why we convert Prometheus COUNTER into Solomon RATE + // TODO: need to fix after server-side aggregation become correct for COUNTERs + Consumer_->OnMetricBegin(EMetricType::RATE); + ConsumeLabels(name, labels); + Consumer_->OnUint64(time, intValue); + Consumer_->OnMetricEnd(); + } + + void ConsumeGauge(TStringBuf name, const TLabelsMap& labels, TInstant time, double value) { + Consumer_->OnMetricBegin(EMetricType::GAUGE); + ConsumeLabels(name, labels); + Consumer_->OnDouble(time, value); + Consumer_->OnMetricEnd(); + } + + void ConsumeHistogram() { + Consumer_->OnMetricBegin(EMetricType::HIST_RATE); + ConsumeLabels(HistogramBuilder_.GetName(), HistogramBuilder_.GetLabels()); + auto time = HistogramBuilder_.GetTime(); + auto hist = HistogramBuilder_.ToSnapshot(); + Consumer_->OnHistogram(time, std::move(hist)); + Consumer_->OnMetricEnd(); + } + + double ParseGoDouble(TStringBuf str) { if (str == TStringBuf("+Inf")) { - return std::numeric_limits<double>::infinity(); + return std::numeric_limits<double>::infinity(); } else if (str == TStringBuf("-Inf")) { - return -std::numeric_limits<double>::infinity(); + return -std::numeric_limits<double>::infinity(); } else if (str == TStringBuf("NaN")) { - return NAN; - } - - double r = 0.0; - if (TryFromString(str, r)) { - return r; - } - Y_PARSER_FAIL("cannot parse double value from '" << str << "\' at line #" << CurrentLine_); - } - - private: - TStringBuf Data_; - IMetricConsumer* Consumer_; + return NAN; + } + + double r = 0.0; + if (TryFromString(str, r)) { + return r; + } + Y_PARSER_FAIL("cannot parse double value from '" << str << "\' at line #" << CurrentLine_); + } + + private: + TStringBuf Data_; + IMetricConsumer* Consumer_; TStringBuf MetricNameLabel_; - THashMap<TString, EPrometheusMetricType> SeenTypes_; - THistogramBuilder HistogramBuilder_; - - ui32 CurrentLine_ = 1; - ui32 CurrentPos_ = 0; - char CurrentByte_ = 0; - }; - } // namespace - + THashMap<TString, EPrometheusMetricType> SeenTypes_; + THistogramBuilder HistogramBuilder_; + + ui32 CurrentLine_ = 1; + ui32 CurrentPos_ = 0; + char CurrentByte_ = 0; + }; + } // namespace + void DecodePrometheus(TStringBuf data, IMetricConsumer* c, TStringBuf metricNameLabel) { TPrometheusReader reader(data, c, metricNameLabel); - reader.Read(); -} - -} // namespace NMonitoring + reader.Read(); +} + +} // namespace NMonitoring diff --git a/library/cpp/monlib/encode/prometheus/prometheus_decoder_ut.cpp b/library/cpp/monlib/encode/prometheus/prometheus_decoder_ut.cpp index 580b9c66179..49c2244fb4e 100644 --- a/library/cpp/monlib/encode/prometheus/prometheus_decoder_ut.cpp +++ b/library/cpp/monlib/encode/prometheus/prometheus_decoder_ut.cpp @@ -1,478 +1,478 @@ -#include "prometheus.h" - -#include <library/cpp/monlib/encode/protobuf/protobuf.h> - +#include "prometheus.h" + +#include <library/cpp/monlib/encode/protobuf/protobuf.h> + #include <library/cpp/testing/unittest/registar.h> - -using namespace NMonitoring; - -#define ASSERT_LABEL_EQUAL(label, name, value) do { \ - UNIT_ASSERT_STRINGS_EQUAL((label).GetName(), name); \ - UNIT_ASSERT_STRINGS_EQUAL((label).GetValue(), value); \ - } while (false) - -#define ASSERT_DOUBLE_POINT(s, time, value) do { \ - UNIT_ASSERT_VALUES_EQUAL((s).GetTime(), (time).MilliSeconds()); \ - UNIT_ASSERT_EQUAL((s).GetValueCase(), NProto::TSingleSample::kFloat64); \ - UNIT_ASSERT_DOUBLES_EQUAL((s).GetFloat64(), value, std::numeric_limits<double>::epsilon()); \ - } while (false) - -#define ASSERT_UINT_POINT(s, time, value) do { \ - UNIT_ASSERT_VALUES_EQUAL((s).GetTime(), (time).MilliSeconds()); \ - UNIT_ASSERT_EQUAL((s).GetValueCase(), NProto::TSingleSample::kUint64); \ - UNIT_ASSERT_VALUES_EQUAL((s).GetUint64(), value); \ - } while (false) - -#define ASSERT_HIST_POINT(s, time, expected) do { \ - UNIT_ASSERT_VALUES_EQUAL((s).GetTime(), time.MilliSeconds()); \ - UNIT_ASSERT_EQUAL((s).GetValueCase(), NProto::TSingleSample::kHistogram);\ - UNIT_ASSERT_VALUES_EQUAL((s).GetHistogram().BoundsSize(), (expected).Count()); \ - UNIT_ASSERT_VALUES_EQUAL((s).GetHistogram().ValuesSize(), (expected).Count()); \ - for (size_t i = 0; i < (s).GetHistogram().BoundsSize(); i++) { \ - UNIT_ASSERT_DOUBLES_EQUAL((s).GetHistogram().GetBounds(i), (expected).UpperBound(i), Min<double>()); \ - UNIT_ASSERT_VALUES_EQUAL((s).GetHistogram().GetValues(i), (expected).Value(i)); \ - } \ - } while (false) - -Y_UNIT_TEST_SUITE(TPrometheusDecoderTest) { - - NProto::TSingleSamplesList Decode(TStringBuf data) { - NProto::TSingleSamplesList samples; - { - IMetricEncoderPtr e = EncoderProtobuf(&samples); - DecodePrometheus(data, e.Get()); - } - return samples; - } - - Y_UNIT_TEST(Empty) { - { - auto samples = Decode(""); - UNIT_ASSERT_EQUAL(samples.SamplesSize(), 0); - } - { - auto samples = Decode("\n"); - UNIT_ASSERT_EQUAL(samples.SamplesSize(), 0); - } - { - auto samples = Decode("\n \n \n"); - UNIT_ASSERT_EQUAL(samples.SamplesSize(), 0); - } - { - auto samples = Decode("\t\n\t\n"); - UNIT_ASSERT_EQUAL(samples.SamplesSize(), 0); - } - } - - Y_UNIT_TEST(Minimal) { - auto samples = Decode( - "minimal_metric 1.234\n" - "another_metric -3e3 103948\n" - "# Even that:\n" - "no_labels{} 3\n" - "# HELP line for non-existing metric will be ignored.\n"); - - UNIT_ASSERT_EQUAL(samples.SamplesSize(), 3); - { - auto& s = samples.GetSamples(0); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); - UNIT_ASSERT_EQUAL(1, s.LabelsSize()); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "minimal_metric"); - ASSERT_DOUBLE_POINT(s, TInstant::Zero(), 1.234); - } - { - auto& s = samples.GetSamples(1); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 1); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "another_metric"); - ASSERT_DOUBLE_POINT(s, TInstant::MilliSeconds(103948), -3000.0); - } - { - auto& s = samples.GetSamples(2); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); - UNIT_ASSERT_EQUAL(1, s.LabelsSize()); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "no_labels"); - ASSERT_DOUBLE_POINT(s, TInstant::Zero(), 3.0); - } - } - - Y_UNIT_TEST(Counter) { - auto samples = Decode( - "# A normal comment.\n" - "#\n" - "# TYPE name counter\n" - "name{labelname=\"val1\",basename=\"basevalue\"} NaN\n" - "name {labelname=\"val2\",basename=\"basevalue\"} 2.3 1234567890\n" - "# HELP name two-line\\n doc str\\\\ing\n"); - - UNIT_ASSERT_EQUAL(samples.SamplesSize(), 2); - - { - auto& s = samples.GetSamples(0); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::RATE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 3); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "name"); - ASSERT_LABEL_EQUAL(s.GetLabels(1), "basename", "basevalue"); - ASSERT_LABEL_EQUAL(s.GetLabels(2), "labelname", "val1"); - ASSERT_UINT_POINT(s, TInstant::Zero(), ui64(0)); - } - { - auto& s = samples.GetSamples(1); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::RATE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 3); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "name"); - ASSERT_LABEL_EQUAL(s.GetLabels(1), "basename", "basevalue"); - ASSERT_LABEL_EQUAL(s.GetLabels(2), "labelname", "val2"); - ASSERT_UINT_POINT(s, TInstant::MilliSeconds(1234567890), i64(2)); - } - } - - Y_UNIT_TEST(Gauge) { - auto samples = Decode( - "# A normal comment.\n" - "#\n" - " # HELP name2 \tdoc str\"ing 2\n" - " # TYPE name2 gauge\n" - "name2{labelname=\"val2\"\t,basename = \"basevalue2\"\t\t} +Inf 54321\n" - "name2{ labelname = \"val1\" , }-Inf\n"); - - UNIT_ASSERT_EQUAL(samples.SamplesSize(), 2); - - { - auto& s = samples.GetSamples(0); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 3); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "name2"); - ASSERT_LABEL_EQUAL(s.GetLabels(1), "basename", "basevalue2"); - ASSERT_LABEL_EQUAL(s.GetLabels(2), "labelname", "val2"); - ASSERT_DOUBLE_POINT(s, TInstant::MilliSeconds(54321), INFINITY); - } - { - auto& s = samples.GetSamples(1); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "name2"); - ASSERT_LABEL_EQUAL(s.GetLabels(1), "labelname", "val1"); - ASSERT_DOUBLE_POINT(s, TInstant::Zero(), -INFINITY); - } - } - - Y_UNIT_TEST(Summary) { - auto samples = Decode( - "# HELP \n" - "# TYPE my_summary summary\n" - "my_summary{n1=\"val1\",quantile=\"0.5\"} 110\n" - "my_summary{n1=\"val1\",quantile=\"0.9\"} 140 1\n" - "my_summary_count{n1=\"val1\"} 42\n" - "my_summary_sum{n1=\"val1\"} 08 15\n" - "# some\n" - "# funny comments\n" - "# HELP\n" - "# HELP my_summary\n" - "# HELP my_summary \n"); - - UNIT_ASSERT_EQUAL(samples.SamplesSize(), 4); - - { - auto& s = samples.GetSamples(0); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 3); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "my_summary"); - ASSERT_LABEL_EQUAL(s.GetLabels(1), "quantile", "0.5"); - ASSERT_LABEL_EQUAL(s.GetLabels(2), "n1", "val1"); - ASSERT_DOUBLE_POINT(s, TInstant::Zero(), 110.0); - } - { - auto& s = samples.GetSamples(1); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 3); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "my_summary"); - ASSERT_LABEL_EQUAL(s.GetLabels(1), "quantile", "0.9"); - ASSERT_LABEL_EQUAL(s.GetLabels(2), "n1", "val1"); - ASSERT_DOUBLE_POINT(s, TInstant::MilliSeconds(1), 140.0); - } - { - auto& s = samples.GetSamples(2); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::RATE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "my_summary_count"); - ASSERT_LABEL_EQUAL(s.GetLabels(1), "n1", "val1"); - ASSERT_UINT_POINT(s, TInstant::Zero(), 42); - } - { - auto& s = samples.GetSamples(3); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "my_summary_sum"); - ASSERT_LABEL_EQUAL(s.GetLabels(1), "n1", "val1"); - ASSERT_DOUBLE_POINT(s, TInstant::MilliSeconds(15), 8.0); - } - } - - Y_UNIT_TEST(Histogram) { - auto samples = Decode( - "# HELP request_duration_microseconds The response latency.\n" - "# TYPE request_duration_microseconds histogram\n" - "request_duration_microseconds_bucket{le=\"0\"} 0\n" - "request_duration_microseconds_bucket{le=\"100\"} 123\n" - "request_duration_microseconds_bucket{le=\"120\"} 412\n" - "request_duration_microseconds_bucket{le=\"144\"} 592\n" - "request_duration_microseconds_bucket{le=\"172.8\"} 1524\n" - "request_duration_microseconds_bucket{le=\"+Inf\"} 2693\n" - "request_duration_microseconds_sum 1.7560473e+06\n" - "request_duration_microseconds_count 2693\n"); - - UNIT_ASSERT_EQUAL(samples.SamplesSize(), 3); - - { - auto& s = samples.GetSamples(0); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 1); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "request_duration_microseconds_sum"); - ASSERT_DOUBLE_POINT(s, TInstant::Zero(), 1756047.3); - } - { - auto& s = samples.GetSamples(1); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::RATE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 1); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "request_duration_microseconds_count"); - ASSERT_UINT_POINT(s, TInstant::Zero(), 2693); - } - { - auto& s = samples.GetSamples(2); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::HIST_RATE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 1); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "request_duration_microseconds"); - auto hist = ExplicitHistogramSnapshot( - { 0, 100, 120, 144, 172.8, HISTOGRAM_INF_BOUND }, - { 0, 123, 289, 180, 932, 1169 }); - ASSERT_HIST_POINT(s, TInstant::Zero(), *hist); - } - } - - Y_UNIT_TEST(HistogramWithLabels) { - auto samples = Decode( - "# A histogram, which has a pretty complex representation in the text format:\n" - "# HELP http_request_duration_seconds A histogram of the request duration.\n" - "# TYPE http_request_duration_seconds histogram\n" - "http_request_duration_seconds_bucket{le=\"0.05\", method=\"POST\"} 24054\n" - "http_request_duration_seconds_bucket{method=\"POST\", le=\"0.1\"} 33444\n" - "http_request_duration_seconds_bucket{le=\"0.2\", method=\"POST\", } 100392\n" - "http_request_duration_seconds_bucket{le=\"0.5\",method=\"POST\",} 129389\n" - "http_request_duration_seconds_bucket{ method=\"POST\", le=\"1\", } 133988\n" - "http_request_duration_seconds_bucket{ le=\"+Inf\", method=\"POST\", } 144320\n" - "http_request_duration_seconds_sum{method=\"POST\"} 53423\n" - "http_request_duration_seconds_count{ method=\"POST\", } 144320\n"); - - UNIT_ASSERT_EQUAL(samples.SamplesSize(), 3); - - { - auto& s = samples.GetSamples(0); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "http_request_duration_seconds_sum"); - ASSERT_LABEL_EQUAL(s.GetLabels(1), "method", "POST"); - ASSERT_DOUBLE_POINT(s, TInstant::Zero(), 53423.0); - } - { - auto& s = samples.GetSamples(1); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::RATE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "http_request_duration_seconds_count"); - ASSERT_LABEL_EQUAL(s.GetLabels(1), "method", "POST"); - ASSERT_UINT_POINT(s, TInstant::Zero(), 144320); - } - { - auto& s = samples.GetSamples(2); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::HIST_RATE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "http_request_duration_seconds"); - ASSERT_LABEL_EQUAL(s.GetLabels(1), "method", "POST"); - auto hist = ExplicitHistogramSnapshot( - { 0.05, 0.1, 0.2, 0.5, 1, HISTOGRAM_INF_BOUND }, - { 24054, 9390, 66948, 28997, 4599, 10332 }); - ASSERT_HIST_POINT(s, TInstant::Zero(), *hist); - } - } - - Y_UNIT_TEST(MultipleHistograms) { - auto samples = Decode( - "# TYPE inboundBytesPerSec histogram\n" - "inboundBytesPerSec_bucket{client=\"mbus\", le=\"10.0\"} 1.0\n" - "inboundBytesPerSec_bucket{client=\"mbus\", le=\"20.0\"} 5.0\n" - "inboundBytesPerSec_bucket{client=\"mbus\", le=\"+Inf\"} 5.0\n" - "inboundBytesPerSec_count{client=\"mbus\"} 5.0\n" - "inboundBytesPerSec_bucket{client=\"grpc\", le=\"10.0\"} 1.0\n" - "inboundBytesPerSec_bucket{client=\"grpc\", le=\"20.0\"} 5.0\n" - "inboundBytesPerSec_bucket{client=\"grpc\", le=\"30.0\"} 5.0\n" - "inboundBytesPerSec_count{client=\"grpc\"} 5.0\n" - "# TYPE outboundBytesPerSec histogram\n" - "outboundBytesPerSec_bucket{client=\"grpc\", le=\"100.0\"} 1.0 1512216000000\n" - "outboundBytesPerSec_bucket{client=\"grpc\", le=\"200.0\"} 1.0 1512216000000\n" - "outboundBytesPerSec_bucket{client=\"grpc\", le=\"+Inf\"} 1.0 1512216000000\n" - "outboundBytesPerSec_count{client=\"grpc\"} 1.0 1512216000000\n"); - - UNIT_ASSERT_EQUAL(samples.SamplesSize(), 6); - - { - auto& s = samples.GetSamples(0); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::RATE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "inboundBytesPerSec_count"); - ASSERT_LABEL_EQUAL(s.GetLabels(1), "client", "mbus"); - ASSERT_UINT_POINT(s, TInstant::Zero(), 5); - } - { - auto& s = samples.GetSamples(1); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::HIST_RATE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "inboundBytesPerSec"); - ASSERT_LABEL_EQUAL(s.GetLabels(1), "client", "mbus"); - auto hist = ExplicitHistogramSnapshot( - { 10, 20, HISTOGRAM_INF_BOUND }, - { 1, 4, 0 }); - ASSERT_HIST_POINT(s, TInstant::Zero(), *hist); - } - { - auto& s = samples.GetSamples(2); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::RATE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "inboundBytesPerSec_count"); - ASSERT_LABEL_EQUAL(s.GetLabels(1), "client", "grpc"); - ASSERT_UINT_POINT(s, TInstant::Zero(), 5); - } - { - auto& s = samples.GetSamples(3); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::HIST_RATE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "inboundBytesPerSec"); - ASSERT_LABEL_EQUAL(s.GetLabels(1), "client", "grpc"); - auto hist = ExplicitHistogramSnapshot( - { 10, 20, 30 }, - { 1, 4, 0 }); - ASSERT_HIST_POINT(s, TInstant::Zero(), *hist); - } - { - auto& s = samples.GetSamples(4); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::RATE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "outboundBytesPerSec_count"); - ASSERT_LABEL_EQUAL(s.GetLabels(1), "client", "grpc"); - ASSERT_UINT_POINT(s, TInstant::Seconds(1512216000), 1) ; - } - { - auto& s = samples.GetSamples(5); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::HIST_RATE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "outboundBytesPerSec"); - ASSERT_LABEL_EQUAL(s.GetLabels(1), "client", "grpc"); - auto hist = ExplicitHistogramSnapshot( - { 100, 200, HISTOGRAM_INF_BOUND }, - { 1, 0, 0 }); - ASSERT_HIST_POINT(s, TInstant::Seconds(1512216000), *hist); - } - } - - Y_UNIT_TEST(MixedTypes) { - auto samples = Decode( - "# HELP http_requests_total The total number of HTTP requests.\n" - "# TYPE http_requests_total counter\n" - "http_requests_total { } 1027 1395066363000\n" - "http_requests_total{method=\"post\",code=\"200\"} 1027 1395066363000\n" - "http_requests_total{method=\"post\",code=\"400\"} 3 1395066363000\n" - "\n" - "# Minimalistic line:\n" - "metric_without_timestamp_and_labels 12.47\n" - "\n" - "# HELP rpc_duration_seconds A summary of the RPC duration in seconds.\n" - "# TYPE rpc_duration_seconds summary\n" - "rpc_duration_seconds{quantile=\"0.01\"} 3102\n" - "rpc_duration_seconds{quantile=\"0.5\"} 4773\n" - "rpc_duration_seconds{quantile=\"0.9\"} 9001\n" - "rpc_duration_seconds_sum 1.7560473e+07\n" - "rpc_duration_seconds_count 2693\n" - "\n" - "# Another mMinimalistic line:\n" - "metric_with_timestamp 12.47 1234567890\n"); - - UNIT_ASSERT_EQUAL(samples.SamplesSize(), 10); - - { - auto& s = samples.GetSamples(0); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::RATE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 1); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "http_requests_total"); - ASSERT_UINT_POINT(s, TInstant::Seconds(1395066363), 1027); - } - { - auto& s = samples.GetSamples(1); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::RATE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 3); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "http_requests_total"); - ASSERT_LABEL_EQUAL(s.GetLabels(1), "method", "post"); - ASSERT_LABEL_EQUAL(s.GetLabels(2), "code", "200"); - ASSERT_UINT_POINT(s, TInstant::Seconds(1395066363), 1027); - } - { - auto& s = samples.GetSamples(2); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::RATE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 3); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "http_requests_total"); - ASSERT_LABEL_EQUAL(s.GetLabels(1), "method", "post"); - ASSERT_LABEL_EQUAL(s.GetLabels(2), "code", "400"); - ASSERT_UINT_POINT(s, TInstant::Seconds(1395066363), 3); - } - { - auto& s = samples.GetSamples(3); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 1); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "metric_without_timestamp_and_labels"); - ASSERT_DOUBLE_POINT(s, TInstant::Zero(), 12.47); - } - { - auto& s = samples.GetSamples(4); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "rpc_duration_seconds"); - ASSERT_LABEL_EQUAL(s.GetLabels(1), "quantile", "0.01"); - ASSERT_DOUBLE_POINT(s, TInstant::Zero(), 3102); - } - { - auto& s = samples.GetSamples(5); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "rpc_duration_seconds"); - ASSERT_LABEL_EQUAL(s.GetLabels(1), "quantile", "0.5"); - ASSERT_DOUBLE_POINT(s, TInstant::Zero(), 4773); - } - { - auto& s = samples.GetSamples(6); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "rpc_duration_seconds"); - ASSERT_LABEL_EQUAL(s.GetLabels(1), "quantile", "0.9"); - ASSERT_DOUBLE_POINT(s, TInstant::Zero(), 9001); - } - { - auto& s = samples.GetSamples(7); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 1); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "rpc_duration_seconds_sum"); - ASSERT_DOUBLE_POINT(s, TInstant::Zero(), 17560473); - } - { - auto& s = samples.GetSamples(8); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::RATE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 1); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "rpc_duration_seconds_count"); - ASSERT_UINT_POINT(s, TInstant::Zero(), 2693); - } - { - auto& s = samples.GetSamples(9); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); - UNIT_ASSERT_EQUAL(s.LabelsSize(), 1); - ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "metric_with_timestamp"); - ASSERT_DOUBLE_POINT(s, TInstant::MilliSeconds(1234567890), 12.47); - } - } -} + +using namespace NMonitoring; + +#define ASSERT_LABEL_EQUAL(label, name, value) do { \ + UNIT_ASSERT_STRINGS_EQUAL((label).GetName(), name); \ + UNIT_ASSERT_STRINGS_EQUAL((label).GetValue(), value); \ + } while (false) + +#define ASSERT_DOUBLE_POINT(s, time, value) do { \ + UNIT_ASSERT_VALUES_EQUAL((s).GetTime(), (time).MilliSeconds()); \ + UNIT_ASSERT_EQUAL((s).GetValueCase(), NProto::TSingleSample::kFloat64); \ + UNIT_ASSERT_DOUBLES_EQUAL((s).GetFloat64(), value, std::numeric_limits<double>::epsilon()); \ + } while (false) + +#define ASSERT_UINT_POINT(s, time, value) do { \ + UNIT_ASSERT_VALUES_EQUAL((s).GetTime(), (time).MilliSeconds()); \ + UNIT_ASSERT_EQUAL((s).GetValueCase(), NProto::TSingleSample::kUint64); \ + UNIT_ASSERT_VALUES_EQUAL((s).GetUint64(), value); \ + } while (false) + +#define ASSERT_HIST_POINT(s, time, expected) do { \ + UNIT_ASSERT_VALUES_EQUAL((s).GetTime(), time.MilliSeconds()); \ + UNIT_ASSERT_EQUAL((s).GetValueCase(), NProto::TSingleSample::kHistogram);\ + UNIT_ASSERT_VALUES_EQUAL((s).GetHistogram().BoundsSize(), (expected).Count()); \ + UNIT_ASSERT_VALUES_EQUAL((s).GetHistogram().ValuesSize(), (expected).Count()); \ + for (size_t i = 0; i < (s).GetHistogram().BoundsSize(); i++) { \ + UNIT_ASSERT_DOUBLES_EQUAL((s).GetHistogram().GetBounds(i), (expected).UpperBound(i), Min<double>()); \ + UNIT_ASSERT_VALUES_EQUAL((s).GetHistogram().GetValues(i), (expected).Value(i)); \ + } \ + } while (false) + +Y_UNIT_TEST_SUITE(TPrometheusDecoderTest) { + + NProto::TSingleSamplesList Decode(TStringBuf data) { + NProto::TSingleSamplesList samples; + { + IMetricEncoderPtr e = EncoderProtobuf(&samples); + DecodePrometheus(data, e.Get()); + } + return samples; + } + + Y_UNIT_TEST(Empty) { + { + auto samples = Decode(""); + UNIT_ASSERT_EQUAL(samples.SamplesSize(), 0); + } + { + auto samples = Decode("\n"); + UNIT_ASSERT_EQUAL(samples.SamplesSize(), 0); + } + { + auto samples = Decode("\n \n \n"); + UNIT_ASSERT_EQUAL(samples.SamplesSize(), 0); + } + { + auto samples = Decode("\t\n\t\n"); + UNIT_ASSERT_EQUAL(samples.SamplesSize(), 0); + } + } + + Y_UNIT_TEST(Minimal) { + auto samples = Decode( + "minimal_metric 1.234\n" + "another_metric -3e3 103948\n" + "# Even that:\n" + "no_labels{} 3\n" + "# HELP line for non-existing metric will be ignored.\n"); + + UNIT_ASSERT_EQUAL(samples.SamplesSize(), 3); + { + auto& s = samples.GetSamples(0); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); + UNIT_ASSERT_EQUAL(1, s.LabelsSize()); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "minimal_metric"); + ASSERT_DOUBLE_POINT(s, TInstant::Zero(), 1.234); + } + { + auto& s = samples.GetSamples(1); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 1); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "another_metric"); + ASSERT_DOUBLE_POINT(s, TInstant::MilliSeconds(103948), -3000.0); + } + { + auto& s = samples.GetSamples(2); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); + UNIT_ASSERT_EQUAL(1, s.LabelsSize()); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "no_labels"); + ASSERT_DOUBLE_POINT(s, TInstant::Zero(), 3.0); + } + } + + Y_UNIT_TEST(Counter) { + auto samples = Decode( + "# A normal comment.\n" + "#\n" + "# TYPE name counter\n" + "name{labelname=\"val1\",basename=\"basevalue\"} NaN\n" + "name {labelname=\"val2\",basename=\"basevalue\"} 2.3 1234567890\n" + "# HELP name two-line\\n doc str\\\\ing\n"); + + UNIT_ASSERT_EQUAL(samples.SamplesSize(), 2); + + { + auto& s = samples.GetSamples(0); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::RATE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 3); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "name"); + ASSERT_LABEL_EQUAL(s.GetLabels(1), "basename", "basevalue"); + ASSERT_LABEL_EQUAL(s.GetLabels(2), "labelname", "val1"); + ASSERT_UINT_POINT(s, TInstant::Zero(), ui64(0)); + } + { + auto& s = samples.GetSamples(1); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::RATE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 3); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "name"); + ASSERT_LABEL_EQUAL(s.GetLabels(1), "basename", "basevalue"); + ASSERT_LABEL_EQUAL(s.GetLabels(2), "labelname", "val2"); + ASSERT_UINT_POINT(s, TInstant::MilliSeconds(1234567890), i64(2)); + } + } + + Y_UNIT_TEST(Gauge) { + auto samples = Decode( + "# A normal comment.\n" + "#\n" + " # HELP name2 \tdoc str\"ing 2\n" + " # TYPE name2 gauge\n" + "name2{labelname=\"val2\"\t,basename = \"basevalue2\"\t\t} +Inf 54321\n" + "name2{ labelname = \"val1\" , }-Inf\n"); + + UNIT_ASSERT_EQUAL(samples.SamplesSize(), 2); + + { + auto& s = samples.GetSamples(0); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 3); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "name2"); + ASSERT_LABEL_EQUAL(s.GetLabels(1), "basename", "basevalue2"); + ASSERT_LABEL_EQUAL(s.GetLabels(2), "labelname", "val2"); + ASSERT_DOUBLE_POINT(s, TInstant::MilliSeconds(54321), INFINITY); + } + { + auto& s = samples.GetSamples(1); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "name2"); + ASSERT_LABEL_EQUAL(s.GetLabels(1), "labelname", "val1"); + ASSERT_DOUBLE_POINT(s, TInstant::Zero(), -INFINITY); + } + } + + Y_UNIT_TEST(Summary) { + auto samples = Decode( + "# HELP \n" + "# TYPE my_summary summary\n" + "my_summary{n1=\"val1\",quantile=\"0.5\"} 110\n" + "my_summary{n1=\"val1\",quantile=\"0.9\"} 140 1\n" + "my_summary_count{n1=\"val1\"} 42\n" + "my_summary_sum{n1=\"val1\"} 08 15\n" + "# some\n" + "# funny comments\n" + "# HELP\n" + "# HELP my_summary\n" + "# HELP my_summary \n"); + + UNIT_ASSERT_EQUAL(samples.SamplesSize(), 4); + + { + auto& s = samples.GetSamples(0); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 3); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "my_summary"); + ASSERT_LABEL_EQUAL(s.GetLabels(1), "quantile", "0.5"); + ASSERT_LABEL_EQUAL(s.GetLabels(2), "n1", "val1"); + ASSERT_DOUBLE_POINT(s, TInstant::Zero(), 110.0); + } + { + auto& s = samples.GetSamples(1); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 3); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "my_summary"); + ASSERT_LABEL_EQUAL(s.GetLabels(1), "quantile", "0.9"); + ASSERT_LABEL_EQUAL(s.GetLabels(2), "n1", "val1"); + ASSERT_DOUBLE_POINT(s, TInstant::MilliSeconds(1), 140.0); + } + { + auto& s = samples.GetSamples(2); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::RATE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "my_summary_count"); + ASSERT_LABEL_EQUAL(s.GetLabels(1), "n1", "val1"); + ASSERT_UINT_POINT(s, TInstant::Zero(), 42); + } + { + auto& s = samples.GetSamples(3); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "my_summary_sum"); + ASSERT_LABEL_EQUAL(s.GetLabels(1), "n1", "val1"); + ASSERT_DOUBLE_POINT(s, TInstant::MilliSeconds(15), 8.0); + } + } + + Y_UNIT_TEST(Histogram) { + auto samples = Decode( + "# HELP request_duration_microseconds The response latency.\n" + "# TYPE request_duration_microseconds histogram\n" + "request_duration_microseconds_bucket{le=\"0\"} 0\n" + "request_duration_microseconds_bucket{le=\"100\"} 123\n" + "request_duration_microseconds_bucket{le=\"120\"} 412\n" + "request_duration_microseconds_bucket{le=\"144\"} 592\n" + "request_duration_microseconds_bucket{le=\"172.8\"} 1524\n" + "request_duration_microseconds_bucket{le=\"+Inf\"} 2693\n" + "request_duration_microseconds_sum 1.7560473e+06\n" + "request_duration_microseconds_count 2693\n"); + + UNIT_ASSERT_EQUAL(samples.SamplesSize(), 3); + + { + auto& s = samples.GetSamples(0); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 1); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "request_duration_microseconds_sum"); + ASSERT_DOUBLE_POINT(s, TInstant::Zero(), 1756047.3); + } + { + auto& s = samples.GetSamples(1); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::RATE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 1); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "request_duration_microseconds_count"); + ASSERT_UINT_POINT(s, TInstant::Zero(), 2693); + } + { + auto& s = samples.GetSamples(2); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::HIST_RATE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 1); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "request_duration_microseconds"); + auto hist = ExplicitHistogramSnapshot( + { 0, 100, 120, 144, 172.8, HISTOGRAM_INF_BOUND }, + { 0, 123, 289, 180, 932, 1169 }); + ASSERT_HIST_POINT(s, TInstant::Zero(), *hist); + } + } + + Y_UNIT_TEST(HistogramWithLabels) { + auto samples = Decode( + "# A histogram, which has a pretty complex representation in the text format:\n" + "# HELP http_request_duration_seconds A histogram of the request duration.\n" + "# TYPE http_request_duration_seconds histogram\n" + "http_request_duration_seconds_bucket{le=\"0.05\", method=\"POST\"} 24054\n" + "http_request_duration_seconds_bucket{method=\"POST\", le=\"0.1\"} 33444\n" + "http_request_duration_seconds_bucket{le=\"0.2\", method=\"POST\", } 100392\n" + "http_request_duration_seconds_bucket{le=\"0.5\",method=\"POST\",} 129389\n" + "http_request_duration_seconds_bucket{ method=\"POST\", le=\"1\", } 133988\n" + "http_request_duration_seconds_bucket{ le=\"+Inf\", method=\"POST\", } 144320\n" + "http_request_duration_seconds_sum{method=\"POST\"} 53423\n" + "http_request_duration_seconds_count{ method=\"POST\", } 144320\n"); + + UNIT_ASSERT_EQUAL(samples.SamplesSize(), 3); + + { + auto& s = samples.GetSamples(0); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "http_request_duration_seconds_sum"); + ASSERT_LABEL_EQUAL(s.GetLabels(1), "method", "POST"); + ASSERT_DOUBLE_POINT(s, TInstant::Zero(), 53423.0); + } + { + auto& s = samples.GetSamples(1); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::RATE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "http_request_duration_seconds_count"); + ASSERT_LABEL_EQUAL(s.GetLabels(1), "method", "POST"); + ASSERT_UINT_POINT(s, TInstant::Zero(), 144320); + } + { + auto& s = samples.GetSamples(2); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::HIST_RATE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "http_request_duration_seconds"); + ASSERT_LABEL_EQUAL(s.GetLabels(1), "method", "POST"); + auto hist = ExplicitHistogramSnapshot( + { 0.05, 0.1, 0.2, 0.5, 1, HISTOGRAM_INF_BOUND }, + { 24054, 9390, 66948, 28997, 4599, 10332 }); + ASSERT_HIST_POINT(s, TInstant::Zero(), *hist); + } + } + + Y_UNIT_TEST(MultipleHistograms) { + auto samples = Decode( + "# TYPE inboundBytesPerSec histogram\n" + "inboundBytesPerSec_bucket{client=\"mbus\", le=\"10.0\"} 1.0\n" + "inboundBytesPerSec_bucket{client=\"mbus\", le=\"20.0\"} 5.0\n" + "inboundBytesPerSec_bucket{client=\"mbus\", le=\"+Inf\"} 5.0\n" + "inboundBytesPerSec_count{client=\"mbus\"} 5.0\n" + "inboundBytesPerSec_bucket{client=\"grpc\", le=\"10.0\"} 1.0\n" + "inboundBytesPerSec_bucket{client=\"grpc\", le=\"20.0\"} 5.0\n" + "inboundBytesPerSec_bucket{client=\"grpc\", le=\"30.0\"} 5.0\n" + "inboundBytesPerSec_count{client=\"grpc\"} 5.0\n" + "# TYPE outboundBytesPerSec histogram\n" + "outboundBytesPerSec_bucket{client=\"grpc\", le=\"100.0\"} 1.0 1512216000000\n" + "outboundBytesPerSec_bucket{client=\"grpc\", le=\"200.0\"} 1.0 1512216000000\n" + "outboundBytesPerSec_bucket{client=\"grpc\", le=\"+Inf\"} 1.0 1512216000000\n" + "outboundBytesPerSec_count{client=\"grpc\"} 1.0 1512216000000\n"); + + UNIT_ASSERT_EQUAL(samples.SamplesSize(), 6); + + { + auto& s = samples.GetSamples(0); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::RATE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "inboundBytesPerSec_count"); + ASSERT_LABEL_EQUAL(s.GetLabels(1), "client", "mbus"); + ASSERT_UINT_POINT(s, TInstant::Zero(), 5); + } + { + auto& s = samples.GetSamples(1); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::HIST_RATE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "inboundBytesPerSec"); + ASSERT_LABEL_EQUAL(s.GetLabels(1), "client", "mbus"); + auto hist = ExplicitHistogramSnapshot( + { 10, 20, HISTOGRAM_INF_BOUND }, + { 1, 4, 0 }); + ASSERT_HIST_POINT(s, TInstant::Zero(), *hist); + } + { + auto& s = samples.GetSamples(2); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::RATE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "inboundBytesPerSec_count"); + ASSERT_LABEL_EQUAL(s.GetLabels(1), "client", "grpc"); + ASSERT_UINT_POINT(s, TInstant::Zero(), 5); + } + { + auto& s = samples.GetSamples(3); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::HIST_RATE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "inboundBytesPerSec"); + ASSERT_LABEL_EQUAL(s.GetLabels(1), "client", "grpc"); + auto hist = ExplicitHistogramSnapshot( + { 10, 20, 30 }, + { 1, 4, 0 }); + ASSERT_HIST_POINT(s, TInstant::Zero(), *hist); + } + { + auto& s = samples.GetSamples(4); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::RATE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "outboundBytesPerSec_count"); + ASSERT_LABEL_EQUAL(s.GetLabels(1), "client", "grpc"); + ASSERT_UINT_POINT(s, TInstant::Seconds(1512216000), 1) ; + } + { + auto& s = samples.GetSamples(5); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::HIST_RATE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "outboundBytesPerSec"); + ASSERT_LABEL_EQUAL(s.GetLabels(1), "client", "grpc"); + auto hist = ExplicitHistogramSnapshot( + { 100, 200, HISTOGRAM_INF_BOUND }, + { 1, 0, 0 }); + ASSERT_HIST_POINT(s, TInstant::Seconds(1512216000), *hist); + } + } + + Y_UNIT_TEST(MixedTypes) { + auto samples = Decode( + "# HELP http_requests_total The total number of HTTP requests.\n" + "# TYPE http_requests_total counter\n" + "http_requests_total { } 1027 1395066363000\n" + "http_requests_total{method=\"post\",code=\"200\"} 1027 1395066363000\n" + "http_requests_total{method=\"post\",code=\"400\"} 3 1395066363000\n" + "\n" + "# Minimalistic line:\n" + "metric_without_timestamp_and_labels 12.47\n" + "\n" + "# HELP rpc_duration_seconds A summary of the RPC duration in seconds.\n" + "# TYPE rpc_duration_seconds summary\n" + "rpc_duration_seconds{quantile=\"0.01\"} 3102\n" + "rpc_duration_seconds{quantile=\"0.5\"} 4773\n" + "rpc_duration_seconds{quantile=\"0.9\"} 9001\n" + "rpc_duration_seconds_sum 1.7560473e+07\n" + "rpc_duration_seconds_count 2693\n" + "\n" + "# Another mMinimalistic line:\n" + "metric_with_timestamp 12.47 1234567890\n"); + + UNIT_ASSERT_EQUAL(samples.SamplesSize(), 10); + + { + auto& s = samples.GetSamples(0); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::RATE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 1); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "http_requests_total"); + ASSERT_UINT_POINT(s, TInstant::Seconds(1395066363), 1027); + } + { + auto& s = samples.GetSamples(1); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::RATE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 3); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "http_requests_total"); + ASSERT_LABEL_EQUAL(s.GetLabels(1), "method", "post"); + ASSERT_LABEL_EQUAL(s.GetLabels(2), "code", "200"); + ASSERT_UINT_POINT(s, TInstant::Seconds(1395066363), 1027); + } + { + auto& s = samples.GetSamples(2); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::RATE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 3); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "http_requests_total"); + ASSERT_LABEL_EQUAL(s.GetLabels(1), "method", "post"); + ASSERT_LABEL_EQUAL(s.GetLabels(2), "code", "400"); + ASSERT_UINT_POINT(s, TInstant::Seconds(1395066363), 3); + } + { + auto& s = samples.GetSamples(3); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 1); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "metric_without_timestamp_and_labels"); + ASSERT_DOUBLE_POINT(s, TInstant::Zero(), 12.47); + } + { + auto& s = samples.GetSamples(4); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "rpc_duration_seconds"); + ASSERT_LABEL_EQUAL(s.GetLabels(1), "quantile", "0.01"); + ASSERT_DOUBLE_POINT(s, TInstant::Zero(), 3102); + } + { + auto& s = samples.GetSamples(5); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "rpc_duration_seconds"); + ASSERT_LABEL_EQUAL(s.GetLabels(1), "quantile", "0.5"); + ASSERT_DOUBLE_POINT(s, TInstant::Zero(), 4773); + } + { + auto& s = samples.GetSamples(6); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 2); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "rpc_duration_seconds"); + ASSERT_LABEL_EQUAL(s.GetLabels(1), "quantile", "0.9"); + ASSERT_DOUBLE_POINT(s, TInstant::Zero(), 9001); + } + { + auto& s = samples.GetSamples(7); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 1); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "rpc_duration_seconds_sum"); + ASSERT_DOUBLE_POINT(s, TInstant::Zero(), 17560473); + } + { + auto& s = samples.GetSamples(8); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::RATE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 1); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "rpc_duration_seconds_count"); + ASSERT_UINT_POINT(s, TInstant::Zero(), 2693); + } + { + auto& s = samples.GetSamples(9); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::EMetricType::GAUGE); + UNIT_ASSERT_EQUAL(s.LabelsSize(), 1); + ASSERT_LABEL_EQUAL(s.GetLabels(0), "sensor", "metric_with_timestamp"); + ASSERT_DOUBLE_POINT(s, TInstant::MilliSeconds(1234567890), 12.47); + } + } +} diff --git a/library/cpp/monlib/encode/prometheus/prometheus_encoder.cpp b/library/cpp/monlib/encode/prometheus/prometheus_encoder.cpp index 3baf90fa2e9..15efeb8c034 100644 --- a/library/cpp/monlib/encode/prometheus/prometheus_encoder.cpp +++ b/library/cpp/monlib/encode/prometheus/prometheus_encoder.cpp @@ -1,95 +1,95 @@ -#include "prometheus.h" -#include "prometheus_model.h" - -#include <library/cpp/monlib/encode/encoder_state.h> -#include <library/cpp/monlib/metrics/labels.h> -#include <library/cpp/monlib/metrics/metric_value.h> - -#include <util/string/cast.h> -#include <util/generic/hash_set.h> - - -namespace NMonitoring { - namespace { - /////////////////////////////////////////////////////////////////////// - // TPrometheusWriter - /////////////////////////////////////////////////////////////////////// - class TPrometheusWriter { - public: - explicit TPrometheusWriter(IOutputStream* out) - : Out_(out) - { - } - - void WriteType(EMetricType type, const TString& name) { - auto r = WrittenTypes_.insert(name); - if (!r.second) { - // type for this metric was already written - return; - } - - Out_->Write("# TYPE "); - WriteMetricName(name); - Out_->Write(' '); - - switch (type) { - case EMetricType::GAUGE: - case EMetricType::IGAUGE: - Out_->Write("gauge"); - break; - case EMetricType::RATE: - case EMetricType::COUNTER: - Out_->Write("counter"); - break; - case EMetricType::HIST: - case EMetricType::HIST_RATE: - Out_->Write("histogram"); - break; +#include "prometheus.h" +#include "prometheus_model.h" + +#include <library/cpp/monlib/encode/encoder_state.h> +#include <library/cpp/monlib/metrics/labels.h> +#include <library/cpp/monlib/metrics/metric_value.h> + +#include <util/string/cast.h> +#include <util/generic/hash_set.h> + + +namespace NMonitoring { + namespace { + /////////////////////////////////////////////////////////////////////// + // TPrometheusWriter + /////////////////////////////////////////////////////////////////////// + class TPrometheusWriter { + public: + explicit TPrometheusWriter(IOutputStream* out) + : Out_(out) + { + } + + void WriteType(EMetricType type, const TString& name) { + auto r = WrittenTypes_.insert(name); + if (!r.second) { + // type for this metric was already written + return; + } + + Out_->Write("# TYPE "); + WriteMetricName(name); + Out_->Write(' '); + + switch (type) { + case EMetricType::GAUGE: + case EMetricType::IGAUGE: + Out_->Write("gauge"); + break; + case EMetricType::RATE: + case EMetricType::COUNTER: + Out_->Write("counter"); + break; + case EMetricType::HIST: + case EMetricType::HIST_RATE: + Out_->Write("histogram"); + break; case EMetricType::LOGHIST: // TODO(@kbalakirev): implement this case break; - case EMetricType::DSUMMARY: + case EMetricType::DSUMMARY: ythrow yexception() << "writing summary type is forbiden"; - case EMetricType::UNKNOWN: - ythrow yexception() << "unknown metric type: " << MetricTypeToStr(type) - << ", name: " << name; - } - Out_->Write('\n'); - } - - void WriteDouble(TStringBuf name, const TLabels& labels, TInstant time, double value) { - WriteValue(name, "", labels, "", "", time, value); - } - - void WriteHistogram(TStringBuf name, const TLabels& labels, TInstant time, IHistogramSnapshot* h) { - Y_ENSURE(!labels.Has(NPrometheus::BUCKET_LABEL), - "histogram metric " << name << " has label '" << - NPrometheus::BUCKET_LABEL << "' which is reserved in Prometheus"); - - double totalCount = 0; - for (ui32 i = 0, count = h->Count(); i < count; i++) { - TBucketBound bound = h->UpperBound(i); - TStringBuf boundStr; - if (bound == HISTOGRAM_INF_BOUND) { + case EMetricType::UNKNOWN: + ythrow yexception() << "unknown metric type: " << MetricTypeToStr(type) + << ", name: " << name; + } + Out_->Write('\n'); + } + + void WriteDouble(TStringBuf name, const TLabels& labels, TInstant time, double value) { + WriteValue(name, "", labels, "", "", time, value); + } + + void WriteHistogram(TStringBuf name, const TLabels& labels, TInstant time, IHistogramSnapshot* h) { + Y_ENSURE(!labels.Has(NPrometheus::BUCKET_LABEL), + "histogram metric " << name << " has label '" << + NPrometheus::BUCKET_LABEL << "' which is reserved in Prometheus"); + + double totalCount = 0; + for (ui32 i = 0, count = h->Count(); i < count; i++) { + TBucketBound bound = h->UpperBound(i); + TStringBuf boundStr; + if (bound == HISTOGRAM_INF_BOUND) { boundStr = TStringBuf("+Inf"); - } else { - size_t len = FloatToString(bound, TmpBuf_, Y_ARRAY_SIZE(TmpBuf_)); - boundStr = TStringBuf(TmpBuf_, len); - } - - TBucketValue value = h->Value(i); - totalCount += static_cast<double>(value); - - WriteValue( - name, NPrometheus::BUCKET_SUFFIX, - labels, NPrometheus::BUCKET_LABEL, boundStr, - time, - totalCount); - } - - WriteValue(name, NPrometheus::COUNT_SUFFIX, labels, "", "", time, totalCount); - } - + } else { + size_t len = FloatToString(bound, TmpBuf_, Y_ARRAY_SIZE(TmpBuf_)); + boundStr = TStringBuf(TmpBuf_, len); + } + + TBucketValue value = h->Value(i); + totalCount += static_cast<double>(value); + + WriteValue( + name, NPrometheus::BUCKET_SUFFIX, + labels, NPrometheus::BUCKET_LABEL, boundStr, + time, + totalCount); + } + + WriteValue(name, NPrometheus::COUNT_SUFFIX, labels, "", "", time, totalCount); + } + void WriteSummaryDouble(TStringBuf name, const TLabels& labels, TInstant time, ISummaryDoubleSnapshot* s) { WriteValue(name, NPrometheus::SUM_SUFFIX, labels, "", "", time, s->GetSum()); WriteValue(name, NPrometheus::MIN_SUFFIX, labels, "", "", time, s->GetMin()); @@ -98,316 +98,316 @@ namespace NMonitoring { WriteValue(name, NPrometheus::COUNT_SUFFIX, labels, "", "", time, s->GetCount()); } - void WriteLn() { - Out_->Write('\n'); - } - - private: - // will replace invalid chars with '_' - void WriteMetricName(TStringBuf name) { - Y_ENSURE(!name.Empty(), "trying to write metric with empty name"); - - char ch = name[0]; - if (NPrometheus::IsValidMetricNameStart(ch)) { - Out_->Write(ch); - } else { - Out_->Write('_'); - } - - for (size_t i = 1, len = name.length(); i < len; i++) { - ch = name[i]; - if (NPrometheus::IsValidMetricNameContinuation(ch)) { - Out_->Write(ch); - } else { - Out_->Write('_'); - } - } - } - - void WriteLabels(const TLabels& labels, TStringBuf addLabelKey, TStringBuf addLabelValue) { - Out_->Write('{'); + void WriteLn() { + Out_->Write('\n'); + } + + private: + // will replace invalid chars with '_' + void WriteMetricName(TStringBuf name) { + Y_ENSURE(!name.Empty(), "trying to write metric with empty name"); + + char ch = name[0]; + if (NPrometheus::IsValidMetricNameStart(ch)) { + Out_->Write(ch); + } else { + Out_->Write('_'); + } + + for (size_t i = 1, len = name.length(); i < len; i++) { + ch = name[i]; + if (NPrometheus::IsValidMetricNameContinuation(ch)) { + Out_->Write(ch); + } else { + Out_->Write('_'); + } + } + } + + void WriteLabels(const TLabels& labels, TStringBuf addLabelKey, TStringBuf addLabelValue) { + Out_->Write('{'); for (auto&& l: labels) { - Out_->Write(l.Name()); - Out_->Write('='); - WriteLabelValue(l.Value()); - Out_->Write(", "); // trailign comma is supported in parsers - } - if (!addLabelKey.Empty() && !addLabelValue.Empty()) { - Out_->Write(addLabelKey); - Out_->Write('='); - WriteLabelValue(addLabelValue); - } - Out_->Write('}'); - } - - void WriteLabelValue(TStringBuf value) { - Out_->Write('"'); - for (char ch: value) { - if (ch == '"') { - Out_->Write("\\\""); - } else if (ch == '\\') { - Out_->Write("\\\\"); - } else if (ch == '\n') { - Out_->Write("\\n"); - } else { - Out_->Write(ch); - } - } - Out_->Write('"'); - } - - void WriteValue( - TStringBuf name, TStringBuf suffix, - const TLabels& labels, TStringBuf addLabelKey, TStringBuf addLabelValue, - TInstant time, double value) - { - // (1) name - WriteMetricName(name); - if (!suffix.Empty()) { - Out_->Write(suffix); - } - - // (2) labels - if (!labels.Empty() || !addLabelKey.Empty()) { - WriteLabels(labels, addLabelKey, addLabelValue); - } - Out_->Write(' '); - - // (3) value - { - size_t len = FloatToString(value, TmpBuf_, Y_ARRAY_SIZE(TmpBuf_)); - Out_->Write(TmpBuf_, len); - } - - // (4) time - if (ui64 timeMillis = time.MilliSeconds()) { - Out_->Write(' '); - size_t len = IntToString<10>(timeMillis, TmpBuf_, Y_ARRAY_SIZE(TmpBuf_)); - Out_->Write(TmpBuf_, len); - } - Out_->Write('\n'); - } - - private: - IOutputStream* Out_; - THashSet<TString> WrittenTypes_; - char TmpBuf_[512]; // used to convert doubles to strings - }; - - /////////////////////////////////////////////////////////////////////// - // TMetricState - /////////////////////////////////////////////////////////////////////// - struct TMetricState { - EMetricType Type = EMetricType::UNKNOWN; - TLabels Labels; - TInstant Time = TInstant::Zero(); - EMetricValueType ValueType = EMetricValueType::UNKNOWN; - TMetricValue Value; - - ~TMetricState() { - ClearValue(); - } - - void Clear() { - Type = EMetricType::UNKNOWN; - Labels.Clear(); - Time = TInstant::Zero(); - ClearValue(); - } - - void ClearValue() { - // TMetricValue does not keep ownership of histogram - if (ValueType == EMetricValueType::HISTOGRAM) { - Value.AsHistogram()->UnRef(); - } else if (ValueType == EMetricValueType::SUMMARY) { + Out_->Write(l.Name()); + Out_->Write('='); + WriteLabelValue(l.Value()); + Out_->Write(", "); // trailign comma is supported in parsers + } + if (!addLabelKey.Empty() && !addLabelValue.Empty()) { + Out_->Write(addLabelKey); + Out_->Write('='); + WriteLabelValue(addLabelValue); + } + Out_->Write('}'); + } + + void WriteLabelValue(TStringBuf value) { + Out_->Write('"'); + for (char ch: value) { + if (ch == '"') { + Out_->Write("\\\""); + } else if (ch == '\\') { + Out_->Write("\\\\"); + } else if (ch == '\n') { + Out_->Write("\\n"); + } else { + Out_->Write(ch); + } + } + Out_->Write('"'); + } + + void WriteValue( + TStringBuf name, TStringBuf suffix, + const TLabels& labels, TStringBuf addLabelKey, TStringBuf addLabelValue, + TInstant time, double value) + { + // (1) name + WriteMetricName(name); + if (!suffix.Empty()) { + Out_->Write(suffix); + } + + // (2) labels + if (!labels.Empty() || !addLabelKey.Empty()) { + WriteLabels(labels, addLabelKey, addLabelValue); + } + Out_->Write(' '); + + // (3) value + { + size_t len = FloatToString(value, TmpBuf_, Y_ARRAY_SIZE(TmpBuf_)); + Out_->Write(TmpBuf_, len); + } + + // (4) time + if (ui64 timeMillis = time.MilliSeconds()) { + Out_->Write(' '); + size_t len = IntToString<10>(timeMillis, TmpBuf_, Y_ARRAY_SIZE(TmpBuf_)); + Out_->Write(TmpBuf_, len); + } + Out_->Write('\n'); + } + + private: + IOutputStream* Out_; + THashSet<TString> WrittenTypes_; + char TmpBuf_[512]; // used to convert doubles to strings + }; + + /////////////////////////////////////////////////////////////////////// + // TMetricState + /////////////////////////////////////////////////////////////////////// + struct TMetricState { + EMetricType Type = EMetricType::UNKNOWN; + TLabels Labels; + TInstant Time = TInstant::Zero(); + EMetricValueType ValueType = EMetricValueType::UNKNOWN; + TMetricValue Value; + + ~TMetricState() { + ClearValue(); + } + + void Clear() { + Type = EMetricType::UNKNOWN; + Labels.Clear(); + Time = TInstant::Zero(); + ClearValue(); + } + + void ClearValue() { + // TMetricValue does not keep ownership of histogram + if (ValueType == EMetricValueType::HISTOGRAM) { + Value.AsHistogram()->UnRef(); + } else if (ValueType == EMetricValueType::SUMMARY) { Value.AsSummaryDouble()->UnRef(); - } - ValueType = EMetricValueType::UNKNOWN; - Value = {}; - } - - template <typename T> - void SetValue(T value) { - // TMetricValue does not keep ownership of histogram - if (ValueType == EMetricValueType::HISTOGRAM) { - Value.AsHistogram()->UnRef(); - } else if (ValueType == EMetricValueType::SUMMARY) { + } + ValueType = EMetricValueType::UNKNOWN; + Value = {}; + } + + template <typename T> + void SetValue(T value) { + // TMetricValue does not keep ownership of histogram + if (ValueType == EMetricValueType::HISTOGRAM) { + Value.AsHistogram()->UnRef(); + } else if (ValueType == EMetricValueType::SUMMARY) { Value.AsSummaryDouble()->UnRef(); - } - ValueType = TValueType<T>::Type; - Value = TMetricValue(value); - if (ValueType == EMetricValueType::HISTOGRAM) { - Value.AsHistogram()->Ref(); - } else if (ValueType == EMetricValueType::SUMMARY) { + } + ValueType = TValueType<T>::Type; + Value = TMetricValue(value); + if (ValueType == EMetricValueType::HISTOGRAM) { + Value.AsHistogram()->Ref(); + } else if (ValueType == EMetricValueType::SUMMARY) { Value.AsSummaryDouble()->Ref(); - } - } - }; - - /////////////////////////////////////////////////////////////////////// - // TPrometheusEncoder - /////////////////////////////////////////////////////////////////////// - class TPrometheusEncoder final: public IMetricEncoder { - public: + } + } + }; + + /////////////////////////////////////////////////////////////////////// + // TPrometheusEncoder + /////////////////////////////////////////////////////////////////////// + class TPrometheusEncoder final: public IMetricEncoder { + public: explicit TPrometheusEncoder(IOutputStream* out, TStringBuf metricNameLabel) - : Writer_(out) + : Writer_(out) , MetricNameLabel_(metricNameLabel) - { - } - - private: - void OnStreamBegin() override { + { + } + + private: + void OnStreamBegin() override { State_.Expect(TEncoderState::EState::ROOT); - } - - void OnStreamEnd() override { + } + + void OnStreamEnd() override { State_.Expect(TEncoderState::EState::ROOT); - Writer_.WriteLn(); - } - - void OnCommonTime(TInstant time) override { + Writer_.WriteLn(); + } + + void OnCommonTime(TInstant time) override { State_.Expect(TEncoderState::EState::ROOT); - CommonTime_ = time; - } - - void OnMetricBegin(EMetricType type) override { - State_.Switch(TEncoderState::EState::ROOT, TEncoderState::EState::METRIC); - MetricState_.Clear(); - MetricState_.Type = type; - } - - void OnMetricEnd() override { - State_.Switch(TEncoderState::EState::METRIC, TEncoderState::EState::ROOT); - WriteMetric(); - } - - void OnLabelsBegin() override { - if (State_ == TEncoderState::EState::METRIC) { - State_ = TEncoderState::EState::METRIC_LABELS; + CommonTime_ = time; + } + + void OnMetricBegin(EMetricType type) override { + State_.Switch(TEncoderState::EState::ROOT, TEncoderState::EState::METRIC); + MetricState_.Clear(); + MetricState_.Type = type; + } + + void OnMetricEnd() override { + State_.Switch(TEncoderState::EState::METRIC, TEncoderState::EState::ROOT); + WriteMetric(); + } + + void OnLabelsBegin() override { + if (State_ == TEncoderState::EState::METRIC) { + State_ = TEncoderState::EState::METRIC_LABELS; } else if (State_ == TEncoderState::EState::ROOT) { State_ = TEncoderState::EState::COMMON_LABELS; - } else { - State_.ThrowInvalid("expected METRIC or ROOT"); - } - } - - void OnLabelsEnd() override { - if (State_ == TEncoderState::EState::METRIC_LABELS) { - State_ = TEncoderState::EState::METRIC; + } else { + State_.ThrowInvalid("expected METRIC or ROOT"); + } + } + + void OnLabelsEnd() override { + if (State_ == TEncoderState::EState::METRIC_LABELS) { + State_ = TEncoderState::EState::METRIC; } else if (State_ == TEncoderState::EState::COMMON_LABELS) { State_ = TEncoderState::EState::ROOT; - } else { - State_.ThrowInvalid("expected LABELS or COMMON_LABELS"); - } - } - - void OnLabel(TStringBuf name, TStringBuf value) override { - if (State_ == TEncoderState::EState::METRIC_LABELS) { - MetricState_.Labels.Add(name, value); + } else { + State_.ThrowInvalid("expected LABELS or COMMON_LABELS"); + } + } + + void OnLabel(TStringBuf name, TStringBuf value) override { + if (State_ == TEncoderState::EState::METRIC_LABELS) { + MetricState_.Labels.Add(name, value); } else if (State_ == TEncoderState::EState::COMMON_LABELS) { - CommonLabels_.Add(name, value); - } else { - State_.ThrowInvalid("expected LABELS or COMMON_LABELS"); - } - } - - void OnDouble(TInstant time, double value) override { - State_.Expect(TEncoderState::EState::METRIC); - MetricState_.Time = time; - MetricState_.SetValue(value); - } - - void OnInt64(TInstant time, i64 value) override { - State_.Expect(TEncoderState::EState::METRIC); - MetricState_.Time = time; - MetricState_.SetValue(value); - } - - void OnUint64(TInstant time, ui64 value) override { - State_.Expect(TEncoderState::EState::METRIC); - MetricState_.Time = time; - MetricState_.SetValue(value); - } - - void OnHistogram(TInstant time, IHistogramSnapshotPtr snapshot) override { - State_.Expect(TEncoderState::EState::METRIC); - MetricState_.Time = time; - MetricState_.SetValue(snapshot.Get()); - } - + CommonLabels_.Add(name, value); + } else { + State_.ThrowInvalid("expected LABELS or COMMON_LABELS"); + } + } + + void OnDouble(TInstant time, double value) override { + State_.Expect(TEncoderState::EState::METRIC); + MetricState_.Time = time; + MetricState_.SetValue(value); + } + + void OnInt64(TInstant time, i64 value) override { + State_.Expect(TEncoderState::EState::METRIC); + MetricState_.Time = time; + MetricState_.SetValue(value); + } + + void OnUint64(TInstant time, ui64 value) override { + State_.Expect(TEncoderState::EState::METRIC); + MetricState_.Time = time; + MetricState_.SetValue(value); + } + + void OnHistogram(TInstant time, IHistogramSnapshotPtr snapshot) override { + State_.Expect(TEncoderState::EState::METRIC); + MetricState_.Time = time; + MetricState_.SetValue(snapshot.Get()); + } + void OnSummaryDouble(TInstant time, ISummaryDoubleSnapshotPtr snapshot) override { - State_.Expect(TEncoderState::EState::METRIC); - MetricState_.Time = time; - MetricState_.SetValue(snapshot.Get()); + State_.Expect(TEncoderState::EState::METRIC); + MetricState_.Time = time; + MetricState_.SetValue(snapshot.Get()); } void OnLogHistogram(TInstant, TLogHistogramSnapshotPtr) override { // TODO(@kbalakirev): implement this function } - void Close() override { - } - - void WriteMetric() { - if (MetricState_.ValueType == EMetricValueType::UNKNOWN) { - return; - } - - // XXX: poor performace + void Close() override { + } + + void WriteMetric() { + if (MetricState_.ValueType == EMetricValueType::UNKNOWN) { + return; + } + + // XXX: poor performace for (auto&& l: CommonLabels_) { - MetricState_.Labels.Add(l.Name(), l.Value()); - } - + MetricState_.Labels.Add(l.Name(), l.Value()); + } + TMaybe<TLabel> nameLabel = MetricState_.Labels.Extract(MetricNameLabel_); - Y_ENSURE(nameLabel, - "labels " << MetricState_.Labels << + Y_ENSURE(nameLabel, + "labels " << MetricState_.Labels << " does not contain label '" << MetricNameLabel_ << '\''); - + const TString& metricName = ToString(nameLabel->Value()); - if (MetricState_.Type != EMetricType::DSUMMARY) { - Writer_.WriteType(MetricState_.Type, metricName); + if (MetricState_.Type != EMetricType::DSUMMARY) { + Writer_.WriteType(MetricState_.Type, metricName); } - - if (MetricState_.Time == TInstant::Zero()) { - MetricState_.Time = CommonTime_; - } - - EMetricType type = MetricState_.Type; - if (type == EMetricType::HIST || type == EMetricType::HIST_RATE) { - Y_ENSURE(MetricState_.ValueType == EMetricValueType::HISTOGRAM, - "invalid value type for histogram: " << int(MetricState_.ValueType)); // TODO: to string conversion - Writer_.WriteHistogram( - metricName, - MetricState_.Labels, - MetricState_.Time, - MetricState_.Value.AsHistogram()); - } else if (type == EMetricType::DSUMMARY) { + + if (MetricState_.Time == TInstant::Zero()) { + MetricState_.Time = CommonTime_; + } + + EMetricType type = MetricState_.Type; + if (type == EMetricType::HIST || type == EMetricType::HIST_RATE) { + Y_ENSURE(MetricState_.ValueType == EMetricValueType::HISTOGRAM, + "invalid value type for histogram: " << int(MetricState_.ValueType)); // TODO: to string conversion + Writer_.WriteHistogram( + metricName, + MetricState_.Labels, + MetricState_.Time, + MetricState_.Value.AsHistogram()); + } else if (type == EMetricType::DSUMMARY) { Writer_.WriteSummaryDouble( metricName, - MetricState_.Labels, - MetricState_.Time, - MetricState_.Value.AsSummaryDouble()); - } else { - Writer_.WriteDouble( - metricName, - MetricState_.Labels, - MetricState_.Time, - MetricState_.Value.AsDouble(MetricState_.ValueType)); - } - } - - private: - TEncoderState State_; - TPrometheusWriter Writer_; + MetricState_.Labels, + MetricState_.Time, + MetricState_.Value.AsSummaryDouble()); + } else { + Writer_.WriteDouble( + metricName, + MetricState_.Labels, + MetricState_.Time, + MetricState_.Value.AsDouble(MetricState_.ValueType)); + } + } + + private: + TEncoderState State_; + TPrometheusWriter Writer_; TString MetricNameLabel_; - TInstant CommonTime_ = TInstant::Zero(); - TLabels CommonLabels_; - TMetricState MetricState_; - }; - } - + TInstant CommonTime_ = TInstant::Zero(); + TLabels CommonLabels_; + TMetricState MetricState_; + }; + } + IMetricEncoderPtr EncoderPrometheus(IOutputStream* out, TStringBuf metricNameLabel) { return MakeHolder<TPrometheusEncoder>(out, metricNameLabel); - } - -} // namespace NMonitoring + } + +} // namespace NMonitoring diff --git a/library/cpp/monlib/encode/prometheus/prometheus_encoder_ut.cpp b/library/cpp/monlib/encode/prometheus/prometheus_encoder_ut.cpp index 6820aa9ba6f..fd9debb060e 100644 --- a/library/cpp/monlib/encode/prometheus/prometheus_encoder_ut.cpp +++ b/library/cpp/monlib/encode/prometheus/prometheus_encoder_ut.cpp @@ -1,119 +1,119 @@ -#include "prometheus.h" - -#include <library/cpp/monlib/encode/protobuf/protobuf.h> -#include <library/cpp/monlib/metrics/metric_value.h> -#include <library/cpp/monlib/metrics/histogram_snapshot.h> - +#include "prometheus.h" + +#include <library/cpp/monlib/encode/protobuf/protobuf.h> +#include <library/cpp/monlib/metrics/metric_value.h> +#include <library/cpp/monlib/metrics/histogram_snapshot.h> + #include <library/cpp/testing/unittest/registar.h> - -#include <util/stream/str.h> - -using namespace NMonitoring; - -Y_UNIT_TEST_SUITE(TPrometheusEncoderTest) { - - template <typename TFunc> - TString EncodeToString(TFunc fn) { - TStringStream ss; - IMetricEncoderPtr encoder = EncoderPrometheus(&ss); - fn(encoder.Get()); - return ss.Str(); - } - + +#include <util/stream/str.h> + +using namespace NMonitoring; + +Y_UNIT_TEST_SUITE(TPrometheusEncoderTest) { + + template <typename TFunc> + TString EncodeToString(TFunc fn) { + TStringStream ss; + IMetricEncoderPtr encoder = EncoderPrometheus(&ss); + fn(encoder.Get()); + return ss.Str(); + } + ISummaryDoubleSnapshotPtr TestSummaryDouble() { return MakeIntrusive<TSummaryDoubleSnapshot>(10.1, -0.45, 0.478, 0.3, 30u); } - Y_UNIT_TEST(Empty) { - auto result = EncodeToString([](IMetricEncoder* e) { - e->OnStreamBegin(); - e->OnStreamEnd(); - }); - UNIT_ASSERT_STRINGS_EQUAL(result, "\n"); - } - - Y_UNIT_TEST(DoubleGauge) { - auto result = EncodeToString([](IMetricEncoder* e) { - e->OnStreamBegin(); - { // no values - e->OnMetricBegin(EMetricType::GAUGE); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "cpuUsage"); - e->OnLabelsEnd(); - } - e->OnMetricEnd(); - } - { // one value no ts - e->OnMetricBegin(EMetricType::GAUGE); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "diskUsage"); - e->OnLabel("disk", "sda1"); - e->OnLabelsEnd(); - } - e->OnDouble(TInstant::Zero(), 1000); - e->OnMetricEnd(); - } - { // one value with ts - e->OnMetricBegin(EMetricType::GAUGE); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "memoryUsage"); - e->OnLabel("host", "solomon-man-00"); - e->OnLabel("dc", "man"); - e->OnLabelsEnd(); - } - e->OnDouble(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:00Z"), 1000); - e->OnMetricEnd(); - } - { // many values - e->OnMetricBegin(EMetricType::GAUGE); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "bytesRx"); - e->OnLabel("host", "solomon-sas-01"); - e->OnLabel("dc", "sas"); - e->OnLabelsEnd(); - } - e->OnDouble(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:00Z"), 2); - e->OnDouble(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:05Z"), 4); - e->OnDouble(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:10Z"), 8); - e->OnMetricEnd(); - } - { // already seen metric name - e->OnMetricBegin(EMetricType::GAUGE); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "diskUsage"); - e->OnLabel("disk", "sdb1"); - e->OnLabelsEnd(); - } - e->OnDouble(TInstant::Zero(), 1001); - e->OnMetricEnd(); - } - { // NaN - e->OnMetricBegin(EMetricType::GAUGE); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "nanValue"); - e->OnLabelsEnd(); - } - e->OnDouble(TInstant::Zero(), NAN); - e->OnMetricEnd(); - } - { // Inf - e->OnMetricBegin(EMetricType::GAUGE); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "infValue"); - e->OnLabelsEnd(); - } - e->OnDouble(TInstant::Zero(), INFINITY); - e->OnMetricEnd(); - } + Y_UNIT_TEST(Empty) { + auto result = EncodeToString([](IMetricEncoder* e) { + e->OnStreamBegin(); + e->OnStreamEnd(); + }); + UNIT_ASSERT_STRINGS_EQUAL(result, "\n"); + } + + Y_UNIT_TEST(DoubleGauge) { + auto result = EncodeToString([](IMetricEncoder* e) { + e->OnStreamBegin(); + { // no values + e->OnMetricBegin(EMetricType::GAUGE); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "cpuUsage"); + e->OnLabelsEnd(); + } + e->OnMetricEnd(); + } + { // one value no ts + e->OnMetricBegin(EMetricType::GAUGE); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "diskUsage"); + e->OnLabel("disk", "sda1"); + e->OnLabelsEnd(); + } + e->OnDouble(TInstant::Zero(), 1000); + e->OnMetricEnd(); + } + { // one value with ts + e->OnMetricBegin(EMetricType::GAUGE); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "memoryUsage"); + e->OnLabel("host", "solomon-man-00"); + e->OnLabel("dc", "man"); + e->OnLabelsEnd(); + } + e->OnDouble(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:00Z"), 1000); + e->OnMetricEnd(); + } + { // many values + e->OnMetricBegin(EMetricType::GAUGE); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "bytesRx"); + e->OnLabel("host", "solomon-sas-01"); + e->OnLabel("dc", "sas"); + e->OnLabelsEnd(); + } + e->OnDouble(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:00Z"), 2); + e->OnDouble(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:05Z"), 4); + e->OnDouble(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:10Z"), 8); + e->OnMetricEnd(); + } + { // already seen metric name + e->OnMetricBegin(EMetricType::GAUGE); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "diskUsage"); + e->OnLabel("disk", "sdb1"); + e->OnLabelsEnd(); + } + e->OnDouble(TInstant::Zero(), 1001); + e->OnMetricEnd(); + } + { // NaN + e->OnMetricBegin(EMetricType::GAUGE); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "nanValue"); + e->OnLabelsEnd(); + } + e->OnDouble(TInstant::Zero(), NAN); + e->OnMetricEnd(); + } + { // Inf + e->OnMetricBegin(EMetricType::GAUGE); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "infValue"); + e->OnLabelsEnd(); + } + e->OnDouble(TInstant::Zero(), INFINITY); + e->OnMetricEnd(); + } { - e->OnMetricBegin(EMetricType::DSUMMARY); + e->OnMetricBegin(EMetricType::DSUMMARY); { e->OnLabelsBegin(); e->OnLabel("sensor", "seconds"); @@ -121,252 +121,252 @@ Y_UNIT_TEST_SUITE(TPrometheusEncoderTest) { e->OnLabelsEnd(); } e->OnSummaryDouble(TInstant::Zero(), TestSummaryDouble()); - e->OnMetricEnd(); + e->OnMetricEnd(); } - e->OnStreamEnd(); - }); - - UNIT_ASSERT_STRINGS_EQUAL(result, - "# TYPE diskUsage gauge\n" - "diskUsage{disk=\"sda1\", } 1000\n" - "# TYPE memoryUsage gauge\n" - "memoryUsage{host=\"solomon-man-00\", dc=\"man\", } 1000 1512216000000\n" - "# TYPE bytesRx gauge\n" - "bytesRx{host=\"solomon-sas-01\", dc=\"sas\", } 8 1512216010000\n" - "diskUsage{disk=\"sdb1\", } 1001\n" - "# TYPE nanValue gauge\n" - "nanValue nan\n" - "# TYPE infValue gauge\n" - "infValue inf\n" + e->OnStreamEnd(); + }); + + UNIT_ASSERT_STRINGS_EQUAL(result, + "# TYPE diskUsage gauge\n" + "diskUsage{disk=\"sda1\", } 1000\n" + "# TYPE memoryUsage gauge\n" + "memoryUsage{host=\"solomon-man-00\", dc=\"man\", } 1000 1512216000000\n" + "# TYPE bytesRx gauge\n" + "bytesRx{host=\"solomon-sas-01\", dc=\"sas\", } 8 1512216010000\n" + "diskUsage{disk=\"sdb1\", } 1001\n" + "# TYPE nanValue gauge\n" + "nanValue nan\n" + "# TYPE infValue gauge\n" + "infValue inf\n" "seconds_sum{disk=\"sdb1\", } 10.1\n" "seconds_min{disk=\"sdb1\", } -0.45\n" "seconds_max{disk=\"sdb1\", } 0.478\n" "seconds_last{disk=\"sdb1\", } 0.3\n" "seconds_count{disk=\"sdb1\", } 30\n" - "\n"); - } - - Y_UNIT_TEST(IntGauges) { - auto result = EncodeToString([](IMetricEncoder* e) { - e->OnStreamBegin(); - { // no values - e->OnMetricBegin(EMetricType::IGAUGE); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "cpuUsage"); - e->OnLabelsEnd(); - } - e->OnMetricEnd(); - } - { // one value no ts - e->OnMetricBegin(EMetricType::IGAUGE); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "diskUsage"); - e->OnLabel("disk", "sda1"); - e->OnLabelsEnd(); - } - e->OnInt64(TInstant::Zero(), 1000); - e->OnMetricEnd(); - } - { // one value with ts - e->OnMetricBegin(EMetricType::IGAUGE); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "memoryUsage"); - e->OnLabel("dc", "man"); - e->OnLabel("host", "solomon-man-00"); - e->OnLabelsEnd(); - } - e->OnInt64(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:00Z"), 1000); - e->OnMetricEnd(); - } - { // many values - e->OnMetricBegin(EMetricType::IGAUGE); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "bytesRx"); - e->OnLabel("dc", "sas"); - e->OnLabel("host", "solomon-sas-01"); - e->OnLabelsEnd(); - } - e->OnInt64(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:00Z"), 2); - e->OnInt64(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:05Z"), 4); - e->OnInt64(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:10Z"), 8); - e->OnMetricEnd(); - } - e->OnStreamEnd(); - }); - - UNIT_ASSERT_STRINGS_EQUAL(result, - "# TYPE diskUsage gauge\n" - "diskUsage{disk=\"sda1\", } 1000\n" - "# TYPE memoryUsage gauge\n" - "memoryUsage{dc=\"man\", host=\"solomon-man-00\", } 1000 1512216000000\n" - "# TYPE bytesRx gauge\n" - "bytesRx{dc=\"sas\", host=\"solomon-sas-01\", } 8 1512216010000\n" - "\n"); - } - - Y_UNIT_TEST(Counters) { - auto result = EncodeToString([](IMetricEncoder* e) { - e->OnStreamBegin(); - { // no values - e->OnMetricBegin(EMetricType::COUNTER); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "cpuUsage"); - e->OnLabelsEnd(); - } - e->OnMetricEnd(); - } - { // one value no ts - e->OnMetricBegin(EMetricType::COUNTER); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "diskUsage"); - e->OnLabel("disk", "sda1"); - e->OnLabelsEnd(); - } - e->OnInt64(TInstant::Zero(), 1000); - e->OnMetricEnd(); - } - { // one value with ts - e->OnMetricBegin(EMetricType::COUNTER); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "memoryUsage"); - e->OnLabel("host", "solomon-man-00"); - e->OnLabel("dc", "man"); - e->OnLabelsEnd(); - } - e->OnInt64(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:00Z"), 1000); - e->OnMetricEnd(); - } - { // many values - e->OnMetricBegin(EMetricType::COUNTER); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "bytesRx"); - e->OnLabel("host", "solomon-sas-01"); - e->OnLabel("dc", "sas"); - e->OnLabelsEnd(); - } - e->OnInt64(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:00Z"), 2); - e->OnInt64(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:05Z"), 4); - e->OnInt64(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:10Z"), 8); - e->OnMetricEnd(); - } - e->OnStreamEnd(); - }); - - UNIT_ASSERT_STRINGS_EQUAL(result, - "# TYPE diskUsage counter\n" - "diskUsage{disk=\"sda1\", } 1000\n" - "# TYPE memoryUsage counter\n" - "memoryUsage{host=\"solomon-man-00\", dc=\"man\", } 1000 1512216000000\n" - "# TYPE bytesRx counter\n" - "bytesRx{host=\"solomon-sas-01\", dc=\"sas\", } 8 1512216010000\n" - "\n"); - } - - Y_UNIT_TEST(Histograms) { - auto result = EncodeToString([](IMetricEncoder* e) { - e->OnStreamBegin(); - { // no values histogram - e->OnMetricBegin(EMetricType::HIST); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "cpuUsage"); - e->OnLabelsEnd(); - } - e->OnMetricEnd(); - } - { // one value no ts - e->OnMetricBegin(EMetricType::HIST); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "inboundBytesPerSec"); - e->OnLabel("client", "mbus"); - e->OnLabelsEnd(); - } - e->OnHistogram( - TInstant::Zero(), - ExplicitHistogramSnapshot({10, 20, HISTOGRAM_INF_BOUND}, {1, 4, 0})); - e->OnMetricEnd(); - } - { // one value no ts no +inf bucket - e->OnMetricBegin(EMetricType::HIST); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "inboundBytesPerSec"); - e->OnLabel("client", "grpc"); - e->OnLabelsEnd(); - } - e->OnHistogram( - TInstant::Zero(), - ExplicitHistogramSnapshot({10, 20, 30}, {1, 4, 0})); - e->OnMetricEnd(); - } - { // one value with ts - e->OnMetricBegin(EMetricType::HIST_RATE); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "outboundBytesPerSec"); - e->OnLabel("client", "grps"); - e->OnLabelsEnd(); - } - e->OnHistogram( - TInstant::ParseIso8601Deprecated("2017-12-02T12:00:00Z"), - ExplicitHistogramSnapshot({100, 200, HISTOGRAM_INF_BOUND}, {1, 0, 0})); - e->OnMetricEnd(); - } - { // many values - e->OnMetricBegin(EMetricType::HIST); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "bytesRx"); - e->OnLabel("host", "solomon-sas-01"); - e->OnLabel("dc", "sas"); - e->OnLabelsEnd(); - } - TBucketBounds bounds = {100, 200, HISTOGRAM_INF_BOUND}; - e->OnHistogram( - TInstant::ParseIso8601Deprecated("2017-12-02T12:00:00Z"), - ExplicitHistogramSnapshot(bounds, {10, 0, 0})); - e->OnHistogram( - TInstant::ParseIso8601Deprecated("2017-12-02T12:00:05Z"), - ExplicitHistogramSnapshot(bounds, {10, 2, 0})); - e->OnHistogram( - TInstant::ParseIso8601Deprecated("2017-12-02T12:00:10Z"), - ExplicitHistogramSnapshot(bounds, {10, 2, 5})); - e->OnMetricEnd(); - } - e->OnStreamEnd(); - }); - - UNIT_ASSERT_STRINGS_EQUAL(result, - "# TYPE inboundBytesPerSec histogram\n" - "inboundBytesPerSec_bucket{client=\"mbus\", le=\"10\"} 1\n" - "inboundBytesPerSec_bucket{client=\"mbus\", le=\"20\"} 5\n" - "inboundBytesPerSec_bucket{client=\"mbus\", le=\"+Inf\"} 5\n" - "inboundBytesPerSec_count{client=\"mbus\", } 5\n" - "inboundBytesPerSec_bucket{client=\"grpc\", le=\"10\"} 1\n" - "inboundBytesPerSec_bucket{client=\"grpc\", le=\"20\"} 5\n" - "inboundBytesPerSec_bucket{client=\"grpc\", le=\"30\"} 5\n" - "inboundBytesPerSec_count{client=\"grpc\", } 5\n" - "# TYPE outboundBytesPerSec histogram\n" - "outboundBytesPerSec_bucket{client=\"grps\", le=\"100\"} 1 1512216000000\n" - "outboundBytesPerSec_bucket{client=\"grps\", le=\"200\"} 1 1512216000000\n" - "outboundBytesPerSec_bucket{client=\"grps\", le=\"+Inf\"} 1 1512216000000\n" - "outboundBytesPerSec_count{client=\"grps\", } 1 1512216000000\n" - "# TYPE bytesRx histogram\n" - "bytesRx_bucket{host=\"solomon-sas-01\", dc=\"sas\", le=\"100\"} 10 1512216010000\n" - "bytesRx_bucket{host=\"solomon-sas-01\", dc=\"sas\", le=\"200\"} 12 1512216010000\n" - "bytesRx_bucket{host=\"solomon-sas-01\", dc=\"sas\", le=\"+Inf\"} 17 1512216010000\n" - "bytesRx_count{host=\"solomon-sas-01\", dc=\"sas\", } 17 1512216010000\n" - "\n"); - } + "\n"); + } + + Y_UNIT_TEST(IntGauges) { + auto result = EncodeToString([](IMetricEncoder* e) { + e->OnStreamBegin(); + { // no values + e->OnMetricBegin(EMetricType::IGAUGE); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "cpuUsage"); + e->OnLabelsEnd(); + } + e->OnMetricEnd(); + } + { // one value no ts + e->OnMetricBegin(EMetricType::IGAUGE); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "diskUsage"); + e->OnLabel("disk", "sda1"); + e->OnLabelsEnd(); + } + e->OnInt64(TInstant::Zero(), 1000); + e->OnMetricEnd(); + } + { // one value with ts + e->OnMetricBegin(EMetricType::IGAUGE); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "memoryUsage"); + e->OnLabel("dc", "man"); + e->OnLabel("host", "solomon-man-00"); + e->OnLabelsEnd(); + } + e->OnInt64(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:00Z"), 1000); + e->OnMetricEnd(); + } + { // many values + e->OnMetricBegin(EMetricType::IGAUGE); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "bytesRx"); + e->OnLabel("dc", "sas"); + e->OnLabel("host", "solomon-sas-01"); + e->OnLabelsEnd(); + } + e->OnInt64(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:00Z"), 2); + e->OnInt64(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:05Z"), 4); + e->OnInt64(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:10Z"), 8); + e->OnMetricEnd(); + } + e->OnStreamEnd(); + }); + + UNIT_ASSERT_STRINGS_EQUAL(result, + "# TYPE diskUsage gauge\n" + "diskUsage{disk=\"sda1\", } 1000\n" + "# TYPE memoryUsage gauge\n" + "memoryUsage{dc=\"man\", host=\"solomon-man-00\", } 1000 1512216000000\n" + "# TYPE bytesRx gauge\n" + "bytesRx{dc=\"sas\", host=\"solomon-sas-01\", } 8 1512216010000\n" + "\n"); + } + + Y_UNIT_TEST(Counters) { + auto result = EncodeToString([](IMetricEncoder* e) { + e->OnStreamBegin(); + { // no values + e->OnMetricBegin(EMetricType::COUNTER); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "cpuUsage"); + e->OnLabelsEnd(); + } + e->OnMetricEnd(); + } + { // one value no ts + e->OnMetricBegin(EMetricType::COUNTER); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "diskUsage"); + e->OnLabel("disk", "sda1"); + e->OnLabelsEnd(); + } + e->OnInt64(TInstant::Zero(), 1000); + e->OnMetricEnd(); + } + { // one value with ts + e->OnMetricBegin(EMetricType::COUNTER); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "memoryUsage"); + e->OnLabel("host", "solomon-man-00"); + e->OnLabel("dc", "man"); + e->OnLabelsEnd(); + } + e->OnInt64(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:00Z"), 1000); + e->OnMetricEnd(); + } + { // many values + e->OnMetricBegin(EMetricType::COUNTER); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "bytesRx"); + e->OnLabel("host", "solomon-sas-01"); + e->OnLabel("dc", "sas"); + e->OnLabelsEnd(); + } + e->OnInt64(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:00Z"), 2); + e->OnInt64(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:05Z"), 4); + e->OnInt64(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:10Z"), 8); + e->OnMetricEnd(); + } + e->OnStreamEnd(); + }); + + UNIT_ASSERT_STRINGS_EQUAL(result, + "# TYPE diskUsage counter\n" + "diskUsage{disk=\"sda1\", } 1000\n" + "# TYPE memoryUsage counter\n" + "memoryUsage{host=\"solomon-man-00\", dc=\"man\", } 1000 1512216000000\n" + "# TYPE bytesRx counter\n" + "bytesRx{host=\"solomon-sas-01\", dc=\"sas\", } 8 1512216010000\n" + "\n"); + } + + Y_UNIT_TEST(Histograms) { + auto result = EncodeToString([](IMetricEncoder* e) { + e->OnStreamBegin(); + { // no values histogram + e->OnMetricBegin(EMetricType::HIST); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "cpuUsage"); + e->OnLabelsEnd(); + } + e->OnMetricEnd(); + } + { // one value no ts + e->OnMetricBegin(EMetricType::HIST); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "inboundBytesPerSec"); + e->OnLabel("client", "mbus"); + e->OnLabelsEnd(); + } + e->OnHistogram( + TInstant::Zero(), + ExplicitHistogramSnapshot({10, 20, HISTOGRAM_INF_BOUND}, {1, 4, 0})); + e->OnMetricEnd(); + } + { // one value no ts no +inf bucket + e->OnMetricBegin(EMetricType::HIST); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "inboundBytesPerSec"); + e->OnLabel("client", "grpc"); + e->OnLabelsEnd(); + } + e->OnHistogram( + TInstant::Zero(), + ExplicitHistogramSnapshot({10, 20, 30}, {1, 4, 0})); + e->OnMetricEnd(); + } + { // one value with ts + e->OnMetricBegin(EMetricType::HIST_RATE); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "outboundBytesPerSec"); + e->OnLabel("client", "grps"); + e->OnLabelsEnd(); + } + e->OnHistogram( + TInstant::ParseIso8601Deprecated("2017-12-02T12:00:00Z"), + ExplicitHistogramSnapshot({100, 200, HISTOGRAM_INF_BOUND}, {1, 0, 0})); + e->OnMetricEnd(); + } + { // many values + e->OnMetricBegin(EMetricType::HIST); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "bytesRx"); + e->OnLabel("host", "solomon-sas-01"); + e->OnLabel("dc", "sas"); + e->OnLabelsEnd(); + } + TBucketBounds bounds = {100, 200, HISTOGRAM_INF_BOUND}; + e->OnHistogram( + TInstant::ParseIso8601Deprecated("2017-12-02T12:00:00Z"), + ExplicitHistogramSnapshot(bounds, {10, 0, 0})); + e->OnHistogram( + TInstant::ParseIso8601Deprecated("2017-12-02T12:00:05Z"), + ExplicitHistogramSnapshot(bounds, {10, 2, 0})); + e->OnHistogram( + TInstant::ParseIso8601Deprecated("2017-12-02T12:00:10Z"), + ExplicitHistogramSnapshot(bounds, {10, 2, 5})); + e->OnMetricEnd(); + } + e->OnStreamEnd(); + }); + + UNIT_ASSERT_STRINGS_EQUAL(result, + "# TYPE inboundBytesPerSec histogram\n" + "inboundBytesPerSec_bucket{client=\"mbus\", le=\"10\"} 1\n" + "inboundBytesPerSec_bucket{client=\"mbus\", le=\"20\"} 5\n" + "inboundBytesPerSec_bucket{client=\"mbus\", le=\"+Inf\"} 5\n" + "inboundBytesPerSec_count{client=\"mbus\", } 5\n" + "inboundBytesPerSec_bucket{client=\"grpc\", le=\"10\"} 1\n" + "inboundBytesPerSec_bucket{client=\"grpc\", le=\"20\"} 5\n" + "inboundBytesPerSec_bucket{client=\"grpc\", le=\"30\"} 5\n" + "inboundBytesPerSec_count{client=\"grpc\", } 5\n" + "# TYPE outboundBytesPerSec histogram\n" + "outboundBytesPerSec_bucket{client=\"grps\", le=\"100\"} 1 1512216000000\n" + "outboundBytesPerSec_bucket{client=\"grps\", le=\"200\"} 1 1512216000000\n" + "outboundBytesPerSec_bucket{client=\"grps\", le=\"+Inf\"} 1 1512216000000\n" + "outboundBytesPerSec_count{client=\"grps\", } 1 1512216000000\n" + "# TYPE bytesRx histogram\n" + "bytesRx_bucket{host=\"solomon-sas-01\", dc=\"sas\", le=\"100\"} 10 1512216010000\n" + "bytesRx_bucket{host=\"solomon-sas-01\", dc=\"sas\", le=\"200\"} 12 1512216010000\n" + "bytesRx_bucket{host=\"solomon-sas-01\", dc=\"sas\", le=\"+Inf\"} 17 1512216010000\n" + "bytesRx_count{host=\"solomon-sas-01\", dc=\"sas\", } 17 1512216010000\n" + "\n"); + } Y_UNIT_TEST(CommonLables) { auto result = EncodeToString([](IMetricEncoder* e) { @@ -411,4 +411,4 @@ two{labels="l2", project="solomon", } 42 1500000000000 )"); } -} +} diff --git a/library/cpp/monlib/encode/prometheus/prometheus_model.h b/library/cpp/monlib/encode/prometheus/prometheus_model.h index f8e55868e77..cb7f2cb15b7 100644 --- a/library/cpp/monlib/encode/prometheus/prometheus_model.h +++ b/library/cpp/monlib/encode/prometheus/prometheus_model.h @@ -1,70 +1,70 @@ -#pragma once - -#include <util/generic/strbuf.h> - - -namespace NMonitoring { -namespace NPrometheus { - - // - // Prometheus specific names and validation rules. - // - // See https://github.com/prometheus/docs/blob/master/content/docs/instrumenting/exposition_formats.md - // and https://github.com/prometheus/common/blob/master/expfmt/text_parse.go - // - +#pragma once + +#include <util/generic/strbuf.h> + + +namespace NMonitoring { +namespace NPrometheus { + + // + // Prometheus specific names and validation rules. + // + // See https://github.com/prometheus/docs/blob/master/content/docs/instrumenting/exposition_formats.md + // and https://github.com/prometheus/common/blob/master/expfmt/text_parse.go + // + inline constexpr TStringBuf BUCKET_SUFFIX = "_bucket"; inline constexpr TStringBuf COUNT_SUFFIX = "_count"; inline constexpr TStringBuf SUM_SUFFIX = "_sum"; inline constexpr TStringBuf MIN_SUFFIX = "_min"; inline constexpr TStringBuf MAX_SUFFIX = "_max"; inline constexpr TStringBuf LAST_SUFFIX = "_last"; - - // Used for the label that defines the upper bound of a bucket of a - // histogram ("le" -> "less or equal"). + + // Used for the label that defines the upper bound of a bucket of a + // histogram ("le" -> "less or equal"). inline constexpr TStringBuf BUCKET_LABEL = "le"; - - - inline bool IsValidLabelNameStart(char ch) { - return (ch >= 'a' && ch <= 'z') || (ch >= 'A' && ch <= 'Z') || ch == '_'; - } - - inline bool IsValidLabelNameContinuation(char ch) { - return IsValidLabelNameStart(ch) || (ch >= '0' && ch <= '9'); - } - - inline bool IsValidMetricNameStart(char ch) { - return IsValidLabelNameStart(ch) || ch == ':'; - } - - inline bool IsValidMetricNameContinuation(char ch) { - return IsValidLabelNameContinuation(ch) || ch == ':'; - } - - inline bool IsSum(TStringBuf name) { - return name.EndsWith(SUM_SUFFIX); - } - - inline bool IsCount(TStringBuf name) { - return name.EndsWith(COUNT_SUFFIX); - } - - inline bool IsBucket(TStringBuf name) { - return name.EndsWith(BUCKET_SUFFIX); - } - - inline TStringBuf ToBaseName(TStringBuf name) { - if (IsBucket(name)) { - return name.SubString(0, name.length() - BUCKET_SUFFIX.length()); - } - if (IsCount(name)) { - return name.SubString(0, name.length() - COUNT_SUFFIX.length()); - } - if (IsSum(name)) { - return name.SubString(0, name.length() - SUM_SUFFIX.length()); - } - return name; - } - -} // namespace NPrometheus -} // namespace NMonitoring + + + inline bool IsValidLabelNameStart(char ch) { + return (ch >= 'a' && ch <= 'z') || (ch >= 'A' && ch <= 'Z') || ch == '_'; + } + + inline bool IsValidLabelNameContinuation(char ch) { + return IsValidLabelNameStart(ch) || (ch >= '0' && ch <= '9'); + } + + inline bool IsValidMetricNameStart(char ch) { + return IsValidLabelNameStart(ch) || ch == ':'; + } + + inline bool IsValidMetricNameContinuation(char ch) { + return IsValidLabelNameContinuation(ch) || ch == ':'; + } + + inline bool IsSum(TStringBuf name) { + return name.EndsWith(SUM_SUFFIX); + } + + inline bool IsCount(TStringBuf name) { + return name.EndsWith(COUNT_SUFFIX); + } + + inline bool IsBucket(TStringBuf name) { + return name.EndsWith(BUCKET_SUFFIX); + } + + inline TStringBuf ToBaseName(TStringBuf name) { + if (IsBucket(name)) { + return name.SubString(0, name.length() - BUCKET_SUFFIX.length()); + } + if (IsCount(name)) { + return name.SubString(0, name.length() - COUNT_SUFFIX.length()); + } + if (IsSum(name)) { + return name.SubString(0, name.length() - SUM_SUFFIX.length()); + } + return name; + } + +} // namespace NPrometheus +} // namespace NMonitoring diff --git a/library/cpp/monlib/encode/prometheus/ut/ya.make b/library/cpp/monlib/encode/prometheus/ut/ya.make index 4d4070d1946..fc468ffb681 100644 --- a/library/cpp/monlib/encode/prometheus/ut/ya.make +++ b/library/cpp/monlib/encode/prometheus/ut/ya.make @@ -1,17 +1,17 @@ -UNITTEST_FOR(library/cpp/monlib/encode/prometheus) - -OWNER( - g:solomon - jamel -) - -SRCS( - prometheus_encoder_ut.cpp - prometheus_decoder_ut.cpp -) - -PEERDIR( - library/cpp/monlib/encode/protobuf -) - -END() +UNITTEST_FOR(library/cpp/monlib/encode/prometheus) + +OWNER( + g:solomon + jamel +) + +SRCS( + prometheus_encoder_ut.cpp + prometheus_decoder_ut.cpp +) + +PEERDIR( + library/cpp/monlib/encode/protobuf +) + +END() diff --git a/library/cpp/monlib/encode/prometheus/ya.make b/library/cpp/monlib/encode/prometheus/ya.make index 03f18f78a46..7f2483b166e 100644 --- a/library/cpp/monlib/encode/prometheus/ya.make +++ b/library/cpp/monlib/encode/prometheus/ya.make @@ -1,17 +1,17 @@ -LIBRARY() - -OWNER( - jamel - g:solomon -) - -SRCS( - prometheus_decoder.cpp - prometheus_encoder.cpp -) - -PEERDIR( - library/cpp/monlib/encode -) - -END() +LIBRARY() + +OWNER( + jamel + g:solomon +) + +SRCS( + prometheus_decoder.cpp + prometheus_encoder.cpp +) + +PEERDIR( + library/cpp/monlib/encode +) + +END() diff --git a/library/cpp/monlib/encode/protobuf/protobuf.h b/library/cpp/monlib/encode/protobuf/protobuf.h index 219b94bee43..3f82cbdd848 100644 --- a/library/cpp/monlib/encode/protobuf/protobuf.h +++ b/library/cpp/monlib/encode/protobuf/protobuf.h @@ -1,16 +1,16 @@ -#pragma once - -#include <library/cpp/monlib/encode/encoder.h> - -#include <library/cpp/monlib/encode/protobuf/protos/samples.pb.h> - -namespace NMonitoring { - namespace NProto { - class TSingleSamplesList; - class TMultiSamplesList; - } - - IMetricEncoderPtr EncoderProtobuf(NProto::TSingleSamplesList* samples); - IMetricEncoderPtr EncoderProtobuf(NProto::TMultiSamplesList* samples); - -} +#pragma once + +#include <library/cpp/monlib/encode/encoder.h> + +#include <library/cpp/monlib/encode/protobuf/protos/samples.pb.h> + +namespace NMonitoring { + namespace NProto { + class TSingleSamplesList; + class TMultiSamplesList; + } + + IMetricEncoderPtr EncoderProtobuf(NProto::TSingleSamplesList* samples); + IMetricEncoderPtr EncoderProtobuf(NProto::TMultiSamplesList* samples); + +} diff --git a/library/cpp/monlib/encode/protobuf/protobuf_encoder.cpp b/library/cpp/monlib/encode/protobuf/protobuf_encoder.cpp index 57fbbba6052..2d11b9d5ba3 100644 --- a/library/cpp/monlib/encode/protobuf/protobuf_encoder.cpp +++ b/library/cpp/monlib/encode/protobuf/protobuf_encoder.cpp @@ -1,42 +1,42 @@ -#include "protobuf.h" - -#include <util/datetime/base.h> - -namespace NMonitoring { - namespace { - NProto::EMetricType ConvertMetricType(EMetricType type) { - switch (type) { - case EMetricType::GAUGE: - return NProto::GAUGE; - case EMetricType::COUNTER: - return NProto::COUNTER; - case EMetricType::RATE: - return NProto::RATE; - case EMetricType::IGAUGE: - return NProto::IGAUGE; - case EMetricType::HIST: +#include "protobuf.h" + +#include <util/datetime/base.h> + +namespace NMonitoring { + namespace { + NProto::EMetricType ConvertMetricType(EMetricType type) { + switch (type) { + case EMetricType::GAUGE: + return NProto::GAUGE; + case EMetricType::COUNTER: + return NProto::COUNTER; + case EMetricType::RATE: + return NProto::RATE; + case EMetricType::IGAUGE: + return NProto::IGAUGE; + case EMetricType::HIST: return NProto::HISTOGRAM; - case EMetricType::HIST_RATE: + case EMetricType::HIST_RATE: return NProto::HIST_RATE; - case EMetricType::DSUMMARY: + case EMetricType::DSUMMARY: return NProto::DSUMMARY; case EMetricType::LOGHIST: return NProto::LOGHISTOGRAM; - case EMetricType::UNKNOWN: - return NProto::UNKNOWN; - } - } - - void FillHistogram( - const IHistogramSnapshot& snapshot, - NProto::THistogram* histogram) - { - for (ui32 i = 0; i < snapshot.Count(); i++) { - histogram->AddBounds(snapshot.UpperBound(i)); - histogram->AddValues(snapshot.Value(i)); - } - } - + case EMetricType::UNKNOWN: + return NProto::UNKNOWN; + } + } + + void FillHistogram( + const IHistogramSnapshot& snapshot, + NProto::THistogram* histogram) + { + for (ui32 i = 0; i < snapshot.Count(); i++) { + histogram->AddBounds(snapshot.UpperBound(i)); + histogram->AddValues(snapshot.Value(i)); + } + } + void FillSummaryDouble(const ISummaryDoubleSnapshot& snapshot, NProto::TSummaryDouble* summary) { summary->SetSum(snapshot.GetSum()); summary->SetMin(snapshot.GetMin()); @@ -54,195 +54,195 @@ namespace NMonitoring { } } - /////////////////////////////////////////////////////////////////////////////// - // TSingleamplesEncoder - /////////////////////////////////////////////////////////////////////////////// - class TSingleSamplesEncoder final: public IMetricEncoder { - public: - TSingleSamplesEncoder(NProto::TSingleSamplesList* samples) - : Samples_(samples) - , Sample_(nullptr) - { - } - - private: - void OnStreamBegin() override { - } - void OnStreamEnd() override { - } - - void OnCommonTime(TInstant time) override { - Samples_->SetCommonTime(time.MilliSeconds()); - } - - void OnMetricBegin(EMetricType type) override { - Sample_ = Samples_->AddSamples(); - Sample_->SetMetricType(ConvertMetricType(type)); - } - - void OnMetricEnd() override { - Sample_ = nullptr; - } - - void OnLabelsBegin() override { - } - void OnLabelsEnd() override { - } - - void OnLabel(TStringBuf name, TStringBuf value) override { - NProto::TLabel* label = (Sample_ == nullptr) - ? Samples_->AddCommonLabels() - : Sample_->AddLabels(); + /////////////////////////////////////////////////////////////////////////////// + // TSingleamplesEncoder + /////////////////////////////////////////////////////////////////////////////// + class TSingleSamplesEncoder final: public IMetricEncoder { + public: + TSingleSamplesEncoder(NProto::TSingleSamplesList* samples) + : Samples_(samples) + , Sample_(nullptr) + { + } + + private: + void OnStreamBegin() override { + } + void OnStreamEnd() override { + } + + void OnCommonTime(TInstant time) override { + Samples_->SetCommonTime(time.MilliSeconds()); + } + + void OnMetricBegin(EMetricType type) override { + Sample_ = Samples_->AddSamples(); + Sample_->SetMetricType(ConvertMetricType(type)); + } + + void OnMetricEnd() override { + Sample_ = nullptr; + } + + void OnLabelsBegin() override { + } + void OnLabelsEnd() override { + } + + void OnLabel(TStringBuf name, TStringBuf value) override { + NProto::TLabel* label = (Sample_ == nullptr) + ? Samples_->AddCommonLabels() + : Sample_->AddLabels(); label->SetName(TString{name}); label->SetValue(TString{value}); - } - - void OnDouble(TInstant time, double value) override { - Y_ENSURE(Sample_, "metric not started"); - Sample_->SetTime(time.MilliSeconds()); - Sample_->SetFloat64(value); - } - - void OnInt64(TInstant time, i64 value) override { - Y_ENSURE(Sample_, "metric not started"); - Sample_->SetTime(time.MilliSeconds()); - Sample_->SetInt64(value); - } - - void OnUint64(TInstant time, ui64 value) override { - Y_ENSURE(Sample_, "metric not started"); - Sample_->SetTime(time.MilliSeconds()); - Sample_->SetUint64(value); - } - - void OnHistogram(TInstant time, IHistogramSnapshotPtr snapshot) override { - Y_ENSURE(Sample_, "metric not started"); - Sample_->SetTime(time.MilliSeconds()); - FillHistogram(*snapshot, Sample_->MutableHistogram()); - } - + } + + void OnDouble(TInstant time, double value) override { + Y_ENSURE(Sample_, "metric not started"); + Sample_->SetTime(time.MilliSeconds()); + Sample_->SetFloat64(value); + } + + void OnInt64(TInstant time, i64 value) override { + Y_ENSURE(Sample_, "metric not started"); + Sample_->SetTime(time.MilliSeconds()); + Sample_->SetInt64(value); + } + + void OnUint64(TInstant time, ui64 value) override { + Y_ENSURE(Sample_, "metric not started"); + Sample_->SetTime(time.MilliSeconds()); + Sample_->SetUint64(value); + } + + void OnHistogram(TInstant time, IHistogramSnapshotPtr snapshot) override { + Y_ENSURE(Sample_, "metric not started"); + Sample_->SetTime(time.MilliSeconds()); + FillHistogram(*snapshot, Sample_->MutableHistogram()); + } + void OnSummaryDouble(TInstant time, ISummaryDoubleSnapshotPtr snapshot) override { - Y_ENSURE(Sample_, "metric not started"); + Y_ENSURE(Sample_, "metric not started"); Sample_->SetTime(time.MilliSeconds()); FillSummaryDouble(*snapshot, Sample_->MutableSummaryDouble()); } void OnLogHistogram(TInstant time, TLogHistogramSnapshotPtr snapshot) override { - Y_ENSURE(Sample_, "metric not started"); + Y_ENSURE(Sample_, "metric not started"); Sample_->SetTime(time.MilliSeconds()); FillLogHistogram(*snapshot, Sample_->MutableLogHistogram()); } - void Close() override { - } - - private: - NProto::TSingleSamplesList* Samples_; - NProto::TSingleSample* Sample_; - }; - - /////////////////////////////////////////////////////////////////////////////// - // TMultiSamplesEncoder - /////////////////////////////////////////////////////////////////////////////// - class TMultiSamplesEncoder final: public IMetricEncoder { - public: - TMultiSamplesEncoder(NProto::TMultiSamplesList* samples) - : Samples_(samples) - , Sample_(nullptr) - { - } - - private: - void OnStreamBegin() override { - } - void OnStreamEnd() override { - } - - void OnCommonTime(TInstant time) override { - Samples_->SetCommonTime(time.MilliSeconds()); - } - - void OnMetricBegin(EMetricType type) override { - Sample_ = Samples_->AddSamples(); - Sample_->SetMetricType(ConvertMetricType(type)); - } - - void OnMetricEnd() override { - Sample_ = nullptr; - } - - void OnLabelsBegin() override { - } - void OnLabelsEnd() override { - } - - void OnLabel(TStringBuf name, TStringBuf value) override { - NProto::TLabel* label = (Sample_ == nullptr) - ? Samples_->AddCommonLabels() - : Sample_->AddLabels(); - + void Close() override { + } + + private: + NProto::TSingleSamplesList* Samples_; + NProto::TSingleSample* Sample_; + }; + + /////////////////////////////////////////////////////////////////////////////// + // TMultiSamplesEncoder + /////////////////////////////////////////////////////////////////////////////// + class TMultiSamplesEncoder final: public IMetricEncoder { + public: + TMultiSamplesEncoder(NProto::TMultiSamplesList* samples) + : Samples_(samples) + , Sample_(nullptr) + { + } + + private: + void OnStreamBegin() override { + } + void OnStreamEnd() override { + } + + void OnCommonTime(TInstant time) override { + Samples_->SetCommonTime(time.MilliSeconds()); + } + + void OnMetricBegin(EMetricType type) override { + Sample_ = Samples_->AddSamples(); + Sample_->SetMetricType(ConvertMetricType(type)); + } + + void OnMetricEnd() override { + Sample_ = nullptr; + } + + void OnLabelsBegin() override { + } + void OnLabelsEnd() override { + } + + void OnLabel(TStringBuf name, TStringBuf value) override { + NProto::TLabel* label = (Sample_ == nullptr) + ? Samples_->AddCommonLabels() + : Sample_->AddLabels(); + label->SetName(TString{name}); label->SetValue(TString{value}); - } - - void OnDouble(TInstant time, double value) override { - Y_ENSURE(Sample_, "metric not started"); - NProto::TPoint* point = Sample_->AddPoints(); - point->SetTime(time.MilliSeconds()); - point->SetFloat64(value); - } - - void OnInt64(TInstant time, i64 value) override { - Y_ENSURE(Sample_, "metric not started"); - NProto::TPoint* point = Sample_->AddPoints(); - point->SetTime(time.MilliSeconds()); - point->SetInt64(value); - } - - void OnUint64(TInstant time, ui64 value) override { - Y_ENSURE(Sample_, "metric not started"); - NProto::TPoint* point = Sample_->AddPoints(); - point->SetTime(time.MilliSeconds()); - point->SetUint64(value); - } - - void OnHistogram(TInstant time, IHistogramSnapshotPtr snapshot) override { - Y_ENSURE(Sample_, "metric not started"); - NProto::TPoint* point = Sample_->AddPoints(); - point->SetTime(time.MilliSeconds()); - FillHistogram(*snapshot, point->MutableHistogram()); - } - + } + + void OnDouble(TInstant time, double value) override { + Y_ENSURE(Sample_, "metric not started"); + NProto::TPoint* point = Sample_->AddPoints(); + point->SetTime(time.MilliSeconds()); + point->SetFloat64(value); + } + + void OnInt64(TInstant time, i64 value) override { + Y_ENSURE(Sample_, "metric not started"); + NProto::TPoint* point = Sample_->AddPoints(); + point->SetTime(time.MilliSeconds()); + point->SetInt64(value); + } + + void OnUint64(TInstant time, ui64 value) override { + Y_ENSURE(Sample_, "metric not started"); + NProto::TPoint* point = Sample_->AddPoints(); + point->SetTime(time.MilliSeconds()); + point->SetUint64(value); + } + + void OnHistogram(TInstant time, IHistogramSnapshotPtr snapshot) override { + Y_ENSURE(Sample_, "metric not started"); + NProto::TPoint* point = Sample_->AddPoints(); + point->SetTime(time.MilliSeconds()); + FillHistogram(*snapshot, point->MutableHistogram()); + } + void OnSummaryDouble(TInstant time, ISummaryDoubleSnapshotPtr snapshot) override { - Y_ENSURE(Sample_, "metric not started"); + Y_ENSURE(Sample_, "metric not started"); NProto::TPoint* point = Sample_->AddPoints(); point->SetTime(time.MilliSeconds()); FillSummaryDouble(*snapshot, point->MutableSummaryDouble()); } void OnLogHistogram(TInstant time, TLogHistogramSnapshotPtr snapshot) override { - Y_ENSURE(Sample_, "metric not started"); + Y_ENSURE(Sample_, "metric not started"); NProto::TPoint* point = Sample_->AddPoints(); point->SetTime(time.MilliSeconds()); FillLogHistogram(*snapshot, point->MutableLogHistogram()); } - void Close() override { - } - - private: - NProto::TMultiSamplesList* Samples_; - NProto::TMultiSample* Sample_; - }; - - } - - IMetricEncoderPtr EncoderProtobuf(NProto::TSingleSamplesList* samples) { + void Close() override { + } + + private: + NProto::TMultiSamplesList* Samples_; + NProto::TMultiSample* Sample_; + }; + + } + + IMetricEncoderPtr EncoderProtobuf(NProto::TSingleSamplesList* samples) { return MakeHolder<TSingleSamplesEncoder>(samples); - } - - IMetricEncoderPtr EncoderProtobuf(NProto::TMultiSamplesList* samples) { + } + + IMetricEncoderPtr EncoderProtobuf(NProto::TMultiSamplesList* samples) { return MakeHolder<TMultiSamplesEncoder>(samples); - } - -} + } + +} diff --git a/library/cpp/monlib/encode/protobuf/protos/samples.proto b/library/cpp/monlib/encode/protobuf/protos/samples.proto index d5f0c6403e3..371f4181d23 100644 --- a/library/cpp/monlib/encode/protobuf/protos/samples.proto +++ b/library/cpp/monlib/encode/protobuf/protos/samples.proto @@ -1,33 +1,33 @@ -syntax = 'proto3'; - -package NMonitoring.NProto; - -option java_package = "ru.yandex.solomon.protos"; -option java_multiple_files = true; -option cc_enable_arenas = true; - -message TLabel { - string Name = 1; - string Value = 2; -} - -enum EMetricType { - UNKNOWN = 0; - GAUGE = 1; - IGAUGE = 2; - COUNTER = 3; - RATE = 4; - HISTOGRAM = 5; +syntax = 'proto3'; + +package NMonitoring.NProto; + +option java_package = "ru.yandex.solomon.protos"; +option java_multiple_files = true; +option cc_enable_arenas = true; + +message TLabel { + string Name = 1; + string Value = 2; +} + +enum EMetricType { + UNKNOWN = 0; + GAUGE = 1; + IGAUGE = 2; + COUNTER = 3; + RATE = 4; + HISTOGRAM = 5; HIST_RATE = 6; DSUMMARY = 7; LOGHISTOGRAM = 8; -} - -message THistogram { - repeated double Bounds = 1; // upper bounds of each bucket - repeated uint64 Values = 2; // values stored in each bucket -} - +} + +message THistogram { + repeated double Bounds = 1; // upper bounds of each bucket + repeated uint64 Values = 2; // values stored in each bucket +} + message TLogHistogram { double Base = 1; uint64 ZerosCount = 2; @@ -43,49 +43,49 @@ message TSummaryDouble { uint64 Count = 5; } -// see TSingleSample -message TPoint { - uint64 Time = 1; - oneof Value { - sfixed64 Int64 = 2; - fixed64 Uint64 = 3; - double Float64 = 4; - THistogram Histogram = 5; +// see TSingleSample +message TPoint { + uint64 Time = 1; + oneof Value { + sfixed64 Int64 = 2; + fixed64 Uint64 = 3; + double Float64 = 4; + THistogram Histogram = 5; TSummaryDouble SummaryDouble = 6; TLogHistogram LogHistogram = 7; - } -} - -message TSingleSample { - repeated TLabel Labels = 1; - EMetricType MetricType = 2; - - // inlined TPoint - uint64 Time = 3; - oneof Value { - sfixed64 Int64 = 4; - fixed64 Uint64 = 5; - double Float64 = 6; - THistogram Histogram = 7; + } +} + +message TSingleSample { + repeated TLabel Labels = 1; + EMetricType MetricType = 2; + + // inlined TPoint + uint64 Time = 3; + oneof Value { + sfixed64 Int64 = 4; + fixed64 Uint64 = 5; + double Float64 = 6; + THistogram Histogram = 7; TSummaryDouble SummaryDouble = 8; TLogHistogram LogHistogram = 9; - } -} - -message TMultiSample { - repeated TLabel Labels = 1; - EMetricType MetricType = 2; - repeated TPoint Points = 3; -} - -message TSingleSamplesList { - uint64 CommonTime = 1; - repeated TLabel CommonLabels = 2; - repeated TSingleSample Samples = 3; -} - -message TMultiSamplesList { - uint64 CommonTime = 1; - repeated TLabel CommonLabels = 2; - repeated TMultiSample Samples = 3; -} + } +} + +message TMultiSample { + repeated TLabel Labels = 1; + EMetricType MetricType = 2; + repeated TPoint Points = 3; +} + +message TSingleSamplesList { + uint64 CommonTime = 1; + repeated TLabel CommonLabels = 2; + repeated TSingleSample Samples = 3; +} + +message TMultiSamplesList { + uint64 CommonTime = 1; + repeated TLabel CommonLabels = 2; + repeated TMultiSample Samples = 3; +} diff --git a/library/cpp/monlib/encode/protobuf/protos/ya.make b/library/cpp/monlib/encode/protobuf/protos/ya.make index 29a87a0b21f..88ff3ddf884 100644 --- a/library/cpp/monlib/encode/protobuf/protos/ya.make +++ b/library/cpp/monlib/encode/protobuf/protos/ya.make @@ -1,14 +1,14 @@ -PROTO_LIBRARY() - +PROTO_LIBRARY() + OWNER( jamel g:solomon ) - -SRCS( - samples.proto -) - + +SRCS( + samples.proto +) + EXCLUDE_TAGS(GO_PROTO) -END() +END() diff --git a/library/cpp/monlib/encode/protobuf/ya.make b/library/cpp/monlib/encode/protobuf/ya.make index fd575ef7319..9354958b6f7 100644 --- a/library/cpp/monlib/encode/protobuf/ya.make +++ b/library/cpp/monlib/encode/protobuf/ya.make @@ -1,17 +1,17 @@ -LIBRARY() - +LIBRARY() + OWNER( jamel g:solomon ) - -SRCS( - protobuf_encoder.cpp -) - -PEERDIR( - library/cpp/monlib/encode - library/cpp/monlib/encode/protobuf/protos -) - -END() + +SRCS( + protobuf_encoder.cpp +) + +PEERDIR( + library/cpp/monlib/encode + library/cpp/monlib/encode/protobuf/protos +) + +END() diff --git a/library/cpp/monlib/encode/spack/compression.cpp b/library/cpp/monlib/encode/spack/compression.cpp index 0ad1eee8667..0d2152fc851 100644 --- a/library/cpp/monlib/encode/spack/compression.cpp +++ b/library/cpp/monlib/encode/spack/compression.cpp @@ -1,213 +1,213 @@ -#include "compression.h" - +#include "compression.h" + #include <util/generic/buffer.h> -#include <util/generic/cast.h> -#include <util/generic/ptr.h> +#include <util/generic/cast.h> +#include <util/generic/ptr.h> #include <util/generic/scope.h> #include <util/generic/size_literals.h> #include <util/stream/format.h> #include <util/stream/output.h> -#include <util/stream/walk.h> - -#include <contrib/libs/lz4/lz4.h> +#include <util/stream/walk.h> + +#include <contrib/libs/lz4/lz4.h> #include <contrib/libs/xxhash/xxhash.h> -#include <contrib/libs/zlib/zlib.h> -#define ZSTD_STATIC_LINKING_ONLY +#include <contrib/libs/zlib/zlib.h> +#define ZSTD_STATIC_LINKING_ONLY #include <contrib/libs/zstd/include/zstd.h> - -namespace NMonitoring { - namespace { - /////////////////////////////////////////////////////////////////////////////// - // Frame - /////////////////////////////////////////////////////////////////////////////// - using TCompressedSize = ui32; - using TUncompressedSize = ui32; - using TCheckSum = ui32; - + +namespace NMonitoring { + namespace { + /////////////////////////////////////////////////////////////////////////////// + // Frame + /////////////////////////////////////////////////////////////////////////////// + using TCompressedSize = ui32; + using TUncompressedSize = ui32; + using TCheckSum = ui32; + constexpr size_t COMPRESSED_FRAME_SIZE_LIMIT = 512_KB; constexpr size_t UNCOMPRESSED_FRAME_SIZE_LIMIT = COMPRESSED_FRAME_SIZE_LIMIT; constexpr size_t FRAME_SIZE_LIMIT = 2_MB; constexpr size_t DEFAULT_FRAME_LEN = 64_KB; - + struct Y_PACKED TFrameHeader { - TCompressedSize CompressedSize; - TUncompressedSize UncompressedSize; - }; - + TCompressedSize CompressedSize; + TUncompressedSize UncompressedSize; + }; + struct Y_PACKED TFrameFooter { - TCheckSum CheckSum; - }; - - /////////////////////////////////////////////////////////////////////////////// - // TBlock - /////////////////////////////////////////////////////////////////////////////// - struct TBlock: public TStringBuf { - template <typename T> - TBlock(T&& t) + TCheckSum CheckSum; + }; + + /////////////////////////////////////////////////////////////////////////////// + // TBlock + /////////////////////////////////////////////////////////////////////////////// + struct TBlock: public TStringBuf { + template <typename T> + TBlock(T&& t) : TStringBuf(t.data(), t.size()) - { + { Y_ENSURE(t.data() != nullptr); - } - + } + char* data() noexcept { return const_cast<char*>(TStringBuf::data()); - } - }; - - /////////////////////////////////////////////////////////////////////////////// - // XXHASH - /////////////////////////////////////////////////////////////////////////////// - struct TXxHash32 { - static TCheckSum Calc(TBlock in) { - static const ui32 SEED = 0x1337c0de; + } + }; + + /////////////////////////////////////////////////////////////////////////////// + // XXHASH + /////////////////////////////////////////////////////////////////////////////// + struct TXxHash32 { + static TCheckSum Calc(TBlock in) { + static const ui32 SEED = 0x1337c0de; return XXH32(in.data(), in.size(), SEED); - } - - static bool Check(TBlock in, TCheckSum checksum) { - return Calc(in) == checksum; - } - }; - - /////////////////////////////////////////////////////////////////////////////// - // Adler32 - /////////////////////////////////////////////////////////////////////////////// - struct TAdler32 { - static TCheckSum Calc(TBlock in) { + } + + static bool Check(TBlock in, TCheckSum checksum) { + return Calc(in) == checksum; + } + }; + + /////////////////////////////////////////////////////////////////////////////// + // Adler32 + /////////////////////////////////////////////////////////////////////////////// + struct TAdler32 { + static TCheckSum Calc(TBlock in) { return adler32(1L, reinterpret_cast<const Bytef*>(in.data()), in.size()); - } - - static bool Check(TBlock in, TCheckSum checksum) { - return Calc(in) == checksum; - } - }; - - /////////////////////////////////////////////////////////////////////////////// - // LZ4 - /////////////////////////////////////////////////////////////////////////////// - struct TLz4Codec { - static size_t MaxCompressedLength(size_t in) { - int result = LZ4_compressBound(static_cast<int>(in)); - Y_ENSURE(result != 0, "lz4 input size is too large"); - return result; - } - - static size_t Compress(TBlock in, TBlock out) { - int rc = LZ4_compress_default( + } + + static bool Check(TBlock in, TCheckSum checksum) { + return Calc(in) == checksum; + } + }; + + /////////////////////////////////////////////////////////////////////////////// + // LZ4 + /////////////////////////////////////////////////////////////////////////////// + struct TLz4Codec { + static size_t MaxCompressedLength(size_t in) { + int result = LZ4_compressBound(static_cast<int>(in)); + Y_ENSURE(result != 0, "lz4 input size is too large"); + return result; + } + + static size_t Compress(TBlock in, TBlock out) { + int rc = LZ4_compress_default( in.data(), out.data(), SafeIntegerCast<int>(in.size()), SafeIntegerCast<int>(out.size())); - Y_ENSURE(rc != 0, "lz4 compression failed"); - return rc; - } - - static void Decompress(TBlock in, TBlock out) { - int rc = LZ4_decompress_safe( + Y_ENSURE(rc != 0, "lz4 compression failed"); + return rc; + } + + static void Decompress(TBlock in, TBlock out) { + int rc = LZ4_decompress_safe( in.data(), out.data(), SafeIntegerCast<int>(in.size()), SafeIntegerCast<int>(out.size())); - Y_ENSURE(rc >= 0, "the lz4 stream is detected malformed"); - } - }; - - /////////////////////////////////////////////////////////////////////////////// - // ZSTD - /////////////////////////////////////////////////////////////////////////////// - struct TZstdCodec { - static const int LEVEL = 11; - - static size_t MaxCompressedLength(size_t in) { - return ZSTD_compressBound(in); - } - - static size_t Compress(TBlock in, TBlock out) { + Y_ENSURE(rc >= 0, "the lz4 stream is detected malformed"); + } + }; + + /////////////////////////////////////////////////////////////////////////////// + // ZSTD + /////////////////////////////////////////////////////////////////////////////// + struct TZstdCodec { + static const int LEVEL = 11; + + static size_t MaxCompressedLength(size_t in) { + return ZSTD_compressBound(in); + } + + static size_t Compress(TBlock in, TBlock out) { size_t rc = ZSTD_compress(out.data(), out.size(), in.data(), in.size(), LEVEL); - if (Y_UNLIKELY(ZSTD_isError(rc))) { + if (Y_UNLIKELY(ZSTD_isError(rc))) { ythrow yexception() << TStringBuf("zstd compression failed: ") - << ZSTD_getErrorName(rc); - } - return rc; - } - - static void Decompress(TBlock in, TBlock out) { + << ZSTD_getErrorName(rc); + } + return rc; + } + + static void Decompress(TBlock in, TBlock out) { size_t rc = ZSTD_decompress(out.data(), out.size(), in.data(), in.size()); - if (Y_UNLIKELY(ZSTD_isError(rc))) { + if (Y_UNLIKELY(ZSTD_isError(rc))) { ythrow yexception() << TStringBuf("zstd decompression failed: ") - << ZSTD_getErrorName(rc); - } + << ZSTD_getErrorName(rc); + } Y_ENSURE(rc == out.size(), "zstd decompressed wrong size"); - } - }; - - /////////////////////////////////////////////////////////////////////////////// - // ZLIB - /////////////////////////////////////////////////////////////////////////////// - struct TZlibCodec { - static const int LEVEL = 6; - - static size_t MaxCompressedLength(size_t in) { - return compressBound(in); - } - - static size_t Compress(TBlock in, TBlock out) { + } + }; + + /////////////////////////////////////////////////////////////////////////////// + // ZLIB + /////////////////////////////////////////////////////////////////////////////// + struct TZlibCodec { + static const int LEVEL = 6; + + static size_t MaxCompressedLength(size_t in) { + return compressBound(in); + } + + static size_t Compress(TBlock in, TBlock out) { uLong ret = out.size(); - int rc = compress2( + int rc = compress2( reinterpret_cast<Bytef*>(out.data()), - &ret, + &ret, reinterpret_cast<const Bytef*>(in.data()), in.size(), - LEVEL); - Y_ENSURE(rc == Z_OK, "zlib compression failed"); - return ret; - } - - static void Decompress(TBlock in, TBlock out) { + LEVEL); + Y_ENSURE(rc == Z_OK, "zlib compression failed"); + return ret; + } + + static void Decompress(TBlock in, TBlock out) { uLong ret = out.size(); - int rc = uncompress( + int rc = uncompress( reinterpret_cast<Bytef*>(out.data()), - &ret, + &ret, reinterpret_cast<const Bytef*>(in.data()), in.size()); - Y_ENSURE(rc == Z_OK, "zlib decompression failed"); + Y_ENSURE(rc == Z_OK, "zlib decompression failed"); Y_ENSURE(ret == out.size(), "zlib decompressed wrong size"); - } - }; - - // - // Framed streams use next frame structure: - // - // +-----------------+-------------------+============+------------------+ - // | compressed size | uncompressed size | data | check sum | - // +-----------------+-------------------+============+------------------+ - // 4 bytes 4 bytes var len 4 bytes - // - - /////////////////////////////////////////////////////////////////////////////// - // TFramedInputStream - /////////////////////////////////////////////////////////////////////////////// - template <typename TCodecAlg, typename TCheckSumAlg> - class TFramedDecompressStream final: public IWalkInput { - public: - explicit TFramedDecompressStream(IInputStream* in) - : In_(in) - { - } - - private: - size_t DoUnboundedNext(const void** ptr) override { - if (!In_) { - return 0; - } - - TFrameHeader header; - In_->LoadOrFail(&header, sizeof(header)); - - if (header.CompressedSize == 0) { - In_ = nullptr; - return 0; - } - + } + }; + + // + // Framed streams use next frame structure: + // + // +-----------------+-------------------+============+------------------+ + // | compressed size | uncompressed size | data | check sum | + // +-----------------+-------------------+============+------------------+ + // 4 bytes 4 bytes var len 4 bytes + // + + /////////////////////////////////////////////////////////////////////////////// + // TFramedInputStream + /////////////////////////////////////////////////////////////////////////////// + template <typename TCodecAlg, typename TCheckSumAlg> + class TFramedDecompressStream final: public IWalkInput { + public: + explicit TFramedDecompressStream(IInputStream* in) + : In_(in) + { + } + + private: + size_t DoUnboundedNext(const void** ptr) override { + if (!In_) { + return 0; + } + + TFrameHeader header; + In_->LoadOrFail(&header, sizeof(header)); + + if (header.CompressedSize == 0) { + In_ = nullptr; + return 0; + } + Y_ENSURE(header.CompressedSize <= COMPRESSED_FRAME_SIZE_LIMIT, "Compressed frame size is limited to " << HumanReadableSize(COMPRESSED_FRAME_SIZE_LIMIT, SF_BYTES) << " but is " << HumanReadableSize(header.CompressedSize, SF_BYTES)); @@ -216,87 +216,87 @@ namespace NMonitoring { << HumanReadableSize(UNCOMPRESSED_FRAME_SIZE_LIMIT, SF_BYTES) << " but is " << HumanReadableSize(header.UncompressedSize, SF_BYTES)); - Compressed_.Resize(header.CompressedSize); - In_->LoadOrFail(Compressed_.Data(), header.CompressedSize); - - TFrameFooter footer; - In_->LoadOrFail(&footer, sizeof(footer)); - Y_ENSURE(TCheckSumAlg::Check(Compressed_, footer.CheckSum), - "corrupted stream: check sum mismatch"); - - Uncompressed_.Resize(header.UncompressedSize); - TCodecAlg::Decompress(Compressed_, Uncompressed_); - - *ptr = Uncompressed_.Data(); - return Uncompressed_.Size(); - } - - private: - IInputStream* In_; - TBuffer Compressed_; - TBuffer Uncompressed_; - }; - - /////////////////////////////////////////////////////////////////////////////// - // TFramedOutputStream - /////////////////////////////////////////////////////////////////////////////// - template <typename TCodecAlg, typename TCheckSumAlg> + Compressed_.Resize(header.CompressedSize); + In_->LoadOrFail(Compressed_.Data(), header.CompressedSize); + + TFrameFooter footer; + In_->LoadOrFail(&footer, sizeof(footer)); + Y_ENSURE(TCheckSumAlg::Check(Compressed_, footer.CheckSum), + "corrupted stream: check sum mismatch"); + + Uncompressed_.Resize(header.UncompressedSize); + TCodecAlg::Decompress(Compressed_, Uncompressed_); + + *ptr = Uncompressed_.Data(); + return Uncompressed_.Size(); + } + + private: + IInputStream* In_; + TBuffer Compressed_; + TBuffer Uncompressed_; + }; + + /////////////////////////////////////////////////////////////////////////////// + // TFramedOutputStream + /////////////////////////////////////////////////////////////////////////////// + template <typename TCodecAlg, typename TCheckSumAlg> class TFramedCompressStream final: public IFramedCompressStream { - public: - explicit TFramedCompressStream(IOutputStream* out) - : Out_(out) - , Uncompressed_(DEFAULT_FRAME_LEN) - { - } - + public: + explicit TFramedCompressStream(IOutputStream* out) + : Out_(out) + , Uncompressed_(DEFAULT_FRAME_LEN) + { + } + ~TFramedCompressStream() override { - try { - Finish(); - } catch (...) { - } - } - - private: - void DoWrite(const void* buf, size_t len) override { - const char* in = static_cast<const char*>(buf); - - while (len != 0) { - const size_t avail = Uncompressed_.Avail(); - if (len < avail) { - Uncompressed_.Append(in, len); - return; - } - - Uncompressed_.Append(in, avail); - Y_ASSERT(Uncompressed_.Avail() == 0); - - in += avail; - len -= avail; - - WriteCompressedFrame(); - } - } - + try { + Finish(); + } catch (...) { + } + } + + private: + void DoWrite(const void* buf, size_t len) override { + const char* in = static_cast<const char*>(buf); + + while (len != 0) { + const size_t avail = Uncompressed_.Avail(); + if (len < avail) { + Uncompressed_.Append(in, len); + return; + } + + Uncompressed_.Append(in, avail); + Y_ASSERT(Uncompressed_.Avail() == 0); + + in += avail; + len -= avail; + + WriteCompressedFrame(); + } + } + void FlushWithoutEmptyFrame() override { - if (Out_ && !Uncompressed_.Empty()) { - WriteCompressedFrame(); - } - } - + if (Out_ && !Uncompressed_.Empty()) { + WriteCompressedFrame(); + } + } + void FinishAndWriteEmptyFrame() override { - if (Out_) { + if (Out_) { Y_DEFER { - Out_ = nullptr; + Out_ = nullptr; }; if (!Uncompressed_.Empty()) { WriteCompressedFrame(); - } + } WriteEmptyFrame(); - } - } - + } + } + void DoFlush() override { FlushWithoutEmptyFrame(); } @@ -305,79 +305,79 @@ namespace NMonitoring { FinishAndWriteEmptyFrame(); } - void WriteCompressedFrame() { - static const auto framePayload = sizeof(TFrameHeader) + sizeof(TFrameFooter); + void WriteCompressedFrame() { + static const auto framePayload = sizeof(TFrameHeader) + sizeof(TFrameFooter); const auto maxFrameSize = ui64(TCodecAlg::MaxCompressedLength(Uncompressed_.Size())) + framePayload; Y_ENSURE(maxFrameSize <= FRAME_SIZE_LIMIT, "Frame size in encoder is limited to " << HumanReadableSize(FRAME_SIZE_LIMIT, SF_BYTES) << " but is " << HumanReadableSize(maxFrameSize, SF_BYTES)); - + Frame_.Resize(maxFrameSize); - // compress - TBlock compressedBlock = Frame_; - compressedBlock.Skip(sizeof(TFrameHeader)); - compressedBlock.Trunc(TCodecAlg::Compress(Uncompressed_, compressedBlock)); - - // add header - auto header = reinterpret_cast<TFrameHeader*>(Frame_.Data()); + // compress + TBlock compressedBlock = Frame_; + compressedBlock.Skip(sizeof(TFrameHeader)); + compressedBlock.Trunc(TCodecAlg::Compress(Uncompressed_, compressedBlock)); + + // add header + auto header = reinterpret_cast<TFrameHeader*>(Frame_.Data()); header->CompressedSize = SafeIntegerCast<TCompressedSize>(compressedBlock.size()); - header->UncompressedSize = SafeIntegerCast<TUncompressedSize>(Uncompressed_.Size()); - - // add footer - auto footer = reinterpret_cast<TFrameFooter*>( - Frame_.Data() + sizeof(TFrameHeader) + header->CompressedSize); - footer->CheckSum = TCheckSumAlg::Calc(compressedBlock); - - // write - Out_->Write(Frame_.Data(), header->CompressedSize + framePayload); - Uncompressed_.Clear(); - } - - void WriteEmptyFrame() { - static const auto framePayload = sizeof(TFrameHeader) + sizeof(TFrameFooter); - char buf[framePayload] = {0}; - Out_->Write(buf, sizeof(buf)); - } - - private: - IOutputStream* Out_; - TBuffer Uncompressed_; - TBuffer Frame_; - }; - - } - - THolder<IInputStream> CompressedInput(IInputStream* in, ECompression alg) { - switch (alg) { - case ECompression::IDENTITY: - return nullptr; - case ECompression::ZLIB: + header->UncompressedSize = SafeIntegerCast<TUncompressedSize>(Uncompressed_.Size()); + + // add footer + auto footer = reinterpret_cast<TFrameFooter*>( + Frame_.Data() + sizeof(TFrameHeader) + header->CompressedSize); + footer->CheckSum = TCheckSumAlg::Calc(compressedBlock); + + // write + Out_->Write(Frame_.Data(), header->CompressedSize + framePayload); + Uncompressed_.Clear(); + } + + void WriteEmptyFrame() { + static const auto framePayload = sizeof(TFrameHeader) + sizeof(TFrameFooter); + char buf[framePayload] = {0}; + Out_->Write(buf, sizeof(buf)); + } + + private: + IOutputStream* Out_; + TBuffer Uncompressed_; + TBuffer Frame_; + }; + + } + + THolder<IInputStream> CompressedInput(IInputStream* in, ECompression alg) { + switch (alg) { + case ECompression::IDENTITY: + return nullptr; + case ECompression::ZLIB: return MakeHolder<TFramedDecompressStream<TZlibCodec, TAdler32>>(in); - case ECompression::ZSTD: + case ECompression::ZSTD: return MakeHolder<TFramedDecompressStream<TZstdCodec, TXxHash32>>(in); - case ECompression::LZ4: + case ECompression::LZ4: return MakeHolder<TFramedDecompressStream<TLz4Codec, TXxHash32>>(in); - case ECompression::UNKNOWN: - return nullptr; - } - Y_FAIL("invalid compression algorithm"); - } - + case ECompression::UNKNOWN: + return nullptr; + } + Y_FAIL("invalid compression algorithm"); + } + THolder<IFramedCompressStream> CompressedOutput(IOutputStream* out, ECompression alg) { - switch (alg) { - case ECompression::IDENTITY: - return nullptr; - case ECompression::ZLIB: + switch (alg) { + case ECompression::IDENTITY: + return nullptr; + case ECompression::ZLIB: return MakeHolder<TFramedCompressStream<TZlibCodec, TAdler32>>(out); - case ECompression::ZSTD: + case ECompression::ZSTD: return MakeHolder<TFramedCompressStream<TZstdCodec, TXxHash32>>(out); - case ECompression::LZ4: + case ECompression::LZ4: return MakeHolder<TFramedCompressStream<TLz4Codec, TXxHash32>>(out); - case ECompression::UNKNOWN: - return nullptr; - } - Y_FAIL("invalid compression algorithm"); - } - -} + case ECompression::UNKNOWN: + return nullptr; + } + Y_FAIL("invalid compression algorithm"); + } + +} diff --git a/library/cpp/monlib/encode/spack/compression.h b/library/cpp/monlib/encode/spack/compression.h index 1f99b0f5a01..f74d8b424e4 100644 --- a/library/cpp/monlib/encode/spack/compression.h +++ b/library/cpp/monlib/encode/spack/compression.h @@ -1,12 +1,12 @@ -#pragma once - -#include "spack_v1.h" - -#include <util/stream/input.h> -#include <util/stream/output.h> - -namespace NMonitoring { - +#pragma once + +#include "spack_v1.h" + +#include <util/stream/input.h> +#include <util/stream/output.h> + +namespace NMonitoring { + class IFramedCompressStream: public IOutputStream { public: virtual void FlushWithoutEmptyFrame() = 0; diff --git a/library/cpp/monlib/encode/spack/fuzz/main.cpp b/library/cpp/monlib/encode/spack/fuzz/main.cpp index ac64b8b2326..6a14afe71c0 100644 --- a/library/cpp/monlib/encode/spack/fuzz/main.cpp +++ b/library/cpp/monlib/encode/spack/fuzz/main.cpp @@ -1,5 +1,5 @@ -#include <library/cpp/monlib/encode/spack/spack_v1.h> -#include <library/cpp/monlib/encode/fake/fake.h> +#include <library/cpp/monlib/encode/spack/spack_v1.h> +#include <library/cpp/monlib/encode/fake/fake.h> #include <util/stream/mem.h> diff --git a/library/cpp/monlib/encode/spack/fuzz/ya.make b/library/cpp/monlib/encode/spack/fuzz/ya.make index c43ee84f36f..99b63eadd5b 100644 --- a/library/cpp/monlib/encode/spack/fuzz/ya.make +++ b/library/cpp/monlib/encode/spack/fuzz/ya.make @@ -10,8 +10,8 @@ FUZZ_OPTS(-rss_limit_mb=1024) SIZE(MEDIUM) PEERDIR( - library/cpp/monlib/encode/spack - library/cpp/monlib/encode/fake + library/cpp/monlib/encode/spack + library/cpp/monlib/encode/fake ) SRCS( diff --git a/library/cpp/monlib/encode/spack/spack_v1.h b/library/cpp/monlib/encode/spack/spack_v1.h index 628f07de5c6..cf1c9417b90 100644 --- a/library/cpp/monlib/encode/spack/spack_v1.h +++ b/library/cpp/monlib/encode/spack/spack_v1.h @@ -1,107 +1,107 @@ -#pragma once - -#include <library/cpp/monlib/encode/encoder.h> -#include <library/cpp/monlib/encode/format.h> -#include <library/cpp/monlib/metrics/metric.h> - -#include <util/generic/yexception.h> - -// -// format specification available here: -// https://wiki.yandex-team.ru/solomon/api/dataformat/spackv1/ -// - -class IInputStream; -class IOutputStream; - -namespace NMonitoring { +#pragma once + +#include <library/cpp/monlib/encode/encoder.h> +#include <library/cpp/monlib/encode/format.h> +#include <library/cpp/monlib/metrics/metric.h> + +#include <util/generic/yexception.h> + +// +// format specification available here: +// https://wiki.yandex-team.ru/solomon/api/dataformat/spackv1/ +// + +class IInputStream; +class IOutputStream; + +namespace NMonitoring { class TSpackDecodeError: public yexception { }; - constexpr auto EncodeMetricType(EMetricType mt) noexcept { - return static_cast<std::underlying_type_t<EMetricType>>(mt); - } - - EMetricType DecodeMetricType(ui8 byte); - + constexpr auto EncodeMetricType(EMetricType mt) noexcept { + return static_cast<std::underlying_type_t<EMetricType>>(mt); + } + + EMetricType DecodeMetricType(ui8 byte); + [[nodiscard]] bool TryDecodeMetricType(ui8 byte, EMetricType* result); - /////////////////////////////////////////////////////////////////////////////// - // EValueType - /////////////////////////////////////////////////////////////////////////////// - enum class EValueType : ui8 { - NONE = 0x00, - ONE_WITHOUT_TS = 0x01, - ONE_WITH_TS = 0x02, - MANY_WITH_TS = 0x03, - }; - - constexpr auto EncodeValueType(EValueType vt) noexcept { - return static_cast<std::underlying_type_t<EValueType>>(vt); - } - - EValueType DecodeValueType(ui8 byte); - + /////////////////////////////////////////////////////////////////////////////// + // EValueType + /////////////////////////////////////////////////////////////////////////////// + enum class EValueType : ui8 { + NONE = 0x00, + ONE_WITHOUT_TS = 0x01, + ONE_WITH_TS = 0x02, + MANY_WITH_TS = 0x03, + }; + + constexpr auto EncodeValueType(EValueType vt) noexcept { + return static_cast<std::underlying_type_t<EValueType>>(vt); + } + + EValueType DecodeValueType(ui8 byte); + [[nodiscard]] bool TryDecodeValueType(ui8 byte, EValueType* result); - /////////////////////////////////////////////////////////////////////////////// - // ETimePrecision - /////////////////////////////////////////////////////////////////////////////// - enum class ETimePrecision : ui8 { - SECONDS = 0x00, - MILLIS = 0x01, - }; - - constexpr auto EncodeTimePrecision(ETimePrecision tp) noexcept { - return static_cast<std::underlying_type_t<ETimePrecision>>(tp); - } - - ETimePrecision DecodeTimePrecision(ui8 byte); - + /////////////////////////////////////////////////////////////////////////////// + // ETimePrecision + /////////////////////////////////////////////////////////////////////////////// + enum class ETimePrecision : ui8 { + SECONDS = 0x00, + MILLIS = 0x01, + }; + + constexpr auto EncodeTimePrecision(ETimePrecision tp) noexcept { + return static_cast<std::underlying_type_t<ETimePrecision>>(tp); + } + + ETimePrecision DecodeTimePrecision(ui8 byte); + [[nodiscard]] bool TryDecodeTimePrecision(ui8 byte, ETimePrecision* result); - /////////////////////////////////////////////////////////////////////////////// - // ECompression - /////////////////////////////////////////////////////////////////////////////// - ui8 EncodeCompression(ECompression c) noexcept; - - ECompression DecodeCompression(ui8 byte); - + /////////////////////////////////////////////////////////////////////////////// + // ECompression + /////////////////////////////////////////////////////////////////////////////// + ui8 EncodeCompression(ECompression c) noexcept; + + ECompression DecodeCompression(ui8 byte); + [[nodiscard]] bool TryDecodeCompression(ui8 byte, ECompression* result); - /////////////////////////////////////////////////////////////////////////////// - // TSpackHeader - /////////////////////////////////////////////////////////////////////////////// + /////////////////////////////////////////////////////////////////////////////// + // TSpackHeader + /////////////////////////////////////////////////////////////////////////////// struct Y_PACKED TSpackHeader { - ui16 Magic = 0x5053; // "SP" + ui16 Magic = 0x5053; // "SP" ui16 Version; // MSB - major version, LSB - minor version - ui16 HeaderSize = sizeof(TSpackHeader); - ui8 TimePrecision; - ui8 Compression; - ui32 LabelNamesSize; - ui32 LabelValuesSize; - ui32 MetricCount; - ui32 PointsCount; - // add new fields here - }; - + ui16 HeaderSize = sizeof(TSpackHeader); + ui8 TimePrecision; + ui8 Compression; + ui32 LabelNamesSize; + ui32 LabelValuesSize; + ui32 MetricCount; + ui32 PointsCount; + // add new fields here + }; + enum ESpackV1Version: ui16 { SV1_00 = 0x0100, SV1_01 = 0x0101, SV1_02 = 0x0102 }; - IMetricEncoderPtr EncoderSpackV1( - IOutputStream* out, - ETimePrecision timePrecision, + IMetricEncoderPtr EncoderSpackV1( + IOutputStream* out, + ETimePrecision timePrecision, ECompression compression, - EMetricsMergingMode mergingMode = EMetricsMergingMode::DEFAULT + EMetricsMergingMode mergingMode = EMetricsMergingMode::DEFAULT ); - + IMetricEncoderPtr EncoderSpackV12( IOutputStream* out, ETimePrecision timePrecision, @@ -109,7 +109,7 @@ namespace NMonitoring { EMetricsMergingMode mergingMode = EMetricsMergingMode::DEFAULT, TStringBuf metricNameLabel = "name" ); - + void DecodeSpackV1(IInputStream* in, IMetricConsumer* c, TStringBuf metricNameLabel = "name"); -} +} diff --git a/library/cpp/monlib/encode/spack/spack_v1_decoder.cpp b/library/cpp/monlib/encode/spack/spack_v1_decoder.cpp index a6dadc08a8a..1f445fc80da 100644 --- a/library/cpp/monlib/encode/spack/spack_v1_decoder.cpp +++ b/library/cpp/monlib/encode/spack/spack_v1_decoder.cpp @@ -1,189 +1,189 @@ -#include "spack_v1.h" -#include "varint.h" -#include "compression.h" - -#include <library/cpp/monlib/encode/buffered/string_pool.h> +#include "spack_v1.h" +#include "varint.h" +#include "compression.h" + +#include <library/cpp/monlib/encode/buffered/string_pool.h> #include <library/cpp/monlib/exception/exception.h> -#include <library/cpp/monlib/metrics/histogram_collector.h> +#include <library/cpp/monlib/metrics/histogram_collector.h> #include <library/cpp/monlib/metrics/metric.h> - -#include <util/generic/yexception.h> -#include <util/generic/buffer.h> + +#include <util/generic/yexception.h> +#include <util/generic/buffer.h> #include <util/generic/size_literals.h> #include <util/stream/format.h> - -#ifndef _little_endian_ -#error Unsupported platform -#endif - -namespace NMonitoring { - namespace { + +#ifndef _little_endian_ +#error Unsupported platform +#endif + +namespace NMonitoring { + namespace { #define DECODE_ENSURE(COND, ...) MONLIB_ENSURE_EX(COND, TSpackDecodeError() << __VA_ARGS__) constexpr ui64 LABEL_SIZE_LIMIT = 128_MB; - /////////////////////////////////////////////////////////////////////// - // TDecoderSpackV1 - /////////////////////////////////////////////////////////////////////// - class TDecoderSpackV1 { - public: + /////////////////////////////////////////////////////////////////////// + // TDecoderSpackV1 + /////////////////////////////////////////////////////////////////////// + class TDecoderSpackV1 { + public: TDecoderSpackV1(IInputStream* in, TStringBuf metricNameLabel) - : In_(in) + : In_(in) , MetricNameLabel_(metricNameLabel) - { - } - - void Decode(IMetricConsumer* c) { - c->OnStreamBegin(); - - // (1) read header + { + } + + void Decode(IMetricConsumer* c) { + c->OnStreamBegin(); + + // (1) read header size_t readBytes = In_->Read(&Header_, sizeof(Header_)); DECODE_ENSURE(readBytes == sizeof(Header_), "not enough data in input stream to read header"); - + ui8 version = ((Header_.Version >> 8) & 0xff); DECODE_ENSURE(version == 1, "versions mismatch (expected: 1, got: " << +version << ')'); - + DECODE_ENSURE(Header_.HeaderSize >= sizeof(Header_), "invalid header size"); if (size_t skipBytes = Header_.HeaderSize - sizeof(Header_)) { DECODE_ENSURE(In_->Skip(skipBytes) == skipBytes, "input stream unexpectedly ended"); - } - - if (Header_.MetricCount == 0) { - // emulate empty stream - c->OnStreamEnd(); - return; - } - - // if compression enabled all below reads must go throught decompressor + } + + if (Header_.MetricCount == 0) { + // emulate empty stream + c->OnStreamEnd(); + return; + } + + // if compression enabled all below reads must go throught decompressor auto compressedIn = CompressedInput(In_, DecodeCompression(Header_.Compression)); - if (compressedIn) { - In_ = compressedIn.Get(); - } - + if (compressedIn) { + In_ = compressedIn.Get(); + } + TimePrecision_ = DecodeTimePrecision(Header_.TimePrecision); - + const ui64 labelSizeTotal = ui64(Header_.LabelNamesSize) + Header_.LabelValuesSize; DECODE_ENSURE(labelSizeTotal <= LABEL_SIZE_LIMIT, "Label names & values size of " << HumanReadableSize(labelSizeTotal, SF_BYTES) << " exceeds the limit which is " << HumanReadableSize(LABEL_SIZE_LIMIT, SF_BYTES)); - // (2) read string pools + // (2) read string pools TVector<char> namesBuf(Header_.LabelNamesSize); readBytes = In_->Load(namesBuf.data(), namesBuf.size()); DECODE_ENSURE(readBytes == Header_.LabelNamesSize, "not enough data to read label names pool"); TStringPool labelNames(namesBuf.data(), namesBuf.size()); - + TVector<char> valuesBuf(Header_.LabelValuesSize); readBytes = In_->Load(valuesBuf.data(), valuesBuf.size()); DECODE_ENSURE(readBytes == Header_.LabelValuesSize, "not enough data to read label values pool"); TStringPool labelValues(valuesBuf.data(), valuesBuf.size()); - - // (3) read common time - c->OnCommonTime(ReadTime()); - - // (4) read common labels - if (ui32 commonLabelsCount = ReadVarint()) { + + // (3) read common time + c->OnCommonTime(ReadTime()); + + // (4) read common labels + if (ui32 commonLabelsCount = ReadVarint()) { c->OnLabelsBegin(); - ReadLabels(labelNames, labelValues, commonLabelsCount, c); + ReadLabels(labelNames, labelValues, commonLabelsCount, c); c->OnLabelsEnd(); - } - - // (5) read metrics - ReadMetrics(labelNames, labelValues, c); - c->OnStreamEnd(); - } - - private: - void ReadMetrics( - const TStringPool& labelNames, - const TStringPool& labelValues, - IMetricConsumer* c) - { - for (ui32 i = 0; i < Header_.MetricCount; i++) { - // (5.1) types byte - ui8 typesByte = ReadFixed<ui8>(); - EMetricType metricType = DecodeMetricType(typesByte >> 2); - EValueType valueType = DecodeValueType(typesByte & 0x03); - - c->OnMetricBegin(metricType); - - // TODO: use it - ReadFixed<ui8>(); // skip flags byte - + } + + // (5) read metrics + ReadMetrics(labelNames, labelValues, c); + c->OnStreamEnd(); + } + + private: + void ReadMetrics( + const TStringPool& labelNames, + const TStringPool& labelValues, + IMetricConsumer* c) + { + for (ui32 i = 0; i < Header_.MetricCount; i++) { + // (5.1) types byte + ui8 typesByte = ReadFixed<ui8>(); + EMetricType metricType = DecodeMetricType(typesByte >> 2); + EValueType valueType = DecodeValueType(typesByte & 0x03); + + c->OnMetricBegin(metricType); + + // TODO: use it + ReadFixed<ui8>(); // skip flags byte + auto metricNameValueIndex = std::numeric_limits<ui32>::max(); if (Header_.Version >= SV1_02) { metricNameValueIndex = ReadVarint(); } - // (5.2) labels - ui32 labelsCount = ReadVarint(); + // (5.2) labels + ui32 labelsCount = ReadVarint(); DECODE_ENSURE(Header_.Version >= SV1_02 || labelsCount > 0, "metric #" << i << " has no labels"); c->OnLabelsBegin(); if (Header_.Version >= SV1_02) { c->OnLabel(MetricNameLabel_, labelValues.Get(metricNameValueIndex)); } - ReadLabels(labelNames, labelValues, labelsCount, c); + ReadLabels(labelNames, labelValues, labelsCount, c); c->OnLabelsEnd(); - - // (5.3) values - switch (valueType) { - case EValueType::NONE: - break; - case EValueType::ONE_WITHOUT_TS: - ReadValue(metricType, TInstant::Zero(), c); - break; - case EValueType::ONE_WITH_TS: { - TInstant time = ReadTime(); - ReadValue(metricType, time, c); - break; - } - case EValueType::MANY_WITH_TS: { - ui32 pointsCount = ReadVarint(); - for (ui32 i = 0; i < pointsCount; i++) { - TInstant time = ReadTime(); - ReadValue(metricType, time, c); - } - break; - } - } - - c->OnMetricEnd(); - } - } - - void ReadValue(EMetricType metricType, TInstant time, IMetricConsumer* c) { - switch (metricType) { - case EMetricType::GAUGE: - c->OnDouble(time, ReadFixed<double>()); - break; - - case EMetricType::IGAUGE: - c->OnInt64(time, ReadFixed<i64>()); - break; - - case EMetricType::COUNTER: - case EMetricType::RATE: - c->OnUint64(time, ReadFixed<ui64>()); - break; - - case EMetricType::DSUMMARY: + + // (5.3) values + switch (valueType) { + case EValueType::NONE: + break; + case EValueType::ONE_WITHOUT_TS: + ReadValue(metricType, TInstant::Zero(), c); + break; + case EValueType::ONE_WITH_TS: { + TInstant time = ReadTime(); + ReadValue(metricType, time, c); + break; + } + case EValueType::MANY_WITH_TS: { + ui32 pointsCount = ReadVarint(); + for (ui32 i = 0; i < pointsCount; i++) { + TInstant time = ReadTime(); + ReadValue(metricType, time, c); + } + break; + } + } + + c->OnMetricEnd(); + } + } + + void ReadValue(EMetricType metricType, TInstant time, IMetricConsumer* c) { + switch (metricType) { + case EMetricType::GAUGE: + c->OnDouble(time, ReadFixed<double>()); + break; + + case EMetricType::IGAUGE: + c->OnInt64(time, ReadFixed<i64>()); + break; + + case EMetricType::COUNTER: + case EMetricType::RATE: + c->OnUint64(time, ReadFixed<ui64>()); + break; + + case EMetricType::DSUMMARY: c->OnSummaryDouble(time, ReadSummaryDouble()); break; - case EMetricType::HIST: - case EMetricType::HIST_RATE: - c->OnHistogram(time, ReadHistogram()); - break; - + case EMetricType::HIST: + case EMetricType::HIST_RATE: + c->OnHistogram(time, ReadHistogram()); + break; + case EMetricType::LOGHIST: c->OnLogHistogram(time, ReadLogHistogram()); break; - default: + default: throw TSpackDecodeError() << "Unsupported metric type: " << metricType; - } - } - + } + } + ISummaryDoubleSnapshotPtr ReadSummaryDouble() { ui64 count = ReadFixed<ui64>(); double sum = ReadFixed<double>(); @@ -198,10 +198,10 @@ namespace NMonitoring { ui64 zerosCount = ReadFixed<ui64>(); int startPower = static_cast<int>(ReadVarint()); ui32 count = ReadVarint(); - // see https://a.yandex-team.ru/arc/trunk/arcadia/infra/yasm/stockpile_client/points.cpp?rev=r8593154#L31 - // and https://a.yandex-team.ru/arc/trunk/arcadia/infra/yasm/common/points/hgram/normal/normal.h?rev=r8268697#L9 - // TODO: share this constant value - Y_ENSURE(count <= 100u, "more than 100 buckets in log histogram: " << count); + // see https://a.yandex-team.ru/arc/trunk/arcadia/infra/yasm/stockpile_client/points.cpp?rev=r8593154#L31 + // and https://a.yandex-team.ru/arc/trunk/arcadia/infra/yasm/common/points/hgram/normal/normal.h?rev=r8268697#L9 + // TODO: share this constant value + Y_ENSURE(count <= 100u, "more than 100 buckets in log histogram: " << count); TVector<double> buckets; buckets.reserve(count); for (ui32 i = 0; i < count; ++i) { @@ -210,17 +210,17 @@ namespace NMonitoring { return MakeIntrusive<TLogHistogramSnapshot>(base, zerosCount, startPower, std::move(buckets)); } - IHistogramSnapshotPtr ReadHistogram() { - ui32 bucketsCount = ReadVarint(); + IHistogramSnapshotPtr ReadHistogram() { + ui32 bucketsCount = ReadVarint(); auto s = TExplicitHistogramSnapshot::New(bucketsCount); - + if (SV1_00 == Header_.Version) { // v1.0 for (ui32 i = 0; i < bucketsCount; i++) { i64 bound = ReadFixed<i64>(); double doubleBound = (bound != Max<i64>()) ? static_cast<double>(bound) : Max<double>(); - + (*s)[i].first = doubleBound; } } else { @@ -228,62 +228,62 @@ namespace NMonitoring { double doubleBound = ReadFixed<double>(); (*s)[i].first = doubleBound; } - } - - - // values - for (ui32 i = 0; i < bucketsCount; i++) { - (*s)[i].second = ReadFixed<ui64>(); - } - return s; - } - - void ReadLabels( - const TStringPool& labelNames, - const TStringPool& labelValues, - ui32 count, - IMetricConsumer* c) - { - for (ui32 i = 0; i < count; i++) { + } + + + // values + for (ui32 i = 0; i < bucketsCount; i++) { + (*s)[i].second = ReadFixed<ui64>(); + } + return s; + } + + void ReadLabels( + const TStringPool& labelNames, + const TStringPool& labelValues, + ui32 count, + IMetricConsumer* c) + { + for (ui32 i = 0; i < count; i++) { auto nameIdx = ReadVarint(); auto valueIdx = ReadVarint(); c->OnLabel(labelNames.Get(nameIdx), labelValues.Get(valueIdx)); - } - } - - TInstant ReadTime() { - switch (TimePrecision_) { - case ETimePrecision::SECONDS: - return TInstant::Seconds(ReadFixed<ui32>()); - case ETimePrecision::MILLIS: - return TInstant::MilliSeconds(ReadFixed<ui64>()); - } - Y_FAIL("invalid time precision"); - } - - template <typename T> - inline T ReadFixed() { - T value; - size_t readBytes = In_->Load(&value, sizeof(T)); + } + } + + TInstant ReadTime() { + switch (TimePrecision_) { + case ETimePrecision::SECONDS: + return TInstant::Seconds(ReadFixed<ui32>()); + case ETimePrecision::MILLIS: + return TInstant::MilliSeconds(ReadFixed<ui64>()); + } + Y_FAIL("invalid time precision"); + } + + template <typename T> + inline T ReadFixed() { + T value; + size_t readBytes = In_->Load(&value, sizeof(T)); DECODE_ENSURE(readBytes == sizeof(T), "no enough data to read " << TypeName<T>()); - return value; - } - - inline ui32 ReadVarint() { - return ReadVarUInt32(In_); - } - - private: - IInputStream* In_; + return value; + } + + inline ui32 ReadVarint() { + return ReadVarUInt32(In_); + } + + private: + IInputStream* In_; TString MetricNameLabel_; - ETimePrecision TimePrecision_; + ETimePrecision TimePrecision_; TSpackHeader Header_; }; // class TDecoderSpackV1 - + #undef DECODE_ENSURE } // namespace - - EValueType DecodeValueType(ui8 byte) { + + EValueType DecodeValueType(ui8 byte) { EValueType result; if (!TryDecodeValueType(byte, &result)) { throw TSpackDecodeError() << "unknown value type: " << byte; @@ -292,32 +292,32 @@ namespace NMonitoring { } bool TryDecodeValueType(ui8 byte, EValueType* result) { - if (byte == EncodeValueType(EValueType::NONE)) { + if (byte == EncodeValueType(EValueType::NONE)) { if (result) { *result = EValueType::NONE; } return true; - } else if (byte == EncodeValueType(EValueType::ONE_WITHOUT_TS)) { + } else if (byte == EncodeValueType(EValueType::ONE_WITHOUT_TS)) { if (result) { *result = EValueType::ONE_WITHOUT_TS; } return true; - } else if (byte == EncodeValueType(EValueType::ONE_WITH_TS)) { + } else if (byte == EncodeValueType(EValueType::ONE_WITH_TS)) { if (result) { *result = EValueType::ONE_WITH_TS; } return true; - } else if (byte == EncodeValueType(EValueType::MANY_WITH_TS)) { + } else if (byte == EncodeValueType(EValueType::MANY_WITH_TS)) { if (result) { *result = EValueType::MANY_WITH_TS; } return true; - } else { + } else { return false; - } - } - - ETimePrecision DecodeTimePrecision(ui8 byte) { + } + } + + ETimePrecision DecodeTimePrecision(ui8 byte) { ETimePrecision result; if (!TryDecodeTimePrecision(byte, &result)) { throw TSpackDecodeError() << "unknown time precision: " << byte; @@ -326,22 +326,22 @@ namespace NMonitoring { } bool TryDecodeTimePrecision(ui8 byte, ETimePrecision* result) { - if (byte == EncodeTimePrecision(ETimePrecision::SECONDS)) { + if (byte == EncodeTimePrecision(ETimePrecision::SECONDS)) { if (result) { *result = ETimePrecision::SECONDS; } return true; - } else if (byte == EncodeTimePrecision(ETimePrecision::MILLIS)) { + } else if (byte == EncodeTimePrecision(ETimePrecision::MILLIS)) { if (result) { *result = ETimePrecision::MILLIS; } return true; - } else { + } else { return false; - } - } - - EMetricType DecodeMetricType(ui8 byte) { + } + } + + EMetricType DecodeMetricType(ui8 byte) { EMetricType result; if (!TryDecodeMetricType(byte, &result)) { throw TSpackDecodeError() << "unknown metric type: " << byte; @@ -350,37 +350,37 @@ namespace NMonitoring { } bool TryDecodeMetricType(ui8 byte, EMetricType* result) { - if (byte == EncodeMetricType(EMetricType::GAUGE)) { + if (byte == EncodeMetricType(EMetricType::GAUGE)) { if (result) { *result = EMetricType::GAUGE; } return true; - } else if (byte == EncodeMetricType(EMetricType::COUNTER)) { + } else if (byte == EncodeMetricType(EMetricType::COUNTER)) { if (result) { *result = EMetricType::COUNTER; } return true; - } else if (byte == EncodeMetricType(EMetricType::RATE)) { + } else if (byte == EncodeMetricType(EMetricType::RATE)) { if (result) { *result = EMetricType::RATE; } return true; - } else if (byte == EncodeMetricType(EMetricType::IGAUGE)) { + } else if (byte == EncodeMetricType(EMetricType::IGAUGE)) { if (result) { *result = EMetricType::IGAUGE; } return true; - } else if (byte == EncodeMetricType(EMetricType::HIST)) { + } else if (byte == EncodeMetricType(EMetricType::HIST)) { if (result) { *result = EMetricType::HIST; } return true; - } else if (byte == EncodeMetricType(EMetricType::HIST_RATE)) { + } else if (byte == EncodeMetricType(EMetricType::HIST_RATE)) { if (result) { *result = EMetricType::HIST_RATE; } return true; - } else if (byte == EncodeMetricType(EMetricType::DSUMMARY)) { + } else if (byte == EncodeMetricType(EMetricType::DSUMMARY)) { if (result) { *result = EMetricType::DSUMMARY; } @@ -390,33 +390,33 @@ namespace NMonitoring { *result = EMetricType::LOGHIST; } return true; - } else if (byte == EncodeMetricType(EMetricType::UNKNOWN)) { + } else if (byte == EncodeMetricType(EMetricType::UNKNOWN)) { if (result) { *result = EMetricType::UNKNOWN; } return true; - } else { + } else { return false; - } - } - - ui8 EncodeCompression(ECompression c) noexcept { - switch (c) { - case ECompression::IDENTITY: - return 0x00; - case ECompression::ZLIB: - return 0x01; - case ECompression::ZSTD: - return 0x02; - case ECompression::LZ4: - return 0x03; - case ECompression::UNKNOWN: - return Max<ui8>(); - } - Y_FAIL(); // for GCC - } - - ECompression DecodeCompression(ui8 byte) { + } + } + + ui8 EncodeCompression(ECompression c) noexcept { + switch (c) { + case ECompression::IDENTITY: + return 0x00; + case ECompression::ZLIB: + return 0x01; + case ECompression::ZSTD: + return 0x02; + case ECompression::LZ4: + return 0x03; + case ECompression::UNKNOWN: + return Max<ui8>(); + } + Y_FAIL(); // for GCC + } + + ECompression DecodeCompression(ui8 byte) { ECompression result; if (!TryDecodeCompression(byte, &result)) { throw TSpackDecodeError() << "unknown compression alg: " << byte; @@ -425,34 +425,34 @@ namespace NMonitoring { } bool TryDecodeCompression(ui8 byte, ECompression* result) { - if (byte == EncodeCompression(ECompression::IDENTITY)) { + if (byte == EncodeCompression(ECompression::IDENTITY)) { if (result) { *result = ECompression::IDENTITY; } return true; - } else if (byte == EncodeCompression(ECompression::ZLIB)) { + } else if (byte == EncodeCompression(ECompression::ZLIB)) { if (result) { *result = ECompression::ZLIB; } return true; - } else if (byte == EncodeCompression(ECompression::ZSTD)) { + } else if (byte == EncodeCompression(ECompression::ZSTD)) { if (result) { *result = ECompression::ZSTD; } return true; - } else if (byte == EncodeCompression(ECompression::LZ4)) { + } else if (byte == EncodeCompression(ECompression::LZ4)) { if (result) { *result = ECompression::LZ4; } return true; - } else { + } else { return false; - } - } - + } + } + void DecodeSpackV1(IInputStream* in, IMetricConsumer* c, TStringBuf metricNameLabel) { TDecoderSpackV1 decoder(in, metricNameLabel); - decoder.Decode(c); - } - -} + decoder.Decode(c); + } + +} diff --git a/library/cpp/monlib/encode/spack/spack_v1_encoder.cpp b/library/cpp/monlib/encode/spack/spack_v1_encoder.cpp index f4f5b88073a..a2b0bb5f50a 100644 --- a/library/cpp/monlib/encode/spack/spack_v1_encoder.cpp +++ b/library/cpp/monlib/encode/spack/spack_v1_encoder.cpp @@ -1,65 +1,65 @@ -#include "spack_v1.h" -#include "compression.h" -#include "varint.h" - -#include <library/cpp/monlib/encode/buffered/buffered_encoder_base.h> - -#include <util/generic/cast.h> -#include <util/datetime/base.h> -#include <util/string/builder.h> - -#ifndef _little_endian_ -#error Unsupported platform -#endif - -namespace NMonitoring { - namespace { - /////////////////////////////////////////////////////////////////////// - // TEncoderSpackV1 - /////////////////////////////////////////////////////////////////////// +#include "spack_v1.h" +#include "compression.h" +#include "varint.h" + +#include <library/cpp/monlib/encode/buffered/buffered_encoder_base.h> + +#include <util/generic/cast.h> +#include <util/datetime/base.h> +#include <util/string/builder.h> + +#ifndef _little_endian_ +#error Unsupported platform +#endif + +namespace NMonitoring { + namespace { + /////////////////////////////////////////////////////////////////////// + // TEncoderSpackV1 + /////////////////////////////////////////////////////////////////////// class TEncoderSpackV1 final: public TBufferedEncoderBase { - public: - TEncoderSpackV1( - IOutputStream* out, - ETimePrecision timePrecision, + public: + TEncoderSpackV1( + IOutputStream* out, + ETimePrecision timePrecision, ECompression compression, EMetricsMergingMode mergingMode, ESpackV1Version version, TStringBuf metricNameLabel ) - : Out_(out) - , TimePrecision_(timePrecision) - , Compression_(compression) + : Out_(out) + , TimePrecision_(timePrecision) + , Compression_(compression) , Version_(version) , MetricName_(Version_ >= SV1_02 ? LabelNamesPool_.PutIfAbsent(metricNameLabel) : nullptr) - { - MetricsMergingMode_ = mergingMode; + { + MetricsMergingMode_ = mergingMode; LabelNamesPool_.SetSorted(true); LabelValuesPool_.SetSorted(true); - } - + } + ~TEncoderSpackV1() override { - Close(); - } - - private: - void OnDouble(TInstant time, double value) override { + Close(); + } + + private: + void OnDouble(TInstant time, double value) override { TBufferedEncoderBase::OnDouble(time, value); - } - - void OnInt64(TInstant time, i64 value) override { - TBufferedEncoderBase::OnInt64(time, value); - } - - void OnUint64(TInstant time, ui64 value) override { + } + + void OnInt64(TInstant time, i64 value) override { + TBufferedEncoderBase::OnInt64(time, value); + } + + void OnUint64(TInstant time, ui64 value) override { TBufferedEncoderBase::OnUint64(time, value); - } - - void OnHistogram(TInstant time, IHistogramSnapshotPtr snapshot) override { - TBufferedEncoderBase::OnHistogram(time, snapshot); - } - + } + + void OnHistogram(TInstant time, IHistogramSnapshotPtr snapshot) override { + TBufferedEncoderBase::OnHistogram(time, snapshot); + } + void OnSummaryDouble(TInstant time, ISummaryDoubleSnapshotPtr snapshot) override { TBufferedEncoderBase::OnSummaryDouble(time, snapshot); } @@ -68,70 +68,70 @@ namespace NMonitoring { TBufferedEncoderBase::OnLogHistogram(time, snapshot); } - void Close() override { - if (Closed_) { - return; - } - Closed_ = true; - + void Close() override { + if (Closed_) { + return; + } + Closed_ = true; + LabelNamesPool_.Build(); LabelValuesPool_.Build(); - + // Sort all points uniquely by ts -- the size can decrease ui64 pointsCount = 0; - for (TMetric& metric : Metrics_) { - if (metric.TimeSeries.Size() > 1) { - metric.TimeSeries.SortByTs(); + for (TMetric& metric : Metrics_) { + if (metric.TimeSeries.Size() > 1) { + metric.TimeSeries.SortByTs(); } - pointsCount += metric.TimeSeries.Size(); + pointsCount += metric.TimeSeries.Size(); } - // (1) write header - TSpackHeader header; + // (1) write header + TSpackHeader header; header.Version = Version_; - header.TimePrecision = EncodeTimePrecision(TimePrecision_); - header.Compression = EncodeCompression(Compression_); - header.LabelNamesSize = static_cast<ui32>( - LabelNamesPool_.BytesSize() + LabelNamesPool_.Count()); - header.LabelValuesSize = static_cast<ui32>( - LabelValuesPool_.BytesSize() + LabelValuesPool_.Count()); - header.MetricCount = Metrics_.size(); + header.TimePrecision = EncodeTimePrecision(TimePrecision_); + header.Compression = EncodeCompression(Compression_); + header.LabelNamesSize = static_cast<ui32>( + LabelNamesPool_.BytesSize() + LabelNamesPool_.Count()); + header.LabelValuesSize = static_cast<ui32>( + LabelValuesPool_.BytesSize() + LabelValuesPool_.Count()); + header.MetricCount = Metrics_.size(); header.PointsCount = pointsCount; - Out_->Write(&header, sizeof(header)); - - // if compression enabled all below writes must go throught compressor - auto compressedOut = CompressedOutput(Out_, Compression_); - if (compressedOut) { - Out_ = compressedOut.Get(); - } - - // (2) write string pools - auto strPoolWrite = [this](TStringBuf str, ui32, ui32) { - Out_->Write(str); - Out_->Write('\0'); - }; - - LabelNamesPool_.ForEach(strPoolWrite); - LabelValuesPool_.ForEach(strPoolWrite); - - // (3) write common time - WriteTime(CommonTime_); - - // (4) write common labels' indexes + Out_->Write(&header, sizeof(header)); + + // if compression enabled all below writes must go throught compressor + auto compressedOut = CompressedOutput(Out_, Compression_); + if (compressedOut) { + Out_ = compressedOut.Get(); + } + + // (2) write string pools + auto strPoolWrite = [this](TStringBuf str, ui32, ui32) { + Out_->Write(str); + Out_->Write('\0'); + }; + + LabelNamesPool_.ForEach(strPoolWrite); + LabelValuesPool_.ForEach(strPoolWrite); + + // (3) write common time + WriteTime(CommonTime_); + + // (4) write common labels' indexes WriteLabels(CommonLabels_, nullptr); - - // (5) write metrics - // metrics count already written in header - for (TMetric& metric : Metrics_) { - // (5.1) types byte - ui8 typesByte = PackTypes(metric); - Out_->Write(&typesByte, sizeof(typesByte)); - - // TODO: implement - ui8 flagsByte = 0x00; - Out_->Write(&flagsByte, sizeof(flagsByte)); - + + // (5) write metrics + // metrics count already written in header + for (TMetric& metric : Metrics_) { + // (5.1) types byte + ui8 typesByte = PackTypes(metric); + Out_->Write(&typesByte, sizeof(typesByte)); + + // TODO: implement + ui8 flagsByte = 0x00; + Out_->Write(&flagsByte, sizeof(flagsByte)); + // v1.2 format addition — metric name if (Version_ >= SV1_02) { const auto it = FindIf(metric.Labels, [&](const auto& l) { @@ -143,53 +143,53 @@ namespace NMonitoring { WriteVarUInt32(Out_, it->Value->Index); } - // (5.2) labels + // (5.2) labels WriteLabels(metric.Labels, MetricName_); - - // (5.3) values - switch (metric.TimeSeries.Size()) { - case 0: - break; - case 1: { - const auto& point = metric.TimeSeries[0]; - if (point.GetTime() != TInstant::Zero()) { - WriteTime(point.GetTime()); - } - EMetricValueType valueType = metric.TimeSeries.GetValueType(); - WriteValue(metric.MetricType, valueType, point.GetValue()); - break; - } - default: - WriteVarUInt32(Out_, static_cast<ui32>(metric.TimeSeries.Size())); - const TMetricTimeSeries& ts = metric.TimeSeries; - EMetricType metricType = metric.MetricType; - ts.ForEach([this, metricType](TInstant time, EMetricValueType valueType, TMetricValue value) { - // workaround for GCC bug - // https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61636 - this->WriteTime(time); - this->WriteValue(metricType, valueType, value); - }); - break; - } - } - } - - // store metric type and values type in one byte - ui8 PackTypes(const TMetric& metric) { - EValueType valueType; - if (metric.TimeSeries.Empty()) { - valueType = EValueType::NONE; - } else if (metric.TimeSeries.Size() == 1) { - TInstant time = metric.TimeSeries[0].GetTime(); - valueType = (time == TInstant::Zero()) - ? EValueType::ONE_WITHOUT_TS - : EValueType::ONE_WITH_TS; - } else { - valueType = EValueType::MANY_WITH_TS; - } - return (static_cast<ui8>(metric.MetricType) << 2) | static_cast<ui8>(valueType); - } - + + // (5.3) values + switch (metric.TimeSeries.Size()) { + case 0: + break; + case 1: { + const auto& point = metric.TimeSeries[0]; + if (point.GetTime() != TInstant::Zero()) { + WriteTime(point.GetTime()); + } + EMetricValueType valueType = metric.TimeSeries.GetValueType(); + WriteValue(metric.MetricType, valueType, point.GetValue()); + break; + } + default: + WriteVarUInt32(Out_, static_cast<ui32>(metric.TimeSeries.Size())); + const TMetricTimeSeries& ts = metric.TimeSeries; + EMetricType metricType = metric.MetricType; + ts.ForEach([this, metricType](TInstant time, EMetricValueType valueType, TMetricValue value) { + // workaround for GCC bug + // https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61636 + this->WriteTime(time); + this->WriteValue(metricType, valueType, value); + }); + break; + } + } + } + + // store metric type and values type in one byte + ui8 PackTypes(const TMetric& metric) { + EValueType valueType; + if (metric.TimeSeries.Empty()) { + valueType = EValueType::NONE; + } else if (metric.TimeSeries.Size() == 1) { + TInstant time = metric.TimeSeries[0].GetTime(); + valueType = (time == TInstant::Zero()) + ? EValueType::ONE_WITHOUT_TS + : EValueType::ONE_WITH_TS; + } else { + valueType = EValueType::MANY_WITH_TS; + } + return (static_cast<ui8>(metric.MetricType) << 2) | static_cast<ui8>(valueType); + } + void WriteLabels(const TPooledLabels& labels, const TPooledStr* skipKey) { WriteVarUInt32(Out_, static_cast<ui32>(skipKey ? labels.size() - 1 : labels.size())); for (auto&& label : labels) { @@ -198,30 +198,30 @@ namespace NMonitoring { } WriteVarUInt32(Out_, label.Key->Index); WriteVarUInt32(Out_, label.Value->Index); - } - } - - void WriteValue(EMetricType metricType, EMetricValueType valueType, TMetricValue value) { - switch (metricType) { - case EMetricType::GAUGE: - WriteFixed(value.AsDouble(valueType)); - break; - - case EMetricType::IGAUGE: - WriteFixed(value.AsInt64(valueType)); - break; - - case EMetricType::COUNTER: - case EMetricType::RATE: - WriteFixed(value.AsUint64(valueType)); - break; - - case EMetricType::HIST: - case EMetricType::HIST_RATE: - WriteHistogram(*value.AsHistogram()); - break; - - case EMetricType::DSUMMARY: + } + } + + void WriteValue(EMetricType metricType, EMetricValueType valueType, TMetricValue value) { + switch (metricType) { + case EMetricType::GAUGE: + WriteFixed(value.AsDouble(valueType)); + break; + + case EMetricType::IGAUGE: + WriteFixed(value.AsInt64(valueType)); + break; + + case EMetricType::COUNTER: + case EMetricType::RATE: + WriteFixed(value.AsUint64(valueType)); + break; + + case EMetricType::HIST: + case EMetricType::HIST_RATE: + WriteHistogram(*value.AsHistogram()); + break; + + case EMetricType::DSUMMARY: WriteSummaryDouble(*value.AsSummaryDouble()); break; @@ -229,44 +229,44 @@ namespace NMonitoring { WriteLogHistogram(*value.AsLogHistogram()); break; - default: - ythrow yexception() << "unsupported metric type: " << metricType; - } - } - - void WriteTime(TInstant instant) { - switch (TimePrecision_) { - case ETimePrecision::SECONDS: { - ui32 time = static_cast<ui32>(instant.Seconds()); - Out_->Write(&time, sizeof(time)); - break; - } - case ETimePrecision::MILLIS: { - ui64 time = static_cast<ui64>(instant.MilliSeconds()); - Out_->Write(&time, sizeof(time)); - } - } - } - - template <typename T> - void WriteFixed(T value) { - Out_->Write(&value, sizeof(value)); - } - - void WriteHistogram(const IHistogramSnapshot& histogram) { - ui32 count = histogram.Count(); - WriteVarUInt32(Out_, count); - - for (ui32 i = 0; i < count; i++) { + default: + ythrow yexception() << "unsupported metric type: " << metricType; + } + } + + void WriteTime(TInstant instant) { + switch (TimePrecision_) { + case ETimePrecision::SECONDS: { + ui32 time = static_cast<ui32>(instant.Seconds()); + Out_->Write(&time, sizeof(time)); + break; + } + case ETimePrecision::MILLIS: { + ui64 time = static_cast<ui64>(instant.MilliSeconds()); + Out_->Write(&time, sizeof(time)); + } + } + } + + template <typename T> + void WriteFixed(T value) { + Out_->Write(&value, sizeof(value)); + } + + void WriteHistogram(const IHistogramSnapshot& histogram) { + ui32 count = histogram.Count(); + WriteVarUInt32(Out_, count); + + for (ui32 i = 0; i < count; i++) { double bound = histogram.UpperBound(i); - Out_->Write(&bound, sizeof(bound)); - } - for (ui32 i = 0; i < count; i++) { - ui64 value = histogram.Value(i); - Out_->Write(&value, sizeof(value)); - } - } - + Out_->Write(&bound, sizeof(bound)); + } + for (ui32 i = 0; i < count; i++) { + ui64 value = histogram.Value(i); + Out_->Write(&value, sizeof(value)); + } + } + void WriteLogHistogram(const TLogHistogramSnapshot& logHist) { WriteFixed(logHist.Base()); WriteFixed(logHist.ZerosCount()); @@ -285,26 +285,26 @@ namespace NMonitoring { WriteFixed(summary.GetLast()); } - private: - IOutputStream* Out_; - ETimePrecision TimePrecision_; - ECompression Compression_; + private: + IOutputStream* Out_; + ETimePrecision TimePrecision_; + ECompression Compression_; ESpackV1Version Version_; const TPooledStr* MetricName_; - bool Closed_ = false; - }; - - } - - IMetricEncoderPtr EncoderSpackV1( - IOutputStream* out, - ETimePrecision timePrecision, + bool Closed_ = false; + }; + + } + + IMetricEncoderPtr EncoderSpackV1( + IOutputStream* out, + ETimePrecision timePrecision, ECompression compression, - EMetricsMergingMode mergingMode + EMetricsMergingMode mergingMode ) { return MakeHolder<TEncoderSpackV1>(out, timePrecision, compression, mergingMode, SV1_01, ""); - } - + } + IMetricEncoderPtr EncoderSpackV12( IOutputStream* out, ETimePrecision timePrecision, @@ -315,4 +315,4 @@ namespace NMonitoring { Y_ENSURE(!metricNameLabel.Empty(), "metricNameLabel can't be empty"); return MakeHolder<TEncoderSpackV1>(out, timePrecision, compression, mergingMode, SV1_02, metricNameLabel); } -} +} diff --git a/library/cpp/monlib/encode/spack/spack_v1_ut.cpp b/library/cpp/monlib/encode/spack/spack_v1_ut.cpp index 0981ac90e2e..fe778eb7e02 100644 --- a/library/cpp/monlib/encode/spack/spack_v1_ut.cpp +++ b/library/cpp/monlib/encode/spack/spack_v1_ut.cpp @@ -1,67 +1,67 @@ -#include "spack_v1.h" - -#include <library/cpp/monlib/encode/protobuf/protobuf.h> -#include <library/cpp/monlib/metrics/labels.h> -#include <library/cpp/monlib/metrics/metric.h> - +#include "spack_v1.h" + +#include <library/cpp/monlib/encode/protobuf/protobuf.h> +#include <library/cpp/monlib/metrics/labels.h> +#include <library/cpp/monlib/metrics/metric.h> + #include <library/cpp/testing/unittest/registar.h> - -#include <util/generic/buffer.h> -#include <util/stream/buffer.h> -#include <util/string/hex.h> - -#include <utility> - -using namespace NMonitoring; - -#define UNIT_ASSERT_BINARY_EQUALS(a, b) \ - do { \ - auto size = Y_ARRAY_SIZE(b); \ - if (Y_UNLIKELY(::memcmp(a, b, size) != 0)) { \ - auto as = HexEncode(a, size); \ - auto bs = HexEncode(b, size); \ - UNIT_FAIL_IMPL("equal assertion failed " #a " == " #b, \ - "\n actual: " << as << "\nexpected: " << bs); \ - } \ - } while (0) - -void AssertLabelEqual(const NProto::TLabel& l, TStringBuf name, TStringBuf value) { - UNIT_ASSERT_STRINGS_EQUAL(l.GetName(), name); - UNIT_ASSERT_STRINGS_EQUAL(l.GetValue(), value); -} - -void AssertPointEqual(const NProto::TPoint& p, TInstant time, double value) { - UNIT_ASSERT_VALUES_EQUAL(p.GetTime(), time.MilliSeconds()); - UNIT_ASSERT_EQUAL(p.GetValueCase(), NProto::TPoint::kFloat64); - UNIT_ASSERT_DOUBLES_EQUAL(p.GetFloat64(), value, std::numeric_limits<double>::epsilon()); -} - -void AssertPointEqual(const NProto::TPoint& p, TInstant time, ui64 value) { - UNIT_ASSERT_VALUES_EQUAL(p.GetTime(), time.MilliSeconds()); - UNIT_ASSERT_EQUAL(p.GetValueCase(), NProto::TPoint::kUint64); - UNIT_ASSERT_VALUES_EQUAL(p.GetUint64(), value); -} - -void AssertPointEqual(const NProto::TPoint& p, TInstant time, i64 value) { - UNIT_ASSERT_VALUES_EQUAL(p.GetTime(), time.MilliSeconds()); - UNIT_ASSERT_EQUAL(p.GetValueCase(), NProto::TPoint::kInt64); - UNIT_ASSERT_VALUES_EQUAL(p.GetInt64(), value); -} - + +#include <util/generic/buffer.h> +#include <util/stream/buffer.h> +#include <util/string/hex.h> + +#include <utility> + +using namespace NMonitoring; + +#define UNIT_ASSERT_BINARY_EQUALS(a, b) \ + do { \ + auto size = Y_ARRAY_SIZE(b); \ + if (Y_UNLIKELY(::memcmp(a, b, size) != 0)) { \ + auto as = HexEncode(a, size); \ + auto bs = HexEncode(b, size); \ + UNIT_FAIL_IMPL("equal assertion failed " #a " == " #b, \ + "\n actual: " << as << "\nexpected: " << bs); \ + } \ + } while (0) + +void AssertLabelEqual(const NProto::TLabel& l, TStringBuf name, TStringBuf value) { + UNIT_ASSERT_STRINGS_EQUAL(l.GetName(), name); + UNIT_ASSERT_STRINGS_EQUAL(l.GetValue(), value); +} + +void AssertPointEqual(const NProto::TPoint& p, TInstant time, double value) { + UNIT_ASSERT_VALUES_EQUAL(p.GetTime(), time.MilliSeconds()); + UNIT_ASSERT_EQUAL(p.GetValueCase(), NProto::TPoint::kFloat64); + UNIT_ASSERT_DOUBLES_EQUAL(p.GetFloat64(), value, std::numeric_limits<double>::epsilon()); +} + +void AssertPointEqual(const NProto::TPoint& p, TInstant time, ui64 value) { + UNIT_ASSERT_VALUES_EQUAL(p.GetTime(), time.MilliSeconds()); + UNIT_ASSERT_EQUAL(p.GetValueCase(), NProto::TPoint::kUint64); + UNIT_ASSERT_VALUES_EQUAL(p.GetUint64(), value); +} + +void AssertPointEqual(const NProto::TPoint& p, TInstant time, i64 value) { + UNIT_ASSERT_VALUES_EQUAL(p.GetTime(), time.MilliSeconds()); + UNIT_ASSERT_EQUAL(p.GetValueCase(), NProto::TPoint::kInt64); + UNIT_ASSERT_VALUES_EQUAL(p.GetInt64(), value); +} + Y_UNIT_TEST_SUITE(TSpackTest) { ui8 expectedHeader_v1_0[] = { - 0x53, 0x50, // magic "SP" (fixed ui16) + 0x53, 0x50, // magic "SP" (fixed ui16) // minor, major - 0x00, 0x01, // version (fixed ui16) - 0x18, 0x00, // header size (fixed ui16) - 0x00, // time precision (fixed ui8) - 0x00, // compression algorithm (fixed ui8) - 0x0d, 0x00, 0x00, 0x00, // label names size (fixed ui32) + 0x00, 0x01, // version (fixed ui16) + 0x18, 0x00, // header size (fixed ui16) + 0x00, // time precision (fixed ui8) + 0x00, // compression algorithm (fixed ui8) + 0x0d, 0x00, 0x00, 0x00, // label names size (fixed ui32) 0x40, 0x00, 0x00, 0x00, // labels values size (fixed ui32) - 0x08, 0x00, 0x00, 0x00, // metric count (fixed ui32) + 0x08, 0x00, 0x00, 0x00, // metric count (fixed ui32) 0x08, 0x00, 0x00, 0x00, // points count (fixed ui32) - }; - + }; + ui8 expectedHeader[] = { 0x53, 0x50, // magic "SP" (fixed ui16) // minor, major @@ -69,105 +69,105 @@ Y_UNIT_TEST_SUITE(TSpackTest) { 0x18, 0x00, // header size (fixed ui16) 0x00, // time precision (fixed ui8) 0x00, // compression algorithm (fixed ui8) - 0x0d, 0x00, 0x00, 0x00, // label names size (fixed ui32) + 0x0d, 0x00, 0x00, 0x00, // label names size (fixed ui32) 0x40, 0x00, 0x00, 0x00, // labels values size (fixed ui32) - 0x08, 0x00, 0x00, 0x00, // metric count (fixed ui32) + 0x08, 0x00, 0x00, 0x00, // metric count (fixed ui32) 0x08, 0x00, 0x00, 0x00, // points count (fixed ui32) }; - ui8 expectedStringPools[] = { - 0x6e, 0x61, 0x6d, 0x65, 0x00, // "name\0" - 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x00, // "project\0" + ui8 expectedStringPools[] = { + 0x6e, 0x61, 0x6d, 0x65, 0x00, // "name\0" + 0x70, 0x72, 0x6f, 0x6a, 0x65, 0x63, 0x74, 0x00, // "project\0" 0x73, 0x6f, 0x6c, 0x6f, 0x6d, 0x6f, 0x6e, 0x00, // "solomon\0" 0x71, 0x31, 0x00, // "q1\0" 0x71, 0x32, 0x00, // "q2\0" 0x71, 0x33, 0x00, // "q3\0" 0x61, 0x6e, 0x73, 0x77, 0x65, 0x72, 0x00, // "answer\0" - 0x72, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, // "responseTimeMillis\0" - 0x54, 0x69, 0x6d, 0x65, 0x4d, 0x69, 0x6c, 0x6c, - 0x69, 0x73, 0x00, + 0x72, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, // "responseTimeMillis\0" + 0x54, 0x69, 0x6d, 0x65, 0x4d, 0x69, 0x6c, 0x6c, + 0x69, 0x73, 0x00, 0x62, 0x79, 0x74, 0x65, 0x73, 0x00, // "bytes\0" 0x74, 0x65, 0x6D, 0x70, 0x65, 0x72, 0x61, 0x74, // "temperature\0" 0x75, 0x72, 0x65, 0x00, 0x6d, 0x73, 0x00, // "ms\0" - }; - - ui8 expectedCommonTime[] = { - 0x00, 0x2f, 0x68, 0x59, // common time in seconds (fixed ui32) - }; - - ui8 expectedCommonLabels[] = { - 0x01, // common labels count (varint) - 0x01, // label name index (varint) + }; + + ui8 expectedCommonTime[] = { + 0x00, 0x2f, 0x68, 0x59, // common time in seconds (fixed ui32) + }; + + ui8 expectedCommonLabels[] = { + 0x01, // common labels count (varint) + 0x01, // label name index (varint) 0x00, // label value index (varint) - }; - - ui8 expectedMetric1[] = { - 0x0C, // types (RATE | NONE) (fixed ui8) - 0x00, // flags (fixed ui8) - 0x01, // metric labels count (varint) - 0x00, // label name index (varint) + }; + + ui8 expectedMetric1[] = { + 0x0C, // types (RATE | NONE) (fixed ui8) + 0x00, // flags (fixed ui8) + 0x01, // metric labels count (varint) + 0x00, // label name index (varint) 0x01, // label value index (varint) - }; - - ui8 expectedMetric2[] = { - 0x09, // types (COUNTER | ONE_WITHOUT_TS) (fixed ui8) - 0x00, // flags (fixed ui8) - 0x01, // metric labels count (varint) - 0x00, // label name index (varint) + }; + + ui8 expectedMetric2[] = { + 0x09, // types (COUNTER | ONE_WITHOUT_TS) (fixed ui8) + 0x00, // flags (fixed ui8) + 0x01, // metric labels count (varint) + 0x00, // label name index (varint) 0x02, // label value index (varint) - 0x11, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // value (fixed ui64) - }; - - ui8 expectedMetric3[] = { - 0x0a, // types (COUNTER | ONE_WITH_TS) (fixed ui8) - 0x00, // flags (fixed ui8) - 0x01, // metric labels count (varint) - 0x00, // label name index (varint) - 0x03, // label value index (varint) - 0x0b, 0x63, 0xfe, 0x59, // time in seconds (fixed ui32) - 0x11, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // value (fixed ui64) - }; - - ui8 expectedMetric4[] = { - 0x07, // types (GAUGE | MANY_WITH_TS) (fixed ui8) - 0x00, // flags (fixed ui8) - 0x01, // metric labels count (varint) - 0x00, // label name index (varint) - 0x04, // label value index (varint) - 0x02, // points count (varint) - 0x0b, 0x63, 0xfe, 0x59, // time in seconds (fixed ui32) - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x45, 0x40, // value (double IEEE754) - 0x1a, 0x63, 0xfe, 0x59, // time in seconds (fixed ui32) - 0x00, 0x00, 0x00, 0x00, 0x00, 0x80, 0x4d, 0x40 // value (double IEEE754) - }; - - ui8 expectedMetric5_v1_0[] = { - 0x16, // types (HIST | ONE_WITH_TS) (fixed ui8) - 0x00, // flags (fixed ui8) - 0x01, // metric labels count (varint) - 0x00, // label name index (varint) + 0x11, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // value (fixed ui64) + }; + + ui8 expectedMetric3[] = { + 0x0a, // types (COUNTER | ONE_WITH_TS) (fixed ui8) + 0x00, // flags (fixed ui8) + 0x01, // metric labels count (varint) + 0x00, // label name index (varint) + 0x03, // label value index (varint) + 0x0b, 0x63, 0xfe, 0x59, // time in seconds (fixed ui32) + 0x11, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // value (fixed ui64) + }; + + ui8 expectedMetric4[] = { + 0x07, // types (GAUGE | MANY_WITH_TS) (fixed ui8) + 0x00, // flags (fixed ui8) + 0x01, // metric labels count (varint) + 0x00, // label name index (varint) + 0x04, // label value index (varint) + 0x02, // points count (varint) + 0x0b, 0x63, 0xfe, 0x59, // time in seconds (fixed ui32) + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x45, 0x40, // value (double IEEE754) + 0x1a, 0x63, 0xfe, 0x59, // time in seconds (fixed ui32) + 0x00, 0x00, 0x00, 0x00, 0x00, 0x80, 0x4d, 0x40 // value (double IEEE754) + }; + + ui8 expectedMetric5_v1_0[] = { + 0x16, // types (HIST | ONE_WITH_TS) (fixed ui8) + 0x00, // flags (fixed ui8) + 0x01, // metric labels count (varint) + 0x00, // label name index (varint) 0x05, // label value index (varint) - 0x0b, 0x63, 0xfe, 0x59, // time in seconds (fixed ui32) - 0x06, // histogram buckets count (varint) + 0x0b, 0x63, 0xfe, 0x59, // time in seconds (fixed ui32) + 0x06, // histogram buckets count (varint) 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // histogram bucket bounds (array of fixed ui64) - 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x10, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x7f, - 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // histogram bucket values - 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x53, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - }; - - ui8 expectedMetric5[] = { + 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x10, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x7f, + 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // histogram bucket values + 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x02, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x53, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + }; + + ui8 expectedMetric5[] = { 0x16, // types (HIST | ONE_WITH_TS) (fixed ui8) 0x00, // flags (fixed ui8) - 0x01, // metric labels count (varint) + 0x01, // metric labels count (varint) 0x00, // label name index (varint) 0x05, // label value index (varint) 0x0b, 0x63, 0xfe, 0x59, // time in seconds (fixed ui32) @@ -186,20 +186,20 @@ Y_UNIT_TEST_SUITE(TSpackTest) { 0x53, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, }; - ui8 expectedMetric6[] = { - 0x12, // types (IGAUGE | ONE_WITH_TS) (fixed ui8) - 0x00, // flags (fixed ui8) - 0x01, // metric labels count (varint) - 0x00, // label name index (varint) + ui8 expectedMetric6[] = { + 0x12, // types (IGAUGE | ONE_WITH_TS) (fixed ui8) + 0x00, // flags (fixed ui8) + 0x01, // metric labels count (varint) + 0x00, // label name index (varint) 0x06, // label value index (varint) - 0x0b, 0x63, 0xfe, 0x59, // time in seconds (fixed ui32) - 0x39, 0x05, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // value (fixed i64) - }; - - ui8 expectedMetric7[] = { + 0x0b, 0x63, 0xfe, 0x59, // time in seconds (fixed ui32) + 0x39, 0x05, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // value (fixed i64) + }; + + ui8 expectedMetric7[] = { 0x1e, // types (DSUMMARY | ONE_WITH_TS) (fixed ui8) 0x00, // flags (fixed ui8) - 0x01, // metric labels count (varint) + 0x01, // metric labels count (varint) 0x00, // label name index (varint) 0x07, // label value index (varint) 0x0b, 0x63, 0xfe, 0x59, // time in seconds (fixed ui32) @@ -210,10 +210,10 @@ Y_UNIT_TEST_SUITE(TSpackTest) { 0x33, 0x33, 0x33, 0x33, 0x33, 0x33, 0xd3, 0x3f, // last (fixed double) }; - ui8 expectedMetric8[] = { + ui8 expectedMetric8[] = { 0x26, // types (LOGHIST | ONE_WITH_TS) (fixed ui8) 0x00, // flags (fixed ui8) - 0x01, // metric labels count (variant) + 0x01, // metric labels count (variant) 0x00, // label name index (variant) 0x08, // label value index (variant) 0x0b, 0x63, 0xfe, 0x59, // time in seconds (fixed ui32) @@ -227,31 +227,31 @@ Y_UNIT_TEST_SUITE(TSpackTest) { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xE0, 0x3F, }; - const size_t expectedSize = - Y_ARRAY_SIZE(expectedHeader) + - Y_ARRAY_SIZE(expectedStringPools) + - Y_ARRAY_SIZE(expectedCommonTime) + - Y_ARRAY_SIZE(expectedCommonLabels) + - Y_ARRAY_SIZE(expectedMetric1) + - Y_ARRAY_SIZE(expectedMetric2) + - Y_ARRAY_SIZE(expectedMetric3) + - Y_ARRAY_SIZE(expectedMetric4) + - Y_ARRAY_SIZE(expectedMetric5) + - Y_ARRAY_SIZE(expectedMetric6) + - Y_ARRAY_SIZE(expectedMetric7) + - Y_ARRAY_SIZE(expectedMetric8); - + const size_t expectedSize = + Y_ARRAY_SIZE(expectedHeader) + + Y_ARRAY_SIZE(expectedStringPools) + + Y_ARRAY_SIZE(expectedCommonTime) + + Y_ARRAY_SIZE(expectedCommonLabels) + + Y_ARRAY_SIZE(expectedMetric1) + + Y_ARRAY_SIZE(expectedMetric2) + + Y_ARRAY_SIZE(expectedMetric3) + + Y_ARRAY_SIZE(expectedMetric4) + + Y_ARRAY_SIZE(expectedMetric5) + + Y_ARRAY_SIZE(expectedMetric6) + + Y_ARRAY_SIZE(expectedMetric7) + + Y_ARRAY_SIZE(expectedMetric8); + const TInstant now = TInstant::ParseIso8601Deprecated("2017-11-05T01:02:03Z"); - - // {1: 1, 2: 1, 4: 2, 8: 4, 16: 8, inf: 83} - IHistogramSnapshotPtr TestHistogram() { - auto h = ExponentialHistogram(6, 2); - for (i64 i = 1; i < 100; i++) { - h->Collect(i); - } - return h->Snapshot(); - } - + + // {1: 1, 2: 1, 4: 2, 8: 4, 16: 8, inf: 83} + IHistogramSnapshotPtr TestHistogram() { + auto h = ExponentialHistogram(6, 2); + for (i64 i = 1; i < 100; i++) { + h->Collect(i); + } + return h->Snapshot(); + } + TLogHistogramSnapshotPtr TestLogHistogram() { TVector buckets{0.5, 0.25, 0.25, 0.5}; return MakeIntrusive<TLogHistogramSnapshot>(1.5, 1u, 0, std::move(buckets)); @@ -262,154 +262,154 @@ Y_UNIT_TEST_SUITE(TSpackTest) { } Y_UNIT_TEST(Encode) { - TBuffer buffer; - TBufferOutput out(buffer); - auto e = EncoderSpackV1( - &out, ETimePrecision::SECONDS, ECompression::IDENTITY); - - e->OnStreamBegin(); - { // common time - e->OnCommonTime(TInstant::Seconds(1500000000)); - } - { // common labels - e->OnLabelsBegin(); - e->OnLabel("project", "solomon"); - e->OnLabelsEnd(); - } - { // metric #1 - e->OnMetricBegin(EMetricType::RATE); - { - e->OnLabelsBegin(); - e->OnLabel("name", "q1"); - e->OnLabelsEnd(); - } - e->OnMetricEnd(); - } - { // metric #2 - e->OnMetricBegin(EMetricType::COUNTER); - { - e->OnLabelsBegin(); - e->OnLabel("name", "q2"); - e->OnLabelsEnd(); - } + TBuffer buffer; + TBufferOutput out(buffer); + auto e = EncoderSpackV1( + &out, ETimePrecision::SECONDS, ECompression::IDENTITY); + + e->OnStreamBegin(); + { // common time + e->OnCommonTime(TInstant::Seconds(1500000000)); + } + { // common labels + e->OnLabelsBegin(); + e->OnLabel("project", "solomon"); + e->OnLabelsEnd(); + } + { // metric #1 + e->OnMetricBegin(EMetricType::RATE); + { + e->OnLabelsBegin(); + e->OnLabel("name", "q1"); + e->OnLabelsEnd(); + } + e->OnMetricEnd(); + } + { // metric #2 + e->OnMetricBegin(EMetricType::COUNTER); + { + e->OnLabelsBegin(); + e->OnLabel("name", "q2"); + e->OnLabelsEnd(); + } // Only the last value will be encoded e->OnUint64(TInstant::Zero(), 10); e->OnUint64(TInstant::Zero(), 13); - e->OnUint64(TInstant::Zero(), 17); - e->OnMetricEnd(); - } - { // metric #3 - e->OnMetricBegin(EMetricType::COUNTER); - { - e->OnLabelsBegin(); - e->OnLabel("name", "q3"); - e->OnLabelsEnd(); - } + e->OnUint64(TInstant::Zero(), 17); + e->OnMetricEnd(); + } + { // metric #3 + e->OnMetricBegin(EMetricType::COUNTER); + { + e->OnLabelsBegin(); + e->OnLabel("name", "q3"); + e->OnLabelsEnd(); + } e->OnUint64(now, 10); e->OnUint64(now, 13); - e->OnUint64(now, 17); - e->OnMetricEnd(); - } - { // metric #4 - e->OnMetricBegin(EMetricType::GAUGE); - { - e->OnLabelsBegin(); - e->OnLabel("name", "answer"); - e->OnLabelsEnd(); - } - e->OnDouble(now, 42); - e->OnDouble(now + TDuration::Seconds(15), 59); - e->OnMetricEnd(); - } - { // metric #5 - e->OnMetricBegin(EMetricType::HIST); - { - e->OnLabelsBegin(); - e->OnLabel("name", "responseTimeMillis"); - e->OnLabelsEnd(); - } - - auto histogram = TestHistogram(); - e->OnHistogram(now, histogram); - e->OnMetricEnd(); - } - { // metric #6 - e->OnMetricBegin(EMetricType::IGAUGE); - { - e->OnLabelsBegin(); - e->OnLabel("name", "bytes"); - e->OnLabelsEnd(); - } - e->OnInt64(now, 1337); - e->OnMetricEnd(); - } - { // metric 7 - e->OnMetricBegin(EMetricType::DSUMMARY); + e->OnUint64(now, 17); + e->OnMetricEnd(); + } + { // metric #4 + e->OnMetricBegin(EMetricType::GAUGE); + { + e->OnLabelsBegin(); + e->OnLabel("name", "answer"); + e->OnLabelsEnd(); + } + e->OnDouble(now, 42); + e->OnDouble(now + TDuration::Seconds(15), 59); + e->OnMetricEnd(); + } + { // metric #5 + e->OnMetricBegin(EMetricType::HIST); + { + e->OnLabelsBegin(); + e->OnLabel("name", "responseTimeMillis"); + e->OnLabelsEnd(); + } + + auto histogram = TestHistogram(); + e->OnHistogram(now, histogram); + e->OnMetricEnd(); + } + { // metric #6 + e->OnMetricBegin(EMetricType::IGAUGE); { e->OnLabelsBegin(); - e->OnLabel("name", "temperature"); + e->OnLabel("name", "bytes"); + e->OnLabelsEnd(); + } + e->OnInt64(now, 1337); + e->OnMetricEnd(); + } + { // metric 7 + e->OnMetricBegin(EMetricType::DSUMMARY); + { + e->OnLabelsBegin(); + e->OnLabel("name", "temperature"); e->OnLabelsEnd(); } e->OnSummaryDouble(now, TestSummaryDouble()); - e->OnMetricEnd(); + e->OnMetricEnd(); } - { // metric 8 + { // metric 8 e->OnMetricBegin(EMetricType::LOGHIST); { e->OnLabelsBegin(); - e->OnLabel("name", "ms"); + e->OnLabel("name", "ms"); e->OnLabelsEnd(); } e->OnLogHistogram(now, TestLogHistogram()); e->OnMetricEnd(); } - e->OnStreamEnd(); - e->Close(); - + e->OnStreamEnd(); + e->Close(); + // Cout << "encoded: " << HexEncode(buffer.Data(), buffer.Size()) << Endl; // Cout << "size: " << buffer.Size() << Endl; - - UNIT_ASSERT_VALUES_EQUAL(buffer.Size(), expectedSize); - - ui8* p = reinterpret_cast<ui8*>(buffer.Data()); - UNIT_ASSERT_BINARY_EQUALS(p, expectedHeader); - p += Y_ARRAY_SIZE(expectedHeader); - - UNIT_ASSERT_BINARY_EQUALS(p, expectedStringPools); - p += Y_ARRAY_SIZE(expectedStringPools); - - UNIT_ASSERT_BINARY_EQUALS(p, expectedCommonTime); - p += Y_ARRAY_SIZE(expectedCommonTime); - - UNIT_ASSERT_BINARY_EQUALS(p, expectedCommonLabels); - p += Y_ARRAY_SIZE(expectedCommonLabels); - - UNIT_ASSERT_BINARY_EQUALS(p, expectedMetric1); - p += Y_ARRAY_SIZE(expectedMetric1); - - UNIT_ASSERT_BINARY_EQUALS(p, expectedMetric2); - p += Y_ARRAY_SIZE(expectedMetric2); - - UNIT_ASSERT_BINARY_EQUALS(p, expectedMetric3); - p += Y_ARRAY_SIZE(expectedMetric3); - - UNIT_ASSERT_BINARY_EQUALS(p, expectedMetric4); - p += Y_ARRAY_SIZE(expectedMetric4); - - UNIT_ASSERT_BINARY_EQUALS(p, expectedMetric5); - p += Y_ARRAY_SIZE(expectedMetric5); - - UNIT_ASSERT_BINARY_EQUALS(p, expectedMetric6); - p += Y_ARRAY_SIZE(expectedMetric6); - - UNIT_ASSERT_BINARY_EQUALS(p, expectedMetric7); - p += Y_ARRAY_SIZE(expectedMetric7); - - UNIT_ASSERT_BINARY_EQUALS(p, expectedMetric8); - p += Y_ARRAY_SIZE(expectedMetric8); - } - - NProto::TMultiSamplesList GetMergingMetricSamples(EMetricsMergingMode mergingMode) { + + UNIT_ASSERT_VALUES_EQUAL(buffer.Size(), expectedSize); + + ui8* p = reinterpret_cast<ui8*>(buffer.Data()); + UNIT_ASSERT_BINARY_EQUALS(p, expectedHeader); + p += Y_ARRAY_SIZE(expectedHeader); + + UNIT_ASSERT_BINARY_EQUALS(p, expectedStringPools); + p += Y_ARRAY_SIZE(expectedStringPools); + + UNIT_ASSERT_BINARY_EQUALS(p, expectedCommonTime); + p += Y_ARRAY_SIZE(expectedCommonTime); + + UNIT_ASSERT_BINARY_EQUALS(p, expectedCommonLabels); + p += Y_ARRAY_SIZE(expectedCommonLabels); + + UNIT_ASSERT_BINARY_EQUALS(p, expectedMetric1); + p += Y_ARRAY_SIZE(expectedMetric1); + + UNIT_ASSERT_BINARY_EQUALS(p, expectedMetric2); + p += Y_ARRAY_SIZE(expectedMetric2); + + UNIT_ASSERT_BINARY_EQUALS(p, expectedMetric3); + p += Y_ARRAY_SIZE(expectedMetric3); + + UNIT_ASSERT_BINARY_EQUALS(p, expectedMetric4); + p += Y_ARRAY_SIZE(expectedMetric4); + + UNIT_ASSERT_BINARY_EQUALS(p, expectedMetric5); + p += Y_ARRAY_SIZE(expectedMetric5); + + UNIT_ASSERT_BINARY_EQUALS(p, expectedMetric6); + p += Y_ARRAY_SIZE(expectedMetric6); + + UNIT_ASSERT_BINARY_EQUALS(p, expectedMetric7); + p += Y_ARRAY_SIZE(expectedMetric7); + + UNIT_ASSERT_BINARY_EQUALS(p, expectedMetric8); + p += Y_ARRAY_SIZE(expectedMetric8); + } + + NProto::TMultiSamplesList GetMergingMetricSamples(EMetricsMergingMode mergingMode) { TBuffer buffer; TBufferOutput out(buffer); @@ -422,29 +422,29 @@ Y_UNIT_TEST_SUITE(TSpackTest) { e->OnStreamBegin(); for (size_t i = 0; i != 3; ++i) { - e->OnMetricBegin(EMetricType::COUNTER); + e->OnMetricBegin(EMetricType::COUNTER); { e->OnLabelsBegin(); - e->OnLabel("name", "my_counter"); + e->OnLabel("name", "my_counter"); e->OnLabelsEnd(); } e->OnUint64(TInstant::Zero() + TDuration::Seconds(i), i + 1); - e->OnMetricEnd(); + e->OnMetricEnd(); } e->OnStreamEnd(); e->Close(); NProto::TMultiSamplesList samples; - IMetricEncoderPtr eProto = EncoderProtobuf(&samples); + IMetricEncoderPtr eProto = EncoderProtobuf(&samples); TBufferInput in(buffer); DecodeSpackV1(&in, eProto.Get()); return samples; } - Y_UNIT_TEST(SpackEncoderMergesMetrics) { + Y_UNIT_TEST(SpackEncoderMergesMetrics) { { - NProto::TMultiSamplesList samples = GetMergingMetricSamples(EMetricsMergingMode::DEFAULT); + NProto::TMultiSamplesList samples = GetMergingMetricSamples(EMetricsMergingMode::DEFAULT); UNIT_ASSERT_EQUAL(samples.SamplesSize(), 3); UNIT_ASSERT_EQUAL(samples.GetSamples(0).GetPoints(0).GetUint64(), 1); @@ -453,7 +453,7 @@ Y_UNIT_TEST_SUITE(TSpackTest) { } { - NProto::TMultiSamplesList samples = GetMergingMetricSamples(EMetricsMergingMode::MERGE_METRICS); + NProto::TMultiSamplesList samples = GetMergingMetricSamples(EMetricsMergingMode::MERGE_METRICS); UNIT_ASSERT_EQUAL(samples.SamplesSize(), 1); @@ -465,29 +465,29 @@ Y_UNIT_TEST_SUITE(TSpackTest) { } void DecodeDataToSamples(NProto::TMultiSamplesList & samples, ui16 version) { - IMetricEncoderPtr e = EncoderProtobuf(&samples); - + IMetricEncoderPtr e = EncoderProtobuf(&samples); + TBuffer data(expectedSize); if (SV1_00 == version) { // v1.0 data.Append(reinterpret_cast<char*>(expectedHeader_v1_0), Y_ARRAY_SIZE(expectedHeader_v1_0)); } else { - data.Append(reinterpret_cast<char*>(expectedHeader), Y_ARRAY_SIZE(expectedHeader)); + data.Append(reinterpret_cast<char*>(expectedHeader), Y_ARRAY_SIZE(expectedHeader)); } data.Append(reinterpret_cast<char*>(expectedStringPools), Y_ARRAY_SIZE(expectedStringPools)); data.Append(reinterpret_cast<char*>(expectedCommonTime), Y_ARRAY_SIZE(expectedCommonTime)); data.Append(reinterpret_cast<char*>(expectedCommonLabels), Y_ARRAY_SIZE(expectedCommonLabels)); - data.Append(reinterpret_cast<char*>(expectedMetric1), Y_ARRAY_SIZE(expectedMetric1)); - data.Append(reinterpret_cast<char*>(expectedMetric2), Y_ARRAY_SIZE(expectedMetric2)); - data.Append(reinterpret_cast<char*>(expectedMetric3), Y_ARRAY_SIZE(expectedMetric3)); - data.Append(reinterpret_cast<char*>(expectedMetric4), Y_ARRAY_SIZE(expectedMetric4)); + data.Append(reinterpret_cast<char*>(expectedMetric1), Y_ARRAY_SIZE(expectedMetric1)); + data.Append(reinterpret_cast<char*>(expectedMetric2), Y_ARRAY_SIZE(expectedMetric2)); + data.Append(reinterpret_cast<char*>(expectedMetric3), Y_ARRAY_SIZE(expectedMetric3)); + data.Append(reinterpret_cast<char*>(expectedMetric4), Y_ARRAY_SIZE(expectedMetric4)); if (SV1_00 == version) { // v1.0 - data.Append(reinterpret_cast<char*>(expectedMetric5_v1_0), Y_ARRAY_SIZE(expectedMetric5_v1_0)); + data.Append(reinterpret_cast<char*>(expectedMetric5_v1_0), Y_ARRAY_SIZE(expectedMetric5_v1_0)); } else { - data.Append(reinterpret_cast<char*>(expectedMetric5), Y_ARRAY_SIZE(expectedMetric5)); - } - data.Append(reinterpret_cast<char*>(expectedMetric6), Y_ARRAY_SIZE(expectedMetric6)); - data.Append(reinterpret_cast<char*>(expectedMetric7), Y_ARRAY_SIZE(expectedMetric7)); - data.Append(reinterpret_cast<char*>(expectedMetric8), Y_ARRAY_SIZE(expectedMetric8)); + data.Append(reinterpret_cast<char*>(expectedMetric5), Y_ARRAY_SIZE(expectedMetric5)); + } + data.Append(reinterpret_cast<char*>(expectedMetric6), Y_ARRAY_SIZE(expectedMetric6)); + data.Append(reinterpret_cast<char*>(expectedMetric7), Y_ARRAY_SIZE(expectedMetric7)); + data.Append(reinterpret_cast<char*>(expectedMetric8), Y_ARRAY_SIZE(expectedMetric8)); TBufferInput in(data); DecodeSpackV1(&in, e.Get()); } @@ -502,85 +502,85 @@ Y_UNIT_TEST_SUITE(TSpackTest) { NProto::TMultiSamplesList samples; DecodeDataToSamples(samples); - UNIT_ASSERT_VALUES_EQUAL( - TInstant::MilliSeconds(samples.GetCommonTime()), - TInstant::Seconds(1500000000)); - - UNIT_ASSERT_VALUES_EQUAL(samples.CommonLabelsSize(), 1); - AssertLabelEqual(samples.GetCommonLabels(0), "project", "solomon"); - + UNIT_ASSERT_VALUES_EQUAL( + TInstant::MilliSeconds(samples.GetCommonTime()), + TInstant::Seconds(1500000000)); + + UNIT_ASSERT_VALUES_EQUAL(samples.CommonLabelsSize(), 1); + AssertLabelEqual(samples.GetCommonLabels(0), "project", "solomon"); + UNIT_ASSERT_VALUES_EQUAL(samples.SamplesSize(), 8); - { - const NProto::TMultiSample& s = samples.GetSamples(0); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::RATE); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); - AssertLabelEqual(s.GetLabels(0), "name", "q1"); - } - { - const NProto::TMultiSample& s = samples.GetSamples(1); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::COUNTER); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); - AssertLabelEqual(s.GetLabels(0), "name", "q2"); - - UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); - AssertPointEqual(s.GetPoints(0), TInstant::Zero(), ui64(17)); - } - { - const NProto::TMultiSample& s = samples.GetSamples(2); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::COUNTER); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); - AssertLabelEqual(s.GetLabels(0), "name", "q3"); - - UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); - AssertPointEqual(s.GetPoints(0), now, ui64(17)); - } - { - const NProto::TMultiSample& s = samples.GetSamples(3); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); - AssertLabelEqual(s.GetLabels(0), "name", "answer"); - - UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 2); - AssertPointEqual(s.GetPoints(0), now, double(42)); - AssertPointEqual(s.GetPoints(1), now + TDuration::Seconds(15), double(59)); - } - { - const NProto::TMultiSample& s = samples.GetSamples(4); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::HISTOGRAM); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); - AssertLabelEqual(s.GetLabels(0), "name", "responseTimeMillis"); - - UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); - - const NProto::TPoint& p = s.GetPoints(0); - UNIT_ASSERT_VALUES_EQUAL(p.GetTime(), now.MilliSeconds()); - UNIT_ASSERT_EQUAL(p.GetValueCase(), NProto::TPoint::kHistogram); - - auto histogram = TestHistogram(); - - const NProto::THistogram& pointHistogram = p.GetHistogram(); - UNIT_ASSERT_VALUES_EQUAL(pointHistogram.BoundsSize(), histogram->Count()); - UNIT_ASSERT_VALUES_EQUAL(pointHistogram.ValuesSize(), histogram->Count()); - - for (size_t i = 0; i < pointHistogram.BoundsSize(); i++) { - UNIT_ASSERT_DOUBLES_EQUAL(pointHistogram.GetBounds(i), histogram->UpperBound(i), Min<double>()); - UNIT_ASSERT_VALUES_EQUAL(pointHistogram.GetValues(i), histogram->Value(i)); - } - } - { - const NProto::TMultiSample& s = samples.GetSamples(5); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::IGAUGE); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); - AssertLabelEqual(s.GetLabels(0), "name", "bytes"); - - UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); - AssertPointEqual(s.GetPoints(0), now, i64(1337)); - } + { + const NProto::TMultiSample& s = samples.GetSamples(0); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::RATE); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); + AssertLabelEqual(s.GetLabels(0), "name", "q1"); + } + { + const NProto::TMultiSample& s = samples.GetSamples(1); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::COUNTER); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); + AssertLabelEqual(s.GetLabels(0), "name", "q2"); + + UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); + AssertPointEqual(s.GetPoints(0), TInstant::Zero(), ui64(17)); + } + { + const NProto::TMultiSample& s = samples.GetSamples(2); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::COUNTER); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); + AssertLabelEqual(s.GetLabels(0), "name", "q3"); + + UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); + AssertPointEqual(s.GetPoints(0), now, ui64(17)); + } + { + const NProto::TMultiSample& s = samples.GetSamples(3); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); + AssertLabelEqual(s.GetLabels(0), "name", "answer"); + + UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 2); + AssertPointEqual(s.GetPoints(0), now, double(42)); + AssertPointEqual(s.GetPoints(1), now + TDuration::Seconds(15), double(59)); + } + { + const NProto::TMultiSample& s = samples.GetSamples(4); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::HISTOGRAM); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); + AssertLabelEqual(s.GetLabels(0), "name", "responseTimeMillis"); + + UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); + + const NProto::TPoint& p = s.GetPoints(0); + UNIT_ASSERT_VALUES_EQUAL(p.GetTime(), now.MilliSeconds()); + UNIT_ASSERT_EQUAL(p.GetValueCase(), NProto::TPoint::kHistogram); + + auto histogram = TestHistogram(); + + const NProto::THistogram& pointHistogram = p.GetHistogram(); + UNIT_ASSERT_VALUES_EQUAL(pointHistogram.BoundsSize(), histogram->Count()); + UNIT_ASSERT_VALUES_EQUAL(pointHistogram.ValuesSize(), histogram->Count()); + + for (size_t i = 0; i < pointHistogram.BoundsSize(); i++) { + UNIT_ASSERT_DOUBLES_EQUAL(pointHistogram.GetBounds(i), histogram->UpperBound(i), Min<double>()); + UNIT_ASSERT_VALUES_EQUAL(pointHistogram.GetValues(i), histogram->Value(i)); + } + } + { + const NProto::TMultiSample& s = samples.GetSamples(5); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::IGAUGE); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); + AssertLabelEqual(s.GetLabels(0), "name", "bytes"); + + UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); + AssertPointEqual(s.GetPoints(0), now, i64(1337)); + } { const NProto::TMultiSample& s = samples.GetSamples(6); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::DSUMMARY); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::DSUMMARY); UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); - AssertLabelEqual(s.GetLabels(0), "name", "temperature"); + AssertLabelEqual(s.GetLabels(0), "name", "temperature"); UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); @@ -602,7 +602,7 @@ Y_UNIT_TEST_SUITE(TSpackTest) { const NProto::TMultiSample& s = samples.GetSamples(7); UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::LOGHISTOGRAM); UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); - AssertLabelEqual(s.GetLabels(0), "name", "ms"); + AssertLabelEqual(s.GetLabels(0), "name", "ms"); UNIT_ASSERT_VALUES_EQUAL(s.PointsSize(), 1); @@ -621,69 +621,69 @@ Y_UNIT_TEST_SUITE(TSpackTest) { UNIT_ASSERT_VALUES_EQUAL(expected->Bucket(i), actual.GetBuckets(i)); } } - } - - void TestCompression(ECompression alg) { - TBuffer buffer; - { - TBufferOutput out(buffer); - auto e = EncoderSpackV1(&out, ETimePrecision::MILLIS, alg); - e->OnStreamBegin(); - { - e->OnMetricBegin(EMetricType::GAUGE); - { - e->OnLabelsBegin(); - e->OnLabel("name", "answer"); - e->OnLabelsEnd(); - } - e->OnDouble(now, 42); - e->OnMetricEnd(); - } - e->OnStreamEnd(); - e->Close(); - } - - auto* header = reinterpret_cast<const TSpackHeader*>(buffer.Data()); - UNIT_ASSERT_EQUAL(DecodeCompression(header->Compression), alg); - - NProto::TMultiSamplesList samples; - { - IMetricEncoderPtr e = EncoderProtobuf(&samples); - TBufferInput in(buffer); - DecodeSpackV1(&in, e.Get()); - } - - UNIT_ASSERT_VALUES_EQUAL( - TInstant::MilliSeconds(samples.GetCommonTime()), - TInstant::Zero()); - - UNIT_ASSERT_VALUES_EQUAL(samples.CommonLabelsSize(), 0); - - UNIT_ASSERT_VALUES_EQUAL(samples.SamplesSize(), 1); - { - const NProto::TMultiSample& s = samples.GetSamples(0); - UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); - UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); - AssertLabelEqual(s.GetLabels(0), "name", "answer"); - AssertPointEqual(s.GetPoints(0), now, 42.0); - } - } - + } + + void TestCompression(ECompression alg) { + TBuffer buffer; + { + TBufferOutput out(buffer); + auto e = EncoderSpackV1(&out, ETimePrecision::MILLIS, alg); + e->OnStreamBegin(); + { + e->OnMetricBegin(EMetricType::GAUGE); + { + e->OnLabelsBegin(); + e->OnLabel("name", "answer"); + e->OnLabelsEnd(); + } + e->OnDouble(now, 42); + e->OnMetricEnd(); + } + e->OnStreamEnd(); + e->Close(); + } + + auto* header = reinterpret_cast<const TSpackHeader*>(buffer.Data()); + UNIT_ASSERT_EQUAL(DecodeCompression(header->Compression), alg); + + NProto::TMultiSamplesList samples; + { + IMetricEncoderPtr e = EncoderProtobuf(&samples); + TBufferInput in(buffer); + DecodeSpackV1(&in, e.Get()); + } + + UNIT_ASSERT_VALUES_EQUAL( + TInstant::MilliSeconds(samples.GetCommonTime()), + TInstant::Zero()); + + UNIT_ASSERT_VALUES_EQUAL(samples.CommonLabelsSize(), 0); + + UNIT_ASSERT_VALUES_EQUAL(samples.SamplesSize(), 1); + { + const NProto::TMultiSample& s = samples.GetSamples(0); + UNIT_ASSERT_EQUAL(s.GetMetricType(), NProto::GAUGE); + UNIT_ASSERT_VALUES_EQUAL(s.LabelsSize(), 1); + AssertLabelEqual(s.GetLabels(0), "name", "answer"); + AssertPointEqual(s.GetPoints(0), now, 42.0); + } + } + Y_UNIT_TEST(CompressionIdentity) { - TestCompression(ECompression::IDENTITY); - } - + TestCompression(ECompression::IDENTITY); + } + Y_UNIT_TEST(CompressionZlib) { - TestCompression(ECompression::ZLIB); - } - + TestCompression(ECompression::ZLIB); + } + Y_UNIT_TEST(CompressionZstd) { - TestCompression(ECompression::ZSTD); - } - + TestCompression(ECompression::ZSTD); + } + Y_UNIT_TEST(CompressionLz4) { - TestCompression(ECompression::LZ4); - } + TestCompression(ECompression::LZ4); + } Y_UNIT_TEST(Decode_v1_0_histograms) { // Check that histogram bounds decoded from different versions are the same @@ -842,4 +842,4 @@ Y_UNIT_TEST_SUITE(TSpackTest) { yexception, "metric name label 's' not found, all metric labels '{m=v, project=solomon}'"); } -} +} diff --git a/library/cpp/monlib/encode/spack/ut/ya.make b/library/cpp/monlib/encode/spack/ut/ya.make index 6f27ddf43ce..980bf546671 100644 --- a/library/cpp/monlib/encode/spack/ut/ya.make +++ b/library/cpp/monlib/encode/spack/ut/ya.make @@ -1,16 +1,16 @@ -UNITTEST_FOR(library/cpp/monlib/encode/spack) - +UNITTEST_FOR(library/cpp/monlib/encode/spack) + OWNER( g:solomon jamel ) - -SRCS( - spack_v1_ut.cpp -) - -PEERDIR( - library/cpp/monlib/encode/protobuf -) - -END() + +SRCS( + spack_v1_ut.cpp +) + +PEERDIR( + library/cpp/monlib/encode/protobuf +) + +END() diff --git a/library/cpp/monlib/encode/spack/varint.cpp b/library/cpp/monlib/encode/spack/varint.cpp index dabfaa9afef..051cf17380c 100644 --- a/library/cpp/monlib/encode/spack/varint.cpp +++ b/library/cpp/monlib/encode/spack/varint.cpp @@ -1,79 +1,79 @@ -#include "varint.h" - -#include <util/generic/yexception.h> -#include <util/stream/input.h> -#include <util/stream/output.h> - -namespace NMonitoring { - ui32 WriteVarUInt32(IOutputStream* output, ui32 value) { - bool stop = false; - ui32 written = 0; - while (!stop) { - ui8 byte = static_cast<ui8>(value | 0x80); - value >>= 7; - if (value == 0) { - stop = true; - byte &= 0x7F; - } - output->Write(byte); - written++; - } - return written; - } - - ui32 ReadVarUInt32(IInputStream* input) { +#include "varint.h" + +#include <util/generic/yexception.h> +#include <util/stream/input.h> +#include <util/stream/output.h> + +namespace NMonitoring { + ui32 WriteVarUInt32(IOutputStream* output, ui32 value) { + bool stop = false; + ui32 written = 0; + while (!stop) { + ui8 byte = static_cast<ui8>(value | 0x80); + value >>= 7; + if (value == 0) { + stop = true; + byte &= 0x7F; + } + output->Write(byte); + written++; + } + return written; + } + + ui32 ReadVarUInt32(IInputStream* input) { ui32 value = 0; switch (TryReadVarUInt32(input, &value)) { case EReadResult::OK: return value; case EReadResult::ERR_OVERFLOW: - ythrow yexception() << "the data is too long to read ui32"; + ythrow yexception() << "the data is too long to read ui32"; case EReadResult::ERR_UNEXPECTED_EOF: - ythrow yexception() << "the data unexpectedly ended"; + ythrow yexception() << "the data unexpectedly ended"; default: ythrow yexception() << "unknown error while reading varint"; } - } - - size_t ReadVarUInt32(const ui8* buf, size_t len, ui32* result) { - size_t count = 0; - ui32 value = 0; - - ui8 byte = 0; - do { - if (7 * count > 8 * sizeof(ui32)) { - ythrow yexception() << "the data is too long to read ui32"; - } - if (count == len) { - ythrow yexception() << "the data unexpectedly ended"; - } - byte = buf[count]; - value |= (static_cast<ui32>(byte & 0x7F)) << (7 * count); - ++count; - } while (byte & 0x80); - - *result = value; - return count; - } - + } + + size_t ReadVarUInt32(const ui8* buf, size_t len, ui32* result) { + size_t count = 0; + ui32 value = 0; + + ui8 byte = 0; + do { + if (7 * count > 8 * sizeof(ui32)) { + ythrow yexception() << "the data is too long to read ui32"; + } + if (count == len) { + ythrow yexception() << "the data unexpectedly ended"; + } + byte = buf[count]; + value |= (static_cast<ui32>(byte & 0x7F)) << (7 * count); + ++count; + } while (byte & 0x80); + + *result = value; + return count; + } + EReadResult TryReadVarUInt32(IInputStream* input, ui32* value) { - size_t count = 0; - ui32 result = 0; - - ui8 byte = 0; - do { - if (7 * count > 8 * sizeof(ui32)) { + size_t count = 0; + ui32 result = 0; + + ui8 byte = 0; + do { + if (7 * count > 8 * sizeof(ui32)) { return EReadResult::ERR_OVERFLOW; - } - if (input->Read(&byte, 1) != 1) { + } + if (input->Read(&byte, 1) != 1) { return EReadResult::ERR_UNEXPECTED_EOF; - } - result |= (static_cast<ui32>(byte & 0x7F)) << (7 * count); - ++count; - } while (byte & 0x80); - - *value = result; + } + result |= (static_cast<ui32>(byte & 0x7F)) << (7 * count); + ++count; + } while (byte & 0x80); + + *value = result; return EReadResult::OK; - } - -} + } + +} diff --git a/library/cpp/monlib/encode/spack/varint.h b/library/cpp/monlib/encode/spack/varint.h index 7bce244b200..7ac522dd6c7 100644 --- a/library/cpp/monlib/encode/spack/varint.h +++ b/library/cpp/monlib/encode/spack/varint.h @@ -1,16 +1,16 @@ -#pragma once - -#include <util/system/types.h> - -class IInputStream; -class IOutputStream; - -namespace NMonitoring { - ui32 WriteVarUInt32(IOutputStream* output, ui32 value); - - ui32 ReadVarUInt32(IInputStream* input); - size_t ReadVarUInt32(const ui8* buf, size_t len, ui32* result); - +#pragma once + +#include <util/system/types.h> + +class IInputStream; +class IOutputStream; + +namespace NMonitoring { + ui32 WriteVarUInt32(IOutputStream* output, ui32 value); + + ui32 ReadVarUInt32(IInputStream* input); + size_t ReadVarUInt32(const ui8* buf, size_t len, ui32* result); + enum class EReadResult { OK, ERR_OVERFLOW, @@ -20,4 +20,4 @@ namespace NMonitoring { [[nodiscard]] EReadResult TryReadVarUInt32(IInputStream* input, ui32* value); -} +} diff --git a/library/cpp/monlib/encode/spack/ya.make b/library/cpp/monlib/encode/spack/ya.make index 1b1a27e7e29..78d3061291a 100644 --- a/library/cpp/monlib/encode/spack/ya.make +++ b/library/cpp/monlib/encode/spack/ya.make @@ -1,25 +1,25 @@ -LIBRARY() - +LIBRARY() + OWNER( g:solomon jamel ) - -SRCS( - spack_v1_decoder.cpp - spack_v1_encoder.cpp - varint.cpp - compression.cpp -) - -PEERDIR( + +SRCS( + spack_v1_decoder.cpp + spack_v1_encoder.cpp + varint.cpp + compression.cpp +) + +PEERDIR( library/cpp/monlib/encode/buffered library/cpp/monlib/exception - contrib/libs/lz4 + contrib/libs/lz4 contrib/libs/xxhash - contrib/libs/zlib - contrib/libs/zstd -) - -END() + contrib/libs/zlib + contrib/libs/zstd +) + +END() diff --git a/library/cpp/monlib/encode/text/text.h b/library/cpp/monlib/encode/text/text.h index 8efc2b18a2a..6b2be3937bd 100644 --- a/library/cpp/monlib/encode/text/text.h +++ b/library/cpp/monlib/encode/text/text.h @@ -1,9 +1,9 @@ -#pragma once - -#include <library/cpp/monlib/encode/encoder.h> - -class IOutputStream; - -namespace NMonitoring { - IMetricEncoderPtr EncoderText(IOutputStream* out, bool humanReadableTs = true); -} +#pragma once + +#include <library/cpp/monlib/encode/encoder.h> + +class IOutputStream; + +namespace NMonitoring { + IMetricEncoderPtr EncoderText(IOutputStream* out, bool humanReadableTs = true); +} diff --git a/library/cpp/monlib/encode/text/text_encoder.cpp b/library/cpp/monlib/encode/text/text_encoder.cpp index cfddcb2405b..10336261f09 100644 --- a/library/cpp/monlib/encode/text/text_encoder.cpp +++ b/library/cpp/monlib/encode/text/text_encoder.cpp @@ -1,148 +1,148 @@ -#include "text.h" - -#include <library/cpp/monlib/encode/encoder_state.h> -#include <library/cpp/monlib/metrics/labels.h> -#include <library/cpp/monlib/metrics/metric_value.h> - -#include <util/datetime/base.h> -#include <util/stream/format.h> - -namespace NMonitoring { - namespace { - class TEncoderText final: public IMetricEncoder { - public: - TEncoderText(IOutputStream* out, bool humanReadableTs) - : Out_(out) - , HumanReadableTs_(humanReadableTs) - { - } - - private: - void OnStreamBegin() override { +#include "text.h" + +#include <library/cpp/monlib/encode/encoder_state.h> +#include <library/cpp/monlib/metrics/labels.h> +#include <library/cpp/monlib/metrics/metric_value.h> + +#include <util/datetime/base.h> +#include <util/stream/format.h> + +namespace NMonitoring { + namespace { + class TEncoderText final: public IMetricEncoder { + public: + TEncoderText(IOutputStream* out, bool humanReadableTs) + : Out_(out) + , HumanReadableTs_(humanReadableTs) + { + } + + private: + void OnStreamBegin() override { State_.Expect(TEncoderState::EState::ROOT); - } - - void OnStreamEnd() override { + } + + void OnStreamEnd() override { State_.Expect(TEncoderState::EState::ROOT); - } - - void OnCommonTime(TInstant time) override { + } + + void OnCommonTime(TInstant time) override { State_.Expect(TEncoderState::EState::ROOT); - CommonTime_ = time; - if (time != TInstant::Zero()) { + CommonTime_ = time; + if (time != TInstant::Zero()) { Out_->Write(TStringBuf("common time: ")); - WriteTime(time); - Out_->Write('\n'); - } - } - - void OnMetricBegin(EMetricType type) override { - State_.Switch(TEncoderState::EState::ROOT, TEncoderState::EState::METRIC); - ClearLastMetricState(); - MetricType_ = type; - } - - void OnMetricEnd() override { - State_.Switch(TEncoderState::EState::METRIC, TEncoderState::EState::ROOT); - WriteMetric(); - } - - void OnLabelsBegin() override { - if (State_ == TEncoderState::EState::METRIC) { - State_ = TEncoderState::EState::METRIC_LABELS; + WriteTime(time); + Out_->Write('\n'); + } + } + + void OnMetricBegin(EMetricType type) override { + State_.Switch(TEncoderState::EState::ROOT, TEncoderState::EState::METRIC); + ClearLastMetricState(); + MetricType_ = type; + } + + void OnMetricEnd() override { + State_.Switch(TEncoderState::EState::METRIC, TEncoderState::EState::ROOT); + WriteMetric(); + } + + void OnLabelsBegin() override { + if (State_ == TEncoderState::EState::METRIC) { + State_ = TEncoderState::EState::METRIC_LABELS; } else if (State_ == TEncoderState::EState::ROOT) { State_ = TEncoderState::EState::COMMON_LABELS; - } else { - State_.ThrowInvalid("expected METRIC or ROOT"); - } - } - - void OnLabelsEnd() override { - if (State_ == TEncoderState::EState::METRIC_LABELS) { - State_ = TEncoderState::EState::METRIC; + } else { + State_.ThrowInvalid("expected METRIC or ROOT"); + } + } + + void OnLabelsEnd() override { + if (State_ == TEncoderState::EState::METRIC_LABELS) { + State_ = TEncoderState::EState::METRIC; } else if (State_ == TEncoderState::EState::COMMON_LABELS) { State_ = TEncoderState::EState::ROOT; Out_->Write(TStringBuf("common labels: ")); - WriteLabels(); - Out_->Write('\n'); - } else { - State_.ThrowInvalid("expected LABELS or COMMON_LABELS"); - } - } - - void OnLabel(TStringBuf name, TStringBuf value) override { - Labels_.Add(name, value); - } - - void OnDouble(TInstant time, double value) override { - State_.Expect(TEncoderState::EState::METRIC); - TimeSeries_.Add(time, value); - } - - void OnInt64(TInstant time, i64 value) override { - State_.Expect(TEncoderState::EState::METRIC); - TimeSeries_.Add(time, value); - } - - void OnUint64(TInstant time, ui64 value) override { - State_.Expect(TEncoderState::EState::METRIC); - TimeSeries_.Add(time, value); - } - - void OnHistogram(TInstant time, IHistogramSnapshotPtr snapshot) override { - State_.Expect(TEncoderState::EState::METRIC); - TimeSeries_.Add(time, snapshot.Get()); - } - + WriteLabels(); + Out_->Write('\n'); + } else { + State_.ThrowInvalid("expected LABELS or COMMON_LABELS"); + } + } + + void OnLabel(TStringBuf name, TStringBuf value) override { + Labels_.Add(name, value); + } + + void OnDouble(TInstant time, double value) override { + State_.Expect(TEncoderState::EState::METRIC); + TimeSeries_.Add(time, value); + } + + void OnInt64(TInstant time, i64 value) override { + State_.Expect(TEncoderState::EState::METRIC); + TimeSeries_.Add(time, value); + } + + void OnUint64(TInstant time, ui64 value) override { + State_.Expect(TEncoderState::EState::METRIC); + TimeSeries_.Add(time, value); + } + + void OnHistogram(TInstant time, IHistogramSnapshotPtr snapshot) override { + State_.Expect(TEncoderState::EState::METRIC); + TimeSeries_.Add(time, snapshot.Get()); + } + void OnSummaryDouble(TInstant time, ISummaryDoubleSnapshotPtr snapshot) override { - State_.Expect(TEncoderState::EState::METRIC); + State_.Expect(TEncoderState::EState::METRIC); TimeSeries_.Add(time, snapshot.Get()); } void OnLogHistogram(TInstant ts, TLogHistogramSnapshotPtr snapshot) override { - State_.Expect(TEncoderState::EState::METRIC); + State_.Expect(TEncoderState::EState::METRIC); TimeSeries_.Add(ts, snapshot.Get()); } - void Close() override { - } - - void WriteTime(TInstant time) { - if (HumanReadableTs_) { - char buf[64]; - auto len = FormatDate8601(buf, sizeof(buf), time.TimeT()); - Out_->Write(buf, len); - } else { - (*Out_) << time.Seconds(); - } - } - - void WriteValue(EMetricValueType type, TMetricValue value) { - switch (type) { - case EMetricValueType::DOUBLE: - (*Out_) << value.AsDouble(); - break; - case EMetricValueType::INT64: - (*Out_) << value.AsInt64(); - break; - case EMetricValueType::UINT64: - (*Out_) << value.AsUint64(); - break; - case EMetricValueType::HISTOGRAM: - (*Out_) << *value.AsHistogram(); - break; - case EMetricValueType::SUMMARY: + void Close() override { + } + + void WriteTime(TInstant time) { + if (HumanReadableTs_) { + char buf[64]; + auto len = FormatDate8601(buf, sizeof(buf), time.TimeT()); + Out_->Write(buf, len); + } else { + (*Out_) << time.Seconds(); + } + } + + void WriteValue(EMetricValueType type, TMetricValue value) { + switch (type) { + case EMetricValueType::DOUBLE: + (*Out_) << value.AsDouble(); + break; + case EMetricValueType::INT64: + (*Out_) << value.AsInt64(); + break; + case EMetricValueType::UINT64: + (*Out_) << value.AsUint64(); + break; + case EMetricValueType::HISTOGRAM: + (*Out_) << *value.AsHistogram(); + break; + case EMetricValueType::SUMMARY: (*Out_) << *value.AsSummaryDouble(); break; case EMetricValueType::LOGHISTOGRAM: (*Out_) << *value.AsLogHistogram(); break; - case EMetricValueType::UNKNOWN: - ythrow yexception() << "unknown metric value type"; - } - } - - void WriteLabels() { + case EMetricValueType::UNKNOWN: + ythrow yexception() << "unknown metric value type"; + } + } + + void WriteLabels() { auto& out = *Out_; const auto size = Labels_.Size(); size_t i = 0; @@ -154,73 +154,73 @@ namespace NMonitoring { ++i; if (i < size) { out << TStringBuf(", "); - } + } }; out << '}'; - } - - void WriteMetric() { - // (1) type - TStringBuf typeStr = MetricTypeToStr(MetricType_); - (*Out_) << LeftPad(typeStr, MaxMetricTypeNameLength) << ' '; - - // (2) name and labels + } + + void WriteMetric() { + // (1) type + TStringBuf typeStr = MetricTypeToStr(MetricType_); + (*Out_) << LeftPad(typeStr, MaxMetricTypeNameLength) << ' '; + + // (2) name and labels auto name = Labels_.Extract(TStringBuf("sensor")); - if (name) { - if (name->Value().find(' ') != TString::npos) { - (*Out_) << '"' << name->Value() << '"'; - } else { - (*Out_) << name->Value(); - } - } - WriteLabels(); - - // (3) values - if (!TimeSeries_.Empty()) { - TimeSeries_.SortByTs(); + if (name) { + if (name->Value().find(' ') != TString::npos) { + (*Out_) << '"' << name->Value() << '"'; + } else { + (*Out_) << name->Value(); + } + } + WriteLabels(); + + // (3) values + if (!TimeSeries_.Empty()) { + TimeSeries_.SortByTs(); Out_->Write(TStringBuf(" [")); - for (size_t i = 0; i < TimeSeries_.Size(); i++) { - if (i > 0) { + for (size_t i = 0; i < TimeSeries_.Size(); i++) { + if (i > 0) { Out_->Write(TStringBuf(", ")); - } - - const auto& point = TimeSeries_[i]; - if (point.GetTime() == CommonTime_ || point.GetTime() == TInstant::Zero()) { - WriteValue(TimeSeries_.GetValueType(), point.GetValue()); - } else { - Out_->Write('('); - WriteTime(point.GetTime()); + } + + const auto& point = TimeSeries_[i]; + if (point.GetTime() == CommonTime_ || point.GetTime() == TInstant::Zero()) { + WriteValue(TimeSeries_.GetValueType(), point.GetValue()); + } else { + Out_->Write('('); + WriteTime(point.GetTime()); Out_->Write(TStringBuf(", ")); - WriteValue(TimeSeries_.GetValueType(), point.GetValue()); - Out_->Write(')'); - } - } - Out_->Write(']'); - } - Out_->Write('\n'); - } - - void ClearLastMetricState() { - MetricType_ = EMetricType::UNKNOWN; - Labels_.Clear(); - TimeSeries_.Clear(); - } - - private: - TEncoderState State_; - IOutputStream* Out_; - bool HumanReadableTs_; - TInstant CommonTime_ = TInstant::Zero(); - EMetricType MetricType_ = EMetricType::UNKNOWN; - TLabels Labels_; - TMetricTimeSeries TimeSeries_; - }; - - } - - IMetricEncoderPtr EncoderText(IOutputStream* out, bool humanReadableTs) { + WriteValue(TimeSeries_.GetValueType(), point.GetValue()); + Out_->Write(')'); + } + } + Out_->Write(']'); + } + Out_->Write('\n'); + } + + void ClearLastMetricState() { + MetricType_ = EMetricType::UNKNOWN; + Labels_.Clear(); + TimeSeries_.Clear(); + } + + private: + TEncoderState State_; + IOutputStream* Out_; + bool HumanReadableTs_; + TInstant CommonTime_ = TInstant::Zero(); + EMetricType MetricType_ = EMetricType::UNKNOWN; + TLabels Labels_; + TMetricTimeSeries TimeSeries_; + }; + + } + + IMetricEncoderPtr EncoderText(IOutputStream* out, bool humanReadableTs) { return MakeHolder<TEncoderText>(out, humanReadableTs); - } - -} + } + +} diff --git a/library/cpp/monlib/encode/text/text_encoder_ut.cpp b/library/cpp/monlib/encode/text/text_encoder_ut.cpp index b817b38285a..554b6f5fa9e 100644 --- a/library/cpp/monlib/encode/text/text_encoder_ut.cpp +++ b/library/cpp/monlib/encode/text/text_encoder_ut.cpp @@ -1,283 +1,283 @@ -#include "text.h" - -#include <library/cpp/monlib/metrics/histogram_collector.h> - +#include "text.h" + +#include <library/cpp/monlib/metrics/histogram_collector.h> + #include <library/cpp/testing/unittest/registar.h> - -using namespace NMonitoring; - + +using namespace NMonitoring; + Y_UNIT_TEST_SUITE(TTextText) { - template <typename TFunc> - TString EncodeToString(bool humanReadableTs, TFunc fn) { - TStringStream ss; - IMetricEncoderPtr encoder = EncoderText(&ss, humanReadableTs); - fn(encoder.Get()); - return ss.Str(); - } - + template <typename TFunc> + TString EncodeToString(bool humanReadableTs, TFunc fn) { + TStringStream ss; + IMetricEncoderPtr encoder = EncoderText(&ss, humanReadableTs); + fn(encoder.Get()); + return ss.Str(); + } + Y_UNIT_TEST(Empty) { - auto result = EncodeToString(true, [](IMetricEncoder* e) { - e->OnStreamBegin(); - e->OnStreamEnd(); - }); - UNIT_ASSERT_STRINGS_EQUAL(result, ""); - } - + auto result = EncodeToString(true, [](IMetricEncoder* e) { + e->OnStreamBegin(); + e->OnStreamEnd(); + }); + UNIT_ASSERT_STRINGS_EQUAL(result, ""); + } + Y_UNIT_TEST(CommonPart) { - auto result = EncodeToString(true, [](IMetricEncoder* e) { - e->OnStreamBegin(); + auto result = EncodeToString(true, [](IMetricEncoder* e) { + e->OnStreamBegin(); e->OnCommonTime(TInstant::ParseIso8601Deprecated("2017-01-02T03:04:05.006Z")); - { - e->OnLabelsBegin(); - e->OnLabel("project", "solomon"); - e->OnLabel("cluster", "man"); - e->OnLabel("service", "stockpile"); - e->OnLabelsEnd(); - } - e->OnStreamEnd(); - }); - UNIT_ASSERT_STRINGS_EQUAL(result, - "common time: 2017-01-02T03:04:05Z\n" - "common labels: {project='solomon', cluster='man', service='stockpile'}\n"); - } - + { + e->OnLabelsBegin(); + e->OnLabel("project", "solomon"); + e->OnLabel("cluster", "man"); + e->OnLabel("service", "stockpile"); + e->OnLabelsEnd(); + } + e->OnStreamEnd(); + }); + UNIT_ASSERT_STRINGS_EQUAL(result, + "common time: 2017-01-02T03:04:05Z\n" + "common labels: {project='solomon', cluster='man', service='stockpile'}\n"); + } + Y_UNIT_TEST(Gauges) { - auto result = EncodeToString(true, [](IMetricEncoder* e) { - e->OnStreamBegin(); - { // no values - e->OnMetricBegin(EMetricType::GAUGE); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "cpuUsage"); - e->OnLabelsEnd(); - } - e->OnMetricEnd(); - } - { // one value no ts - e->OnMetricBegin(EMetricType::GAUGE); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "diskUsage"); - e->OnLabel("disk", "sda1"); - e->OnLabelsEnd(); - } - e->OnDouble(TInstant::Zero(), 1000); - e->OnMetricEnd(); - } - { // one value with ts - e->OnMetricBegin(EMetricType::GAUGE); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "memoryUsage"); - e->OnLabel("host", "solomon-man-00"); - e->OnLabel("dc", "man"); - e->OnLabelsEnd(); - } + auto result = EncodeToString(true, [](IMetricEncoder* e) { + e->OnStreamBegin(); + { // no values + e->OnMetricBegin(EMetricType::GAUGE); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "cpuUsage"); + e->OnLabelsEnd(); + } + e->OnMetricEnd(); + } + { // one value no ts + e->OnMetricBegin(EMetricType::GAUGE); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "diskUsage"); + e->OnLabel("disk", "sda1"); + e->OnLabelsEnd(); + } + e->OnDouble(TInstant::Zero(), 1000); + e->OnMetricEnd(); + } + { // one value with ts + e->OnMetricBegin(EMetricType::GAUGE); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "memoryUsage"); + e->OnLabel("host", "solomon-man-00"); + e->OnLabel("dc", "man"); + e->OnLabelsEnd(); + } + e->OnDouble(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:00Z"), 1000); + e->OnMetricEnd(); + } + { // many values + e->OnMetricBegin(EMetricType::GAUGE); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "bytesRx"); + e->OnLabel("host", "solomon-sas-01"); + e->OnLabel("dc", "sas"); + e->OnLabelsEnd(); + } + e->OnDouble(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:00Z"), 2); + e->OnDouble(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:05Z"), 4); + e->OnDouble(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:10Z"), 8); + e->OnMetricEnd(); + } + e->OnStreamEnd(); + }); + UNIT_ASSERT_STRINGS_EQUAL(result, + " GAUGE cpuUsage{}\n" + " GAUGE diskUsage{disk='sda1'} [1000]\n" + " GAUGE memoryUsage{host='solomon-man-00', dc='man'} [(2017-12-02T12:00:00Z, 1000)]\n" + " GAUGE bytesRx{host='solomon-sas-01', dc='sas'} [(2017-12-02T12:00:00Z, 2), (2017-12-02T12:00:05Z, 4), (2017-12-02T12:00:10Z, 8)]\n"); + } + + Y_UNIT_TEST(IntGauges) { + auto result = EncodeToString(true, [](IMetricEncoder* e) { + e->OnStreamBegin(); + { // no values + e->OnMetricBegin(EMetricType::IGAUGE); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "cpuUsage"); + e->OnLabelsEnd(); + } + e->OnMetricEnd(); + } + { // one value no ts + e->OnMetricBegin(EMetricType::IGAUGE); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "diskUsage"); + e->OnLabel("disk", "sda1"); + e->OnLabelsEnd(); + } + e->OnDouble(TInstant::Zero(), 1000); + e->OnMetricEnd(); + } + { // one value with ts + e->OnMetricBegin(EMetricType::IGAUGE); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "memoryUsage"); + e->OnLabel("host", "solomon-man-00"); + e->OnLabel("dc", "man"); + e->OnLabelsEnd(); + } e->OnDouble(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:00Z"), 1000); - e->OnMetricEnd(); - } - { // many values - e->OnMetricBegin(EMetricType::GAUGE); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "bytesRx"); - e->OnLabel("host", "solomon-sas-01"); - e->OnLabel("dc", "sas"); - e->OnLabelsEnd(); - } + e->OnMetricEnd(); + } + { // many values + e->OnMetricBegin(EMetricType::IGAUGE); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "bytesRx"); + e->OnLabel("host", "solomon-sas-01"); + e->OnLabel("dc", "sas"); + e->OnLabelsEnd(); + } e->OnDouble(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:00Z"), 2); e->OnDouble(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:05Z"), 4); e->OnDouble(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:10Z"), 8); - e->OnMetricEnd(); - } - e->OnStreamEnd(); - }); - UNIT_ASSERT_STRINGS_EQUAL(result, - " GAUGE cpuUsage{}\n" - " GAUGE diskUsage{disk='sda1'} [1000]\n" - " GAUGE memoryUsage{host='solomon-man-00', dc='man'} [(2017-12-02T12:00:00Z, 1000)]\n" - " GAUGE bytesRx{host='solomon-sas-01', dc='sas'} [(2017-12-02T12:00:00Z, 2), (2017-12-02T12:00:05Z, 4), (2017-12-02T12:00:10Z, 8)]\n"); - } - - Y_UNIT_TEST(IntGauges) { - auto result = EncodeToString(true, [](IMetricEncoder* e) { - e->OnStreamBegin(); - { // no values - e->OnMetricBegin(EMetricType::IGAUGE); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "cpuUsage"); - e->OnLabelsEnd(); - } - e->OnMetricEnd(); - } - { // one value no ts - e->OnMetricBegin(EMetricType::IGAUGE); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "diskUsage"); - e->OnLabel("disk", "sda1"); - e->OnLabelsEnd(); - } - e->OnDouble(TInstant::Zero(), 1000); - e->OnMetricEnd(); - } - { // one value with ts - e->OnMetricBegin(EMetricType::IGAUGE); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "memoryUsage"); - e->OnLabel("host", "solomon-man-00"); - e->OnLabel("dc", "man"); - e->OnLabelsEnd(); - } - e->OnDouble(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:00Z"), 1000); - e->OnMetricEnd(); - } - { // many values - e->OnMetricBegin(EMetricType::IGAUGE); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "bytesRx"); - e->OnLabel("host", "solomon-sas-01"); - e->OnLabel("dc", "sas"); - e->OnLabelsEnd(); - } - e->OnDouble(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:00Z"), 2); - e->OnDouble(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:05Z"), 4); - e->OnDouble(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:10Z"), 8); - e->OnMetricEnd(); - } - e->OnStreamEnd(); - }); - UNIT_ASSERT_STRINGS_EQUAL(result, - " IGAUGE cpuUsage{}\n" - " IGAUGE diskUsage{disk='sda1'} [1000]\n" - " IGAUGE memoryUsage{host='solomon-man-00', dc='man'} [(2017-12-02T12:00:00Z, 1000)]\n" - " IGAUGE bytesRx{host='solomon-sas-01', dc='sas'} [(2017-12-02T12:00:00Z, 2), (2017-12-02T12:00:05Z, 4), (2017-12-02T12:00:10Z, 8)]\n"); - } - + e->OnMetricEnd(); + } + e->OnStreamEnd(); + }); + UNIT_ASSERT_STRINGS_EQUAL(result, + " IGAUGE cpuUsage{}\n" + " IGAUGE diskUsage{disk='sda1'} [1000]\n" + " IGAUGE memoryUsage{host='solomon-man-00', dc='man'} [(2017-12-02T12:00:00Z, 1000)]\n" + " IGAUGE bytesRx{host='solomon-sas-01', dc='sas'} [(2017-12-02T12:00:00Z, 2), (2017-12-02T12:00:05Z, 4), (2017-12-02T12:00:10Z, 8)]\n"); + } + Y_UNIT_TEST(Counters) { - auto doEncode = [](IMetricEncoder* e) { - e->OnStreamBegin(); - { // no values - e->OnMetricBegin(EMetricType::COUNTER); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "cpuUsage"); - e->OnLabelsEnd(); - } - e->OnMetricEnd(); - } - { // one value no ts - e->OnMetricBegin(EMetricType::COUNTER); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "diskUsage"); - e->OnLabel("disk", "sda1"); - e->OnLabelsEnd(); - } - e->OnUint64(TInstant::Zero(), 1000); - e->OnMetricEnd(); - } - { // one value with ts - e->OnMetricBegin(EMetricType::COUNTER); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "memoryUsage"); - e->OnLabel("host", "solomon-man-00"); - e->OnLabel("dc", "man"); - e->OnLabelsEnd(); - } + auto doEncode = [](IMetricEncoder* e) { + e->OnStreamBegin(); + { // no values + e->OnMetricBegin(EMetricType::COUNTER); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "cpuUsage"); + e->OnLabelsEnd(); + } + e->OnMetricEnd(); + } + { // one value no ts + e->OnMetricBegin(EMetricType::COUNTER); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "diskUsage"); + e->OnLabel("disk", "sda1"); + e->OnLabelsEnd(); + } + e->OnUint64(TInstant::Zero(), 1000); + e->OnMetricEnd(); + } + { // one value with ts + e->OnMetricBegin(EMetricType::COUNTER); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "memoryUsage"); + e->OnLabel("host", "solomon-man-00"); + e->OnLabel("dc", "man"); + e->OnLabelsEnd(); + } e->OnUint64(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:00Z"), 1000); - e->OnMetricEnd(); - } - { // many values - e->OnMetricBegin(EMetricType::COUNTER); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "bytesRx"); - e->OnLabel("host", "solomon-sas-01"); - e->OnLabel("dc", "sas"); - e->OnLabelsEnd(); - } + e->OnMetricEnd(); + } + { // many values + e->OnMetricBegin(EMetricType::COUNTER); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "bytesRx"); + e->OnLabel("host", "solomon-sas-01"); + e->OnLabel("dc", "sas"); + e->OnLabelsEnd(); + } e->OnUint64(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:00Z"), 2); e->OnUint64(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:05Z"), 4); e->OnUint64(TInstant::ParseIso8601Deprecated("2017-12-02T12:00:10Z"), 8); - e->OnMetricEnd(); - } - e->OnStreamEnd(); - }; - - auto result1 = EncodeToString(false, doEncode); - UNIT_ASSERT_STRINGS_EQUAL(result1, - " COUNTER cpuUsage{}\n" - " COUNTER diskUsage{disk='sda1'} [1000]\n" - " COUNTER memoryUsage{host='solomon-man-00', dc='man'} [(1512216000, 1000)]\n" - " COUNTER bytesRx{host='solomon-sas-01', dc='sas'} [(1512216000, 2), (1512216005, 4), (1512216010, 8)]\n"); - - auto result2 = EncodeToString(true, doEncode); - UNIT_ASSERT_STRINGS_EQUAL(result2, - " COUNTER cpuUsage{}\n" - " COUNTER diskUsage{disk='sda1'} [1000]\n" - " COUNTER memoryUsage{host='solomon-man-00', dc='man'} [(2017-12-02T12:00:00Z, 1000)]\n" - " COUNTER bytesRx{host='solomon-sas-01', dc='sas'} [(2017-12-02T12:00:00Z, 2), (2017-12-02T12:00:05Z, 4), (2017-12-02T12:00:10Z, 8)]\n"); - } - - Y_UNIT_TEST(Histograms) { - auto h = ExplicitHistogram({1, 2, 3, 4, 5}); - h->Collect(3); - h->Collect(5, 7); - h->Collect(13); - auto s = h->Snapshot(); - - TString result = EncodeToString(true, [s](IMetricEncoder* e) { - e->OnStreamBegin(); - { - e->OnMetricBegin(EMetricType::HIST); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "readTimeMillis"); - e->OnLabelsEnd(); - } - e->OnHistogram(TInstant::Zero(), s); - e->OnMetricEnd(); - } - { - e->OnMetricBegin(EMetricType::HIST_RATE); - { - e->OnLabelsBegin(); - e->OnLabel("sensor", "writeTimeMillis"); - e->OnLabelsEnd(); - } - e->OnHistogram(TInstant::Zero(), s); - e->OnMetricEnd(); - } - e->OnStreamEnd(); - }); - - UNIT_ASSERT_STRINGS_EQUAL(result, - " HIST readTimeMillis{} [{1: 0, 2: 0, 3: 1, 4: 0, 5: 7, inf: 1}]\n" - "HIST_RATE writeTimeMillis{} [{1: 0, 2: 0, 3: 1, 4: 0, 5: 7, inf: 1}]\n"); - } + e->OnMetricEnd(); + } + e->OnStreamEnd(); + }; + + auto result1 = EncodeToString(false, doEncode); + UNIT_ASSERT_STRINGS_EQUAL(result1, + " COUNTER cpuUsage{}\n" + " COUNTER diskUsage{disk='sda1'} [1000]\n" + " COUNTER memoryUsage{host='solomon-man-00', dc='man'} [(1512216000, 1000)]\n" + " COUNTER bytesRx{host='solomon-sas-01', dc='sas'} [(1512216000, 2), (1512216005, 4), (1512216010, 8)]\n"); + + auto result2 = EncodeToString(true, doEncode); + UNIT_ASSERT_STRINGS_EQUAL(result2, + " COUNTER cpuUsage{}\n" + " COUNTER diskUsage{disk='sda1'} [1000]\n" + " COUNTER memoryUsage{host='solomon-man-00', dc='man'} [(2017-12-02T12:00:00Z, 1000)]\n" + " COUNTER bytesRx{host='solomon-sas-01', dc='sas'} [(2017-12-02T12:00:00Z, 2), (2017-12-02T12:00:05Z, 4), (2017-12-02T12:00:10Z, 8)]\n"); + } + + Y_UNIT_TEST(Histograms) { + auto h = ExplicitHistogram({1, 2, 3, 4, 5}); + h->Collect(3); + h->Collect(5, 7); + h->Collect(13); + auto s = h->Snapshot(); + + TString result = EncodeToString(true, [s](IMetricEncoder* e) { + e->OnStreamBegin(); + { + e->OnMetricBegin(EMetricType::HIST); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "readTimeMillis"); + e->OnLabelsEnd(); + } + e->OnHistogram(TInstant::Zero(), s); + e->OnMetricEnd(); + } + { + e->OnMetricBegin(EMetricType::HIST_RATE); + { + e->OnLabelsBegin(); + e->OnLabel("sensor", "writeTimeMillis"); + e->OnLabelsEnd(); + } + e->OnHistogram(TInstant::Zero(), s); + e->OnMetricEnd(); + } + e->OnStreamEnd(); + }); + + UNIT_ASSERT_STRINGS_EQUAL(result, + " HIST readTimeMillis{} [{1: 0, 2: 0, 3: 1, 4: 0, 5: 7, inf: 1}]\n" + "HIST_RATE writeTimeMillis{} [{1: 0, 2: 0, 3: 1, 4: 0, 5: 7, inf: 1}]\n"); + } Y_UNIT_TEST(Summary) { auto s = MakeIntrusive<TSummaryDoubleSnapshot>(10.1, -0.45, 0.478, 0.3, 30u); - TString result = EncodeToString(true, [s](IMetricEncoder* e) { + TString result = EncodeToString(true, [s](IMetricEncoder* e) { e->OnStreamBegin(); { - e->OnMetricBegin(EMetricType::DSUMMARY); + e->OnMetricBegin(EMetricType::DSUMMARY); { e->OnLabelsBegin(); e->OnLabel("sensor", "temperature"); e->OnLabelsEnd(); } e->OnSummaryDouble(TInstant::Zero(), s); - e->OnMetricEnd(); + e->OnMetricEnd(); } e->OnStreamEnd(); }); UNIT_ASSERT_STRINGS_EQUAL(result, " DSUMMARY temperature{} [{sum: 10.1, min: -0.45, max: 0.478, last: 0.3, count: 30}]\n"); } -} +} diff --git a/library/cpp/monlib/encode/text/ut/ya.make b/library/cpp/monlib/encode/text/ut/ya.make index 3d507800e09..df23a252d12 100644 --- a/library/cpp/monlib/encode/text/ut/ya.make +++ b/library/cpp/monlib/encode/text/ut/ya.make @@ -1,12 +1,12 @@ -UNITTEST_FOR(library/cpp/monlib/encode/text) - +UNITTEST_FOR(library/cpp/monlib/encode/text) + OWNER( g:solomon jamel ) - -SRCS( - text_encoder_ut.cpp -) - -END() + +SRCS( + text_encoder_ut.cpp +) + +END() diff --git a/library/cpp/monlib/encode/text/ya.make b/library/cpp/monlib/encode/text/ya.make index 85ae8043ecc..d296c78c1b4 100644 --- a/library/cpp/monlib/encode/text/ya.make +++ b/library/cpp/monlib/encode/text/ya.make @@ -1,16 +1,16 @@ -LIBRARY() - +LIBRARY() + OWNER( g:solomon jamel ) - -SRCS( - text_encoder.cpp -) - -PEERDIR( - library/cpp/monlib/encode -) - -END() + +SRCS( + text_encoder.cpp +) + +PEERDIR( + library/cpp/monlib/encode +) + +END() diff --git a/library/cpp/monlib/encode/unistat/unistat.h b/library/cpp/monlib/encode/unistat/unistat.h index c550dc3b6a0..1c43b7fa1b7 100644 --- a/library/cpp/monlib/encode/unistat/unistat.h +++ b/library/cpp/monlib/encode/unistat/unistat.h @@ -6,8 +6,8 @@ namespace NMonitoring { /// Decodes unistat-style metrics /// https://wiki.yandex-team.ru/golovan/stat-handle - void DecodeUnistat(TStringBuf data, class IMetricConsumer* c, TInstant ts = TInstant::Zero()); + void DecodeUnistat(TStringBuf data, class IMetricConsumer* c, TInstant ts = TInstant::Zero()); /// Assumes consumer's stream is open by the caller - void DecodeUnistatToStream(TStringBuf data, class IMetricConsumer* c, TInstant = TInstant::Zero()); + void DecodeUnistatToStream(TStringBuf data, class IMetricConsumer* c, TInstant = TInstant::Zero()); } diff --git a/library/cpp/monlib/encode/unistat/unistat_decoder.cpp b/library/cpp/monlib/encode/unistat/unistat_decoder.cpp index e97ca884da0..b2344b0905c 100644 --- a/library/cpp/monlib/encode/unistat/unistat_decoder.cpp +++ b/library/cpp/monlib/encode/unistat/unistat_decoder.cpp @@ -1,10 +1,10 @@ #include "unistat.h" -#include <library/cpp/monlib/metrics/histogram_collector.h> -#include <library/cpp/monlib/metrics/labels.h> -#include <library/cpp/monlib/metrics/metric_type.h> -#include <library/cpp/monlib/metrics/metric_value.h> -#include <library/cpp/monlib/metrics/metric_consumer.h> +#include <library/cpp/monlib/metrics/histogram_collector.h> +#include <library/cpp/monlib/metrics/labels.h> +#include <library/cpp/monlib/metrics/metric_type.h> +#include <library/cpp/monlib/metrics/metric_value.h> +#include <library/cpp/monlib/metrics/metric_consumer.h> #include <library/cpp/json/json_reader.h> @@ -86,7 +86,7 @@ namespace NMonitoring { class TDecoderUnistat { private: public: - explicit TDecoderUnistat(IMetricConsumer* consumer, IInputStream* is, TInstant ts) + explicit TDecoderUnistat(IMetricConsumer* consumer, IInputStream* is, TInstant ts) : Consumer_{consumer} , Timestamp_{ts} { ReadJsonTree(is, &Json_, /* throw */ true); @@ -120,19 +120,19 @@ namespace NMonitoring { private: void OnScalar(const TJsonValue& jsonValue) { if (MetricContext_.IsDeriv) { - MetricContext_.Type = EMetricType::RATE; - MetricContext_.Value = TMetricValue{ExtractUi64(jsonValue)}; + MetricContext_.Type = EMetricType::RATE; + MetricContext_.Value = TMetricValue{ExtractUi64(jsonValue)}; } else { - MetricContext_.Type = EMetricType::GAUGE; - MetricContext_.Value = TMetricValue{ExtractDouble(jsonValue)}; + MetricContext_.Type = EMetricType::GAUGE; + MetricContext_.Value = TMetricValue{ExtractDouble(jsonValue)}; } } void OnHistogram(const TJsonValue& jsonHist) { if (MetricContext_.IsDeriv) { - MetricContext_.Type = EMetricType::HIST_RATE; + MetricContext_.Type = EMetricType::HIST_RATE; } else { - MetricContext_.Type = EMetricType::HIST; + MetricContext_.Type = EMetricType::HIST; } auto histogramBuilder = THistogramBuilder(); @@ -147,7 +147,7 @@ namespace NMonitoring { } MetricContext_.Histogram = histogramBuilder.Finalize(); - MetricContext_.Value = TMetricValue{MetricContext_.Histogram.Get()}; + MetricContext_.Value = TMetricValue{MetricContext_.Histogram.Get()}; } bool IsDeriv(TStringBuf name) { @@ -190,7 +190,7 @@ namespace NMonitoring { private: void WriteValue() { - Consumer_->OnMetricBegin(MetricContext_.Type); + Consumer_->OnMetricBegin(MetricContext_.Type); Consumer_->OnLabelsBegin(); Consumer_->OnLabel("sensor", TString{MetricContext_.Name}); @@ -200,37 +200,37 @@ namespace NMonitoring { Consumer_->OnLabelsEnd(); - switch (MetricContext_.Type) { - case EMetricType::GAUGE: + switch (MetricContext_.Type) { + case EMetricType::GAUGE: Consumer_->OnDouble(Timestamp_, MetricContext_.Value.AsDouble()); break; - case EMetricType::RATE: + case EMetricType::RATE: Consumer_->OnUint64(Timestamp_, MetricContext_.Value.AsUint64()); break; - case EMetricType::HIST: - case EMetricType::HIST_RATE: + case EMetricType::HIST: + case EMetricType::HIST_RATE: Consumer_->OnHistogram(Timestamp_, MetricContext_.Value.AsHistogram()); break; case EMetricType::LOGHIST: - case EMetricType::DSUMMARY: - case EMetricType::IGAUGE: - case EMetricType::COUNTER: - case EMetricType::UNKNOWN: - ythrow yexception() << "Unexpected metric type: " << MetricContext_.Type; + case EMetricType::DSUMMARY: + case EMetricType::IGAUGE: + case EMetricType::COUNTER: + case EMetricType::UNKNOWN: + ythrow yexception() << "Unexpected metric type: " << MetricContext_.Type; } - Consumer_->OnMetricEnd(); + Consumer_->OnMetricEnd(); } private: - IMetricConsumer* Consumer_; + IMetricConsumer* Consumer_; NJson::TJsonValue Json_; TInstant Timestamp_; struct { TStringBuf Name; - EMetricType Type{EMetricType::UNKNOWN}; - TMetricValue Value; + EMetricType Type{EMetricType::UNKNOWN}; + TMetricValue Value; bool IsDeriv{false}; TLabels Labels; IHistogramSnapshotPtr Histogram; @@ -239,13 +239,13 @@ namespace NMonitoring { } - void DecodeUnistat(TStringBuf data, IMetricConsumer* c, TInstant ts) { + void DecodeUnistat(TStringBuf data, IMetricConsumer* c, TInstant ts) { c->OnStreamBegin(); DecodeUnistatToStream(data, c, ts); c->OnStreamEnd(); } - void DecodeUnistatToStream(TStringBuf data, IMetricConsumer* c, TInstant ts) { + void DecodeUnistatToStream(TStringBuf data, IMetricConsumer* c, TInstant ts) { TMemoryInput in{data.data(), data.size()}; TDecoderUnistat decoder(c, &in, ts); decoder.Decode(); diff --git a/library/cpp/monlib/encode/unistat/unistat_ut.cpp b/library/cpp/monlib/encode/unistat/unistat_ut.cpp index df71da3b47e..dbbc238bf32 100644 --- a/library/cpp/monlib/encode/unistat/unistat_ut.cpp +++ b/library/cpp/monlib/encode/unistat/unistat_ut.cpp @@ -1,7 +1,7 @@ #include "unistat.h" -#include <library/cpp/monlib/encode/protobuf/protobuf.h> -#include <library/cpp/monlib/metrics/labels.h> +#include <library/cpp/monlib/encode/protobuf/protobuf.h> +#include <library/cpp/monlib/metrics/labels.h> #include <library/cpp/testing/unittest/registar.h> @@ -18,7 +18,7 @@ Y_UNIT_TEST_SUITE(TUnistatDecoderTest) { UNIT_ASSERT_VALUES_EQUAL(samples.SamplesSize(), 1); auto sample = samples.GetSamples(0); - UNIT_ASSERT_EQUAL(sample.GetMetricType(), NProto::GAUGE); + UNIT_ASSERT_EQUAL(sample.GetMetricType(), NProto::GAUGE); UNIT_ASSERT_VALUES_EQUAL(sample.PointsSize(), 1); UNIT_ASSERT_VALUES_EQUAL(sample.LabelsSize(), 1); @@ -83,7 +83,7 @@ Y_UNIT_TEST_SUITE(TUnistatDecoderTest) { DecodeUnistat(input, encoder.Get()); auto sample = samples.GetSamples(0); - UNIT_ASSERT_EQUAL(sample.GetMetricType(), NProto::HIST_RATE); + UNIT_ASSERT_EQUAL(sample.GetMetricType(), NProto::HIST_RATE); UNIT_ASSERT_VALUES_EQUAL(sample.PointsSize(), 1); UNIT_ASSERT_VALUES_EQUAL(sample.LabelsSize(), 1); @@ -114,7 +114,7 @@ Y_UNIT_TEST_SUITE(TUnistatDecoderTest) { DecodeUnistat(input, encoder.Get()); auto sample = samples.GetSamples(0); - UNIT_ASSERT_EQUAL(sample.GetMetricType(), NProto::HISTOGRAM); + UNIT_ASSERT_EQUAL(sample.GetMetricType(), NProto::HISTOGRAM); UNIT_ASSERT_VALUES_EQUAL(sample.PointsSize(), 1); UNIT_ASSERT_VALUES_EQUAL(sample.LabelsSize(), 1); } @@ -159,7 +159,7 @@ Y_UNIT_TEST_SUITE(TUnistatDecoderTest) { UNIT_ASSERT_VALUES_EQUAL(samples.SamplesSize(), 2); auto sample = samples.GetSamples(0); - UNIT_ASSERT_EQUAL(sample.GetMetricType(), NProto::GAUGE); + UNIT_ASSERT_EQUAL(sample.GetMetricType(), NProto::GAUGE); UNIT_ASSERT_VALUES_EQUAL(sample.PointsSize(), 1); UNIT_ASSERT_VALUES_EQUAL(sample.LabelsSize(), 1); @@ -170,7 +170,7 @@ Y_UNIT_TEST_SUITE(TUnistatDecoderTest) { UNIT_ASSERT_VALUES_EQUAL(label.GetValue(), "something_axxx"); sample = samples.GetSamples(1); - UNIT_ASSERT_EQUAL(sample.GetMetricType(), NProto::RATE); + UNIT_ASSERT_EQUAL(sample.GetMetricType(), NProto::RATE); UNIT_ASSERT_VALUES_EQUAL(sample.PointsSize(), 1); UNIT_ASSERT_VALUES_EQUAL(sample.LabelsSize(), 1); @@ -190,7 +190,7 @@ Y_UNIT_TEST_SUITE(TUnistatDecoderTest) { UNIT_ASSERT_VALUES_EQUAL(samples.SamplesSize(), 1); auto sample = samples.GetSamples(0); - UNIT_ASSERT_EQUAL(sample.GetMetricType(), NProto::RATE); + UNIT_ASSERT_EQUAL(sample.GetMetricType(), NProto::RATE); UNIT_ASSERT_VALUES_EQUAL(sample.PointsSize(), 1); UNIT_ASSERT_VALUES_EQUAL(sample.LabelsSize(), 1); @@ -210,7 +210,7 @@ Y_UNIT_TEST_SUITE(TUnistatDecoderTest) { UNIT_ASSERT_VALUES_EQUAL(samples.SamplesSize(), 1); auto sample = samples.GetSamples(0); - UNIT_ASSERT_EQUAL(sample.GetMetricType(), NProto::GAUGE); + UNIT_ASSERT_EQUAL(sample.GetMetricType(), NProto::GAUGE); UNIT_ASSERT_VALUES_EQUAL(sample.PointsSize(), 1); UNIT_ASSERT_VALUES_EQUAL(sample.LabelsSize(), 1); diff --git a/library/cpp/monlib/encode/unistat/ut/ya.make b/library/cpp/monlib/encode/unistat/ut/ya.make index d9b2b516580..a652139f45f 100644 --- a/library/cpp/monlib/encode/unistat/ut/ya.make +++ b/library/cpp/monlib/encode/unistat/ut/ya.make @@ -1,4 +1,4 @@ -UNITTEST_FOR(library/cpp/monlib/encode/unistat) +UNITTEST_FOR(library/cpp/monlib/encode/unistat) OWNER( msherbakov @@ -10,7 +10,7 @@ SRCS( ) PEERDIR( - library/cpp/monlib/encode/protobuf + library/cpp/monlib/encode/protobuf ) END() diff --git a/library/cpp/monlib/encode/unistat/ya.make b/library/cpp/monlib/encode/unistat/ya.make index ae3c1c40da0..4ac2edadf40 100644 --- a/library/cpp/monlib/encode/unistat/ya.make +++ b/library/cpp/monlib/encode/unistat/ya.make @@ -8,7 +8,7 @@ OWNER( PEERDIR( contrib/libs/re2 library/cpp/json - library/cpp/monlib/metrics + library/cpp/monlib/metrics ) SRCS( diff --git a/library/cpp/monlib/encode/ut/ya.make b/library/cpp/monlib/encode/ut/ya.make index b83f62f2a2d..1990386d760 100644 --- a/library/cpp/monlib/encode/ut/ya.make +++ b/library/cpp/monlib/encode/ut/ya.make @@ -1,12 +1,12 @@ -UNITTEST_FOR(library/cpp/monlib/encode) - +UNITTEST_FOR(library/cpp/monlib/encode) + OWNER( jamel g:solomon ) - -SRCS( - format_ut.cpp -) - -END() + +SRCS( + format_ut.cpp +) + +END() diff --git a/library/cpp/monlib/encode/ya.make b/library/cpp/monlib/encode/ya.make index a0dae3582fa..d1bb09f07b8 100644 --- a/library/cpp/monlib/encode/ya.make +++ b/library/cpp/monlib/encode/ya.make @@ -1,20 +1,20 @@ -LIBRARY() - +LIBRARY() + OWNER( g:solomon jamel ) - -SRCS( - encoder.cpp - encoder_state.cpp - format.cpp -) - -PEERDIR( - library/cpp/monlib/metrics -) - + +SRCS( + encoder.cpp + encoder_state.cpp + format.cpp +) + +PEERDIR( + library/cpp/monlib/metrics +) + GENERATE_ENUM_SERIALIZATION_WITH_HEADER(encoder_state_enum.h) -END() +END() diff --git a/library/cpp/monlib/messagebus/mon_messagebus.cpp b/library/cpp/monlib/messagebus/mon_messagebus.cpp index 414f3f6bb45..355b4386cd9 100644 --- a/library/cpp/monlib/messagebus/mon_messagebus.cpp +++ b/library/cpp/monlib/messagebus/mon_messagebus.cpp @@ -6,6 +6,6 @@ using namespace NMonitoring; void TBusNgMonPage::Output(NMonitoring::IMonHttpRequest& request) { NBus::TBusWww::TOptionalParams params; - params.ParentLinks.push_back(NBus::TBusWww::TLink{"/", request.GetServiceTitle()}); + params.ParentLinks.push_back(NBus::TBusWww::TLink{"/", request.GetServiceTitle()}); BusWww->ServeHttp(request.Output(), request.GetParams(), params); } diff --git a/library/cpp/monlib/messagebus/mon_messagebus.h b/library/cpp/monlib/messagebus/mon_messagebus.h index 2cf46ecbbc2..e1fa73c69fb 100644 --- a/library/cpp/monlib/messagebus/mon_messagebus.h +++ b/library/cpp/monlib/messagebus/mon_messagebus.h @@ -3,44 +3,44 @@ #include <library/cpp/messagebus/ybus.h> #include <library/cpp/messagebus/www/www.h> -#include <library/cpp/monlib/service/pages/mon_page.h> +#include <library/cpp/monlib/service/pages/mon_page.h> namespace NMonitoring { - template <class TBusSmth> - class TBusSmthMonPage: public NMonitoring::IMonPage { - private: - TBusSmth* Smth; - - public: - explicit TBusSmthMonPage(const TString& name, const TString& title, TBusSmth* smth) - : IMonPage("msgbus/" + name, title) - , Smth(smth) - { - } + template <class TBusSmth> + class TBusSmthMonPage: public NMonitoring::IMonPage { + private: + TBusSmth* Smth; + + public: + explicit TBusSmthMonPage(const TString& name, const TString& title, TBusSmth* smth) + : IMonPage("msgbus/" + name, title) + , Smth(smth) + { + } void Output(NMonitoring::IMonHttpRequest& request) override { - Y_UNUSED(request); + Y_UNUSED(request); request.Output() << NMonitoring::HTTPOKHTML; request.Output() << "<h2>" << Title << "</h2>"; request.Output() << "<pre>"; request.Output() << Smth->GetStatus(); request.Output() << "</pre>"; - } - }; - - using TBusQueueMonPage = TBusSmthMonPage<NBus::TBusMessageQueue>; - using TBusModuleMonPage = TBusSmthMonPage<NBus::TBusModule>; - - class TBusNgMonPage: public NMonitoring::IMonPage { - public: - TIntrusivePtr<NBus::TBusWww> BusWww; - - public: - TBusNgMonPage() - : IMonPage("messagebus", "MessageBus") - , BusWww(new NBus::TBusWww) - { - } + } + }; + + using TBusQueueMonPage = TBusSmthMonPage<NBus::TBusMessageQueue>; + using TBusModuleMonPage = TBusSmthMonPage<NBus::TBusModule>; + + class TBusNgMonPage: public NMonitoring::IMonPage { + public: + TIntrusivePtr<NBus::TBusWww> BusWww; + + public: + TBusNgMonPage() + : IMonPage("messagebus", "MessageBus") + , BusWww(new NBus::TBusWww) + { + } void Output(NMonitoring::IMonHttpRequest& request) override; - }; - -} + }; + +} diff --git a/library/cpp/monlib/messagebus/mon_service_messagebus.cpp b/library/cpp/monlib/messagebus/mon_service_messagebus.cpp index 36ab81b012e..4dd144ebe83 100644 --- a/library/cpp/monlib/messagebus/mon_service_messagebus.cpp +++ b/library/cpp/monlib/messagebus/mon_service_messagebus.cpp @@ -4,5 +4,5 @@ using namespace NMonitoring; TMonServiceMessageBus::TMonServiceMessageBus(ui16 port, const TString& title) : TMonService2(port, title) -{ -} +{ +} diff --git a/library/cpp/monlib/messagebus/mon_service_messagebus.h b/library/cpp/monlib/messagebus/mon_service_messagebus.h index 32c7b4cb703..fe791e8a9b8 100644 --- a/library/cpp/monlib/messagebus/mon_service_messagebus.h +++ b/library/cpp/monlib/messagebus/mon_service_messagebus.h @@ -1,46 +1,46 @@ #pragma once -#include "mon_messagebus.h" - -#include <library/cpp/monlib/service/monservice.h> - +#include "mon_messagebus.h" + +#include <library/cpp/monlib/service/monservice.h> + #include <util/system/mutex.h> namespace NMonitoring { - class TMonServiceMessageBus: public TMonService2 { - private: - TMutex Mtx; - TIntrusivePtr<NMonitoring::TBusNgMonPage> BusNgMonPage; - - public: - TMonServiceMessageBus(ui16 port, const TString& title); - - private: - NBus::TBusWww* RegisterBusNgMonPage() { - TGuard<TMutex> g(Mtx); - if (!BusNgMonPage) { - BusNgMonPage = new NMonitoring::TBusNgMonPage(); - Register(BusNgMonPage.Get()); - } - return BusNgMonPage->BusWww.Get(); - } - - public: - void RegisterClientSession(NBus::TBusClientSessionPtr clientSession) { - RegisterBusNgMonPage()->RegisterClientSession(clientSession); + class TMonServiceMessageBus: public TMonService2 { + private: + TMutex Mtx; + TIntrusivePtr<NMonitoring::TBusNgMonPage> BusNgMonPage; + + public: + TMonServiceMessageBus(ui16 port, const TString& title); + + private: + NBus::TBusWww* RegisterBusNgMonPage() { + TGuard<TMutex> g(Mtx); + if (!BusNgMonPage) { + BusNgMonPage = new NMonitoring::TBusNgMonPage(); + Register(BusNgMonPage.Get()); + } + return BusNgMonPage->BusWww.Get(); } - void RegisterServerSession(NBus::TBusServerSessionPtr serverSession) { - RegisterBusNgMonPage()->RegisterServerSession(serverSession); - } + public: + void RegisterClientSession(NBus::TBusClientSessionPtr clientSession) { + RegisterBusNgMonPage()->RegisterClientSession(clientSession); + } - void RegisterQueue(NBus::TBusMessageQueuePtr queue) { - RegisterBusNgMonPage()->RegisterQueue(queue); - } + void RegisterServerSession(NBus::TBusServerSessionPtr serverSession) { + RegisterBusNgMonPage()->RegisterServerSession(serverSession); + } - void RegisterModule(NBus::TBusModule* module) { - RegisterBusNgMonPage()->RegisterModule(module); - } - }; + void RegisterQueue(NBus::TBusMessageQueuePtr queue) { + RegisterBusNgMonPage()->RegisterQueue(queue); + } + + void RegisterModule(NBus::TBusModule* module) { + RegisterBusNgMonPage()->RegisterModule(module); + } + }; -} +} diff --git a/library/cpp/monlib/messagebus/ya.make b/library/cpp/monlib/messagebus/ya.make index 79889e38a79..a0b5362296d 100644 --- a/library/cpp/monlib/messagebus/ya.make +++ b/library/cpp/monlib/messagebus/ya.make @@ -10,7 +10,7 @@ SRCS( PEERDIR( library/cpp/messagebus library/cpp/messagebus/www - library/cpp/monlib/dynamic_counters + library/cpp/monlib/dynamic_counters ) END() diff --git a/library/cpp/monlib/metrics/atomics_array.h b/library/cpp/monlib/metrics/atomics_array.h index c9193f0a332..f19aebf2915 100644 --- a/library/cpp/monlib/metrics/atomics_array.h +++ b/library/cpp/monlib/metrics/atomics_array.h @@ -1,52 +1,52 @@ -#pragma once - -#include <util/generic/ptr.h> -#include <util/generic/vector.h> - +#pragma once + +#include <util/generic/ptr.h> +#include <util/generic/vector.h> + #include <atomic> -namespace NMonitoring { - class TAtomicsArray { - public: - explicit TAtomicsArray(size_t size) +namespace NMonitoring { + class TAtomicsArray { + public: + explicit TAtomicsArray(size_t size) : Values_(new std::atomic<ui64>[size]) - , Size_(size) - { - for (size_t i = 0; i < Size_; i++) { + , Size_(size) + { + for (size_t i = 0; i < Size_; i++) { Values_[i].store(0, std::memory_order_relaxed); - } - } - - ui64 operator[](size_t index) const noexcept { - Y_VERIFY_DEBUG(index < Size_); + } + } + + ui64 operator[](size_t index) const noexcept { + Y_VERIFY_DEBUG(index < Size_); return Values_[index].load(std::memory_order_relaxed); - } - - size_t Size() const noexcept { - return Size_; - } - - void Add(size_t index, ui32 count) noexcept { - Y_VERIFY_DEBUG(index < Size_); + } + + size_t Size() const noexcept { + return Size_; + } + + void Add(size_t index, ui32 count) noexcept { + Y_VERIFY_DEBUG(index < Size_); Values_[index].fetch_add(count, std::memory_order_relaxed); - } - + } + void Reset() noexcept { for (size_t i = 0; i < Size_; i++) { Values_[i].store(0, std::memory_order_relaxed); } } - TVector<ui64> Copy() const { - TVector<ui64> copy(Reserve(Size_)); - for (size_t i = 0; i < Size_; i++) { + TVector<ui64> Copy() const { + TVector<ui64> copy(Reserve(Size_)); + for (size_t i = 0; i < Size_; i++) { copy.push_back(Values_[i].load(std::memory_order_relaxed)); - } - return copy; - } - - private: + } + return copy; + } + + private: TArrayHolder<std::atomic<ui64>> Values_; - size_t Size_; - }; -} + size_t Size_; + }; +} diff --git a/library/cpp/monlib/metrics/ewma.cpp b/library/cpp/monlib/metrics/ewma.cpp index 4ee78c0c3d4..8a296c32258 100644 --- a/library/cpp/monlib/metrics/ewma.cpp +++ b/library/cpp/monlib/metrics/ewma.cpp @@ -1,5 +1,5 @@ #include "ewma.h" -#include "metric.h" +#include "metric.h" #include <atomic> #include <cmath> @@ -14,12 +14,12 @@ namespace { class TExpMovingAverage final: public IExpMovingAverage { public: - explicit TExpMovingAverage(IGauge* metric, double alpha, TDuration interval) - : Metric_{metric} + explicit TExpMovingAverage(IGauge* metric, double alpha, TDuration interval) + : Metric_{metric} , Alpha_{alpha} , Interval_{interval.Seconds()} { - Y_VERIFY(metric != nullptr, "Passing nullptr metric is not allowed"); + Y_VERIFY(metric != nullptr, "Passing nullptr metric is not allowed"); } ~TExpMovingAverage() override = default; @@ -30,11 +30,11 @@ namespace { const double instantRate = double(current) / Interval_; if (Y_UNLIKELY(!IsInitialized())) { - Metric_->Set(instantRate); + Metric_->Set(instantRate); Init_ = true; } else { - const double currentRate = Metric_->Get(); - Metric_->Set(Alpha_ * (instantRate - currentRate) + currentRate); + const double currentRate = Metric_->Get(); + Metric_->Set(Alpha_ * (instantRate - currentRate) + currentRate); } } @@ -44,7 +44,7 @@ namespace { } double Rate() const override { - return Metric_->Get(); + return Metric_->Get(); } void Reset() override { @@ -61,7 +61,7 @@ namespace { std::atomic<i64> Uncounted_{0}; std::atomic<bool> Init_{false}; - IGauge* Metric_{nullptr}; + IGauge* Metric_{nullptr}; double Alpha_; ui64 Interval_; }; @@ -132,19 +132,19 @@ namespace { return Ewma_->Rate(); } - IExpMovingAveragePtr OneMinuteEwma(IGauge* metric) { - return MakeHolder<TExpMovingAverage>(metric, ALPHA1, DEFAULT_INTERVAL); + IExpMovingAveragePtr OneMinuteEwma(IGauge* metric) { + return MakeHolder<TExpMovingAverage>(metric, ALPHA1, DEFAULT_INTERVAL); } - IExpMovingAveragePtr FiveMinuteEwma(IGauge* metric) { - return MakeHolder<TExpMovingAverage>(metric, ALPHA5, DEFAULT_INTERVAL); + IExpMovingAveragePtr FiveMinuteEwma(IGauge* metric) { + return MakeHolder<TExpMovingAverage>(metric, ALPHA5, DEFAULT_INTERVAL); } - IExpMovingAveragePtr FiveteenMinuteEwma(IGauge* metric) { - return MakeHolder<TExpMovingAverage>(metric, ALPHA15, DEFAULT_INTERVAL); + IExpMovingAveragePtr FiveteenMinuteEwma(IGauge* metric) { + return MakeHolder<TExpMovingAverage>(metric, ALPHA15, DEFAULT_INTERVAL); } - IExpMovingAveragePtr CreateEwma(IGauge* metric, double alpha, TDuration interval) { - return MakeHolder<TExpMovingAverage>(metric, alpha, interval); + IExpMovingAveragePtr CreateEwma(IGauge* metric, double alpha, TDuration interval) { + return MakeHolder<TExpMovingAverage>(metric, alpha, interval); } } // namespace NMonitoring diff --git a/library/cpp/monlib/metrics/ewma_ut.cpp b/library/cpp/monlib/metrics/ewma_ut.cpp index 56e292f9fdc..01ef2478f78 100644 --- a/library/cpp/monlib/metrics/ewma_ut.cpp +++ b/library/cpp/monlib/metrics/ewma_ut.cpp @@ -1,5 +1,5 @@ #include "ewma.h" -#include "metric.h" +#include "metric.h" #include <library/cpp/testing/unittest/registar.h> @@ -15,9 +15,9 @@ void ElapseMinute(IExpMovingAverage& ewma) { Y_UNIT_TEST_SUITE(TEwmaTest) { Y_UNIT_TEST(OneMinute) { - TGauge gauge; + TGauge gauge; - auto ewma = OneMinuteEwma(&gauge); + auto ewma = OneMinuteEwma(&gauge); ewma->Update(3); ewma->Tick(); @@ -47,9 +47,9 @@ Y_UNIT_TEST_SUITE(TEwmaTest) { } Y_UNIT_TEST(FiveMinutes) { - TGauge gauge; + TGauge gauge; - auto ewma = FiveMinuteEwma(&gauge); + auto ewma = FiveMinuteEwma(&gauge); ewma->Update(3); ewma->Tick(); @@ -79,9 +79,9 @@ Y_UNIT_TEST_SUITE(TEwmaTest) { } Y_UNIT_TEST(FiveteenMinutes) { - TGauge gauge; + TGauge gauge; - auto ewma = FiveteenMinuteEwma(&gauge); + auto ewma = FiveteenMinuteEwma(&gauge); ewma->Update(3); ewma->Tick(); diff --git a/library/cpp/monlib/metrics/fake.cpp b/library/cpp/monlib/metrics/fake.cpp index 41522314ca3..b6f5e37af89 100644 --- a/library/cpp/monlib/metrics/fake.cpp +++ b/library/cpp/monlib/metrics/fake.cpp @@ -1,9 +1,9 @@ #include "fake.h" namespace NMonitoring { - - IGauge* TFakeMetricRegistry::Gauge(ILabelsPtr labels) { - return Metric<TFakeGauge, EMetricType::GAUGE>(std::move(labels)); + + IGauge* TFakeMetricRegistry::Gauge(ILabelsPtr labels) { + return Metric<TFakeGauge, EMetricType::GAUGE>(std::move(labels)); } ILazyGauge* TFakeMetricRegistry::LazyGauge(ILabelsPtr labels, std::function<double()> supplier) { @@ -11,8 +11,8 @@ namespace NMonitoring { return Metric<TFakeLazyGauge, EMetricType::GAUGE>(std::move(labels)); } - IIntGauge* TFakeMetricRegistry::IntGauge(ILabelsPtr labels) { - return Metric<TFakeIntGauge, EMetricType::IGAUGE>(std::move(labels)); + IIntGauge* TFakeMetricRegistry::IntGauge(ILabelsPtr labels) { + return Metric<TFakeIntGauge, EMetricType::IGAUGE>(std::move(labels)); } ILazyIntGauge* TFakeMetricRegistry::LazyIntGauge(ILabelsPtr labels, std::function<i64()> supplier) { @@ -20,8 +20,8 @@ namespace NMonitoring { return Metric<TFakeLazyIntGauge, EMetricType::IGAUGE>(std::move(labels)); } - ICounter* TFakeMetricRegistry::Counter(ILabelsPtr labels) { - return Metric<TFakeCounter, EMetricType::COUNTER>(std::move(labels)); + ICounter* TFakeMetricRegistry::Counter(ILabelsPtr labels) { + return Metric<TFakeCounter, EMetricType::COUNTER>(std::move(labels)); } ILazyCounter* TFakeMetricRegistry::LazyCounter(ILabelsPtr labels, std::function<ui64()> supplier) { @@ -29,8 +29,8 @@ namespace NMonitoring { return Metric<TFakeLazyCounter, EMetricType::COUNTER>(std::move(labels)); } - IRate* TFakeMetricRegistry::Rate(ILabelsPtr labels) { - return Metric<TFakeRate, EMetricType::RATE>(std::move(labels)); + IRate* TFakeMetricRegistry::Rate(ILabelsPtr labels) { + return Metric<TFakeRate, EMetricType::RATE>(std::move(labels)); } ILazyRate* TFakeMetricRegistry::LazyRate(ILabelsPtr labels, std::function<ui64()> supplier) { @@ -38,46 +38,46 @@ namespace NMonitoring { return Metric<TFakeLazyRate, EMetricType::RATE>(std::move(labels)); } - IHistogram* TFakeMetricRegistry::HistogramCounter(ILabelsPtr labels, IHistogramCollectorPtr collector) { + IHistogram* TFakeMetricRegistry::HistogramCounter(ILabelsPtr labels, IHistogramCollectorPtr collector) { Y_UNUSED(collector); - return Metric<TFakeHistogram, EMetricType::HIST>(std::move(labels), false); + return Metric<TFakeHistogram, EMetricType::HIST>(std::move(labels), false); } - void TFakeMetricRegistry::RemoveMetric(const ILabels& labels) noexcept { + void TFakeMetricRegistry::RemoveMetric(const ILabels& labels) noexcept { TWriteGuard g{Lock_}; - Metrics_.erase(labels); + Metrics_.erase(labels); } - void TFakeMetricRegistry::Accept(TInstant time, IMetricConsumer* consumer) const { + void TFakeMetricRegistry::Accept(TInstant time, IMetricConsumer* consumer) const { Y_UNUSED(time); consumer->OnStreamBegin(); consumer->OnStreamEnd(); } - IHistogram* TFakeMetricRegistry::HistogramRate(ILabelsPtr labels, IHistogramCollectorPtr collector) { + IHistogram* TFakeMetricRegistry::HistogramRate(ILabelsPtr labels, IHistogramCollectorPtr collector) { Y_UNUSED(collector); - return Metric<TFakeHistogram, EMetricType::HIST_RATE>(std::move(labels), true); + return Metric<TFakeHistogram, EMetricType::HIST_RATE>(std::move(labels), true); } - void TFakeMetricRegistry::Append(TInstant time, IMetricConsumer* consumer) const { + void TFakeMetricRegistry::Append(TInstant time, IMetricConsumer* consumer) const { Y_UNUSED(time, consumer); } - const TLabels& TFakeMetricRegistry::CommonLabels() const noexcept { + const TLabels& TFakeMetricRegistry::CommonLabels() const noexcept { return CommonLabels_; } - template <typename TMetric, EMetricType type, typename TLabelsType, typename... Args> - TMetric* TFakeMetricRegistry::Metric(TLabelsType&& labels, Args&&... args) { + template <typename TMetric, EMetricType type, typename TLabelsType, typename... Args> + TMetric* TFakeMetricRegistry::Metric(TLabelsType&& labels, Args&&... args) { { TReadGuard g{Lock_}; - auto it = Metrics_.find(labels); - if (it != Metrics_.end()) { - Y_ENSURE(it->second->Type() == type, "cannot create metric " << labels - << " with type " << MetricTypeToStr(type) - << ", because registry already has same metric with type " << MetricTypeToStr(it->second->Type())); - return static_cast<TMetric*>(it->second.Get()); + auto it = Metrics_.find(labels); + if (it != Metrics_.end()) { + Y_ENSURE(it->second->Type() == type, "cannot create metric " << labels + << " with type " << MetricTypeToStr(type) + << ", because registry already has same metric with type " << MetricTypeToStr(it->second->Type())); + return static_cast<TMetric*>(it->second.Get()); } } @@ -86,15 +86,15 @@ namespace NMonitoring { IMetricPtr metric = MakeHolder<TMetric>(std::forward<Args>(args)...); - // decltype(Metrics_)::iterator breaks build on windows - THashMap<ILabelsPtr, IMetricPtr>::iterator it; + // decltype(Metrics_)::iterator breaks build on windows + THashMap<ILabelsPtr, IMetricPtr>::iterator it; if constexpr (!std::is_convertible_v<TLabelsType, ILabelsPtr>) { - it = Metrics_.emplace(new TLabels{std::forward<TLabelsType>(labels)}, std::move(metric)).first; + it = Metrics_.emplace(new TLabels{std::forward<TLabelsType>(labels)}, std::move(metric)).first; } else { - it = Metrics_.emplace(std::forward<TLabelsType>(labels), std::move(metric)).first; + it = Metrics_.emplace(std::forward<TLabelsType>(labels), std::move(metric)).first; } - return static_cast<TMetric*>(it->second.Get()); + return static_cast<TMetric*>(it->second.Get()); } } } // namespace NMonitoring diff --git a/library/cpp/monlib/metrics/fake.h b/library/cpp/monlib/metrics/fake.h index c2e40b267b4..61ba4f2bd48 100644 --- a/library/cpp/monlib/metrics/fake.h +++ b/library/cpp/monlib/metrics/fake.h @@ -1,21 +1,21 @@ #pragma once -#include "metric.h" -#include "metric_registry.h" +#include "metric.h" +#include "metric_registry.h" namespace NMonitoring { - class TFakeMetricRegistry: public IMetricRegistry { + class TFakeMetricRegistry: public IMetricRegistry { public: - TFakeMetricRegistry() noexcept - : CommonLabels_{0} - { - } - - explicit TFakeMetricRegistry(TLabels commonLabels) noexcept - : CommonLabels_{std::move(commonLabels)} - { - } - + TFakeMetricRegistry() noexcept + : CommonLabels_{0} + { + } + + explicit TFakeMetricRegistry(TLabels commonLabels) noexcept + : CommonLabels_{std::move(commonLabels)} + { + } + IGauge* Gauge(ILabelsPtr labels) override; ILazyGauge* LazyGauge(ILabelsPtr labels, std::function<double()> supplier) override; IIntGauge* IntGauge(ILabelsPtr labels) override; @@ -32,25 +32,25 @@ namespace NMonitoring { IHistogram* HistogramRate( ILabelsPtr labels, IHistogramCollectorPtr collector) override; - void Accept(TInstant time, IMetricConsumer* consumer) const override; - void Append(TInstant time, IMetricConsumer* consumer) const override; + void Accept(TInstant time, IMetricConsumer* consumer) const override; + void Append(TInstant time, IMetricConsumer* consumer) const override; const TLabels& CommonLabels() const noexcept override; - void RemoveMetric(const ILabels& labels) noexcept override; + void RemoveMetric(const ILabels& labels) noexcept override; private: TRWMutex Lock_; - THashMap<ILabelsPtr, IMetricPtr> Metrics_; + THashMap<ILabelsPtr, IMetricPtr> Metrics_; - template <typename TMetric, EMetricType type, typename TLabelsType, typename... Args> - TMetric* Metric(TLabelsType&& labels, Args&&... args); + template <typename TMetric, EMetricType type, typename TLabelsType, typename... Args> + TMetric* Metric(TLabelsType&& labels, Args&&... args); const TLabels CommonLabels_; }; template <typename TBase> struct TFakeAcceptor: TBase { - void Accept(TInstant time, IMetricConsumer* consumer) const override { + void Accept(TInstant time, IMetricConsumer* consumer) const override { Y_UNUSED(time, consumer); } }; @@ -85,9 +85,9 @@ namespace NMonitoring { ui64 Get() const noexcept override { return 0; } - - void Reset() noexcept override { - } + + void Reset() noexcept override { + } }; struct TFakeLazyRate final: public TFakeAcceptor<ILazyRate> { @@ -123,11 +123,11 @@ namespace NMonitoring { { } - void Record(double value) override { + void Record(double value) override { Y_UNUSED(value); } - void Record(double value, ui32 count) override { + void Record(double value, ui32 count) override { Y_UNUSED(value, count); } @@ -135,7 +135,7 @@ namespace NMonitoring { return nullptr; } - void Accept(TInstant time, IMetricConsumer* consumer) const override { + void Accept(TInstant time, IMetricConsumer* consumer) const override { Y_UNUSED(time, consumer); } @@ -148,7 +148,7 @@ namespace NMonitoring { Y_UNUSED(n); return 0; } - + ui64 Get() const noexcept override { return 0; } diff --git a/library/cpp/monlib/metrics/fake_ut.cpp b/library/cpp/monlib/metrics/fake_ut.cpp index af164f46284..c3368ca3021 100644 --- a/library/cpp/monlib/metrics/fake_ut.cpp +++ b/library/cpp/monlib/metrics/fake_ut.cpp @@ -1,34 +1,34 @@ -#include "fake.h" - -#include <library/cpp/testing/unittest/registar.h> - -#include <util/generic/ptr.h> - -using namespace NMonitoring; - -Y_UNIT_TEST_SUITE(TFakeTest) { - - Y_UNIT_TEST(CreateOnStack) { - TFakeMetricRegistry registry; - } - - Y_UNIT_TEST(CreateOnHeap) { - auto registry = MakeAtomicShared<TFakeMetricRegistry>(); - UNIT_ASSERT(registry); - } - - Y_UNIT_TEST(Gauge) { - TFakeMetricRegistry registry(TLabels{{"common", "label"}}); - - IGauge* g = registry.Gauge(MakeLabels({{"my", "gauge"}})); - UNIT_ASSERT(g); - - UNIT_ASSERT_DOUBLES_EQUAL(g->Get(), 0.0, 1E-6); - g->Set(12.34); - UNIT_ASSERT_DOUBLES_EQUAL(g->Get(), 0.0, 1E-6); // no changes - - double val = g->Add(1.2); - UNIT_ASSERT_DOUBLES_EQUAL(g->Get(), 0.0, 1E-6); - UNIT_ASSERT_DOUBLES_EQUAL(val, 0.0, 1E-6); - } -} +#include "fake.h" + +#include <library/cpp/testing/unittest/registar.h> + +#include <util/generic/ptr.h> + +using namespace NMonitoring; + +Y_UNIT_TEST_SUITE(TFakeTest) { + + Y_UNIT_TEST(CreateOnStack) { + TFakeMetricRegistry registry; + } + + Y_UNIT_TEST(CreateOnHeap) { + auto registry = MakeAtomicShared<TFakeMetricRegistry>(); + UNIT_ASSERT(registry); + } + + Y_UNIT_TEST(Gauge) { + TFakeMetricRegistry registry(TLabels{{"common", "label"}}); + + IGauge* g = registry.Gauge(MakeLabels({{"my", "gauge"}})); + UNIT_ASSERT(g); + + UNIT_ASSERT_DOUBLES_EQUAL(g->Get(), 0.0, 1E-6); + g->Set(12.34); + UNIT_ASSERT_DOUBLES_EQUAL(g->Get(), 0.0, 1E-6); // no changes + + double val = g->Add(1.2); + UNIT_ASSERT_DOUBLES_EQUAL(g->Get(), 0.0, 1E-6); + UNIT_ASSERT_DOUBLES_EQUAL(val, 0.0, 1E-6); + } +} diff --git a/library/cpp/monlib/metrics/fwd.h b/library/cpp/monlib/metrics/fwd.h index 91b0ad9fcb2..b4327ee5d5e 100644 --- a/library/cpp/monlib/metrics/fwd.h +++ b/library/cpp/monlib/metrics/fwd.h @@ -1,40 +1,40 @@ -#pragma once - -namespace NMonitoring { - - struct ILabel; - struct ILabels; - - class ICounter; - class IGauge; - class IHistogram; - class IIntGauge; - class ILazyCounter; - class ILazyGauge; - class ILazyIntGauge; - class ILazyRate; - class IMetric; - class IRate; - class TCounter; - class TGauge; - class THistogram; - class TIntGauge; - class TLazyCounter; - class TLazyGauge; - class TLazyIntGauge; - class TLazyRate; - class TRate; - - class IMetricSupplier; - class IMetricFactory; - class IMetricConsumer; - - class IMetricRegistry; - class TMetricRegistry; - - class IHistogramCollector; - class IHistogramSnapshot; - - class IExpMovingAverage; - -} // namespace NMonitoring +#pragma once + +namespace NMonitoring { + + struct ILabel; + struct ILabels; + + class ICounter; + class IGauge; + class IHistogram; + class IIntGauge; + class ILazyCounter; + class ILazyGauge; + class ILazyIntGauge; + class ILazyRate; + class IMetric; + class IRate; + class TCounter; + class TGauge; + class THistogram; + class TIntGauge; + class TLazyCounter; + class TLazyGauge; + class TLazyIntGauge; + class TLazyRate; + class TRate; + + class IMetricSupplier; + class IMetricFactory; + class IMetricConsumer; + + class IMetricRegistry; + class TMetricRegistry; + + class IHistogramCollector; + class IHistogramSnapshot; + + class IExpMovingAverage; + +} // namespace NMonitoring diff --git a/library/cpp/monlib/metrics/histogram_collector.h b/library/cpp/monlib/metrics/histogram_collector.h index b1d628ce9ef..9f6bbbdfb73 100644 --- a/library/cpp/monlib/metrics/histogram_collector.h +++ b/library/cpp/monlib/metrics/histogram_collector.h @@ -1,119 +1,119 @@ -#pragma once - -#include "histogram_snapshot.h" - -namespace NMonitoring { - - /////////////////////////////////////////////////////////////////////////// - // IHistogramCollector - /////////////////////////////////////////////////////////////////////////// - class IHistogramCollector { - public: - virtual ~IHistogramCollector() = default; - - /** - * Store {@code count} times given {@code value} in this collector. - */ +#pragma once + +#include "histogram_snapshot.h" + +namespace NMonitoring { + + /////////////////////////////////////////////////////////////////////////// + // IHistogramCollector + /////////////////////////////////////////////////////////////////////////// + class IHistogramCollector { + public: + virtual ~IHistogramCollector() = default; + + /** + * Store {@code count} times given {@code value} in this collector. + */ virtual void Collect(double value, ui32 count) = 0; - - /** - * Store given {@code value} in this collector. - */ - void Collect(double value) { - Collect(value, 1); - } - - /** - * Add counts from snapshot into this collector - */ - void Collect(const IHistogramSnapshot& snapshot) { - for (ui32 i = 0; i < snapshot.Count(); i++) { - Collect(snapshot.UpperBound(i), snapshot.Value(i)); - } - } - - /** + + /** + * Store given {@code value} in this collector. + */ + void Collect(double value) { + Collect(value, 1); + } + + /** + * Add counts from snapshot into this collector + */ + void Collect(const IHistogramSnapshot& snapshot) { + for (ui32 i = 0; i < snapshot.Count(); i++) { + Collect(snapshot.UpperBound(i), snapshot.Value(i)); + } + } + + /** * Reset collector values */ virtual void Reset() = 0; /** - * @return snapshot of the state of this collector. - */ - virtual IHistogramSnapshotPtr Snapshot() const = 0; - }; - - using IHistogramCollectorPtr = THolder<IHistogramCollector>; - - /////////////////////////////////////////////////////////////////////////// - // free functions - /////////////////////////////////////////////////////////////////////////// - - /** - * <p>Creates histogram collector for a set of buckets with arbitrary - * bounds.</p> - * - * <p>Defines {@code bounds.size() + 1} buckets with these boundaries for - * bucket i:</p> - * <ul> - * <li>Upper bound (0 <= i < N-1): {@code bounds[i]}</li> - * <li>Lower bound (1 <= i < N): {@code bounds[i - 1]}</li> - * </ul> - * - * <p>For example, if the list of boundaries is:</p> - * <pre>0, 1, 2, 5, 10, 20</pre> - * - * <p>then there are five finite buckets with the following ranges:</p> - * <pre>(-INF, 0], (0, 1], (1, 2], (2, 5], (5, 10], (10, 20], (20, +INF)</pre> - * - * @param bounds array of upper bounds for buckets. Values must be sorted. - */ + * @return snapshot of the state of this collector. + */ + virtual IHistogramSnapshotPtr Snapshot() const = 0; + }; + + using IHistogramCollectorPtr = THolder<IHistogramCollector>; + + /////////////////////////////////////////////////////////////////////////// + // free functions + /////////////////////////////////////////////////////////////////////////// + + /** + * <p>Creates histogram collector for a set of buckets with arbitrary + * bounds.</p> + * + * <p>Defines {@code bounds.size() + 1} buckets with these boundaries for + * bucket i:</p> + * <ul> + * <li>Upper bound (0 <= i < N-1): {@code bounds[i]}</li> + * <li>Lower bound (1 <= i < N): {@code bounds[i - 1]}</li> + * </ul> + * + * <p>For example, if the list of boundaries is:</p> + * <pre>0, 1, 2, 5, 10, 20</pre> + * + * <p>then there are five finite buckets with the following ranges:</p> + * <pre>(-INF, 0], (0, 1], (1, 2], (2, 5], (5, 10], (10, 20], (20, +INF)</pre> + * + * @param bounds array of upper bounds for buckets. Values must be sorted. + */ IHistogramCollectorPtr ExplicitHistogram(TBucketBounds bounds); - - /** - * <p>Creates histogram collector for a sequence of buckets that have a - * width proportional to the value of the lower bound.</p> - * - * <p>Defines {@code bucketsCount} buckets with these boundaries for bucket i:</p> - * <ul> - * <li>Upper bound (0 <= i < N-1): {@code scale * (base ^ i)}</li> - * <li>Lower bound (1 <= i < N): {@code scale * (base ^ (i - 1))}</li> - * </ul> - * - * <p>For example, if {@code bucketsCount=6}, {@code base=2}, and {@code scale=3}, - * then the bucket ranges are as follows:</p> - * - * <pre>(-INF, 3], (3, 6], (6, 12], (12, 24], (24, 48], (48, +INF)</pre> - * - * @param bucketsCount the total number of buckets. The value must be >= 2. - * @param base the exponential growth factor for the buckets width. - * The value must be >= 1.0. - * @param scale the linear scale for the buckets. The value must be >= 1.0. - */ - IHistogramCollectorPtr ExponentialHistogram( - ui32 bucketsCount, double base, double scale = 1.0); - - /** - * <p>Creates histogram collector for a sequence of buckets that all have - * the same width (except overflow and underflow).</p> - * - * <p>Defines {@code bucketsCount} buckets with these boundaries for bucket i:</p> - * <ul> - * <li>Upper bound (0 <= i < N-1): {@code startValue + bucketWidth * i}</li> - * <li>Lower bound (1 <= i < N): {@code startValue + bucketWidth * (i - 1)}</li> - * </ul> - * - * <p>For example, if {@code bucketsCount=6}, {@code startValue=5}, and - * {@code bucketWidth=15}, then the bucket ranges are as follows:</p> - * - * <pre>(-INF, 5], (5, 20], (20, 35], (35, 50], (50, 65], (65, +INF)</pre> - * - * @param bucketsCount the total number of buckets. The value must be >= 2. - * @param startValue the upper boundary of the first bucket. - * @param bucketWidth the difference between the upper and lower bounds for - * each bucket. The value must be >= 1. - */ - IHistogramCollectorPtr LinearHistogram( + + /** + * <p>Creates histogram collector for a sequence of buckets that have a + * width proportional to the value of the lower bound.</p> + * + * <p>Defines {@code bucketsCount} buckets with these boundaries for bucket i:</p> + * <ul> + * <li>Upper bound (0 <= i < N-1): {@code scale * (base ^ i)}</li> + * <li>Lower bound (1 <= i < N): {@code scale * (base ^ (i - 1))}</li> + * </ul> + * + * <p>For example, if {@code bucketsCount=6}, {@code base=2}, and {@code scale=3}, + * then the bucket ranges are as follows:</p> + * + * <pre>(-INF, 3], (3, 6], (6, 12], (12, 24], (24, 48], (48, +INF)</pre> + * + * @param bucketsCount the total number of buckets. The value must be >= 2. + * @param base the exponential growth factor for the buckets width. + * The value must be >= 1.0. + * @param scale the linear scale for the buckets. The value must be >= 1.0. + */ + IHistogramCollectorPtr ExponentialHistogram( + ui32 bucketsCount, double base, double scale = 1.0); + + /** + * <p>Creates histogram collector for a sequence of buckets that all have + * the same width (except overflow and underflow).</p> + * + * <p>Defines {@code bucketsCount} buckets with these boundaries for bucket i:</p> + * <ul> + * <li>Upper bound (0 <= i < N-1): {@code startValue + bucketWidth * i}</li> + * <li>Lower bound (1 <= i < N): {@code startValue + bucketWidth * (i - 1)}</li> + * </ul> + * + * <p>For example, if {@code bucketsCount=6}, {@code startValue=5}, and + * {@code bucketWidth=15}, then the bucket ranges are as follows:</p> + * + * <pre>(-INF, 5], (5, 20], (20, 35], (35, 50], (50, 65], (65, +INF)</pre> + * + * @param bucketsCount the total number of buckets. The value must be >= 2. + * @param startValue the upper boundary of the first bucket. + * @param bucketWidth the difference between the upper and lower bounds for + * each bucket. The value must be >= 1. + */ + IHistogramCollectorPtr LinearHistogram( ui32 bucketsCount, TBucketBound startValue, TBucketBound bucketWidth); - + } // namespace NMonitoring diff --git a/library/cpp/monlib/metrics/histogram_collector_explicit.cpp b/library/cpp/monlib/metrics/histogram_collector_explicit.cpp index 90c41a235a5..377fc233eff 100644 --- a/library/cpp/monlib/metrics/histogram_collector_explicit.cpp +++ b/library/cpp/monlib/metrics/histogram_collector_explicit.cpp @@ -1,55 +1,55 @@ -#include "histogram_collector.h" -#include "atomics_array.h" - -#include <util/generic/algorithm.h> -#include <util/generic/vector.h> -#include <util/generic/yexception.h> -#include <util/generic/ylimits.h> - -namespace NMonitoring { - - /////////////////////////////////////////////////////////////////////////// - // TExplicitHistogramCollector - /////////////////////////////////////////////////////////////////////////// - class TExplicitHistogramCollector: public IHistogramCollector { - public: +#include "histogram_collector.h" +#include "atomics_array.h" + +#include <util/generic/algorithm.h> +#include <util/generic/vector.h> +#include <util/generic/yexception.h> +#include <util/generic/ylimits.h> + +namespace NMonitoring { + + /////////////////////////////////////////////////////////////////////////// + // TExplicitHistogramCollector + /////////////////////////////////////////////////////////////////////////// + class TExplicitHistogramCollector: public IHistogramCollector { + public: TExplicitHistogramCollector(TBucketBounds bounds) : Values_(bounds.size() + 1) , Bounds_(std::move(bounds)) - { - // add one bucket as +INF - Bounds_.push_back(Max<TBucketBound>()); - } - + { + // add one bucket as +INF + Bounds_.push_back(Max<TBucketBound>()); + } + void Collect(double value, ui32 count) override { auto it = LowerBound(Bounds_.begin(), Bounds_.end(), value); - auto index = std::distance(Bounds_.begin(), it); - Values_.Add(index, count); - } - + auto index = std::distance(Bounds_.begin(), it); + Values_.Add(index, count); + } + void Reset() override { Values_.Reset(); } - IHistogramSnapshotPtr Snapshot() const override { + IHistogramSnapshotPtr Snapshot() const override { auto values = Values_.Copy(); return ExplicitHistogramSnapshot(Bounds_, values); - } - - private: + } + + private: TAtomicsArray Values_; - TBucketBounds Bounds_; - }; - - IHistogramCollectorPtr ExplicitHistogram(TBucketBounds bounds) { - Y_ENSURE(bounds.size() >= 1, - "explicit histogram must contain at least one bucket"); + TBucketBounds Bounds_; + }; + + IHistogramCollectorPtr ExplicitHistogram(TBucketBounds bounds) { + Y_ENSURE(bounds.size() >= 1, + "explicit histogram must contain at least one bucket"); Y_ENSURE(bounds.size() <= HISTOGRAM_MAX_BUCKETS_COUNT, "buckets count must be <=" << HISTOGRAM_MAX_BUCKETS_COUNT - << ", but got: " << bounds.size()); - Y_ENSURE(IsSorted(bounds.begin(), bounds.end()), - "bounds for explicit histogram must be sorted"); - + << ", but got: " << bounds.size()); + Y_ENSURE(IsSorted(bounds.begin(), bounds.end()), + "bounds for explicit histogram must be sorted"); + return MakeHolder<TExplicitHistogramCollector>(bounds); - } -} + } +} diff --git a/library/cpp/monlib/metrics/histogram_collector_exponential.cpp b/library/cpp/monlib/metrics/histogram_collector_exponential.cpp index a6b929e8f3e..2f8a50a5f9d 100644 --- a/library/cpp/monlib/metrics/histogram_collector_exponential.cpp +++ b/library/cpp/monlib/metrics/histogram_collector_exponential.cpp @@ -1,68 +1,68 @@ -#include "histogram_collector.h" -#include "atomics_array.h" - -#include <util/generic/algorithm.h> -#include <util/generic/vector.h> -#include <util/generic/yexception.h> -#include <util/generic/ylimits.h> - -namespace NMonitoring { - /////////////////////////////////////////////////////////////////////////// - // TExponentialHistogramCollector - /////////////////////////////////////////////////////////////////////////// - class TExponentialHistogramCollector: public IHistogramCollector { - public: - TExponentialHistogramCollector(ui32 bucketsCount, double base, double scale) - : Values_(bucketsCount) - , Base_(base) - , Scale_(scale) +#include "histogram_collector.h" +#include "atomics_array.h" + +#include <util/generic/algorithm.h> +#include <util/generic/vector.h> +#include <util/generic/yexception.h> +#include <util/generic/ylimits.h> + +namespace NMonitoring { + /////////////////////////////////////////////////////////////////////////// + // TExponentialHistogramCollector + /////////////////////////////////////////////////////////////////////////// + class TExponentialHistogramCollector: public IHistogramCollector { + public: + TExponentialHistogramCollector(ui32 bucketsCount, double base, double scale) + : Values_(bucketsCount) + , Base_(base) + , Scale_(scale) , MinValue_(scale) , MaxValue_(scale * std::pow(base, bucketsCount - 2)) - , LogOfBase_(std::log(base)) - { - } - + , LogOfBase_(std::log(base)) + { + } + void Collect(double value, ui32 count) override { - ui32 index = Max<ui32>(); - if (value <= MinValue_) { - index = 0; - } else if (value > MaxValue_) { - index = Values_.Size() - 1; - } else { - double logBase = std::log(value / Scale_) / LogOfBase_; - index = static_cast<ui32>(std::ceil(logBase)); - } - Values_.Add(index, count); - } - + ui32 index = Max<ui32>(); + if (value <= MinValue_) { + index = 0; + } else if (value > MaxValue_) { + index = Values_.Size() - 1; + } else { + double logBase = std::log(value / Scale_) / LogOfBase_; + index = static_cast<ui32>(std::ceil(logBase)); + } + Values_.Add(index, count); + } + void Reset() override { Values_.Reset(); } - IHistogramSnapshotPtr Snapshot() const override { - return new TExponentialHistogramSnapshot(Base_, Scale_, Values_.Copy()); - } - - private: - TAtomicsArray Values_; - double Base_; - double Scale_; + IHistogramSnapshotPtr Snapshot() const override { + return new TExponentialHistogramSnapshot(Base_, Scale_, Values_.Copy()); + } + + private: + TAtomicsArray Values_; + double Base_; + double Scale_; TBucketBound MinValue_; TBucketBound MaxValue_; - double LogOfBase_; - }; - - IHistogramCollectorPtr ExponentialHistogram( - ui32 bucketsCount, double base, double scale) - { - Y_ENSURE(bucketsCount >= 2, - "exponential histogram must contain at least two buckets"); + double LogOfBase_; + }; + + IHistogramCollectorPtr ExponentialHistogram( + ui32 bucketsCount, double base, double scale) + { + Y_ENSURE(bucketsCount >= 2, + "exponential histogram must contain at least two buckets"); Y_ENSURE(bucketsCount <= HISTOGRAM_MAX_BUCKETS_COUNT, "buckets count must be <=" << HISTOGRAM_MAX_BUCKETS_COUNT - << ", but got: " << bucketsCount); - Y_ENSURE(base > 1.0, "base must be > 1.0, got: " << base); - Y_ENSURE(scale >= 1.0, "scale must be >= 1.0, got: " << scale); - - return MakeHolder<TExponentialHistogramCollector>(bucketsCount, base, scale); - } -} + << ", but got: " << bucketsCount); + Y_ENSURE(base > 1.0, "base must be > 1.0, got: " << base); + Y_ENSURE(scale >= 1.0, "scale must be >= 1.0, got: " << scale); + + return MakeHolder<TExponentialHistogramCollector>(bucketsCount, base, scale); + } +} diff --git a/library/cpp/monlib/metrics/histogram_collector_linear.cpp b/library/cpp/monlib/metrics/histogram_collector_linear.cpp index c0afa292418..f8ad86f3a48 100644 --- a/library/cpp/monlib/metrics/histogram_collector_linear.cpp +++ b/library/cpp/monlib/metrics/histogram_collector_linear.cpp @@ -1,67 +1,67 @@ -#include "histogram_collector.h" -#include "atomics_array.h" - -#include <util/generic/algorithm.h> -#include <util/generic/vector.h> -#include <util/generic/yexception.h> -#include <util/generic/ylimits.h> - -#include <cmath> - -namespace NMonitoring { - /////////////////////////////////////////////////////////////////////////// - // TLinearHistogramCollector - /////////////////////////////////////////////////////////////////////////// - class TLinearHistogramCollector: public IHistogramCollector { - public: - TLinearHistogramCollector( +#include "histogram_collector.h" +#include "atomics_array.h" + +#include <util/generic/algorithm.h> +#include <util/generic/vector.h> +#include <util/generic/yexception.h> +#include <util/generic/ylimits.h> + +#include <cmath> + +namespace NMonitoring { + /////////////////////////////////////////////////////////////////////////// + // TLinearHistogramCollector + /////////////////////////////////////////////////////////////////////////// + class TLinearHistogramCollector: public IHistogramCollector { + public: + TLinearHistogramCollector( ui32 bucketsCount, TBucketBound startValue, TBucketBound bucketWidth) - : Values_(bucketsCount) - , StartValue_(startValue) - , BucketWidth_(bucketWidth) - , MaxValue_(startValue + bucketWidth * (bucketsCount - 2)) - { - } - + : Values_(bucketsCount) + , StartValue_(startValue) + , BucketWidth_(bucketWidth) + , MaxValue_(startValue + bucketWidth * (bucketsCount - 2)) + { + } + void Collect(double value, ui32 count) override { - ui32 index = Max<ui32>(); - if (value <= StartValue_) { - index = 0; - } else if (value > MaxValue_) { - index = Values_.Size() - 1; - } else { + ui32 index = Max<ui32>(); + if (value <= StartValue_) { + index = 0; + } else if (value > MaxValue_) { + index = Values_.Size() - 1; + } else { double buckets = (value - StartValue_) / BucketWidth_; - index = static_cast<ui32>(std::ceil(buckets)); - } - Values_.Add(index, count); - } - + index = static_cast<ui32>(std::ceil(buckets)); + } + Values_.Add(index, count); + } + void Reset() override { Values_.Reset(); } - IHistogramSnapshotPtr Snapshot() const override { - return new TLinearHistogramSnapshot( - StartValue_, BucketWidth_, Values_.Copy()); - } - - private: - TAtomicsArray Values_; + IHistogramSnapshotPtr Snapshot() const override { + return new TLinearHistogramSnapshot( + StartValue_, BucketWidth_, Values_.Copy()); + } + + private: + TAtomicsArray Values_; TBucketBound StartValue_; double BucketWidth_; TBucketBound MaxValue_; - }; - - IHistogramCollectorPtr LinearHistogram( + }; + + IHistogramCollectorPtr LinearHistogram( ui32 bucketsCount, TBucketBound startValue, TBucketBound bucketWidth) - { - Y_ENSURE(bucketsCount >= 2, - "linear histogram must contain at least two buckets"); + { + Y_ENSURE(bucketsCount >= 2, + "linear histogram must contain at least two buckets"); Y_ENSURE(bucketsCount <= HISTOGRAM_MAX_BUCKETS_COUNT, "buckets count must be <=" << HISTOGRAM_MAX_BUCKETS_COUNT - << ", but got: " << bucketsCount); - Y_ENSURE(bucketWidth >= 1, "bucketWidth must be >= 1, got: " << bucketWidth); - + << ", but got: " << bucketsCount); + Y_ENSURE(bucketWidth >= 1, "bucketWidth must be >= 1, got: " << bucketWidth); + return MakeHolder<TLinearHistogramCollector>(bucketsCount, startValue, bucketWidth); - } -} + } +} diff --git a/library/cpp/monlib/metrics/histogram_collector_ut.cpp b/library/cpp/monlib/metrics/histogram_collector_ut.cpp index bfbc1a5f2e9..1cf66507fa2 100644 --- a/library/cpp/monlib/metrics/histogram_collector_ut.cpp +++ b/library/cpp/monlib/metrics/histogram_collector_ut.cpp @@ -1,114 +1,114 @@ -#include "histogram_collector.h" - +#include "histogram_collector.h" + #include <library/cpp/testing/unittest/registar.h> - -using namespace NMonitoring; - -Y_UNIT_TEST_SUITE(THistogramCollectorTest) { - void CheckSnapshot( - const IHistogramSnapshot& s, - const TBucketBounds& bounds, - const TBucketValues& values) { - UNIT_ASSERT_VALUES_EQUAL(bounds.size(), values.size()); - UNIT_ASSERT_VALUES_EQUAL(bounds.size(), s.Count()); - - double epsilon = std::numeric_limits<double>::epsilon(); - for (ui32 i = 0; i < s.Count(); i++) { - UNIT_ASSERT_DOUBLES_EQUAL(bounds[i], s.UpperBound(i), epsilon); - UNIT_ASSERT_VALUES_EQUAL(values[i], s.Value(i)); - } - } - - Y_UNIT_TEST(Explicit) { - auto histogram = ExplicitHistogram({0, 1, 2, 5, 10, 20}); - histogram->Collect(-2); - histogram->Collect(-1); - histogram->Collect(0); - histogram->Collect(1); - histogram->Collect(20); - histogram->Collect(21); - histogram->Collect(1000); - - TBucketBounds expectedBounds = {0, 1, 2, 5, 10, 20, Max<TBucketBound>()}; - TBucketValues expectedValues = {3, 1, 0, 0, 0, 1, 2}; - - CheckSnapshot(*histogram->Snapshot(), expectedBounds, expectedValues); - } - - Y_UNIT_TEST(ExplicitWithFloadBounds) { - auto histogram = ExplicitHistogram({0.1, 0.5, 1, 1.7, 10}); - histogram->Collect(0.3, 2); - histogram->Collect(0.01); - histogram->Collect(0.9); - histogram->Collect(0.6); - histogram->Collect(1.1); - histogram->Collect(0.7); - histogram->Collect(2.71); - - TBucketBounds expectedBounds = {0.1, 0.5, 1, 1.7, 10, Max<TBucketBound>()}; - TBucketValues expectedValues = {1, 2, 3, 1, 1, 0}; - - CheckSnapshot(*histogram->Snapshot(), expectedBounds, expectedValues); - } - - Y_UNIT_TEST(Exponential) { - auto histogram = ExponentialHistogram(6, 2.0, 3.0); - histogram->Collect(-1); - histogram->Collect(0); - histogram->Collect(1); - histogram->Collect(3); - histogram->Collect(4); - histogram->Collect(5); - histogram->Collect(22); - histogram->Collect(23); - histogram->Collect(24); - histogram->Collect(50); - histogram->Collect(100); - histogram->Collect(1000); - - TBucketBounds expectedBounds = {3, 6, 12, 24, 48, Max<TBucketBound>()}; - TBucketValues expectedValues = {4, 2, 0, 3, 0, 3}; - - CheckSnapshot(*histogram->Snapshot(), expectedBounds, expectedValues); - } - - Y_UNIT_TEST(Linear) { - auto histogram = LinearHistogram(6, 5, 15); - histogram->Collect(-1); - histogram->Collect(0); - histogram->Collect(1); - histogram->Collect(4); - histogram->Collect(5); - histogram->Collect(6); - histogram->Collect(64); - histogram->Collect(65); - histogram->Collect(66); - histogram->Collect(100); - histogram->Collect(1000); - - TBucketBounds expectedBounds = {5, 20, 35, 50, 65, Max<TBucketBound>()}; - TBucketValues expectedValues = {5, 1, 0, 0, 2, 3}; - - CheckSnapshot(*histogram->Snapshot(), expectedBounds, expectedValues); - } - - Y_UNIT_TEST(SnapshotOutput) { - auto histogram = ExplicitHistogram({0, 1, 2, 5, 10, 20}); - histogram->Collect(-2); - histogram->Collect(-1); - histogram->Collect(0); - histogram->Collect(1); - histogram->Collect(20); - histogram->Collect(21); - histogram->Collect(1000); - - auto snapshot = histogram->Snapshot(); - - TStringStream ss; - ss << *snapshot; - - UNIT_ASSERT_STRINGS_EQUAL( - "{0: 3, 1: 1, 2: 0, 5: 0, 10: 0, 20: 1, inf: 2}", - ss.Str()); - } -} + +using namespace NMonitoring; + +Y_UNIT_TEST_SUITE(THistogramCollectorTest) { + void CheckSnapshot( + const IHistogramSnapshot& s, + const TBucketBounds& bounds, + const TBucketValues& values) { + UNIT_ASSERT_VALUES_EQUAL(bounds.size(), values.size()); + UNIT_ASSERT_VALUES_EQUAL(bounds.size(), s.Count()); + + double epsilon = std::numeric_limits<double>::epsilon(); + for (ui32 i = 0; i < s.Count(); i++) { + UNIT_ASSERT_DOUBLES_EQUAL(bounds[i], s.UpperBound(i), epsilon); + UNIT_ASSERT_VALUES_EQUAL(values[i], s.Value(i)); + } + } + + Y_UNIT_TEST(Explicit) { + auto histogram = ExplicitHistogram({0, 1, 2, 5, 10, 20}); + histogram->Collect(-2); + histogram->Collect(-1); + histogram->Collect(0); + histogram->Collect(1); + histogram->Collect(20); + histogram->Collect(21); + histogram->Collect(1000); + + TBucketBounds expectedBounds = {0, 1, 2, 5, 10, 20, Max<TBucketBound>()}; + TBucketValues expectedValues = {3, 1, 0, 0, 0, 1, 2}; + + CheckSnapshot(*histogram->Snapshot(), expectedBounds, expectedValues); + } + + Y_UNIT_TEST(ExplicitWithFloadBounds) { + auto histogram = ExplicitHistogram({0.1, 0.5, 1, 1.7, 10}); + histogram->Collect(0.3, 2); + histogram->Collect(0.01); + histogram->Collect(0.9); + histogram->Collect(0.6); + histogram->Collect(1.1); + histogram->Collect(0.7); + histogram->Collect(2.71); + + TBucketBounds expectedBounds = {0.1, 0.5, 1, 1.7, 10, Max<TBucketBound>()}; + TBucketValues expectedValues = {1, 2, 3, 1, 1, 0}; + + CheckSnapshot(*histogram->Snapshot(), expectedBounds, expectedValues); + } + + Y_UNIT_TEST(Exponential) { + auto histogram = ExponentialHistogram(6, 2.0, 3.0); + histogram->Collect(-1); + histogram->Collect(0); + histogram->Collect(1); + histogram->Collect(3); + histogram->Collect(4); + histogram->Collect(5); + histogram->Collect(22); + histogram->Collect(23); + histogram->Collect(24); + histogram->Collect(50); + histogram->Collect(100); + histogram->Collect(1000); + + TBucketBounds expectedBounds = {3, 6, 12, 24, 48, Max<TBucketBound>()}; + TBucketValues expectedValues = {4, 2, 0, 3, 0, 3}; + + CheckSnapshot(*histogram->Snapshot(), expectedBounds, expectedValues); + } + + Y_UNIT_TEST(Linear) { + auto histogram = LinearHistogram(6, 5, 15); + histogram->Collect(-1); + histogram->Collect(0); + histogram->Collect(1); + histogram->Collect(4); + histogram->Collect(5); + histogram->Collect(6); + histogram->Collect(64); + histogram->Collect(65); + histogram->Collect(66); + histogram->Collect(100); + histogram->Collect(1000); + + TBucketBounds expectedBounds = {5, 20, 35, 50, 65, Max<TBucketBound>()}; + TBucketValues expectedValues = {5, 1, 0, 0, 2, 3}; + + CheckSnapshot(*histogram->Snapshot(), expectedBounds, expectedValues); + } + + Y_UNIT_TEST(SnapshotOutput) { + auto histogram = ExplicitHistogram({0, 1, 2, 5, 10, 20}); + histogram->Collect(-2); + histogram->Collect(-1); + histogram->Collect(0); + histogram->Collect(1); + histogram->Collect(20); + histogram->Collect(21); + histogram->Collect(1000); + + auto snapshot = histogram->Snapshot(); + + TStringStream ss; + ss << *snapshot; + + UNIT_ASSERT_STRINGS_EQUAL( + "{0: 3, 1: 1, 2: 0, 5: 0, 10: 0, 20: 1, inf: 2}", + ss.Str()); + } +} diff --git a/library/cpp/monlib/metrics/histogram_snapshot.cpp b/library/cpp/monlib/metrics/histogram_snapshot.cpp index df787bedaff..75b58115464 100644 --- a/library/cpp/monlib/metrics/histogram_snapshot.cpp +++ b/library/cpp/monlib/metrics/histogram_snapshot.cpp @@ -1,27 +1,27 @@ -#include "histogram_snapshot.h" - +#include "histogram_snapshot.h" + #include <util/stream/output.h> #include <iostream> -namespace NMonitoring { - +namespace NMonitoring { + IHistogramSnapshotPtr ExplicitHistogramSnapshot(TConstArrayRef<TBucketBound> bounds, TConstArrayRef<TBucketValue> values) { Y_ENSURE(bounds.size() == values.size(), "mismatched sizes: bounds(" << bounds.size() << ") != buckets(" << values.size() << ')'); - + auto snapshot = TExplicitHistogramSnapshot::New(bounds.size()); - + for (size_t i = 0; i != bounds.size(); ++i) { (*snapshot)[i].first = bounds[i]; (*snapshot)[i].second = values[i]; } - + return snapshot; - } - + } + } // namespace NMonitoring namespace { diff --git a/library/cpp/monlib/metrics/histogram_snapshot.h b/library/cpp/monlib/metrics/histogram_snapshot.h index 1f092a60d54..e8acf6ac2b0 100644 --- a/library/cpp/monlib/metrics/histogram_snapshot.h +++ b/library/cpp/monlib/metrics/histogram_snapshot.h @@ -1,50 +1,50 @@ -#pragma once - +#pragma once + #include <util/generic/array_ref.h> -#include <util/generic/ptr.h> -#include <util/generic/vector.h> +#include <util/generic/ptr.h> +#include <util/generic/vector.h> #include <util/generic/yexception.h> - + #include <cmath> -#include <limits> - - -namespace NMonitoring { - - using TBucketBound = double; - using TBucketValue = ui64; - - using TBucketBounds = TVector<TBucketBound>; - using TBucketValues = TVector<TBucketValue>; - - constexpr ui32 HISTOGRAM_MAX_BUCKETS_COUNT = 51; - constexpr TBucketBound HISTOGRAM_INF_BOUND = std::numeric_limits<TBucketBound>::max(); - - /////////////////////////////////////////////////////////////////////////// - // IHistogramSnapshot - /////////////////////////////////////////////////////////////////////////// - class IHistogramSnapshot: public TAtomicRefCount<IHistogramSnapshot> { - public: - virtual ~IHistogramSnapshot() = default; - - /** - * @return buckets count. - */ - virtual ui32 Count() const = 0; - - /** - * @return upper bound for the bucket with particular index. - */ - virtual TBucketBound UpperBound(ui32 index) const = 0; - - /** - * @return value stored in the bucket with particular index. - */ - virtual TBucketValue Value(ui32 index) const = 0; - }; - - using IHistogramSnapshotPtr = TIntrusivePtr<IHistogramSnapshot>; - +#include <limits> + + +namespace NMonitoring { + + using TBucketBound = double; + using TBucketValue = ui64; + + using TBucketBounds = TVector<TBucketBound>; + using TBucketValues = TVector<TBucketValue>; + + constexpr ui32 HISTOGRAM_MAX_BUCKETS_COUNT = 51; + constexpr TBucketBound HISTOGRAM_INF_BOUND = std::numeric_limits<TBucketBound>::max(); + + /////////////////////////////////////////////////////////////////////////// + // IHistogramSnapshot + /////////////////////////////////////////////////////////////////////////// + class IHistogramSnapshot: public TAtomicRefCount<IHistogramSnapshot> { + public: + virtual ~IHistogramSnapshot() = default; + + /** + * @return buckets count. + */ + virtual ui32 Count() const = 0; + + /** + * @return upper bound for the bucket with particular index. + */ + virtual TBucketBound UpperBound(ui32 index) const = 0; + + /** + * @return value stored in the bucket with particular index. + */ + virtual TBucketValue Value(ui32 index) const = 0; + }; + + using IHistogramSnapshotPtr = TIntrusivePtr<IHistogramSnapshot>; + /////////////////////////////////////////////////////////////////////////////// // TLinearHistogramSnapshot /////////////////////////////////////////////////////////////////////////////// @@ -85,9 +85,9 @@ namespace NMonitoring { TBucketValues Values_; }; - /////////////////////////////////////////////////////////////////////////// + /////////////////////////////////////////////////////////////////////////// // TExponentialHistogramSnapshot - /////////////////////////////////////////////////////////////////////////// + /////////////////////////////////////////////////////////////////////////// class TExponentialHistogramSnapshot: public IHistogramSnapshot { public: TExponentialHistogramSnapshot( diff --git a/library/cpp/monlib/metrics/labels.cpp b/library/cpp/monlib/metrics/labels.cpp index 226bf5df266..1eaadb7cba0 100644 --- a/library/cpp/monlib/metrics/labels.cpp +++ b/library/cpp/monlib/metrics/labels.cpp @@ -1,31 +1,31 @@ -#include "labels.h" - +#include "labels.h" + #include <util/stream/output.h> #include <util/string/split.h> - -static void OutputLabels(IOutputStream& out, const NMonitoring::ILabels& labels) { - size_t i = 0; + +static void OutputLabels(IOutputStream& out, const NMonitoring::ILabels& labels) { + size_t i = 0; out << '{'; - for (const auto& label: labels) { - if (i++ > 0) { + for (const auto& label: labels) { + if (i++ > 0) { out << TStringBuf(", "); } - out << label; - } + out << label; + } out << '}'; } template <> -void Out<NMonitoring::ILabelsPtr>(IOutputStream& out, const NMonitoring::ILabelsPtr& labels) { - OutputLabels(out, *labels); -} - -template <> -void Out<NMonitoring::ILabels>(IOutputStream& out, const NMonitoring::ILabels& labels) { - OutputLabels(out, labels); -} - -template <> +void Out<NMonitoring::ILabelsPtr>(IOutputStream& out, const NMonitoring::ILabelsPtr& labels) { + OutputLabels(out, *labels); +} + +template <> +void Out<NMonitoring::ILabels>(IOutputStream& out, const NMonitoring::ILabels& labels) { + OutputLabels(out, labels); +} + +template <> void Out<NMonitoring::ILabel>(IOutputStream& out, const NMonitoring::ILabel& labels) { out << labels.Name() << "=" << labels.Value(); } diff --git a/library/cpp/monlib/metrics/labels.h b/library/cpp/monlib/metrics/labels.h index 5afb7e332ec..63dc997c280 100644 --- a/library/cpp/monlib/metrics/labels.h +++ b/library/cpp/monlib/metrics/labels.h @@ -1,19 +1,19 @@ -#pragma once - -#include <util/digest/multi.h> -#include <util/digest/sequence.h> -#include <util/generic/algorithm.h> -#include <util/generic/maybe.h> -#include <util/generic/string.h> -#include <util/generic/vector.h> +#pragma once + +#include <util/digest/multi.h> +#include <util/digest/sequence.h> +#include <util/generic/algorithm.h> +#include <util/generic/maybe.h> +#include <util/generic/string.h> +#include <util/generic/vector.h> #include <util/stream/output.h> #include <util/string/builder.h> #include <util/string/strip.h> - + #include <optional> #include <type_traits> -namespace NMonitoring { +namespace NMonitoring { struct ILabel { virtual ~ILabel() = default; @@ -21,38 +21,38 @@ namespace NMonitoring { virtual TStringBuf Value() const noexcept = 0; }; - /////////////////////////////////////////////////////////////////////////// - // TLabel - /////////////////////////////////////////////////////////////////////////// + /////////////////////////////////////////////////////////////////////////// + // TLabel + /////////////////////////////////////////////////////////////////////////// template <typename TStringBackend> class TLabelImpl: public ILabel { - public: + public: using TStringType = TStringBackend; TLabelImpl() = default; - + inline TLabelImpl(TStringBuf name, TStringBuf value) : Name_{name} , Value_{value} - { - } - + { + } + inline bool operator==(const TLabelImpl& rhs) const noexcept { - return Name_ == rhs.Name_ && Value_ == rhs.Value_; - } - + return Name_ == rhs.Name_ && Value_ == rhs.Value_; + } + inline bool operator!=(const TLabelImpl& rhs) const noexcept { - return !(*this == rhs); - } - + return !(*this == rhs); + } + inline TStringBuf Name() const noexcept { - return Name_; - } - + return Name_; + } + inline TStringBuf Value() const noexcept { - return Value_; - } - + return Value_; + } + inline const TStringBackend& NameStr() const { return Name_; } @@ -61,18 +61,18 @@ namespace NMonitoring { return Value_; } - inline size_t Hash() const noexcept { - return MultiHash(Name_, Value_); - } - + inline size_t Hash() const noexcept { + return MultiHash(Name_, Value_); + } + TStringBackend ToString() const { TStringBackend buf = Name_; buf += '='; buf += Value_; - + return buf; } - + static TLabelImpl FromString(TStringBuf str) { TStringBuf name, value; Y_ENSURE(str.TrySplit('=', name, value), @@ -107,11 +107,11 @@ namespace NMonitoring { return true; } - private: + private: TStringBackend Name_; TStringBackend Value_; - }; - + }; + using TLabel = TLabelImpl<TString>; struct ILabels { @@ -158,12 +158,12 @@ namespace NMonitoring { virtual ~ILabels() = default; - virtual bool Add(TStringBuf name, TStringBuf value) noexcept = 0; + virtual bool Add(TStringBuf name, TStringBuf value) noexcept = 0; virtual bool Add(const ILabel& label) noexcept { return Add(label.Name(), label.Value()); } - virtual bool Has(TStringBuf name) const noexcept = 0; + virtual bool Has(TStringBuf name) const noexcept = 0; virtual size_t Size() const noexcept = 0; virtual bool Empty() const noexcept = 0; @@ -171,7 +171,7 @@ namespace NMonitoring { virtual size_t Hash() const noexcept = 0; - virtual std::optional<const ILabel*> Get(TStringBuf name) const = 0; + virtual std::optional<const ILabel*> Get(TStringBuf name) const = 0; // NB: there's no guarantee that indices are preserved after any object modification virtual const ILabel* Get(size_t idx) const = 0; @@ -188,16 +188,16 @@ namespace NMonitoring { bool TryLoadLabelsFromString(TStringBuf sb, ILabels& labels); bool TryLoadLabelsFromString(IInputStream& is, ILabels& labels); - /////////////////////////////////////////////////////////////////////////// - // TLabels - /////////////////////////////////////////////////////////////////////////// + /////////////////////////////////////////////////////////////////////////// + // TLabels + /////////////////////////////////////////////////////////////////////////// template <typename TStringBackend> class TLabelsImpl: public ILabels { - public: + public: using value_type = TLabelImpl<TStringBackend>; - + TLabelsImpl() = default; - + explicit TLabelsImpl(::NDetail::TReserveTag rt) : Labels_(std::move(rt)) {} @@ -222,13 +222,13 @@ namespace NMonitoring { inline bool operator==(const TLabelsImpl& rhs) const { return Labels_ == rhs.Labels_; - } - + } + inline bool operator!=(const TLabelsImpl& rhs) const { return Labels_ != rhs.Labels_; - } - - bool Add(TStringBuf name, TStringBuf value) noexcept override { + } + + bool Add(TStringBuf name, TStringBuf value) noexcept override { if (Has(name)) { return false; } @@ -239,7 +239,7 @@ namespace NMonitoring { using ILabels::Add; - bool Has(TStringBuf name) const noexcept override { + bool Has(TStringBuf name) const noexcept override { auto it = FindIf(Labels_, [name](const TLabelImpl<TStringBackend>& label) { return name == TStringBuf{label.Name()}; }); @@ -264,7 +264,7 @@ namespace NMonitoring { return *it; } - std::optional<const ILabel*> Get(TStringBuf name) const override { + std::optional<const ILabel*> Get(TStringBuf name) const override { auto it = FindIf(Labels_, [name] (auto&& l) { return name == l.Name(); }); @@ -300,20 +300,20 @@ namespace NMonitoring { inline size_t Hash() const noexcept override { return TSimpleRangeHash()(Labels_); - } - - inline TLabel* Data() const noexcept { + } + + inline TLabel* Data() const noexcept { return const_cast<TLabel*>(Labels_.data()); - } - + } + inline size_t Size() const noexcept override { return Labels_.size(); - } - + } + inline bool Empty() const noexcept override { return Labels_.empty(); - } - + } + inline void Clear() noexcept override { Labels_.clear(); }; @@ -391,26 +391,26 @@ namespace NMonitoring { private: TVector<TLabelImpl<TStringBackend>> Labels_; - }; + }; using TLabels = TLabelsImpl<TString>; using ILabelsPtr = THolder<ILabels>; - - template <typename T> - ILabelsPtr MakeLabels() { - return MakeHolder<TLabelsImpl<T>>(); - } - - template <typename T> - ILabelsPtr MakeLabels(std::initializer_list<TLabelImpl<T>> labels) { - return MakeHolder<TLabelsImpl<T>>(labels); - } - - inline ILabelsPtr MakeLabels(TLabels&& labels) { - return MakeHolder<TLabels>(std::move(labels)); - } -} - + + template <typename T> + ILabelsPtr MakeLabels() { + return MakeHolder<TLabelsImpl<T>>(); + } + + template <typename T> + ILabelsPtr MakeLabels(std::initializer_list<TLabelImpl<T>> labels) { + return MakeHolder<TLabelsImpl<T>>(labels); + } + + inline ILabelsPtr MakeLabels(TLabels&& labels) { + return MakeHolder<TLabels>(std::move(labels)); + } +} + template<> struct THash<NMonitoring::ILabelsPtr> { size_t operator()(const NMonitoring::ILabelsPtr& labels) const noexcept { @@ -432,14 +432,14 @@ struct THash<NMonitoring::TLabelsImpl<TStringBackend>> { template <typename TStringBackend> struct THash<NMonitoring::TLabelImpl<TStringBackend>> { inline size_t operator()(const NMonitoring::TLabelImpl<TStringBackend>& label) const noexcept { - return label.Hash(); - } -}; - + return label.Hash(); + } +}; + inline bool operator==(const NMonitoring::ILabels& lhs, const NMonitoring::ILabels& rhs) { if (lhs.Size() != rhs.Size()) { return false; - } + } for (auto&& l : lhs) { auto rl = rhs.Get(l.Name()); @@ -468,7 +468,7 @@ struct TEqualTo<NMonitoring::ILabelsPtr> { bool operator()(const NMonitoring::ILabels& lhs, const NMonitoring::ILabelsPtr& rhs) { return lhs == *rhs; } -}; +}; #define Y_MONLIB_DEFINE_LABELS_OUT(T) \ template <> \ diff --git a/library/cpp/monlib/metrics/labels_ut.cpp b/library/cpp/monlib/metrics/labels_ut.cpp index 8c96c23bfb6..f0e4f532aba 100644 --- a/library/cpp/monlib/metrics/labels_ut.cpp +++ b/library/cpp/monlib/metrics/labels_ut.cpp @@ -1,194 +1,194 @@ -#include "labels.h" - +#include "labels.h" + #include <library/cpp/testing/unittest/registar.h> - -using namespace NMonitoring; - + +using namespace NMonitoring; + Y_UNIT_TEST_SUITE(TLabelsTest) { - TLabel pSolomon("project", "solomon"); - TLabel pKikimr("project", "kikimr"); - + TLabel pSolomon("project", "solomon"); + TLabel pKikimr("project", "kikimr"); + Y_UNIT_TEST(Equals) { - UNIT_ASSERT(pSolomon == TLabel("project", "solomon")); - - UNIT_ASSERT_STRINGS_EQUAL(pSolomon.Name(), "project"); - UNIT_ASSERT_STRINGS_EQUAL(pSolomon.Value(), "solomon"); - - UNIT_ASSERT(pSolomon != pKikimr); - } - + UNIT_ASSERT(pSolomon == TLabel("project", "solomon")); + + UNIT_ASSERT_STRINGS_EQUAL(pSolomon.Name(), "project"); + UNIT_ASSERT_STRINGS_EQUAL(pSolomon.Value(), "solomon"); + + UNIT_ASSERT(pSolomon != pKikimr); + } + Y_UNIT_TEST(ToString) { - UNIT_ASSERT_STRINGS_EQUAL(pSolomon.ToString(), "project=solomon"); - UNIT_ASSERT_STRINGS_EQUAL(pKikimr.ToString(), "project=kikimr"); - } - + UNIT_ASSERT_STRINGS_EQUAL(pSolomon.ToString(), "project=solomon"); + UNIT_ASSERT_STRINGS_EQUAL(pKikimr.ToString(), "project=kikimr"); + } + Y_UNIT_TEST(FromString) { - auto pYql = TLabel::FromString("project=yql"); - UNIT_ASSERT_EQUAL(pYql, TLabel("project", "yql")); - - UNIT_ASSERT_EQUAL(TLabel::FromString("k=v"), TLabel("k", "v")); - UNIT_ASSERT_EQUAL(TLabel::FromString("k=v "), TLabel("k", "v")); - UNIT_ASSERT_EQUAL(TLabel::FromString("k= v"), TLabel("k", "v")); - UNIT_ASSERT_EQUAL(TLabel::FromString("k =v"), TLabel("k", "v")); - UNIT_ASSERT_EQUAL(TLabel::FromString(" k=v"), TLabel("k", "v")); - UNIT_ASSERT_EQUAL(TLabel::FromString(" k = v "), TLabel("k", "v")); - - UNIT_ASSERT_EXCEPTION_CONTAINS( - TLabel::FromString(""), - yexception, - "invalid label string format"); - - UNIT_ASSERT_EXCEPTION_CONTAINS( - TLabel::FromString("k v"), - yexception, - "invalid label string format"); - - UNIT_ASSERT_EXCEPTION_CONTAINS( - TLabel::FromString(" =v"), - yexception, - "label name cannot be empty"); - - UNIT_ASSERT_EXCEPTION_CONTAINS( - TLabel::FromString("k= "), - yexception, - "label value cannot be empty"); - } - + auto pYql = TLabel::FromString("project=yql"); + UNIT_ASSERT_EQUAL(pYql, TLabel("project", "yql")); + + UNIT_ASSERT_EQUAL(TLabel::FromString("k=v"), TLabel("k", "v")); + UNIT_ASSERT_EQUAL(TLabel::FromString("k=v "), TLabel("k", "v")); + UNIT_ASSERT_EQUAL(TLabel::FromString("k= v"), TLabel("k", "v")); + UNIT_ASSERT_EQUAL(TLabel::FromString("k =v"), TLabel("k", "v")); + UNIT_ASSERT_EQUAL(TLabel::FromString(" k=v"), TLabel("k", "v")); + UNIT_ASSERT_EQUAL(TLabel::FromString(" k = v "), TLabel("k", "v")); + + UNIT_ASSERT_EXCEPTION_CONTAINS( + TLabel::FromString(""), + yexception, + "invalid label string format"); + + UNIT_ASSERT_EXCEPTION_CONTAINS( + TLabel::FromString("k v"), + yexception, + "invalid label string format"); + + UNIT_ASSERT_EXCEPTION_CONTAINS( + TLabel::FromString(" =v"), + yexception, + "label name cannot be empty"); + + UNIT_ASSERT_EXCEPTION_CONTAINS( + TLabel::FromString("k= "), + yexception, + "label value cannot be empty"); + } + Y_UNIT_TEST(TryFromString) { - TLabel pYql; - UNIT_ASSERT(TLabel::TryFromString("project=yql", pYql)); - UNIT_ASSERT_EQUAL(pYql, TLabel("project", "yql")); - - { - TLabel label; - UNIT_ASSERT(TLabel::TryFromString("k=v", label)); - UNIT_ASSERT_EQUAL(label, TLabel("k", "v")); - } - { - TLabel label; - UNIT_ASSERT(TLabel::TryFromString("k=v ", label)); - UNIT_ASSERT_EQUAL(label, TLabel("k", "v")); - } - { - TLabel label; - UNIT_ASSERT(TLabel::TryFromString("k= v", label)); - UNIT_ASSERT_EQUAL(label, TLabel("k", "v")); - } - { - TLabel label; - UNIT_ASSERT(TLabel::TryFromString("k =v", label)); - UNIT_ASSERT_EQUAL(label, TLabel("k", "v")); - } - { - TLabel label; - UNIT_ASSERT(TLabel::TryFromString(" k=v", label)); - UNIT_ASSERT_EQUAL(label, TLabel("k", "v")); - } - { - TLabel label; - UNIT_ASSERT(TLabel::TryFromString(" k = v ", label)); - UNIT_ASSERT_EQUAL(label, TLabel("k", "v")); - } - } - + TLabel pYql; + UNIT_ASSERT(TLabel::TryFromString("project=yql", pYql)); + UNIT_ASSERT_EQUAL(pYql, TLabel("project", "yql")); + + { + TLabel label; + UNIT_ASSERT(TLabel::TryFromString("k=v", label)); + UNIT_ASSERT_EQUAL(label, TLabel("k", "v")); + } + { + TLabel label; + UNIT_ASSERT(TLabel::TryFromString("k=v ", label)); + UNIT_ASSERT_EQUAL(label, TLabel("k", "v")); + } + { + TLabel label; + UNIT_ASSERT(TLabel::TryFromString("k= v", label)); + UNIT_ASSERT_EQUAL(label, TLabel("k", "v")); + } + { + TLabel label; + UNIT_ASSERT(TLabel::TryFromString("k =v", label)); + UNIT_ASSERT_EQUAL(label, TLabel("k", "v")); + } + { + TLabel label; + UNIT_ASSERT(TLabel::TryFromString(" k=v", label)); + UNIT_ASSERT_EQUAL(label, TLabel("k", "v")); + } + { + TLabel label; + UNIT_ASSERT(TLabel::TryFromString(" k = v ", label)); + UNIT_ASSERT_EQUAL(label, TLabel("k", "v")); + } + } + Y_UNIT_TEST(Labels) { - TLabels labels; + TLabels labels; UNIT_ASSERT(labels.Add(TStringBuf("name1"), TStringBuf("value1"))); - UNIT_ASSERT(labels.Size() == 1); + UNIT_ASSERT(labels.Size() == 1); UNIT_ASSERT(labels.Has(TStringBuf("name1"))); - { - auto l = labels.Find("name1"); - UNIT_ASSERT(l.Defined()); - UNIT_ASSERT_STRINGS_EQUAL(l->Name(), "name1"); - UNIT_ASSERT_STRINGS_EQUAL(l->Value(), "value1"); - } - { - auto l = labels.Find("name2"); - UNIT_ASSERT(!l.Defined()); - } - - // duplicated name + { + auto l = labels.Find("name1"); + UNIT_ASSERT(l.Defined()); + UNIT_ASSERT_STRINGS_EQUAL(l->Name(), "name1"); + UNIT_ASSERT_STRINGS_EQUAL(l->Value(), "value1"); + } + { + auto l = labels.Find("name2"); + UNIT_ASSERT(!l.Defined()); + } + + // duplicated name UNIT_ASSERT(!labels.Add(TStringBuf("name1"), TStringBuf("value2"))); - UNIT_ASSERT(labels.Size() == 1); - + UNIT_ASSERT(labels.Size() == 1); + UNIT_ASSERT(labels.Add(TStringBuf("name2"), TStringBuf("value2"))); - UNIT_ASSERT(labels.Size() == 2); + UNIT_ASSERT(labels.Size() == 2); UNIT_ASSERT(labels.Has(TStringBuf("name2"))); - { - auto l = labels.Find("name2"); - UNIT_ASSERT(l.Defined()); - UNIT_ASSERT_STRINGS_EQUAL(l->Name(), "name2"); - UNIT_ASSERT_STRINGS_EQUAL(l->Value(), "value2"); - } - - UNIT_ASSERT_EQUAL(labels[0], TLabel("name1", "value1")); - UNIT_ASSERT_EQUAL(labels[1], TLabel("name2", "value2")); - + { + auto l = labels.Find("name2"); + UNIT_ASSERT(l.Defined()); + UNIT_ASSERT_STRINGS_EQUAL(l->Name(), "name2"); + UNIT_ASSERT_STRINGS_EQUAL(l->Value(), "value2"); + } + + UNIT_ASSERT_EQUAL(labels[0], TLabel("name1", "value1")); + UNIT_ASSERT_EQUAL(labels[1], TLabel("name2", "value2")); + TVector<TLabel> labelsCopy; for (auto&& label : labels) { labelsCopy.emplace_back(label.Name(), label.Value()); - } - + } + UNIT_ASSERT_EQUAL(labelsCopy, TVector<TLabel>({ - {"name1", "value1"}, - {"name2", "value2"}, - })); - } - + {"name1", "value1"}, + {"name2", "value2"}, + })); + } + Y_UNIT_TEST(Hash) { - TLabel label("name", "value"); - UNIT_ASSERT_EQUAL(ULL(2378153472115172159), label.Hash()); - - { - TLabels labels = {{"name", "value"}}; - UNIT_ASSERT_EQUAL(ULL(5420514431458887014), labels.Hash()); - } - { - TLabels labels = {{"name1", "value1"}, {"name2", "value2"}}; - UNIT_ASSERT_EQUAL(ULL(2226975250396609813), labels.Hash()); - } - } - - Y_UNIT_TEST(MakeEmptyLabels) { - { - auto labels = MakeLabels<TString>(); - UNIT_ASSERT(labels); - UNIT_ASSERT(labels->Empty()); - UNIT_ASSERT_VALUES_EQUAL(labels->Size(), 0); - } - { - auto labels = MakeLabels<TStringBuf>(); - UNIT_ASSERT(labels); - UNIT_ASSERT(labels->Empty()); - UNIT_ASSERT_VALUES_EQUAL(labels->Size(), 0); - } - } - - Y_UNIT_TEST(MakeLabelsFromInitializerList) { - auto labels = MakeLabels<TString>({{"my", "label"}}); - UNIT_ASSERT(labels); - UNIT_ASSERT(!labels->Empty()); - UNIT_ASSERT_VALUES_EQUAL(labels->Size(), 1); - - UNIT_ASSERT(labels->Has("my")); - - auto label = labels->Get("my"); - UNIT_ASSERT(label.has_value()); - UNIT_ASSERT_STRINGS_EQUAL((*label)->Name(), "my"); - UNIT_ASSERT_STRINGS_EQUAL((*label)->Value(), "label"); - } - - Y_UNIT_TEST(MakeLabelsFromOtherLabel) { - auto labels = MakeLabels({{"my", "label"}}); - UNIT_ASSERT(labels); - UNIT_ASSERT(!labels->Empty()); - UNIT_ASSERT_VALUES_EQUAL(labels->Size(), 1); - - UNIT_ASSERT(labels->Has("my")); - - auto label = labels->Get("my"); - UNIT_ASSERT(label.has_value()); - UNIT_ASSERT_STRINGS_EQUAL((*label)->Name(), "my"); - UNIT_ASSERT_STRINGS_EQUAL((*label)->Value(), "label"); - } -} + TLabel label("name", "value"); + UNIT_ASSERT_EQUAL(ULL(2378153472115172159), label.Hash()); + + { + TLabels labels = {{"name", "value"}}; + UNIT_ASSERT_EQUAL(ULL(5420514431458887014), labels.Hash()); + } + { + TLabels labels = {{"name1", "value1"}, {"name2", "value2"}}; + UNIT_ASSERT_EQUAL(ULL(2226975250396609813), labels.Hash()); + } + } + + Y_UNIT_TEST(MakeEmptyLabels) { + { + auto labels = MakeLabels<TString>(); + UNIT_ASSERT(labels); + UNIT_ASSERT(labels->Empty()); + UNIT_ASSERT_VALUES_EQUAL(labels->Size(), 0); + } + { + auto labels = MakeLabels<TStringBuf>(); + UNIT_ASSERT(labels); + UNIT_ASSERT(labels->Empty()); + UNIT_ASSERT_VALUES_EQUAL(labels->Size(), 0); + } + } + + Y_UNIT_TEST(MakeLabelsFromInitializerList) { + auto labels = MakeLabels<TString>({{"my", "label"}}); + UNIT_ASSERT(labels); + UNIT_ASSERT(!labels->Empty()); + UNIT_ASSERT_VALUES_EQUAL(labels->Size(), 1); + + UNIT_ASSERT(labels->Has("my")); + + auto label = labels->Get("my"); + UNIT_ASSERT(label.has_value()); + UNIT_ASSERT_STRINGS_EQUAL((*label)->Name(), "my"); + UNIT_ASSERT_STRINGS_EQUAL((*label)->Value(), "label"); + } + + Y_UNIT_TEST(MakeLabelsFromOtherLabel) { + auto labels = MakeLabels({{"my", "label"}}); + UNIT_ASSERT(labels); + UNIT_ASSERT(!labels->Empty()); + UNIT_ASSERT_VALUES_EQUAL(labels->Size(), 1); + + UNIT_ASSERT(labels->Has("my")); + + auto label = labels->Get("my"); + UNIT_ASSERT(label.has_value()); + UNIT_ASSERT_STRINGS_EQUAL((*label)->Name(), "my"); + UNIT_ASSERT_STRINGS_EQUAL((*label)->Value(), "label"); + } +} diff --git a/library/cpp/monlib/metrics/log_histogram_collector.h b/library/cpp/monlib/metrics/log_histogram_collector.h index 13b656af35f..b81f84ebf31 100644 --- a/library/cpp/monlib/metrics/log_histogram_collector.h +++ b/library/cpp/monlib/metrics/log_histogram_collector.h @@ -25,9 +25,9 @@ namespace NMonitoring { Merge(logHist); } - bool Collect(double value) { + bool Collect(double value) { std::lock_guard guard(Mutex_); - return CollectDouble(value); + return CollectDouble(value); } TLogHistogramSnapshotPtr Snapshot() const { @@ -64,16 +64,16 @@ namespace NMonitoring { ++Buckets_[idx]; } - bool CollectDouble(double value) { - if (Y_UNLIKELY(std::isnan(value) || std::isinf(value))) { - return false; - } + bool CollectDouble(double value) { + if (Y_UNLIKELY(std::isnan(value) || std::isinf(value))) { + return false; + } if (value <= 0.0) { ++CountZero_; } else { CollectPositiveDouble(value); } - return true; + return true; } void Merge(TLogHistogramSnapshot* logHist) { diff --git a/library/cpp/monlib/metrics/metric.h b/library/cpp/monlib/metrics/metric.h index f4c7dc37f2b..b8ce12d7533 100644 --- a/library/cpp/monlib/metrics/metric.h +++ b/library/cpp/monlib/metrics/metric.h @@ -1,28 +1,28 @@ -#pragma once - -#include "metric_consumer.h" - -#include <util/datetime/base.h> -#include <util/generic/ptr.h> - -namespace NMonitoring { - /////////////////////////////////////////////////////////////////////////////// - // IMetric - /////////////////////////////////////////////////////////////////////////////// - class IMetric { - public: - virtual ~IMetric() = default; - - virtual EMetricType Type() const noexcept = 0; - virtual void Accept(TInstant time, IMetricConsumer* consumer) const = 0; - }; - - using IMetricPtr = THolder<IMetric>; - - class IGauge: public IMetric { +#pragma once + +#include "metric_consumer.h" + +#include <util/datetime/base.h> +#include <util/generic/ptr.h> + +namespace NMonitoring { + /////////////////////////////////////////////////////////////////////////////// + // IMetric + /////////////////////////////////////////////////////////////////////////////// + class IMetric { + public: + virtual ~IMetric() = default; + + virtual EMetricType Type() const noexcept = 0; + virtual void Accept(TInstant time, IMetricConsumer* consumer) const = 0; + }; + + using IMetricPtr = THolder<IMetric>; + + class IGauge: public IMetric { public: - EMetricType Type() const noexcept final { - return EMetricType::GAUGE; + EMetricType Type() const noexcept final { + return EMetricType::GAUGE; } virtual double Add(double n) noexcept = 0; @@ -41,10 +41,10 @@ namespace NMonitoring { virtual double Get() const noexcept = 0; }; - class IIntGauge: public IMetric { + class IIntGauge: public IMetric { public: - EMetricType Type() const noexcept final { - return EMetricType::IGAUGE; + EMetricType Type() const noexcept final { + return EMetricType::IGAUGE; } virtual i64 Add(i64 n) noexcept = 0; @@ -72,10 +72,10 @@ namespace NMonitoring { virtual i64 Get() const noexcept = 0; }; - class ICounter: public IMetric { + class ICounter: public IMetric { public: - EMetricType Type() const noexcept final { - return EMetricType::COUNTER; + EMetricType Type() const noexcept final { + return EMetricType::COUNTER; } virtual ui64 Inc() noexcept { @@ -96,10 +96,10 @@ namespace NMonitoring { virtual ui64 Get() const noexcept = 0; }; - class IRate: public IMetric { + class IRate: public IMetric { public: - EMetricType Type() const noexcept final { - return EMetricType::RATE; + EMetricType Type() const noexcept final { + return EMetricType::RATE; } virtual ui64 Inc() noexcept { @@ -108,7 +108,7 @@ namespace NMonitoring { virtual ui64 Add(ui64 n) noexcept = 0; virtual ui64 Get() const noexcept = 0; - virtual void Reset() noexcept = 0; + virtual void Reset() noexcept = 0; }; class ILazyRate: public IMetric { @@ -120,19 +120,19 @@ namespace NMonitoring { virtual ui64 Get() const noexcept = 0; }; - class IHistogram: public IMetric { + class IHistogram: public IMetric { public: explicit IHistogram(bool isRate) : IsRate_{isRate} { } - EMetricType Type() const noexcept final { - return IsRate_ ? EMetricType::HIST_RATE : EMetricType::HIST; + EMetricType Type() const noexcept final { + return IsRate_ ? EMetricType::HIST_RATE : EMetricType::HIST; } - virtual void Record(double value) = 0; - virtual void Record(double value, ui32 count) = 0; + virtual void Record(double value) = 0; + virtual void Record(double value, ui32 count) = 0; virtual IHistogramSnapshotPtr TakeSnapshot() const = 0; virtual void Reset() = 0; @@ -140,15 +140,15 @@ namespace NMonitoring { const bool IsRate_; }; - /////////////////////////////////////////////////////////////////////////////// - // TGauge - /////////////////////////////////////////////////////////////////////////////// + /////////////////////////////////////////////////////////////////////////////// + // TGauge + /////////////////////////////////////////////////////////////////////////////// class TGauge final: public IGauge { - public: - explicit TGauge(double value = 0.0) { - Set(value); - } - + public: + explicit TGauge(double value = 0.0) { + Set(value); + } + double Add(double n) noexcept override { double newValue; double oldValue = Get(); @@ -162,21 +162,21 @@ namespace NMonitoring { void Set(double n) noexcept override { Value_.store(n, std::memory_order_relaxed); - } - + } + double Get() const noexcept override { return Value_.load(std::memory_order_relaxed); - } - - void Accept(TInstant time, IMetricConsumer* consumer) const override { - consumer->OnDouble(time, Get()); - } - - private: + } + + void Accept(TInstant time, IMetricConsumer* consumer) const override { + consumer->OnDouble(time, Get()); + } + + private: std::atomic<double> Value_; - }; - - /////////////////////////////////////////////////////////////////////////////// + }; + + /////////////////////////////////////////////////////////////////////////////// // TLazyGauge /////////////////////////////////////////////////////////////////////////////// class TLazyGauge final: public ILazyGauge { @@ -199,35 +199,35 @@ namespace NMonitoring { }; /////////////////////////////////////////////////////////////////////////////// - // TIntGauge - /////////////////////////////////////////////////////////////////////////////// + // TIntGauge + /////////////////////////////////////////////////////////////////////////////// class TIntGauge final: public IIntGauge { - public: - explicit TIntGauge(i64 value = 0) { - Set(value); - } - + public: + explicit TIntGauge(i64 value = 0) { + Set(value); + } + i64 Add(i64 n) noexcept override { return Value_.fetch_add(n, std::memory_order_relaxed) + n; } void Set(i64 value) noexcept override { Value_.store(value, std::memory_order_relaxed); - } - + } + i64 Get() const noexcept override { return Value_.load(std::memory_order_relaxed); - } - - void Accept(TInstant time, IMetricConsumer* consumer) const override { - consumer->OnInt64(time, Get()); - } - - private: + } + + void Accept(TInstant time, IMetricConsumer* consumer) const override { + consumer->OnInt64(time, Get()); + } + + private: std::atomic_int64_t Value_; - }; - - /////////////////////////////////////////////////////////////////////////////// + }; + + /////////////////////////////////////////////////////////////////////////////// // TLazyIntGauge /////////////////////////////////////////////////////////////////////////////// class TLazyIntGauge final: public ILazyIntGauge { @@ -250,35 +250,35 @@ namespace NMonitoring { }; /////////////////////////////////////////////////////////////////////////////// - // TCounter - /////////////////////////////////////////////////////////////////////////////// - class TCounter final: public ICounter { - public: - explicit TCounter(ui64 value = 0) { + // TCounter + /////////////////////////////////////////////////////////////////////////////// + class TCounter final: public ICounter { + public: + explicit TCounter(ui64 value = 0) { Value_.store(value, std::memory_order_relaxed); - } - + } + ui64 Add(ui64 n) noexcept override { return Value_.fetch_add(n, std::memory_order_relaxed) + n; - } - + } + ui64 Get() const noexcept override { return Value_.load(std::memory_order_relaxed); - } - + } + void Reset() noexcept override { Value_.store(0, std::memory_order_relaxed); - } - - void Accept(TInstant time, IMetricConsumer* consumer) const override { - consumer->OnUint64(time, Get()); - } - - private: + } + + void Accept(TInstant time, IMetricConsumer* consumer) const override { + consumer->OnUint64(time, Get()); + } + + private: std::atomic_uint64_t Value_; - }; - - /////////////////////////////////////////////////////////////////////////////// + }; + + /////////////////////////////////////////////////////////////////////////////// // TLazyCounter /////////////////////////////////////////////////////////////////////////////// class TLazyCounter final: public ILazyCounter { @@ -301,35 +301,35 @@ namespace NMonitoring { }; /////////////////////////////////////////////////////////////////////////////// - // TRate - /////////////////////////////////////////////////////////////////////////////// + // TRate + /////////////////////////////////////////////////////////////////////////////// class TRate final: public IRate { - public: - explicit TRate(ui64 value = 0) { + public: + explicit TRate(ui64 value = 0) { Value_.store(value, std::memory_order_relaxed); - } - + } + ui64 Add(ui64 n) noexcept override { return Value_.fetch_add(n, std::memory_order_relaxed) + n; - } - + } + ui64 Get() const noexcept override { return Value_.load(std::memory_order_relaxed); - } - - void Reset() noexcept override { - Value_.store(0, std::memory_order_relaxed); - } - - void Accept(TInstant time, IMetricConsumer* consumer) const override { - consumer->OnUint64(time, Get()); - } - - private: + } + + void Reset() noexcept override { + Value_.store(0, std::memory_order_relaxed); + } + + void Accept(TInstant time, IMetricConsumer* consumer) const override { + consumer->OnUint64(time, Get()); + } + + private: std::atomic_uint64_t Value_; - }; - - /////////////////////////////////////////////////////////////////////////////// + }; + + /////////////////////////////////////////////////////////////////////////////// // TLazyRate /////////////////////////////////////////////////////////////////////////////// class TLazyRate final: public ILazyRate { @@ -352,28 +352,28 @@ namespace NMonitoring { }; /////////////////////////////////////////////////////////////////////////////// - // THistogram - /////////////////////////////////////////////////////////////////////////////// + // THistogram + /////////////////////////////////////////////////////////////////////////////// class THistogram final: public IHistogram { - public: - THistogram(IHistogramCollectorPtr collector, bool isRate) + public: + THistogram(IHistogramCollectorPtr collector, bool isRate) : IHistogram(isRate) , Collector_(std::move(collector)) - { - } - - void Record(double value) override { - Collector_->Collect(value); - } - - void Record(double value, ui32 count) override { - Collector_->Collect(value, count); - } - - void Accept(TInstant time, IMetricConsumer* consumer) const override { + { + } + + void Record(double value) override { + Collector_->Collect(value); + } + + void Record(double value, ui32 count) override { + Collector_->Collect(value, count); + } + + void Accept(TInstant time, IMetricConsumer* consumer) const override { consumer->OnHistogram(time, TakeSnapshot()); - } - + } + IHistogramSnapshotPtr TakeSnapshot() const override { return Collector_->Snapshot(); } @@ -382,7 +382,7 @@ namespace NMonitoring { Collector_->Reset(); } - private: - IHistogramCollectorPtr Collector_; - }; -} + private: + IHistogramCollectorPtr Collector_; + }; +} diff --git a/library/cpp/monlib/metrics/metric_consumer.cpp b/library/cpp/monlib/metrics/metric_consumer.cpp index 4289efdad98..121ee368f02 100644 --- a/library/cpp/monlib/metrics/metric_consumer.cpp +++ b/library/cpp/monlib/metrics/metric_consumer.cpp @@ -8,7 +8,7 @@ namespace NMonitoring { Y_ENSURE(false, "Not implemented"); } - std::pair<ui32, ui32> IMetricConsumer::PrepareLabel(TStringBuf name, TStringBuf value) { + std::pair<ui32, ui32> IMetricConsumer::PrepareLabel(TStringBuf name, TStringBuf value) { Y_UNUSED(name, value); Y_ENSURE(false, "Not implemented"); } diff --git a/library/cpp/monlib/metrics/metric_consumer.h b/library/cpp/monlib/metrics/metric_consumer.h index 87deed785dc..f7a727585ad 100644 --- a/library/cpp/monlib/metrics/metric_consumer.h +++ b/library/cpp/monlib/metrics/metric_consumer.h @@ -1,40 +1,40 @@ -#pragma once - -#include "metric_type.h" -#include "histogram_collector.h" +#pragma once + +#include "metric_type.h" +#include "histogram_collector.h" #include "summary_collector.h" #include "log_histogram_snapshot.h" - -class TInstant; - -namespace NMonitoring { - class IMetricConsumer { - public: - virtual ~IMetricConsumer() = default; - - virtual void OnStreamBegin() = 0; - virtual void OnStreamEnd() = 0; - - virtual void OnCommonTime(TInstant time) = 0; - - virtual void OnMetricBegin(EMetricType type) = 0; - virtual void OnMetricEnd() = 0; - - virtual void OnLabelsBegin() = 0; - virtual void OnLabelsEnd() = 0; - virtual void OnLabel(TStringBuf name, TStringBuf value) = 0; + +class TInstant; + +namespace NMonitoring { + class IMetricConsumer { + public: + virtual ~IMetricConsumer() = default; + + virtual void OnStreamBegin() = 0; + virtual void OnStreamEnd() = 0; + + virtual void OnCommonTime(TInstant time) = 0; + + virtual void OnMetricBegin(EMetricType type) = 0; + virtual void OnMetricEnd() = 0; + + virtual void OnLabelsBegin() = 0; + virtual void OnLabelsEnd() = 0; + virtual void OnLabel(TStringBuf name, TStringBuf value) = 0; virtual void OnLabel(ui32 name, ui32 value); - virtual std::pair<ui32, ui32> PrepareLabel(TStringBuf name, TStringBuf value); - - virtual void OnDouble(TInstant time, double value) = 0; - virtual void OnInt64(TInstant time, i64 value) = 0; - virtual void OnUint64(TInstant time, ui64 value) = 0; - - virtual void OnHistogram(TInstant time, IHistogramSnapshotPtr snapshot) = 0; + virtual std::pair<ui32, ui32> PrepareLabel(TStringBuf name, TStringBuf value); + + virtual void OnDouble(TInstant time, double value) = 0; + virtual void OnInt64(TInstant time, i64 value) = 0; + virtual void OnUint64(TInstant time, ui64 value) = 0; + + virtual void OnHistogram(TInstant time, IHistogramSnapshotPtr snapshot) = 0; virtual void OnLogHistogram(TInstant time, TLogHistogramSnapshotPtr snapshot) = 0; - virtual void OnSummaryDouble(TInstant time, ISummaryDoubleSnapshotPtr snapshot) = 0; - }; - - using IMetricConsumerPtr = THolder<IMetricConsumer>; - -} + virtual void OnSummaryDouble(TInstant time, ISummaryDoubleSnapshotPtr snapshot) = 0; + }; + + using IMetricConsumerPtr = THolder<IMetricConsumer>; + +} diff --git a/library/cpp/monlib/metrics/metric_registry.cpp b/library/cpp/monlib/metrics/metric_registry.cpp index 091a7b52829..b083163a7b2 100644 --- a/library/cpp/monlib/metrics/metric_registry.cpp +++ b/library/cpp/monlib/metrics/metric_registry.cpp @@ -1,41 +1,41 @@ -#include "metric_registry.h" - +#include "metric_registry.h" + #include <memory> -namespace NMonitoring { - namespace { - void ConsumeLabels(IMetricConsumer* consumer, const ILabels& labels) { - for (auto&& label: labels) { - consumer->OnLabel(label.Name(), label.Value()); - } - } - - template <typename TLabelsConsumer> - void ConsumeMetric(TInstant time, IMetricConsumer* consumer, IMetric* metric, TLabelsConsumer&& labelsConsumer) { - consumer->OnMetricBegin(metric->Type()); - - // (1) add labels - consumer->OnLabelsBegin(); - labelsConsumer(); - consumer->OnLabelsEnd(); - - // (2) add time and value - metric->Accept(time, consumer); - consumer->OnMetricEnd(); - } - } - +namespace NMonitoring { + namespace { + void ConsumeLabels(IMetricConsumer* consumer, const ILabels& labels) { + for (auto&& label: labels) { + consumer->OnLabel(label.Name(), label.Value()); + } + } + + template <typename TLabelsConsumer> + void ConsumeMetric(TInstant time, IMetricConsumer* consumer, IMetric* metric, TLabelsConsumer&& labelsConsumer) { + consumer->OnMetricBegin(metric->Type()); + + // (1) add labels + consumer->OnLabelsBegin(); + labelsConsumer(); + consumer->OnLabelsEnd(); + + // (2) add time and value + metric->Accept(time, consumer); + consumer->OnMetricEnd(); + } + } + void WriteLabels(IMetricConsumer* consumer, const ILabels& labels) { consumer->OnLabelsBegin(); - ConsumeLabels(consumer, labels); + ConsumeLabels(consumer, labels); consumer->OnLabelsEnd(); } - TMetricRegistry::TMetricRegistry() = default; - TMetricRegistry::~TMetricRegistry() = default; + TMetricRegistry::TMetricRegistry() = default; + TMetricRegistry::~TMetricRegistry() = default; - TMetricRegistry::TMetricRegistry(const TLabels& commonLabels) - : TMetricRegistry{} + TMetricRegistry::TMetricRegistry(const TLabels& commonLabels) + : TMetricRegistry{} { CommonLabels_ = commonLabels; } @@ -44,14 +44,14 @@ namespace NMonitoring { return Singleton<TMetricRegistry>(); } - TGauge* TMetricRegistry::Gauge(TLabels labels) { - return Metric<TGauge, EMetricType::GAUGE>(std::move(labels)); - } - - TGauge* TMetricRegistry::Gauge(ILabelsPtr labels) { - return Metric<TGauge, EMetricType::GAUGE>(std::move(labels)); - } - + TGauge* TMetricRegistry::Gauge(TLabels labels) { + return Metric<TGauge, EMetricType::GAUGE>(std::move(labels)); + } + + TGauge* TMetricRegistry::Gauge(ILabelsPtr labels) { + return Metric<TGauge, EMetricType::GAUGE>(std::move(labels)); + } + TLazyGauge* TMetricRegistry::LazyGauge(TLabels labels, std::function<double()> supplier) { return Metric<TLazyGauge, EMetricType::GAUGE>(std::move(labels), std::move(supplier)); } @@ -60,14 +60,14 @@ namespace NMonitoring { return Metric<TLazyGauge, EMetricType::GAUGE>(std::move(labels), std::move(supplier)); } - TIntGauge* TMetricRegistry::IntGauge(TLabels labels) { - return Metric<TIntGauge, EMetricType::IGAUGE>(std::move(labels)); - } - - TIntGauge* TMetricRegistry::IntGauge(ILabelsPtr labels) { - return Metric<TIntGauge, EMetricType::IGAUGE>(std::move(labels)); - } - + TIntGauge* TMetricRegistry::IntGauge(TLabels labels) { + return Metric<TIntGauge, EMetricType::IGAUGE>(std::move(labels)); + } + + TIntGauge* TMetricRegistry::IntGauge(ILabelsPtr labels) { + return Metric<TIntGauge, EMetricType::IGAUGE>(std::move(labels)); + } + TLazyIntGauge* TMetricRegistry::LazyIntGauge(TLabels labels, std::function<i64()> supplier) { return Metric<TLazyIntGauge, EMetricType::GAUGE>(std::move(labels), std::move(supplier)); } @@ -76,12 +76,12 @@ namespace NMonitoring { return Metric<TLazyIntGauge, EMetricType::GAUGE>(std::move(labels), std::move(supplier)); } - TCounter* TMetricRegistry::Counter(TLabels labels) { - return Metric<TCounter, EMetricType::COUNTER>(std::move(labels)); + TCounter* TMetricRegistry::Counter(TLabels labels) { + return Metric<TCounter, EMetricType::COUNTER>(std::move(labels)); } - TCounter* TMetricRegistry::Counter(ILabelsPtr labels) { - return Metric<TCounter, EMetricType::COUNTER>(std::move(labels)); + TCounter* TMetricRegistry::Counter(ILabelsPtr labels) { + return Metric<TCounter, EMetricType::COUNTER>(std::move(labels)); } TLazyCounter* TMetricRegistry::LazyCounter(TLabels labels, std::function<ui64()> supplier) { @@ -92,12 +92,12 @@ namespace NMonitoring { return Metric<TLazyCounter, EMetricType::COUNTER>(std::move(labels), std::move(supplier)); } - TRate* TMetricRegistry::Rate(TLabels labels) { - return Metric<TRate, EMetricType::RATE>(std::move(labels)); + TRate* TMetricRegistry::Rate(TLabels labels) { + return Metric<TRate, EMetricType::RATE>(std::move(labels)); } - TRate* TMetricRegistry::Rate(ILabelsPtr labels) { - return Metric<TRate, EMetricType::RATE>(std::move(labels)); + TRate* TMetricRegistry::Rate(ILabelsPtr labels) { + return Metric<TRate, EMetricType::RATE>(std::move(labels)); } TLazyRate* TMetricRegistry::LazyRate(TLabels labels, std::function<ui64()> supplier) { @@ -108,20 +108,20 @@ namespace NMonitoring { return Metric<TLazyRate, EMetricType::RATE>(std::move(labels), std::move(supplier)); } - THistogram* TMetricRegistry::HistogramCounter(TLabels labels, IHistogramCollectorPtr collector) { - return Metric<THistogram, EMetricType::HIST>(std::move(labels), std::move(collector), false); - } - - THistogram* TMetricRegistry::HistogramCounter(ILabelsPtr labels, IHistogramCollectorPtr collector) { - return Metric<THistogram, EMetricType::HIST>(std::move(labels), std::move(collector), false); + THistogram* TMetricRegistry::HistogramCounter(TLabels labels, IHistogramCollectorPtr collector) { + return Metric<THistogram, EMetricType::HIST>(std::move(labels), std::move(collector), false); } - THistogram* TMetricRegistry::HistogramRate(TLabels labels, IHistogramCollectorPtr collector) { - return Metric<THistogram, EMetricType::HIST_RATE>(std::move(labels), std::move(collector), true); - } - - THistogram* TMetricRegistry::HistogramRate(ILabelsPtr labels, IHistogramCollectorPtr collector) { - return Metric<THistogram, EMetricType::HIST_RATE>(std::move(labels), std::move(collector), true); + THistogram* TMetricRegistry::HistogramCounter(ILabelsPtr labels, IHistogramCollectorPtr collector) { + return Metric<THistogram, EMetricType::HIST>(std::move(labels), std::move(collector), false); + } + + THistogram* TMetricRegistry::HistogramRate(TLabels labels, IHistogramCollectorPtr collector) { + return Metric<THistogram, EMetricType::HIST_RATE>(std::move(labels), std::move(collector), true); + } + + THistogram* TMetricRegistry::HistogramRate(ILabelsPtr labels, IHistogramCollectorPtr collector) { + return Metric<THistogram, EMetricType::HIST_RATE>(std::move(labels), std::move(collector), true); } void TMetricRegistry::Reset() { @@ -157,74 +157,74 @@ namespace NMonitoring { Metrics_.clear(); } - template <typename TMetric, EMetricType type, typename TLabelsType, typename... Args> - TMetric* TMetricRegistry::Metric(TLabelsType&& labels, Args&&... args) { - { + template <typename TMetric, EMetricType type, typename TLabelsType, typename... Args> + TMetric* TMetricRegistry::Metric(TLabelsType&& labels, Args&&... args) { + { TReadGuard g{Lock_}; - auto it = Metrics_.find(labels); - if (it != Metrics_.end()) { - Y_ENSURE(it->second->Type() == type, "cannot create metric " << labels - << " with type " << MetricTypeToStr(type) - << ", because registry already has same metric with type " << MetricTypeToStr(it->second->Type())); - return static_cast<TMetric*>(it->second.Get()); - } - } - - { - IMetricPtr metric = MakeHolder<TMetric>(std::forward<Args>(args)...); - + auto it = Metrics_.find(labels); + if (it != Metrics_.end()) { + Y_ENSURE(it->second->Type() == type, "cannot create metric " << labels + << " with type " << MetricTypeToStr(type) + << ", because registry already has same metric with type " << MetricTypeToStr(it->second->Type())); + return static_cast<TMetric*>(it->second.Get()); + } + } + + { + IMetricPtr metric = MakeHolder<TMetric>(std::forward<Args>(args)...); + TWriteGuard g{Lock_}; - // decltype(Metrics_)::iterator breaks build on windows - THashMap<ILabelsPtr, IMetricPtr>::iterator it; + // decltype(Metrics_)::iterator breaks build on windows + THashMap<ILabelsPtr, IMetricPtr>::iterator it; if constexpr (!std::is_convertible_v<TLabelsType, ILabelsPtr>) { - it = Metrics_.emplace(new TLabels{std::forward<TLabelsType>(labels)}, std::move(metric)).first; + it = Metrics_.emplace(new TLabels{std::forward<TLabelsType>(labels)}, std::move(metric)).first; } else { - it = Metrics_.emplace(std::forward<TLabelsType>(labels), std::move(metric)).first; + it = Metrics_.emplace(std::forward<TLabelsType>(labels), std::move(metric)).first; } - return static_cast<TMetric*>(it->second.Get()); - } - } + return static_cast<TMetric*>(it->second.Get()); + } + } - void TMetricRegistry::RemoveMetric(const ILabels& labels) noexcept { + void TMetricRegistry::RemoveMetric(const ILabels& labels) noexcept { TWriteGuard g{Lock_}; - Metrics_.erase(labels); + Metrics_.erase(labels); } - void TMetricRegistry::Accept(TInstant time, IMetricConsumer* consumer) const { + void TMetricRegistry::Accept(TInstant time, IMetricConsumer* consumer) const { consumer->OnStreamBegin(); if (!CommonLabels_.Empty()) { - consumer->OnLabelsBegin(); - ConsumeLabels(consumer, CommonLabels_); - consumer->OnLabelsEnd(); + consumer->OnLabelsBegin(); + ConsumeLabels(consumer, CommonLabels_); + consumer->OnLabelsEnd(); + } + + { + TReadGuard g{Lock_}; + for (const auto& it: Metrics_) { + ILabels* labels = it.first.Get(); + IMetric* metric = it.second.Get(); + ConsumeMetric(time, consumer, metric, [&]() { + ConsumeLabels(consumer, *labels); + }); + } } - { - TReadGuard g{Lock_}; - for (const auto& it: Metrics_) { - ILabels* labels = it.first.Get(); - IMetric* metric = it.second.Get(); - ConsumeMetric(time, consumer, metric, [&]() { - ConsumeLabels(consumer, *labels); - }); - } - } - consumer->OnStreamEnd(); } - void TMetricRegistry::Append(TInstant time, IMetricConsumer* consumer) const { + void TMetricRegistry::Append(TInstant time, IMetricConsumer* consumer) const { TReadGuard g{Lock_}; - for (const auto& it: Metrics_) { - ILabels* labels = it.first.Get(); - IMetric* metric = it.second.Get(); - ConsumeMetric(time, consumer, metric, [&]() { - ConsumeLabels(consumer, CommonLabels_); - ConsumeLabels(consumer, *labels); - }); - } - } -} + for (const auto& it: Metrics_) { + ILabels* labels = it.first.Get(); + IMetric* metric = it.second.Get(); + ConsumeMetric(time, consumer, metric, [&]() { + ConsumeLabels(consumer, CommonLabels_); + ConsumeLabels(consumer, *labels); + }); + } + } +} diff --git a/library/cpp/monlib/metrics/metric_registry.h b/library/cpp/monlib/metrics/metric_registry.h index e308065da9a..670cf8651e1 100644 --- a/library/cpp/monlib/metrics/metric_registry.h +++ b/library/cpp/monlib/metrics/metric_registry.h @@ -1,17 +1,17 @@ -#pragma once - -#include "labels.h" -#include "metric.h" - +#pragma once + +#include "labels.h" +#include "metric.h" + #include <util/system/rwlock.h> - + #include <library/cpp/threading/light_rw_lock/lightrwlock.h> -namespace NMonitoring { - class IMetricFactory { +namespace NMonitoring { + class IMetricFactory { public: - virtual ~IMetricFactory() = default; + virtual ~IMetricFactory() = default; virtual IGauge* Gauge(ILabelsPtr labels) = 0; virtual ILazyGauge* LazyGauge(ILabelsPtr labels, std::function<double()> supplier) = 0; @@ -36,27 +36,27 @@ namespace NMonitoring { public: virtual ~IMetricSupplier() = default; - virtual void Accept(TInstant time, IMetricConsumer* consumer) const = 0; - virtual void Append(TInstant time, IMetricConsumer* consumer) const = 0; + virtual void Accept(TInstant time, IMetricConsumer* consumer) const = 0; + virtual void Append(TInstant time, IMetricConsumer* consumer) const = 0; }; class IMetricRegistry: public IMetricSupplier, public IMetricFactory { public: virtual const TLabels& CommonLabels() const noexcept = 0; - virtual void RemoveMetric(const ILabels& labels) noexcept = 0; + virtual void RemoveMetric(const ILabels& labels) noexcept = 0; }; - /////////////////////////////////////////////////////////////////////////////// - // TMetricRegistry - /////////////////////////////////////////////////////////////////////////////// - class TMetricRegistry: public IMetricRegistry { - public: - TMetricRegistry(); - ~TMetricRegistry(); - - explicit TMetricRegistry(const TLabels& commonLabels); - + /////////////////////////////////////////////////////////////////////////////// + // TMetricRegistry + /////////////////////////////////////////////////////////////////////////////// + class TMetricRegistry: public IMetricRegistry { + public: + TMetricRegistry(); + ~TMetricRegistry(); + + explicit TMetricRegistry(const TLabels& commonLabels); + /** * Get a global metrics registry instance. */ @@ -66,19 +66,19 @@ namespace NMonitoring { TLazyGauge* LazyGauge(TLabels labels, std::function<double()> supplier); TIntGauge* IntGauge(TLabels labels); TLazyIntGauge* LazyIntGauge(TLabels labels, std::function<i64()> supplier); - TCounter* Counter(TLabels labels); + TCounter* Counter(TLabels labels); TLazyCounter* LazyCounter(TLabels labels, std::function<ui64()> supplier); TRate* Rate(TLabels labels); TLazyRate* LazyRate(TLabels labels, std::function<ui64()> supplier); - - THistogram* HistogramCounter( + + THistogram* HistogramCounter( TLabels labels, - IHistogramCollectorPtr collector); - - THistogram* HistogramRate( + IHistogramCollectorPtr collector); + + THistogram* HistogramRate( TLabels labels, - IHistogramCollectorPtr collector); - + IHistogramCollectorPtr collector); + /** * Set all registered metrics to zero */ @@ -88,21 +88,21 @@ namespace NMonitoring { */ void Clear(); - void Accept(TInstant time, IMetricConsumer* consumer) const override; - void Append(TInstant time, IMetricConsumer* consumer) const override; - + void Accept(TInstant time, IMetricConsumer* consumer) const override; + void Append(TInstant time, IMetricConsumer* consumer) const override; + const TLabels& CommonLabels() const noexcept override { - return CommonLabels_; - } - - void RemoveMetric(const ILabels& labels) noexcept override; + return CommonLabels_; + } - private: + void RemoveMetric(const ILabels& labels) noexcept override; + + private: TGauge* Gauge(ILabelsPtr labels) override; TLazyGauge* LazyGauge(ILabelsPtr labels, std::function<double()> supplier) override; TIntGauge* IntGauge(ILabelsPtr labels) override; TLazyIntGauge* LazyIntGauge(ILabelsPtr labels, std::function<i64()> supplier) override; - TCounter* Counter(ILabelsPtr labels) override; + TCounter* Counter(ILabelsPtr labels) override; TLazyCounter* LazyCounter(ILabelsPtr labels, std::function<ui64()> supplier) override; TRate* Rate(ILabelsPtr labels) override; TLazyRate* LazyRate(ILabelsPtr labels, std::function<ui64()> supplier) override; @@ -117,13 +117,13 @@ namespace NMonitoring { private: TRWMutex Lock_; - THashMap<ILabelsPtr, IMetricPtr> Metrics_; + THashMap<ILabelsPtr, IMetricPtr> Metrics_; - template <typename TMetric, EMetricType type, typename TLabelsType, typename... Args> - TMetric* Metric(TLabelsType&& labels, Args&&... args); - - TLabels CommonLabels_; - }; + template <typename TMetric, EMetricType type, typename TLabelsType, typename... Args> + TMetric* Metric(TLabelsType&& labels, Args&&... args); + + TLabels CommonLabels_; + }; void WriteLabels(IMetricConsumer* consumer, const ILabels& labels); -} +} diff --git a/library/cpp/monlib/metrics/metric_registry_ut.cpp b/library/cpp/monlib/metrics/metric_registry_ut.cpp index e10d2efc025..86d9a52ec0c 100644 --- a/library/cpp/monlib/metrics/metric_registry_ut.cpp +++ b/library/cpp/monlib/metrics/metric_registry_ut.cpp @@ -1,15 +1,15 @@ -#include "metric_registry.h" - -#include <library/cpp/monlib/encode/protobuf/protobuf.h> -#include <library/cpp/monlib/encode/json/json.h> +#include "metric_registry.h" + +#include <library/cpp/monlib/encode/protobuf/protobuf.h> +#include <library/cpp/monlib/encode/json/json.h> #include <library/cpp/resource/resource.h> - + #include <library/cpp/testing/unittest/registar.h> - -#include <util/stream/str.h> - -using namespace NMonitoring; - + +#include <util/stream/str.h> + +using namespace NMonitoring; + template<> void Out<NMonitoring::NProto::TSingleSample::ValueCase>(IOutputStream& os, NMonitoring::NProto::TSingleSample::ValueCase val) { switch (val) { @@ -37,14 +37,14 @@ void Out<NMonitoring::NProto::TSingleSample::ValueCase>(IOutputStream& os, NMoni } } -Y_UNIT_TEST_SUITE(TMetricRegistryTest) { +Y_UNIT_TEST_SUITE(TMetricRegistryTest) { Y_UNIT_TEST(Gauge) { - TMetricRegistry registry(TLabels{{"common", "label"}}); - TGauge* g = registry.Gauge({{"my", "gauge"}}); - - UNIT_ASSERT_DOUBLES_EQUAL(g->Get(), 0.0, 1E-6); - g->Set(12.34); - UNIT_ASSERT_DOUBLES_EQUAL(g->Get(), 12.34, 1E-6); + TMetricRegistry registry(TLabels{{"common", "label"}}); + TGauge* g = registry.Gauge({{"my", "gauge"}}); + + UNIT_ASSERT_DOUBLES_EQUAL(g->Get(), 0.0, 1E-6); + g->Set(12.34); + UNIT_ASSERT_DOUBLES_EQUAL(g->Get(), 12.34, 1E-6); double val; @@ -55,8 +55,8 @@ Y_UNIT_TEST_SUITE(TMetricRegistryTest) { val = g->Add(-3.47); UNIT_ASSERT_DOUBLES_EQUAL(g->Get(), 10.07, 1E-6); UNIT_ASSERT_DOUBLES_EQUAL(g->Get(), val, 1E-6); - } - + } + Y_UNIT_TEST(LazyGauge) { TMetricRegistry registry(TLabels{{"common", "label"}}); double val = 0.0; @@ -76,7 +76,7 @@ Y_UNIT_TEST_SUITE(TMetricRegistryTest) { } Y_UNIT_TEST(IntGauge) { - TMetricRegistry registry(TLabels{{"common", "label"}}); + TMetricRegistry registry(TLabels{{"common", "label"}}); TIntGauge* g = registry.IntGauge({{"my", "gauge"}}); UNIT_ASSERT_VALUES_EQUAL(g->Get(), 0); @@ -123,16 +123,16 @@ Y_UNIT_TEST_SUITE(TMetricRegistryTest) { } Y_UNIT_TEST(Counter) { - TMetricRegistry registry(TLabels{{"common", "label"}}); - TCounter* c = registry.Counter({{"my", "counter"}}); - + TMetricRegistry registry(TLabels{{"common", "label"}}); + TCounter* c = registry.Counter({{"my", "counter"}}); + UNIT_ASSERT_VALUES_EQUAL(c->Get(), 0); UNIT_ASSERT_VALUES_EQUAL(c->Inc(), 1); UNIT_ASSERT_VALUES_EQUAL(c->Get(), 1); UNIT_ASSERT_VALUES_EQUAL(c->Add(10), 11); UNIT_ASSERT_VALUES_EQUAL(c->Get(), 11); - } - + } + Y_UNIT_TEST(LazyCounter) { TMetricRegistry registry(TLabels{{"common", "label"}}); ui64 val = 0; @@ -156,9 +156,9 @@ Y_UNIT_TEST_SUITE(TMetricRegistryTest) { } Y_UNIT_TEST(DoubleCounter) { - TMetricRegistry registry(TLabels{{"common", "label"}}); + TMetricRegistry registry(TLabels{{"common", "label"}}); - TCounter* c = registry.Counter({{"my", "counter"}}); + TCounter* c = registry.Counter({{"my", "counter"}}); UNIT_ASSERT_VALUES_EQUAL(c->Get(), 0); c->Add(10); @@ -167,19 +167,19 @@ Y_UNIT_TEST_SUITE(TMetricRegistryTest) { } Y_UNIT_TEST(Sample) { - TMetricRegistry registry(TLabels{{"common", "label"}}); - - TGauge* g = registry.Gauge({{"my", "gauge"}}); - g->Set(12.34); - - TCounter* c = registry.Counter({{"my", "counter"}}); - c->Add(10); - - NProto::TSingleSamplesList samples; - auto encoder = EncoderProtobuf(&samples); - auto now = TInstant::Now(); - registry.Accept(now, encoder.Get()); - + TMetricRegistry registry(TLabels{{"common", "label"}}); + + TGauge* g = registry.Gauge({{"my", "gauge"}}); + g->Set(12.34); + + TCounter* c = registry.Counter({{"my", "counter"}}); + c->Add(10); + + NProto::TSingleSamplesList samples; + auto encoder = EncoderProtobuf(&samples); + auto now = TInstant::Now(); + registry.Accept(now, encoder.Get()); + UNIT_ASSERT_VALUES_EQUAL(samples.SamplesSize(), 2); UNIT_ASSERT_VALUES_EQUAL(samples.CommonLabelsSize(), 1); { @@ -187,65 +187,65 @@ Y_UNIT_TEST_SUITE(TMetricRegistryTest) { UNIT_ASSERT_STRINGS_EQUAL(label.GetName(), "common"); UNIT_ASSERT_STRINGS_EQUAL(label.GetValue(), "label"); } - - for (const NProto::TSingleSample& sample : samples.GetSamples()) { + + for (const NProto::TSingleSample& sample : samples.GetSamples()) { UNIT_ASSERT_VALUES_EQUAL(sample.LabelsSize(), 1); UNIT_ASSERT_VALUES_EQUAL(sample.GetTime(), now.MilliSeconds()); - - if (sample.GetMetricType() == NProto::GAUGE) { + + if (sample.GetMetricType() == NProto::GAUGE) { UNIT_ASSERT_VALUES_EQUAL(sample.GetValueCase(), NProto::TSingleSample::kFloat64); - UNIT_ASSERT_DOUBLES_EQUAL(sample.GetFloat64(), 12.34, 1E-6); - + UNIT_ASSERT_DOUBLES_EQUAL(sample.GetFloat64(), 12.34, 1E-6); + const NProto::TLabel& label = sample.GetLabels(0); - UNIT_ASSERT_STRINGS_EQUAL(label.GetName(), "my"); - UNIT_ASSERT_STRINGS_EQUAL(label.GetValue(), "gauge"); - } else if (sample.GetMetricType() == NProto::COUNTER) { + UNIT_ASSERT_STRINGS_EQUAL(label.GetName(), "my"); + UNIT_ASSERT_STRINGS_EQUAL(label.GetValue(), "gauge"); + } else if (sample.GetMetricType() == NProto::COUNTER) { UNIT_ASSERT_VALUES_EQUAL(sample.GetValueCase(), NProto::TSingleSample::kUint64); UNIT_ASSERT_VALUES_EQUAL(sample.GetUint64(), 10); - + const NProto::TLabel& label = sample.GetLabels(0); - UNIT_ASSERT_STRINGS_EQUAL(label.GetName(), "my"); - UNIT_ASSERT_STRINGS_EQUAL(label.GetValue(), "counter"); - } else { - UNIT_FAIL("unexpected sample type"); - } - } - } - - Y_UNIT_TEST(Histograms) { - TMetricRegistry registry(TLabels{{"common", "label"}}); - - THistogram* h1 = registry.HistogramCounter( - {{"sensor", "readTimeMillis"}}, - ExponentialHistogram(5, 2)); - - THistogram* h2 = registry.HistogramRate( - {{"sensor", "writeTimeMillis"}}, - ExplicitHistogram({1, 5, 15, 20, 25})); - - for (i64 i = 0; i < 100; i++) { - h1->Record(i); - h2->Record(i); - } - - TStringStream ss; - { - auto encoder = EncoderJson(&ss, 2); - registry.Accept(TInstant::Zero(), encoder.Get()); - } - ss << '\n'; - - UNIT_ASSERT_NO_DIFF(ss.Str(), NResource::Find("/histograms.json")); - } - + UNIT_ASSERT_STRINGS_EQUAL(label.GetName(), "my"); + UNIT_ASSERT_STRINGS_EQUAL(label.GetValue(), "counter"); + } else { + UNIT_FAIL("unexpected sample type"); + } + } + } + + Y_UNIT_TEST(Histograms) { + TMetricRegistry registry(TLabels{{"common", "label"}}); + + THistogram* h1 = registry.HistogramCounter( + {{"sensor", "readTimeMillis"}}, + ExponentialHistogram(5, 2)); + + THistogram* h2 = registry.HistogramRate( + {{"sensor", "writeTimeMillis"}}, + ExplicitHistogram({1, 5, 15, 20, 25})); + + for (i64 i = 0; i < 100; i++) { + h1->Record(i); + h2->Record(i); + } + + TStringStream ss; + { + auto encoder = EncoderJson(&ss, 2); + registry.Accept(TInstant::Zero(), encoder.Get()); + } + ss << '\n'; + + UNIT_ASSERT_NO_DIFF(ss.Str(), NResource::Find("/histograms.json")); + } + Y_UNIT_TEST(StreamingEncoderTest) { const TString expected { "{\"commonLabels\":{\"common\":\"label\"}," "\"sensors\":[{\"kind\":\"GAUGE\",\"labels\":{\"my\":\"gauge\"},\"value\":12.34}]}" }; - TMetricRegistry registry(TLabels{{"common", "label"}}); + TMetricRegistry registry(TLabels{{"common", "label"}}); TGauge* g = registry.Gauge({{"my", "gauge"}}); g->Set(12.34); @@ -257,48 +257,48 @@ Y_UNIT_TEST_SUITE(TMetricRegistryTest) { UNIT_ASSERT_STRINGS_EQUAL(os.Str(), expected); } - Y_UNIT_TEST(CreatingSameMetricWithDifferentTypesShouldThrow) { - TMetricRegistry registry; - + Y_UNIT_TEST(CreatingSameMetricWithDifferentTypesShouldThrow) { + TMetricRegistry registry; + registry.Gauge({{"foo", "bar"}}); UNIT_ASSERT_EXCEPTION(registry.Counter({{"foo", "bar"}}), yexception); - + registry.HistogramCounter({{"bar", "baz"}}, nullptr); UNIT_ASSERT_EXCEPTION(registry.HistogramRate({{"bar", "baz"}}, nullptr), yexception); - } - - Y_UNIT_TEST(EncodeRegistryWithCommonLabels) { - TMetricRegistry registry(TLabels{{"common", "label"}}); - - TGauge* g = registry.Gauge({{"my", "gauge"}}); - g->Set(12.34); - - // Append() adds common labels to each metric, allowing to combine - // several metric registries in one resulting blob - { - TStringStream os; - auto encoder = EncoderJson(&os); - encoder->OnStreamBegin(); - registry.Append(TInstant::Zero(), encoder.Get()); - encoder->OnStreamEnd(); - - UNIT_ASSERT_STRINGS_EQUAL( - os.Str(), - "{\"sensors\":[{\"kind\":\"GAUGE\",\"labels\":{\"common\":\"label\",\"my\":\"gauge\"},\"value\":12.34}]}"); - } - - // Accept() adds common labels to the beginning of the blob - { - TStringStream os; - auto encoder = EncoderJson(&os); - registry.Accept(TInstant::Zero(), encoder.Get()); - - UNIT_ASSERT_STRINGS_EQUAL( - os.Str(), - "{\"commonLabels\":{\"common\":\"label\"}," - "\"sensors\":[{\"kind\":\"GAUGE\",\"labels\":{\"my\":\"gauge\"},\"value\":12.34}]}"); - } - } + } + + Y_UNIT_TEST(EncodeRegistryWithCommonLabels) { + TMetricRegistry registry(TLabels{{"common", "label"}}); + + TGauge* g = registry.Gauge({{"my", "gauge"}}); + g->Set(12.34); + + // Append() adds common labels to each metric, allowing to combine + // several metric registries in one resulting blob + { + TStringStream os; + auto encoder = EncoderJson(&os); + encoder->OnStreamBegin(); + registry.Append(TInstant::Zero(), encoder.Get()); + encoder->OnStreamEnd(); + + UNIT_ASSERT_STRINGS_EQUAL( + os.Str(), + "{\"sensors\":[{\"kind\":\"GAUGE\",\"labels\":{\"common\":\"label\",\"my\":\"gauge\"},\"value\":12.34}]}"); + } + + // Accept() adds common labels to the beginning of the blob + { + TStringStream os; + auto encoder = EncoderJson(&os); + registry.Accept(TInstant::Zero(), encoder.Get()); + + UNIT_ASSERT_STRINGS_EQUAL( + os.Str(), + "{\"commonLabels\":{\"common\":\"label\"}," + "\"sensors\":[{\"kind\":\"GAUGE\",\"labels\":{\"my\":\"gauge\"},\"value\":12.34}]}"); + } + } Y_UNIT_TEST(MetricsRegistryClear) { TMetricRegistry registry; @@ -316,4 +316,4 @@ Y_UNIT_TEST_SUITE(TMetricRegistryTest) { UNIT_ASSERT(samples.SamplesSize() == 0); } -} +} diff --git a/library/cpp/monlib/metrics/metric_sub_registry.h b/library/cpp/monlib/metrics/metric_sub_registry.h index 70397623362..e83eeeafb25 100644 --- a/library/cpp/monlib/metrics/metric_sub_registry.h +++ b/library/cpp/monlib/metrics/metric_sub_registry.h @@ -1,116 +1,116 @@ -#pragma once - -#include "metric_registry.h" - -namespace NMonitoring { - -/** - * This registry is wrapping given delegate registry to add common labels - * to all created metrics through this sub registry. - */ -class TMetricSubRegistry final: public IMetricRegistry { -public: - /** - * Do not keep ownership of the given delegate. - */ - TMetricSubRegistry(TLabels commonLabels, IMetricRegistry* delegate) noexcept - : CommonLabels_{std::move(commonLabels)} - , DelegatePtr_{delegate} - { - } - - /** - * Keeps ownership of the given delegate. - */ - TMetricSubRegistry(TLabels commonLabels, std::shared_ptr<IMetricRegistry> delegate) noexcept - : CommonLabels_{std::move(commonLabels)} - , Delegate_{std::move(delegate)} - , DelegatePtr_{Delegate_.get()} - { - } - - IGauge* Gauge(ILabelsPtr labels) override { - AddCommonLabels(labels.Get()); - return DelegatePtr_->Gauge(std::move(labels)); - } - - ILazyGauge* LazyGauge(ILabelsPtr labels, std::function<double()> supplier) override { - AddCommonLabels(labels.Get()); - return DelegatePtr_->LazyGauge(std::move(labels), std::move(supplier)); - } - - IIntGauge* IntGauge(ILabelsPtr labels) override { - AddCommonLabels(labels.Get()); - return DelegatePtr_->IntGauge(std::move(labels)); - } - - ILazyIntGauge* LazyIntGauge(ILabelsPtr labels, std::function<i64()> supplier) override { - AddCommonLabels(labels.Get()); - return DelegatePtr_->LazyIntGauge(std::move(labels), std::move(supplier)); - } - - ICounter* Counter(ILabelsPtr labels) override { - AddCommonLabels(labels.Get()); - return DelegatePtr_->Counter(std::move(labels)); - } - - ILazyCounter* LazyCounter(ILabelsPtr labels, std::function<ui64()> supplier) override { - AddCommonLabels(labels.Get()); - return DelegatePtr_->LazyCounter(std::move(labels), std::move(supplier)); - } - - IRate* Rate(ILabelsPtr labels) override { - AddCommonLabels(labels.Get()); - return DelegatePtr_->Rate(std::move(labels)); - } - - ILazyRate* LazyRate(ILabelsPtr labels, std::function<ui64()> supplier) override { - AddCommonLabels(labels.Get()); - return DelegatePtr_->LazyRate(std::move(labels), std::move(supplier)); - } - - IHistogram* HistogramCounter(ILabelsPtr labels, IHistogramCollectorPtr collector) override { - AddCommonLabels(labels.Get()); - return DelegatePtr_->HistogramCounter(std::move(labels), std::move(collector)); - } - - IHistogram* HistogramRate(ILabelsPtr labels, IHistogramCollectorPtr collector) override { - AddCommonLabels(labels.Get()); - return DelegatePtr_->HistogramRate(std::move(labels), std::move(collector)); - } - - void Accept(TInstant time, IMetricConsumer* consumer) const override { - DelegatePtr_->Accept(time, consumer); - } - - void Append(TInstant time, IMetricConsumer* consumer) const override { - DelegatePtr_->Append(time, consumer); - } - - const TLabels& CommonLabels() const noexcept override { - return CommonLabels_; - } - - void RemoveMetric(const ILabels& labels) noexcept override { - TLabelsImpl<TStringBuf> toRemove; - for (auto& l: labels) { - toRemove.Add(l); - } - AddCommonLabels(&toRemove); - DelegatePtr_->RemoveMetric(toRemove); - } - -private: - void AddCommonLabels(ILabels* labels) const { - for (auto& label: CommonLabels_) { - labels->Add(label); - } - } - -private: - const TLabels CommonLabels_; - std::shared_ptr<IMetricRegistry> Delegate_; - IMetricRegistry* DelegatePtr_; -}; - -} // namespace NMonitoring +#pragma once + +#include "metric_registry.h" + +namespace NMonitoring { + +/** + * This registry is wrapping given delegate registry to add common labels + * to all created metrics through this sub registry. + */ +class TMetricSubRegistry final: public IMetricRegistry { +public: + /** + * Do not keep ownership of the given delegate. + */ + TMetricSubRegistry(TLabels commonLabels, IMetricRegistry* delegate) noexcept + : CommonLabels_{std::move(commonLabels)} + , DelegatePtr_{delegate} + { + } + + /** + * Keeps ownership of the given delegate. + */ + TMetricSubRegistry(TLabels commonLabels, std::shared_ptr<IMetricRegistry> delegate) noexcept + : CommonLabels_{std::move(commonLabels)} + , Delegate_{std::move(delegate)} + , DelegatePtr_{Delegate_.get()} + { + } + + IGauge* Gauge(ILabelsPtr labels) override { + AddCommonLabels(labels.Get()); + return DelegatePtr_->Gauge(std::move(labels)); + } + + ILazyGauge* LazyGauge(ILabelsPtr labels, std::function<double()> supplier) override { + AddCommonLabels(labels.Get()); + return DelegatePtr_->LazyGauge(std::move(labels), std::move(supplier)); + } + + IIntGauge* IntGauge(ILabelsPtr labels) override { + AddCommonLabels(labels.Get()); + return DelegatePtr_->IntGauge(std::move(labels)); + } + + ILazyIntGauge* LazyIntGauge(ILabelsPtr labels, std::function<i64()> supplier) override { + AddCommonLabels(labels.Get()); + return DelegatePtr_->LazyIntGauge(std::move(labels), std::move(supplier)); + } + + ICounter* Counter(ILabelsPtr labels) override { + AddCommonLabels(labels.Get()); + return DelegatePtr_->Counter(std::move(labels)); + } + + ILazyCounter* LazyCounter(ILabelsPtr labels, std::function<ui64()> supplier) override { + AddCommonLabels(labels.Get()); + return DelegatePtr_->LazyCounter(std::move(labels), std::move(supplier)); + } + + IRate* Rate(ILabelsPtr labels) override { + AddCommonLabels(labels.Get()); + return DelegatePtr_->Rate(std::move(labels)); + } + + ILazyRate* LazyRate(ILabelsPtr labels, std::function<ui64()> supplier) override { + AddCommonLabels(labels.Get()); + return DelegatePtr_->LazyRate(std::move(labels), std::move(supplier)); + } + + IHistogram* HistogramCounter(ILabelsPtr labels, IHistogramCollectorPtr collector) override { + AddCommonLabels(labels.Get()); + return DelegatePtr_->HistogramCounter(std::move(labels), std::move(collector)); + } + + IHistogram* HistogramRate(ILabelsPtr labels, IHistogramCollectorPtr collector) override { + AddCommonLabels(labels.Get()); + return DelegatePtr_->HistogramRate(std::move(labels), std::move(collector)); + } + + void Accept(TInstant time, IMetricConsumer* consumer) const override { + DelegatePtr_->Accept(time, consumer); + } + + void Append(TInstant time, IMetricConsumer* consumer) const override { + DelegatePtr_->Append(time, consumer); + } + + const TLabels& CommonLabels() const noexcept override { + return CommonLabels_; + } + + void RemoveMetric(const ILabels& labels) noexcept override { + TLabelsImpl<TStringBuf> toRemove; + for (auto& l: labels) { + toRemove.Add(l); + } + AddCommonLabels(&toRemove); + DelegatePtr_->RemoveMetric(toRemove); + } + +private: + void AddCommonLabels(ILabels* labels) const { + for (auto& label: CommonLabels_) { + labels->Add(label); + } + } + +private: + const TLabels CommonLabels_; + std::shared_ptr<IMetricRegistry> Delegate_; + IMetricRegistry* DelegatePtr_; +}; + +} // namespace NMonitoring diff --git a/library/cpp/monlib/metrics/metric_sub_registry_ut.cpp b/library/cpp/monlib/metrics/metric_sub_registry_ut.cpp index af6d4fa462b..0c5d48b8765 100644 --- a/library/cpp/monlib/metrics/metric_sub_registry_ut.cpp +++ b/library/cpp/monlib/metrics/metric_sub_registry_ut.cpp @@ -1,65 +1,65 @@ -#include "metric_sub_registry.h" - -#include <library/cpp/testing/unittest/registar.h> - -using namespace NMonitoring; - -Y_UNIT_TEST_SUITE(TMetricSubRegistryTest) { - Y_UNIT_TEST(WrapRegistry) { - TMetricRegistry registry; - - { - TMetricSubRegistry subRegistry{{{"common", "label"}}, ®istry}; - IIntGauge* g = subRegistry.IntGauge(MakeLabels({{"my", "gauge"}})); - UNIT_ASSERT(g); - g->Set(42); - } - - TIntGauge* g = registry.IntGauge({{"my", "gauge"}, {"common", "label"}}); - UNIT_ASSERT(g); - UNIT_ASSERT_VALUES_EQUAL(g->Get(), 42); - } - - Y_UNIT_TEST(CommonLabelsDoNotOverrideGeneralLabel) { - TMetricRegistry registry; - - { - TMetricSubRegistry subRegistry{{{"common", "label"}, {"my", "notOverride"}}, ®istry}; - IIntGauge* g = subRegistry.IntGauge(MakeLabels({{"my", "gauge"}})); - UNIT_ASSERT(g); - g->Set(1234); - } - - TIntGauge* knownGauge = registry.IntGauge({{"my", "gauge"}, {"common", "label"}}); - UNIT_ASSERT(knownGauge); - UNIT_ASSERT_VALUES_EQUAL(knownGauge->Get(), 1234); - - TIntGauge* newGauge = registry.IntGauge({{"common", "label"}, {"my", "notOverride"}}); - UNIT_ASSERT(newGauge); - UNIT_ASSERT_VALUES_EQUAL(newGauge->Get(), 0); - } - - Y_UNIT_TEST(RemoveMetric) { - TMetricRegistry registry; - - { - TMetricSubRegistry subRegistry{{{"common", "label"}}, ®istry}; - IIntGauge* g = subRegistry.IntGauge(MakeLabels({{"my", "gauge"}})); - UNIT_ASSERT(g); - g->Set(1234); - } - - IIntGauge* g1 = registry.IntGauge({{"my", "gauge"}, {"common", "label"}}); - UNIT_ASSERT(g1); - UNIT_ASSERT_VALUES_EQUAL(g1->Get(), 1234); - - { - TMetricSubRegistry subRegistry{{{"common", "label"}}, ®istry}; - subRegistry.RemoveMetric(TLabels{{"my", "gauge"}}); - } - - IIntGauge* g2 = registry.IntGauge({{"my", "gauge"}, {"common", "label"}}); - UNIT_ASSERT(g2); - UNIT_ASSERT_VALUES_EQUAL(g2->Get(), 0); - } -} +#include "metric_sub_registry.h" + +#include <library/cpp/testing/unittest/registar.h> + +using namespace NMonitoring; + +Y_UNIT_TEST_SUITE(TMetricSubRegistryTest) { + Y_UNIT_TEST(WrapRegistry) { + TMetricRegistry registry; + + { + TMetricSubRegistry subRegistry{{{"common", "label"}}, ®istry}; + IIntGauge* g = subRegistry.IntGauge(MakeLabels({{"my", "gauge"}})); + UNIT_ASSERT(g); + g->Set(42); + } + + TIntGauge* g = registry.IntGauge({{"my", "gauge"}, {"common", "label"}}); + UNIT_ASSERT(g); + UNIT_ASSERT_VALUES_EQUAL(g->Get(), 42); + } + + Y_UNIT_TEST(CommonLabelsDoNotOverrideGeneralLabel) { + TMetricRegistry registry; + + { + TMetricSubRegistry subRegistry{{{"common", "label"}, {"my", "notOverride"}}, ®istry}; + IIntGauge* g = subRegistry.IntGauge(MakeLabels({{"my", "gauge"}})); + UNIT_ASSERT(g); + g->Set(1234); + } + + TIntGauge* knownGauge = registry.IntGauge({{"my", "gauge"}, {"common", "label"}}); + UNIT_ASSERT(knownGauge); + UNIT_ASSERT_VALUES_EQUAL(knownGauge->Get(), 1234); + + TIntGauge* newGauge = registry.IntGauge({{"common", "label"}, {"my", "notOverride"}}); + UNIT_ASSERT(newGauge); + UNIT_ASSERT_VALUES_EQUAL(newGauge->Get(), 0); + } + + Y_UNIT_TEST(RemoveMetric) { + TMetricRegistry registry; + + { + TMetricSubRegistry subRegistry{{{"common", "label"}}, ®istry}; + IIntGauge* g = subRegistry.IntGauge(MakeLabels({{"my", "gauge"}})); + UNIT_ASSERT(g); + g->Set(1234); + } + + IIntGauge* g1 = registry.IntGauge({{"my", "gauge"}, {"common", "label"}}); + UNIT_ASSERT(g1); + UNIT_ASSERT_VALUES_EQUAL(g1->Get(), 1234); + + { + TMetricSubRegistry subRegistry{{{"common", "label"}}, ®istry}; + subRegistry.RemoveMetric(TLabels{{"my", "gauge"}}); + } + + IIntGauge* g2 = registry.IntGauge({{"my", "gauge"}, {"common", "label"}}); + UNIT_ASSERT(g2); + UNIT_ASSERT_VALUES_EQUAL(g2->Get(), 0); + } +} diff --git a/library/cpp/monlib/metrics/metric_type.cpp b/library/cpp/monlib/metrics/metric_type.cpp index c903ae3dea9..a8a546e843c 100644 --- a/library/cpp/monlib/metrics/metric_type.cpp +++ b/library/cpp/monlib/metrics/metric_type.cpp @@ -1,57 +1,57 @@ -#include "metric_type.h" - -#include <util/generic/strbuf.h> -#include <util/generic/yexception.h> -#include <util/stream/output.h> - -namespace NMonitoring { - TStringBuf MetricTypeToStr(EMetricType type) { - switch (type) { - case EMetricType::GAUGE: +#include "metric_type.h" + +#include <util/generic/strbuf.h> +#include <util/generic/yexception.h> +#include <util/stream/output.h> + +namespace NMonitoring { + TStringBuf MetricTypeToStr(EMetricType type) { + switch (type) { + case EMetricType::GAUGE: return TStringBuf("GAUGE"); - case EMetricType::COUNTER: + case EMetricType::COUNTER: return TStringBuf("COUNTER"); - case EMetricType::RATE: + case EMetricType::RATE: return TStringBuf("RATE"); - case EMetricType::IGAUGE: + case EMetricType::IGAUGE: return TStringBuf("IGAUGE"); - case EMetricType::HIST: + case EMetricType::HIST: return TStringBuf("HIST"); - case EMetricType::HIST_RATE: + case EMetricType::HIST_RATE: return TStringBuf("HIST_RATE"); - case EMetricType::DSUMMARY: + case EMetricType::DSUMMARY: return TStringBuf("DSUMMARY"); case EMetricType::LOGHIST: return TStringBuf("LOGHIST"); - default: + default: return TStringBuf("UNKNOWN"); - } - } - - EMetricType MetricTypeFromStr(TStringBuf str) { + } + } + + EMetricType MetricTypeFromStr(TStringBuf str) { if (str == TStringBuf("GAUGE") || str == TStringBuf("DGAUGE")) { - return EMetricType::GAUGE; + return EMetricType::GAUGE; } else if (str == TStringBuf("COUNTER")) { - return EMetricType::COUNTER; + return EMetricType::COUNTER; } else if (str == TStringBuf("RATE")) { - return EMetricType::RATE; + return EMetricType::RATE; } else if (str == TStringBuf("IGAUGE")) { - return EMetricType::IGAUGE; + return EMetricType::IGAUGE; } else if (str == TStringBuf("HIST")) { - return EMetricType::HIST; + return EMetricType::HIST; } else if (str == TStringBuf("HIST_RATE")) { - return EMetricType::HIST_RATE; + return EMetricType::HIST_RATE; } else if (str == TStringBuf("DSUMMARY")) { - return EMetricType::DSUMMARY; + return EMetricType::DSUMMARY; } else if (str == TStringBuf("LOGHIST")) { return EMetricType::LOGHIST; - } else { - ythrow yexception() << "unknown metric type: " << str; - } - } -} - -template <> -void Out<NMonitoring::EMetricType>(IOutputStream& o, NMonitoring::EMetricType t) { - o << NMonitoring::MetricTypeToStr(t); -} + } else { + ythrow yexception() << "unknown metric type: " << str; + } + } +} + +template <> +void Out<NMonitoring::EMetricType>(IOutputStream& o, NMonitoring::EMetricType t) { + o << NMonitoring::MetricTypeToStr(t); +} diff --git a/library/cpp/monlib/metrics/metric_type.h b/library/cpp/monlib/metrics/metric_type.h index 00a8187af18..1984c42c1e0 100644 --- a/library/cpp/monlib/metrics/metric_type.h +++ b/library/cpp/monlib/metrics/metric_type.h @@ -1,25 +1,25 @@ -#pragma once - -#include <util/generic/fwd.h> - -namespace NMonitoring { - - constexpr ui32 MaxMetricTypeNameLength = 9; - - enum class EMetricType { - UNKNOWN = 0, - GAUGE = 1, - COUNTER = 2, - RATE = 3, - IGAUGE = 4, - HIST = 5, - HIST_RATE = 6, +#pragma once + +#include <util/generic/fwd.h> + +namespace NMonitoring { + + constexpr ui32 MaxMetricTypeNameLength = 9; + + enum class EMetricType { + UNKNOWN = 0, + GAUGE = 1, + COUNTER = 2, + RATE = 3, + IGAUGE = 4, + HIST = 5, + HIST_RATE = 6, DSUMMARY = 7, // ISUMMARY = 8, reserved LOGHIST = 9, - }; - - TStringBuf MetricTypeToStr(EMetricType type); - EMetricType MetricTypeFromStr(TStringBuf str); - -} + }; + + TStringBuf MetricTypeToStr(EMetricType type); + EMetricType MetricTypeFromStr(TStringBuf str); + +} diff --git a/library/cpp/monlib/metrics/metric_value.cpp b/library/cpp/monlib/metrics/metric_value.cpp index 1ea2ed41422..b95d7011c60 100644 --- a/library/cpp/monlib/metrics/metric_value.cpp +++ b/library/cpp/monlib/metrics/metric_value.cpp @@ -1,27 +1,27 @@ -#include "metric_value.h" - - -namespace NMonitoring { - void TMetricTimeSeries::SortByTs() { +#include "metric_value.h" + + +namespace NMonitoring { + void TMetricTimeSeries::SortByTs() { SortPointsByTs(ValueType_, Points_); - } - - void TMetricTimeSeries::Clear() noexcept { - if (ValueType_ == EMetricValueType::HISTOGRAM) { - for (TPoint& p: Points_) { - SnapshotUnRef<EMetricValueType::HISTOGRAM>(p); - } - } else if (ValueType_ == EMetricValueType::SUMMARY) { + } + + void TMetricTimeSeries::Clear() noexcept { + if (ValueType_ == EMetricValueType::HISTOGRAM) { + for (TPoint& p: Points_) { + SnapshotUnRef<EMetricValueType::HISTOGRAM>(p); + } + } else if (ValueType_ == EMetricValueType::SUMMARY) { for (TPoint& p: Points_) { - SnapshotUnRef<EMetricValueType::SUMMARY>(p); + SnapshotUnRef<EMetricValueType::SUMMARY>(p); } } else if (ValueType_ == EMetricValueType::LOGHISTOGRAM) { for (TPoint& p: Points_) { SnapshotUnRef<EMetricValueType::LOGHISTOGRAM>(p); } - } + } - Points_.clear(); - ValueType_ = EMetricValueType::UNKNOWN; - } -} + Points_.clear(); + ValueType_ = EMetricValueType::UNKNOWN; + } +} diff --git a/library/cpp/monlib/metrics/metric_value.h b/library/cpp/monlib/metrics/metric_value.h index d7b2f6a6808..607fcc86022 100644 --- a/library/cpp/monlib/metrics/metric_value.h +++ b/library/cpp/monlib/metrics/metric_value.h @@ -1,17 +1,17 @@ -#pragma once - -#include "histogram_collector.h" -#include "metric_value_type.h" +#pragma once + +#include "histogram_collector.h" +#include "metric_value_type.h" #include "summary_collector.h" #include "log_histogram_snapshot.h" - -#include <util/datetime/base.h> -#include <util/generic/algorithm.h> -#include <util/generic/vector.h> + +#include <util/datetime/base.h> +#include <util/generic/algorithm.h> +#include <util/generic/vector.h> #include <util/generic/cast.h> #include <util/generic/ymath.h> - -namespace NMonitoring { + +namespace NMonitoring { namespace NPrivate { template <typename T> T FromFloatSafe(double d) { @@ -25,71 +25,71 @@ namespace NMonitoring { }; } // namespace NPrivate - template <typename T, typename Enable = void> - struct TValueType; - - template <> - struct TValueType<double> { - static constexpr auto Type = EMetricValueType::DOUBLE; - }; - - template <> - struct TValueType<i64> { - static constexpr auto Type = EMetricValueType::INT64; - }; - - template <> - struct TValueType<ui64> { - static constexpr auto Type = EMetricValueType::UINT64; - }; - + template <typename T, typename Enable = void> + struct TValueType; + + template <> + struct TValueType<double> { + static constexpr auto Type = EMetricValueType::DOUBLE; + }; + + template <> + struct TValueType<i64> { + static constexpr auto Type = EMetricValueType::INT64; + }; + + template <> + struct TValueType<ui64> { + static constexpr auto Type = EMetricValueType::UINT64; + }; + template <> struct TValueType<TLogHistogramSnapshot*> { static constexpr auto Type = EMetricValueType::LOGHISTOGRAM; }; - template <typename T> - struct TValueType<T*, typename std::enable_if_t<std::is_base_of<IHistogramSnapshot, T>::value>> { - static constexpr auto Type = EMetricValueType::HISTOGRAM; - }; - + template <typename T> + struct TValueType<T*, typename std::enable_if_t<std::is_base_of<IHistogramSnapshot, T>::value>> { + static constexpr auto Type = EMetricValueType::HISTOGRAM; + }; + template <typename T> struct TValueType<T*, typename std::enable_if_t<std::is_base_of<ISummaryDoubleSnapshot, T>::value>> { - static constexpr auto Type = EMetricValueType::SUMMARY; + static constexpr auto Type = EMetricValueType::SUMMARY; }; - - /////////////////////////////////////////////////////////////////////////// - // TMetricValue - /////////////////////////////////////////////////////////////////////////// - // TMetricValue represents a generic value. It does not contain type + + /////////////////////////////////////////////////////////////////////////// + // TMetricValue + /////////////////////////////////////////////////////////////////////////// + // TMetricValue represents a generic value. It does not contain type // information about a value. This is done to minimize object footprint. // To read an actual value from the object the type must be checked // first or provided to AsXxxx(type) member-functions. // This class does not hold an ownership of an IHistogramSnapshot or // SummarySnapshot, so this must be done somewhere outside. - class TMetricValue { - public: - TMetricValue() noexcept { - Value_.Uint64 = 0; - } - - explicit TMetricValue(double value) noexcept { - Value_.Double = value; - } - - explicit TMetricValue(i64 value) noexcept { - Value_.Int64 = value; - } - - explicit TMetricValue(ui64 value) noexcept { - Value_.Uint64 = value; - } - - explicit TMetricValue(IHistogramSnapshot* histogram) noexcept { - Value_.Histogram = histogram; - } - - explicit TMetricValue(ISummaryDoubleSnapshot* summary) noexcept { + class TMetricValue { + public: + TMetricValue() noexcept { + Value_.Uint64 = 0; + } + + explicit TMetricValue(double value) noexcept { + Value_.Double = value; + } + + explicit TMetricValue(i64 value) noexcept { + Value_.Int64 = value; + } + + explicit TMetricValue(ui64 value) noexcept { + Value_.Uint64 = value; + } + + explicit TMetricValue(IHistogramSnapshot* histogram) noexcept { + Value_.Histogram = histogram; + } + + explicit TMetricValue(ISummaryDoubleSnapshot* summary) noexcept { Value_.Summary = summary; } @@ -97,90 +97,90 @@ namespace NMonitoring { Value_.LogHistogram = logHist; } - double AsDouble() const noexcept { - return Value_.Double; - } - - // will cast value into double, current value type is determined by - // the given type argument - double AsDouble(EMetricValueType type) const { - switch (type) { - case EMetricValueType::DOUBLE: - return Value_.Double; - case EMetricValueType::INT64: - return static_cast<double>(Value_.Int64); - case EMetricValueType::UINT64: - return static_cast<double>(Value_.Uint64); - case EMetricValueType::HISTOGRAM: - ythrow yexception() << "histogram cannot be casted to Double"; - case EMetricValueType::SUMMARY: + double AsDouble() const noexcept { + return Value_.Double; + } + + // will cast value into double, current value type is determined by + // the given type argument + double AsDouble(EMetricValueType type) const { + switch (type) { + case EMetricValueType::DOUBLE: + return Value_.Double; + case EMetricValueType::INT64: + return static_cast<double>(Value_.Int64); + case EMetricValueType::UINT64: + return static_cast<double>(Value_.Uint64); + case EMetricValueType::HISTOGRAM: + ythrow yexception() << "histogram cannot be casted to Double"; + case EMetricValueType::SUMMARY: ythrow yexception() << "summary cannot be casted to Double"; case EMetricValueType::LOGHISTOGRAM: ythrow yexception() << "loghistogram cannot be casted to Double"; - case EMetricValueType::UNKNOWN: - ythrow yexception() << "unknown value type"; - } - Y_FAIL(); // for GCC - } - - ui64 AsUint64() const noexcept { - return Value_.Uint64; - } - - // will cast value into uint64, current value's type is determined by - // the given type argument - ui64 AsUint64(EMetricValueType type) const { - switch (type) { - case EMetricValueType::DOUBLE: + case EMetricValueType::UNKNOWN: + ythrow yexception() << "unknown value type"; + } + Y_FAIL(); // for GCC + } + + ui64 AsUint64() const noexcept { + return Value_.Uint64; + } + + // will cast value into uint64, current value's type is determined by + // the given type argument + ui64 AsUint64(EMetricValueType type) const { + switch (type) { + case EMetricValueType::DOUBLE: return NPrivate::FromFloatSafe<ui64>(Value_.Double); - case EMetricValueType::INT64: + case EMetricValueType::INT64: return SafeIntegerCast<ui64>(Value_.Int64); - case EMetricValueType::UINT64: - return Value_.Uint64; - case EMetricValueType::HISTOGRAM: - ythrow yexception() << "histogram cannot be casted to Uint64"; - case EMetricValueType::SUMMARY: + case EMetricValueType::UINT64: + return Value_.Uint64; + case EMetricValueType::HISTOGRAM: + ythrow yexception() << "histogram cannot be casted to Uint64"; + case EMetricValueType::SUMMARY: ythrow yexception() << "summary cannot be casted to Uint64"; case EMetricValueType::LOGHISTOGRAM: ythrow yexception() << "loghistogram cannot be casted to Uint64"; - case EMetricValueType::UNKNOWN: - ythrow yexception() << "unknown value type"; - } - Y_FAIL(); // for GCC - } - - i64 AsInt64() const noexcept { - return Value_.Int64; - } - - // will cast value into int64, current value's type is determined by - // the given type argument - i64 AsInt64(EMetricValueType type) const { - switch (type) { - case EMetricValueType::DOUBLE: + case EMetricValueType::UNKNOWN: + ythrow yexception() << "unknown value type"; + } + Y_FAIL(); // for GCC + } + + i64 AsInt64() const noexcept { + return Value_.Int64; + } + + // will cast value into int64, current value's type is determined by + // the given type argument + i64 AsInt64(EMetricValueType type) const { + switch (type) { + case EMetricValueType::DOUBLE: return NPrivate::FromFloatSafe<i64>(Value_.Double); - case EMetricValueType::INT64: - return Value_.Int64; - case EMetricValueType::UINT64: + case EMetricValueType::INT64: + return Value_.Int64; + case EMetricValueType::UINT64: return SafeIntegerCast<i64>(Value_.Uint64); - case EMetricValueType::HISTOGRAM: - ythrow yexception() << "histogram cannot be casted to Int64"; - case EMetricValueType::SUMMARY: + case EMetricValueType::HISTOGRAM: + ythrow yexception() << "histogram cannot be casted to Int64"; + case EMetricValueType::SUMMARY: ythrow yexception() << "summary cannot be casted to Int64"; case EMetricValueType::LOGHISTOGRAM: ythrow yexception() << "loghistogram cannot be casted to Int64"; - case EMetricValueType::UNKNOWN: - ythrow yexception() << "unknown value type"; - } - Y_FAIL(); // for GCC - } - - IHistogramSnapshot* AsHistogram() const noexcept { - return Value_.Histogram; - } - - IHistogramSnapshot* AsHistogram(EMetricValueType type) const { - if (type != EMetricValueType::HISTOGRAM) { + case EMetricValueType::UNKNOWN: + ythrow yexception() << "unknown value type"; + } + Y_FAIL(); // for GCC + } + + IHistogramSnapshot* AsHistogram() const noexcept { + return Value_.Histogram; + } + + IHistogramSnapshot* AsHistogram(EMetricValueType type) const { + if (type != EMetricValueType::HISTOGRAM) { ythrow yexception() << type << " cannot be casted to Histogram"; } @@ -191,8 +191,8 @@ namespace NMonitoring { return Value_.Summary; } - ISummaryDoubleSnapshot* AsSummaryDouble(EMetricValueType type) const { - if (type != EMetricValueType::SUMMARY) { + ISummaryDoubleSnapshot* AsSummaryDouble(EMetricValueType type) const { + if (type != EMetricValueType::SUMMARY) { ythrow yexception() << type << " cannot be casted to SummaryDouble"; } @@ -211,35 +211,35 @@ namespace NMonitoring { return Value_.LogHistogram; } - protected: - union { - double Double; - i64 Int64; - ui64 Uint64; - IHistogramSnapshot* Histogram; + protected: + union { + double Double; + i64 Int64; + ui64 Uint64; + IHistogramSnapshot* Histogram; ISummaryDoubleSnapshot* Summary; TLogHistogramSnapshot* LogHistogram; - } Value_; - }; - + } Value_; + }; + /////////////////////////////////////////////////////////////////////////// - // TMetricValueWithType + // TMetricValueWithType /////////////////////////////////////////////////////////////////////////// - // Same as TMetricValue, but this type holds an ownership of + // Same as TMetricValue, but this type holds an ownership of // snapshots and contains value type information. - class TMetricValueWithType: private TMetricValue, public TMoveOnly { + class TMetricValueWithType: private TMetricValue, public TMoveOnly { public: - using TBase = TMetricValue; + using TBase = TMetricValue; template <typename T> - explicit TMetricValueWithType(T value) + explicit TMetricValueWithType(T value) : TBase(value) , ValueType_{TValueType<T>::Type} { Ref(); } - TMetricValueWithType(TMetricValueWithType&& other) + TMetricValueWithType(TMetricValueWithType&& other) : TBase(std::move(other)) , ValueType_{other.ValueType_} { @@ -247,7 +247,7 @@ namespace NMonitoring { other.Clear(); } - TMetricValueWithType& operator=(TMetricValueWithType&& other) { + TMetricValueWithType& operator=(TMetricValueWithType&& other) { TBase::operator=(other); ValueType_ = other.ValueType_; @@ -257,16 +257,16 @@ namespace NMonitoring { return *this; } - ~TMetricValueWithType() { + ~TMetricValueWithType() { UnRef(); } void Clear() { UnRef(); - ValueType_ = EMetricValueType::UNKNOWN; + ValueType_ = EMetricValueType::UNKNOWN; } - EMetricValueType GetType() const noexcept { + EMetricValueType GetType() const noexcept { return ValueType_; } @@ -296,9 +296,9 @@ namespace NMonitoring { private: void Ref() { - if (ValueType_ == EMetricValueType::SUMMARY) { + if (ValueType_ == EMetricValueType::SUMMARY) { TBase::AsSummaryDouble()->Ref(); - } else if (ValueType_ == EMetricValueType::HISTOGRAM) { + } else if (ValueType_ == EMetricValueType::HISTOGRAM) { TBase::AsHistogram()->Ref(); } else if (ValueType_ == EMetricValueType::LOGHISTOGRAM) { TBase::AsLogHistogram()->Ref(); @@ -306,9 +306,9 @@ namespace NMonitoring { } void UnRef() { - if (ValueType_ == EMetricValueType::SUMMARY) { + if (ValueType_ == EMetricValueType::SUMMARY) { TBase::AsSummaryDouble()->UnRef(); - } else if (ValueType_ == EMetricValueType::HISTOGRAM) { + } else if (ValueType_ == EMetricValueType::HISTOGRAM) { TBase::AsHistogram()->UnRef(); } else if (ValueType_ == EMetricValueType::LOGHISTOGRAM) { TBase::AsLogHistogram()->UnRef(); @@ -316,114 +316,114 @@ namespace NMonitoring { } private: - EMetricValueType ValueType_ = EMetricValueType::UNKNOWN; + EMetricValueType ValueType_ = EMetricValueType::UNKNOWN; }; - static_assert(sizeof(TMetricValue) == sizeof(ui64), - "expected size of TMetricValue is one machine word"); - - /////////////////////////////////////////////////////////////////////////// - // TMetricTimeSeries - /////////////////////////////////////////////////////////////////////////// - class TMetricTimeSeries: private TMoveOnly { - public: - class TPoint { - public: - TPoint() - : Time_(TInstant::Zero()) - { - } - - template <typename T> - TPoint(TInstant time, T value) - : Time_(time) - , Value_(value) - { - } - - TInstant GetTime() const noexcept { - return Time_; - } - - TMetricValue GetValue() const noexcept { - return Value_; - } - - void ClearValue() { - Value_ = {}; - } - - private: - TInstant Time_; - TMetricValue Value_; - }; - - public: - TMetricTimeSeries() = default; - - TMetricTimeSeries(TMetricTimeSeries&& rhs) noexcept - : ValueType_(rhs.ValueType_) - , Points_(std::move(rhs.Points_)) - { - rhs.ValueType_ = EMetricValueType::UNKNOWN; - } - - TMetricTimeSeries& operator=(TMetricTimeSeries&& rhs) noexcept { - Clear(); - - ValueType_ = rhs.ValueType_; - rhs.ValueType_ = EMetricValueType::UNKNOWN; - - Points_ = std::move(rhs.Points_); - return *this; - } - - ~TMetricTimeSeries() { - Clear(); - } - - template <typename T> - void Add(TInstant time, T value) { + static_assert(sizeof(TMetricValue) == sizeof(ui64), + "expected size of TMetricValue is one machine word"); + + /////////////////////////////////////////////////////////////////////////// + // TMetricTimeSeries + /////////////////////////////////////////////////////////////////////////// + class TMetricTimeSeries: private TMoveOnly { + public: + class TPoint { + public: + TPoint() + : Time_(TInstant::Zero()) + { + } + + template <typename T> + TPoint(TInstant time, T value) + : Time_(time) + , Value_(value) + { + } + + TInstant GetTime() const noexcept { + return Time_; + } + + TMetricValue GetValue() const noexcept { + return Value_; + } + + void ClearValue() { + Value_ = {}; + } + + private: + TInstant Time_; + TMetricValue Value_; + }; + + public: + TMetricTimeSeries() = default; + + TMetricTimeSeries(TMetricTimeSeries&& rhs) noexcept + : ValueType_(rhs.ValueType_) + , Points_(std::move(rhs.Points_)) + { + rhs.ValueType_ = EMetricValueType::UNKNOWN; + } + + TMetricTimeSeries& operator=(TMetricTimeSeries&& rhs) noexcept { + Clear(); + + ValueType_ = rhs.ValueType_; + rhs.ValueType_ = EMetricValueType::UNKNOWN; + + Points_ = std::move(rhs.Points_); + return *this; + } + + ~TMetricTimeSeries() { + Clear(); + } + + template <typename T> + void Add(TInstant time, T value) { Add(TPoint(time, value), TValueType<T>::Type); } void Add(TPoint point, EMetricValueType valueType) { - if (Empty()) { + if (Empty()) { ValueType_ = valueType; - } else { + } else { CheckTypes(ValueType_, valueType); - } + } Points_.push_back(point); - - if (ValueType_ == EMetricValueType::SUMMARY) { + + if (ValueType_ == EMetricValueType::SUMMARY) { TPoint& p = Points_.back(); p.GetValue().AsSummaryDouble()->Ref(); - } else if (ValueType_ == EMetricValueType::HISTOGRAM) { + } else if (ValueType_ == EMetricValueType::HISTOGRAM) { TPoint& p = Points_.back(); p.GetValue().AsHistogram()->Ref(); } else if (ValueType_ == EMetricValueType::LOGHISTOGRAM) { TPoint& p = Points_.back(); p.GetValue().AsLogHistogram()->Ref(); - } - } - - void CopyFrom(const TMetricTimeSeries& other) { + } + } + + void CopyFrom(const TMetricTimeSeries& other) { if (Empty()) { ValueType_ = other.ValueType_; } else { CheckTypes(GetValueType(), other.GetValueType()); } - - size_t prevSize = Points_.size(); - Copy(std::begin(other.Points_), std::end(other.Points_), - std::back_inserter(Points_)); - - if (ValueType_ == EMetricValueType::HISTOGRAM) { - for (size_t i = prevSize; i < Points_.size(); i++) { - TPoint& point = Points_[i]; - point.GetValue().AsHistogram()->Ref(); - } - } else if (ValueType_ == EMetricValueType::SUMMARY) { + + size_t prevSize = Points_.size(); + Copy(std::begin(other.Points_), std::end(other.Points_), + std::back_inserter(Points_)); + + if (ValueType_ == EMetricValueType::HISTOGRAM) { + for (size_t i = prevSize; i < Points_.size(); i++) { + TPoint& point = Points_[i]; + point.GetValue().AsHistogram()->Ref(); + } + } else if (ValueType_ == EMetricValueType::SUMMARY) { for (size_t i = prevSize; i < Points_.size(); ++i) { TPoint& point = Points_[i]; point.GetValue().AsSummaryDouble()->Ref(); @@ -436,56 +436,56 @@ namespace NMonitoring { } } - template <typename TConsumer> - void ForEach(TConsumer c) const { - for (const auto& point : Points_) { - c(point.GetTime(), ValueType_, point.GetValue()); - } - } - - bool Empty() const noexcept { - return Points_.empty(); - } - - size_t Size() const noexcept { - return Points_.size(); - } - + template <typename TConsumer> + void ForEach(TConsumer c) const { + for (const auto& point : Points_) { + c(point.GetTime(), ValueType_, point.GetValue()); + } + } + + bool Empty() const noexcept { + return Points_.empty(); + } + + size_t Size() const noexcept { + return Points_.size(); + } + size_t Capacity() const noexcept { return Points_.capacity(); } - const TPoint& operator[](size_t index) const noexcept { - return Points_[index]; - } - - void SortByTs(); - - void Clear() noexcept; - - EMetricValueType GetValueType() const noexcept { - return ValueType_; - } - - private: - static void CheckTypes(EMetricValueType t1, EMetricValueType t2) { - Y_ENSURE(t1 == t2, + const TPoint& operator[](size_t index) const noexcept { + return Points_[index]; + } + + void SortByTs(); + + void Clear() noexcept; + + EMetricValueType GetValueType() const noexcept { + return ValueType_; + } + + private: + static void CheckTypes(EMetricValueType t1, EMetricValueType t2) { + Y_ENSURE(t1 == t2, "Series type mismatch: expected " << t1 << ", but got " << t2); - } - - private: - EMetricValueType ValueType_ = EMetricValueType::UNKNOWN; - TVector<TPoint> Points_; - }; - - template <EMetricValueType valueType, typename TPoint> + } + + private: + EMetricValueType ValueType_ = EMetricValueType::UNKNOWN; + TVector<TPoint> Points_; + }; + + template <EMetricValueType valueType, typename TPoint> static inline void SnapshotUnRef(TPoint& point) { - if constexpr (valueType == EMetricValueType::HISTOGRAM) { + if constexpr (valueType == EMetricValueType::HISTOGRAM) { if (auto* hist = point.GetValue().AsHistogram()) { hist->UnRef(); } - } else if constexpr (valueType == EMetricValueType::SUMMARY) { + } else if constexpr (valueType == EMetricValueType::SUMMARY) { if (auto* summary = point.GetValue().AsSummaryDouble()) { summary->UnRef(); } @@ -496,14 +496,14 @@ namespace NMonitoring { } } - template <EMetricValueType valueType, typename TPoint> + template <EMetricValueType valueType, typename TPoint> static void EraseDuplicates(TVector<TPoint>& points) { // we have to manually clean reference to a snapshot from point // while removing duplicates auto result = points.rbegin(); for (auto it = result + 1; it != points.rend(); ++it) { if (result->GetTime() != it->GetTime() && ++result != it) { - SnapshotUnRef<valueType>(*result); + SnapshotUnRef<valueType>(*result); *result = *it; // (2) copy it->ClearValue(); // (3) clean pointer in the source } @@ -511,13 +511,13 @@ namespace NMonitoring { // erase tail points for (auto it = result + 1; it != points.rend(); ++it) { - SnapshotUnRef<valueType>(*it); + SnapshotUnRef<valueType>(*it); } points.erase(points.begin(), (result + 1).base()); } template <typename TPoint> - void SortPointsByTs(EMetricValueType valueType, TVector<TPoint>& points) { + void SortPointsByTs(EMetricValueType valueType, TVector<TPoint>& points) { if (points.size() < 2) { return; } @@ -530,13 +530,13 @@ namespace NMonitoring { points.erase(points.begin(), it.base()); } else { StableSortBy(points, NPrivate::POINT_KEY_FN); - if (valueType == EMetricValueType::HISTOGRAM) { - EraseDuplicates<EMetricValueType::HISTOGRAM>(points); + if (valueType == EMetricValueType::HISTOGRAM) { + EraseDuplicates<EMetricValueType::HISTOGRAM>(points); } else if (valueType == EMetricValueType::LOGHISTOGRAM) { EraseDuplicates<EMetricValueType::LOGHISTOGRAM>(points); } else { - EraseDuplicates<EMetricValueType::SUMMARY>(points); + EraseDuplicates<EMetricValueType::SUMMARY>(points); } } } -} +} diff --git a/library/cpp/monlib/metrics/metric_value_type.h b/library/cpp/monlib/metrics/metric_value_type.h index b17d1ca04a9..ab30a958c25 100644 --- a/library/cpp/monlib/metrics/metric_value_type.h +++ b/library/cpp/monlib/metrics/metric_value_type.h @@ -3,7 +3,7 @@ namespace NMonitoring { -enum class EMetricValueType { +enum class EMetricValueType { UNKNOWN, DOUBLE, INT64, diff --git a/library/cpp/monlib/metrics/metric_value_ut.cpp b/library/cpp/monlib/metrics/metric_value_ut.cpp index 6db9c583ad3..49b47c40574 100644 --- a/library/cpp/monlib/metrics/metric_value_ut.cpp +++ b/library/cpp/monlib/metrics/metric_value_ut.cpp @@ -1,33 +1,33 @@ -#include "metric_value.h" - +#include "metric_value.h" + #include <library/cpp/testing/unittest/registar.h> - -using namespace NMonitoring; - -Y_UNIT_TEST_SUITE(TMetricValueTest) { - - class TTestHistogram: public IHistogramSnapshot { + +using namespace NMonitoring; + +Y_UNIT_TEST_SUITE(TMetricValueTest) { + + class TTestHistogram: public IHistogramSnapshot { public: TTestHistogram(ui32 count = 1) : Count_{count} {} private: - ui32 Count() const override { + ui32 Count() const override { return Count_; - } - - TBucketBound UpperBound(ui32 /*index*/) const override { - return 1234.56; - } - - TBucketValue Value(ui32 /*index*/) const override { - return 42; - } + } + + TBucketBound UpperBound(ui32 /*index*/) const override { + return 1234.56; + } + + TBucketValue Value(ui32 /*index*/) const override { + return 42; + } ui32 Count_{0}; - }; - + }; + IHistogramSnapshotPtr MakeHistogramSnapshot() { return MakeIntrusive<TTestHistogram>(); } @@ -44,46 +44,46 @@ Y_UNIT_TEST_SUITE(TMetricValueTest) { return MakeIntrusive<TLogHistogramSnapshot>(1.5, 0u, 0, buckets); } - Y_UNIT_TEST(Sorted) { - auto ts1 = TInstant::Now(); - auto ts2 = ts1 + TDuration::Seconds(1); - - TMetricTimeSeries timeSeries; - timeSeries.Add(ts1, 3.14159); - timeSeries.Add(ts1, 6.28318); - timeSeries.Add(ts2, 2.71828); - - UNIT_ASSERT_EQUAL(timeSeries.Size(), 3); - - timeSeries.SortByTs(); - UNIT_ASSERT_EQUAL(timeSeries.Size(), 2); - - UNIT_ASSERT_EQUAL(ts1, timeSeries[0].GetTime()); + Y_UNIT_TEST(Sorted) { + auto ts1 = TInstant::Now(); + auto ts2 = ts1 + TDuration::Seconds(1); + + TMetricTimeSeries timeSeries; + timeSeries.Add(ts1, 3.14159); + timeSeries.Add(ts1, 6.28318); + timeSeries.Add(ts2, 2.71828); + + UNIT_ASSERT_EQUAL(timeSeries.Size(), 3); + + timeSeries.SortByTs(); + UNIT_ASSERT_EQUAL(timeSeries.Size(), 2); + + UNIT_ASSERT_EQUAL(ts1, timeSeries[0].GetTime()); UNIT_ASSERT_DOUBLES_EQUAL(6.28318, timeSeries[0].GetValue().AsDouble(), Min<double>()); - - UNIT_ASSERT_EQUAL(ts2, timeSeries[1].GetTime()); - UNIT_ASSERT_DOUBLES_EQUAL(2.71828, timeSeries[1].GetValue().AsDouble(), Min<double>()); - } - - Y_UNIT_TEST(Histograms) { - auto ts = TInstant::Now(); - auto histogram = MakeIntrusive<TTestHistogram>(); - - UNIT_ASSERT_VALUES_EQUAL(1, histogram->RefCount()); - { - TMetricTimeSeries timeSeries; - timeSeries.Add(ts, histogram.Get()); - UNIT_ASSERT_VALUES_EQUAL(2, histogram->RefCount()); - } - UNIT_ASSERT_VALUES_EQUAL(1, histogram->RefCount()); - } - + + UNIT_ASSERT_EQUAL(ts2, timeSeries[1].GetTime()); + UNIT_ASSERT_DOUBLES_EQUAL(2.71828, timeSeries[1].GetValue().AsDouble(), Min<double>()); + } + + Y_UNIT_TEST(Histograms) { + auto ts = TInstant::Now(); + auto histogram = MakeIntrusive<TTestHistogram>(); + + UNIT_ASSERT_VALUES_EQUAL(1, histogram->RefCount()); + { + TMetricTimeSeries timeSeries; + timeSeries.Add(ts, histogram.Get()); + UNIT_ASSERT_VALUES_EQUAL(2, histogram->RefCount()); + } + UNIT_ASSERT_VALUES_EQUAL(1, histogram->RefCount()); + } + Y_UNIT_TEST(Summary) { auto ts = TInstant::Now(); auto summary = MakeSummarySnapshot(); UNIT_ASSERT_VALUES_EQUAL(1, summary->RefCount()); { - TMetricTimeSeries timeSeries; + TMetricTimeSeries timeSeries; timeSeries.Add(ts, summary.Get()); UNIT_ASSERT_VALUES_EQUAL(2, summary->RefCount()); } @@ -102,73 +102,73 @@ Y_UNIT_TEST_SUITE(TMetricValueTest) { UNIT_ASSERT_VALUES_EQUAL(1, logHist->RefCount()); } - Y_UNIT_TEST(TimeSeriesMovable) { - auto ts = TInstant::Now(); - auto histogram = MakeIntrusive<TTestHistogram>(); - - UNIT_ASSERT_VALUES_EQUAL(1, histogram->RefCount()); - { - TMetricTimeSeries timeSeriesA; - timeSeriesA.Add(ts, histogram.Get()); - UNIT_ASSERT_VALUES_EQUAL(2, histogram->RefCount()); - - TMetricTimeSeries timeSeriesB = std::move(timeSeriesA); - UNIT_ASSERT_VALUES_EQUAL(2, histogram->RefCount()); - - UNIT_ASSERT_VALUES_EQUAL(1, timeSeriesB.Size()); - UNIT_ASSERT_EQUAL(EMetricValueType::HISTOGRAM, timeSeriesB.GetValueType()); - - UNIT_ASSERT_VALUES_EQUAL(0, timeSeriesA.Size()); - UNIT_ASSERT_EQUAL(EMetricValueType::UNKNOWN, timeSeriesA.GetValueType()); - } - UNIT_ASSERT_VALUES_EQUAL(1, histogram->RefCount()); - } - - Y_UNIT_TEST(HistogramsUnique) { - auto ts1 = TInstant::Now(); - auto ts2 = ts1 + TDuration::Seconds(1); - auto ts3 = ts2 + TDuration::Seconds(1); - - auto h1 = MakeIntrusive<TTestHistogram>(); - auto h2 = MakeIntrusive<TTestHistogram>(); - auto h3 = MakeIntrusive<TTestHistogram>(); - - UNIT_ASSERT_VALUES_EQUAL(1, h1->RefCount()); - UNIT_ASSERT_VALUES_EQUAL(1, h2->RefCount()); - UNIT_ASSERT_VALUES_EQUAL(1, h3->RefCount()); - - { - TMetricTimeSeries timeSeries; - timeSeries.Add(ts1, h1.Get()); // drop at the head - timeSeries.Add(ts1, h1.Get()); - timeSeries.Add(ts1, h1.Get()); - - timeSeries.Add(ts2, h2.Get()); // drop in the middle - timeSeries.Add(ts2, h2.Get()); - timeSeries.Add(ts2, h2.Get()); - - timeSeries.Add(ts3, h3.Get()); // drop at the end - timeSeries.Add(ts3, h3.Get()); - timeSeries.Add(ts3, h3.Get()); - - UNIT_ASSERT_EQUAL(timeSeries.Size(), 9); - - UNIT_ASSERT_VALUES_EQUAL(4, h1->RefCount()); - UNIT_ASSERT_VALUES_EQUAL(4, h2->RefCount()); - UNIT_ASSERT_VALUES_EQUAL(4, h3->RefCount()); - - timeSeries.SortByTs(); - UNIT_ASSERT_EQUAL(timeSeries.Size(), 3); - - UNIT_ASSERT_VALUES_EQUAL(2, h1->RefCount()); - UNIT_ASSERT_VALUES_EQUAL(2, h2->RefCount()); - UNIT_ASSERT_VALUES_EQUAL(2, h3->RefCount()); - } - - UNIT_ASSERT_VALUES_EQUAL(1, h1->RefCount()); - UNIT_ASSERT_VALUES_EQUAL(1, h2->RefCount()); - UNIT_ASSERT_VALUES_EQUAL(1, h3->RefCount()); - } + Y_UNIT_TEST(TimeSeriesMovable) { + auto ts = TInstant::Now(); + auto histogram = MakeIntrusive<TTestHistogram>(); + + UNIT_ASSERT_VALUES_EQUAL(1, histogram->RefCount()); + { + TMetricTimeSeries timeSeriesA; + timeSeriesA.Add(ts, histogram.Get()); + UNIT_ASSERT_VALUES_EQUAL(2, histogram->RefCount()); + + TMetricTimeSeries timeSeriesB = std::move(timeSeriesA); + UNIT_ASSERT_VALUES_EQUAL(2, histogram->RefCount()); + + UNIT_ASSERT_VALUES_EQUAL(1, timeSeriesB.Size()); + UNIT_ASSERT_EQUAL(EMetricValueType::HISTOGRAM, timeSeriesB.GetValueType()); + + UNIT_ASSERT_VALUES_EQUAL(0, timeSeriesA.Size()); + UNIT_ASSERT_EQUAL(EMetricValueType::UNKNOWN, timeSeriesA.GetValueType()); + } + UNIT_ASSERT_VALUES_EQUAL(1, histogram->RefCount()); + } + + Y_UNIT_TEST(HistogramsUnique) { + auto ts1 = TInstant::Now(); + auto ts2 = ts1 + TDuration::Seconds(1); + auto ts3 = ts2 + TDuration::Seconds(1); + + auto h1 = MakeIntrusive<TTestHistogram>(); + auto h2 = MakeIntrusive<TTestHistogram>(); + auto h3 = MakeIntrusive<TTestHistogram>(); + + UNIT_ASSERT_VALUES_EQUAL(1, h1->RefCount()); + UNIT_ASSERT_VALUES_EQUAL(1, h2->RefCount()); + UNIT_ASSERT_VALUES_EQUAL(1, h3->RefCount()); + + { + TMetricTimeSeries timeSeries; + timeSeries.Add(ts1, h1.Get()); // drop at the head + timeSeries.Add(ts1, h1.Get()); + timeSeries.Add(ts1, h1.Get()); + + timeSeries.Add(ts2, h2.Get()); // drop in the middle + timeSeries.Add(ts2, h2.Get()); + timeSeries.Add(ts2, h2.Get()); + + timeSeries.Add(ts3, h3.Get()); // drop at the end + timeSeries.Add(ts3, h3.Get()); + timeSeries.Add(ts3, h3.Get()); + + UNIT_ASSERT_EQUAL(timeSeries.Size(), 9); + + UNIT_ASSERT_VALUES_EQUAL(4, h1->RefCount()); + UNIT_ASSERT_VALUES_EQUAL(4, h2->RefCount()); + UNIT_ASSERT_VALUES_EQUAL(4, h3->RefCount()); + + timeSeries.SortByTs(); + UNIT_ASSERT_EQUAL(timeSeries.Size(), 3); + + UNIT_ASSERT_VALUES_EQUAL(2, h1->RefCount()); + UNIT_ASSERT_VALUES_EQUAL(2, h2->RefCount()); + UNIT_ASSERT_VALUES_EQUAL(2, h3->RefCount()); + } + + UNIT_ASSERT_VALUES_EQUAL(1, h1->RefCount()); + UNIT_ASSERT_VALUES_EQUAL(1, h2->RefCount()); + UNIT_ASSERT_VALUES_EQUAL(1, h3->RefCount()); + } Y_UNIT_TEST(LogHistogramsUnique) { auto ts1 = TInstant::Now(); @@ -230,7 +230,7 @@ Y_UNIT_TEST_SUITE(TMetricValueTest) { UNIT_ASSERT_VALUES_EQUAL(1, h3->RefCount()); { - TMetricTimeSeries timeSeries; + TMetricTimeSeries timeSeries; timeSeries.Add(ts1, h1.Get()); // drop at the head timeSeries.Add(ts1, h1.Get()); timeSeries.Add(ts1, h1.Get()); @@ -278,7 +278,7 @@ Y_UNIT_TEST_SUITE(TMetricValueTest) { auto h7 = MakeIntrusive<TTestHistogram>(7u); { - TMetricTimeSeries timeSeries; + TMetricTimeSeries timeSeries; timeSeries.Add(ts1, h1.Get()); timeSeries.Add(ts1, h2.Get()); @@ -356,7 +356,7 @@ Y_UNIT_TEST_SUITE(TMetricValueTest) { auto h7 = MakeSummarySnapshot(7u); { - TMetricTimeSeries timeSeries; + TMetricTimeSeries timeSeries; timeSeries.Add(ts1, h1.Get()); timeSeries.Add(ts1, h2.Get()); @@ -379,27 +379,27 @@ Y_UNIT_TEST_SUITE(TMetricValueTest) { } } - Y_UNIT_TEST(TMetricValueWithType) { + Y_UNIT_TEST(TMetricValueWithType) { // correct usage { double value = 1.23; - TMetricValueWithType v{value}; + TMetricValueWithType v{value}; - UNIT_ASSERT_VALUES_EQUAL(v.GetType(), EMetricValueType::DOUBLE); + UNIT_ASSERT_VALUES_EQUAL(v.GetType(), EMetricValueType::DOUBLE); UNIT_ASSERT_VALUES_EQUAL(v.AsDouble(), value); } { ui64 value = 12; - TMetricValueWithType v{value}; + TMetricValueWithType v{value}; - UNIT_ASSERT_VALUES_EQUAL(v.GetType(), EMetricValueType::UINT64); + UNIT_ASSERT_VALUES_EQUAL(v.GetType(), EMetricValueType::UINT64); UNIT_ASSERT_VALUES_EQUAL(v.AsUint64(), value); } { i64 value = i64(-12); - TMetricValueWithType v{value}; + TMetricValueWithType v{value}; - UNIT_ASSERT_VALUES_EQUAL(v.GetType(), EMetricValueType::INT64); + UNIT_ASSERT_VALUES_EQUAL(v.GetType(), EMetricValueType::INT64); UNIT_ASSERT_VALUES_EQUAL(v.AsInt64(), value); } { @@ -408,11 +408,11 @@ Y_UNIT_TEST_SUITE(TMetricValueTest) { { auto value = h.Get(); - TMetricValueWithType v{value}; + TMetricValueWithType v{value}; UNIT_ASSERT_VALUES_EQUAL(h.RefCount(), 2); - UNIT_ASSERT_VALUES_EQUAL(v.GetType(), EMetricValueType::HISTOGRAM); + UNIT_ASSERT_VALUES_EQUAL(v.GetType(), EMetricValueType::HISTOGRAM); UNIT_ASSERT_VALUES_EQUAL(v.AsHistogram(), value); } @@ -425,11 +425,11 @@ Y_UNIT_TEST_SUITE(TMetricValueTest) { UNIT_ASSERT_VALUES_EQUAL(s.RefCount(), 1); { - TMetricValueWithType v{value}; + TMetricValueWithType v{value}; UNIT_ASSERT_VALUES_EQUAL(s.RefCount(), 2); - UNIT_ASSERT_VALUES_EQUAL(v.GetType(), EMetricValueType::SUMMARY); + UNIT_ASSERT_VALUES_EQUAL(v.GetType(), EMetricValueType::SUMMARY); UNIT_ASSERT_VALUES_EQUAL(v.AsSummaryDouble(), value); } @@ -442,7 +442,7 @@ Y_UNIT_TEST_SUITE(TMetricValueTest) { UNIT_ASSERT_VALUES_EQUAL(s.RefCount(), 1); { - TMetricValueWithType v{value}; + TMetricValueWithType v{value}; UNIT_ASSERT_VALUES_EQUAL(s.RefCount(), 2); @@ -457,19 +457,19 @@ Y_UNIT_TEST_SUITE(TMetricValueTest) { auto value = s.Get(); { - TMetricValueWithType v1{ui64{1}}; + TMetricValueWithType v1{ui64{1}}; UNIT_ASSERT_VALUES_EQUAL(s.RefCount(), 1); { - TMetricValueWithType v2{value}; + TMetricValueWithType v2{value}; UNIT_ASSERT_VALUES_EQUAL(s.RefCount(), 2); v1 = std::move(v2); UNIT_ASSERT_VALUES_EQUAL(s.RefCount(), 2); UNIT_ASSERT_VALUES_EQUAL(v1.AsSummaryDouble(), value); - UNIT_ASSERT_VALUES_EQUAL(v1.GetType(), EMetricValueType::SUMMARY); - UNIT_ASSERT_VALUES_EQUAL(v2.GetType(), EMetricValueType::UNKNOWN); + UNIT_ASSERT_VALUES_EQUAL(v1.GetType(), EMetricValueType::SUMMARY); + UNIT_ASSERT_VALUES_EQUAL(v2.GetType(), EMetricValueType::UNKNOWN); } UNIT_ASSERT_VALUES_EQUAL(s.RefCount(), 2); @@ -480,14 +480,14 @@ Y_UNIT_TEST_SUITE(TMetricValueTest) { // incorrect usage { - TMetricValueWithType v{1.23}; + TMetricValueWithType v{1.23}; UNIT_ASSERT_EXCEPTION(v.AsHistogram(), yexception); UNIT_ASSERT_EXCEPTION(v.AsSummaryDouble(), yexception); } { auto h = MakeHistogramSnapshot(); - TMetricValueWithType v{h.Get()}; + TMetricValueWithType v{h.Get()}; UNIT_ASSERT_EXCEPTION(v.AsUint64(), yexception); UNIT_ASSERT_EXCEPTION(v.AsInt64(), yexception); @@ -496,7 +496,7 @@ Y_UNIT_TEST_SUITE(TMetricValueTest) { } { auto s = MakeSummarySnapshot(); - TMetricValueWithType v{s.Get()}; + TMetricValueWithType v{s.Get()}; UNIT_ASSERT_EXCEPTION(v.AsUint64(), yexception); UNIT_ASSERT_EXCEPTION(v.AsInt64(), yexception); @@ -504,4 +504,4 @@ Y_UNIT_TEST_SUITE(TMetricValueTest) { UNIT_ASSERT_EXCEPTION(v.AsHistogram(), yexception); } } -} +} diff --git a/library/cpp/monlib/metrics/timer.h b/library/cpp/monlib/metrics/timer.h index 35d7824facf..5c4e26e37bf 100644 --- a/library/cpp/monlib/metrics/timer.h +++ b/library/cpp/monlib/metrics/timer.h @@ -1,6 +1,6 @@ #pragma once -#include "metric.h" +#include "metric.h" #include <util/generic/typetraits.h> @@ -12,59 +12,59 @@ namespace NMonitoring { /** * A timing scope to record elapsed time since creation. */ - template <typename TMetric, + template <typename TMetric, typename Resolution = std::chrono::milliseconds, typename Clock = std::chrono::high_resolution_clock> - class TMetricTimerScope { + class TMetricTimerScope { public: - explicit TMetricTimerScope(TMetric* metric) - : Metric_(metric) + explicit TMetricTimerScope(TMetric* metric) + : Metric_(metric) , StartTime_(Clock::now()) { - Y_ENSURE(Metric_); + Y_ENSURE(Metric_); } - TMetricTimerScope(TMetricTimerScope&) = delete; - TMetricTimerScope& operator=(const TMetricTimerScope&) = delete; + TMetricTimerScope(TMetricTimerScope&) = delete; + TMetricTimerScope& operator=(const TMetricTimerScope&) = delete; - TMetricTimerScope(TMetricTimerScope&& other) { + TMetricTimerScope(TMetricTimerScope&& other) { *this = std::move(other); } - TMetricTimerScope& operator=(TMetricTimerScope&& other) { - Metric_ = other.Metric_; - other.Metric_ = nullptr; + TMetricTimerScope& operator=(TMetricTimerScope&& other) { + Metric_ = other.Metric_; + other.Metric_ = nullptr; StartTime_ = std::move(other.StartTime_); return *this; } void Record() { - Y_VERIFY_DEBUG(Metric_); - if (Metric_ == nullptr) { + Y_VERIFY_DEBUG(Metric_); + if (Metric_ == nullptr) { return; } auto duration = std::chrono::duration_cast<Resolution>(Clock::now() - StartTime_).count(); - if constexpr (std::is_same<TMetric, TGauge>::value) { - Metric_->Set(duration); - } else if constexpr (std::is_same<TMetric, TIntGauge>::value) { - Metric_->Set(duration); - } else if constexpr (std::is_same<TMetric, TCounter>::value) { - Metric_->Add(duration); - } else if constexpr (std::is_same<TMetric, TRate>::value) { - Metric_->Add(duration); - } else if constexpr (std::is_same<TMetric, THistogram>::value) { - Metric_->Record(duration); + if constexpr (std::is_same<TMetric, TGauge>::value) { + Metric_->Set(duration); + } else if constexpr (std::is_same<TMetric, TIntGauge>::value) { + Metric_->Set(duration); + } else if constexpr (std::is_same<TMetric, TCounter>::value) { + Metric_->Add(duration); + } else if constexpr (std::is_same<TMetric, TRate>::value) { + Metric_->Add(duration); + } else if constexpr (std::is_same<TMetric, THistogram>::value) { + Metric_->Record(duration); } else { static_assert(TDependentFalse<TMetric>, "Not supported metric type"); } - Metric_ = nullptr; + Metric_ = nullptr; } - ~TMetricTimerScope() { - if (Metric_ == nullptr) { + ~TMetricTimerScope() { + if (Metric_ == nullptr) { return; } @@ -72,7 +72,7 @@ namespace NMonitoring { } private: - TMetric* Metric_{nullptr}; + TMetric* Metric_{nullptr}; typename Clock::time_point StartTime_; }; @@ -80,18 +80,18 @@ namespace NMonitoring { * @brief A class that is supposed to use to measure execution time of an asynchronuous operation. * * In order to be able to capture an object into a lambda which is then passed to TFuture::Subscribe/Apply, - * the object must be copy constructible (limitation of the std::function class). So, we cannot use the TMetricTimerScope + * the object must be copy constructible (limitation of the std::function class). So, we cannot use the TMetricTimerScope * with the abovementioned functions without storing it in a shared pointer or somewhere else. This class works around this * issue with wrapping the timer with a auto_ptr-like hack Also, Record is const so that one doesn't need to make every lambda mutable * just to record time measurement. */ - template <typename TMetric, + template <typename TMetric, typename Resolution = std::chrono::milliseconds, typename Clock = std::chrono::high_resolution_clock> class TFutureFriendlyTimer { public: - explicit TFutureFriendlyTimer(TMetric* metric) - : Impl_{metric} + explicit TFutureFriendlyTimer(TMetric* metric) + : Impl_{metric} { } @@ -112,16 +112,16 @@ namespace NMonitoring { } private: - mutable TMetricTimerScope<TMetric, Resolution, Clock> Impl_; + mutable TMetricTimerScope<TMetric, Resolution, Clock> Impl_; }; - template <typename TMetric> - TMetricTimerScope<TMetric> ScopeTimer(TMetric* metric) { - return TMetricTimerScope<TMetric>{metric}; + template <typename TMetric> + TMetricTimerScope<TMetric> ScopeTimer(TMetric* metric) { + return TMetricTimerScope<TMetric>{metric}; } - template <typename TMetric> - TFutureFriendlyTimer<TMetric> FutureTimer(TMetric* metric) { - return TFutureFriendlyTimer<TMetric>{metric}; + template <typename TMetric> + TFutureFriendlyTimer<TMetric> FutureTimer(TMetric* metric) { + return TFutureFriendlyTimer<TMetric>{metric}; } } diff --git a/library/cpp/monlib/metrics/timer_ut.cpp b/library/cpp/monlib/metrics/timer_ut.cpp index f72c965feee..c244a8c9e12 100644 --- a/library/cpp/monlib/metrics/timer_ut.cpp +++ b/library/cpp/monlib/metrics/timer_ut.cpp @@ -29,13 +29,13 @@ Y_UNIT_TEST_SUITE(TTimerTest) { TGauge gauge(0); { - TMetricTimerScope<TGauge, milliseconds, TTestClock> t{&gauge}; + TMetricTimerScope<TGauge, milliseconds, TTestClock> t{&gauge}; TTestClock::TimePoint += milliseconds(10); } UNIT_ASSERT_EQUAL(10, gauge.Get()); { - TMetricTimerScope<TGauge, milliseconds, TTestClock> t{&gauge}; + TMetricTimerScope<TGauge, milliseconds, TTestClock> t{&gauge}; TTestClock::TimePoint += milliseconds(20); } UNIT_ASSERT_EQUAL(20, gauge.Get()); @@ -46,13 +46,13 @@ Y_UNIT_TEST_SUITE(TTimerTest) { TIntGauge gauge(0); { - TMetricTimerScope<TIntGauge, milliseconds, TTestClock> t{&gauge}; + TMetricTimerScope<TIntGauge, milliseconds, TTestClock> t{&gauge}; TTestClock::TimePoint += milliseconds(10); } UNIT_ASSERT_EQUAL(10, gauge.Get()); { - TMetricTimerScope<TIntGauge, milliseconds, TTestClock> t{&gauge}; + TMetricTimerScope<TIntGauge, milliseconds, TTestClock> t{&gauge}; TTestClock::TimePoint += milliseconds(20); } UNIT_ASSERT_EQUAL(20, gauge.Get()); @@ -61,15 +61,15 @@ Y_UNIT_TEST_SUITE(TTimerTest) { Y_UNIT_TEST(CounterNew) { TTestClock::TimePoint = TTestClock::time_point::min(); - TCounter counter(0); + TCounter counter(0); { - TMetricTimerScope<TCounter, milliseconds, TTestClock> t{&counter}; + TMetricTimerScope<TCounter, milliseconds, TTestClock> t{&counter}; TTestClock::TimePoint += milliseconds(10); } UNIT_ASSERT_EQUAL(10, counter.Get()); { - TMetricTimerScope<TCounter, milliseconds, TTestClock> t{&counter}; + TMetricTimerScope<TCounter, milliseconds, TTestClock> t{&counter}; TTestClock::TimePoint += milliseconds(20); } UNIT_ASSERT_EQUAL(30, counter.Get()); @@ -80,13 +80,13 @@ Y_UNIT_TEST_SUITE(TTimerTest) { TRate rate(0); { - TMetricTimerScope<TRate, milliseconds, TTestClock> t{&rate}; + TMetricTimerScope<TRate, milliseconds, TTestClock> t{&rate}; TTestClock::TimePoint += milliseconds(10); } UNIT_ASSERT_EQUAL(10, rate.Get()); { - TMetricTimerScope<TRate, milliseconds, TTestClock> t{&rate}; + TMetricTimerScope<TRate, milliseconds, TTestClock> t{&rate}; TTestClock::TimePoint += milliseconds(20); } UNIT_ASSERT_EQUAL(30, rate.Get()); @@ -104,13 +104,13 @@ Y_UNIT_TEST_SUITE(TTimerTest) { THistogram histogram(ExplicitHistogram({10, 20, 30}), true); { - TMetricTimerScope<THistogram, milliseconds, TTestClock> t{&histogram}; + TMetricTimerScope<THistogram, milliseconds, TTestClock> t{&histogram}; TTestClock::TimePoint += milliseconds(5); } assertHistogram({1, 0, 0, 0}, histogram.TakeSnapshot()); { - TMetricTimerScope<THistogram, milliseconds, TTestClock> t{&histogram}; + TMetricTimerScope<THistogram, milliseconds, TTestClock> t{&histogram}; TTestClock::TimePoint += milliseconds(15); } assertHistogram({1, 1, 0, 0}, histogram.TakeSnapshot()); @@ -119,9 +119,9 @@ Y_UNIT_TEST_SUITE(TTimerTest) { Y_UNIT_TEST(Moving) { TTestClock::TimePoint = TTestClock::time_point::min(); - TCounter counter(0); + TCounter counter(0); { - TMetricTimerScope<TCounter, milliseconds, TTestClock> t{&counter}; + TMetricTimerScope<TCounter, milliseconds, TTestClock> t{&counter}; [tt = std::move(t)] { TTestClock::TimePoint += milliseconds(5); Y_UNUSED(tt); @@ -137,9 +137,9 @@ Y_UNIT_TEST_SUITE(TTimerTest) { TTestClock::TimePoint = TTestClock::time_point::min(); auto pool = CreateThreadPool(1); - TCounter counter(0); + TCounter counter(0); { - TFutureFriendlyTimer<TCounter, milliseconds, TTestClock> t{&counter}; + TFutureFriendlyTimer<TCounter, milliseconds, TTestClock> t{&counter}; auto f = Async([=] { return; diff --git a/library/cpp/monlib/metrics/ut/histograms.json b/library/cpp/monlib/metrics/ut/histograms.json index 074273cfc4f..a6e8b78fea0 100644 --- a/library/cpp/monlib/metrics/ut/histograms.json +++ b/library/cpp/monlib/metrics/ut/histograms.json @@ -1,61 +1,61 @@ -{ - "commonLabels": - { - "common":"label" - }, - "sensors": - [ - { - "kind":"HIST", - "labels": - { - "sensor":"readTimeMillis" - }, - "hist": - { - "bounds": - [ - 1, - 2, - 4, - 8 - ], - "buckets": - [ - 2, - 1, - 2, - 4 - ], - "inf":91 - } - }, - { - "kind":"HIST_RATE", - "labels": - { - "sensor":"writeTimeMillis" - }, - "hist": - { - "bounds": - [ - 1, - 5, - 15, - 20, - 25 - ], - "buckets": - [ - 2, - 4, - 10, - 5, - 5 - ], - "inf":74 - } - } - ] -} +{ + "commonLabels": + { + "common":"label" + }, + "sensors": + [ + { + "kind":"HIST", + "labels": + { + "sensor":"readTimeMillis" + }, + "hist": + { + "bounds": + [ + 1, + 2, + 4, + 8 + ], + "buckets": + [ + 2, + 1, + 2, + 4 + ], + "inf":91 + } + }, + { + "kind":"HIST_RATE", + "labels": + { + "sensor":"writeTimeMillis" + }, + "hist": + { + "bounds": + [ + 1, + 5, + 15, + 20, + 25 + ], + "buckets": + [ + 2, + 4, + 10, + 5, + 5 + ], + "inf":74 + } + } + ] +} diff --git a/library/cpp/monlib/metrics/ut/ya.make b/library/cpp/monlib/metrics/ut/ya.make index 2f57822c059..aec9974fbd5 100644 --- a/library/cpp/monlib/metrics/ut/ya.make +++ b/library/cpp/monlib/metrics/ut/ya.make @@ -1,32 +1,32 @@ -UNITTEST_FOR(library/cpp/monlib/metrics) - +UNITTEST_FOR(library/cpp/monlib/metrics) + OWNER( jamel g:solomon ) - -SRCS( - ewma_ut.cpp - fake_ut.cpp - histogram_collector_ut.cpp - labels_ut.cpp + +SRCS( + ewma_ut.cpp + fake_ut.cpp + histogram_collector_ut.cpp + labels_ut.cpp log_histogram_collector_ut.cpp - metric_registry_ut.cpp - metric_sub_registry_ut.cpp - metric_value_ut.cpp - summary_collector_ut.cpp + metric_registry_ut.cpp + metric_sub_registry_ut.cpp + metric_value_ut.cpp + summary_collector_ut.cpp timer_ut.cpp -) - -RESOURCE( - histograms.json /histograms.json -) - -PEERDIR( +) + +RESOURCE( + histograms.json /histograms.json +) + +PEERDIR( library/cpp/resource - library/cpp/monlib/encode/protobuf - library/cpp/monlib/encode/json + library/cpp/monlib/encode/protobuf + library/cpp/monlib/encode/json library/cpp/threading/future -) - -END() +) + +END() diff --git a/library/cpp/monlib/metrics/ya.make b/library/cpp/monlib/metrics/ya.make index a4bacb0180d..0e1fa143f9f 100644 --- a/library/cpp/monlib/metrics/ya.make +++ b/library/cpp/monlib/metrics/ya.make @@ -1,26 +1,26 @@ -LIBRARY() - +LIBRARY() + OWNER( g:solomon jamel ) - -GENERATE_ENUM_SERIALIZATION_WITH_HEADER(metric_value_type.h) -SRCS( - ewma.cpp - fake.cpp - histogram_collector_explicit.cpp - histogram_collector_exponential.cpp - histogram_collector_linear.cpp - histogram_snapshot.cpp +GENERATE_ENUM_SERIALIZATION_WITH_HEADER(metric_value_type.h) + +SRCS( + ewma.cpp + fake.cpp + histogram_collector_explicit.cpp + histogram_collector_exponential.cpp + histogram_collector_linear.cpp + histogram_snapshot.cpp log_histogram_snapshot.cpp - labels.cpp - metric_registry.cpp + labels.cpp + metric_registry.cpp metric_consumer.cpp - metric_type.cpp - metric_value.cpp - summary_snapshot.cpp -) - -END() + metric_type.cpp + metric_value.cpp + summary_snapshot.cpp +) + +END() diff --git a/library/cpp/monlib/service/auth/tvm/auth.cpp b/library/cpp/monlib/service/auth/tvm/auth.cpp index 22f126eaade..e071c11ebc8 100644 --- a/library/cpp/monlib/service/auth/tvm/auth.cpp +++ b/library/cpp/monlib/service/auth/tvm/auth.cpp @@ -33,7 +33,7 @@ namespace { return AllowedClients_.contains(clientId); } - TCheckedServiceTicket CheckServiceTicket(TStringBuf ticket) override { + TCheckedServiceTicket CheckServiceTicket(TStringBuf ticket) override { return Tvm_->CheckServiceTicket(ticket); } diff --git a/library/cpp/monlib/service/auth/tvm/auth.h b/library/cpp/monlib/service/auth/tvm/auth.h index e9b71ef2d84..432beff9d6d 100644 --- a/library/cpp/monlib/service/auth/tvm/auth.h +++ b/library/cpp/monlib/service/auth/tvm/auth.h @@ -8,7 +8,7 @@ namespace NMonitoring { struct ITvmManager { virtual ~ITvmManager() = default; virtual bool IsAllowedClient(NTvmAuth::TTvmId clientId) = 0; - virtual NTvmAuth::TCheckedServiceTicket CheckServiceTicket(TStringBuf ticket) = 0; + virtual NTvmAuth::TCheckedServiceTicket CheckServiceTicket(TStringBuf ticket) = 0; }; THolder<ITvmManager> CreateDefaultTvmManager( diff --git a/library/cpp/monlib/service/format.h b/library/cpp/monlib/service/format.h index 16b6c12c489..0044b586b19 100644 --- a/library/cpp/monlib/service/format.h +++ b/library/cpp/monlib/service/format.h @@ -1,6 +1,6 @@ #pragma once -#include <library/cpp/monlib/encode/format.h> +#include <library/cpp/monlib/encode/format.h> #include <util/string/ascii.h> #include <util/generic/yexception.h> diff --git a/library/cpp/monlib/service/mon_service_http_request.cpp b/library/cpp/monlib/service/mon_service_http_request.cpp index a0164afaec8..5d805631d9d 100644 --- a/library/cpp/monlib/service/mon_service_http_request.cpp +++ b/library/cpp/monlib/service/mon_service_http_request.cpp @@ -1,4 +1,4 @@ -#include "mon_service_http_request.h" +#include "mon_service_http_request.h" #include "monservice.h" using namespace NMonitoring; diff --git a/library/cpp/monlib/service/mon_service_http_request.h b/library/cpp/monlib/service/mon_service_http_request.h index 80fce276752..b4f2f8f0c59 100644 --- a/library/cpp/monlib/service/mon_service_http_request.h +++ b/library/cpp/monlib/service/mon_service_http_request.h @@ -1,12 +1,12 @@ #pragma once -#include "service.h" - +#include "service.h" + #include <util/stream/output.h> namespace NMonitoring { - class TMonService2; - class IMonPage; + class TMonService2; + class IMonPage; // XXX: IHttpRequest is already taken struct IMonHttpRequest { @@ -34,26 +34,26 @@ namespace NMonitoring { }; struct TMonService2HttpRequest: IMonHttpRequest { - IOutputStream* const Out; - const IHttpRequest* const HttpRequest; - TMonService2* const MonService; - IMonPage* const MonPage; - const TString PathInfo; - TMonService2HttpRequest* const Parent; - - TMonService2HttpRequest( + IOutputStream* const Out; + const IHttpRequest* const HttpRequest; + TMonService2* const MonService; + IMonPage* const MonPage; + const TString PathInfo; + TMonService2HttpRequest* const Parent; + + TMonService2HttpRequest( IOutputStream* out, const IHttpRequest* httpRequest, TMonService2* monService, IMonPage* monPage, const TString& pathInfo, TMonService2HttpRequest* parent) - : Out(out) - , HttpRequest(httpRequest) - , MonService(monService) - , MonPage(monPage) - , PathInfo(pathInfo) - , Parent(parent) - { - } + : Out(out) + , HttpRequest(httpRequest) + , MonService(monService) + , MonPage(monPage) + , PathInfo(pathInfo) + , Parent(parent) + { + } ~TMonService2HttpRequest() override; @@ -85,6 +85,6 @@ namespace NMonitoring { } TString GetServiceTitle() const override; - }; + }; -} +} diff --git a/library/cpp/monlib/service/monservice.cpp b/library/cpp/monlib/service/monservice.cpp index 5c8fd6aaeae..d1b9cda1d21 100644 --- a/library/cpp/monlib/service/monservice.cpp +++ b/library/cpp/monlib/service/monservice.cpp @@ -1,21 +1,21 @@ #include "monservice.h" - + #include <library/cpp/malloc/api/malloc.h> #include <library/cpp/string_utils/base64/base64.h> #include <library/cpp/svnversion/svnversion.h> - + #include <util/generic/map.h> #include <util/generic/ptr.h> #include <util/system/hostname.h> - + #include <google/protobuf/text_format.h> using namespace NMonitoring; TMonService2::TMonService2(ui16 port, const TString& host, ui32 threads, const TString& title, THolder<IAuthProvider> auth) : TMonService2(HttpServerOptions(port, host, threads), title, std::move(auth)) -{ -} +{ +} TMonService2::TMonService2(const THttpServerOptions& options, const TString& title, THolder<IAuthProvider> auth) : NMonitoring::TMtHttpServer(options, std::bind(&TMonService2::ServeRequest, this, std::placeholders::_1, std::placeholders::_2)) @@ -46,14 +46,14 @@ TMonService2::TMonService2(ui16 port, ui32 threads, const TString& title, THolde TMonService2::TMonService2(ui16 port, const TString& title, THolder<IAuthProvider> auth) : TMonService2(port, TString(), 0, title, std::move(auth)) -{ -} +{ +} -void TMonService2::OutputIndex(IOutputStream& out) { +void TMonService2::OutputIndex(IOutputStream& out) { IndexMonPage->OutputIndex(out, true); } -void TMonService2::OutputIndexPage(IOutputStream& out) { +void TMonService2::OutputIndexPage(IOutputStream& out) { out << HTTPOKHTML; out << "<html>\n"; IndexMonPage->OutputHead(out); @@ -61,7 +61,7 @@ void TMonService2::OutputIndexPage(IOutputStream& out) { out << "</html>\n"; } -void TMonService2::OutputIndexBody(IOutputStream& out) { +void TMonService2::OutputIndexBody(IOutputStream& out) { out << "<body>\n"; // part of common navbar @@ -77,7 +77,7 @@ void TMonService2::OutputIndexBody(IOutputStream& out) { << "</body>\n"; } -void TMonService2::ServeRequest(IOutputStream& out, const NMonitoring::IHttpRequest& request) { +void TMonService2::ServeRequest(IOutputStream& out, const NMonitoring::IHttpRequest& request) { TString path = request.GetPath(); Y_VERIFY(path.StartsWith('/')); @@ -99,12 +99,12 @@ void TMonService2::ServeRequest(IOutputStream& out, const NMonitoring::IHttpRequ OutputIndexPage(out); } else { TMonService2HttpRequest monService2HttpRequest( - &out, &request, this, IndexMonPage.Get(), path, nullptr); + &out, &request, this, IndexMonPage.Get(), path, nullptr); IndexMonPage->Output(monService2HttpRequest); } } -void TMonService2::Register(IMonPage* page) { +void TMonService2::Register(IMonPage* page) { IndexMonPage->Register(page); } diff --git a/library/cpp/monlib/service/monservice.h b/library/cpp/monlib/service/monservice.h index 492bd9bec76..8f5e52fcdb2 100644 --- a/library/cpp/monlib/service/monservice.h +++ b/library/cpp/monlib/service/monservice.h @@ -1,73 +1,73 @@ #pragma once -#include "service.h" +#include "service.h" #include "auth.h" -#include "mon_service_http_request.h" - -#include <library/cpp/monlib/service/pages/index_mon_page.h> -#include <library/cpp/monlib/service/pages/mon_page.h> - +#include "mon_service_http_request.h" + +#include <library/cpp/monlib/service/pages/index_mon_page.h> +#include <library/cpp/monlib/service/pages/mon_page.h> + #include <util/system/progname.h> - + #include <functional> namespace NMonitoring { - class TMonService2: public TMtHttpServer { - protected: - const TString Title; - char StartTime[26]; - TIntrusivePtr<TIndexMonPage> IndexMonPage; + class TMonService2: public TMtHttpServer { + protected: + const TString Title; + char StartTime[26]; + TIntrusivePtr<TIndexMonPage> IndexMonPage; THolder<IAuthProvider> AuthProvider_; - public: + public: static THttpServerOptions HttpServerOptions(ui16 port, const TString& host, ui32 threads) { - THttpServerOptions opts(port); + THttpServerOptions opts(port); if (!host.empty()) { opts.SetHost(host); } - opts.SetClientTimeout(TDuration::Minutes(1)); - opts.EnableCompression(true); - opts.SetThreads(threads); + opts.SetClientTimeout(TDuration::Minutes(1)); + opts.EnableCompression(true); + opts.SetThreads(threads); opts.SetMaxConnections(std::max<ui32>(100, threads)); opts.EnableRejectExcessConnections(true); - return opts; - } + return opts; + } static THttpServerOptions HttpServerOptions(ui16 port, ui32 threads) { return HttpServerOptions(port, TString(), threads); } - public: + public: explicit TMonService2(ui16 port, const TString& title = GetProgramName(), THolder<IAuthProvider> auth = nullptr); explicit TMonService2(ui16 port, ui32 threads, const TString& title = GetProgramName(), THolder<IAuthProvider> auth = nullptr); explicit TMonService2(ui16 port, const TString& host, ui32 threads, const TString& title = GetProgramName(), THolder<IAuthProvider> auth = nullptr); explicit TMonService2(const THttpServerOptions& options, const TString& title = GetProgramName(), THolder<IAuthProvider> auth = nullptr); explicit TMonService2(const THttpServerOptions& options, TSimpleSharedPtr<IThreadPool> pool, const TString& title = GetProgramName(), THolder<IAuthProvider> auth = nullptr); - ~TMonService2() override { - } + ~TMonService2() override { + } - const char* GetStartTime() const { - return StartTime; - } + const char* GetStartTime() const { + return StartTime; + } - const TString& GetTitle() const { - return Title; - } + const TString& GetTitle() const { + return Title; + } - virtual void ServeRequest(IOutputStream& out, const NMonitoring::IHttpRequest& request); - virtual void OutputIndex(IOutputStream& out); - virtual void OutputIndexPage(IOutputStream& out); - virtual void OutputIndexBody(IOutputStream& out); + virtual void ServeRequest(IOutputStream& out, const NMonitoring::IHttpRequest& request); + virtual void OutputIndex(IOutputStream& out); + virtual void OutputIndexPage(IOutputStream& out); + virtual void OutputIndexBody(IOutputStream& out); - void Register(IMonPage* page); + void Register(IMonPage* page); void Register(TMonPagePtr page); - TIndexMonPage* RegisterIndexPage(const TString& path, const TString& title); + TIndexMonPage* RegisterIndexPage(const TString& path, const TString& title); - IMonPage* FindPage(const TString& relativePath); - TIndexMonPage* FindIndexPage(const TString& relativePath); - void SortPages(); - }; + IMonPage* FindPage(const TString& relativePath); + TIndexMonPage* FindIndexPage(const TString& relativePath); + void SortPages(); + }; -} +} diff --git a/library/cpp/monlib/service/pages/diag_mon_page.h b/library/cpp/monlib/service/pages/diag_mon_page.h index 2d90d5850db..761194d4ecf 100644 --- a/library/cpp/monlib/service/pages/diag_mon_page.h +++ b/library/cpp/monlib/service/pages/diag_mon_page.h @@ -3,14 +3,14 @@ #include "pre_mon_page.h" namespace NMonitoring { - // internal diagnostics page - struct TDiagMonPage: public TPreMonPage { - TDiagMonPage() - : TPreMonPage("diag", "Diagnostics Page") - { - } + // internal diagnostics page + struct TDiagMonPage: public TPreMonPage { + TDiagMonPage() + : TPreMonPage("diag", "Diagnostics Page") + { + } void OutputText(IOutputStream& out, NMonitoring::IMonHttpRequest&) override; - }; + }; -} +} diff --git a/library/cpp/monlib/service/pages/html_mon_page.cpp b/library/cpp/monlib/service/pages/html_mon_page.cpp index 8c178e135f5..eb4eb3b66c6 100644 --- a/library/cpp/monlib/service/pages/html_mon_page.cpp +++ b/library/cpp/monlib/service/pages/html_mon_page.cpp @@ -1,6 +1,6 @@ #include "html_mon_page.h" -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> using namespace NMonitoring; @@ -20,8 +20,8 @@ void THtmlMonPage::Output(NMonitoring::IMonHttpRequest& request) { out << "<script language='javascript' type='text/javascript' src='https://yastatic.net/bootstrap/3.3.1/js/bootstrap.min.js'></script>\n"; if (OutputTableSorterJsCss) { - out << "<link rel='stylesheet' href='/jquery.tablesorter.css'>\n"; - out << "<script language='javascript' type='text/javascript' src='/jquery.tablesorter.js'></script>\n"; + out << "<link rel='stylesheet' href='/jquery.tablesorter.css'>\n"; + out << "<script language='javascript' type='text/javascript' src='/jquery.tablesorter.js'></script>\n"; } out << "<style type=\"text/css\">\n"; @@ -34,7 +34,7 @@ void THtmlMonPage::Output(NMonitoring::IMonHttpRequest& request) { out << "</style>\n"; } BODY() { - OutputNavBar(out); + OutputNavBar(out); DIV_CLASS("container") { if (!!Title) { diff --git a/library/cpp/monlib/service/pages/html_mon_page.h b/library/cpp/monlib/service/pages/html_mon_page.h index 305af355cb0..e87c53b62bd 100644 --- a/library/cpp/monlib/service/pages/html_mon_page.h +++ b/library/cpp/monlib/service/pages/html_mon_page.h @@ -3,14 +3,14 @@ #include "mon_page.h" namespace NMonitoring { - struct THtmlMonPage: public IMonPage { - THtmlMonPage(const TString& path, - const TString& title = TString(), - bool outputTableSorterJsCss = false) + struct THtmlMonPage: public IMonPage { + THtmlMonPage(const TString& path, + const TString& title = TString(), + bool outputTableSorterJsCss = false) : IMonPage(path, title) - , OutputTableSorterJsCss(outputTableSorterJsCss) - { - } + , OutputTableSorterJsCss(outputTableSorterJsCss) + { + } void Output(NMonitoring::IMonHttpRequest& request) override; @@ -19,7 +19,7 @@ namespace NMonitoring { virtual void OutputContent(NMonitoring::IMonHttpRequest& request) = 0; - bool OutputTableSorterJsCss; - }; + bool OutputTableSorterJsCss; + }; -} +} diff --git a/library/cpp/monlib/service/pages/index_mon_page.cpp b/library/cpp/monlib/service/pages/index_mon_page.cpp index 65f6c1ea59b..83ff8b529ae 100644 --- a/library/cpp/monlib/service/pages/index_mon_page.cpp +++ b/library/cpp/monlib/service/pages/index_mon_page.cpp @@ -1,7 +1,7 @@ -#include "index_mon_page.h" - +#include "index_mon_page.h" + #include <util/generic/cast.h> -#include <util/string/ascii.h> +#include <util/string/ascii.h> using namespace NMonitoring; @@ -51,7 +51,7 @@ void TIndexMonPage::Output(IMonHttpRequest& request) { } } -void TIndexMonPage::OutputIndex(IOutputStream& out, bool pathEndsWithSlash) { +void TIndexMonPage::OutputIndex(IOutputStream& out, bool pathEndsWithSlash) { TGuard<TMutex> g(Mtx); for (auto& Page : Pages) { IMonPage* page = Page.Get(); @@ -65,7 +65,7 @@ void TIndexMonPage::OutputIndex(IOutputStream& out, bool pathEndsWithSlash) { } } -void TIndexMonPage::Register(TMonPagePtr page) { +void TIndexMonPage::Register(TMonPagePtr page) { TGuard<TMutex> g(Mtx); auto insres = PagesByPath.insert(std::make_pair("/" + page->GetPath(), page)); if (insres.second) { @@ -80,10 +80,10 @@ void TIndexMonPage::Register(TMonPagePtr page) { // this already present, replace it insres.first->second = page; } - page->Parent = this; + page->Parent = this; } -TIndexMonPage* TIndexMonPage::RegisterIndexPage(const TString& path, const TString& title) { +TIndexMonPage* TIndexMonPage::RegisterIndexPage(const TString& path, const TString& title) { TGuard<TMutex> g(Mtx); TIndexMonPage* page = VerifyDynamicCast<TIndexMonPage*>(FindPage(path)); if (page) { @@ -116,7 +116,7 @@ void TIndexMonPage::OutputCommonJsCss(IOutputStream& out) { out << "<script language='javascript' type='text/javascript' src='https://yastatic.net/bootstrap/3.3.1/js/bootstrap.min.js'></script>\n"; } -void TIndexMonPage::OutputHead(IOutputStream& out) { +void TIndexMonPage::OutputHead(IOutputStream& out) { out << "<head>\n"; OutputCommonJsCss(out); out << "<title>" << Title << "</title>\n"; @@ -128,19 +128,19 @@ void TIndexMonPage::OutputBody(IMonHttpRequest& req) { out << "<body>\n"; // part of common navbar - OutputNavBar(out); + OutputNavBar(out); out << "<div class='container'>\n" - << "<h2>" << Title << "</h2>\n"; + << "<h2>" << Title << "</h2>\n"; OutputIndex(out, req.GetPathInfo().EndsWith('/')); out << "<div>\n" << "</body>\n"; } -void TIndexMonPage::SortPages() { +void TIndexMonPage::SortPages() { TGuard<TMutex> g(Mtx); std::sort(Pages.begin(), Pages.end(), [](const TMonPagePtr& a, const TMonPagePtr& b) { - return AsciiCompareIgnoreCase(a->GetTitle(), b->GetTitle()) < 0; + return AsciiCompareIgnoreCase(a->GetTitle(), b->GetTitle()) < 0; }); } diff --git a/library/cpp/monlib/service/pages/index_mon_page.h b/library/cpp/monlib/service/pages/index_mon_page.h index 6831fd15ca5..bf514a31056 100644 --- a/library/cpp/monlib/service/pages/index_mon_page.h +++ b/library/cpp/monlib/service/pages/index_mon_page.h @@ -3,36 +3,36 @@ #include "mon_page.h" namespace NMonitoring { - struct TIndexMonPage: public IMonPage { - TMutex Mtx; - typedef TVector<TMonPagePtr> TPages; - TPages Pages; - typedef THashMap<TString, TMonPagePtr> TPagesByPath; - TPagesByPath PagesByPath; + struct TIndexMonPage: public IMonPage { + TMutex Mtx; + typedef TVector<TMonPagePtr> TPages; + TPages Pages; + typedef THashMap<TString, TMonPagePtr> TPagesByPath; + TPagesByPath PagesByPath; - TIndexMonPage(const TString& path, const TString& title) - : IMonPage(path, title) - { - } + TIndexMonPage(const TString& path, const TString& title) + : IMonPage(path, title) + { + } - ~TIndexMonPage() override { - } + ~TIndexMonPage() override { + } void Output(IMonHttpRequest& request) override; void OutputIndexPage(IMonHttpRequest& request); - virtual void OutputIndex(IOutputStream& out, bool pathEndsWithSlash); - virtual void OutputCommonJsCss(IOutputStream& out); - void OutputHead(IOutputStream& out); + virtual void OutputIndex(IOutputStream& out, bool pathEndsWithSlash); + virtual void OutputCommonJsCss(IOutputStream& out); + void OutputHead(IOutputStream& out); void OutputBody(IMonHttpRequest& out); - void Register(TMonPagePtr page); - TIndexMonPage* RegisterIndexPage(const TString& path, const TString& title); + void Register(TMonPagePtr page); + TIndexMonPage* RegisterIndexPage(const TString& path, const TString& title); - IMonPage* FindPage(const TString& relativePath); - TIndexMonPage* FindIndexPage(const TString& relativePath); + IMonPage* FindPage(const TString& relativePath); + TIndexMonPage* FindIndexPage(const TString& relativePath); - void SortPages(); + void SortPages(); void ClearPages(); - }; + }; -} +} diff --git a/library/cpp/monlib/service/pages/mon_page.cpp b/library/cpp/monlib/service/pages/mon_page.cpp index 1bffd42027c..72033b1699a 100644 --- a/library/cpp/monlib/service/pages/mon_page.cpp +++ b/library/cpp/monlib/service/pages/mon_page.cpp @@ -9,28 +9,28 @@ IMonPage::IMonPage(const TString& path, const TString& title) Y_VERIFY(!Path.StartsWith('/')); Y_VERIFY(!Path.EndsWith('/')); } - -void IMonPage::OutputNavBar(IOutputStream& out) { - TVector<const IMonPage*> parents; - for (const IMonPage* p = this; p; p = p->Parent) { - parents.push_back(p); - } - std::reverse(parents.begin(), parents.end()); - - out << "<ol class='breadcrumb'>\n"; - - TString absolutePath; - for (size_t i = 0; i < parents.size(); ++i) { - const TString& title = parents[i]->GetTitle(); - if (i == parents.size() - 1) { - out << "<li>" << title << "</li>\n"; - } else { - if (!absolutePath.EndsWith('/')) { - absolutePath += '/'; - } - absolutePath += parents[i]->GetPath(); - out << "<li class='active'><a href='" << absolutePath << "'>" << title << "</a></li>\n"; - } - } - out << "</ol>\n"; -} + +void IMonPage::OutputNavBar(IOutputStream& out) { + TVector<const IMonPage*> parents; + for (const IMonPage* p = this; p; p = p->Parent) { + parents.push_back(p); + } + std::reverse(parents.begin(), parents.end()); + + out << "<ol class='breadcrumb'>\n"; + + TString absolutePath; + for (size_t i = 0; i < parents.size(); ++i) { + const TString& title = parents[i]->GetTitle(); + if (i == parents.size() - 1) { + out << "<li>" << title << "</li>\n"; + } else { + if (!absolutePath.EndsWith('/')) { + absolutePath += '/'; + } + absolutePath += parents[i]->GetPath(); + out << "<li class='active'><a href='" << absolutePath << "'>" << title << "</a></li>\n"; + } + } + out << "</ol>\n"; +} diff --git a/library/cpp/monlib/service/pages/mon_page.h b/library/cpp/monlib/service/pages/mon_page.h index da151e5255d..e396612bb0d 100644 --- a/library/cpp/monlib/service/pages/mon_page.h +++ b/library/cpp/monlib/service/pages/mon_page.h @@ -1,66 +1,66 @@ #pragma once -#include <library/cpp/monlib/service/service.h> -#include <library/cpp/monlib/service/mon_service_http_request.h> - +#include <library/cpp/monlib/service/service.h> +#include <library/cpp/monlib/service/mon_service_http_request.h> + #include <util/generic/string.h> #include <util/generic/ptr.h> namespace NMonitoring { - static const char HTTPOKTEXT[] = "HTTP/1.1 200 Ok\r\nContent-Type: text/plain\r\nConnection: Close\r\n\r\n"; - static const char HTTPOKBIN[] = "HTTP/1.1 200 Ok\r\nContent-Type: application/octet-stream\r\nConnection: Close\r\n\r\n"; - static const char HTTPOKHTML[] = "HTTP/1.1 200 Ok\r\nContent-Type: text/html\r\nConnection: Close\r\n\r\n"; - static const char HTTPOKJSON[] = "HTTP/1.1 200 Ok\r\nContent-Type: application/json\r\nConnection: Close\r\n\r\n"; - static const char HTTPOKSPACK[] = "HTTP/1.1 200 Ok\r\nContent-Type: application/x-solomon-spack\r\nConnection: Close\r\n\r\n"; + static const char HTTPOKTEXT[] = "HTTP/1.1 200 Ok\r\nContent-Type: text/plain\r\nConnection: Close\r\n\r\n"; + static const char HTTPOKBIN[] = "HTTP/1.1 200 Ok\r\nContent-Type: application/octet-stream\r\nConnection: Close\r\n\r\n"; + static const char HTTPOKHTML[] = "HTTP/1.1 200 Ok\r\nContent-Type: text/html\r\nConnection: Close\r\n\r\n"; + static const char HTTPOKJSON[] = "HTTP/1.1 200 Ok\r\nContent-Type: application/json\r\nConnection: Close\r\n\r\n"; + static const char HTTPOKSPACK[] = "HTTP/1.1 200 Ok\r\nContent-Type: application/x-solomon-spack\r\nConnection: Close\r\n\r\n"; static const char HTTPOKPROMETHEUS[] = "HTTP/1.1 200 Ok\r\nContent-Type: text/plain\r\nConnection: Close\r\n\r\n"; - static const char HTTPOKJAVASCRIPT[] = "HTTP/1.1 200 Ok\r\nContent-Type: text/javascript\r\nConnection: Close\r\n\r\n"; - static const char HTTPOKCSS[] = "HTTP/1.1 200 Ok\r\nContent-Type: text/css\r\nConnection: Close\r\n\r\n"; - static const char HTTPNOCONTENT[] = "HTTP/1.1 204 No content\r\nConnection: Close\r\n\r\n"; - static const char HTTPNOTFOUND[] = "HTTP/1.1 404 Invalid URI\r\nConnection: Close\r\n\r\nInvalid URL\r\n"; + static const char HTTPOKJAVASCRIPT[] = "HTTP/1.1 200 Ok\r\nContent-Type: text/javascript\r\nConnection: Close\r\n\r\n"; + static const char HTTPOKCSS[] = "HTTP/1.1 200 Ok\r\nContent-Type: text/css\r\nConnection: Close\r\n\r\n"; + static const char HTTPNOCONTENT[] = "HTTP/1.1 204 No content\r\nConnection: Close\r\n\r\n"; + static const char HTTPNOTFOUND[] = "HTTP/1.1 404 Invalid URI\r\nConnection: Close\r\n\r\nInvalid URL\r\n"; static const char HTTPUNAUTHORIZED[] = "HTTP/1.1 401 Unauthorized\r\nConnection: Close\r\n\r\nUnauthorized\r\n"; static const char HTTPFORBIDDEN[] = "HTTP/1.1 403 Forbidden\r\nConnection: Close\r\n\r\nForbidden\r\n"; - // Fonts - static const char HTTPOKFONTEOT[] = "HTTP/1.1 200 Ok\r\nContent-Type: application/vnd.ms-fontobject\r\nConnection: Close\r\n\r\n"; - static const char HTTPOKFONTTTF[] = "HTTP/1.1 200 Ok\r\nContent-Type: application/x-font-ttf\r\nConnection: Close\r\n\r\n"; - static const char HTTPOKFONTWOFF[] = "HTTP/1.1 200 Ok\r\nContent-Type: application/font-woff\r\nConnection: Close\r\n\r\n"; - static const char HTTPOKFONTWOFF2[] = "HTTP/1.1 200 Ok\r\nContent-Type: application/font-woff2\r\nConnection: Close\r\n\r\n"; + // Fonts + static const char HTTPOKFONTEOT[] = "HTTP/1.1 200 Ok\r\nContent-Type: application/vnd.ms-fontobject\r\nConnection: Close\r\n\r\n"; + static const char HTTPOKFONTTTF[] = "HTTP/1.1 200 Ok\r\nContent-Type: application/x-font-ttf\r\nConnection: Close\r\n\r\n"; + static const char HTTPOKFONTWOFF[] = "HTTP/1.1 200 Ok\r\nContent-Type: application/font-woff\r\nConnection: Close\r\n\r\n"; + static const char HTTPOKFONTWOFF2[] = "HTTP/1.1 200 Ok\r\nContent-Type: application/font-woff2\r\nConnection: Close\r\n\r\n"; + + // Images + static const char HTTPOKPNG[] = "HTTP/1.1 200 Ok\r\nContent-Type: image/png\r\nConnection: Close\r\n\r\n"; + static const char HTTPOKSVG[] = "HTTP/1.1 200 Ok\r\nContent-Type: image/svg+xml\r\nConnection: Close\r\n\r\n"; - // Images - static const char HTTPOKPNG[] = "HTTP/1.1 200 Ok\r\nContent-Type: image/png\r\nConnection: Close\r\n\r\n"; - static const char HTTPOKSVG[] = "HTTP/1.1 200 Ok\r\nContent-Type: image/svg+xml\r\nConnection: Close\r\n\r\n"; + class IMonPage; - class IMonPage; + using TMonPagePtr = TIntrusivePtr<IMonPage>; - using TMonPagePtr = TIntrusivePtr<IMonPage>; + class IMonPage: public TAtomicRefCount<IMonPage> { + public: + const TString Path; + const TString Title; + const IMonPage* Parent = nullptr; - class IMonPage: public TAtomicRefCount<IMonPage> { - public: - const TString Path; - const TString Title; - const IMonPage* Parent = nullptr; + public: + IMonPage(const TString& path, const TString& title = TString()); - public: - IMonPage(const TString& path, const TString& title = TString()); + virtual ~IMonPage() { + } - virtual ~IMonPage() { - } + void OutputNavBar(IOutputStream& out); - void OutputNavBar(IOutputStream& out); - - virtual const TString& GetPath() const { - return Path; - } + virtual const TString& GetPath() const { + return Path; + } - virtual const TString& GetTitle() const { - return Title; - } + virtual const TString& GetTitle() const { + return Title; + } - bool IsInIndex() const { + bool IsInIndex() const { return !Title.empty(); - } + } virtual void Output(IMonHttpRequest& request) = 0; - }; + }; } diff --git a/library/cpp/monlib/service/pages/pre_mon_page.h b/library/cpp/monlib/service/pages/pre_mon_page.h index 7f4199287d3..c9a923d39ac 100644 --- a/library/cpp/monlib/service/pages/pre_mon_page.h +++ b/library/cpp/monlib/service/pages/pre_mon_page.h @@ -3,25 +3,25 @@ #include "html_mon_page.h" namespace NMonitoring { - struct TPreMonPage: public THtmlMonPage { - TPreMonPage(const TString& path, - const TString& title = TString(), - bool preTag = true, - bool outputTableSorterJsCss = false) - : THtmlMonPage(path, title, outputTableSorterJsCss) - , PreTag(preTag) - { - } + struct TPreMonPage: public THtmlMonPage { + TPreMonPage(const TString& path, + const TString& title = TString(), + bool preTag = true, + bool outputTableSorterJsCss = false) + : THtmlMonPage(path, title, outputTableSorterJsCss) + , PreTag(preTag) + { + } void OutputContent(NMonitoring::IMonHttpRequest& request) override; - // hook to customize output + // hook to customize output virtual void BeforePre(NMonitoring::IMonHttpRequest& request); - // put your text here + // put your text here virtual void OutputText(IOutputStream& out, NMonitoring::IMonHttpRequest&) = 0; - const bool PreTag; - }; + const bool PreTag; + }; -} +} diff --git a/library/cpp/monlib/service/pages/registry_mon_page.cpp b/library/cpp/monlib/service/pages/registry_mon_page.cpp index e05c80d07fd..c59e50f622e 100644 --- a/library/cpp/monlib/service/pages/registry_mon_page.cpp +++ b/library/cpp/monlib/service/pages/registry_mon_page.cpp @@ -1,18 +1,18 @@ #include "registry_mon_page.h" -#include <library/cpp/monlib/encode/text/text.h> -#include <library/cpp/monlib/encode/json/json.h> +#include <library/cpp/monlib/encode/text/text.h> +#include <library/cpp/monlib/encode/json/json.h> #include <library/cpp/monlib/encode/prometheus/prometheus.h> -#include <library/cpp/monlib/encode/spack/spack_v1.h> +#include <library/cpp/monlib/encode/spack/spack_v1.h> #include <library/cpp/monlib/service/format.h> namespace NMonitoring { - void TMetricRegistryPage::Output(NMonitoring::IMonHttpRequest& request) { + void TMetricRegistryPage::Output(NMonitoring::IMonHttpRequest& request) { const auto formatStr = TStringBuf{request.GetPathInfo()}.RNextTok('/'); auto& out = request.Output(); if (!formatStr.empty()) { - IMetricEncoderPtr encoder; + IMetricEncoderPtr encoder; TString resp; if (formatStr == TStringBuf("json")) { @@ -26,7 +26,7 @@ namespace NMonitoring { resp = HTTPOKPROMETHEUS; encoder = NMonitoring::EncoderPrometheus(&out); } else { - ythrow yexception() << "unsupported metric encoding format: " << formatStr; + ythrow yexception() << "unsupported metric encoding format: " << formatStr; } out.Write(resp); @@ -38,8 +38,8 @@ namespace NMonitoring { } } - void TMetricRegistryPage::OutputText(IOutputStream& out, NMonitoring::IMonHttpRequest&) { - IMetricEncoderPtr encoder = NMonitoring::EncoderText(&out); + void TMetricRegistryPage::OutputText(IOutputStream& out, NMonitoring::IMonHttpRequest&) { + IMetricEncoderPtr encoder = NMonitoring::EncoderText(&out); RegistryRawPtr_->Accept(TInstant::Zero(), encoder.Get()); } diff --git a/library/cpp/monlib/service/pages/registry_mon_page.h b/library/cpp/monlib/service/pages/registry_mon_page.h index c370dfc97b6..2d26d3319c9 100644 --- a/library/cpp/monlib/service/pages/registry_mon_page.h +++ b/library/cpp/monlib/service/pages/registry_mon_page.h @@ -2,11 +2,11 @@ #include "pre_mon_page.h" -#include <library/cpp/monlib/metrics/metric_registry.h> +#include <library/cpp/monlib/metrics/metric_registry.h> namespace NMonitoring { // For now this class can only enumerate all metrics without any grouping or serve JSON/Spack/Prometheus - class TMetricRegistryPage: public TPreMonPage { + class TMetricRegistryPage: public TPreMonPage { public: TMetricRegistryPage(const TString& path, const TString& title, TAtomicSharedPtr<IMetricSupplier> registry) : TPreMonPage(path, title) diff --git a/library/cpp/monlib/service/pages/resource_mon_page.cpp b/library/cpp/monlib/service/pages/resource_mon_page.cpp index b8f7c73b25a..ec4ac6a1a73 100644 --- a/library/cpp/monlib/service/pages/resource_mon_page.cpp +++ b/library/cpp/monlib/service/pages/resource_mon_page.cpp @@ -4,7 +4,7 @@ using namespace NMonitoring; void TResourceMonPage::Output(NMonitoring::IMonHttpRequest& request) { IOutputStream& out = request.Output(); - switch (ResourceType) { + switch (ResourceType) { case TEXT: out << HTTPOKTEXT; break; diff --git a/library/cpp/monlib/service/pages/resource_mon_page.h b/library/cpp/monlib/service/pages/resource_mon_page.h index 43b1dfc9c40..f6ab67200e4 100644 --- a/library/cpp/monlib/service/pages/resource_mon_page.h +++ b/library/cpp/monlib/service/pages/resource_mon_page.h @@ -5,39 +5,39 @@ #include <library/cpp/resource/resource.h> namespace NMonitoring { - struct TResourceMonPage: public IMonPage { - public: - enum EResourceType { - BINARY, - TEXT, - JSON, - CSS, - JAVASCRIPT, - - FONT_EOT, - FONT_TTF, - FONT_WOFF, - FONT_WOFF2, - - PNG, - SVG - }; - - TResourceMonPage(const TString& path, const TString& resourceName, - const EResourceType& resourceType = BINARY) - : IMonPage(path, "") - , ResourceName(resourceName) - , ResourceType(resourceType) - { - } + struct TResourceMonPage: public IMonPage { + public: + enum EResourceType { + BINARY, + TEXT, + JSON, + CSS, + JAVASCRIPT, + + FONT_EOT, + FONT_TTF, + FONT_WOFF, + FONT_WOFF2, + + PNG, + SVG + }; + + TResourceMonPage(const TString& path, const TString& resourceName, + const EResourceType& resourceType = BINARY) + : IMonPage(path, "") + , ResourceName(resourceName) + , ResourceType(resourceType) + { + } void Output(NMonitoring::IMonHttpRequest& request) override; void NotFound(NMonitoring::IMonHttpRequest& request) const; - private: - TString ResourceName; - EResourceType ResourceType; - }; + private: + TString ResourceName; + EResourceType ResourceType; + }; -} +} diff --git a/library/cpp/monlib/service/pages/tablesorter/css_mon_page.h b/library/cpp/monlib/service/pages/tablesorter/css_mon_page.h index abd1ed27b55..c2c8330089f 100644 --- a/library/cpp/monlib/service/pages/tablesorter/css_mon_page.h +++ b/library/cpp/monlib/service/pages/tablesorter/css_mon_page.h @@ -1,13 +1,13 @@ #pragma once -#include <library/cpp/monlib/service/pages/resource_mon_page.h> +#include <library/cpp/monlib/service/pages/resource_mon_page.h> namespace NMonitoring { - struct TTablesorterCssMonPage: public TResourceMonPage { - TTablesorterCssMonPage() - : TResourceMonPage("jquery.tablesorter.css", "jquery.tablesorter.css", CSS) - { - } - }; + struct TTablesorterCssMonPage: public TResourceMonPage { + TTablesorterCssMonPage() + : TResourceMonPage("jquery.tablesorter.css", "jquery.tablesorter.css", CSS) + { + } + }; -} +} diff --git a/library/cpp/monlib/service/pages/tablesorter/js_mon_page.h b/library/cpp/monlib/service/pages/tablesorter/js_mon_page.h index 79a1a2f6677..f8a1d8254e2 100644 --- a/library/cpp/monlib/service/pages/tablesorter/js_mon_page.h +++ b/library/cpp/monlib/service/pages/tablesorter/js_mon_page.h @@ -1,13 +1,13 @@ #pragma once -#include <library/cpp/monlib/service/pages/resource_mon_page.h> +#include <library/cpp/monlib/service/pages/resource_mon_page.h> namespace NMonitoring { - struct TTablesorterJsMonPage: public TResourceMonPage { - TTablesorterJsMonPage() - : TResourceMonPage("jquery.tablesorter.js", "jquery.tablesorter.js", JAVASCRIPT) - { - } - }; + struct TTablesorterJsMonPage: public TResourceMonPage { + TTablesorterJsMonPage() + : TResourceMonPage("jquery.tablesorter.js", "jquery.tablesorter.js", JAVASCRIPT) + { + } + }; -} +} diff --git a/library/cpp/monlib/service/pages/tablesorter/ya.make b/library/cpp/monlib/service/pages/tablesorter/ya.make index a07e43750a6..b5b6a64da89 100644 --- a/library/cpp/monlib/service/pages/tablesorter/ya.make +++ b/library/cpp/monlib/service/pages/tablesorter/ya.make @@ -8,7 +8,7 @@ RESOURCE( ) PEERDIR( - library/cpp/monlib/dynamic_counters + library/cpp/monlib/dynamic_counters ) END() diff --git a/library/cpp/monlib/service/pages/templates.cpp b/library/cpp/monlib/service/pages/templates.cpp index 17442c8c9b9..ece12bea710 100644 --- a/library/cpp/monlib/service/pages/templates.cpp +++ b/library/cpp/monlib/service/pages/templates.cpp @@ -32,4 +32,4 @@ namespace NMonitoring { extern const char DTermTag[] = "dt"; extern const char DDescTag[] = "dd"; -} +} diff --git a/library/cpp/monlib/service/pages/templates.h b/library/cpp/monlib/service/pages/templates.h index 356cd13483c..b4656f059fa 100644 --- a/library/cpp/monlib/service/pages/templates.h +++ b/library/cpp/monlib/service/pages/templates.h @@ -7,76 +7,76 @@ #define WITH_SCOPED_I(var, value, label) \ if (auto var = (value)) { \ - Y_UNUSED(var); \ - goto label; \ - } else \ - label \ - : - -#define TAG(name) WITH_SCOPED(tmp, NMonitoring::name(__stream)) -#define TAG_CLASS(name, cls) WITH_SCOPED(tmp, NMonitoring::name(__stream, cls)) -#define TAG_CLASS_STYLE(name, cls, style) WITH_SCOPED(tmp, NMonitoring::name(__stream, {{"class", cls}, {"style", style}})) -#define TAG_CLASS_ID(name, cls, id) WITH_SCOPED(tmp, NMonitoring::name(__stream, cls, "", id)) -#define TAG_CLASS_FOR(name, cls, for0) WITH_SCOPED(tmp, NMonitoring::name(__stream, cls, for0)) -#define TAG_ATTRS(name, ...) WITH_SCOPED(tmp, NMonitoring::name(__stream, ##__VA_ARGS__)) - -#define HTML(str) WITH_SCOPED(__stream, NMonitoring::TOutputStreamRef(str)) - -#define HEAD() TAG(THead) -#define BODY() TAG(TBody) -#define HTML_TAG() TAG(THtml) -#define DIV() TAG(TDiv) -#define DIV_CLASS(cls) TAG_CLASS(TDiv, cls) + Y_UNUSED(var); \ + goto label; \ + } else \ + label \ + : + +#define TAG(name) WITH_SCOPED(tmp, NMonitoring::name(__stream)) +#define TAG_CLASS(name, cls) WITH_SCOPED(tmp, NMonitoring::name(__stream, cls)) +#define TAG_CLASS_STYLE(name, cls, style) WITH_SCOPED(tmp, NMonitoring::name(__stream, {{"class", cls}, {"style", style}})) +#define TAG_CLASS_ID(name, cls, id) WITH_SCOPED(tmp, NMonitoring::name(__stream, cls, "", id)) +#define TAG_CLASS_FOR(name, cls, for0) WITH_SCOPED(tmp, NMonitoring::name(__stream, cls, for0)) +#define TAG_ATTRS(name, ...) WITH_SCOPED(tmp, NMonitoring::name(__stream, ##__VA_ARGS__)) + +#define HTML(str) WITH_SCOPED(__stream, NMonitoring::TOutputStreamRef(str)) + +#define HEAD() TAG(THead) +#define BODY() TAG(TBody) +#define HTML_TAG() TAG(THtml) +#define DIV() TAG(TDiv) +#define DIV_CLASS(cls) TAG_CLASS(TDiv, cls) #define DIV_CLASS_ID(cls, id) TAG_CLASS_ID(TDiv, cls, id) -#define PRE() TAG(TPre) -#define TABLE() TAG(TTable) -#define TABLE_CLASS(cls) TAG_CLASS(TTable, cls) -#define TABLE_SORTABLE() TABLE_CLASS("table-sortable") +#define PRE() TAG(TPre) +#define TABLE() TAG(TTable) +#define TABLE_CLASS(cls) TAG_CLASS(TTable, cls) +#define TABLE_SORTABLE() TABLE_CLASS("table-sortable") #define TABLE_SORTABLE_CLASS(cls) TABLE_CLASS(cls " table-sortable") -#define TABLEHEAD() TAG(TTableHead) -#define TABLEHEAD_CLASS(cls) TAG_CLASS(TTableHead, cls) -#define TABLEBODY() TAG(TTableBody) -#define TABLEBODY_CLASS(cls) TAG_CLASS(TTableBody, cls) -#define TABLER() TAG(TTableR) -#define TABLER_CLASS(cls) TAG_CLASS(TTableR, cls) -#define TABLED() TAG(TTableD) -#define TABLED_CLASS(cls) TAG_CLASS(TTableD, cls) -#define TABLED_ATTRS(...) TAG_ATTRS(TTableD, ##__VA_ARGS__) -#define TABLEH() TAG(TTableH) -#define TABLEH_CLASS(cls) TAG_CLASS(TTableH, cls) -#define FORM() TAG(TFormC) -#define FORM_CLASS(cls) TAG_CLASS(TFormC, cls) -#define LABEL() TAG(TLabelC) -#define LABEL_CLASS(cls) TAG_CLASS(TLabelC, cls) +#define TABLEHEAD() TAG(TTableHead) +#define TABLEHEAD_CLASS(cls) TAG_CLASS(TTableHead, cls) +#define TABLEBODY() TAG(TTableBody) +#define TABLEBODY_CLASS(cls) TAG_CLASS(TTableBody, cls) +#define TABLER() TAG(TTableR) +#define TABLER_CLASS(cls) TAG_CLASS(TTableR, cls) +#define TABLED() TAG(TTableD) +#define TABLED_CLASS(cls) TAG_CLASS(TTableD, cls) +#define TABLED_ATTRS(...) TAG_ATTRS(TTableD, ##__VA_ARGS__) +#define TABLEH() TAG(TTableH) +#define TABLEH_CLASS(cls) TAG_CLASS(TTableH, cls) +#define FORM() TAG(TFormC) +#define FORM_CLASS(cls) TAG_CLASS(TFormC, cls) +#define LABEL() TAG(TLabelC) +#define LABEL_CLASS(cls) TAG_CLASS(TLabelC, cls) #define LABEL_CLASS_FOR(cls, for0) TAG_CLASS_FOR(TLabelC, cls, for0) -#define SPAN_CLASS(cls) TAG_CLASS(TSpanC, cls) +#define SPAN_CLASS(cls) TAG_CLASS(TSpanC, cls) #define SPAN_CLASS_STYLE(cls, style) TAG_CLASS_STYLE(TSpanC, cls, style) -#define PARA() TAG(TPara) -#define PARA_CLASS(cls) TAG_CLASS(TPara, cls) - -#define H1() TAG(TH1) -#define H1_CLASS(cls) TAG_CLASS(TH1, cls) -#define H2() TAG(TH2) -#define H2_CLASS(cls) TAG_CLASS(TH2, cls) -#define H3() TAG(TH3) -#define H3_CLASS(cls) TAG_CLASS(TH3, cls) -#define H4() TAG(TH4) -#define H4_CLASS(cls) TAG_CLASS(TH4, cls) -#define H5() TAG(TH5) -#define H5_CLASS(cls) TAG_CLASS(TH5, cls) -#define H6() TAG(TH6) -#define H6_CLASS(cls) TAG_CLASS(TH6, cls) - -#define SMALL() TAG(TSMALL) -#define STRONG() TAG(TSTRONG) - -#define LI() TAG(TLIST) -#define LI_CLASS(cls) TAG_CLASS(TLIST, cls) -#define UL() TAG(TULIST) -#define UL_CLASS(cls) TAG_CLASS(TULIST, cls) -#define OL() TAG(TOLIST) -#define OL_CLASS(cls) TAG_CLASS(TOLIST, cls) +#define PARA() TAG(TPara) +#define PARA_CLASS(cls) TAG_CLASS(TPara, cls) + +#define H1() TAG(TH1) +#define H1_CLASS(cls) TAG_CLASS(TH1, cls) +#define H2() TAG(TH2) +#define H2_CLASS(cls) TAG_CLASS(TH2, cls) +#define H3() TAG(TH3) +#define H3_CLASS(cls) TAG_CLASS(TH3, cls) +#define H4() TAG(TH4) +#define H4_CLASS(cls) TAG_CLASS(TH4, cls) +#define H5() TAG(TH5) +#define H5_CLASS(cls) TAG_CLASS(TH5, cls) +#define H6() TAG(TH6) +#define H6_CLASS(cls) TAG_CLASS(TH6, cls) + +#define SMALL() TAG(TSMALL) +#define STRONG() TAG(TSTRONG) + +#define LI() TAG(TLIST) +#define LI_CLASS(cls) TAG_CLASS(TLIST, cls) +#define UL() TAG(TULIST) +#define UL_CLASS(cls) TAG_CLASS(TULIST, cls) +#define OL() TAG(TOLIST) +#define OL_CLASS(cls) TAG_CLASS(TOLIST, cls) #define DL() TAG(DLIST) #define DL_CLASS(cls) TAG_CLASS(DLIST, cls) @@ -85,8 +85,8 @@ #define DD() TAG(DDESC) #define DD_CLASS(cls) TAG_CLASS(DDESC, cls) -#define CAPTION() TAG(TCaption) -#define CAPTION_CLASS(cls) CAPTION_CLASS(TCaption, cls) +#define CAPTION() TAG(TCaption) +#define CAPTION_CLASS(cls) CAPTION_CLASS(TCaption, cls) #define HTML_OUTPUT_PARAM(str, param) str << #param << ": " << param << "<br/>" #define HTML_OUTPUT_TIME_PARAM(str, param) str << #param << ": " << ToStringLocalTimeUpToSeconds(param) << "<br/>" @@ -116,9 +116,9 @@ namespace NMonitoring { IOutputStream& Str; }; - template <const char* tag> + template <const char* tag> struct TTag { - TTag(IOutputStream& str, TStringBuf cls = "", TStringBuf for0 = "", TStringBuf id = "") + TTag(IOutputStream& str, TStringBuf cls = "", TStringBuf for0 = "", TStringBuf id = "") : Str(str) { Str << "<" << tag; @@ -137,7 +137,7 @@ namespace NMonitoring { Str << ">"; } - TTag(IOutputStream& str, std::initializer_list<std::pair<TStringBuf, TStringBuf>> attributes) + TTag(IOutputStream& str, std::initializer_list<std::pair<TStringBuf, TStringBuf>> attributes) : Str(str) { Str << "<" << tag; @@ -152,20 +152,20 @@ namespace NMonitoring { ~TTag() { try { Str << "</" << tag << ">"; - } catch (...) { - } + } catch (...) { + } } explicit inline operator bool() const noexcept { return true; // just to work with WITH_SCOPED } - IOutputStream& Str; + IOutputStream& Str; }; // a nice class for creating collapsable regions of html output struct TCollapsedButton { - TCollapsedButton(IOutputStream& str, const TString& targetId, const TString& buttonText) + TCollapsedButton(IOutputStream& str, const TString& targetId, const TString& buttonText) : Str(str) { Str << "<button type='button' class='btn' data-toggle='collapse' data-target='#" << targetId << "'>" @@ -176,22 +176,22 @@ namespace NMonitoring { ~TCollapsedButton() { try { Str << "</div>"; - } catch (...) { - } + } catch (...) { + } } explicit inline operator bool() const noexcept { return true; // just to work with WITH_SCOPED } - IOutputStream& Str; + IOutputStream& Str; }; struct TOutputStreamRef { TOutputStreamRef(IOutputStream& str) : Str(str) - { - } + { + } inline operator IOutputStream&() noexcept { return Str; @@ -265,4 +265,4 @@ namespace NMonitoring { typedef TTag<DListTag> DLIST; typedef TTag<DTermTag> DTERM; typedef TTag<DDescTag> DDESC; -} +} diff --git a/library/cpp/monlib/service/pages/version_mon_page.h b/library/cpp/monlib/service/pages/version_mon_page.h index 1ccfc6698e9..f7649947e4c 100644 --- a/library/cpp/monlib/service/pages/version_mon_page.h +++ b/library/cpp/monlib/service/pages/version_mon_page.h @@ -3,13 +3,13 @@ #include "pre_mon_page.h" namespace NMonitoring { - struct TVersionMonPage: public TPreMonPage { - TVersionMonPage(const TString& path = "ver", const TString& title = "Version") - : TPreMonPage(path, title) - { - } + struct TVersionMonPage: public TPreMonPage { + TVersionMonPage(const TString& path = "ver", const TString& title = "Version") + : TPreMonPage(path, title) + { + } void OutputText(IOutputStream& out, NMonitoring::IMonHttpRequest&) override; - }; + }; -} +} diff --git a/library/cpp/monlib/service/pages/ya.make b/library/cpp/monlib/service/pages/ya.make index b9b6ca5646e..48d44a0838b 100644 --- a/library/cpp/monlib/service/pages/ya.make +++ b/library/cpp/monlib/service/pages/ya.make @@ -1,31 +1,31 @@ -LIBRARY() - +LIBRARY() + OWNER(g:solomon) - + NO_WSHADOW() -SRCS( - diag_mon_page.cpp - html_mon_page.cpp - index_mon_page.cpp - mon_page.cpp - pre_mon_page.cpp - resource_mon_page.cpp - templates.cpp - version_mon_page.cpp +SRCS( + diag_mon_page.cpp + html_mon_page.cpp + index_mon_page.cpp + mon_page.cpp + pre_mon_page.cpp + resource_mon_page.cpp + templates.cpp + version_mon_page.cpp registry_mon_page.cpp -) - -PEERDIR( +) + +PEERDIR( library/cpp/build_info library/cpp/malloc/api library/cpp/svnversion library/cpp/resource - library/cpp/monlib/service - library/cpp/monlib/encode/json - library/cpp/monlib/encode/text - library/cpp/monlib/encode/spack + library/cpp/monlib/service + library/cpp/monlib/encode/json + library/cpp/monlib/encode/text + library/cpp/monlib/encode/spack library/cpp/monlib/encode/prometheus -) - -END() +) + +END() diff --git a/library/cpp/monlib/service/service.cpp b/library/cpp/monlib/service/service.cpp index 0db0e206d02..929efbf8162 100644 --- a/library/cpp/monlib/service/service.cpp +++ b/library/cpp/monlib/service/service.cpp @@ -7,13 +7,13 @@ #include <library/cpp/uri/http_url.h> #include <util/generic/buffer.h> -#include <util/stream/str.h> +#include <util/stream/str.h> #include <util/stream/buffer.h> -#include <util/stream/zerocopy.h> -#include <util/string/vector.h> - +#include <util/stream/zerocopy.h> +#include <util/string/vector.h> + namespace NMonitoring { - class THttpClient: public IHttpRequest { + class THttpClient: public IHttpRequest { public: void ServeRequest(THttpInput& in, IOutputStream& out, const NAddr::IRemoteAddr* remoteAddr, const THandler& Handler) { try { @@ -36,7 +36,7 @@ namespace NMonitoring { } Headers = &in.Headers(); CgiParams.Scan(Url.Get(THttpURL::FieldQuery)); - } catch (...) { + } catch (...) { out << "HTTP/1.1 500 Internal server error\r\nConnection: Close\r\n\r\n"; YSYSLOG(TLOG_ERR, "THttpClient: internal error while serving monitoring request: %s", CurrentExceptionMessage().data()); } @@ -46,7 +46,7 @@ namespace NMonitoring { Handler(out, *this); out.Finish(); - } catch (...) { + } catch (...) { auto msg = CurrentExceptionMessage(); out << "HTTP/1.1 500 Internal server error\r\nConnection: Close\r\n\r\n" << msg; out.Finish(); @@ -102,7 +102,7 @@ namespace NMonitoring { /* TCoHttpServer */ - class TCoHttpServer::TConnection: public THttpClient { + class TCoHttpServer::TConnection: public THttpClient { public: TConnection(const TCoHttpServer::TAcceptFull& acc, const TCoHttpServer& parent) : Socket(acc.S->Release()) @@ -111,7 +111,7 @@ namespace NMonitoring { { } - void operator()(TCont* c) { + void operator()(TCont* c) { try { THolder<TConnection> me(this); TContIO io(Socket, c); @@ -122,11 +122,11 @@ namespace NMonitoring { ServeRequest(in, s, RemoteAddr, Parent.Handler); out << s.Str(); out.Finish(); - } catch (...) { + } catch (...) { YSYSLOG(TLOG_WARNING, "TCoHttpServer::TConnection: error: %s\n", CurrentExceptionMessage().data()); } } - + private: TSocketHolder Socket; const NAddr::IRemoteAddr* RemoteAddr; @@ -171,15 +171,15 @@ namespace NMonitoring { TSocket sock(addr); TSocketOutput sock_out(sock); TSocketInput sock_in(sock); - sock_out << "GET " << request.GetURI() << " HTTP/1.0\r\n\r\n"; + sock_out << "GET " << request.GetURI() << " HTTP/1.0\r\n\r\n"; THttpInput http_in(&sock_in); try { out << "HTTP/1.1 200 Ok\nConnection: Close\n\n"; TransferData(&http_in, &out); - } catch (...) { + } catch (...) { YSYSLOG(TLOG_DEBUG, "TCoHttpServer: while getting data from backend: %s", CurrentExceptionMessage().data()); } - } catch (const yexception& /*e*/) { + } catch (const yexception& /*e*/) { out << "HTTP/1.1 500 Internal server error\nConnection: Close\n\n"; YSYSLOG(TLOG_DEBUG, "TCoHttpServer: while getting data from backend: %s", CurrentExceptionMessage().data()); } @@ -187,7 +187,7 @@ namespace NMonitoring { /* TMtHttpServer */ - class TMtHttpServer::TConnection: public TClientRequest, public THttpClient { + class TMtHttpServer::TConnection: public TClientRequest, public THttpClient { public: TConnection(const TMtHttpServer& parent) : Parent(parent) @@ -198,7 +198,7 @@ namespace NMonitoring { ServeRequest(Input(), Output(), NAddr::GetPeerAddr(Socket()).Get(), Parent.Handler); return true; } - + private: const TMtHttpServer& Parent; }; @@ -215,32 +215,32 @@ namespace NMonitoring { { } - bool TMtHttpServer::Start() { - return THttpServer::Start(); - } - - void TMtHttpServer::StartOrThrow() { - if (!Start()) { - const auto& opts = THttpServer::Options(); - TNetworkAddress addr = opts.Host + bool TMtHttpServer::Start() { + return THttpServer::Start(); + } + + void TMtHttpServer::StartOrThrow() { + if (!Start()) { + const auto& opts = THttpServer::Options(); + TNetworkAddress addr = opts.Host ? TNetworkAddress(opts.Host, opts.Port) : TNetworkAddress(opts.Port); - ythrow TSystemError(GetErrorCode()) << addr; - } - } - - void TMtHttpServer::Stop() { - THttpServer::Stop(); - } - + ythrow TSystemError(GetErrorCode()) << addr; + } + } + + void TMtHttpServer::Stop() { + THttpServer::Stop(); + } + TClientRequest* TMtHttpServer::CreateClient() { return new TConnection(*this); } /* TService */ - TMonService::TMonService(TContExecutor& executor, TIpPort internalPort, TIpPort externalPort, - THandler coHandler, THandler mtHandler) + TMonService::TMonService(TContExecutor& executor, TIpPort internalPort, TIpPort externalPort, + THandler coHandler, THandler mtHandler) : CoServer(executor, "127.0.0.1", internalPort, std::move(coHandler)) , MtServer(THttpServerOptions(externalPort), std::bind(&TMonService::DispatchRequest, this, std::placeholders::_1, std::placeholders::_2)) , MtHandler(std::move(mtHandler)) @@ -261,8 +261,8 @@ namespace NMonitoring { if (strcmp(request.GetPath(), "/") == 0) { out << "HTTP/1.1 200 Ok\nConnection: Close\n\n"; MtHandler(out, request); - } else + } else CoServer.ProcessRequest(out, request); } -} +} diff --git a/library/cpp/monlib/service/service.h b/library/cpp/monlib/service/service.h index 1bff927199d..2f66dddaf87 100644 --- a/library/cpp/monlib/service/service.h +++ b/library/cpp/monlib/service/service.h @@ -9,14 +9,14 @@ #include <util/network/ip.h> #include <library/cpp/cgiparam/cgiparam.h> -#include <functional> +#include <functional> struct TMonitor; namespace NMonitoring { struct IHttpRequest { - virtual ~IHttpRequest() { - } + virtual ~IHttpRequest() { + } virtual const char* GetURI() const = 0; virtual const char* GetPath() const = 0; virtual const TCgiParameters& GetParams() const = 0; @@ -28,9 +28,9 @@ namespace NMonitoring { }; // first param - output stream to write result to // second param - URL of request - typedef std::function<void(IOutputStream&, const IHttpRequest&)> THandler; + typedef std::function<void(IOutputStream&, const IHttpRequest&)> THandler; - class TCoHttpServer: private TContListener::ICallBack { + class TCoHttpServer: private TContListener::ICallBack { public: // initialize and schedule coroutines for execution TCoHttpServer(TContExecutor& executor, const TString& bindAddr, TIpPort port, THandler handler); @@ -42,14 +42,14 @@ namespace NMonitoring { // @note this call may be blocking; don't use inside coroutines // @throws may throw in case of connection error, etc void ProcessRequest(IOutputStream&, const IHttpRequest&); - + private: class TConnection; // ICallBack implementation void OnAcceptFull(const TAcceptFull& a) override; void OnError() override; - + private: TContExecutor& Executor; TContListener Listener; @@ -58,30 +58,30 @@ namespace NMonitoring { TIpPort Port; }; - class TMtHttpServer: public THttpServer, private THttpServer::ICallBack { + class TMtHttpServer: public THttpServer, private THttpServer::ICallBack { public: TMtHttpServer(const TOptions& options, THandler handler, IThreadFactory* pool = nullptr); TMtHttpServer(const TOptions& options, THandler handler, TSimpleSharedPtr<IThreadPool> pool); - - /** - * This will cause the server start to accept incoming connections. - * - * @return true if the port binding was successfull, - * false otherwise. - */ - bool Start(); - - /** - * Same as Start() member-function, but will throw TSystemError if - * there were some errors. - */ - void StartOrThrow(); - - /** - * Stops the server from accepting new connections. - */ - void Stop(); - + + /** + * This will cause the server start to accept incoming connections. + * + * @return true if the port binding was successfull, + * false otherwise. + */ + bool Start(); + + /** + * Same as Start() member-function, but will throw TSystemError if + * there were some errors. + */ + void StartOrThrow(); + + /** + * Stops the server from accepting new connections. + */ + void Stop(); + private: class TConnection; TClientRequest* CreateClient() override; @@ -95,18 +95,18 @@ namespace NMonitoring { // will be served in a coroutine context class TMonService { public: - TMonService(TContExecutor& executor, TIpPort internalPort, TIpPort externalPort, - THandler coHandler, THandler mtHandler); + TMonService(TContExecutor& executor, TIpPort internalPort, TIpPort externalPort, + THandler coHandler, THandler mtHandler); void Start(); void Stop(); - + protected: void DispatchRequest(IOutputStream& out, const IHttpRequest&); - + private: TCoHttpServer CoServer; TMtHttpServer MtServer; THandler MtHandler; }; -} +} diff --git a/library/cpp/monlib/service/ya.make b/library/cpp/monlib/service/ya.make index 7240be80713..ad088fc2c6c 100644 --- a/library/cpp/monlib/service/ya.make +++ b/library/cpp/monlib/service/ya.make @@ -1,18 +1,18 @@ -LIBRARY() - +LIBRARY() + OWNER(g:solomon) - -SRCS( - monservice.cpp - mon_service_http_request.cpp - service.cpp + +SRCS( + monservice.cpp + mon_service_http_request.cpp + service.cpp format.cpp auth.cpp -) - -PEERDIR( +) + +PEERDIR( library/cpp/string_utils/base64 - contrib/libs/protobuf + contrib/libs/protobuf library/cpp/coroutine/engine library/cpp/coroutine/listener library/cpp/http/fetch @@ -23,6 +23,6 @@ PEERDIR( library/cpp/svnversion library/cpp/uri library/cpp/cgiparam -) - -END() +) + +END() diff --git a/library/cpp/monlib/ya.make b/library/cpp/monlib/ya.make index 12986ec20c5..9bd236d6fda 100644 --- a/library/cpp/monlib/ya.make +++ b/library/cpp/monlib/ya.make @@ -3,43 +3,43 @@ OWNER( jamel ) -RECURSE( +RECURSE( consumers - counters - counters/ut - deprecated - dynamic_counters + counters + counters/ut + deprecated + dynamic_counters dynamic_counters/percentile dynamic_counters/percentile/ut - dynamic_counters/ut - encode - encode/buffered - encode/buffered/ut + dynamic_counters/ut + encode + encode/buffered + encode/buffered/ut encode/fake encode/fuzz - encode/json - encode/json/ut + encode/json + encode/json/ut encode/legacy_protobuf encode/legacy_protobuf/ut - encode/prometheus - encode/prometheus/ut + encode/prometheus + encode/prometheus/ut encode/protobuf - encode/spack - encode/spack/ut - encode/text - encode/text/ut + encode/spack + encode/spack/ut + encode/text + encode/text/ut encode/unistat encode/unistat/ut - encode/ut - example + encode/ut + example exception - libtimestats/ut - metrics - metrics/ut - messagebus - push_client - service + libtimestats/ut + metrics + metrics/ut + messagebus + push_client + service service/auth/tvm - service/pages - service/pages/tablesorter + service/pages + service/pages/tablesorter ) diff --git a/library/cpp/on_disk/chunks/chunks_ut.cpp b/library/cpp/on_disk/chunks/chunks_ut.cpp index 5b2c7d04d1a..f727647f7f2 100644 --- a/library/cpp/on_disk/chunks/chunks_ut.cpp +++ b/library/cpp/on_disk/chunks/chunks_ut.cpp @@ -55,7 +55,7 @@ public: TPlainHash<ui64, ui16> reader(temp); ui16 value = 0; UNIT_ASSERT(reader.Find(5, &value)); - UNIT_ASSERT_EQUAL(7, value); + UNIT_ASSERT_EQUAL(7, value); UNIT_ASSERT(!reader.Find(6, &value)); } } diff --git a/library/cpp/packers/ut/packers_ut.cpp b/library/cpp/packers/ut/packers_ut.cpp index 1203ba95c11..18ce2150d1e 100644 --- a/library/cpp/packers/ut/packers_ut.cpp +++ b/library/cpp/packers/ut/packers_ut.cpp @@ -54,7 +54,7 @@ void TPackersTest::TestPacker(const TData& data) { TData dataTmp; TPacker().UnpackLeaf(buf.Get(), dataTmp); - UNIT_ASSERT(data == dataTmp); + UNIT_ASSERT(data == dataTmp); } template <class TData, class TPacker> diff --git a/library/cpp/scheme/tests/ut/scheme_json_ut.cpp b/library/cpp/scheme/tests/ut/scheme_json_ut.cpp index 23f6e1a3463..daeb2654f9a 100644 --- a/library/cpp/scheme/tests/ut/scheme_json_ut.cpp +++ b/library/cpp/scheme/tests/ut/scheme_json_ut.cpp @@ -60,7 +60,7 @@ Y_UNIT_TEST_SUITE(TSchemeJsonTest) { } { const char* json = "[a:b]"; - UNIT_ASSERT(NSc::TValue::FromJson(json).IsNull()); + UNIT_ASSERT(NSc::TValue::FromJson(json).IsNull()); } { UNIT_ASSERT_VALUES_EQUAL("{\n \"a\" : \"b\",\n \"c\" : \"d\"\n}", diff --git a/library/cpp/scheme/tests/ut/scheme_merge_ut.cpp b/library/cpp/scheme/tests/ut/scheme_merge_ut.cpp index 8658e6c6e86..2a06cf110da 100644 --- a/library/cpp/scheme/tests/ut/scheme_merge_ut.cpp +++ b/library/cpp/scheme/tests/ut/scheme_merge_ut.cpp @@ -30,7 +30,7 @@ Y_UNIT_TEST_SUITE(TSchemeMergeTest) { TStringBuf data = "{ a : [ { b : 1, d : { e : -1.e5 } }, { f : 0, g : [ h, i ] } ] }"; NSc::TValue v = NSc::TValue::FromJson(data); UNIT_ASSERT_VALUES_EQUAL(v.ToJson(true), v.Clone().ToJson(true)); - UNIT_ASSERT(v.Has("a")); + UNIT_ASSERT(v.Has("a")); UNIT_ASSERT(v["a"].Has(1)); UNIT_ASSERT(v["a"][0].Has("b")); UNIT_ASSERT(v["a"][0].Has("d")); @@ -62,8 +62,8 @@ Y_UNIT_TEST_SUITE(TSchemeMergeTest) { UNIT_ASSERT_C(NSc::TValue::Equal(v["a"], v2), Sprintf("\n%s\n!=\n%s\n", v["a"].ToJson().data(), v2.ToJson().data())); } - UNIT_ASSERT(v.Has("a")); - UNIT_ASSERT(v.Has("q")); + UNIT_ASSERT(v.Has("a")); + UNIT_ASSERT(v.Has("q")); UNIT_ASSERT(TStringBuf("r") == v["q"]); UNIT_ASSERT(v["a"].Has(1)); UNIT_ASSERT(!v["a"][0].Has("b")); diff --git a/library/cpp/scheme/tests/ut/scheme_path_ut.cpp b/library/cpp/scheme/tests/ut/scheme_path_ut.cpp index 45026fbdb26..0d4d79d4833 100644 --- a/library/cpp/scheme/tests/ut/scheme_path_ut.cpp +++ b/library/cpp/scheme/tests/ut/scheme_path_ut.cpp @@ -18,9 +18,9 @@ Y_UNIT_TEST_SUITE(TSchemePathTest) { *v.TrySelectOrAdd(path) = 1; NSc::NUt::AssertSchemeJson(expected, v); UNIT_ASSERT(v.PathExists(path)); - UNIT_ASSERT(1 == v.TrySelectOrAdd(path)->GetNumber()); - UNIT_ASSERT(1 == v.TrySelect(path).GetNumber()); - UNIT_ASSERT(1 == v.TrySelectAndDelete(path).GetNumber()); + UNIT_ASSERT(1 == v.TrySelectOrAdd(path)->GetNumber()); + UNIT_ASSERT(1 == v.TrySelect(path).GetNumber()); + UNIT_ASSERT(1 == v.TrySelectAndDelete(path).GetNumber()); UNIT_ASSERT(NSc::TValue::Same(v.TrySelectAndDelete(path), NSc::Null())); NSc::NUt::AssertSchemeJson(delexpected, v); UNIT_ASSERT(!v.PathExists(path)); @@ -29,7 +29,7 @@ Y_UNIT_TEST_SUITE(TSchemePathTest) { Y_UNIT_TEST(TestSelect) { NSc::TValue v; - UNIT_ASSERT(!v.PathValid(" ")); + UNIT_ASSERT(!v.PathValid(" ")); UNIT_ASSERT(v.PathExists("")); UNIT_ASSERT(v.PathExists("//")); diff --git a/library/cpp/scheme/tests/ut/scheme_ut.cpp b/library/cpp/scheme/tests/ut/scheme_ut.cpp index e49ac959ff0..1a5d07c31bc 100644 --- a/library/cpp/scheme/tests/ut/scheme_ut.cpp +++ b/library/cpp/scheme/tests/ut/scheme_ut.cpp @@ -526,7 +526,7 @@ Y_UNIT_TEST_SUITE(TSchemeTest) { UNIT_ASSERT(2 > NSc::TValue(1)); UNIT_ASSERT(NSc::TValue(1) < 2); - UNIT_ASSERT(TString("test") == NSc::TValue("test")); + UNIT_ASSERT(TString("test") == NSc::TValue("test")); } Y_UNIT_TEST(TestDestructor) { diff --git a/library/cpp/sse/ut/test.cpp b/library/cpp/sse/ut/test.cpp index f72fd1cf912..33c999d284b 100644 --- a/library/cpp/sse/ut/test.cpp +++ b/library/cpp/sse/ut/test.cpp @@ -1858,7 +1858,7 @@ void TSSEEmulTest::Test_mm_storeu_pd() { for (size_t shift = 0; shift != 3; ++shift) { _mm_storeu_pd(&res[shift], value); for (size_t j = 0; j != 2; ++j) { - UNIT_ASSERT_EQUAL_C(res[j + shift], valueBits[i + j], "res: " << HexEncode(&res[shift], 16) << " vs etalon: " << HexEncode(&valueBits[i], 16)); + UNIT_ASSERT_EQUAL_C(res[j + shift], valueBits[i + j], "res: " << HexEncode(&res[shift], 16) << " vs etalon: " << HexEncode(&valueBits[i], 16)); } } } diff --git a/library/cpp/string_utils/url/url.cpp b/library/cpp/string_utils/url/url.cpp index dc3d5aa00ee..85f4ac5d693 100644 --- a/library/cpp/string_utils/url/url.cpp +++ b/library/cpp/string_utils/url/url.cpp @@ -206,35 +206,35 @@ void SeparateUrlFromQueryAndFragment(const TStringBuf url, TStringBuf& sanitized } } -bool TryGetSchemeHostAndPort(const TStringBuf url, TStringBuf& scheme, TStringBuf& host, ui16& port) { - const size_t schemeSize = GetSchemePrefixSize(url); - if (schemeSize != 0) { - scheme = url.Head(schemeSize); - } - - TStringBuf portStr; - TStringBuf hostAndPort = GetHostAndPort(url.Tail(schemeSize)); +bool TryGetSchemeHostAndPort(const TStringBuf url, TStringBuf& scheme, TStringBuf& host, ui16& port) { + const size_t schemeSize = GetSchemePrefixSize(url); + if (schemeSize != 0) { + scheme = url.Head(schemeSize); + } + + TStringBuf portStr; + TStringBuf hostAndPort = GetHostAndPort(url.Tail(schemeSize)); if (hostAndPort && hostAndPort.back() != ']' && hostAndPort.TryRSplit(':', host, portStr)) { - // URL has port - if (!TryFromString(portStr, port)) { - return false; - } - } else { - host = hostAndPort; + // URL has port + if (!TryFromString(portStr, port)) { + return false; + } + } else { + host = hostAndPort; if (scheme == TStringBuf("https://")) { - port = 443; + port = 443; } else if (scheme == TStringBuf("http://")) { - port = 80; - } - } - return true; -} - -void GetSchemeHostAndPort(const TStringBuf url, TStringBuf& scheme, TStringBuf& host, ui16& port) { - bool isOk = TryGetSchemeHostAndPort(url, scheme, host, port); - Y_ENSURE(isOk, "cannot parse port number from URL: " << url); -} - + port = 80; + } + } + return true; +} + +void GetSchemeHostAndPort(const TStringBuf url, TStringBuf& scheme, TStringBuf& host, ui16& port) { + bool isOk = TryGetSchemeHostAndPort(url, scheme, host, port); + Y_ENSURE(isOk, "cannot parse port number from URL: " << url); +} + TStringBuf GetOnlyHost(const TStringBuf url) noexcept { return GetHost(CutSchemePrefix(url)); } diff --git a/library/cpp/string_utils/url/url.h b/library/cpp/string_utils/url/url.h index 5e797e374aa..84137ccc57d 100644 --- a/library/cpp/string_utils/url/url.h +++ b/library/cpp/string_utils/url/url.h @@ -65,14 +65,14 @@ TString AddSchemePrefix(const TString& url); Y_PURE_FUNCTION TStringBuf GetHost(const TStringBuf url) noexcept; - + Y_PURE_FUNCTION TStringBuf GetHostAndPort(const TStringBuf url) noexcept; Y_PURE_FUNCTION TStringBuf GetSchemeHostAndPort(const TStringBuf url, bool trimHttp = true, bool trimDefaultPort = true) noexcept; -/** +/** * Splits URL to host and path * * @param[in] url any URL @@ -93,35 +93,35 @@ void SplitUrlToHostAndPath(const TStringBuf url, TString& host, TString& path); void SeparateUrlFromQueryAndFragment(const TStringBuf url, TStringBuf& sanitizedUrl, TStringBuf& query, TStringBuf& fragment); /** - * Extracts scheme, host and port from URL. - * - * Port will be parsed from URL with checks against ui16 overflow. If URL doesn't - * contain port it will be determined by one of the known schemes (currently - * https:// and http:// only). - * Given parameters will not be modified if URL has no appropriate components. - * - * @param[in] url any URL - * @param[out] scheme URL scheme - * @param[out] host host name - * @param[out] port parsed port number - * @return false if present port number cannot be parsed into ui16 - * true otherwise. - */ -bool TryGetSchemeHostAndPort(const TStringBuf url, TStringBuf& scheme, TStringBuf& host, ui16& port); - -/** - * Extracts scheme, host and port from URL. - * + * Extracts scheme, host and port from URL. + * + * Port will be parsed from URL with checks against ui16 overflow. If URL doesn't + * contain port it will be determined by one of the known schemes (currently + * https:// and http:// only). + * Given parameters will not be modified if URL has no appropriate components. + * + * @param[in] url any URL + * @param[out] scheme URL scheme + * @param[out] host host name + * @param[out] port parsed port number + * @return false if present port number cannot be parsed into ui16 + * true otherwise. + */ +bool TryGetSchemeHostAndPort(const TStringBuf url, TStringBuf& scheme, TStringBuf& host, ui16& port); + +/** + * Extracts scheme, host and port from URL. + * * This function perform the same actions as TryGetSchemeHostAndPort(), but in - * case of impossibility to parse port number throws yexception. - * - * @param[in] url any URL - * @param[out] scheme URL scheme - * @param[out] host host name - * @param[out] port parsed port number - * @throws yexception if present port number cannot be parsed into ui16. - */ -void GetSchemeHostAndPort(const TStringBuf url, TStringBuf& scheme, TStringBuf& host, ui16& port); + * case of impossibility to parse port number throws yexception. + * + * @param[in] url any URL + * @param[out] scheme URL scheme + * @param[out] host host name + * @param[out] port parsed port number + * @throws yexception if present port number cannot be parsed into ui16. + */ +void GetSchemeHostAndPort(const TStringBuf url, TStringBuf& scheme, TStringBuf& host, ui16& port); Y_PURE_FUNCTION TStringBuf GetPathAndQuery(const TStringBuf url, bool trimFragment = true) noexcept; diff --git a/library/cpp/string_utils/url/url_ut.cpp b/library/cpp/string_utils/url/url_ut.cpp index 3e64d3e2c54..15880138939 100644 --- a/library/cpp/string_utils/url/url_ut.cpp +++ b/library/cpp/string_utils/url/url_ut.cpp @@ -126,7 +126,7 @@ Y_UNIT_TEST_SUITE(TUtilUrlTest) { UNIT_ASSERT_VALUES_EQUAL("m", CutMPrefix("m")); UNIT_ASSERT_VALUES_EQUAL("ya.ru", CutMPrefix("m.ya.ru")); } - + Y_UNIT_TEST(TestSplitUrlToHostAndPath) { TStringBuf host, path; @@ -176,46 +176,46 @@ Y_UNIT_TEST_SUITE(TUtilUrlTest) { } Y_UNIT_TEST(TestGetSchemeHostAndPort) { - { // all components are present - TStringBuf scheme("unknown"), host("unknown"); - ui16 port = 0; - GetSchemeHostAndPort("https://ya.ru:8080/bebe", scheme, host, port); - UNIT_ASSERT_VALUES_EQUAL(scheme, "https://"); - UNIT_ASSERT_VALUES_EQUAL(host, "ya.ru"); - UNIT_ASSERT_VALUES_EQUAL(port, 8080); - } - { // scheme is abset - TStringBuf scheme("unknown"), host("unknown"); - ui16 port = 0; - GetSchemeHostAndPort("ya.ru:8080/bebe", scheme, host, port); - UNIT_ASSERT_VALUES_EQUAL(scheme, "unknown"); - UNIT_ASSERT_VALUES_EQUAL(host, "ya.ru"); - UNIT_ASSERT_VALUES_EQUAL(port, 8080); - } - { // scheme and port are absent - TStringBuf scheme("unknown"), host("unknown"); - ui16 port = 0; - GetSchemeHostAndPort("ya.ru/bebe", scheme, host, port); - UNIT_ASSERT_VALUES_EQUAL(scheme, "unknown"); - UNIT_ASSERT_VALUES_EQUAL(host, "ya.ru"); - UNIT_ASSERT_VALUES_EQUAL(port, 0); - } - { // port is absent, but returned its default value for HTTP - TStringBuf scheme("unknown"), host("unknown"); - ui16 port = 0; - GetSchemeHostAndPort("http://ya.ru/bebe", scheme, host, port); - UNIT_ASSERT_VALUES_EQUAL(scheme, "http://"); - UNIT_ASSERT_VALUES_EQUAL(host, "ya.ru"); - UNIT_ASSERT_VALUES_EQUAL(port, 80); - } - { // port is absent, but returned its default value for HTTPS - TStringBuf scheme("unknown"), host("unknown"); - ui16 port = 0; - GetSchemeHostAndPort("https://ya.ru/bebe", scheme, host, port); - UNIT_ASSERT_VALUES_EQUAL(scheme, "https://"); - UNIT_ASSERT_VALUES_EQUAL(host, "ya.ru"); - UNIT_ASSERT_VALUES_EQUAL(port, 443); - } + { // all components are present + TStringBuf scheme("unknown"), host("unknown"); + ui16 port = 0; + GetSchemeHostAndPort("https://ya.ru:8080/bebe", scheme, host, port); + UNIT_ASSERT_VALUES_EQUAL(scheme, "https://"); + UNIT_ASSERT_VALUES_EQUAL(host, "ya.ru"); + UNIT_ASSERT_VALUES_EQUAL(port, 8080); + } + { // scheme is abset + TStringBuf scheme("unknown"), host("unknown"); + ui16 port = 0; + GetSchemeHostAndPort("ya.ru:8080/bebe", scheme, host, port); + UNIT_ASSERT_VALUES_EQUAL(scheme, "unknown"); + UNIT_ASSERT_VALUES_EQUAL(host, "ya.ru"); + UNIT_ASSERT_VALUES_EQUAL(port, 8080); + } + { // scheme and port are absent + TStringBuf scheme("unknown"), host("unknown"); + ui16 port = 0; + GetSchemeHostAndPort("ya.ru/bebe", scheme, host, port); + UNIT_ASSERT_VALUES_EQUAL(scheme, "unknown"); + UNIT_ASSERT_VALUES_EQUAL(host, "ya.ru"); + UNIT_ASSERT_VALUES_EQUAL(port, 0); + } + { // port is absent, but returned its default value for HTTP + TStringBuf scheme("unknown"), host("unknown"); + ui16 port = 0; + GetSchemeHostAndPort("http://ya.ru/bebe", scheme, host, port); + UNIT_ASSERT_VALUES_EQUAL(scheme, "http://"); + UNIT_ASSERT_VALUES_EQUAL(host, "ya.ru"); + UNIT_ASSERT_VALUES_EQUAL(port, 80); + } + { // port is absent, but returned its default value for HTTPS + TStringBuf scheme("unknown"), host("unknown"); + ui16 port = 0; + GetSchemeHostAndPort("https://ya.ru/bebe", scheme, host, port); + UNIT_ASSERT_VALUES_EQUAL(scheme, "https://"); + UNIT_ASSERT_VALUES_EQUAL(host, "ya.ru"); + UNIT_ASSERT_VALUES_EQUAL(port, 443); + } { // ipv6 TStringBuf scheme("unknown"), host("unknown"); ui16 port = 0; @@ -240,14 +240,14 @@ Y_UNIT_TEST_SUITE(TUtilUrlTest) { UNIT_ASSERT_VALUES_EQUAL(host, ""); UNIT_ASSERT_VALUES_EQUAL(port, 0); } - // port overflow - auto testCase = []() { - TStringBuf scheme("unknown"), host("unknown"); - ui16 port = 0; - GetSchemeHostAndPort("https://ya.ru:65536/bebe", scheme, host, port); - }; - UNIT_ASSERT_EXCEPTION(testCase(), yexception); - } + // port overflow + auto testCase = []() { + TStringBuf scheme("unknown"), host("unknown"); + ui16 port = 0; + GetSchemeHostAndPort("https://ya.ru:65536/bebe", scheme, host, port); + }; + UNIT_ASSERT_EXCEPTION(testCase(), yexception); + } Y_UNIT_TEST(TestCutUrlPrefixes) { UNIT_ASSERT_VALUES_EQUAL("ya.ru/bebe", CutUrlPrefixes("http://ya.ru/bebe")); diff --git a/library/cpp/testing/unittest/registar.h b/library/cpp/testing/unittest/registar.h index ec74ef88dc0..44517a00924 100644 --- a/library/cpp/testing/unittest/registar.h +++ b/library/cpp/testing/unittest/registar.h @@ -375,26 +375,26 @@ public: \ #define UNIT_FAIL_NONFATAL(M) UNIT_FAIL_NONFATAL_IMPL("forced failure", M) //types -#define UNIT_ASSERT_TYPES_EQUAL(A, B) \ - do { \ - if (!std::is_same<A, B>::value) { \ +#define UNIT_ASSERT_TYPES_EQUAL(A, B) \ + do { \ + if (!std::is_same<A, B>::value) { \ UNIT_FAIL_IMPL("types equal assertion failed", (::TStringBuilder() << #A << " (" << TypeName<A>() << ") != " << #B << " (" << TypeName<B>() << ")").data()); \ - } \ - } while (false) + } \ + } while (false) //doubles // UNIT_ASSERT_DOUBLES_EQUAL_DEPRECATED* macros do not handle NaNs correctly (see IGNIETFERRO-1419) and are for backward compatibility // only. Consider switching to regular UNIT_ASSERT_DOUBLES_EQUAL* macros if you're still using the deprecated version. #define UNIT_ASSERT_DOUBLES_EQUAL_DEPRECATED_C(E, A, D, C) \ - do { \ - if (std::abs((E) - (A)) > (D)) { \ - const auto _es = ToString((long double)(E)); \ - const auto _as = ToString((long double)(A)); \ - const auto _ds = ToString((long double)(D)); \ + do { \ + if (std::abs((E) - (A)) > (D)) { \ + const auto _es = ToString((long double)(E)); \ + const auto _as = ToString((long double)(A)); \ + const auto _ds = ToString((long double)(D)); \ auto&& failMsg = Sprintf("std::abs(%s - %s) > %s %s", _es.data(), _as.data(), _ds.data(), (::TStringBuilder() << C).data()); \ - UNIT_FAIL_IMPL("assertion failure", failMsg); \ - } \ - } while (false) + UNIT_FAIL_IMPL("assertion failure", failMsg); \ + } \ + } while (false) #define UNIT_ASSERT_DOUBLES_EQUAL_DEPRECATED(E, A, D) UNIT_ASSERT_DOUBLES_EQUAL_DEPRECATED_C(E, A, D, "") @@ -472,72 +472,72 @@ public: \ #define UNIT_ASSERT_STRINGS_UNEQUAL(A, B) UNIT_ASSERT_STRINGS_UNEQUAL_C(A, B, "") //bool -#define UNIT_ASSERT_C(A, C) \ - do { \ - if (!(A)) { \ +#define UNIT_ASSERT_C(A, C) \ + do { \ + if (!(A)) { \ UNIT_FAIL_IMPL("assertion failed", Sprintf("(%s) %s", #A, (::TStringBuilder() << C).data())); \ - } \ - } while (false) + } \ + } while (false) #define UNIT_ASSERT(A) UNIT_ASSERT_C(A, "") //general -#define UNIT_ASSERT_EQUAL_C(A, B, C) \ - do { \ - if (!((A) == (B))) { \ +#define UNIT_ASSERT_EQUAL_C(A, B, C) \ + do { \ + if (!((A) == (B))) { \ UNIT_FAIL_IMPL("equal assertion failed", Sprintf("%s == %s %s", #A, #B, (::TStringBuilder() << C).data())); \ - } \ - } while (false) + } \ + } while (false) #define UNIT_ASSERT_EQUAL(A, B) UNIT_ASSERT_EQUAL_C(A, B, "") -#define UNIT_ASSERT_UNEQUAL_C(A, B, C) \ - do { \ - if ((A) == (B)) { \ +#define UNIT_ASSERT_UNEQUAL_C(A, B, C) \ + do { \ + if ((A) == (B)) { \ UNIT_FAIL_IMPL("unequal assertion failed", Sprintf("%s != %s %s", #A, #B, (::TStringBuilder() << C).data()));\ - } \ - } while (false) + } \ + } while (false) #define UNIT_ASSERT_UNEQUAL(A, B) UNIT_ASSERT_UNEQUAL_C(A, B, "") -#define UNIT_ASSERT_LT_C(A, B, C) \ - do { \ - if (!((A) < (B))) { \ +#define UNIT_ASSERT_LT_C(A, B, C) \ + do { \ + if (!((A) < (B))) { \ UNIT_FAIL_IMPL("less-than assertion failed", Sprintf("%s < %s %s", #A, #B, (::TStringBuilder() << C).data())); \ - } \ - } while (false) + } \ + } while (false) #define UNIT_ASSERT_LT(A, B) UNIT_ASSERT_LT_C(A, B, "") -#define UNIT_ASSERT_LE_C(A, B, C) \ - do { \ - if (!((A) <= (B))) { \ +#define UNIT_ASSERT_LE_C(A, B, C) \ + do { \ + if (!((A) <= (B))) { \ UNIT_FAIL_IMPL("less-or-equal assertion failed", Sprintf("%s <= %s %s", #A, #B, (::TStringBuilder() << C).data())); \ - } \ - } while (false) + } \ + } while (false) #define UNIT_ASSERT_LE(A, B) UNIT_ASSERT_LE_C(A, B, "") -#define UNIT_ASSERT_GT_C(A, B, C) \ - do { \ - if (!((A) > (B))) { \ +#define UNIT_ASSERT_GT_C(A, B, C) \ + do { \ + if (!((A) > (B))) { \ UNIT_FAIL_IMPL("greater-than assertion failed", Sprintf("%s > %s %s", #A, #B, (::TStringBuilder() << C).data())); \ - } \ - } while (false) + } \ + } while (false) #define UNIT_ASSERT_GT(A, B) UNIT_ASSERT_GT_C(A, B, "") #define UNIT_ASSERT_GE_C(A, B, C) \ - do { \ - if (!((A) >= (B))) { \ + do { \ + if (!((A) >= (B))) { \ UNIT_FAIL_IMPL("greater-or-equal assertion failed", Sprintf("%s >= %s %s", #A, #B, (::TStringBuilder() << C).data())); \ - } \ - } while (false) + } \ + } while (false) #define UNIT_ASSERT_GE(A, B) UNIT_ASSERT_GE_C(A, B, "") #define UNIT_CHECK_GENERATED_EXCEPTION_C(A, E, C) \ - do { \ + do { \ try { \ (void)(A); \ } catch (const ::NUnitTest::TAssertException&) { \ @@ -546,20 +546,20 @@ public: \ break; \ } \ UNIT_ASSERT_C(0, "Exception hasn't been thrown, but it should have happened " << C); \ - } while (false) + } while (false) #define UNIT_CHECK_GENERATED_EXCEPTION(A, E) UNIT_CHECK_GENERATED_EXCEPTION_C(A, E, "") -#define UNIT_CHECK_GENERATED_NO_EXCEPTION_C(A, E, C) \ - do { \ - try { \ - (void)(A); \ - } catch (const ::NUnitTest::TAssertException&) { \ - throw; \ - } catch (const E&) { \ - UNIT_ASSERT_C(0, "Exception has been thrown, but it shouldn't have happened " << C); \ - } \ - } while (false) +#define UNIT_CHECK_GENERATED_NO_EXCEPTION_C(A, E, C) \ + do { \ + try { \ + (void)(A); \ + } catch (const ::NUnitTest::TAssertException&) { \ + throw; \ + } catch (const E&) { \ + UNIT_ASSERT_C(0, "Exception has been thrown, but it shouldn't have happened " << C); \ + } \ + } while (false) #define UNIT_CHECK_GENERATED_NO_EXCEPTION(A, E) UNIT_CHECK_GENERATED_NO_EXCEPTION_C(A, E, "and exception message is:\n" << CurrentExceptionMessage()) diff --git a/library/python/ya.make b/library/python/ya.make index 02fdb9095ca..2e1eb6e0e1f 100644 --- a/library/python/ya.make +++ b/library/python/ya.make @@ -119,7 +119,7 @@ RECURSE( metrics_framework mime_types monitoring - monlib + monlib monlib/examples monlib/so murmurhash @@ -169,14 +169,14 @@ RECURSE( selenium_ui_test sendmsg stubmaker - solomon + solomon spack spyt ssh_client ssh_sign startrek_python_client startrek_python_client/tests_int - statface_client + statface_client step strings strings/ut diff --git a/library/ya.make b/library/ya.make index 54897a22610..86f833bd02d 100644 --- a/library/ya.make +++ b/library/ya.make @@ -11,7 +11,7 @@ RECURSE( IF (NOT SANITIZER_TYPE) RECURSE( go - java + java ) ENDIF() diff --git a/util/draft/datetime_ut.cpp b/util/draft/datetime_ut.cpp index 76f05083021..a5e065ef6e8 100644 --- a/util/draft/datetime_ut.cpp +++ b/util/draft/datetime_ut.cpp @@ -54,11 +54,11 @@ Y_UNIT_TEST_SUITE(TSimpleTMTest) { Y_UNIT_TEST(TestLeap) { using namespace NDatetime; - UNIT_ASSERT(LeapYearAD(2000)); - UNIT_ASSERT(LeapYearAD(2012)); - UNIT_ASSERT(!LeapYearAD(1999)); - UNIT_ASSERT(LeapYearAD(2004)); - UNIT_ASSERT(!LeapYearAD(1900)); + UNIT_ASSERT(LeapYearAD(2000)); + UNIT_ASSERT(LeapYearAD(2012)); + UNIT_ASSERT(!LeapYearAD(1999)); + UNIT_ASSERT(LeapYearAD(2004)); + UNIT_ASSERT(!LeapYearAD(1900)); } Y_UNIT_TEST(TestYDayConversion) { diff --git a/util/generic/hash_ut.cpp b/util/generic/hash_ut.cpp index 81b30cf9f17..0551d587708 100644 --- a/util/generic/hash_ut.cpp +++ b/util/generic/hash_ut.cpp @@ -361,9 +361,9 @@ void THashTest::TestHMMapHas() { m.insert(std::pair<const char, int>('X', 10)); m.insert(std::pair<const char, int>('X', 20)); m.insert(std::pair<const char, int>('Y', 32)); - UNIT_ASSERT(m.contains('X')); - UNIT_ASSERT(m.contains('Y')); - UNIT_ASSERT(!m.contains('Z')); + UNIT_ASSERT(m.contains('X')); + UNIT_ASSERT(m.contains('Y')); + UNIT_ASSERT(!m.contains('Z')); } void THashTest::TestHSetConstructorsAndAssignments() { diff --git a/util/generic/iterator_ut.cpp b/util/generic/iterator_ut.cpp index 8c43f15b364..00be19e10ec 100644 --- a/util/generic/iterator_ut.cpp +++ b/util/generic/iterator_ut.cpp @@ -58,6 +58,6 @@ Y_UNIT_TEST_SUITE(TInputRangeAdaptor) { UNIT_ASSERT_VALUES_EQUAL(part, *expected_part); ++expected_part; } - UNIT_ASSERT(expected_part == expected.end()); + UNIT_ASSERT(expected_part == expected.end()); } } diff --git a/util/generic/lazy_value_ut.cpp b/util/generic/lazy_value_ut.cpp index 586040a8c2f..f6135880c3c 100644 --- a/util/generic/lazy_value_ut.cpp +++ b/util/generic/lazy_value_ut.cpp @@ -39,7 +39,7 @@ Y_UNIT_TEST_SUITE(TLazyValueTestSuite) { TLazyValue<int> notEmptyCopy = value; UNIT_ASSERT(notEmptyCopy); - UNIT_ASSERT_EQUAL(*notEmptyCopy, 5); + UNIT_ASSERT_EQUAL(*notEmptyCopy, 5); } struct TCopyCounter { diff --git a/util/generic/yexception_ut.cpp b/util/generic/yexception_ut.cpp index 4d827795832..cb3e29fed84 100644 --- a/util/generic/yexception_ut.cpp +++ b/util/generic/yexception_ut.cpp @@ -66,7 +66,7 @@ private: throw; } } catch (...) { - UNIT_ASSERT(CurrentExceptionMessage().Contains("ithappens")); + UNIT_ASSERT(CurrentExceptionMessage().Contains("ithappens")); } } diff --git a/util/network/socket_ut.cpp b/util/network/socket_ut.cpp index c3911dd197c..6b20e11f70a 100644 --- a/util/network/socket_ut.cpp +++ b/util/network/socket_ut.cpp @@ -169,7 +169,7 @@ void TSockTest::TestBrokenPipe() { void TSockTest::TestClose() { SOCKET socks[2]; - UNIT_ASSERT_EQUAL(SocketPair(socks), 0); + UNIT_ASSERT_EQUAL(SocketPair(socks), 0); TSocket receiver(socks[1]); UNIT_ASSERT_EQUAL(static_cast<SOCKET>(receiver), socks[1]); diff --git a/util/system/guard_ut.cpp b/util/system/guard_ut.cpp index cab4ca9e37e..404ede99ab9 100644 --- a/util/system/guard_ut.cpp +++ b/util/system/guard_ut.cpp @@ -13,7 +13,7 @@ struct TTestGuard: public TTestBase { UNIT_TEST(TestSync) UNIT_TEST(TestUnguard) UNIT_TEST(TestTryReadGuard) - UNIT_TEST(TestWithLock) + UNIT_TEST(TestWithLock) UNIT_TEST(TestWithLockScope); UNIT_TEST_SUITE_END(); @@ -119,11 +119,11 @@ struct TTestGuard: public TTestBase { { TTryGuard<TGuardChecker> guard(checker); UNIT_ASSERT(checker.guarded); - UNIT_ASSERT(guard.WasAcquired()); - { + UNIT_ASSERT(guard.WasAcquired()); + { TTryGuard<TGuardChecker> guard2(checker); UNIT_ASSERT(checker.guarded); - UNIT_ASSERT(!guard2.WasAcquired()); + UNIT_ASSERT(!guard2.WasAcquired()); } UNIT_ASSERT(checker.guarded); } @@ -152,21 +152,21 @@ struct TTestGuard: public TTestBase { TTryReadGuard tryGuard(mutex); UNIT_ASSERT(tryGuard.WasAcquired()); } - - int WithLockIncrement(TGuardChecker& m, int n) { + + int WithLockIncrement(TGuardChecker& m, int n) { with_lock (m) { - UNIT_ASSERT(m.guarded); - return n + 1; - } - } - - void TestWithLock() { - TGuardChecker m; - int n = 42; - n = WithLockIncrement(m, n); - UNIT_ASSERT(!m.guarded); - UNIT_ASSERT_EQUAL(n, 43); - } + UNIT_ASSERT(m.guarded); + return n + 1; + } + } + + void TestWithLock() { + TGuardChecker m; + int n = 42; + n = WithLockIncrement(m, n); + UNIT_ASSERT(!m.guarded); + UNIT_ASSERT_EQUAL(n, 43); + } void TestWithLockScope() { auto Guard = [](auto) { UNIT_FAIL("Non global Guard used"); return 0; }; diff --git a/util/system/mem_info_ut.cpp b/util/system/mem_info_ut.cpp index 2dcfc9c5c4e..a1100ba6468 100644 --- a/util/system/mem_info_ut.cpp +++ b/util/system/mem_info_ut.cpp @@ -13,8 +13,8 @@ class TMemInfoTest: public NUnitTest::TTestBase { TMemInfo stats = GetMemInfo(); - UNIT_ASSERT(stats.RSS >= NSystemInfo::GetPageSize()); - UNIT_ASSERT(stats.VMS >= stats.RSS); + UNIT_ASSERT(stats.RSS >= NSystemInfo::GetPageSize()); + UNIT_ASSERT(stats.VMS >= stats.RSS); } }; diff --git a/ydb/core/actorlib_impl/actor_bootstrapped_ut.cpp b/ydb/core/actorlib_impl/actor_bootstrapped_ut.cpp index b50e005b222..990330b321b 100644 --- a/ydb/core/actorlib_impl/actor_bootstrapped_ut.cpp +++ b/ydb/core/actorlib_impl/actor_bootstrapped_ut.cpp @@ -9,7 +9,7 @@ Y_UNIT_TEST_SUITE(ActorBootstrapped) { class TTestBoostrapped: public TActorBootstrapped<TTestBoostrapped> { public: - TTestBoostrapped(const TActorId& edge) + TTestBoostrapped(const TActorId& edge) : Edge(edge) {} @@ -17,21 +17,21 @@ Y_UNIT_TEST_SUITE(ActorBootstrapped) { ctx.Send(Edge, new TEvents::TEvWakeup); } - TActorId Edge; + TActorId Edge; }; class TTestBoostrappedParent: public TActorBootstrapped<TTestBoostrappedParent> { public: - TTestBoostrappedParent(const TActorId& edge) + TTestBoostrappedParent(const TActorId& edge) : Edge(edge) {} - void Bootstrap(const TActorId&, const TActorContext& ctx) { + void Bootstrap(const TActorId&, const TActorContext& ctx) { ctx.Send(Edge, new TEvents::TEvWakeup); } - TActorId Edge; + TActorId Edge; }; template <typename TDerivedActor> diff --git a/ydb/core/actorlib_impl/actor_tracker.cpp b/ydb/core/actorlib_impl/actor_tracker.cpp index e61b2559dce..99787dfce19 100644 --- a/ydb/core/actorlib_impl/actor_tracker.cpp +++ b/ydb/core/actorlib_impl/actor_tracker.cpp @@ -12,7 +12,7 @@ namespace NActors { void TActorTracker::Handle(TEvTrackActor::TPtr& ev, const TActorContext& ctx) { // insert newly created actor into registered actors set and ensure it is not duplicate one - const TActorId& newSubactorId = ev->Get()->NewSubactorId; + const TActorId& newSubactorId = ev->Get()->NewSubactorId; const bool inserted = RegisteredActors.insert(newSubactorId).second; Y_VERIFY(inserted); @@ -42,7 +42,7 @@ namespace NActors { // propagate TEvPoisonPill to all currently registered actors; request delivery tracking as each actor may die // before processing TEvPosionPill message - for (const TActorId& subactorId : RegisteredActors) { + for (const TActorId& subactorId : RegisteredActors) { ctx.Send(subactorId, new TEvents::TEvPoisonPill, IEventHandle::MakeFlags(0, IEventHandle::FlagTrackDelivery)); } @@ -56,7 +56,7 @@ namespace NActors { RemoveActorFromTrackList(ev->Sender, ctx); } - void TActorTracker::RemoveActorFromTrackList(const TActorId& subactorId, const TActorContext& ctx) { + void TActorTracker::RemoveActorFromTrackList(const TActorId& subactorId, const TActorContext& ctx) { // erase subactor from set and check if we have finished processing PoisonPill (if we are processing it) const ui32 numErased = RegisteredActors.erase(subactorId); Y_VERIFY(numErased == 1); @@ -110,17 +110,17 @@ namespace NActors { return true; } - TActorId TActorTracker::RegisterSubactor(THolder<TTrackedActorBase>&& subactor, const TActorContext& ctx, + TActorId TActorTracker::RegisterSubactor(THolder<TTrackedActorBase>&& subactor, const TActorContext& ctx, TMailboxType::EType mailboxType, ui32 poolId) { if (!PreRegister()) { - return TActorId(); + return TActorId(); } // bind new actor to this tracker subactor->BindToTracker(this); // we create new actor and register it in pool now - TActorId subactorId = ctx.Register(subactor.Release(), mailboxType, poolId); + TActorId subactorId = ctx.Register(subactor.Release(), mailboxType, poolId); // send TEvTrackActor message to tracker actor ctx.Send(ActorId, new TEvTrackActor(subactorId)); @@ -128,9 +128,9 @@ namespace NActors { return subactorId; } - TActorId TActorTracker::RegisterLocalSubactor(THolder<TTrackedActorBase>&& subactor, const TActorContext& ctx) { + TActorId TActorTracker::RegisterLocalSubactor(THolder<TTrackedActorBase>&& subactor, const TActorContext& ctx) { if (!PreRegister()) { - return TActorId(); + return TActorId(); } // bind new actor to this tracker @@ -153,13 +153,13 @@ namespace NActors { - TActorId TTrackedActorBase::RegisterSubactor(THolder<TTrackedActorBase>&& subactor, const TActorContext& ctx, + TActorId TTrackedActorBase::RegisterSubactor(THolder<TTrackedActorBase>&& subactor, const TActorContext& ctx, TMailboxType::EType mailboxType, ui32 poolId) { Y_VERIFY(Tracker); return Tracker->RegisterSubactor(std::move(subactor), ctx, mailboxType, poolId); } - TActorId TTrackedActorBase::RegisterLocalSubactor(THolder<TTrackedActorBase>&& subactor, const TActorContext& ctx) { + TActorId TTrackedActorBase::RegisterLocalSubactor(THolder<TTrackedActorBase>&& subactor, const TActorContext& ctx) { Y_VERIFY(Tracker); return Tracker->RegisterLocalSubactor(std::move(subactor), ctx); } @@ -178,7 +178,7 @@ namespace NActors { IActor::Die(ctx); } - TAutoPtr<IEventHandle> TTrackedActorBase::AfterRegister(const TActorId& self, const TActorId& parent) { + TAutoPtr<IEventHandle> TTrackedActorBase::AfterRegister(const TActorId& self, const TActorId& parent) { // send TEvBootstrap event locally return new IEventHandle(self, parent, new TEvents::TEvBootstrap); } diff --git a/ydb/core/actorlib_impl/actor_tracker.h b/ydb/core/actorlib_impl/actor_tracker.h index 7996eca1b12..66e953740d4 100644 --- a/ydb/core/actorlib_impl/actor_tracker.h +++ b/ydb/core/actorlib_impl/actor_tracker.h @@ -11,9 +11,9 @@ namespace NActors { // TEvTrackActor is sent by child actor in Register[Local]Subactor to an actor tracker to notify it about new // tracked subactor in pool. struct TEvTrackActor : TEventLocal<TEvTrackActor, TEvents::TSystem::TrackActor> { - const TActorId NewSubactorId; + const TActorId NewSubactorId; - TEvTrackActor(const TActorId& newSubactorId) + TEvTrackActor(const TActorId& newSubactorId) : NewSubactorId(newSubactorId) {} }; @@ -25,17 +25,17 @@ namespace NActors { class TActorTracker { // our killer -- the one who sent TEvPoisonPill here - TActorId KillerActorId; + TActorId KillerActorId; // a set of registered child actors we are tracking - TSet<TActorId> RegisteredActors; + TSet<TActorId> RegisteredActors; // number of in flight TEvTrackActor messages coming to tracker, but not yet processed; high bit indicates if // we are stopping and can't register new actors TAtomic NumInFlightTracks = 0; // actor id for this tracker - TActorId ActorId; + TActorId ActorId; public: void BindToActor(const TActorContext& ctx); @@ -59,11 +59,11 @@ namespace NActors { // register subactor inside this tracker on a separate mailbox; should be called instead of ExecutorThread's // method - TActorId RegisterSubactor(THolder<TTrackedActorBase>&& subactor, const TActorContext& ctx, + TActorId RegisterSubactor(THolder<TTrackedActorBase>&& subactor, const TActorContext& ctx, TMailboxType::EType mailboxType = TMailboxType::Simple, ui32 poolId = Max<ui32>()); // register subactor inside the same mailbox as of the caller - TActorId RegisterLocalSubactor(THolder<TTrackedActorBase>&& subactor, const TActorContext& ctx); + TActorId RegisterLocalSubactor(THolder<TTrackedActorBase>&& subactor, const TActorContext& ctx); private: bool PreRegister(); @@ -71,7 +71,7 @@ namespace NActors { void Handle(TEvUntrackActor::TPtr& ev, const TActorContext& ctx); void Handle(TEvents::TEvPoisonPill::TPtr& ev, const TActorContext& ctx); void Handle(TEvents::TEvPoisonTaken::TPtr& ev, const TActorContext& ctx); - void RemoveActorFromTrackList(const TActorId& subactorId, const TActorContext& ctx); + void RemoveActorFromTrackList(const TActorId& subactorId, const TActorContext& ctx); void CheckIfPoisonPillDone(const TActorContext& ctx); private: @@ -90,9 +90,9 @@ namespace NActors { {} // subactor registration helpers - TActorId RegisterSubactor(THolder<TTrackedActorBase>&& subactor, const TActorContext& ctx, + TActorId RegisterSubactor(THolder<TTrackedActorBase>&& subactor, const TActorContext& ctx, TMailboxType::EType mailboxType = TMailboxType::Simple, ui32 poolId = Max<ui32>()); - TActorId RegisterLocalSubactor(THolder<TTrackedActorBase>&& subactor, const TActorContext& ctx); + TActorId RegisterLocalSubactor(THolder<TTrackedActorBase>&& subactor, const TActorContext& ctx); // an override for tracked actors that also informs tracker about the death of tracked actor void Die(const TActorContext& ctx) override; @@ -101,7 +101,7 @@ namespace NActors { // internally it calls Die() to terminate this actor void HandlePoison(TEvents::TEvPoisonPill::TPtr& ev, const TActorContext& ctx); - TAutoPtr<IEventHandle> AfterRegister(const TActorId& self, const TActorId& /*parent*/) override; + TAutoPtr<IEventHandle> AfterRegister(const TActorId& self, const TActorId& /*parent*/) override; virtual void AfterBootstrap(const TActorContext& ctx) = 0; private: diff --git a/ydb/core/actorlib_impl/actor_tracker_ut.cpp b/ydb/core/actorlib_impl/actor_tracker_ut.cpp index 8a467c6c1aa..d804e3b9240 100644 --- a/ydb/core/actorlib_impl/actor_tracker_ut.cpp +++ b/ydb/core/actorlib_impl/actor_tracker_ut.cpp @@ -51,7 +51,7 @@ public: void Bootstrap(const TActorContext& ctx) { // create some subworkers for (int i = 0; i < 3; ++i) { - TActorId actorId = RegisterSubactor(MakeHolder<TSubworkerActor>(), ctx); + TActorId actorId = RegisterSubactor(MakeHolder<TSubworkerActor>(), ctx); if (!actorId) { Die(ctx); } @@ -79,7 +79,7 @@ public: // create some workers for (int i = 0; i < 2; ++i) { - TActorId actorId = Tracker.RegisterSubactor(MakeHolder<TWorkerActor>(), ctx); + TActorId actorId = Tracker.RegisterSubactor(MakeHolder<TWorkerActor>(), ctx); if (!actorId) { // a race has occured Die(ctx); @@ -105,8 +105,8 @@ Y_UNIT_TEST_SUITE(TActorTracker) { { TTestBasicRuntime runtime; runtime.Initialize(NKikimr::TAppPrepare().Unwrap()); - TActorId managerId = runtime.Register(new TManagerActor); - TActorId edge = runtime.AllocateEdgeActor(); + TActorId managerId = runtime.Register(new TManagerActor); + TActorId edge = runtime.AllocateEdgeActor(); runtime.Schedule(new IEventHandle(managerId, edge, new TEvents::TEvPoisonPill), TDuration::Seconds(1)); runtime.DispatchEvents(); TAutoPtr<IEventHandle> handle; diff --git a/ydb/core/actorlib_impl/destruct_actor.h b/ydb/core/actorlib_impl/destruct_actor.h index 30b841d8296..adc236911d5 100644 --- a/ydb/core/actorlib_impl/destruct_actor.h +++ b/ydb/core/actorlib_impl/destruct_actor.h @@ -25,8 +25,8 @@ private: } }; -TActorId GetDestructActorID() noexcept { - return TActorId(0, "destructor"); +TActorId GetDestructActorID() noexcept { + return TActorId(0, "destructor"); } } diff --git a/ydb/core/actorlib_impl/load_network.cpp b/ydb/core/actorlib_impl/load_network.cpp index af572dc8ba9..ae6f1b248a3 100644 --- a/ydb/core/actorlib_impl/load_network.cpp +++ b/ydb/core/actorlib_impl/load_network.cpp @@ -6,8 +6,8 @@ namespace { -NActors::TActorId GetLoadNetworkActorID(ui32 myNodeId) noexcept { - return NActors::TActorId(myNodeId, "loadnetwork"); +NActors::TActorId GetLoadNetworkActorID(ui32 myNodeId) noexcept { + return NActors::TActorId(myNodeId, "loadnetwork"); } using NActors::IEventHandle; diff --git a/ydb/core/actorlib_impl/long_timer.cpp b/ydb/core/actorlib_impl/long_timer.cpp index 651fd822c3c..e3b139579fb 100644 --- a/ydb/core/actorlib_impl/long_timer.cpp +++ b/ydb/core/actorlib_impl/long_timer.cpp @@ -65,15 +65,15 @@ public: const TActorId& parentId) { if (delta.Seconds() < ThresholdSec) { // direct scheduling w/o creating actor - ctx.ExecutorThread.Schedule(delta, ev, cookie); - return TActorId(); + ctx.ExecutorThread.Schedule(delta, ev, cookie); + return TActorId(); } TMonotonic now = ctx.Monotonic(); TMonotonic signalTime = now + delta; ui64 semirandomNumber = parentId.LocalId(); const TActorId timerActorID = ctx.ExecutorThread.ActorSystem->Register(new TLongTimer(now, signalTime, ev, cookie), TMailboxType::HTSwap, poolId, semirandomNumber, parentId); - ctx.ExecutorThread.Schedule(TDuration::Seconds(ThresholdSec), new IEventHandle(timerActorID, timerActorID, new TEvents::TEvWakeup())); + ctx.ExecutorThread.Schedule(TDuration::Seconds(ThresholdSec), new IEventHandle(timerActorID, timerActorID, new TEvents::TEvWakeup())); return timerActorID; } @@ -98,7 +98,7 @@ public: } }; -TActorId CreateLongTimer(const TActorContext &ctx, TDuration delta, TAutoPtr<IEventHandle> ev, ui32 poolId, ISchedulerCookie *cookie) { +TActorId CreateLongTimer(const TActorContext &ctx, TDuration delta, TAutoPtr<IEventHandle> ev, ui32 poolId, ISchedulerCookie *cookie) { return TLongTimer::Create(ctx, delta, ev, poolId, cookie, ctx.SelfID); } diff --git a/ydb/core/actorlib_impl/long_timer.h b/ydb/core/actorlib_impl/long_timer.h index 7ed40c7f219..9b04c301217 100644 --- a/ydb/core/actorlib_impl/long_timer.h +++ b/ydb/core/actorlib_impl/long_timer.h @@ -3,7 +3,7 @@ namespace NActors { -TActorId CreateLongTimer(const TActorContext &ctx, TDuration delta, TAutoPtr<IEventHandle> ev, ui32 poolId = 0, ISchedulerCookie *cookie = nullptr); +TActorId CreateLongTimer(const TActorContext &ctx, TDuration delta, TAutoPtr<IEventHandle> ev, ui32 poolId = 0, ISchedulerCookie *cookie = nullptr); // uses TlsActivationContext, note that by default we use current pool TActorId CreateLongTimer(TDuration delta, TAutoPtr<IEventHandle> ev, ui32 poolId = Max<ui32>(), ISchedulerCookie *cookie = nullptr); diff --git a/ydb/core/actorlib_impl/mad_squirrel.cpp b/ydb/core/actorlib_impl/mad_squirrel.cpp index dc9e3290866..3f9629d0996 100644 --- a/ydb/core/actorlib_impl/mad_squirrel.cpp +++ b/ydb/core/actorlib_impl/mad_squirrel.cpp @@ -5,7 +5,7 @@ namespace NActors { class TMadSquirrel : public TActor<TMadSquirrel> { - TAutoPtr<IEventHandle> AfterRegister(const TActorId &self, const TActorId &) override { + TAutoPtr<IEventHandle> AfterRegister(const TActorId &self, const TActorId &) override { return new IEventHandle(self, self, new TEvents::TEvWakeup()); } diff --git a/ydb/core/actorlib_impl/mad_squirrel.h b/ydb/core/actorlib_impl/mad_squirrel.h index 1303f143ab2..cce45d4aaa8 100644 --- a/ydb/core/actorlib_impl/mad_squirrel.h +++ b/ydb/core/actorlib_impl/mad_squirrel.h @@ -4,7 +4,7 @@ namespace NActors { IActor* CreateMadSquirrel(); // will send messages to itself in round -IActor* CreateMelancholicGopher(double surveyForSeconds, const TActorId &reportTo); // will spin for survey period and then wakeup next in line +IActor* CreateMelancholicGopher(double surveyForSeconds, const TActorId &reportTo); // will spin for survey period and then wakeup next in line IActor* CreateGopherMother(const TVector<std::pair<ui32, double>> &lineProfile, ui32 lines, ui32 shotsInRound); // would spawn gophers according to profile (poolid, period) in lines number } diff --git a/ydb/core/actorlib_impl/melancholic_gopher.cpp b/ydb/core/actorlib_impl/melancholic_gopher.cpp index eeee8cba0f8..52a895aae4a 100644 --- a/ydb/core/actorlib_impl/melancholic_gopher.cpp +++ b/ydb/core/actorlib_impl/melancholic_gopher.cpp @@ -11,7 +11,7 @@ namespace NActors { class TMelancholicGopher : public TActor<TMelancholicGopher> { const double SurveyForSeconds; - const TActorId ReportTo; + const TActorId ReportTo; void Round(const TActorContext &ctx) { if (SurveyForSeconds > 0.0) { @@ -26,7 +26,7 @@ public: return ACTORLIB_COMMON; } - TMelancholicGopher(double surveyForSeconds, const TActorId &reportTo) + TMelancholicGopher(double surveyForSeconds, const TActorId &reportTo) : TActor(&TThis::StateFunc) , SurveyForSeconds(surveyForSeconds) , ReportTo(reportTo) @@ -44,7 +44,7 @@ class TGopherMother : public TActorBootstrapped<TGopherMother> { const TVector<std::pair<ui32, double>> LineProfile; const ui32 Lines; const ui32 ShotsInRound; - TVector<TActorId> HeadGophers; + TVector<TActorId> HeadGophers; ui32 WaitFor; TInstant RoundStart; @@ -83,8 +83,8 @@ public: void Bootstrap(const TActorContext &ctx) { HeadGophers.reserve(Lines); for (ui32 lines = Lines; lines > 0; --lines) { - HeadGophers.push_back(TActorId()); - TActorId &head = HeadGophers.back(); + HeadGophers.push_back(TActorId()); + TActorId &head = HeadGophers.back(); head = ctx.SelfID; for (const auto &xpair : LineProfile) head = ctx.ExecutorThread.ActorSystem->Register(CreateMelancholicGopher(xpair.second, head), TMailboxType::Simple, xpair.first); @@ -101,7 +101,7 @@ public: } }; -IActor* CreateMelancholicGopher(double surveyForSeconds, const TActorId &reportTo) { +IActor* CreateMelancholicGopher(double surveyForSeconds, const TActorId &reportTo) { return new TMelancholicGopher(surveyForSeconds, reportTo); } diff --git a/ydb/core/actorlib_impl/name_service_client_protocol.h b/ydb/core/actorlib_impl/name_service_client_protocol.h index dafa10e5c30..c46ffe9b030 100644 --- a/ydb/core/actorlib_impl/name_service_client_protocol.h +++ b/ydb/core/actorlib_impl/name_service_client_protocol.h @@ -22,7 +22,7 @@ public: MemLogPrintF("TResolveClientProtocol send name request: %s, %u", msg->Address.data(), (int)msg->Port); - ctx.Send(GetNameserviceActorId(), msg); + ctx.Send(GetNameserviceActorId(), msg); orig->template AddMsgProtocol<TEvAddressInfo>( &TOrigActor::template CallProtocolStateFunc< diff --git a/ydb/core/actorlib_impl/node_identifier.cpp b/ydb/core/actorlib_impl/node_identifier.cpp index 68215f7ae4b..410fbede7cc 100644 --- a/ydb/core/actorlib_impl/node_identifier.cpp +++ b/ydb/core/actorlib_impl/node_identifier.cpp @@ -137,7 +137,7 @@ class TNodeIdentifier : public TActorBootstrapped<TNodeIdentifier> { catch (const std::exception&) { } - TActorId whiteboardServiceId = NNodeWhiteboard::MakeNodeWhiteboardServiceId(ctx.SelfID.NodeId()); + TActorId whiteboardServiceId = NNodeWhiteboard::MakeNodeWhiteboardServiceId(ctx.SelfID.NodeId()); ctx.Send(whiteboardServiceId, new NNodeWhiteboard::TEvWhiteboard::TEvSystemStateUpdate(systemStateInfo)); } diff --git a/ydb/core/actorlib_impl/node_identifier.h b/ydb/core/actorlib_impl/node_identifier.h index be78dd7971e..7981886b93a 100644 --- a/ydb/core/actorlib_impl/node_identifier.h +++ b/ydb/core/actorlib_impl/node_identifier.h @@ -3,9 +3,9 @@ namespace NKikimr { -inline NActors::TActorId MakeNodeIdentifierServiceId() { +inline NActors::TActorId MakeNodeIdentifierServiceId() { char x[12] = {'n','o','d','e','i','d','e','n','t','i','f','r'}; - return TActorId(0, TStringBuf(x, 12)); + return TActorId(0, TStringBuf(x, 12)); } NActors::IActor* CreateNodeIdentifier(); diff --git a/ydb/core/actorlib_impl/router_rr.h b/ydb/core/actorlib_impl/router_rr.h index 96d4987f339..f78f61d2be5 100644 --- a/ydb/core/actorlib_impl/router_rr.h +++ b/ydb/core/actorlib_impl/router_rr.h @@ -48,18 +48,18 @@ namespace NActors { NActors::IActor *actor = new TTargetActor(); // register the actor - NActors::TActorId actorId = ctx.ExecutorThread.RegisterActor(actor, mbType, threadPoolId); + NActors::TActorId actorId = ctx.ExecutorThread.RegisterActor(actor, mbType, threadPoolId); Actors.push_back(actorId); } } - virtual TAutoPtr<NActors::IEventHandle> AfterRegister(const NActors::TActorId &self, const TActorId& parentId) override { + virtual TAutoPtr<NActors::IEventHandle> AfterRegister(const NActors::TActorId &self, const TActorId& parentId) override { Y_UNUSED(parentId); return new NActors::IEventHandle(self, self, new TEvents::TEvBootstrap(), 0); } protected: - TVector<NActors::TActorId> Actors; + TVector<NActors::TActorId> Actors; int Pos; }; diff --git a/ydb/core/actorlib_impl/test_interconnect_ut.cpp b/ydb/core/actorlib_impl/test_interconnect_ut.cpp index 31bc7107251..0d9d3535ef6 100644 --- a/ydb/core/actorlib_impl/test_interconnect_ut.cpp +++ b/ydb/core/actorlib_impl/test_interconnect_ut.cpp @@ -51,7 +51,7 @@ Y_UNIT_TEST_SUITE(TInterconnectTest) { class TFlooder : public NActors::TActor<TFlooder> { public: - TFlooder(const TActorId& peer, const TActorId& edge, unsigned count) noexcept + TFlooder(const TActorId& peer, const TActorId& edge, unsigned count) noexcept : TActor(&TThis::InitFunc), Peer(peer), Edge(edge), Counter(count), Responses(0ULL) {} @@ -74,7 +74,7 @@ Y_UNIT_TEST_SUITE(TInterconnectTest) { } } - TAutoPtr<IEventHandle> AfterRegister(const TActorId &self, const TActorId& parentId) noexcept override { + TAutoPtr<IEventHandle> AfterRegister(const TActorId &self, const TActorId& parentId) noexcept override { return new IEventHandle(self, parentId, new TEvents::TEvBootstrap, 0); } @@ -101,7 +101,7 @@ Y_UNIT_TEST_SUITE(TInterconnectTest) { } } - const TActorId Peer, Edge; + const TActorId Peer, Edge; const unsigned Counter; unsigned Responses; }; @@ -515,7 +515,7 @@ Y_UNIT_TEST_SUITE(TInterconnectTest) { TAutoPtr<IEventHandle> handle; runtime.GrabEdgeEvent<TEvInterconnect::TEvNodeConnected>(handle); - runtime.Send(new IEventHandle(TActorId(runtime.GetNodeId(1), "null"), + runtime.Send(new IEventHandle(TActorId(runtime.GetNodeId(1), "null"), edge, new TEvents::TEvPing, IEventHandle::FlagTrackDelivery, 13), @@ -640,7 +640,7 @@ Y_UNIT_TEST_SUITE(TInterconnectTest) { auto event = new TEvResolveAddress; event->Address = "localhost"; event->Port = 80; - runtime.Send(new IEventHandle(GetNameserviceActorId(), edge, event), 0); + runtime.Send(new IEventHandle(GetNameserviceActorId(), edge, event), 0); TAutoPtr<IEventHandle> handle; const auto reply = runtime.GrabEdgeEvent<TEvAddressInfo>(handle); UNIT_ASSERT_VALUES_EQUAL(NAddr::PrintHostAndPort(*reply->Address), @@ -688,7 +688,7 @@ Y_UNIT_TEST_SUITE(TInterconnectTest) { msg->Record.SetMeta(meta); msg->Record.AddPayloadId(msg->AddPayload(TRope(rope1))); msg->Record.AddPayloadId(msg->AddPayload(TRope(rope2))); - runtime.Send(new IEventHandle(edge, TActorId(), msg.Release()), 0); + runtime.Send(new IEventHandle(edge, TActorId(), msg.Release()), 0); } TAutoPtr<IEventHandle> handle; diff --git a/ydb/core/actorlib_impl/test_protocols_ut.cpp b/ydb/core/actorlib_impl/test_protocols_ut.cpp index 7646370c82c..2e622406a05 100644 --- a/ydb/core/actorlib_impl/test_protocols_ut.cpp +++ b/ydb/core/actorlib_impl/test_protocols_ut.cpp @@ -136,7 +136,7 @@ Y_UNIT_TEST_SUITE(TestProtocols) { NAddr::IRemoteAddrPtr Address; TString Error; - TActorId Edge; + TActorId Edge; }; @@ -211,7 +211,7 @@ Y_UNIT_TEST_SUITE(TestProtocols) { NAddr::IRemoteAddrRef Address; TString Error; - TActorId Edge; + TActorId Edge; TIntrusivePtr<NInterconnect::TStreamSocket> Socket; ui64 Port; }; @@ -338,7 +338,7 @@ Y_UNIT_TEST_SUITE(TestProtocols) { public: TString Error; TString Data; - TActorId Edge; + TActorId Edge; ui16 Port; private: @@ -673,7 +673,7 @@ Y_UNIT_TEST_SUITE(TestProtocols) { public: TString Error; - TActorId Edge; + TActorId Edge; ui16 Port; private: diff --git a/ydb/core/actorlib_impl/ya.make b/ydb/core/actorlib_impl/ya.make index 274191fd0df..ba63f39617f 100644 --- a/ydb/core/actorlib_impl/ya.make +++ b/ydb/core/actorlib_impl/ya.make @@ -46,8 +46,8 @@ PEERDIR( library/cpp/lwtrace library/cpp/lwtrace/mon library/cpp/messagebus/monitoring - library/cpp/monlib/dynamic_counters - library/cpp/monlib/service/pages/tablesorter + library/cpp/monlib/dynamic_counters + library/cpp/monlib/service/pages/tablesorter library/cpp/packedtypes library/cpp/sliding_window ydb/core/base diff --git a/ydb/core/base/appdata.cpp b/ydb/core/base/appdata.cpp index 261bdce8603..f9e517fc424 100644 --- a/ydb/core/base/appdata.cpp +++ b/ydb/core/base/appdata.cpp @@ -1,13 +1,13 @@ #include "appdata.h" -#include "tablet_types.h" +#include "tablet_types.h" namespace NKikimr { -TAppData::TAppData( - ui32 sysPoolId, ui32 userPoolId, ui32 ioPoolId, ui32 batchPoolId, +TAppData::TAppData( + ui32 sysPoolId, ui32 userPoolId, ui32 ioPoolId, ui32 batchPoolId, TMap<TString, ui32> servicePools, - const NScheme::TTypeRegistry* typeRegistry, - const NMiniKQL::IFunctionRegistry* functionRegistry, + const NScheme::TTypeRegistry* typeRegistry, + const NMiniKQL::IFunctionRegistry* functionRegistry, const TFormatFactory* formatFactory, TProgramShouldContinue *kikimrShouldContinue) : Magic(MagicTag) @@ -17,7 +17,7 @@ TAppData::TAppData( , BatchPoolId(batchPoolId) , ServicePools(servicePools) , TypeRegistry(typeRegistry) - , FunctionRegistry(functionRegistry) + , FunctionRegistry(functionRegistry) , FormatFactory(formatFactory) , ProxySchemeCacheNodes(Max<ui64>() / 4) , ProxySchemeCacheDistrNodes(Max<ui64>() / 4) diff --git a/ydb/core/base/appdata.h b/ydb/core/base/appdata.h index f09121b8792..c666f7468c0 100644 --- a/ydb/core/base/appdata.h +++ b/ydb/core/base/appdata.h @@ -1,5 +1,5 @@ #pragma once - + #include "defs.h" #include "channel_profiles.h" #include "domain.h" @@ -8,7 +8,7 @@ #include "tablet_types.h" #include "resource_profile.h" #include "event_filter.h" - + #include <ydb/core/control/immediate_control_board_impl.h> #include <ydb/core/grpc_services/grpc_helper.h> #include <ydb/core/protos/auth.pb.h> @@ -24,7 +24,7 @@ #include <library/cpp/actors/util/should_continue.h> #include <library/cpp/random_provider/random_provider.h> #include <library/cpp/time_provider/time_provider.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> namespace NActors { class TMon; @@ -40,14 +40,14 @@ namespace NYdb { namespace NKikimr { -namespace NScheme { - class TTypeRegistry; -} - -namespace NMiniKQL { - class IFunctionRegistry; -} - +namespace NScheme { + class TTypeRegistry; +} + +namespace NMiniKQL { + class IFunctionRegistry; +} + namespace NDataShard { class IExportFactory; } @@ -86,7 +86,7 @@ struct TAppData { const NPQ::IPersQueueMirrorReaderFactory* PersQueueMirrorReaderFactory = nullptr; NYdb::TDriver* YdbDriver = nullptr; const NPDisk::IIoContextFactory* IoContextFactory = nullptr; - + struct TDefaultTabletTypes { TTabletTypes::EType SchemeShard; TTabletTypes::EType DataShard; @@ -171,11 +171,11 @@ struct TAppData { TKikimrScopeId LocalScopeId; - TAppData( - ui32 sysPoolId, ui32 userPoolId, ui32 ioPoolId, ui32 batchPoolId, + TAppData( + ui32 sysPoolId, ui32 userPoolId, ui32 ioPoolId, ui32 batchPoolId, TMap<TString, ui32> servicePools, - const NScheme::TTypeRegistry* typeRegistry, - const NMiniKQL::IFunctionRegistry* functionRegistry, + const NScheme::TTypeRegistry* typeRegistry, + const NMiniKQL::IFunctionRegistry* functionRegistry, const TFormatFactory* formatFactory, TProgramShouldContinue *kikimrShouldContinue); }; diff --git a/ydb/core/base/blobstorage.h b/ydb/core/base/blobstorage.h index bc15baee598..a2faee326e5 100644 --- a/ydb/core/base/blobstorage.h +++ b/ydb/core/base/blobstorage.h @@ -2,9 +2,9 @@ #include "defs.h" #include "blobstorage_pdisk_category.h" -#include "events.h" -#include "tablet_types.h" -#include "logoblob.h" +#include "events.h" +#include "tablet_types.h" +#include "logoblob.h" #include "pathid.h" #include <ydb/core/base/services/blobstorage_service_id.h> @@ -362,7 +362,7 @@ public: return str.Str(); } - TActorId BSProxyIDForChannel(ui32 channel, ui32 generation) const; + TActorId BSProxyIDForChannel(ui32 channel, ui32 generation) const; bool operator<(const TTabletStorageInfo &other) const noexcept { if (Version != 0 && other.Version != 0) { @@ -392,14 +392,14 @@ public: ui64 HiveId = 0; }; -inline TActorId TTabletStorageInfo::BSProxyIDForChannel(ui32 channel, ui32 generation) const { +inline TActorId TTabletStorageInfo::BSProxyIDForChannel(ui32 channel, ui32 generation) const { const ui32 group = GroupFor(channel, generation); Y_VERIFY(group != Max<ui32>()); - const TActorId proxy = MakeBlobStorageProxyID(group); + const TActorId proxy = MakeBlobStorageProxyID(group); return proxy; } -inline ui32 GroupIDFromBlobStorageProxyID(TActorId actorId) { +inline ui32 GroupIDFromBlobStorageProxyID(TActorId actorId) { ui32 blobStorageGroup = ui32( ((actorId.RawX1() >> (7 * 8)) & 0xff) | (((actorId.RawX2() >> (0 * 8)) & 0xff) << 8) | @@ -427,12 +427,12 @@ inline bool SendToBSProxy(TActorId sender, TActorId recipient, IEventBase *ev, u return TActivationContext::Send(CreateEventForBSProxy(sender, recipient, ev, cookie, std::move(traceId))); } -inline bool SendToBSProxy(const TActorContext &ctx, TActorId recipient, IEventBase *ev, ui64 cookie = 0, +inline bool SendToBSProxy(const TActorContext &ctx, TActorId recipient, IEventBase *ev, ui64 cookie = 0, NWilson::TTraceId traceId = {}) { return ctx.Send(CreateEventForBSProxy(ctx.SelfID, recipient, ev, cookie, std::move(traceId))); } -inline bool SendToBSProxy(TActorId sender, ui32 groupId, IEventBase *ev, ui64 cookie = 0, NWilson::TTraceId traceId = {}) { +inline bool SendToBSProxy(TActorId sender, ui32 groupId, IEventBase *ev, ui64 cookie = 0, NWilson::TTraceId traceId = {}) { return TActivationContext::Send(CreateEventForBSProxy(sender, groupId, ev, cookie, std::move(traceId))); } diff --git a/ydb/core/base/board_lookup.cpp b/ydb/core/base/board_lookup.cpp index 5e8c07224d4..890b88f6b9e 100644 --- a/ydb/core/base/board_lookup.cpp +++ b/ydb/core/base/board_lookup.cpp @@ -21,7 +21,7 @@ namespace NKikimr { class TBoardLookupActor : public TActorBootstrapped<TBoardLookupActor> { const TString Path; - const TActorId Owner; + const TActorId Owner; const EBoardLookupMode Mode; const ui32 StateStorageGroupId; @@ -33,12 +33,12 @@ class TBoardLookupActor : public TActorBootstrapped<TBoardLookupActor> { }; struct TReplica { - TActorId Replica; + TActorId Replica; EReplicaState State = EReplicaState::Unknown; }; TVector<TReplica> Replicas; - TMap<TActorId, TEvStateStorage::TEvBoardInfo::TInfoEntry> Info; + TMap<TActorId, TEvStateStorage::TEvBoardInfo::TInfoEntry> Info; ui32 WaitForReplicasToSuccess; @@ -83,8 +83,8 @@ class TBoardLookupActor : public TActorBootstrapped<TBoardLookupActor> { Replicas.resize(msg->Replicas.size()); for (auto idx : xrange(msg->Replicas.size())) { - const TActorId &replica = msg->Replicas[idx]; - Send(replica, new TEvStateStorage::TEvReplicaBoardLookup(Path, TActorId(), false), IEventHandle::FlagTrackDelivery | IEventHandle::FlagSubscribeOnSession, idx); + const TActorId &replica = msg->Replicas[idx]; + Send(replica, new TEvStateStorage::TEvReplicaBoardLookup(Path, TActorId(), false), IEventHandle::FlagTrackDelivery | IEventHandle::FlagSubscribeOnSession, idx); Replicas[idx].Replica = replica; Replicas[idx].State = EReplicaState::Unknown; } @@ -128,7 +128,7 @@ class TBoardLookupActor : public TActorBootstrapped<TBoardLookupActor> { ++Stats.HasInfo; for (auto &x : record.GetInfo()) { - const TActorId oid = ActorIdFromProto(x.GetOwner()); + const TActorId oid = ActorIdFromProto(x.GetOwner()); Info[oid].Payload = x.GetPayload(); } } @@ -170,7 +170,7 @@ public: return NKikimrServices::TActivity::BOARD_LOOKUP_ACTOR; } - TBoardLookupActor(const TString &path, TActorId owner, EBoardLookupMode mode, ui32 groupId) + TBoardLookupActor(const TString &path, TActorId owner, EBoardLookupMode mode, ui32 groupId) : Path(path) , Owner(owner) , Mode(mode) @@ -178,7 +178,7 @@ public: {} void Bootstrap() { - const TActorId proxyId = MakeStateStorageProxyID(StateStorageGroupId); + const TActorId proxyId = MakeStateStorageProxyID(StateStorageGroupId); Send(proxyId, new TEvStateStorage::TEvResolveBoard(Path), IEventHandle::FlagTrackDelivery); Become(&TThis::StateResolve); } @@ -201,7 +201,7 @@ public: } }; -IActor* CreateBoardLookupActor(const TString &path, const TActorId &owner, ui32 groupId, EBoardLookupMode mode, bool sub, bool useNodeSubsriptions) { +IActor* CreateBoardLookupActor(const TString &path, const TActorId &owner, ui32 groupId, EBoardLookupMode mode, bool sub, bool useNodeSubsriptions) { Y_UNUSED(useNodeSubsriptions); Y_VERIFY(!sub, "subscribe mode for board lookup not implemented yet"); return new TBoardLookupActor(path, owner, mode, groupId); diff --git a/ydb/core/base/board_publish.cpp b/ydb/core/base/board_publish.cpp index 7b6077b8672..cdd4bf6e489 100644 --- a/ydb/core/base/board_publish.cpp +++ b/ydb/core/base/board_publish.cpp @@ -22,8 +22,8 @@ namespace NKikimr { class TBoardReplicaPublishActor : public TActorBootstrapped<TBoardReplicaPublishActor> { const TString Path; TString Payload; - const TActorId Replica; - const TActorId PublishActor; + const TActorId Replica; + const TActorId PublishActor; ui64 Round; @@ -51,7 +51,7 @@ public: return NKikimrServices::TActivity::BOARD_REPLICA_PUBLISH_ACTOR; } - TBoardReplicaPublishActor(const TString &path, const TString &payload, TActorId replica, TActorId publishActor) + TBoardReplicaPublishActor(const TString &path, const TString &payload, TActorId replica, TActorId publishActor) : Path(path) , Payload(payload) , Replica(replica) @@ -78,12 +78,12 @@ public: class TBoardPublishActor : public TActorBootstrapped<TBoardPublishActor> { const TString Path; const TString Payload; - const TActorId Owner; + const TActorId Owner; const ui32 StateStorageGroupId; const ui32 TtlMs; const bool Register; - TMap<TActorId, TActorId> ReplicaPublishActors; // replica -> publish actor + TMap<TActorId, TActorId> ReplicaPublishActors; // replica -> publish actor void PassAway() override { for (auto &xpair : ReplicaPublishActors) { @@ -105,10 +105,10 @@ class TBoardPublishActor : public TActorBootstrapped<TBoardPublishActor> { if (msg->Replicas.empty()) { BLOG_ERROR("publish on unconfigured statestorage board service " << StateStorageGroupId); } else { - TMap<TActorId, TActorId> updated; + TMap<TActorId, TActorId> updated; for (auto &replicaId : msg->Replicas) { - const TActorId *known = ReplicaPublishActors.FindPtr(replicaId); + const TActorId *known = ReplicaPublishActors.FindPtr(replicaId); if (known && *known) updated[replicaId] = *known; else @@ -122,11 +122,11 @@ class TBoardPublishActor : public TActorBootstrapped<TBoardPublishActor> { } bool ResolveGone(TEvents::TEvGone::TPtr &ev) { - const TActorId sender = ev->Sender; + const TActorId sender = ev->Sender; for (auto &xpair : ReplicaPublishActors) { if (xpair.second == sender) { - xpair.second = TActorId(); + xpair.second = TActorId(); return true; } } @@ -136,7 +136,7 @@ class TBoardPublishActor : public TActorBootstrapped<TBoardPublishActor> { void CalmGone(TEvents::TEvGone::TPtr &ev) { if (ResolveGone(ev)) { - const TActorId proxyId = MakeStateStorageProxyID(StateStorageGroupId); + const TActorId proxyId = MakeStateStorageProxyID(StateStorageGroupId); const ui32 flags = IEventHandle::FlagTrackDelivery; TAutoPtr<IEventHandle> x = new IEventHandle(proxyId, SelfId(), new TEvStateStorage::TEvResolveBoard(Path), flags); TActivationContext::Schedule(TDuration::MilliSeconds(50), x); @@ -150,7 +150,7 @@ public: return NKikimrServices::TActivity::BOARD_PUBLISH_ACTOR; } - TBoardPublishActor(const TString &path, const TString &payload, const TActorId &owner, ui32 groupId, ui32 ttlMs, bool reg) + TBoardPublishActor(const TString &path, const TString &payload, const TActorId &owner, ui32 groupId, ui32 ttlMs, bool reg) : Path(path) , Payload(payload) , Owner(owner) @@ -163,7 +163,7 @@ public: } void Bootstrap() { - const TActorId proxyId = MakeStateStorageProxyID(StateStorageGroupId); + const TActorId proxyId = MakeStateStorageProxyID(StateStorageGroupId); Send(proxyId, new TEvStateStorage::TEvResolveBoard(Path), IEventHandle::FlagTrackDelivery); Become(&TThis::StateResolve); } @@ -185,7 +185,7 @@ public: } }; -IActor* CreateBoardPublishActor(const TString &path, const TString &payload, const TActorId &owner, ui32 groupId, ui32 ttlMs, bool reg) { +IActor* CreateBoardPublishActor(const TString &path, const TString &payload, const TActorId &owner, ui32 groupId, ui32 ttlMs, bool reg) { return new TBoardPublishActor(path, payload, owner, groupId, ttlMs, reg); } diff --git a/ydb/core/base/board_replica.cpp b/ydb/core/base/board_replica.cpp index 42a7764b8fc..c38fd20e351 100644 --- a/ydb/core/base/board_replica.cpp +++ b/ydb/core/base/board_replica.cpp @@ -18,12 +18,12 @@ namespace NKikimr { class TBoardReplicaActor : public TActor<TBoardReplicaActor> { - using TOwnerIndex = TMap<TActorId, ui32, TActorId::TOrderedCmp>; + using TOwnerIndex = TMap<TActorId, ui32, TActorId::TOrderedCmp>; using TPathIndex = TMap<TString, TSet<ui32>>; struct TEntry { TString Payload; - TActorId Owner; + TActorId Owner; TOwnerIndex::iterator OwnerIt; TPathIndex::iterator PathIt; }; @@ -51,7 +51,7 @@ class TBoardReplicaActor : public TActor<TBoardReplicaActor> { void Handle(TEvStateStorage::TEvReplicaBoardPublish::TPtr &ev) { auto &record = ev->Get()->Record; const TString &path = record.GetPath(); - const TActorId &owner = ev->Sender; + const TActorId &owner = ev->Sender; if (!record.GetRegister()) { BLOG_ERROR("free floating entries not implemented yet"); @@ -69,13 +69,13 @@ class TBoardReplicaActor : public TActor<TBoardReplicaActor> { } entry.Payload = record.GetPayload(); - Y_VERIFY_DEBUG(entry.Owner == ActorIdFromProto(record.GetOwner())); + Y_VERIFY_DEBUG(entry.Owner == ActorIdFromProto(record.GetOwner())); } else { const ui32 entryIndex = AllocateEntry(); TEntry &entry = Entries[entryIndex]; entry.Payload = record.GetPayload(); - entry.Owner = ActorIdFromProto(record.GetOwner()); + entry.Owner = ActorIdFromProto(record.GetOwner()); auto ownerInsPairIt = IndexOwner.emplace(owner, entryIndex); entry.OwnerIt = ownerInsPairIt.first; @@ -118,7 +118,7 @@ class TBoardReplicaActor : public TActor<TBoardReplicaActor> { IndexOwner.erase(entry.OwnerIt); TString().swap(entry.Payload); - entry.Owner = TActorId(); + entry.Owner = TActorId(); entry.PathIt = IndexPath.end(); entry.OwnerIt = IndexOwner.end(); @@ -171,7 +171,7 @@ class TBoardReplicaActor : public TActor<TBoardReplicaActor> { for (ui32 entryIndex : pathIt->second) { const TEntry &entry = Entries[entryIndex]; auto *ex = info->Add(); - ActorIdToProto(entry.Owner, ex->MutableOwner()); + ActorIdToProto(entry.Owner, ex->MutableOwner()); ex->SetPayload(entry.Payload); } @@ -189,7 +189,7 @@ class TBoardReplicaActor : public TActor<TBoardReplicaActor> { void Handle(TEvInterconnect::TEvNodeDisconnected::TPtr &ev) { auto *msg = ev->Get(); const ui32 nodeId = msg->NodeId; - auto ownerIt = IndexOwner.lower_bound(TActorId(nodeId, 0, 0, 0)); + auto ownerIt = IndexOwner.lower_bound(TActorId(nodeId, 0, 0, 0)); while (ownerIt != IndexOwner.end() && ownerIt->first.NodeId() == nodeId) { const ui32 entryToCleanupIndex = ownerIt->second; ++ownerIt; diff --git a/ydb/core/base/counters.h b/ydb/core/base/counters.h index 5a4376eefe4..01dc8c4c0f7 100644 --- a/ydb/core/base/counters.h +++ b/ydb/core/base/counters.h @@ -1,7 +1,7 @@ #pragma once #include "defs.h" -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <util/generic/hash_set.h> diff --git a/ydb/core/base/hive.h b/ydb/core/base/hive.h index b471011a759..7464c76699d 100644 --- a/ydb/core/base/hive.h +++ b/ydb/core/base/hive.h @@ -269,16 +269,16 @@ namespace NKikimr { Record.SetTabletID(tabletId); } - TEvStopTablet(ui64 tabletId, const TActorId &actorToNotify) { + TEvStopTablet(ui64 tabletId, const TActorId &actorToNotify) { Record.SetTabletID(tabletId); - ActorIdToProto(actorToNotify, Record.MutableActorToNotify()); + ActorIdToProto(actorToNotify, Record.MutableActorToNotify()); } TString ToString() const { TStringStream str; str << "{EvStopTablet TabletID: " << Record.GetTabletID(); if (Record.HasActorToNotify()) { - str << " ActorToNotify: " << ActorIdFromProto(Record.GetActorToNotify()).ToString(); + str << " ActorToNotify: " << ActorIdFromProto(Record.GetActorToNotify()).ToString(); } str << "}"; return str.Str(); @@ -806,5 +806,5 @@ namespace NKikimr { }; }; - IActor* CreateDefaultHive(const TActorId &tablet, TTabletStorageInfo *info); + IActor* CreateDefaultHive(const TActorId &tablet, TTabletStorageInfo *info); } diff --git a/ydb/core/base/pool_stats_collector.h b/ydb/core/base/pool_stats_collector.h index 5fd26ad119f..cbf06f9f513 100644 --- a/ydb/core/base/pool_stats_collector.h +++ b/ydb/core/base/pool_stats_collector.h @@ -1,7 +1,7 @@ #pragma once #include "defs.h" #include <library/cpp/actors/core/actorsystem.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> namespace NActors { struct TActorSystemSetup; diff --git a/ydb/core/base/quoter.cpp b/ydb/core/base/quoter.cpp index 2b662b76fb2..c31333fe567 100644 --- a/ydb/core/base/quoter.cpp +++ b/ydb/core/base/quoter.cpp @@ -2,9 +2,9 @@ namespace NKikimr { -TActorId MakeQuoterServiceID() { +TActorId MakeQuoterServiceID() { char x[12] = { 'q', 'u', 'o', 't', 'e', 'r', 's', 'v', 'c' }; - return TActorId(0, TStringBuf(x, 12)); + return TActorId(0, TStringBuf(x, 12)); } ui64 TEvQuota::TResourceLeaf::MakeTaggedRateRes(ui32 tag, ui32 rate) { diff --git a/ydb/core/base/quoter.h b/ydb/core/base/quoter.h index 24a0fc1eb15..41a19775bc8 100644 --- a/ydb/core/base/quoter.h +++ b/ydb/core/base/quoter.h @@ -249,6 +249,6 @@ struct TEvQuota { }; // -TActorId MakeQuoterServiceID(); +TActorId MakeQuoterServiceID(); } diff --git a/ydb/core/base/services/blobstorage_service_id.h b/ydb/core/base/services/blobstorage_service_id.h index 313a3de03ca..43f6eceda65 100644 --- a/ydb/core/base/services/blobstorage_service_id.h +++ b/ydb/core/base/services/blobstorage_service_id.h @@ -5,34 +5,34 @@ namespace NKikimr { -inline TActorId MakeBlobStorageProxyID(ui32 blobStorageGroup) { +inline TActorId MakeBlobStorageProxyID(ui32 blobStorageGroup) { char x[12] = {'b', 's', 'p', 'r', 'o', 'x', 'y' }; x[7] = (char)blobStorageGroup; x[8] = (char)(blobStorageGroup >> 8); x[9] = (char)(blobStorageGroup >> 16); x[10] = (char)(blobStorageGroup >> 24); - return TActorId(0, TStringBuf(x, 12)); + return TActorId(0, TStringBuf(x, 12)); } -inline TActorId MakeBlobStorageLoadID(ui32 nodeId) { +inline TActorId MakeBlobStorageLoadID(ui32 nodeId) { char x[12] = {'b', 's', 'l', 'o', 'a', 'd', 'd', 0}; x[8] = (char)(nodeId >> 24); x[9] = (char)(nodeId >> 16); x[10] = (char)(nodeId >> 8); x[11] = (char)nodeId; - return TActorId(nodeId, TStringBuf(x, 12)); + return TActorId(nodeId, TStringBuf(x, 12)); } -inline TActorId MakeBlobStorageFailureInjectionID(ui32 nodeId) { +inline TActorId MakeBlobStorageFailureInjectionID(ui32 nodeId) { char x[12] = {'b', 's', 'F', 'a', 'i', 'l', 'I', 'n'}; x[8] = (char)(nodeId >> 24); x[9] = (char)(nodeId >> 16); x[10] = (char)(nodeId >> 8); x[11] = (char)nodeId; - return TActorId(nodeId, TStringBuf(x, 12)); + return TActorId(nodeId, TStringBuf(x, 12)); } -inline TActorId MakeBlobStorageVDiskID(ui32 node, ui32 pDiskID, ui32 vDiskSlotID) { +inline TActorId MakeBlobStorageVDiskID(ui32 node, ui32 pDiskID, ui32 vDiskSlotID) { char x[12] = {'b','s','v','d'}; x[4] = (char)pDiskID; x[5] = (char)(pDiskID >> 8); @@ -42,7 +42,7 @@ inline TActorId MakeBlobStorageVDiskID(ui32 node, ui32 pDiskID, ui32 vDiskSlotID x[9] = (char)(vDiskSlotID >> 8); x[10] = (char)(vDiskSlotID >> 16); x[11] = (char)(vDiskSlotID >> 24); - return TActorId(node, TStringBuf(x, 12)); + return TActorId(node, TStringBuf(x, 12)); } inline std::tuple<ui32, ui32, ui32> DecomposeVDiskServiceId(const TActorId& actorId) { @@ -58,23 +58,23 @@ inline std::tuple<ui32, ui32, ui32> DecomposeVDiskServiceId(const TActorId& acto return {nodeId, pdiskId, vslotId}; } -inline TActorId MakeBlobStoragePDiskID(ui32 node, ui32 pDiskID) { +inline TActorId MakeBlobStoragePDiskID(ui32 node, ui32 pDiskID) { char x[12] = {'b','s','p','d','i','s','k', 0}; x[8] = (char)pDiskID; x[9] = (char)(pDiskID >> 8); x[10] = (char)(pDiskID >> 16); x[11] = (char)(pDiskID >> 24); - return TActorId(node, TStringBuf(x, 12)); + return TActorId(node, TStringBuf(x, 12)); } -inline TActorId MakeBlobStorageReplBrokerID() { +inline TActorId MakeBlobStorageReplBrokerID() { char x[12] = {'b', 's', 'r', 'e', 'p', 'l', 'b', 'r', 'o', 'k', 'e', 'r'}; - return TActorId(0, TStringBuf(x, 12)); + return TActorId(0, TStringBuf(x, 12)); } -inline TActorId MakeBlobStorageNodeWardenID(ui32 node) { +inline TActorId MakeBlobStorageNodeWardenID(ui32 node) { char x[12] = {'b','s','n','o','d','e','c','n','t','r','l','r'}; - return TActorId(node, TStringBuf(x, 12)); + return TActorId(node, TStringBuf(x, 12)); } } // namespace NKikimr diff --git a/ydb/core/base/shared_quota.h b/ydb/core/base/shared_quota.h index 03c45c9f657..a565954bb54 100644 --- a/ydb/core/base/shared_quota.h +++ b/ydb/core/base/shared_quota.h @@ -1,7 +1,7 @@ #pragma once #include "defs.h" -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <util/generic/ptr.h> diff --git a/ydb/core/base/statestorage.cpp b/ydb/core/base/statestorage.cpp index fc73bd7053a..de39a327ae1 100644 --- a/ydb/core/base/statestorage.cpp +++ b/ydb/core/base/statestorage.cpp @@ -71,7 +71,7 @@ void TStateStorageInfo::SelectReplicas(ui64 tabletId, TSelection *selection) con if (selection->Sz < NToSelect) { selection->Status.Reset(new TStateStorageInfo::TSelection::EStatus[NToSelect]); - selection->SelectedReplicas.Reset(new TActorId[NToSelect]); + selection->SelectedReplicas.Reset(new TActorId[NToSelect]); } selection->Sz = NToSelect; @@ -89,7 +89,7 @@ void TStateStorageInfo::SelectReplicas(ui64 tabletId, TSelection *selection) con } } -TActorId TStateStorageInfo::TRing::SelectReplica(ui32 hash) const { +TActorId TStateStorageInfo::TRing::SelectReplica(ui32 hash) const { if (Replicas.size() == 1) return Replicas[0]; @@ -101,12 +101,12 @@ TActorId TStateStorageInfo::TRing::SelectReplica(ui32 hash) const { } } -TList<TActorId> TStateStorageInfo::SelectAllReplicas() const { +TList<TActorId> TStateStorageInfo::SelectAllReplicas() const { // TODO: we really need this method in such way? - TList<TActorId> replicas; + TList<TActorId> replicas; for (auto &ring : Rings) { - for (TActorId replica : ring.Replicas) + for (TActorId replica : ring.Replicas) replicas.push_back(replica); } @@ -115,7 +115,7 @@ TList<TActorId> TStateStorageInfo::SelectAllReplicas() const { ui32 TStateStorageInfo::TRing::ContentHash() const { ui64 hash = 17; - for (TActorId replica : Replicas) { + for (TActorId replica : Replicas) { hash = Hash64to32((hash << 32) | replica.Hash32()); } return static_cast<ui32>(hash); @@ -233,7 +233,7 @@ static void CopyStateStorageRingInfo( for (ui32 inode = 0, enode = source.NodeSize(); inode != enode; ++inode) { serviceId[depth] = (inode + 1); - const TActorId replicaActorID = TActorId(source.GetNode(inode), TStringBuf(serviceId, serviceId + 12)); + const TActorId replicaActorID = TActorId(source.GetNode(inode), TStringBuf(serviceId, serviceId + 12)); info->Rings[inode].Replicas.push_back(replicaActorID); } @@ -243,12 +243,12 @@ static void CopyStateStorageRingInfo( Y_FAIL("must have rings or legacy node config"); } -TIntrusivePtr<TStateStorageInfo> BuildStateStorageInfo(char (&namePrefix)[TActorId::MaxServiceIDLength], const NKikimrConfig::TDomainsConfig::TStateStorage& config) { +TIntrusivePtr<TStateStorageInfo> BuildStateStorageInfo(char (&namePrefix)[TActorId::MaxServiceIDLength], const NKikimrConfig::TDomainsConfig::TStateStorage& config) { TIntrusivePtr<TStateStorageInfo> info = new TStateStorageInfo(); info->StateStorageGroup = config.GetSSId(); const size_t offset = FindIndex(namePrefix, char()); - Y_VERIFY(offset != NPOS && (offset + sizeof(ui32)) < TActorId::MaxServiceIDLength); + Y_VERIFY(offset != NPOS && (offset + sizeof(ui32)) < TActorId::MaxServiceIDLength); memcpy(namePrefix + offset, reinterpret_cast<const char *>(&info->StateStorageGroup), sizeof(ui32)); CopyStateStorageRingInfo(config.GetRing(), info.Get(), namePrefix, offset + sizeof(ui32)); @@ -261,9 +261,9 @@ void BuildStateStorageInfos(const NKikimrConfig::TDomainsConfig::TStateStorage& TIntrusivePtr<TStateStorageInfo> &boardInfo, TIntrusivePtr<TStateStorageInfo> &schemeBoardInfo) { - char ssr[TActorId::MaxServiceIDLength] = { 's', 's', 'r' }; // state storage replica - char ssb[TActorId::MaxServiceIDLength] = { 's', 's', 'b' }; // state storage board - char sbr[TActorId::MaxServiceIDLength] = { 's', 'b', 'r' }; // scheme board replica + char ssr[TActorId::MaxServiceIDLength] = { 's', 's', 'r' }; // state storage replica + char ssb[TActorId::MaxServiceIDLength] = { 's', 's', 'b' }; // state storage board + char sbr[TActorId::MaxServiceIDLength] = { 's', 'b', 'r' }; // scheme board replica stateStorageInfo = BuildStateStorageInfo(ssr, config); boardInfo = BuildStateStorageInfo(ssb, config); diff --git a/ydb/core/base/statestorage.h b/ydb/core/base/statestorage.h index bc5f967f1aa..9b3e3b6b1b7 100644 --- a/ydb/core/base/statestorage.h +++ b/ydb/core/base/statestorage.h @@ -393,7 +393,7 @@ struct TEvStateStorage { }; struct TEvResponseReplicasDumps : public TEventLocal<TEvResponseReplicasDumps, EvResponseReplicasDumps> { - TVector<std::pair<TActorId, TAutoPtr<TEvReplicaDump>>> ReplicasDumps; + TVector<std::pair<TActorId, TAutoPtr<TEvReplicaDump>>> ReplicasDumps; }; struct TEvReplicaRegFollower : public TEventPB<TEvReplicaRegFollower, NKikimrStateStorage::TEvRegisterFollower, EvReplicaRegFollower> { @@ -433,7 +433,7 @@ struct TEvStateStorage { const EStatus Status; const TString Path; - TMap<TActorId, TInfoEntry> InfoEntries; + TMap<TActorId, TInfoEntry> InfoEntries; TEvBoardInfo(EStatus status, const TString &path) : Status(status) @@ -458,7 +458,7 @@ struct TStateStorageInfo : public TThrRefBase { }; ui32 Sz; - TArrayHolder<TActorId> SelectedReplicas; + TArrayHolder<TActorId> SelectedReplicas; TArrayHolder<EStatus> Status; TSelection() @@ -467,15 +467,15 @@ struct TStateStorageInfo : public TThrRefBase { void MergeReply(EStatus status, EStatus *owner, ui64 targetCookie, bool resetOld); - const TActorId* begin() const { return SelectedReplicas.Get(); } - const TActorId* end() const { return SelectedReplicas.Get() + Sz; } + const TActorId* begin() const { return SelectedReplicas.Get(); } + const TActorId* end() const { return SelectedReplicas.Get() + Sz; } }; struct TRing { bool UseRingSpecificNodeSelection; - TVector<TActorId> Replicas; + TVector<TActorId> Replicas; - TActorId SelectReplica(ui32 hash) const; + TActorId SelectReplica(ui32 hash) const; ui32 ContentHash() const; }; @@ -484,7 +484,7 @@ struct TStateStorageInfo : public TThrRefBase { TVector<TRing> Rings; void SelectReplicas(ui64 tabletId, TSelection *selection) const; - TList<TActorId> SelectAllReplicas() const; + TList<TActorId> SelectAllReplicas() const; ui32 ContentHash() const; TStateStorageInfo() @@ -506,7 +506,7 @@ enum class EBoardLookupMode { MajorityDoubleTime, }; -TIntrusivePtr<TStateStorageInfo> BuildStateStorageInfo(char (&namePrefix)[TActorId::MaxServiceIDLength], const NKikimrConfig::TDomainsConfig::TStateStorage& config); +TIntrusivePtr<TStateStorageInfo> BuildStateStorageInfo(char (&namePrefix)[TActorId::MaxServiceIDLength], const NKikimrConfig::TDomainsConfig::TStateStorage& config); void BuildStateStorageInfos(const NKikimrConfig::TDomainsConfig::TStateStorage& config, TIntrusivePtr<TStateStorageInfo> &stateStorageInfo, TIntrusivePtr<TStateStorageInfo> &boardInfo, @@ -517,13 +517,13 @@ IActor* CreateStateStorageWarden(const TIntrusivePtr<TStateStorageInfo> &info, c IActor* CreateStateStorageProxy(const TIntrusivePtr<TStateStorageInfo> &info, const TIntrusivePtr<TStateStorageInfo> &board, const TIntrusivePtr<TStateStorageInfo> &schemeBoard); IActor* CreateStateStorageProxyStub(); IActor* CreateStateStorageReplica(const TIntrusivePtr<TStateStorageInfo> &info, ui32 replicaIndex); -IActor* CreateStateStorageMonitoringActor(ui64 targetTablet, const TActorId &sender, const TString &query); +IActor* CreateStateStorageMonitoringActor(ui64 targetTablet, const TActorId &sender, const TString &query); IActor* CreateStateStorageTabletGuardian(ui64 tabletId, const TActorId &leader, const TActorId &tabletLeader, ui32 generation); IActor* CreateStateStorageFollowerGuardian(ui64 tabletId, const TActorId &follower); // created as followerCandidate IActor* CreateStateStorageBoardReplica(const TIntrusivePtr<TStateStorageInfo> &, ui32); IActor* CreateSchemeBoardReplica(const TIntrusivePtr<TStateStorageInfo>&, ui32); -IActor* CreateBoardLookupActor(const TString &path, const TActorId &owner, ui32 groupId, EBoardLookupMode mode, bool sub, bool useNodeSubscriptions); -IActor* CreateBoardPublishActor(const TString &path, const TString &payload, const TActorId &owner, ui32 groupId, ui32 ttlMs, bool reg); +IActor* CreateBoardLookupActor(const TString &path, const TActorId &owner, ui32 groupId, EBoardLookupMode mode, bool sub, bool useNodeSubscriptions); +IActor* CreateBoardPublishActor(const TString &path, const TString &payload, const TActorId &owner, ui32 groupId, ui32 ttlMs, bool reg); TString MakeEndpointsBoardPath(const TString &database); diff --git a/ydb/core/base/statestorage_guardian.cpp b/ydb/core/base/statestorage_guardian.cpp index 0dd8642f28f..f1df97896b5 100644 --- a/ydb/core/base/statestorage_guardian.cpp +++ b/ydb/core/base/statestorage_guardian.cpp @@ -56,7 +56,7 @@ struct TGuardedInfo : public TAtomicRefCount<TGuardedInfo> { struct TFollowerInfo : public TAtomicRefCount<TGuardedInfo> { const ui64 TabletID; const TActorId Follower; - const TActorId Tablet; + const TActorId Tablet; const bool IsCandidate; TFollowerInfo(ui64 tabletId, TActorId follower, TActorId tablet, bool isCandidate) @@ -69,8 +69,8 @@ struct TFollowerInfo : public TAtomicRefCount<TGuardedInfo> { class TReplicaGuardian : public TActorBootstrapped<TReplicaGuardian> { TIntrusiveConstPtr<TGuardedInfo> Info; - const TActorId Replica; - const TActorId Guard; + const TActorId Replica; + const TActorId Guard; ui64 Signature; TInstant DowntimeFrom; @@ -80,7 +80,7 @@ class TReplicaGuardian : public TActorBootstrapped<TReplicaGuardian> { Send(TActivationContext::InterconnectProxy(Replica.NodeId()), new TEvents::TEvUnsubscribe); if (KIKIMR_ALLOW_SSREPLICA_PROBES) { - const TActorId ssProxyId = MakeStateStorageProxyID(StateStorageGroupFromTabletID(Info->TabletID)); + const TActorId ssProxyId = MakeStateStorageProxyID(StateStorageGroupFromTabletID(Info->TabletID)); Send(ssProxyId, new TEvStateStorage::TEvReplicaProbeUnsubscribe(Replica)); } @@ -89,7 +89,7 @@ class TReplicaGuardian : public TActorBootstrapped<TReplicaGuardian> { void RequestInfo() { if (KIKIMR_ALLOW_SSREPLICA_PROBES) { - const TActorId ssProxyId = MakeStateStorageProxyID(StateStorageGroupFromTabletID(Info->TabletID)); + const TActorId ssProxyId = MakeStateStorageProxyID(StateStorageGroupFromTabletID(Info->TabletID)); Send(ssProxyId, new TEvStateStorage::TEvReplicaProbeSubscribe(Replica)); Become(&TThis::StateLookup); } else { @@ -175,7 +175,7 @@ public: return NKikimrServices::TActivity::SS_REPLICA_GUARDIAN; } - TReplicaGuardian(TGuardedInfo *info, TActorId replica, TActorId guard) + TReplicaGuardian(TGuardedInfo *info, TActorId replica, TActorId guard) : Info(info) , Replica(replica) , Guard(guard) @@ -230,8 +230,8 @@ public: class TFollowerGuardian : public TActorBootstrapped<TFollowerGuardian> { TIntrusiveConstPtr<TFollowerInfo> Info; - const TActorId Replica; - const TActorId Guard; + const TActorId Replica; + const TActorId Guard; TInstant DowntimeFrom; @@ -246,7 +246,7 @@ class TFollowerGuardian : public TActorBootstrapped<TFollowerGuardian> { void UpdateInfo() { if (KIKIMR_ALLOW_SSREPLICA_PROBES) { - const TActorId ssProxyId = MakeStateStorageProxyID(StateStorageGroupFromTabletID(Info->TabletID)); + const TActorId ssProxyId = MakeStateStorageProxyID(StateStorageGroupFromTabletID(Info->TabletID)); Send(ssProxyId, new TEvStateStorage::TEvReplicaProbeSubscribe(Replica)); Become(&TThis::StateCalm); } else { @@ -279,7 +279,7 @@ class TFollowerGuardian : public TActorBootstrapped<TFollowerGuardian> { Send(TActivationContext::InterconnectProxy(Replica.NodeId()), new TEvents::TEvUnsubscribe()); if (KIKIMR_ALLOW_SSREPLICA_PROBES) { - const TActorId ssProxyId = MakeStateStorageProxyID(StateStorageGroupFromTabletID(Info->TabletID)); + const TActorId ssProxyId = MakeStateStorageProxyID(StateStorageGroupFromTabletID(Info->TabletID)); Send(ssProxyId, new TEvStateStorage::TEvReplicaProbeUnsubscribe(Replica)); } @@ -339,12 +339,12 @@ class TTabletGuardian : public TActorBootstrapped<TTabletGuardian> { TIntrusivePtr<TGuardedInfo> Info; TIntrusivePtr<TFollowerInfo> FollowerInfo; - TVector<std::pair<TActorId, TActorId>> ReplicaGuardians; // replica -> guardian, position dependant so vector + TVector<std::pair<TActorId, TActorId>> ReplicaGuardians; // replica -> guardian, position dependant so vector ui32 ReplicasOnlineThreshold; THolder<TFollowerTracker> FollowerTracker; - TActorId Launcher() const { + TActorId Launcher() const { return Info ? Info->Leader : FollowerInfo->Follower; } @@ -356,12 +356,12 @@ class TTabletGuardian : public TActorBootstrapped<TTabletGuardian> { } void Handle(TEvStateStorage::TEvResolveReplicasList::TPtr &ev) { - const TVector<TActorId> &replicasList = ev->Get()->Replicas; + const TVector<TActorId> &replicasList = ev->Get()->Replicas; Y_VERIFY(!replicasList.empty(), "must not happens, guardian must be created over active tablet"); const ui32 replicaSz = replicasList.size(); - TVector<std::pair<TActorId, TActorId>> updatedReplicaGuardians; + TVector<std::pair<TActorId, TActorId>> updatedReplicaGuardians; updatedReplicaGuardians.reserve(replicaSz); const bool inspectCurrent = (ReplicaGuardians.size() == replicaSz); @@ -374,11 +374,11 @@ class TTabletGuardian : public TActorBootstrapped<TTabletGuardian> { } for (ui32 idx : xrange(replicasList.size())) { - const TActorId replica = replicasList[idx]; + const TActorId replica = replicasList[idx]; if (inspectCurrent && ReplicaGuardians[idx].first == replica && ReplicaGuardians[idx].second) { updatedReplicaGuardians.emplace_back(ReplicaGuardians[idx]); - ReplicaGuardians[idx].second = TActorId(); + ReplicaGuardians[idx].second = TActorId(); } else { if (Info) updatedReplicaGuardians.emplace_back(replica, RegisterWithSameMailbox(new TReplicaGuardian(Info.Get(), replica, SelfId()))); @@ -406,13 +406,13 @@ class TTabletGuardian : public TActorBootstrapped<TTabletGuardian> { Y_FAIL("must not happens, guardian must be created over active tablet"); } - bool ReplicaDown(TActorId guardian) { + bool ReplicaDown(TActorId guardian) { ui32 replicasOnline = 0; bool ret = false; for (auto it = ReplicaGuardians.begin(), end = ReplicaGuardians.end(); it != end; ++it) { if (it->second == guardian) { - it->second = TActorId(); + it->second = TActorId(); ret = true; } else if (it->second) { ++replicasOnline; @@ -433,7 +433,7 @@ class TTabletGuardian : public TActorBootstrapped<TTabletGuardian> { void SendResolveRequest(TDuration delay) { const ui64 tabletId = Info ? Info->TabletID : FollowerInfo->TabletID; const ui64 stateStorageGroup = StateStorageGroupFromTabletID(tabletId); - const TActorId proxyActorID = MakeStateStorageProxyID(stateStorageGroup); + const TActorId proxyActorID = MakeStateStorageProxyID(stateStorageGroup); if (delay == TDuration::Zero()) { Send(proxyActorID, new TEvStateStorage::TEvResolveReplicas(tabletId), IEventHandle::FlagTrackDelivery); @@ -463,19 +463,19 @@ class TTabletGuardian : public TActorBootstrapped<TTabletGuardian> { Y_VERIFY(FollowerTracker); const NKikimrStateStorage::TEvInfo &record = ev->Get()->Record; - const TActorId guardian = ev->Sender; + const TActorId guardian = ev->Sender; for (ui32 idx : xrange(ReplicaGuardians.size())) { if (ReplicaGuardians[idx].second != guardian) continue; - TVector<TActorId> reported; + TVector<TActorId> reported; reported.reserve(record.FollowerSize() + record.FollowerCandidatesSize()); for (const auto &x : record.GetFollower()) { - reported.emplace_back(ActorIdFromProto(x)); + reported.emplace_back(ActorIdFromProto(x)); } for (const auto &x : record.GetFollowerCandidates()) { - reported.emplace_back(ActorIdFromProto(x)); + reported.emplace_back(ActorIdFromProto(x)); } Sort(reported); @@ -521,7 +521,7 @@ class TTabletGuardian : public TActorBootstrapped<TTabletGuardian> { return; for (auto &xpair : ReplicaGuardians) { - const TActorId guardian = xpair.second; + const TActorId guardian = xpair.second; Send(guardian, new TEvPrivate::TEvRefreshFollowerState(FollowerInfo)); } } diff --git a/ydb/core/base/statestorage_guardian_impl.h b/ydb/core/base/statestorage_guardian_impl.h index b6a34db6e05..0bdda91eadb 100644 --- a/ydb/core/base/statestorage_guardian_impl.h +++ b/ydb/core/base/statestorage_guardian_impl.h @@ -10,7 +10,7 @@ struct TFollowerTracker { TVector<TVector<TActorId>> Reported; // reported followers by replica index TMap<TActorId, ui32> Merged; // follower -> referenced by - bool AddMerged(TActorId x) { + bool AddMerged(TActorId x) { auto itPair = Merged.emplace(x, 1); if (itPair.second) { return true; @@ -20,7 +20,7 @@ struct TFollowerTracker { } } - bool DelMerged(TActorId x) { + bool DelMerged(TActorId x) { auto it = Merged.find(x); Y_VERIFY(it != Merged.end(), "follower tracker consistency broken"); @@ -37,22 +37,22 @@ public: : Reported(replicas) {} - const TMap<TActorId, ui32>& GetMerged() const { + const TMap<TActorId, ui32>& GetMerged() const { return Merged; } // update reported list for replica, returns true if smth changed // reported must be sorted - bool Merge(ui32 replicaIdx, TVector<TActorId> &reported) { + bool Merge(ui32 replicaIdx, TVector<TActorId> &reported) { bool changed = false; - TVector<TActorId> &old = Reported[replicaIdx]; + TVector<TActorId> &old = Reported[replicaIdx]; const ui32 oldSz = old.size(); bool gotDuplicates = false; ui32 oldIdx = 0; - TActorId prevReported; - for (TActorId x : reported) { + TActorId prevReported; + for (TActorId x : reported) { if (x == prevReported) { // skip duplicated gotDuplicates = true; continue; diff --git a/ydb/core/base/statestorage_guardian_impl_ut.cpp b/ydb/core/base/statestorage_guardian_impl_ut.cpp index 1abdd534142..00638ab480c 100644 --- a/ydb/core/base/statestorage_guardian_impl_ut.cpp +++ b/ydb/core/base/statestorage_guardian_impl_ut.cpp @@ -30,9 +30,9 @@ namespace NKikimr { auto merged = tracker.GetMerged(); UNIT_ASSERT(merged.size() == 2); - UNIT_ASSERT(merged.FindPtr(TActorId(1, 1, 1, 1)) != nullptr); - UNIT_ASSERT(merged.FindPtr(TActorId(1, 1, 5, 1)) != nullptr); - UNIT_ASSERT(merged.FindPtr(TActorId(1, 1, 10, 1)) == nullptr); + UNIT_ASSERT(merged.FindPtr(TActorId(1, 1, 1, 1)) != nullptr); + UNIT_ASSERT(merged.FindPtr(TActorId(1, 1, 5, 1)) != nullptr); + UNIT_ASSERT(merged.FindPtr(TActorId(1, 1, 10, 1)) == nullptr); } Y_UNIT_TEST(FollowerTrackerDuplicates) { diff --git a/ydb/core/base/statestorage_impl.h b/ydb/core/base/statestorage_impl.h index 5e64ab79c06..2a2aa22f6a7 100644 --- a/ydb/core/base/statestorage_impl.h +++ b/ydb/core/base/statestorage_impl.h @@ -4,11 +4,11 @@ namespace NKikimr { -inline TActorId MakeStateStorageReplicaID(ui32 node, ui64 stateStorageGroup, ui32 replicaIndex) { +inline TActorId MakeStateStorageReplicaID(ui32 node, ui64 stateStorageGroup, ui32 replicaIndex) { char x[12] = { 's', 't', 's' }; x[3] = (char)stateStorageGroup; memcpy(x + 5, &replicaIndex, sizeof(ui32)); - return TActorId(node, TStringBuf(x, 12)); + return TActorId(node, TStringBuf(x, 12)); } struct TEvStateStorage::TEvReplicaInfo : public TEventPB<TEvStateStorage::TEvReplicaInfo, NKikimrStateStorage::TEvInfo, TEvStateStorage::EvReplicaInfo> { @@ -80,33 +80,33 @@ struct TEvStateStorage::TEvUpdateGroupConfig : public TEventLocal<TEvUpdateGroup }; struct TEvStateStorage::TEvReplicaProbeSubscribe : public TEventLocal<TEvReplicaProbeSubscribe, EvReplicaProbeSubscribe> { - const TActorId ReplicaId; + const TActorId ReplicaId; - TEvReplicaProbeSubscribe(TActorId replicaId) + TEvReplicaProbeSubscribe(TActorId replicaId) : ReplicaId(replicaId) {} }; struct TEvStateStorage::TEvReplicaProbeUnsubscribe : public TEventLocal<TEvReplicaProbeUnsubscribe, EvReplicaProbeUnsubscribe> { - const TActorId ReplicaId; + const TActorId ReplicaId; - TEvReplicaProbeUnsubscribe(TActorId replicaId) + TEvReplicaProbeUnsubscribe(TActorId replicaId) : ReplicaId(replicaId) {} }; struct TEvStateStorage::TEvReplicaProbeConnected : public TEventLocal<TEvReplicaProbeConnected, EvReplicaProbeConnected> { - const TActorId ReplicaId; + const TActorId ReplicaId; - TEvReplicaProbeConnected(TActorId replicaId) + TEvReplicaProbeConnected(TActorId replicaId) : ReplicaId(replicaId) {} }; struct TEvStateStorage::TEvReplicaProbeDisconnected : public TEventLocal<TEvReplicaProbeDisconnected, EvReplicaProbeDisconnected> { - const TActorId ReplicaId; + const TActorId ReplicaId; - TEvReplicaProbeDisconnected(TActorId replicaId) + TEvReplicaProbeDisconnected(TActorId replicaId) : ReplicaId(replicaId) {} }; @@ -150,7 +150,7 @@ struct TEvStateStorage::TEvResolveSchemeBoard : public TEventLocal<TEvResolveSch }; struct TEvStateStorage::TEvResolveReplicasList : public TEventLocal<TEvResolveReplicasList, EvResolveReplicasList> { - TVector<TActorId> Replicas; + TVector<TActorId> Replicas; ui32 ConfigContentHash = Max<ui32>(); }; @@ -267,13 +267,13 @@ struct TEvStateStorage::TEvReplicaBoardPublish : public TEventPB<TEvStateStorage TEvReplicaBoardPublish() {} - TEvReplicaBoardPublish(const TString &path, const TString &payload, ui64 ttlMs, bool reg, TActorId owner) + TEvReplicaBoardPublish(const TString &path, const TString &payload, ui64 ttlMs, bool reg, TActorId owner) { Record.SetPath(path); Record.SetPayload(payload); Record.SetTtlMs(ttlMs); Record.SetRegister(reg); - ActorIdToProto(owner, Record.MutableOwner()); + ActorIdToProto(owner, Record.MutableOwner()); } }; @@ -281,10 +281,10 @@ struct TEvStateStorage::TEvReplicaBoardLookup : public TEventPB<TEvStateStorage: TEvReplicaBoardLookup() {} - TEvReplicaBoardLookup(const TString &path, TActorId owner, bool sub) + TEvReplicaBoardLookup(const TString &path, TActorId owner, bool sub) { Record.SetPath(path); - ActorIdToProto(owner, Record.MutableOwner()); + ActorIdToProto(owner, Record.MutableOwner()); Record.SetSubscribe(sub); } }; @@ -310,6 +310,6 @@ struct TEvStateStorage::TEvReplicaBoardInfo : public TEventPB<TEvStateStorage::T } }; -IActor* CreateStateStorageReplicaProbe(TActorId replica); +IActor* CreateStateStorageReplicaProbe(TActorId replica); } diff --git a/ydb/core/base/statestorage_monitoring.cpp b/ydb/core/base/statestorage_monitoring.cpp index a4a98a19dbb..5bcc3ec67fa 100644 --- a/ydb/core/base/statestorage_monitoring.cpp +++ b/ydb/core/base/statestorage_monitoring.cpp @@ -2,7 +2,7 @@ #include "tabletid.h" #include <ydb/core/protos/services.pb.h> #include <library/cpp/actors/core/interconnect.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <library/cpp/actors/core/mon.h> #include <library/cpp/actors/core/hfunc.h> #include <library/cpp/actors/core/actor_bootstrapped.h> @@ -11,7 +11,7 @@ namespace NKikimr { class TStateStorageMonitoringActor : public TActorBootstrapped<TStateStorageMonitoringActor> { struct TReplicaInfo { - TActorId ActorID; + TActorId ActorID; TInstant ReplyTime; TActorId CurrentLeader; @@ -22,7 +22,7 @@ class TStateStorageMonitoringActor : public TActorBootstrapped<TStateStorageMoni bool Locked; ui64 LockedFor; - TReplicaInfo(const TActorId &x) + TReplicaInfo(const TActorId &x) : ActorID(x) , ReplyTime(TInstant::MicroSeconds(Max<ui64>())) , CurrentLeader() @@ -35,7 +35,7 @@ class TStateStorageMonitoringActor : public TActorBootstrapped<TStateStorageMoni }; const ui64 TabletID; - const TActorId Sender; + const TActorId Sender; const TString Query; TInstant BeginMoment; @@ -126,7 +126,7 @@ class TStateStorageMonitoringActor : public TActorBootstrapped<TStateStorageMoni } void Handle(TEvStateStorage::TEvResolveReplicasList::TPtr &ev, const TActorContext &ctx) { - const TVector<TActorId> &replicasList = ev->Get()->Replicas; + const TVector<TActorId> &replicasList = ev->Get()->Replicas; if (replicasList.empty()) return Reply("empty replica list", ctx); @@ -138,7 +138,7 @@ class TStateStorageMonitoringActor : public TActorBootstrapped<TStateStorageMoni ReplicasInfo.reserve(replicasList.size()); for (ui64 cookie = 0, e = replicasList.size(); cookie < e; ++cookie) { - const TActorId &replica = replicasList[cookie]; + const TActorId &replica = replicasList[cookie]; ReplicasInfo.push_back(replica); ctx.Send(replica, new TEvStateStorage::TEvReplicaLookup(TabletID, cookie)); } @@ -202,7 +202,7 @@ public: return NKikimrServices::TActivity::SS_MON; } - TStateStorageMonitoringActor(ui64 tabletId, const TActorId &sender, const TString &query) + TStateStorageMonitoringActor(ui64 tabletId, const TActorId &sender, const TString &query) : TabletID(tabletId) , Sender(sender) , Query(query) @@ -213,7 +213,7 @@ public: void Bootstrap(const TActorContext &ctx) { // try to send monitoring request to proxy const ui64 stateStorageGroup = StateStorageGroupFromTabletID(TabletID); - const TActorId proxyActorID = MakeStateStorageProxyID(stateStorageGroup); + const TActorId proxyActorID = MakeStateStorageProxyID(stateStorageGroup); BeginMoment = ctx.Now(); @@ -239,7 +239,7 @@ public: } }; -IActor* CreateStateStorageMonitoringActor(ui64 targetTablet, const TActorId &sender, const TString &query) { +IActor* CreateStateStorageMonitoringActor(ui64 targetTablet, const TActorId &sender, const TString &query) { return new TStateStorageMonitoringActor(targetTablet, sender, query); } diff --git a/ydb/core/base/statestorage_proxy.cpp b/ydb/core/base/statestorage_proxy.cpp index afb72b8acef..13d396cb0f1 100644 --- a/ydb/core/base/statestorage_proxy.cpp +++ b/ydb/core/base/statestorage_proxy.cpp @@ -40,7 +40,7 @@ class TStateStorageProxyRequest : public TActor<TStateStorageProxyRequest> { ui32 SuggestedStep; TActorId SuggestedLeader; TActorId SuggestedLeaderTablet; - TActorId Source; + TActorId Source; ui32 Replicas; THolder<TStateStorageInfo::TSelection> ReplicaSelection; @@ -600,9 +600,9 @@ public: class TStateStorageDumpRequest : public TActorBootstrapped<TStateStorageDumpRequest> { protected: - const TActorId Sender; + const TActorId Sender; TIntrusivePtr<TStateStorageInfo> Info; - TList<TActorId> AllReplicas; + TList<TActorId> AllReplicas; TAutoPtr<TEvStateStorage::TEvResponseReplicasDumps> Response; ui64 UndeliveredCount; @@ -611,7 +611,7 @@ public: return NKikimrServices::TActivity::TABLET_FORWARDING_ACTOR; } - TStateStorageDumpRequest(const TActorId &sender, const TIntrusivePtr<TStateStorageInfo> &info) + TStateStorageDumpRequest(const TActorId &sender, const TIntrusivePtr<TStateStorageInfo> &info) : Sender(sender) , Info(info) , UndeliveredCount(0) @@ -627,7 +627,7 @@ public: AllReplicas = Info->SelectAllReplicas(); if (!AllReplicas.empty()) { Response->ReplicasDumps.reserve(AllReplicas.size()); - for (const TActorId &replica : AllReplicas) { + for (const TActorId &replica : AllReplicas) { Send(replica, new TEvStateStorage::TEvReplicaDumpRequest(), IEventHandle::FlagTrackDelivery); } Schedule(TDuration::Seconds(60), new TEvents::TEvWakeup()); @@ -667,9 +667,9 @@ public: class TStateStorageDeleteRequest : public TActorBootstrapped<TStateStorageDeleteRequest> { protected: - const TActorId Sender; + const TActorId Sender; TIntrusivePtr<TStateStorageInfo> Info; - TList<TActorId> AllReplicas; + TList<TActorId> AllReplicas; ui32 Count; ui32 UndeliveredCount; ui64 TabletID; @@ -679,7 +679,7 @@ public: return NKikimrServices::TActivity::TABLET_FORWARDING_ACTOR; } - TStateStorageDeleteRequest(const TActorId &sender, const TIntrusivePtr<TStateStorageInfo> &info, ui64 tabletId) + TStateStorageDeleteRequest(const TActorId &sender, const TIntrusivePtr<TStateStorageInfo> &info, ui64 tabletId) : Sender(sender) , Info(info) , Count(0) @@ -695,7 +695,7 @@ public: void Bootstrap() { AllReplicas = Info->SelectAllReplicas(); if (!AllReplicas.empty()) { - for (const TActorId &replica : AllReplicas) { + for (const TActorId &replica : AllReplicas) { Send(replica, new TEvStateStorage::TEvReplicaDelete(TabletID), IEventHandle::FlagTrackDelivery); } Schedule(TDuration::Seconds(60), new TEvents::TEvWakeup()); @@ -740,7 +740,7 @@ class TStateStorageProxy : public TActor<TStateStorageProxy> { TIntrusivePtr<TStateStorageInfo> FlowControlledInfo; - TMap<TActorId, TActorId> ReplicaProbes; + TMap<TActorId, TActorId> ReplicaProbes; void Handle(TEvStateStorage::TEvRequestReplicasDumps::TPtr &ev) { TActivationContext::Register(new TStateStorageDumpRequest(ev->Sender, Info)); @@ -866,8 +866,8 @@ class TStateStorageProxy : public TActor<TStateStorageProxy> { ReplicaProbes.clear(); for (auto &ring : Info->Rings) - for (const TActorId replicaId : ring.Replicas) { - const TActorId probeId = sys->Register(CreateStateStorageReplicaProbe(replicaId)); + for (const TActorId replicaId : ring.Replicas) { + const TActorId probeId = sys->Register(CreateStateStorageReplicaProbe(replicaId)); ReplicaProbes.emplace(replicaId, probeId); } } @@ -911,7 +911,7 @@ class TStateStorageProxy : public TActor<TStateStorageProxy> { for (const ui32 srcSize = srcring.Replicas.size(); replicaIdx < srcSize; ++replicaIdx) { if (checkRing && srcring.Replicas[replicaIdx] == oldring->Replicas[replicaIdx]) { ctring.Replicas[replicaIdx] = fcring->Replicas[replicaIdx]; - fcring->Replicas[replicaIdx] = TActorId(); + fcring->Replicas[replicaIdx] = TActorId(); } else { if (fcring && replicaIdx < fcring->Replicas.size()) Send(fcring->Replicas[replicaIdx], new TEvents::TEvPoison()); @@ -934,7 +934,7 @@ class TStateStorageProxy : public TActor<TStateStorageProxy> { } if (FlowControlledInfo) { for (const ui32 oldSize = FlowControlledInfo->Rings.size(); ringIdx < oldSize; ++ringIdx) { - for (TActorId outdated : FlowControlledInfo->Rings[oldSize].Replicas) + for (TActorId outdated : FlowControlledInfo->Rings[oldSize].Replicas) Send(outdated, new TEvents::TEvPoison()); } } @@ -942,7 +942,7 @@ class TStateStorageProxy : public TActor<TStateStorageProxy> { FlowControlledInfo = std::move(updated); } - void Registered(TActorSystem* sys, const TActorId&) { + void Registered(TActorSystem* sys, const TActorId&) { RegisterDerivedServices(sys, nullptr); } public: @@ -992,8 +992,8 @@ class TStateStorageProxyStub : public TActor<TStateStorageProxyStub> { Send(ev->Sender, new TEvStateStorage::TEvInfo( NKikimrProto::ERROR, - tabletId, cookie, TActorId(), TActorId(), 0, 0, false, 0, - nullptr, 0, TMap<TActorId, TActorId>())); + tabletId, cookie, TActorId(), TActorId(), 0, 0, false, 0, + nullptr, 0, TMap<TActorId, TActorId>())); } public: static constexpr NKikimrServices::TActivity::EType ActorActivityType() { diff --git a/ydb/core/base/statestorage_replica.cpp b/ydb/core/base/statestorage_replica.cpp index f0be6416c00..aca41acb16a 100644 --- a/ydb/core/base/statestorage_replica.cpp +++ b/ydb/core/base/statestorage_replica.cpp @@ -1,20 +1,20 @@ #include "statestorage_impl.h" #include "tabletid.h" -#include "appdata.h" +#include "appdata.h" #include "tablet.h" - + #include <ydb/core/protos/services.pb.h> #include <library/cpp/actors/core/interconnect.h> #include <library/cpp/actors/core/hfunc.h> #include <library/cpp/actors/core/log.h> - + #include <util/generic/map.h> #include <util/generic/hash_set.h> #if defined BLOG_D || defined BLOG_I || defined BLOG_ERROR || defined BLOG_TRACE #error log macro definition clash #endif - + #define BLOG_D(stream) LOG_DEBUG_S(*TlsActivationContext, NKikimrServices::STATESTORAGE, stream) #define BLOG_I(stream) LOG_INFO_S(*TlsActivationContext, NKikimrServices::STATESTORAGE, stream) #define BLOG_W(stream) LOG_WARN_S(*TlsActivationContext, NKikimrServices::STATESTORAGE, stream) @@ -51,7 +51,7 @@ class TStateStorageReplica : public TActor<TStateStorageReplica> { TTabletState::ETabletState TabletState; TActorId CurrentLeader; TActorId CurrentLeaderTablet; - TActorId CurrentGuardian; + TActorId CurrentGuardian; ui32 CurrentGeneration; ui32 CurrentStep; ui64 LockedFrom; @@ -81,7 +81,7 @@ class TStateStorageReplica : public TActor<TStateStorageReplica> { return ok; } - void NotifyWithTabletInfo(const TActorId &recp, ui64 tabletId, ui64 cookie, const TEntry *entry) { + void NotifyWithTabletInfo(const TActorId &recp, ui64 tabletId, ui64 cookie, const TEntry *entry) { THolder<TEvStateStorage::TEvReplicaInfo> msg; if (entry && entry->CurrentLeader) { const bool locked = (entry->TabletState == TTabletState::Locked); @@ -111,7 +111,7 @@ class TStateStorageReplica : public TActor<TStateStorageReplica> { Send(recp, msg.Release()); } - void ReplyWithStatus(const TActorId &recp, ui64 tabletId, ui64 cookie, NKikimrProto::EReplyStatus status) { + void ReplyWithStatus(const TActorId &recp, ui64 tabletId, ui64 cookie, NKikimrProto::EReplyStatus status) { THolder<TEvStateStorage::TEvReplicaInfo> msg(new TEvStateStorage::TEvReplicaInfo(tabletId, status)); msg->Record.SetCookie(cookie); msg->Record.SetSignature(Signature()); @@ -299,7 +299,7 @@ class TStateStorageReplica : public TActor<TStateStorageReplica> { BLOG_D("Replica::Handle ev: " << msg->ToString()); const ui64 tabletId = msg->Record.GetTabletID(); Y_VERIFY_DEBUG(StateStorageGroupFromTabletID(tabletId) == Info->StateStorageGroup); - const TActorId &sender = ev->Sender; + const TActorId &sender = ev->Sender; if (CheckSignature(msg)) { TEntry &x = Tablets[tabletId]; @@ -318,7 +318,7 @@ class TStateStorageReplica : public TActor<TStateStorageReplica> { x.CurrentStep = 0; x.CurrentLeader = proposedLeader; x.CurrentLeaderTablet = TActorId(); - x.CurrentGuardian = TActorId(); + x.CurrentGuardian = TActorId(); x.TabletState = TTabletState::Locked; x.LockedFrom = TActivationContext::Now().MicroSeconds(); } diff --git a/ydb/core/base/statestorage_replica_probe.cpp b/ydb/core/base/statestorage_replica_probe.cpp index 89777c598cf..959c5a77ace 100644 --- a/ydb/core/base/statestorage_replica_probe.cpp +++ b/ydb/core/base/statestorage_replica_probe.cpp @@ -22,12 +22,12 @@ namespace NKikimr { class TStateStorageReplicaProbe : public TActorBootstrapped<TStateStorageReplicaProbe> { - const TActorId ReplicaId; + const TActorId ReplicaId; - TSet<TActorId> Subscribers; + TSet<TActorId> Subscribers; void PassAway() override { - for (TActorId x : Subscribers) + for (TActorId x : Subscribers) Send(x, new TEvStateStorage::TEvReplicaProbeDisconnected(ReplicaId)); Subscribers.clear(); Send(TActivationContext::InterconnectProxy(ReplicaId.NodeId()), new TEvents::TEvUnsubscribe()); @@ -49,7 +49,7 @@ class TStateStorageReplicaProbe : public TActorBootstrapped<TStateStorageReplica void HandlePong() { // we can receive outdated pong, we don't care much, it's best effort - for (TActorId x : Subscribers) + for (TActorId x : Subscribers) Send(x, new TEvStateStorage::TEvReplicaProbeConnected(ReplicaId)); Subscribers.clear(); } @@ -67,7 +67,7 @@ public: Become(&TThis::StateWait); } - TStateStorageReplicaProbe(TActorId replicaId) + TStateStorageReplicaProbe(TActorId replicaId) : ReplicaId(replicaId) {} @@ -96,7 +96,7 @@ public: } }; -IActor* CreateStateStorageReplicaProbe(TActorId replica) { +IActor* CreateStateStorageReplicaProbe(TActorId replica) { return new TStateStorageReplicaProbe(replica); } diff --git a/ydb/core/base/statestorage_ut.cpp b/ydb/core/base/statestorage_ut.cpp index a6c42f2a3be..94a49082db3 100644 --- a/ydb/core/base/statestorage_ut.cpp +++ b/ydb/core/base/statestorage_ut.cpp @@ -14,7 +14,7 @@ Y_UNIT_TEST_SUITE(TStateStorageConfig) { info->Rings.resize(replicas); for (ui32 i : xrange(replicas)) { for (ui32 j : xrange(replicasInRing)) { - info->Rings[i].Replicas.push_back(TActorId(i, i, i + j, i)); + info->Rings[i].Replicas.push_back(TActorId(i, i, i + j, i)); info->Rings[i].UseRingSpecificNodeSelection = useRingSpecificNodeSelection; } } @@ -90,7 +90,7 @@ Y_UNIT_TEST_SUITE(TStateStorageConfig) { } double UniformityRun(ui32 replicas, ui32 nToSelect, ui32 replicasInRing, bool useRingSpecificNodeSelection) { - THashMap<TActorId, ui32> history; + THashMap<TActorId, ui32> history; TStateStorageInfo info; FillStateStorageInfo(&info, replicas, nToSelect, replicasInRing, useRingSpecificNodeSelection); diff --git a/ydb/core/base/statestorage_warden.cpp b/ydb/core/base/statestorage_warden.cpp index 5ef088ac770..972a74e1fb4 100644 --- a/ydb/core/base/statestorage_warden.cpp +++ b/ydb/core/base/statestorage_warden.cpp @@ -60,16 +60,16 @@ class TStateStorageWarden : public TActorBootstrapped<TStateStorageWarden> { // should kill? if (ringIdx < current->Rings.size() && replicaIdx < current->Rings[ringIdx].Replicas.size()) { - const TActorId outdated = current->Rings[ringSz].Replicas[replicaIdx]; + const TActorId outdated = current->Rings[ringSz].Replicas[replicaIdx]; if (outdated.NodeId() == selfNode) { - sys->RegisterLocalService(outdated, TActorId()); + sys->RegisterLocalService(outdated, TActorId()); Send(outdated, new TEvents::TEvPoison()); } } // must start? if (replicas[replicaIdx].NodeId() == selfNode) { - const TActorId replicaActorId = Register(createFunc(updated.Get(), index), TMailboxType::ReadAsFilled, sysPoolId); + const TActorId replicaActorId = Register(createFunc(updated.Get(), index), TMailboxType::ReadAsFilled, sysPoolId); sys->RegisterLocalService(replicas[replicaIdx], replicaActorId); } } diff --git a/ydb/core/base/tablet.h b/ydb/core/base/tablet.h index 34c963c22a6..602e39c6000 100644 --- a/ydb/core/base/tablet.h +++ b/ydb/core/base/tablet.h @@ -182,7 +182,7 @@ struct TEvTablet { TIntrusivePtr<TDependencyGraph> DependencyGraph; - const TActorId Launcher; + const TActorId Launcher; TIntrusivePtr<TTabletStorageInfo> TabletStorageInfo; TResourceProfilesPtr ResourceProfiles; TSharedQuotaPtr TxCacheQuota; @@ -194,7 +194,7 @@ struct TEvTablet { ui64 tabletId, ui32 generation, TDependencyGraph *dependencyGraph, - const TActorId& launcher, + const TActorId& launcher, TIntrusivePtr<TTabletStorageInfo> info, TResourceProfilesPtr profiles = nullptr, TSharedQuotaPtr txCacheQuota = nullptr, @@ -216,7 +216,7 @@ struct TEvTablet { struct TEvRestored : public TEventLocal<TEvRestored, EvRestored> { const ui64 TabletID; const ui32 Generation; - const TActorId UserTabletActor; + const TActorId UserTabletActor; const bool Follower; TEvRestored(ui64 tabletId, ui32 generation, const TActorId &userTabletActor, bool follower) @@ -389,7 +389,7 @@ struct TEvTablet { struct TEvFollowerUpdateState : public TEventLocal<TEvFollowerUpdateState, EvFollowerUpdateState> { const bool IsCandidate; const TActorId FollowerActor; - const TActorId TabletActor; + const TActorId TabletActor; TEvFollowerUpdateState(bool isCandidate, TActorId followerActor, TActorId tabletActor) : IsCandidate(isCandidate) @@ -622,7 +622,7 @@ struct TEvTablet { const ui32 FollowerID; const ui32 Generation; - const TActorId Launcher; + const TActorId Launcher; // must be present one of: or loaded graph or snapshot follower update TIntrusivePtr<TDependencyGraph> DependencyGraph; @@ -780,8 +780,8 @@ struct TEvTablet { }; IActor* CreateTabletKiller(ui64 tabletId, ui32 nodeId = 0, ui32 maxGeneration = Max<ui32>()); -IActor* CreateTabletDSChecker(const TActorId &replyTo, TTabletStorageInfo *info); -IActor* CreateTabletReqReset(const TActorId &replyTo, const TIntrusivePtr<TTabletStorageInfo> &tabletStorageInfo, ui32 knownGeneration = 0); +IActor* CreateTabletDSChecker(const TActorId &replyTo, TTabletStorageInfo *info); +IActor* CreateTabletReqReset(const TActorId &replyTo, const TIntrusivePtr<TTabletStorageInfo> &tabletStorageInfo, ui32 knownGeneration = 0); } diff --git a/ydb/core/base/tablet_killer.cpp b/ydb/core/base/tablet_killer.cpp index 53801bfc281..d46e2c5c895 100644 --- a/ydb/core/base/tablet_killer.cpp +++ b/ydb/core/base/tablet_killer.cpp @@ -46,7 +46,7 @@ public: {} void Bootstrap(const TActorContext &ctx) { - const TActorId stateStorageProxyId = MakeStateStorageProxyID(StateStorageGroupFromTabletID(TabletId)); + const TActorId stateStorageProxyId = MakeStateStorageProxyID(StateStorageGroupFromTabletID(TabletId)); ctx.Send(stateStorageProxyId, new TEvStateStorage::TEvLookup(TabletId, 0)); Become(&TTabletKillRequest::StateFunc); } diff --git a/ydb/core/base/tablet_pipe.h b/ydb/core/base/tablet_pipe.h index 49f04bd1d03..a0420b86724 100644 --- a/ydb/core/base/tablet_pipe.h +++ b/ydb/core/base/tablet_pipe.h @@ -1,13 +1,13 @@ #pragma once - + #include "defs.h" -#include "events.h" - +#include "events.h" + #include <ydb/core/protos/base.pb.h> #include <ydb/core/protos/tablet_pipe.pb.h> #include <library/cpp/actors/core/event_local.h> - + namespace NKikimr { struct TEvTabletPipe { @@ -44,7 +44,7 @@ namespace NKikimr { TEvConnect(ui64 tabletId, const TActorId& clientId, ui32 features = 0) { Record.SetTabletId(tabletId); - ActorIdToProto(clientId, Record.MutableClientId()); + ActorIdToProto(clientId, Record.MutableClientId()); if (features) { Record.SetFeatures(features); } @@ -58,8 +58,8 @@ namespace NKikimr { { Record.SetStatus(status); Record.SetTabletId(tabletId); - ActorIdToProto(clientId, Record.MutableClientId()); - ActorIdToProto(serverId, Record.MutableServerId()); + ActorIdToProto(clientId, Record.MutableClientId()); + ActorIdToProto(serverId, Record.MutableServerId()); Record.SetLeader(leader); } }; @@ -67,11 +67,11 @@ namespace NKikimr { struct TEvPush : public TEventPB<TEvPush, NKikimrTabletPipe::TEvPush, EvPush> { TEvPush() {} - TEvPush(ui64 tabletId, ui32 type, const TActorId& sender, const TStringBuf& buffer, ui64 cookie, bool extendedFormat) + TEvPush(ui64 tabletId, ui32 type, const TActorId& sender, const TStringBuf& buffer, ui64 cookie, bool extendedFormat) { Record.SetTabletId(tabletId); Record.SetType(type); - ActorIdToProto(sender, Record.MutableSender()); + ActorIdToProto(sender, Record.MutableSender()); Record.SetBuffer(buffer.data(), buffer.size()); Record.SetCookie(cookie); Record.SetExtendedFormat(extendedFormat); @@ -85,11 +85,11 @@ namespace NKikimr { struct TEvPeerClosed : public TEventPB<TEvPeerClosed, NKikimrTabletPipe::TEvPeerClosed, EvPeerClosed> { TEvPeerClosed() {} - TEvPeerClosed(ui64 tabletId, const TActorId& clientId, const TActorId& serverId) + TEvPeerClosed(ui64 tabletId, const TActorId& clientId, const TActorId& serverId) { Record.SetTabletId(tabletId); - ActorIdToProto(clientId, Record.MutableClientId()); - ActorIdToProto(serverId, Record.MutableServerId()); + ActorIdToProto(clientId, Record.MutableClientId()); + ActorIdToProto(serverId, Record.MutableServerId()); } }; @@ -125,34 +125,34 @@ namespace NKikimr { const ui64 TabletId; const NKikimrProto::EReplyStatus Status; - const TActorId ClientId; - const TActorId ServerId; + const TActorId ClientId; + const TActorId ServerId; const bool Leader; const bool Dead; }; struct TEvServerConnected : public TEventLocal<TEvServerConnected, EvServerConnected> { - TEvServerConnected(ui64 tabletId, const TActorId& clientId, const TActorId& serverId) + TEvServerConnected(ui64 tabletId, const TActorId& clientId, const TActorId& serverId) : TabletId(tabletId) , ClientId(clientId) , ServerId(serverId) {} const ui64 TabletId; - const TActorId ClientId; - const TActorId ServerId; + const TActorId ClientId; + const TActorId ServerId; }; struct TEvClientDestroyed : public TEventLocal<TEvClientDestroyed, EvClientDestroyed> { - TEvClientDestroyed(ui64 tabletId, const TActorId& clientId, const TActorId& serverId) + TEvClientDestroyed(ui64 tabletId, const TActorId& clientId, const TActorId& serverId) : TabletId(tabletId) , ClientId(clientId) , ServerId(serverId) {} const ui64 TabletId; - const TActorId ClientId; - const TActorId ServerId; + const TActorId ClientId; + const TActorId ServerId; }; struct TEvClientShuttingDown : public TEventLocal<TEvClientShuttingDown, EvClientShuttingDown> { @@ -170,27 +170,27 @@ namespace NKikimr { }; struct TEvServerDisconnected : public TEventLocal<TEvServerDisconnected, EvServerDisconnected> { - TEvServerDisconnected(ui64 tabletId, const TActorId& clientId, const TActorId& serverId) + TEvServerDisconnected(ui64 tabletId, const TActorId& clientId, const TActorId& serverId) : TabletId(tabletId) , ClientId(clientId) , ServerId(serverId) {} const ui64 TabletId; - const TActorId ClientId; - const TActorId ServerId; + const TActorId ClientId; + const TActorId ServerId; }; struct TEvServerDestroyed : public TEventLocal<TEvServerDestroyed, EvServerDestroyed> { - TEvServerDestroyed(ui64 tabletId, const TActorId& clientId, const TActorId& serverId) + TEvServerDestroyed(ui64 tabletId, const TActorId& clientId, const TActorId& serverId) : TabletId(tabletId) , ClientId(clientId) , ServerId(serverId) {} const ui64 TabletId; - const TActorId ClientId; - const TActorId ServerId; + const TActorId ClientId; + const TActorId ServerId; }; struct TEvActivate : public TEventLocal<TEvActivate, EvActivate> { @@ -202,8 +202,8 @@ namespace NKikimr { {} const ui64 TabletId; - const TActorId OwnerId; - const TActorId RecipientId; + const TActorId OwnerId; + const TActorId RecipientId; const bool Leader; }; @@ -281,7 +281,7 @@ namespace NKikimr { // Creates and activated server, returns serverId. // Created server will forward messages to the specified recipent. - virtual TActorId Accept(TEvTabletPipe::TEvConnect::TPtr &ev, + virtual TActorId Accept(TEvTabletPipe::TEvConnect::TPtr &ev, TActorIdentity owner, TActorId recipient, bool leader = true) = 0; // Rejects connect with an error. @@ -295,7 +295,7 @@ namespace NKikimr { // Creates an inactive server, returns server Id. // Owner of context is not captured at this time. - virtual TActorId Enqueue(TEvTabletPipe::TEvConnect::TPtr &ev, TActorIdentity owner) = 0; + virtual TActorId Enqueue(TEvTabletPipe::TEvConnect::TPtr &ev, TActorIdentity owner) = 0; // Activates all inactive servers, created by Enqueue. // All activated servers will forward messages to the specified recipent. @@ -362,26 +362,26 @@ namespace NKikimr { }; // Returns client actor. - IActor* CreateClient(const TActorId& owner, ui64 tabletId, const TClientConfig& config = TClientConfig()); + IActor* CreateClient(const TActorId& owner, ui64 tabletId, const TClientConfig& config = TClientConfig()); // Sends data via client actor. // Payload will be delivered as a event to the owner of server. // Sender field in that event will be taken from the passed context. // Recipient field in that event will be filled by serverId. - void SendData(const TActorContext& ctx, const TActorId& clientId, IEventBase* payload, ui64 cookie = 0); - void SendData(const TActorContext& ctx, const TActorId& clientId, ui32 eventType, TIntrusivePtr<TEventSerializedData> buffer, ui64 cookie = 0); - void SendData(TActorId self, TActorId clientId, IEventBase* payload, ui64 cookie = 0); + void SendData(const TActorContext& ctx, const TActorId& clientId, IEventBase* payload, ui64 cookie = 0); + void SendData(const TActorContext& ctx, const TActorId& clientId, ui32 eventType, TIntrusivePtr<TEventSerializedData> buffer, ui64 cookie = 0); + void SendData(TActorId self, TActorId clientId, IEventBase* payload, ui64 cookie = 0); void SendData(TActorId self, TActorId clientId, THolder<IEventBase>&& payload, ui64 cookie = 0); void SendDataWithSeqNo(TActorId self, TActorId clientId, IEventBase* payload, ui64 seqNo, ui64 cookie = 0); - void SendData(TActorId self, TActorId clientId, ui32 eventType, TIntrusivePtr<TEventSerializedData> buffer, ui64 cookie = 0); + void SendData(TActorId self, TActorId clientId, ui32 eventType, TIntrusivePtr<TEventSerializedData> buffer, ui64 cookie = 0); // Shutdown client actor. - void ShutdownClient(const TActorContext& ctx, const TActorId& clientId); + void ShutdownClient(const TActorContext& ctx, const TActorId& clientId); // Destroys client actor. - void CloseClient(const TActorContext& ctx, const TActorId& clientId); - void CloseClient(TActorIdentity self, TActorId clientId); - void CloseAndForgetClient(TActorIdentity self, TActorId &clientId); + void CloseClient(const TActorContext& ctx, const TActorId& clientId); + void CloseClient(TActorIdentity self, TActorId clientId); + void CloseAndForgetClient(TActorIdentity self, TActorId &clientId); // Returns server actor in inactive state. IActor* CreateServer(ui64 tabletId, const TActorId& clientId, const TActorId& interconnectSession, ui32 features, ui64 connectCookie); @@ -390,7 +390,7 @@ namespace NKikimr { void ActivateServer(ui64 tabletId, TActorId serverId, TActorIdentity owner, TActorId recipientId, bool leader); // Destroys server actor. - void CloseServer(TActorIdentity owner, TActorId serverId); + void CloseServer(TActorIdentity owner, TActorId serverId); // Returns a controller for handling connects. IConnectAcceptor* CreateConnectAcceptor(ui64 tabletId); diff --git a/ydb/core/base/tablet_resolver.h b/ydb/core/base/tablet_resolver.h index 6d0bf728d4a..24744cef478 100644 --- a/ydb/core/base/tablet_resolver.h +++ b/ydb/core/base/tablet_resolver.h @@ -137,9 +137,9 @@ struct TEvTabletResolver { struct TEvTabletProblem : public TEventLocal<TEvTabletProblem, EvTabletProblem> { const ui64 TabletID; - const TActorId TabletActor; + const TActorId TabletActor; - TEvTabletProblem(ui64 tabletId, const TActorId &tabletActor) + TEvTabletProblem(ui64 tabletId, const TActorId &tabletActor) : TabletID(tabletId) , TabletActor(tabletActor) {} @@ -174,8 +174,8 @@ struct TEvTabletResolver { const NKikimrProto::EReplyStatus Status; ui64 TabletID; - TActorId TabletActor; - TActorId Tablet; + TActorId TabletActor; + TActorId Tablet; ui64 CacheEpoch; TEvForwardResult(NKikimrProto::EReplyStatus status, ui64 tabletId) @@ -214,6 +214,6 @@ struct TTabletResolverConfig : public TThrRefBase { }; IActor* CreateTabletResolver(const TIntrusivePtr<TTabletResolverConfig> &config); -TActorId MakeTabletResolverID(); +TActorId MakeTabletResolverID(); } diff --git a/ydb/core/base/tablet_status_checker.cpp b/ydb/core/base/tablet_status_checker.cpp index 2910d0b1a47..bfb51b8e852 100644 --- a/ydb/core/base/tablet_status_checker.cpp +++ b/ydb/core/base/tablet_status_checker.cpp @@ -9,7 +9,7 @@ namespace NKikimr { class TTabletStatusCheckRequest : public TActorBootstrapped<TTabletStatusCheckRequest> { private: - const TActorId ReplyTo; + const TActorId ReplyTo; TIntrusiveConstPtr<TTabletStorageInfo> Info; ui32 RequestsLeft; TVector<ui32> LightYellowMoveGroups; @@ -38,7 +38,7 @@ public: return NKikimrServices::TActivity::TABLET_ACTOR; } - TTabletStatusCheckRequest(const TActorId &replyTo, TTabletStorageInfo *info) + TTabletStatusCheckRequest(const TActorId &replyTo, TTabletStorageInfo *info) : ReplyTo(replyTo) , Info(info) , RequestsLeft(0) @@ -64,7 +64,7 @@ public: } }; -IActor* CreateTabletDSChecker(const TActorId &replyTo, TTabletStorageInfo *info) { +IActor* CreateTabletDSChecker(const TActorId &replyTo, TTabletStorageInfo *info) { return new TTabletStatusCheckRequest(replyTo, info); } diff --git a/ydb/core/base/tabletid.h b/ydb/core/base/tabletid.h index be0a5fbd39e..cd0a7cb58f0 100644 --- a/ydb/core/base/tabletid.h +++ b/ydb/core/base/tabletid.h @@ -87,11 +87,11 @@ namespace NKikimr { // TODO: think about encoding scheme for sibling group hive - inline TActorId MakeStateStorageProxyID(ui64 stateStorageGroup) { + inline TActorId MakeStateStorageProxyID(ui64 stateStorageGroup) { Y_VERIFY_DEBUG(stateStorageGroup < (1ull << 8ull)); char x[12] = { 's', 't', 's', 'p', 'r', 'o', 'x', 'y' }; x[8] = (char)stateStorageGroup; - return TActorId(0, TStringBuf(x, 12)); + return TActorId(0, TStringBuf(x, 12)); } } diff --git a/ydb/core/base/ticket_parser.h b/ydb/core/base/ticket_parser.h index c31b3124591..62004b3a89d 100644 --- a/ydb/core/base/ticket_parser.h +++ b/ydb/core/base/ticket_parser.h @@ -152,9 +152,9 @@ namespace NKikimr { }; }; - inline NActors::TActorId MakeTicketParserID() { + inline NActors::TActorId MakeTicketParserID() { const char name[12] = "ticketparse"; - return NActors::TActorId(0, TStringBuf(name, 12)); + return NActors::TActorId(0, TStringBuf(name, 12)); } } diff --git a/ydb/core/base/tracing.h b/ydb/core/base/tracing.h index 021d786cf74..40420e45414 100644 --- a/ydb/core/base/tracing.h +++ b/ydb/core/base/tracing.h @@ -1,7 +1,7 @@ #pragma once #include "defs.h" #include "traceid.h" -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <library/cpp/threading/light_rw_lock/lightrwlock.h> namespace NKikimr { diff --git a/ydb/core/base/user_registry.h b/ydb/core/base/user_registry.h index 5ed1abf9726..74484c3d06c 100644 --- a/ydb/core/base/user_registry.h +++ b/ydb/core/base/user_registry.h @@ -39,9 +39,9 @@ namespace NKikimr { }; }; - inline NActors::TActorId MakeUserRegistryID() { + inline NActors::TActorId MakeUserRegistryID() { const char name[12] = "userregistr"; - return NActors::TActorId(0, TStringBuf(name, 12)); + return NActors::TActorId(0, TStringBuf(name, 12)); } IActor* CreateUserRegistry(const TString& query); diff --git a/ydb/core/blobstorage/backpressure/event.cpp b/ydb/core/blobstorage/backpressure/event.cpp index 9156a767fa5..cc04981615a 100644 --- a/ydb/core/blobstorage/backpressure/event.cpp +++ b/ydb/core/blobstorage/backpressure/event.cpp @@ -25,7 +25,7 @@ IEventBase *TEventHolder::MakeErrorReply(NKikimrProto::EReplyStatus status, cons return Apply(callback); } -void TEventHolder::SendToVDisk(const TActorContext& ctx, const TActorId& remoteVDisk, ui64 queueCookie, ui64 msgId, +void TEventHolder::SendToVDisk(const TActorContext& ctx, const TActorId& remoteVDisk, ui64 queueCookie, ui64 msgId, ui64 sequenceId, bool sendMeCostSettings, NWilson::TTraceId traceId, const NBackpressure::TQueueClientId& clientId, const THPTimer& processingTimer) { // check that we are not discarded yet diff --git a/ydb/core/blobstorage/backpressure/event.h b/ydb/core/blobstorage/backpressure/event.h index b5c6815b407..7b8ea72d4c2 100644 --- a/ydb/core/blobstorage/backpressure/event.h +++ b/ydb/core/blobstorage/backpressure/event.h @@ -33,7 +33,7 @@ inline NLWTrace::TOrbit MoveOrbit<TEvBlobStorage::TEvVPut::TPtr>(TEvBlobStorage: class TEventHolder { ui32 Type; ui32 ByteSize; - TActorId Sender; + TActorId Sender; ui64 Cookie; ui32 InterconnectChannel; mutable NLWTrace::TOrbit Orbit; @@ -122,7 +122,7 @@ public: return Type; } - const TActorId& GetSender() const { + const TActorId& GetSender() const { return Sender; } @@ -134,7 +134,7 @@ public: return Orbit; } - void SendToVDisk(const TActorContext& ctx, const TActorId& remoteVDisk, ui64 queueCookie, ui64 msgId, ui64 sequenceId, + void SendToVDisk(const TActorContext& ctx, const TActorId& remoteVDisk, ui64 queueCookie, ui64 msgId, ui64 sequenceId, bool sendMeCostSettings, NWilson::TTraceId traceId, const NBackpressure::TQueueClientId& clientId, const THPTimer& processingTimer); diff --git a/ydb/core/blobstorage/backpressure/queue.cpp b/ydb/core/blobstorage/backpressure/queue.cpp index 4b2f9f00822..c573b7b16b4 100644 --- a/ydb/core/blobstorage/backpressure/queue.cpp +++ b/ydb/core/blobstorage/backpressure/queue.cpp @@ -115,7 +115,7 @@ void TBlobStorageQueue::SetItemQueue(TItem& item, EItemQueue newQueue) { } } -void TBlobStorageQueue::SendToVDisk(const TActorContext& ctx, const TActorId& remoteVDisk, IActor *actor) { +void TBlobStorageQueue::SendToVDisk(const TActorContext& ctx, const TActorId& remoteVDisk, IActor *actor) { const TInstant now = ctx.Now(); const bool sendMeCostSettings = now >= CostSettingsUpdate; @@ -220,7 +220,7 @@ bool TBlobStorageQueue::Expecting(ui64 msgId, ui64 sequenceId) const { return InFlightLookup.count(std::make_pair(sequenceId, msgId)); } -bool TBlobStorageQueue::OnResponse(ui64 msgId, ui64 sequenceId, ui64 cookie, TActorId *outSender, ui64 *outCookie, +bool TBlobStorageQueue::OnResponse(ui64 msgId, ui64 sequenceId, ui64 cookie, TActorId *outSender, ui64 *outCookie, TDuration *processingTime) { const auto lookupIt = InFlightLookup.find(std::make_pair(sequenceId, msgId)); Y_VERIFY(lookupIt != InFlightLookup.end()); @@ -328,7 +328,7 @@ TBlobStorageQueue::TItemList::iterator TBlobStorageQueue::EraseItem(TItemList& q return nextIter; } -void TBlobStorageQueue::Prune(const TActorId& sender) { +void TBlobStorageQueue::Prune(const TActorId& sender) { TSenderMap::TIterator it = SenderToItems.LowerBound(sender); while (it != SenderToItems.End()) { TItem& item = static_cast<TItem&>(*it++); diff --git a/ydb/core/blobstorage/backpressure/queue.h b/ydb/core/blobstorage/backpressure/queue.h index 182e77ca5ed..209346cc2db 100644 --- a/ydb/core/blobstorage/backpressure/queue.h +++ b/ydb/core/blobstorage/backpressure/queue.h @@ -32,8 +32,8 @@ class TBlobStorageQueue { }; template<typename TDerived> - struct TSenderNode : public TRbTreeItem<TSenderNode<TDerived>, TCompare<TActorId>> { - const TActorId& GetKey() const { + struct TSenderNode : public TRbTreeItem<TSenderNode<TDerived>, TCompare<TActorId>> { + const TActorId& GetKey() const { return static_cast<const TDerived&>(*this).Event.GetSender(); } }; @@ -102,7 +102,7 @@ class TBlobStorageQueue { {} }; - using TSenderMap = TRbTree<TSenderNode<TItem>, TCompare<TActorId>>; + using TSenderMap = TRbTree<TSenderNode<TItem>, TCompare<TActorId>>; TQueues Queues; TSenderMap SenderToItems; @@ -190,11 +190,11 @@ public: void SetItemQueue(TItem& item, EItemQueue newQueue); - void SendToVDisk(const TActorContext& ctx, const TActorId& remoteVDisk, IActor *actor); + void SendToVDisk(const TActorContext& ctx, const TActorId& remoteVDisk, IActor *actor); void ReplyWithError(TItem& item, NKikimrProto::EReplyStatus status, const TString& errorReason, const TActorContext& ctx); bool Expecting(ui64 msgId, ui64 sequenceId) const; - bool OnResponse(ui64 msgId, ui64 sequenceId, ui64 cookie, TActorId *outSender, ui64 *outCookie, TDuration *processingTime); + bool OnResponse(ui64 msgId, ui64 sequenceId, ui64 cookie, TActorId *outSender, ui64 *outCookie, TDuration *processingTime); void Unwind(ui64 failedMsgId, ui64 failedSequenceId, ui64 expectedMsgId, ui64 expectedSequenceId); @@ -231,7 +231,7 @@ public: } TItemList::iterator EraseItem(TItemList& queue, TItemList::iterator it); - void Prune(const TActorId& sender); + void Prune(const TActorId& sender); TMaybe<TDuration> GetWorstRequestProcessingTime() const; }; diff --git a/ydb/core/blobstorage/backpressure/queue_backpressure_client.cpp b/ydb/core/blobstorage/backpressure/queue_backpressure_client.cpp index 71e024f8dd8..200b85f616d 100644 --- a/ydb/core/blobstorage/backpressure/queue_backpressure_client.cpp +++ b/ydb/core/blobstorage/backpressure/queue_backpressure_client.cpp @@ -32,16 +32,16 @@ class TVDiskBackpressureClientActor : public TActorBootstrapped<TVDiskBackpressu const TString QueueName; const NMonitoring::TDynamicCounterPtr Counters; TBlobStorageQueue Queue; - TActorId BlobStorageProxy; + TActorId BlobStorageProxy; const TVDiskIdShort VDiskIdShort; - TActorId RemoteVDisk; + TActorId RemoteVDisk; TVDiskID VDiskId; NKikimrBlobStorage::EVDiskQueueId QueueId; const TDuration QueueWatchdogTimeout; ui64 CheckReadinessCookie = 1; TIntrusivePtr<NBackpressure::TFlowRecord> FlowRecord; const ui32 InterconnectChannel; - TActorId SessionId; + TActorId SessionId; std::optional<NKikimrBlobStorage::TGroupInfo> RecentGroup; TIntrusivePtr<TBlobStorageGroupInfo> Info; TBlobStorageGroupType GType; @@ -90,7 +90,7 @@ public: Y_VERIFY(Info); } - void Bootstrap(const TActorId& parent, const TActorContext& ctx) { + void Bootstrap(const TActorId& parent, const TActorContext& ctx) { ApplyGroupInfo(*std::exchange(Info, nullptr)); QLOG_INFO_S("BSQ01", "starting parent# " << parent); InitCounters(); @@ -397,7 +397,7 @@ private: // reset watchdog back to default interval -- we've got successful response ResetWatchdogTimer(ctx.Now()); - TActorId sender; + TActorId sender; ui64 cookie = 0; TDuration processingTime; bool isOk = Queue.OnResponse(msgId, sequenceId, ev->Cookie, &sender, &cookie, &processingTime); @@ -588,7 +588,7 @@ private: //////////////////////////////////////////////////////////////////////// struct TStatusRequestItem { - TActorId Sender; + TActorId Sender; ui64 Cookie; }; @@ -642,7 +642,7 @@ private: Pump(ctx); } - void SendStatusErrorResponse(const TActorId& sender, ui64 cookie, NKikimrProto::EReplyStatus status, + void SendStatusErrorResponse(const TActorId& sender, ui64 cookie, NKikimrProto::EReplyStatus status, const TActorContext& ctx) { QLOG_DEBUG_S("BSQ34", "Status# " << status << " VDiskId# " << VDiskId diff --git a/ydb/core/blobstorage/backpressure/queue_backpressure_client_ut.cpp b/ydb/core/blobstorage/backpressure/queue_backpressure_client_ut.cpp index d0a2a7c528a..af14a13f2d0 100644 --- a/ydb/core/blobstorage/backpressure/queue_backpressure_client_ut.cpp +++ b/ydb/core/blobstorage/backpressure/queue_backpressure_client_ut.cpp @@ -1,6 +1,6 @@ #include "queue_backpressure_client.h" #include <library/cpp/testing/unittest/registar.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <library/cpp/actors/core/actorsystem.h> #include <library/cpp/actors/core/executor_pool_basic.h> #include <library/cpp/actors/core/executor_pool_io.h> @@ -21,12 +21,12 @@ public: using TFilterFunc = std::function<bool (IEventHandle&, const TActorContext&)>; private: - TActorId QueueId; - TActorId VDiskId; + TActorId QueueId; + TActorId VDiskId; TFilterFunc FilterFunc; public: - TFilterActor(const TActorId& queueId, const TActorId& vdiskId, TFilterFunc&& filterFunc) + TFilterActor(const TActorId& queueId, const TActorId& vdiskId, TFilterFunc&& filterFunc) : QueueId(queueId) , VDiskId(vdiskId) , FilterFunc(filterFunc) @@ -71,12 +71,12 @@ class TProxyActor : public TActorBootstrapped<TProxyActor> { const TIntrusivePtr<TBlobStorageGroupInfo> Info; const TVDiskID VDiskId; const TIntrusivePtr<NMonitoring::TDynamicCounters> Counters; - const TActorId QueueActorId; + const TActorId QueueActorId; public: TProxyActor(TIntrusivePtr<TBlobStorageGroupInfo> info, const TVDiskID& vdiskId, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, - const TActorId& queueActorId) + const TActorId& queueActorId) : Info(std::move(info)) , VDiskId(vdiskId) , Counters(std::move(counters)) @@ -129,13 +129,13 @@ class TQueueTestRuntime { ui32 ChunkSize; ui64 PDiskGuid; NPDisk::TKey PDiskKey; - TActorId PDiskId; + TActorId PDiskId; std::unique_ptr<TAppData> AppData; TVDiskID VDiskId; - TActorId VDiskActorId; - TActorId QueueActorId; - TActorId FilterActorId; - TActorId ProxyActorId; + TActorId VDiskActorId; + TActorId QueueActorId; + TActorId FilterActorId; + TActorId ProxyActorId; TProxyActor *Proxy; TIntrusivePtr<NPDisk::TSectorMap> SectorMap; @@ -175,12 +175,12 @@ public: //////////////////////////////////////////////////////////////////////////////////////////////////////////////// // BlobStorage group info - FilterActorId = TActorId{1, "filter"}; - QueueActorId = TActorId{1, "queue"}; - ProxyActorId = TActorId{1, "proxy"}; - VDiskActorId = TActorId{1, "vdisk"}; + FilterActorId = TActorId{1, "filter"}; + QueueActorId = TActorId{1, "queue"}; + ProxyActorId = TActorId{1, "proxy"}; + VDiskActorId = TActorId{1, "vdisk"}; - TVector<TActorId> actorIds{ + TVector<TActorId> actorIds{ {FilterActorId} // actor id for our single VDisk that will be used by queue }; @@ -217,7 +217,7 @@ public: //////////////////////////////////////////////////////////////////////////////////////////////////////////////// // LOGGER - NActors::TActorId loggerActorId{1, "logger"}; + NActors::TActorId loggerActorId{1, "logger"}; TIntrusivePtr<NActors::NLog::TSettings> logSettings{new NActors::NLog::TSettings{loggerActorId, NKikimrServices::LOGGER, NActors::NLog::PRI_ERROR, NActors::NLog::PRI_ERROR, 0}}; logSettings->Append( @@ -268,11 +268,11 @@ public: ActorSystem->Send(ev.release()); } - const TActorId& GetProxyActorId() const { + const TActorId& GetProxyActorId() const { return ProxyActorId; } - const TActorId& GetQueueActorId() const { + const TActorId& GetQueueActorId() const { return QueueActorId; } diff --git a/ydb/core/blobstorage/backpressure/queue_backpressure_server.h b/ydb/core/blobstorage/backpressure/queue_backpressure_server.h index 95980a1c80a..c1f8a7e3952 100644 --- a/ydb/core/blobstorage/backpressure/queue_backpressure_server.h +++ b/ydb/core/blobstorage/backpressure/queue_backpressure_server.h @@ -3,7 +3,7 @@ #include "queue_backpressure_common.h" #include <ydb/core/blobstorage/groupinfo/blobstorage_groupinfo.h> #include <ydb/core/base/blobstorage.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> namespace NKikimr { @@ -96,7 +96,7 @@ namespace NKikimr { TStatPtr GlobalStatPtr; public: const TClientId ClientId; - const TActorId ActorId; + const TActorId ActorId; private: TMessageId ExpectedMsgId; ui64 Cost; @@ -143,7 +143,7 @@ namespace NKikimr { } public: - TWindow(TStatPtr globalStatPtr, const TClientId &clientId, const TActorId& actorId, + TWindow(TStatPtr globalStatPtr, const TClientId &clientId, const TActorId& actorId, ui64 percentThreshold, ui64 defLowWatermark, ui64 defHighWatermark, ui64 costChangeUntilFrozen, ui64 costChangeUntilDeath, TDuration windowTimeout, TInstant now) : GlobalStatPtr(globalStatPtr) @@ -413,7 +413,7 @@ namespace NKikimr { } } - TFeedback Push(const TClientId& clientId, const TActorId& actorId, const TMessageId &msgId, ui64 cost, TInstant now) { + TFeedback Push(const TClientId& clientId, const TActorId& actorId, const TMessageId &msgId, ui64 cost, TInstant now) { Y_VERIFY(actorId); ClearRecalculateCache(); diff --git a/ydb/core/blobstorage/backpressure/unisched.cpp b/ydb/core/blobstorage/backpressure/unisched.cpp index 10d99176617..0a0190fa1c3 100644 --- a/ydb/core/blobstorage/backpressure/unisched.cpp +++ b/ydb/core/blobstorage/backpressure/unisched.cpp @@ -14,10 +14,10 @@ namespace NKikimr { public: struct TEvRegister : TEventLocal<TEvRegister, EvRegister> { - const TActorId ActorId; + const TActorId ActorId; TIntrusivePtr<NBackpressure::TFlowRecord> FlowRecord; - TEvRegister(const TActorId& actorId, TIntrusivePtr<NBackpressure::TFlowRecord> flowRecord) + TEvRegister(const TActorId& actorId, TIntrusivePtr<NBackpressure::TFlowRecord> flowRecord) : ActorId(actorId) , FlowRecord(std::move(flowRecord)) {} @@ -26,7 +26,7 @@ namespace NKikimr { private: class TSubschedulerActor : public TActorBootstrapped<TSubschedulerActor> { const TDuration Interval; - THashMap<TActorId, TIntrusivePtr<NBackpressure::TFlowRecord>> Subscribers; + THashMap<TActorId, TIntrusivePtr<NBackpressure::TFlowRecord>> Subscribers; public: TSubschedulerActor(TDuration interval) @@ -73,10 +73,10 @@ namespace NKikimr { private: struct TSubschedulerInfo { - TActorId ActorId; + TActorId ActorId; size_t NumSubscribers = 0; - TSubschedulerInfo(const TActorId& actorId) + TSubschedulerInfo(const TActorId& actorId) : ActorId(actorId) {} @@ -89,11 +89,11 @@ namespace NKikimr { private: const size_t MaxActorsPerSubscheduler = 1024; const TDuration Interval; - THashMap<TActorId, TActorId> SubscriberToSubscheduler; // subscriber to subscheduler actor id map + THashMap<TActorId, TActorId> SubscriberToSubscheduler; // subscriber to subscheduler actor id map using TOrderedSubschedulerSet = TSet<TSubschedulerInfo>; TOrderedSubschedulerSet PartialSubschedulers; TOrderedSubschedulerSet FullSubschedulers; - THashMap<TActorId, TOrderedSubschedulerSet::iterator> SubschedulerToOrderedSetPosition; + THashMap<TActorId, TOrderedSubschedulerSet::iterator> SubschedulerToOrderedSetPosition; public: TUniversalSchedulerActor(const TDuration interval = TDuration::Seconds(1)) @@ -108,7 +108,7 @@ namespace NKikimr { } void RegisterSubscriber(const TActorId& actorId, std::unique_ptr<TEvRegister> msg, const TActorContext& ctx) { - TActorId subschedulerActorId; + TActorId subschedulerActorId; // first, check if we have partially filled subscheduler actor -- if so, find the most filled one and stuff // it with one more subscriber @@ -147,7 +147,7 @@ namespace NKikimr { Y_VERIFY(it != SubscriberToSubscheduler.end()); // erase it from the map, remembering its actor id - const TActorId subschedulerActorId = it->second; + const TActorId subschedulerActorId = it->second; SubscriberToSubscheduler.erase(it); // send unregister message to the actor @@ -191,7 +191,7 @@ namespace NKikimr { return new TUniversalSchedulerActor; } - bool RegisterActorInUniversalScheduler(const TActorId& actorId, TIntrusivePtr<NBackpressure::TFlowRecord> flowRecord, + bool RegisterActorInUniversalScheduler(const TActorId& actorId, TIntrusivePtr<NBackpressure::TFlowRecord> flowRecord, TActorSystem *actorSystem) { return actorSystem->Send(MakeUniversalSchedulerActorId(), new TUniversalSchedulerActor::TEvRegister(actorId, std::move(flowRecord))); diff --git a/ydb/core/blobstorage/backpressure/unisched.h b/ydb/core/blobstorage/backpressure/unisched.h index f05cd1d4c8f..9331b43e70e 100644 --- a/ydb/core/blobstorage/backpressure/unisched.h +++ b/ydb/core/blobstorage/backpressure/unisched.h @@ -8,14 +8,14 @@ namespace NKikimr { - inline TActorId MakeUniversalSchedulerActorId() { + inline TActorId MakeUniversalSchedulerActorId() { char name[12] = {'U', 'n', 'i', 'S', 'c', 'h', 'e', 'd', 'A', 'c', 't', 'r'}; - return TActorId(0, TStringBuf(name, sizeof(name))); + return TActorId(0, TStringBuf(name, sizeof(name))); } IActor *CreateUniversalSchedulerActor(); - bool RegisterActorInUniversalScheduler(const TActorId& actorId, TIntrusivePtr<NBackpressure::TFlowRecord> flowRecord, + bool RegisterActorInUniversalScheduler(const TActorId& actorId, TIntrusivePtr<NBackpressure::TFlowRecord> flowRecord, TActorSystem *actorSystem); } // NKikimr diff --git a/ydb/core/blobstorage/backpressure/ut_client/backpressure_ut.cpp b/ydb/core/blobstorage/backpressure/ut_client/backpressure_ut.cpp index a874ac764cc..1a0760c6cf9 100644 --- a/ydb/core/blobstorage/backpressure/ut_client/backpressure_ut.cpp +++ b/ydb/core/blobstorage/backpressure/ut_client/backpressure_ut.cpp @@ -13,7 +13,7 @@ Y_UNIT_TEST_SUITE(Backpressure) { const TVDiskID vdiskId(0, 1, 0, 0, 0); TActorId vdiskActorId = runtime.Register(new TSkeletonFrontMockActor, TActorId(), 0, std::nullopt, 1); - std::vector<TActorId> clients; + std::vector<TActorId> clients; ui64 clientId = 1; const TInstant simDuration = TInstant::Hours(6); @@ -77,7 +77,7 @@ Y_UNIT_TEST_SUITE(Backpressure) { } // terminate loader actors and the disk - for (const TActorId& actorId : std::exchange(clients, {})) { + for (const TActorId& actorId : std::exchange(clients, {})) { runtime.Send(new IEventHandle(TEvents::TSystem::Poison, 0, actorId, {}, {}, 0), 1); } runtime.Send(new IEventHandle(TEvents::TSystem::Poison, 0, vdiskActorId, {}, {}, 0), 1); diff --git a/ydb/core/blobstorage/backpressure/ut_client/loader.h b/ydb/core/blobstorage/backpressure/ut_client/loader.h index e8d027c46ec..2372d31d6c5 100644 --- a/ydb/core/blobstorage/backpressure/ut_client/loader.h +++ b/ydb/core/blobstorage/backpressure/ut_client/loader.h @@ -12,11 +12,11 @@ class TLoaderActor : public TActorBootstrapped<TLoaderActor> { const NBackpressure::TQueueClientId ClientId; const TVDiskID VDiskId; - const TActorId VDiskActorId; + const TActorId VDiskActorId; TIntrusivePtr<NMonitoring::TDynamicCounters> Counters; TBSProxyContextPtr BSProxyCtx; TIntrusivePtr<NBackpressure::TFlowRecord> FlowRecord; - TActorId QueueId; + TActorId QueueId; ui32 InFlightRemain = 16; bool Ready = false; ui32 BlobIdx = 1; @@ -24,7 +24,7 @@ class TLoaderActor : public TActorBootstrapped<TLoaderActor> { std::deque<TLogoBlobID> RequestQ; public: - TLoaderActor(NBackpressure::TQueueClientId clientId, TVDiskID vdiskId, TActorId vdiskActorId) + TLoaderActor(NBackpressure::TQueueClientId clientId, TVDiskID vdiskId, TActorId vdiskActorId) : ClientId(std::move(clientId)) , VDiskId(std::move(vdiskId)) , VDiskActorId(std::move(vdiskActorId)) diff --git a/ydb/core/blobstorage/backpressure/ut_client/skeleton_front_mock.h b/ydb/core/blobstorage/backpressure/ut_client/skeleton_front_mock.h index 6c41247bee7..290c529ea56 100644 --- a/ydb/core/blobstorage/backpressure/ut_client/skeleton_front_mock.h +++ b/ydb/core/blobstorage/backpressure/ut_client/skeleton_front_mock.h @@ -11,7 +11,7 @@ class TSkeletonFrontMockActor : public TActorBootstrapped<TSkeletonFrontMockActo struct TEvProcessQueue : TEventLocal<TEvProcessQueue, EvProcessQueue> {}; bool Ready = false; - std::set<TActorId> Notify; + std::set<TActorId> Notify; std::optional<TCostModel> CostModel; NBackpressure::TQueueBackpressure<NBackpressure::TQueueClientId> Server; @@ -20,7 +20,7 @@ class TSkeletonFrontMockActor : public TActorBootstrapped<TSkeletonFrontMockActo NBackpressure::TMessageId MessageId; ui64 Cost; std::unique_ptr<IEventHandle> Result; - TActorId Sender; + TActorId Sender; }; std::deque<TOperation> Operations; @@ -41,7 +41,7 @@ public: void HandleWakeup() { LOG_DEBUG(*TlsActivationContext, NActorsServices::TEST, "HandleWakeup"); Ready = true; - for (const TActorId& id : std::exchange(Notify, {})) { + for (const TActorId& id : std::exchange(Notify, {})) { Send(id, new TEvBlobStorage::TEvVReadyNotify); } } diff --git a/ydb/core/blobstorage/base/common_latency_hist_bounds.h b/ydb/core/blobstorage/base/common_latency_hist_bounds.h index 29611c1d975..1f44e119197 100644 --- a/ydb/core/blobstorage/base/common_latency_hist_bounds.h +++ b/ydb/core/blobstorage/base/common_latency_hist_bounds.h @@ -3,7 +3,7 @@ #include "defs.h" #include <ydb/core/base/blobstorage.h> -#include <library/cpp/monlib/metrics/histogram_snapshot.h> +#include <library/cpp/monlib/metrics/histogram_snapshot.h> namespace NKikimr { diff --git a/ydb/core/blobstorage/base/html.cpp b/ydb/core/blobstorage/base/html.cpp index 48c27a1fdf5..6658530a949 100644 --- a/ydb/core/blobstorage/base/html.cpp +++ b/ydb/core/blobstorage/base/html.cpp @@ -1,6 +1,6 @@ #include "html.h" #include <library/cpp/actors/core/mon.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> namespace NKikimr { diff --git a/ydb/core/blobstorage/base/transparent.h b/ydb/core/blobstorage/base/transparent.h index bf7c27de539..2590ec7d9cf 100644 --- a/ydb/core/blobstorage/base/transparent.h +++ b/ydb/core/blobstorage/base/transparent.h @@ -5,7 +5,7 @@ #include <util/generic/ptr.h> #include <util/generic/buffer.h> #include <util/memory/pool.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <ydb/core/blobstorage/vdisk/common/memusage.h> #include <ydb/core/blobstorage/base/ptr.h> diff --git a/ydb/core/blobstorage/base/utility.h b/ydb/core/blobstorage/base/utility.h index 22b7003fcc4..afb519dcb86 100644 --- a/ydb/core/blobstorage/base/utility.h +++ b/ydb/core/blobstorage/base/utility.h @@ -14,9 +14,9 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// // RunInBatchPool //////////////////////////////////////////////////////////////////////////// - inline TActorId RunInBatchPool(const TActorContext &ctx, IActor *actor) { + inline TActorId RunInBatchPool(const TActorContext &ctx, IActor *actor) { ui32 poolId = AppData(ctx)->BatchPoolId; - TActorId actorID = ctx.Register(actor, TMailboxType::HTSwap, poolId); + TActorId actorID = ctx.Register(actor, TMailboxType::HTSwap, poolId); return actorID; } @@ -79,9 +79,9 @@ namespace NKikimr { class TChildrenSweeper { protected: unsigned UndertakerCounter; - TActorId UndertakerNotifyId; + TActorId UndertakerNotifyId; - void SetupUndertakerCounter(const TActorContext &ctx, const TActorId notifyId, unsigned eventsToWait) { + void SetupUndertakerCounter(const TActorContext &ctx, const TActorId notifyId, unsigned eventsToWait) { UndertakerCounter = eventsToWait; UndertakerNotifyId = notifyId; UndertakerCheckToDie(ctx); @@ -131,7 +131,7 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// class TActorNotify : public IActorNotify { public: - TActorNotify(TActorSystem *as, TActorId notifyId) + TActorNotify(TActorSystem *as, TActorId notifyId) : ActorSystem(as) , NotifyId(notifyId) {} @@ -144,7 +144,7 @@ namespace NKikimr { private: TActorSystem *ActorSystem = nullptr; - TActorId NotifyId; + TActorId NotifyId; }; } // NKikimr diff --git a/ydb/core/blobstorage/base/vdisk_sync_common.h b/ydb/core/blobstorage/base/vdisk_sync_common.h index 413a56a87ba..9e0d2cf21de 100644 --- a/ydb/core/blobstorage/base/vdisk_sync_common.h +++ b/ydb/core/blobstorage/base/vdisk_sync_common.h @@ -7,7 +7,7 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////////////////// // dictionary used to determine BS_QUEUE actor id for a specific VDisk inside group // (identified by stable TVDiskIdShort) - using TQueueActorMap = THashMap<TVDiskIdShort, TActorId>; + using TQueueActorMap = THashMap<TVDiskIdShort, TActorId>; using TQueueActorMapPtr = std::shared_ptr<TQueueActorMap>; } // NKikimr diff --git a/ydb/core/blobstorage/base/wilson_events.h b/ydb/core/blobstorage/base/wilson_events.h index c91a4f5b7f0..7ed95284632 100644 --- a/ydb/core/blobstorage/base/wilson_events.h +++ b/ydb/core/blobstorage/base/wilson_events.h @@ -13,7 +13,7 @@ #define DECLARE_ACTOR_EVENT(EVENT_TYPE, ...) \ DECLARE_WILSON_EVENT(EVENT_TYPE, \ (::NKikimrServices::TActivity::EType, ActivityType), \ - (::NActors::TActorId, ActorId), \ + (::NActors::TActorId, ActorId), \ ##__VA_ARGS__ \ ) diff --git a/ydb/core/blobstorage/crypto/chacha.cpp b/ydb/core/blobstorage/crypto/chacha.cpp index 59241499b54..8945982887e 100644 --- a/ydb/core/blobstorage/crypto/chacha.cpp +++ b/ydb/core/blobstorage/crypto/chacha.cpp @@ -1,68 +1,68 @@ -/* -chacha-merged.c version 20080118 -D. J. Bernstein -Public domain. -*/ - -#include "chacha.h" +/* +chacha-merged.c version 20080118 +D. J. Bernstein +Public domain. +*/ + +#include "chacha.h" #include "secured_block.h" - -#include <util/system/yassert.h> - -#define U32C(v) (v##U) -#define U32V(v) ((ui32)(v) & U32C(0xFFFFFFFF)) - -#define ROTL32(v, n) \ - (U32V((v) << (n)) | ((v) >> (32 - (n)))) - -#define U8TO32_LITTLE(p) (*(ui32*)(p)) -#define U32TO8_LITTLE(p, v) (*(ui32*)(p) = (v)) - -#define ROTATE(v,c) (ROTL32(v,c)) -#define XOR(v,w) ((v) ^ (w)) -#define PLUS(v,w) (U32V((v) + (w))) -#define PLUSONE(v) (PLUS((v),1)) - -#define QUARTERROUND(a,b,c,d) \ - a = PLUS(a,b); d = ROTATE(XOR(d,a),16); \ - c = PLUS(c,d); b = ROTATE(XOR(b,c),12); \ - a = PLUS(a,b); d = ROTATE(XOR(d,a), 8); \ - c = PLUS(c,d); b = ROTATE(XOR(b,c), 7); - - -static const char sigma[] = "expand 32-byte k"; -static const char tau[] = "expand 16-byte k"; - + +#include <util/system/yassert.h> + +#define U32C(v) (v##U) +#define U32V(v) ((ui32)(v) & U32C(0xFFFFFFFF)) + +#define ROTL32(v, n) \ + (U32V((v) << (n)) | ((v) >> (32 - (n)))) + +#define U8TO32_LITTLE(p) (*(ui32*)(p)) +#define U32TO8_LITTLE(p, v) (*(ui32*)(p) = (v)) + +#define ROTATE(v,c) (ROTL32(v,c)) +#define XOR(v,w) ((v) ^ (w)) +#define PLUS(v,w) (U32V((v) + (w))) +#define PLUSONE(v) (PLUS((v),1)) + +#define QUARTERROUND(a,b,c,d) \ + a = PLUS(a,b); d = ROTATE(XOR(d,a),16); \ + c = PLUS(c,d); b = ROTATE(XOR(b,c),12); \ + a = PLUS(a,b); d = ROTATE(XOR(d,a), 8); \ + c = PLUS(c,d); b = ROTATE(XOR(b,c), 7); + + +static const char sigma[] = "expand 32-byte k"; +static const char tau[] = "expand 16-byte k"; + #if (!defined(_win_) && !defined(_arm64_)) -constexpr size_t ChaCha::KEY_SIZE; -constexpr size_t ChaCha::BLOCK_SIZE; +constexpr size_t ChaCha::KEY_SIZE; +constexpr size_t ChaCha::BLOCK_SIZE; #endif - -void ChaCha::SetKey(const ui8* key, size_t size) -{ - const char *constants; + +void ChaCha::SetKey(const ui8* key, size_t size) +{ + const char *constants; Y_ASSERT((size == 16 || size == 32) && "key must be 16 or 32 bytes long"); - - state_[4] = U8TO32_LITTLE(key + 0); - state_[5] = U8TO32_LITTLE(key + 4); - state_[6] = U8TO32_LITTLE(key + 8); - state_[7] = U8TO32_LITTLE(key + 12); - if (size == 32) { /* recommended */ - key += 16; - constants = sigma; - } else { /* size == 128 */ - constants = tau; - } - state_[8] = U8TO32_LITTLE(key + 0); - state_[9] = U8TO32_LITTLE(key + 4); - state_[10] = U8TO32_LITTLE(key + 8); - state_[11] = U8TO32_LITTLE(key + 12); - state_[0] = U8TO32_LITTLE(constants + 0); - state_[1] = U8TO32_LITTLE(constants + 4); - state_[2] = U8TO32_LITTLE(constants + 8); - state_[3] = U8TO32_LITTLE(constants + 12); -} - + + state_[4] = U8TO32_LITTLE(key + 0); + state_[5] = U8TO32_LITTLE(key + 4); + state_[6] = U8TO32_LITTLE(key + 8); + state_[7] = U8TO32_LITTLE(key + 12); + if (size == 32) { /* recommended */ + key += 16; + constants = sigma; + } else { /* size == 128 */ + constants = tau; + } + state_[8] = U8TO32_LITTLE(key + 0); + state_[9] = U8TO32_LITTLE(key + 4); + state_[10] = U8TO32_LITTLE(key + 8); + state_[11] = U8TO32_LITTLE(key + 12); + state_[0] = U8TO32_LITTLE(constants + 0); + state_[1] = U8TO32_LITTLE(constants + 4); + state_[2] = U8TO32_LITTLE(constants + 8); + state_[3] = U8TO32_LITTLE(constants + 12); +} + void ChaCha::SetIV(const ui8* iv, const ui8* blockIdx) { state_[12] = U8TO32_LITTLE(blockIdx + 0); @@ -71,149 +71,149 @@ void ChaCha::SetIV(const ui8* iv, const ui8* blockIdx) state_[15] = U8TO32_LITTLE(iv + 4); } -void ChaCha::SetIV(const ui8* iv) -{ - state_[12] = 0; - state_[13] = 0; - state_[14] = U8TO32_LITTLE(iv + 0); - state_[15] = U8TO32_LITTLE(iv + 4); -} - -void ChaCha::Encipher(const ui8* plaintext, ui8* ciphertext, size_t size) -{ - ui32 x0, x1, x2, x3, x4, x5, x6, x7, x8, x9, x10, x11, x12, x13, x14, x15; - ui32 j0, j1, j2, j3, j4, j5, j6, j7, j8, j9, j10, j11, j12, j13, j14, j15; - ui8* ctarget = nullptr; - ui8 tmp[64]; - size_t i; - - if (!size) return; - - j0 = state_[0]; - j1 = state_[1]; - j2 = state_[2]; - j3 = state_[3]; - j4 = state_[4]; - j5 = state_[5]; - j6 = state_[6]; - j7 = state_[7]; - j8 = state_[8]; - j9 = state_[9]; - j10 = state_[10]; - j11 = state_[11]; - j12 = state_[12]; - j13 = state_[13]; - j14 = state_[14]; - j15 = state_[15]; - - for (;;) { - if (size < 64) { - for (i = 0; i < size; ++i) tmp[i] = plaintext[i]; - plaintext = tmp; - ctarget = ciphertext; - ciphertext = tmp; - } - x0 = j0; - x1 = j1; - x2 = j2; - x3 = j3; - x4 = j4; - x5 = j5; - x6 = j6; - x7 = j7; - x8 = j8; - x9 = j9; - x10 = j10; - x11 = j11; - x12 = j12; - x13 = j13; - x14 = j14; - x15 = j15; - for (i = rounds_; i > 0; i -= 2) { - QUARTERROUND( x0, x4, x8,x12) - QUARTERROUND( x1, x5, x9,x13) - QUARTERROUND( x2, x6,x10,x14) - QUARTERROUND( x3, x7,x11,x15) - QUARTERROUND( x0, x5,x10,x15) - QUARTERROUND( x1, x6,x11,x12) - QUARTERROUND( x2, x7, x8,x13) - QUARTERROUND( x3, x4, x9,x14) - } - x0 = PLUS(x0,j0); - x1 = PLUS(x1,j1); - x2 = PLUS(x2,j2); - x3 = PLUS(x3,j3); - x4 = PLUS(x4,j4); - x5 = PLUS(x5,j5); - x6 = PLUS(x6,j6); - x7 = PLUS(x7,j7); - x8 = PLUS(x8,j8); - x9 = PLUS(x9,j9); - x10 = PLUS(x10,j10); - x11 = PLUS(x11,j11); - x12 = PLUS(x12,j12); - x13 = PLUS(x13,j13); - x14 = PLUS(x14,j14); - x15 = PLUS(x15,j15); - - x0 = XOR(x0,U8TO32_LITTLE(plaintext + 0)); - x1 = XOR(x1,U8TO32_LITTLE(plaintext + 4)); - x2 = XOR(x2,U8TO32_LITTLE(plaintext + 8)); - x3 = XOR(x3,U8TO32_LITTLE(plaintext + 12)); - x4 = XOR(x4,U8TO32_LITTLE(plaintext + 16)); - x5 = XOR(x5,U8TO32_LITTLE(plaintext + 20)); - x6 = XOR(x6,U8TO32_LITTLE(plaintext + 24)); - x7 = XOR(x7,U8TO32_LITTLE(plaintext + 28)); - x8 = XOR(x8,U8TO32_LITTLE(plaintext + 32)); - x9 = XOR(x9,U8TO32_LITTLE(plaintext + 36)); - x10 = XOR(x10,U8TO32_LITTLE(plaintext + 40)); - x11 = XOR(x11,U8TO32_LITTLE(plaintext + 44)); - x12 = XOR(x12,U8TO32_LITTLE(plaintext + 48)); - x13 = XOR(x13,U8TO32_LITTLE(plaintext + 52)); - x14 = XOR(x14,U8TO32_LITTLE(plaintext + 56)); - x15 = XOR(x15,U8TO32_LITTLE(plaintext + 60)); - - j12 = PLUSONE(j12); - if (!j12) { - j13 = PLUSONE(j13); - /* stopping at 2^70 bytes per nonce is user's responsibility */ - } - - U32TO8_LITTLE(ciphertext + 0,x0); - U32TO8_LITTLE(ciphertext + 4,x1); - U32TO8_LITTLE(ciphertext + 8,x2); - U32TO8_LITTLE(ciphertext + 12,x3); - U32TO8_LITTLE(ciphertext + 16,x4); - U32TO8_LITTLE(ciphertext + 20,x5); - U32TO8_LITTLE(ciphertext + 24,x6); - U32TO8_LITTLE(ciphertext + 28,x7); - U32TO8_LITTLE(ciphertext + 32,x8); - U32TO8_LITTLE(ciphertext + 36,x9); - U32TO8_LITTLE(ciphertext + 40,x10); - U32TO8_LITTLE(ciphertext + 44,x11); - U32TO8_LITTLE(ciphertext + 48,x12); - U32TO8_LITTLE(ciphertext + 52,x13); - U32TO8_LITTLE(ciphertext + 56,x14); - U32TO8_LITTLE(ciphertext + 60,x15); - - if (size <= 64) { - if (size < 64) { - for (i = 0; i < size; ++i) ctarget[i] = ciphertext[i]; - } - state_[12] = j12; - state_[13] = j13; - return; - } - size -= 64; - ciphertext += 64; - plaintext += 64; - } -} - -void ChaCha::Decipher(const ui8* ciphertext, ui8* plaintext, size_t size) -{ - Encipher(ciphertext, plaintext, size); -} +void ChaCha::SetIV(const ui8* iv) +{ + state_[12] = 0; + state_[13] = 0; + state_[14] = U8TO32_LITTLE(iv + 0); + state_[15] = U8TO32_LITTLE(iv + 4); +} + +void ChaCha::Encipher(const ui8* plaintext, ui8* ciphertext, size_t size) +{ + ui32 x0, x1, x2, x3, x4, x5, x6, x7, x8, x9, x10, x11, x12, x13, x14, x15; + ui32 j0, j1, j2, j3, j4, j5, j6, j7, j8, j9, j10, j11, j12, j13, j14, j15; + ui8* ctarget = nullptr; + ui8 tmp[64]; + size_t i; + + if (!size) return; + + j0 = state_[0]; + j1 = state_[1]; + j2 = state_[2]; + j3 = state_[3]; + j4 = state_[4]; + j5 = state_[5]; + j6 = state_[6]; + j7 = state_[7]; + j8 = state_[8]; + j9 = state_[9]; + j10 = state_[10]; + j11 = state_[11]; + j12 = state_[12]; + j13 = state_[13]; + j14 = state_[14]; + j15 = state_[15]; + + for (;;) { + if (size < 64) { + for (i = 0; i < size; ++i) tmp[i] = plaintext[i]; + plaintext = tmp; + ctarget = ciphertext; + ciphertext = tmp; + } + x0 = j0; + x1 = j1; + x2 = j2; + x3 = j3; + x4 = j4; + x5 = j5; + x6 = j6; + x7 = j7; + x8 = j8; + x9 = j9; + x10 = j10; + x11 = j11; + x12 = j12; + x13 = j13; + x14 = j14; + x15 = j15; + for (i = rounds_; i > 0; i -= 2) { + QUARTERROUND( x0, x4, x8,x12) + QUARTERROUND( x1, x5, x9,x13) + QUARTERROUND( x2, x6,x10,x14) + QUARTERROUND( x3, x7,x11,x15) + QUARTERROUND( x0, x5,x10,x15) + QUARTERROUND( x1, x6,x11,x12) + QUARTERROUND( x2, x7, x8,x13) + QUARTERROUND( x3, x4, x9,x14) + } + x0 = PLUS(x0,j0); + x1 = PLUS(x1,j1); + x2 = PLUS(x2,j2); + x3 = PLUS(x3,j3); + x4 = PLUS(x4,j4); + x5 = PLUS(x5,j5); + x6 = PLUS(x6,j6); + x7 = PLUS(x7,j7); + x8 = PLUS(x8,j8); + x9 = PLUS(x9,j9); + x10 = PLUS(x10,j10); + x11 = PLUS(x11,j11); + x12 = PLUS(x12,j12); + x13 = PLUS(x13,j13); + x14 = PLUS(x14,j14); + x15 = PLUS(x15,j15); + + x0 = XOR(x0,U8TO32_LITTLE(plaintext + 0)); + x1 = XOR(x1,U8TO32_LITTLE(plaintext + 4)); + x2 = XOR(x2,U8TO32_LITTLE(plaintext + 8)); + x3 = XOR(x3,U8TO32_LITTLE(plaintext + 12)); + x4 = XOR(x4,U8TO32_LITTLE(plaintext + 16)); + x5 = XOR(x5,U8TO32_LITTLE(plaintext + 20)); + x6 = XOR(x6,U8TO32_LITTLE(plaintext + 24)); + x7 = XOR(x7,U8TO32_LITTLE(plaintext + 28)); + x8 = XOR(x8,U8TO32_LITTLE(plaintext + 32)); + x9 = XOR(x9,U8TO32_LITTLE(plaintext + 36)); + x10 = XOR(x10,U8TO32_LITTLE(plaintext + 40)); + x11 = XOR(x11,U8TO32_LITTLE(plaintext + 44)); + x12 = XOR(x12,U8TO32_LITTLE(plaintext + 48)); + x13 = XOR(x13,U8TO32_LITTLE(plaintext + 52)); + x14 = XOR(x14,U8TO32_LITTLE(plaintext + 56)); + x15 = XOR(x15,U8TO32_LITTLE(plaintext + 60)); + + j12 = PLUSONE(j12); + if (!j12) { + j13 = PLUSONE(j13); + /* stopping at 2^70 bytes per nonce is user's responsibility */ + } + + U32TO8_LITTLE(ciphertext + 0,x0); + U32TO8_LITTLE(ciphertext + 4,x1); + U32TO8_LITTLE(ciphertext + 8,x2); + U32TO8_LITTLE(ciphertext + 12,x3); + U32TO8_LITTLE(ciphertext + 16,x4); + U32TO8_LITTLE(ciphertext + 20,x5); + U32TO8_LITTLE(ciphertext + 24,x6); + U32TO8_LITTLE(ciphertext + 28,x7); + U32TO8_LITTLE(ciphertext + 32,x8); + U32TO8_LITTLE(ciphertext + 36,x9); + U32TO8_LITTLE(ciphertext + 40,x10); + U32TO8_LITTLE(ciphertext + 44,x11); + U32TO8_LITTLE(ciphertext + 48,x12); + U32TO8_LITTLE(ciphertext + 52,x13); + U32TO8_LITTLE(ciphertext + 56,x14); + U32TO8_LITTLE(ciphertext + 60,x15); + + if (size <= 64) { + if (size < 64) { + for (i = 0; i < size; ++i) ctarget[i] = ciphertext[i]; + } + state_[12] = j12; + state_[13] = j13; + return; + } + size -= 64; + ciphertext += 64; + plaintext += 64; + } +} + +void ChaCha::Decipher(const ui8* ciphertext, ui8* plaintext, size_t size) +{ + Encipher(ciphertext, plaintext, size); +} ChaCha::~ChaCha() { SecureWipeBuffer((ui8*)&state_[4], 32); diff --git a/ydb/core/blobstorage/crypto/chacha.h b/ydb/core/blobstorage/crypto/chacha.h index c937e653d98..e527f96444c 100644 --- a/ydb/core/blobstorage/crypto/chacha.h +++ b/ydb/core/blobstorage/crypto/chacha.h @@ -1,34 +1,34 @@ -#pragma once - -#include <util/system/types.h> - -#define CHACHA_MINKEYLEN 16 -#define CHACHA_NONCELEN 8 -#define CHACHA_CTRLEN 8 -#define CHACHA_STATELEN (CHACHA_NONCELEN + CHACHA_CTRLEN) -#define CHACHA_BLOCKLEN 64 -#define CHACHA_ROUNDS 8 - - -class ChaCha -{ -public: - using NonceType = ui64; - static constexpr size_t KEY_SIZE = 32; - static constexpr size_t BLOCK_SIZE = 64; - -public: - ChaCha(ui8 rounds = CHACHA_ROUNDS): rounds_(rounds) {} - - void SetKey(const ui8* key, size_t size); +#pragma once + +#include <util/system/types.h> + +#define CHACHA_MINKEYLEN 16 +#define CHACHA_NONCELEN 8 +#define CHACHA_CTRLEN 8 +#define CHACHA_STATELEN (CHACHA_NONCELEN + CHACHA_CTRLEN) +#define CHACHA_BLOCKLEN 64 +#define CHACHA_ROUNDS 8 + + +class ChaCha +{ +public: + using NonceType = ui64; + static constexpr size_t KEY_SIZE = 32; + static constexpr size_t BLOCK_SIZE = 64; + +public: + ChaCha(ui8 rounds = CHACHA_ROUNDS): rounds_(rounds) {} + + void SetKey(const ui8* key, size_t size); void SetIV(const ui8* iv, const ui8* blockIdx); - void SetIV(const ui8* iv); - void Encipher(const ui8* plaintext, ui8* ciphertext, size_t size); - void Decipher(const ui8* ciphertext, ui8* plaintext, size_t size); - + void SetIV(const ui8* iv); + void Encipher(const ui8* plaintext, ui8* ciphertext, size_t size); + void Decipher(const ui8* ciphertext, ui8* plaintext, size_t size); + ~ChaCha(); -private: - ui32 state_[16]; - ui8 rounds_; -}; +private: + ui32 state_[16]; + ui8 rounds_; +}; diff --git a/ydb/core/blobstorage/crypto/chacha_ut.cpp b/ydb/core/blobstorage/crypto/chacha_ut.cpp index 83d04a9ea3d..18db0f8df64 100644 --- a/ydb/core/blobstorage/crypto/chacha_ut.cpp +++ b/ydb/core/blobstorage/crypto/chacha_ut.cpp @@ -1,114 +1,114 @@ -#include "chacha.h" -#include "secured_block.h" +#include "chacha.h" +#include "secured_block.h" #include <ydb/core/blobstorage/crypto/ut/ut_helpers.h> #include <ydb/core/blobstorage/crypto/ut/chacha_test_vectors.h> - - + + Y_UNIT_TEST_SUITE(TChaCha) -{ - void RunTest(int rounds, const ui8 key[KEY_SIZE], const ui8 iv[IV_SIZE], - const ui8 expected[][DATA_SIZE]) - { - ui8 data[DATA_SIZE] = { 0x00 }; - ui8 buf[DATA_SIZE]; - - ChaCha cipher(rounds); - cipher.SetIV(iv); - cipher.SetKey(key, KEY_SIZE); - - cipher.Encipher(data, buf, sizeof(data)); - UNIT_ASSERT_ARRAYS_EQUAL(buf, expected[0], sizeof(buf)); - - cipher.Encipher(data, buf, sizeof(data)); - UNIT_ASSERT_ARRAYS_EQUAL(buf, expected[1], sizeof(buf)); - } - - // TC1: All zero key and IV. +{ + void RunTest(int rounds, const ui8 key[KEY_SIZE], const ui8 iv[IV_SIZE], + const ui8 expected[][DATA_SIZE]) + { + ui8 data[DATA_SIZE] = { 0x00 }; + ui8 buf[DATA_SIZE]; + + ChaCha cipher(rounds); + cipher.SetIV(iv); + cipher.SetKey(key, KEY_SIZE); + + cipher.Encipher(data, buf, sizeof(data)); + UNIT_ASSERT_ARRAYS_EQUAL(buf, expected[0], sizeof(buf)); + + cipher.Encipher(data, buf, sizeof(data)); + UNIT_ASSERT_ARRAYS_EQUAL(buf, expected[1], sizeof(buf)); + } + + // TC1: All zero key and IV. Y_UNIT_TEST(KeystreamTest1) { - RunTest(8, tc1_key, tc1_iv, tc1_expected_8); - RunTest(12, tc1_key, tc1_iv, tc1_expected_12); - RunTest(20, tc1_key, tc1_iv, tc1_expected_20); - } - - // TC2: Single bit in key set. All zero IV + RunTest(8, tc1_key, tc1_iv, tc1_expected_8); + RunTest(12, tc1_key, tc1_iv, tc1_expected_12); + RunTest(20, tc1_key, tc1_iv, tc1_expected_20); + } + + // TC2: Single bit in key set. All zero IV Y_UNIT_TEST(KeystreamTest2) { - RunTest(8, tc2_key, tc2_iv, tc2_expected_8); - RunTest(12, tc2_key, tc2_iv, tc2_expected_12); - RunTest(20, tc2_key, tc2_iv, tc2_expected_20); - } - - // TC3: Single bit in IV set. All zero key + RunTest(8, tc2_key, tc2_iv, tc2_expected_8); + RunTest(12, tc2_key, tc2_iv, tc2_expected_12); + RunTest(20, tc2_key, tc2_iv, tc2_expected_20); + } + + // TC3: Single bit in IV set. All zero key Y_UNIT_TEST(KeystreamTest3) { - RunTest(8, tc3_key, tc3_iv, tc3_expected_8); - RunTest(12, tc3_key, tc3_iv, tc3_expected_12); - RunTest(20, tc3_key, tc3_iv, tc3_expected_20); - } - - // TC4: All bits in key and IV are set + RunTest(8, tc3_key, tc3_iv, tc3_expected_8); + RunTest(12, tc3_key, tc3_iv, tc3_expected_12); + RunTest(20, tc3_key, tc3_iv, tc3_expected_20); + } + + // TC4: All bits in key and IV are set Y_UNIT_TEST(KeystreamTest4) { - RunTest(8, tc4_key, tc4_iv, tc4_expected_8); - RunTest(12, tc4_key, tc4_iv, tc4_expected_12); - RunTest(20, tc4_key, tc4_iv, tc4_expected_20); - } - - // TC5: Every even bit set in key and IV + RunTest(8, tc4_key, tc4_iv, tc4_expected_8); + RunTest(12, tc4_key, tc4_iv, tc4_expected_12); + RunTest(20, tc4_key, tc4_iv, tc4_expected_20); + } + + // TC5: Every even bit set in key and IV Y_UNIT_TEST(KeystreamTest5) { - RunTest(8, tc5_key, tc5_iv, tc5_expected_8); - RunTest(12, tc5_key, tc5_iv, tc5_expected_12); - RunTest(20, tc5_key, tc5_iv, tc5_expected_20); - } - - // TC6: Every odd bit set in key and IV + RunTest(8, tc5_key, tc5_iv, tc5_expected_8); + RunTest(12, tc5_key, tc5_iv, tc5_expected_12); + RunTest(20, tc5_key, tc5_iv, tc5_expected_20); + } + + // TC6: Every odd bit set in key and IV Y_UNIT_TEST(KeystreamTest6) { - RunTest(8, tc6_key, tc6_iv, tc6_expected_8); - RunTest(12, tc6_key, tc6_iv, tc6_expected_12); - RunTest(20, tc6_key, tc6_iv, tc6_expected_20); - } - - // TC7: Sequence patterns in key and IV + RunTest(8, tc6_key, tc6_iv, tc6_expected_8); + RunTest(12, tc6_key, tc6_iv, tc6_expected_12); + RunTest(20, tc6_key, tc6_iv, tc6_expected_20); + } + + // TC7: Sequence patterns in key and IV Y_UNIT_TEST(KeystreamTest7) { - RunTest(8, tc7_key, tc7_iv, tc7_expected_8); - RunTest(12, tc7_key, tc7_iv, tc7_expected_12); - RunTest(20, tc7_key, tc7_iv, tc7_expected_20); - } - - // TC8: key: 'All your base are belong to us!, IV: 'IETF2013' + RunTest(8, tc7_key, tc7_iv, tc7_expected_8); + RunTest(12, tc7_key, tc7_iv, tc7_expected_12); + RunTest(20, tc7_key, tc7_iv, tc7_expected_20); + } + + // TC8: key: 'All your base are belong to us!, IV: 'IETF2013' Y_UNIT_TEST(KeystreamTest8) { - RunTest(8, tc8_key, tc8_iv, tc8_expected_8); - RunTest(12, tc8_key, tc8_iv, tc8_expected_12); - RunTest(20, tc8_key, tc8_iv, tc8_expected_20); - } - + RunTest(8, tc8_key, tc8_iv, tc8_expected_8); + RunTest(12, tc8_key, tc8_iv, tc8_expected_12); + RunTest(20, tc8_key, tc8_iv, tc8_expected_20); + } + Y_UNIT_TEST(MultiEncipherOneDecipher) { TStringBuf lorem = - "Lorem ipsum dolor sit amet, consectetur adipisicing elit, " - "sed do eiusmod tempor incididunt ut labore et dolore magna " - "aliqua. Ut enim ad minim veniam, quis nostrud exercitation " - "ullamco laboris nisi ut aliquip ex ea commodo consequat. " - "Duis aute irure dolor in reprehenderit in voluptate velit " - "esse cillum dolore eu fugiat nulla pariatur. Excepteur sint " - "occaecat cupidatat non proident, sunt in culpa qui officia " + "Lorem ipsum dolor sit amet, consectetur adipisicing elit, " + "sed do eiusmod tempor incididunt ut labore et dolore magna " + "aliqua. Ut enim ad minim veniam, quis nostrud exercitation " + "ullamco laboris nisi ut aliquip ex ea commodo consequat. " + "Duis aute irure dolor in reprehenderit in voluptate velit " + "esse cillum dolore eu fugiat nulla pariatur. Excepteur sint " + "occaecat cupidatat non proident, sunt in culpa qui officia " "deserunt mollit anim id est laborum."; - + TSecuredBlock<> buf(lorem.data(), lorem.size()); - - ChaCha cipher; - cipher.SetIV(tc8_iv); - cipher.SetKey(tc8_key, KEY_SIZE); - - for (size_t i = 0; i < buf.Size(); ) { - size_t len = Min(ChaCha::BLOCK_SIZE, buf.Size() - i); - cipher.Encipher(buf.Data() + i, buf.Data() + i, len); - i += len; - } - - UNIT_ASSERT_UNEQUAL(buf.AsStringBuf(), lorem); - - cipher.SetIV(tc8_iv); - cipher.Decipher(buf, buf, buf.Size()); - - UNIT_ASSERT_EQUAL(buf.AsStringBuf(), lorem); - } + + ChaCha cipher; + cipher.SetIV(tc8_iv); + cipher.SetKey(tc8_key, KEY_SIZE); + + for (size_t i = 0; i < buf.Size(); ) { + size_t len = Min(ChaCha::BLOCK_SIZE, buf.Size() - i); + cipher.Encipher(buf.Data() + i, buf.Data() + i, len); + i += len; + } + + UNIT_ASSERT_UNEQUAL(buf.AsStringBuf(), lorem); + + cipher.SetIV(tc8_iv); + cipher.Decipher(buf, buf, buf.Size()); + + UNIT_ASSERT_EQUAL(buf.AsStringBuf(), lorem); + } Y_UNIT_TEST(SecondBlock) { TStringBuf plaintext = @@ -135,4 +135,4 @@ Y_UNIT_TEST_SUITE(TChaCha) UNIT_ASSERT_EQUAL(((ui8*)buf)[i], plaintext.data()[i]); } } -} +} diff --git a/ydb/core/blobstorage/crypto/chacha_vec.cpp b/ydb/core/blobstorage/crypto/chacha_vec.cpp index 33e752339b1..870c63e2e43 100644 --- a/ydb/core/blobstorage/crypto/chacha_vec.cpp +++ b/ydb/core/blobstorage/crypto/chacha_vec.cpp @@ -1,103 +1,103 @@ -/* -chacha-merged.c version 20080118 -D. J. Bernstein -Public domain. -*/ - -#include "chacha_vec.h" +/* +chacha-merged.c version 20080118 +D. J. Bernstein +Public domain. +*/ + +#include "chacha_vec.h" #include "secured_block.h" - + #include <util/system/align.h> -#include <util/system/yassert.h> - - -/* Chacha implementation for 16-byte vectors by Ted Krovetz (ted@krovetz.net). - * Assumes 32-bit int, 64-bit long long. Public domain. Modified: 2013.06.19. - * Chacha is an improvement on the stream cipher Salsa, described at - * http://cr.yp.to/papers.html#chacha - */ -#include <string.h> - -/* This implementation is designed for Neon, SSE and AltiVec machines. The - * following specify how to do certain vector operations efficiently on - * each architecture, using intrinsics. - * This implementation supports parallel processing of multiple blocks, - * including potentially using general-purpose registers. - */ +#include <util/system/yassert.h> + + +/* Chacha implementation for 16-byte vectors by Ted Krovetz (ted@krovetz.net). + * Assumes 32-bit int, 64-bit long long. Public domain. Modified: 2013.06.19. + * Chacha is an improvement on the stream cipher Salsa, described at + * http://cr.yp.to/papers.html#chacha + */ +#include <string.h> + +/* This implementation is designed for Neon, SSE and AltiVec machines. The + * following specify how to do certain vector operations efficiently on + * each architecture, using intrinsics. + * This implementation supports parallel processing of multiple blocks, + * including potentially using general-purpose registers. + */ #if (__ARM_NEON__ || defined(_arm64_)) -# include <arm_neon.h> -# define ONE (vec)vsetq_lane_u32(1,vdupq_n_u32(0),0) +# include <arm_neon.h> +# define ONE (vec)vsetq_lane_u32(1,vdupq_n_u32(0),0) # define NONCE(p, bp) (vec)vcombine_u32(vcreate_u32(*(uint64_t *)(bp)), vcreate_u32(*(uint64_t *)(p))) -# define ROTV1(x) (vec)vextq_u32((uint32x4_t)x,(uint32x4_t)x,1) -# define ROTV2(x) (vec)vextq_u32((uint32x4_t)x,(uint32x4_t)x,2) -# define ROTV3(x) (vec)vextq_u32((uint32x4_t)x,(uint32x4_t)x,3) -# define ROTW16(x) (vec)vrev32q_u16((uint16x8_t)x) -# if __clang__ -# define ROTW7(x) (x << ((vec){ 7, 7, 7, 7})) ^ (x >> ((vec){25,25,25,25})) -# define ROTW8(x) (x << ((vec){ 8, 8, 8, 8})) ^ (x >> ((vec){24,24,24,24})) -# define ROTW12(x) (x << ((vec){12,12,12,12})) ^ (x >> ((vec){20,20,20,20})) -# else -# define ROTW7(x) (vec)vsriq_n_u32(vshlq_n_u32((uint32x4_t)x,7),(uint32x4_t)x,25) -# define ROTW8(x) (vec)vsriq_n_u32(vshlq_n_u32((uint32x4_t)x,8),(uint32x4_t)x,24) -# define ROTW12(x) (vec)vsriq_n_u32(vshlq_n_u32((uint32x4_t)x,12),(uint32x4_t)x,20) -# endif -#elif __SSE2__ -# include <emmintrin.h> -# define ONE (vec)_mm_set_epi32(0,0,0,1) +# define ROTV1(x) (vec)vextq_u32((uint32x4_t)x,(uint32x4_t)x,1) +# define ROTV2(x) (vec)vextq_u32((uint32x4_t)x,(uint32x4_t)x,2) +# define ROTV3(x) (vec)vextq_u32((uint32x4_t)x,(uint32x4_t)x,3) +# define ROTW16(x) (vec)vrev32q_u16((uint16x8_t)x) +# if __clang__ +# define ROTW7(x) (x << ((vec){ 7, 7, 7, 7})) ^ (x >> ((vec){25,25,25,25})) +# define ROTW8(x) (x << ((vec){ 8, 8, 8, 8})) ^ (x >> ((vec){24,24,24,24})) +# define ROTW12(x) (x << ((vec){12,12,12,12})) ^ (x >> ((vec){20,20,20,20})) +# else +# define ROTW7(x) (vec)vsriq_n_u32(vshlq_n_u32((uint32x4_t)x,7),(uint32x4_t)x,25) +# define ROTW8(x) (vec)vsriq_n_u32(vshlq_n_u32((uint32x4_t)x,8),(uint32x4_t)x,24) +# define ROTW12(x) (vec)vsriq_n_u32(vshlq_n_u32((uint32x4_t)x,12),(uint32x4_t)x,20) +# endif +#elif __SSE2__ +# include <emmintrin.h> +# define ONE (vec)_mm_set_epi32(0,0,0,1) # define NONCE(p, bp) (vec)(_mm_slli_si128(_mm_loadl_epi64((__m128i *)(p)),8) ^ _mm_loadl_epi64((__m128i *)(bp))) -# define ROTV1(x) (vec)_mm_shuffle_epi32((__m128i)x,_MM_SHUFFLE(0,3,2,1)) -# define ROTV2(x) (vec)_mm_shuffle_epi32((__m128i)x,_MM_SHUFFLE(1,0,3,2)) -# define ROTV3(x) (vec)_mm_shuffle_epi32((__m128i)x,_MM_SHUFFLE(2,1,0,3)) -# define ROTW7(x) (vec)(_mm_slli_epi32((__m128i)x, 7) ^ _mm_srli_epi32((__m128i)x,25)) -# define ROTW12(x) (vec)(_mm_slli_epi32((__m128i)x,12) ^ _mm_srli_epi32((__m128i)x,20)) +# define ROTV1(x) (vec)_mm_shuffle_epi32((__m128i)x,_MM_SHUFFLE(0,3,2,1)) +# define ROTV2(x) (vec)_mm_shuffle_epi32((__m128i)x,_MM_SHUFFLE(1,0,3,2)) +# define ROTV3(x) (vec)_mm_shuffle_epi32((__m128i)x,_MM_SHUFFLE(2,1,0,3)) +# define ROTW7(x) (vec)(_mm_slli_epi32((__m128i)x, 7) ^ _mm_srli_epi32((__m128i)x,25)) +# define ROTW12(x) (vec)(_mm_slli_epi32((__m128i)x,12) ^ _mm_srli_epi32((__m128i)x,20)) # define SHIFT_LEFT(x) (vec)_mm_bslli_si128((__m128i)(x), 8) # define SHIFT_RIGHT(x) (vec)_mm_bsrli_si128((__m128i)(x), 8) # define SET(x0, x1) (vec)_mm_set_epi64x((x0), (x1)) -# if __SSSE3__ -# include <tmmintrin.h> -# define ROTW8(x) (vec)_mm_shuffle_epi8((__m128i)x,_mm_set_epi8(14,13,12,15,10,9,8,11,6,5,4,7,2,1,0,3)) -# define ROTW16(x) (vec)_mm_shuffle_epi8((__m128i)x,_mm_set_epi8(13,12,15,14,9,8,11,10,5,4,7,6,1,0,3,2)) -# else -# define ROTW8(x) (vec)(_mm_slli_epi32((__m128i)x, 8) ^ _mm_srli_epi32((__m128i)x,24)) -# define ROTW16(x) (vec)(_mm_slli_epi32((__m128i)x,16) ^ _mm_srli_epi32((__m128i)x,16)) -# endif -#else +# if __SSSE3__ +# include <tmmintrin.h> +# define ROTW8(x) (vec)_mm_shuffle_epi8((__m128i)x,_mm_set_epi8(14,13,12,15,10,9,8,11,6,5,4,7,2,1,0,3)) +# define ROTW16(x) (vec)_mm_shuffle_epi8((__m128i)x,_mm_set_epi8(13,12,15,14,9,8,11,10,5,4,7,6,1,0,3,2)) +# else +# define ROTW8(x) (vec)(_mm_slli_epi32((__m128i)x, 8) ^ _mm_srli_epi32((__m128i)x,24)) +# define ROTW16(x) (vec)(_mm_slli_epi32((__m128i)x,16) ^ _mm_srli_epi32((__m128i)x,16)) +# endif +#else # error -- Implementation supports only machines with neon or SSE2 -#endif - -#ifndef REVV_BE -# define REVV_BE(x) (x) -#endif - -#ifndef REVW_BE -# define REVW_BE(x) (x) -#endif - -#define DQROUND_VECTORS(a,b,c,d) \ - a += b; d ^= a; d = ROTW16(d); \ - c += d; b ^= c; b = ROTW12(b); \ - a += b; d ^= a; d = ROTW8(d); \ - c += d; b ^= c; b = ROTW7(b); \ - b = ROTV1(b); c = ROTV2(c); d = ROTV3(d); \ - a += b; d ^= a; d = ROTW16(d); \ - c += d; b ^= c; b = ROTW12(b); \ - a += b; d ^= a; d = ROTW8(d); \ - c += d; b ^= c; b = ROTW7(b); \ - b = ROTV3(b); c = ROTV2(c); d = ROTV1(d); - -#define QROUND_WORDS(a,b,c,d) \ - a = a+b; d ^= a; d = d<<16 | d>>16; \ - c = c+d; b ^= c; b = b<<12 | b>>20; \ - a = a+b; d ^= a; d = d<< 8 | d>>24; \ - c = c+d; b ^= c; b = b<< 7 | b>>25; - -#define WRITE_XOR(in, op, d, v0, v1, v2, v3) \ +#endif + +#ifndef REVV_BE +# define REVV_BE(x) (x) +#endif + +#ifndef REVW_BE +# define REVW_BE(x) (x) +#endif + +#define DQROUND_VECTORS(a,b,c,d) \ + a += b; d ^= a; d = ROTW16(d); \ + c += d; b ^= c; b = ROTW12(b); \ + a += b; d ^= a; d = ROTW8(d); \ + c += d; b ^= c; b = ROTW7(b); \ + b = ROTV1(b); c = ROTV2(c); d = ROTV3(d); \ + a += b; d ^= a; d = ROTW16(d); \ + c += d; b ^= c; b = ROTW12(b); \ + a += b; d ^= a; d = ROTW8(d); \ + c += d; b ^= c; b = ROTW7(b); \ + b = ROTV3(b); c = ROTV2(c); d = ROTV1(d); + +#define QROUND_WORDS(a,b,c,d) \ + a = a+b; d ^= a; d = d<<16 | d>>16; \ + c = c+d; b ^= c; b = b<<12 | b>>20; \ + a = a+b; d ^= a; d = d<< 8 | d>>24; \ + c = c+d; b ^= c; b = b<< 7 | b>>25; + +#define WRITE_XOR(in, op, d, v0, v1, v2, v3) \ *(vec *)(op + d + 0) = *(vec *)(in + d + 0) ^ v0; \ *(vec *)(op + d + 4) = *(vec *)(in + d + 4) ^ v1; \ *(vec *)(op + d + 8) = *(vec *)(in + d + 8) ^ v2; \ *(vec *)(op + d + 12) = *(vec *)(in + d + 12) ^ v3; - + template<bool Aligned> Y_FORCE_INLINE void WriteXor(ui32 *op, ui32 *ip, vec v0, vec v1, vec v2, vec v3, @@ -114,7 +114,7 @@ Y_FORCE_INLINE void WriteXor(ui32 *op, ui32 *ip, *(vec *)(op + 12) = i_v[3] ^ v3; } } - + template<bool Aligned, bool IsFirst, bool IsLast> Y_FORCE_INLINE void ReadW(ui32 *ip, vec i_v[4], vec& next_i_v) { if constexpr (Aligned) { @@ -149,56 +149,56 @@ Y_FORCE_INLINE void ReadW(ui32 *ip, vec i_v[4], vec& next_i_v) { } -constexpr size_t ChaChaVec::KEY_SIZE; -constexpr size_t ChaChaVec::BLOCK_SIZE; - +constexpr size_t ChaChaVec::KEY_SIZE; +constexpr size_t ChaChaVec::BLOCK_SIZE; + alignas(16) ui32 chacha_const[] = { - 0x61707865, 0x3320646e, 0x79622d32, 0x6b206574 -}; - -void ChaChaVec::SetKey(const ui8* key, size_t size) -{ + 0x61707865, 0x3320646e, 0x79622d32, 0x6b206574 +}; + +void ChaChaVec::SetKey(const ui8* key, size_t size) +{ Y_ASSERT((size == KEY_SIZE) && "key must be 32 bytes long"); - + alignas(16) ui8 aligned_key[KEY_SIZE]; - memcpy(aligned_key, key, size); - - ui32 *kp; + memcpy(aligned_key, key, size); + + ui32 *kp; #if ( __ARM_NEON__ || __SSE2__ || defined(_arm64_)) - kp = (ui32*)aligned_key; - #else + kp = (ui32*)aligned_key; + #else alignas(16) ui32 k[4]; ((vec *)k)[0] = ((vec *)aligned_key)[0]; ((vec *)k)[1] = ((vec *)aligned_key)[1]; - kp = (ui32*)k; - #endif - s0_ = *(vec *)chacha_const; - s1_ = ((vec *)kp)[0]; - s2_ = ((vec *)kp)[1]; + kp = (ui32*)k; + #endif + s0_ = *(vec *)chacha_const; + s1_ = ((vec *)kp)[0]; + s2_ = ((vec *)kp)[1]; SecureWipeBuffer(aligned_key, KEY_SIZE); -} - +} + void ChaChaVec::SetIV(const ui8* iv, const ui8* blockIdx) -{ - ui32 *np; +{ + ui32 *np; ui32 *bp; #if ( __ARM_NEON__ || __SSE2__ || defined(_arm64_)) - np = (ui32*)iv; + np = (ui32*)iv; bp = (ui32*)blockIdx; - #else + #else alignas(16) ui32 nonce[2]; - nonce[0] = REVW_BE(((ui32*)iv)[0]); - nonce[1] = REVW_BE(((ui32*)iv)[1]); - np = (ui32*)nonce; + nonce[0] = REVW_BE(((ui32*)iv)[0]); + nonce[1] = REVW_BE(((ui32*)iv)[1]); + np = (ui32*)nonce; alignas(16) ui32 idx[2]; idx[0] = REVW_BE(((ui32*)blockIdx)[0]); idx[1] = REVW_BE(((ui32*)blockIdx)[1]); bp = (ui32*)idx; - #endif + #endif s3_ = NONCE(np, bp); -} - +} + void ChaChaVec::SetIV(const ui8* iv) { const ui8 zero[8] = {0, 0, 0, 0, 0, 0, 0, 0}; @@ -207,8 +207,8 @@ void ChaChaVec::SetIV(const ui8* iv) { template<bool Aligned> void ChaChaVec::EncipherImpl(const ui8* plaintext, ui8* ciphertext, size_t len) -{ - size_t iters, i; +{ + size_t iters, i; ui32* ip = (ui32*)AlignUp<intptr_t>(intptr_t(plaintext), 16); ui32 *op = (ui32*)ciphertext; @@ -379,139 +379,139 @@ void ChaChaVec::EncipherImpl(const ui8* plaintext, ui8* ciphertext, size_t len) void ChaChaVec::EncipherOld(const ui8* plaintext, ui8* ciphertext, size_t len) { size_t iters, i; - ui32 *op=(ui32*)ciphertext, *ip=(ui32*)plaintext; - - for (iters = 0; iters < len/(CHACHA_BPI * BLOCK_SIZE); iters++) { - vec v0,v1,v2,v3,v4,v5,v6,v7; - v4 = v0 = s0_; v5 = v1 = s1_; v6 = v2 = s2_; v3 = s3_; - v7 = v3 + ONE; - #if CHACHA_VBPI > 2 - vec v8,v9,v10,v11; - v8 = v4; v9 = v5; v10 = v6; - v11 = v7 + ONE; - #endif - #if CHACHA_VBPI > 3 - vec v12,v13,v14,v15; - v12 = v8; v13 = v9; v14 = v10; - v15 = v11 + ONE; - #endif - #if CHACHA_GPR_TOO - ui32* kp = (ui32*)&s1_; - ui32* np = (ui32*)&s3_ + 2; - + ui32 *op=(ui32*)ciphertext, *ip=(ui32*)plaintext; + + for (iters = 0; iters < len/(CHACHA_BPI * BLOCK_SIZE); iters++) { + vec v0,v1,v2,v3,v4,v5,v6,v7; + v4 = v0 = s0_; v5 = v1 = s1_; v6 = v2 = s2_; v3 = s3_; + v7 = v3 + ONE; + #if CHACHA_VBPI > 2 + vec v8,v9,v10,v11; + v8 = v4; v9 = v5; v10 = v6; + v11 = v7 + ONE; + #endif + #if CHACHA_VBPI > 3 + vec v12,v13,v14,v15; + v12 = v8; v13 = v9; v14 = v10; + v15 = v11 + ONE; + #endif + #if CHACHA_GPR_TOO + ui32* kp = (ui32*)&s1_; + ui32* np = (ui32*)&s3_ + 2; + ui32 x0, x1, x2, x3, x4, x5, x6, x7, x8, - x9, x10, x11, x12, x13, x14, x15; - x0 = chacha_const[0]; x1 = chacha_const[1]; - x2 = chacha_const[2]; x3 = chacha_const[3]; - x4 = kp[0]; x5 = kp[1]; x6 = kp[2]; x7 = kp[3]; - x8 = kp[4]; x9 = kp[5]; x10 = kp[6]; x11 = kp[7]; - x12 = CHACHA_BPI*iters+(CHACHA_BPI-1); x13 = 0; x14 = np[0]; x15 = np[1]; - #endif - for (i = rounds_/2; i; i--) { - DQROUND_VECTORS(v0,v1,v2,v3) - DQROUND_VECTORS(v4,v5,v6,v7) - #if CHACHA_VBPI > 2 - DQROUND_VECTORS(v8,v9,v10,v11) - #endif - #if CHACHA_VBPI > 3 - DQROUND_VECTORS(v12,v13,v14,v15) - #endif - #if CHACHA_GPR_TOO - QROUND_WORDS(x0, x4, x8, x12) - QROUND_WORDS(x1, x5, x9, x13) - QROUND_WORDS(x2, x6, x10, x14) - QROUND_WORDS(x3, x7, x11, x15) - QROUND_WORDS(x0, x5, x10, x15) - QROUND_WORDS(x1, x6, x11, x12) - QROUND_WORDS(x2, x7, x8, x13) - QROUND_WORDS(x3, x4, x9, x14) - #endif - } - WRITE_XOR(ip, op, 0, v0+s0_, v1+s1_, v2+s2_, v3+s3_) - s3_ += ONE; - WRITE_XOR(ip, op, 16, v4+s0_, v5+s1_, v6+s2_, v7+s3_) - s3_ += ONE; - #if CHACHA_VBPI > 2 - WRITE_XOR(ip, op, 32, v8+s0_, v9+s1_, v10+s2_, v11+s3_) - s3_ += ONE; - #endif - #if CHACHA_VBPI > 3 - WRITE_XOR(ip, op, 48, v12+s0_, v13+s1_, v14+s2_, v15+s3_) - s3_ += ONE; - #endif - ip += CHACHA_VBPI*16; - op += CHACHA_VBPI*16; - #if CHACHA_GPR_TOO - op[0] = REVW_BE(REVW_BE(ip[0]) ^ (x0 + chacha_const[0])); - op[1] = REVW_BE(REVW_BE(ip[1]) ^ (x1 + chacha_const[1])); - op[2] = REVW_BE(REVW_BE(ip[2]) ^ (x2 + chacha_const[2])); - op[3] = REVW_BE(REVW_BE(ip[3]) ^ (x3 + chacha_const[3])); - op[4] = REVW_BE(REVW_BE(ip[4]) ^ (x4 + kp[0])); - op[5] = REVW_BE(REVW_BE(ip[5]) ^ (x5 + kp[1])); - op[6] = REVW_BE(REVW_BE(ip[6]) ^ (x6 + kp[2])); - op[7] = REVW_BE(REVW_BE(ip[7]) ^ (x7 + kp[3])); - op[8] = REVW_BE(REVW_BE(ip[8]) ^ (x8 + kp[4])); - op[9] = REVW_BE(REVW_BE(ip[9]) ^ (x9 + kp[5])); - op[10] = REVW_BE(REVW_BE(ip[10]) ^ (x10 + kp[6])); - op[11] = REVW_BE(REVW_BE(ip[11]) ^ (x11 + kp[7])); - op[12] = REVW_BE(REVW_BE(ip[12]) ^ (x12 + CHACHA_BPI*iters+(CHACHA_BPI-1))); - op[13] = REVW_BE(REVW_BE(ip[13]) ^ (x13)); - op[14] = REVW_BE(REVW_BE(ip[14]) ^ (x14 + np[0])); - op[15] = REVW_BE(REVW_BE(ip[15]) ^ (x15 + np[1])); - s3_ += ONE; - ip += 16; - op += 16; - #endif - } - for (iters = len % (CHACHA_BPI*BLOCK_SIZE)/BLOCK_SIZE; iters != 0; iters--) { - vec v0 = s0_, v1 = s1_, v2 = s2_, v3 = s3_; - for (i = rounds_/2; i; i--) { - DQROUND_VECTORS(v0,v1,v2,v3) - } - WRITE_XOR(ip, op, 0, v0+s0_, v1+s1_, v2+s2_, v3+s3_) - s3_ += ONE; - ip += 16; - op += 16; - } - len = len % BLOCK_SIZE; - if (len) { + x9, x10, x11, x12, x13, x14, x15; + x0 = chacha_const[0]; x1 = chacha_const[1]; + x2 = chacha_const[2]; x3 = chacha_const[3]; + x4 = kp[0]; x5 = kp[1]; x6 = kp[2]; x7 = kp[3]; + x8 = kp[4]; x9 = kp[5]; x10 = kp[6]; x11 = kp[7]; + x12 = CHACHA_BPI*iters+(CHACHA_BPI-1); x13 = 0; x14 = np[0]; x15 = np[1]; + #endif + for (i = rounds_/2; i; i--) { + DQROUND_VECTORS(v0,v1,v2,v3) + DQROUND_VECTORS(v4,v5,v6,v7) + #if CHACHA_VBPI > 2 + DQROUND_VECTORS(v8,v9,v10,v11) + #endif + #if CHACHA_VBPI > 3 + DQROUND_VECTORS(v12,v13,v14,v15) + #endif + #if CHACHA_GPR_TOO + QROUND_WORDS(x0, x4, x8, x12) + QROUND_WORDS(x1, x5, x9, x13) + QROUND_WORDS(x2, x6, x10, x14) + QROUND_WORDS(x3, x7, x11, x15) + QROUND_WORDS(x0, x5, x10, x15) + QROUND_WORDS(x1, x6, x11, x12) + QROUND_WORDS(x2, x7, x8, x13) + QROUND_WORDS(x3, x4, x9, x14) + #endif + } + WRITE_XOR(ip, op, 0, v0+s0_, v1+s1_, v2+s2_, v3+s3_) + s3_ += ONE; + WRITE_XOR(ip, op, 16, v4+s0_, v5+s1_, v6+s2_, v7+s3_) + s3_ += ONE; + #if CHACHA_VBPI > 2 + WRITE_XOR(ip, op, 32, v8+s0_, v9+s1_, v10+s2_, v11+s3_) + s3_ += ONE; + #endif + #if CHACHA_VBPI > 3 + WRITE_XOR(ip, op, 48, v12+s0_, v13+s1_, v14+s2_, v15+s3_) + s3_ += ONE; + #endif + ip += CHACHA_VBPI*16; + op += CHACHA_VBPI*16; + #if CHACHA_GPR_TOO + op[0] = REVW_BE(REVW_BE(ip[0]) ^ (x0 + chacha_const[0])); + op[1] = REVW_BE(REVW_BE(ip[1]) ^ (x1 + chacha_const[1])); + op[2] = REVW_BE(REVW_BE(ip[2]) ^ (x2 + chacha_const[2])); + op[3] = REVW_BE(REVW_BE(ip[3]) ^ (x3 + chacha_const[3])); + op[4] = REVW_BE(REVW_BE(ip[4]) ^ (x4 + kp[0])); + op[5] = REVW_BE(REVW_BE(ip[5]) ^ (x5 + kp[1])); + op[6] = REVW_BE(REVW_BE(ip[6]) ^ (x6 + kp[2])); + op[7] = REVW_BE(REVW_BE(ip[7]) ^ (x7 + kp[3])); + op[8] = REVW_BE(REVW_BE(ip[8]) ^ (x8 + kp[4])); + op[9] = REVW_BE(REVW_BE(ip[9]) ^ (x9 + kp[5])); + op[10] = REVW_BE(REVW_BE(ip[10]) ^ (x10 + kp[6])); + op[11] = REVW_BE(REVW_BE(ip[11]) ^ (x11 + kp[7])); + op[12] = REVW_BE(REVW_BE(ip[12]) ^ (x12 + CHACHA_BPI*iters+(CHACHA_BPI-1))); + op[13] = REVW_BE(REVW_BE(ip[13]) ^ (x13)); + op[14] = REVW_BE(REVW_BE(ip[14]) ^ (x14 + np[0])); + op[15] = REVW_BE(REVW_BE(ip[15]) ^ (x15 + np[1])); + s3_ += ONE; + ip += 16; + op += 16; + #endif + } + for (iters = len % (CHACHA_BPI*BLOCK_SIZE)/BLOCK_SIZE; iters != 0; iters--) { + vec v0 = s0_, v1 = s1_, v2 = s2_, v3 = s3_; + for (i = rounds_/2; i; i--) { + DQROUND_VECTORS(v0,v1,v2,v3) + } + WRITE_XOR(ip, op, 0, v0+s0_, v1+s1_, v2+s2_, v3+s3_) + s3_ += ONE; + ip += 16; + op += 16; + } + len = len % BLOCK_SIZE; + if (len) { alignas(16) char buf[16]; - vec tail; - vec v0, v1, v2, v3; - v0 = s0_; v1 = s1_; v2 = s2_; v3 = s3_; - for (i = rounds_/2; i; i--) { - DQROUND_VECTORS(v0,v1,v2,v3) - } - if (len >= 32) { - *(vec *)(op + 0) = *(vec *)(ip + 0) ^ REVV_BE(v0 + s0_); - *(vec *)(op + 4) = *(vec *)(ip + 4) ^ REVV_BE(v1 + s1_); - if (len >= 48) { - *(vec *)(op + 8) = *(vec *)(ip + 8) ^ REVV_BE(v2 + s2_); - tail = REVV_BE(v3 + s3_); - op += 12; - ip += 12; - len -= 48; - } else { - tail = REVV_BE(v2 + s2_); - op += 8; - ip += 8; - len -= 32; - } - } else if (len >= 16) { - *(vec *)(op + 0) = *(vec *)(ip + 0) ^ REVV_BE(v0 + s0_); - tail = REVV_BE(v1 + s1_); - op += 4; - ip += 4; - len -= 16; - } else tail = REVV_BE(v0 + s0_); - memcpy(buf, ip, len); - void *bp = buf; - *(vec *)bp = tail ^ *(vec *)bp; - memcpy(op, buf, len); + vec tail; + vec v0, v1, v2, v3; + v0 = s0_; v1 = s1_; v2 = s2_; v3 = s3_; + for (i = rounds_/2; i; i--) { + DQROUND_VECTORS(v0,v1,v2,v3) + } + if (len >= 32) { + *(vec *)(op + 0) = *(vec *)(ip + 0) ^ REVV_BE(v0 + s0_); + *(vec *)(op + 4) = *(vec *)(ip + 4) ^ REVV_BE(v1 + s1_); + if (len >= 48) { + *(vec *)(op + 8) = *(vec *)(ip + 8) ^ REVV_BE(v2 + s2_); + tail = REVV_BE(v3 + s3_); + op += 12; + ip += 12; + len -= 48; + } else { + tail = REVV_BE(v2 + s2_); + op += 8; + ip += 8; + len -= 32; + } + } else if (len >= 16) { + *(vec *)(op + 0) = *(vec *)(ip + 0) ^ REVV_BE(v0 + s0_); + tail = REVV_BE(v1 + s1_); + op += 4; + ip += 4; + len -= 16; + } else tail = REVV_BE(v0 + s0_); + memcpy(buf, ip, len); + void *bp = buf; + *(vec *)bp = tail ^ *(vec *)bp; + memcpy(op, buf, len); SecureWipeBuffer(buf, 16); - } -} - + } +} + void ChaChaVec::Encipher(const ui8* plaintext, ui8* ciphertext, size_t len) { const ui32 input_unalignment = intptr_t(plaintext) % 16; @@ -524,10 +524,10 @@ void ChaChaVec::Encipher(const ui8* plaintext, ui8* ciphertext, size_t len) } } -void ChaChaVec::Decipher(const ui8* ciphertext, ui8* plaintext, size_t len) -{ - Encipher(ciphertext, plaintext, len); -} +void ChaChaVec::Decipher(const ui8* ciphertext, ui8* plaintext, size_t len) +{ + Encipher(ciphertext, plaintext, len); +} ChaChaVec::~ChaChaVec() { SecureWipeBuffer((ui8*)&s1_, 16); diff --git a/ydb/core/blobstorage/crypto/chacha_vec.h b/ydb/core/blobstorage/crypto/chacha_vec.h index 6b6f89fe603..7bcaf9dcdf3 100644 --- a/ydb/core/blobstorage/crypto/chacha_vec.h +++ b/ydb/core/blobstorage/crypto/chacha_vec.h @@ -1,55 +1,55 @@ -#pragma once - -#include <util/system/types.h> - +#pragma once + +#include <util/system/types.h> + #if (__ARM_NEON__ || defined(_arm64_)) -# define CHACHA_GPR_TOO 1 -# define CHACHA_VBPI 2 -#elif __ALTIVEC__ -# define CHACHA_GPR_TOO 1 -# define CHACHA_VBPI 3 -#elif __SSE2__ -# define CHACHA_GPR_TOO 0 -# if __clang__ -# define CHACHA_VBPI 4 -# else -# define CHACHA_VBPI 3 -# endif -#endif - -// Blocks computed per loop iteration -#define CHACHA_BPI (CHACHA_VBPI + CHACHA_GPR_TOO) - -// 8 (high speed), 20 (conservative), 12 (middle) -#define CHACHA_ROUNDS 8 - - -// Architecture-neutral way to specify 16-byte vector of ints -using vec = ui32 __attribute__ ((vector_size (16))); - -class ChaChaVec -{ -public: - using NonceType = ui64; - static constexpr size_t KEY_SIZE = 32; - static constexpr size_t BLOCK_SIZE = 64; - -public: - ChaChaVec(ui8 rounds = CHACHA_ROUNDS): rounds_(rounds) {} - - void SetKey(const ui8* key, size_t size); +# define CHACHA_GPR_TOO 1 +# define CHACHA_VBPI 2 +#elif __ALTIVEC__ +# define CHACHA_GPR_TOO 1 +# define CHACHA_VBPI 3 +#elif __SSE2__ +# define CHACHA_GPR_TOO 0 +# if __clang__ +# define CHACHA_VBPI 4 +# else +# define CHACHA_VBPI 3 +# endif +#endif + +// Blocks computed per loop iteration +#define CHACHA_BPI (CHACHA_VBPI + CHACHA_GPR_TOO) + +// 8 (high speed), 20 (conservative), 12 (middle) +#define CHACHA_ROUNDS 8 + + +// Architecture-neutral way to specify 16-byte vector of ints +using vec = ui32 __attribute__ ((vector_size (16))); + +class ChaChaVec +{ +public: + using NonceType = ui64; + static constexpr size_t KEY_SIZE = 32; + static constexpr size_t BLOCK_SIZE = 64; + +public: + ChaChaVec(ui8 rounds = CHACHA_ROUNDS): rounds_(rounds) {} + + void SetKey(const ui8* key, size_t size); void SetIV(const ui8* iv, const ui8* blockIdx); - void SetIV(const ui8* iv); - void Encipher(const ui8* plaintext, ui8* ciphertext, size_t size); + void SetIV(const ui8* iv); + void Encipher(const ui8* plaintext, ui8* ciphertext, size_t size); // Only for tests void EncipherOld(const ui8* plaintext, ui8* ciphertext, size_t size); - void Decipher(const ui8* ciphertext, ui8* plaintext, size_t size); - + void Decipher(const ui8* ciphertext, ui8* plaintext, size_t size); + ~ChaChaVec(); -private: +private: template<bool Aligned> void EncipherImpl(const ui8* plaintext, ui8* ciphertext, size_t len); - vec s0_, s1_, s2_, s3_; - ui8 rounds_; -}; + vec s0_, s1_, s2_, s3_; + ui8 rounds_; +}; diff --git a/ydb/core/blobstorage/crypto/chacha_vec_ut.cpp b/ydb/core/blobstorage/crypto/chacha_vec_ut.cpp index 75ba245c9af..823348e8321 100644 --- a/ydb/core/blobstorage/crypto/chacha_vec_ut.cpp +++ b/ydb/core/blobstorage/crypto/chacha_vec_ut.cpp @@ -1,116 +1,116 @@ #include <util/random/fast.h> -#include "chacha_vec.h" -#include "secured_block.h" +#include "chacha_vec.h" +#include "secured_block.h" #include <ydb/core/blobstorage/crypto/ut/ut_helpers.h> #include <ydb/core/blobstorage/crypto/ut/chacha_test_vectors.h> - - + + Y_UNIT_TEST_SUITE(TChaChaVec) -{ - void RunTest(int rounds, const ui8 key[KEY_SIZE], const ui8 iv[IV_SIZE], - const ui8 expected[][DATA_SIZE]) - { - ui8 data[DATA_SIZE] = { 0x00 }; - ui8 buf[DATA_SIZE]; - - ChaChaVec cipher(rounds); - cipher.SetIV(iv); - cipher.SetKey(key, KEY_SIZE); - - cipher.Encipher(data, buf, sizeof(data)); - UNIT_ASSERT_ARRAYS_EQUAL(buf, expected[0], sizeof(buf)); - - cipher.Encipher(data, buf, sizeof(data)); - UNIT_ASSERT_ARRAYS_EQUAL(buf, expected[1], sizeof(buf)); - } - - // TC1: All zero key and IV. +{ + void RunTest(int rounds, const ui8 key[KEY_SIZE], const ui8 iv[IV_SIZE], + const ui8 expected[][DATA_SIZE]) + { + ui8 data[DATA_SIZE] = { 0x00 }; + ui8 buf[DATA_SIZE]; + + ChaChaVec cipher(rounds); + cipher.SetIV(iv); + cipher.SetKey(key, KEY_SIZE); + + cipher.Encipher(data, buf, sizeof(data)); + UNIT_ASSERT_ARRAYS_EQUAL(buf, expected[0], sizeof(buf)); + + cipher.Encipher(data, buf, sizeof(data)); + UNIT_ASSERT_ARRAYS_EQUAL(buf, expected[1], sizeof(buf)); + } + + // TC1: All zero key and IV. Y_UNIT_TEST(KeystreamTest1) { - RunTest(8, tc1_key, tc1_iv, tc1_expected_8); - RunTest(12, tc1_key, tc1_iv, tc1_expected_12); - RunTest(20, tc1_key, tc1_iv, tc1_expected_20); - } - - // TC2: Single bit in key set. All zero IV + RunTest(8, tc1_key, tc1_iv, tc1_expected_8); + RunTest(12, tc1_key, tc1_iv, tc1_expected_12); + RunTest(20, tc1_key, tc1_iv, tc1_expected_20); + } + + // TC2: Single bit in key set. All zero IV Y_UNIT_TEST(KeystreamTest2) { - RunTest(8, tc2_key, tc2_iv, tc2_expected_8); - RunTest(12, tc2_key, tc2_iv, tc2_expected_12); - RunTest(20, tc2_key, tc2_iv, tc2_expected_20); - } - - // TC3: Single bit in IV set. All zero key + RunTest(8, tc2_key, tc2_iv, tc2_expected_8); + RunTest(12, tc2_key, tc2_iv, tc2_expected_12); + RunTest(20, tc2_key, tc2_iv, tc2_expected_20); + } + + // TC3: Single bit in IV set. All zero key Y_UNIT_TEST(KeystreamTest3) { - RunTest(8, tc3_key, tc3_iv, tc3_expected_8); - RunTest(12, tc3_key, tc3_iv, tc3_expected_12); - RunTest(20, tc3_key, tc3_iv, tc3_expected_20); - } - - // TC4: All bits in key and IV are set + RunTest(8, tc3_key, tc3_iv, tc3_expected_8); + RunTest(12, tc3_key, tc3_iv, tc3_expected_12); + RunTest(20, tc3_key, tc3_iv, tc3_expected_20); + } + + // TC4: All bits in key and IV are set Y_UNIT_TEST(KeystreamTest4) { - RunTest(8, tc4_key, tc4_iv, tc4_expected_8); - RunTest(12, tc4_key, tc4_iv, tc4_expected_12); - RunTest(20, tc4_key, tc4_iv, tc4_expected_20); - } - - // TC5: Every even bit set in key and IV + RunTest(8, tc4_key, tc4_iv, tc4_expected_8); + RunTest(12, tc4_key, tc4_iv, tc4_expected_12); + RunTest(20, tc4_key, tc4_iv, tc4_expected_20); + } + + // TC5: Every even bit set in key and IV Y_UNIT_TEST(KeystreamTest5) { - RunTest(8, tc5_key, tc5_iv, tc5_expected_8); - RunTest(12, tc5_key, tc5_iv, tc5_expected_12); - RunTest(20, tc5_key, tc5_iv, tc5_expected_20); - } - - // TC6: Every odd bit set in key and IV + RunTest(8, tc5_key, tc5_iv, tc5_expected_8); + RunTest(12, tc5_key, tc5_iv, tc5_expected_12); + RunTest(20, tc5_key, tc5_iv, tc5_expected_20); + } + + // TC6: Every odd bit set in key and IV Y_UNIT_TEST(KeystreamTest6) { - RunTest(8, tc6_key, tc6_iv, tc6_expected_8); - RunTest(12, tc6_key, tc6_iv, tc6_expected_12); - RunTest(20, tc6_key, tc6_iv, tc6_expected_20); - } - - // TC7: Sequence patterns in key and IV + RunTest(8, tc6_key, tc6_iv, tc6_expected_8); + RunTest(12, tc6_key, tc6_iv, tc6_expected_12); + RunTest(20, tc6_key, tc6_iv, tc6_expected_20); + } + + // TC7: Sequence patterns in key and IV Y_UNIT_TEST(KeystreamTest7) { - RunTest(8, tc7_key, tc7_iv, tc7_expected_8); - RunTest(12, tc7_key, tc7_iv, tc7_expected_12); - RunTest(20, tc7_key, tc7_iv, tc7_expected_20); - } - - // TC8: key: 'All your base are belong to us!, IV: 'IETF2013' + RunTest(8, tc7_key, tc7_iv, tc7_expected_8); + RunTest(12, tc7_key, tc7_iv, tc7_expected_12); + RunTest(20, tc7_key, tc7_iv, tc7_expected_20); + } + + // TC8: key: 'All your base are belong to us!, IV: 'IETF2013' Y_UNIT_TEST(KeystreamTest8) { - RunTest(8, tc8_key, tc8_iv, tc8_expected_8); - RunTest(12, tc8_key, tc8_iv, tc8_expected_12); - RunTest(20, tc8_key, tc8_iv, tc8_expected_20); - } - + RunTest(8, tc8_key, tc8_iv, tc8_expected_8); + RunTest(12, tc8_key, tc8_iv, tc8_expected_12); + RunTest(20, tc8_key, tc8_iv, tc8_expected_20); + } + Y_UNIT_TEST(MultiEncipherOneDecipher) { TStringBuf lorem = - "Lorem ipsum dolor sit amet, consectetur adipisicing elit, " - "sed do eiusmod tempor incididunt ut labore et dolore magna " - "aliqua. Ut enim ad minim veniam, quis nostrud exercitation " - "ullamco laboris nisi ut aliquip ex ea commodo consequat. " - "Duis aute irure dolor in reprehenderit in voluptate velit " - "esse cillum dolore eu fugiat nulla pariatur. Excepteur sint " - "occaecat cupidatat non proident, sunt in culpa qui officia " + "Lorem ipsum dolor sit amet, consectetur adipisicing elit, " + "sed do eiusmod tempor incididunt ut labore et dolore magna " + "aliqua. Ut enim ad minim veniam, quis nostrud exercitation " + "ullamco laboris nisi ut aliquip ex ea commodo consequat. " + "Duis aute irure dolor in reprehenderit in voluptate velit " + "esse cillum dolore eu fugiat nulla pariatur. Excepteur sint " + "occaecat cupidatat non proident, sunt in culpa qui officia " "deserunt mollit anim id est laborum."; - + TSecuredBlock<> buf(lorem.data(), lorem.size()); - - ChaChaVec cipher; - cipher.SetIV(tc8_iv); - cipher.SetKey(tc8_key, KEY_SIZE); - - for (size_t i = 0; i < buf.Size(); ) { - size_t len = Min(ChaChaVec::BLOCK_SIZE, buf.Size() - i); - cipher.Encipher(buf.Data() + i, buf.Data() + i, len); - i += len; - } - - UNIT_ASSERT_UNEQUAL(buf.AsStringBuf(), lorem); - - cipher.SetIV(tc8_iv); - cipher.Decipher(buf, buf, buf.Size()); - - UNIT_ASSERT_EQUAL(buf.AsStringBuf(), lorem); - } + + ChaChaVec cipher; + cipher.SetIV(tc8_iv); + cipher.SetKey(tc8_key, KEY_SIZE); + + for (size_t i = 0; i < buf.Size(); ) { + size_t len = Min(ChaChaVec::BLOCK_SIZE, buf.Size() - i); + cipher.Encipher(buf.Data() + i, buf.Data() + i, len); + i += len; + } + + UNIT_ASSERT_UNEQUAL(buf.AsStringBuf(), lorem); + + cipher.SetIV(tc8_iv); + cipher.Decipher(buf, buf, buf.Size()); + + UNIT_ASSERT_EQUAL(buf.AsStringBuf(), lorem); + } Y_UNIT_TEST(SecondBlock) { TStringBuf plaintext = @@ -176,4 +176,4 @@ Y_UNIT_TEST_SUITE(TChaChaVec) UNIT_ASSERT_ARRAYS_EQUAL(bufOrig.Data(), bufOutNew.Data(), size); } } -} +} diff --git a/ydb/core/blobstorage/crypto/poly1305.cpp b/ydb/core/blobstorage/crypto/poly1305.cpp index aa6dbb27f42..60ed46cb436 100644 --- a/ydb/core/blobstorage/crypto/poly1305.cpp +++ b/ydb/core/blobstorage/crypto/poly1305.cpp @@ -1,281 +1,281 @@ -/* - * Public Domain poly1305 from Andrew Moon - * Based on poly1305-donna.c, poly1305-donna-32.h and poly1305-donna.h from: - * https://github.com/floodyberry/poly1305-donna - */ - -#include "poly1305.h" - -#include <util/system/yassert.h> - -#include <string.h> - - -struct poly1305_state_st -{ - ui32 r0, r1, r2, r3, r4; - ui32 s1, s2, s3, s4; - ui32 h0, h1, h2, h3, h4; - ui8 buf[16]; - unsigned int buf_used; - ui8 key[16]; -}; - - -static uint32_t U8TO32_LE(const uint8_t *m) { - uint32_t r; - memcpy(&r, m, sizeof(r)); - return r; -} - -static void U32TO8_LE(uint8_t *m, uint32_t v) { memcpy(m, &v, sizeof(v)); } - -inline ui64 mul32x32_64(ui32 a, ui32 b) { return (ui64)a * b; } - -/* poly1305_blocks updates |state| given some amount of input data. This - * function may only be called with a |len| that is not a multiple of 16 at the - * end of the data. Otherwise the input must be buffered into 16 byte blocks. */ -static void poly1305_update(poly1305_state_st *state, const ui8 *in, size_t len) -{ - ui32 t0, t1, t2, t3; - ui64 t[5]; - ui32 b; - ui64 c; - size_t j; - - if (len < 16) { - goto poly1305_donna_atmost15bytes; - } - -poly1305_donna_16bytes: - t0 = U8TO32_LE(in); - t1 = U8TO32_LE(in + 4); - t2 = U8TO32_LE(in + 8); - t3 = U8TO32_LE(in + 12); - - in += 16; - len -= 16; - - state->h0 += t0 & 0x3ffffff; - state->h1 += ((((ui64)t1 << 32) | t0) >> 26) & 0x3ffffff; - state->h2 += ((((ui64)t2 << 32) | t1) >> 20) & 0x3ffffff; - state->h3 += ((((ui64)t3 << 32) | t2) >> 14) & 0x3ffffff; - state->h4 += (t3 >> 8) | (1 << 24); - -poly1305_donna_mul: - t[0] = mul32x32_64(state->h0, state->r0) + mul32x32_64(state->h1, state->s4) + - mul32x32_64(state->h2, state->s3) + mul32x32_64(state->h3, state->s2) + - mul32x32_64(state->h4, state->s1); - t[1] = mul32x32_64(state->h0, state->r1) + mul32x32_64(state->h1, state->r0) + - mul32x32_64(state->h2, state->s4) + mul32x32_64(state->h3, state->s3) + - mul32x32_64(state->h4, state->s2); - t[2] = mul32x32_64(state->h0, state->r2) + mul32x32_64(state->h1, state->r1) + - mul32x32_64(state->h2, state->r0) + mul32x32_64(state->h3, state->s4) + - mul32x32_64(state->h4, state->s3); - t[3] = mul32x32_64(state->h0, state->r3) + mul32x32_64(state->h1, state->r2) + - mul32x32_64(state->h2, state->r1) + mul32x32_64(state->h3, state->r0) + - mul32x32_64(state->h4, state->s4); - t[4] = mul32x32_64(state->h0, state->r4) + mul32x32_64(state->h1, state->r3) + - mul32x32_64(state->h2, state->r2) + mul32x32_64(state->h3, state->r1) + - mul32x32_64(state->h4, state->r0); - - state->h0 = (ui32)t[0] & 0x3ffffff; - c = (t[0] >> 26); - t[1] += c; - state->h1 = (ui32)t[1] & 0x3ffffff; - b = (ui32)(t[1] >> 26); - t[2] += b; - state->h2 = (ui32)t[2] & 0x3ffffff; - b = (ui32)(t[2] >> 26); - t[3] += b; - state->h3 = (ui32)t[3] & 0x3ffffff; - b = (ui32)(t[3] >> 26); - t[4] += b; - state->h4 = (ui32)t[4] & 0x3ffffff; - b = (ui32)(t[4] >> 26); - state->h0 += b * 5; - - if (len >= 16) { - goto poly1305_donna_16bytes; - } - - /* final bytes */ -poly1305_donna_atmost15bytes: - if (!len) { - return; - } - - ui8 mp[16]; - for (j = 0; j < len; j++) { - mp[j] = in[j]; - } - mp[j++] = 1; - for (; j < 16; j++) { - mp[j] = 0; - } - len = 0; - - t0 = U8TO32_LE(&mp[0] + 0); - t1 = U8TO32_LE(mp + 4); - t2 = U8TO32_LE(mp + 8); - t3 = U8TO32_LE(mp + 12); - - state->h0 += t0 & 0x3ffffff; - state->h1 += ((((ui64)t1 << 32) | t0) >> 26) & 0x3ffffff; - state->h2 += ((((ui64)t2 << 32) | t1) >> 20) & 0x3ffffff; - state->h3 += ((((ui64)t3 << 32) | t2) >> 14) & 0x3ffffff; - state->h4 += (t3 >> 8); - - goto poly1305_donna_mul; -} - +/* + * Public Domain poly1305 from Andrew Moon + * Based on poly1305-donna.c, poly1305-donna-32.h and poly1305-donna.h from: + * https://github.com/floodyberry/poly1305-donna + */ + +#include "poly1305.h" + +#include <util/system/yassert.h> + +#include <string.h> + + +struct poly1305_state_st +{ + ui32 r0, r1, r2, r3, r4; + ui32 s1, s2, s3, s4; + ui32 h0, h1, h2, h3, h4; + ui8 buf[16]; + unsigned int buf_used; + ui8 key[16]; +}; + + +static uint32_t U8TO32_LE(const uint8_t *m) { + uint32_t r; + memcpy(&r, m, sizeof(r)); + return r; +} + +static void U32TO8_LE(uint8_t *m, uint32_t v) { memcpy(m, &v, sizeof(v)); } + +inline ui64 mul32x32_64(ui32 a, ui32 b) { return (ui64)a * b; } + +/* poly1305_blocks updates |state| given some amount of input data. This + * function may only be called with a |len| that is not a multiple of 16 at the + * end of the data. Otherwise the input must be buffered into 16 byte blocks. */ +static void poly1305_update(poly1305_state_st *state, const ui8 *in, size_t len) +{ + ui32 t0, t1, t2, t3; + ui64 t[5]; + ui32 b; + ui64 c; + size_t j; + + if (len < 16) { + goto poly1305_donna_atmost15bytes; + } + +poly1305_donna_16bytes: + t0 = U8TO32_LE(in); + t1 = U8TO32_LE(in + 4); + t2 = U8TO32_LE(in + 8); + t3 = U8TO32_LE(in + 12); + + in += 16; + len -= 16; + + state->h0 += t0 & 0x3ffffff; + state->h1 += ((((ui64)t1 << 32) | t0) >> 26) & 0x3ffffff; + state->h2 += ((((ui64)t2 << 32) | t1) >> 20) & 0x3ffffff; + state->h3 += ((((ui64)t3 << 32) | t2) >> 14) & 0x3ffffff; + state->h4 += (t3 >> 8) | (1 << 24); + +poly1305_donna_mul: + t[0] = mul32x32_64(state->h0, state->r0) + mul32x32_64(state->h1, state->s4) + + mul32x32_64(state->h2, state->s3) + mul32x32_64(state->h3, state->s2) + + mul32x32_64(state->h4, state->s1); + t[1] = mul32x32_64(state->h0, state->r1) + mul32x32_64(state->h1, state->r0) + + mul32x32_64(state->h2, state->s4) + mul32x32_64(state->h3, state->s3) + + mul32x32_64(state->h4, state->s2); + t[2] = mul32x32_64(state->h0, state->r2) + mul32x32_64(state->h1, state->r1) + + mul32x32_64(state->h2, state->r0) + mul32x32_64(state->h3, state->s4) + + mul32x32_64(state->h4, state->s3); + t[3] = mul32x32_64(state->h0, state->r3) + mul32x32_64(state->h1, state->r2) + + mul32x32_64(state->h2, state->r1) + mul32x32_64(state->h3, state->r0) + + mul32x32_64(state->h4, state->s4); + t[4] = mul32x32_64(state->h0, state->r4) + mul32x32_64(state->h1, state->r3) + + mul32x32_64(state->h2, state->r2) + mul32x32_64(state->h3, state->r1) + + mul32x32_64(state->h4, state->r0); + + state->h0 = (ui32)t[0] & 0x3ffffff; + c = (t[0] >> 26); + t[1] += c; + state->h1 = (ui32)t[1] & 0x3ffffff; + b = (ui32)(t[1] >> 26); + t[2] += b; + state->h2 = (ui32)t[2] & 0x3ffffff; + b = (ui32)(t[2] >> 26); + t[3] += b; + state->h3 = (ui32)t[3] & 0x3ffffff; + b = (ui32)(t[3] >> 26); + t[4] += b; + state->h4 = (ui32)t[4] & 0x3ffffff; + b = (ui32)(t[4] >> 26); + state->h0 += b * 5; + + if (len >= 16) { + goto poly1305_donna_16bytes; + } + + /* final bytes */ +poly1305_donna_atmost15bytes: + if (!len) { + return; + } + + ui8 mp[16]; + for (j = 0; j < len; j++) { + mp[j] = in[j]; + } + mp[j++] = 1; + for (; j < 16; j++) { + mp[j] = 0; + } + len = 0; + + t0 = U8TO32_LE(&mp[0] + 0); + t1 = U8TO32_LE(mp + 4); + t2 = U8TO32_LE(mp + 8); + t3 = U8TO32_LE(mp + 12); + + state->h0 += t0 & 0x3ffffff; + state->h1 += ((((ui64)t1 << 32) | t0) >> 26) & 0x3ffffff; + state->h2 += ((((ui64)t2 << 32) | t1) >> 20) & 0x3ffffff; + state->h3 += ((((ui64)t3 << 32) | t2) >> 14) & 0x3ffffff; + state->h4 += (t3 >> 8); + + goto poly1305_donna_mul; +} + #if (!defined(_win_) && !defined(_arm64_)) -constexpr size_t Poly1305::KEY_SIZE; -constexpr size_t Poly1305::MAC_SIZE; +constexpr size_t Poly1305::KEY_SIZE; +constexpr size_t Poly1305::MAC_SIZE; #endif - -void Poly1305::SetKey(const ui8* key, size_t size) -{ + +void Poly1305::SetKey(const ui8* key, size_t size) +{ Y_ASSERT((size == KEY_SIZE) && "key must be 32 bytes long"); - - poly1305_state_st *st = (poly1305_state_st *)&state; - ui32 t0, t1, t2, t3; - - t0 = U8TO32_LE(key + 0); - t1 = U8TO32_LE(key + 4); - t2 = U8TO32_LE(key + 8); - t3 = U8TO32_LE(key + 12); - - /* precompute multipliers */ - st->r0 = t0 & 0x3ffffff; - t0 >>= 26; - t0 |= t1 << 6; - st->r1 = t0 & 0x3ffff03; - t1 >>= 20; - t1 |= t2 << 12; - st->r2 = t1 & 0x3ffc0ff; - t2 >>= 14; - t2 |= t3 << 18; - st->r3 = t2 & 0x3f03fff; - t3 >>= 8; - st->r4 = t3 & 0x00fffff; - - st->s1 = st->r1 * 5; - st->s2 = st->r2 * 5; - st->s3 = st->r3 * 5; - st->s4 = st->r4 * 5; - - /* init state */ - st->h0 = 0; - st->h1 = 0; - st->h2 = 0; - st->h3 = 0; - st->h4 = 0; - - st->buf_used = 0; - memcpy(st->key, key + 16, sizeof(st->key)); -} - -void Poly1305::Update(const ui8* msg, size_t size) -{ - struct poly1305_state_st *st = (struct poly1305_state_st *)state; - - - if (st->buf_used) { - size_t todo = 16 - st->buf_used; - if (todo > size) { - todo = size; - } - for (size_t i = 0; i < todo; i++) { - st->buf[st->buf_used + i] = msg[i]; - } - st->buf_used += todo; - size -= todo; - msg += todo; - - if (st->buf_used == 16) { - poly1305_update(st, st->buf, 16); - st->buf_used = 0; - } - } - - if (size >= 16) { - size_t todo = size & ~0xf; - poly1305_update(st, msg, todo); - msg += todo; - size &= 0xf; - } - - if (size) { - for (size_t i = 0; i < size; i++) { - st->buf[i] = msg[i]; - } - st->buf_used = size; - } -} - -void Poly1305::Finish(ui8 mac[MAC_SIZE]) -{ - struct poly1305_state_st *st = (struct poly1305_state_st *)state; - ui64 f0, f1, f2, f3; - ui32 g0, g1, g2, g3, g4; - ui32 b, nb; - - if (st->buf_used) { - poly1305_update(st, st->buf, st->buf_used); - } - - b = st->h0 >> 26; - st->h0 = st->h0 & 0x3ffffff; - st->h1 += b; - b = st->h1 >> 26; - st->h1 = st->h1 & 0x3ffffff; - st->h2 += b; - b = st->h2 >> 26; - st->h2 = st->h2 & 0x3ffffff; - st->h3 += b; - b = st->h3 >> 26; - st->h3 = st->h3 & 0x3ffffff; - st->h4 += b; - b = st->h4 >> 26; - st->h4 = st->h4 & 0x3ffffff; - st->h0 += b * 5; - - g0 = st->h0 + 5; - b = g0 >> 26; - g0 &= 0x3ffffff; - g1 = st->h1 + b; - b = g1 >> 26; - g1 &= 0x3ffffff; - g2 = st->h2 + b; - b = g2 >> 26; - g2 &= 0x3ffffff; - g3 = st->h3 + b; - b = g3 >> 26; - g3 &= 0x3ffffff; - g4 = st->h4 + b - (1 << 26); - - b = (g4 >> 31) - 1; - nb = ~b; - st->h0 = (st->h0 & nb) | (g0 & b); - st->h1 = (st->h1 & nb) | (g1 & b); - st->h2 = (st->h2 & nb) | (g2 & b); - st->h3 = (st->h3 & nb) | (g3 & b); - st->h4 = (st->h4 & nb) | (g4 & b); - - f0 = ((st->h0) | (st->h1 << 26)) + (ui64)U8TO32_LE(&st->key[0]); - f1 = ((st->h1 >> 6) | (st->h2 << 20)) + - (ui64)U8TO32_LE(&st->key[4]); - f2 = ((st->h2 >> 12) | (st->h3 << 14)) + - (ui64)U8TO32_LE(&st->key[8]); - f3 = ((st->h3 >> 18) | (st->h4 << 8)) + - (ui64)U8TO32_LE(&st->key[12]); - - U32TO8_LE(&mac[0], f0); - f1 += (f0 >> 32); - U32TO8_LE(&mac[4], f1); - f2 += (f1 >> 32); - U32TO8_LE(&mac[8], f2); - f3 += (f2 >> 32); - U32TO8_LE(&mac[12], f3); -} + + poly1305_state_st *st = (poly1305_state_st *)&state; + ui32 t0, t1, t2, t3; + + t0 = U8TO32_LE(key + 0); + t1 = U8TO32_LE(key + 4); + t2 = U8TO32_LE(key + 8); + t3 = U8TO32_LE(key + 12); + + /* precompute multipliers */ + st->r0 = t0 & 0x3ffffff; + t0 >>= 26; + t0 |= t1 << 6; + st->r1 = t0 & 0x3ffff03; + t1 >>= 20; + t1 |= t2 << 12; + st->r2 = t1 & 0x3ffc0ff; + t2 >>= 14; + t2 |= t3 << 18; + st->r3 = t2 & 0x3f03fff; + t3 >>= 8; + st->r4 = t3 & 0x00fffff; + + st->s1 = st->r1 * 5; + st->s2 = st->r2 * 5; + st->s3 = st->r3 * 5; + st->s4 = st->r4 * 5; + + /* init state */ + st->h0 = 0; + st->h1 = 0; + st->h2 = 0; + st->h3 = 0; + st->h4 = 0; + + st->buf_used = 0; + memcpy(st->key, key + 16, sizeof(st->key)); +} + +void Poly1305::Update(const ui8* msg, size_t size) +{ + struct poly1305_state_st *st = (struct poly1305_state_st *)state; + + + if (st->buf_used) { + size_t todo = 16 - st->buf_used; + if (todo > size) { + todo = size; + } + for (size_t i = 0; i < todo; i++) { + st->buf[st->buf_used + i] = msg[i]; + } + st->buf_used += todo; + size -= todo; + msg += todo; + + if (st->buf_used == 16) { + poly1305_update(st, st->buf, 16); + st->buf_used = 0; + } + } + + if (size >= 16) { + size_t todo = size & ~0xf; + poly1305_update(st, msg, todo); + msg += todo; + size &= 0xf; + } + + if (size) { + for (size_t i = 0; i < size; i++) { + st->buf[i] = msg[i]; + } + st->buf_used = size; + } +} + +void Poly1305::Finish(ui8 mac[MAC_SIZE]) +{ + struct poly1305_state_st *st = (struct poly1305_state_st *)state; + ui64 f0, f1, f2, f3; + ui32 g0, g1, g2, g3, g4; + ui32 b, nb; + + if (st->buf_used) { + poly1305_update(st, st->buf, st->buf_used); + } + + b = st->h0 >> 26; + st->h0 = st->h0 & 0x3ffffff; + st->h1 += b; + b = st->h1 >> 26; + st->h1 = st->h1 & 0x3ffffff; + st->h2 += b; + b = st->h2 >> 26; + st->h2 = st->h2 & 0x3ffffff; + st->h3 += b; + b = st->h3 >> 26; + st->h3 = st->h3 & 0x3ffffff; + st->h4 += b; + b = st->h4 >> 26; + st->h4 = st->h4 & 0x3ffffff; + st->h0 += b * 5; + + g0 = st->h0 + 5; + b = g0 >> 26; + g0 &= 0x3ffffff; + g1 = st->h1 + b; + b = g1 >> 26; + g1 &= 0x3ffffff; + g2 = st->h2 + b; + b = g2 >> 26; + g2 &= 0x3ffffff; + g3 = st->h3 + b; + b = g3 >> 26; + g3 &= 0x3ffffff; + g4 = st->h4 + b - (1 << 26); + + b = (g4 >> 31) - 1; + nb = ~b; + st->h0 = (st->h0 & nb) | (g0 & b); + st->h1 = (st->h1 & nb) | (g1 & b); + st->h2 = (st->h2 & nb) | (g2 & b); + st->h3 = (st->h3 & nb) | (g3 & b); + st->h4 = (st->h4 & nb) | (g4 & b); + + f0 = ((st->h0) | (st->h1 << 26)) + (ui64)U8TO32_LE(&st->key[0]); + f1 = ((st->h1 >> 6) | (st->h2 << 20)) + + (ui64)U8TO32_LE(&st->key[4]); + f2 = ((st->h2 >> 12) | (st->h3 << 14)) + + (ui64)U8TO32_LE(&st->key[8]); + f3 = ((st->h3 >> 18) | (st->h4 << 8)) + + (ui64)U8TO32_LE(&st->key[12]); + + U32TO8_LE(&mac[0], f0); + f1 += (f0 >> 32); + U32TO8_LE(&mac[4], f1); + f2 += (f1 >> 32); + U32TO8_LE(&mac[8], f2); + f3 += (f2 >> 32); + U32TO8_LE(&mac[12], f3); +} diff --git a/ydb/core/blobstorage/crypto/poly1305.h b/ydb/core/blobstorage/crypto/poly1305.h index 60c971f630d..da19bdc8c5b 100644 --- a/ydb/core/blobstorage/crypto/poly1305.h +++ b/ydb/core/blobstorage/crypto/poly1305.h @@ -1,21 +1,21 @@ -#pragma once - -#include <util/system/types.h> - - -using poly1305_state = ui8[512]; - -class Poly1305 -{ -public: - static constexpr size_t KEY_SIZE = 32; - static constexpr size_t MAC_SIZE = 16; - -public: - void SetKey(const ui8* key, size_t size); - void Update(const ui8* msg, size_t size); - void Finish(ui8 mac[MAC_SIZE]); - -private: - poly1305_state state; -}; +#pragma once + +#include <util/system/types.h> + + +using poly1305_state = ui8[512]; + +class Poly1305 +{ +public: + static constexpr size_t KEY_SIZE = 32; + static constexpr size_t MAC_SIZE = 16; + +public: + void SetKey(const ui8* key, size_t size); + void Update(const ui8* msg, size_t size); + void Finish(ui8 mac[MAC_SIZE]); + +private: + poly1305_state state; +}; diff --git a/ydb/core/blobstorage/crypto/poly1305_ut.cpp b/ydb/core/blobstorage/crypto/poly1305_ut.cpp index 8c663517066..41632d764b6 100644 --- a/ydb/core/blobstorage/crypto/poly1305_ut.cpp +++ b/ydb/core/blobstorage/crypto/poly1305_ut.cpp @@ -1,44 +1,44 @@ -#include "poly1305.h" +#include "poly1305.h" #include <ydb/core/blobstorage/crypto/ut/ut_helpers.h> #include <ydb/core/blobstorage/crypto/ut/poly1305_test_vectors.h> - - + + Y_UNIT_TEST_SUITE(TPoly1305) -{ - inline void RunTest(const ui8* input, size_t input_size, - const ui8 key[32], const ui8 expectedMac[16]) - { - ui8 mac[16]; - - Poly1305 poly; - poly.SetKey(key, 32); - poly.Update(input, input_size); - poly.Finish(mac); - - UNIT_ASSERT_ARRAYS_EQUAL(mac, expectedMac, sizeof(mac)); - - // test that at the second time we get the same result - - poly.SetKey(key, 32); - poly.Update(input, input_size); - poly.Finish(mac); - - UNIT_ASSERT_ARRAYS_EQUAL(mac, expectedMac, sizeof(mac)); - } - +{ + inline void RunTest(const ui8* input, size_t input_size, + const ui8 key[32], const ui8 expectedMac[16]) + { + ui8 mac[16]; + + Poly1305 poly; + poly.SetKey(key, 32); + poly.Update(input, input_size); + poly.Finish(mac); + + UNIT_ASSERT_ARRAYS_EQUAL(mac, expectedMac, sizeof(mac)); + + // test that at the second time we get the same result + + poly.SetKey(key, 32); + poly.Update(input, input_size); + poly.Finish(mac); + + UNIT_ASSERT_ARRAYS_EQUAL(mac, expectedMac, sizeof(mac)); + } + Y_UNIT_TEST(TestVector1) { - RunTest(tc1_data, sizeof(tc1_data), tc1_key, tc1_tag); - } - + RunTest(tc1_data, sizeof(tc1_data), tc1_key, tc1_tag); + } + Y_UNIT_TEST(TestVector2) { - RunTest(tc2_data, sizeof(tc2_data) - 1, tc2_key, tc2_tag); - } - + RunTest(tc2_data, sizeof(tc2_data) - 1, tc2_key, tc2_tag); + } + Y_UNIT_TEST(TestVector3) { - RunTest(tc3_data, sizeof(tc3_data) - 1, tc3_key, tc3_tag); - } - + RunTest(tc3_data, sizeof(tc3_data) - 1, tc3_key, tc3_tag); + } + Y_UNIT_TEST(TestVector4) { - RunTest(tc4_data, sizeof(tc4_data) - 1, tc4_key, tc4_tag); - } -} + RunTest(tc4_data, sizeof(tc4_data) - 1, tc4_key, tc4_tag); + } +} diff --git a/ydb/core/blobstorage/crypto/poly1305_vec.cpp b/ydb/core/blobstorage/crypto/poly1305_vec.cpp index 7404232e0aa..25f8308f793 100644 --- a/ydb/core/blobstorage/crypto/poly1305_vec.cpp +++ b/ydb/core/blobstorage/crypto/poly1305_vec.cpp @@ -1,665 +1,665 @@ -/* - * Public Domain Poly1305Vec from Andrew Moon - * Based on Poly1305Vec-donna.c, Poly1305Vec-donna-32.h and Poly1305Vec-donna.h from: - * https://github.com/floodyberry/Poly1305Vec-donna - */ - -#include "poly1305_vec.h" - -#include <util/system/yassert.h> +/* + * Public Domain Poly1305Vec from Andrew Moon + * Based on Poly1305Vec-donna.c, Poly1305Vec-donna-32.h and Poly1305Vec-donna.h from: + * https://github.com/floodyberry/Poly1305Vec-donna + */ + +#include "poly1305_vec.h" + +#include <util/system/yassert.h> #include <util/system/unaligned_mem.h> - -/* This implementation of poly1305 is by Andrew Moon - * (https://github.com/floodyberry/poly1305-donna) and released as public - * domain. It implements SIMD vectorization based on the algorithm described in - * http://cr.yp.to/papers.html#neoncrypto. Unrolled to 2 powers, i.e. 64 byte - * block size -*/ - + +/* This implementation of poly1305 is by Andrew Moon + * (https://github.com/floodyberry/poly1305-donna) and released as public + * domain. It implements SIMD vectorization based on the algorithm described in + * http://cr.yp.to/papers.html#neoncrypto. Unrolled to 2 powers, i.e. 64 byte + * block size +*/ + #include <library/cpp/sse/sse.h> // The header chooses appropriate SSE support -#include <stdint.h> - -#define INLINE inline -#define U8TO64_LE(m) (*(uint64_t*)(m)) -#define U8TO32_LE(m) (*(uint32_t*)(m)) -#define U64TO8_LE(m,v) (*(uint64_t*)(m)) = v - -typedef __m128i xmmi; -typedef unsigned __int128 uint128_t; - +#include <stdint.h> + +#define INLINE inline +#define U8TO64_LE(m) (*(uint64_t*)(m)) +#define U8TO32_LE(m) (*(uint32_t*)(m)) +#define U64TO8_LE(m,v) (*(uint64_t*)(m)) = v + +typedef __m128i xmmi; +typedef unsigned __int128 uint128_t; + alignas(16) static const uint32_t poly1305_x64_sse2_message_mask[4] = - {(1 << 26) - 1, 0, (1 << 26) - 1, 0}; + {(1 << 26) - 1, 0, (1 << 26) - 1, 0}; alignas(16) static const uint32_t poly1305_x64_sse2_5[4] = {5, 0, 5, 0}; alignas(16) static const uint32_t poly1305_x64_sse2_1shl128[4] = - {(1 << 24), 0, (1 << 24), 0}; - -static uint128_t INLINE add128(uint128_t a, uint128_t b) -{ - return a + b; -} - -static uint128_t INLINE add128_64(uint128_t a, uint64_t b) -{ - return a + b; -} - -static uint128_t INLINE mul64x64_128(uint64_t a, uint64_t b) -{ - return (uint128_t)a * b; -} - -static uint64_t INLINE lo128(uint128_t a) -{ - return (uint64_t)a; -} - -static uint64_t INLINE shr128(uint128_t v, const int shift) -{ - return (uint64_t)(v >> shift); -} - -static uint64_t INLINE shr128_pair(uint64_t hi, uint64_t lo, const int shift) -{ - return (uint64_t)((((uint128_t)hi << 64) | lo) >> shift); -} - -typedef struct poly1305_power_t -{ - union { - xmmi v; - uint64_t u[2]; - uint32_t d[4]; - } R20,R21,R22,R23,R24,S21,S22,S23,S24; -} poly1305_power; - -typedef struct poly1305_state_internal_t -{ - poly1305_power P[2]; /* 288 bytes, top 32 bit halves unused = 144 - bytes of free storage */ - union { - xmmi H[5]; /* 80 bytes */ - uint64_t HH[10]; - }; - /* uint64_t r0,r1,r2; [24 bytes] */ - /* uint64_t pad0,pad1; [16 bytes] */ - uint64_t started; /* 8 bytes */ - uint64_t leftover; /* 8 bytes */ - uint8_t buffer[64]; /* 64 bytes */ -} poly1305_state_internal; /* 448 bytes total + 63 bytes for - alignment = 511 bytes raw */ - -static poly1305_state_internal INLINE -*poly1305_aligned_state(poly1305_state *state) -{ - return (poly1305_state_internal *)(((uint64_t)state + 63) & ~63); -} - -/* copy 0-63 bytes */ -static void INLINE -poly1305_block_copy(uint8_t *dst, const uint8_t *src, size_t bytes) -{ - if (bytes & 32) { + {(1 << 24), 0, (1 << 24), 0}; + +static uint128_t INLINE add128(uint128_t a, uint128_t b) +{ + return a + b; +} + +static uint128_t INLINE add128_64(uint128_t a, uint64_t b) +{ + return a + b; +} + +static uint128_t INLINE mul64x64_128(uint64_t a, uint64_t b) +{ + return (uint128_t)a * b; +} + +static uint64_t INLINE lo128(uint128_t a) +{ + return (uint64_t)a; +} + +static uint64_t INLINE shr128(uint128_t v, const int shift) +{ + return (uint64_t)(v >> shift); +} + +static uint64_t INLINE shr128_pair(uint64_t hi, uint64_t lo, const int shift) +{ + return (uint64_t)((((uint128_t)hi << 64) | lo) >> shift); +} + +typedef struct poly1305_power_t +{ + union { + xmmi v; + uint64_t u[2]; + uint32_t d[4]; + } R20,R21,R22,R23,R24,S21,S22,S23,S24; +} poly1305_power; + +typedef struct poly1305_state_internal_t +{ + poly1305_power P[2]; /* 288 bytes, top 32 bit halves unused = 144 + bytes of free storage */ + union { + xmmi H[5]; /* 80 bytes */ + uint64_t HH[10]; + }; + /* uint64_t r0,r1,r2; [24 bytes] */ + /* uint64_t pad0,pad1; [16 bytes] */ + uint64_t started; /* 8 bytes */ + uint64_t leftover; /* 8 bytes */ + uint8_t buffer[64]; /* 64 bytes */ +} poly1305_state_internal; /* 448 bytes total + 63 bytes for + alignment = 511 bytes raw */ + +static poly1305_state_internal INLINE +*poly1305_aligned_state(poly1305_state *state) +{ + return (poly1305_state_internal *)(((uint64_t)state + 63) & ~63); +} + +/* copy 0-63 bytes */ +static void INLINE +poly1305_block_copy(uint8_t *dst, const uint8_t *src, size_t bytes) +{ + if (bytes & 32) { _mm_storeu_si128((xmmi *)(dst + 0), _mm_loadu_si128((xmmi *)(src + 0))); _mm_storeu_si128((xmmi *)(dst + 16), _mm_loadu_si128((xmmi *)(src + 16))); - dst += 32; + dst += 32; src += 32; - } - if (bytes & 16) { - _mm_storeu_si128((xmmi *)dst, + } + if (bytes & 16) { + _mm_storeu_si128((xmmi *)dst, _mm_loadu_si128((xmmi *)(src))); - dst += 16; + dst += 16; src += 16; - } - if (bytes & 8) { + } + if (bytes & 8) { WriteUnaligned<uint64_t>(dst, ReadUnaligned<uint64_t>(src)); - dst += 8; + dst += 8; src += 8; - } - if (bytes & 4) { + } + if (bytes & 4) { WriteUnaligned<uint32_t>(dst, ReadUnaligned<uint32_t>(src)); - dst += 4; + dst += 4; src += 4; - } - if (bytes & 2) { + } + if (bytes & 2) { WriteUnaligned<uint16_t>(dst, ReadUnaligned<uint16_t>(src)); - dst += 2; + dst += 2; src += 2; - } - if (bytes & 1) { + } + if (bytes & 1) { WriteUnaligned<uint8_t>(dst, ReadUnaligned<uint8_t>(src)); - } -} - -/* zero 0-15 bytes */ -static void INLINE poly1305_block_zero(uint8_t *dst, size_t bytes) -{ + } +} + +/* zero 0-15 bytes */ +static void INLINE poly1305_block_zero(uint8_t *dst, size_t bytes) +{ if (bytes & 1) { *( uint8_t *)dst = 0; dst++; } if (bytes & 2) { *(uint16_t *)dst = 0; dst += 2; } if (bytes & 4) { *(uint32_t *)dst = 0; dst += 4; } - if (bytes & 8) { *(uint64_t *)dst = 0; dst += 8; } -} - -static size_t INLINE poly1305_min(size_t a, size_t b) -{ - return (a < b) ? a : b; -} - -static void -poly1305_first_block(poly1305_state_internal *st, const uint8_t *m) -{ - const xmmi MMASK = - _mm_load_si128((xmmi *)poly1305_x64_sse2_message_mask); - const xmmi FIVE = _mm_load_si128((xmmi*)poly1305_x64_sse2_5); - const xmmi HIBIT = _mm_load_si128((xmmi*)poly1305_x64_sse2_1shl128); - xmmi T5,T6; - poly1305_power *p; - uint128_t d[3]; - uint64_t r0,r1,r2; - uint64_t r20,r21,r22,s22; - uint64_t pad0,pad1; - uint64_t c; - uint64_t i; - - /* pull out stored info */ - p = &st->P[1]; - - r0 = ((uint64_t)p->R20.d[3] << 32) | (uint64_t)p->R20.d[1]; - r1 = ((uint64_t)p->R21.d[3] << 32) | (uint64_t)p->R21.d[1]; - r2 = ((uint64_t)p->R22.d[3] << 32) | (uint64_t)p->R22.d[1]; - pad0 = ((uint64_t)p->R23.d[3] << 32) | (uint64_t)p->R23.d[1]; - pad1 = ((uint64_t)p->R24.d[3] << 32) | (uint64_t)p->R24.d[1]; - - /* compute powers r^2,r^4 */ - r20 = r0; - r21 = r1; - r22 = r2; - for (i = 0; i < 2; i++) { - s22 = r22 * (5 << 2); - - d[0] = add128(mul64x64_128(r20, r20), mul64x64_128(r21 * 2, s22)); - d[1] = add128(mul64x64_128(r22, s22), mul64x64_128(r20 * 2, r21)); - d[2] = add128(mul64x64_128(r21, r21), mul64x64_128(r22 * 2, r20)); - - r20 = lo128(d[0]) & 0xfffffffffff; c = shr128(d[0], 44); - d[1] = add128_64(d[1], c); r21 = lo128(d[1]) & 0xfffffffffff; c = shr128(d[1], 44); - d[2] = add128_64(d[2], c); r22 = lo128(d[2]) & 0x3ffffffffff; c = shr128(d[2], 42); - r20 += c * 5; c = (r20 >> 44); r20 = r20 & 0xfffffffffff; - r21 += c; - - p->R20.v = _mm_shuffle_epi32(_mm_cvtsi32_si128((uint32_t)( r20 ) & 0x3ffffff), _MM_SHUFFLE(1,0,1,0)); - p->R21.v = _mm_shuffle_epi32(_mm_cvtsi32_si128((uint32_t)((r20 >> 26) | (r21 << 18)) & 0x3ffffff), _MM_SHUFFLE(1,0,1,0)); - p->R22.v = _mm_shuffle_epi32(_mm_cvtsi32_si128((uint32_t)((r21 >> 8) ) & 0x3ffffff), _MM_SHUFFLE(1,0,1,0)); - p->R23.v = _mm_shuffle_epi32(_mm_cvtsi32_si128((uint32_t)((r21 >> 34) | (r22 << 10)) & 0x3ffffff), _MM_SHUFFLE(1,0,1,0)); - p->R24.v = _mm_shuffle_epi32(_mm_cvtsi32_si128((uint32_t)((r22 >> 16) ) ), _MM_SHUFFLE(1,0,1,0)); - p->S21.v = _mm_mul_epu32(p->R21.v, FIVE); - p->S22.v = _mm_mul_epu32(p->R22.v, FIVE); - p->S23.v = _mm_mul_epu32(p->R23.v, FIVE); - p->S24.v = _mm_mul_epu32(p->R24.v, FIVE); - p--; - } - - /* put saved info back */ - p = &st->P[1]; - p->R20.d[1] = (uint32_t)(r0 ); - p->R20.d[3] = (uint32_t)(r0 >> 32); - p->R21.d[1] = (uint32_t)(r1 ); - p->R21.d[3] = (uint32_t)(r1 >> 32); - p->R22.d[1] = (uint32_t)(r2 ); - p->R22.d[3] = (uint32_t)(r2 >> 32); - p->R23.d[1] = (uint32_t)(pad0 ); - p->R23.d[3] = (uint32_t)(pad0 >> 32); - p->R24.d[1] = (uint32_t)(pad1 ); - p->R24.d[3] = (uint32_t)(pad1 >> 32); - - /* H = [Mx,My] */ - T5 = _mm_unpacklo_epi64(_mm_loadl_epi64((xmmi *)(m + 0)), _mm_loadl_epi64((xmmi *)(m + 16))); - T6 = _mm_unpacklo_epi64(_mm_loadl_epi64((xmmi *)(m + 8)), _mm_loadl_epi64((xmmi *)(m + 24))); - st->H[0] = _mm_and_si128(MMASK, T5); - st->H[1] = _mm_and_si128(MMASK, _mm_srli_epi64(T5, 26)); - T5 = _mm_or_si128(_mm_srli_epi64(T5, 52), _mm_slli_epi64(T6, 12)); - st->H[2] = _mm_and_si128(MMASK, T5); - st->H[3] = _mm_and_si128(MMASK, _mm_srli_epi64(T5, 26)); - st->H[4] = _mm_or_si128(_mm_srli_epi64(T6, 40), HIBIT); -} - -Y_NO_SANITIZE("memory") static void -poly1305_blocks(poly1305_state_internal *st, const uint8_t *m, size_t bytes) -{ - const xmmi MMASK = _mm_load_si128((xmmi *)poly1305_x64_sse2_message_mask); - const xmmi FIVE = _mm_load_si128((xmmi*)poly1305_x64_sse2_5); - const xmmi HIBIT = _mm_load_si128((xmmi*)poly1305_x64_sse2_1shl128); - - poly1305_power *p; - xmmi H0,H1,H2,H3,H4; - xmmi T0,T1,T2,T3,T4,T5,T6; - xmmi M0,M1,M2,M3,M4; - xmmi C1,C2; - - H0 = st->H[0]; - H1 = st->H[1]; - H2 = st->H[2]; - H3 = st->H[3]; - H4 = st->H[4]; - - while (bytes >= 64) { - /* H *= [r^4,r^4] */ - p = &st->P[0]; - T0 = _mm_mul_epu32(H0, p->R20.v); - T1 = _mm_mul_epu32(H0, p->R21.v); - T2 = _mm_mul_epu32(H0, p->R22.v); - T3 = _mm_mul_epu32(H0, p->R23.v); - T4 = _mm_mul_epu32(H0, p->R24.v); - T5 = _mm_mul_epu32(H1, p->S24.v); T6 = _mm_mul_epu32(H1, p->R20.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); - T5 = _mm_mul_epu32(H2, p->S23.v); T6 = _mm_mul_epu32(H2, p->S24.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); - T5 = _mm_mul_epu32(H3, p->S22.v); T6 = _mm_mul_epu32(H3, p->S23.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); - T5 = _mm_mul_epu32(H4, p->S21.v); T6 = _mm_mul_epu32(H4, p->S22.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); - T5 = _mm_mul_epu32(H1, p->R21.v); T6 = _mm_mul_epu32(H1, p->R22.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); - T5 = _mm_mul_epu32(H2, p->R20.v); T6 = _mm_mul_epu32(H2, p->R21.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); - T5 = _mm_mul_epu32(H3, p->S24.v); T6 = _mm_mul_epu32(H3, p->R20.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); - T5 = _mm_mul_epu32(H4, p->S23.v); T6 = _mm_mul_epu32(H4, p->S24.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); - T5 = _mm_mul_epu32(H1, p->R23.v); T4 = _mm_add_epi64(T4, T5); - T5 = _mm_mul_epu32(H2, p->R22.v); T4 = _mm_add_epi64(T4, T5); - T5 = _mm_mul_epu32(H3, p->R21.v); T4 = _mm_add_epi64(T4, T5); - T5 = _mm_mul_epu32(H4, p->R20.v); T4 = _mm_add_epi64(T4, T5); - - /* H += [Mx,My]*[r^2,r^2] */ - T5 = _mm_unpacklo_epi64(_mm_loadl_epi64((xmmi *)(m + 0)), _mm_loadl_epi64((xmmi *)(m + 16))); - T6 = _mm_unpacklo_epi64(_mm_loadl_epi64((xmmi *)(m + 8)), _mm_loadl_epi64((xmmi *)(m + 24))); - M0 = _mm_and_si128(MMASK, T5); - M1 = _mm_and_si128(MMASK, _mm_srli_epi64(T5, 26)); - T5 = _mm_or_si128(_mm_srli_epi64(T5, 52), _mm_slli_epi64(T6, 12)); - M2 = _mm_and_si128(MMASK, T5); - M3 = _mm_and_si128(MMASK, _mm_srli_epi64(T5, 26)); - M4 = _mm_or_si128(_mm_srli_epi64(T6, 40), HIBIT); - - p = &st->P[1]; - T5 = _mm_mul_epu32(M0, p->R20.v); T6 = _mm_mul_epu32(M0, p->R21.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); - T5 = _mm_mul_epu32(M1, p->S24.v); T6 = _mm_mul_epu32(M1, p->R20.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); - T5 = _mm_mul_epu32(M2, p->S23.v); T6 = _mm_mul_epu32(M2, p->S24.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); - T5 = _mm_mul_epu32(M3, p->S22.v); T6 = _mm_mul_epu32(M3, p->S23.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); - T5 = _mm_mul_epu32(M4, p->S21.v); T6 = _mm_mul_epu32(M4, p->S22.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); - T5 = _mm_mul_epu32(M0, p->R22.v); T6 = _mm_mul_epu32(M0, p->R23.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); - T5 = _mm_mul_epu32(M1, p->R21.v); T6 = _mm_mul_epu32(M1, p->R22.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); - T5 = _mm_mul_epu32(M2, p->R20.v); T6 = _mm_mul_epu32(M2, p->R21.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); - T5 = _mm_mul_epu32(M3, p->S24.v); T6 = _mm_mul_epu32(M3, p->R20.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); - T5 = _mm_mul_epu32(M4, p->S23.v); T6 = _mm_mul_epu32(M4, p->S24.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); - T5 = _mm_mul_epu32(M0, p->R24.v); T4 = _mm_add_epi64(T4, T5); - T5 = _mm_mul_epu32(M1, p->R23.v); T4 = _mm_add_epi64(T4, T5); - T5 = _mm_mul_epu32(M2, p->R22.v); T4 = _mm_add_epi64(T4, T5); - T5 = _mm_mul_epu32(M3, p->R21.v); T4 = _mm_add_epi64(T4, T5); - T5 = _mm_mul_epu32(M4, p->R20.v); T4 = _mm_add_epi64(T4, T5); - - /* H += [Mx,My] */ - T5 = _mm_unpacklo_epi64(_mm_loadl_epi64((xmmi *)(m + 32)), _mm_loadl_epi64((xmmi *)(m + 48))); - T6 = _mm_unpacklo_epi64(_mm_loadl_epi64((xmmi *)(m + 40)), _mm_loadl_epi64((xmmi *)(m + 56))); - M0 = _mm_and_si128(MMASK, T5); - M1 = _mm_and_si128(MMASK, _mm_srli_epi64(T5, 26)); - T5 = _mm_or_si128(_mm_srli_epi64(T5, 52), _mm_slli_epi64(T6, 12)); - M2 = _mm_and_si128(MMASK, T5); - M3 = _mm_and_si128(MMASK, _mm_srli_epi64(T5, 26)); - M4 = _mm_or_si128(_mm_srli_epi64(T6, 40), HIBIT); - - T0 = _mm_add_epi64(T0, M0); - T1 = _mm_add_epi64(T1, M1); - T2 = _mm_add_epi64(T2, M2); - T3 = _mm_add_epi64(T3, M3); - T4 = _mm_add_epi64(T4, M4); - - /* reduce */ - C1 = _mm_srli_epi64(T0, 26); C2 = _mm_srli_epi64(T3, 26); T0 = _mm_and_si128(T0, MMASK); T3 = _mm_and_si128(T3, MMASK); T1 = _mm_add_epi64(T1, C1); T4 = _mm_add_epi64(T4, C2); - C1 = _mm_srli_epi64(T1, 26); C2 = _mm_srli_epi64(T4, 26); T1 = _mm_and_si128(T1, MMASK); T4 = _mm_and_si128(T4, MMASK); T2 = _mm_add_epi64(T2, C1); T0 = _mm_add_epi64(T0, _mm_mul_epu32(C2, FIVE)); - C1 = _mm_srli_epi64(T2, 26); C2 = _mm_srli_epi64(T0, 26); T2 = _mm_and_si128(T2, MMASK); T0 = _mm_and_si128(T0, MMASK); T3 = _mm_add_epi64(T3, C1); T1 = _mm_add_epi64(T1, C2); - C1 = _mm_srli_epi64(T3, 26); T3 = _mm_and_si128(T3, MMASK); T4 = _mm_add_epi64(T4, C1); - - /* H = (H*[r^4,r^4] + [Mx,My]*[r^2,r^2] + [Mx,My]) */ - H0 = T0; - H1 = T1; - H2 = T2; - H3 = T3; - H4 = T4; - - m += 64; - bytes -= 64; - } - - st->H[0] = H0; - st->H[1] = H1; - st->H[2] = H2; - st->H[3] = H3; - st->H[4] = H4; -} - -static size_t -poly1305_combine(poly1305_state_internal *st, const uint8_t *m, size_t bytes) -{ - const xmmi MMASK = - _mm_load_si128((xmmi *)poly1305_x64_sse2_message_mask); - const xmmi HIBIT = _mm_load_si128((xmmi*)poly1305_x64_sse2_1shl128); - const xmmi FIVE = _mm_load_si128((xmmi*)poly1305_x64_sse2_5); - - poly1305_power *p; - xmmi H0,H1,H2,H3,H4; - xmmi M0,M1,M2,M3,M4; - xmmi T0,T1,T2,T3,T4,T5,T6; - xmmi C1,C2; - - uint64_t r0,r1,r2; - uint64_t t0,t1,t2,t3,t4; - uint64_t c; - size_t consumed = 0; - - H0 = st->H[0]; - H1 = st->H[1]; - H2 = st->H[2]; - H3 = st->H[3]; - H4 = st->H[4]; - - /* p = [r^2,r^2] */ - p = &st->P[1]; - - if (bytes >= 32) { - /* H *= [r^2,r^2] */ - T0 = _mm_mul_epu32(H0, p->R20.v); - T1 = _mm_mul_epu32(H0, p->R21.v); - T2 = _mm_mul_epu32(H0, p->R22.v); - T3 = _mm_mul_epu32(H0, p->R23.v); - T4 = _mm_mul_epu32(H0, p->R24.v); - T5 = _mm_mul_epu32(H1, p->S24.v); T6 = _mm_mul_epu32(H1, p->R20.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); - T5 = _mm_mul_epu32(H2, p->S23.v); T6 = _mm_mul_epu32(H2, p->S24.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); - T5 = _mm_mul_epu32(H3, p->S22.v); T6 = _mm_mul_epu32(H3, p->S23.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); - T5 = _mm_mul_epu32(H4, p->S21.v); T6 = _mm_mul_epu32(H4, p->S22.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); - T5 = _mm_mul_epu32(H1, p->R21.v); T6 = _mm_mul_epu32(H1, p->R22.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); - T5 = _mm_mul_epu32(H2, p->R20.v); T6 = _mm_mul_epu32(H2, p->R21.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); - T5 = _mm_mul_epu32(H3, p->S24.v); T6 = _mm_mul_epu32(H3, p->R20.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); - T5 = _mm_mul_epu32(H4, p->S23.v); T6 = _mm_mul_epu32(H4, p->S24.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); - T5 = _mm_mul_epu32(H1, p->R23.v); T4 = _mm_add_epi64(T4, T5); - T5 = _mm_mul_epu32(H2, p->R22.v); T4 = _mm_add_epi64(T4, T5); - T5 = _mm_mul_epu32(H3, p->R21.v); T4 = _mm_add_epi64(T4, T5); - T5 = _mm_mul_epu32(H4, p->R20.v); T4 = _mm_add_epi64(T4, T5); - - /* H += [Mx,My] */ - T5 = _mm_unpacklo_epi64(_mm_loadl_epi64((xmmi *)(m + 0)), _mm_loadl_epi64((xmmi *)(m + 16))); - T6 = _mm_unpacklo_epi64(_mm_loadl_epi64((xmmi *)(m + 8)), _mm_loadl_epi64((xmmi *)(m + 24))); - M0 = _mm_and_si128(MMASK, T5); - M1 = _mm_and_si128(MMASK, _mm_srli_epi64(T5, 26)); - T5 = _mm_or_si128(_mm_srli_epi64(T5, 52), _mm_slli_epi64(T6, 12)); - M2 = _mm_and_si128(MMASK, T5); - M3 = _mm_and_si128(MMASK, _mm_srli_epi64(T5, 26)); - M4 = _mm_or_si128(_mm_srli_epi64(T6, 40), HIBIT); - - T0 = _mm_add_epi64(T0, M0); - T1 = _mm_add_epi64(T1, M1); - T2 = _mm_add_epi64(T2, M2); - T3 = _mm_add_epi64(T3, M3); - T4 = _mm_add_epi64(T4, M4); - - /* reduce */ - C1 = _mm_srli_epi64(T0, 26); C2 = _mm_srli_epi64(T3, 26); T0 = _mm_and_si128(T0, MMASK); T3 = _mm_and_si128(T3, MMASK); T1 = _mm_add_epi64(T1, C1); T4 = _mm_add_epi64(T4, C2); - C1 = _mm_srli_epi64(T1, 26); C2 = _mm_srli_epi64(T4, 26); T1 = _mm_and_si128(T1, MMASK); T4 = _mm_and_si128(T4, MMASK); T2 = _mm_add_epi64(T2, C1); T0 = _mm_add_epi64(T0, _mm_mul_epu32(C2, FIVE)); - C1 = _mm_srli_epi64(T2, 26); C2 = _mm_srli_epi64(T0, 26); T2 = _mm_and_si128(T2, MMASK); T0 = _mm_and_si128(T0, MMASK); T3 = _mm_add_epi64(T3, C1); T1 = _mm_add_epi64(T1, C2); - C1 = _mm_srli_epi64(T3, 26); T3 = _mm_and_si128(T3, MMASK); T4 = _mm_add_epi64(T4, C1); - - /* H = (H*[r^2,r^2] + [Mx,My]) */ - H0 = T0; - H1 = T1; - H2 = T2; - H3 = T3; - H4 = T4; - - consumed = 32; - } - - /* finalize, H *= [r^2,r] */ - r0 = ((uint64_t)p->R20.d[3] << 32) | (uint64_t)p->R20.d[1]; - r1 = ((uint64_t)p->R21.d[3] << 32) | (uint64_t)p->R21.d[1]; - r2 = ((uint64_t)p->R22.d[3] << 32) | (uint64_t)p->R22.d[1]; - - p->R20.d[2] = (uint32_t)( r0 ) & 0x3ffffff; - p->R21.d[2] = (uint32_t)((r0 >> 26) | (r1 << 18)) & 0x3ffffff; - p->R22.d[2] = (uint32_t)((r1 >> 8) ) & 0x3ffffff; - p->R23.d[2] = (uint32_t)((r1 >> 34) | (r2 << 10)) & 0x3ffffff; - p->R24.d[2] = (uint32_t)((r2 >> 16) ) ; - p->S21.d[2] = p->R21.d[2] * 5; - p->S22.d[2] = p->R22.d[2] * 5; - p->S23.d[2] = p->R23.d[2] * 5; - p->S24.d[2] = p->R24.d[2] * 5; - - /* H *= [r^2,r] */ - T0 = _mm_mul_epu32(H0, p->R20.v); - T1 = _mm_mul_epu32(H0, p->R21.v); - T2 = _mm_mul_epu32(H0, p->R22.v); - T3 = _mm_mul_epu32(H0, p->R23.v); - T4 = _mm_mul_epu32(H0, p->R24.v); - T5 = _mm_mul_epu32(H1, p->S24.v); T6 = _mm_mul_epu32(H1, p->R20.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); - T5 = _mm_mul_epu32(H2, p->S23.v); T6 = _mm_mul_epu32(H2, p->S24.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); - T5 = _mm_mul_epu32(H3, p->S22.v); T6 = _mm_mul_epu32(H3, p->S23.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); - T5 = _mm_mul_epu32(H4, p->S21.v); T6 = _mm_mul_epu32(H4, p->S22.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); - T5 = _mm_mul_epu32(H1, p->R21.v); T6 = _mm_mul_epu32(H1, p->R22.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); - T5 = _mm_mul_epu32(H2, p->R20.v); T6 = _mm_mul_epu32(H2, p->R21.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); - T5 = _mm_mul_epu32(H3, p->S24.v); T6 = _mm_mul_epu32(H3, p->R20.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); - T5 = _mm_mul_epu32(H4, p->S23.v); T6 = _mm_mul_epu32(H4, p->S24.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); - T5 = _mm_mul_epu32(H1, p->R23.v); T4 = _mm_add_epi64(T4, T5); - T5 = _mm_mul_epu32(H2, p->R22.v); T4 = _mm_add_epi64(T4, T5); - T5 = _mm_mul_epu32(H3, p->R21.v); T4 = _mm_add_epi64(T4, T5); - T5 = _mm_mul_epu32(H4, p->R20.v); T4 = _mm_add_epi64(T4, T5); - - C1 = _mm_srli_epi64(T0, 26); C2 = _mm_srli_epi64(T3, 26); T0 = _mm_and_si128(T0, MMASK); T3 = _mm_and_si128(T3, MMASK); T1 = _mm_add_epi64(T1, C1); T4 = _mm_add_epi64(T4, C2); - C1 = _mm_srli_epi64(T1, 26); C2 = _mm_srli_epi64(T4, 26); T1 = _mm_and_si128(T1, MMASK); T4 = _mm_and_si128(T4, MMASK); T2 = _mm_add_epi64(T2, C1); T0 = _mm_add_epi64(T0, _mm_mul_epu32(C2, FIVE)); - C1 = _mm_srli_epi64(T2, 26); C2 = _mm_srli_epi64(T0, 26); T2 = _mm_and_si128(T2, MMASK); T0 = _mm_and_si128(T0, MMASK); T3 = _mm_add_epi64(T3, C1); T1 = _mm_add_epi64(T1, C2); - C1 = _mm_srli_epi64(T3, 26); T3 = _mm_and_si128(T3, MMASK); T4 = _mm_add_epi64(T4, C1); - - /* H = H[0]+H[1] */ - H0 = _mm_add_epi64(T0, _mm_srli_si128(T0, 8)); - H1 = _mm_add_epi64(T1, _mm_srli_si128(T1, 8)); - H2 = _mm_add_epi64(T2, _mm_srli_si128(T2, 8)); - H3 = _mm_add_epi64(T3, _mm_srli_si128(T3, 8)); - H4 = _mm_add_epi64(T4, _mm_srli_si128(T4, 8)); - - t0 = _mm_cvtsi128_si32(H0) ; c = (t0 >> 26); t0 &= 0x3ffffff; - t1 = _mm_cvtsi128_si32(H1) + c; c = (t1 >> 26); t1 &= 0x3ffffff; - t2 = _mm_cvtsi128_si32(H2) + c; c = (t2 >> 26); t2 &= 0x3ffffff; - t3 = _mm_cvtsi128_si32(H3) + c; c = (t3 >> 26); t3 &= 0x3ffffff; - t4 = _mm_cvtsi128_si32(H4) + c; c = (t4 >> 26); t4 &= 0x3ffffff; - t0 = t0 + (c * 5); c = (t0 >> 26); t0 &= 0x3ffffff; - t1 = t1 + c; - - st->HH[0] = ((t0 ) | (t1 << 26) ) & 0xfffffffffffull; - st->HH[1] = ((t1 >> 18) | (t2 << 8) | (t3 << 34)) & 0xfffffffffffull; - st->HH[2] = ((t3 >> 10) | (t4 << 16) ) & 0x3ffffffffffull; - - return consumed; -} - -constexpr size_t Poly1305Vec::KEY_SIZE; -constexpr size_t Poly1305Vec::MAC_SIZE; - -void Poly1305Vec::SetKey(const ui8* key, size_t size) -{ + if (bytes & 8) { *(uint64_t *)dst = 0; dst += 8; } +} + +static size_t INLINE poly1305_min(size_t a, size_t b) +{ + return (a < b) ? a : b; +} + +static void +poly1305_first_block(poly1305_state_internal *st, const uint8_t *m) +{ + const xmmi MMASK = + _mm_load_si128((xmmi *)poly1305_x64_sse2_message_mask); + const xmmi FIVE = _mm_load_si128((xmmi*)poly1305_x64_sse2_5); + const xmmi HIBIT = _mm_load_si128((xmmi*)poly1305_x64_sse2_1shl128); + xmmi T5,T6; + poly1305_power *p; + uint128_t d[3]; + uint64_t r0,r1,r2; + uint64_t r20,r21,r22,s22; + uint64_t pad0,pad1; + uint64_t c; + uint64_t i; + + /* pull out stored info */ + p = &st->P[1]; + + r0 = ((uint64_t)p->R20.d[3] << 32) | (uint64_t)p->R20.d[1]; + r1 = ((uint64_t)p->R21.d[3] << 32) | (uint64_t)p->R21.d[1]; + r2 = ((uint64_t)p->R22.d[3] << 32) | (uint64_t)p->R22.d[1]; + pad0 = ((uint64_t)p->R23.d[3] << 32) | (uint64_t)p->R23.d[1]; + pad1 = ((uint64_t)p->R24.d[3] << 32) | (uint64_t)p->R24.d[1]; + + /* compute powers r^2,r^4 */ + r20 = r0; + r21 = r1; + r22 = r2; + for (i = 0; i < 2; i++) { + s22 = r22 * (5 << 2); + + d[0] = add128(mul64x64_128(r20, r20), mul64x64_128(r21 * 2, s22)); + d[1] = add128(mul64x64_128(r22, s22), mul64x64_128(r20 * 2, r21)); + d[2] = add128(mul64x64_128(r21, r21), mul64x64_128(r22 * 2, r20)); + + r20 = lo128(d[0]) & 0xfffffffffff; c = shr128(d[0], 44); + d[1] = add128_64(d[1], c); r21 = lo128(d[1]) & 0xfffffffffff; c = shr128(d[1], 44); + d[2] = add128_64(d[2], c); r22 = lo128(d[2]) & 0x3ffffffffff; c = shr128(d[2], 42); + r20 += c * 5; c = (r20 >> 44); r20 = r20 & 0xfffffffffff; + r21 += c; + + p->R20.v = _mm_shuffle_epi32(_mm_cvtsi32_si128((uint32_t)( r20 ) & 0x3ffffff), _MM_SHUFFLE(1,0,1,0)); + p->R21.v = _mm_shuffle_epi32(_mm_cvtsi32_si128((uint32_t)((r20 >> 26) | (r21 << 18)) & 0x3ffffff), _MM_SHUFFLE(1,0,1,0)); + p->R22.v = _mm_shuffle_epi32(_mm_cvtsi32_si128((uint32_t)((r21 >> 8) ) & 0x3ffffff), _MM_SHUFFLE(1,0,1,0)); + p->R23.v = _mm_shuffle_epi32(_mm_cvtsi32_si128((uint32_t)((r21 >> 34) | (r22 << 10)) & 0x3ffffff), _MM_SHUFFLE(1,0,1,0)); + p->R24.v = _mm_shuffle_epi32(_mm_cvtsi32_si128((uint32_t)((r22 >> 16) ) ), _MM_SHUFFLE(1,0,1,0)); + p->S21.v = _mm_mul_epu32(p->R21.v, FIVE); + p->S22.v = _mm_mul_epu32(p->R22.v, FIVE); + p->S23.v = _mm_mul_epu32(p->R23.v, FIVE); + p->S24.v = _mm_mul_epu32(p->R24.v, FIVE); + p--; + } + + /* put saved info back */ + p = &st->P[1]; + p->R20.d[1] = (uint32_t)(r0 ); + p->R20.d[3] = (uint32_t)(r0 >> 32); + p->R21.d[1] = (uint32_t)(r1 ); + p->R21.d[3] = (uint32_t)(r1 >> 32); + p->R22.d[1] = (uint32_t)(r2 ); + p->R22.d[3] = (uint32_t)(r2 >> 32); + p->R23.d[1] = (uint32_t)(pad0 ); + p->R23.d[3] = (uint32_t)(pad0 >> 32); + p->R24.d[1] = (uint32_t)(pad1 ); + p->R24.d[3] = (uint32_t)(pad1 >> 32); + + /* H = [Mx,My] */ + T5 = _mm_unpacklo_epi64(_mm_loadl_epi64((xmmi *)(m + 0)), _mm_loadl_epi64((xmmi *)(m + 16))); + T6 = _mm_unpacklo_epi64(_mm_loadl_epi64((xmmi *)(m + 8)), _mm_loadl_epi64((xmmi *)(m + 24))); + st->H[0] = _mm_and_si128(MMASK, T5); + st->H[1] = _mm_and_si128(MMASK, _mm_srli_epi64(T5, 26)); + T5 = _mm_or_si128(_mm_srli_epi64(T5, 52), _mm_slli_epi64(T6, 12)); + st->H[2] = _mm_and_si128(MMASK, T5); + st->H[3] = _mm_and_si128(MMASK, _mm_srli_epi64(T5, 26)); + st->H[4] = _mm_or_si128(_mm_srli_epi64(T6, 40), HIBIT); +} + +Y_NO_SANITIZE("memory") static void +poly1305_blocks(poly1305_state_internal *st, const uint8_t *m, size_t bytes) +{ + const xmmi MMASK = _mm_load_si128((xmmi *)poly1305_x64_sse2_message_mask); + const xmmi FIVE = _mm_load_si128((xmmi*)poly1305_x64_sse2_5); + const xmmi HIBIT = _mm_load_si128((xmmi*)poly1305_x64_sse2_1shl128); + + poly1305_power *p; + xmmi H0,H1,H2,H3,H4; + xmmi T0,T1,T2,T3,T4,T5,T6; + xmmi M0,M1,M2,M3,M4; + xmmi C1,C2; + + H0 = st->H[0]; + H1 = st->H[1]; + H2 = st->H[2]; + H3 = st->H[3]; + H4 = st->H[4]; + + while (bytes >= 64) { + /* H *= [r^4,r^4] */ + p = &st->P[0]; + T0 = _mm_mul_epu32(H0, p->R20.v); + T1 = _mm_mul_epu32(H0, p->R21.v); + T2 = _mm_mul_epu32(H0, p->R22.v); + T3 = _mm_mul_epu32(H0, p->R23.v); + T4 = _mm_mul_epu32(H0, p->R24.v); + T5 = _mm_mul_epu32(H1, p->S24.v); T6 = _mm_mul_epu32(H1, p->R20.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); + T5 = _mm_mul_epu32(H2, p->S23.v); T6 = _mm_mul_epu32(H2, p->S24.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); + T5 = _mm_mul_epu32(H3, p->S22.v); T6 = _mm_mul_epu32(H3, p->S23.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); + T5 = _mm_mul_epu32(H4, p->S21.v); T6 = _mm_mul_epu32(H4, p->S22.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); + T5 = _mm_mul_epu32(H1, p->R21.v); T6 = _mm_mul_epu32(H1, p->R22.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); + T5 = _mm_mul_epu32(H2, p->R20.v); T6 = _mm_mul_epu32(H2, p->R21.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); + T5 = _mm_mul_epu32(H3, p->S24.v); T6 = _mm_mul_epu32(H3, p->R20.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); + T5 = _mm_mul_epu32(H4, p->S23.v); T6 = _mm_mul_epu32(H4, p->S24.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); + T5 = _mm_mul_epu32(H1, p->R23.v); T4 = _mm_add_epi64(T4, T5); + T5 = _mm_mul_epu32(H2, p->R22.v); T4 = _mm_add_epi64(T4, T5); + T5 = _mm_mul_epu32(H3, p->R21.v); T4 = _mm_add_epi64(T4, T5); + T5 = _mm_mul_epu32(H4, p->R20.v); T4 = _mm_add_epi64(T4, T5); + + /* H += [Mx,My]*[r^2,r^2] */ + T5 = _mm_unpacklo_epi64(_mm_loadl_epi64((xmmi *)(m + 0)), _mm_loadl_epi64((xmmi *)(m + 16))); + T6 = _mm_unpacklo_epi64(_mm_loadl_epi64((xmmi *)(m + 8)), _mm_loadl_epi64((xmmi *)(m + 24))); + M0 = _mm_and_si128(MMASK, T5); + M1 = _mm_and_si128(MMASK, _mm_srli_epi64(T5, 26)); + T5 = _mm_or_si128(_mm_srli_epi64(T5, 52), _mm_slli_epi64(T6, 12)); + M2 = _mm_and_si128(MMASK, T5); + M3 = _mm_and_si128(MMASK, _mm_srli_epi64(T5, 26)); + M4 = _mm_or_si128(_mm_srli_epi64(T6, 40), HIBIT); + + p = &st->P[1]; + T5 = _mm_mul_epu32(M0, p->R20.v); T6 = _mm_mul_epu32(M0, p->R21.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); + T5 = _mm_mul_epu32(M1, p->S24.v); T6 = _mm_mul_epu32(M1, p->R20.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); + T5 = _mm_mul_epu32(M2, p->S23.v); T6 = _mm_mul_epu32(M2, p->S24.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); + T5 = _mm_mul_epu32(M3, p->S22.v); T6 = _mm_mul_epu32(M3, p->S23.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); + T5 = _mm_mul_epu32(M4, p->S21.v); T6 = _mm_mul_epu32(M4, p->S22.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); + T5 = _mm_mul_epu32(M0, p->R22.v); T6 = _mm_mul_epu32(M0, p->R23.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); + T5 = _mm_mul_epu32(M1, p->R21.v); T6 = _mm_mul_epu32(M1, p->R22.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); + T5 = _mm_mul_epu32(M2, p->R20.v); T6 = _mm_mul_epu32(M2, p->R21.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); + T5 = _mm_mul_epu32(M3, p->S24.v); T6 = _mm_mul_epu32(M3, p->R20.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); + T5 = _mm_mul_epu32(M4, p->S23.v); T6 = _mm_mul_epu32(M4, p->S24.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); + T5 = _mm_mul_epu32(M0, p->R24.v); T4 = _mm_add_epi64(T4, T5); + T5 = _mm_mul_epu32(M1, p->R23.v); T4 = _mm_add_epi64(T4, T5); + T5 = _mm_mul_epu32(M2, p->R22.v); T4 = _mm_add_epi64(T4, T5); + T5 = _mm_mul_epu32(M3, p->R21.v); T4 = _mm_add_epi64(T4, T5); + T5 = _mm_mul_epu32(M4, p->R20.v); T4 = _mm_add_epi64(T4, T5); + + /* H += [Mx,My] */ + T5 = _mm_unpacklo_epi64(_mm_loadl_epi64((xmmi *)(m + 32)), _mm_loadl_epi64((xmmi *)(m + 48))); + T6 = _mm_unpacklo_epi64(_mm_loadl_epi64((xmmi *)(m + 40)), _mm_loadl_epi64((xmmi *)(m + 56))); + M0 = _mm_and_si128(MMASK, T5); + M1 = _mm_and_si128(MMASK, _mm_srli_epi64(T5, 26)); + T5 = _mm_or_si128(_mm_srli_epi64(T5, 52), _mm_slli_epi64(T6, 12)); + M2 = _mm_and_si128(MMASK, T5); + M3 = _mm_and_si128(MMASK, _mm_srli_epi64(T5, 26)); + M4 = _mm_or_si128(_mm_srli_epi64(T6, 40), HIBIT); + + T0 = _mm_add_epi64(T0, M0); + T1 = _mm_add_epi64(T1, M1); + T2 = _mm_add_epi64(T2, M2); + T3 = _mm_add_epi64(T3, M3); + T4 = _mm_add_epi64(T4, M4); + + /* reduce */ + C1 = _mm_srli_epi64(T0, 26); C2 = _mm_srli_epi64(T3, 26); T0 = _mm_and_si128(T0, MMASK); T3 = _mm_and_si128(T3, MMASK); T1 = _mm_add_epi64(T1, C1); T4 = _mm_add_epi64(T4, C2); + C1 = _mm_srli_epi64(T1, 26); C2 = _mm_srli_epi64(T4, 26); T1 = _mm_and_si128(T1, MMASK); T4 = _mm_and_si128(T4, MMASK); T2 = _mm_add_epi64(T2, C1); T0 = _mm_add_epi64(T0, _mm_mul_epu32(C2, FIVE)); + C1 = _mm_srli_epi64(T2, 26); C2 = _mm_srli_epi64(T0, 26); T2 = _mm_and_si128(T2, MMASK); T0 = _mm_and_si128(T0, MMASK); T3 = _mm_add_epi64(T3, C1); T1 = _mm_add_epi64(T1, C2); + C1 = _mm_srli_epi64(T3, 26); T3 = _mm_and_si128(T3, MMASK); T4 = _mm_add_epi64(T4, C1); + + /* H = (H*[r^4,r^4] + [Mx,My]*[r^2,r^2] + [Mx,My]) */ + H0 = T0; + H1 = T1; + H2 = T2; + H3 = T3; + H4 = T4; + + m += 64; + bytes -= 64; + } + + st->H[0] = H0; + st->H[1] = H1; + st->H[2] = H2; + st->H[3] = H3; + st->H[4] = H4; +} + +static size_t +poly1305_combine(poly1305_state_internal *st, const uint8_t *m, size_t bytes) +{ + const xmmi MMASK = + _mm_load_si128((xmmi *)poly1305_x64_sse2_message_mask); + const xmmi HIBIT = _mm_load_si128((xmmi*)poly1305_x64_sse2_1shl128); + const xmmi FIVE = _mm_load_si128((xmmi*)poly1305_x64_sse2_5); + + poly1305_power *p; + xmmi H0,H1,H2,H3,H4; + xmmi M0,M1,M2,M3,M4; + xmmi T0,T1,T2,T3,T4,T5,T6; + xmmi C1,C2; + + uint64_t r0,r1,r2; + uint64_t t0,t1,t2,t3,t4; + uint64_t c; + size_t consumed = 0; + + H0 = st->H[0]; + H1 = st->H[1]; + H2 = st->H[2]; + H3 = st->H[3]; + H4 = st->H[4]; + + /* p = [r^2,r^2] */ + p = &st->P[1]; + + if (bytes >= 32) { + /* H *= [r^2,r^2] */ + T0 = _mm_mul_epu32(H0, p->R20.v); + T1 = _mm_mul_epu32(H0, p->R21.v); + T2 = _mm_mul_epu32(H0, p->R22.v); + T3 = _mm_mul_epu32(H0, p->R23.v); + T4 = _mm_mul_epu32(H0, p->R24.v); + T5 = _mm_mul_epu32(H1, p->S24.v); T6 = _mm_mul_epu32(H1, p->R20.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); + T5 = _mm_mul_epu32(H2, p->S23.v); T6 = _mm_mul_epu32(H2, p->S24.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); + T5 = _mm_mul_epu32(H3, p->S22.v); T6 = _mm_mul_epu32(H3, p->S23.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); + T5 = _mm_mul_epu32(H4, p->S21.v); T6 = _mm_mul_epu32(H4, p->S22.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); + T5 = _mm_mul_epu32(H1, p->R21.v); T6 = _mm_mul_epu32(H1, p->R22.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); + T5 = _mm_mul_epu32(H2, p->R20.v); T6 = _mm_mul_epu32(H2, p->R21.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); + T5 = _mm_mul_epu32(H3, p->S24.v); T6 = _mm_mul_epu32(H3, p->R20.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); + T5 = _mm_mul_epu32(H4, p->S23.v); T6 = _mm_mul_epu32(H4, p->S24.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); + T5 = _mm_mul_epu32(H1, p->R23.v); T4 = _mm_add_epi64(T4, T5); + T5 = _mm_mul_epu32(H2, p->R22.v); T4 = _mm_add_epi64(T4, T5); + T5 = _mm_mul_epu32(H3, p->R21.v); T4 = _mm_add_epi64(T4, T5); + T5 = _mm_mul_epu32(H4, p->R20.v); T4 = _mm_add_epi64(T4, T5); + + /* H += [Mx,My] */ + T5 = _mm_unpacklo_epi64(_mm_loadl_epi64((xmmi *)(m + 0)), _mm_loadl_epi64((xmmi *)(m + 16))); + T6 = _mm_unpacklo_epi64(_mm_loadl_epi64((xmmi *)(m + 8)), _mm_loadl_epi64((xmmi *)(m + 24))); + M0 = _mm_and_si128(MMASK, T5); + M1 = _mm_and_si128(MMASK, _mm_srli_epi64(T5, 26)); + T5 = _mm_or_si128(_mm_srli_epi64(T5, 52), _mm_slli_epi64(T6, 12)); + M2 = _mm_and_si128(MMASK, T5); + M3 = _mm_and_si128(MMASK, _mm_srli_epi64(T5, 26)); + M4 = _mm_or_si128(_mm_srli_epi64(T6, 40), HIBIT); + + T0 = _mm_add_epi64(T0, M0); + T1 = _mm_add_epi64(T1, M1); + T2 = _mm_add_epi64(T2, M2); + T3 = _mm_add_epi64(T3, M3); + T4 = _mm_add_epi64(T4, M4); + + /* reduce */ + C1 = _mm_srli_epi64(T0, 26); C2 = _mm_srli_epi64(T3, 26); T0 = _mm_and_si128(T0, MMASK); T3 = _mm_and_si128(T3, MMASK); T1 = _mm_add_epi64(T1, C1); T4 = _mm_add_epi64(T4, C2); + C1 = _mm_srli_epi64(T1, 26); C2 = _mm_srli_epi64(T4, 26); T1 = _mm_and_si128(T1, MMASK); T4 = _mm_and_si128(T4, MMASK); T2 = _mm_add_epi64(T2, C1); T0 = _mm_add_epi64(T0, _mm_mul_epu32(C2, FIVE)); + C1 = _mm_srli_epi64(T2, 26); C2 = _mm_srli_epi64(T0, 26); T2 = _mm_and_si128(T2, MMASK); T0 = _mm_and_si128(T0, MMASK); T3 = _mm_add_epi64(T3, C1); T1 = _mm_add_epi64(T1, C2); + C1 = _mm_srli_epi64(T3, 26); T3 = _mm_and_si128(T3, MMASK); T4 = _mm_add_epi64(T4, C1); + + /* H = (H*[r^2,r^2] + [Mx,My]) */ + H0 = T0; + H1 = T1; + H2 = T2; + H3 = T3; + H4 = T4; + + consumed = 32; + } + + /* finalize, H *= [r^2,r] */ + r0 = ((uint64_t)p->R20.d[3] << 32) | (uint64_t)p->R20.d[1]; + r1 = ((uint64_t)p->R21.d[3] << 32) | (uint64_t)p->R21.d[1]; + r2 = ((uint64_t)p->R22.d[3] << 32) | (uint64_t)p->R22.d[1]; + + p->R20.d[2] = (uint32_t)( r0 ) & 0x3ffffff; + p->R21.d[2] = (uint32_t)((r0 >> 26) | (r1 << 18)) & 0x3ffffff; + p->R22.d[2] = (uint32_t)((r1 >> 8) ) & 0x3ffffff; + p->R23.d[2] = (uint32_t)((r1 >> 34) | (r2 << 10)) & 0x3ffffff; + p->R24.d[2] = (uint32_t)((r2 >> 16) ) ; + p->S21.d[2] = p->R21.d[2] * 5; + p->S22.d[2] = p->R22.d[2] * 5; + p->S23.d[2] = p->R23.d[2] * 5; + p->S24.d[2] = p->R24.d[2] * 5; + + /* H *= [r^2,r] */ + T0 = _mm_mul_epu32(H0, p->R20.v); + T1 = _mm_mul_epu32(H0, p->R21.v); + T2 = _mm_mul_epu32(H0, p->R22.v); + T3 = _mm_mul_epu32(H0, p->R23.v); + T4 = _mm_mul_epu32(H0, p->R24.v); + T5 = _mm_mul_epu32(H1, p->S24.v); T6 = _mm_mul_epu32(H1, p->R20.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); + T5 = _mm_mul_epu32(H2, p->S23.v); T6 = _mm_mul_epu32(H2, p->S24.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); + T5 = _mm_mul_epu32(H3, p->S22.v); T6 = _mm_mul_epu32(H3, p->S23.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); + T5 = _mm_mul_epu32(H4, p->S21.v); T6 = _mm_mul_epu32(H4, p->S22.v); T0 = _mm_add_epi64(T0, T5); T1 = _mm_add_epi64(T1, T6); + T5 = _mm_mul_epu32(H1, p->R21.v); T6 = _mm_mul_epu32(H1, p->R22.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); + T5 = _mm_mul_epu32(H2, p->R20.v); T6 = _mm_mul_epu32(H2, p->R21.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); + T5 = _mm_mul_epu32(H3, p->S24.v); T6 = _mm_mul_epu32(H3, p->R20.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); + T5 = _mm_mul_epu32(H4, p->S23.v); T6 = _mm_mul_epu32(H4, p->S24.v); T2 = _mm_add_epi64(T2, T5); T3 = _mm_add_epi64(T3, T6); + T5 = _mm_mul_epu32(H1, p->R23.v); T4 = _mm_add_epi64(T4, T5); + T5 = _mm_mul_epu32(H2, p->R22.v); T4 = _mm_add_epi64(T4, T5); + T5 = _mm_mul_epu32(H3, p->R21.v); T4 = _mm_add_epi64(T4, T5); + T5 = _mm_mul_epu32(H4, p->R20.v); T4 = _mm_add_epi64(T4, T5); + + C1 = _mm_srli_epi64(T0, 26); C2 = _mm_srli_epi64(T3, 26); T0 = _mm_and_si128(T0, MMASK); T3 = _mm_and_si128(T3, MMASK); T1 = _mm_add_epi64(T1, C1); T4 = _mm_add_epi64(T4, C2); + C1 = _mm_srli_epi64(T1, 26); C2 = _mm_srli_epi64(T4, 26); T1 = _mm_and_si128(T1, MMASK); T4 = _mm_and_si128(T4, MMASK); T2 = _mm_add_epi64(T2, C1); T0 = _mm_add_epi64(T0, _mm_mul_epu32(C2, FIVE)); + C1 = _mm_srli_epi64(T2, 26); C2 = _mm_srli_epi64(T0, 26); T2 = _mm_and_si128(T2, MMASK); T0 = _mm_and_si128(T0, MMASK); T3 = _mm_add_epi64(T3, C1); T1 = _mm_add_epi64(T1, C2); + C1 = _mm_srli_epi64(T3, 26); T3 = _mm_and_si128(T3, MMASK); T4 = _mm_add_epi64(T4, C1); + + /* H = H[0]+H[1] */ + H0 = _mm_add_epi64(T0, _mm_srli_si128(T0, 8)); + H1 = _mm_add_epi64(T1, _mm_srli_si128(T1, 8)); + H2 = _mm_add_epi64(T2, _mm_srli_si128(T2, 8)); + H3 = _mm_add_epi64(T3, _mm_srli_si128(T3, 8)); + H4 = _mm_add_epi64(T4, _mm_srli_si128(T4, 8)); + + t0 = _mm_cvtsi128_si32(H0) ; c = (t0 >> 26); t0 &= 0x3ffffff; + t1 = _mm_cvtsi128_si32(H1) + c; c = (t1 >> 26); t1 &= 0x3ffffff; + t2 = _mm_cvtsi128_si32(H2) + c; c = (t2 >> 26); t2 &= 0x3ffffff; + t3 = _mm_cvtsi128_si32(H3) + c; c = (t3 >> 26); t3 &= 0x3ffffff; + t4 = _mm_cvtsi128_si32(H4) + c; c = (t4 >> 26); t4 &= 0x3ffffff; + t0 = t0 + (c * 5); c = (t0 >> 26); t0 &= 0x3ffffff; + t1 = t1 + c; + + st->HH[0] = ((t0 ) | (t1 << 26) ) & 0xfffffffffffull; + st->HH[1] = ((t1 >> 18) | (t2 << 8) | (t3 << 34)) & 0xfffffffffffull; + st->HH[2] = ((t3 >> 10) | (t4 << 16) ) & 0x3ffffffffffull; + + return consumed; +} + +constexpr size_t Poly1305Vec::KEY_SIZE; +constexpr size_t Poly1305Vec::MAC_SIZE; + +void Poly1305Vec::SetKey(const ui8* key, size_t size) +{ Y_ASSERT((size == KEY_SIZE) && "key must be 32 bytes long"); - - poly1305_state_internal *st = poly1305_aligned_state(&state); - poly1305_power *p; - uint64_t r0,r1,r2; - uint64_t t0,t1; - - /* clamp key */ - t0 = U8TO64_LE(key + 0); - t1 = U8TO64_LE(key + 8); - r0 = t0 & 0xffc0fffffff; t0 >>= 44; t0 |= t1 << 20; - r1 = t0 & 0xfffffc0ffff; t1 >>= 24; - r2 = t1 & 0x00ffffffc0f; - - /* store r in un-used space of st->P[1] */ - p = &st->P[1]; - p->R20.d[1] = (uint32_t)(r0 ); - p->R20.d[3] = (uint32_t)(r0 >> 32); - p->R21.d[1] = (uint32_t)(r1 ); - p->R21.d[3] = (uint32_t)(r1 >> 32); - p->R22.d[1] = (uint32_t)(r2 ); - p->R22.d[3] = (uint32_t)(r2 >> 32); - - /* store pad */ - p->R23.d[1] = U8TO32_LE(key + 16); - p->R23.d[3] = U8TO32_LE(key + 20); - p->R24.d[1] = U8TO32_LE(key + 24); - p->R24.d[3] = U8TO32_LE(key + 28); - - /* H = 0 */ - st->H[0] = _mm_setzero_si128(); - st->H[1] = _mm_setzero_si128(); - st->H[2] = _mm_setzero_si128(); - st->H[3] = _mm_setzero_si128(); - st->H[4] = _mm_setzero_si128(); - - st->started = 0; - st->leftover = 0; -} - -void Poly1305Vec::Update(const ui8* msg, size_t size) -{ - poly1305_state_internal *st = poly1305_aligned_state(&state); - size_t want; - - /* need at least 32 initial bytes to start the accelerated branch */ - if (!st->started) { - if ((st->leftover == 0) && (size > 32)) { - poly1305_first_block(st, msg); - msg += 32; - size -= 32; - } else { - want = poly1305_min(32 - st->leftover, size); - poly1305_block_copy(st->buffer + st->leftover, msg, want); - size -= want; - msg += want; - st->leftover += want; - if ((st->leftover < 32) || (size == 0)) - return; - poly1305_first_block(st, st->buffer); - st->leftover = 0; - } - st->started = 1; - } - - /* handle leftover */ - if (st->leftover) { - want = poly1305_min(64 - st->leftover, size); - poly1305_block_copy(st->buffer + st->leftover, msg, want); - size -= want; - msg += want; - st->leftover += want; - if (st->leftover < 64) - return; - poly1305_blocks(st, st->buffer, 64); - st->leftover = 0; - } - - /* process 64 byte blocks */ - if (size >= 64) { - want = (size & ~63); - poly1305_blocks(st, msg, want); - msg += want; - size -= want; - } - - if (size) { - poly1305_block_copy(st->buffer + st->leftover, msg, size); - st->leftover += size; - } -} - -void Poly1305Vec::Finish(ui8 mac[MAC_SIZE]) -{ - poly1305_state_internal *st = poly1305_aligned_state(&state); - size_t leftover = st->leftover; - uint8_t *m = st->buffer; - uint128_t d[3]; - uint64_t h0,h1,h2; - uint64_t t0,t1; - uint64_t g0,g1,g2,c,nc; - uint64_t r0,r1,r2,s1,s2; - poly1305_power *p; - - if (st->started) { - size_t consumed = poly1305_combine(st, m, leftover); - leftover -= consumed; - m += consumed; - } - - /* st->HH will either be 0 or have the combined result */ - h0 = st->HH[0]; - h1 = st->HH[1]; - h2 = st->HH[2]; - - p = &st->P[1]; - r0 = ((uint64_t)p->R20.d[3] << 32) | (uint64_t)p->R20.d[1]; - r1 = ((uint64_t)p->R21.d[3] << 32) | (uint64_t)p->R21.d[1]; - r2 = ((uint64_t)p->R22.d[3] << 32) | (uint64_t)p->R22.d[1]; - s1 = r1 * (5 << 2); - s2 = r2 * (5 << 2); - - if (leftover < 16) - goto poly1305_donna_atmost15bytes; - -poly1305_donna_atleast16bytes: - t0 = U8TO64_LE(m + 0); - t1 = U8TO64_LE(m + 8); - h0 += t0 & 0xfffffffffff; - t0 = shr128_pair(t1, t0, 44); - h1 += t0 & 0xfffffffffff; - h2 += (t1 >> 24) | ((uint64_t)1 << 40); - -poly1305_donna_mul: - d[0] = add128(add128(mul64x64_128(h0, r0), mul64x64_128(h1, s2)), mul64x64_128(h2, s1)); - d[1] = add128(add128(mul64x64_128(h0, r1), mul64x64_128(h1, r0)), mul64x64_128(h2, s2)); - d[2] = add128(add128(mul64x64_128(h0, r2), mul64x64_128(h1, r1)), mul64x64_128(h2, r0)); - h0 = lo128(d[0]) & 0xfffffffffff; c = shr128(d[0], 44); - d[1] = add128_64(d[1], c); h1 = lo128(d[1]) & 0xfffffffffff; c = shr128(d[1], 44); - d[2] = add128_64(d[2], c); h2 = lo128(d[2]) & 0x3ffffffffff; c = shr128(d[2], 42); - h0 += c * 5; - - m += 16; - leftover -= 16; - if (leftover >= 16) goto poly1305_donna_atleast16bytes; - - /* final bytes */ -poly1305_donna_atmost15bytes: - if (!leftover) goto poly1305_donna_finish; - - m[leftover++] = 1; - poly1305_block_zero(m + leftover, 16 - leftover); - leftover = 16; - - t0 = U8TO64_LE(m+0); - t1 = U8TO64_LE(m+8); - h0 += t0 & 0xfffffffffff; t0 = shr128_pair(t1, t0, 44); - h1 += t0 & 0xfffffffffff; - h2 += (t1 >> 24); - - goto poly1305_donna_mul; - -poly1305_donna_finish: - c = (h0 >> 44); h0 &= 0xfffffffffff; - h1 += c; c = (h1 >> 44); h1 &= 0xfffffffffff; - h2 += c; c = (h2 >> 42); h2 &= 0x3ffffffffff; - h0 += c * 5; - - g0 = h0 + 5; c = (g0 >> 44); g0 &= 0xfffffffffff; - g1 = h1 + c; c = (g1 >> 44); g1 &= 0xfffffffffff; - g2 = h2 + c - ((uint64_t)1 << 42); - - c = (g2 >> 63) - 1; - nc = ~c; - h0 = (h0 & nc) | (g0 & c); - h1 = (h1 & nc) | (g1 & c); - h2 = (h2 & nc) | (g2 & c); - - /* pad */ - t0 = ((uint64_t)p->R23.d[3] << 32) | (uint64_t)p->R23.d[1]; - t1 = ((uint64_t)p->R24.d[3] << 32) | (uint64_t)p->R24.d[1]; - h0 += (t0 & 0xfffffffffff) ; c = (h0 >> 44); h0 &= 0xfffffffffff; t0 = shr128_pair(t1, t0, 44); - h1 += (t0 & 0xfffffffffff) + c; c = (h1 >> 44); h1 &= 0xfffffffffff; t1 = (t1 >> 24); - h2 += (t1 ) + c; - - U64TO8_LE(mac + 0, ((h0 ) | (h1 << 44))); - U64TO8_LE(mac + 8, ((h1 >> 20) | (h2 << 24))); -} + + poly1305_state_internal *st = poly1305_aligned_state(&state); + poly1305_power *p; + uint64_t r0,r1,r2; + uint64_t t0,t1; + + /* clamp key */ + t0 = U8TO64_LE(key + 0); + t1 = U8TO64_LE(key + 8); + r0 = t0 & 0xffc0fffffff; t0 >>= 44; t0 |= t1 << 20; + r1 = t0 & 0xfffffc0ffff; t1 >>= 24; + r2 = t1 & 0x00ffffffc0f; + + /* store r in un-used space of st->P[1] */ + p = &st->P[1]; + p->R20.d[1] = (uint32_t)(r0 ); + p->R20.d[3] = (uint32_t)(r0 >> 32); + p->R21.d[1] = (uint32_t)(r1 ); + p->R21.d[3] = (uint32_t)(r1 >> 32); + p->R22.d[1] = (uint32_t)(r2 ); + p->R22.d[3] = (uint32_t)(r2 >> 32); + + /* store pad */ + p->R23.d[1] = U8TO32_LE(key + 16); + p->R23.d[3] = U8TO32_LE(key + 20); + p->R24.d[1] = U8TO32_LE(key + 24); + p->R24.d[3] = U8TO32_LE(key + 28); + + /* H = 0 */ + st->H[0] = _mm_setzero_si128(); + st->H[1] = _mm_setzero_si128(); + st->H[2] = _mm_setzero_si128(); + st->H[3] = _mm_setzero_si128(); + st->H[4] = _mm_setzero_si128(); + + st->started = 0; + st->leftover = 0; +} + +void Poly1305Vec::Update(const ui8* msg, size_t size) +{ + poly1305_state_internal *st = poly1305_aligned_state(&state); + size_t want; + + /* need at least 32 initial bytes to start the accelerated branch */ + if (!st->started) { + if ((st->leftover == 0) && (size > 32)) { + poly1305_first_block(st, msg); + msg += 32; + size -= 32; + } else { + want = poly1305_min(32 - st->leftover, size); + poly1305_block_copy(st->buffer + st->leftover, msg, want); + size -= want; + msg += want; + st->leftover += want; + if ((st->leftover < 32) || (size == 0)) + return; + poly1305_first_block(st, st->buffer); + st->leftover = 0; + } + st->started = 1; + } + + /* handle leftover */ + if (st->leftover) { + want = poly1305_min(64 - st->leftover, size); + poly1305_block_copy(st->buffer + st->leftover, msg, want); + size -= want; + msg += want; + st->leftover += want; + if (st->leftover < 64) + return; + poly1305_blocks(st, st->buffer, 64); + st->leftover = 0; + } + + /* process 64 byte blocks */ + if (size >= 64) { + want = (size & ~63); + poly1305_blocks(st, msg, want); + msg += want; + size -= want; + } + + if (size) { + poly1305_block_copy(st->buffer + st->leftover, msg, size); + st->leftover += size; + } +} + +void Poly1305Vec::Finish(ui8 mac[MAC_SIZE]) +{ + poly1305_state_internal *st = poly1305_aligned_state(&state); + size_t leftover = st->leftover; + uint8_t *m = st->buffer; + uint128_t d[3]; + uint64_t h0,h1,h2; + uint64_t t0,t1; + uint64_t g0,g1,g2,c,nc; + uint64_t r0,r1,r2,s1,s2; + poly1305_power *p; + + if (st->started) { + size_t consumed = poly1305_combine(st, m, leftover); + leftover -= consumed; + m += consumed; + } + + /* st->HH will either be 0 or have the combined result */ + h0 = st->HH[0]; + h1 = st->HH[1]; + h2 = st->HH[2]; + + p = &st->P[1]; + r0 = ((uint64_t)p->R20.d[3] << 32) | (uint64_t)p->R20.d[1]; + r1 = ((uint64_t)p->R21.d[3] << 32) | (uint64_t)p->R21.d[1]; + r2 = ((uint64_t)p->R22.d[3] << 32) | (uint64_t)p->R22.d[1]; + s1 = r1 * (5 << 2); + s2 = r2 * (5 << 2); + + if (leftover < 16) + goto poly1305_donna_atmost15bytes; + +poly1305_donna_atleast16bytes: + t0 = U8TO64_LE(m + 0); + t1 = U8TO64_LE(m + 8); + h0 += t0 & 0xfffffffffff; + t0 = shr128_pair(t1, t0, 44); + h1 += t0 & 0xfffffffffff; + h2 += (t1 >> 24) | ((uint64_t)1 << 40); + +poly1305_donna_mul: + d[0] = add128(add128(mul64x64_128(h0, r0), mul64x64_128(h1, s2)), mul64x64_128(h2, s1)); + d[1] = add128(add128(mul64x64_128(h0, r1), mul64x64_128(h1, r0)), mul64x64_128(h2, s2)); + d[2] = add128(add128(mul64x64_128(h0, r2), mul64x64_128(h1, r1)), mul64x64_128(h2, r0)); + h0 = lo128(d[0]) & 0xfffffffffff; c = shr128(d[0], 44); + d[1] = add128_64(d[1], c); h1 = lo128(d[1]) & 0xfffffffffff; c = shr128(d[1], 44); + d[2] = add128_64(d[2], c); h2 = lo128(d[2]) & 0x3ffffffffff; c = shr128(d[2], 42); + h0 += c * 5; + + m += 16; + leftover -= 16; + if (leftover >= 16) goto poly1305_donna_atleast16bytes; + + /* final bytes */ +poly1305_donna_atmost15bytes: + if (!leftover) goto poly1305_donna_finish; + + m[leftover++] = 1; + poly1305_block_zero(m + leftover, 16 - leftover); + leftover = 16; + + t0 = U8TO64_LE(m+0); + t1 = U8TO64_LE(m+8); + h0 += t0 & 0xfffffffffff; t0 = shr128_pair(t1, t0, 44); + h1 += t0 & 0xfffffffffff; + h2 += (t1 >> 24); + + goto poly1305_donna_mul; + +poly1305_donna_finish: + c = (h0 >> 44); h0 &= 0xfffffffffff; + h1 += c; c = (h1 >> 44); h1 &= 0xfffffffffff; + h2 += c; c = (h2 >> 42); h2 &= 0x3ffffffffff; + h0 += c * 5; + + g0 = h0 + 5; c = (g0 >> 44); g0 &= 0xfffffffffff; + g1 = h1 + c; c = (g1 >> 44); g1 &= 0xfffffffffff; + g2 = h2 + c - ((uint64_t)1 << 42); + + c = (g2 >> 63) - 1; + nc = ~c; + h0 = (h0 & nc) | (g0 & c); + h1 = (h1 & nc) | (g1 & c); + h2 = (h2 & nc) | (g2 & c); + + /* pad */ + t0 = ((uint64_t)p->R23.d[3] << 32) | (uint64_t)p->R23.d[1]; + t1 = ((uint64_t)p->R24.d[3] << 32) | (uint64_t)p->R24.d[1]; + h0 += (t0 & 0xfffffffffff) ; c = (h0 >> 44); h0 &= 0xfffffffffff; t0 = shr128_pair(t1, t0, 44); + h1 += (t0 & 0xfffffffffff) + c; c = (h1 >> 44); h1 &= 0xfffffffffff; t1 = (t1 >> 24); + h2 += (t1 ) + c; + + U64TO8_LE(mac + 0, ((h0 ) | (h1 << 44))); + U64TO8_LE(mac + 8, ((h1 >> 20) | (h2 << 24))); +} diff --git a/ydb/core/blobstorage/crypto/poly1305_vec.h b/ydb/core/blobstorage/crypto/poly1305_vec.h index ae6632e2a0e..c0b61b84fa2 100644 --- a/ydb/core/blobstorage/crypto/poly1305_vec.h +++ b/ydb/core/blobstorage/crypto/poly1305_vec.h @@ -1,21 +1,21 @@ -#pragma once - -#include <util/system/types.h> - - -using poly1305_state = ui8[512]; - -class Poly1305Vec -{ -public: - static constexpr size_t KEY_SIZE = 32; - static constexpr size_t MAC_SIZE = 16; - -public: - void SetKey(const ui8* key, size_t size); - void Update(const ui8* msg, size_t size); - void Finish(ui8 mac[MAC_SIZE]); - -private: - poly1305_state state; -}; +#pragma once + +#include <util/system/types.h> + + +using poly1305_state = ui8[512]; + +class Poly1305Vec +{ +public: + static constexpr size_t KEY_SIZE = 32; + static constexpr size_t MAC_SIZE = 16; + +public: + void SetKey(const ui8* key, size_t size); + void Update(const ui8* msg, size_t size); + void Finish(ui8 mac[MAC_SIZE]); + +private: + poly1305_state state; +}; diff --git a/ydb/core/blobstorage/crypto/poly1305_vec_ut.cpp b/ydb/core/blobstorage/crypto/poly1305_vec_ut.cpp index 7956340c208..491c49b4c7f 100644 --- a/ydb/core/blobstorage/crypto/poly1305_vec_ut.cpp +++ b/ydb/core/blobstorage/crypto/poly1305_vec_ut.cpp @@ -1,44 +1,44 @@ -#include "poly1305_vec.h" +#include "poly1305_vec.h" #include <ydb/core/blobstorage/crypto/ut/ut_helpers.h> #include <ydb/core/blobstorage/crypto/ut/poly1305_test_vectors.h> - - + + Y_UNIT_TEST_SUITE(TPoly1305Vec) -{ - inline void RunTest(const ui8* input, size_t input_size, - const ui8 key[32], const ui8 expectedMac[16]) - { - ui8 mac[16]; - - Poly1305Vec poly; - poly.SetKey(key, 32); - poly.Update(input, input_size); - poly.Finish(mac); - - UNIT_ASSERT_ARRAYS_EQUAL(mac, expectedMac, sizeof(mac)); - - // test that at the second time we get the same result - - poly.SetKey(key, 32); - poly.Update(input, input_size); - poly.Finish(mac); - - UNIT_ASSERT_ARRAYS_EQUAL(mac, expectedMac, sizeof(mac)); - } - +{ + inline void RunTest(const ui8* input, size_t input_size, + const ui8 key[32], const ui8 expectedMac[16]) + { + ui8 mac[16]; + + Poly1305Vec poly; + poly.SetKey(key, 32); + poly.Update(input, input_size); + poly.Finish(mac); + + UNIT_ASSERT_ARRAYS_EQUAL(mac, expectedMac, sizeof(mac)); + + // test that at the second time we get the same result + + poly.SetKey(key, 32); + poly.Update(input, input_size); + poly.Finish(mac); + + UNIT_ASSERT_ARRAYS_EQUAL(mac, expectedMac, sizeof(mac)); + } + Y_UNIT_TEST(TestVector1) { - RunTest(tc1_data, sizeof(tc1_data), tc1_key, tc1_tag); - } - + RunTest(tc1_data, sizeof(tc1_data), tc1_key, tc1_tag); + } + Y_UNIT_TEST(TestVector2) { - RunTest(tc2_data, sizeof(tc2_data) - 1, tc2_key, tc2_tag); - } - + RunTest(tc2_data, sizeof(tc2_data) - 1, tc2_key, tc2_tag); + } + Y_UNIT_TEST(TestVector3) { - RunTest(tc3_data, sizeof(tc3_data) - 1, tc3_key, tc3_tag); - } - + RunTest(tc3_data, sizeof(tc3_data) - 1, tc3_key, tc3_tag); + } + Y_UNIT_TEST(TestVector4) { - RunTest(tc4_data, sizeof(tc4_data) - 1, tc4_key, tc4_tag); - } -} + RunTest(tc4_data, sizeof(tc4_data) - 1, tc4_key, tc4_tag); + } +} diff --git a/ydb/core/blobstorage/crypto/secured_block.cpp b/ydb/core/blobstorage/crypto/secured_block.cpp index 1b273692197..5c52c4a4296 100644 --- a/ydb/core/blobstorage/crypto/secured_block.cpp +++ b/ydb/core/blobstorage/crypto/secured_block.cpp @@ -1,96 +1,96 @@ -#include "secured_block.h" - -#include <new> - - -#if defined(_MSC_VER) -# if _MSC_VER == 1200 -# include <malloc.h> -# endif -# if _MSC_VER > 1200 || defined(_mm_free) -# define MSVC6PP_OR_LATER // VC 6 processor pack or later -# else -# define MSVC6_NO_PP // VC 6 without processor pack -# endif -#endif - -// how to allocate 16-byte aligned memory (for SSE2) -#if defined(MSVC6PP_OR_LATER) -# define MM_MALLOC_AVAILABLE -#elif defined(__FreeBSD__) || defined(__NetBSD__) || defined(__OpenBSD__) -# define MALLOC_ALIGNMENT_IS_16 -#elif defined(__linux__) || defined(__sun__) || defined(__CYGWIN__) -# define MEMALIGN_AVAILABLE -# include <stdlib.h> -#else -# define NO_ALIGNED_ALLOC -#endif - - - -void CallNewHandler() -{ -#if !(defined(_MSC_VER) && (_MSC_VER < 1300)) - using std::new_handler; - using std::set_new_handler; -#endif - - new_handler newHandler = set_new_handler(nullptr); - if (newHandler) - set_new_handler(newHandler); - - if (newHandler) - newHandler(); - else - throw std::bad_alloc(); -} - -ui8* AlignedAllocate(size_t size) -{ - ui8* p; -#ifdef MM_MALLOC_AVAILABLE - while (!(p = (ui8*)_mm_malloc(size, 16))) -#elif defined(MEMALIGN_AVAILABLE) - while (posix_memalign((void**)&p, 16, size)) -#elif defined(MALLOC_ALIGNMENT_IS_16) - while (!(p = (ui8*)malloc(size))) -#else - while (!(p = (ui8*)malloc(size + 16))) -#endif - CallNewHandler(); - -#ifdef NO_ALIGNED_ALLOC - size_t adjustment = 16 - ((size_t)p % 16); - p += adjustment; - p[-1] = (ui8)adjustment; -#endif - +#include "secured_block.h" + +#include <new> + + +#if defined(_MSC_VER) +# if _MSC_VER == 1200 +# include <malloc.h> +# endif +# if _MSC_VER > 1200 || defined(_mm_free) +# define MSVC6PP_OR_LATER // VC 6 processor pack or later +# else +# define MSVC6_NO_PP // VC 6 without processor pack +# endif +#endif + +// how to allocate 16-byte aligned memory (for SSE2) +#if defined(MSVC6PP_OR_LATER) +# define MM_MALLOC_AVAILABLE +#elif defined(__FreeBSD__) || defined(__NetBSD__) || defined(__OpenBSD__) +# define MALLOC_ALIGNMENT_IS_16 +#elif defined(__linux__) || defined(__sun__) || defined(__CYGWIN__) +# define MEMALIGN_AVAILABLE +# include <stdlib.h> +#else +# define NO_ALIGNED_ALLOC +#endif + + + +void CallNewHandler() +{ +#if !(defined(_MSC_VER) && (_MSC_VER < 1300)) + using std::new_handler; + using std::set_new_handler; +#endif + + new_handler newHandler = set_new_handler(nullptr); + if (newHandler) + set_new_handler(newHandler); + + if (newHandler) + newHandler(); + else + throw std::bad_alloc(); +} + +ui8* AlignedAllocate(size_t size) +{ + ui8* p; +#ifdef MM_MALLOC_AVAILABLE + while (!(p = (ui8*)_mm_malloc(size, 16))) +#elif defined(MEMALIGN_AVAILABLE) + while (posix_memalign((void**)&p, 16, size)) +#elif defined(MALLOC_ALIGNMENT_IS_16) + while (!(p = (ui8*)malloc(size))) +#else + while (!(p = (ui8*)malloc(size + 16))) +#endif + CallNewHandler(); + +#ifdef NO_ALIGNED_ALLOC + size_t adjustment = 16 - ((size_t)p % 16); + p += adjustment; + p[-1] = (ui8)adjustment; +#endif + Y_ASSERT(IsAlignedOn(p, 16)); - return p; -} - -void AlignedDeallocate(ui8* p) -{ -#ifdef MM_MALLOC_AVAILABLE - _mm_free(p); -#elif defined(NO_ALIGNED_ALLOC) - p = (ui8*)p - ((ui8*)p)[-1]; - free(p); -#else - free(p); -#endif -} - -ui8* UnalignedAllocate(size_t size) -{ - ui8* p; - while (!(p = (ui8*)malloc(size))) - CallNewHandler(); - return p; -} - -void UnalignedDeallocate(ui8* p) -{ - free(p); -} - + return p; +} + +void AlignedDeallocate(ui8* p) +{ +#ifdef MM_MALLOC_AVAILABLE + _mm_free(p); +#elif defined(NO_ALIGNED_ALLOC) + p = (ui8*)p - ((ui8*)p)[-1]; + free(p); +#else + free(p); +#endif +} + +ui8* UnalignedAllocate(size_t size) +{ + ui8* p; + while (!(p = (ui8*)malloc(size))) + CallNewHandler(); + return p; +} + +void UnalignedDeallocate(ui8* p) +{ + free(p); +} + diff --git a/ydb/core/blobstorage/crypto/secured_block.h b/ydb/core/blobstorage/crypto/secured_block.h index f0717ba2526..9992d70933d 100644 --- a/ydb/core/blobstorage/crypto/secured_block.h +++ b/ydb/core/blobstorage/crypto/secured_block.h @@ -1,307 +1,307 @@ -#pragma once - -#include <util/generic/bitops.h> -#include <util/system/types.h> -#include <util/system/yassert.h> -#include <util/system/compiler.h> -#include <util/generic/utility.h> -#include <util/generic/strbuf.h> +#pragma once + +#include <util/generic/bitops.h> +#include <util/system/types.h> +#include <util/system/yassert.h> +#include <util/system/compiler.h> +#include <util/generic/utility.h> +#include <util/generic/strbuf.h> #include <util/generic/cast.h> - + #ifdef _win_ #include <windows.h> #endif - -template <class T1, class T2> -inline T2 ModPowerOf2(const T1& a, const T2& b) -{ + +template <class T1, class T2> +inline T2 ModPowerOf2(const T1& a, const T2& b) +{ Y_ASSERT(IsPowerOf2(b)); - return T2(a) & (b-1); -} - -inline bool IsAlignedOn(const void* p, unsigned int alignment) -{ - if (alignment==1 || IsPowerOf2(alignment)) - return ModPowerOf2((size_t)p, alignment) == 0; - - return (size_t)p % alignment == 0; -} - -ui8* AlignedAllocate(size_t size); -void AlignedDeallocate(ui8* p); -ui8* UnalignedAllocate(size_t size); -void UnalignedDeallocate(ui8* p); - -inline void SecureWipeBuffer(ui8* buf, size_t size) -{ -#ifdef _win_ + return T2(a) & (b-1); +} + +inline bool IsAlignedOn(const void* p, unsigned int alignment) +{ + if (alignment==1 || IsPowerOf2(alignment)) + return ModPowerOf2((size_t)p, alignment) == 0; + + return (size_t)p % alignment == 0; +} + +ui8* AlignedAllocate(size_t size); +void AlignedDeallocate(ui8* p); +ui8* UnalignedAllocate(size_t size); +void UnalignedDeallocate(ui8* p); + +inline void SecureWipeBuffer(ui8* buf, size_t size) +{ +#ifdef _win_ ::SecureZeroMemory(buf, size); -#else - ::memset(buf, 0, size); - - /* As best as we can tell, this is sufficient to break any optimisations that - might try to eliminate "superfluous" memsets. If there's an easy way to - detect memset_s, it would be better to use that. */ - __asm__ __volatile__("" : : "r"(buf) : "memory"); -#endif -} - -inline void SecureWipeBuffer(char* buf, size_t size) -{ +#else + ::memset(buf, 0, size); + + /* As best as we can tell, this is sufficient to break any optimisations that + might try to eliminate "superfluous" memsets. If there's an easy way to + detect memset_s, it would be better to use that. */ + __asm__ __volatile__("" : : "r"(buf) : "memory"); +#endif +} + +inline void SecureWipeBuffer(char* buf, size_t size) +{ SecureWipeBuffer(reinterpret_cast<ui8*>(buf), size); -} - -class TAlignedAllocator -{ -public: - ui8* Allocate(size_t nbytes) { - if (nbytes == 0) - return nullptr; - - if (nbytes >= 16) - return AlignedAllocate(nbytes); - - return UnalignedAllocate(nbytes); - } - - void Deallocate(ui8* ptr, size_t nbytes) { - SecureWipeBuffer(ptr, nbytes); - - if (nbytes >= 16) - return AlignedDeallocate(ptr); - - UnalignedDeallocate(ptr); - } - - ui8* Reallocate(ui8* ptr, size_t oldSize, size_t newSize, bool preserve) { - if (oldSize == newSize) - return ptr; - - if (preserve) { - ui8* newPtr = Allocate(newSize); - memcpy(newPtr, ptr, Min(oldSize, newSize)); - Deallocate(ptr, oldSize); - return newPtr; - } - - Deallocate(ptr, oldSize); - return Allocate(newSize); - } -}; - -template <size_t nbytes, size_t alignment = 8> -class TFixedAllocator -{ -public: - ui8* Allocate(size_t n) { +} + +class TAlignedAllocator +{ +public: + ui8* Allocate(size_t nbytes) { + if (nbytes == 0) + return nullptr; + + if (nbytes >= 16) + return AlignedAllocate(nbytes); + + return UnalignedAllocate(nbytes); + } + + void Deallocate(ui8* ptr, size_t nbytes) { + SecureWipeBuffer(ptr, nbytes); + + if (nbytes >= 16) + return AlignedDeallocate(ptr); + + UnalignedDeallocate(ptr); + } + + ui8* Reallocate(ui8* ptr, size_t oldSize, size_t newSize, bool preserve) { + if (oldSize == newSize) + return ptr; + + if (preserve) { + ui8* newPtr = Allocate(newSize); + memcpy(newPtr, ptr, Min(oldSize, newSize)); + Deallocate(ptr, oldSize); + return newPtr; + } + + Deallocate(ptr, oldSize); + return Allocate(newSize); + } +}; + +template <size_t nbytes, size_t alignment = 8> +class TFixedAllocator +{ +public: + ui8* Allocate(size_t n) { Y_UNUSED(n); - - // TODO: use static assert + + // TODO: use static assert Y_ASSERT(IsAlignedOn(Array_, alignment)); - return GetAlignedArray(); - } - - void Deallocate(ui8* ptr, size_t n) { + return GetAlignedArray(); + } + + void Deallocate(ui8* ptr, size_t n) { Y_UNUSED(n); Y_UNUSED(ptr); - + ui8 *p = GetAlignedArray(); SecureWipeBuffer(p, nbytes); - } - - ui8* Reallocate(ui8* ptr, size_t oldSize, size_t newSize, bool preserve) { + } + + ui8* Reallocate(ui8* ptr, size_t oldSize, size_t newSize, bool preserve) { Y_UNUSED(preserve); - - if (oldSize > newSize) - SecureWipeBuffer(ptr + newSize, oldSize - newSize); - return ptr; - } - -private: - ui8* GetAlignedArray() { - return (ui8*)(Array_ + (0 - (size_t)Array_) % 16); - } - + + if (oldSize > newSize) + SecureWipeBuffer(ptr + newSize, oldSize - newSize); + return ptr; + } + +private: + ui8* GetAlignedArray() { + return (ui8*)(Array_ + (0 - (size_t)Array_) % 16); + } + alignas(alignment) ui8 Array_[nbytes + alignment]; -}; - - -template <typename Allocator> -class TBaseBlock -{ -public: - typedef ui8 value_type; - typedef ui8* iterator; - typedef const ui8* const_iterator; - typedef size_t size_type; - -public: - explicit TBaseBlock(size_t size = 0) - : Size_(size) - { - Ptr_ = Allocator_.Allocate(size); - Clean(); - } - - TBaseBlock(const TBaseBlock<Allocator>& other) - : TBaseBlock(other.Size_) - { - memcpy(Ptr_, other.Ptr_, Size_); - } - - TBaseBlock(const ui8* ptr, size_t size) - : TBaseBlock(size) - { - if (ptr == nullptr) - memset(Ptr_, 0, size); - else - memcpy(Ptr_, ptr, size); - } - - ~TBaseBlock() { - Allocator_.Deallocate(Ptr_, Size_); - } - - operator const void*() const { return Ptr_; } - operator void*() { return Ptr_; } - - operator const ui8*() const { return Ptr_; } - operator ui8*() { return Ptr_; } - - iterator begin() { return Ptr_; } - const_iterator begin() const { return Ptr_; } - - iterator end() { return Ptr_ + Size_; } - const_iterator end() const { return Ptr_ + Size_; } - - ui8* Data() { return Ptr_; } - const ui8* Data() const { return Ptr_; } - - size_t Size() const { return Size_; } - bool Empty() const { return Size_ == 0; } - - void Assign(const ui8* ptr, size_t size) { - New(size); - memcpy(Ptr_, ptr, size); - } - - void Assign(const TBaseBlock<Allocator>& other) { - if (this != &other) { - New(other.Size_); - memcpy(Ptr_, other.Ptr_, other.Size_); - } - } - - TBaseBlock<Allocator>& operator=(const TBaseBlock<Allocator>& other) { - Assign(other); - return *this; - } - - const TBaseBlock<Allocator> operator+(const TBaseBlock<Allocator>& other) { - TBaseBlock<Allocator> result(Size_ + other.Size_); - memcpy(result.Ptr_, Ptr_, Size_); - memcpy(result.Ptr_ + Size_, other.Ptr_, other.Size_); - return result; - } - - bool operator==(const TBaseBlock<Allocator>& other) const { - bool sameSize = (Size_ == other.Size_); +}; + + +template <typename Allocator> +class TBaseBlock +{ +public: + typedef ui8 value_type; + typedef ui8* iterator; + typedef const ui8* const_iterator; + typedef size_t size_type; + +public: + explicit TBaseBlock(size_t size = 0) + : Size_(size) + { + Ptr_ = Allocator_.Allocate(size); + Clean(); + } + + TBaseBlock(const TBaseBlock<Allocator>& other) + : TBaseBlock(other.Size_) + { + memcpy(Ptr_, other.Ptr_, Size_); + } + + TBaseBlock(const ui8* ptr, size_t size) + : TBaseBlock(size) + { + if (ptr == nullptr) + memset(Ptr_, 0, size); + else + memcpy(Ptr_, ptr, size); + } + + ~TBaseBlock() { + Allocator_.Deallocate(Ptr_, Size_); + } + + operator const void*() const { return Ptr_; } + operator void*() { return Ptr_; } + + operator const ui8*() const { return Ptr_; } + operator ui8*() { return Ptr_; } + + iterator begin() { return Ptr_; } + const_iterator begin() const { return Ptr_; } + + iterator end() { return Ptr_ + Size_; } + const_iterator end() const { return Ptr_ + Size_; } + + ui8* Data() { return Ptr_; } + const ui8* Data() const { return Ptr_; } + + size_t Size() const { return Size_; } + bool Empty() const { return Size_ == 0; } + + void Assign(const ui8* ptr, size_t size) { + New(size); + memcpy(Ptr_, ptr, size); + } + + void Assign(const TBaseBlock<Allocator>& other) { + if (this != &other) { + New(other.Size_); + memcpy(Ptr_, other.Ptr_, other.Size_); + } + } + + TBaseBlock<Allocator>& operator=(const TBaseBlock<Allocator>& other) { + Assign(other); + return *this; + } + + const TBaseBlock<Allocator> operator+(const TBaseBlock<Allocator>& other) { + TBaseBlock<Allocator> result(Size_ + other.Size_); + memcpy(result.Ptr_, Ptr_, Size_); + memcpy(result.Ptr_ + Size_, other.Ptr_, other.Size_); + return result; + } + + bool operator==(const TBaseBlock<Allocator>& other) const { + bool sameSize = (Size_ == other.Size_); return sameSize && !memcmp(Ptr_, other.Ptr_, Size_); - } - - bool operator!=(const TBaseBlock<Allocator>& other) const { - return !operator==(other); - } - - void New(size_t newSize) { - Ptr_ = Allocator_.Reallocate(Ptr_, Size_, newSize, false); - Size_ = newSize; - } - - void Clean() { - memset(Ptr_, 0, Size_); - } - - void CleanNew(size_t newSize) { - New(newSize); - memset(Ptr_, 0, Size_); - } - - void Grow(size_t newSize) { - if (newSize > Size_) { - Ptr_ = Allocator_.Reallocate(Ptr_, Size_, newSize, true); - Size_ = newSize; - } - } - - void CleanGrow(size_t newSize) { - if (newSize > Size_) { - Ptr_ = Allocator_.Reallocate(Ptr_, Size_, newSize, true); - memset(Ptr_ + Size_, 0, newSize - Size_); - Size_ = newSize; - } - } - - void Resize(size_t newSize) { - Ptr_ = Allocator_.Reallocate(Ptr_, Size_, newSize, true); - Size_ = newSize; - } - - void Swap(TBaseBlock<Allocator>& other) { - DoSwap(Allocator_, other.Allocator_); - DoSwap(Size_, other.Size_); - DoSwap(Ptr_, other.Ptr_); - } - - TStringBuf AsStringBuf() const { + } + + bool operator!=(const TBaseBlock<Allocator>& other) const { + return !operator==(other); + } + + void New(size_t newSize) { + Ptr_ = Allocator_.Reallocate(Ptr_, Size_, newSize, false); + Size_ = newSize; + } + + void Clean() { + memset(Ptr_, 0, Size_); + } + + void CleanNew(size_t newSize) { + New(newSize); + memset(Ptr_, 0, Size_); + } + + void Grow(size_t newSize) { + if (newSize > Size_) { + Ptr_ = Allocator_.Reallocate(Ptr_, Size_, newSize, true); + Size_ = newSize; + } + } + + void CleanGrow(size_t newSize) { + if (newSize > Size_) { + Ptr_ = Allocator_.Reallocate(Ptr_, Size_, newSize, true); + memset(Ptr_ + Size_, 0, newSize - Size_); + Size_ = newSize; + } + } + + void Resize(size_t newSize) { + Ptr_ = Allocator_.Reallocate(Ptr_, Size_, newSize, true); + Size_ = newSize; + } + + void Swap(TBaseBlock<Allocator>& other) { + DoSwap(Allocator_, other.Allocator_); + DoSwap(Size_, other.Size_); + DoSwap(Ptr_, other.Ptr_); + } + + TStringBuf AsStringBuf() const { return { reinterpret_cast<const char*>(Data()), Size() }; - } - -private: - Allocator Allocator_; - size_t Size_; - ui8* Ptr_; -}; - - -template <typename A = TAlignedAllocator> -class TSecuredBlock : public TBaseBlock<A> -{ -public: - explicit TSecuredBlock(size_t size = 0) - : TBaseBlock<A>(size) {} - - TSecuredBlock(const ui8* data, size_t size) - : TBaseBlock<A>(data, size) {} - - TSecuredBlock(const char* data, size_t size) + } + +private: + Allocator Allocator_; + size_t Size_; + ui8* Ptr_; +}; + + +template <typename A = TAlignedAllocator> +class TSecuredBlock : public TBaseBlock<A> +{ +public: + explicit TSecuredBlock(size_t size = 0) + : TBaseBlock<A>(size) {} + + TSecuredBlock(const ui8* data, size_t size) + : TBaseBlock<A>(data, size) {} + + TSecuredBlock(const char* data, size_t size) : TBaseBlock<A>(reinterpret_cast<const ui8*>(data), size) {} - - template<typename U> - void Append(const TBaseBlock<U>& other) { - size_t oldSize = this->Size(); - this->Grow(oldSize + other.Size()); - memcpy(this->Data() + oldSize, other.Data(), other.Size()); - } - - void Append(const char* data, size_t size) { - size_t oldSize = this->Size(); - this->Grow(oldSize + size); - memcpy(this->Data() + oldSize, data, size); - } -}; - - -template<size_t nbytes, typename A = TFixedAllocator<nbytes>> -class TFixedSecuredBlock : public TBaseBlock<A> -{ -public: - TFixedSecuredBlock() - : TBaseBlock<A>(nbytes) {} - - explicit TFixedSecuredBlock(const ui8* data) - : TBaseBlock<A>(data, nbytes) {} - - explicit TFixedSecuredBlock(const char* data) + + template<typename U> + void Append(const TBaseBlock<U>& other) { + size_t oldSize = this->Size(); + this->Grow(oldSize + other.Size()); + memcpy(this->Data() + oldSize, other.Data(), other.Size()); + } + + void Append(const char* data, size_t size) { + size_t oldSize = this->Size(); + this->Grow(oldSize + size); + memcpy(this->Data() + oldSize, data, size); + } +}; + + +template<size_t nbytes, typename A = TFixedAllocator<nbytes>> +class TFixedSecuredBlock : public TBaseBlock<A> +{ +public: + TFixedSecuredBlock() + : TBaseBlock<A>(nbytes) {} + + explicit TFixedSecuredBlock(const ui8* data) + : TBaseBlock<A>(data, nbytes) {} + + explicit TFixedSecuredBlock(const char* data) : TBaseBlock<A>(reinterpret_cast<const ui8*>(data), nbytes) {} -}; +}; diff --git a/ydb/core/blobstorage/crypto/ut/chacha_test_vectors.h b/ydb/core/blobstorage/crypto/ut/chacha_test_vectors.h index 2a6e8ab2d6d..58c89343d8c 100644 --- a/ydb/core/blobstorage/crypto/ut/chacha_test_vectors.h +++ b/ydb/core/blobstorage/crypto/ut/chacha_test_vectors.h @@ -1,709 +1,709 @@ -#pragma once - -// -// See http://tools.ietf.org/html/draft-strombergson-chacha-test-vectors-01 -// for source data -// - -#include <util/system/types.h> - -#define KEY_SIZE 32 -#define IV_SIZE 8 -#define DATA_SIZE 64 - - -//----------------------------------------------------------------------------- -// TC1: All zero key and IV. -//----------------------------------------------------------------------------- - -const ui8 tc1_key[KEY_SIZE] = { - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, -}; - -const ui8 tc1_iv[IV_SIZE] = { - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, -}; - -const ui8 tc1_expected_8[][DATA_SIZE] = { - { - 0x3e, 0x00, 0xef, 0x2f, 0x89, 0x5f, 0x40, 0xd6, - 0x7f, 0x5b, 0xb8, 0xe8, 0x1f, 0x09, 0xa5, 0xa1, - 0x2c, 0x84, 0x0e, 0xc3, 0xce, 0x9a, 0x7f, 0x3b, - 0x18, 0x1b, 0xe1, 0x88, 0xef, 0x71, 0x1a, 0x1e, - 0x98, 0x4c, 0xe1, 0x72, 0xb9, 0x21, 0x6f, 0x41, - 0x9f, 0x44, 0x53, 0x67, 0x45, 0x6d, 0x56, 0x19, - 0x31, 0x4a, 0x42, 0xa3, 0xda, 0x86, 0xb0, 0x01, - 0x38, 0x7b, 0xfd, 0xb8, 0x0e, 0x0c, 0xfe, 0x42, - - }, - { - 0xd2, 0xae, 0xfa, 0x0d, 0xea, 0xa5, 0xc1, 0x51, - 0xbf, 0x0a, 0xdb, 0x6c, 0x01, 0xf2, 0xa5, 0xad, - 0xc0, 0xfd, 0x58, 0x12, 0x59, 0xf9, 0xa2, 0xaa, - 0xdc, 0xf2, 0x0f, 0x8f, 0xd5, 0x66, 0xa2, 0x6b, - 0x50, 0x32, 0xec, 0x38, 0xbb, 0xc5, 0xda, 0x98, - 0xee, 0x0c, 0x6f, 0x56, 0x8b, 0x87, 0x2a, 0x65, - 0xa0, 0x8a, 0xbf, 0x25, 0x1d, 0xeb, 0x21, 0xbb, - 0x4b, 0x56, 0xe5, 0xd8, 0x82, 0x1e, 0x68, 0xaa, - } -}; - -const ui8 tc1_expected_12[][DATA_SIZE] = { - { - 0x9b, 0xf4, 0x9a, 0x6a, 0x07, 0x55, 0xf9, 0x53, - 0x81, 0x1f, 0xce, 0x12, 0x5f, 0x26, 0x83, 0xd5, - 0x04, 0x29, 0xc3, 0xbb, 0x49, 0xe0, 0x74, 0x14, - 0x7e, 0x00, 0x89, 0xa5, 0x2e, 0xae, 0x15, 0x5f, - 0x05, 0x64, 0xf8, 0x79, 0xd2, 0x7a, 0xe3, 0xc0, - 0x2c, 0xe8, 0x28, 0x34, 0xac, 0xfa, 0x8c, 0x79, - 0x3a, 0x62, 0x9f, 0x2c, 0xa0, 0xde, 0x69, 0x19, - 0x61, 0x0b, 0xe8, 0x2f, 0x41, 0x13, 0x26, 0xbe, - - }, - { - 0x0b, 0xd5, 0x88, 0x41, 0x20, 0x3e, 0x74, 0xfe, - 0x86, 0xfc, 0x71, 0x33, 0x8c, 0xe0, 0x17, 0x3d, - 0xc6, 0x28, 0xeb, 0xb7, 0x19, 0xbd, 0xcb, 0xcc, - 0x15, 0x15, 0x85, 0x21, 0x4c, 0xc0, 0x89, 0xb4, - 0x42, 0x25, 0x8d, 0xcd, 0xa1, 0x4c, 0xf1, 0x11, - 0xc6, 0x02, 0xb8, 0x97, 0x1b, 0x8c, 0xc8, 0x43, - 0xe9, 0x1e, 0x46, 0xca, 0x90, 0x51, 0x51, 0xc0, - 0x27, 0x44, 0xa6, 0xb0, 0x17, 0xe6, 0x93, 0x16, - } -}; - -const ui8 tc1_expected_20[][DATA_SIZE] = { - { - 0x76, 0xb8, 0xe0, 0xad, 0xa0, 0xf1, 0x3d, 0x90, - 0x40, 0x5d, 0x6a, 0xe5, 0x53, 0x86, 0xbd, 0x28, - 0xbd, 0xd2, 0x19, 0xb8, 0xa0, 0x8d, 0xed, 0x1a, - 0xa8, 0x36, 0xef, 0xcc, 0x8b, 0x77, 0x0d, 0xc7, - 0xda, 0x41, 0x59, 0x7c, 0x51, 0x57, 0x48, 0x8d, - 0x77, 0x24, 0xe0, 0x3f, 0xb8, 0xd8, 0x4a, 0x37, - 0x6a, 0x43, 0xb8, 0xf4, 0x15, 0x18, 0xa1, 0x1c, - 0xc3, 0x87, 0xb6, 0x69, 0xb2, 0xee, 0x65, 0x86, - - }, - { - 0x9f, 0x07, 0xe7, 0xbe, 0x55, 0x51, 0x38, 0x7a, - 0x98, 0xba, 0x97, 0x7c, 0x73, 0x2d, 0x08, 0x0d, - 0xcb, 0x0f, 0x29, 0xa0, 0x48, 0xe3, 0x65, 0x69, - 0x12, 0xc6, 0x53, 0x3e, 0x32, 0xee, 0x7a, 0xed, - 0x29, 0xb7, 0x21, 0x76, 0x9c, 0xe6, 0x4e, 0x43, - 0xd5, 0x71, 0x33, 0xb0, 0x74, 0xd8, 0x39, 0xd5, - 0x31, 0xed, 0x1f, 0x28, 0x51, 0x0a, 0xfb, 0x45, - 0xac, 0xe1, 0x0a, 0x1f, 0x4b, 0x79, 0x4d, 0x6f, - } -}; - -//----------------------------------------------------------------------------- -// TC2: Single bit in key set. All zero IV. -//----------------------------------------------------------------------------- - -const ui8 tc2_key[KEY_SIZE] = { - 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, -}; - -const ui8 tc2_iv[IV_SIZE] = { - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, -}; - -const ui8 tc2_expected_8[][DATA_SIZE] = { - { - 0xcf, 0x5e, 0xe9, 0xa0, 0x49, 0x4a, 0xa9, 0x61, - 0x3e, 0x05, 0xd5, 0xed, 0x72, 0x5b, 0x80, 0x4b, - 0x12, 0xf4, 0xa4, 0x65, 0xee, 0x63, 0x5a, 0xcc, - 0x3a, 0x31, 0x1d, 0xe8, 0x74, 0x04, 0x89, 0xea, - 0x28, 0x9d, 0x04, 0xf4, 0x3c, 0x75, 0x18, 0xdb, - 0x56, 0xeb, 0x44, 0x33, 0xe4, 0x98, 0xa1, 0x23, - 0x8c, 0xd8, 0x46, 0x4d, 0x37, 0x63, 0xdd, 0xbb, - 0x92, 0x22, 0xee, 0x3b, 0xd8, 0xfa, 0xe3, 0xc8, - - }, - { - 0xb4, 0x35, 0x5a, 0x7d, 0x93, 0xdd, 0x88, 0x67, - 0x08, 0x9e, 0xe6, 0x43, 0x55, 0x8b, 0x95, 0x75, - 0x4e, 0xfa, 0x2b, 0xd1, 0xa8, 0xa1, 0xe2, 0xd7, - 0x5b, 0xcd, 0xb3, 0x20, 0x15, 0x54, 0x26, 0x38, - 0x29, 0x19, 0x41, 0xfe, 0xb4, 0x99, 0x65, 0x58, - 0x7c, 0x4f, 0xdf, 0xe2, 0x19, 0xcf, 0x0e, 0xc1, - 0x32, 0xa6, 0xcd, 0x4d, 0xc0, 0x67, 0x39, 0x2e, - 0x67, 0x98, 0x2f, 0xe5, 0x32, 0x78, 0xc0, 0xb4, - } -}; - -const ui8 tc2_expected_12[][DATA_SIZE] = { - { - 0x12, 0x05, 0x6e, 0x59, 0x5d, 0x56, 0xb0, 0xf6, - 0xee, 0xf0, 0x90, 0xf0, 0xcd, 0x25, 0xa2, 0x09, - 0x49, 0x24, 0x8c, 0x27, 0x90, 0x52, 0x5d, 0x0f, - 0x93, 0x02, 0x18, 0xff, 0x0b, 0x4d, 0xdd, 0x10, - 0xa6, 0x00, 0x22, 0x39, 0xd9, 0xa4, 0x54, 0xe2, - 0x9e, 0x10, 0x7a, 0x7d, 0x06, 0xfe, 0xfd, 0xfe, - 0xf0, 0x21, 0x0f, 0xeb, 0xa0, 0x44, 0xf9, 0xf2, - 0x9b, 0x17, 0x72, 0xc9, 0x60, 0xdc, 0x29, 0xc0, - - }, - { - 0x0c, 0x73, 0x66, 0xc5, 0xcb, 0xc6, 0x04, 0x24, - 0x0e, 0x66, 0x5e, 0xb0, 0x2a, 0x69, 0x37, 0x2a, - 0x7a, 0xf9, 0x79, 0xb2, 0x6f, 0xbb, 0x78, 0x09, - 0x2a, 0xc7, 0xc4, 0xb8, 0x80, 0x29, 0xa7, 0xc8, - 0x54, 0x51, 0x3b, 0xc2, 0x17, 0xbb, 0xfc, 0x7d, - 0x90, 0x43, 0x2e, 0x30, 0x8e, 0xba, 0x15, 0xaf, - 0xc6, 0x5a, 0xeb, 0x48, 0xef, 0x10, 0x0d, 0x56, - 0x01, 0xe6, 0xaf, 0xba, 0x25, 0x71, 0x17, 0xa9, - } -}; - -const ui8 tc2_expected_20[][DATA_SIZE] = { - { - 0xc5, 0xd3, 0x0a, 0x7c, 0xe1, 0xec, 0x11, 0x93, - 0x78, 0xc8, 0x4f, 0x48, 0x7d, 0x77, 0x5a, 0x85, - 0x42, 0xf1, 0x3e, 0xce, 0x23, 0x8a, 0x94, 0x55, - 0xe8, 0x22, 0x9e, 0x88, 0x8d, 0xe8, 0x5b, 0xbd, - 0x29, 0xeb, 0x63, 0xd0, 0xa1, 0x7a, 0x5b, 0x99, - 0x9b, 0x52, 0xda, 0x22, 0xbe, 0x40, 0x23, 0xeb, - 0x07, 0x62, 0x0a, 0x54, 0xf6, 0xfa, 0x6a, 0xd8, - 0x73, 0x7b, 0x71, 0xeb, 0x04, 0x64, 0xda, 0xc0, - - }, - { - 0x10, 0xf6, 0x56, 0xe6, 0xd1, 0xfd, 0x55, 0x05, - 0x3e, 0x50, 0xc4, 0x87, 0x5c, 0x99, 0x30, 0xa3, - 0x3f, 0x6d, 0x02, 0x63, 0xbd, 0x14, 0xdf, 0xd6, - 0xab, 0x8c, 0x70, 0x52, 0x1c, 0x19, 0x33, 0x8b, - 0x23, 0x08, 0xb9, 0x5c, 0xf8, 0xd0, 0xbb, 0x7d, - 0x20, 0x2d, 0x21, 0x02, 0x78, 0x0e, 0xa3, 0x52, - 0x8f, 0x1c, 0xb4, 0x85, 0x60, 0xf7, 0x6b, 0x20, - 0xf3, 0x82, 0xb9, 0x42, 0x50, 0x0f, 0xce, 0xac, - } -}; - -//----------------------------------------------------------------------------- -// TC3: Single bit in IV set. All zero key. -//----------------------------------------------------------------------------- - -const ui8 tc3_key[KEY_SIZE] = { - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, -}; - -const ui8 tc3_iv[IV_SIZE] = { - 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, -}; - -const ui8 tc3_expected_8[][DATA_SIZE] = { - { - 0x2b, 0x8f, 0x4b, 0xb3, 0x79, 0x83, 0x06, 0xca, - 0x51, 0x30, 0xd4, 0x7c, 0x4f, 0x8d, 0x4e, 0xd1, - 0x3a, 0xa0, 0xed, 0xcc, 0xc1, 0xbe, 0x69, 0x42, - 0x09, 0x0f, 0xae, 0xec, 0xa0, 0xd7, 0x59, 0x9b, - 0x7f, 0xf0, 0xfe, 0x61, 0x6b, 0xb2, 0x5a, 0xa0, - 0x15, 0x3a, 0xd6, 0xfd, 0xc8, 0x8b, 0x95, 0x49, - 0x03, 0xc2, 0x24, 0x26, 0xd4, 0x78, 0xb9, 0x7b, - 0x22, 0xb8, 0xf9, 0xb1, 0xdb, 0x00, 0xcf, 0x06, - - }, - { - 0x47, 0x0b, 0xdf, 0xfb, 0xc4, 0x88, 0xa8, 0xb7, - 0xc7, 0x01, 0xeb, 0xf4, 0x06, 0x1d, 0x75, 0xc5, - 0x96, 0x91, 0x86, 0x49, 0x7c, 0x95, 0x36, 0x78, - 0x09, 0xaf, 0xa8, 0x0b, 0xd8, 0x43, 0xb0, 0x40, - 0xa7, 0x9a, 0xbc, 0x6e, 0x73, 0xa9, 0x17, 0x57, - 0xf1, 0xdb, 0x73, 0xc8, 0xea, 0xcf, 0xa5, 0x43, - 0xb3, 0x8f, 0x28, 0x9d, 0x06, 0x5a, 0xb2, 0xf3, - 0x03, 0x2d, 0x37, 0x7b, 0x8c, 0x37, 0xfe, 0x46, - } -}; - -const ui8 tc3_expected_12[][DATA_SIZE] = { - { - 0x64, 0xb8, 0xbd, 0xf8, 0x7b, 0x82, 0x8c, 0x4b, - 0x6d, 0xba, 0xf7, 0xef, 0x69, 0x8d, 0xe0, 0x3d, - 0xf8, 0xb3, 0x3f, 0x63, 0x57, 0x14, 0x41, 0x8f, - 0x98, 0x36, 0xad, 0xe5, 0x9b, 0xe1, 0x29, 0x69, - 0x46, 0xc9, 0x53, 0xa0, 0xf3, 0x8e, 0xcf, 0xfc, - 0x9e, 0xcb, 0x98, 0xe8, 0x1d, 0x5d, 0x99, 0xa5, - 0xed, 0xfc, 0x8f, 0x9a, 0x0a, 0x45, 0xb9, 0xe4, - 0x1e, 0xf3, 0xb3, 0x1f, 0x02, 0x8f, 0x1d, 0x0f, - - }, - { - 0x55, 0x9d, 0xb4, 0xa7, 0xf2, 0x22, 0xc4, 0x42, - 0xfe, 0x23, 0xb9, 0xa2, 0x59, 0x6a, 0x88, 0x28, - 0x51, 0x22, 0xee, 0x4f, 0x13, 0x63, 0x89, 0x6e, - 0xa7, 0x7c, 0xa1, 0x50, 0x91, 0x2a, 0xc7, 0x23, - 0xbf, 0xf0, 0x4b, 0x02, 0x6a, 0x2f, 0x80, 0x7e, - 0x03, 0xb2, 0x9c, 0x02, 0x07, 0x7d, 0x7b, 0x06, - 0xfc, 0x1a, 0xb9, 0x82, 0x7c, 0x13, 0xc8, 0x01, - 0x3a, 0x6d, 0x83, 0xbd, 0x3b, 0x52, 0xa2, 0x6f, - } -}; - -const ui8 tc3_expected_20[][DATA_SIZE] = { - { - 0xef, 0x3f, 0xdf, 0xd6, 0xc6, 0x15, 0x78, 0xfb, - 0xf5, 0xcf, 0x35, 0xbd, 0x3d, 0xd3, 0x3b, 0x80, - 0x09, 0x63, 0x16, 0x34, 0xd2, 0x1e, 0x42, 0xac, - 0x33, 0x96, 0x0b, 0xd1, 0x38, 0xe5, 0x0d, 0x32, - 0x11, 0x1e, 0x4c, 0xaf, 0x23, 0x7e, 0xe5, 0x3c, - 0xa8, 0xad, 0x64, 0x26, 0x19, 0x4a, 0x88, 0x54, - 0x5d, 0xdc, 0x49, 0x7a, 0x0b, 0x46, 0x6e, 0x7d, - 0x6b, 0xbd, 0xb0, 0x04, 0x1b, 0x2f, 0x58, 0x6b, - - }, - { - 0x53, 0x05, 0xe5, 0xe4, 0x4a, 0xff, 0x19, 0xb2, - 0x35, 0x93, 0x61, 0x44, 0x67, 0x5e, 0xfb, 0xe4, - 0x40, 0x9e, 0xb7, 0xe8, 0xe5, 0xf1, 0x43, 0x0f, - 0x5f, 0x58, 0x36, 0xae, 0xb4, 0x9b, 0xb5, 0x32, - 0x8b, 0x01, 0x7c, 0x4b, 0x9d, 0xc1, 0x1f, 0x8a, - 0x03, 0x86, 0x3f, 0xa8, 0x03, 0xdc, 0x71, 0xd5, - 0x72, 0x6b, 0x2b, 0x6b, 0x31, 0xaa, 0x32, 0x70, - 0x8a, 0xfe, 0x5a, 0xf1, 0xd6, 0xb6, 0x90, 0x58, - } -}; - -//----------------------------------------------------------------------------- -// TC4: All bits in key and IV are set. -//----------------------------------------------------------------------------- - -const ui8 tc4_key[KEY_SIZE] = { - 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, - 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, - 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, - 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, -}; - -const ui8 tc4_iv[IV_SIZE] = { - 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, -}; - -const ui8 tc4_expected_8[][DATA_SIZE] = { - { - 0xe1, 0x63, 0xbb, 0xf8, 0xc9, 0xa7, 0x39, 0xd1, - 0x89, 0x25, 0xee, 0x83, 0x62, 0xda, 0xd2, 0xcd, - 0xc9, 0x73, 0xdf, 0x05, 0x22, 0x5a, 0xfb, 0x2a, - 0xa2, 0x63, 0x96, 0xf2, 0xa9, 0x84, 0x9a, 0x4a, - 0x44, 0x5e, 0x05, 0x47, 0xd3, 0x1c, 0x16, 0x23, - 0xc5, 0x37, 0xdf, 0x4b, 0xa8, 0x5c, 0x70, 0xa9, - 0x88, 0x4a, 0x35, 0xbc, 0xbf, 0x3d, 0xfa, 0xb0, - 0x77, 0xe9, 0x8b, 0x0f, 0x68, 0x13, 0x5f, 0x54, - - }, - { - 0x81, 0xd4, 0x93, 0x3f, 0x8b, 0x32, 0x2a, 0xc0, - 0xcd, 0x76, 0x2c, 0x27, 0x23, 0x5c, 0xe2, 0xb3, - 0x15, 0x34, 0xe0, 0x24, 0x4a, 0x9a, 0x2f, 0x1f, - 0xd5, 0xe9, 0x44, 0x98, 0xd4, 0x7f, 0xf1, 0x08, - 0x79, 0x0c, 0x00, 0x9c, 0xf9, 0xe1, 0xa3, 0x48, - 0x03, 0x2a, 0x76, 0x94, 0xcb, 0x28, 0x02, 0x4c, - 0xd9, 0x6d, 0x34, 0x98, 0x36, 0x1e, 0xdb, 0x17, - 0x85, 0xaf, 0x75, 0x2d, 0x18, 0x7a, 0xb5, 0x4b, - } -}; - -const ui8 tc4_expected_12[][DATA_SIZE] = { - { - 0x04, 0xbf, 0x88, 0xda, 0xe8, 0xe4, 0x7a, 0x22, - 0x8f, 0xa4, 0x7b, 0x7e, 0x63, 0x79, 0x43, 0x4b, - 0xa6, 0x64, 0xa7, 0xd2, 0x8f, 0x4d, 0xab, 0x84, - 0xe5, 0xf8, 0xb4, 0x64, 0xad, 0xd2, 0x0c, 0x3a, - 0xca, 0xa6, 0x9c, 0x5a, 0xb2, 0x21, 0xa2, 0x3a, - 0x57, 0xeb, 0x5f, 0x34, 0x5c, 0x96, 0xf4, 0xd1, - 0x32, 0x2d, 0x0a, 0x2f, 0xf7, 0xa9, 0xcd, 0x43, - 0x40, 0x1c, 0xd5, 0x36, 0x63, 0x9a, 0x61, 0x5a, - - }, - { - 0x5c, 0x94, 0x29, 0xb5, 0x5c, 0xa3, 0xc1, 0xb5, - 0x53, 0x54, 0x55, 0x96, 0x69, 0xa1, 0x54, 0xac, - 0xa4, 0x6c, 0xd7, 0x61, 0xc4, 0x1a, 0xb8, 0xac, - 0xe3, 0x85, 0x36, 0x3b, 0x95, 0x67, 0x5f, 0x06, - 0x8e, 0x18, 0xdb, 0x5a, 0x67, 0x3c, 0x11, 0x29, - 0x1b, 0xd4, 0x18, 0x78, 0x92, 0xa9, 0xa3, 0xa3, - 0x35, 0x14, 0xf3, 0x71, 0x2b, 0x26, 0xc1, 0x30, - 0x26, 0x10, 0x32, 0x98, 0xed, 0x76, 0xbc, 0x9a, - } -}; - -const ui8 tc4_expected_20[][DATA_SIZE] = { - { - 0xd9, 0xbf, 0x3f, 0x6b, 0xce, 0x6e, 0xd0, 0xb5, - 0x42, 0x54, 0x55, 0x77, 0x67, 0xfb, 0x57, 0x44, - 0x3d, 0xd4, 0x77, 0x89, 0x11, 0xb6, 0x06, 0x05, - 0x5c, 0x39, 0xcc, 0x25, 0xe6, 0x74, 0xb8, 0x36, - 0x3f, 0xea, 0xbc, 0x57, 0xfd, 0xe5, 0x4f, 0x79, - 0x0c, 0x52, 0xc8, 0xae, 0x43, 0x24, 0x0b, 0x79, - 0xd4, 0x90, 0x42, 0xb7, 0x77, 0xbf, 0xd6, 0xcb, - 0x80, 0xe9, 0x31, 0x27, 0x0b, 0x7f, 0x50, 0xeb, - - }, - { - 0x5b, 0xac, 0x2a, 0xcd, 0x86, 0xa8, 0x36, 0xc5, - 0xdc, 0x98, 0xc1, 0x16, 0xc1, 0x21, 0x7e, 0xc3, - 0x1d, 0x3a, 0x63, 0xa9, 0x45, 0x13, 0x19, 0xf0, - 0x97, 0xf3, 0xb4, 0xd6, 0xda, 0xb0, 0x77, 0x87, - 0x19, 0x47, 0x7d, 0x24, 0xd2, 0x4b, 0x40, 0x3a, - 0x12, 0x24, 0x1d, 0x7c, 0xca, 0x06, 0x4f, 0x79, - 0x0f, 0x1d, 0x51, 0xcc, 0xaf, 0xf6, 0xb1, 0x66, - 0x7d, 0x4b, 0xbc, 0xa1, 0x95, 0x8c, 0x43, 0x06, - } -}; - -//----------------------------------------------------------------------------- -// TC5: Every even bit set in key and IV. -//----------------------------------------------------------------------------- - -const ui8 tc5_key[KEY_SIZE] = { - 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, - 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, - 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, - 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, -}; - -const ui8 tc5_iv[IV_SIZE] = { - 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, -}; - -const ui8 tc5_expected_8[][DATA_SIZE] = { - { - 0x7c, 0xb7, 0x82, 0x14, 0xe4, 0xd3, 0x46, 0x5b, - 0x6d, 0xc6, 0x2c, 0xf7, 0xa1, 0x53, 0x8c, 0x88, - 0x99, 0x69, 0x52, 0xb4, 0xfb, 0x72, 0xcb, 0x61, - 0x05, 0xf1, 0x24, 0x3c, 0xe3, 0x44, 0x2e, 0x29, - 0x75, 0xa5, 0x9e, 0xbc, 0xd2, 0xb2, 0xa5, 0x98, - 0x29, 0x0d, 0x75, 0x38, 0x49, 0x1f, 0xe6, 0x5b, - 0xdb, 0xfe, 0xfd, 0x06, 0x0d, 0x88, 0x79, 0x81, - 0x20, 0xa7, 0x0d, 0x04, 0x9d, 0xc2, 0x67, 0x7d, - - }, - { - 0xd4, 0x8f, 0xf5, 0xa2, 0x51, 0x3e, 0x49, 0x7a, - 0x5d, 0x54, 0x80, 0x2d, 0x74, 0x84, 0xc4, 0xf1, - 0x08, 0x39, 0x44, 0xd8, 0xd0, 0xd1, 0x4d, 0x64, - 0x82, 0xce, 0x09, 0xf7, 0xe5, 0xeb, 0xf2, 0x0b, - 0x29, 0x80, 0x7d, 0x62, 0xc3, 0x18, 0x74, 0xd0, - 0x2f, 0x5d, 0x3c, 0xc8, 0x53, 0x81, 0xa7, 0x45, - 0xec, 0xbc, 0x60, 0x52, 0x52, 0x05, 0xe3, 0x00, - 0xa7, 0x69, 0x61, 0xbf, 0xe5, 0x1a, 0xc0, 0x7c, - } -}; - -const ui8 tc5_expected_12[][DATA_SIZE] = { - { - 0xa6, 0x00, 0xf0, 0x77, 0x27, 0xff, 0x93, 0xf3, - 0xda, 0x00, 0xdd, 0x74, 0xcc, 0x3e, 0x8b, 0xfb, - 0x5c, 0xa7, 0x30, 0x2f, 0x6a, 0x0a, 0x29, 0x44, - 0x95, 0x3d, 0xe0, 0x04, 0x50, 0xee, 0xcd, 0x40, - 0xb8, 0x60, 0xf6, 0x60, 0x49, 0xf2, 0xea, 0xed, - 0x63, 0xb2, 0xef, 0x39, 0xcc, 0x31, 0x0d, 0x2c, - 0x48, 0x8f, 0x5d, 0x9a, 0x24, 0x1b, 0x61, 0x5d, - 0xc0, 0xab, 0x70, 0xf9, 0x21, 0xb9, 0x1b, 0x95, - - }, - { - 0x14, 0x0e, 0xff, 0x4a, 0xa4, 0x95, 0xac, 0x61, - 0x28, 0x9b, 0x6b, 0xc5, 0x7d, 0xe0, 0x72, 0x41, - 0x9d, 0x09, 0xda, 0xa7, 0xa7, 0x24, 0x39, 0x90, - 0xda, 0xf3, 0x48, 0xa8, 0xf2, 0x83, 0x1e, 0x59, - 0x7c, 0xf3, 0x79, 0xb3, 0xb2, 0x84, 0xf0, 0x0b, - 0xda, 0x27, 0xa4, 0xc6, 0x80, 0x85, 0x37, 0x4a, - 0x8a, 0x5c, 0x38, 0xde, 0xd6, 0x2d, 0x11, 0x41, - 0xca, 0xe0, 0xbb, 0x83, 0x8d, 0xdc, 0x22, 0x32, - } -}; - -const ui8 tc5_expected_20[][DATA_SIZE] = { - { - 0xbe, 0xa9, 0x41, 0x1a, 0xa4, 0x53, 0xc5, 0x43, - 0x4a, 0x5a, 0xe8, 0xc9, 0x28, 0x62, 0xf5, 0x64, - 0x39, 0x68, 0x55, 0xa9, 0xea, 0x6e, 0x22, 0xd6, - 0xd3, 0xb5, 0x0a, 0xe1, 0xb3, 0x66, 0x33, 0x11, - 0xa4, 0xa3, 0x60, 0x6c, 0x67, 0x1d, 0x60, 0x5c, - 0xe1, 0x6c, 0x3a, 0xec, 0xe8, 0xe6, 0x1e, 0xa1, - 0x45, 0xc5, 0x97, 0x75, 0x01, 0x7b, 0xee, 0x2f, - 0xa6, 0xf8, 0x8a, 0xfc, 0x75, 0x80, 0x69, 0xf7, - - }, - { - 0xe0, 0xb8, 0xf6, 0x76, 0xe6, 0x44, 0x21, 0x6f, - 0x4d, 0x2a, 0x34, 0x22, 0xd7, 0xfa, 0x36, 0xc6, - 0xc4, 0x93, 0x1a, 0xca, 0x95, 0x0e, 0x9d, 0xa4, - 0x27, 0x88, 0xe6, 0xd0, 0xb6, 0xd1, 0xcd, 0x83, - 0x8e, 0xf6, 0x52, 0xe9, 0x7b, 0x14, 0x5b, 0x14, - 0x87, 0x1e, 0xae, 0x6c, 0x68, 0x04, 0xc7, 0x00, - 0x4d, 0xb5, 0xac, 0x2f, 0xce, 0x4c, 0x68, 0xc7, - 0x26, 0xd0, 0x04, 0xb1, 0x0f, 0xca, 0xba, 0x86, - } -}; - -//----------------------------------------------------------------------------- -// TC6: Every odd bit set in key and IV. -//----------------------------------------------------------------------------- - -const ui8 tc6_key[KEY_SIZE] = { - 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, - 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, - 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, - 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, -}; - -const ui8 tc6_iv[IV_SIZE] = { - 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, -}; - -const ui8 tc6_expected_8[][DATA_SIZE] = { - { - 0x40, 0xf9, 0xab, 0x86, 0xc8, 0xf9, 0xa1, 0xa0, - 0xcd, 0xc0, 0x5a, 0x75, 0xe5, 0x53, 0x1b, 0x61, - 0x2d, 0x71, 0xef, 0x7f, 0x0c, 0xf9, 0xe3, 0x87, - 0xdf, 0x6e, 0xd6, 0x97, 0x2f, 0x0a, 0xae, 0x21, - 0x31, 0x1a, 0xa5, 0x81, 0xf8, 0x16, 0xc9, 0x0e, - 0x8a, 0x99, 0xde, 0x99, 0x0b, 0x6b, 0x95, 0xaa, - 0xc9, 0x24, 0x50, 0xf4, 0xe1, 0x12, 0x71, 0x26, - 0x67, 0xb8, 0x04, 0xc9, 0x9e, 0x9c, 0x6e, 0xda, - - }, - { - 0xf8, 0xd1, 0x44, 0xf5, 0x60, 0xc8, 0xc0, 0xea, - 0x36, 0x88, 0x0d, 0x3b, 0x77, 0x87, 0x4c, 0x9a, - 0x91, 0x03, 0xd1, 0x47, 0xf6, 0xde, 0xd3, 0x86, - 0x28, 0x48, 0x01, 0xa4, 0xee, 0x15, 0x8e, 0x5e, - 0xa4, 0xf9, 0xc0, 0x93, 0xfc, 0x55, 0xfd, 0x34, - 0x4c, 0x33, 0x34, 0x9d, 0xc5, 0xb6, 0x99, 0xe2, - 0x1d, 0xc8, 0x3b, 0x42, 0x96, 0xf9, 0x2e, 0xe3, - 0xec, 0xab, 0xf3, 0xd5, 0x1f, 0x95, 0xfe, 0x3f, - } -}; - -const ui8 tc6_expected_12[][DATA_SIZE] = { - { - 0x85, 0x65, 0x05, 0xb0, 0x1d, 0x3b, 0x47, 0xaa, - 0xe0, 0x3d, 0x6a, 0x97, 0xaa, 0x0f, 0x03, 0x3a, - 0x9a, 0xdc, 0xc9, 0x43, 0x77, 0xba, 0xbd, 0x86, - 0x08, 0x86, 0x4f, 0xb3, 0xf6, 0x25, 0xb6, 0xe3, - 0x14, 0xf0, 0x86, 0x15, 0x8f, 0x9f, 0x72, 0x5d, - 0x81, 0x1e, 0xeb, 0x95, 0x3b, 0x7f, 0x74, 0x70, - 0x76, 0xe4, 0xc3, 0xf6, 0x39, 0xfa, 0x84, 0x1f, - 0xad, 0x6c, 0x9a, 0x70, 0x9e, 0x62, 0x13, 0x97, - - }, - { - 0x6d, 0xd6, 0xee, 0x9b, 0x5e, 0x1e, 0x2e, 0x67, - 0x6b, 0x1c, 0x9e, 0x2b, 0x82, 0xc2, 0xe9, 0x6c, - 0x16, 0x48, 0x43, 0x7b, 0xff, 0x2f, 0x01, 0x26, - 0xb7, 0x4e, 0x8c, 0xe0, 0xa9, 0xb0, 0x6d, 0x17, - 0x20, 0xac, 0x0b, 0x6f, 0x09, 0x08, 0x6f, 0x28, - 0xbc, 0x20, 0x15, 0x87, 0xf0, 0x53, 0x5e, 0xd9, - 0x38, 0x52, 0x70, 0xd0, 0x8b, 0x4a, 0x93, 0x82, - 0xf1, 0x8f, 0x82, 0xdb, 0xde, 0x18, 0x21, 0x0e, - } -}; - -const ui8 tc6_expected_20[][DATA_SIZE] = { - { - 0x9a, 0xa2, 0xa9, 0xf6, 0x56, 0xef, 0xde, 0x5a, - 0xa7, 0x59, 0x1c, 0x5f, 0xed, 0x4b, 0x35, 0xae, - 0xa2, 0x89, 0x5d, 0xec, 0x7c, 0xb4, 0x54, 0x3b, - 0x9e, 0x9f, 0x21, 0xf5, 0xe7, 0xbc, 0xbc, 0xf3, - 0xc4, 0x3c, 0x74, 0x8a, 0x97, 0x08, 0x88, 0xf8, - 0x24, 0x83, 0x93, 0xa0, 0x9d, 0x43, 0xe0, 0xb7, - 0xe1, 0x64, 0xbc, 0x4d, 0x0b, 0x0f, 0xb2, 0x40, - 0xa2, 0xd7, 0x21, 0x15, 0xc4, 0x80, 0x89, 0x06, - - }, - { - 0x72, 0x18, 0x44, 0x89, 0x44, 0x05, 0x45, 0xd0, - 0x21, 0xd9, 0x7e, 0xf6, 0xb6, 0x93, 0xdf, 0xe5, - 0xb2, 0xc1, 0x32, 0xd4, 0x7e, 0x6f, 0x04, 0x1c, - 0x90, 0x63, 0x65, 0x1f, 0x96, 0xb6, 0x23, 0xe6, - 0x2a, 0x11, 0x99, 0x9a, 0x23, 0xb6, 0xf7, 0xc4, - 0x61, 0xb2, 0x15, 0x30, 0x26, 0xad, 0x5e, 0x86, - 0x6a, 0x2e, 0x59, 0x7e, 0xd0, 0x7b, 0x84, 0x01, - 0xde, 0xc6, 0x3a, 0x09, 0x34, 0xc6, 0xb2, 0xa9, - } -}; - -//----------------------------------------------------------------------------- -// TC7: Sequence patterns in key and IV. -//----------------------------------------------------------------------------- - -const ui8 tc7_key[KEY_SIZE] = { - 0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, - 0x88, 0x99, 0xaa, 0xbb, 0xcc, 0xdd, 0xee, 0xff, - 0xff, 0xee, 0xdd, 0xcc, 0xbb, 0xaa, 0x99, 0x88, - 0x77, 0x66, 0x55, 0x44, 0x33, 0x22, 0x11, 0x00, -}; - -const ui8 tc7_iv[IV_SIZE] = { - 0x0f, 0x1e, 0x2d, 0x3c, 0x4b, 0x5a, 0x69, 0x78, -}; - -const ui8 tc7_expected_8[][DATA_SIZE] = { - { - 0xdb, 0x43, 0xad, 0x9d, 0x1e, 0x84, 0x2d, 0x12, - 0x72, 0xe4, 0x53, 0x0e, 0x27, 0x6b, 0x3f, 0x56, - 0x8f, 0x88, 0x59, 0xb3, 0xf7, 0xcf, 0x6d, 0x9d, - 0x2c, 0x74, 0xfa, 0x53, 0x80, 0x8c, 0xb5, 0x15, - 0x7a, 0x8e, 0xbf, 0x46, 0xad, 0x3d, 0xcc, 0x4b, - 0x6c, 0x7d, 0xad, 0xde, 0x13, 0x17, 0x84, 0xb0, - 0x12, 0x0e, 0x0e, 0x22, 0xf6, 0xd5, 0xf9, 0xff, - 0xa7, 0x40, 0x7d, 0x4a, 0x21, 0xb6, 0x95, 0xd9, - - }, - { - 0xc5, 0xdd, 0x30, 0xbf, 0x55, 0x61, 0x2f, 0xab, - 0x9b, 0xdd, 0x11, 0x89, 0x20, 0xc1, 0x98, 0x16, - 0x47, 0x0c, 0x7f, 0x5d, 0xcd, 0x42, 0x32, 0x5d, - 0xbb, 0xed, 0x8c, 0x57, 0xa5, 0x62, 0x81, 0xc1, - 0x44, 0xcb, 0x0f, 0x03, 0xe8, 0x1b, 0x30, 0x04, - 0x62, 0x4e, 0x06, 0x50, 0xa1, 0xce, 0x5a, 0xfa, - 0xf9, 0xa7, 0xcd, 0x81, 0x63, 0xf6, 0xdb, 0xd7, - 0x26, 0x02, 0x25, 0x7d, 0xd9, 0x6e, 0x47, 0x1e, - } -}; - -const ui8 tc7_expected_12[][DATA_SIZE] = { - { - 0x7e, 0xd1, 0x2a, 0x3a, 0x63, 0x91, 0x2a, 0xe9, - 0x41, 0xba, 0x6d, 0x4c, 0x0d, 0x5e, 0x86, 0x2e, - 0x56, 0x8b, 0x0e, 0x55, 0x89, 0x34, 0x69, 0x35, - 0x50, 0x5f, 0x06, 0x4b, 0x8c, 0x26, 0x98, 0xdb, - 0xf7, 0xd8, 0x50, 0x66, 0x7d, 0x8e, 0x67, 0xbe, - 0x63, 0x9f, 0x3b, 0x4f, 0x6a, 0x16, 0xf9, 0x2e, - 0x65, 0xea, 0x80, 0xf6, 0xc7, 0x42, 0x94, 0x45, - 0xda, 0x1f, 0xc2, 0xc1, 0xb9, 0x36, 0x50, 0x40, - - }, - { - 0xe3, 0x2e, 0x50, 0xc4, 0x10, 0x6f, 0x3b, 0x3d, - 0xa1, 0xce, 0x7c, 0xcb, 0x1e, 0x71, 0x40, 0xb1, - 0x53, 0x49, 0x3c, 0x0f, 0x3a, 0xd9, 0xa9, 0xbc, - 0xff, 0x07, 0x7e, 0xc4, 0x59, 0x6f, 0x1d, 0x0f, - 0x29, 0xbf, 0x9c, 0xba, 0xa5, 0x02, 0x82, 0x0f, - 0x73, 0x2a, 0xf5, 0xa9, 0x3c, 0x49, 0xee, 0xe3, - 0x3d, 0x1c, 0x4f, 0x12, 0xaf, 0x3b, 0x42, 0x97, - 0xaf, 0x91, 0xfe, 0x41, 0xea, 0x9e, 0x94, 0xa2, - } -}; - -const ui8 tc7_expected_20[][DATA_SIZE] = { - { - 0x9f, 0xad, 0xf4, 0x09, 0xc0, 0x08, 0x11, 0xd0, - 0x04, 0x31, 0xd6, 0x7e, 0xfb, 0xd8, 0x8f, 0xba, - 0x59, 0x21, 0x8d, 0x5d, 0x67, 0x08, 0xb1, 0xd6, - 0x85, 0x86, 0x3f, 0xab, 0xbb, 0x0e, 0x96, 0x1e, - 0xea, 0x48, 0x0f, 0xd6, 0xfb, 0x53, 0x2b, 0xfd, - 0x49, 0x4b, 0x21, 0x51, 0x01, 0x50, 0x57, 0x42, - 0x3a, 0xb6, 0x0a, 0x63, 0xfe, 0x4f, 0x55, 0xf7, - 0xa2, 0x12, 0xe2, 0x16, 0x7c, 0xca, 0xb9, 0x31, - - }, - { - 0xfb, 0xfd, 0x29, 0xcf, 0x7b, 0xc1, 0xd2, 0x79, - 0xed, 0xdf, 0x25, 0xdd, 0x31, 0x6b, 0xb8, 0x84, - 0x3d, 0x6e, 0xde, 0xe0, 0xbd, 0x1e, 0xf1, 0x21, - 0xd1, 0x2f, 0xa1, 0x7c, 0xbc, 0x2c, 0x57, 0x4c, - 0xcc, 0xab, 0x5e, 0x27, 0x51, 0x67, 0xb0, 0x8b, - 0xd6, 0x86, 0xf8, 0xa0, 0x9d, 0xf8, 0x7e, 0xc3, - 0xff, 0xb3, 0x53, 0x61, 0xb9, 0x4e, 0xbf, 0xa1, - 0x3f, 0xec, 0x0e, 0x48, 0x89, 0xd1, 0x8d, 0xa5, - } -}; - -//----------------------------------------------------------------------------- -// TC8: key: 'All your base are belong to us!, IV: 'IETF2013' -//----------------------------------------------------------------------------- - -const ui8 tc8_key[KEY_SIZE] = { - 0xc4, 0x6e, 0xc1, 0xb1, 0x8c, 0xe8, 0xa8, 0x78, - 0x72, 0x5a, 0x37, 0xe7, 0x80, 0xdf, 0xb7, 0x35, - 0x1f, 0x68, 0xed, 0x2e, 0x19, 0x4c, 0x79, 0xfb, - 0xc6, 0xae, 0xbe, 0xe1, 0xa6, 0x67, 0x97, 0x5d, -}; - -const ui8 tc8_iv[IV_SIZE] = { - 0x1a, 0xda, 0x31, 0xd5, 0xcf, 0x68, 0x82, 0x21, -}; - -const ui8 tc8_expected_8[][DATA_SIZE] = { - { - 0x83, 0x87, 0x51, 0xb4, 0x2d, 0x8d, 0xdd, 0x8a, - 0x3d, 0x77, 0xf4, 0x88, 0x25, 0xa2, 0xba, 0x75, - 0x2c, 0xf4, 0x04, 0x7c, 0xb3, 0x08, 0xa5, 0x97, - 0x8e, 0xf2, 0x74, 0x97, 0x3b, 0xe3, 0x74, 0xc9, - 0x6a, 0xd8, 0x48, 0x06, 0x58, 0x71, 0x41, 0x7b, - 0x08, 0xf0, 0x34, 0xe6, 0x81, 0xfe, 0x46, 0xa9, - 0x3f, 0x7d, 0x5c, 0x61, 0xd1, 0x30, 0x66, 0x14, - 0xd4, 0xaa, 0xf2, 0x57, 0xa7, 0xcf, 0xf0, 0x8b, - - }, - { - 0x16, 0xf2, 0xfd, 0xa1, 0x70, 0xcc, 0x18, 0xa4, - 0xb5, 0x8a, 0x26, 0x67, 0xed, 0x96, 0x27, 0x74, - 0xaf, 0x79, 0x2a, 0x6e, 0x7f, 0x3c, 0x77, 0x99, - 0x25, 0x40, 0x71, 0x1a, 0x7a, 0x13, 0x6d, 0x7e, - 0x8a, 0x2f, 0x8d, 0x3f, 0x93, 0x81, 0x67, 0x09, - 0xd4, 0x5a, 0x3f, 0xa5, 0xf8, 0xce, 0x72, 0xfd, - 0xe1, 0x5b, 0xe7, 0xb8, 0x41, 0xac, 0xba, 0x3a, - 0x2a, 0xbd, 0x55, 0x72, 0x28, 0xd9, 0xfe, 0x4f, - } -}; - -const ui8 tc8_expected_12[][DATA_SIZE] = { - { - 0x14, 0x82, 0x07, 0x27, 0x84, 0xbc, 0x6d, 0x06, - 0xb4, 0xe7, 0x3b, 0xdc, 0x11, 0x8b, 0xc0, 0x10, - 0x3c, 0x79, 0x76, 0x78, 0x6c, 0xa9, 0x18, 0xe0, - 0x69, 0x86, 0xaa, 0x25, 0x1f, 0x7e, 0x9c, 0xc1, - 0xb2, 0x74, 0x9a, 0x0a, 0x16, 0xee, 0x83, 0xb4, - 0x24, 0x2d, 0x2e, 0x99, 0xb0, 0x8d, 0x7c, 0x20, - 0x09, 0x2b, 0x80, 0xbc, 0x46, 0x6c, 0x87, 0x28, - 0x3b, 0x61, 0xb1, 0xb3, 0x9d, 0x0f, 0xfb, 0xab, - - }, - { - 0xd9, 0x4b, 0x11, 0x6b, 0xc1, 0xeb, 0xdb, 0x32, - 0x9b, 0x9e, 0x4f, 0x62, 0x0d, 0xb6, 0x95, 0x54, - 0x4a, 0x8e, 0x3d, 0x9b, 0x68, 0x47, 0x3d, 0x0c, - 0x97, 0x5a, 0x46, 0xad, 0x96, 0x6e, 0xd6, 0x31, - 0xe4, 0x2a, 0xff, 0x53, 0x0a, 0xd5, 0xea, 0xc7, - 0xd8, 0x04, 0x7a, 0xdf, 0xa1, 0xe5, 0x11, 0x3c, - 0x91, 0xf3, 0xe3, 0xb8, 0x83, 0xf1, 0xd1, 0x89, - 0xac, 0x1c, 0x8f, 0xe0, 0x7b, 0xa5, 0xa4, 0x2b, - } -}; - -const ui8 tc8_expected_20[][DATA_SIZE] = { - { - 0xf6, 0x3a, 0x89, 0xb7, 0x5c, 0x22, 0x71, 0xf9, - 0x36, 0x88, 0x16, 0x54, 0x2b, 0xa5, 0x2f, 0x06, - 0xed, 0x49, 0x24, 0x17, 0x92, 0x30, 0x2b, 0x00, - 0xb5, 0xe8, 0xf8, 0x0a, 0xe9, 0xa4, 0x73, 0xaf, - 0xc2, 0x5b, 0x21, 0x8f, 0x51, 0x9a, 0xf0, 0xfd, - 0xd4, 0x06, 0x36, 0x2e, 0x8d, 0x69, 0xde, 0x7f, - 0x54, 0xc6, 0x04, 0xa6, 0xe0, 0x0f, 0x35, 0x3f, - 0x11, 0x0f, 0x77, 0x1b, 0xdc, 0xa8, 0xab, 0x92, - - }, - { - 0xe5, 0xfb, 0xc3, 0x4e, 0x60, 0xa1, 0xd9, 0xa9, - 0xdb, 0x17, 0x34, 0x5b, 0x0a, 0x40, 0x27, 0x36, - 0x85, 0x3b, 0xf9, 0x10, 0xb0, 0x60, 0xbd, 0xf1, - 0xf8, 0x97, 0xb6, 0x29, 0x0f, 0x01, 0xd1, 0x38, - 0xae, 0x2c, 0x4c, 0x90, 0x22, 0x5b, 0xa9, 0xea, - 0x14, 0xd5, 0x18, 0xf5, 0x59, 0x29, 0xde, 0xa0, - 0x98, 0xca, 0x7a, 0x6c, 0xcf, 0xe6, 0x12, 0x27, - 0x05, 0x3c, 0x84, 0xe4, 0x9a, 0x4a, 0x33, 0x32, - } -}; +#pragma once + +// +// See http://tools.ietf.org/html/draft-strombergson-chacha-test-vectors-01 +// for source data +// + +#include <util/system/types.h> + +#define KEY_SIZE 32 +#define IV_SIZE 8 +#define DATA_SIZE 64 + + +//----------------------------------------------------------------------------- +// TC1: All zero key and IV. +//----------------------------------------------------------------------------- + +const ui8 tc1_key[KEY_SIZE] = { + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, +}; + +const ui8 tc1_iv[IV_SIZE] = { + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, +}; + +const ui8 tc1_expected_8[][DATA_SIZE] = { + { + 0x3e, 0x00, 0xef, 0x2f, 0x89, 0x5f, 0x40, 0xd6, + 0x7f, 0x5b, 0xb8, 0xe8, 0x1f, 0x09, 0xa5, 0xa1, + 0x2c, 0x84, 0x0e, 0xc3, 0xce, 0x9a, 0x7f, 0x3b, + 0x18, 0x1b, 0xe1, 0x88, 0xef, 0x71, 0x1a, 0x1e, + 0x98, 0x4c, 0xe1, 0x72, 0xb9, 0x21, 0x6f, 0x41, + 0x9f, 0x44, 0x53, 0x67, 0x45, 0x6d, 0x56, 0x19, + 0x31, 0x4a, 0x42, 0xa3, 0xda, 0x86, 0xb0, 0x01, + 0x38, 0x7b, 0xfd, 0xb8, 0x0e, 0x0c, 0xfe, 0x42, + + }, + { + 0xd2, 0xae, 0xfa, 0x0d, 0xea, 0xa5, 0xc1, 0x51, + 0xbf, 0x0a, 0xdb, 0x6c, 0x01, 0xf2, 0xa5, 0xad, + 0xc0, 0xfd, 0x58, 0x12, 0x59, 0xf9, 0xa2, 0xaa, + 0xdc, 0xf2, 0x0f, 0x8f, 0xd5, 0x66, 0xa2, 0x6b, + 0x50, 0x32, 0xec, 0x38, 0xbb, 0xc5, 0xda, 0x98, + 0xee, 0x0c, 0x6f, 0x56, 0x8b, 0x87, 0x2a, 0x65, + 0xa0, 0x8a, 0xbf, 0x25, 0x1d, 0xeb, 0x21, 0xbb, + 0x4b, 0x56, 0xe5, 0xd8, 0x82, 0x1e, 0x68, 0xaa, + } +}; + +const ui8 tc1_expected_12[][DATA_SIZE] = { + { + 0x9b, 0xf4, 0x9a, 0x6a, 0x07, 0x55, 0xf9, 0x53, + 0x81, 0x1f, 0xce, 0x12, 0x5f, 0x26, 0x83, 0xd5, + 0x04, 0x29, 0xc3, 0xbb, 0x49, 0xe0, 0x74, 0x14, + 0x7e, 0x00, 0x89, 0xa5, 0x2e, 0xae, 0x15, 0x5f, + 0x05, 0x64, 0xf8, 0x79, 0xd2, 0x7a, 0xe3, 0xc0, + 0x2c, 0xe8, 0x28, 0x34, 0xac, 0xfa, 0x8c, 0x79, + 0x3a, 0x62, 0x9f, 0x2c, 0xa0, 0xde, 0x69, 0x19, + 0x61, 0x0b, 0xe8, 0x2f, 0x41, 0x13, 0x26, 0xbe, + + }, + { + 0x0b, 0xd5, 0x88, 0x41, 0x20, 0x3e, 0x74, 0xfe, + 0x86, 0xfc, 0x71, 0x33, 0x8c, 0xe0, 0x17, 0x3d, + 0xc6, 0x28, 0xeb, 0xb7, 0x19, 0xbd, 0xcb, 0xcc, + 0x15, 0x15, 0x85, 0x21, 0x4c, 0xc0, 0x89, 0xb4, + 0x42, 0x25, 0x8d, 0xcd, 0xa1, 0x4c, 0xf1, 0x11, + 0xc6, 0x02, 0xb8, 0x97, 0x1b, 0x8c, 0xc8, 0x43, + 0xe9, 0x1e, 0x46, 0xca, 0x90, 0x51, 0x51, 0xc0, + 0x27, 0x44, 0xa6, 0xb0, 0x17, 0xe6, 0x93, 0x16, + } +}; + +const ui8 tc1_expected_20[][DATA_SIZE] = { + { + 0x76, 0xb8, 0xe0, 0xad, 0xa0, 0xf1, 0x3d, 0x90, + 0x40, 0x5d, 0x6a, 0xe5, 0x53, 0x86, 0xbd, 0x28, + 0xbd, 0xd2, 0x19, 0xb8, 0xa0, 0x8d, 0xed, 0x1a, + 0xa8, 0x36, 0xef, 0xcc, 0x8b, 0x77, 0x0d, 0xc7, + 0xda, 0x41, 0x59, 0x7c, 0x51, 0x57, 0x48, 0x8d, + 0x77, 0x24, 0xe0, 0x3f, 0xb8, 0xd8, 0x4a, 0x37, + 0x6a, 0x43, 0xb8, 0xf4, 0x15, 0x18, 0xa1, 0x1c, + 0xc3, 0x87, 0xb6, 0x69, 0xb2, 0xee, 0x65, 0x86, + + }, + { + 0x9f, 0x07, 0xe7, 0xbe, 0x55, 0x51, 0x38, 0x7a, + 0x98, 0xba, 0x97, 0x7c, 0x73, 0x2d, 0x08, 0x0d, + 0xcb, 0x0f, 0x29, 0xa0, 0x48, 0xe3, 0x65, 0x69, + 0x12, 0xc6, 0x53, 0x3e, 0x32, 0xee, 0x7a, 0xed, + 0x29, 0xb7, 0x21, 0x76, 0x9c, 0xe6, 0x4e, 0x43, + 0xd5, 0x71, 0x33, 0xb0, 0x74, 0xd8, 0x39, 0xd5, + 0x31, 0xed, 0x1f, 0x28, 0x51, 0x0a, 0xfb, 0x45, + 0xac, 0xe1, 0x0a, 0x1f, 0x4b, 0x79, 0x4d, 0x6f, + } +}; + +//----------------------------------------------------------------------------- +// TC2: Single bit in key set. All zero IV. +//----------------------------------------------------------------------------- + +const ui8 tc2_key[KEY_SIZE] = { + 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, +}; + +const ui8 tc2_iv[IV_SIZE] = { + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, +}; + +const ui8 tc2_expected_8[][DATA_SIZE] = { + { + 0xcf, 0x5e, 0xe9, 0xa0, 0x49, 0x4a, 0xa9, 0x61, + 0x3e, 0x05, 0xd5, 0xed, 0x72, 0x5b, 0x80, 0x4b, + 0x12, 0xf4, 0xa4, 0x65, 0xee, 0x63, 0x5a, 0xcc, + 0x3a, 0x31, 0x1d, 0xe8, 0x74, 0x04, 0x89, 0xea, + 0x28, 0x9d, 0x04, 0xf4, 0x3c, 0x75, 0x18, 0xdb, + 0x56, 0xeb, 0x44, 0x33, 0xe4, 0x98, 0xa1, 0x23, + 0x8c, 0xd8, 0x46, 0x4d, 0x37, 0x63, 0xdd, 0xbb, + 0x92, 0x22, 0xee, 0x3b, 0xd8, 0xfa, 0xe3, 0xc8, + + }, + { + 0xb4, 0x35, 0x5a, 0x7d, 0x93, 0xdd, 0x88, 0x67, + 0x08, 0x9e, 0xe6, 0x43, 0x55, 0x8b, 0x95, 0x75, + 0x4e, 0xfa, 0x2b, 0xd1, 0xa8, 0xa1, 0xe2, 0xd7, + 0x5b, 0xcd, 0xb3, 0x20, 0x15, 0x54, 0x26, 0x38, + 0x29, 0x19, 0x41, 0xfe, 0xb4, 0x99, 0x65, 0x58, + 0x7c, 0x4f, 0xdf, 0xe2, 0x19, 0xcf, 0x0e, 0xc1, + 0x32, 0xa6, 0xcd, 0x4d, 0xc0, 0x67, 0x39, 0x2e, + 0x67, 0x98, 0x2f, 0xe5, 0x32, 0x78, 0xc0, 0xb4, + } +}; + +const ui8 tc2_expected_12[][DATA_SIZE] = { + { + 0x12, 0x05, 0x6e, 0x59, 0x5d, 0x56, 0xb0, 0xf6, + 0xee, 0xf0, 0x90, 0xf0, 0xcd, 0x25, 0xa2, 0x09, + 0x49, 0x24, 0x8c, 0x27, 0x90, 0x52, 0x5d, 0x0f, + 0x93, 0x02, 0x18, 0xff, 0x0b, 0x4d, 0xdd, 0x10, + 0xa6, 0x00, 0x22, 0x39, 0xd9, 0xa4, 0x54, 0xe2, + 0x9e, 0x10, 0x7a, 0x7d, 0x06, 0xfe, 0xfd, 0xfe, + 0xf0, 0x21, 0x0f, 0xeb, 0xa0, 0x44, 0xf9, 0xf2, + 0x9b, 0x17, 0x72, 0xc9, 0x60, 0xdc, 0x29, 0xc0, + + }, + { + 0x0c, 0x73, 0x66, 0xc5, 0xcb, 0xc6, 0x04, 0x24, + 0x0e, 0x66, 0x5e, 0xb0, 0x2a, 0x69, 0x37, 0x2a, + 0x7a, 0xf9, 0x79, 0xb2, 0x6f, 0xbb, 0x78, 0x09, + 0x2a, 0xc7, 0xc4, 0xb8, 0x80, 0x29, 0xa7, 0xc8, + 0x54, 0x51, 0x3b, 0xc2, 0x17, 0xbb, 0xfc, 0x7d, + 0x90, 0x43, 0x2e, 0x30, 0x8e, 0xba, 0x15, 0xaf, + 0xc6, 0x5a, 0xeb, 0x48, 0xef, 0x10, 0x0d, 0x56, + 0x01, 0xe6, 0xaf, 0xba, 0x25, 0x71, 0x17, 0xa9, + } +}; + +const ui8 tc2_expected_20[][DATA_SIZE] = { + { + 0xc5, 0xd3, 0x0a, 0x7c, 0xe1, 0xec, 0x11, 0x93, + 0x78, 0xc8, 0x4f, 0x48, 0x7d, 0x77, 0x5a, 0x85, + 0x42, 0xf1, 0x3e, 0xce, 0x23, 0x8a, 0x94, 0x55, + 0xe8, 0x22, 0x9e, 0x88, 0x8d, 0xe8, 0x5b, 0xbd, + 0x29, 0xeb, 0x63, 0xd0, 0xa1, 0x7a, 0x5b, 0x99, + 0x9b, 0x52, 0xda, 0x22, 0xbe, 0x40, 0x23, 0xeb, + 0x07, 0x62, 0x0a, 0x54, 0xf6, 0xfa, 0x6a, 0xd8, + 0x73, 0x7b, 0x71, 0xeb, 0x04, 0x64, 0xda, 0xc0, + + }, + { + 0x10, 0xf6, 0x56, 0xe6, 0xd1, 0xfd, 0x55, 0x05, + 0x3e, 0x50, 0xc4, 0x87, 0x5c, 0x99, 0x30, 0xa3, + 0x3f, 0x6d, 0x02, 0x63, 0xbd, 0x14, 0xdf, 0xd6, + 0xab, 0x8c, 0x70, 0x52, 0x1c, 0x19, 0x33, 0x8b, + 0x23, 0x08, 0xb9, 0x5c, 0xf8, 0xd0, 0xbb, 0x7d, + 0x20, 0x2d, 0x21, 0x02, 0x78, 0x0e, 0xa3, 0x52, + 0x8f, 0x1c, 0xb4, 0x85, 0x60, 0xf7, 0x6b, 0x20, + 0xf3, 0x82, 0xb9, 0x42, 0x50, 0x0f, 0xce, 0xac, + } +}; + +//----------------------------------------------------------------------------- +// TC3: Single bit in IV set. All zero key. +//----------------------------------------------------------------------------- + +const ui8 tc3_key[KEY_SIZE] = { + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, +}; + +const ui8 tc3_iv[IV_SIZE] = { + 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, +}; + +const ui8 tc3_expected_8[][DATA_SIZE] = { + { + 0x2b, 0x8f, 0x4b, 0xb3, 0x79, 0x83, 0x06, 0xca, + 0x51, 0x30, 0xd4, 0x7c, 0x4f, 0x8d, 0x4e, 0xd1, + 0x3a, 0xa0, 0xed, 0xcc, 0xc1, 0xbe, 0x69, 0x42, + 0x09, 0x0f, 0xae, 0xec, 0xa0, 0xd7, 0x59, 0x9b, + 0x7f, 0xf0, 0xfe, 0x61, 0x6b, 0xb2, 0x5a, 0xa0, + 0x15, 0x3a, 0xd6, 0xfd, 0xc8, 0x8b, 0x95, 0x49, + 0x03, 0xc2, 0x24, 0x26, 0xd4, 0x78, 0xb9, 0x7b, + 0x22, 0xb8, 0xf9, 0xb1, 0xdb, 0x00, 0xcf, 0x06, + + }, + { + 0x47, 0x0b, 0xdf, 0xfb, 0xc4, 0x88, 0xa8, 0xb7, + 0xc7, 0x01, 0xeb, 0xf4, 0x06, 0x1d, 0x75, 0xc5, + 0x96, 0x91, 0x86, 0x49, 0x7c, 0x95, 0x36, 0x78, + 0x09, 0xaf, 0xa8, 0x0b, 0xd8, 0x43, 0xb0, 0x40, + 0xa7, 0x9a, 0xbc, 0x6e, 0x73, 0xa9, 0x17, 0x57, + 0xf1, 0xdb, 0x73, 0xc8, 0xea, 0xcf, 0xa5, 0x43, + 0xb3, 0x8f, 0x28, 0x9d, 0x06, 0x5a, 0xb2, 0xf3, + 0x03, 0x2d, 0x37, 0x7b, 0x8c, 0x37, 0xfe, 0x46, + } +}; + +const ui8 tc3_expected_12[][DATA_SIZE] = { + { + 0x64, 0xb8, 0xbd, 0xf8, 0x7b, 0x82, 0x8c, 0x4b, + 0x6d, 0xba, 0xf7, 0xef, 0x69, 0x8d, 0xe0, 0x3d, + 0xf8, 0xb3, 0x3f, 0x63, 0x57, 0x14, 0x41, 0x8f, + 0x98, 0x36, 0xad, 0xe5, 0x9b, 0xe1, 0x29, 0x69, + 0x46, 0xc9, 0x53, 0xa0, 0xf3, 0x8e, 0xcf, 0xfc, + 0x9e, 0xcb, 0x98, 0xe8, 0x1d, 0x5d, 0x99, 0xa5, + 0xed, 0xfc, 0x8f, 0x9a, 0x0a, 0x45, 0xb9, 0xe4, + 0x1e, 0xf3, 0xb3, 0x1f, 0x02, 0x8f, 0x1d, 0x0f, + + }, + { + 0x55, 0x9d, 0xb4, 0xa7, 0xf2, 0x22, 0xc4, 0x42, + 0xfe, 0x23, 0xb9, 0xa2, 0x59, 0x6a, 0x88, 0x28, + 0x51, 0x22, 0xee, 0x4f, 0x13, 0x63, 0x89, 0x6e, + 0xa7, 0x7c, 0xa1, 0x50, 0x91, 0x2a, 0xc7, 0x23, + 0xbf, 0xf0, 0x4b, 0x02, 0x6a, 0x2f, 0x80, 0x7e, + 0x03, 0xb2, 0x9c, 0x02, 0x07, 0x7d, 0x7b, 0x06, + 0xfc, 0x1a, 0xb9, 0x82, 0x7c, 0x13, 0xc8, 0x01, + 0x3a, 0x6d, 0x83, 0xbd, 0x3b, 0x52, 0xa2, 0x6f, + } +}; + +const ui8 tc3_expected_20[][DATA_SIZE] = { + { + 0xef, 0x3f, 0xdf, 0xd6, 0xc6, 0x15, 0x78, 0xfb, + 0xf5, 0xcf, 0x35, 0xbd, 0x3d, 0xd3, 0x3b, 0x80, + 0x09, 0x63, 0x16, 0x34, 0xd2, 0x1e, 0x42, 0xac, + 0x33, 0x96, 0x0b, 0xd1, 0x38, 0xe5, 0x0d, 0x32, + 0x11, 0x1e, 0x4c, 0xaf, 0x23, 0x7e, 0xe5, 0x3c, + 0xa8, 0xad, 0x64, 0x26, 0x19, 0x4a, 0x88, 0x54, + 0x5d, 0xdc, 0x49, 0x7a, 0x0b, 0x46, 0x6e, 0x7d, + 0x6b, 0xbd, 0xb0, 0x04, 0x1b, 0x2f, 0x58, 0x6b, + + }, + { + 0x53, 0x05, 0xe5, 0xe4, 0x4a, 0xff, 0x19, 0xb2, + 0x35, 0x93, 0x61, 0x44, 0x67, 0x5e, 0xfb, 0xe4, + 0x40, 0x9e, 0xb7, 0xe8, 0xe5, 0xf1, 0x43, 0x0f, + 0x5f, 0x58, 0x36, 0xae, 0xb4, 0x9b, 0xb5, 0x32, + 0x8b, 0x01, 0x7c, 0x4b, 0x9d, 0xc1, 0x1f, 0x8a, + 0x03, 0x86, 0x3f, 0xa8, 0x03, 0xdc, 0x71, 0xd5, + 0x72, 0x6b, 0x2b, 0x6b, 0x31, 0xaa, 0x32, 0x70, + 0x8a, 0xfe, 0x5a, 0xf1, 0xd6, 0xb6, 0x90, 0x58, + } +}; + +//----------------------------------------------------------------------------- +// TC4: All bits in key and IV are set. +//----------------------------------------------------------------------------- + +const ui8 tc4_key[KEY_SIZE] = { + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, +}; + +const ui8 tc4_iv[IV_SIZE] = { + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, +}; + +const ui8 tc4_expected_8[][DATA_SIZE] = { + { + 0xe1, 0x63, 0xbb, 0xf8, 0xc9, 0xa7, 0x39, 0xd1, + 0x89, 0x25, 0xee, 0x83, 0x62, 0xda, 0xd2, 0xcd, + 0xc9, 0x73, 0xdf, 0x05, 0x22, 0x5a, 0xfb, 0x2a, + 0xa2, 0x63, 0x96, 0xf2, 0xa9, 0x84, 0x9a, 0x4a, + 0x44, 0x5e, 0x05, 0x47, 0xd3, 0x1c, 0x16, 0x23, + 0xc5, 0x37, 0xdf, 0x4b, 0xa8, 0x5c, 0x70, 0xa9, + 0x88, 0x4a, 0x35, 0xbc, 0xbf, 0x3d, 0xfa, 0xb0, + 0x77, 0xe9, 0x8b, 0x0f, 0x68, 0x13, 0x5f, 0x54, + + }, + { + 0x81, 0xd4, 0x93, 0x3f, 0x8b, 0x32, 0x2a, 0xc0, + 0xcd, 0x76, 0x2c, 0x27, 0x23, 0x5c, 0xe2, 0xb3, + 0x15, 0x34, 0xe0, 0x24, 0x4a, 0x9a, 0x2f, 0x1f, + 0xd5, 0xe9, 0x44, 0x98, 0xd4, 0x7f, 0xf1, 0x08, + 0x79, 0x0c, 0x00, 0x9c, 0xf9, 0xe1, 0xa3, 0x48, + 0x03, 0x2a, 0x76, 0x94, 0xcb, 0x28, 0x02, 0x4c, + 0xd9, 0x6d, 0x34, 0x98, 0x36, 0x1e, 0xdb, 0x17, + 0x85, 0xaf, 0x75, 0x2d, 0x18, 0x7a, 0xb5, 0x4b, + } +}; + +const ui8 tc4_expected_12[][DATA_SIZE] = { + { + 0x04, 0xbf, 0x88, 0xda, 0xe8, 0xe4, 0x7a, 0x22, + 0x8f, 0xa4, 0x7b, 0x7e, 0x63, 0x79, 0x43, 0x4b, + 0xa6, 0x64, 0xa7, 0xd2, 0x8f, 0x4d, 0xab, 0x84, + 0xe5, 0xf8, 0xb4, 0x64, 0xad, 0xd2, 0x0c, 0x3a, + 0xca, 0xa6, 0x9c, 0x5a, 0xb2, 0x21, 0xa2, 0x3a, + 0x57, 0xeb, 0x5f, 0x34, 0x5c, 0x96, 0xf4, 0xd1, + 0x32, 0x2d, 0x0a, 0x2f, 0xf7, 0xa9, 0xcd, 0x43, + 0x40, 0x1c, 0xd5, 0x36, 0x63, 0x9a, 0x61, 0x5a, + + }, + { + 0x5c, 0x94, 0x29, 0xb5, 0x5c, 0xa3, 0xc1, 0xb5, + 0x53, 0x54, 0x55, 0x96, 0x69, 0xa1, 0x54, 0xac, + 0xa4, 0x6c, 0xd7, 0x61, 0xc4, 0x1a, 0xb8, 0xac, + 0xe3, 0x85, 0x36, 0x3b, 0x95, 0x67, 0x5f, 0x06, + 0x8e, 0x18, 0xdb, 0x5a, 0x67, 0x3c, 0x11, 0x29, + 0x1b, 0xd4, 0x18, 0x78, 0x92, 0xa9, 0xa3, 0xa3, + 0x35, 0x14, 0xf3, 0x71, 0x2b, 0x26, 0xc1, 0x30, + 0x26, 0x10, 0x32, 0x98, 0xed, 0x76, 0xbc, 0x9a, + } +}; + +const ui8 tc4_expected_20[][DATA_SIZE] = { + { + 0xd9, 0xbf, 0x3f, 0x6b, 0xce, 0x6e, 0xd0, 0xb5, + 0x42, 0x54, 0x55, 0x77, 0x67, 0xfb, 0x57, 0x44, + 0x3d, 0xd4, 0x77, 0x89, 0x11, 0xb6, 0x06, 0x05, + 0x5c, 0x39, 0xcc, 0x25, 0xe6, 0x74, 0xb8, 0x36, + 0x3f, 0xea, 0xbc, 0x57, 0xfd, 0xe5, 0x4f, 0x79, + 0x0c, 0x52, 0xc8, 0xae, 0x43, 0x24, 0x0b, 0x79, + 0xd4, 0x90, 0x42, 0xb7, 0x77, 0xbf, 0xd6, 0xcb, + 0x80, 0xe9, 0x31, 0x27, 0x0b, 0x7f, 0x50, 0xeb, + + }, + { + 0x5b, 0xac, 0x2a, 0xcd, 0x86, 0xa8, 0x36, 0xc5, + 0xdc, 0x98, 0xc1, 0x16, 0xc1, 0x21, 0x7e, 0xc3, + 0x1d, 0x3a, 0x63, 0xa9, 0x45, 0x13, 0x19, 0xf0, + 0x97, 0xf3, 0xb4, 0xd6, 0xda, 0xb0, 0x77, 0x87, + 0x19, 0x47, 0x7d, 0x24, 0xd2, 0x4b, 0x40, 0x3a, + 0x12, 0x24, 0x1d, 0x7c, 0xca, 0x06, 0x4f, 0x79, + 0x0f, 0x1d, 0x51, 0xcc, 0xaf, 0xf6, 0xb1, 0x66, + 0x7d, 0x4b, 0xbc, 0xa1, 0x95, 0x8c, 0x43, 0x06, + } +}; + +//----------------------------------------------------------------------------- +// TC5: Every even bit set in key and IV. +//----------------------------------------------------------------------------- + +const ui8 tc5_key[KEY_SIZE] = { + 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, + 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, + 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, + 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, +}; + +const ui8 tc5_iv[IV_SIZE] = { + 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, +}; + +const ui8 tc5_expected_8[][DATA_SIZE] = { + { + 0x7c, 0xb7, 0x82, 0x14, 0xe4, 0xd3, 0x46, 0x5b, + 0x6d, 0xc6, 0x2c, 0xf7, 0xa1, 0x53, 0x8c, 0x88, + 0x99, 0x69, 0x52, 0xb4, 0xfb, 0x72, 0xcb, 0x61, + 0x05, 0xf1, 0x24, 0x3c, 0xe3, 0x44, 0x2e, 0x29, + 0x75, 0xa5, 0x9e, 0xbc, 0xd2, 0xb2, 0xa5, 0x98, + 0x29, 0x0d, 0x75, 0x38, 0x49, 0x1f, 0xe6, 0x5b, + 0xdb, 0xfe, 0xfd, 0x06, 0x0d, 0x88, 0x79, 0x81, + 0x20, 0xa7, 0x0d, 0x04, 0x9d, 0xc2, 0x67, 0x7d, + + }, + { + 0xd4, 0x8f, 0xf5, 0xa2, 0x51, 0x3e, 0x49, 0x7a, + 0x5d, 0x54, 0x80, 0x2d, 0x74, 0x84, 0xc4, 0xf1, + 0x08, 0x39, 0x44, 0xd8, 0xd0, 0xd1, 0x4d, 0x64, + 0x82, 0xce, 0x09, 0xf7, 0xe5, 0xeb, 0xf2, 0x0b, + 0x29, 0x80, 0x7d, 0x62, 0xc3, 0x18, 0x74, 0xd0, + 0x2f, 0x5d, 0x3c, 0xc8, 0x53, 0x81, 0xa7, 0x45, + 0xec, 0xbc, 0x60, 0x52, 0x52, 0x05, 0xe3, 0x00, + 0xa7, 0x69, 0x61, 0xbf, 0xe5, 0x1a, 0xc0, 0x7c, + } +}; + +const ui8 tc5_expected_12[][DATA_SIZE] = { + { + 0xa6, 0x00, 0xf0, 0x77, 0x27, 0xff, 0x93, 0xf3, + 0xda, 0x00, 0xdd, 0x74, 0xcc, 0x3e, 0x8b, 0xfb, + 0x5c, 0xa7, 0x30, 0x2f, 0x6a, 0x0a, 0x29, 0x44, + 0x95, 0x3d, 0xe0, 0x04, 0x50, 0xee, 0xcd, 0x40, + 0xb8, 0x60, 0xf6, 0x60, 0x49, 0xf2, 0xea, 0xed, + 0x63, 0xb2, 0xef, 0x39, 0xcc, 0x31, 0x0d, 0x2c, + 0x48, 0x8f, 0x5d, 0x9a, 0x24, 0x1b, 0x61, 0x5d, + 0xc0, 0xab, 0x70, 0xf9, 0x21, 0xb9, 0x1b, 0x95, + + }, + { + 0x14, 0x0e, 0xff, 0x4a, 0xa4, 0x95, 0xac, 0x61, + 0x28, 0x9b, 0x6b, 0xc5, 0x7d, 0xe0, 0x72, 0x41, + 0x9d, 0x09, 0xda, 0xa7, 0xa7, 0x24, 0x39, 0x90, + 0xda, 0xf3, 0x48, 0xa8, 0xf2, 0x83, 0x1e, 0x59, + 0x7c, 0xf3, 0x79, 0xb3, 0xb2, 0x84, 0xf0, 0x0b, + 0xda, 0x27, 0xa4, 0xc6, 0x80, 0x85, 0x37, 0x4a, + 0x8a, 0x5c, 0x38, 0xde, 0xd6, 0x2d, 0x11, 0x41, + 0xca, 0xe0, 0xbb, 0x83, 0x8d, 0xdc, 0x22, 0x32, + } +}; + +const ui8 tc5_expected_20[][DATA_SIZE] = { + { + 0xbe, 0xa9, 0x41, 0x1a, 0xa4, 0x53, 0xc5, 0x43, + 0x4a, 0x5a, 0xe8, 0xc9, 0x28, 0x62, 0xf5, 0x64, + 0x39, 0x68, 0x55, 0xa9, 0xea, 0x6e, 0x22, 0xd6, + 0xd3, 0xb5, 0x0a, 0xe1, 0xb3, 0x66, 0x33, 0x11, + 0xa4, 0xa3, 0x60, 0x6c, 0x67, 0x1d, 0x60, 0x5c, + 0xe1, 0x6c, 0x3a, 0xec, 0xe8, 0xe6, 0x1e, 0xa1, + 0x45, 0xc5, 0x97, 0x75, 0x01, 0x7b, 0xee, 0x2f, + 0xa6, 0xf8, 0x8a, 0xfc, 0x75, 0x80, 0x69, 0xf7, + + }, + { + 0xe0, 0xb8, 0xf6, 0x76, 0xe6, 0x44, 0x21, 0x6f, + 0x4d, 0x2a, 0x34, 0x22, 0xd7, 0xfa, 0x36, 0xc6, + 0xc4, 0x93, 0x1a, 0xca, 0x95, 0x0e, 0x9d, 0xa4, + 0x27, 0x88, 0xe6, 0xd0, 0xb6, 0xd1, 0xcd, 0x83, + 0x8e, 0xf6, 0x52, 0xe9, 0x7b, 0x14, 0x5b, 0x14, + 0x87, 0x1e, 0xae, 0x6c, 0x68, 0x04, 0xc7, 0x00, + 0x4d, 0xb5, 0xac, 0x2f, 0xce, 0x4c, 0x68, 0xc7, + 0x26, 0xd0, 0x04, 0xb1, 0x0f, 0xca, 0xba, 0x86, + } +}; + +//----------------------------------------------------------------------------- +// TC6: Every odd bit set in key and IV. +//----------------------------------------------------------------------------- + +const ui8 tc6_key[KEY_SIZE] = { + 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, + 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, + 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, + 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, +}; + +const ui8 tc6_iv[IV_SIZE] = { + 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, 0xaa, +}; + +const ui8 tc6_expected_8[][DATA_SIZE] = { + { + 0x40, 0xf9, 0xab, 0x86, 0xc8, 0xf9, 0xa1, 0xa0, + 0xcd, 0xc0, 0x5a, 0x75, 0xe5, 0x53, 0x1b, 0x61, + 0x2d, 0x71, 0xef, 0x7f, 0x0c, 0xf9, 0xe3, 0x87, + 0xdf, 0x6e, 0xd6, 0x97, 0x2f, 0x0a, 0xae, 0x21, + 0x31, 0x1a, 0xa5, 0x81, 0xf8, 0x16, 0xc9, 0x0e, + 0x8a, 0x99, 0xde, 0x99, 0x0b, 0x6b, 0x95, 0xaa, + 0xc9, 0x24, 0x50, 0xf4, 0xe1, 0x12, 0x71, 0x26, + 0x67, 0xb8, 0x04, 0xc9, 0x9e, 0x9c, 0x6e, 0xda, + + }, + { + 0xf8, 0xd1, 0x44, 0xf5, 0x60, 0xc8, 0xc0, 0xea, + 0x36, 0x88, 0x0d, 0x3b, 0x77, 0x87, 0x4c, 0x9a, + 0x91, 0x03, 0xd1, 0x47, 0xf6, 0xde, 0xd3, 0x86, + 0x28, 0x48, 0x01, 0xa4, 0xee, 0x15, 0x8e, 0x5e, + 0xa4, 0xf9, 0xc0, 0x93, 0xfc, 0x55, 0xfd, 0x34, + 0x4c, 0x33, 0x34, 0x9d, 0xc5, 0xb6, 0x99, 0xe2, + 0x1d, 0xc8, 0x3b, 0x42, 0x96, 0xf9, 0x2e, 0xe3, + 0xec, 0xab, 0xf3, 0xd5, 0x1f, 0x95, 0xfe, 0x3f, + } +}; + +const ui8 tc6_expected_12[][DATA_SIZE] = { + { + 0x85, 0x65, 0x05, 0xb0, 0x1d, 0x3b, 0x47, 0xaa, + 0xe0, 0x3d, 0x6a, 0x97, 0xaa, 0x0f, 0x03, 0x3a, + 0x9a, 0xdc, 0xc9, 0x43, 0x77, 0xba, 0xbd, 0x86, + 0x08, 0x86, 0x4f, 0xb3, 0xf6, 0x25, 0xb6, 0xe3, + 0x14, 0xf0, 0x86, 0x15, 0x8f, 0x9f, 0x72, 0x5d, + 0x81, 0x1e, 0xeb, 0x95, 0x3b, 0x7f, 0x74, 0x70, + 0x76, 0xe4, 0xc3, 0xf6, 0x39, 0xfa, 0x84, 0x1f, + 0xad, 0x6c, 0x9a, 0x70, 0x9e, 0x62, 0x13, 0x97, + + }, + { + 0x6d, 0xd6, 0xee, 0x9b, 0x5e, 0x1e, 0x2e, 0x67, + 0x6b, 0x1c, 0x9e, 0x2b, 0x82, 0xc2, 0xe9, 0x6c, + 0x16, 0x48, 0x43, 0x7b, 0xff, 0x2f, 0x01, 0x26, + 0xb7, 0x4e, 0x8c, 0xe0, 0xa9, 0xb0, 0x6d, 0x17, + 0x20, 0xac, 0x0b, 0x6f, 0x09, 0x08, 0x6f, 0x28, + 0xbc, 0x20, 0x15, 0x87, 0xf0, 0x53, 0x5e, 0xd9, + 0x38, 0x52, 0x70, 0xd0, 0x8b, 0x4a, 0x93, 0x82, + 0xf1, 0x8f, 0x82, 0xdb, 0xde, 0x18, 0x21, 0x0e, + } +}; + +const ui8 tc6_expected_20[][DATA_SIZE] = { + { + 0x9a, 0xa2, 0xa9, 0xf6, 0x56, 0xef, 0xde, 0x5a, + 0xa7, 0x59, 0x1c, 0x5f, 0xed, 0x4b, 0x35, 0xae, + 0xa2, 0x89, 0x5d, 0xec, 0x7c, 0xb4, 0x54, 0x3b, + 0x9e, 0x9f, 0x21, 0xf5, 0xe7, 0xbc, 0xbc, 0xf3, + 0xc4, 0x3c, 0x74, 0x8a, 0x97, 0x08, 0x88, 0xf8, + 0x24, 0x83, 0x93, 0xa0, 0x9d, 0x43, 0xe0, 0xb7, + 0xe1, 0x64, 0xbc, 0x4d, 0x0b, 0x0f, 0xb2, 0x40, + 0xa2, 0xd7, 0x21, 0x15, 0xc4, 0x80, 0x89, 0x06, + + }, + { + 0x72, 0x18, 0x44, 0x89, 0x44, 0x05, 0x45, 0xd0, + 0x21, 0xd9, 0x7e, 0xf6, 0xb6, 0x93, 0xdf, 0xe5, + 0xb2, 0xc1, 0x32, 0xd4, 0x7e, 0x6f, 0x04, 0x1c, + 0x90, 0x63, 0x65, 0x1f, 0x96, 0xb6, 0x23, 0xe6, + 0x2a, 0x11, 0x99, 0x9a, 0x23, 0xb6, 0xf7, 0xc4, + 0x61, 0xb2, 0x15, 0x30, 0x26, 0xad, 0x5e, 0x86, + 0x6a, 0x2e, 0x59, 0x7e, 0xd0, 0x7b, 0x84, 0x01, + 0xde, 0xc6, 0x3a, 0x09, 0x34, 0xc6, 0xb2, 0xa9, + } +}; + +//----------------------------------------------------------------------------- +// TC7: Sequence patterns in key and IV. +//----------------------------------------------------------------------------- + +const ui8 tc7_key[KEY_SIZE] = { + 0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, + 0x88, 0x99, 0xaa, 0xbb, 0xcc, 0xdd, 0xee, 0xff, + 0xff, 0xee, 0xdd, 0xcc, 0xbb, 0xaa, 0x99, 0x88, + 0x77, 0x66, 0x55, 0x44, 0x33, 0x22, 0x11, 0x00, +}; + +const ui8 tc7_iv[IV_SIZE] = { + 0x0f, 0x1e, 0x2d, 0x3c, 0x4b, 0x5a, 0x69, 0x78, +}; + +const ui8 tc7_expected_8[][DATA_SIZE] = { + { + 0xdb, 0x43, 0xad, 0x9d, 0x1e, 0x84, 0x2d, 0x12, + 0x72, 0xe4, 0x53, 0x0e, 0x27, 0x6b, 0x3f, 0x56, + 0x8f, 0x88, 0x59, 0xb3, 0xf7, 0xcf, 0x6d, 0x9d, + 0x2c, 0x74, 0xfa, 0x53, 0x80, 0x8c, 0xb5, 0x15, + 0x7a, 0x8e, 0xbf, 0x46, 0xad, 0x3d, 0xcc, 0x4b, + 0x6c, 0x7d, 0xad, 0xde, 0x13, 0x17, 0x84, 0xb0, + 0x12, 0x0e, 0x0e, 0x22, 0xf6, 0xd5, 0xf9, 0xff, + 0xa7, 0x40, 0x7d, 0x4a, 0x21, 0xb6, 0x95, 0xd9, + + }, + { + 0xc5, 0xdd, 0x30, 0xbf, 0x55, 0x61, 0x2f, 0xab, + 0x9b, 0xdd, 0x11, 0x89, 0x20, 0xc1, 0x98, 0x16, + 0x47, 0x0c, 0x7f, 0x5d, 0xcd, 0x42, 0x32, 0x5d, + 0xbb, 0xed, 0x8c, 0x57, 0xa5, 0x62, 0x81, 0xc1, + 0x44, 0xcb, 0x0f, 0x03, 0xe8, 0x1b, 0x30, 0x04, + 0x62, 0x4e, 0x06, 0x50, 0xa1, 0xce, 0x5a, 0xfa, + 0xf9, 0xa7, 0xcd, 0x81, 0x63, 0xf6, 0xdb, 0xd7, + 0x26, 0x02, 0x25, 0x7d, 0xd9, 0x6e, 0x47, 0x1e, + } +}; + +const ui8 tc7_expected_12[][DATA_SIZE] = { + { + 0x7e, 0xd1, 0x2a, 0x3a, 0x63, 0x91, 0x2a, 0xe9, + 0x41, 0xba, 0x6d, 0x4c, 0x0d, 0x5e, 0x86, 0x2e, + 0x56, 0x8b, 0x0e, 0x55, 0x89, 0x34, 0x69, 0x35, + 0x50, 0x5f, 0x06, 0x4b, 0x8c, 0x26, 0x98, 0xdb, + 0xf7, 0xd8, 0x50, 0x66, 0x7d, 0x8e, 0x67, 0xbe, + 0x63, 0x9f, 0x3b, 0x4f, 0x6a, 0x16, 0xf9, 0x2e, + 0x65, 0xea, 0x80, 0xf6, 0xc7, 0x42, 0x94, 0x45, + 0xda, 0x1f, 0xc2, 0xc1, 0xb9, 0x36, 0x50, 0x40, + + }, + { + 0xe3, 0x2e, 0x50, 0xc4, 0x10, 0x6f, 0x3b, 0x3d, + 0xa1, 0xce, 0x7c, 0xcb, 0x1e, 0x71, 0x40, 0xb1, + 0x53, 0x49, 0x3c, 0x0f, 0x3a, 0xd9, 0xa9, 0xbc, + 0xff, 0x07, 0x7e, 0xc4, 0x59, 0x6f, 0x1d, 0x0f, + 0x29, 0xbf, 0x9c, 0xba, 0xa5, 0x02, 0x82, 0x0f, + 0x73, 0x2a, 0xf5, 0xa9, 0x3c, 0x49, 0xee, 0xe3, + 0x3d, 0x1c, 0x4f, 0x12, 0xaf, 0x3b, 0x42, 0x97, + 0xaf, 0x91, 0xfe, 0x41, 0xea, 0x9e, 0x94, 0xa2, + } +}; + +const ui8 tc7_expected_20[][DATA_SIZE] = { + { + 0x9f, 0xad, 0xf4, 0x09, 0xc0, 0x08, 0x11, 0xd0, + 0x04, 0x31, 0xd6, 0x7e, 0xfb, 0xd8, 0x8f, 0xba, + 0x59, 0x21, 0x8d, 0x5d, 0x67, 0x08, 0xb1, 0xd6, + 0x85, 0x86, 0x3f, 0xab, 0xbb, 0x0e, 0x96, 0x1e, + 0xea, 0x48, 0x0f, 0xd6, 0xfb, 0x53, 0x2b, 0xfd, + 0x49, 0x4b, 0x21, 0x51, 0x01, 0x50, 0x57, 0x42, + 0x3a, 0xb6, 0x0a, 0x63, 0xfe, 0x4f, 0x55, 0xf7, + 0xa2, 0x12, 0xe2, 0x16, 0x7c, 0xca, 0xb9, 0x31, + + }, + { + 0xfb, 0xfd, 0x29, 0xcf, 0x7b, 0xc1, 0xd2, 0x79, + 0xed, 0xdf, 0x25, 0xdd, 0x31, 0x6b, 0xb8, 0x84, + 0x3d, 0x6e, 0xde, 0xe0, 0xbd, 0x1e, 0xf1, 0x21, + 0xd1, 0x2f, 0xa1, 0x7c, 0xbc, 0x2c, 0x57, 0x4c, + 0xcc, 0xab, 0x5e, 0x27, 0x51, 0x67, 0xb0, 0x8b, + 0xd6, 0x86, 0xf8, 0xa0, 0x9d, 0xf8, 0x7e, 0xc3, + 0xff, 0xb3, 0x53, 0x61, 0xb9, 0x4e, 0xbf, 0xa1, + 0x3f, 0xec, 0x0e, 0x48, 0x89, 0xd1, 0x8d, 0xa5, + } +}; + +//----------------------------------------------------------------------------- +// TC8: key: 'All your base are belong to us!, IV: 'IETF2013' +//----------------------------------------------------------------------------- + +const ui8 tc8_key[KEY_SIZE] = { + 0xc4, 0x6e, 0xc1, 0xb1, 0x8c, 0xe8, 0xa8, 0x78, + 0x72, 0x5a, 0x37, 0xe7, 0x80, 0xdf, 0xb7, 0x35, + 0x1f, 0x68, 0xed, 0x2e, 0x19, 0x4c, 0x79, 0xfb, + 0xc6, 0xae, 0xbe, 0xe1, 0xa6, 0x67, 0x97, 0x5d, +}; + +const ui8 tc8_iv[IV_SIZE] = { + 0x1a, 0xda, 0x31, 0xd5, 0xcf, 0x68, 0x82, 0x21, +}; + +const ui8 tc8_expected_8[][DATA_SIZE] = { + { + 0x83, 0x87, 0x51, 0xb4, 0x2d, 0x8d, 0xdd, 0x8a, + 0x3d, 0x77, 0xf4, 0x88, 0x25, 0xa2, 0xba, 0x75, + 0x2c, 0xf4, 0x04, 0x7c, 0xb3, 0x08, 0xa5, 0x97, + 0x8e, 0xf2, 0x74, 0x97, 0x3b, 0xe3, 0x74, 0xc9, + 0x6a, 0xd8, 0x48, 0x06, 0x58, 0x71, 0x41, 0x7b, + 0x08, 0xf0, 0x34, 0xe6, 0x81, 0xfe, 0x46, 0xa9, + 0x3f, 0x7d, 0x5c, 0x61, 0xd1, 0x30, 0x66, 0x14, + 0xd4, 0xaa, 0xf2, 0x57, 0xa7, 0xcf, 0xf0, 0x8b, + + }, + { + 0x16, 0xf2, 0xfd, 0xa1, 0x70, 0xcc, 0x18, 0xa4, + 0xb5, 0x8a, 0x26, 0x67, 0xed, 0x96, 0x27, 0x74, + 0xaf, 0x79, 0x2a, 0x6e, 0x7f, 0x3c, 0x77, 0x99, + 0x25, 0x40, 0x71, 0x1a, 0x7a, 0x13, 0x6d, 0x7e, + 0x8a, 0x2f, 0x8d, 0x3f, 0x93, 0x81, 0x67, 0x09, + 0xd4, 0x5a, 0x3f, 0xa5, 0xf8, 0xce, 0x72, 0xfd, + 0xe1, 0x5b, 0xe7, 0xb8, 0x41, 0xac, 0xba, 0x3a, + 0x2a, 0xbd, 0x55, 0x72, 0x28, 0xd9, 0xfe, 0x4f, + } +}; + +const ui8 tc8_expected_12[][DATA_SIZE] = { + { + 0x14, 0x82, 0x07, 0x27, 0x84, 0xbc, 0x6d, 0x06, + 0xb4, 0xe7, 0x3b, 0xdc, 0x11, 0x8b, 0xc0, 0x10, + 0x3c, 0x79, 0x76, 0x78, 0x6c, 0xa9, 0x18, 0xe0, + 0x69, 0x86, 0xaa, 0x25, 0x1f, 0x7e, 0x9c, 0xc1, + 0xb2, 0x74, 0x9a, 0x0a, 0x16, 0xee, 0x83, 0xb4, + 0x24, 0x2d, 0x2e, 0x99, 0xb0, 0x8d, 0x7c, 0x20, + 0x09, 0x2b, 0x80, 0xbc, 0x46, 0x6c, 0x87, 0x28, + 0x3b, 0x61, 0xb1, 0xb3, 0x9d, 0x0f, 0xfb, 0xab, + + }, + { + 0xd9, 0x4b, 0x11, 0x6b, 0xc1, 0xeb, 0xdb, 0x32, + 0x9b, 0x9e, 0x4f, 0x62, 0x0d, 0xb6, 0x95, 0x54, + 0x4a, 0x8e, 0x3d, 0x9b, 0x68, 0x47, 0x3d, 0x0c, + 0x97, 0x5a, 0x46, 0xad, 0x96, 0x6e, 0xd6, 0x31, + 0xe4, 0x2a, 0xff, 0x53, 0x0a, 0xd5, 0xea, 0xc7, + 0xd8, 0x04, 0x7a, 0xdf, 0xa1, 0xe5, 0x11, 0x3c, + 0x91, 0xf3, 0xe3, 0xb8, 0x83, 0xf1, 0xd1, 0x89, + 0xac, 0x1c, 0x8f, 0xe0, 0x7b, 0xa5, 0xa4, 0x2b, + } +}; + +const ui8 tc8_expected_20[][DATA_SIZE] = { + { + 0xf6, 0x3a, 0x89, 0xb7, 0x5c, 0x22, 0x71, 0xf9, + 0x36, 0x88, 0x16, 0x54, 0x2b, 0xa5, 0x2f, 0x06, + 0xed, 0x49, 0x24, 0x17, 0x92, 0x30, 0x2b, 0x00, + 0xb5, 0xe8, 0xf8, 0x0a, 0xe9, 0xa4, 0x73, 0xaf, + 0xc2, 0x5b, 0x21, 0x8f, 0x51, 0x9a, 0xf0, 0xfd, + 0xd4, 0x06, 0x36, 0x2e, 0x8d, 0x69, 0xde, 0x7f, + 0x54, 0xc6, 0x04, 0xa6, 0xe0, 0x0f, 0x35, 0x3f, + 0x11, 0x0f, 0x77, 0x1b, 0xdc, 0xa8, 0xab, 0x92, + + }, + { + 0xe5, 0xfb, 0xc3, 0x4e, 0x60, 0xa1, 0xd9, 0xa9, + 0xdb, 0x17, 0x34, 0x5b, 0x0a, 0x40, 0x27, 0x36, + 0x85, 0x3b, 0xf9, 0x10, 0xb0, 0x60, 0xbd, 0xf1, + 0xf8, 0x97, 0xb6, 0x29, 0x0f, 0x01, 0xd1, 0x38, + 0xae, 0x2c, 0x4c, 0x90, 0x22, 0x5b, 0xa9, 0xea, + 0x14, 0xd5, 0x18, 0xf5, 0x59, 0x29, 0xde, 0xa0, + 0x98, 0xca, 0x7a, 0x6c, 0xcf, 0xe6, 0x12, 0x27, + 0x05, 0x3c, 0x84, 0xe4, 0x9a, 0x4a, 0x33, 0x32, + } +}; diff --git a/ydb/core/blobstorage/crypto/ut/poly1305_test_vectors.h b/ydb/core/blobstorage/crypto/ut/poly1305_test_vectors.h index 5c493dc7f81..37284d569ec 100644 --- a/ydb/core/blobstorage/crypto/ut/poly1305_test_vectors.h +++ b/ydb/core/blobstorage/crypto/ut/poly1305_test_vectors.h @@ -1,94 +1,94 @@ -#pragma once - -// -// See Appendix A.3 at https://tools.ietf.org/html/draft-nir-cfrg-chacha20-poly1305-06 -// for source data -// - -#include <util/system/types.h> - -#define KEY_SIZE 32 -#define TAG_SIZE 16 - - -//----------------------------------------------------------------------------- -// TEST CASE #1 -//----------------------------------------------------------------------------- - -const ui8 tc1_key[KEY_SIZE] = { 0x00 }; - -const ui8 tc1_data[64] = { 0x00 }; - -const ui8 tc1_tag[TAG_SIZE] = { 0x00 }; - -//----------------------------------------------------------------------------- -// TEST CASE #2 -//----------------------------------------------------------------------------- - -const ui8 tc2_key[KEY_SIZE] = { - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x36, 0xe5, 0xf6, 0xb5, 0xc5, 0xe0, 0x60, 0x70, - 0xf0, 0xef, 0xca, 0x96, 0x22, 0x7a, 0x86, 0x3e, -}; - -const ui8 tc2_data[] = - "Any submission to the IETF intended by the Contributor " - "for publication as all or part of an IETF Internet-Draft" - "or RFC and any statement made within the context of an IETF " - "activity is considered an \"IETF Contribution\". Such statements " - "include oral statements in IETF sessions, as well as written and " - "electronic communications made at any time or place, which are " - "addressed to"; - -const ui8 tc2_tag[TAG_SIZE] = { - 0x36, 0xe5, 0xf6, 0xb5, 0xc5, 0xe0, 0x60, 0x70, - 0xf0, 0xef, 0xca, 0x96, 0x22, 0x7a, 0x86, 0x3e, -}; - -//----------------------------------------------------------------------------- -// TEST CASE #3 -//----------------------------------------------------------------------------- - -const ui8 tc3_key[KEY_SIZE] = { - 0x36, 0xe5, 0xf6, 0xb5, 0xc5, 0xe0, 0x60, 0x70, - 0xf0, 0xef, 0xca, 0x96, 0x22, 0x7a, 0x86, 0x3e, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, -}; - -const ui8 tc3_data[] = - "Any submission to the IETF intended by the Contributor " - "for publication as all or part of an IETF Internet-Draft " - "or RFC and any statement made within the context of an IETF " - "activity is considered an \"IETF Contribution\". Such statements " - "include oral statements in IETF sessions, as well as written and " - "electronic communications made at any time or place, which are " - "addressed to"; - -const ui8 tc3_tag[TAG_SIZE] = { - 0xf3, 0x47, 0x7e, 0x7c, 0xd9, 0x54, 0x17, 0xaf, - 0x89, 0xa6, 0xb8, 0x79, 0x4c, 0x31, 0x0c, 0xf0, -}; - -//----------------------------------------------------------------------------- -// TEST CASE #4 -//----------------------------------------------------------------------------- - -const ui8 tc4_key[KEY_SIZE] = { - 0x1c, 0x92, 0x40, 0xa5, 0xeb, 0x55, 0xd3, 0x8a, - 0xf3, 0x33, 0x88, 0x86, 0x04, 0xf6, 0xb5, 0xf0, - 0x47, 0x39, 0x17, 0xc1, 0x40, 0x2b, 0x80, 0x09, - 0x9d, 0xca, 0x5c, 0xbc, 0x20, 0x70, 0x75, 0xc0, -}; - -const ui8 tc4_data[] = - "'Twas brillig, and the slithy toves\n" - "Did gyre and gimble in the wabe:\n" - "All mimsy were the borogoves,\n" - "And the mome raths outgrabe."; - -const ui8 tc4_tag[TAG_SIZE] = { - 0x45, 0x41, 0x66, 0x9a, 0x7e, 0xaa, 0xee, 0x61, - 0xe7, 0x08, 0xdc, 0x7c, 0xbc, 0xc5, 0xeb, 0x62, -}; +#pragma once + +// +// See Appendix A.3 at https://tools.ietf.org/html/draft-nir-cfrg-chacha20-poly1305-06 +// for source data +// + +#include <util/system/types.h> + +#define KEY_SIZE 32 +#define TAG_SIZE 16 + + +//----------------------------------------------------------------------------- +// TEST CASE #1 +//----------------------------------------------------------------------------- + +const ui8 tc1_key[KEY_SIZE] = { 0x00 }; + +const ui8 tc1_data[64] = { 0x00 }; + +const ui8 tc1_tag[TAG_SIZE] = { 0x00 }; + +//----------------------------------------------------------------------------- +// TEST CASE #2 +//----------------------------------------------------------------------------- + +const ui8 tc2_key[KEY_SIZE] = { + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x36, 0xe5, 0xf6, 0xb5, 0xc5, 0xe0, 0x60, 0x70, + 0xf0, 0xef, 0xca, 0x96, 0x22, 0x7a, 0x86, 0x3e, +}; + +const ui8 tc2_data[] = + "Any submission to the IETF intended by the Contributor " + "for publication as all or part of an IETF Internet-Draft" + "or RFC and any statement made within the context of an IETF " + "activity is considered an \"IETF Contribution\". Such statements " + "include oral statements in IETF sessions, as well as written and " + "electronic communications made at any time or place, which are " + "addressed to"; + +const ui8 tc2_tag[TAG_SIZE] = { + 0x36, 0xe5, 0xf6, 0xb5, 0xc5, 0xe0, 0x60, 0x70, + 0xf0, 0xef, 0xca, 0x96, 0x22, 0x7a, 0x86, 0x3e, +}; + +//----------------------------------------------------------------------------- +// TEST CASE #3 +//----------------------------------------------------------------------------- + +const ui8 tc3_key[KEY_SIZE] = { + 0x36, 0xe5, 0xf6, 0xb5, 0xc5, 0xe0, 0x60, 0x70, + 0xf0, 0xef, 0xca, 0x96, 0x22, 0x7a, 0x86, 0x3e, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, +}; + +const ui8 tc3_data[] = + "Any submission to the IETF intended by the Contributor " + "for publication as all or part of an IETF Internet-Draft " + "or RFC and any statement made within the context of an IETF " + "activity is considered an \"IETF Contribution\". Such statements " + "include oral statements in IETF sessions, as well as written and " + "electronic communications made at any time or place, which are " + "addressed to"; + +const ui8 tc3_tag[TAG_SIZE] = { + 0xf3, 0x47, 0x7e, 0x7c, 0xd9, 0x54, 0x17, 0xaf, + 0x89, 0xa6, 0xb8, 0x79, 0x4c, 0x31, 0x0c, 0xf0, +}; + +//----------------------------------------------------------------------------- +// TEST CASE #4 +//----------------------------------------------------------------------------- + +const ui8 tc4_key[KEY_SIZE] = { + 0x1c, 0x92, 0x40, 0xa5, 0xeb, 0x55, 0xd3, 0x8a, + 0xf3, 0x33, 0x88, 0x86, 0x04, 0xf6, 0xb5, 0xf0, + 0x47, 0x39, 0x17, 0xc1, 0x40, 0x2b, 0x80, 0x09, + 0x9d, 0xca, 0x5c, 0xbc, 0x20, 0x70, 0x75, 0xc0, +}; + +const ui8 tc4_data[] = + "'Twas brillig, and the slithy toves\n" + "Did gyre and gimble in the wabe:\n" + "All mimsy were the borogoves,\n" + "And the mome raths outgrabe."; + +const ui8 tc4_tag[TAG_SIZE] = { + 0x45, 0x41, 0x66, 0x9a, 0x7e, 0xaa, 0xee, 0x61, + 0xe7, 0x08, 0xdc, 0x7c, 0xbc, 0xc5, 0xeb, 0x62, +}; diff --git a/ydb/core/blobstorage/crypto/ut/ut_helpers.h b/ydb/core/blobstorage/crypto/ut/ut_helpers.h index 920a80fc00d..e9695541df2 100644 --- a/ydb/core/blobstorage/crypto/ut/ut_helpers.h +++ b/ydb/core/blobstorage/crypto/ut/ut_helpers.h @@ -1,9 +1,9 @@ -#pragma once - +#pragma once + #include <library/cpp/actors/util/rope.h> #include <library/cpp/testing/unittest/registar.h> #include <util/system/align.h> - + #include <utility> class TAlignedBuf { @@ -11,7 +11,7 @@ class TAlignedBuf { ui8 *AlignedBuf; size_t BufSize; size_t Align; - + void AllocBuf() { Buf = new ui8[BufSize + Align]; AlignedBuf = reinterpret_cast<ui8*>(AlignUp<intptr_t>(intptr_t(Buf), Align)); @@ -96,4 +96,4 @@ void inline Print(const ui8* out, size_t size) { " a[" << i << "]# " << Hex((A)[i], HF_FULL) << " != " \ " b[" << i << "]# " << Hex((B)[i], HF_FULL)); \ } \ - } while (0) + } while (0) diff --git a/ydb/core/blobstorage/crypto/ut/ya.make b/ydb/core/blobstorage/crypto/ut/ya.make index 470e0d17147..30ec24f750b 100644 --- a/ydb/core/blobstorage/crypto/ut/ya.make +++ b/ydb/core/blobstorage/crypto/ut/ya.make @@ -1,22 +1,22 @@ UNITTEST_FOR(ydb/core/blobstorage/crypto) - -OWNER(g:kikimr) - + +OWNER(g:kikimr) + IF (NOT OS_WINDOWS) - SRCS( - chacha_ut.cpp - chacha_vec_ut.cpp - crypto_rope_ut.cpp - crypto_ut.cpp - poly1305_ut.cpp - poly1305_vec_ut.cpp - ) -ELSE() - SRCS( - chacha_ut.cpp - crypto_ut.cpp - poly1305_ut.cpp - ) -ENDIF() - -END() + SRCS( + chacha_ut.cpp + chacha_vec_ut.cpp + crypto_rope_ut.cpp + crypto_ut.cpp + poly1305_ut.cpp + poly1305_vec_ut.cpp + ) +ELSE() + SRCS( + chacha_ut.cpp + crypto_ut.cpp + poly1305_ut.cpp + ) +ENDIF() + +END() diff --git a/ydb/core/blobstorage/crypto/ya.make b/ydb/core/blobstorage/crypto/ya.make index fcca0f28188..4540b3e8373 100644 --- a/ydb/core/blobstorage/crypto/ya.make +++ b/ydb/core/blobstorage/crypto/ya.make @@ -1,34 +1,34 @@ -LIBRARY() - -OWNER(g:kikimr) - +LIBRARY() + +OWNER(g:kikimr) + IF (NOT OS_WINDOWS AND NOT ARCH_ARM64) - SRCS( - chacha.cpp - crypto.cpp - crypto.h - poly1305.cpp - chacha_vec.cpp - poly1305_vec.cpp - secured_block.cpp - ) -ELSE() - SRCS( - chacha.cpp - crypto.cpp - crypto.h - poly1305.cpp - secured_block.cpp - ) -ENDIF() - + SRCS( + chacha.cpp + crypto.cpp + crypto.h + poly1305.cpp + chacha_vec.cpp + poly1305_vec.cpp + secured_block.cpp + ) +ELSE() + SRCS( + chacha.cpp + crypto.cpp + crypto.h + poly1305.cpp + secured_block.cpp + ) +ENDIF() + PEERDIR( contrib/libs/t1ha library/cpp/sse ) -END() - +END() + RECURSE_FOR_TESTS( ut ) diff --git a/ydb/core/blobstorage/dsproxy/dsproxy.h b/ydb/core/blobstorage/dsproxy/dsproxy.h index c4523b9178c..27592e971f9 100644 --- a/ydb/core/blobstorage/dsproxy/dsproxy.h +++ b/ydb/core/blobstorage/dsproxy/dsproxy.h @@ -138,7 +138,7 @@ public: } TBlobStorageGroupRequestActor(TIntrusivePtr<TBlobStorageGroupInfo> info, TIntrusivePtr<TGroupQueues> groupQueues, - TIntrusivePtr<TBlobStorageGroupProxyMon> mon, const TActorId& source, ui64 cookie, NWilson::TTraceId traceId, + TIntrusivePtr<TBlobStorageGroupProxyMon> mon, const TActorId& source, ui64 cookie, NWilson::TTraceId traceId, NKikimrServices::EServiceKikimr logComponent, bool logAccEnabled, TMaybe<TGroupStat::EKind> latencyQueueKind, TInstant now, TIntrusivePtr<TStoragePoolCounters> &storagePoolCounters, ui32 restartCounter) : Info(std::move(info)) @@ -163,7 +163,7 @@ public: Mon->CountEvent(request, ev); } - TActorId GetVDiskActorId(const TVDiskIdShort &shortId) const { + TActorId GetVDiskActorId(const TVDiskIdShort &shortId) const { return Info->GetActorId(shortId); } @@ -506,7 +506,7 @@ protected: const ui32 RestartCounter = 0; private: - const TActorId Source; + const TActorId Source; const ui64 Cookie; THashMap<TActorId, ui32> InvolvedQueues; ui32 RequestsInFlight = 0; diff --git a/ydb/core/blobstorage/dsproxy/dsproxy_collect.cpp b/ydb/core/blobstorage/dsproxy/dsproxy_collect.cpp index 0752fc5ab2e..d12c819a745 100644 --- a/ydb/core/blobstorage/dsproxy/dsproxy_collect.cpp +++ b/ydb/core/blobstorage/dsproxy/dsproxy_collect.cpp @@ -98,7 +98,7 @@ class TBlobStorageGroupCollectGarbageRequest : public TBlobStorageGroupRequestAc SendCollectGarbageRequest(vdiskId); } } - + void CheckProgress() { Y_VERIFY(Dead || ResponsesReceived < RequestsSent, "No more unreplied vdisk requests!" " QuorumTracker# %s RequestsSent# %" PRIu32 " ResponsesReceived# %" PRIu32, diff --git a/ydb/core/blobstorage/dsproxy/dsproxy_discover.cpp b/ydb/core/blobstorage/dsproxy/dsproxy_discover.cpp index 3b8bee5c022..2605a19a2e9 100644 --- a/ydb/core/blobstorage/dsproxy/dsproxy_discover.cpp +++ b/ydb/core/blobstorage/dsproxy/dsproxy_discover.cpp @@ -707,7 +707,7 @@ class TBlobStorageGroupDiscoverRequest : public TBlobStorageGroupRequestActor<TB TVDiskID vDiskId = Info->CreateVDiskID(vId); if (!curVDisk.IsError && !curVDisk.IsAllRead && !curVDisk.IsMoreRequested) { - const TActorId &vdisk = Info->GetActorId(vDiskId); + const TActorId &vdisk = Info->GetActorId(vDiskId); auto msg = TEvBlobStorage::TEvVGet::CreateRangeIndexQuery(vDiskId, Deadline, NKikimrBlobStorage::EGetHandleClass::Discover, TEvBlobStorage::TEvVGet::EFlags::ShowInternals, diff --git a/ydb/core/blobstorage/dsproxy/dsproxy_mon.h b/ydb/core/blobstorage/dsproxy/dsproxy_mon.h index 7f74602b683..941bea6664a 100644 --- a/ydb/core/blobstorage/dsproxy/dsproxy_mon.h +++ b/ydb/core/blobstorage/dsproxy/dsproxy_mon.h @@ -11,7 +11,7 @@ #include <ydb/core/protos/node_whiteboard.pb.h> #include <ydb/core/util/throughput_meter.h> #include <ydb/core/mon/mon.h> -#include <library/cpp/monlib/dynamic_counters/percentile/percentile_lg.h> +#include <library/cpp/monlib/dynamic_counters/percentile/percentile_lg.h> #include <util/generic/ptr.h> namespace NKikimr { diff --git a/ydb/core/blobstorage/dsproxy/dsproxy_monactor.cpp b/ydb/core/blobstorage/dsproxy/dsproxy_monactor.cpp index 9d63b5fa042..96b492c415a 100644 --- a/ydb/core/blobstorage/dsproxy/dsproxy_monactor.cpp +++ b/ydb/core/blobstorage/dsproxy/dsproxy_monactor.cpp @@ -11,7 +11,7 @@ #include <library/cpp/actors/core/mon.h> #include <library/cpp/actors/interconnect/interconnect.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> namespace NKikimr { diff --git a/ydb/core/blobstorage/dsproxy/dsproxy_nodemon.h b/ydb/core/blobstorage/dsproxy/dsproxy_nodemon.h index 6b8fda7ff01..150da172fb8 100644 --- a/ydb/core/blobstorage/dsproxy/dsproxy_nodemon.h +++ b/ydb/core/blobstorage/dsproxy/dsproxy_nodemon.h @@ -12,8 +12,8 @@ #include <ydb/core/util/throughput_meter.h> #include <ydb/core/mon/mon.h> -#include <library/cpp/monlib/dynamic_counters/percentile/percentile.h> -#include <library/cpp/monlib/metrics/histogram_snapshot.h> +#include <library/cpp/monlib/dynamic_counters/percentile/percentile.h> +#include <library/cpp/monlib/metrics/histogram_snapshot.h> #include <util/generic/ptr.h> diff --git a/ydb/core/blobstorage/dsproxy/dsproxy_nodemonactor.cpp b/ydb/core/blobstorage/dsproxy/dsproxy_nodemonactor.cpp index 19d23cb7d6d..5bf4b098362 100644 --- a/ydb/core/blobstorage/dsproxy/dsproxy_nodemonactor.cpp +++ b/ydb/core/blobstorage/dsproxy/dsproxy_nodemonactor.cpp @@ -7,7 +7,7 @@ #include <library/cpp/actors/core/hfunc.h> #include <library/cpp/actors/core/mon.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> namespace NKikimr { diff --git a/ydb/core/blobstorage/dsproxy/dsproxy_put.cpp b/ydb/core/blobstorage/dsproxy/dsproxy_put.cpp index 5332a22eb19..d9e0e2b3202 100644 --- a/ydb/core/blobstorage/dsproxy/dsproxy_put.cpp +++ b/ydb/core/blobstorage/dsproxy/dsproxy_put.cpp @@ -495,7 +495,7 @@ public: TIntrusivePtr<TStoragePoolCounters> &storagePoolCounters, NKikimrBlobStorage::EPutHandleClass handleClass, TEvBlobStorage::TEvPut::ETactic tactic, bool enableRequestMod3x3ForMinLatecy) - : TBlobStorageGroupRequestActor(info, state, mon, TActorId(), 0, NWilson::TTraceId(), + : TBlobStorageGroupRequestActor(info, state, mon, TActorId(), 0, NWilson::TTraceId(), NKikimrServices::BS_PROXY_PUT, false, latencyQueueKind, now, storagePoolCounters, MaxRestartCounter(events)) , PutImpl(info, state, events, mon, handleClass, tactic, enableRequestMod3x3ForMinLatecy) diff --git a/ydb/core/blobstorage/dsproxy/dsproxy_timestats.h b/ydb/core/blobstorage/dsproxy/dsproxy_timestats.h index f0825ae5772..943b93858df 100644 --- a/ydb/core/blobstorage/dsproxy/dsproxy_timestats.h +++ b/ydb/core/blobstorage/dsproxy/dsproxy_timestats.h @@ -5,7 +5,7 @@ #include <ydb/core/base/appdata.h> #include <ydb/core/base/blobstorage.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/generic/queue.h> #include <util/datetime/parser.h> diff --git a/ydb/core/blobstorage/dsproxy/ut/dsproxy_fault_tolerance_ut_runtime.h b/ydb/core/blobstorage/dsproxy/ut/dsproxy_fault_tolerance_ut_runtime.h index f7f61089f86..bc33af4b86c 100644 --- a/ydb/core/blobstorage/dsproxy/ut/dsproxy_fault_tolerance_ut_runtime.h +++ b/ydb/core/blobstorage/dsproxy/ut/dsproxy_fault_tolerance_ut_runtime.h @@ -24,7 +24,7 @@ class TFaultToleranceTestRuntime { public: TIntrusivePtr<NMonitoring::TDynamicCounters> Counters; TIntrusivePtr<TBlobStorageGroupInfo> GroupInfo; - TVector<std::pair<TVDiskID, TActorId>> VDisks; + TVector<std::pair<TVDiskID, TActorId>> VDisks; std::unique_ptr<TAppData> AppData; std::unique_ptr<TActorSystem> ActorSystem; TProgramShouldContinue KikimrShouldContinue; @@ -32,12 +32,12 @@ public: void Setup(TBlobStorageGroupType groupType, ui32 numFailDomains, ui32 numVDisksPerFailDomain, ui32 numRealms) { Counters = new NMonitoring::TDynamicCounters; - TIntrusivePtr<NScheme::TTypeRegistry> typeRegistry(new NScheme::TKikimrTypeRegistry()); + TIntrusivePtr<NScheme::TTypeRegistry> typeRegistry(new NScheme::TKikimrTypeRegistry()); auto functionRegistry = NKikimr::NMiniKQL::CreateFunctionRegistry(NKikimr::NMiniKQL::CreateBuiltinRegistry()); AppData.reset(new NKikimr::TAppData(0, 1, 2, 3, TMap<TString, ui32>(), typeRegistry.Get(), functionRegistry.Get(), nullptr, &KikimrShouldContinue)); AppData->Counters = Counters; - NActors::TActorId loggerActorId = TActorId(1, "logger"); + NActors::TActorId loggerActorId = TActorId(1, "logger"); TIntrusivePtr<NLog::TSettings> logSettings = new NLog::TSettings(loggerActorId, NKikimrServices::LOGGER, (IsVerbose ? NLog::PRI_NOTICE : NLog::PRI_CRIT), NLog::PRI_DEBUG, 0); diff --git a/ydb/core/blobstorage/dsproxy/ut/dsproxy_put_ut.cpp b/ydb/core/blobstorage/dsproxy/ut/dsproxy_put_ut.cpp index eda7899e281..d2d39a35504 100644 --- a/ydb/core/blobstorage/dsproxy/ut/dsproxy_put_ut.cpp +++ b/ydb/core/blobstorage/dsproxy/ut/dsproxy_put_ut.cpp @@ -114,7 +114,7 @@ void TestPutMaxPartCountOnHandoff(TErasureType::EErasureSpecies erasureSpecies) TDeque<std::unique_ptr<TEvBlobStorage::TEvVPut>> nextVPuts; TEvBlobStorage::TEvVPut& vPut = *vPuts[vPutIdx]; - TActorId sender; + TActorId sender; TEvBlobStorage::TEvVPutResult vPutResult; NKikimrProto::EReplyStatus status = group.OnVPut(vPut); vPutResult.MakeError(status, TString(), vPut.Record); @@ -273,7 +273,7 @@ struct TTestPutAllOk { break; } - TActorId sender; + TActorId sender; TDeque<std::unique_ptr<TEvBlobStorage::TEvVPut>> vPuts2; putImpl.OnVPutEventResult(LogCtx, sender, *vPutResults[resIdx], vPuts2, putResults); if (putResults.size()) { @@ -304,7 +304,7 @@ struct TTestPutAllOk { break; } - TActorId sender; + TActorId sender; TDeque<std::unique_ptr<TEvBlobStorage::TEvVMultiPut>> vMultiPuts2; putImpl.OnVPutEventResult(LogCtx, sender, *vMultiPutResults[resIdx], vMultiPuts2, putResults); if (putResults.size() == BlobIds.size()) { diff --git a/ydb/core/blobstorage/dsproxy/ut/dsproxy_sequence_ut.cpp b/ydb/core/blobstorage/dsproxy/ut/dsproxy_sequence_ut.cpp index 92e584b97fd..fdd7a92ea30 100644 --- a/ydb/core/blobstorage/dsproxy/ut/dsproxy_sequence_ut.cpp +++ b/ydb/core/blobstorage/dsproxy/ut/dsproxy_sequence_ut.cpp @@ -89,8 +89,8 @@ struct TGetRangeQuery { struct TGetRequest { TVector<TGetQuery> Queries; TVector<TGetRangeQuery> RangeQueries; - TActorId Sender; - TActorId ActorId; + TActorId Sender; + TActorId ActorId; TVDiskID VDiskId; ui64 Cookie; ui64 RecordCookie; @@ -136,8 +136,8 @@ struct TGetRequest { }; struct TVDiskState { - TActorId Sender; - TActorId ActorId; + TActorId Sender; + TActorId ActorId; TLogoBlobID LogoBlobId; TVDiskID VDiskId; ui64 LastCookie; @@ -276,7 +276,7 @@ void SendVGetResult(ui32 vDiskIdx, NKikimrProto::EReplyStatus status, ui32 partI void SendVGetResult(ui32 blobIdx, ui32 vDiskIdx, NKikimrProto::EReplyStatus status, TVector<TVector<TVDiskState>> &blobSubgroups, - TMap<TActorId, TGetRequest> &lastRequest, TTestActorRuntime &runtime) { + TMap<TActorId, TGetRequest> &lastRequest, TTestActorRuntime &runtime) { TGetRequest &request = lastRequest[blobSubgroups[blobIdx][vDiskIdx].ActorId]; if (!request.IsValid) { return; @@ -375,8 +375,8 @@ void SendVPutResultEvent(TTestActorRuntime &runtime, TVDiskState &vdisk, NKikimr void PrepareBlobSubgroup(TLogoBlobID logoblobid, TString data, TVector<TVDiskState> &subgroup, TTestActorRuntime &runtime, TBlobStorageGroupType type) { - TActorId proxy = MakeBlobStorageProxyID(GROUP_ID); - TActorId sender = runtime.AllocateEdgeActor(0); + TActorId proxy = MakeBlobStorageProxyID(GROUP_ID); + TActorId sender = runtime.AllocateEdgeActor(0); runtime.Send(new IEventHandle(proxy, sender, new TEvBlobStorage::TEvPut(logoblobid, data, TInstant::Max()))); subgroup.resize(type.BlobSubgroupSize()); @@ -751,8 +751,8 @@ Y_UNIT_TEST(TestGivenBlock42GetThenVGetResponseParts2523Nodata4ThenGetOk) { TBlobStorageGroupType type = {TErasureType::Erasure4Plus2Block}; Setup(runtime, type); - TActorId proxy = MakeBlobStorageProxyID(GROUP_ID); - TActorId sender = runtime.AllocateEdgeActor(0); + TActorId proxy = MakeBlobStorageProxyID(GROUP_ID); + TActorId sender = runtime.AllocateEdgeActor(0); TString data("xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"); TLogoBlobID logoblobid(1, 0, 0, 0, (ui32)data.size(), 0); @@ -835,8 +835,8 @@ void MakeTestProtobufSizeWithMultiGet(const TVector<TBlobPack> &packs, TVector<u TBlobStorageGroupType type(TErasureType::ErasureNone); Setup(runtime, type); - TActorId proxy = MakeBlobStorageProxyID(GROUP_ID); - TActorId sender = runtime.AllocateEdgeActor(0); + TActorId proxy = MakeBlobStorageProxyID(GROUP_ID); + TActorId sender = runtime.AllocateEdgeActor(0); TVector<TString> datas(packs.size()); for (ui32 idx = 0; idx < packs.size(); ++idx) { @@ -918,7 +918,7 @@ void MakeTestProtobufSizeWithMultiGet(const TVector<TBlobPack> &packs, TVector<u } for (ui64 idx = 0; idx < vGetInformations.size(); ++idx) { - TMap<TActorId, TGetRequest> lastRequest; + TMap<TActorId, TGetRequest> lastRequest; TAutoPtr<IEventHandle> handle; auto vget = runtime.GrabEdgeEventRethrow<TEvBlobStorage::TEvVGet>(handle); UNIT_ASSERT_C(vget->Record.ExtremeQueriesSize(), "vget without queries vGet# " << vget->ToString()); @@ -979,8 +979,8 @@ Y_UNIT_TEST(TestGivenStripe42GetThenVGetResponsePartsNodata263451ThenGetOk) { TBlobStorageGroupType type = {TErasureType::Erasure4Plus2Stripe}; Setup(runtime, type); - TActorId proxy = MakeBlobStorageProxyID(GROUP_ID); - TActorId sender = runtime.AllocateEdgeActor(0); + TActorId proxy = MakeBlobStorageProxyID(GROUP_ID); + TActorId sender = runtime.AllocateEdgeActor(0); TString data; data.resize(1209816, 'x'); @@ -1025,8 +1025,8 @@ Y_UNIT_TEST(TestGivenStripe42WhenGet2PartsOfBlobThenGetOk) { TBlobStorageGroupType type = {TErasureType::Erasure4Plus2Stripe}; Setup(runtime, type); - TActorId proxy = MakeBlobStorageProxyID(GROUP_ID); - TActorId sender = runtime.AllocateEdgeActor(0); + TActorId proxy = MakeBlobStorageProxyID(GROUP_ID); + TActorId sender = runtime.AllocateEdgeActor(0); TString data; data.resize(1209816, 'x'); @@ -1044,7 +1044,7 @@ Y_UNIT_TEST(TestGivenStripe42WhenGet2PartsOfBlobThenGetOk) { TVector<TVector<TVDiskState>> blobSubgroups; blobSubgroups.resize(logoblobids.size()); - TMap<TActorId, TGetRequest> lastRequest; + TMap<TActorId, TGetRequest> lastRequest; for (ui32 i = 0; i < logoblobids.size(); ++i) { PrepareBlobSubgroup(logoblobids[i], data, blobSubgroups[i], runtime, type); @@ -1090,8 +1090,8 @@ Y_UNIT_TEST(TestGivenBlock42IntersectingPutWhenNodataOkThenOk) { TBlobStorageGroupType type = {TErasureType::Erasure4Plus2Block}; Setup(runtime, type); - TActorId proxy = MakeBlobStorageProxyID(GROUP_ID); - TActorId sender = runtime.AllocateEdgeActor(0); + TActorId proxy = MakeBlobStorageProxyID(GROUP_ID); + TActorId sender = runtime.AllocateEdgeActor(0); TString data; data.resize(1000, 'x'); @@ -1105,7 +1105,7 @@ Y_UNIT_TEST(TestGivenBlock42IntersectingPutWhenNodataOkThenOk) { } TVector<TVector<TVDiskState>> blobSubgroups; blobSubgroups.resize(logoblobids.size()); - TMap<TActorId, TGetRequest> lastRequest; + TMap<TActorId, TGetRequest> lastRequest; for (ui32 i = 0; i < logoblobids.size(); ++i) { PrepareBlobSubgroup(logoblobids[i], data, blobSubgroups[i], runtime, type); @@ -1150,8 +1150,8 @@ Y_UNIT_TEST(TestGivenBlock42PutWhenPartialGetThenSingleDiskRequestOk) { TBlobStorageGroupType type = {TErasureType::Erasure4Plus2Block}; Setup(runtime, type); - TActorId proxy = MakeBlobStorageProxyID(GROUP_ID); - TActorId sender = runtime.AllocateEdgeActor(0); + TActorId proxy = MakeBlobStorageProxyID(GROUP_ID); + TActorId sender = runtime.AllocateEdgeActor(0); TString data; data.resize(400 << 10); @@ -1165,7 +1165,7 @@ Y_UNIT_TEST(TestGivenBlock42PutWhenPartialGetThenSingleDiskRequestOk) { for (ui32 step = 0; step < 32; ++step) { for (ui32 part = 0; part < 4; ++part) { for (ui32 disk = 0; disk < 3; ++disk) { - TMap<TActorId, TGetRequest> lastRequest; + TMap<TActorId, TGetRequest> lastRequest; // Send Get ui32 shift = ((25 + part * 100) << 10) + step; ui32 size = 50 << 10; @@ -1259,8 +1259,8 @@ Y_UNIT_TEST(TestGivenBlock42Put6PartsOnOneVDiskWhenDiscoverThenRecoverFirst) { TBlobStorageGroupType type = {TErasureType::Erasure4Plus2Block}; Setup(runtime, type); - TActorId proxy = MakeBlobStorageProxyID(GROUP_ID); - TActorId sender = runtime.AllocateEdgeActor(0); + TActorId proxy = MakeBlobStorageProxyID(GROUP_ID); + TActorId sender = runtime.AllocateEdgeActor(0); TString data; data.resize(400 << 10); @@ -1280,7 +1280,7 @@ Y_UNIT_TEST(TestGivenBlock42Put6PartsOnOneVDiskWhenDiscoverThenRecoverFirst) { proxy, sender, new TEvBlobStorage::TEvDiscover(tabletId, minGeneration, true, false, TInstant::Max(), 0))); // Receive VGet - TMap<TActorId, TGetRequest> lastRequest; + TMap<TActorId, TGetRequest> lastRequest; for (ui32 i = 0; i < 8; ++i) { TAutoPtr<IEventHandle> handle; auto vget = runtime.GrabEdgeEventRethrow<TEvBlobStorage::TEvVGet>(handle); @@ -1293,7 +1293,7 @@ Y_UNIT_TEST(TestGivenBlock42Put6PartsOnOneVDiskWhenDiscoverThenRecoverFirst) { } runtime.EnableScheduleForActor(lastRequest.begin()->second.Sender, true); - TActorId firstHandoffActorId = blobSubgroups[0][6].ActorId; + TActorId firstHandoffActorId = blobSubgroups[0][6].ActorId; { // Send 6 part VGetResult from the first handoff vDiskIdx# 6 diff --git a/ydb/core/blobstorage/dsproxy/ut/dsproxy_vdisk_mock_ut.h b/ydb/core/blobstorage/dsproxy/ut/dsproxy_vdisk_mock_ut.h index afd8b858e7c..0a17c046e71 100644 --- a/ydb/core/blobstorage/dsproxy/ut/dsproxy_vdisk_mock_ut.h +++ b/ydb/core/blobstorage/dsproxy/ut/dsproxy_vdisk_mock_ut.h @@ -37,7 +37,7 @@ public: Info = info; } - TActorId GetActorId() { + TActorId GetActorId() { return Info->GetActorId(Info->GetOrderNumber(VDiskId)); } diff --git a/ydb/core/blobstorage/dsproxy/ut_fat/dsproxy_ut.cpp b/ydb/core/blobstorage/dsproxy/ut_fat/dsproxy_ut.cpp index 29bf986ba6e..13b1bec3563 100644 --- a/ydb/core/blobstorage/dsproxy/ut_fat/dsproxy_ut.cpp +++ b/ydb/core/blobstorage/dsproxy/ut_fat/dsproxy_ut.cpp @@ -136,8 +136,8 @@ struct TTestArgs { struct TTestEnvironment : public TThrRefBase { ui32 VDiskCount; ui64 DeadVDisksMask; - TVector<TActorId> VDisks; - TVector<TActorId> PDisks; + TVector<TActorId> VDisks; + TVector<TActorId> PDisks; TVector<TVDiskID> VDiskIds; bool ShouldBeUnwritable; @@ -148,8 +148,8 @@ struct TTestEnvironment : public TThrRefBase { ui32 FailedDiskCount; TBlobStorageGroupType GroupType; - TActorId ProxyId; - TActorId ProxyTestId; + TActorId ProxyId; + TActorId ProxyTestId; TSystemEvent DoneEvent; yexception LastException; @@ -237,7 +237,7 @@ protected: ui32 BlockedGeneration; bool LogoBlobsCompacted; TStorageStatusFlags StatusFlags; - TActorId Sender; + TActorId Sender; TVDiskID VDiskId; bool IsConnected; TIntrusivePtr<TGroupQueues> GroupQueues; @@ -261,12 +261,12 @@ protected: BlockedGeneration = 0; LogoBlobsCompacted = false; StatusFlags.Raw = 0; - Sender = TActorId(); + Sender = TActorId(); } }; TResponseData LastResponse; - const TActorId Proxy; + const TActorId Proxy; const TBlobStorageGroupType::EErasureSpecies ErasureSpecies; TIntrusivePtr<TBlobStorageGroupInfo> BsInfo; int TestStep; @@ -615,7 +615,7 @@ protected: } public: - TTestBlobStorageProxy(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxy(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TActor(&TThis::StateRegister) , Proxy(proxy) @@ -675,7 +675,7 @@ class TTestBlobStorageProxyBlockSet : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyBlockSet(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyBlockSet(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -726,7 +726,7 @@ class TTestBlobStorageProxyBlockCheck : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyBlockCheck(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyBlockCheck(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -761,7 +761,7 @@ class TTestIterativePut: public TTestBlobStorageProxy { } public: - TTestIterativePut(const TActorId& proxy, const TIntrusivePtr<TBlobStorageGroupInfo>& bsInfo, + TTestIterativePut(const TActorId& proxy, const TIntrusivePtr<TBlobStorageGroupInfo>& bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) { @@ -798,7 +798,7 @@ class TTestInFlightPuts: public TTestBlobStorageProxy { } public: - TTestInFlightPuts(const TActorId& proxy, const TIntrusivePtr<TBlobStorageGroupInfo>& bsInfo, + TTestInFlightPuts(const TActorId& proxy, const TIntrusivePtr<TBlobStorageGroupInfo>& bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) { @@ -847,7 +847,7 @@ class TTestOrderGet: public TTestBlobStorageProxy { } public: - TTestOrderGet(const TActorId& proxy, const TIntrusivePtr<TBlobStorageGroupInfo>& bsInfo, + TTestOrderGet(const TActorId& proxy, const TIntrusivePtr<TBlobStorageGroupInfo>& bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) { @@ -887,7 +887,7 @@ class TTestRangeGet: public TTestBlobStorageProxy { } public: - TTestRangeGet(const TActorId& proxy, const TIntrusivePtr<TBlobStorageGroupInfo>& bsInfo, + TTestRangeGet(const TActorId& proxy, const TIntrusivePtr<TBlobStorageGroupInfo>& bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) { @@ -998,7 +998,7 @@ class TTestBlobStorageProxyBlock : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyBlock(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyBlock(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -1036,7 +1036,7 @@ class TTestBlobStorageProxyPut : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyPut(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyPut(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -1074,7 +1074,7 @@ class TTestBlobStorageProxyPutInvalidSize : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyPutInvalidSize(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyPutInvalidSize(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -1109,7 +1109,7 @@ class TTestBlobStorageProxyPutFail : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyPutFail(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyPutFail(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -1146,7 +1146,7 @@ class TTestBlobStorageProxyGet : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyGet(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyGet(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -1181,7 +1181,7 @@ class TTestBlobStorageProxyGetFail : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyGetFail(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyGetFail(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -1216,7 +1216,7 @@ class TTestBlobStorageProxyGetTimeout : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyGetTimeout(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyGetTimeout(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -1271,7 +1271,7 @@ class TTestBlobStorageProxyPutGetMany: public TTestBlobStorageProxy { ++TestStep; } public: - TTestBlobStorageProxyPutGetMany(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyPutGetMany(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -1364,7 +1364,7 @@ class TTestBlobStorageProxyPutGetStatus: public TTestBlobStorageProxy { ++TestStep; } public: - TTestBlobStorageProxyPutGetStatus(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyPutGetStatus(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -1433,7 +1433,7 @@ class TTestBlobStorageProxyVPutVGet : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyVPutVGet(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyVPutVGet(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -1530,7 +1530,7 @@ class TTestBlobStorageProxyVPutVGetLimit : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyVPutVGetLimit(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyVPutVGetLimit(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) , Iteration(0) @@ -1644,7 +1644,7 @@ private: TestStep += 10; } public: - TTestBlobStorageProxyVPut(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyVPut(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -1727,7 +1727,7 @@ class TTestBlobStorageProxyVGet : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyVGet(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyVGet(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -1814,7 +1814,7 @@ class TTestBlobStorageProxyVGetFail : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyVGetFail(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyVGetFail(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -1898,7 +1898,7 @@ class TTestBlobStorageProxyVBlockVPutVGet : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyVBlockVPutVGet(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyVBlockVPutVGet(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -1937,7 +1937,7 @@ class TTestBlobStorageProxyGarbageMark : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyGarbageMark(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyGarbageMark(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -1974,7 +1974,7 @@ class TTestBlobStorageProxyGarbageUnmark : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyGarbageUnmark(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyGarbageUnmark(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -2007,7 +2007,7 @@ class TTestBlobStorageProxyGarbageCollect : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyGarbageCollect(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyGarbageCollect(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -2054,7 +2054,7 @@ class TTestBlobStorageProxyGarbageCollectHuge : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyGarbageCollectHuge(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyGarbageCollectHuge(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -2097,7 +2097,7 @@ class TTestBlobStorageProxyEmptyGet : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyEmptyGet(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyEmptyGet(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -2230,7 +2230,7 @@ class TTestBlobStorageProxyGarbageCollectComplex : public TTestBlobStorageProxy TestStep += 10; } public: - TTestBlobStorageProxyGarbageCollectComplex(const TActorId &proxy, + TTestBlobStorageProxyGarbageCollectComplex(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) @@ -2296,7 +2296,7 @@ class TTestBlobStorageProxyGarbageCollectAfterLargeData : public TTestBlobStorag TestStep += 10; } public: - TTestBlobStorageProxyGarbageCollectAfterLargeData(const TActorId &proxy, + TTestBlobStorageProxyGarbageCollectAfterLargeData(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) @@ -2358,7 +2358,7 @@ class TTestBlobStorageProxySimpleDiscover : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxySimpleDiscover(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxySimpleDiscover(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -2414,7 +2414,7 @@ class TTestBlobStorageProxyDiscover : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyDiscover(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyDiscover(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -2450,7 +2450,7 @@ class TTestBlobStorageProxyDiscoverFail : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyDiscoverFail(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyDiscoverFail(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -2486,7 +2486,7 @@ class TTestBlobStorageProxyDiscoverEmpty : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyDiscoverEmpty(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyDiscoverEmpty(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -2522,7 +2522,7 @@ class TTestBlobStorageProxyDiscoverTimeout : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyDiscoverTimeout(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyDiscoverTimeout(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -2675,7 +2675,7 @@ class TTestBlobStorageProxyLongTailDiscoverPut : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyLongTailDiscoverPut(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyLongTailDiscoverPut(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) , MsgIdx(0) @@ -2720,7 +2720,7 @@ class TTestBlobStorageProxyLongTailDiscover : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyLongTailDiscover(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyLongTailDiscover(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -2774,7 +2774,7 @@ class TTestBlobStorageProxyPartialGet : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyPartialGet(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyPartialGet(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -2808,7 +2808,7 @@ class TTestEmptyRange : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestEmptyRange(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestEmptyRange(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -3155,7 +3155,7 @@ class TTestBlobStorageProxyBasic1 : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyBasic1(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyBasic1(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -3202,7 +3202,7 @@ class TTestGetMultipart : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestGetMultipart(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestGetMultipart(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -3253,7 +3253,7 @@ class TTestVDiskCompacted : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestVDiskCompacted(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestVDiskCompacted(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -3337,7 +3337,7 @@ class TTestBlobStorageProxyVPutVCollectVGetRace : public TTestBlobStorageProxy { TestStep += 10; } public: - TTestBlobStorageProxyVPutVCollectVGetRace(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, + TTestBlobStorageProxyVPutVCollectVGetRace(const TActorId &proxy, const TIntrusivePtr<TBlobStorageGroupInfo> &bsInfo, const TIntrusivePtr<TTestEnvironment> &env, const TIntrusivePtr<ITestParametrs> ¶metrs) : TTestBlobStorageProxy(proxy, bsInfo, env, parametrs) {} @@ -4074,11 +4074,11 @@ public: ui64 nodeCount = nameserverTable->StaticNodeTable.size() + 1; setup->LocalServices.emplace_back( - GetNameserviceActorId(), + GetNameserviceActorId(), TActorSetupCmd(CreateNameserverTable(nameserverTable), TMailboxType::ReadAsFilled, 0) ); TIntrusivePtr<TInterconnectProxyCommon> icCommon = new TInterconnectProxyCommon(); - icCommon->NameserviceId = GetNameserviceActorId(); + icCommon->NameserviceId = GetNameserviceActorId(); icCommon->MonCounters = counters.GetSubgroup("counters", "interconnect"); icCommon->TechnicalSelfHostName = "127.0.0.1"; setup->Interconnect.ProxyActors.resize(nodeCount); @@ -4092,11 +4092,11 @@ public: } TActorSetupCmd profilerSetup(CreateProfilerActor(nullptr, "."), TMailboxType::Simple, 0); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeProfilerID(nodeId), profilerSetup)); + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeProfilerID(nodeId), profilerSetup)); - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); TActorSetupCmd nameserviceSetup(CreateNameserverTable(nameserverTable), TMailboxType::Simple, 0); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(nameserviceId, nameserviceSetup)); + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(nameserviceId, nameserviceSetup)); return setup; } @@ -4104,7 +4104,7 @@ public: TIntrusivePtr<NActors::NLog::TSettings> AddLoggerActor(THolder<TActorSystemSetup> &setup, NMonitoring::TDynamicCounters &counters) { - NActors::TActorId loggerActorId = NActors::TActorId(setup->NodeId, "logger"); + NActors::TActorId loggerActorId = NActors::TActorId(setup->NodeId, "logger"); TIntrusivePtr<NActors::NLog::TSettings> logSettings( new NActors::NLog::TSettings(loggerActorId, NKikimrServices::LOGGER, IsVerbose ? NLog::PRI_ERROR : NLog::PRI_CRIT, @@ -4140,7 +4140,7 @@ public: NActors::CreateStderrBackend(), counters.GetSubgroup("counters", "utils")); NActors::TActorSetupCmd loggerActorCmd(loggerActor, NActors::TMailboxType::Simple, 2); - std::pair<NActors::TActorId, NActors::TActorSetupCmd> loggerActorPair(loggerActorId, loggerActorCmd); + std::pair<NActors::TActorId, NActors::TActorSetupCmd> loggerActorPair(loggerActorId, loggerActorCmd); setup->LocalServices.push_back(loggerActorPair); return logSettings; @@ -4191,7 +4191,7 @@ public: std::unique_ptr<IActor> proxyActor{CreateBlobStorageGroupProxyConfigured(TIntrusivePtr(bsInfo), false, dsProxyNodeMon, TIntrusivePtr(storagePoolCounters), args.EnablePutBatching, DefaultEnableVPatch)}; TActorSetupCmd bsproxySetup(proxyActor.release(), TMailboxType::Revolving, 3); - setup1->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(env->ProxyId, bsproxySetup)); + setup1->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(env->ProxyId, bsproxySetup)); TTempDir tempDir; NPDisk::TKey mainKey = 123; @@ -4212,7 +4212,7 @@ public: ui32 chunkSize = 32 << 20; ui64 diskSizeBytes = 32ull << 30ull; ui64 pDiskCategory = 0; - TActorId pdiskId = env->PDisks[i]; + TActorId pdiskId = env->PDisks[i]; TString filePath = databaseDirectory + "/pdisk.dat"; if (!SectorMapByPath[filePath]) { SectorMapByPath[filePath].Reset(new NPDisk::TSectorMap(diskSizeBytes)); @@ -4230,7 +4230,7 @@ public: TActorSetupCmd pDiskSetup( CreatePDisk(pDiskConfig.Get(), mainKey, counters), TMailboxType::Revolving, 0); - setup2->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(pdiskId, pDiskSetup)); + setup2->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(pdiskId, pDiskSetup)); TVDiskConfig::TBaseInfo baseInfo( env->VDiskIds[i], @@ -4254,13 +4254,13 @@ public: IActor* vDisk = CreateVDisk(vDiskConfig, bsInfo, counters); TActorSetupCmd vDiskSetup(vDisk, TMailboxType::Revolving, 0); - setup2->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(env->VDisks[i], vDiskSetup)); + setup2->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(env->VDisks[i], vDiskSetup)); } } TActorSetupCmd proxyTestSetup(new T(env->ProxyId, bsInfo.Get(), env, args.Parametrs), TMailboxType::Revolving, 0); - setup1->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(env->ProxyTestId, proxyTestSetup)); + setup1->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(env->ProxyTestId, proxyTestSetup)); ////////////////////////////////////////////////////////////////////////////// GetServiceCounters(counters, "utils"); @@ -4277,7 +4277,7 @@ public: EnableActorCallstack(); try { VERBOSE_COUT("Sending TEvBoot to testproxy"); - actorSystem1->Send(env->ProxyTestId, new TEvTablet::TEvBoot(MakeTabletID(0, 0, 1), 0, nullptr, TActorId(), + actorSystem1->Send(env->ProxyTestId, new TEvTablet::TEvBoot(MakeTabletID(0, 0, 1), 0, nullptr, TActorId(), nullptr)); VERBOSE_COUT("Done"); diff --git a/ydb/core/blobstorage/dsproxy/ya.make b/ydb/core/blobstorage/dsproxy/ya.make index 6292b314294..6e165c7f5d1 100644 --- a/ydb/core/blobstorage/dsproxy/ya.make +++ b/ydb/core/blobstorage/dsproxy/ya.make @@ -64,7 +64,7 @@ SRCS( ) PEERDIR( - library/cpp/monlib/dynamic_counters/percentile + library/cpp/monlib/dynamic_counters/percentile ydb/core/base ydb/core/blobstorage/backpressure ydb/core/blobstorage/base diff --git a/ydb/core/blobstorage/groupinfo/blobstorage_groupinfo.cpp b/ydb/core/blobstorage/groupinfo/blobstorage_groupinfo.cpp index 7590a84d672..054d5be6653 100644 --- a/ydb/core/blobstorage/groupinfo/blobstorage_groupinfo.cpp +++ b/ydb/core/blobstorage/groupinfo/blobstorage_groupinfo.cpp @@ -470,7 +470,7 @@ IBlobToDiskMapper *TBlobStorageGroupInfo::TTopology::CreateMapper(TBlobStorageGr case TBlobStorageGroupType::ErasureMirror3dc: return IBlobToDiskMapper::CreateMirror3dcMapper(topology); - + default: Y_FAIL("unexpected erasure type 0x%08" PRIx32, static_cast<ui32>(gtype.GetErasure())); } @@ -554,7 +554,7 @@ TBlobStorageGroupInfo::TDynamicInfo::TDynamicInfo(ui32 groupId, ui32 groupGen) // TBlobStorageGroupInfo //////////////////////////////////////////////////////////////////////////// TBlobStorageGroupInfo::TBlobStorageGroupInfo(TBlobStorageGroupType gtype, ui32 numVDisksPerFailDomain, - ui32 numFailDomains, ui32 numFailRealms, const TVector<TActorId> *vdiskIds, EEncryptionMode encryptionMode, + ui32 numFailDomains, ui32 numFailRealms, const TVector<TActorId> *vdiskIds, EEncryptionMode encryptionMode, ELifeCyclePhase lifeCyclePhase, TCypherKey key) : GroupID(0) , GroupGeneration(1) @@ -841,11 +841,11 @@ TVDiskID TBlobStorageGroupInfo::GetVDiskId(const TVDiskIdShort &vd) const { return TVDiskID(GroupID, GroupGeneration, vd); } -TActorId TBlobStorageGroupInfo::GetActorId(ui32 orderNumber) const { +TActorId TBlobStorageGroupInfo::GetActorId(ui32 orderNumber) const { return Dynamic.ServiceIdForOrderNumber[orderNumber]; } -TActorId TBlobStorageGroupInfo::GetActorId(const TVDiskIdShort &vd) const { +TActorId TBlobStorageGroupInfo::GetActorId(const TVDiskIdShort &vd) const { return GetActorId(Topology->GetOrderNumber(vd)); } diff --git a/ydb/core/blobstorage/groupinfo/blobstorage_groupinfo.h b/ydb/core/blobstorage/groupinfo/blobstorage_groupinfo.h index 64bff909a40..e2d1445be4e 100644 --- a/ydb/core/blobstorage/groupinfo/blobstorage_groupinfo.h +++ b/ydb/core/blobstorage/groupinfo/blobstorage_groupinfo.h @@ -76,7 +76,7 @@ public: // TYPES //////////////////////////////////////////////////////////////////////////////////////////////////////////////////// using TVDiskIds = TStackVec<TVDiskID, 16>; - using TServiceIds = TStackVec<TActorId, 16>; + using TServiceIds = TStackVec<TActorId, 16>; using TOrderNums = TStackVec<ui32, 16>; enum EBlobStateFlags { @@ -273,8 +273,8 @@ public: const ui32 GroupId; // blobstorage group generation const ui32 GroupGeneration; - // map to quickly get Service id (TActorId) from its order number inside TTopology - TVector<TActorId> ServiceIdForOrderNumber; + // map to quickly get Service id (TActorId) from its order number inside TTopology + TVector<TActorId> ServiceIdForOrderNumber; TDynamicInfo(ui32 groupId, ui32 groupGen); TDynamicInfo(const TDynamicInfo&) = default; @@ -282,7 +282,7 @@ public: TDynamicInfo &operator =(TDynamicInfo&&) = default; ~TDynamicInfo() = default; - void PushBackActorId(const TActorId &aid) { + void PushBackActorId(const TActorId &aid) { ServiceIdForOrderNumber.push_back(aid); } }; @@ -299,7 +299,7 @@ public: // for testing purposes; numFailDomains = 0 automatically selects possible minimum for provided erasure; groupId=0 // and groupGen=1 for constructed group explicit TBlobStorageGroupInfo(TBlobStorageGroupType gtype, ui32 numVDisksPerFailDomain = 1, - ui32 numFailDomains = 0, ui32 numFailRealms = 1, const TVector<TActorId> *vdiskIds = nullptr, + ui32 numFailDomains = 0, ui32 numFailRealms = 1, const TVector<TActorId> *vdiskIds = nullptr, EEncryptionMode encryptionMode = EEM_ENC_V1, ELifeCyclePhase lifeCyclePhase = ELCP_IN_USE, TCypherKey key = TCypherKey((const ui8*)"TestKey", 8)); @@ -351,9 +351,9 @@ public: // obtain full vdisk id having just short vdisk id TVDiskID GetVDiskId(const TVDiskIdShort &vd) const; // get ActorID by specified orderNumber (in range 0...TotalVDisks) - TActorId GetActorId(ui32 orderNumber) const; + TActorId GetActorId(ui32 orderNumber) const; // get ActorID having just short vdisk id - TActorId GetActorId(const TVDiskIdShort &vdisk) const; + TActorId GetActorId(const TVDiskIdShort &vdisk) const; // get the total number of VDisks in this group ui32 GetTotalVDisksNum() const; // get the total number of fail domains in this group diff --git a/ydb/core/blobstorage/groupinfo/blobstorage_groupinfo_blobmap_ut.cpp b/ydb/core/blobstorage/groupinfo/blobstorage_groupinfo_blobmap_ut.cpp index 628dbc55924..9147c6a8715 100644 --- a/ydb/core/blobstorage/groupinfo/blobstorage_groupinfo_blobmap_ut.cpp +++ b/ydb/core/blobstorage/groupinfo/blobstorage_groupinfo_blobmap_ut.cpp @@ -25,7 +25,7 @@ namespace { } } - bool PickSubgroup(ui32 hash, ui32 vdiskSz, TVDiskID *outVDisk, TActorId *outServiceIds) const { + bool PickSubgroup(ui32 hash, ui32 vdiskSz, TVDiskID *outVDisk, TActorId *outServiceIds) const { const ui32 subgroupSz = BlobSubgroupSize; Y_VERIFY(vdiskSz == subgroupSz); diff --git a/ydb/core/blobstorage/incrhuge/incrhuge.h b/ydb/core/blobstorage/incrhuge/incrhuge.h index ad13a89e034..c2f5848e72e 100644 --- a/ydb/core/blobstorage/incrhuge/incrhuge.h +++ b/ydb/core/blobstorage/incrhuge/incrhuge.h @@ -180,16 +180,16 @@ namespace NKikimr { // helper function used to construct incremental huge blob keeper's actod id on a local pdisk with specific // identifier - inline NActors::TActorId MakeIncrHugeKeeperId(ui32 pdiskId) { + inline NActors::TActorId MakeIncrHugeKeeperId(ui32 pdiskId) { char x[12] = {'b', 's', 'I', 'n', 'c', 'r', 'H', 'K', 0, 0, 0, 0}; x[8] = pdiskId; x[9] = pdiskId >> 8; x[10] = pdiskId >> 16; x[11] = pdiskId >> 24; - return NActors::TActorId(0, TStringBuf(x, 12)); + return NActors::TActorId(0, TStringBuf(x, 12)); } - inline ui32 PDiskIdFromIncrHugeKeeperId(const TActorId& keeperId) { + inline ui32 PDiskIdFromIncrHugeKeeperId(const TActorId& keeperId) { ui64 raw2 = keeperId.RawX2(); Y_VERIFY(raw2 < (ui64(1) << 32)); ui32 pdiskId = raw2; diff --git a/ydb/core/blobstorage/incrhuge/incrhuge_keeper.cpp b/ydb/core/blobstorage/incrhuge/incrhuge_keeper.cpp index 5b382b9cec8..05aafde94d1 100644 --- a/ydb/core/blobstorage/incrhuge/incrhuge_keeper.cpp +++ b/ydb/core/blobstorage/incrhuge/incrhuge_keeper.cpp @@ -4,7 +4,7 @@ #include <library/cpp/actors/core/actor_bootstrapped.h> #include <ydb/core/blobstorage/pdisk/blobstorage_pdisk.h> #include <library/cpp/digest/crc32c/crc32c.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/generic/queue.h> #include <util/generic/bitmap.h> @@ -156,7 +156,7 @@ namespace NKikimr { } void TKeeper::HandlePoison(const TActorContext& ctx) { - for (const TActorId& actorId : State.ChildActors) { + for (const TActorId& actorId : State.ChildActors) { ctx.Send(actorId, new TEvents::TEvPoisonPill); } Die(ctx); diff --git a/ydb/core/blobstorage/incrhuge/incrhuge_keeper_common.h b/ydb/core/blobstorage/incrhuge/incrhuge_keeper_common.h index 711dad56836..bc9f706f26b 100644 --- a/ydb/core/blobstorage/incrhuge/incrhuge_keeper_common.h +++ b/ydb/core/blobstorage/incrhuge/incrhuge_keeper_common.h @@ -23,7 +23,7 @@ namespace NKikimr { // initial incremental huge keeper settings struct TKeeperSettings { ui32 PDiskId; // PDisk id - TActorId PDiskActorId; // PDisk actor id + TActorId PDiskActorId; // PDisk actor id ui64 PDiskGuid; // PDisk GUID ui32 MinHugeBlobInBytes; // minimal blob size we will receive ui32 MinCleanChunks; // minimal number of clean chunks we will hold @@ -186,7 +186,7 @@ namespace NKikimr { THashMap<TIncrHugeBlobId, bool> DefragWriteInProgress; // set of spawned children actors - THashSet<TActorId> ChildActors; + THashSet<TActorId> ChildActors; TKeeperCommonState(const TKeeperSettings& settings) : Settings(settings) diff --git a/ydb/core/blobstorage/incrhuge/incrhuge_keeper_defrag.cpp b/ydb/core/blobstorage/incrhuge/incrhuge_keeper_defrag.cpp index 1a81233ab75..0cb43beb2ad 100644 --- a/ydb/core/blobstorage/incrhuge/incrhuge_keeper_defrag.cpp +++ b/ydb/core/blobstorage/incrhuge/incrhuge_keeper_defrag.cpp @@ -77,7 +77,7 @@ namespace NKikimr { IHLOG_DEBUG(ctx, "starting ChunkInProgress# %" PRIu32 " efficiency# %.3lf", ChunkInProgress, worstEfficiency); // create chunk scanner to read index - const TActorId actorId = ctx.Register(CreateRecoveryScanActor(ChunkInProgress, true, ChunkInProgressSerNum, + const TActorId actorId = ctx.Register(CreateRecoveryScanActor(ChunkInProgress, true, ChunkInProgressSerNum, static_cast<ui64>(EScanCookie::Defrag), Keeper.State), TMailboxType::HTSwap, AppData(ctx)->BatchPoolId); const bool inserted = Keeper.State.ChildActors.insert(actorId).second; Y_VERIFY(inserted); @@ -98,7 +98,7 @@ namespace NKikimr { } } - void TDefragmenter::ApplyScan(const TActorId& sender, TEvIncrHugeScanResult& msg, const TActorContext& ctx) { + void TDefragmenter::ApplyScan(const TActorId& sender, TEvIncrHugeScanResult& msg, const TActorContext& ctx) { const size_t num = Keeper.State.ChildActors.erase(sender); Y_VERIFY(num == 1); diff --git a/ydb/core/blobstorage/incrhuge/incrhuge_keeper_defrag.h b/ydb/core/blobstorage/incrhuge/incrhuge_keeper_defrag.h index 715247fc5af..e6bb8355a4f 100644 --- a/ydb/core/blobstorage/incrhuge/incrhuge_keeper_defrag.h +++ b/ydb/core/blobstorage/incrhuge/incrhuge_keeper_defrag.h @@ -44,7 +44,7 @@ namespace NKikimr { // cleaned void UpdateChunkState(TChunkIdx chunkIdx, const TActorContext& ctx); - void ApplyScan(const TActorId& sender, TEvIncrHugeScanResult& msg, const TActorContext& ctx); + void ApplyScan(const TActorId& sender, TEvIncrHugeScanResult& msg, const TActorContext& ctx); void HandleControlDefrag(TEvIncrHugeControlDefrag::TPtr& ev, const TActorContext& ctx); diff --git a/ydb/core/blobstorage/incrhuge/incrhuge_keeper_delete.cpp b/ydb/core/blobstorage/incrhuge/incrhuge_keeper_delete.cpp index 493e2450de4..141aa8f0b97 100644 --- a/ydb/core/blobstorage/incrhuge/incrhuge_keeper_delete.cpp +++ b/ydb/core/blobstorage/incrhuge/incrhuge_keeper_delete.cpp @@ -14,7 +14,7 @@ namespace NKikimr { }; struct TDeleter::TDeleteQueueItem { - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; const ui8 Owner; const ui64 SeqNo; diff --git a/ydb/core/blobstorage/incrhuge/incrhuge_keeper_read.h b/ydb/core/blobstorage/incrhuge/incrhuge_keeper_read.h index c64573df4eb..d6e104d7d79 100644 --- a/ydb/core/blobstorage/incrhuge/incrhuge_keeper_read.h +++ b/ydb/core/blobstorage/incrhuge/incrhuge_keeper_read.h @@ -17,7 +17,7 @@ namespace NKikimr { TIncrHugeBlobId Id; ui32 Offset; ui32 Size; - TActorId Sender; + TActorId Sender; ui64 Cookie; }; using TReadQueue = TQueue<TReadQueueItem>; diff --git a/ydb/core/blobstorage/incrhuge/incrhuge_keeper_recovery.cpp b/ydb/core/blobstorage/incrhuge/incrhuge_keeper_recovery.cpp index 18e23933456..afdf92e80d7 100644 --- a/ydb/core/blobstorage/incrhuge/incrhuge_keeper_recovery.cpp +++ b/ydb/core/blobstorage/incrhuge/incrhuge_keeper_recovery.cpp @@ -80,7 +80,7 @@ namespace NKikimr { item.ScanQueue.pop(); // issue scan request - const TActorId actorId = ctx.Register(CreateRecoveryScanActor(chunkIdx, true, chunk->ChunkSerNum, + const TActorId actorId = ctx.Register(CreateRecoveryScanActor(chunkIdx, true, chunk->ChunkSerNum, static_cast<ui64>(EScanCookie::Recovery), Keeper.State)); const bool added = Keeper.State.ChildActors.insert(actorId).second; Y_VERIFY(added); @@ -144,7 +144,7 @@ namespace NKikimr { Y_VERIFY(status == NKikimrProto::OK); static const TMaybe<ui64> none; - const TActorId actorId = ctx.Register(CreateRecoveryReadLogActor(Keeper.State.Settings.PDiskActorId, + const TActorId actorId = ctx.Register(CreateRecoveryReadLogActor(Keeper.State.Settings.PDiskActorId, Keeper.State.PDiskParams->Owner, Keeper.State.PDiskParams->OwnerRound, chunks ? chunks->Lsn : none, deletes ? deletes->Lsn : none)); const bool added = Keeper.State.ChildActors.insert(actorId).second; @@ -152,7 +152,7 @@ namespace NKikimr { IHLOG_INFO(ctx, "[IncrHugeKeeper PDisk# %09" PRIu32 "] starting ReadLog", Keeper.State.Settings.PDiskId); } - void TRecovery::ApplyReadLog(const TActorId& sender, TEvIncrHugeReadLogResult& msg, const TActorContext& ctx) { + void TRecovery::ApplyReadLog(const TActorId& sender, TEvIncrHugeReadLogResult& msg, const TActorContext& ctx) { const size_t num = Keeper.State.ChildActors.erase(sender); Y_VERIFY(num == 1); @@ -287,7 +287,7 @@ namespace NKikimr { ProcessScanQueue(ctx); } - void TRecovery::ApplyScan(const TActorId& sender, TEvIncrHugeScanResult& msg, const TActorContext& ctx) { + void TRecovery::ApplyScan(const TActorId& sender, TEvIncrHugeScanResult& msg, const TActorContext& ctx) { const size_t num = Keeper.State.ChildActors.erase(sender); Y_VERIFY(num == 1); @@ -460,7 +460,7 @@ namespace NKikimr { if (ScanBytesInFlight && ScanBytesInFlight + bytes > Keeper.State.PDiskParams->ChunkSize) { return false; } - const TActorId actorId = ctx.Register(CreateRecoveryScanActor(item.ChunkIdx, item.IndexOnly, + const TActorId actorId = ctx.Register(CreateRecoveryScanActor(item.ChunkIdx, item.IndexOnly, item.ChunkSerNum, static_cast<ui64>(EScanCookie::Recovery), Keeper.State)); const bool added = Keeper.State.ChildActors.insert(actorId).second; Y_VERIFY(added); diff --git a/ydb/core/blobstorage/incrhuge/incrhuge_keeper_recovery.h b/ydb/core/blobstorage/incrhuge/incrhuge_keeper_recovery.h index 81990b6e297..53463b80c90 100644 --- a/ydb/core/blobstorage/incrhuge/incrhuge_keeper_recovery.h +++ b/ydb/core/blobstorage/incrhuge/incrhuge_keeper_recovery.h @@ -16,7 +16,7 @@ namespace NKikimr { TVDiskID VDiskId; ui8 Owner; ui64 FirstLsn; - TActorId Sender; + TActorId Sender; ui64 Cookie; TQueue<std::pair<TChunkIdx, TChunkInfo *>> ScanQueue; THashMap<TChunkIdx, TDynBitMap> DeletedItemsMap; @@ -61,7 +61,7 @@ namespace NKikimr { TChunkIdx ChunkIdx; TChunkInfo& Chunk; }; - THashMap<TActorId, TScanInfo> ScannerMap; + THashMap<TActorId, TScanInfo> ScannerMap; public: TRecovery(TKeeper& keeper); @@ -72,10 +72,10 @@ namespace NKikimr { const TActorContext& ctx); // handle read log result - void ApplyReadLog(const TActorId& sender, TEvIncrHugeReadLogResult& msg, const TActorContext& ctx); + void ApplyReadLog(const TActorId& sender, TEvIncrHugeReadLogResult& msg, const TActorContext& ctx); // handle scan result - void ApplyScan(const TActorId& sender, TEvIncrHugeScanResult& msg, const TActorContext& ctx); + void ApplyScan(const TActorId& sender, TEvIncrHugeScanResult& msg, const TActorContext& ctx); // handle init message from client void HandleInit(TEvIncrHugeInit::TPtr& ev, const TActorContext& ctx); diff --git a/ydb/core/blobstorage/incrhuge/incrhuge_keeper_recovery_read_log.cpp b/ydb/core/blobstorage/incrhuge/incrhuge_keeper_recovery_read_log.cpp index 9fb3c502949..b5499213caf 100644 --- a/ydb/core/blobstorage/incrhuge/incrhuge_keeper_recovery_read_log.cpp +++ b/ydb/core/blobstorage/incrhuge/incrhuge_keeper_recovery_read_log.cpp @@ -6,10 +6,10 @@ namespace NKikimr { namespace NIncrHuge { class TReadLogActor : public TActor<TReadLogActor> { - const TActorId PDiskActorId; + const TActorId PDiskActorId; const ui8 Owner; const NPDisk::TOwnerRound OwnerRound; - TActorId KeeperActorId; + TActorId KeeperActorId; TMaybe<ui64> ChunksEntrypointLsn; TChunkRecordMerger ChunkMerger; TMaybe<ui64> DeletesEntrypointLsn; @@ -20,7 +20,7 @@ namespace NKikimr { return NKikimrServices::TActivity::BS_INCR_HUGE_KEEPER_RECOVERY_READ_LOG; } - TReadLogActor(const TActorId& pdiskActorId, ui8 owner, NPDisk::TOwnerRound ownerRound, + TReadLogActor(const TActorId& pdiskActorId, ui8 owner, NPDisk::TOwnerRound ownerRound, TMaybe<ui64> chunksEntrypointLsn, TMaybe<ui64> deletesEntrypointLsn) : TActor<TReadLogActor>(&TReadLogActor::StateFunc) , PDiskActorId(pdiskActorId) @@ -31,7 +31,7 @@ namespace NKikimr { {} private: - TAutoPtr<IEventHandle> AfterRegister(const TActorId& self, const TActorId& parentId) override { + TAutoPtr<IEventHandle> AfterRegister(const TActorId& self, const TActorId& parentId) override { return new IEventHandle(self, parentId, new TEvents::TEvBootstrap); } @@ -132,7 +132,7 @@ namespace NKikimr { ) }; - IActor *CreateRecoveryReadLogActor(const TActorId& pdiskActorId, ui8 owner, NPDisk::TOwnerRound ownerRound, + IActor *CreateRecoveryReadLogActor(const TActorId& pdiskActorId, ui8 owner, NPDisk::TOwnerRound ownerRound, TMaybe<ui64> chunksEntrypointLsn, TMaybe<ui64> deletesEntrypointLsn) { return new TReadLogActor(pdiskActorId, owner, ownerRound, chunksEntrypointLsn, deletesEntrypointLsn); } diff --git a/ydb/core/blobstorage/incrhuge/incrhuge_keeper_recovery_read_log.h b/ydb/core/blobstorage/incrhuge/incrhuge_keeper_recovery_read_log.h index 5604b09383d..551e8ad6412 100644 --- a/ydb/core/blobstorage/incrhuge/incrhuge_keeper_recovery_read_log.h +++ b/ydb/core/blobstorage/incrhuge/incrhuge_keeper_recovery_read_log.h @@ -11,7 +11,7 @@ namespace NKikimr { namespace NIncrHuge { - NActors::IActor *CreateRecoveryReadLogActor(const NActors::TActorId& pdiskActorId, ui8 owner, + NActors::IActor *CreateRecoveryReadLogActor(const NActors::TActorId& pdiskActorId, ui8 owner, NPDisk::TOwnerRound ownerRound, TMaybe<ui64> chunksEntrypointLsn, TMaybe<ui64> deletesEntrypointLsn); } // NIncrHuge diff --git a/ydb/core/blobstorage/incrhuge/incrhuge_keeper_recovery_scan.cpp b/ydb/core/blobstorage/incrhuge/incrhuge_keeper_recovery_scan.cpp index 15d6cd0e2e0..eb60105861b 100644 --- a/ydb/core/blobstorage/incrhuge/incrhuge_keeper_recovery_scan.cpp +++ b/ydb/core/blobstorage/incrhuge/incrhuge_keeper_recovery_scan.cpp @@ -10,7 +10,7 @@ namespace NKikimr { const TChunkSerNum ChunkSerNum; const ui64 Cookie; const TKeeperCommonState& State; - TActorId KeeperActorId; + TActorId KeeperActorId; public: static constexpr NKikimrServices::TActivity::EType ActorActivityType() { @@ -28,7 +28,7 @@ namespace NKikimr { {} private: - TAutoPtr<IEventHandle> AfterRegister(const TActorId& self, const TActorId& parentId) override { + TAutoPtr<IEventHandle> AfterRegister(const TActorId& self, const TActorId& parentId) override { return new IEventHandle(self, parentId, new TEvents::TEvBootstrap); } diff --git a/ydb/core/blobstorage/incrhuge/incrhuge_keeper_write.cpp b/ydb/core/blobstorage/incrhuge/incrhuge_keeper_write.cpp index 19d9af273e1..ca6cedf5d8f 100644 --- a/ydb/core/blobstorage/incrhuge/incrhuge_keeper_write.cpp +++ b/ydb/core/blobstorage/incrhuge/incrhuge_keeper_write.cpp @@ -40,7 +40,7 @@ namespace NKikimr { TWritePayloadPtr Payload; // the actor who sent this request - TActorId Sender; + TActorId Sender; // cookie of request; the same for response ui64 Cookie; diff --git a/ydb/core/blobstorage/incrhuge/ut/faulty_pdisk.h b/ydb/core/blobstorage/incrhuge/ut/faulty_pdisk.h index 16424c030dc..73735697022 100644 --- a/ydb/core/blobstorage/incrhuge/ut/faulty_pdisk.h +++ b/ydb/core/blobstorage/incrhuge/ut/faulty_pdisk.h @@ -1,12 +1,12 @@ #pragma once class TFaultyPDiskActor : public TActor<TFaultyPDiskActor> { - TActorId PDiskId; + TActorId PDiskId; ui32 Counter; TManualEvent *Event; public: - TFaultyPDiskActor(const TActorId& pdiskId, ui32 counter, TManualEvent *event) + TFaultyPDiskActor(const TActorId& pdiskId, ui32 counter, TManualEvent *event) : TActor<TFaultyPDiskActor>(&TFaultyPDiskActor::StateFunc) , PDiskId(pdiskId) , Counter(counter) diff --git a/ydb/core/blobstorage/incrhuge/ut/incrhuge_basic_ut.cpp b/ydb/core/blobstorage/incrhuge/ut/incrhuge_basic_ut.cpp index 6fe6de8fc42..59d3cb999cf 100644 --- a/ydb/core/blobstorage/incrhuge/ut/incrhuge_basic_ut.cpp +++ b/ydb/core/blobstorage/incrhuge/ut/incrhuge_basic_ut.cpp @@ -29,8 +29,8 @@ public: ui64 DiskSize; ui64 PDiskGuid; ui64 PDiskKey; - TActorId PDiskId; - TActorId KeeperId; + TActorId PDiskId; + TActorId KeeperId; std::unique_ptr<TActorSystem> ActorSystem; TTempDir TempDir; std::unique_ptr<TAppData> AppData; @@ -66,10 +66,10 @@ public: TActorSetupCmd pDiskSetup(CreatePDisk(pDiskConfig.Get(), PDiskKey, Counters), TMailboxType::Revolving, 0); setup->LocalServices.emplace_back(PDiskId, pDiskSetup); - TActorId pdiskActorId; + TActorId pdiskActorId; if (counter) { char x[12] = {'f', 'a', 'u', 'l', 't', 'y', 'p', 'd', 'i', 's', 'k'}; - pdiskActorId = TActorId(0, x); + pdiskActorId = TActorId(0, x); setup->LocalServices.emplace_back(pdiskActorId, TActorSetupCmd(new TFaultyPDiskActor(PDiskId, counter, event), TMailboxType::Simple, 0)); } else { @@ -93,7 +93,7 @@ public: //////////////////////////////////////////////////////////////////////////////////////////////////////////////// // LOGGER - NActors::TActorId loggerActorId{1, "logger"}; + NActors::TActorId loggerActorId{1, "logger"}; TIntrusivePtr<NActors::NLog::TSettings> logSettings{new NActors::NLog::TSettings{loggerActorId, NKikimrServices::LOGGER, NActors::NLog::PRI_ERROR, NActors::NLog::PRI_ERROR, 0}}; logSettings->Append( diff --git a/ydb/core/blobstorage/incrhuge/ut/test_actor_concurrent.h b/ydb/core/blobstorage/incrhuge/ut/test_actor_concurrent.h index 163f5d3a21f..d7f370706fe 100644 --- a/ydb/core/blobstorage/incrhuge/ut/test_actor_concurrent.h +++ b/ydb/core/blobstorage/incrhuge/ut/test_actor_concurrent.h @@ -50,7 +50,7 @@ public: private: TVDiskID VDiskId; - TActorId KeeperId; + TActorId KeeperId; TManualEvent *Event; ui8 Owner = 1; TTestActorState& State; @@ -70,7 +70,7 @@ private: }; public: - TTestActorConcurrent(const TActorId& keeperId, TManualEvent *event, TTestActorState& state, ui32 numActions, + TTestActorConcurrent(const TActorId& keeperId, TManualEvent *event, TTestActorState& state, ui32 numActions, ui32 generation, ui32 writeScore = 10, ui32 deleteScore = 10, ui32 readScore = 5) : KeeperId(keeperId) , Event(event) diff --git a/ydb/core/blobstorage/incrhuge/ut/test_actor_seq.h b/ydb/core/blobstorage/incrhuge/ut/test_actor_seq.h index 8d55887624f..9bf87cbd1b7 100644 --- a/ydb/core/blobstorage/incrhuge/ut/test_actor_seq.h +++ b/ydb/core/blobstorage/incrhuge/ut/test_actor_seq.h @@ -28,7 +28,7 @@ public: }; private: - TActorId KeeperId; + TActorId KeeperId; TTestActorState& State; ui8 Owner = 1; ui32 NumReadsPending = 0; @@ -39,7 +39,7 @@ private: TManualEvent *Event; public: - TTestActorSeq(const TActorId& keeperId, TTestActorState& state, ui32 generation, TManualEvent *event) + TTestActorSeq(const TActorId& keeperId, TTestActorState& state, ui32 generation, TManualEvent *event) : KeeperId(keeperId) , State(state) , Rng(42) diff --git a/ydb/core/blobstorage/nodewarden/blobstorage_node_warden_ut.cpp b/ydb/core/blobstorage/nodewarden/blobstorage_node_warden_ut.cpp index 57544f978b0..e3c74cce7b4 100644 --- a/ydb/core/blobstorage/nodewarden/blobstorage_node_warden_ut.cpp +++ b/ydb/core/blobstorage/nodewarden/blobstorage_node_warden_ut.cpp @@ -243,7 +243,7 @@ void SetupServices(TTestActorRuntime &runtime, TString extraPath, TIntrusivePtr< runtime.Initialize(app.Unwrap()); for (ui32 nodeIndex = 0; nodeIndex < runtime.GetNodeCount(); ++nodeIndex) { - TActorId localActor = runtime.GetLocalServiceId( + TActorId localActor = runtime.GetLocalServiceId( MakeBlobStorageNodeWardenID(runtime.GetNodeId(nodeIndex)), nodeIndex); runtime.EnableScheduleForActor(localActor, true); } @@ -300,11 +300,11 @@ Y_UNIT_TEST_SUITE(TBlobStorageWardenTest) { return pdiskId; } - void Put(TTestActorRuntime &runtime, TActorId &sender, ui32 groupId, TLogoBlobID logoBlobId, TString data, NKikimrProto::EReplyStatus expectAnsver = NKikimrProto::OK) { + void Put(TTestActorRuntime &runtime, TActorId &sender, ui32 groupId, TLogoBlobID logoBlobId, TString data, NKikimrProto::EReplyStatus expectAnsver = NKikimrProto::OK) { VERBOSE_COUT(" Sending TEvPut"); - TActorId proxy = MakeBlobStorageProxyID(groupId); + TActorId proxy = MakeBlobStorageProxyID(groupId); ui32 nodeId = sender.NodeId(); - TActorId nodeWarden = MakeBlobStorageNodeWardenID(nodeId); + TActorId nodeWarden = MakeBlobStorageNodeWardenID(nodeId); ui64 cookie = 6543210; runtime.Send(new IEventHandle(proxy, sender, new TEvBlobStorage::TEvPut(logoBlobId, data, TInstant::Max()), @@ -322,7 +322,7 @@ Y_UNIT_TEST_SUITE(TBlobStorageWardenTest) { auto stateStorage = runtime.GetAppData().DomainsInfo->GetDefaultStateStorageGroup(domainId); NKikimrBlobStorage::TDefineStoragePool storagePool = runtime.GetAppData().DomainsInfo->GetDomain(domainId).StoragePoolTypes.at(kind); - TActorId edge = runtime.AllocateEdgeActor(); + TActorId edge = runtime.AllocateEdgeActor(); auto request = std::make_unique<TEvBlobStorage::TEvControllerConfigRequest>(); Y_VERIFY(storagePool.GetKind() == kind); storagePool.ClearStoragePoolId(); @@ -342,7 +342,7 @@ Y_UNIT_TEST_SUITE(TBlobStorageWardenTest) { ui32 GetGroupFromPool(TTestBasicRuntime& runtime, ui32 domainId, TString poolName) { auto stateStorage = runtime.GetAppData().DomainsInfo->GetDefaultStateStorageGroup(domainId); - TActorId edge = runtime.AllocateEdgeActor(); + TActorId edge = runtime.AllocateEdgeActor(); auto selectGroups = std::make_unique<TEvBlobStorage::TEvControllerSelectGroups>(); auto *record = &selectGroups->Record; record->SetReturnAllMatchingGroups(true); @@ -358,12 +358,12 @@ Y_UNIT_TEST_SUITE(TBlobStorageWardenTest) { return reply->Get()->Record.GetMatchingGroups(0).GetGroups(0).GetGroupID(); } - void SendToBsProxy(TTestBasicRuntime& runtime, TActorId sender, ui32 groupId, IEventBase *ev, ui64 cookie = 0) { + void SendToBsProxy(TTestBasicRuntime& runtime, TActorId sender, ui32 groupId, IEventBase *ev, ui64 cookie = 0) { auto flags = NActors::IEventHandle::FlagTrackDelivery | NActors::IEventHandle::FlagForwardOnNondelivery; - TActorId recipient = MakeBlobStorageProxyID(groupId); - TActorId nodeWarden = MakeBlobStorageNodeWardenID(sender.NodeId()); + TActorId recipient = MakeBlobStorageProxyID(groupId); + TActorId nodeWarden = MakeBlobStorageNodeWardenID(sender.NodeId()); return runtime.Send(new IEventHandle(recipient, sender, ev, flags, cookie, &nodeWarden, {}), sender.NodeId() - runtime.GetNodeId(0)); } @@ -371,7 +371,7 @@ Y_UNIT_TEST_SUITE(TBlobStorageWardenTest) { NKikimrBlobStorage::TDefineStoragePool DescribeStoragePool(TTestBasicRuntime& runtime, ui32 domainId, const TString& name) { auto stateStorage = runtime.GetAppData().DomainsInfo->GetDefaultStateStorageGroup(domainId); - TActorId edge = runtime.AllocateEdgeActor(); + TActorId edge = runtime.AllocateEdgeActor(); auto selectGroups = std::make_unique<TEvBlobStorage::TEvControllerConfigRequest>(); auto* request = selectGroups->Record.MutableRequest(); auto* readPool = request->AddCommand()->MutableReadStoragePool(); @@ -389,7 +389,7 @@ Y_UNIT_TEST_SUITE(TBlobStorageWardenTest) { void RemoveStoragePool(TTestBasicRuntime& runtime, ui32 domainId, const NKikimrBlobStorage::TDefineStoragePool& storagePool) { auto stateStorage = runtime.GetAppData().DomainsInfo->GetDefaultStateStorageGroup(domainId); - TActorId edge = runtime.AllocateEdgeActor(); + TActorId edge = runtime.AllocateEdgeActor(); auto selectGroups = std::make_unique<TEvBlobStorage::TEvControllerConfigRequest>(); auto* request = selectGroups->Record.MutableRequest(); auto* deletePool = request->AddCommand()->MutableDeleteStoragePool(); @@ -646,8 +646,8 @@ Y_UNIT_TEST_SUITE(TBlobStorageWardenTest) { TTestBasicRuntime runtime(2, false); TIntrusivePtr<NPDisk::TSectorMap> sectorMap(new NPDisk::TSectorMap(32ull << 30ull)); Setup(runtime, "SectorMap:new_pdisk", sectorMap); - TActorId sender0 = runtime.AllocateEdgeActor(0); -// TActorId sender1 = runtime.AllocateEdgeActor(1); + TActorId sender0 = runtime.AllocateEdgeActor(0); +// TActorId sender1 = runtime.AllocateEdgeActor(1); VERBOSE_COUT(" Formatting pdisk"); FormatPDiskRandomKeys(tempDir() + "/new_pdisk.dat", sectorMap->DeviceSize, 32 << 20, 1, false, sectorMap); @@ -667,7 +667,7 @@ Y_UNIT_TEST_SUITE(TBlobStorageWardenTest) { EntropyPool().Read(&guid2, sizeof(guid2)); } ui32 nodeId = runtime.GetNodeId(0); - TActorId pDiskActorId = MakeBlobStoragePDiskID(nodeId, pDiskId); + TActorId pDiskActorId = MakeBlobStoragePDiskID(nodeId, pDiskId); for (;;) { runtime.Send(new IEventHandle(pDiskActorId, sender0, new NPDisk::TEvYardInit(1, vDiskId, guid)), 0); TAutoPtr<IEventHandle> handle; diff --git a/ydb/core/blobstorage/nodewarden/group_stat_aggregator.cpp b/ydb/core/blobstorage/nodewarden/group_stat_aggregator.cpp index eb306c49581..9f63143b322 100644 --- a/ydb/core/blobstorage/nodewarden/group_stat_aggregator.cpp +++ b/ydb/core/blobstorage/nodewarden/group_stat_aggregator.cpp @@ -15,7 +15,7 @@ namespace { }; const ui32 GroupId; - const TActorId VDiskServiceId; + const TActorId VDiskServiceId; const TDuration ReportPeriod; TGroupStat Accum; THashMap<ui32, TNodeStat> PerNodeStat; @@ -25,7 +25,7 @@ namespace { return NKikimrServices::TActivity::NODE_WARDEN_STATAGGR_ACTOR; } - TGroupStatAggregatorActor(ui32 groupId, const TActorId& vdiskServiceId, TDuration reportPeriod = TDuration::Seconds(10)) + TGroupStatAggregatorActor(ui32 groupId, const TActorId& vdiskServiceId, TDuration reportPeriod = TDuration::Seconds(10)) : GroupId(groupId) , VDiskServiceId(vdiskServiceId) , ReportPeriod(reportPeriod) @@ -39,7 +39,7 @@ namespace { TEvGroupStatReport *msg = ev->Get(); TGroupStat stat; if (msg->GetGroupId() == GroupId && msg->GetStat(stat)) { - const TActorId& sender = ev->Sender; + const TActorId& sender = ev->Sender; const ui32 nodeId = sender.NodeId(); auto it = PerNodeStat.find(nodeId); if (it != PerNodeStat.end()) { @@ -88,7 +88,7 @@ namespace { namespace NKikimr { - IActor *CreateGroupStatAggregatorActor(ui32 groupId, const TActorId& vdiskServiceId) { + IActor *CreateGroupStatAggregatorActor(ui32 groupId, const TActorId& vdiskServiceId) { return new TGroupStatAggregatorActor(groupId, vdiskServiceId); } diff --git a/ydb/core/blobstorage/nodewarden/group_stat_aggregator.h b/ydb/core/blobstorage/nodewarden/group_stat_aggregator.h index 9715470d1f8..24c1f0a93e7 100644 --- a/ydb/core/blobstorage/nodewarden/group_stat_aggregator.h +++ b/ydb/core/blobstorage/nodewarden/group_stat_aggregator.h @@ -13,10 +13,10 @@ namespace NKikimr { struct TEvGroupStatReport : TEventPB<TEvGroupStatReport, NKikimrBlobStorage::TEvGroupStatReport, TEvBlobStorage::EvGroupStatReport> { TEvGroupStatReport() = default; - TEvGroupStatReport(const TActorId& vdiskServiceId, ui32 groupId, const TGroupStat& stat) + TEvGroupStatReport(const TActorId& vdiskServiceId, ui32 groupId, const TGroupStat& stat) { Record.SetGroupId(groupId); - ActorIdToProto(vdiskServiceId, Record.MutableVDiskServiceId()); + ActorIdToProto(vdiskServiceId, Record.MutableVDiskServiceId()); stat.Serialize(&Record); } @@ -28,8 +28,8 @@ namespace NKikimr { return Record.GetGroupId(); } - TActorId GetVDiskServiceId() const { - return ActorIdFromProto(Record.GetVDiskServiceId()); + TActorId GetVDiskServiceId() const { + return ActorIdFromProto(Record.GetVDiskServiceId()); } }; @@ -59,6 +59,6 @@ namespace NKikimr { return TActorId(vdiskServiceId.NodeId(), TStringBuf(x, 12)); } - NActors::IActor *CreateGroupStatAggregatorActor(ui32 groupId, const TActorId& vdiskServiceId); + NActors::IActor *CreateGroupStatAggregatorActor(ui32 groupId, const TActorId& vdiskServiceId); } // NKikimr diff --git a/ydb/core/blobstorage/nodewarden/node_warden_impl.h b/ydb/core/blobstorage/nodewarden/node_warden_impl.h index 5ef72fc7d80..be67e5ef670 100644 --- a/ydb/core/blobstorage/nodewarden/node_warden_impl.h +++ b/ydb/core/blobstorage/nodewarden/node_warden_impl.h @@ -66,7 +66,7 @@ namespace NKikimr::NStorage { class TNodeWarden : public TActorBootstrapped<TNodeWarden> { TIntrusivePtr<TNodeWardenConfig> Cfg; TIntrusivePtr<TDsProxyNodeMon> DsProxyNodeMon; - TActorId DsProxyNodeMonActor; + TActorId DsProxyNodeMonActor; TIntrusivePtr<TDsProxyPerPoolCounters> DsProxyPerPoolCounters; //////////////////////////////////////////////////////////////////////////////////////////////////////////////// @@ -185,8 +185,8 @@ namespace NKikimr::NStorage { TGroupStat Stat; }; - TSet<TActorId> RunningVDiskServiceIds; - TMap<TActorId, TAggregatorInfo> PerAggregatorInfo; + TSet<TActorId> RunningVDiskServiceIds; + TMap<TActorId, TAggregatorInfo> PerAggregatorInfo; void ReportLatencies(); void Handle(TEvGroupStatReport::TPtr ev); @@ -315,7 +315,7 @@ namespace NKikimr::NStorage { //////////////////////////////////////////////////////////////////////////////////////////////////////////////// std::queue<std::unique_ptr<IActor>> SyncOpQ; - TActorId SyncActorId; + TActorId SyncActorId; void InvokeSyncOp(std::unique_ptr<IActor> actor); void Handle(TEvents::TEvInvokeResult::TPtr ev); @@ -381,9 +381,9 @@ namespace NKikimr::NStorage { //////////////////////////////////////////////////////////////////////////////////////////////////////////////// - std::map<TActorId, std::deque<std::unique_ptr<IEventHandle>>> PendingMessageQ; + std::map<TActorId, std::deque<std::unique_ptr<IEventHandle>>> PendingMessageQ; - void RegisterPendingActor(const TActorId& actorId); + void RegisterPendingActor(const TActorId& actorId); void EnqueuePendingMessage(TAutoPtr<IEventHandle> ev); void IssuePendingMessages(const TActorId& actorId); diff --git a/ydb/core/blobstorage/nodewarden/node_warden_resource.cpp b/ydb/core/blobstorage/nodewarden/node_warden_resource.cpp index c641ea5f9d2..d8179b7af81 100644 --- a/ydb/core/blobstorage/nodewarden/node_warden_resource.cpp +++ b/ydb/core/blobstorage/nodewarden/node_warden_resource.cpp @@ -10,7 +10,7 @@ using namespace NKikimr; using namespace NStorage; -void TNodeWarden::RegisterPendingActor(const TActorId& actorId) { +void TNodeWarden::RegisterPendingActor(const TActorId& actorId) { const bool inserted = PendingMessageQ.emplace(actorId, std::deque<std::unique_ptr<IEventHandle>>()).second; Y_VERIFY(inserted); } @@ -84,7 +84,7 @@ void TNodeWarden::ApplyServiceSet(const NKikimrBlobStorage::TNodeWardenServiceSe } void TNodeWarden::HandleIncrHugeInit(NIncrHuge::TEvIncrHugeInit::TPtr ev) { - const TActorId keeperId = ev->GetForwardOnNondeliveryRecipient(); + const TActorId keeperId = ev->GetForwardOnNondeliveryRecipient(); const ui32 pdiskId = NIncrHuge::PDiskIdFromIncrHugeKeeperId(keeperId); // find local pdisk config to extract GUID diff --git a/ydb/core/blobstorage/nodewarden/node_warden_stat_aggr.cpp b/ydb/core/blobstorage/nodewarden/node_warden_stat_aggr.cpp index 00ce97add75..6a357489fc5 100644 --- a/ydb/core/blobstorage/nodewarden/node_warden_stat_aggr.cpp +++ b/ydb/core/blobstorage/nodewarden/node_warden_stat_aggr.cpp @@ -15,7 +15,7 @@ void TNodeWarden::ReportLatencies() { auto& record = ev->Record; auto *pb = record.AddPerGroupReport(); pb->SetGroupId(info.GroupId); - ActorIdToProto(kv.first, pb->MutableVDiskServiceId()); + ActorIdToProto(kv.first, pb->MutableVDiskServiceId()); info.Stat.Serialize(pb); } @@ -26,7 +26,7 @@ void TNodeWarden::ReportLatencies() { void TNodeWarden::Handle(TEvGroupStatReport::TPtr ev) { TEvGroupStatReport *msg = ev->Get(); - TActorId vdiskServiceId = msg->GetVDiskServiceId(); + TActorId vdiskServiceId = msg->GetVDiskServiceId(); TGroupStat stat; if (RunningVDiskServiceIds.count(vdiskServiceId) && msg->GetStat(stat)) { @@ -39,7 +39,7 @@ void TNodeWarden::Handle(TEvGroupStatReport::TPtr ev) { void TNodeWarden::StartAggregator(const TActorId& vdiskServiceId, ui32 groupId) { if (RunningVDiskServiceIds.emplace(vdiskServiceId).second) { - const TActorId groupStatAggregatorId = MakeGroupStatAggregatorId(vdiskServiceId); + const TActorId groupStatAggregatorId = MakeGroupStatAggregatorId(vdiskServiceId); const TActorId actorId = Register(CreateGroupStatAggregatorActor(groupId, vdiskServiceId), TMailboxType::Revolving, AppData()->SystemPoolId); TlsActivationContext->ExecutorThread.ActorSystem->RegisterLocalService(groupStatAggregatorId, actorId); @@ -48,7 +48,7 @@ void TNodeWarden::StartAggregator(const TActorId& vdiskServiceId, ui32 groupId) void TNodeWarden::StopAggregator(const TActorId& vdiskServiceId) { if (RunningVDiskServiceIds.erase(vdiskServiceId)) { - const TActorId groupStatAggregatorId = MakeGroupStatAggregatorId(vdiskServiceId); + const TActorId groupStatAggregatorId = MakeGroupStatAggregatorId(vdiskServiceId); TActivationContext::Send(new IEventHandle(TEvents::TSystem::Poison, 0, groupStatAggregatorId, {}, nullptr, 0)); PerAggregatorInfo.erase(groupStatAggregatorId); } diff --git a/ydb/core/blobstorage/nodewarden/ut_sequence/dsproxy_config_retrieval.cpp b/ydb/core/blobstorage/nodewarden/ut_sequence/dsproxy_config_retrieval.cpp index 6408202e63c..040411e06c3 100644 --- a/ydb/core/blobstorage/nodewarden/ut_sequence/dsproxy_config_retrieval.cpp +++ b/ydb/core/blobstorage/nodewarden/ut_sequence/dsproxy_config_retrieval.cpp @@ -83,9 +83,9 @@ void SetupServices(TTestBasicRuntime& runtime) { // setup box and storage pool for testing { - TActorId edge = runtime.AllocateEdgeActor(); + TActorId edge = runtime.AllocateEdgeActor(); - runtime.Send(new IEventHandle(GetNameserviceActorId(), edge, new TEvInterconnect::TEvListNodes)); + runtime.Send(new IEventHandle(GetNameserviceActorId(), edge, new TEvInterconnect::TEvListNodes)); TAutoPtr<IEventHandle> handleNodesInfo; auto nodesInfo = runtime.GrabEdgeEventRethrow<TEvInterconnect::TEvNodesInfo>(handleNodesInfo); @@ -137,9 +137,9 @@ Y_UNIT_TEST_SUITE(NodeWardenDsProxyConfigRetrieval) { const ui64 tabletId = MakeBSControllerID(0); bool allowConfiguring = false; - TActorId nodeWardenId; + TActorId nodeWardenId; TTestActorRuntimeBase::TRegistrationObserver prevReg = runtime.SetRegistrationObserverFunc( - [&](TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId) { + [&](TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId) { if (IActor *actor = runtime.FindActor(actorId); dynamic_cast<NKikimr::NStorage::TNodeWarden*>(actor)) { UNIT_ASSERT(!nodeWardenId); nodeWardenId = actorId; @@ -149,7 +149,7 @@ Y_UNIT_TEST_SUITE(NodeWardenDsProxyConfigRetrieval) { return prevReg(runtime, parentId, actorId); }); - TActorId clientId; + TActorId clientId; TTestActorRuntimeBase::TEventObserver prev = runtime.SetObserverFunc( [&](TTestActorRuntimeBase& runtime, TAutoPtr<IEventHandle>& ev) { if (auto *msg = ev->CastAsLocal<TEvBlobStorage::TEvControllerNodeServiceSetUpdate>()) { @@ -183,11 +183,11 @@ Y_UNIT_TEST_SUITE(NodeWardenDsProxyConfigRetrieval) { runtime.DispatchEvents(opts); // trigger event - const TActorId recip = MakeBlobStorageProxyID(groupId); - const TActorId sender = runtime.AllocateEdgeActor(0); - const TActorId warden = MakeBlobStorageNodeWardenID(runtime.GetNodeId(0)); + const TActorId recip = MakeBlobStorageProxyID(groupId); + const TActorId sender = runtime.AllocateEdgeActor(0); + const TActorId warden = MakeBlobStorageNodeWardenID(runtime.GetNodeId(0)); Cerr << "=== Breaking pipe ===" << Endl; - runtime.Send(new IEventHandle(TEvents::TSystem::Poison, 0, clientId, TActorId(), {}, 0)); + runtime.Send(new IEventHandle(TEvents::TSystem::Poison, 0, clientId, TActorId(), {}, 0)); Cerr << "=== Sending put ===" << Endl; runtime.Send(new IEventHandle(recip, sender, new TEvBlobStorage::TEvPut(TLogoBlobID(1, 1, 1, 1, 1, 1), "1", TInstant::Max()), IEventHandle::FlagForwardOnNondelivery, 0, &warden), 0, true); diff --git a/ydb/core/blobstorage/other/mon_blob_range_page.cpp b/ydb/core/blobstorage/other/mon_blob_range_page.cpp index 01f5551cb32..f4797d62e24 100644 --- a/ydb/core/blobstorage/other/mon_blob_range_page.cpp +++ b/ydb/core/blobstorage/other/mon_blob_range_page.cpp @@ -1,8 +1,8 @@ #include "mon_blob_range_page.h" #include <ydb/core/base/blobstorage.h> #include <library/cpp/json/writer/json.h> -#include <library/cpp/json/writer/json_value.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/json/writer/json_value.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <library/cpp/threading/future/future.h> namespace NKikimr { diff --git a/ydb/core/blobstorage/other/mon_blob_range_page.h b/ydb/core/blobstorage/other/mon_blob_range_page.h index 5c987b865a8..06b0dc98ef4 100644 --- a/ydb/core/blobstorage/other/mon_blob_range_page.h +++ b/ydb/core/blobstorage/other/mon_blob_range_page.h @@ -2,7 +2,7 @@ #include "defs.h" -#include <library/cpp/monlib/service/pages/mon_page.h> +#include <library/cpp/monlib/service/pages/mon_page.h> namespace NKikimr { diff --git a/ydb/core/blobstorage/other/mon_get_blob_page.cpp b/ydb/core/blobstorage/other/mon_get_blob_page.cpp index 7805f4d6071..95308879a33 100644 --- a/ydb/core/blobstorage/other/mon_get_blob_page.cpp +++ b/ydb/core/blobstorage/other/mon_get_blob_page.cpp @@ -1,6 +1,6 @@ #include "mon_get_blob_page.h" #include <ydb/core/base/blobstorage.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <library/cpp/threading/future/future.h> namespace NKikimr { diff --git a/ydb/core/blobstorage/other/mon_get_blob_page.h b/ydb/core/blobstorage/other/mon_get_blob_page.h index 9c36290cae4..d822023ab78 100644 --- a/ydb/core/blobstorage/other/mon_get_blob_page.h +++ b/ydb/core/blobstorage/other/mon_get_blob_page.h @@ -2,7 +2,7 @@ #include "defs.h" -#include <library/cpp/monlib/service/pages/mon_page.h> +#include <library/cpp/monlib/service/pages/mon_page.h> namespace NKikimr { diff --git a/ydb/core/blobstorage/other/mon_vdisk_stream.cpp b/ydb/core/blobstorage/other/mon_vdisk_stream.cpp index 3825d66891d..90b95b4926e 100644 --- a/ydb/core/blobstorage/other/mon_vdisk_stream.cpp +++ b/ydb/core/blobstorage/other/mon_vdisk_stream.cpp @@ -1,7 +1,7 @@ #include "mon_get_blob_page.h" #include <ydb/core/base/blobstorage.h> #include <ydb/core/blobstorage/base/blobstorage_events.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <library/cpp/threading/future/future.h> namespace NKikimr { diff --git a/ydb/core/blobstorage/other/mon_vdisk_stream.h b/ydb/core/blobstorage/other/mon_vdisk_stream.h index c0aa152e88f..3014fd48f5a 100644 --- a/ydb/core/blobstorage/other/mon_vdisk_stream.h +++ b/ydb/core/blobstorage/other/mon_vdisk_stream.h @@ -2,7 +2,7 @@ #include "defs.h" -#include <library/cpp/monlib/service/pages/mon_page.h> +#include <library/cpp/monlib/service/pages/mon_page.h> namespace NKikimr { diff --git a/ydb/core/blobstorage/other/ya.make b/ydb/core/blobstorage/other/ya.make index 7989d231f8f..650c118dade 100644 --- a/ydb/core/blobstorage/other/ya.make +++ b/ydb/core/blobstorage/other/ya.make @@ -7,7 +7,7 @@ OWNER( PEERDIR( library/cpp/json/writer - library/cpp/monlib/service/pages + library/cpp/monlib/service/pages library/cpp/threading/future ydb/core/base ydb/core/blobstorage/base diff --git a/ydb/core/blobstorage/pdisk/blobstorage_pdisk.h b/ydb/core/blobstorage/pdisk/blobstorage_pdisk.h index 08f384fd186..8e722ce42a0 100644 --- a/ydb/core/blobstorage/pdisk/blobstorage_pdisk.h +++ b/ydb/core/blobstorage/pdisk/blobstorage_pdisk.h @@ -10,7 +10,7 @@ #include <ydb/core/blobstorage/base/bufferwithgaps.h> #include <ydb/core/blobstorage/base/transparent.h> #include <ydb/core/blobstorage/base/batched_vec.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <util/generic/map.h> @@ -134,7 +134,7 @@ struct TEvYardInit : public TEventLocal<TEvYardInit, TEvBlobStorage::EvYardInit> TOwnerRound OwnerRound; TVDiskID VDisk; ui64 PDiskGuid; - TActorId CutLogID; // ask this actor about log cut + TActorId CutLogID; // ask this actor about log cut TActorId WhiteboardProxyId; ui32 SlotId; diff --git a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_actor.cpp b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_actor.cpp index 9288540222c..d92b19c6cca 100644 --- a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_actor.cpp +++ b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_actor.cpp @@ -28,7 +28,7 @@ #include <library/cpp/actors/core/actor_bootstrapped.h> #include <library/cpp/actors/core/hfunc.h> #include <library/cpp/actors/core/mon.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/generic/algorithm.h> #include <util/random/entropy.h> @@ -49,8 +49,8 @@ class TPDiskActor : public TActorBootstrapped<TPDiskActor> { TOwnerRound OwnerRound; TVDiskID VDisk; ui64 PDiskGuid; - TActorId Sender; - TActorId CutLogId; + TActorId Sender; + TActorId CutLogId; TActorId WhiteboardProxyId; ui32 SlotId; @@ -78,8 +78,8 @@ class TPDiskActor : public TActorBootstrapped<TPDiskActor> { bool IsFormattingNow = false; std::function<void()> PendingRestartResponse; - TActorId NodeWhiteboardServiceId; - TActorId NodeWardenServiceId; + TActorId NodeWhiteboardServiceId; + TActorId NodeWardenServiceId; THolder<IEventHandle> ControledStartResult; @@ -340,13 +340,13 @@ public: LOG_WARN_S(*TlsActivationContext, NKikimrServices::BS_PDISK, "PDiskId# " << PDisk->PDiskId << PDisk->ErrorStr); // Is used to pass parameters into formatting thread, because TThread can pass only void* - using TCookieType = std::tuple<TPDiskActor*, TActorSystem*, TActorId>; + using TCookieType = std::tuple<TPDiskActor*, TActorSystem*, TActorId>; FormattingThread.Reset(new TThread( [] (void *cookie) -> void* { auto params = static_cast<TCookieType*>(cookie); TPDiskActor *actor = std::get<0>(*params); TActorSystem *actorSystem = std::get<1>(*params); - TActorId pDiskActor = std::get<2>(*params); + TActorId pDiskActor = std::get<2>(*params); delete params; NPDisk::TKey chunkKey; @@ -1084,9 +1084,9 @@ IActor* CreatePDisk(const TIntrusivePtr<TPDiskConfig> &cfg, const NPDisk::TKey & void TRealPDiskServiceFactory::Create(const TActorContext &ctx, ui32 pDiskID, const TIntrusivePtr<TPDiskConfig> &cfg, const NPDisk::TKey &mainKey, ui32 poolId, ui32 nodeId) { - TActorId actorId = ctx.ExecutorThread.RegisterActor( + TActorId actorId = ctx.ExecutorThread.RegisterActor( CreatePDisk(cfg, mainKey, AppData(ctx)->Counters), TMailboxType::ReadAsFilled, poolId); - TActorId pDiskServiceId = MakeBlobStoragePDiskID(nodeId, pDiskID); + TActorId pDiskServiceId = MakeBlobStoragePDiskID(nodeId, pDiskID); ctx.ExecutorThread.ActorSystem->RegisterLocalService(pDiskServiceId, actorId); } diff --git a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_completion_impl.h b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_completion_impl.h index 3671ec2b854..2baba160c50 100644 --- a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_completion_impl.h +++ b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_completion_impl.h @@ -24,14 +24,14 @@ class TPDisk; class TCompletionEventSender : public TCompletionAction { TPDisk *PDisk; - const TActorId Recipient; + const TActorId Recipient; THolder<IEventBase> Event; NMonitoring::TDynamicCounters::TCounterPtr Counter; public: THolder<TRequestBase> Req; - TCompletionEventSender(TPDisk *pDisk, const TActorId &recipient, IEventBase *event, + TCompletionEventSender(TPDisk *pDisk, const TActorId &recipient, IEventBase *event, NMonitoring::TDynamicCounters::TCounterPtr &counter) : PDisk(pDisk) , Recipient(recipient) @@ -39,7 +39,7 @@ public: , Counter(counter) {} - TCompletionEventSender(TPDisk *pDisk, const TActorId &recipient, IEventBase *event) + TCompletionEventSender(TPDisk *pDisk, const TActorId &recipient, IEventBase *event) : PDisk(pDisk) , Recipient(recipient) , Event(event) @@ -66,7 +66,7 @@ public: }; class TCompletionChunkWrite : public TCompletionAction { - const TActorId Recipient; + const TActorId Recipient; THolder<TEvChunkWriteResult> Event; TPDiskMon *Mon; ui32 PDiskId; diff --git a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_factory.h b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_factory.h index 76dea7670d0..6d472eab412 100644 --- a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_factory.h +++ b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_factory.h @@ -2,7 +2,7 @@ #include "defs.h" #include "blobstorage_pdisk_config.h" #include "blobstorage_pdisk_defs.h" -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> namespace NActors { struct TActorSetupCmd; diff --git a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_free_chunks.h b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_free_chunks.h index a93a86935b0..f5dbe53bc14 100644 --- a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_free_chunks.h +++ b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_free_chunks.h @@ -1,7 +1,7 @@ #pragma once #include "defs.h" -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <util/generic/algorithm.h> #include <util/generic/queue.h> diff --git a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_impl.h b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_impl.h index 3437f32609b..a9b910cb0c1 100644 --- a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_impl.h +++ b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_impl.h @@ -44,7 +44,7 @@ class TCompletionEventSender; class TPDisk : public IPDisk { public: ui32 PDiskId; - TActorId PDiskActor; + TActorId PDiskActor; // Monitoring TPDiskMon Mon; @@ -182,11 +182,11 @@ public: bool IsFormatMagicValid(ui8 *magicData, ui32 magicDataSize); // Called by actor bool CheckGuid(TString *outReason); // Called by actor bool CheckFormatComplete(); // Called by actor - void ReadSysLog(const TActorId &pDiskActor); // Called by actor + void ReadSysLog(const TActorId &pDiskActor); // Called by actor void ProcessChunk0(const TEvReadLogResult &readLogResult); void PrintChunksDebugInfo(); TString ProcessReadSysLogResult(ui64 &outWritePosition, ui64 &outLsn, const TEvReadLogResult &readLogResult); - void ReadAndParseMainLog(const TActorId &pDiskActor); + void ReadAndParseMainLog(const TActorId &pDiskActor); // Called by the log reader on success with the current chunkOwnerMap. void ProcessChunkOwnerMap(TMap<ui32, TChunkState> &chunkOwnerMap); void InitLogChunksInfo(); diff --git a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_internal_interface.h b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_internal_interface.h index ea07a4c3eb3..df2f0885694 100644 --- a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_internal_interface.h +++ b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_internal_interface.h @@ -38,9 +38,9 @@ struct TEvWhiteboardReportResult : struct TEvHttpInfoResult : public TEventLocal<TEvHttpInfoResult, TEvBlobStorage::EvHttpInfoResult> { TAutoPtr<NMon::TEvHttpInfoRes> HttpInfoRes; - const TActorId EndCustomer; + const TActorId EndCustomer; - TEvHttpInfoResult(const TActorId &endCustomer) + TEvHttpInfoResult(const TActorId &endCustomer) : HttpInfoRes(nullptr) , EndCustomer(endCustomer) {} diff --git a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_logreader.cpp b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_logreader.cpp index 6c3c98a5da0..caf4b581fa3 100644 --- a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_logreader.cpp +++ b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_logreader.cpp @@ -283,7 +283,7 @@ public: } }; -TLogReader::TLogReader(bool isInitial,TPDisk *pDisk, TActorSystem * const actorSystem, const TActorId &replyTo, NPDisk::TOwner owner, +TLogReader::TLogReader(bool isInitial,TPDisk *pDisk, TActorSystem * const actorSystem, const TActorId &replyTo, NPDisk::TOwner owner, TLogPosition ownerLogStartPosition, EOwnerGroupType ownerGroupType, TLogPosition position, ui64 sizeLimit, ui64 lastNonce, ui32 logEndChunkIdx, ui64 logEndSectorIdx, TReqId reqId, TVector<TChunkIdx> &&chunksToRead, ui64 firstLsnToKeep, ui64 firstNonceToKeep, TVDiskID ownerVDiskId) diff --git a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_logreader.h b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_logreader.h index 607ffaf091a..a95ff19be13 100644 --- a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_logreader.h +++ b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_logreader.h @@ -49,7 +49,7 @@ class TLogReader : public TLogReaderBase { bool IsInitial; TPDisk * const PDisk; TActorSystem * const ActorSystem; - const TActorId ReplyTo; + const TActorId ReplyTo; TOwner Owner; TLogPosition OwnerLogStartPosition; diff --git a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_params.cpp b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_params.cpp index 22caf6e536e..e762b4e7038 100644 --- a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_params.cpp +++ b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_params.cpp @@ -1,5 +1,5 @@ #include "blobstorage_pdisk_params.h" -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> namespace NKikimr { diff --git a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_req_creator.h b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_req_creator.h index 34f5113ef0d..af5402a4f81 100644 --- a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_req_creator.h +++ b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_req_creator.h @@ -207,7 +207,7 @@ public: return CreateFromArgs<TChunkTrim>(chunkIdx, offset, size); } - [[nodiscard]] TLogWrite* CreateLogWrite(NPDisk::TEvLog &ev, const TActorId &sender, double& burstMs, NWilson::TTraceId traceId) { + [[nodiscard]] TLogWrite* CreateLogWrite(NPDisk::TEvLog &ev, const TActorId &sender, double& burstMs, NWilson::TTraceId traceId) { TReqId reqId(TReqId::LogWrite, AtomicIncrement(LastReqId)); LOG_DEBUG(*ActorSystem, NKikimrServices::BS_PDISK, "PDiskId# %" PRIu32 " %s Sender# %" PRIu64 " ReqId# %" PRIu64, (ui32)PDiskId, ev.ToString().c_str(), (ui64)sender.LocalId(), (ui64)reqId.Id); @@ -221,7 +221,7 @@ public: return NewRequest(new TLogWrite(ev, sender, AtomicGet(*EstimatedLogChunkIdx), reqId, std::move(traceId)), &burstMs); } - [[nodiscard]] TChunkRead* CreateChunkRead(const NPDisk::TEvChunkRead &ev, const TActorId &sender, double& burstMs, + [[nodiscard]] TChunkRead* CreateChunkRead(const NPDisk::TEvChunkRead &ev, const TActorId &sender, double& burstMs, NWilson::TTraceId traceId) { TReqId reqId(TReqId::ChunkRead, AtomicIncrement(LastReqId)); LOG_DEBUG(*ActorSystem, NKikimrServices::BS_PDISK, "PDiskId# %" PRIu32 " %s Sender# %" PRIu64 " ReqId# %" PRIu64, @@ -235,7 +235,7 @@ public: return NewRequest(read, &burstMs); } - [[nodiscard]] TChunkWrite* CreateChunkWrite(const NPDisk::TEvChunkWrite &ev, const TActorId &sender, double& burstMs, + [[nodiscard]] TChunkWrite* CreateChunkWrite(const NPDisk::TEvChunkWrite &ev, const TActorId &sender, double& burstMs, NWilson::TTraceId traceId) { TReqId reqId(TReqId::ChunkWrite, AtomicIncrement(LastReqId)); LOG_DEBUG(*ActorSystem, NKikimrServices::BS_PDISK, "PDiskId# %" PRIu32 " %s Sender# %" PRIu64 " ReqId# %" PRIu64, diff --git a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_requestimpl.h b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_requestimpl.h index 7ae70e62963..61c0996e081 100644 --- a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_requestimpl.h +++ b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_requestimpl.h @@ -30,7 +30,7 @@ enum class EOwnerGroupType { class TRequestBase : public TThrRefBase { public: // Identification - const TActorId Sender; + const TActorId Sender; const TReqId ReqId; TOwner Owner; TOwnerRound OwnerRound; @@ -57,7 +57,7 @@ public: mutable NWilson::TTraceId TraceId; mutable NLWTrace::TOrbit Orbit; public: - TRequestBase(const TActorId &sender, TReqId reqId, TOwner owner, TOwnerRound ownerRound, ui8 priorityClass, + TRequestBase(const TActorId &sender, TReqId reqId, TOwner owner, TOwnerRound ownerRound, ui8 priorityClass, NWilson::TTraceId traceId = {}) : Sender(sender) , ReqId(reqId) @@ -119,11 +119,11 @@ class TYardInit : public TRequestBase { public: TVDiskID VDisk; ui64 PDiskGuid; - TActorId CutLogId; + TActorId CutLogId; TActorId WhiteboardProxyId; ui32 SlotId; - TYardInit(const NPDisk::TEvYardInit &ev, const TActorId &sender, TAtomicBase reqIdx) + TYardInit(const NPDisk::TEvYardInit &ev, const TActorId &sender, TAtomicBase reqIdx) : TRequestBase(sender, TReqId(TReqId::YardInit, reqIdx), 0, ev.OwnerRound, NPriInternal::Other) , VDisk(ev.VDisk) , PDiskGuid(ev.PDiskGuid) @@ -148,7 +148,7 @@ public: // class TCheckSpace : public TRequestBase { public: - TCheckSpace(const NPDisk::TEvCheckSpace &ev, const TActorId &sender, TAtomicBase reqIdx) + TCheckSpace(const NPDisk::TEvCheckSpace &ev, const TActorId &sender, TAtomicBase reqIdx) : TRequestBase(sender, TReqId(TReqId::CheckSpace, reqIdx), ev.Owner, ev.OwnerRound, NPriInternal::Other) { Y_UNUSED(ev); @@ -167,7 +167,7 @@ public: TLogPosition Position; ui64 SizeLimit; - TLogRead(const NPDisk::TEvReadLog &ev, const TActorId &sender, TAtomicBase reqIdx) + TLogRead(const NPDisk::TEvReadLog &ev, const TActorId &sender, TAtomicBase reqIdx) : TRequestBase(sender, TReqId(TReqId::LogRead, reqIdx), ev.Owner, ev.OwnerRound, NPriInternal::LogRead) , Position(ev.Position) , SizeLimit(ev.SizeLimit) @@ -189,7 +189,7 @@ public: TCompletionAction *CompletionAction; TReqId ReqId; - TLogReadContinue(const NPDisk::TEvReadLogContinue &ev, const TActorId &sender, TAtomicBase /*reqIdx*/) + TLogReadContinue(const NPDisk::TEvReadLogContinue &ev, const TActorId &sender, TAtomicBase /*reqIdx*/) : TRequestBase(sender, ev.ReqId, 0, 0, NPriInternal::LogRead) , Data(ev.Data) , Size(ev.Size) @@ -210,7 +210,7 @@ class TLogReadResultProcess : public TRequestBase { public: NPDisk::TEvReadLogResult::TPtr ReadLogResult; - TLogReadResultProcess(NPDisk::TEvReadLogResult::TPtr &ev, const TActorId &sender, TAtomicBase reqIdx) + TLogReadResultProcess(NPDisk::TEvReadLogResult::TPtr &ev, const TActorId &sender, TAtomicBase reqIdx) : TRequestBase(sender, TReqId(TReqId::LogReadResultProcess, reqIdx), 0, 0, NPriInternal::LogRead) , ReadLogResult(std::move(ev)) @@ -231,7 +231,7 @@ public: ui64 Offset; TCompletionAction *CompletionAction; - TLogSectorRestore(const NPDisk::TEvLogSectorRestore &ev, const TActorId &sender, TAtomicBase reqIdx) + TLogSectorRestore(const NPDisk::TEvLogSectorRestore &ev, const TActorId &sender, TAtomicBase reqIdx) : TRequestBase(sender, TReqId(TReqId::LogSectorRestore, reqIdx), 0, 0, NPriInternal::LogRead) , Data(ev.Data) , Size(ev.Size) @@ -264,7 +264,7 @@ public: THolder<NPDisk::TEvLogResult> Result; std::function<void()> OnDestroy; - TLogWrite(NPDisk::TEvLog &ev, const TActorId &sender, ui32 estimatedChunkIdx, TReqId reqId, NWilson::TTraceId traceId) + TLogWrite(NPDisk::TEvLog &ev, const TActorId &sender, ui32 estimatedChunkIdx, TReqId reqId, NWilson::TTraceId traceId) : TRequestBase(sender, reqId, ev.Owner, ev.OwnerRound, NPriInternal::LogWrite, std::move(traceId)) , Signature(ev.Signature) , EstimatedChunkIdx(estimatedChunkIdx) @@ -344,7 +344,7 @@ public: const ui64 DoubleFreeCanary; - TChunkRead(const NPDisk::TEvChunkRead &ev, const TActorId &sender, TReqId reqId, NWilson::TTraceId traceId) + TChunkRead(const NPDisk::TEvChunkRead &ev, const TActorId &sender, TReqId reqId, NWilson::TTraceId traceId) : TRequestBase(sender, reqId, ev.Owner, ev.OwnerRound, ev.PriorityClass, std::move(traceId)) , ChunkIdx(ev.ChunkIdx) , Offset(ev.Offset) @@ -457,7 +457,7 @@ public: THolder<NPDisk::TCompletionAction> Completion; - TChunkWrite(const NPDisk::TEvChunkWrite &ev, const TActorId &sender, TReqId reqId, NWilson::TTraceId traceId) + TChunkWrite(const NPDisk::TEvChunkWrite &ev, const TActorId &sender, TReqId reqId, NWilson::TTraceId traceId) : TRequestBase(sender, reqId, ev.Owner, ev.OwnerRound, ev.PriorityClass, std::move(traceId)) , ChunkIdx(ev.ChunkIdx) , Offset(ev.Offset) @@ -579,7 +579,7 @@ public: // class THarakiri : public TRequestBase { public: - THarakiri(const NPDisk::TEvHarakiri &ev, const TActorId &sender, TAtomicBase reqIdx) + THarakiri(const NPDisk::TEvHarakiri &ev, const TActorId &sender, TAtomicBase reqIdx) : TRequestBase(sender, TReqId(TReqId::Harakiri, reqIdx), ev.Owner, ev.OwnerRound, NPriInternal::Other) {} @@ -597,7 +597,7 @@ public: TOwnerRound SlayOwnerRound; ui32 PDiskId; ui32 VSlotId; - TSlay(const NPDisk::TEvSlay &ev, const TActorId &sender, TAtomicBase reqIdx) + TSlay(const NPDisk::TEvSlay &ev, const TActorId &sender, TAtomicBase reqIdx) : TRequestBase(sender, TReqId(TReqId::Slay, reqIdx), OwnerUnallocated, ev.SlayOwnerRound, NPriInternal::Other) , VDiskId(ev.VDiskId) , SlayOwnerRound(ev.SlayOwnerRound) @@ -620,7 +620,7 @@ public: ui32 End; ui32 Count; - TChunksLock(const NPDisk::TEvChunksLock &ev, const TActorId &sender, TAtomicBase reqIdx) + TChunksLock(const NPDisk::TEvChunksLock &ev, const TActorId &sender, TAtomicBase reqIdx) : TRequestBase(sender, TReqId(TReqId::ChunksLock, reqIdx), 0, 0, NPriInternal::Other) , LockByRange(ev.LockByRange) , Begin(ev.Begin) @@ -638,7 +638,7 @@ public: // class TChunksUnlock : public TRequestBase { public: - TChunksUnlock(const NPDisk::TEvChunksUnlock &ev, const TActorId &sender, TAtomicBase reqIdx) + TChunksUnlock(const NPDisk::TEvChunksUnlock &ev, const TActorId &sender, TAtomicBase reqIdx) : TRequestBase(sender, TReqId(TReqId::ChunksUnlock, reqIdx), 0, 0, NPriInternal::Other) { Y_UNUSED(ev); @@ -656,7 +656,7 @@ class TChunkReserve : public TRequestBase { public: ui32 SizeChunks; - TChunkReserve(const NPDisk::TEvChunkReserve &ev, const TActorId &sender, TAtomicBase reqIdx) + TChunkReserve(const NPDisk::TEvChunkReserve &ev, const TActorId &sender, TAtomicBase reqIdx) : TRequestBase(sender, TReqId(TReqId::ChunkReserve, reqIdx), ev.Owner, ev.OwnerRound, NPriInternal::Other) , SizeChunks(ev.SizeChunks) {} @@ -688,7 +688,7 @@ public: // class THttpInfo : public TRequestBase { public: - const TActorId EndCustomer; + const TActorId EndCustomer; TStringStream OutputString; TString DeviceFlagStr; TString RealtimeFlagStr; @@ -732,7 +732,7 @@ class TUndelivered : public TRequestBase { public: TAutoPtr<TEvents::TEvUndelivered> Event; - TUndelivered(TEvents::TEvUndelivered::TPtr ev, const TActorId &sender, TAtomicBase reqIdx) + TUndelivered(TEvents::TEvUndelivered::TPtr ev, const TActorId &sender, TAtomicBase reqIdx) : TRequestBase(sender, TReqId(TReqId::Undelivered, reqIdx), 0u, 0u, NPriInternal::Other) , Event(ev->Release()) {} @@ -750,7 +750,7 @@ public: ui32 Action; void *Cookie; - TYardControl(const NPDisk::TEvYardControl &ev, const TActorId &sender, TAtomicBase reqIdx) + TYardControl(const NPDisk::TEvYardControl &ev, const TActorId &sender, TAtomicBase reqIdx) : TRequestBase(sender, TReqId(TReqId::YardControl, reqIdx), 0, 0, NPriInternal::Other) , Action(ev.Action) , Cookie(ev.Cookie) @@ -766,7 +766,7 @@ public: // class TAskForCutLog : public TRequestBase { public: - TAskForCutLog(const NPDisk::TEvAskForCutLog &ev, const TActorId &sender, TAtomicBase reqIdx) + TAskForCutLog(const NPDisk::TEvAskForCutLog &ev, const TActorId &sender, TAtomicBase reqIdx) : TRequestBase(sender, TReqId(TReqId::AskForCutLog, reqIdx), ev.Owner, ev.OwnerRound, NPriInternal::Other) {} @@ -785,7 +785,7 @@ public: TPDiskSchedulerConfig SchedulerCfg; - TConfigureScheduler(const NPDisk::TEvConfigureScheduler &ev, const TActorId &sender, TAtomicBase reqIdx) + TConfigureScheduler(const NPDisk::TEvConfigureScheduler &ev, const TActorId &sender, TAtomicBase reqIdx) : TRequestBase(sender, TReqId(TReqId::ConfigureScheduler, reqIdx), 0, 0, NPriInternal::Other) , OwnerId(ev.Owner) , OwnerRound(ev.OwnerRound) @@ -793,7 +793,7 @@ public: {} TConfigureScheduler(TOwner ownerId, TOwnerRound ownerRound) - : TRequestBase(TActorId(), TReqId(TReqId::InnerConfigureScheduler, 0), 0, 0, NPriInternal::Other) + : TRequestBase(TActorId(), TReqId(TReqId::InnerConfigureScheduler, 0), 0, 0, NPriInternal::Other) , OwnerId(ownerId) , OwnerRound(ownerRound) {} @@ -812,7 +812,7 @@ public: TVector<ui32> CommitedLogChunks; TCommitLogChunks(TVector<ui32>&& commitedLogChunks, TAtomicBase reqIdx) - : TRequestBase(TActorId(), TReqId(TReqId::CommitLogChunks, reqIdx), OwnerSystem, 0, NPriInternal::Other) + : TRequestBase(TActorId(), TReqId(TReqId::CommitLogChunks, reqIdx), OwnerSystem, 0, NPriInternal::Other) , CommitedLogChunks(std::move(commitedLogChunks)) {} @@ -833,7 +833,7 @@ public: TReleaseChunks(const TLogChunkInfo& gapStart, const TLogChunkInfo& gapEnd, TVector<TChunkIdx> chunksToRelease, TAtomicBase reqIdx) - : TRequestBase(TActorId(), TReqId(TReqId::ReleaseChunks, reqIdx), OwnerSystem, 0, NPriInternal::Other) + : TRequestBase(TActorId(), TReqId(TReqId::ReleaseChunks, reqIdx), OwnerSystem, 0, NPriInternal::Other) , GapStart(gapStart) , GapEnd(gapEnd) , ChunksToRelease(std::move(chunksToRelease)) @@ -841,7 +841,7 @@ public: {} TReleaseChunks(TVector<TChunkIdx> chunksToRelease, TAtomicBase reqIdx) - : TRequestBase(TActorId(), TReqId(TReqId::ReleaseChunks, reqIdx), OwnerSystem, 0, NPriInternal::Other) + : TRequestBase(TActorId(), TReqId(TReqId::ReleaseChunks, reqIdx), OwnerSystem, 0, NPriInternal::Other) , ChunksToRelease(std::move(chunksToRelease)) , IsChunksFromLogSplice(false) {} diff --git a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_sectorrestorator.cpp b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_sectorrestorator.cpp index bac38caf08a..11e11261343 100644 --- a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_sectorrestorator.cpp +++ b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_sectorrestorator.cpp @@ -7,7 +7,7 @@ namespace NPDisk { TSectorRestorator::TSectorRestorator(const bool isTrippleCopy, const ui32 erasureDataParts, const bool isErasureEncode, const TDiskFormat &format, - TActorSystem *actorSystem, const TActorId pDiskActorId, ui32 pDiskId, TPDiskMon *mon, + TActorSystem *actorSystem, const TActorId pDiskActorId, ui32 pDiskId, TPDiskMon *mon, TBufferPool *bufferPool) : IsTrippleCopy(isTrippleCopy) , ErasureDataParts(erasureDataParts) diff --git a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_sectorrestorator.h b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_sectorrestorator.h index 2e69b49b64b..e0936e8bc30 100644 --- a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_sectorrestorator.h +++ b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_sectorrestorator.h @@ -19,7 +19,7 @@ struct TSectorRestorator { ui32 RestoredSectorFlags; const TDiskFormat &Format; TActorSystem *ActorSystem; - const TActorId PDiskActorId; + const TActorId PDiskActorId; ui32 PDiskId; bool IsErasureEncode; TPDiskMon *Mon; @@ -27,7 +27,7 @@ struct TSectorRestorator { TSectorRestorator(const bool isTrippleCopy, const ui32 erasureDataParts, const bool isErasureEncode, const TDiskFormat &format, - TActorSystem *actorSystem, const TActorId pDiskActorId, ui32 pDiskId, TPDiskMon *mon, + TActorSystem *actorSystem, const TActorId pDiskActorId, ui32 pDiskId, TPDiskMon *mon, TBufferPool *bufferPool); TSectorRestorator(const bool isTrippleCopy, const ui32 erasureDataParts, diff --git a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_syslogreader.h b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_syslogreader.h index f8f90479279..909ff8aee6a 100644 --- a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_syslogreader.h +++ b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_syslogreader.h @@ -88,7 +88,7 @@ class TPDisk; class TSysLogReader : public TThrRefBase { TPDisk *const PDisk; TActorSystem *const ActorSystem; - const TActorId ReplyTo; + const TActorId ReplyTo; const TReqId ReqId; THolder<TEvReadLogResult> Result; diff --git a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_tools.cpp b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_tools.cpp index c2d9c43f632..152deec9997 100644 --- a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_tools.cpp +++ b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_tools.cpp @@ -26,7 +26,7 @@ #include <library/cpp/actors/core/hfunc.h> #include <library/cpp/actors/core/mon.h> #include <library/cpp/actors/core/scheduler_basic.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/generic/algorithm.h> #include <util/generic/queue.h> diff --git a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_ut_base_test.h b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_ut_base_test.h index b400d5448b8..e6c75e60003 100644 --- a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_ut_base_test.h +++ b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_ut_base_test.h @@ -48,7 +48,7 @@ public: } protected: - const TActorId Yard; + const TActorId Yard; const TVDiskID VDiskID; int TestStep; diff --git a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_ut_config.h b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_ut_config.h index 27ae7a68bd2..e33fa94a2db 100644 --- a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_ut_config.h +++ b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_ut_config.h @@ -7,9 +7,9 @@ namespace NKikimr { struct TTestConfig : public TThrRefBase { TVDiskID VDiskID; - TActorId YardActorID; + TActorId YardActorID; - TTestConfig(const TVDiskID &vDiskID, const TActorId &yardActorID) + TTestConfig(const TVDiskID &vDiskID, const TActorId &yardActorID) : VDiskID(vDiskID) , YardActorID(yardActorID) {} diff --git a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_ut_run.cpp b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_ut_run.cpp index 168e118f87b..5935d13229e 100644 --- a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_ut_run.cpp +++ b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_ut_run.cpp @@ -17,8 +17,8 @@ namespace NPDisk { void Run(TVector<IActor*> tests, TTestRunConfig runCfg) { TTempDir tempDir; - TVector<TActorId> testIds; - TActorId pDiskId; + TVector<TActorId> testIds; + TActorId pDiskId; TAppData appData(0, 0, 0, 0, TMap<TString, ui32>(), nullptr, nullptr, nullptr, nullptr); auto ioContext = std::make_shared<NPDisk::TIoContextFactoryOSS>(); appData.IoContextFactory = ioContext.get(); @@ -55,9 +55,9 @@ void Run(TVector<IActor*> tests, TTestRunConfig runCfg) { setup1->Executors[2].Reset(new TIOExecutorPool(2, 10)); setup1->Scheduler.Reset(new TBasicSchedulerThread(TSchedulerConfig(512, 100))); - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); TActorSetupCmd nameserviceSetup(CreateNameserverTable(nameserverTable), TMailboxType::Simple, 0); - setup1->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(nameserviceId, nameserviceSetup)); + setup1->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(nameserviceId, nameserviceSetup)); TString dataPath; if (!runCfg.TestContext->IsFormatedDiskExpected()) { @@ -88,12 +88,12 @@ void Run(TVector<IActor*> tests, TTestRunConfig runCfg) { TActorSetupCmd pDiskSetup(CreatePDisk(pDiskConfig.Get(), NPDisk::YdbDefaultPDiskSequence, mainCounters), TMailboxType::Revolving, 0); - setup1->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(pDiskId, pDiskSetup)); + setup1->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(pDiskId, pDiskSetup)); for (ui32 i = 0; i < runCfg.Instances; ++i) { testIds[i] = MakeBlobStorageProxyID(1 + i); TActorSetupCmd testSetup(tests[i], TMailboxType::Revolving, 0); - setup1->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(testIds[i], testSetup)); + setup1->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(testIds[i], testSetup)); } AtomicSet(doneCounter, 0); @@ -103,7 +103,7 @@ void Run(TVector<IActor*> tests, TTestRunConfig runCfg) { /////////////////////// LOGGER /////////////////////////////////////////////// GetServiceCounters(counters, "utils"); - NActors::TActorId loggerActorId = NActors::TActorId(1, "logger"); + NActors::TActorId loggerActorId = NActors::TActorId(1, "logger"); TIntrusivePtr<NActors::NLog::TSettings> logSettings( new NActors::NLog::TSettings(loggerActorId, NKikimrServices::LOGGER, NActors::NLog::PRI_ERROR, NActors::NLog::PRI_ERROR, 0)); @@ -130,7 +130,7 @@ void Run(TVector<IActor*> tests, TTestRunConfig runCfg) { IsLowVerbose ? NActors::CreateStderrBackend() : NActors::CreateNullBackend(), GetServiceCounters(counters, "utils")); NActors::TActorSetupCmd loggerActorCmd(loggerActor, NActors::TMailboxType::Simple, 2); - std::pair<NActors::TActorId, NActors::TActorSetupCmd> loggerActorPair(loggerActorId, loggerActorCmd); + std::pair<NActors::TActorId, NActors::TActorSetupCmd> loggerActorPair(loggerActorId, loggerActorCmd); setup1->LocalServices.push_back(loggerActorPair); ////////////////////////////////////////////////////////////////////////////// @@ -154,7 +154,7 @@ void Run(TVector<IActor*> tests, TTestRunConfig runCfg) { VERBOSE_COUT("Sending TEvBoot to test"); for (ui32 i = 0; i < runCfg.Instances; ++i) { actorSystem1->Send(testIds[i], new TEvTablet::TEvBoot( - MakeTabletID(0, 0, 1), 0, nullptr, TActorId(), nullptr)); + MakeTabletID(0, 0, 1), 0, nullptr, TActorId(), nullptr)); } TAtomicBase doneCount = 0; diff --git a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_ut_run.h b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_ut_run.h index 51298c218a3..28418e3f58a 100644 --- a/ydb/core/blobstorage/pdisk/blobstorage_pdisk_ut_run.h +++ b/ydb/core/blobstorage/pdisk/blobstorage_pdisk_ut_run.h @@ -29,7 +29,7 @@ void Run(TVector<IActor*> tests, TTestRunConfig runCfg); template <class T> void Run(TTestRunConfig runCfg) { - const TActorId pDiskId = MakeBlobStoragePDiskID(1, 1); + const TActorId pDiskId = MakeBlobStoragePDiskID(1, 1); TVector<IActor*> tests; for (ui32 i = 0; i < runCfg.Instances; ++i) { TIntrusivePtr<TTestConfig> testConfig = new TTestConfig(TVDiskID(0, 1, 0, 0, i), pDiskId); diff --git a/ydb/core/blobstorage/pdisk/ya.make b/ydb/core/blobstorage/pdisk/ya.make index e9d6a3cba55..464191b8c04 100644 --- a/ydb/core/blobstorage/pdisk/ya.make +++ b/ydb/core/blobstorage/pdisk/ya.make @@ -13,8 +13,8 @@ PEERDIR( library/cpp/actors/util library/cpp/containers/stack_vector library/cpp/lwtrace - library/cpp/monlib/dynamic_counters/percentile - library/cpp/monlib/service/pages + library/cpp/monlib/dynamic_counters/percentile + library/cpp/monlib/service/pages util ydb/core/base ydb/core/base/services diff --git a/ydb/core/blobstorage/storagepoolmon/ya.make b/ydb/core/blobstorage/storagepoolmon/ya.make index b3f3bbf0fd9..47e3f08dee1 100644 --- a/ydb/core/blobstorage/storagepoolmon/ya.make +++ b/ydb/core/blobstorage/storagepoolmon/ya.make @@ -11,7 +11,7 @@ SRCS( ) PEERDIR( - library/cpp/monlib/dynamic_counters + library/cpp/monlib/dynamic_counters ydb/core/base ydb/core/blobstorage/base ) diff --git a/ydb/core/blobstorage/testload/test_load_actor.cpp b/ydb/core/blobstorage/testload/test_load_actor.cpp index a66efb63530..c8788b13104 100644 --- a/ydb/core/blobstorage/testload/test_load_actor.cpp +++ b/ydb/core/blobstorage/testload/test_load_actor.cpp @@ -6,7 +6,7 @@ #include <ydb/public/lib/base/msgbus.h> #include <google/protobuf/text_format.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> namespace NKikimr { @@ -21,9 +21,9 @@ class TLoadActor : public TActorBootstrapped<TLoadActor> { // per-request info struct THttpInfoRequest { - TActorId Origin; // who asked for status + TActorId Origin; // who asked for status int SubRequestId; // origin subrequest id - THashMap<TActorId, TActorInfo> ActorMap; // per-actor status + THashMap<TActorId, TActorInfo> ActorMap; // per-actor status ui32 HttpInfoResPending; // number of requests pending TString ErrorMessage; }; @@ -38,7 +38,7 @@ class TLoadActor : public TActorBootstrapped<TLoadActor> { TVector<TFinishedTestInfo> FinishedTests; // currently running load actors - TMap<ui64, TActorId> LoadActors; + TMap<ui64, TActorId> LoadActors; // next HTTP request identifier ui32 NextRequestId; diff --git a/ydb/core/blobstorage/testload/test_load_actor.h b/ydb/core/blobstorage/testload/test_load_actor.h index d081bfc0a5f..38b7fb3d305 100644 --- a/ydb/core/blobstorage/testload/test_load_actor.h +++ b/ydb/core/blobstorage/testload/test_load_actor.h @@ -2,7 +2,7 @@ #include "defs.h" #include <ydb/core/base/blobstorage.h> -#include <library/cpp/monlib/dynamic_counters/percentile/percentile_lg.h> +#include <library/cpp/monlib/dynamic_counters/percentile/percentile_lg.h> #include <cmath> namespace NKikimr { @@ -31,22 +31,22 @@ namespace NKikimr { NActors::IActor *CreateTestLoadActor(const TIntrusivePtr<NMonitoring::TDynamicCounters>& counters); NActors::IActor *CreateWriterTestLoad(const NKikimrBlobStorage::TEvTestLoadRequest::TLoadStart& cmd, - const NActors::TActorId& parent, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, ui64 tag); + const NActors::TActorId& parent, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, ui64 tag); NActors::IActor *CreatePDiskWriterTestLoad(const NKikimrBlobStorage::TEvTestLoadRequest::TPDiskLoadStart& cmd, - const NActors::TActorId& parent, const TIntrusivePtr<NMonitoring::TDynamicCounters>& counters, + const NActors::TActorId& parent, const TIntrusivePtr<NMonitoring::TDynamicCounters>& counters, ui64 index, ui64 tag); NActors::IActor *CreatePDiskLogWriterTestLoad(const NKikimrBlobStorage::TEvTestLoadRequest::TPDiskLogLoadStart& cmd, - const NActors::TActorId& parent, const TIntrusivePtr<NMonitoring::TDynamicCounters>& counters, + const NActors::TActorId& parent, const TIntrusivePtr<NMonitoring::TDynamicCounters>& counters, ui64 index, ui64 tag); NActors::IActor *CreatePDiskReaderTestLoad(const NKikimrBlobStorage::TEvTestLoadRequest::TPDiskReadLoadStart& cmd, - const NActors::TActorId& parent, const TIntrusivePtr<NMonitoring::TDynamicCounters>& counters, + const NActors::TActorId& parent, const TIntrusivePtr<NMonitoring::TDynamicCounters>& counters, ui64 index, ui64 tag); NActors::IActor *CreateVDiskWriterTestLoad(const NKikimrBlobStorage::TEvTestLoadRequest::TVDiskLoadStart& cmd, - const NActors::TActorId& parent, ui64 tag); + const NActors::TActorId& parent, ui64 tag); NActors::IActor *CreateKeyValueWriterTestLoad(const NKikimrBlobStorage::TEvTestLoadRequest::TKeyValueLoadStart& cmd, const NActors::TActorId& parent, const TIntrusivePtr<NMonitoring::TDynamicCounters>& counters, diff --git a/ydb/core/blobstorage/testload/test_load_pdisk_log.cpp b/ydb/core/blobstorage/testload/test_load_pdisk_log.cpp index e40a5b7c111..33fc363c935 100644 --- a/ydb/core/blobstorage/testload/test_load_pdisk_log.cpp +++ b/ydb/core/blobstorage/testload/test_load_pdisk_log.cpp @@ -3,7 +3,7 @@ #include <ydb/core/base/counters.h> #include <ydb/core/blobstorage/pdisk/blobstorage_pdisk.h> #include <ydb/core/blobstorage/base/blobstorage_events.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/random/fast.h> #include <util/generic/queue.h> @@ -277,7 +277,7 @@ class TPDiskLogWriterTestLoadActor : public TActorBootstrapped<TPDiskLogWriterTe ui64 WrittenBytes = 0; - const TActorId Parent; + const TActorId Parent; ui64 Tag; ui32 DurationSeconds; i32 OwnerInitInProgress = 0; @@ -306,7 +306,7 @@ public: } TPDiskLogWriterTestLoadActor(const NKikimrBlobStorage::TEvTestLoadRequest::TPDiskLogLoadStart& cmd, - const TActorId& parent, const TIntrusivePtr<NMonitoring::TDynamicCounters>& counters, ui64 index, ui64 tag) + const TActorId& parent, const TIntrusivePtr<NMonitoring::TDynamicCounters>& counters, ui64 index, ui64 tag) : Parent(parent) , Tag(tag) , Rng(Now().GetValue()) @@ -694,7 +694,7 @@ public: }; IActor *CreatePDiskLogWriterTestLoad(const NKikimrBlobStorage::TEvTestLoadRequest::TPDiskLogLoadStart& cmd, - const TActorId& parent, const TIntrusivePtr<NMonitoring::TDynamicCounters>& counters, ui64 index, ui64 tag) { + const TActorId& parent, const TIntrusivePtr<NMonitoring::TDynamicCounters>& counters, ui64 index, ui64 tag) { return new TPDiskLogWriterTestLoadActor(cmd, parent, counters, index, tag); } diff --git a/ydb/core/blobstorage/testload/test_load_pdisk_read.cpp b/ydb/core/blobstorage/testload/test_load_pdisk_read.cpp index 6a2fa28e4f6..3b446a74c2d 100644 --- a/ydb/core/blobstorage/testload/test_load_pdisk_read.cpp +++ b/ydb/core/blobstorage/testload/test_load_pdisk_read.cpp @@ -5,7 +5,7 @@ #include <ydb/core/control/immediate_control_board_impl.h> #include <ydb/core/blobstorage/pdisk/blobstorage_pdisk.h> #include <ydb/core/blobstorage/base/blobstorage_events.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/random/fast.h> #include <util/generic/queue.h> @@ -69,7 +69,7 @@ class TPDiskReaderTestLoadActor : public TActorBootstrapped<TPDiskReaderTestLoad THashMap<ui64, TRequestInfo> RequestInfo; ui64 NextRequestIdx = 0; - const TActorId Parent; + const TActorId Parent; ui64 Tag; ui32 DurationSeconds; ui32 IntervalMsMin = 0; @@ -121,7 +121,7 @@ public: return NKikimrServices::TActivity::BS_LOAD_PDISK_READ; } - TPDiskReaderTestLoadActor(const NKikimrBlobStorage::TEvTestLoadRequest::TPDiskReadLoadStart& cmd, const TActorId& parent, + TPDiskReaderTestLoadActor(const NKikimrBlobStorage::TEvTestLoadRequest::TPDiskReadLoadStart& cmd, const TActorId& parent, const TIntrusivePtr<NMonitoring::TDynamicCounters>& counters, ui64 index, ui64 tag) : Parent(parent) , Tag(tag) @@ -585,7 +585,7 @@ public: }; IActor *CreatePDiskReaderTestLoad(const NKikimrBlobStorage::TEvTestLoadRequest::TPDiskReadLoadStart& cmd, - const TActorId& parent, const TIntrusivePtr<NMonitoring::TDynamicCounters>& counters, + const TActorId& parent, const TIntrusivePtr<NMonitoring::TDynamicCounters>& counters, ui64 index, ui64 tag) { return new TPDiskReaderTestLoadActor(cmd, parent, counters, index, tag); } diff --git a/ydb/core/blobstorage/testload/test_load_pdisk_write.cpp b/ydb/core/blobstorage/testload/test_load_pdisk_write.cpp index 3aff7355c00..8ea16e014cb 100644 --- a/ydb/core/blobstorage/testload/test_load_pdisk_write.cpp +++ b/ydb/core/blobstorage/testload/test_load_pdisk_write.cpp @@ -3,7 +3,7 @@ #include <ydb/core/base/counters.h> #include <ydb/core/blobstorage/pdisk/blobstorage_pdisk.h> #include <ydb/core/blobstorage/base/blobstorage_events.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/random/fast.h> #include <util/generic/queue.h> @@ -75,7 +75,7 @@ class TPDiskWriterTestLoadActor : public TActorBootstrapped<TPDiskWriterTestLoad THashMap<ui64, TRequestInfo> RequestInfo; ui64 NextRequestIdx = 0; - const TActorId Parent; + const TActorId Parent; ui64 Tag; ui32 DurationSeconds; ui32 IntervalMsMin = 0; @@ -131,7 +131,7 @@ public: return NKikimrServices::TActivity::BS_LOAD_PDISK_WRITE; } - TPDiskWriterTestLoadActor(const NKikimrBlobStorage::TEvTestLoadRequest::TPDiskLoadStart& cmd, const TActorId& parent, + TPDiskWriterTestLoadActor(const NKikimrBlobStorage::TEvTestLoadRequest::TPDiskLoadStart& cmd, const TActorId& parent, const TIntrusivePtr<NMonitoring::TDynamicCounters>& counters, ui64 index, ui64 tag) : Parent(parent) , Tag(tag) @@ -639,7 +639,7 @@ public: }; IActor *CreatePDiskWriterTestLoad(const NKikimrBlobStorage::TEvTestLoadRequest::TPDiskLoadStart& cmd, - const TActorId& parent, const TIntrusivePtr<NMonitoring::TDynamicCounters>& counters, ui64 index, ui64 tag) { + const TActorId& parent, const TIntrusivePtr<NMonitoring::TDynamicCounters>& counters, ui64 index, ui64 tag) { return new TPDiskWriterTestLoadActor(cmd, parent, counters, index, tag); } diff --git a/ydb/core/blobstorage/testload/test_load_quantile.h b/ydb/core/blobstorage/testload/test_load_quantile.h index 252988f1bbb..0d0f0207c6f 100644 --- a/ydb/core/blobstorage/testload/test_load_quantile.h +++ b/ydb/core/blobstorage/testload/test_load_quantile.h @@ -3,7 +3,7 @@ #include "defs.h" #include "test_load_time_series.h" -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> namespace NKikimr { diff --git a/ydb/core/blobstorage/testload/test_load_vdisk_write.cpp b/ydb/core/blobstorage/testload/test_load_vdisk_write.cpp index 52d33ff12c5..39ecca9dbda 100644 --- a/ydb/core/blobstorage/testload/test_load_vdisk_write.cpp +++ b/ydb/core/blobstorage/testload/test_load_vdisk_write.cpp @@ -6,7 +6,7 @@ #include <ydb/core/blobstorage/backpressure/queue_backpressure_client.h> #include <ydb/core/base/appdata.h> #include <ydb/core/base/interconnect_channels.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> namespace NKikimr { @@ -24,14 +24,14 @@ namespace NKikimr { struct TEvTryToCollect : public TEventLocal<TEvTryToCollect, EvTryToCollect> {}; - const TActorId ParentActorId; + const TActorId ParentActorId; const ui64 Tag; const TIntrusivePtr<TBlobStorageGroupInfo> Info; const TVDiskID VDiskId; - const TActorId VDiskActorId; + const TActorId VDiskActorId; const TBlobStorageGroupType GType; - TActorId QueueActorId; + TActorId QueueActorId; const ui64 TabletId; const ui32 Channel; @@ -76,7 +76,7 @@ namespace NKikimr { } TVDiskLoadActor(const NKikimrBlobStorage::TEvTestLoadRequest::TVDiskLoadStart& cmd, - const NActors::TActorId& parent, ui64 tag) + const NActors::TActorId& parent, ui64 tag) : ParentActorId(parent) , Tag(tag) , Info(TBlobStorageGroupInfo::Parse(cmd.GetGroupInfo(), nullptr, nullptr)) @@ -340,7 +340,7 @@ namespace NKikimr { } // <anonymous> IActor *CreateVDiskWriterTestLoad(const NKikimrBlobStorage::TEvTestLoadRequest::TVDiskLoadStart& cmd, - const NActors::TActorId& parent, ui64 tag) { + const NActors::TActorId& parent, ui64 tag) { return new TVDiskLoadActor(cmd, parent, tag); } diff --git a/ydb/core/blobstorage/testload/test_load_write.cpp b/ydb/core/blobstorage/testload/test_load_write.cpp index 5c4cb4c2ee6..3169c1609ed 100644 --- a/ydb/core/blobstorage/testload/test_load_write.cpp +++ b/ydb/core/blobstorage/testload/test_load_write.cpp @@ -9,7 +9,7 @@ #include <google/protobuf/text_format.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/datetime/cputimer.h> #include <util/generic/queue.h> @@ -748,7 +748,7 @@ class TLogWriterTestLoadActor : public TActorBootstrapped<TLogWriterTestLoadActo TString ConfingString; const ui64 Tag; - const TActorId Parent; + const TActorId Parent; TMaybe<TDuration> TestDuration; @@ -770,7 +770,7 @@ public: return NKikimrServices::TActivity::BS_LOAD_ACTOR; } - TLogWriterTestLoadActor(const NKikimrBlobStorage::TEvTestLoadRequest::TLoadStart& cmd, const TActorId& parent, + TLogWriterTestLoadActor(const NKikimrBlobStorage::TEvTestLoadRequest::TLoadStart& cmd, const TActorId& parent, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, ui64 tag) : Tag(tag) , Parent(parent) @@ -970,7 +970,7 @@ public: ) }; -IActor *CreateWriterTestLoad(const NKikimrBlobStorage::TEvTestLoadRequest::TLoadStart& cmd, const TActorId& parent, +IActor *CreateWriterTestLoad(const NKikimrBlobStorage::TEvTestLoadRequest::TLoadStart& cmd, const TActorId& parent, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, ui64 tag) { return new TLogWriterTestLoadActor(cmd, parent, std::move(counters), tag); } diff --git a/ydb/core/blobstorage/testload/ya.make b/ydb/core/blobstorage/testload/ya.make index 53dd93c8995..f1448fe2404 100644 --- a/ydb/core/blobstorage/testload/ya.make +++ b/ydb/core/blobstorage/testload/ya.make @@ -4,8 +4,8 @@ OWNER(fomichev) PEERDIR( contrib/libs/protobuf - library/cpp/monlib/dynamic_counters/percentile - library/cpp/monlib/service/pages + library/cpp/monlib/dynamic_counters/percentile + library/cpp/monlib/service/pages ydb/core/base ydb/core/blobstorage/backpressure ydb/core/blobstorage/base diff --git a/ydb/core/blobstorage/ut_pdiskfit/lib/basic_test.cpp b/ydb/core/blobstorage/ut_pdiskfit/lib/basic_test.cpp index 74945479086..94eae0767aa 100644 --- a/ydb/core/blobstorage/ut_pdiskfit/lib/basic_test.cpp +++ b/ydb/core/blobstorage/ut_pdiskfit/lib/basic_test.cpp @@ -11,7 +11,7 @@ class TFakeVDisk , public TObjectWithState { const TVDiskID VDiskId; - const TActorId PDiskServiceId; + const TActorId PDiskServiceId; const ui64 PDiskGuid; TStateManager *StateManager; TIntrusivePtr<TPDiskParams> PDiskParams; diff --git a/ydb/core/blobstorage/ut_pdiskfit/lib/basic_test.h b/ydb/core/blobstorage/ut_pdiskfit/lib/basic_test.h index eb3ee8f8276..17a25ef9fbe 100644 --- a/ydb/core/blobstorage/ut_pdiskfit/lib/basic_test.h +++ b/ydb/core/blobstorage/ut_pdiskfit/lib/basic_test.h @@ -26,7 +26,7 @@ struct TFakeVDiskParams { }; -IActor *CreateFakeVDisk(const TVDiskID& vdiskId, const TActorId& pdiskServiceId, ui64 pdiskGuid, +IActor *CreateFakeVDisk(const TVDiskID& vdiskId, const TActorId& pdiskServiceId, ui64 pdiskGuid, TStateManager *stateManager, TFakeVDiskParams params); class TBasicTest : public TActorBootstrapped<TBasicTest> { @@ -34,7 +34,7 @@ class TBasicTest : public TActorBootstrapped<TBasicTest> { TStateManager *StateManager = nullptr; TIntrusivePtr<NMonitoring::TDynamicCounters> Counters; TIntrusivePtr<TPDiskConfig> PDiskConfig; - TActorId PDiskServiceId; + TActorId PDiskServiceId; const ui32 NumVDisks; bool InduceLogSplicing; @@ -58,7 +58,7 @@ public: void Bootstrap(const TActorContext& ctx) { CreatePDiskActor(ctx); - TVector<TActorId> actors; + TVector<TActorId> actors; for (ui32 i = 0; i < NumVDisks; ++i) { TVDiskID vdiskId(i, 0, 0, 0, 0); TFakeVDiskParams params; @@ -70,11 +70,11 @@ public: params.LogsToBeSent = 100; } } - TActorId actorId = ctx.ExecutorThread.ActorSystem->Register(CreateFakeVDisk(vdiskId, PDiskServiceId, + TActorId actorId = ctx.ExecutorThread.ActorSystem->Register(CreateFakeVDisk(vdiskId, PDiskServiceId, PDiskConfig->PDiskGuid, StateManager, params)); actors.push_back(actorId); } - for (const TActorId& actor : actors) { + for (const TActorId& actor : actors) { ctx.Send(actor, new TEvents::TEvBootstrap); } Become(&TBasicTest::StateFunc); diff --git a/ydb/core/blobstorage/ut_pdiskfit/lib/fail_injection_test.h b/ydb/core/blobstorage/ut_pdiskfit/lib/fail_injection_test.h index 521cb9edcfb..7e0bfa05e06 100644 --- a/ydb/core/blobstorage/ut_pdiskfit/lib/fail_injection_test.h +++ b/ydb/core/blobstorage/ut_pdiskfit/lib/fail_injection_test.h @@ -13,7 +13,7 @@ #include <library/cpp/actors/protos/services_common.pb.h> #include <library/cpp/lwtrace/all.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <library/cpp/testing/unittest/registar.h> #include <util/folder/tempdir.h> @@ -185,7 +185,7 @@ struct TPDiskFailureInjectionTest { setup->Scheduler = new TBasicSchedulerThread(TSchedulerConfig(512, 100)); // initialize logger settings - const TActorId loggerId(setup->NodeId, "logger"); + const TActorId loggerId(setup->NodeId, "logger"); TIntrusivePtr<NLog::TSettings> loggerSettings = new NLog::TSettings(loggerId, NKikimrServices::LOGGER, NActors::NLog::PRI_NOTICE, NActors::NLog::PRI_DEBUG, 0); diff --git a/ydb/core/blobstorage/ut_vdisk/lib/astest.h b/ydb/core/blobstorage/ut_vdisk/lib/astest.h index 9ba8650e398..2cf254eff2c 100644 --- a/ydb/core/blobstorage/ut_vdisk/lib/astest.h +++ b/ydb/core/blobstorage/ut_vdisk/lib/astest.h @@ -13,7 +13,7 @@ #include <ydb/core/scheme_types/scheme_type_registry.h> #include <ydb/core/scheme/scheme_type_registry.h> #include <ydb/library/pdisk_io/aio.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/system/event.h> @@ -61,13 +61,13 @@ inline void TTestWithActorSystem::Run(NActors::IActor *testActor) { setup1->Executors[3].Reset(new TBasicExecutorPool(3, 8, 20)); setup1->Scheduler.Reset(new TBasicSchedulerThread(TSchedulerConfig(512, 100))); - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); TActorSetupCmd nameserviceSetup(CreateNameserverTable(nameserverTable), TMailboxType::Simple, 0); - setup1->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(nameserviceId, nameserviceSetup)); + setup1->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(nameserviceId, nameserviceSetup)); ///////////////////////// LOGGER /////////////////////////////////////////////// - NActors::TActorId loggerActorId = NActors::TActorId(1, "logger"); + NActors::TActorId loggerActorId = NActors::TActorId(1, "logger"); TIntrusivePtr<NActors::NLog::TSettings> logSettings; logSettings.Reset(new NActors::NLog::TSettings(loggerActorId, NKikimrServices::LOGGER, NActors::NLog::PRI_ERROR, NActors::NLog::PRI_DEBUG, 0)); // NOTICE @@ -88,14 +88,14 @@ inline void TTestWithActorSystem::Run(NActors::IActor *testActor) { NActors::CreateStderrBackend(), Counters->GetSubgroup("logger", "counters")); NActors::TActorSetupCmd loggerActorCmd(loggerActor, NActors::TMailboxType::Simple, 0); - std::pair<NActors::TActorId, NActors::TActorSetupCmd> loggerActorPair(loggerActorId, loggerActorCmd); + std::pair<NActors::TActorId, NActors::TActorSetupCmd> loggerActorPair(loggerActorId, loggerActorCmd); setup1->LocalServices.push_back(loggerActorPair); ////////////////////////////////////////////////////////////////////////////// ///////////////////////// SETUP TEST ACTOR /////////////////////////////////// - NActors::TActorId testActorId = NActors::TActorId(1, "test123"); + NActors::TActorId testActorId = NActors::TActorId(1, "test123"); TActorSetupCmd testActorSetup(testActor, TMailboxType::Simple, 0); - setup1->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(testActorId, testActorSetup)); + setup1->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(testActorId, testActorSetup)); ////////////////////////////////////////////////////////////////////////////// ///////////////////////// TYPE REGISTRY ////////////////////////////////////// diff --git a/ydb/core/blobstorage/ut_vdisk/lib/helpers.cpp b/ydb/core/blobstorage/ut_vdisk/lib/helpers.cpp index 6c83a22616f..b73da3df8db 100644 --- a/ydb/core/blobstorage/ut_vdisk/lib/helpers.cpp +++ b/ydb/core/blobstorage/ut_vdisk/lib/helpers.cpp @@ -20,7 +20,7 @@ static inline TString LimitData(const TString &data) { //////////////////////////////////////////////////////////////////////////////////////////////////////////////// template <class TMsg> bool PDiskGenStatusHandlerDefault(const TActorContext &ctx, - const TActorId ¬ifyId, + const TActorId ¬ifyId, typename TMsg::TPtr &ev) { Y_UNUSED(notifyId); Y_VERIFY(ev->Get()->Record.GetStatus() == NKikimrProto::OK, "Status=%d (%s)", ev->Get()->Record.GetStatus(), @@ -31,7 +31,7 @@ bool PDiskGenStatusHandlerDefault(const TActorContext &ctx, template <class TMsg> bool PDiskGenStatusHandlerErrorAware(const TActorContext &ctx, - const TActorId ¬ifyId, + const TActorId ¬ifyId, typename TMsg::TPtr &ev) { auto status = ev->Get()->Record.GetStatus(); if (status != NKikimrProto::OK) { @@ -44,7 +44,7 @@ bool PDiskGenStatusHandlerErrorAware(const TActorContext &ctx, template <class TMsg> bool PDiskGenStatusHandleYellowMoveZone(const TActorContext &ctx, - const TActorId ¬ifyId, + const TActorId ¬ifyId, typename TMsg::TPtr &ev) { auto status = ev->Get()->Record.GetStatus(); auto flags = ev->Get()->Record.GetStatusFlags(); @@ -62,19 +62,19 @@ bool PDiskGenStatusHandleYellowMoveZone(const TActorContext &ctx, // PDisk Put status handlers //////////////////////////////////////////////////////////////////////////////////////////////////////////////// bool PDiskPutStatusHandlerDefault(const TActorContext &ctx, - const TActorId ¬ifyId, + const TActorId ¬ifyId, TEvBlobStorage::TEvVPutResult::TPtr &ev) { return PDiskGenStatusHandlerDefault<TEvBlobStorage::TEvVPutResult>(ctx, notifyId, ev); } bool PDiskPutStatusHandlerErrorAware(const TActorContext &ctx, - const TActorId ¬ifyId, + const TActorId ¬ifyId, TEvBlobStorage::TEvVPutResult::TPtr &ev) { return PDiskGenStatusHandlerErrorAware<TEvBlobStorage::TEvVPutResult>(ctx, notifyId, ev); } bool PDiskPutStatusHandlerYellowMoveZone(const NActors::TActorContext &ctx, - const NActors::TActorId ¬ifyId, + const NActors::TActorId ¬ifyId, NKikimr::TEvBlobStorage::TEvVPutResult::TPtr &ev) { return PDiskGenStatusHandleYellowMoveZone<TEvBlobStorage::TEvVPutResult>(ctx, notifyId, ev); } @@ -84,12 +84,12 @@ bool PDiskPutStatusHandlerYellowMoveZone(const NActors::TActorContext &ctx, // PDisk Get status handlers //////////////////////////////////////////////////////////////////////////////////////////////////////////////// bool PDiskGetStatusHandlerDefault(const TActorContext &ctx, - const TActorId ¬ifyId, + const TActorId ¬ifyId, TEvBlobStorage::TEvVGetResult::TPtr &ev) { return PDiskGenStatusHandlerDefault<TEvBlobStorage::TEvVGetResult>(ctx, notifyId, ev);} bool PDiskGetStatusHandlerErrorAware(const TActorContext &ctx, - const TActorId ¬ifyId, + const TActorId ¬ifyId, TEvBlobStorage::TEvVGetResult::TPtr &ev) { return PDiskGenStatusHandlerErrorAware<TEvBlobStorage::TEvVGetResult>(ctx, notifyId, ev); } @@ -99,7 +99,7 @@ bool PDiskGetStatusHandlerErrorAware(const TActorContext &ctx, /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// class TRangeGet : public TActorBootstrapped<TRangeGet> { - const TActorId NotifyID; + const TActorId NotifyID; const TAllVDisks::TVDiskInstance VDiskInfo; const TLogoBlobID ReadFrom; const TLogoBlobID ReadTo; @@ -166,7 +166,7 @@ class TRangeGet : public TActorBootstrapped<TRangeGet> { ) public: - TRangeGet(const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, + TRangeGet(const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, const TLogoBlobID &readFrom, const TLogoBlobID &readTo, bool /*indexOnly*/, ui32 maxResults) : TActorBootstrapped<TRangeGet>() , NotifyID(notifyID) @@ -177,7 +177,7 @@ public: {} }; -IActor *CreateRangeGet(const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, +IActor *CreateRangeGet(const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, const TLogoBlobID &readFrom, const TLogoBlobID &readTo, bool indexOnly, ui32 maxResults) { Y_VERIFY(indexOnly); return new TRangeGet(notifyID, vdiskInfo, readFrom, readTo, indexOnly, maxResults); @@ -191,7 +191,7 @@ class TManyPuts : public TActorBootstrapped<TManyPuts> { }; TConfiguration *Conf; - TActorId NotifyID; + TActorId NotifyID; const TAllVDisks::TVDiskInstance VDiskInfo; std::shared_ptr<TVector<TMsgPackInfo>> MsgPacks; @@ -205,7 +205,7 @@ class TManyPuts : public TActorBootstrapped<TManyPuts> { const TDuration RequestTimeout; TVector<TPut> Puts; ui32 PutIdx; - TActorId QueueActorId; + TActorId QueueActorId; bool Started = false; // how many deadline statuses we got ui64 RequestDeadlines = 0; @@ -317,7 +317,7 @@ class TManyPuts : public TActorBootstrapped<TManyPuts> { } public: - TManyPuts(TConfiguration *conf, const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, + TManyPuts(TConfiguration *conf, const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, ui32 msgDataSize, ui32 msgNum, ui64 tabletId, ui32 channel, ui32 gen, std::shared_ptr<IPutHandleClassGenerator> cls, std::shared_ptr<TSet<ui32>> badSteps, TDuration requestTimeout) @@ -337,7 +337,7 @@ public: Init(); } - TManyPuts(TConfiguration *conf, const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, + TManyPuts(TConfiguration *conf, const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, std::shared_ptr<TVector<TMsgPackInfo>> msgPacks, ui64 tabletId, ui32 channel, ui32 gen, std::shared_ptr<IPutHandleClassGenerator> cls, std::shared_ptr<TSet<ui32>> badSteps, TDuration requestTimeout) @@ -358,7 +358,7 @@ public: } }; -IActor *CreateManyPuts(TConfiguration *conf, const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, +IActor *CreateManyPuts(TConfiguration *conf, const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, ui32 msgDataSize, ui32 msgNum, ui64 tabletId, ui32 channel, ui32 gen, std::shared_ptr<IPutHandleClassGenerator> cls, std::shared_ptr<TSet<ui32>> badSteps, TDuration requestTimeout) { @@ -366,7 +366,7 @@ IActor *CreateManyPuts(TConfiguration *conf, const TActorId ¬ifyID, const TAl requestTimeout); } -IActor *CreateManyPuts(TConfiguration *conf, const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, +IActor *CreateManyPuts(TConfiguration *conf, const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, std::shared_ptr<TVector<TMsgPackInfo>> msgPacks, ui64 tabletId, ui32 channel, ui32 gen, std::shared_ptr<IPutHandleClassGenerator> cls, std::shared_ptr<TSet<ui32>> badSteps, TDuration requestTimeout) { @@ -376,7 +376,7 @@ IActor *CreateManyPuts(TConfiguration *conf, const TActorId ¬ifyID, const TAl /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// class TManyMultiPuts : public TActorBootstrapped<TManyMultiPuts> { TConfiguration *Conf; - TActorId NotifyID; + TActorId NotifyID; const TAllVDisks::TVDiskInstance VDiskInfo; const ui32 MsgDataSize; const ui32 MsgNum; @@ -390,7 +390,7 @@ class TManyMultiPuts : public TActorBootstrapped<TManyMultiPuts> { TVector<ui32> Steps; ui32 Step; TString MsgData; - TActorId QueueActorId; + TActorId QueueActorId; bool Started = false; // how many deadline statuses we got ui64 RequestDeadlines = 0; @@ -511,7 +511,7 @@ class TManyMultiPuts : public TActorBootstrapped<TManyMultiPuts> { } public: - TManyMultiPuts(TConfiguration *conf, const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, + TManyMultiPuts(TConfiguration *conf, const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, ui32 msgDataSize, ui32 msgNum, ui32 batchSize, ui64 tabletId, ui32 channel, ui32 gen, std::shared_ptr<IPutHandleClassGenerator> cls, std::shared_ptr<TSet<ui32>> badSteps, TDuration requestTimeout) @@ -542,7 +542,7 @@ public: } }; -IActor *CreateManyMultiPuts(TConfiguration *conf, const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, +IActor *CreateManyMultiPuts(TConfiguration *conf, const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, ui32 msgDataSize, ui32 msgNum, ui32 batchSize, ui64 tabletId, ui32 channel, ui32 gen, std::shared_ptr<IPutHandleClassGenerator> cls, std::shared_ptr<TSet<ui32>> badSteps, TDuration requestTimeout) { @@ -552,7 +552,7 @@ IActor *CreateManyMultiPuts(TConfiguration *conf, const TActorId ¬ifyID, cons /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// class TManyGets : public TActorBootstrapped<TManyGets> { - TActorId NotifyID; + TActorId NotifyID; const TAllVDisks::TVDiskInstance VDiskInfo; const ui32 MsgDataSize; const ui32 MsgNum; @@ -624,7 +624,7 @@ class TManyGets : public TActorBootstrapped<TManyGets> { ) public: - TManyGets(const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, ui32 msgDataSize, ui32 msgNum, + TManyGets(const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, ui32 msgDataSize, ui32 msgNum, ui64 tabletId, ui32 channel, ui32 gen, std::shared_ptr<TSet<ui32>> badSteps) : TActorBootstrapped<TManyGets>() , NotifyID(notifyID) @@ -644,14 +644,14 @@ public: } }; -IActor *CreateManyGets(const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, ui32 msgDataSize, +IActor *CreateManyGets(const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, ui32 msgDataSize, ui32 msgNum, ui64 tabletId, ui32 channel, ui32 gen, std::shared_ptr<TSet<ui32>> badSteps) { return new TManyGets(notifyID, vdiskInfo, msgDataSize, msgNum, tabletId, channel, gen, badSteps); } /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// class TGet : public TActorBootstrapped<TGet> { - TActorId NotifyID; + TActorId NotifyID; const TAllVDisks::TVDiskInstance VDiskInfo; ui32 MsgNum; std::shared_ptr<TVector<TMsgPackInfo>> MsgPacks; @@ -752,7 +752,7 @@ class TGet : public TActorBootstrapped<TGet> { } public: - TGet(const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, ui32 msgDataSize, ui32 msgNum, + TGet(const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, ui32 msgDataSize, ui32 msgNum, ui64 tabletId, ui32 channel, ui32 gen, ui64 shift, bool withErrorResponse) : TActorBootstrapped<TGet>() , NotifyID(notifyID) @@ -767,7 +767,7 @@ public: Init(); } - TGet(const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, + TGet(const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, std::shared_ptr<TVector<TMsgPackInfo>> msgPacks, ui64 tabletId, ui32 channel, ui32 gen, ui64 shift, bool withErrorResponse) : TActorBootstrapped<TGet>() @@ -784,12 +784,12 @@ public: } }; -IActor *CreateGet(const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, ui32 msgDataSize, +IActor *CreateGet(const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, ui32 msgDataSize, ui32 msgNum, ui64 tabletId, ui32 channel, ui32 gen, ui64 shift, bool withErrorResponse) { return new TGet(notifyID, vdiskInfo, msgDataSize, msgNum, tabletId, channel, gen, shift, withErrorResponse); } -IActor *CreateGet(const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, +IActor *CreateGet(const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, std::shared_ptr<TVector<TMsgPackInfo>> msgPacks, ui64 tabletId, ui32 channel, ui32 gen, ui64 shift, bool withErrorResponse) { return new TGet(notifyID, vdiskInfo, msgPacks, tabletId, channel, gen, shift, withErrorResponse); @@ -797,7 +797,7 @@ IActor *CreateGet(const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vd /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// class TPutGC : public TActorBootstrapped<TPutGC> { - const NActors::TActorId NotifyID; + const NActors::TActorId NotifyID; const TAllVDisks::TVDiskInstance VDiskInfo; ui64 TabletID; ui32 RecGen; @@ -832,7 +832,7 @@ class TPutGC : public TActorBootstrapped<TPutGC> { ) public: - TPutGC(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, + TPutGC(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, ui64 tabletID, ui32 recGen, ui32 recGenCounter, ui32 channel, bool collect, ui32 collectGen, ui32 collectStep, TAutoPtr<TVector<TLogoBlobID>> keep, TAutoPtr<TVector<TLogoBlobID>> doNotKeep) : TActorBootstrapped<TPutGC>() @@ -850,7 +850,7 @@ public: {} }; -NActors::IActor *CreatePutGC(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, +NActors::IActor *CreatePutGC(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, ui64 tabletID, ui32 recGen, ui32 recGenCounter, ui32 channel, bool collect, ui32 collectGen, ui32 collectStep, TAutoPtr<TVector<TLogoBlobID>> keep, TAutoPtr<TVector<TLogoBlobID>> doNotKeep) { @@ -862,7 +862,7 @@ NActors::IActor *CreatePutGC(const NActors::TActorId ¬ifyID, const TAllVDisks /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// class TWaitForCompactionOneDisk : public TActorBootstrapped<TWaitForCompactionOneDisk> { - TActorId NotifyID; + TActorId NotifyID; const TAllVDisks::TVDiskInstance VDiskInfo; friend class TActorBootstrapped<TWaitForCompactionOneDisk>; @@ -911,20 +911,20 @@ class TWaitForCompactionOneDisk : public TActorBootstrapped<TWaitForCompactionOn ) public: - TWaitForCompactionOneDisk(const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo) + TWaitForCompactionOneDisk(const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo) : TActorBootstrapped<TWaitForCompactionOneDisk>() , NotifyID(notifyID) , VDiskInfo(vdiskInfo) {} }; -NActors::IActor *CreateWaitForCompaction(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo) { +NActors::IActor *CreateWaitForCompaction(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo) { return new TWaitForCompactionOneDisk(notifyID, vdiskInfo); } /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// class TWaitForCompaction : public TActorBootstrapped<TWaitForCompaction> { - TActorId NotifyID; + TActorId NotifyID; TConfiguration *Conf; ui32 Counter; @@ -955,7 +955,7 @@ class TWaitForCompaction : public TActorBootstrapped<TWaitForCompaction> { ) public: - TWaitForCompaction(const TActorId ¬ifyID, TConfiguration *conf) + TWaitForCompaction(const TActorId ¬ifyID, TConfiguration *conf) : TActorBootstrapped<TWaitForCompaction>() , NotifyID(notifyID) , Conf(conf) @@ -963,7 +963,7 @@ public: {} }; -NActors::IActor *CreateWaitForCompaction(const NActors::TActorId ¬ifyID, TConfiguration *conf) { +NActors::IActor *CreateWaitForCompaction(const NActors::TActorId ¬ifyID, TConfiguration *conf) { return new TWaitForCompaction(notifyID, conf); } @@ -1079,14 +1079,14 @@ NActors::IActor *CreateDefrag( class TWaitForSync : public TActorBootstrapped<TWaitForSync> { // info we get from single VDisk struct TInfo { - const TActorId ActorId; + const TActorId ActorId; const TVDiskID VDiskId; bool SetUp = false; TVector<TSyncState> SyncStates; NSyncLog::TLogEssence SyncLogEssence; - TInfo(const TActorId &id, const TVDiskID &vd) + TInfo(const TActorId &id, const TVDiskID &vd) : ActorId(id) , VDiskId(vd) {} @@ -1190,7 +1190,7 @@ class TWaitForSync : public TActorBootstrapped<TWaitForSync> { }; - TActorId NotifyID; + TActorId NotifyID; ui32 Counter = 0; TMutualSyncState MutualSyncState; @@ -1235,14 +1235,14 @@ class TWaitForSync : public TActorBootstrapped<TWaitForSync> { ) public: - TWaitForSync(const TActorId ¬ifyID, TConfiguration *conf) + TWaitForSync(const TActorId ¬ifyID, TConfiguration *conf) : TActorBootstrapped<TWaitForSync>() , NotifyID(notifyID) , MutualSyncState(conf) {} }; -NActors::IActor *CreateWaitForSync(const NActors::TActorId ¬ifyID, TConfiguration *conf) { +NActors::IActor *CreateWaitForSync(const NActors::TActorId ¬ifyID, TConfiguration *conf) { return new TWaitForSync(notifyID, conf); } @@ -1251,7 +1251,7 @@ NActors::IActor *CreateWaitForSync(const NActors::TActorId ¬ifyID, TConfigura /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// class TCheckDbEmptynessActor : public TActorBootstrapped<TCheckDbEmptynessActor> { - const TActorId NotifyID; + const TActorId NotifyID; const TAllVDisks::TVDiskInstance VDiskInfo; const bool ExpectEmpty; @@ -1280,7 +1280,7 @@ class TCheckDbEmptynessActor : public TActorBootstrapped<TCheckDbEmptynessActor> ) public: - TCheckDbEmptynessActor(const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, bool expectEmpty) + TCheckDbEmptynessActor(const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, bool expectEmpty) : TActorBootstrapped<TCheckDbEmptynessActor>() , NotifyID(notifyID) , VDiskInfo(vdiskInfo) @@ -1289,7 +1289,7 @@ public: }; -NActors::IActor *CreateCheckDbEmptyness(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, +NActors::IActor *CreateCheckDbEmptyness(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, bool expectEmpty) { return new TCheckDbEmptynessActor(notifyID, vdiskInfo, expectEmpty); } @@ -1298,7 +1298,7 @@ NActors::IActor *CreateCheckDbEmptyness(const NActors::TActorId ¬ifyID, const /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// class TPutGCToCorrespondingVDisksActor : public TActorBootstrapped<TPutGCToCorrespondingVDisksActor> { - const NActors::TActorId NotifyID; + const NActors::TActorId NotifyID; TConfiguration *Conf; ui32 Counter; ui64 TabletID; @@ -1347,7 +1347,7 @@ class TPutGCToCorrespondingVDisksActor : public TActorBootstrapped<TPutGCToCorre ) public: - TPutGCToCorrespondingVDisksActor(const NActors::TActorId ¬ifyID, TConfiguration *conf, ui64 tabletID, + TPutGCToCorrespondingVDisksActor(const NActors::TActorId ¬ifyID, TConfiguration *conf, ui64 tabletID, ui32 recGen, ui32 recGenCounter, ui32 channel, bool collect, ui32 collectGen, ui32 collectStep, TAutoPtr<TVector<NKikimr::TLogoBlobID>> keep, TAutoPtr<TVector<NKikimr::TLogoBlobID>> doNotKeep) @@ -1368,7 +1368,7 @@ public: }; -IActor *PutGCToCorrespondingVDisks(const NActors::TActorId ¬ifyID, TConfiguration *conf, ui64 tabletID, +IActor *PutGCToCorrespondingVDisks(const NActors::TActorId ¬ifyID, TConfiguration *conf, ui64 tabletID, ui32 recGen, ui32 recGenCounter, ui32 channel, bool collect, ui32 collectGen, ui32 collectStep, TAutoPtr<TVector<NKikimr::TLogoBlobID>> keep, TAutoPtr<TVector<NKikimr::TLogoBlobID>> doNotKeep) { @@ -1376,7 +1376,7 @@ IActor *PutGCToCorrespondingVDisks(const NActors::TActorId ¬ifyID, TConfigura collectGen, collectStep, keep, doNotKeep); } -NActors::IActor *PutGCToCorrespondingVDisks(const NActors::TActorId ¬ifyID, TConfiguration *conf, +NActors::IActor *PutGCToCorrespondingVDisks(const NActors::TActorId ¬ifyID, TConfiguration *conf, const TGCSettings &settings, TAutoPtr<TVector<NKikimr::TLogoBlobID>> keep, TAutoPtr<TVector<NKikimr::TLogoBlobID>> doNotKeep) { @@ -1390,7 +1390,7 @@ NActors::IActor *PutGCToCorrespondingVDisks(const NActors::TActorId ¬ifyID, T /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// -void PutLogoBlobToVDisk(const TActorContext &ctx, const TActorId &actorID, const TVDiskID &vdiskID, +void PutLogoBlobToVDisk(const TActorContext &ctx, const TActorId &actorID, const TVDiskID &vdiskID, const TLogoBlobID &id, const TString &data, NKikimrBlobStorage::EPutHandleClass cls) { LOG_DEBUG(ctx, NActorsServices::TEST, " Sending TEvPut: id=%s data='%s'", id.ToString().data(), LimitData(data).data()); ctx.Send(actorID, new TEvBlobStorage::TEvVPut(id, data, vdiskID, false, nullptr, TInstant::Max(), cls)); @@ -1440,7 +1440,7 @@ ui32 GetLogoBlobFromCorrespondingVDisks(const NActors::TActorContext &ctx, NKiki /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// -TDataSnapshot::TItem::TItem(const TVDiskID &vdisk, const TActorId &service, const NKikimr::TLogoBlobID &id, +TDataSnapshot::TItem::TItem(const TVDiskID &vdisk, const TActorId &service, const NKikimr::TLogoBlobID &id, const TString &data, const TIngress &ingress) : VDiskID(vdisk) , ServiceID(service) @@ -1459,7 +1459,7 @@ TDataSnapshot::TDataSnapshot(TIntrusivePtr<NKikimr::TBlobStorageGroupInfo> info) : Info(info) {} -void TDataSnapshot::PutExact(const TVDiskID &vdisk, const TActorId &service, const NKikimr::TLogoBlobID &id, +void TDataSnapshot::PutExact(const TVDiskID &vdisk, const TActorId &service, const NKikimr::TLogoBlobID &id, const TString &data, const TIngress &ingress) { Y_ASSERT(id.PartId() != 0); Data.push_back(TItem(vdisk, service, id, data, ingress)); @@ -1508,7 +1508,7 @@ TDataSnapshot::TIterator TDataSnapshot::end() { //////////////////////////////////////////////////////////////////////////////////////////////////////////////// class TLoadDataSnapshotActor : public TActorBootstrapped<TLoadDataSnapshotActor> { private: - const NActors::TActorId NotifyID; + const NActors::TActorId NotifyID; TConfiguration *Conf; TDataSnapshotPtr DataPtr; NKikimrBlobStorage::EPutHandleClass HandleClass; @@ -1551,7 +1551,7 @@ private: ) public: - TLoadDataSnapshotActor(const NActors::TActorId ¬ifyID, TConfiguration *conf, TDataSnapshotPtr dataPtr, + TLoadDataSnapshotActor(const NActors::TActorId ¬ifyID, TConfiguration *conf, TDataSnapshotPtr dataPtr, NKikimrBlobStorage::EPutHandleClass cls, TPDiskPutStatusHandler hndl) : TActorBootstrapped<TLoadDataSnapshotActor>() , NotifyID(notifyID) @@ -1563,7 +1563,7 @@ public: {} }; -NActors::IActor *CreateLoadDataSnapshot(const NActors::TActorId ¬ifyID, TConfiguration *conf, +NActors::IActor *CreateLoadDataSnapshot(const NActors::TActorId ¬ifyID, TConfiguration *conf, TDataSnapshotPtr dataPtr, NKikimrBlobStorage::EPutHandleClass cls, TPDiskPutStatusHandler hndl) { return new TLoadDataSnapshotActor(notifyID, conf, dataPtr, cls, hndl); @@ -1573,7 +1573,7 @@ NActors::IActor *CreateLoadDataSnapshot(const NActors::TActorId ¬ifyID, TConf /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// class TCheckDataSnapshotActor : public TActorBootstrapped<TCheckDataSnapshotActor> { - const NActors::TActorId NotifyID; + const NActors::TActorId NotifyID; TConfiguration *Conf; TDataSnapshotPtr DataPtr; TDataSnapshot::TIterator Cur; @@ -1662,7 +1662,7 @@ class TCheckDataSnapshotActor : public TActorBootstrapped<TCheckDataSnapshotActo ) public: - TCheckDataSnapshotActor(const NActors::TActorId ¬ifyID, TConfiguration *conf, TDataSnapshotPtr dataPtr, + TCheckDataSnapshotActor(const NActors::TActorId ¬ifyID, TConfiguration *conf, TDataSnapshotPtr dataPtr, TPDiskGetStatusHandler hndl) : TActorBootstrapped<TCheckDataSnapshotActor>() , NotifyID(notifyID) @@ -1674,7 +1674,7 @@ public: {} }; -NActors::IActor *CreateCheckDataSnapshot(const NActors::TActorId ¬ifyID, TConfiguration *conf, +NActors::IActor *CreateCheckDataSnapshot(const NActors::TActorId ¬ifyID, TConfiguration *conf, TDataSnapshotPtr dataPtr, TPDiskGetStatusHandler hndl) { return new TCheckDataSnapshotActor(notifyID, conf, dataPtr, hndl); } @@ -1851,7 +1851,7 @@ TSyncRunner::TSyncRunner(TActorSystem *system, TConfiguration *conf) WorkerID = ActorSystem->Register(new TSyncRunActor(_Event, ReturnValue)); } -TActorId TSyncRunner::NotifyID() const { +TActorId TSyncRunner::NotifyID() const { return WorkerID; } @@ -1900,7 +1900,7 @@ TSyncTestWithSmallCommonDataset::TSyncTestWithSmallCommonDataset(TConfiguration class TManyPutsToOneVDiskActor : public TActorBootstrapped<TManyPutsToOneVDiskActor> { typedef TManyPutsToOneVDiskActor TThis; - const NActors::TActorId NotifyID; + const NActors::TActorId NotifyID; const TAllVDisks::TVDiskInstance VDiskInfo; const IDataSet *DataSet; ui8 PartId; @@ -1945,7 +1945,7 @@ class TManyPutsToOneVDiskActor : public TActorBootstrapped<TManyPutsToOneVDiskAc ) public: - TManyPutsToOneVDiskActor(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, + TManyPutsToOneVDiskActor(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, const IDataSet *dataSet, ui8 partId, NKikimrBlobStorage::EPutHandleClass cls) : TActorBootstrapped<TManyPutsToOneVDiskActor>() , NotifyID(notifyID) @@ -1958,7 +1958,7 @@ public: {} }; -NActors::IActor *ManyPutsToOneVDisk(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, +NActors::IActor *ManyPutsToOneVDisk(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, const IDataSet *dataSet, ui8 partId, NKikimrBlobStorage::EPutHandleClass cls) { return new TManyPutsToOneVDiskActor(notifyID, vdiskInfo, dataSet, partId, cls); } @@ -1970,7 +1970,7 @@ NActors::IActor *ManyPutsToOneVDisk(const NActors::TActorId ¬ifyID, const TAl class TManyPutsToCorrespondingVDisksActor : public TActorBootstrapped<TManyPutsToCorrespondingVDisksActor> { typedef TManyPutsToCorrespondingVDisksActor TThis; - const NActors::TActorId NotifyID; + const NActors::TActorId NotifyID; TConfiguration *Conf; const IDataSet *DataSet; ui32 InFlight; @@ -2049,7 +2049,7 @@ class TManyPutsToCorrespondingVDisksActor : public TActorBootstrapped<TManyPutsT ) public: - TManyPutsToCorrespondingVDisksActor(const NActors::TActorId ¬ifyID, + TManyPutsToCorrespondingVDisksActor(const NActors::TActorId ¬ifyID, TConfiguration *conf, const IDataSet *dataSet, TPDiskPutStatusHandler hndl, @@ -2069,7 +2069,7 @@ public: {} }; -NActors::IActor *ManyPutsToCorrespondingVDisks(const NActors::TActorId ¬ifyID, TConfiguration *conf, +NActors::IActor *ManyPutsToCorrespondingVDisks(const NActors::TActorId ¬ifyID, TConfiguration *conf, const IDataSet *dataSet, TPDiskPutStatusHandler hndl, ui32 inFlight) { return new TManyPutsToCorrespondingVDisksActor(notifyID, conf, dataSet, hndl, inFlight); } diff --git a/ydb/core/blobstorage/ut_vdisk/lib/helpers.h b/ydb/core/blobstorage/ut_vdisk/lib/helpers.h index 95962e804c5..ed8792b838d 100644 --- a/ydb/core/blobstorage/ut_vdisk/lib/helpers.h +++ b/ydb/core/blobstorage/ut_vdisk/lib/helpers.h @@ -37,44 +37,44 @@ private: // PDisk Put status handlers //////////////////////////////////////////////////////////////////////////////////////////////////////////////// using TPDiskPutStatusHandler = bool (*)(const NActors::TActorContext &, - const NActors::TActorId ¬ifyId, + const NActors::TActorId ¬ifyId, NKikimr::TEvBlobStorage::TEvVPutResult::TPtr &); bool PDiskPutStatusHandlerDefault(const NActors::TActorContext &ctx, - const NActors::TActorId ¬ifyId, + const NActors::TActorId ¬ifyId, NKikimr::TEvBlobStorage::TEvVPutResult::TPtr &ev); bool PDiskPutStatusHandlerErrorAware(const NActors::TActorContext &ctx, - const NActors::TActorId ¬ifyId, + const NActors::TActorId ¬ifyId, NKikimr::TEvBlobStorage::TEvVPutResult::TPtr &ev); bool PDiskPutStatusHandlerYellowMoveZone(const NActors::TActorContext &ctx, - const NActors::TActorId ¬ifyId, + const NActors::TActorId ¬ifyId, NKikimr::TEvBlobStorage::TEvVPutResult::TPtr &ev); //////////////////////////////////////////////////////////////////////////////////////////////////////////////// // PDisk Get status handlers //////////////////////////////////////////////////////////////////////////////////////////////////////////////// using TPDiskGetStatusHandler = bool (*)(const NActors::TActorContext &, - const NActors::TActorId ¬ifyId, + const NActors::TActorId ¬ifyId, NKikimr::TEvBlobStorage::TEvVGetResult::TPtr &); bool PDiskGetStatusHandlerDefault(const NActors::TActorContext &ctx, - const NActors::TActorId ¬ifyId, + const NActors::TActorId ¬ifyId, NKikimr::TEvBlobStorage::TEvVGetResult::TPtr &ev); bool PDiskGetStatusHandlerErrorAware(const NActors::TActorContext &ctx, - const NActors::TActorId ¬ifyId, + const NActors::TActorId ¬ifyId, NKikimr::TEvBlobStorage::TEvVGetResult::TPtr &ev); //////////////////////////////////////////////////////////////////////////////////////////////////////////////// -NActors::IActor *CreateRangeGet(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, +NActors::IActor *CreateRangeGet(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, const NKikimr::TLogoBlobID &readFrom, const NKikimr::TLogoBlobID &readTo, bool indexOnly, ui32 maxResults); -NActors::IActor *CreateManyPuts(TConfiguration *conf, const NActors::TActorId ¬ifyID, +NActors::IActor *CreateManyPuts(TConfiguration *conf, const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, ui32 msgDataSize, ui32 msgNum, ui64 tabletId, ui32 channel, ui32 gen, std::shared_ptr<IPutHandleClassGenerator> cls, std::shared_ptr<TSet<ui32>> badSteps, @@ -95,40 +95,40 @@ struct TMsgPackInfo { }; -NActors::IActor *CreateManyPuts(TConfiguration *conf, const NActors::TActorId ¬ifyID, +NActors::IActor *CreateManyPuts(TConfiguration *conf, const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, std::shared_ptr<TVector<TMsgPackInfo>> msgPacks, ui64 tabletId, ui32 channel, ui32 gen, std::shared_ptr<IPutHandleClassGenerator> cls, std::shared_ptr<TSet<ui32>> badSteps, TDuration requestTimeout); -NActors::IActor *CreateManyMultiPuts(TConfiguration *conf, const NActors::TActorId ¬ifyID, +NActors::IActor *CreateManyMultiPuts(TConfiguration *conf, const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, ui32 msgDataSize, ui32 msgNum, ui32 batchSize, ui64 tabletId, ui32 channel, ui32 gen, std::shared_ptr<IPutHandleClassGenerator> cls, std::shared_ptr<TSet<ui32>> badSteps, TDuration requestTimeout); -NActors::IActor *CreateManyGets(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, +NActors::IActor *CreateManyGets(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, ui32 msgDataSize, ui32 msgNum, ui64 tabletId, ui32 channel, ui32 gen, std::shared_ptr<TSet<ui32>> badSteps); -NActors::IActor *CreateGet(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, +NActors::IActor *CreateGet(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, ui32 msgDataSize, ui32 msgNum, ui64 tabletId, ui32 channel, ui32 gen, ui64 shift, bool withErrorResponse); -NActors::IActor *CreateGet(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, +NActors::IActor *CreateGet(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, std::shared_ptr<TVector<TMsgPackInfo>> msgPacks, ui64 tabletId, ui32 channel, ui32 gen, ui64 shift, bool withErrorResponse); -NActors::IActor *CreatePutGC(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, +NActors::IActor *CreatePutGC(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, ui64 tabletID, ui32 recGen, ui32 recGenCounter, ui32 channel, bool collect, ui32 collectGen, ui32 collectStep, TAutoPtr<TVector<NKikimr::TLogoBlobID>> keep, TAutoPtr<TVector<NKikimr::TLogoBlobID>> doNotKeep); -NActors::IActor *CreateWaitForCompaction(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo); -NActors::IActor *CreateWaitForCompaction(const NActors::TActorId ¬ifyID, TConfiguration *conf); -NActors::IActor *CreateWaitForSync(const NActors::TActorId ¬ifyID, TConfiguration *conf); -NActors::IActor *CreateCheckDbEmptyness(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, +NActors::IActor *CreateWaitForCompaction(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo); +NActors::IActor *CreateWaitForCompaction(const NActors::TActorId ¬ifyID, TConfiguration *conf); +NActors::IActor *CreateWaitForSync(const NActors::TActorId ¬ifyID, TConfiguration *conf); +NActors::IActor *CreateCheckDbEmptyness(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, bool expectEmpty); NActors::IActor *CreateDefrag(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, bool full, std::function<void(NKikimr::TEvBlobStorage::TEvVDefragResult::TPtr &)> check); @@ -136,14 +136,14 @@ NActors::IActor *CreateDefrag(const NActors::TActorId ¬ifyID, TConfiguration bool full, std::function<void(NKikimr::TEvBlobStorage::TEvVDefragResult::TPtr &)> check); template <class TRequestFunc, class TCheckResponseFunc> -NActors::IActor *CreateOneGet(const NActors::TActorId ¬ifyID, TRequestFunc req, TCheckResponseFunc check); +NActors::IActor *CreateOneGet(const NActors::TActorId ¬ifyID, TRequestFunc req, TCheckResponseFunc check); //////////////////////////////////////////////////////////////////////////////////////////////////////////////// -NActors::IActor *ManyPutsToOneVDisk(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, +NActors::IActor *ManyPutsToOneVDisk(const NActors::TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, const IDataSet *dataSet, ui8 partId, NKikimrBlobStorage::EPutHandleClass cls); -NActors::IActor *ManyPutsToCorrespondingVDisks(const NActors::TActorId ¬ifyID, +NActors::IActor *ManyPutsToCorrespondingVDisks(const NActors::TActorId ¬ifyID, TConfiguration *conf, const IDataSet *dataSet, TPDiskPutStatusHandler hndl = PDiskPutStatusHandlerDefault, @@ -158,12 +158,12 @@ struct TGCSettings { ui32 CollectGen; ui32 CollectStep; }; -NActors::IActor *PutGCToCorrespondingVDisks(const NActors::TActorId ¬ifyID, TConfiguration *conf, ui64 tabletID, +NActors::IActor *PutGCToCorrespondingVDisks(const NActors::TActorId ¬ifyID, TConfiguration *conf, ui64 tabletID, ui32 recGen, ui32 recGenCounter, ui32 channel, bool collect, ui32 collectGen, ui32 collectStep, TAutoPtr<TVector<NKikimr::TLogoBlobID>> keep, TAutoPtr<TVector<NKikimr::TLogoBlobID>> doNotKeep); -NActors::IActor *PutGCToCorrespondingVDisks(const NActors::TActorId ¬ifyID, TConfiguration *conf, +NActors::IActor *PutGCToCorrespondingVDisks(const NActors::TActorId ¬ifyID, TConfiguration *conf, const TGCSettings &settings, TAutoPtr<TVector<NKikimr::TLogoBlobID>> keep, TAutoPtr<TVector<NKikimr::TLogoBlobID>> doNotKeep); @@ -174,7 +174,7 @@ NActors::IActor *PutGCToCorrespondingVDisks(const NActors::TActorId ¬ifyID, T //////////////////////////////////////////////////////////////////////////////////////////////////////////////// -void PutLogoBlobToVDisk(const NActors::TActorContext &ctx, const NActors::TActorId &actorID, +void PutLogoBlobToVDisk(const NActors::TActorContext &ctx, const NActors::TActorId &actorID, const NKikimr::TVDiskID &vdiskID, const NKikimr::TLogoBlobID &id, const TString &data, NKikimrBlobStorage::EPutHandleClass cls); @@ -195,12 +195,12 @@ class TDataSnapshot : public TThrRefBase { public: struct TItem { NKikimr::TVDiskID VDiskID; - NKikimr::TActorId ServiceID; + NKikimr::TActorId ServiceID; NKikimr::TLogoBlobID Id; // exact part TString Data; NKikimr::TIngress Ingress; - TItem(const NKikimr::TVDiskID &vdisk, const NKikimr::TActorId &service, const NKikimr::TLogoBlobID &id, + TItem(const NKikimr::TVDiskID &vdisk, const NKikimr::TActorId &service, const NKikimr::TLogoBlobID &id, const TString &data, const NKikimr::TIngress &ingress); }; @@ -210,7 +210,7 @@ public: typedef TData::iterator TIterator; TDataSnapshot(TIntrusivePtr<NKikimr::TBlobStorageGroupInfo> info); - void PutExact(const NKikimr::TVDiskID &vdisk, const NKikimr::TActorId &service, const NKikimr::TLogoBlobID &id, + void PutExact(const NKikimr::TVDiskID &vdisk, const NKikimr::TActorId &service, const NKikimr::TLogoBlobID &id, const TString &data, const NKikimr::TIngress &ingress = NKikimr::TIngress(0)); // must provide exact partId void PutCorresponding(const NKikimr::TLogoBlobID &id, const TString &data); void SortAndCheck(); @@ -225,10 +225,10 @@ private: typedef TIntrusivePtr<TDataSnapshot> TDataSnapshotPtr; //////////////////////////////////////////////////////////////////////////////////////////////////////////////// -NActors::IActor *CreateLoadDataSnapshot(const NActors::TActorId ¬ifyID, TConfiguration *conf, +NActors::IActor *CreateLoadDataSnapshot(const NActors::TActorId ¬ifyID, TConfiguration *conf, TDataSnapshotPtr dataPtr, NKikimrBlobStorage::EPutHandleClass cls, TPDiskPutStatusHandler hndl = &PDiskPutStatusHandlerDefault); -NActors::IActor *CreateCheckDataSnapshot(const NActors::TActorId ¬ifyID, TConfiguration *conf, TDataSnapshotPtr dataPtr, +NActors::IActor *CreateCheckDataSnapshot(const NActors::TActorId ¬ifyID, TConfiguration *conf, TDataSnapshotPtr dataPtr, TPDiskGetStatusHandler hndl = &PDiskGetStatusHandlerDefault); //////////////////////////////////////////////////////////////////////////////////////////////////////////////// @@ -292,13 +292,13 @@ public: }; TSyncRunner(NActors::TActorSystem *system, TConfiguration *conf); - NActors::TActorId NotifyID() const; + NActors::TActorId NotifyID() const; // returns pair<Id, Status> TReturnValue Run(const NActors::TActorContext &ctx, TAutoPtr<NActors::IActor> actor); private: NActors::TActorSystem *ActorSystem; - NActors::TActorId WorkerID; + NActors::TActorId WorkerID; std::shared_ptr<TSystemEvent> _Event; std::shared_ptr<TReturnValue> ReturnValue; TConfiguration *Conf; diff --git a/ydb/core/blobstorage/ut_vdisk/lib/helpers_impl.h b/ydb/core/blobstorage/ut_vdisk/lib/helpers_impl.h index feb0c4f6665..7f1a87d5e34 100644 --- a/ydb/core/blobstorage/ut_vdisk/lib/helpers_impl.h +++ b/ydb/core/blobstorage/ut_vdisk/lib/helpers_impl.h @@ -11,7 +11,7 @@ namespace NKikimr { class TOneGetActor : public TActorBootstrapped<TOneGetActor<TRequestFunc, TCheckResponseFunc> > { protected: typedef TOneGetActor<TRequestFunc, TCheckResponseFunc> TThis; - const TActorId NotifyID; + const TActorId NotifyID; TRequestFunc RequestFunc; TCheckResponseFunc CheckResponseFunc; @@ -35,7 +35,7 @@ namespace NKikimr { ) public: - TOneGetActor(const TActorId ¬ifyID, TRequestFunc requestFunc, TCheckResponseFunc checkRespFunc) + TOneGetActor(const TActorId ¬ifyID, TRequestFunc requestFunc, TCheckResponseFunc checkRespFunc) : TActorBootstrapped<TOneGetActor>() , NotifyID(notifyID) , RequestFunc(requestFunc) @@ -46,7 +46,7 @@ namespace NKikimr { } // NKikimr template <class TRequestFunc, class TCheckResponseFunc> -NActors::IActor *CreateOneGet(const NActors::TActorId ¬ifyID, TRequestFunc req, TCheckResponseFunc check) { +NActors::IActor *CreateOneGet(const NActors::TActorId ¬ifyID, TRequestFunc req, TCheckResponseFunc check) { return new NKikimr::TOneGetActor<TRequestFunc, TCheckResponseFunc>(notifyID, req, check); } diff --git a/ydb/core/blobstorage/ut_vdisk/lib/prepare.cpp b/ydb/core/blobstorage/ut_vdisk/lib/prepare.cpp index 346849664c5..b2c020c3d21 100644 --- a/ydb/core/blobstorage/ut_vdisk/lib/prepare.cpp +++ b/ydb/core/blobstorage/ut_vdisk/lib/prepare.cpp @@ -158,7 +158,7 @@ void TAllPDisks::ActorSetupCmd(NActors::TActorSystemSetup *setup, ui32 node, pDiskConfig->WriteCacheSwitch = NKikimrBlobStorage::TPDiskConfig::DoNotTouch; TActorSetupCmd pDiskSetup(CreatePDisk(pDiskConfig.Get(), NPDisk::YdbDefaultPDiskSequence, counters), TMailboxType::Revolving, 0); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(inst.PDiskActorID, pDiskSetup)); + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(inst.PDiskActorID, pDiskSetup)); } } @@ -209,7 +209,7 @@ void TAllVDisks::ActorSetupCmd(NActors::TActorSystemSetup *setup, NKikimr::TBlob TVDiskInstance &vdisk = VDisks[i]; if (vdisk.Initialized) { TActorSetupCmd vdiskSetup(CreateVDisk(vdisk.Cfg.Get(), groupInfo, counters), TMailboxType::Revolving, 0); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(vdisk.ActorID, vdiskSetup)); + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(vdisk.ActorID, vdiskSetup)); } } } @@ -294,9 +294,9 @@ void TConfiguration::Prepare(IVDiskSetup *vdiskSetup, bool newPDisks, bool runRe setup1->Executors[3].Reset(new TBasicExecutorPool(3, 8, 20)); setup1->Scheduler.Reset(new TBasicSchedulerThread(TSchedulerConfig(512, 100))); - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); TActorSetupCmd nameserviceSetup(CreateNameserverTable(nameserverTable), TMailboxType::Simple, 0); - setup1->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(nameserviceId, nameserviceSetup)); + setup1->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(nameserviceId, nameserviceSetup)); ui64 initOwnerRound = 1; // setup pdisks @@ -315,7 +315,7 @@ void TConfiguration::Prepare(IVDiskSetup *vdiskSetup, bool newPDisks, bool runRe VDisks->ActorSetupCmd(setup1.Get(), GroupInfo.Get(), Counters); ///////////////////////// LOGGER /////////////////////////////////////////////// - NActors::TActorId loggerActorId = NActors::TActorId(1, "logger"); + NActors::TActorId loggerActorId = NActors::TActorId(1, "logger"); TIntrusivePtr<NActors::NLog::TSettings> logSettings; logSettings.Reset(new NActors::NLog::TSettings(loggerActorId, NKikimrServices::LOGGER, NActors::NLog::PRI_ERROR, NActors::NLog::PRI_DEBUG, 0)); // NOTICE @@ -354,7 +354,7 @@ void TConfiguration::Prepare(IVDiskSetup *vdiskSetup, bool newPDisks, bool runRe NActors::CreateStderrBackend(), Counters->GetSubgroup("logger", "counters")); NActors::TActorSetupCmd loggerActorCmd(loggerActor, NActors::TMailboxType::Simple, 0); - std::pair<NActors::TActorId, NActors::TActorSetupCmd> loggerActorPair(loggerActorId, loggerActorCmd); + std::pair<NActors::TActorId, NActors::TActorSetupCmd> loggerActorPair(loggerActorId, loggerActorCmd); setup1->LocalServices.push_back(loggerActorPair); ////////////////////////////////////////////////////////////////////////////// @@ -369,7 +369,7 @@ void TConfiguration::Prepare(IVDiskSetup *vdiskSetup, bool newPDisks, bool runRe loggerActor->Log(Now(), NKikimr::NLog::PRI_NOTICE, NActorsServices::TEST, "Monitoring settings set up"); ////////////////////////////////////////////////////////////////////////////// - TIntrusivePtr<NScheme::TTypeRegistry> typeRegistry(new NScheme::TKikimrTypeRegistry()); + TIntrusivePtr<NScheme::TTypeRegistry> typeRegistry(new NScheme::TKikimrTypeRegistry()); AppData.reset(new NKikimr::TAppData(0, 1, 2, 3, TMap<TString, ui32>(), typeRegistry.Get(), nullptr, nullptr, &KikimrShouldContinue)); AppData->Counters = Counters; diff --git a/ydb/core/blobstorage/ut_vdisk/lib/prepare.h b/ydb/core/blobstorage/ut_vdisk/lib/prepare.h index 2bae0147bb5..fc5da8d3421 100644 --- a/ydb/core/blobstorage/ut_vdisk/lib/prepare.h +++ b/ydb/core/blobstorage/ut_vdisk/lib/prepare.h @@ -45,7 +45,7 @@ private: // TOnePDisk ////////////////////////////////////////////////////////////////////////////////////// struct TOnePDisk { - NActors::TActorId PDiskActorID; + NActors::TActorId PDiskActorID; const ui32 PDiskID = 0; ui64 PDiskGuid = 0; const TString Filename; @@ -87,7 +87,7 @@ struct IVDiskSetup; class TAllVDisks { public: struct TVDiskInstance { - NActors::TActorId ActorID; + NActors::TActorId ActorID; NKikimr::TVDiskID VDiskID; bool Initialized = false; TIntrusivePtr<NKikimr::TVDiskConfig> Cfg; diff --git a/ydb/core/blobstorage/ut_vdisk/lib/test_brokendevice.cpp b/ydb/core/blobstorage/ut_vdisk/lib/test_brokendevice.cpp index 333fae2e7a8..74bfddbcab8 100644 --- a/ydb/core/blobstorage/ut_vdisk/lib/test_brokendevice.cpp +++ b/ydb/core/blobstorage/ut_vdisk/lib/test_brokendevice.cpp @@ -17,7 +17,7 @@ protected: private: const TAllVDisks::TVDiskInstance VDiskInfo; IDataSetPtr DataSetPtr; - const TActorId PDiskId; + const TActorId PDiskId; TAutoPtr<IDataSet::TIterator> It; void Send(const TActorContext &ctx) { @@ -27,7 +27,7 @@ protected: } public: - TSender(const TAllVDisks::TVDiskInstance vdiskInfo, IDataSetPtr dataSetPtr, const TActorId &pdiskId) + TSender(const TAllVDisks::TVDiskInstance vdiskInfo, IDataSetPtr dataSetPtr, const TActorId &pdiskId) : VDiskInfo(vdiskInfo) , DataSetPtr(dataSetPtr) , PDiskId(pdiskId) diff --git a/ydb/core/blobstorage/ut_vdisk/lib/test_huge.cpp b/ydb/core/blobstorage/ut_vdisk/lib/test_huge.cpp index 3427c3c3a7a..7364483b0c5 100644 --- a/ydb/core/blobstorage/ut_vdisk/lib/test_huge.cpp +++ b/ydb/core/blobstorage/ut_vdisk/lib/test_huge.cpp @@ -20,7 +20,7 @@ public: // returns true if finished virtual bool GenerateTask(const TActorContext &ctx) = 0; virtual void Handle(TEvHullLogHugeBlob::TPtr &ev, const TActorContext &ctx) = 0; - virtual void SetHugeKeeperId(const TActorId &hugeKeeperId) { + virtual void SetHugeKeeperId(const TActorId &hugeKeeperId) { HugeKeeperId = hugeKeeperId; } @@ -34,7 +34,7 @@ public: protected: const TAllVDisks::TVDiskInstance &VDiskInstance; - TActorId HugeKeeperId; + TActorId HugeKeeperId; }; ///////////////////////////////////////////////////////////////////////////////////////////////////////// @@ -57,7 +57,7 @@ public: TString abcdefghkj(CreateData("abcdefghkj", minHugeBlobSize, true)); const TLogoBlobID logoBlobId(0, 1, 10, 0, abcdefghkj.size(), 0, 1); ctx.Send(HugeKeeperId, - new TEvHullWriteHugeBlob(TActorId(), 0, logoBlobId, TIngress(), + new TEvHullWriteHugeBlob(TActorId(), 0, logoBlobId, TIngress(), TRope(abcdefghkj), false, NKikimrBlobStorage::EPutHandleClass::AsyncBlob, std::make_unique<TEvBlobStorage::TEvVPutResult>())); @@ -97,10 +97,10 @@ struct THugeModuleContext { TIntrusivePtr<NKikimr::TVDiskConfig> Config; TVDiskContextPtr VCtx; TPDiskCtxPtr PDiskCtx; - TActorId MainID; - TActorId LoggerID; - TActorId LogCutterID; - TActorId HugeKeeperID; + TActorId MainID; + TActorId LoggerID; + TActorId LogCutterID; + TActorId HugeKeeperID; THugeModuleContext(TConfiguration *conf) : Conf(conf) diff --git a/ydb/core/blobstorage/ut_vdisk/lib/test_repl.cpp b/ydb/core/blobstorage/ut_vdisk/lib/test_repl.cpp index cf98feae9fe..ae4288ab25b 100644 --- a/ydb/core/blobstorage/ut_vdisk/lib/test_repl.cpp +++ b/ydb/core/blobstorage/ut_vdisk/lib/test_repl.cpp @@ -175,7 +175,7 @@ void TReadUntilSuccess::operator ()(TConfiguration *conf) { class TVDiskReplProxyReaderActor : public TActorBootstrapped<TVDiskReplProxyReaderActor> { private: - const TActorId NotifyID; + const TActorId NotifyID; const TAllVDisks::TVDiskInstance VDiskInfo; TConfiguration *Conf; TVDiskContextPtr VCtx; @@ -183,7 +183,7 @@ private: TVDiskProxyPtr Proxy; IDataSet *DataSet; TAutoPtr<TExpectedSet> ExpectedSetPtr; - TActorId QueueId; + TActorId QueueId; bool Running = false; /* void Put(const NKikimr::TLogoBlobID &id, NKikimrProto::EReplyStatus status, const TString &data); @@ -266,7 +266,7 @@ private: ) public: - TVDiskReplProxyReaderActor(TConfiguration *conf, const TActorId ¬ifyID, + TVDiskReplProxyReaderActor(TConfiguration *conf, const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, TAutoPtr<TExpectedSet> expSetPtr, IDataSet *dataSet) : TActorBootstrapped<TVDiskReplProxyReaderActor>() @@ -280,7 +280,7 @@ public: {} }; -IActor *CreateVDiskReplProxyReader(TConfiguration *conf, const TActorId ¬ifyID, +IActor *CreateVDiskReplProxyReader(TConfiguration *conf, const TActorId ¬ifyID, const TAllVDisks::TVDiskInstance &vdiskInfo, TAutoPtr<TExpectedSet> expSetPtr, IDataSet *dataSet) { return new TVDiskReplProxyReaderActor(conf, notifyID, vdiskInfo, expSetPtr, dataSet); diff --git a/ydb/core/blobstorage/ut_vdisk/lib/test_synclog.cpp b/ydb/core/blobstorage/ut_vdisk/lib/test_synclog.cpp index 162a706372a..b5608c382d2 100644 --- a/ydb/core/blobstorage/ut_vdisk/lib/test_synclog.cpp +++ b/ydb/core/blobstorage/ut_vdisk/lib/test_synclog.cpp @@ -24,8 +24,8 @@ public: TPDiskCtxPtr PDiskCtx; TIntrusivePtr<TLsnMngr> LsnMngr; TVDiskID SelfVDiskId; - TActorId LoggerId; - TActorId SyncLogId; + TActorId LoggerId; + TActorId SyncLogId; }; /////////////////////////////////////////////////////////////////////////////////////////////// @@ -35,7 +35,7 @@ class TDataWriterActor : public TActorBootstrapped<TDataWriterActor> { friend class TActorBootstrapped<TDataWriterActor>; const TDuration Period = TDuration::MilliSeconds(100); std::shared_ptr<TTestContext> TestCtx; - TActorId ParentId; + TActorId ParentId; ui32 Generation = 0; ui32 Iterations = 10; @@ -153,8 +153,8 @@ class TSyncLogTestWriteActor : public TActorBootstrapped<TSyncLogTestWriteActor> std::shared_ptr<TTestContext> TestCtx = std::make_shared<TTestContext>(); TVDiskContextPtr VCtx; TIntrusivePtr<TVDiskConfig> VDiskConfig; - TActorId LogCutterId; - TActorId DataWriterId; + TActorId LogCutterId; + TActorId DataWriterId; TIntrusivePtr<NKikimr::TDb> Db; diff --git a/ydb/core/blobstorage/ut_vdisk/mon_reregister_ut.cpp b/ydb/core/blobstorage/ut_vdisk/mon_reregister_ut.cpp index 89e9b203db0..36a2eca81f8 100644 --- a/ydb/core/blobstorage/ut_vdisk/mon_reregister_ut.cpp +++ b/ydb/core/blobstorage/ut_vdisk/mon_reregister_ut.cpp @@ -11,7 +11,7 @@ #include <ydb/core/scheme_types/scheme_type_registry.h> #include <ydb/core/scheme/scheme_type_registry.h> #include <ydb/library/yql/minikql/mkql_function_registry.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/system/event.h> #include <ydb/core/blobstorage/ut_vdisk/lib/astest.h> #include <ydb/core/blobstorage/ut_vdisk/lib/http_client.h> @@ -30,7 +30,7 @@ Y_UNIT_TEST_SUITE(TMonitoring) { class TMyWorker : public TActorBootstrapped<TMyWorker> { private: const unsigned Incarnation; - const TActorId ParentId; + const TActorId ParentId; friend class TActorBootstrapped<TMyWorker>; @@ -73,7 +73,7 @@ Y_UNIT_TEST_SUITE(TMonitoring) { ) public: - TMyWorker(unsigned incarnation, const TActorId &parentId) + TMyWorker(unsigned incarnation, const TActorId &parentId) : TActorBootstrapped<TThis>() , Incarnation(incarnation) , ParentId(parentId) @@ -87,7 +87,7 @@ Y_UNIT_TEST_SUITE(TMonitoring) { //////////////////////////////////////////////////////////////////// class TMyTest : public TActorBootstrapped<TMyTest> { std::shared_ptr<TTestWithActorSystem> Env; - TActorId WorkerId; + TActorId WorkerId; friend class TActorBootstrapped<TMyTest>; diff --git a/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubis.cpp b/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubis.cpp index d70fcef8f03..2ea94df9a89 100644 --- a/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubis.cpp +++ b/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubis.cpp @@ -34,8 +34,8 @@ namespace NKikimr { TIntrusivePtr<THullCtx> HullCtx; TQueueActorMapPtr QueueActorMapPtr; TActiveActors ActiveActors; - const TActorId ParentId; - const TActorId SkeletonId; + const TActorId ParentId; + const TActorId SkeletonId; TLogoBlobID Pos; // start position TLogoBlobsSnapshot LogoBlobsSnap; TBarriersSnapshot BarriersSnap; @@ -244,8 +244,8 @@ namespace NKikimr { TAnubisQuantumActor(const TIntrusivePtr<THullCtx> &hullCtx, const TQueueActorMapPtr &queueActorMapPtr, - const TActorId &parentId, - const TActorId &skeletonId, + const TActorId &parentId, + const TActorId &skeletonId, const TLogoBlobID &pos, TLogoBlobsSnapshot &&logoBlobsSnap, TBarriersSnapshot &&barriersSnap, @@ -273,8 +273,8 @@ namespace NKikimr { // mapping from TVDiskIdShort to proxy TQueueActorMapPtr QueueActorMapPtr; TActiveActors ActiveActors; - const TActorId ParentId; - const TActorId SkeletonId; + const TActorId ParentId; + const TActorId SkeletonId; const ui32 ReplInterconnectChannel; const ui64 AnubisOsirisMaxInFly; TLogoBlobID Pos; @@ -376,8 +376,8 @@ namespace NKikimr { TAnubis(const TIntrusivePtr<THullCtx> &hullCtx, const TIntrusivePtr<TBlobStorageGroupInfo> &ginfo, - const TActorId &parentId, - const TActorId &skeletonId, + const TActorId &parentId, + const TActorId &skeletonId, ui32 replInterconnectChannel, ui64 anubisOsirisMaxInFly) : TActorBootstrapped<TAnubis>() @@ -395,8 +395,8 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// IActor* CreateAnubis(const TIntrusivePtr<THullCtx> &hullCtx, const TIntrusivePtr<TBlobStorageGroupInfo> &ginfo, - const TActorId &parentId, - const TActorId &skeletonId, + const TActorId &parentId, + const TActorId &skeletonId, ui32 replInterconnectChannel, ui64 anubisOsirisMaxInFly) { return new TAnubis(hullCtx, ginfo, parentId, skeletonId, replInterconnectChannel, anubisOsirisMaxInFly); diff --git a/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubis.h b/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubis.h index 829d31bf0f0..ca6ec2d9e1f 100644 --- a/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubis.h +++ b/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubis.h @@ -66,8 +66,8 @@ namespace NKikimr { struct THullCtx; IActor* CreateAnubis(const TIntrusivePtr<THullCtx> &hullCtx, const TIntrusivePtr<TBlobStorageGroupInfo> &ginfo, - const TActorId &parentId, - const TActorId &skeletonId, + const TActorId &parentId, + const TActorId &skeletonId, ui32 replInterconnectChannel, ui64 anubisOsirisMaxInFly); diff --git a/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubisfinder.cpp b/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubisfinder.cpp index 3fe9d142c13..a07206a22b2 100644 --- a/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubisfinder.cpp +++ b/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubisfinder.cpp @@ -127,7 +127,7 @@ namespace NKikimr { class TAnubisCandidatesFinderActor : public TActorBootstrapped<TAnubisCandidatesFinderActor> { TIntrusivePtr<THullCtx> HullCtx; TActiveActors ActiveActors; - const TActorId ParentId; + const TActorId ParentId; TLogoBlobID Pos; // start position TLogoBlobsSnapshot LogoBlobsSnap; TBarriersSnapshot BarriersSnap; @@ -152,7 +152,7 @@ namespace NKikimr { } TAnubisCandidatesFinderActor(const TIntrusivePtr<THullCtx> &hullCtx, - const TActorId &parentId, + const TActorId &parentId, const TLogoBlobID &pos, TLogoBlobsSnapshot &&logoBlobsSnap, TBarriersSnapshot &&barriersSnap) @@ -170,7 +170,7 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// IActor *CreateAnubisCandidatesFinder( const TIntrusivePtr<THullCtx> &hullCtx, - const TActorId &parentId, + const TActorId &parentId, const TLogoBlobID &pos, TLogoBlobsSnapshot &&logoBlobsSnap, TBarriersSnapshot &&barriersSnap) { diff --git a/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubisfinder.h b/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubisfinder.h index d09cf30bfec..b09fd9d50e8 100644 --- a/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubisfinder.h +++ b/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubisfinder.h @@ -82,7 +82,7 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// IActor *CreateAnubisCandidatesFinder( const TIntrusivePtr<THullCtx> &hullCtx, - const TActorId &parentId, + const TActorId &parentId, const TLogoBlobID &pos, TLogoBlobsSnapshot &&logoBlobsSnap, TBarriersSnapshot &&barriersSnap); diff --git a/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubisproxy.cpp b/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubisproxy.cpp index ab3586109f3..cdb854a099b 100644 --- a/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubisproxy.cpp +++ b/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubisproxy.cpp @@ -3,7 +3,7 @@ #include <ydb/core/blobstorage/backpressure/queue_backpressure_client.h> #include <ydb/core/blobstorage/base/utility.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <library/cpp/actors/core/mon.h> #include <ydb/core/protos/blobstorage.pb.h> @@ -22,9 +22,9 @@ namespace NKikimr { TVDiskID TargetVDiskId; TVDiskID SelfVDiskId; TActiveActors ActiveActors; - TActorId CliId; // backpressure client id + TActorId CliId; // backpressure client id std::unique_ptr<IActor> Actor; // backpressure client actor - TActorId RequestFrom; + TActorId RequestFrom; friend class TActorBootstrapped<TAnubisProxyActor>; @@ -37,7 +37,7 @@ namespace NKikimr { void Handle(TEvAnubisVGet::TPtr &ev, const TActorContext &ctx) { using TEvVGet = TEvBlobStorage::TEvVGet; - Y_VERIFY(RequestFrom == TActorId()); + Y_VERIFY(RequestFrom == TActorId()); const auto eclass = NKikimrBlobStorage::EGetHandleClass::AsyncRead; auto msg = TEvVGet::CreateExtremeIndexQuery(TargetVDiskId, TInstant::Max(), eclass); msg->Record.SetSuppressBarrierCheck(true); @@ -50,7 +50,7 @@ namespace NKikimr { } void Handle(TEvBlobStorage::TEvVGetResult::TPtr &ev, const TActorContext &ctx) { - Y_VERIFY(RequestFrom != TActorId()); + Y_VERIFY(RequestFrom != TActorId()); // check for RACE and update status if required NKikimrBlobStorage::TEvVGetResult &record = ev->Get()->Record; @@ -64,7 +64,7 @@ namespace NKikimr { // reply with result ctx.Send(RequestFrom, new TEvAnubisVGetResult(TargetVDiskIdShort, ev)); - RequestFrom = TActorId(); + RequestFrom = TActorId(); } void HandlePoison(TEvents::TEvPoisonPill::TPtr &ev, const TActorContext &ctx) { diff --git a/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubisrunner.cpp b/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubisrunner.cpp index f334ff3ac1b..f1bbfa34c64 100644 --- a/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubisrunner.cpp +++ b/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubisrunner.cpp @@ -4,7 +4,7 @@ #include <ydb/core/blobstorage/vdisk/common/vdisk_events.h> #include <ydb/core/blobstorage/groupinfo/blobstorage_groupinfo_sets.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <library/cpp/actors/core/mon.h> using namespace NKikimrServices; @@ -17,15 +17,15 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// class TAnubisRunnerHttpInfoActor : public TActorBootstrapped<TAnubisRunnerHttpInfoActor> { NMon::TEvHttpInfo::TPtr Ev; - const TActorId ReplyId; - const TActorId NotifyId; - const TActorId AnubisId; + const TActorId ReplyId; + const TActorId NotifyId; + const TActorId AnubisId; const TString CurRunnerState; friend class TActorBootstrapped<TAnubisRunnerHttpInfoActor>; void Bootstrap(const TActorContext &ctx) { - if (AnubisId == TActorId()) { + if (AnubisId == TActorId()) { RenderHtmlAndReply(ctx, TString()); } else { Become(&TThis::StateFunc); @@ -86,8 +86,8 @@ namespace NKikimr { TAnubisRunnerHttpInfoActor( NMon::TEvHttpInfo::TPtr &ev, - const TActorId ¬ifyId, - const TActorId &anubisId, + const TActorId ¬ifyId, + const TActorId &anubisId, const TString &curRunnerState) : TActorBootstrapped<TAnubisRunnerHttpInfoActor>() , Ev(ev) @@ -147,7 +147,7 @@ namespace NKikimr { TIntrusivePtr<TBlobStorageGroupInfo> GInfo; TQuorumForAnubisTracker QuorumTracker; TActiveActors ActiveActors; - TActorId AnubisId; + TActorId AnubisId; bool RetryAnubis = true; bool AnubisScheduled = false; @@ -176,7 +176,7 @@ namespace NKikimr { void Handle(TEvAnubisDone::TPtr &ev, const TActorContext &ctx) { ActiveActors.Erase(ev->Sender); - AnubisId = TActorId(); + AnubisId = TActorId(); RetryAnubis = ev->Get()->Issues.HaveProblems(); // check if need to schedule Anubis later on if (RetryAnubis && !AnubisScheduled) { @@ -218,9 +218,9 @@ namespace NKikimr { if (AnubisId) { ActiveActors.Erase(AnubisId); ctx.Send(AnubisId, new TEvents::TEvPoisonPill()); - AnubisId = TActorId(); + AnubisId = TActorId(); RunAnubis(ctx); - Y_VERIFY(AnubisId != TActorId()); + Y_VERIFY(AnubisId != TActorId()); } } diff --git a/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubisrunner.h b/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubisrunner.h index d72ae5bd946..c2429164368 100644 --- a/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubisrunner.h +++ b/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_anubisrunner.h @@ -48,12 +48,12 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// struct TAnubisCtx { TIntrusivePtr<THullCtx> HullCtx; - TActorId SkeletonId; + TActorId SkeletonId; ui32 ReplInterconnectChannel; ui64 AnubisOsirisMaxInFly; TDuration AnubisTimeout; - TAnubisCtx(const TIntrusivePtr<THullCtx> &hullCtx, const TActorId &skeletonId, + TAnubisCtx(const TIntrusivePtr<THullCtx> &hullCtx, const TActorId &skeletonId, ui32 replInterconnectChannel, ui64 anubisOsirisMaxInFly, TDuration anubisTimeout) : HullCtx(hullCtx) , SkeletonId(skeletonId) diff --git a/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_osiris.cpp b/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_osiris.cpp index f29f6b71d68..019c048d491 100644 --- a/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_osiris.cpp +++ b/ydb/core/blobstorage/vdisk/anubis_osiris/blobstorage_osiris.cpp @@ -137,9 +137,9 @@ namespace NKikimr { TIntrusivePtr<THullCtx> HullCtx; const ui64 ConfirmedLsn; - const TActorId NotifyId; - const TActorId ParentId; - const TActorId SkeletonId; + const TActorId NotifyId; + const TActorId ParentId; + const TActorId SkeletonId; std::shared_ptr<TLogoBlobFilterForOsiris> LogoBlobFilter; TSaviour<TLogoBlobFilterForOsiris> Saviour; ui64 InFly = 0; diff --git a/ydb/core/blobstorage/vdisk/anubis_osiris/ya.make b/ydb/core/blobstorage/vdisk/anubis_osiris/ya.make index 08849c4f2e4..e59b3a060ec 100644 --- a/ydb/core/blobstorage/vdisk/anubis_osiris/ya.make +++ b/ydb/core/blobstorage/vdisk/anubis_osiris/ya.make @@ -4,7 +4,7 @@ OWNER(fomichev) PEERDIR( library/cpp/actors/core - library/cpp/monlib/service/pages + library/cpp/monlib/service/pages ydb/core/base ydb/core/blobstorage/backpressure ydb/core/blobstorage/base diff --git a/ydb/core/blobstorage/vdisk/common/blobstorage_dblogcutter.cpp b/ydb/core/blobstorage/vdisk/common/blobstorage_dblogcutter.cpp index daadf638464..5f176b61b75 100644 --- a/ydb/core/blobstorage/vdisk/common/blobstorage_dblogcutter.cpp +++ b/ydb/core/blobstorage/vdisk/common/blobstorage_dblogcutter.cpp @@ -6,7 +6,7 @@ #include "vdisk_lsnmngr.h" #include <ydb/core/blobstorage/base/utility.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> namespace NKikimr { diff --git a/ydb/core/blobstorage/vdisk/common/blobstorage_dblogcutter.h b/ydb/core/blobstorage/vdisk/common/blobstorage_dblogcutter.h index 060c1d2e5ac..b98c5061259 100644 --- a/ydb/core/blobstorage/vdisk/common/blobstorage_dblogcutter.h +++ b/ydb/core/blobstorage/vdisk/common/blobstorage_dblogcutter.h @@ -44,7 +44,7 @@ namespace NKikimr { TPDiskCtxPtr PDiskCtx; TIntrusivePtr<TLsnMngr> LsnMngr; TIntrusivePtr<TVDiskConfig> Config; - TActorId LoggerId; + TActorId LoggerId; }; IActor* CreateRecoveryLogCutter(TLogCutterCtx &&logCutterCtx); diff --git a/ydb/core/blobstorage/vdisk/common/blobstorage_status.cpp b/ydb/core/blobstorage/vdisk/common/blobstorage_status.cpp index df588b96d7a..4d770a40370 100644 --- a/ydb/core/blobstorage/vdisk/common/blobstorage_status.cpp +++ b/ydb/core/blobstorage/vdisk/common/blobstorage_status.cpp @@ -11,15 +11,15 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// class TStatusRequestHandler : public TActorBootstrapped<TStatusRequestHandler> { TIntrusivePtr<TVDiskContext> VCtx; - const TActorId SkeletonId; - const TActorId SyncerId; - const TActorId SyncLogId; + const TActorId SkeletonId; + const TActorId SyncerId; + const TActorId SyncLogId; std::shared_ptr<NMonGroup::TVDiskIFaceGroup> IFaceMonGroup; const TVDiskID SelfVDiskId; const ui64 IncarnationGuid; const TIntrusivePtr<TBlobStorageGroupInfo> GroupInfo; TEvBlobStorage::TEvVStatus::TPtr Ev; - const TActorId NotifyId; + const TActorId NotifyId; const TInstant Now; const bool ReplDone; unsigned Counter; @@ -54,8 +54,8 @@ namespace NKikimr { Become(&TThis::StateFunc); } - void SendLocalStatusRequest(const TActorContext &ctx, const TActorId &actor) { - if (actor != TActorId()) { + void SendLocalStatusRequest(const TActorContext &ctx, const TActorId &actor) { + if (actor != TActorId()) { ctx.Send(actor, new TEvLocalStatus()); Counter++; } @@ -93,15 +93,15 @@ namespace NKikimr { TStatusRequestHandler( const TIntrusivePtr<TVDiskContext> &vctx, - const TActorId &skeletonId, - const TActorId &syncerId, - const TActorId &syncLogId, + const TActorId &skeletonId, + const TActorId &syncerId, + const TActorId &syncLogId, const std::shared_ptr<NMonGroup::TVDiskIFaceGroup> &ifaceMonGroup, const TVDiskID selfVDiskId, const ui64 incarnationGuid, const TIntrusivePtr<TBlobStorageGroupInfo>& groupInfo, TEvBlobStorage::TEvVStatus::TPtr &ev, - const TActorId ¬ifyId, + const TActorId ¬ifyId, const TInstant &now, bool replDone) : TActorBootstrapped<TStatusRequestHandler>() @@ -123,15 +123,15 @@ namespace NKikimr { IActor *CreateStatusRequestHandler( const TIntrusivePtr<TVDiskContext> &vctx, - const TActorId &skeletonId, - const TActorId &syncerId, - const TActorId &syncLogId, + const TActorId &skeletonId, + const TActorId &syncerId, + const TActorId &syncLogId, const std::shared_ptr<NMonGroup::TVDiskIFaceGroup> &ifaceMonGroup, const TVDiskID selfVDiskId, const ui64 incarnationGuid, const TIntrusivePtr<TBlobStorageGroupInfo>& groupInfo, TEvBlobStorage::TEvVStatus::TPtr &ev, - const TActorId ¬ifyId, + const TActorId ¬ifyId, const TInstant &now, bool replDone) { return new TStatusRequestHandler(vctx, skeletonId, syncerId, syncLogId, ifaceMonGroup, selfVDiskId, diff --git a/ydb/core/blobstorage/vdisk/common/blobstorage_status.h b/ydb/core/blobstorage/vdisk/common/blobstorage_status.h index 691c406578c..413aab478ee 100644 --- a/ydb/core/blobstorage/vdisk/common/blobstorage_status.h +++ b/ydb/core/blobstorage/vdisk/common/blobstorage_status.h @@ -19,15 +19,15 @@ namespace NKikimr { IActor *CreateStatusRequestHandler( const TIntrusivePtr<TVDiskContext> &vctx, - const TActorId &skeletonId, - const TActorId &syncerId, - const TActorId &syncLogId, + const TActorId &skeletonId, + const TActorId &syncerId, + const TActorId &syncLogId, const std::shared_ptr<NMonGroup::TVDiskIFaceGroup> &ifaceMonGroup, const TVDiskID selfVDiskId, const ui64 incarnationGuid, const TIntrusivePtr<TBlobStorageGroupInfo>& groupInfo, TEvBlobStorage::TEvVStatus::TPtr &ev, - const TActorId ¬ifyId, + const TActorId ¬ifyId, const TInstant &now, bool replDone); diff --git a/ydb/core/blobstorage/vdisk/common/memusage.h b/ydb/core/blobstorage/vdisk/common/memusage.h index 79f459ac49f..b045c1e3d50 100644 --- a/ydb/core/blobstorage/vdisk/common/memusage.h +++ b/ydb/core/blobstorage/vdisk/common/memusage.h @@ -2,7 +2,7 @@ #include "defs.h" -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <util/generic/buffer.h> #include <util/generic/string.h> diff --git a/ydb/core/blobstorage/vdisk/common/sublog.h b/ydb/core/blobstorage/vdisk/common/sublog.h index 786d6e8ca8a..d9d5310682e 100644 --- a/ydb/core/blobstorage/vdisk/common/sublog.h +++ b/ydb/core/blobstorage/vdisk/common/sublog.h @@ -69,7 +69,7 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////// class TSublogLineHolder { public: - TSublogLineHolder(const TActorId &aid, const TActorContext &ctx) + TSublogLineHolder(const TActorId &aid, const TActorContext &ctx) : Aid(aid) , Ctx(ctx) , Ev(std::make_unique<TEvSublogLine>()) @@ -83,7 +83,7 @@ namespace NKikimr { return Ev->Stream; } private: - TActorId Aid; + TActorId Aid; const TActorContext &Ctx; std::unique_ptr<TEvSublogLine> Ev; }; diff --git a/ydb/core/blobstorage/vdisk/common/vdisk_config.h b/ydb/core/blobstorage/vdisk/common/vdisk_config.h index 1e48afcef85..ff4bb4f0d61 100644 --- a/ydb/core/blobstorage/vdisk/common/vdisk_config.h +++ b/ydb/core/blobstorage/vdisk/common/vdisk_config.h @@ -18,7 +18,7 @@ namespace NKikimr { // Base info provided while constructing TVDiskConfig struct TBaseInfo { TVDiskIdShort VDiskIdShort; - TActorId PDiskActorID; + TActorId PDiskActorID; ui64 InitOwnerRound = 0; ui64 PDiskGuid = 0; ui32 PDiskId = 0; @@ -42,7 +42,7 @@ namespace NKikimr { TBaseInfo( const TVDiskIdShort &vDiskIdShort, - const TActorId &pDiskActorId, + const TActorId &pDiskActorId, ui64 pDiskGuid, ui32 pdiskId, TPDiskCategory::EDeviceType deviceType, diff --git a/ydb/core/blobstorage/vdisk/common/vdisk_context.cpp b/ydb/core/blobstorage/vdisk/common/vdisk_context.cpp index 330afea4091..0635f0ec9e8 100644 --- a/ydb/core/blobstorage/vdisk/common/vdisk_context.cpp +++ b/ydb/core/blobstorage/vdisk/common/vdisk_context.cpp @@ -20,7 +20,7 @@ namespace NKikimr { } TVDiskContext::TVDiskContext( - const TActorId &vdiskActorId, + const TActorId &vdiskActorId, std::shared_ptr<TBlobStorageGroupInfo::TTopology> top, const TIntrusivePtr<NMonitoring::TDynamicCounters>& vdiskCounters, const TVDiskID &selfVDisk, diff --git a/ydb/core/blobstorage/vdisk/common/vdisk_context.h b/ydb/core/blobstorage/vdisk/common/vdisk_context.h index 58dff0a4ce6..7bb7dfd6704 100644 --- a/ydb/core/blobstorage/vdisk/common/vdisk_context.h +++ b/ydb/core/blobstorage/vdisk/common/vdisk_context.h @@ -31,7 +31,7 @@ namespace NKikimr { class TVDiskContext : public TBSProxyContext { public: // ActorId of the main VDisk actor (currently ActorId of SkeletonFront) - const TActorId VDiskActorId; + const TActorId VDiskActorId; const std::shared_ptr<TBlobStorageGroupInfo::TTopology> Top; const TIntrusivePtr<NMonitoring::TDynamicCounters> VDiskCounters; const TIntrusivePtr<NMonitoring::TDynamicCounters> VDiskMemCounters; @@ -78,7 +78,7 @@ namespace NKikimr { public: TVDiskContext( - const TActorId &vdiskActorId, + const TActorId &vdiskActorId, std::shared_ptr<TBlobStorageGroupInfo::TTopology> top, const TIntrusivePtr<NMonitoring::TDynamicCounters>& vdiskCounters, const TVDiskID &selfVDisk, diff --git a/ydb/core/blobstorage/vdisk/common/vdisk_events.h b/ydb/core/blobstorage/vdisk/common/vdisk_events.h index 0407f7c16c3..47fbd959c96 100644 --- a/ydb/core/blobstorage/vdisk/common/vdisk_events.h +++ b/ydb/core/blobstorage/vdisk/common/vdisk_events.h @@ -211,7 +211,7 @@ namespace NKikimr { template <class TClientId> struct TWindowStatus { TClientId ClientId; - TActorId ActorId; + TActorId ActorId; EStatus Status; bool Notify; ui64 ActualWindowSize; @@ -230,7 +230,7 @@ namespace NKikimr { , FailedMsgId() {} - void Set(const TClientId &clientId, const TActorId& actorId, EStatus status, bool notify, + void Set(const TClientId &clientId, const TActorId& actorId, EStatus status, bool notify, ui64 actualWindowSize, ui64 maxWindowSize, const TMessageId &expectedMsgId, const TMessageId &failedMsgId) { ClientId = clientId; diff --git a/ydb/core/blobstorage/vdisk/common/vdisk_histogram_latency.h b/ydb/core/blobstorage/vdisk/common/vdisk_histogram_latency.h index 04e43edeb52..d0388fac73f 100644 --- a/ydb/core/blobstorage/vdisk/common/vdisk_histogram_latency.h +++ b/ydb/core/blobstorage/vdisk/common/vdisk_histogram_latency.h @@ -4,8 +4,8 @@ #include <ydb/core/base/blobstorage.h> -#include <library/cpp/monlib/dynamic_counters/percentile/percentile.h> -#include <library/cpp/monlib/metrics/histogram_collector.h> +#include <library/cpp/monlib/dynamic_counters/percentile/percentile.h> +#include <library/cpp/monlib/metrics/histogram_collector.h> namespace NKikimr { namespace NVDiskMon { diff --git a/ydb/core/blobstorage/vdisk/common/vdisk_hulllogctx.h b/ydb/core/blobstorage/vdisk/common/vdisk_hulllogctx.h index f895d2e327c..06a06930f44 100644 --- a/ydb/core/blobstorage/vdisk/common/vdisk_hulllogctx.h +++ b/ydb/core/blobstorage/vdisk/common/vdisk_hulllogctx.h @@ -9,16 +9,16 @@ namespace NKikimr { struct THullLogCtx { TIntrusivePtr<TVDiskContext> VCtx; TPDiskCtxPtr PDiskCtx; - TActorId SkeletonId; - TActorId SyncLogId; - TActorId HugeKeeperId; + TActorId SkeletonId; + TActorId SyncLogId; + TActorId HugeKeeperId; THullLogCtx( const TIntrusivePtr<TVDiskContext> &vctx, const TPDiskCtxPtr &pdiskCtx, - TActorId skeletonId, - TActorId syncLogId, - TActorId hugeKeeperId) + TActorId skeletonId, + TActorId syncLogId, + TActorId hugeKeeperId) : VCtx(vctx) , PDiskCtx(pdiskCtx) , SkeletonId(skeletonId) diff --git a/ydb/core/blobstorage/vdisk/common/vdisk_pdiskctx.h b/ydb/core/blobstorage/vdisk/common/vdisk_pdiskctx.h index f16169c2f66..369118e5f37 100644 --- a/ydb/core/blobstorage/vdisk/common/vdisk_pdiskctx.h +++ b/ydb/core/blobstorage/vdisk/common/vdisk_pdiskctx.h @@ -17,9 +17,9 @@ namespace NKikimr { // PDisk related constants const TIntrusivePtr<TPDiskParams> Dsk; // PDisk Actor id - const TActorId PDiskId; + const TActorId PDiskId; - TPDiskCtx(TIntrusivePtr<TPDiskParams> dsk, TActorId pDiskId) + TPDiskCtx(TIntrusivePtr<TPDiskParams> dsk, TActorId pDiskId) : Dsk(std::move(dsk)) , PDiskId(pDiskId) {} diff --git a/ydb/core/blobstorage/vdisk/common/vdisk_recoverylogwriter.cpp b/ydb/core/blobstorage/vdisk/common/vdisk_recoverylogwriter.cpp index d9c314e64f9..f0ec9c15e40 100644 --- a/ydb/core/blobstorage/vdisk/common/vdisk_recoverylogwriter.cpp +++ b/ydb/core/blobstorage/vdisk/common/vdisk_recoverylogwriter.cpp @@ -85,8 +85,8 @@ LWTRACE_USING(BLOBSTORAGE_PROVIDER); typedef TPriorityQueue<TQueueItem, TVector<TQueueItem>, TQueueItem::TGreater> TQueueType; - const TActorId YardID; - const TActorId SkeletonID; + const TActorId YardID; + const TActorId SkeletonID; const NPDisk::TOwner Owner; const NPDisk::TOwnerRound OwnerRound; ui64 CurSentLsn = 0; @@ -206,7 +206,7 @@ LWTRACE_USING(BLOBSTORAGE_PROVIDER); return NKikimrServices::TActivity::BS_RECOVERY_LOG_WRITER; } - TRecoveryLogWriter(const TActorId &yardID, const TActorId &skeletonID, + TRecoveryLogWriter(const TActorId &yardID, const TActorId &skeletonID, NPDisk::TOwner owner, NPDisk::TOwnerRound ownerRound, ui64 startLsn, TIntrusivePtr<NMonitoring::TDynamicCounters> counters) : TActorBootstrapped<TRecoveryLogWriter>() @@ -221,7 +221,7 @@ LWTRACE_USING(BLOBSTORAGE_PROVIDER); {} }; - IActor* CreateRecoveryLogWriter(const TActorId &yardID, const TActorId &skeletonID, NPDisk::TOwner owner, + IActor* CreateRecoveryLogWriter(const TActorId &yardID, const TActorId &skeletonID, NPDisk::TOwner owner, NPDisk::TOwnerRound ownerRound, ui64 startLsn, TIntrusivePtr<NMonitoring::TDynamicCounters> counters) { return new TRecoveryLogWriter(yardID, skeletonID, owner, ownerRound, startLsn, counters); diff --git a/ydb/core/blobstorage/vdisk/common/vdisk_recoverylogwriter.h b/ydb/core/blobstorage/vdisk/common/vdisk_recoverylogwriter.h index 2bba7c75447..c38ff753d46 100644 --- a/ydb/core/blobstorage/vdisk/common/vdisk_recoverylogwriter.h +++ b/ydb/core/blobstorage/vdisk/common/vdisk_recoverylogwriter.h @@ -6,7 +6,7 @@ namespace NKikimr { // Writing via LogWriter makes proper ordering of Lsns - IActor* CreateRecoveryLogWriter(const TActorId &yardID, const TActorId &skeletonID, NPDisk::TOwner owner, + IActor* CreateRecoveryLogWriter(const TActorId &yardID, const TActorId &skeletonID, NPDisk::TOwner owner, NPDisk::TOwnerRound ownerRound, ui64 startLsn, TIntrusivePtr<NMonitoring::TDynamicCounters> counters); diff --git a/ydb/core/blobstorage/vdisk/common/vdisk_syncneighbors.h b/ydb/core/blobstorage/vdisk/common/vdisk_syncneighbors.h index 89d928905c5..b014da97b04 100644 --- a/ydb/core/blobstorage/vdisk/common/vdisk_syncneighbors.h +++ b/ydb/core/blobstorage/vdisk/common/vdisk_syncneighbors.h @@ -9,7 +9,7 @@ #include <util/generic/vector.h> #include <util/stream/str.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <type_traits> #include <iterator> diff --git a/ydb/core/blobstorage/vdisk/common/ya.make b/ydb/core/blobstorage/vdisk/common/ya.make index 238461b723a..4fb4364ee2a 100644 --- a/ydb/core/blobstorage/vdisk/common/ya.make +++ b/ydb/core/blobstorage/vdisk/common/ya.make @@ -4,7 +4,7 @@ OWNER(g:kikimr) PEERDIR( library/cpp/actors/protos - library/cpp/monlib/service/pages + library/cpp/monlib/service/pages ydb/core/base ydb/core/blobstorage/vdisk/hulldb/base ydb/core/protos diff --git a/ydb/core/blobstorage/vdisk/handoff/handoff_basic.cpp b/ydb/core/blobstorage/vdisk/handoff/handoff_basic.cpp index 6774b447dd3..da29a8822b2 100644 --- a/ydb/core/blobstorage/vdisk/handoff/handoff_basic.cpp +++ b/ydb/core/blobstorage/vdisk/handoff/handoff_basic.cpp @@ -2,7 +2,7 @@ #include <ydb/core/blobstorage/vdisk/common/vdisk_syncneighbors.h> #include <ydb/core/blobstorage/base/utility.h> #include <ydb/core/base/appdata.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> namespace NKikimr { diff --git a/ydb/core/blobstorage/vdisk/handoff/handoff_basic.h b/ydb/core/blobstorage/vdisk/handoff/handoff_basic.h index 688c0504fc5..0a8380a8fcb 100644 --- a/ydb/core/blobstorage/vdisk/handoff/handoff_basic.h +++ b/ydb/core/blobstorage/vdisk/handoff/handoff_basic.h @@ -79,7 +79,7 @@ namespace NKikimr { TAtomic EarlyDiscardCounter; TVDiskID SelfVDiskID; TVDiskID TargetVDiskID; - TActorId ProxyID; + TActorId ProxyID; ui32 MaxBytes; bool Initialized; diff --git a/ydb/core/blobstorage/vdisk/handoff/handoff_delegate.cpp b/ydb/core/blobstorage/vdisk/handoff/handoff_delegate.cpp index fac469e2279..f37aca976ac 100644 --- a/ydb/core/blobstorage/vdisk/handoff/handoff_delegate.cpp +++ b/ydb/core/blobstorage/vdisk/handoff/handoff_delegate.cpp @@ -18,7 +18,7 @@ namespace NKikimr { const TVDiskID SelfVDisk; // FIXME: switch to TVDiskIdShort TIntrusivePtr<TBlobStorageGroupInfo> Info; NHandoff::TProxiesPtr ProxiesPtr; - TActorId MonActorID = {}; + TActorId MonActorID = {}; const THandoffParams Params; bool ProxiesStarted = false; @@ -75,7 +75,7 @@ namespace NKikimr { return ref.Get().Restore(ctx, id, fullDataSize, std::move(data)); } - TActorId THandoffDelegate::GetMonActorID() const { + TActorId THandoffDelegate::GetMonActorID() const { return Fields->MonActorID; } diff --git a/ydb/core/blobstorage/vdisk/handoff/handoff_delegate.h b/ydb/core/blobstorage/vdisk/handoff/handoff_delegate.h index f0ebe89c3a0..eaedc8bb04b 100644 --- a/ydb/core/blobstorage/vdisk/handoff/handoff_delegate.h +++ b/ydb/core/blobstorage/vdisk/handoff/handoff_delegate.h @@ -41,7 +41,7 @@ namespace NKikimr { const TLogoBlobID &id, ui64 fullDataSize, TRope&& data); - TActorId GetMonActorID() const; + TActorId GetMonActorID() const; private: struct TFields; diff --git a/ydb/core/blobstorage/vdisk/handoff/handoff_map.h b/ydb/core/blobstorage/vdisk/handoff/handoff_map.h index 44cb3fbf4ae..9e7480f60fe 100644 --- a/ydb/core/blobstorage/vdisk/handoff/handoff_map.h +++ b/ydb/core/blobstorage/vdisk/handoff/handoff_map.h @@ -112,7 +112,7 @@ namespace NKikimr { // Finish transforming items, shutdown proxy void Finish(const TActorContext &ctx) { // do nothing by default, all work is done in template specialization for logo blobs - Y_VERIFY_DEBUG(ProxyID == TActorId()); + Y_VERIFY_DEBUG(ProxyID == TActorId()); ctx.Send(NotifyID, new TEvHandoffSyncLogFinished(false)); } @@ -121,7 +121,7 @@ namespace NKikimr { const std::shared_ptr<TBlobStorageGroupInfo::TTopology> Top; TIntrusivePtr<THandoffDelegate> HandoffDelegate; const bool RunHandoff; - const TActorId SkeletonId; + const TActorId SkeletonId; // TODO: we can store MoveMap and DelMap in a more compact way // Hints: 1. most of values are zero (we don't want to move or delete) // 2. we can store only 2 bit for every local part, not 8 bits for the whole logoblob key @@ -130,8 +130,8 @@ namespace NKikimr { unsigned Counter; TTransformedItem TrRes; TStat Stat; - TActorId ProxyID; - TActorId NotifyID; + TActorId ProxyID; + TActorId NotifyID; }; @@ -302,7 +302,7 @@ namespace NKikimr { template <> template <class TIterator> - inline TActorId THandoffMap<TKeyLogoBlob, TMemRecLogoBlob>::BuildMap( + inline TActorId THandoffMap<TKeyLogoBlob, TMemRecLogoBlob>::BuildMap( const TActorContext &ctx, const TLevelIndexSnapshot &levelSnap, const TIterator &i, @@ -371,10 +371,10 @@ namespace NKikimr { "THandoffMap: finished: %s", Stat.ToStringRuntimeStat().data())); if (RunHandoff) { - Y_VERIFY_DEBUG(ProxyID != TActorId()); + Y_VERIFY_DEBUG(ProxyID != TActorId()); ctx.Send(ProxyID, new TEvHandoffSyncLogDel()); // i.e. finish } else { - Y_VERIFY_DEBUG(ProxyID == TActorId()); + Y_VERIFY_DEBUG(ProxyID == TActorId()); ctx.Send(NotifyID, new TEvHandoffSyncLogFinished(false)); } } diff --git a/ydb/core/blobstorage/vdisk/handoff/handoff_mon.cpp b/ydb/core/blobstorage/vdisk/handoff/handoff_mon.cpp index be259652eed..5aa0e92348f 100644 --- a/ydb/core/blobstorage/vdisk/handoff/handoff_mon.cpp +++ b/ydb/core/blobstorage/vdisk/handoff/handoff_mon.cpp @@ -29,7 +29,7 @@ namespace NKikimr { typedef NSync::TVDiskNeighbors<TInfo> TCells; - const TActorId ParentId; + const TActorId ParentId; TProxiesPtr ProxiesPtr; NMon::TEvHttpInfo::TPtr Ev; TCells Cells; @@ -114,7 +114,7 @@ namespace NKikimr { return NKikimrServices::TActivity::BS_HANDOFF_MON_REQUEST; } - THandoffMonRequestActor(const TActorId &parentId, + THandoffMonRequestActor(const TActorId &parentId, const TVDiskID &selfVDisk, const std::shared_ptr<TBlobStorageGroupInfo::TTopology> &top, TProxiesPtr proxiesPtr, diff --git a/ydb/core/blobstorage/vdisk/handoff/handoff_synclogproxy.cpp b/ydb/core/blobstorage/vdisk/handoff/handoff_synclogproxy.cpp index 3e27876ff37..4d4f55b88bf 100644 --- a/ydb/core/blobstorage/vdisk/handoff/handoff_synclogproxy.cpp +++ b/ydb/core/blobstorage/vdisk/handoff/handoff_synclogproxy.cpp @@ -14,8 +14,8 @@ namespace NKikimr { }; typedef TDeque<TItem> TQueueType; - const TActorId SkeletonId; - const TActorId NotifyId; + const TActorId SkeletonId; + const TActorId NotifyId; ui64 OrderId = 0; bool Finished = false; TQueueType Queue = {}; @@ -67,8 +67,8 @@ namespace NKikimr { return NKikimrServices::TActivity::BS_HANDOFF_SYNCLOG_PROXY; } - THandoffSyncLogProxy(const TActorId &skeletonId, - const TActorId ¬ifyId) + THandoffSyncLogProxy(const TActorId &skeletonId, + const TActorId ¬ifyId) : TActor<THandoffSyncLogProxy>(&TThis::StateFunc) , SkeletonId(skeletonId) , NotifyId(notifyId) @@ -76,7 +76,7 @@ namespace NKikimr { } }; - IActor* CreateHandoffSyncLogProxy(const TActorId &skeletonId, const TActorId ¬ifyId) { + IActor* CreateHandoffSyncLogProxy(const TActorId &skeletonId, const TActorId ¬ifyId) { return new THandoffSyncLogProxy(skeletonId, notifyId); } diff --git a/ydb/core/blobstorage/vdisk/handoff/handoff_synclogproxy.h b/ydb/core/blobstorage/vdisk/handoff/handoff_synclogproxy.h index 8a690684e12..16d4f8da460 100644 --- a/ydb/core/blobstorage/vdisk/handoff/handoff_synclogproxy.h +++ b/ydb/core/blobstorage/vdisk/handoff/handoff_synclogproxy.h @@ -46,7 +46,7 @@ namespace NKikimr { {} }; - IActor* CreateHandoffSyncLogProxy(const TActorId &skeletonId, const TActorId ¬ifyId); + IActor* CreateHandoffSyncLogProxy(const TActorId &skeletonId, const TActorId ¬ifyId); } // NKikimr diff --git a/ydb/core/blobstorage/vdisk/handoff/ya.make b/ydb/core/blobstorage/vdisk/handoff/ya.make index d1fcadff285..61be4cc530f 100644 --- a/ydb/core/blobstorage/vdisk/handoff/ya.make +++ b/ydb/core/blobstorage/vdisk/handoff/ya.make @@ -4,7 +4,7 @@ OWNER(fomichev) PEERDIR( library/cpp/actors/core - library/cpp/monlib/service/pages + library/cpp/monlib/service/pages ydb/core/base ydb/core/blobstorage/backpressure ydb/core/blobstorage/groupinfo diff --git a/ydb/core/blobstorage/vdisk/huge/blobstorage_hullhuge.cpp b/ydb/core/blobstorage/vdisk/huge/blobstorage_hullhuge.cpp index 3064ddaf5f0..30d2aeece40 100644 --- a/ydb/core/blobstorage/vdisk/huge/blobstorage_hullhuge.cpp +++ b/ydb/core/blobstorage/vdisk/huge/blobstorage_hullhuge.cpp @@ -10,7 +10,7 @@ #include <ydb/core/blobstorage/vdisk/common/vdisk_lsnmngr.h> #include <ydb/core/blobstorage/vdisk/common/blobstorage_dblogcutter.h> #include <ydb/core/blobstorage/vdisk/hulldb/base/blobstorage_blob.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> using namespace NKikimrServices; using namespace NKikimr::NHuge; @@ -148,7 +148,7 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// class THullHugeBlobWriter : public TActorBootstrapped<THullHugeBlobWriter> { std::shared_ptr<THugeKeeperCtx> HugeKeeperCtx; - const TActorId NotifyID; + const TActorId NotifyID; const NHuge::THugeSlot HugeSlot; std::unique_ptr<TEvHullWriteHugeBlob> Item; ui64 WriteId; @@ -224,7 +224,7 @@ namespace NKikimr { THullHugeBlobWriter( std::shared_ptr<THugeKeeperCtx> hugeKeeperCtx, - const TActorId ¬ifyID, + const TActorId ¬ifyID, const NHuge::THugeSlot &hugeSlot, std::unique_ptr<TEvHullWriteHugeBlob> item, ui64 wId) @@ -245,7 +245,7 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// class THullHugeBlobChunkAllocator : public TActorBootstrapped<THullHugeBlobChunkAllocator> { std::shared_ptr<THugeKeeperCtx> HugeKeeperCtx; - const TActorId NotifyID; + const TActorId NotifyID; ui64 Lsn; std::shared_ptr<THullHugeKeeperPersState> Pers; ui32 ChunkId = 0; @@ -336,7 +336,7 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// class THullHugeBlobChunkDestroyer : public TActorBootstrapped<THullHugeBlobChunkDestroyer> { std::shared_ptr<THugeKeeperCtx> HugeKeeperCtx; - const TActorId NotifyID; + const TActorId NotifyID; TVector<ui32> ChunksToFree; const ui64 Lsn; @@ -410,7 +410,7 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// class THullHugeBlobEntryPointSaver : public TActorBootstrapped<THullHugeBlobEntryPointSaver> { std::shared_ptr<THugeKeeperCtx> HugeKeeperCtx; - const TActorId NotifyID; + const TActorId NotifyID; const ui64 EntryPointLsn; const TString Serialized; @@ -599,9 +599,9 @@ namespace NKikimr { TIntrusivePtr<TVDiskContext> vctx, TPDiskCtxPtr pdiskCtx, TIntrusivePtr<TLsnMngr> lsnMngr, - TActorId skeletonId, - TActorId loggerId, - TActorId logCutterId, + TActorId skeletonId, + TActorId loggerId, + TActorId logCutterId, const TString &localRecoveryInfoDbg) : VCtx(std::move(vctx)) , PDiskCtx(std::move(pdiskCtx)) diff --git a/ydb/core/blobstorage/vdisk/huge/blobstorage_hullhuge.h b/ydb/core/blobstorage/vdisk/huge/blobstorage_hullhuge.h index 740ba93533e..66f3c049594 100644 --- a/ydb/core/blobstorage/vdisk/huge/blobstorage_hullhuge.h +++ b/ydb/core/blobstorage/vdisk/huge/blobstorage_hullhuge.h @@ -13,7 +13,7 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// class TEvHullWriteHugeBlob : public TEventLocal<TEvHullWriteHugeBlob, TEvBlobStorage::EvHullWriteHugeBlob>, public TIntrusiveListItem<TEvHullWriteHugeBlob> { public: - const TActorId SenderId; + const TActorId SenderId; const ui64 Cookie; const TLogoBlobID LogoBlobId; const TIngress Ingress; @@ -22,7 +22,7 @@ namespace NKikimr { const NKikimrBlobStorage::EPutHandleClass HandleClass; std::unique_ptr<TEvBlobStorage::TEvVPutResult> Result; - TEvHullWriteHugeBlob(const TActorId &senderId, + TEvHullWriteHugeBlob(const TActorId &senderId, ui64 cookie, const TLogoBlobID &logoBlobId, const TIngress &ingress, @@ -61,7 +61,7 @@ namespace NKikimr { const TIngress Ingress; const TDiskPart HugeBlob; const bool IgnoreBlock; - const TActorId OrigClient; + const TActorId OrigClient; const ui64 OrigCookie; std::unique_ptr<TEvBlobStorage::TEvVPutResult> Result; @@ -70,7 +70,7 @@ namespace NKikimr { const TIngress &ingress, const TDiskPart &hugeBlob, bool ignoreBlock, - const TActorId &origClient, + const TActorId &origClient, ui64 origCookie, std::unique_ptr<TEvBlobStorage::TEvVPutResult> result) : WriteId(writeId) @@ -195,9 +195,9 @@ namespace NKikimr { TIntrusivePtr<TVDiskContext> VCtx; TPDiskCtxPtr PDiskCtx; TIntrusivePtr<TLsnMngr> LsnMngr; - TActorId SkeletonId; - TActorId LoggerId; - TActorId LogCutterId; + TActorId SkeletonId; + TActorId LoggerId; + TActorId LogCutterId; const TString LocalRecoveryInfoDbg; NMonGroup::TLsmHullGroup LsmHullGroup; NMonGroup::TDskOutOfSpaceGroup DskOutOfSpaceGroup; @@ -206,9 +206,9 @@ namespace NKikimr { TIntrusivePtr<TVDiskContext> vctx, TPDiskCtxPtr pdiskCtx, TIntrusivePtr<TLsnMngr> lsnMngr, - TActorId skeletonId, - TActorId loggerId, - TActorId logCutterId, + TActorId skeletonId, + TActorId loggerId, + TActorId logCutterId, const TString &localRecoveryInfoDbg); ~THugeKeeperCtx(); }; diff --git a/ydb/core/blobstorage/vdisk/huge/blobstorage_hullhugedelete.cpp b/ydb/core/blobstorage/vdisk/huge/blobstorage_hullhugedelete.cpp index 79f6b363c0c..3c5e904a279 100644 --- a/ydb/core/blobstorage/vdisk/huge/blobstorage_hullhugedelete.cpp +++ b/ydb/core/blobstorage/vdisk/huge/blobstorage_hullhugedelete.cpp @@ -4,7 +4,7 @@ namespace NKikimr { class TDelayedHugeBlobDeleterActor : public TActor<TDelayedHugeBlobDeleterActor> { // pointer to database general data; actually we need only HugeKeeperID from that data - const TActorId HugeKeeperId; + const TActorId HugeKeeperId; // pointer to shared deleter state, it is primarily created in TLevelIndex const TIntrusivePtr<TDelayedHugeBlobDeleterInfo> Info; @@ -14,7 +14,7 @@ namespace NKikimr { return NKikimrServices::TActivity::BS_DELAYED_HUGE_BLOB_DELETER; } - TDelayedHugeBlobDeleterActor(const TActorId &hugeKeeperId, + TDelayedHugeBlobDeleterActor(const TActorId &hugeKeeperId, TIntrusivePtr<TDelayedHugeBlobDeleterInfo> info) : TActor(&TDelayedHugeBlobDeleterActor::StateFunc) , HugeKeeperId(hugeKeeperId) @@ -36,7 +36,7 @@ namespace NKikimr { ) }; - IActor *CreateDelayedHugeBlobDeleterActor(const TActorId &hugeKeeperId, + IActor *CreateDelayedHugeBlobDeleterActor(const TActorId &hugeKeeperId, TIntrusivePtr<TDelayedHugeBlobDeleterInfo> info) { return new TDelayedHugeBlobDeleterActor(hugeKeeperId, std::move(info)); } diff --git a/ydb/core/blobstorage/vdisk/huge/blobstorage_hullhugedelete.h b/ydb/core/blobstorage/vdisk/huge/blobstorage_hullhugedelete.h index e3d1be6a695..f0f6ebc326e 100644 --- a/ydb/core/blobstorage/vdisk/huge/blobstorage_hullhugedelete.h +++ b/ydb/core/blobstorage/vdisk/huge/blobstorage_hullhugedelete.h @@ -3,7 +3,7 @@ #include "defs.h" #include "blobstorage_hullhuge.h" -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <ydb/core/base/blobstorage.h> #include <util/generic/queue.h> @@ -50,7 +50,7 @@ namespace NKikimr { ui64 LastDeletionLsn = 0; // delayed huge blob deleter actor id - TActorId ActorId; + TActorId ActorId; // a queue of removed huge blobs per compaction struct TRemovedHugeBlobsQueueItem { @@ -67,12 +67,12 @@ namespace NKikimr { TDeque<TRemovedHugeBlobsQueueItem> RemovedHugeBlobsQueue; public: - void SetActorId(const TActorId& actorId) { + void SetActorId(const TActorId& actorId) { Y_VERIFY(!ActorId); ActorId = actorId; } - const TActorId& GetActorId() const { + const TActorId& GetActorId() const { return ActorId; } @@ -182,7 +182,7 @@ namespace NKikimr { private: friend class TDelayedHugeBlobDeleterActor; - void ReleaseSnapshot(ui64 cookie, const TActorContext& ctx, const TActorId& hugeKeeperId) { + void ReleaseSnapshot(ui64 cookie, const TActorContext& ctx, const TActorId& hugeKeeperId) { auto it = CurrentSnapshots.find(cookie); Y_VERIFY(it != CurrentSnapshots.end() && it->second > 0); if (!--it->second) { @@ -191,7 +191,7 @@ namespace NKikimr { } } - void ProcessRemovedHugeBlobsQueue(const TActorContext& ctx, const TActorId& hugeKeeperId) { + void ProcessRemovedHugeBlobsQueue(const TActorContext& ctx, const TActorId& hugeKeeperId) { // if we have no snapshots, we can safely process all messages; otherwise we can process only those messages // which do not have snapshots created before the point of compaction while (RemovedHugeBlobsQueue) { @@ -222,11 +222,11 @@ namespace NKikimr { // implemented in blobstorage_hull.h ~TDelayedHugeBlobDeleterNotifier() { - ActorSystem->Send(new IEventHandle(Info->GetActorId(), TActorId(), new TEvHullReleaseSnapshot(Cookie))); + ActorSystem->Send(new IEventHandle(Info->GetActorId(), TActorId(), new TEvHullReleaseSnapshot(Cookie))); } }; - IActor *CreateDelayedHugeBlobDeleterActor(const TActorId &hugeKeeperId, + IActor *CreateDelayedHugeBlobDeleterActor(const TActorId &hugeKeeperId, TIntrusivePtr<TDelayedHugeBlobDeleterInfo> info); } // NKikimr diff --git a/ydb/core/blobstorage/vdisk/huge/blobstorage_hullhugeheap.cpp b/ydb/core/blobstorage/vdisk/huge/blobstorage_hullhugeheap.cpp index 27c40771587..e60200de987 100644 --- a/ydb/core/blobstorage/vdisk/huge/blobstorage_hullhugeheap.cpp +++ b/ydb/core/blobstorage/vdisk/huge/blobstorage_hullhugeheap.cpp @@ -1,6 +1,6 @@ #include "blobstorage_hullhugeheap.h" -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> namespace NKikimr { namespace NHuge { diff --git a/ydb/core/blobstorage/vdisk/huge/ya.make b/ydb/core/blobstorage/vdisk/huge/ya.make index 1f50ae5bf54..78fc6697c51 100644 --- a/ydb/core/blobstorage/vdisk/huge/ya.make +++ b/ydb/core/blobstorage/vdisk/huge/ya.make @@ -6,7 +6,7 @@ OWNER( ) PEERDIR( - library/cpp/monlib/service/pages + library/cpp/monlib/service/pages ydb/core/base ydb/core/blobstorage/vdisk/common ydb/core/blobstorage/vdisk/hulldb/base diff --git a/ydb/core/blobstorage/vdisk/hulldb/base/blobstorage_hullsatisfactionrank.cpp b/ydb/core/blobstorage/vdisk/hulldb/base/blobstorage_hullsatisfactionrank.cpp index 1a9e37aa85c..74ed554356b 100644 --- a/ydb/core/blobstorage/vdisk/hulldb/base/blobstorage_hullsatisfactionrank.cpp +++ b/ydb/core/blobstorage/vdisk/hulldb/base/blobstorage_hullsatisfactionrank.cpp @@ -1,5 +1,5 @@ #include "blobstorage_hullsatisfactionrank.h" -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/stream/output.h> using namespace NKikimrServices; diff --git a/ydb/core/blobstorage/vdisk/hulldb/base/ya.make b/ydb/core/blobstorage/vdisk/hulldb/base/ya.make index 9dfdd0d7ba0..18949cc0031 100644 --- a/ydb/core/blobstorage/vdisk/hulldb/base/ya.make +++ b/ydb/core/blobstorage/vdisk/hulldb/base/ya.make @@ -3,7 +3,7 @@ LIBRARY() OWNER(fomichev) PEERDIR( - library/cpp/monlib/service/pages + library/cpp/monlib/service/pages ydb/core/base ydb/core/blobstorage/pdisk ) diff --git a/ydb/core/blobstorage/vdisk/hulldb/compstrat/hulldb_compstrat_selector.h b/ydb/core/blobstorage/vdisk/hulldb/compstrat/hulldb_compstrat_selector.h index 51aa623257f..f269d15b952 100644 --- a/ydb/core/blobstorage/vdisk/hulldb/compstrat/hulldb_compstrat_selector.h +++ b/ydb/core/blobstorage/vdisk/hulldb/compstrat/hulldb_compstrat_selector.h @@ -93,7 +93,7 @@ namespace NKikimr { TSelectorParams Params; TLevelIndexSnapshot LevelSnap; TBarriersSnapshot BarriersSnap; - const TActorId RecipientID; + const TActorId RecipientID; std::unique_ptr<TCompactionTask> CompactionTask; void Bootstrap(const TActorContext &ctx) { diff --git a/ydb/core/blobstorage/vdisk/hulldb/fresh/fresh_appendix.h b/ydb/core/blobstorage/vdisk/hulldb/fresh/fresh_appendix.h index 404b3c9e10f..a4da075ad42 100644 --- a/ydb/core/blobstorage/vdisk/hulldb/fresh/fresh_appendix.h +++ b/ydb/core/blobstorage/vdisk/hulldb/fresh/fresh_appendix.h @@ -9,7 +9,7 @@ // FIXME #include <ydb/core/blobstorage/vdisk/hulldb/generic/blobstorage_hullrecmerger.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <ydb/core/blobstorage/base/ptr.h> namespace NKikimr { diff --git a/ydb/core/blobstorage/vdisk/hulldb/fresh/fresh_segment_impl.h b/ydb/core/blobstorage/vdisk/hulldb/fresh/fresh_segment_impl.h index 228f4f11b46..3ebf8e8bea3 100644 --- a/ydb/core/blobstorage/vdisk/hulldb/fresh/fresh_segment_impl.h +++ b/ydb/core/blobstorage/vdisk/hulldb/fresh/fresh_segment_impl.h @@ -8,7 +8,7 @@ #include <ydb/core/blobstorage/vdisk/hulldb/base/hullbase_logoblob.h> #include <ydb/core/blobstorage/vdisk/hulldb/base/blobstorage_blob.h> #include <library/cpp/threading/skip_list/skiplist.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/system/align.h> #include <util/generic/set.h> diff --git a/ydb/core/blobstorage/vdisk/hulldb/generic/hullds_idx.h b/ydb/core/blobstorage/vdisk/hulldb/generic/hullds_idx.h index f3b22f056ec..b3b44759129 100644 --- a/ydb/core/blobstorage/vdisk/hulldb/generic/hullds_idx.h +++ b/ydb/core/blobstorage/vdisk/hulldb/generic/hullds_idx.h @@ -179,7 +179,7 @@ namespace NKikimr { TLevelSlicePtr CurSlice; TList<TIntrusivePtr<TLevelSegment>> UncommittedReplSegments; - TActorId LIActor; + TActorId LIActor; ui64 CurEntryPointLsn = ui64(-1); ui64 PrevEntryPointLsn = ui64(-1); diff --git a/ydb/core/blobstorage/vdisk/hulldb/generic/hullds_sstslice.h b/ydb/core/blobstorage/vdisk/hulldb/generic/hullds_sstslice.h index a29ede173b7..83d7253d3bf 100644 --- a/ydb/core/blobstorage/vdisk/hulldb/generic/hullds_sstslice.h +++ b/ydb/core/blobstorage/vdisk/hulldb/generic/hullds_sstslice.h @@ -189,7 +189,7 @@ namespace NKikimr { // when slice is destroyed we notify CommitterId about this event (i.e. ChunksToDelete are not used anymore) TActorSystem * /*const*/ ActorSystem; - TActorId CommitterId; + TActorId CommitterId; // In ChunksToDelete we store chunks that are old and subject for deletion, // but previous snapshot can still use them TVector<ui32> ChunksToDelete; @@ -242,7 +242,7 @@ namespace NKikimr { } } - void SetUpCommitter(TActorSystem * /*const*/ system, const TActorId &id) { + void SetUpCommitter(TActorSystem * /*const*/ system, const TActorId &id) { ActorSystem = system; CommitterId = id; } diff --git a/ydb/core/blobstorage/vdisk/hulldb/hulldb_bulksst_add.h b/ydb/core/blobstorage/vdisk/hulldb/hulldb_bulksst_add.h index b45dfff41cc..dcf23836d63 100644 --- a/ydb/core/blobstorage/vdisk/hulldb/hulldb_bulksst_add.h +++ b/ydb/core/blobstorage/vdisk/hulldb/hulldb_bulksst_add.h @@ -122,7 +122,7 @@ namespace NKikimr { // SSTables to add TAddBulkSstEssence Essence; // actor ID to notify after commit - TActorId NotifyId; + TActorId NotifyId; TEvAddBulkSst() = default; diff --git a/ydb/core/blobstorage/vdisk/hulldb/hulldb_bulksstmngr.cpp b/ydb/core/blobstorage/vdisk/hulldb/hulldb_bulksstmngr.cpp index 0526adda6c6..3cc7d07a196 100644 --- a/ydb/core/blobstorage/vdisk/hulldb/hulldb_bulksstmngr.cpp +++ b/ydb/core/blobstorage/vdisk/hulldb/hulldb_bulksstmngr.cpp @@ -25,9 +25,9 @@ namespace NKikimr { TVDiskContextPtr VCtx; TPDiskCtxPtr PDiskCtx; - const TActorId LocalRecoveryActorId; + const TActorId LocalRecoveryActorId; TQueue<TBulkSegmentLoadQueueItem> BulkSegmentLoadQueue; - THashMap<TActorId, TBulkSegmentLoadQueueItem> BulkSegmentLoadInFlight; + THashMap<TActorId, TBulkSegmentLoadQueueItem> BulkSegmentLoadInFlight; TVector<TLevelSegmentPtr> Segments; TActiveActors ActiveActors; @@ -36,7 +36,7 @@ namespace NKikimr { return NKikimrServices::TActivity::BS_BULK_SST_LOADER; } - TLoaderActor(TVDiskContextPtr &&vctx, TPDiskCtxPtr &&pdiskCtx, const TActorId& localRecoveryActorId) + TLoaderActor(TVDiskContextPtr &&vctx, TPDiskCtxPtr &&pdiskCtx, const TActorId& localRecoveryActorId) : VCtx(std::move(vctx)) , PDiskCtx(std::move(pdiskCtx)) , LocalRecoveryActorId(localRecoveryActorId) @@ -147,7 +147,7 @@ namespace NKikimr { IActor *TBulkFormedSstInfoSet::CreateLoaderActor(TVDiskContextPtr vctx, TPDiskCtxPtr pdiskCtx, ui64 syncLogMaxLsnStored, - const TActorId& localRecoveryActorId) { + const TActorId& localRecoveryActorId) { auto loader = std::make_unique<NLoaderActor::TLoaderActor>(std::move(vctx), std::move(pdiskCtx), localRecoveryActorId); diff --git a/ydb/core/blobstorage/vdisk/hulldb/hulldb_bulksstmngr.h b/ydb/core/blobstorage/vdisk/hulldb/hulldb_bulksstmngr.h index a44ee46efd4..22038ef95ca 100644 --- a/ydb/core/blobstorage/vdisk/hulldb/hulldb_bulksstmngr.h +++ b/ydb/core/blobstorage/vdisk/hulldb/hulldb_bulksstmngr.h @@ -134,7 +134,7 @@ namespace NKikimr { // create actor which loads bulk-formed segments necessary for SyncLog recovery IActor *CreateLoaderActor(TVDiskContextPtr vctx, TPDiskCtxPtr pdiskCtx, ui64 syncLogMaxLsnStored, - const TActorId& localRecoveryActorId); + const TActorId& localRecoveryActorId); void SerializeToProto(NKikimrVDiskData::TBulkFormedSstInfoSet &pb) const; static bool ConvertToProto(NKikimrVDiskData::TBulkFormedSstInfoSet &pb, const char *begin, const char *end); diff --git a/ydb/core/blobstorage/vdisk/hullop/blobstorage_hull.cpp b/ydb/core/blobstorage/vdisk/hullop/blobstorage_hull.cpp index 9c8bac0167c..33a8e1153b5 100644 --- a/ydb/core/blobstorage/vdisk/hullop/blobstorage_hull.cpp +++ b/ydb/core/blobstorage/vdisk/hullop/blobstorage_hull.cpp @@ -41,7 +41,7 @@ namespace NKikimr { TIntrusivePtr<TLsnMngr> &&lsnMngr, TPDiskCtxPtr &&pdiskCtx, TIntrusivePtr<THandoffDelegate> &&handoffDelegate, - const TActorId skeletonId, + const TActorId skeletonId, bool runHandoff, TActorSystem *as, bool barrierValidation) @@ -62,7 +62,7 @@ namespace NKikimr { BarriersRunTimeCtx->CutRecoveryLog(ctx, std::unique_ptr<NPDisk::TEvCutLog>(msg->Clone())); } - void SetLogNotifierActorId(const TActorId &aid) { + void SetLogNotifierActorId(const TActorId &aid) { LogoBlobsRunTimeCtx->SetLogNotifierActorId(aid); BlocksRunTimeCtx->SetLogNotifierActorId(aid); BarriersRunTimeCtx->SetLogNotifierActorId(aid); @@ -76,7 +76,7 @@ namespace NKikimr { TIntrusivePtr<TLsnMngr> lsnMngr, TPDiskCtxPtr pdiskCtx, TIntrusivePtr<THandoffDelegate> handoffDelegate, - const TActorId skeletonId, + const TActorId skeletonId, bool runHandoff, THullDbRecovery &&uncond, TActorSystem *as, @@ -108,8 +108,8 @@ namespace NKikimr { TIntrusivePtr<TVDiskConfig> config, std::shared_ptr<THullLogCtx> hullLogCtx, std::shared_ptr<NSyncLog::TSyncLogFirstLsnToKeep> syncLogFirstLsnToKeep, - TActorId loggerId, - TActorId logCutterId, + TActorId loggerId, + TActorId logCutterId, const TActorContext &ctx) { TActiveActors activeActors; @@ -148,7 +148,7 @@ namespace NKikimr { void THull::PostponeReplyUntilCommitted( IEventBase *msg, - const TActorId &recipient, + const TActorId &recipient, ui64 recipientCookie, ui64 lsn) { diff --git a/ydb/core/blobstorage/vdisk/hullop/blobstorage_hull.h b/ydb/core/blobstorage/vdisk/hullop/blobstorage_hull.h index 0073131d610..309c3d52c2f 100644 --- a/ydb/core/blobstorage/vdisk/hullop/blobstorage_hull.h +++ b/ydb/core/blobstorage/vdisk/hullop/blobstorage_hull.h @@ -61,7 +61,7 @@ namespace NKikimr { TIntrusivePtr<TLsnMngr> lsnMngr, TPDiskCtxPtr pdiskCtx, TIntrusivePtr<THandoffDelegate> handoffDelegate, - const TActorId skeletonId, + const TActorId skeletonId, bool runHandoff, THullDbRecovery &&uncond, TActorSystem *as, @@ -77,14 +77,14 @@ namespace NKikimr { TIntrusivePtr<TVDiskConfig> config, std::shared_ptr<THullLogCtx> hullLogCtx, std::shared_ptr<NSyncLog::TSyncLogFirstLsnToKeep> syncLogFirstLsnToKeep, - TActorId loggerId, - TActorId logCutterId, + TActorId loggerId, + TActorId logCutterId, const TActorContext &ctx); // Request from PDisk to cut the recovery log void CutRecoveryLog(const TActorContext &ctx, std::unique_ptr<NPDisk::TEvCutLog> msg); - void PostponeReplyUntilCommitted(IEventBase *msg, const TActorId &recipient, ui64 recipientCookie, ui64 lsn); + void PostponeReplyUntilCommitted(IEventBase *msg, const TActorId &recipient, ui64 recipientCookie, ui64 lsn); //////////////////////////////////////////////////////////////////////// // LogoBlobs @@ -134,7 +134,7 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////// // Blocks //////////////////////////////////////////////////////////////////////// - using TReplySender = std::function<void (const TActorId &, ui64, IEventBase *)>; + using TReplySender = std::function<void (const TActorId &, ui64, IEventBase *)>; THullCheckStatus CheckBlockCmdAndAllocLsn( ui64 tabletID, diff --git a/ydb/core/blobstorage/vdisk/hullop/blobstorage_hullactor.cpp b/ydb/core/blobstorage/vdisk/hullop/blobstorage_hullactor.cpp index 2872215b283..e0118b3acf6 100644 --- a/ydb/core/blobstorage/vdisk/hullop/blobstorage_hullactor.cpp +++ b/ydb/core/blobstorage/vdisk/hullop/blobstorage_hullactor.cpp @@ -656,7 +656,7 @@ namespace NKikimr { TIntrusivePtr<TVDiskConfig> config, TIntrusivePtr<THullDs> hullDs, std::shared_ptr<THullLogCtx> hullLogCtx, - TActorId loggerId, + TActorId loggerId, std::shared_ptr<TRunTimeCtx> rtCtx, std::shared_ptr<NSyncLog::TSyncLogFirstLsnToKeep> syncLogFirstLsnToKeep) : TActorBootstrapped<TThis>() @@ -684,7 +684,7 @@ namespace NKikimr { TIntrusivePtr<TVDiskConfig> config, TIntrusivePtr<THullDs> hullDs, std::shared_ptr<THullLogCtx> hullLogCtx, - TActorId loggerId, + TActorId loggerId, std::shared_ptr<TLevelIndexRunTimeCtx<TKeyLogoBlob, TMemRecLogoBlob>> rtCtx, std::shared_ptr<NSyncLog::TSyncLogFirstLsnToKeep> syncLogFirstLsnToKeep) { @@ -696,7 +696,7 @@ namespace NKikimr { TIntrusivePtr<TVDiskConfig> config, TIntrusivePtr<THullDs> hullDs, std::shared_ptr<THullLogCtx> hullLogCtx, - TActorId loggerId, + TActorId loggerId, std::shared_ptr<TLevelIndexRunTimeCtx<TKeyBlock, TMemRecBlock>> rtCtx, std::shared_ptr<NSyncLog::TSyncLogFirstLsnToKeep> syncLogFirstLsnToKeep) { @@ -708,7 +708,7 @@ namespace NKikimr { TIntrusivePtr<TVDiskConfig> config, TIntrusivePtr<THullDs> hullDs, std::shared_ptr<THullLogCtx> hullLogCtx, - TActorId loggerId, + TActorId loggerId, std::shared_ptr<TLevelIndexRunTimeCtx<TKeyBarrier, TMemRecBarrier>> rtCtx, std::shared_ptr<NSyncLog::TSyncLogFirstLsnToKeep> syncLogFirstLsnToKeep) { diff --git a/ydb/core/blobstorage/vdisk/hullop/blobstorage_hullactor.h b/ydb/core/blobstorage/vdisk/hullop/blobstorage_hullactor.h index a30b3baf842..5749a70ecd2 100644 --- a/ydb/core/blobstorage/vdisk/hullop/blobstorage_hullactor.h +++ b/ydb/core/blobstorage/vdisk/hullop/blobstorage_hullactor.h @@ -21,19 +21,19 @@ namespace NKikimr { TIntrusivePtr<TLsnMngr> LsnMngr; TPDiskCtxPtr PDiskCtx; TIntrusivePtr<THandoffDelegate> HandoffDelegate; - const TActorId SkeletonId; + const TActorId SkeletonId; const bool RunHandoff; const TIntrusivePtr<TLevelIndex<TKey, TMemRec>> LevelIndex; ui64 FreeUpToLsn = 0; private: // ActorId of the LogCutterNotifier, that aggregates cut log lsn for the Hull database - TActorId LogNotifierActorId; + TActorId LogNotifierActorId; public: TLevelIndexRunTimeCtx(TIntrusivePtr<TLsnMngr> lsnMngr, TPDiskCtxPtr pdiskCtx, TIntrusivePtr<THandoffDelegate> handoffDelegate, - const TActorId skeletonId, + const TActorId skeletonId, bool runHandoff, TIntrusivePtr<TLevelIndex<TKey, TMemRec>> levelIndex) : LsnMngr(std::move(lsnMngr)) @@ -59,11 +59,11 @@ namespace NKikimr { return FreeUpToLsn; } - void SetLogNotifierActorId(const TActorId &aid) { + void SetLogNotifierActorId(const TActorId &aid) { LogNotifierActorId = aid; } - TActorId GetLogNotifierActorId() const { + TActorId GetLogNotifierActorId() const { Y_VERIFY(LogNotifierActorId); return LogNotifierActorId; } @@ -107,7 +107,7 @@ namespace NKikimr { TIntrusivePtr<TVDiskConfig> config, TIntrusivePtr<THullDs> hullDs, std::shared_ptr<THullLogCtx> hullLogCtx, - TActorId loggerId, + TActorId loggerId, std::shared_ptr<TLevelIndexRunTimeCtx<TKeyLogoBlob, TMemRecLogoBlob>> rtCtx, std::shared_ptr<NSyncLog::TSyncLogFirstLsnToKeep> syncLogFirstLsnToKeep); @@ -115,7 +115,7 @@ namespace NKikimr { TIntrusivePtr<TVDiskConfig> config, TIntrusivePtr<THullDs> hullDs, std::shared_ptr<THullLogCtx> hullLogCtx, - TActorId loggerId, + TActorId loggerId, std::shared_ptr<TLevelIndexRunTimeCtx<TKeyBlock, TMemRecBlock>> rtCtx, std::shared_ptr<NSyncLog::TSyncLogFirstLsnToKeep> syncLogFirstLsnToKeep); @@ -123,7 +123,7 @@ namespace NKikimr { TIntrusivePtr<TVDiskConfig> config, TIntrusivePtr<THullDs> hullDs, std::shared_ptr<THullLogCtx> hullLogCtx, - TActorId loggerId, + TActorId loggerId, std::shared_ptr<TLevelIndexRunTimeCtx<TKeyBarrier, TMemRecBarrier>> rtCtx, std::shared_ptr<NSyncLog::TSyncLogFirstLsnToKeep> syncLogFirstLsnToKeep); diff --git a/ydb/core/blobstorage/vdisk/hullop/blobstorage_hullcommit.h b/ydb/core/blobstorage/vdisk/hullop/blobstorage_hullcommit.h index 0098f4d0767..8b094e893bc 100644 --- a/ydb/core/blobstorage/vdisk/hullop/blobstorage_hullcommit.h +++ b/ydb/core/blobstorage/vdisk/hullop/blobstorage_hullcommit.h @@ -19,14 +19,14 @@ namespace NKikimr { TPDiskCtxPtr PDiskCtx; THullCtxPtr HullCtx; TIntrusivePtr<TLsnMngr> LsnMngr; - const TActorId LoggerId; - const TActorId HugeKeeperId; + const TActorId LoggerId; + const TActorId HugeKeeperId; THullDbCommitterCtx( TPDiskCtxPtr pdiskCtx, THullCtxPtr hullCtx, TIntrusivePtr<TLsnMngr> lsnMngr, - const TActorId &loggerId, + const TActorId &loggerId, const TActorId hugeKeeperId) : PDiskCtx(std::move(pdiskCtx)) , HullCtx(std::move(hullCtx)) @@ -88,8 +88,8 @@ namespace NKikimr { std::shared_ptr<THullLogCtx> HullLogCtx; THullDbCommitterCtxPtr Ctx; TIntrusivePtr<TLevelIndex> LevelIndex; - TActorId NotifyID; - TActorId SecondNotifyID; + TActorId NotifyID; + TActorId SecondNotifyID; THullCommitMeta Metadata; std::unique_ptr<NPDisk::TEvLog> CommitMsg; TLsnSeg LsnSeg; @@ -274,8 +274,8 @@ namespace NKikimr { std::shared_ptr<THullLogCtx> hullLogCtx, THullDbCommitterCtxPtr ctx, TIntrusivePtr<TLevelIndex> levelIndex, - const TActorId& notifyID, - const TActorId& secondNotifyID, + const TActorId& notifyID, + const TActorId& secondNotifyID, THullCommitMeta&& metadata, const TString &callerInfo) : HullLogCtx(std::move(hullLogCtx)) @@ -305,13 +305,13 @@ namespace NKikimr { std::shared_ptr<THullLogCtx> hullLogCtx, THullDbCommitterCtxPtr ctx, TIntrusivePtr<typename TBase::TLevelIndex> levelIndex, - const TActorId ¬ifyID, + const TActorId ¬ifyID, const TString &callerInfo) : TBase(std::move(hullLogCtx), std::move(ctx), std::move(levelIndex), notifyID, - TActorId(), + TActorId(), typename TBase::THullCommitMeta(TVector<ui32>(), TVector<ui32>(), TDiskPartVec()), callerInfo) {} @@ -336,7 +336,7 @@ namespace NKikimr { std::shared_ptr<THullLogCtx> hullLogCtx, THullDbCommitterCtxPtr ctx, TIntrusivePtr<typename TBase::TLevelIndex> levelIndex, - const TActorId& notifyID, + const TActorId& notifyID, TVector<ui32>&& chunksAdded, TVector<ui32>&& chunksDeleted, TDiskPartVec&& removedHugeBlobs, @@ -345,7 +345,7 @@ namespace NKikimr { std::move(ctx), std::move(levelIndex), notifyID, - TActorId(), + TActorId(), typename TBase::THullCommitMeta(std::move(chunksAdded), std::move(chunksDeleted), std::move(removedHugeBlobs)), @@ -400,7 +400,7 @@ namespace NKikimr { std::shared_ptr<THullLogCtx> hullLogCtx, THullDbCommitterCtxPtr ctx, TIntrusivePtr<typename TBase::TLevelIndex> levelIndex, - const TActorId& notifyID, + const TActorId& notifyID, TVector<ui32>&& chunksAdded, TVector<ui32>&& chunksDeleted, TDiskPartVec&& removedHugeBlobs, @@ -409,7 +409,7 @@ namespace NKikimr { std::move(ctx), std::move(levelIndex), notifyID, - TActorId(), + TActorId(), typename TBase::THullCommitMeta(std::move(chunksAdded), std::move(chunksDeleted), std::move(removedHugeBlobs)), @@ -439,12 +439,12 @@ namespace NKikimr { std::shared_ptr<THullLogCtx> hullLogCtx, THullDbCommitterCtxPtr ctx, TIntrusivePtr<typename TBase::TLevelIndex> levelIndex, - const TActorId& notifyID, + const TActorId& notifyID, TVector<ui32>&& chunksAdded, TVector<ui32>&& chunksDeleted, TIntrusivePtr<TLevelSegment> replSst, ui32 numRecoveredBlobs, - const TActorId& secondNotifyID) + const TActorId& secondNotifyID) : TBase(std::move(hullLogCtx), std::move(ctx), std::move(levelIndex), diff --git a/ydb/core/blobstorage/vdisk/hullop/blobstorage_hullcompact.h b/ydb/core/blobstorage/vdisk/hullop/blobstorage_hullcompact.h index f538fb7a5b1..fca2f00daa9 100644 --- a/ydb/core/blobstorage/vdisk/hullop/blobstorage_hullcompact.h +++ b/ydb/core/blobstorage/vdisk/hullop/blobstorage_hullcompact.h @@ -65,7 +65,7 @@ namespace NKikimr { THullCtxPtr HullCtx; TPDiskCtxPtr PDiskCtx; - const TActorId LIActor; + const TActorId LIActor; // FreshSegment to compact if any TIntrusivePtr<TFreshSegment> FreshSegment; std::shared_ptr<TFreshSegmentSnapshot> FreshSegmentSnap; diff --git a/ydb/core/blobstorage/vdisk/hullop/blobstorage_hullload.h b/ydb/core/blobstorage/vdisk/hullop/blobstorage_hullload.h index 771f4409be1..d08b3a04d77 100644 --- a/ydb/core/blobstorage/vdisk/hullop/blobstorage_hullload.h +++ b/ydb/core/blobstorage/vdisk/hullop/blobstorage_hullload.h @@ -106,7 +106,7 @@ namespace NKikimr { const TVDiskContextPtr VCtx; const TPDiskCtxPtr PDiskCtx; TLevelSegment *LevelSegment; - TActorId Recipient; + TActorId Recipient; TString Origin; bool FirstRead; ui32 RestToReadIndex; @@ -302,7 +302,7 @@ namespace NKikimr { const TVDiskContextPtr &vctx, const TPDiskCtxPtr &pdiskCtx, TLevelSegment *levelSegment, - const TActorId &recipient, + const TActorId &recipient, const TString &origin) : TActorBootstrapped<TThis>() , VCtx(vctx) @@ -359,7 +359,7 @@ namespace NKikimr { const TVDiskContextPtr VCtx; const TPDiskCtxPtr PDiskCtx; TOrderedLevelSegments *Segs; - TActorId Recipient; + TActorId Recipient; ui32 Pos; ui32 Size; TActiveActors ActiveActors; @@ -370,7 +370,7 @@ namespace NKikimr { if (Pos < Size) { std::unique_ptr<TLevelSegmentLoader> actor(new TLevelSegmentLoader(VCtx, PDiskCtx, Segs->Segments[Pos].Get(), ctx.SelfID, "OrderedLevelSegmentsLoader")); - NActors::TActorId aid = ctx.Register(actor.Release()); + NActors::TActorId aid = ctx.Register(actor.Release()); ActiveActors.Insert(aid); ++Pos; } else { @@ -410,7 +410,7 @@ namespace NKikimr { const TVDiskContextPtr vctx, const TPDiskCtxPtr pdiskCtx, TOrderedLevelSegments *levelSegmentVec, - const TActorId &recipient) + const TActorId &recipient) : TActorBootstrapped<TThis>() , VCtx(vctx) , PDiskCtx(pdiskCtx) @@ -440,7 +440,7 @@ namespace NKikimr { const TVDiskContextPtr VCtx; const TPDiskCtxPtr PDiskCtx; TUnorderedLevelSegments *Segs; - TActorId Recipient; + TActorId Recipient; TIterator Pos; TIterator End; TActiveActors ActiveActors; @@ -450,7 +450,7 @@ namespace NKikimr { void Process(const TActorContext &ctx) { if (Pos != End) { std::unique_ptr<TLevelSegmentLoader> actor(new TLevelSegmentLoader(VCtx, PDiskCtx, Pos->Get(), ctx.SelfID)); - NActors::TActorId aid = ctx.Register(actor.Release()); + NActors::TActorId aid = ctx.Register(actor.Release()); ActiveActors.Insert(aid); } else { ctx.Send(Recipient, new THullSegmentsLoaded(Segs)); @@ -488,7 +488,7 @@ namespace NKikimr { TUnorderedLevelSegmentsLoader( const TVDiskContextPtr &vctx, const TPDiskCtxPtr &pdiskCtx, - TUnorderedLevelSegments *segs, const TActorId &recipient) + TUnorderedLevelSegments *segs, const TActorId &recipient) : TActorBootstrapped<TThis>() , VCtx(vctx) , PDiskCtx(pdiskCtx) @@ -528,7 +528,7 @@ namespace NKikimr { const TVDiskContextPtr VCtx; const TPDiskCtxPtr PDiskCtx; TLevelIndex *LevelIndex; - TActorId Recipient; + TActorId Recipient; TSstIterator It; TActiveActors ActiveActors; @@ -581,7 +581,7 @@ namespace NKikimr { const TVDiskContextPtr vctx, const TPDiskCtxPtr pdiskCtx, TLevelIndex *levelIndex, - const TActorId &recipient) + const TActorId &recipient) : TActorBootstrapped<TThis>() , VCtx(vctx) , PDiskCtx(pdiskCtx) diff --git a/ydb/core/blobstorage/vdisk/hullop/blobstorage_hulllog.cpp b/ydb/core/blobstorage/vdisk/hullop/blobstorage_hulllog.cpp index 5dd46824282..bbdf962ce56 100644 --- a/ydb/core/blobstorage/vdisk/hullop/blobstorage_hulllog.cpp +++ b/ydb/core/blobstorage/vdisk/hullop/blobstorage_hulllog.cpp @@ -6,14 +6,14 @@ namespace NKikimr { class THullCallback : public NPDisk::TEvLog::ICallback { const TLsnSeg Seg; TIntrusivePtr<TVDiskContext> VCtx; - const TActorId SkeletonId; - const TActorId SyncLogId; - const TActorId HugeKeeperId; + const TActorId SkeletonId; + const TActorId SyncLogId; + const TActorId HugeKeeperId; std::unique_ptr<IEventBase> SyncLogMsg; std::unique_ptr<TEvHullHugeBlobLogged> HugeKeeperNotice; public: - THullCallback(TLsnSeg seg, const TIntrusivePtr<TVDiskContext> &vctx, TActorId skeletonId, TActorId syncLogId, + THullCallback(TLsnSeg seg, const TIntrusivePtr<TVDiskContext> &vctx, TActorId skeletonId, TActorId syncLogId, TActorId hugeKeeperId, std::unique_ptr<IEventBase> syncLogMsg, std::unique_ptr<TEvHullHugeBlobLogged> hugeKeeperNotice) : Seg(seg) @@ -73,7 +73,7 @@ namespace NKikimr { std::unique_ptr<IEventBase> syncLogMsg) { auto callback = std::make_unique<THullCallback>(seg, hullLogCtx->VCtx, hullLogCtx->SkeletonId, - hullLogCtx->SyncLogId, TActorId(), std::move(syncLogMsg), nullptr); + hullLogCtx->SyncLogId, TActorId(), std::move(syncLogMsg), nullptr); return std::make_unique<NPDisk::TEvLog>(hullLogCtx->PDiskCtx->Dsk->Owner, hullLogCtx->PDiskCtx->Dsk->OwnerRound, diff --git a/ydb/core/blobstorage/vdisk/hullop/blobstorage_hulllogcutternotify.cpp b/ydb/core/blobstorage/vdisk/hullop/blobstorage_hulllogcutternotify.cpp index 9c42bbd92a2..4904c6b363a 100644 --- a/ydb/core/blobstorage/vdisk/hullop/blobstorage_hulllogcutternotify.cpp +++ b/ydb/core/blobstorage/vdisk/hullop/blobstorage_hulllogcutternotify.cpp @@ -10,7 +10,7 @@ namespace NKikimr { // must be run on the same mailbox with TSkeleton class THullLogCutterNotifier : public TActor<THullLogCutterNotifier> { const TVDiskContextPtr VCtx; - const TActorId LogCutterId; + const TActorId LogCutterId; TIntrusivePtr<THullDs> HullDs; TMaybe<ui64> PreviousCutLsn; @@ -55,7 +55,7 @@ namespace NKikimr { THullLogCutterNotifier( const TVDiskContextPtr &vctx, - const TActorId &logCutterId, + const TActorId &logCutterId, TIntrusivePtr<THullDs> hullDs) : TActor<THullLogCutterNotifier>(&TThis::StateFunc) , VCtx(vctx) @@ -70,7 +70,7 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// IActor* CreateHullLogCutterNotifier( const TVDiskContextPtr &vctx, - const TActorId &logCutterId, + const TActorId &logCutterId, TIntrusivePtr<THullDs> hullDs) { Y_VERIFY(logCutterId); return new THullLogCutterNotifier(vctx, logCutterId, hullDs); diff --git a/ydb/core/blobstorage/vdisk/hullop/blobstorage_hulllogcutternotify.h b/ydb/core/blobstorage/vdisk/hullop/blobstorage_hulllogcutternotify.h index 00439a35637..5165696eecb 100644 --- a/ydb/core/blobstorage/vdisk/hullop/blobstorage_hulllogcutternotify.h +++ b/ydb/core/blobstorage/vdisk/hullop/blobstorage_hulllogcutternotify.h @@ -14,7 +14,7 @@ namespace NKikimr { IActor* CreateHullLogCutterNotifier( const TVDiskContextPtr &vctx, - const TActorId &logCutterId, + const TActorId &logCutterId, TIntrusivePtr<THullDs> hullDs); } // NKikimr diff --git a/ydb/core/blobstorage/vdisk/hullop/hullop_delayedresp.h b/ydb/core/blobstorage/vdisk/hullop/hullop_delayedresp.h index b9356c5c4ea..8d65c474c9b 100644 --- a/ydb/core/blobstorage/vdisk/hullop/hullop_delayedresp.h +++ b/ydb/core/blobstorage/vdisk/hullop/hullop_delayedresp.h @@ -11,9 +11,9 @@ namespace NKikimr { /////////////////////////////////////////////////////////////////////////////////////////////////////// class TDelayedResponses { public: - using TAction = std::function<void (const TActorId &id, ui64 cookie, IEventBase *msg)>; + using TAction = std::function<void (const TActorId &id, ui64 cookie, IEventBase *msg)>; - void Put(IEventBase *msg, const TActorId &recipient, ui64 recipientCookie, ui64 lsn) { + void Put(IEventBase *msg, const TActorId &recipient, ui64 recipientCookie, ui64 lsn) { Map.emplace(lsn, TValue {recipient, recipientCookie, std::unique_ptr<IEventBase>(msg)}); } @@ -30,7 +30,7 @@ namespace NKikimr { private: struct TValue { - TActorId Recipient; + TActorId Recipient; ui64 RecipientCookie; std::unique_ptr<IEventBase> Msg; }; diff --git a/ydb/core/blobstorage/vdisk/hullop/hullop_delayedresp_ut.cpp b/ydb/core/blobstorage/vdisk/hullop/hullop_delayedresp_ut.cpp index 3672a209c17..f95ace015a2 100644 --- a/ydb/core/blobstorage/vdisk/hullop/hullop_delayedresp_ut.cpp +++ b/ydb/core/blobstorage/vdisk/hullop/hullop_delayedresp_ut.cpp @@ -10,7 +10,7 @@ namespace NKikimr { Y_UNIT_TEST(Test) { TVector<ui64> res; - auto action = [&res] (const TActorId &actorId, ui64 cookie, IEventBase *msg) { + auto action = [&res] (const TActorId &actorId, ui64 cookie, IEventBase *msg) { Y_UNUSED(actorId); STR << "cookie# " << cookie << "\n"; res.push_back(cookie); @@ -18,12 +18,12 @@ namespace NKikimr { }; auto dr = std::make_unique<TDelayedResponses>(); - dr->Put(nullptr, TActorId(), 1, 500); - dr->Put(nullptr, TActorId(), 2, 500); - dr->Put(nullptr, TActorId(), 3, 501); - dr->Put(nullptr, TActorId(), 4, 500); + dr->Put(nullptr, TActorId(), 1, 500); + dr->Put(nullptr, TActorId(), 2, 500); + dr->Put(nullptr, TActorId(), 3, 501); + dr->Put(nullptr, TActorId(), 4, 500); dr->ConfirmLsn(500, action); - dr->Put(nullptr, TActorId(), 5, 502); + dr->Put(nullptr, TActorId(), 5, 502); dr->ConfirmLsn(501, action); dr->ConfirmLsn(502, action); diff --git a/ydb/core/blobstorage/vdisk/ingress/blobstorage_ingress_matrix_ut.cpp b/ydb/core/blobstorage/vdisk/ingress/blobstorage_ingress_matrix_ut.cpp index 5a661cdb49a..e3373f054c6 100644 --- a/ydb/core/blobstorage/vdisk/ingress/blobstorage_ingress_matrix_ut.cpp +++ b/ydb/core/blobstorage/vdisk/ingress/blobstorage_ingress_matrix_ut.cpp @@ -254,7 +254,7 @@ namespace NKikimr { UNIT_ASSERT(m.GetColumn(2) == TVectorType(0x01, 8)); } - void DebugPrint(const TVector<std::pair<TVDiskID, TActorId> > &vec) { + void DebugPrint(const TVector<std::pair<TVDiskID, TActorId> > &vec) { for (unsigned i = 0; i < vec.size(); i++) { fprintf(stderr, "%s\n", vec[i].first.ToString().data()); } diff --git a/ydb/core/blobstorage/vdisk/localrecovery/localrecovery_defs.cpp b/ydb/core/blobstorage/vdisk/localrecovery/localrecovery_defs.cpp index 28c40b06bf0..f75654db66c 100644 --- a/ydb/core/blobstorage/vdisk/localrecovery/localrecovery_defs.cpp +++ b/ydb/core/blobstorage/vdisk/localrecovery/localrecovery_defs.cpp @@ -1,7 +1,7 @@ #include "localrecovery_defs.h" #include <ydb/core/blobstorage/vdisk/common/vdisk_mon.h> #include <ydb/core/blobstorage/pdisk/blobstorage_pdisk_internal_interface.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> namespace NKikimr { diff --git a/ydb/core/blobstorage/vdisk/localrecovery/localrecovery_logreplay.cpp b/ydb/core/blobstorage/vdisk/localrecovery/localrecovery_logreplay.cpp index 052fa52cca9..b9f9460b402 100644 --- a/ydb/core/blobstorage/vdisk/localrecovery/localrecovery_logreplay.cpp +++ b/ydb/core/blobstorage/vdisk/localrecovery/localrecovery_logreplay.cpp @@ -55,7 +55,7 @@ namespace NKikimr { {} }; - const TActorId ParentId; + const TActorId ParentId; std::shared_ptr<TLocalRecoveryContext> LocRecCtx; TActiveActors ActiveActors; NPDisk::TLogPosition PrevLogPos = {0, 0}; diff --git a/ydb/core/blobstorage/vdisk/localrecovery/localrecovery_public.cpp b/ydb/core/blobstorage/vdisk/localrecovery/localrecovery_public.cpp index 2242847f32f..b9bdafdbb13 100644 --- a/ydb/core/blobstorage/vdisk/localrecovery/localrecovery_public.cpp +++ b/ydb/core/blobstorage/vdisk/localrecovery/localrecovery_public.cpp @@ -66,7 +66,7 @@ namespace NKikimr { TIntrusivePtr<TVDiskConfig> Config; // generation independent self VDiskId (it is required for Yard init only) const TVDiskID SelfVDiskId; - const TActorId SkeletonId; + const TActorId SkeletonId; const TActorId SkeletonFrontId; std::shared_ptr<TLocalRecoveryContext> LocRecCtx; std::shared_ptr<THugeBlobCtx> HugeBlobCtx; @@ -226,7 +226,7 @@ namespace NKikimr { template <class TMetaBase, class TLoader, int signature> bool InitMetabase(const TStartingPoints &startingPoints, TIntrusivePtr<TMetaBase> &metabase, - bool &initFlag, NMonitoring::TDeprecatedCounter &counter, bool &emptyDb, + bool &initFlag, NMonitoring::TDeprecatedCounter &counter, bool &emptyDb, ui64 freshBufSize, ui64 compThreshold, const TActorContext &ctx) { TStartingPoints::const_iterator it; // Settings @@ -570,7 +570,7 @@ namespace NKikimr { auto ev = std::make_unique<NPDisk::TEvYardInit>(Config->BaseInfo.InitOwnerRound, SelfVDiskId, Config->BaseInfo.PDiskGuid, SkeletonId, SkeletonFrontId, Config->BaseInfo.VDiskSlotId); - const TActorId nodeWardenId = MakeBlobStorageNodeWardenID(SelfId().NodeId()); + const TActorId nodeWardenId = MakeBlobStorageNodeWardenID(SelfId().NodeId()); auto handle = std::make_unique<IEventHandle>(Config->BaseInfo.PDiskActorID, SelfId(), ev.release(), IEventHandle::FlagForwardOnNondelivery, 0, &nodeWardenId); if (const TDuration delay = Config->BaseInfo.YardInitDelay; delay != TDuration::Zero()) { @@ -659,7 +659,7 @@ namespace NKikimr { const TIntrusivePtr<TVDiskContext> &vctx, const TIntrusivePtr<TVDiskConfig> &config, const TVDiskID &selfVDiskId, - const TActorId &skeletonId, + const TActorId &skeletonId, const TActorId skeletonFrontId, std::shared_ptr<TRopeArena> arena) : TActorBootstrapped<TDatabaseLocalRecovery>() @@ -682,7 +682,7 @@ namespace NKikimr { const TIntrusivePtr<TVDiskContext> &vctx, const TIntrusivePtr<TVDiskConfig> &config, const TVDiskID &selfVDiskId, - const TActorId &skeletonId, + const TActorId &skeletonId, const TActorId skeletonFrontId, std::shared_ptr<TRopeArena> arena) { return new TDatabaseLocalRecovery(vctx, config, selfVDiskId, skeletonId, skeletonFrontId, std::move(arena)); diff --git a/ydb/core/blobstorage/vdisk/localrecovery/localrecovery_public.h b/ydb/core/blobstorage/vdisk/localrecovery/localrecovery_public.h index d756c0d0882..adc87c73a86 100644 --- a/ydb/core/blobstorage/vdisk/localrecovery/localrecovery_public.h +++ b/ydb/core/blobstorage/vdisk/localrecovery/localrecovery_public.h @@ -76,7 +76,7 @@ namespace NKikimr { const TIntrusivePtr<TVDiskContext> &vctx, const TIntrusivePtr<TVDiskConfig> &config, const TVDiskID &selfVDiskId, - const TActorId &skeletonId, + const TActorId &skeletonId, const TActorId skeletonFrontId, std::shared_ptr<TRopeArena> arena); diff --git a/ydb/core/blobstorage/vdisk/localrecovery/localrecovery_readbulksst.cpp b/ydb/core/blobstorage/vdisk/localrecovery/localrecovery_readbulksst.cpp index 915d391e2e6..f9c300960d9 100644 --- a/ydb/core/blobstorage/vdisk/localrecovery/localrecovery_readbulksst.cpp +++ b/ydb/core/blobstorage/vdisk/localrecovery/localrecovery_readbulksst.cpp @@ -16,7 +16,7 @@ namespace NKikimr { const TVDiskContextPtr VCtx; const TPDiskCtxPtr PDiskCtx; - TActorId Recipient; + TActorId Recipient; const ui64 RecoveryLogRecLsn; const TString Origin; // Index of sst being loaded @@ -78,7 +78,7 @@ namespace NKikimr { const TPDiskCtxPtr &pdiskCtx, const google::protobuf::RepeatedPtrField< NKikimrVDiskData::TAddBulkSstRecoveryLogRec_TSstAndRecsNum> &proto, - const TActorId &recipient, + const TActorId &recipient, const ui64 recoveryLogRecLsn, const TString &origin) : TActorBootstrapped<TThis>() @@ -108,7 +108,7 @@ namespace NKikimr { // Recovery Log Record NKikimrVDiskData::TAddBulkSstRecoveryLogRec Proto; // Reply to Recipient - TActorId Recipient; + TActorId Recipient; // We handle recovery log record with this lsn const ui64 RecoveryLogRecLsn; // Number of databases/actors run @@ -191,7 +191,7 @@ namespace NKikimr { const TVDiskContextPtr &vctx, const TPDiskCtxPtr &pdiskCtx, const NKikimrVDiskData::TAddBulkSstRecoveryLogRec &proto, - const TActorId &recipient, + const TActorId &recipient, const ui64 recoveryLogRecLsn, bool loadLogoBlobs, bool loadBlocks, @@ -215,7 +215,7 @@ namespace NKikimr { const TVDiskContextPtr &vctx, const TPDiskCtxPtr &pdiskCtx, const NKikimrVDiskData::TAddBulkSstRecoveryLogRec &proto, - const TActorId &recipient, + const TActorId &recipient, const ui64 recoveryLogRecLsn, bool loadLogoBlobs, bool loadBlocks, diff --git a/ydb/core/blobstorage/vdisk/localrecovery/localrecovery_readbulksst.h b/ydb/core/blobstorage/vdisk/localrecovery/localrecovery_readbulksst.h index 3f66a8815f7..c9691087a43 100644 --- a/ydb/core/blobstorage/vdisk/localrecovery/localrecovery_readbulksst.h +++ b/ydb/core/blobstorage/vdisk/localrecovery/localrecovery_readbulksst.h @@ -35,7 +35,7 @@ namespace NKikimr { const TVDiskContextPtr &vctx, const TPDiskCtxPtr &pdiskCtx, const NKikimrVDiskData::TAddBulkSstRecoveryLogRec &proto, - const TActorId &recipient, + const TActorId &recipient, const ui64 recoveryLogRecLsn, bool loadLogoBlobs, bool loadBlocks, diff --git a/ydb/core/blobstorage/vdisk/query/query_barrier.cpp b/ydb/core/blobstorage/vdisk/query/query_barrier.cpp index 3b35eb0cd46..c469bb8f404 100644 --- a/ydb/core/blobstorage/vdisk/query/query_barrier.cpp +++ b/ydb/core/blobstorage/vdisk/query/query_barrier.cpp @@ -15,7 +15,7 @@ namespace NKikimr { typedef ::NKikimr::TIndexRecordMerger<TKeyBarrier, TMemRecBarrier> TIndexRecordMerger; TIntrusivePtr<THullCtx> HullCtx; - const TActorId ParentId; + const TActorId ParentId; TBarriersSnapshot BarriersSnap; TEvBlobStorage::TEvVGetBarrier::TPtr Ev; NKikimrBlobStorage::TEvVGetBarrier &Record; @@ -60,7 +60,7 @@ namespace NKikimr { TLevelIndexBarrierQuery( TIntrusivePtr<THullCtx> &hullCtx, - const TActorId &parentId, + const TActorId &parentId, TBarriersSnapshot &&barriersSnap, TEvBlobStorage::TEvVGetBarrier::TPtr &ev, std::unique_ptr<TEvBlobStorage::TEvVGetBarrierResult> result) @@ -79,7 +79,7 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// IActor *CreateLevelIndexBarrierQueryActor( TIntrusivePtr<THullCtx> &hullCtx, - const TActorId &parentId, + const TActorId &parentId, TBarriersSnapshot &&barriersSnap, TEvBlobStorage::TEvVGetBarrier::TPtr &ev, std::unique_ptr<TEvBlobStorage::TEvVGetBarrierResult> result) { diff --git a/ydb/core/blobstorage/vdisk/query/query_base.h b/ydb/core/blobstorage/vdisk/query/query_base.h index 69d571eac45..092b93964f2 100644 --- a/ydb/core/blobstorage/vdisk/query/query_base.h +++ b/ydb/core/blobstorage/vdisk/query/query_base.h @@ -15,7 +15,7 @@ namespace NKikimr { class TLevelIndexQueryBase { protected: std::shared_ptr<TQueryCtx> QueryCtx; - const TActorId ParentId; + const TActorId ParentId; TLogoBlobsSnapshot LogoBlobsSnapshot; TBarriersSnapshot BarriersSnapshot; TReadBatcherCtxPtr BatcherCtx; @@ -27,7 +27,7 @@ namespace NKikimr { TLevelIndexQueryBase( std::shared_ptr<TQueryCtx> &queryCtx, - const TActorId &parentId, + const TActorId &parentId, TLogoBlobsSnapshot &&logoBlobsSnapshot, TBarriersSnapshot &&barrierSnapshot, TEvBlobStorage::TEvVGet::TPtr &ev, diff --git a/ydb/core/blobstorage/vdisk/query/query_dumpdb.h b/ydb/core/blobstorage/vdisk/query/query_dumpdb.h index 069c287791f..bba8a7fbdcd 100644 --- a/ydb/core/blobstorage/vdisk/query/query_dumpdb.h +++ b/ydb/core/blobstorage/vdisk/query/query_dumpdb.h @@ -57,7 +57,7 @@ namespace NKikimr { } TLevelIndexDumpActor(TIntrusivePtr<THullCtx> hullCtx, - const TActorId &parentId, + const TActorId &parentId, TLevelIndexSnapshot &&snapshot, TEvBlobStorage::TEvVDbStat::TPtr &ev, std::unique_ptr<TEvBlobStorage::TEvVDbStatResult> result) @@ -71,7 +71,7 @@ namespace NKikimr { private: TIntrusivePtr<THullCtx> HullCtx; - const TActorId ParentId; + const TActorId ParentId; TLevelIndexSnapshot Snapshot; TEvBlobStorage::TEvVDbStat::TPtr Ev; std::unique_ptr<TEvBlobStorage::TEvVDbStatResult> Result; diff --git a/ydb/core/blobstorage/vdisk/query/query_extr.cpp b/ydb/core/blobstorage/vdisk/query/query_extr.cpp index 0cc5782cfe5..196b6d2054a 100644 --- a/ydb/core/blobstorage/vdisk/query/query_extr.cpp +++ b/ydb/core/blobstorage/vdisk/query/query_extr.cpp @@ -73,7 +73,7 @@ namespace NKikimr { TLevelIndexExtremeQueryViaBatcherBase( std::shared_ptr<TQueryCtx> &queryCtx, - const TActorId &parentId, + const TActorId &parentId, TLogoBlobsSnapshot &&logoBlobsSnapshot, TBarriersSnapshot &&barrierSnapshot, TEvBlobStorage::TEvVGet::TPtr &ev, @@ -144,7 +144,7 @@ namespace NKikimr { TLevelIndexExtremeQueryViaBatcherIndexOnly( std::shared_ptr<TQueryCtx> &queryCtx, - const TActorId &parentId, + const TActorId &parentId, TLogoBlobsSnapshot &&logoBlobsSnapshot, TBarriersSnapshot &&barrierSnapshot, TEvBlobStorage::TEvVGet::TPtr &ev, @@ -382,7 +382,7 @@ namespace NKikimr { TLevelIndexExtremeQueryViaBatcherMergeData( std::shared_ptr<TQueryCtx> &queryCtx, - const TActorId &parentId, + const TActorId &parentId, TLogoBlobsSnapshot &&logoBlobsSnapshot, TBarriersSnapshot &&barrierSnapshot, TEvBlobStorage::TEvVGet::TPtr &ev, @@ -403,7 +403,7 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// IActor *CreateLevelIndexExtremeQueryActor( std::shared_ptr<TQueryCtx> &queryCtx, - const TActorId &parentId, + const TActorId &parentId, TLogoBlobsSnapshot &&logoBlobsSnapshot, TBarriersSnapshot &&barrierSnapshot, TEvBlobStorage::TEvVGet::TPtr &ev, diff --git a/ydb/core/blobstorage/vdisk/query/query_public.cpp b/ydb/core/blobstorage/vdisk/query/query_public.cpp index 4d1d23c125b..25761b685eb 100644 --- a/ydb/core/blobstorage/vdisk/query/query_public.cpp +++ b/ydb/core/blobstorage/vdisk/query/query_public.cpp @@ -13,7 +13,7 @@ namespace NKikimr { // Extreme Query Declaration IActor *CreateLevelIndexExtremeQueryActor( std::shared_ptr<TQueryCtx> &queryCtx, - const TActorId &parentId, + const TActorId &parentId, TLogoBlobsSnapshot &&logoBlobsSnapshot, TBarriersSnapshot &&barrierSnapshot, TEvBlobStorage::TEvVGet::TPtr &ev, @@ -23,7 +23,7 @@ namespace NKikimr { // Range Query Declaration IActor *CreateLevelIndexRangeQueryActor( std::shared_ptr<TQueryCtx> &queryCtx, - const TActorId &parentId, + const TActorId &parentId, TLogoBlobsSnapshot &&logoBlobsSnapshot, TBarriersSnapshot &&barrierSnapshot, TEvBlobStorage::TEvVGet::TPtr &ev, @@ -83,7 +83,7 @@ namespace NKikimr { TReadQueryKeepChecker &&keepChecker, const TActorContext &ctx, THullDsSnap &&fullSnap, - const TActorId &parentId, + const TActorId &parentId, TEvBlobStorage::TEvVGet::TPtr &ev, std::unique_ptr<TEvBlobStorage::TEvVGetResult> result, TActorId replSchedulerId) { @@ -167,7 +167,7 @@ namespace NKikimr { const TIntrusivePtr<THullCtx> &hullCtx, const TActorContext &ctx, TLevelIndexSnapshot<TKey, TMemRec> &&levelSnap, - const TActorId &parentId, + const TActorId &parentId, TEvBlobStorage::TEvVDbStat::TPtr &ev, std::unique_ptr<TEvBlobStorage::TEvVDbStatResult> result, const IActor& actor) @@ -199,7 +199,7 @@ namespace NKikimr { const std::shared_ptr<THugeBlobCtx> &hugeBlobCtx, const TActorContext &ctx, THullDsSnap &&fullSnap, - const TActorId &parentId, + const TActorId &parentId, TEvBlobStorage::TEvVDbStat::TPtr &ev, std::unique_ptr<TEvBlobStorage::TEvVDbStatResult> result, const IActor& actor) { diff --git a/ydb/core/blobstorage/vdisk/query/query_public.h b/ydb/core/blobstorage/vdisk/query/query_public.h index 31688befc37..38e498ad718 100644 --- a/ydb/core/blobstorage/vdisk/query/query_public.h +++ b/ydb/core/blobstorage/vdisk/query/query_public.h @@ -43,7 +43,7 @@ namespace NKikimr { TReadQueryKeepChecker &&keepChecker, const TActorContext &ctx, THullDsSnap &&fullSnap, - const TActorId &parentId, + const TActorId &parentId, TEvBlobStorage::TEvVGet::TPtr &ev, std::unique_ptr<TEvBlobStorage::TEvVGetResult> result, TActorId replSchedulerId); @@ -60,7 +60,7 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// IActor *CreateLevelIndexBarrierQueryActor( TIntrusivePtr<THullCtx> &hullCtx, - const TActorId &parentId, + const TActorId &parentId, TBarriersSnapshot &&barriersSnap, TEvBlobStorage::TEvVGetBarrier::TPtr &ev, std::unique_ptr<TEvBlobStorage::TEvVGetBarrierResult> result); @@ -81,7 +81,7 @@ namespace NKikimr { const std::shared_ptr<THugeBlobCtx> &hugeBlobCtx, const TActorContext &ctx, THullDsSnap &&fullSnap, - const TActorId &parentId, + const TActorId &parentId, TEvBlobStorage::TEvVDbStat::TPtr &ev, std::unique_ptr<TEvBlobStorage::TEvVDbStatResult> result, const IActor& actor); diff --git a/ydb/core/blobstorage/vdisk/query/query_range.cpp b/ydb/core/blobstorage/vdisk/query/query_range.cpp index 5455f8bf9a0..4513187b550 100644 --- a/ydb/core/blobstorage/vdisk/query/query_range.cpp +++ b/ydb/core/blobstorage/vdisk/query/query_range.cpp @@ -52,7 +52,7 @@ namespace NKikimr { TLevelIndexRangeQueryViaBatcherBase( std::shared_ptr<TQueryCtx> &queryCtx, - const TActorId &parentId, + const TActorId &parentId, TLogoBlobsSnapshot &&logoBlobsSnapshot, TBarriersSnapshot &&barriersSnapshot, TEvBlobStorage::TEvVGet::TPtr &ev, @@ -143,7 +143,7 @@ namespace NKikimr { TLevelIndexRangeQueryViaBatcherIndexOnly( std::shared_ptr<TQueryCtx> &queryCtx, - const TActorId &parentId, + const TActorId &parentId, TLogoBlobsSnapshot &&logoBlobsSnapshot, TBarriersSnapshot &&barriersSnapshot, TEvBlobStorage::TEvVGet::TPtr &ev, @@ -163,7 +163,7 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// IActor *CreateLevelIndexRangeQueryActor( std::shared_ptr<TQueryCtx> &queryCtx, - const TActorId &parentId, + const TActorId &parentId, TLogoBlobsSnapshot &&logoBlobsSnapshot, TBarriersSnapshot &&barriersSnapshot, TEvBlobStorage::TEvVGet::TPtr &ev, diff --git a/ydb/core/blobstorage/vdisk/query/query_readactor.cpp b/ydb/core/blobstorage/vdisk/query/query_readactor.cpp index d75d6e5d671..dfe5cff3c0b 100644 --- a/ydb/core/blobstorage/vdisk/query/query_readactor.cpp +++ b/ydb/core/blobstorage/vdisk/query/query_readactor.cpp @@ -14,7 +14,7 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// class TTReadBatcherActor : public TActorBootstrapped<TTReadBatcherActor> { TReadBatcherCtxPtr Ctx; - const TActorId NotifyID; + const TActorId NotifyID; std::shared_ptr<TReadBatcherResult> Result; const ui8 Priority; NWilson::TTraceId TraceId; @@ -118,7 +118,7 @@ namespace NKikimr { TTReadBatcherActor( TReadBatcherCtxPtr ctx, - const TActorId notifyID, + const TActorId notifyID, std::shared_ptr<TReadBatcherResult> result, ui8 priority, NWilson::TTraceId traceId, @@ -135,7 +135,7 @@ namespace NKikimr { IActor *CreateReadBatcherActor( TReadBatcherCtxPtr ctx, - const TActorId notifyID, + const TActorId notifyID, std::shared_ptr<TReadBatcherResult> result, ui8 priority, NWilson::TTraceId traceId, diff --git a/ydb/core/blobstorage/vdisk/query/query_readactor.h b/ydb/core/blobstorage/vdisk/query/query_readactor.h index 6700dfd9ac2..614625ee8b3 100644 --- a/ydb/core/blobstorage/vdisk/query/query_readactor.h +++ b/ydb/core/blobstorage/vdisk/query/query_readactor.h @@ -7,7 +7,7 @@ namespace NKikimr { IActor *CreateReadBatcherActor( TReadBatcherCtxPtr ctx, - const TActorId notifyID, + const TActorId notifyID, std::shared_ptr<TReadBatcherResult> result, ui8 priority, NWilson::TTraceId traceId, diff --git a/ydb/core/blobstorage/vdisk/query/query_readbatch.cpp b/ydb/core/blobstorage/vdisk/query/query_readbatch.cpp index 4e06d08514d..a883191f156 100644 --- a/ydb/core/blobstorage/vdisk/query/query_readbatch.cpp +++ b/ydb/core/blobstorage/vdisk/query/query_readbatch.cpp @@ -218,7 +218,7 @@ namespace NKikimr { return true; } - IActor *TReadBatcher::CreateAsyncDataReader(const TActorId ¬ifyID, + IActor *TReadBatcher::CreateAsyncDataReader(const TActorId ¬ifyID, ui8 priority, NWilson::TTraceId traceId, bool isRepl) { diff --git a/ydb/core/blobstorage/vdisk/query/query_statdb.h b/ydb/core/blobstorage/vdisk/query/query_statdb.h index a6f85891cdd..67aa2aeec54 100644 --- a/ydb/core/blobstorage/vdisk/query/query_statdb.h +++ b/ydb/core/blobstorage/vdisk/query/query_statdb.h @@ -42,7 +42,7 @@ namespace NKikimr { TLevelIndexStatActor( const TIntrusivePtr<THullCtx> &hullCtx, - const TActorId &parentId, + const TActorId &parentId, TLevelIndexSnapshot &&snapshot, TEvBlobStorage::TEvVDbStat::TPtr &ev, std::unique_ptr<TEvBlobStorage::TEvVDbStatResult> result) @@ -56,7 +56,7 @@ namespace NKikimr { private: TIntrusivePtr<THullCtx> HullCtx; - const TActorId ParentId; + const TActorId ParentId; TLevelIndexSnapshot Snapshot; TEvBlobStorage::TEvVDbStat::TPtr Ev; std::unique_ptr<TEvBlobStorage::TEvVDbStatResult> Result; diff --git a/ydb/core/blobstorage/vdisk/query/query_stathuge.cpp b/ydb/core/blobstorage/vdisk/query/query_stathuge.cpp index e6e2b3c048f..e4f14dba802 100644 --- a/ydb/core/blobstorage/vdisk/query/query_stathuge.cpp +++ b/ydb/core/blobstorage/vdisk/query/query_stathuge.cpp @@ -37,7 +37,7 @@ namespace NKikimr { THugeStatActor( TIntrusivePtr<THullCtx> hullCtx, const std::shared_ptr<THugeBlobCtx> &hugeBlobCtx, - const TActorId &parentId, + const TActorId &parentId, THullDsSnap &&fullSnap, TEvBlobStorage::TEvVDbStat::TPtr &ev, std::unique_ptr<TEvBlobStorage::TEvVDbStatResult> result) @@ -56,7 +56,7 @@ namespace NKikimr { TIntrusivePtr<THullCtx> HullCtx; std::shared_ptr<THugeBlobCtx> HugeBlobCtx; - const TActorId ParentId; + const TActorId ParentId; THullDsSnap FullSnap; TEvBlobStorage::TEvVDbStat::TPtr Ev; std::unique_ptr<TEvBlobStorage::TEvVDbStatResult> Result; @@ -397,7 +397,7 @@ namespace NKikimr { IActor *CreateHugeStatActor( TIntrusivePtr<THullCtx> hullCtx, const std::shared_ptr<THugeBlobCtx> &hugeBlobCtx, - const TActorId &parentId, + const TActorId &parentId, THullDsSnap &&fullSnap, TEvBlobStorage::TEvVDbStat::TPtr &ev, std::unique_ptr<TEvBlobStorage::TEvVDbStatResult> result) { diff --git a/ydb/core/blobstorage/vdisk/query/query_stathuge.h b/ydb/core/blobstorage/vdisk/query/query_stathuge.h index 0a2cf436547..eb6985db648 100644 --- a/ydb/core/blobstorage/vdisk/query/query_stathuge.h +++ b/ydb/core/blobstorage/vdisk/query/query_stathuge.h @@ -13,7 +13,7 @@ namespace NKikimr { IActor *CreateHugeStatActor( TIntrusivePtr<THullCtx> hullCtx, const std::shared_ptr<THugeBlobCtx> &hugeBlobCtx, - const TActorId &parentId, + const TActorId &parentId, THullDsSnap &&fullSnap, TEvBlobStorage::TEvVDbStat::TPtr &ev, std::unique_ptr<TEvBlobStorage::TEvVDbStatResult> result); diff --git a/ydb/core/blobstorage/vdisk/query/query_stattablet.cpp b/ydb/core/blobstorage/vdisk/query/query_stattablet.cpp index de8972e1082..16e25fb1a65 100644 --- a/ydb/core/blobstorage/vdisk/query/query_stattablet.cpp +++ b/ydb/core/blobstorage/vdisk/query/query_stattablet.cpp @@ -39,7 +39,7 @@ namespace NKikimr { } TTabletStatActor(TIntrusivePtr<THullCtx> hullCtx, - const TActorId &parentId, + const TActorId &parentId, THullDsSnap &&fullSnap, TEvBlobStorage::TEvVDbStat::TPtr &ev, std::unique_ptr<TEvBlobStorage::TEvVDbStatResult> result) @@ -56,7 +56,7 @@ namespace NKikimr { class TAggr; TIntrusivePtr<THullCtx> HullCtx; - const TActorId ParentId; + const TActorId ParentId; THullDsSnap FullSnap; TEvBlobStorage::TEvVDbStat::TPtr Ev; std::unique_ptr<TEvBlobStorage::TEvVDbStatResult> Result; @@ -362,7 +362,7 @@ namespace NKikimr { // CreateTabletStatActor //////////////////////////////////////////////////////////////////////////// IActor *CreateTabletStatActor(TIntrusivePtr<THullCtx> hullCtx, - const TActorId &parentId, + const TActorId &parentId, THullDsSnap &&fullSnap, TEvBlobStorage::TEvVDbStat::TPtr &ev, std::unique_ptr<TEvBlobStorage::TEvVDbStatResult> result) { diff --git a/ydb/core/blobstorage/vdisk/query/query_stattablet.h b/ydb/core/blobstorage/vdisk/query/query_stattablet.h index 6412b8c893f..1d6ae9e22ae 100644 --- a/ydb/core/blobstorage/vdisk/query/query_stattablet.h +++ b/ydb/core/blobstorage/vdisk/query/query_stattablet.h @@ -10,7 +10,7 @@ namespace NKikimr { // Report all this VDisk knows about the requested Tablet //////////////////////////////////////////////////////////////////////////// IActor *CreateTabletStatActor(TIntrusivePtr<THullCtx> hullCtx, - const TActorId &parentId, + const TActorId &parentId, THullDsSnap &&fullSnap, TEvBlobStorage::TEvVDbStat::TPtr &ev, std::unique_ptr<TEvBlobStorage::TEvVDbStatResult> result); diff --git a/ydb/core/blobstorage/vdisk/query/query_stream.h b/ydb/core/blobstorage/vdisk/query/query_stream.h index f407d424e13..1f5236284d0 100644 --- a/ydb/core/blobstorage/vdisk/query/query_stream.h +++ b/ydb/core/blobstorage/vdisk/query/query_stream.h @@ -14,7 +14,7 @@ namespace NKikimr { TIntrusivePtr<THullCtx> HullCtx; std::optional<THullDsSnap> FullSnap; - TActorId SkeletonId; + TActorId SkeletonId; const TString StreamId; TInstant Deadline; ui64 SequenceId = 1; @@ -269,7 +269,7 @@ namespace NKikimr { , StreamId(std::move(ev->Get()->StreamId)) {} - void Bootstrap(const TActorId& parentId, const TActorContext& ctx) { + void Bootstrap(const TActorId& parentId, const TActorContext& ctx) { Become(&TLevelIndexStreamActor::StateFunc); // record all snapped data into the processor and release snapshot diff --git a/ydb/core/blobstorage/vdisk/repl/blobstorage_hullrepljob.cpp b/ydb/core/blobstorage/vdisk/repl/blobstorage_hullrepljob.cpp index 20f542f65a7..3a95c22022f 100644 --- a/ydb/core/blobstorage/vdisk/repl/blobstorage_hullrepljob.cpp +++ b/ydb/core/blobstorage/vdisk/repl/blobstorage_hullrepljob.cpp @@ -40,7 +40,7 @@ namespace NKikimr { std::unique_ptr<TRecoveryMachine> RecoveryMachine; std::shared_ptr<TReplCtx> ReplCtx; TIntrusivePtr<TBlobStorageGroupInfo> GInfo; - TActorId Recipient; + TActorId Recipient; TLogoBlobID StartKey; TEvReplFinished::TInfoPtr ReplInfo; TBlobIdQueuePtr BlobsToReplicatePtr; @@ -210,7 +210,7 @@ namespace NKikimr { std::shared_ptr<TReplCtx> ReplCtx; TIntrusivePtr<TBlobStorageGroupInfo> GInfo; - const TActorId ParentId; + const TActorId ParentId; const TLogoBlobID StartKey; TVector<TVDiskProxyPtr> MergeHeap; TEvReplFinished::TInfoPtr ReplInfo; @@ -791,7 +791,7 @@ namespace NKikimr { THullReplJobActor( std::shared_ptr<TReplCtx> replCtx, - const TActorId &parentId, + const TActorId &parentId, const TLogoBlobID &startKey, TQueueActorMapPtr&& queueActorMapPtr, TBlobIdQueuePtr&& blobsToReplicatePtr, @@ -825,7 +825,7 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// IActor *CreateReplJobActor( std::shared_ptr<TReplCtx> replCtx, - const TActorId &parentId, + const TActorId &parentId, const TLogoBlobID &startKey, TQueueActorMapPtr queueActorMapPtr, TBlobIdQueuePtr blobsToReplicatePtr, diff --git a/ydb/core/blobstorage/vdisk/repl/blobstorage_hullrepljob.h b/ydb/core/blobstorage/vdisk/repl/blobstorage_hullrepljob.h index ce330ba9c3e..8f7d43fb29a 100644 --- a/ydb/core/blobstorage/vdisk/repl/blobstorage_hullrepljob.h +++ b/ydb/core/blobstorage/vdisk/repl/blobstorage_hullrepljob.h @@ -12,7 +12,7 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// IActor *CreateReplJobActor( std::shared_ptr<TReplCtx> replCtx, - const TActorId &parentId, + const TActorId &parentId, const TLogoBlobID &startKey, TQueueActorMapPtr queueActorMapPtr, TBlobIdQueuePtr blobsToReplicatePtr, diff --git a/ydb/core/blobstorage/vdisk/repl/blobstorage_hullreplwritesst_ut.cpp b/ydb/core/blobstorage/vdisk/repl/blobstorage_hullreplwritesst_ut.cpp index 5aecf2ccf7f..cacb1405938 100644 --- a/ydb/core/blobstorage/vdisk/repl/blobstorage_hullreplwritesst_ut.cpp +++ b/ydb/core/blobstorage/vdisk/repl/blobstorage_hullreplwritesst_ut.cpp @@ -23,7 +23,7 @@ std::shared_ptr<TReplCtx> CreateReplCtx(TVector<TVDiskID>& vdisks, const TIntrus hugeBlobCtx, nullptr, info, - TActorId(), + TActorId(), vdiskCfg, std::make_shared<std::atomic_uint64_t>()); return replCtx; diff --git a/ydb/core/blobstorage/vdisk/repl/blobstorage_repl.cpp b/ydb/core/blobstorage/vdisk/repl/blobstorage_repl.cpp index 58e1f3b8651..34be219abce 100644 --- a/ydb/core/blobstorage/vdisk/repl/blobstorage_repl.cpp +++ b/ydb/core/blobstorage/vdisk/repl/blobstorage_repl.cpp @@ -10,7 +10,7 @@ #include <ydb/core/blobstorage/vdisk/common/vdisk_queues.h> #include <ydb/core/blobstorage/vdisk/hulldb/base/hullbase_logoblob.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/generic/queue.h> #include <util/generic/deque.h> @@ -163,7 +163,7 @@ namespace NKikimr { TQueueActorMapPtr QueueActorMapPtr; TBlobIdQueuePtr BlobsToReplicatePtr; TBlobIdQueuePtr UnreplicatedBlobsPtr = std::make_shared<TBlobIdQueue>(); - TActorId ReplJobActorId; + TActorId ReplJobActorId; std::list<std::optional<TDonorQueueItem>> DonorQueue; std::deque<std::pair<TVDiskID, TActorId>> Donors; std::set<TVDiskID> ConnectedPeerDisks, ConnectedDonorDisks; diff --git a/ydb/core/blobstorage/vdisk/repl/blobstorage_replbroker.cpp b/ydb/core/blobstorage/vdisk/repl/blobstorage_replbroker.cpp index f713b68194d..543f0a537db 100644 --- a/ydb/core/blobstorage/vdisk/repl/blobstorage_replbroker.cpp +++ b/ydb/core/blobstorage/vdisk/repl/blobstorage_replbroker.cpp @@ -8,15 +8,15 @@ namespace NKikimr { class TReplBroker : public TActor<TReplBroker> { // queue of senders waiting for token per PDisk; the first one in queue always has a token - THashMap<ui32, TDeque<TActorId>> VDiskQ; - THashMap<TActorId, ui32> SenderToPDisk; + THashMap<ui32, TDeque<TActorId>> VDiskQ; + THashMap<TActorId, ui32> SenderToPDisk; struct TMemQueueItem { - TActorId Sender; + TActorId Sender; ui64 Cookie; ui64 Bytes; - TMemQueueItem(const TActorId& sender, ui64 cookie, ui64 bytes) + TMemQueueItem(const TActorId& sender, ui64 cookie, ui64 bytes) : Sender(sender) , Cookie(cookie) , Bytes(bytes) @@ -26,10 +26,10 @@ namespace NKikimr { i64 MemFree; struct TMemToken { - TActorId Sender; + TActorId Sender; ui64 Bytes; - TMemToken(const TActorId& sender, ui64 bytes) + TMemToken(const TActorId& sender, ui64 bytes) : Sender(sender) , Bytes(bytes) {} diff --git a/ydb/core/blobstorage/vdisk/repl/blobstorage_replctx.h b/ydb/core/blobstorage/vdisk/repl/blobstorage_replctx.h index 0cb9ae5595a..55406e76ec8 100644 --- a/ydb/core/blobstorage/vdisk/repl/blobstorage_replctx.h +++ b/ydb/core/blobstorage/vdisk/repl/blobstorage_replctx.h @@ -16,7 +16,7 @@ namespace NKikimr { std::shared_ptr<THugeBlobCtx> HugeBlobCtx; TIntrusivePtr<THullDs> HullDs; TIntrusivePtr<TBlobStorageGroupInfo> GInfo; - TActorId SkeletonId; + TActorId SkeletonId; NMonGroup::TReplGroup MonGroup; // settings @@ -30,7 +30,7 @@ namespace NKikimr { std::shared_ptr<THugeBlobCtx> hugeBlobCtx, TIntrusivePtr<THullDs> hullDs, TIntrusivePtr<TBlobStorageGroupInfo> info, - const TActorId &skeletonId, + const TActorId &skeletonId, TIntrusivePtr<TVDiskConfig> vdiskCfg, std::shared_ptr<std::atomic_uint64_t> pdiskWriteBytes, bool pausedAtStart = false) diff --git a/ydb/core/blobstorage/vdisk/repl/blobstorage_replproxy.cpp b/ydb/core/blobstorage/vdisk/repl/blobstorage_replproxy.cpp index d8ae59404eb..1f82e552bac 100644 --- a/ydb/core/blobstorage/vdisk/repl/blobstorage_replproxy.cpp +++ b/ydb/core/blobstorage/vdisk/repl/blobstorage_replproxy.cpp @@ -20,7 +20,7 @@ namespace NKikimr { std::shared_ptr<TReplCtx> replCtx, TTrackableVector<TVDiskProxy::TScheduledBlob>&& ids, const TVDiskID& vdiskId, - const TActorId& serviceId); + const TActorId& serviceId); //////////////////////////////////////////////////////////////////////////// // TVDiskProxy @@ -114,7 +114,7 @@ namespace NKikimr { TActorId Recipient; TTrackableVector<TVDiskProxy::TScheduledBlob> Ids; TVDiskID VDiskId; - TActorId ServiceId; + TActorId ServiceId; TProxyStat Stat; ui32 SendIdx; ui32 CurPosIdx; @@ -443,7 +443,7 @@ namespace NKikimr { std::shared_ptr<TReplCtx> replCtx, TTrackableVector<TVDiskProxy::TScheduledBlob>&& ids, const TVDiskID& vdiskId, - const TActorId& serviceId) + const TActorId& serviceId) : TActorBootstrapped<TVDiskProxyActor>() , ReplCtx(std::move(replCtx)) , GType(ReplCtx->VCtx->Top->GType) @@ -469,7 +469,7 @@ namespace NKikimr { IActor *CreateVDiskProxyActor(std::shared_ptr<TReplCtx> replCtx, TTrackableVector<TVDiskProxy::TScheduledBlob>&& ids, const TVDiskID& vdiskId, - const TActorId& serviceId) { + const TActorId& serviceId) { return new TVDiskProxyActor(std::move(replCtx), std::move(ids), vdiskId, serviceId); } diff --git a/ydb/core/blobstorage/vdisk/repl/blobstorage_replproxy.h b/ydb/core/blobstorage/vdisk/repl/blobstorage_replproxy.h index 120ec6060f7..e05f54cd079 100644 --- a/ydb/core/blobstorage/vdisk/repl/blobstorage_replproxy.h +++ b/ydb/core/blobstorage/vdisk/repl/blobstorage_replproxy.h @@ -261,7 +261,7 @@ namespace NKikimr { TVDiskProxy( std::shared_ptr<TReplCtx> replCtx, const TVDiskID &vdisk, - const TActorId &serviceID); + const TActorId &serviceID); TActorId Run(const TActorId& parentId); void SendNextRequest(); diff --git a/ydb/core/blobstorage/vdisk/repl/blobstorage_replrecoverymachine_ut.cpp b/ydb/core/blobstorage/vdisk/repl/blobstorage_replrecoverymachine_ut.cpp index f8a40ff21cf..3ff7efb53e2 100644 --- a/ydb/core/blobstorage/vdisk/repl/blobstorage_replrecoverymachine_ut.cpp +++ b/ydb/core/blobstorage/vdisk/repl/blobstorage_replrecoverymachine_ut.cpp @@ -73,7 +73,7 @@ namespace NKikimr { hugeBlobCtx, nullptr, info, - TActorId(), + TActorId(), vdiskCfg, std::make_unique<std::atomic_uint64_t>()); diff --git a/ydb/core/blobstorage/vdisk/repl/ya.make b/ydb/core/blobstorage/vdisk/repl/ya.make index 8245ee674ec..8c237578909 100644 --- a/ydb/core/blobstorage/vdisk/repl/ya.make +++ b/ydb/core/blobstorage/vdisk/repl/ya.make @@ -4,7 +4,7 @@ OWNER(fomichev) PEERDIR( library/cpp/actors/core - library/cpp/monlib/service/pages + library/cpp/monlib/service/pages ydb/core/base ydb/core/blobstorage/backpressure ydb/core/blobstorage/groupinfo diff --git a/ydb/core/blobstorage/vdisk/skeleton/blobstorage_db.h b/ydb/core/blobstorage/vdisk/skeleton/blobstorage_db.h index 5231a4bc6d6..2bd127bbba3 100644 --- a/ydb/core/blobstorage/vdisk/skeleton/blobstorage_db.h +++ b/ydb/core/blobstorage/vdisk/skeleton/blobstorage_db.h @@ -16,7 +16,7 @@ namespace NKikimr { // TGuardedActorID //////////////////////////////////////////////////////////////////////////// class TGuardedActorID { - TActorId ActorID; + TActorId ActorID; TAtomic SetUp; public: @@ -25,13 +25,13 @@ namespace NKikimr { , SetUp(0) {} - void Set(const TActorId &id) { + void Set(const TActorId &id) { ActorID = id; AtomicAdd(SetUp, 1); } - operator TActorId() { - return SetUp ? ActorID : TActorId(); + operator TActorId() { + return SetUp ? ActorID : TActorId(); } operator bool() { diff --git a/ydb/core/blobstorage/vdisk/skeleton/blobstorage_monactors.cpp b/ydb/core/blobstorage/vdisk/skeleton/blobstorage_monactors.cpp index ac90be8c0d5..44b3c58c9c4 100644 --- a/ydb/core/blobstorage/vdisk/skeleton/blobstorage_monactors.cpp +++ b/ydb/core/blobstorage/vdisk/skeleton/blobstorage_monactors.cpp @@ -6,7 +6,7 @@ #include <library/cpp/actors/core/hfunc.h> #include <library/cpp/actors/core/actor_bootstrapped.h> #include <library/cpp/actors/core/mon.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> namespace NKikimr { @@ -14,7 +14,7 @@ namespace NKikimr { // TMonErrorActor //////////////////////////////////////////////////////////////////////////// class TMonErrorActor : public TActorBootstrapped<TMonErrorActor> { - const TActorId NotifyId; + const TActorId NotifyId; NMon::TEvHttpInfo::TPtr Ev; const TString Explanation; @@ -31,7 +31,7 @@ namespace NKikimr { return NKikimrServices::TActivity::BS_MON_ERROR; } - TMonErrorActor(const TActorId ¬ifyId, + TMonErrorActor(const TActorId ¬ifyId, NMon::TEvHttpInfo::TPtr &ev, const TString &explanation) : TActorBootstrapped<TMonErrorActor>() @@ -47,8 +47,8 @@ namespace NKikimr { class TSkeletonMonMainPageActor : public TActorBootstrapped<TSkeletonMonMainPageActor> { TIntrusivePtr<TDb> Db; NMon::TEvHttpInfo::TPtr Ev; - const TActorId NotifyId; - const TActorId LocalRecovActorID; + const TActorId NotifyId; + const TActorId LocalRecovActorID; unsigned Counter; TString SkeletonState; @@ -69,7 +69,7 @@ namespace NKikimr { void Bootstrap(const TActorContext &ctx) { // send requests to all actors - if (bool(TActorId(Db->SkeletonID))) { + if (bool(TActorId(Db->SkeletonID))) { ctx.Send(Db->SkeletonID, new NMon::TEvHttpInfo(Ev->Get()->Request, TDbMon::SkeletonStateId)); Counter++; ctx.Send(Db->SkeletonID, new NMon::TEvHttpInfo(Ev->Get()->Request, TDbMon::HullInfoId)); @@ -78,27 +78,27 @@ namespace NKikimr { Counter++; } - if (bool(TActorId(Db->SyncerID))) { + if (bool(TActorId(Db->SyncerID))) { ctx.Send(Db->SyncerID, new NMon::TEvHttpInfo(Ev->Get()->Request, TDbMon::SyncerInfoId)); Counter++; } - if (bool(TActorId(Db->SyncLogID))) { + if (bool(TActorId(Db->SyncLogID))) { ctx.Send(Db->SyncLogID, new NMon::TEvHttpInfo(Ev->Get()->Request, TDbMon::SyncLogId)); Counter++; } - if (bool(TActorId(Db->ReplID))) { + if (bool(TActorId(Db->ReplID))) { ctx.Send(Db->ReplID, new NMon::TEvHttpInfo(Ev->Get()->Request, TDbMon::ReplId)); Counter++; } - if (bool(TActorId(Db->LogCutterID))) { + if (bool(TActorId(Db->LogCutterID))) { ctx.Send(Db->LogCutterID, new NMon::TEvHttpInfo(Ev->Get()->Request, TDbMon::LogCutterId)); Counter++; } - if (bool(TActorId(Db->HugeKeeperID))) { + if (bool(TActorId(Db->HugeKeeperID))) { ctx.Send(Db->HugeKeeperID, new NMon::TEvHttpInfo(Ev->Get()->Request, TDbMon::HugeKeeperId)); Counter++; } @@ -108,7 +108,7 @@ namespace NKikimr { Counter++; } - if (bool(TActorId(Db->DskSpaceTrackerID))) { + if (bool(TActorId(Db->DskSpaceTrackerID))) { ctx.Send(Db->DskSpaceTrackerID, new NMon::TEvHttpInfo(Ev->Get()->Request, TDbMon::DskSpaceTrackerId)); Counter++; } @@ -118,7 +118,7 @@ namespace NKikimr { Counter++; } - if (bool(TActorId(Db->AnubisRunnerID))) { + if (bool(TActorId(Db->AnubisRunnerID))) { ctx.Send(Db->AnubisRunnerID, new NMon::TEvHttpInfo(Ev->Get()->Request, TDbMon::AnubisRunnerId)); Counter++; } @@ -224,8 +224,8 @@ namespace NKikimr { TSkeletonMonMainPageActor(TIntrusivePtr<TDb> &db, NMon::TEvHttpInfo::TPtr &ev, - const TActorId notifyId, - const TActorId &localRecovActorID) + const TActorId notifyId, + const TActorId &localRecovActorID) : TActorBootstrapped<TSkeletonMonMainPageActor>() , Db(db) , Ev(ev) @@ -237,8 +237,8 @@ namespace NKikimr { IActor* CreateSkeletonMonRequestHandler(TIntrusivePtr<TDb> &db, NMon::TEvHttpInfo::TPtr &ev, - const TActorId notifyId, - const TActorId &localRecovActorID) { + const TActorId notifyId, + const TActorId &localRecovActorID) { return new TSkeletonMonMainPageActor(db, ev, notifyId, localRecovActorID); } @@ -301,8 +301,8 @@ namespace NKikimr { const TVDiskID SelfVDiskId; TIntrusivePtr<TVDiskConfig> Cfg; std::shared_ptr<TBlobStorageGroupInfo::TTopology> Top; - const TActorId NotifyId; - const TActorId SkeletonFrontID; + const TActorId NotifyId; + const TActorId SkeletonFrontID; NMon::TEvHttpInfo::TPtr Ev; TLogoBlobID From; TLogoBlobID To; @@ -479,10 +479,10 @@ namespace NKikimr { } TSkeletonFrontMonLogoBlobsQueryActor(const TVDiskID &selfVDiskId, - const TActorId ¬ifyId, + const TActorId ¬ifyId, TIntrusivePtr<TVDiskConfig> cfg, const std::shared_ptr<TBlobStorageGroupInfo::TTopology> &top, - const TActorId &skeletonFrontID, + const TActorId &skeletonFrontID, NMon::TEvHttpInfo::TPtr &ev) : TActorBootstrapped<TSkeletonFrontMonLogoBlobsQueryActor>() , SelfVDiskId(selfVDiskId) @@ -507,8 +507,8 @@ namespace NKikimr { const TVDiskID SelfVDiskId; TIntrusivePtr<TVDiskConfig> Cfg; std::shared_ptr<TBlobStorageGroupInfo::TTopology> Top; - const TActorId NotifyId; - const TActorId SkeletonFrontID; + const TActorId NotifyId; + const TActorId SkeletonFrontID; NMon::TEvHttpInfo::TPtr Ev; TKeyBarrier From; TKeyBarrier To; @@ -668,10 +668,10 @@ namespace NKikimr { } TSkeletonFrontMonBarriersQueryActor(const TVDiskID &selfVDiskId, - const TActorId ¬ifyId, + const TActorId ¬ifyId, TIntrusivePtr<TVDiskConfig> cfg, const std::shared_ptr<TBlobStorageGroupInfo::TTopology> &top, - const TActorId &skeletonFrontID, + const TActorId &skeletonFrontID, NMon::TEvHttpInfo::TPtr &ev) : TActorBootstrapped<TSkeletonFrontMonBarriersQueryActor>() , SelfVDiskId(selfVDiskId) @@ -695,8 +695,8 @@ namespace NKikimr { class TSkeletonFrontMonDbStatActor : public TActorBootstrapped<TSkeletonFrontMonDbStatActor> { const TVDiskID SelfVDiskId; TIntrusivePtr<TVDiskConfig> Cfg; - const TActorId NotifyId; - const TActorId SkeletonFrontID; + const TActorId NotifyId; + const TActorId SkeletonFrontID; NMon::TEvHttpInfo::TPtr Ev; const NKikimrBlobStorage::EDbStatAction Action; const TString Dbname; @@ -874,9 +874,9 @@ namespace NKikimr { } TSkeletonFrontMonDbStatActor(const TVDiskID &selfVDiskId, - const TActorId ¬ifyId, + const TActorId ¬ifyId, TIntrusivePtr<TVDiskConfig> cfg, - const TActorId &skeletonFrontID, + const TActorId &skeletonFrontID, NMon::TEvHttpInfo::TPtr &ev, NKikimrBlobStorage::EDbStatAction action, const TString &dbname) @@ -895,8 +895,8 @@ namespace NKikimr { // TSkeletonFrontMonMainPageActor //////////////////////////////////////////////////////////////////////////// class TSkeletonFrontMonMainPageActor : public TActorBootstrapped<TSkeletonFrontMonMainPageActor> { - const TActorId NotifyId; - const TActorId SkeletonID; + const TActorId NotifyId; + const TActorId SkeletonID; NMon::TEvHttpInfo::TPtr Ev; TString SkeletonFrontAnswer; TString SkeletonAnswer; @@ -960,8 +960,8 @@ namespace NKikimr { return NKikimrServices::TActivity::BS_MON_SF_MAIN_PAGE; } - TSkeletonFrontMonMainPageActor(const TActorId ¬ifyId, - const TActorId &skeletonID, + TSkeletonFrontMonMainPageActor(const TActorId ¬ifyId, + const TActorId &skeletonID, NMon::TEvHttpInfo::TPtr &ev, const TString &frontHtml) : TActorBootstrapped<TSkeletonFrontMonMainPageActor>() @@ -976,9 +976,9 @@ namespace NKikimr { // SKELETON FRONT MON REQUEST HANDLER //////////////////////////////////////////////////////////////////////////// IActor* CreateFrontSkeletonMonRequestHandler(const TVDiskID &selfVDiskId, - const TActorId ¬ifyId, - const TActorId &skeletonID, - const TActorId &skeletonFrontID, + const TActorId ¬ifyId, + const TActorId &skeletonID, + const TActorId &skeletonFrontID, TIntrusivePtr<TVDiskConfig> cfg, const std::shared_ptr<TBlobStorageGroupInfo::TTopology> &top, NMon::TEvHttpInfo::TPtr &ev, diff --git a/ydb/core/blobstorage/vdisk/skeleton/blobstorage_monactors.h b/ydb/core/blobstorage/vdisk/skeleton/blobstorage_monactors.h index 5e6c2142bac..95eacad7b58 100644 --- a/ydb/core/blobstorage/vdisk/skeleton/blobstorage_monactors.h +++ b/ydb/core/blobstorage/vdisk/skeleton/blobstorage_monactors.h @@ -10,12 +10,12 @@ namespace NKikimr { struct TVDiskConfig; IActor *CreateSkeletonMonRequestHandler(TIntrusivePtr<TDb> &db, NMon::TEvHttpInfo::TPtr &ev, - const TActorId notifyId, - const TActorId &localRecovActorID); + const TActorId notifyId, + const TActorId &localRecovActorID); IActor *CreateFrontSkeletonMonRequestHandler(const TVDiskID &selfVDiskId, - const TActorId ¬ifyId, - const TActorId &skeletonID, - const TActorId &skeletonFrontID, + const TActorId ¬ifyId, + const TActorId &skeletonID, + const TActorId &skeletonFrontID, TIntrusivePtr<TVDiskConfig> cfg, const std::shared_ptr<TBlobStorageGroupInfo::TTopology> &top, NMon::TEvHttpInfo::TPtr &ev, diff --git a/ydb/core/blobstorage/vdisk/skeleton/blobstorage_skeleton.cpp b/ydb/core/blobstorage/vdisk/skeleton/blobstorage_skeleton.cpp index 77cad57b798..96b813d3172 100644 --- a/ydb/core/blobstorage/vdisk/skeleton/blobstorage_skeleton.cpp +++ b/ydb/core/blobstorage/vdisk/skeleton/blobstorage_skeleton.cpp @@ -43,7 +43,7 @@ #include <ydb/core/blobstorage/pdisk/blobstorage_pdisk_internal_interface.h> #include <ydb/core/protos/node_whiteboard.pb.h> #include <ydb/core/node_whiteboard/node_whiteboard.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/generic/intrlist.h> @@ -295,14 +295,14 @@ namespace NKikimr { TLoggedRecVPut* CreateLoggedRec(TLsnSeg seg, bool confirmSyncLogAlso, const TLogoBlobID &id, const TIngress &ingress, TRope &&buffer, std::unique_ptr<TEvBlobStorage::TEvVPutResult> res, - const TActorId &sender, ui64 cookie) + const TActorId &sender, ui64 cookie) { return new TLoggedRecVPut(seg, confirmSyncLogAlso, id, ingress, std::move(buffer), std::move(res), sender, cookie); } TLoggedRecVMultiPutItem* CreateLoggedRec(TLsnSeg seg, bool confirmSyncLogAlso, const TLogoBlobID &id, const TIngress &ingress, TRope &&buffer, std::unique_ptr<TEvVMultiPutItemResult> res, - const TActorId &sender, ui64 cookie) + const TActorId &sender, ui64 cookie) { return new TLoggedRecVMultiPutItem(seg, confirmSyncLogAlso, id, ingress, std::move(buffer), std::move(res), sender, cookie); @@ -525,7 +525,7 @@ namespace NKikimr { IActor* vMultiPutActor = CreateSkeletonVMultiPutActor(SelfId(), statuses, oosStatus, ev, SkeletonFrontIDPtr, IFaceMonGroup->MultiPutResMsgsPtr(), Db->GetVDiskIncarnationGuid()); - NActors::TActorId vMultiPutActorId = ctx.Register(vMultiPutActor); + NActors::TActorId vMultiPutActorId = ctx.Register(vMultiPutActor); TLsnSeg lsnBatch; if (lsnCount) { @@ -1152,11 +1152,11 @@ namespace NKikimr { // STREAM QUERIES //////////////////////////////////////////////////////////////////////// - THashMap<TString, TActorId> MonStreamActors; + THashMap<TString, TActorId> MonStreamActors; void Handle(TEvBlobStorage::TEvMonStreamQuery::TPtr& ev, const TActorContext& ctx) { - TActorId& actorId = MonStreamActors[ev->Get()->StreamId]; - if (actorId == TActorId()) { + TActorId& actorId = MonStreamActors[ev->Get()->StreamId]; + if (actorId == TActorId()) { actorId = RunInBatchPool(ctx, CreateMonStreamActor(Hull->GetIndexSnapshot(), ev)); ActiveActors.insert(actorId); } @@ -1618,7 +1618,7 @@ namespace NKikimr { void Handle(TEvBlobStorage::TEvLocalRecoveryDone::TPtr &ev, const TActorContext &ctx) { LocalRecovInfo = ev->Get()->RecovInfo; - LocalDbRecoveryID = TActorId(); + LocalDbRecoveryID = TActorId(); ActiveActors.Erase(ev->Sender); PDiskCtx = ev->Get()->PDiskCtx; @@ -1678,14 +1678,14 @@ namespace NKikimr { // run LogCutter in the same mailbox TLogCutterCtx logCutterCtx = {VCtx, PDiskCtx, Db->LsnMngr, Config, - (TActorId)(Db->LoggerID)}; + (TActorId)(Db->LoggerID)}; Db->LogCutterID.Set(ctx.RegisterWithSameMailbox(CreateRecoveryLogCutter(std::move(logCutterCtx)))); ActiveActors.Insert(Db->LogCutterID); // keep forever // run HugeBlobKeeper TString localRecovInfoStr = Db->LocalRecoveryInfo ? Db->LocalRecoveryInfo->ToString() : TString("{}"); auto hugeKeeperCtx = std::make_shared<THugeKeeperCtx>(VCtx, PDiskCtx, Db->LsnMngr, - ctx.SelfID, (TActorId)(Db->LoggerID), (TActorId)(Db->LogCutterID), + ctx.SelfID, (TActorId)(Db->LoggerID), (TActorId)(Db->LogCutterID), localRecovInfoStr); auto hugeKeeper = CreateHullHugeBlobKeeper(hugeKeeperCtx, ev->Get()->RepairedHuge); Db->HugeKeeperID.Set(ctx.Register(hugeKeeper)); @@ -1856,7 +1856,7 @@ namespace NKikimr { switch (auto subrequest = ev->Get()->SubRequestId) { case 0: { // calculate id for the actor who'll tell us about local recovery - TActorId locRecovActor = LocalDbRecoveryID ? LocalDbRecoveryID : ctx.SelfID; + TActorId locRecovActor = LocalDbRecoveryID ? LocalDbRecoveryID : ctx.SelfID; auto aid = ctx.Register(CreateSkeletonMonRequestHandler(Db, ev, ctx.SelfID, locRecovActor)); ActiveActors.Insert(aid); break; @@ -2414,7 +2414,7 @@ namespace NKikimr { TSkeleton(TIntrusivePtr<TVDiskConfig> cfg, TIntrusivePtr<TBlobStorageGroupInfo> info, - const TActorId &skeletonFrontID, + const TActorId &skeletonFrontID, const TVDiskContextPtr &vctx) : TActorBootstrapped<TSkeleton>() , Config(cfg) @@ -2423,7 +2423,7 @@ namespace NKikimr { , GInfo(info) , Hull() , LocalRecovInfo() - , SkeletonFrontIDPtr(new TActorId(skeletonFrontID)) + , SkeletonFrontIDPtr(new TActorId(skeletonFrontID)) , LocalDbRecoveryID() , NodeWardenServiceId(MakeBlobStorageNodeWardenID(vctx->NodeId)) , SelfVDiskId(GInfo->GetVDiskId(VCtx->ShortSelfVDisk)) @@ -2454,8 +2454,8 @@ namespace NKikimr { TIntrusivePtr<TLocalRecoveryInfo> LocalRecovInfo; // just info we got after local recovery std::unique_ptr<TOverloadHandler> OverloadHandler; TActorIDPtr SkeletonFrontIDPtr; - TActorId LocalDbRecoveryID; - const TActorId NodeWardenServiceId; + TActorId LocalDbRecoveryID; + const TActorId NodeWardenServiceId; TVDiskID SelfVDiskId; TMaybe<ui64> DbBirthLsn; TActiveActors ActiveActors; @@ -2484,7 +2484,7 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// IActor* CreateVDiskSkeleton(const TIntrusivePtr<TVDiskConfig> &cfg, const TIntrusivePtr<TBlobStorageGroupInfo> &info, - const TActorId &skeletonFrontID, + const TActorId &skeletonFrontID, const TVDiskContextPtr &vctx) { return new TSkeleton(cfg, info, skeletonFrontID, vctx); } diff --git a/ydb/core/blobstorage/vdisk/skeleton/blobstorage_skeleton.h b/ydb/core/blobstorage/vdisk/skeleton/blobstorage_skeleton.h index 99eae9a70e1..5c9a6a81562 100644 --- a/ydb/core/blobstorage/vdisk/skeleton/blobstorage_skeleton.h +++ b/ydb/core/blobstorage/vdisk/skeleton/blobstorage_skeleton.h @@ -12,7 +12,7 @@ namespace NKikimr { IActor* CreateVDiskSkeleton(const TIntrusivePtr<TVDiskConfig> &cfg, const TIntrusivePtr<TBlobStorageGroupInfo> &info, - const TActorId &skeletonFrontID, + const TActorId &skeletonFrontID, const TVDiskContextPtr &vctx); } // NKikimr diff --git a/ydb/core/blobstorage/vdisk/skeleton/blobstorage_skeletonfront.cpp b/ydb/core/blobstorage/vdisk/skeleton/blobstorage_skeletonfront.cpp index db4d68d4343..08c32a8921e 100644 --- a/ydb/core/blobstorage/vdisk/skeleton/blobstorage_skeletonfront.cpp +++ b/ydb/core/blobstorage/vdisk/skeleton/blobstorage_skeletonfront.cpp @@ -22,7 +22,7 @@ #include <ydb/core/base/counters.h> #include <ydb/core/node_whiteboard/node_whiteboard.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/generic/set.h> #include <util/generic/maybe.h> @@ -592,7 +592,7 @@ namespace NKikimr { TIntrusivePtr<TBlobStorageGroupInfo> GInfo; std::shared_ptr<TBlobStorageGroupInfo::TTopology> Top; TVDiskID SelfVDiskId; - TActorId SkeletonId; + TActorId SkeletonId; TIntrusivePtr<NMonitoring::TDynamicCounters> VDiskCounters; TIntrusivePtr<NMonitoring::TDynamicCounters> SkeletonFrontGroup; NMonitoring::TDynamicCounters::TCounterPtr AccessDeniedMessages; @@ -621,7 +621,7 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////// // NOTIFICATIONS //////////////////////////////////////////////////////////////////////// - using TNotificationIDs = TSet<TActorId>; + using TNotificationIDs = TSet<TActorId>; TNotificationIDs NotificationIDs; template <class T> diff --git a/ydb/core/blobstorage/vdisk/skeleton/blobstorage_skeletonfront.h b/ydb/core/blobstorage/vdisk/skeleton/blobstorage_skeletonfront.h index b50852710ec..9044230ab1e 100644 --- a/ydb/core/blobstorage/vdisk/skeleton/blobstorage_skeletonfront.h +++ b/ydb/core/blobstorage/vdisk/skeleton/blobstorage_skeletonfront.h @@ -3,7 +3,7 @@ #include <ydb/core/base/blobstorage.h> #include <ydb/core/blobstorage/vdisk/common/blobstorage_vdisk_guids.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> namespace NKikimr { diff --git a/ydb/core/blobstorage/vdisk/skeleton/blobstorage_syncfull.cpp b/ydb/core/blobstorage/vdisk/skeleton/blobstorage_syncfull.cpp index adc9a68b018..5881368772c 100644 --- a/ydb/core/blobstorage/vdisk/skeleton/blobstorage_syncfull.cpp +++ b/ydb/core/blobstorage/vdisk/skeleton/blobstorage_syncfull.cpp @@ -37,8 +37,8 @@ namespace NKikimr { class THullSyncFullActor : public TActorBootstrapped<THullSyncFullActor> { TIntrusivePtr<TVDiskConfig> Config; TIntrusivePtr<THullCtx> HullCtx; - const TActorId ParentId; - const TActorId Recipient; + const TActorId ParentId; + const TActorId Recipient; THullDsSnap FullSnap; // keys are subject to change during the processing TKeyLogoBlob KeyLogoBlob; @@ -172,9 +172,9 @@ namespace NKikimr { THullSyncFullActor( const TIntrusivePtr<TVDiskConfig> &config, const TIntrusivePtr<THullCtx> &hullCtx, - const TActorId &parentId, + const TActorId &parentId, const TVDiskID &sourceVDisk, - const TActorId &recipient, + const TActorId &recipient, THullDsSnap &&fullSnap, const TKeyLogoBlob &keyLogoBlob, const TKeyBlock &keyBlock, @@ -204,9 +204,9 @@ namespace NKikimr { IActor *CreateHullSyncFullActor( const TIntrusivePtr<TVDiskConfig> &config, const TIntrusivePtr<THullCtx> &hullCtx, - const TActorId &parentId, + const TActorId &parentId, const TVDiskID &sourceVDisk, - const TActorId &recipient, + const TActorId &recipient, THullDsSnap &&fullSnap, const TKeyLogoBlob &keyLogoBlob, const TKeyBlock &keyBlock, diff --git a/ydb/core/blobstorage/vdisk/skeleton/blobstorage_syncfull.h b/ydb/core/blobstorage/vdisk/skeleton/blobstorage_syncfull.h index 723691ed3ec..50d8ea5acce 100644 --- a/ydb/core/blobstorage/vdisk/skeleton/blobstorage_syncfull.h +++ b/ydb/core/blobstorage/vdisk/skeleton/blobstorage_syncfull.h @@ -9,9 +9,9 @@ namespace NKikimr { IActor *CreateHullSyncFullActor( const TIntrusivePtr<TVDiskConfig> &config, const TIntrusivePtr<THullCtx> &hullCtx, - const TActorId &parentId, + const TActorId &parentId, const TVDiskID &sourceVDisk, - const TActorId &recipient, + const TActorId &recipient, THullDsSnap &&fullSnap, const TKeyLogoBlob &keyLogoBlob, const TKeyBlock &keyBlock, diff --git a/ydb/core/blobstorage/vdisk/skeleton/blobstorage_syncfullhandler.cpp b/ydb/core/blobstorage/vdisk/skeleton/blobstorage_syncfullhandler.cpp index beac6e6a9d1..11f708e717d 100644 --- a/ydb/core/blobstorage/vdisk/skeleton/blobstorage_syncfullhandler.cpp +++ b/ydb/core/blobstorage/vdisk/skeleton/blobstorage_syncfullhandler.cpp @@ -17,7 +17,7 @@ namespace NKikimr { TIntrusivePtr<TDb> Db; TIntrusivePtr<THullCtx> HullCtx; const TVDiskID SelfVDiskId; - const TActorId ParentId; + const TActorId ParentId; std::shared_ptr<THull> Hull; std::shared_ptr<NMonGroup::TVDiskIFaceGroup> IFaceMonGroup; TEvBlobStorage::TEvVSyncFull::TPtr Ev; @@ -42,7 +42,7 @@ namespace NKikimr { void Bootstrap(const TActorContext &ctx) { IFaceMonGroup->SyncFullMsgs()++; - TActorId recipient = Ev->Sender; + TActorId recipient = Ev->Sender; const ui64 cookie = Ev->Cookie; TSyncState clientSyncState(SyncStateFromSyncState(Record.GetSyncState())); @@ -97,7 +97,7 @@ namespace NKikimr { } // parse stage and keys - TActorId recipient = Ev->Sender; + TActorId recipient = Ev->Sender; const NKikimrBlobStorage::ESyncFullStage stage = Record.GetStage(); const TLogoBlobID logoBlobFrom = LogoBlobIDFromLogoBlobID(Record.GetLogoBlobFrom()); const ui64 blockTabletFrom = Record.GetBlockTabletFrom(); @@ -158,7 +158,7 @@ namespace NKikimr { TVSyncFullHandler(const TIntrusivePtr<TDb> &db, const TIntrusivePtr<THullCtx> &hullCtx, const TVDiskID &selfVDiskId, - const TActorId &parentId, + const TActorId &parentId, const std::shared_ptr<THull> &hull, const std::shared_ptr<NMonGroup::TVDiskIFaceGroup> &ifaceMonGroup, TEvBlobStorage::TEvVSyncFull::TPtr &ev, @@ -190,7 +190,7 @@ namespace NKikimr { IActor *CreateHullSyncFullHandler(const TIntrusivePtr<TDb> &db, const TIntrusivePtr<THullCtx> &hullCtx, const TVDiskID &selfVDiskId, - const TActorId &parentId, + const TActorId &parentId, const std::shared_ptr<THull> &hull, const std::shared_ptr<NMonGroup::TVDiskIFaceGroup> &ifaceMonGroup, TEvBlobStorage::TEvVSyncFull::TPtr &ev, diff --git a/ydb/core/blobstorage/vdisk/skeleton/blobstorage_syncfullhandler.h b/ydb/core/blobstorage/vdisk/skeleton/blobstorage_syncfullhandler.h index dfe0212954b..7d4c29253aa 100644 --- a/ydb/core/blobstorage/vdisk/skeleton/blobstorage_syncfullhandler.h +++ b/ydb/core/blobstorage/vdisk/skeleton/blobstorage_syncfullhandler.h @@ -15,7 +15,7 @@ namespace NKikimr { IActor *CreateHullSyncFullHandler(const TIntrusivePtr<TDb> &db, const TIntrusivePtr<THullCtx> &hullCtx, const TVDiskID &selfVDiskId, - const TActorId &parentId, + const TActorId &parentId, const std::shared_ptr<THull> &hull, const std::shared_ptr<NMonGroup::TVDiskIFaceGroup> &ifaceMonGroup, TEvBlobStorage::TEvVSyncFull::TPtr &ev, diff --git a/ydb/core/blobstorage/vdisk/skeleton/skeleton_loggedrec.cpp b/ydb/core/blobstorage/vdisk/skeleton/skeleton_loggedrec.cpp index f2da333effc..ce4615fb5d4 100644 --- a/ydb/core/blobstorage/vdisk/skeleton/skeleton_loggedrec.cpp +++ b/ydb/core/blobstorage/vdisk/skeleton/skeleton_loggedrec.cpp @@ -24,7 +24,7 @@ namespace NKikimr { const TIngress &ingress, TRope &&buffer, std::unique_ptr<TEvBlobStorage::TEvVPutResult> result, - const TActorId &recipient, + const TActorId &recipient, ui64 recipientCookie) : ILoggedRec(seg, confirmSyncLogAlso) , Id(id) @@ -57,7 +57,7 @@ namespace NKikimr { const TIngress &ingress, TRope &&buffer, std::unique_ptr<TEvVMultiPutItemResult> result, - const TActorId &recipient, + const TActorId &recipient, ui64 recipientCookie) : ILoggedRec(seg, confirmSyncLogAlso) , Id(id) @@ -88,7 +88,7 @@ namespace NKikimr { TLoggedRecVPutHuge::TLoggedRecVPutHuge( TLsnSeg seg, bool confirmSyncLogAlso, - const TActorId &hugeKeeperId, + const TActorId &hugeKeeperId, TEvHullLogHugeBlob::TPtr ev) : ILoggedRec(seg, confirmSyncLogAlso) , HugeKeeperId(hugeKeeperId) @@ -121,7 +121,7 @@ namespace NKikimr { ui32 gen, ui64 issuerGuid, std::unique_ptr<TEvBlobStorage::TEvVBlockResult> result, - const TActorId &recipient, + const TActorId &recipient, ui64 recipientCookie) : ILoggedRec(seg, confirmSyncLogAlso) , TabletId(tabletId) @@ -133,7 +133,7 @@ namespace NKikimr { {} void TLoggedRecVBlock::Replay(THull &hull, const TActorContext &ctx, const IActor& actor) { - auto replySender = [&ctx, &actor] (const TActorId &id, ui64 cookie, IEventBase *msg) { + auto replySender = [&ctx, &actor] (const TActorId &id, ui64 cookie, IEventBase *msg) { SendVDiskResponse(ctx, id, msg, actor, cookie); }; @@ -184,7 +184,7 @@ namespace NKikimr { {} void TLoggedRecLocalSyncData::Replay(THull &hull, const TActorContext &ctx, const IActor& actor) { - auto replySender = [&ctx, &actor] (const TActorId &id, ui64 cookie, IEventBase *msg) { + auto replySender = [&ctx, &actor] (const TActorId &id, ui64 cookie, IEventBase *msg) { SendVDiskResponse(ctx, id, msg, actor, cookie); }; @@ -240,7 +240,7 @@ namespace NKikimr { TLsnSeg seg, bool confirmSyncLogAlso, std::unique_ptr<TEvDelLogoBlobDataSyncLogResult> result, - const TActorId &recipient, + const TActorId &recipient, ui64 recipientCookie) : ILoggedRec(seg, confirmSyncLogAlso) , Result(std::move(result)) diff --git a/ydb/core/blobstorage/vdisk/skeleton/skeleton_loggedrec.h b/ydb/core/blobstorage/vdisk/skeleton/skeleton_loggedrec.h index f30f62b25f9..50a7a06d2af 100644 --- a/ydb/core/blobstorage/vdisk/skeleton/skeleton_loggedrec.h +++ b/ydb/core/blobstorage/vdisk/skeleton/skeleton_loggedrec.h @@ -57,7 +57,7 @@ namespace NKikimr { TIngress Ingress; TRope Buffer; std::unique_ptr<TEvBlobStorage::TEvVPutResult> Result; - TActorId Recipient; + TActorId Recipient; ui64 RecipientCookie; }; @@ -76,7 +76,7 @@ namespace NKikimr { TIngress Ingress; TRope Buffer; std::unique_ptr<TEvVMultiPutItemResult> Result; - TActorId Recipient; + TActorId Recipient; ui64 RecipientCookie; }; @@ -85,12 +85,12 @@ namespace NKikimr { /////////////////////////////////////////////////////////////////////////////////////////////////////// class TLoggedRecVPutHuge : public ILoggedRec { public: - TLoggedRecVPutHuge(TLsnSeg seg, bool confirmSyncLogAlso, const TActorId &hugeKeeperId, + TLoggedRecVPutHuge(TLsnSeg seg, bool confirmSyncLogAlso, const TActorId &hugeKeeperId, TEvHullLogHugeBlob::TPtr ev); void Replay(THull &hull, const TActorContext &ctx, const IActor& actor) override; private: - const TActorId HugeKeeperId; + const TActorId HugeKeeperId; TEvHullLogHugeBlob::TPtr Ev; }; @@ -108,7 +108,7 @@ namespace NKikimr { ui32 Gen; ui64 IssuerGuid; std::unique_ptr<TEvBlobStorage::TEvVBlockResult> Result; - TActorId Recipient; + TActorId Recipient; ui64 RecipientCookie; }; @@ -182,7 +182,7 @@ namespace NKikimr { private: std::unique_ptr<TEvDelLogoBlobDataSyncLogResult> Result; - TActorId Recipient; + TActorId Recipient; ui64 RecipientCookie; }; diff --git a/ydb/core/blobstorage/vdisk/skeleton/skeleton_oos_tracker.cpp b/ydb/core/blobstorage/vdisk/skeleton/skeleton_oos_tracker.cpp index 0bb571d3f43..8af5bd8dc22 100644 --- a/ydb/core/blobstorage/vdisk/skeleton/skeleton_oos_tracker.cpp +++ b/ydb/core/blobstorage/vdisk/skeleton/skeleton_oos_tracker.cpp @@ -5,7 +5,7 @@ #include <ydb/core/blobstorage/vdisk/common/vdisk_mongroups.h> #include <ydb/core/blobstorage/vdisk/common/vdisk_mon.h> #include <ydb/core/blobstorage/vdisk/common/vdisk_pdiskctx.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> namespace NKikimr { diff --git a/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer.cpp b/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer.cpp index a0b3da11fc6..bf0179b4ed9 100644 --- a/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer.cpp +++ b/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer.cpp @@ -12,7 +12,7 @@ #include <ydb/core/blobstorage/vdisk/common/vdisk_events.h> #include <ydb/core/blobstorage/vdisk/synclog/blobstorage_synclog_public_events.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <library/cpp/actors/core/mon.h> using namespace NKikimrServices; @@ -55,7 +55,7 @@ namespace NKikimr { private: struct TItemAndSeq { - TActorId ActorId; + TActorId ActorId; std::unique_ptr<TItem> Item; ui64 SeqNum; @@ -79,15 +79,15 @@ namespace NKikimr { class TSyncerHttpInfoActor : public TActorBootstrapped<TSyncerHttpInfoActor> { TIntrusivePtr<TSyncerContext> SyncerCtx; NMon::TEvHttpInfo::TPtr Ev; - const TActorId ReplyId; - const TActorId NotifyId; - const TActorId SchedulerId; + const TActorId ReplyId; + const TActorId NotifyId; + const TActorId SchedulerId; const TString LogAndPhase; friend class TActorBootstrapped<TSyncerHttpInfoActor>; void Bootstrap(const TActorContext &ctx) { - if (SchedulerId == TActorId()) { + if (SchedulerId == TActorId()) { RenderHtmlAndReply(ctx, TString()); } else { Become(&TThis::StateFunc); @@ -137,8 +137,8 @@ namespace NKikimr { TSyncerHttpInfoActor(TIntrusivePtr<TSyncerContext> &sc, NMon::TEvHttpInfo::TPtr &ev, - const TActorId ¬ifyId, - const TActorId &schedulerId, + const TActorId ¬ifyId, + const TActorId &schedulerId, const TString &logAndPhase) : TActorBootstrapped<TSyncerHttpInfoActor>() , SyncerCtx(sc) @@ -169,11 +169,11 @@ namespace NKikimr { TIntrusivePtr<TBlobStorageGroupInfo> GInfo; TIntrusivePtr<TSyncerData> SyncerData; TLocalSyncerState LocalSyncerState; - TActorId CommitterId; - TActorId SchedulerId; - TActorId GuidRecoveryId; - TActorId RecoverLostDataId; - TVector<TActorId> PropagatorIds; + TActorId CommitterId; + TActorId SchedulerId; + TActorId GuidRecoveryId; + TActorId RecoverLostDataId; + TVector<TActorId> PropagatorIds; EPhase Phase = TPhaseVal::PhaseNone; TActiveActors ActiveActors; std::unique_ptr<NSyncer::TOutcome> GuidRecovOutcome; @@ -203,7 +203,7 @@ namespace NKikimr { PropagatorIds.reserve(SyncerCtx->VCtx->Top->GetTotalVDisksNum()); for (auto &x : neighbors) { const TVDiskID vd = GInfo->GetVDiskId(x.OrderNumber); - const TActorId va = GInfo->GetActorId(x.OrderNumber); + const TActorId va = GInfo->GetActorId(x.OrderNumber); auto aid = ctx.Register(CreateSyncerGuidPropagator(SyncerCtx->VCtx, selfVDiskId, vd, @@ -228,7 +228,7 @@ namespace NKikimr { void Handle(TEvVDiskGuidRecovered::TPtr &ev, const TActorContext &ctx) { ActiveActors.Erase(ev->Sender); - GuidRecoveryId = TActorId(); + GuidRecoveryId = TActorId(); GuidRecovOutcome = std::make_unique<NSyncer::TOutcome>(std::move(ev->Get()->Outcome)); switch (GuidRecovOutcome->Decision) { @@ -295,7 +295,7 @@ namespace NKikimr { void Handle(TEvSyncerLostDataRecovered::TPtr &ev, const TActorContext &ctx) { ActiveActors.Erase(ev->Sender); - RecoverLostDataId = TActorId(); + RecoverLostDataId = TActorId(); LocalSyncerState = ev->Get()->LocalSyncerState; StandardMode(ctx); } @@ -431,7 +431,7 @@ namespace NKikimr { void Handle(NMon::TEvHttpInfo::TPtr &ev, const TActorContext &ctx) { Y_VERIFY(ev->Get()->SubRequestId == TDbMon::SyncerInfoId); - TActorId schId; + TActorId schId; switch (Phase) { case TPhaseVal::PhaseStandardMode: { schId = SchedulerId; @@ -515,7 +515,7 @@ namespace NKikimr { GInfo = msg->NewInfo; // reconfigure guid recovery actor - Y_VERIFY(GuidRecoveryId != TActorId()); + Y_VERIFY(GuidRecoveryId != TActorId()); ctx.Send(GuidRecoveryId, msg->Clone()); } @@ -538,7 +538,7 @@ namespace NKikimr { GInfo = msg->NewInfo; // reconfigure guid recovery actor - Y_VERIFY(RecoverLostDataId != TActorId()); + Y_VERIFY(RecoverLostDataId != TActorId()); ctx.Send(RecoverLostDataId, msg->Clone()); } diff --git a/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_committer.cpp b/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_committer.cpp index d8046e76783..0c394f24443 100644 --- a/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_committer.cpp +++ b/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_committer.cpp @@ -100,9 +100,9 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// class TSyncerCommitter : public TActorBootstrapped<TSyncerCommitter> { struct TNotify { - const TActorId ActorId; + const TActorId ActorId; const void *Cookie = nullptr; - TNotify(const TActorId aid, const void *cookie) + TNotify(const TActorId aid, const void *cookie) : ActorId(aid) , Cookie(cookie) {} diff --git a/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_data.cpp b/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_data.cpp index 1f321c20b12..4de7b60fbe8 100644 --- a/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_data.cpp +++ b/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_data.cpp @@ -162,7 +162,7 @@ namespace NKikimr { // TSyncNeighbors //////////////////////////////////////////////////////////////////////////// TSyncNeighbors::TSyncNeighbors(const TString &logPrefix, - const TActorId ¬ifyId, + const TActorId ¬ifyId, const TVDiskIdShort &self, std::shared_ptr<TBlobStorageGroupInfo::TTopology> top) : LogPrefix(logPrefix) @@ -288,7 +288,7 @@ namespace NKikimr { } TSyncerData::TSyncerData(const TString &logPrefix, - const TActorId ¬ifyId, + const TActorId ¬ifyId, const TVDiskIdShort &selfVDisk, std::shared_ptr<TBlobStorageGroupInfo::TTopology> top, const TString &entryPoint) @@ -304,7 +304,7 @@ namespace NKikimr { } bool TSyncerData::CheckEntryPoint(const TString &logPrefix, - const TActorId ¬ifyId, + const TActorId ¬ifyId, const TVDiskIdShort &selfVDisk, std::shared_ptr<TBlobStorageGroupInfo::TTopology> top, const TString &entryPoint) { diff --git a/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_data.h b/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_data.h index 313afd4f663..c623859a60c 100644 --- a/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_data.h +++ b/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_data.h @@ -116,13 +116,13 @@ namespace NKikimr { private: const TString LogPrefix; - const TActorId NotifyId; + const TActorId NotifyId; TNeighbors Neighbors; NSync::TQuorumTracker QuorumTracker; public: TSyncNeighbors(const TString &logPrefix, - const TActorId ¬ifyId, + const TActorId ¬ifyId, const TVDiskIdShort &self, std::shared_ptr<TBlobStorageGroupInfo::TTopology> top); @@ -188,10 +188,10 @@ namespace NKikimr { struct TSyncerData : public TThrRefBase { TSyncNeighborsPtr Neighbors; NSyncer::TLocalSyncerState LocalSyncerState; - const TActorId NotifyId; + const TActorId NotifyId; TSyncerData(const TString &logPrefix, - const TActorId ¬ifyId, + const TActorId ¬ifyId, const TVDiskIdShort &selfVDisk, std::shared_ptr<TBlobStorageGroupInfo::TTopology> top, const TString &entryPoint = TString()); @@ -205,7 +205,7 @@ namespace NKikimr { // check and cut signature static TString WithoutSignature(const TString &entryPoint); static bool CheckEntryPoint(const TString &logPrefix, - const TActorId ¬ifyId, + const TActorId ¬ifyId, const TVDiskIdShort &selfVDisk, std::shared_ptr<TBlobStorageGroupInfo::TTopology> top, const TString &entryPoint); diff --git a/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_data_ut.cpp b/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_data_ut.cpp index 07d73eddc18..0da0c427ee3 100644 --- a/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_data_ut.cpp +++ b/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_data_ut.cpp @@ -17,14 +17,14 @@ Y_UNIT_TEST_SUITE(TSyncNeighborsTests) { const TVDiskID self = vd; // fill in values - TSyncNeighbors n("Prefix", TActorId(), self, info->PickTopology()); + TSyncNeighbors n("Prefix", TActorId(), self, info->PickTopology()); ui64 i = 123456u; for (auto &x: n) { x.Get().PeerSyncState.SchTime = TInstant::MicroSeconds(++i); } // old serialize/parse - TSyncNeighbors n2("Prefix", TActorId(), self, info->PickTopology()); + TSyncNeighbors n2("Prefix", TActorId(), self, info->PickTopology()); { TStringStream output; n.OldSerialize(output, info.Get()); @@ -36,7 +36,7 @@ Y_UNIT_TEST_SUITE(TSyncNeighborsTests) { } // new serialize/parse - TSyncNeighbors n3("Prefix", TActorId(), self, info->PickTopology()); + TSyncNeighbors n3("Prefix", TActorId(), self, info->PickTopology()); { TStringStream output; n2.Serialize(output, info.Get()); diff --git a/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_recoverlostdata.cpp b/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_recoverlostdata.cpp index 515c66e683f..1b3286397de 100644 --- a/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_recoverlostdata.cpp +++ b/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_recoverlostdata.cpp @@ -17,21 +17,21 @@ namespace NKikimr { namespace { struct TPeerState { - TActorId ProxyId; + TActorId ProxyId; // We keep NSyncer::TPeerSyncState to facilitate standard Full Sync Job, // that we use in Syncer NSyncer::TPeerSyncState PeerSyncState; void FullSyncDone() { - ProxyId = TActorId(); + ProxyId = TActorId(); } - void SetProxyId(const TActorId &aid) { + void SetProxyId(const TActorId &aid) { ProxyId = aid; } bool IsFinished() const { - return ProxyId == TActorId(); + return ProxyId == TActorId(); } }; @@ -100,8 +100,8 @@ namespace NKikimr { TIntrusivePtr<TSyncerContext> SyncerCtx; TIntrusivePtr<TBlobStorageGroupInfo> GInfo; TSyncFullRecoverState State; - const TActorId CommitterId; - const TActorId NotifyId; + const TActorId CommitterId; + const TActorId NotifyId; const TVDiskEternalGuid Guid; ui64 DbBirthLsn = 0; std::shared_ptr<TSjCtx> JobCtx; @@ -269,8 +269,8 @@ namespace NKikimr { TSyncerRecoverLostDataActor(const TIntrusivePtr<TSyncerContext> &sc, const TIntrusivePtr<TBlobStorageGroupInfo> &info, - const TActorId &committerId, - const TActorId ¬ifyId, + const TActorId &committerId, + const TActorId ¬ifyId, TVDiskEternalGuid guid) : TActorBootstrapped<TSyncerRecoverLostDataActor>() , SyncerCtx(sc) @@ -289,8 +289,8 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// IActor *CreateSyncerRecoverLostDataActor(const TIntrusivePtr<TSyncerContext> &sc, const TIntrusivePtr<TBlobStorageGroupInfo> &info, - const TActorId &committerId, - const TActorId ¬ifyId, + const TActorId &committerId, + const TActorId ¬ifyId, TVDiskEternalGuid guid) { return new TSyncerRecoverLostDataActor(sc, info, committerId, notifyId, guid); } diff --git a/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_recoverlostdata.h b/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_recoverlostdata.h index be3cd68c6e7..b81f62d072e 100644 --- a/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_recoverlostdata.h +++ b/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_recoverlostdata.h @@ -28,8 +28,8 @@ namespace NKikimr { class TBlobStorageGroupInfo; IActor *CreateSyncerRecoverLostDataActor(const TIntrusivePtr<TSyncerContext> &sc, const TIntrusivePtr<TBlobStorageGroupInfo> &info, - const TActorId &committerId, - const TActorId ¬ifyId, + const TActorId &committerId, + const TActorId ¬ifyId, TVDiskEternalGuid guid); } // NKikimr diff --git a/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_recoverlostdata_proxy.cpp b/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_recoverlostdata_proxy.cpp index a9fa75ecaea..2d0e205c7b5 100644 --- a/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_recoverlostdata_proxy.cpp +++ b/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_recoverlostdata_proxy.cpp @@ -37,13 +37,13 @@ namespace NKikimr { TIntrusivePtr<TSyncerContext> SyncerCtx; const NSyncer::TPeerSyncState PeerSyncState; - const TActorId CommitterId; - const TActorId NotifyId; + const TActorId CommitterId; + const TActorId NotifyId; TActiveActors ActiveActors; std::shared_ptr<TSjCtx> JobCtx; // Target VDiskId and ActorId are reconfigurable TVDiskID TargetVDiskId; - TActorId TargetActorId; + TActorId TargetActorId; void CreateAndRunTask(const TActorContext &ctx) { // create task @@ -175,11 +175,11 @@ namespace NKikimr { TSyncerRLDFullSyncProxyActor(const TIntrusivePtr<TSyncerContext> &sc, const TPeerSyncState& peerSyncState, - const TActorId &committerId, - const TActorId ¬ifyId, + const TActorId &committerId, + const TActorId ¬ifyId, const std::shared_ptr<TSjCtx> &jobCtx, const TVDiskID &targetVDiskId, - const TActorId &targetActorId) + const TActorId &targetActorId) : TActorBootstrapped<TSyncerRLDFullSyncProxyActor>() , SyncerCtx(sc) , PeerSyncState(peerSyncState) @@ -197,11 +197,11 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// IActor *CreateProxyForFullSyncWithPeer(const TIntrusivePtr<TSyncerContext> &sc, const TPeerSyncState& peerSyncState, - const TActorId &committerId, - const TActorId ¬ifyId, + const TActorId &committerId, + const TActorId ¬ifyId, const std::shared_ptr<TSjCtx> &jobCtx, const TVDiskID &targetVDiskId, - const TActorId &targetActorId) { + const TActorId &targetActorId) { return new TSyncerRLDFullSyncProxyActor(sc, peerSyncState, committerId, notifyId, jobCtx, targetVDiskId, targetActorId); } diff --git a/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_recoverlostdata_proxy.h b/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_recoverlostdata_proxy.h index b1f1e23144f..eb5605a4708 100644 --- a/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_recoverlostdata_proxy.h +++ b/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_recoverlostdata_proxy.h @@ -35,10 +35,10 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// IActor *CreateProxyForFullSyncWithPeer(const TIntrusivePtr<TSyncerContext> &sc, const NSyncer::TPeerSyncState& peerSyncState, - const TActorId &committerId, - const TActorId ¬ifyId, + const TActorId &committerId, + const TActorId ¬ifyId, const std::shared_ptr<NSyncer::TSjCtx> &jobCtx, const TVDiskID &targetVDiskId, - const TActorId &targetActorId); + const TActorId &targetActorId); } // NKikimr diff --git a/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_scheduler.cpp b/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_scheduler.cpp index 2bb448b6855..92ae94e89f7 100644 --- a/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_scheduler.cpp +++ b/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_scheduler.cpp @@ -9,7 +9,7 @@ #include <library/cpp/actors/core/mon.h> #include <library/cpp/actors/core/interconnect.h> #include <library/cpp/actors/interconnect/interconnect.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/generic/queue.h> #include <util/generic/deque.h> @@ -26,7 +26,7 @@ namespace NKikimr { TIntrusivePtr<TBlobStorageGroupInfo> GInfo; TIntrusivePtr<TSyncerData> SyncerData; NMon::TEvHttpInfo::TPtr Ev; - const TActorId NotifyId; + const TActorId NotifyId; friend class TActorBootstrapped<TSyncerSchedulerHttpActor>; @@ -49,7 +49,7 @@ namespace NKikimr { auto vd = GInfo.GetVDiskId(it->VDiskIdShort); str << "VDiskId: " << vd.ToStringWOGeneration() << "<br>"; // output node info - TActorId aid = GInfo.GetActorId(it->VDiskIdShort); + TActorId aid = GInfo.GetActorId(it->VDiskIdShort); ui32 nodeId = aid.NodeId(); using TNodeInfo = TEvInterconnect::TNodeInfo; const TNodeInfo *info = NodesInfo->Get()->GetNodeInfo(nodeId); @@ -72,7 +72,7 @@ namespace NKikimr { }; void Bootstrap(const TActorContext &ctx) { - ctx.Send(GetNameserviceActorId(), new TEvInterconnect::TEvListNodes()); + ctx.Send(GetNameserviceActorId(), new TEvInterconnect::TEvListNodes()); Become(&TThis::StateFunc); } @@ -104,7 +104,7 @@ namespace NKikimr { const TIntrusivePtr<TBlobStorageGroupInfo> &ginfo, const TIntrusivePtr<TSyncerData> &syncerData, NMon::TEvHttpInfo::TPtr &ev, - const TActorId ¬ifyId) + const TActorId ¬ifyId) : TActorBootstrapped<TSyncerSchedulerHttpActor>() , SyncerContext(sc) , GInfo(ginfo) @@ -136,8 +136,8 @@ namespace NKikimr { class TSyncerCommitterProxy : public TActorBootstrapped<TSyncerCommitterProxy> { friend class TActorBootstrapped<TSyncerCommitterProxy>; - const TActorId NotifyId; - const TActorId CommitterId; + const TActorId NotifyId; + const TActorId CommitterId; std::unique_ptr<TSyncerJobTask> Task; void Bootstrap(const TActorContext &ctx) { @@ -167,8 +167,8 @@ namespace NKikimr { return NKikimrServices::TActivity::BS_SYNCER_COMMITTER_PROXY; } - TSyncerCommitterProxy(const TActorId ¬ifyId, - const TActorId &committerId, + TSyncerCommitterProxy(const TActorId ¬ifyId, + const TActorId &committerId, std::unique_ptr<TSyncerJobTask> task) : TActorBootstrapped<TSyncerCommitterProxy>() , NotifyId(notifyId) @@ -204,7 +204,7 @@ namespace NKikimr { TSchedulerQueue SchedulerQueue; TActiveActors ActiveActors; const TDuration SyncTimeInterval; - TActorId CommitterId; + TActorId CommitterId; bool Scheduled; std::shared_ptr<TSjCtx> JobCtx; @@ -369,7 +369,7 @@ namespace NKikimr { TSyncerScheduler(const TIntrusivePtr<TSyncerContext> &sc, const TIntrusivePtr<TBlobStorageGroupInfo> &info, const TIntrusivePtr<TSyncerData> &syncerData, - const TActorId &committerId) + const TActorId &committerId) : TActorBootstrapped<TSyncerScheduler>() , SyncerContext(sc) , GInfo(info) @@ -390,7 +390,7 @@ namespace NKikimr { IActor* CreateSyncerSchedulerActor(const TIntrusivePtr<TSyncerContext> &sc, const TIntrusivePtr<TBlobStorageGroupInfo> &info, const TIntrusivePtr<TSyncerData> &syncerData, - const TActorId &committerId) { + const TActorId &committerId) { return new TSyncerScheduler(sc, info, syncerData, committerId); } diff --git a/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_scheduler.h b/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_scheduler.h index 6289d5eebcf..8705e4a573a 100644 --- a/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_scheduler.h +++ b/ydb/core/blobstorage/vdisk/syncer/blobstorage_syncer_scheduler.h @@ -13,6 +13,6 @@ namespace NKikimr { IActor* CreateSyncerSchedulerActor(const TIntrusivePtr<TSyncerContext> &sc, const TIntrusivePtr<TBlobStorageGroupInfo> &info, const TIntrusivePtr<TSyncerData> &syncerData, - const TActorId &committerId); + const TActorId &committerId); } // NKikimr diff --git a/ydb/core/blobstorage/vdisk/syncer/guid_firstrun.cpp b/ydb/core/blobstorage/vdisk/syncer/guid_firstrun.cpp index a6a9fc66ab7..79a4df2abe3 100644 --- a/ydb/core/blobstorage/vdisk/syncer/guid_firstrun.cpp +++ b/ydb/core/blobstorage/vdisk/syncer/guid_firstrun.cpp @@ -62,18 +62,18 @@ namespace NKikimr { // ActorID of a proxy that communicates with the given VDisk - TActorId ProxyId; + TActorId ProxyId; // if we got response from a vdisk bool GotResponse = false; TVDiskState() = default; - void SetActorID(const TActorId &proxyId) { + void SetActorID(const TActorId &proxyId) { ProxyId = proxyId; } void Clear() { - ProxyId = TActorId(); + ProxyId = TActorId(); GotResponse = false; } @@ -266,8 +266,8 @@ namespace NKikimr { TIntrusivePtr<TVDiskContext> VCtx; TIntrusivePtr<TBlobStorageGroupInfo> GInfo; - const TActorId CommitterId; - const TActorId NotifyId; + const TActorId CommitterId; + const TActorId NotifyId; TVDiskGuidFirstRunState FirstRunState; EWaitFor WaitFor = WaitNotSet; @@ -550,8 +550,8 @@ namespace NKikimr { TVDiskGuidFirstRunActor(TIntrusivePtr<TVDiskContext> vctx, TIntrusivePtr<TBlobStorageGroupInfo> info, - const TActorId &committerId, - const TActorId ¬ifyId, + const TActorId &committerId, + const TActorId ¬ifyId, EFirstRunStep startStep, TVDiskEternalGuid guid) : TActorBootstrapped<TVDiskGuidFirstRunActor>() @@ -565,8 +565,8 @@ namespace NKikimr { IActor *CreateVDiskGuidFirstRunActor(TIntrusivePtr<TVDiskContext> vctx, TIntrusivePtr<TBlobStorageGroupInfo> info, - const TActorId &committerId, - const TActorId ¬ifyId, + const TActorId &committerId, + const TActorId ¬ifyId, EFirstRunStep startStep, TVDiskEternalGuid guid) { return new TVDiskGuidFirstRunActor(std::move(vctx), std::move(info), committerId, notifyId, startStep, guid); diff --git a/ydb/core/blobstorage/vdisk/syncer/guid_firstrun.h b/ydb/core/blobstorage/vdisk/syncer/guid_firstrun.h index 6c3663e90c6..7745dff6cdf 100644 --- a/ydb/core/blobstorage/vdisk/syncer/guid_firstrun.h +++ b/ydb/core/blobstorage/vdisk/syncer/guid_firstrun.h @@ -51,8 +51,8 @@ namespace NKikimr { class TVDiskContext; IActor *CreateVDiskGuidFirstRunActor(TIntrusivePtr<TVDiskContext> vctx, TIntrusivePtr<TBlobStorageGroupInfo> info, - const TActorId &committerId, - const TActorId ¬ifyId, + const TActorId &committerId, + const TActorId ¬ifyId, NSyncer::EFirstRunStep startStep, TVDiskEternalGuid guid); diff --git a/ydb/core/blobstorage/vdisk/syncer/guid_propagator.cpp b/ydb/core/blobstorage/vdisk/syncer/guid_propagator.cpp index 996c997b7ac..75be9753075 100644 --- a/ydb/core/blobstorage/vdisk/syncer/guid_propagator.cpp +++ b/ydb/core/blobstorage/vdisk/syncer/guid_propagator.cpp @@ -16,12 +16,12 @@ namespace NKikimr { // vdisk/actor ids are reconfigurable, they are changed during group reconfiguration TVDiskID SelfVDiskId; TVDiskID TargetVDiskId; - TActorId TargetServiceId; + TActorId TargetServiceId; // const values we propagate const NKikimrBlobStorage::TSyncGuidInfo::EState State; const TVDiskEternalGuid Guid; // actors we borned - TActorId WriterId; + TActorId WriterId; TActiveActors ActiveActors; const TDuration WaitPeriod = TDuration::Seconds(10); @@ -52,7 +52,7 @@ namespace NKikimr { void Handle(TEvVDiskGuidWritten::TPtr &ev, const TActorContext &ctx) { ActiveActors.Erase(ev->Sender); - WriterId = TActorId(); + WriterId = TActorId(); auto *msg = ev->Get(); Y_VERIFY(msg->State == State && msg->Guid == Guid && @@ -119,7 +119,7 @@ namespace NKikimr { TSyncerGuidPropagator(TIntrusivePtr<TVDiskContext> vctx, const TVDiskID &selfVDiskId, const TVDiskID &targetVDiskId, - const TActorId &targetServiceId, + const TActorId &targetServiceId, NKikimrBlobStorage::TSyncGuidInfo::EState state, TVDiskEternalGuid guid) : TActorBootstrapped<TSyncerGuidPropagator>() @@ -139,7 +139,7 @@ namespace NKikimr { IActor* CreateSyncerGuidPropagator(TIntrusivePtr<TVDiskContext> vctx, const TVDiskID &selfVDiskId, const TVDiskID &targetVDiskId, - const TActorId &targetServiceId, + const TActorId &targetServiceId, NKikimrBlobStorage::TSyncGuidInfo::EState state, TVDiskEternalGuid guid) { return new TSyncerGuidPropagator(std::move(vctx), selfVDiskId, targetVDiskId, targetServiceId, diff --git a/ydb/core/blobstorage/vdisk/syncer/guid_propagator.h b/ydb/core/blobstorage/vdisk/syncer/guid_propagator.h index 2f0b9eadc2c..de67b4611a9 100644 --- a/ydb/core/blobstorage/vdisk/syncer/guid_propagator.h +++ b/ydb/core/blobstorage/vdisk/syncer/guid_propagator.h @@ -17,7 +17,7 @@ namespace NKikimr { IActor* CreateSyncerGuidPropagator(TIntrusivePtr<TVDiskContext> vctx, const TVDiskID &selfVDiskId, const TVDiskID &targetVDiskId, - const TActorId &targetServiceId, + const TActorId &targetServiceId, NKikimrBlobStorage::TSyncGuidInfo::EState state, TVDiskEternalGuid guid); diff --git a/ydb/core/blobstorage/vdisk/syncer/guid_proxybase.h b/ydb/core/blobstorage/vdisk/syncer/guid_proxybase.h index 92e3deb4387..ffdd4b26c7e 100644 --- a/ydb/core/blobstorage/vdisk/syncer/guid_proxybase.h +++ b/ydb/core/blobstorage/vdisk/syncer/guid_proxybase.h @@ -17,9 +17,9 @@ namespace NKikimr { // these fields below are reconfigurable TVDiskID SelfVDiskId; TVDiskID TargetVDiskId; - TActorId TargetServiceId; + TActorId TargetServiceId; // notify this actor when done - const TActorId NotifyId; + const TActorId NotifyId; const TDuration RetryPeriod = TDuration::MilliSeconds(50); TTrivialLogThrottler LogThrottler = { TDuration::Minutes(1) }; @@ -166,8 +166,8 @@ namespace NKikimr { TVDiskGuidProxyBase(TIntrusivePtr<TVDiskContext> vctx, const TVDiskID &selfVDiskId, const TVDiskID &targetVDiskId, - const TActorId &targetServiceId, - const TActorId ¬ifyId) + const TActorId &targetServiceId, + const TActorId ¬ifyId) : TActorBootstrapped<TVDiskGuidProxyBase>() , VCtx(vctx) , SelfVDiskId(selfVDiskId) diff --git a/ydb/core/blobstorage/vdisk/syncer/guid_proxyobtain.cpp b/ydb/core/blobstorage/vdisk/syncer/guid_proxyobtain.cpp index c55c5a11991..29f2e3bfbc0 100644 --- a/ydb/core/blobstorage/vdisk/syncer/guid_proxyobtain.cpp +++ b/ydb/core/blobstorage/vdisk/syncer/guid_proxyobtain.cpp @@ -36,8 +36,8 @@ namespace NKikimr { TObtainVDiskGuidProxy(TIntrusivePtr<TVDiskContext> vctx, const TVDiskID &selfVDiskId, const TVDiskID &targetVDiskId, - const TActorId &targetServiceId, - const TActorId ¬ifyId) + const TActorId &targetServiceId, + const TActorId ¬ifyId) : TVDiskGuidProxyBase(std::move(vctx), selfVDiskId, targetVDiskId, targetServiceId, notifyId) {} }; @@ -49,8 +49,8 @@ namespace NKikimr { IActor* CreateProxyForObtainingVDiskGuid(TIntrusivePtr<TVDiskContext> vctx, const TVDiskID &selfVDiskId, const TVDiskID &targetVDiskId, - const TActorId &targetServiceId, - const TActorId ¬ifyId) { + const TActorId &targetServiceId, + const TActorId ¬ifyId) { return new TObtainVDiskGuidProxy(std::move(vctx), selfVDiskId, targetVDiskId, targetServiceId, notifyId); } diff --git a/ydb/core/blobstorage/vdisk/syncer/guid_proxyobtain.h b/ydb/core/blobstorage/vdisk/syncer/guid_proxyobtain.h index 7e9ff452efd..6ee396c2850 100644 --- a/ydb/core/blobstorage/vdisk/syncer/guid_proxyobtain.h +++ b/ydb/core/blobstorage/vdisk/syncer/guid_proxyobtain.h @@ -53,7 +53,7 @@ namespace NKikimr { IActor* CreateProxyForObtainingVDiskGuid(TIntrusivePtr<TVDiskContext> vctx, const TVDiskID &selfVDiskId, const TVDiskID &targetVDiskId, - const TActorId &targetServiceId, - const TActorId ¬ifyId); + const TActorId &targetServiceId, + const TActorId ¬ifyId); } // NKikimr diff --git a/ydb/core/blobstorage/vdisk/syncer/guid_proxywrite.cpp b/ydb/core/blobstorage/vdisk/syncer/guid_proxywrite.cpp index 54f6c23b347..8b2d29b0616 100644 --- a/ydb/core/blobstorage/vdisk/syncer/guid_proxywrite.cpp +++ b/ydb/core/blobstorage/vdisk/syncer/guid_proxywrite.cpp @@ -34,8 +34,8 @@ namespace NKikimr { TWriteVDiskGuidProxy(TIntrusivePtr<TVDiskContext> vctx, const TVDiskID &selfVDiskId, const TVDiskID &targetVDiskId, - const TActorId &targetServiceId, - const TActorId ¬ifyId, + const TActorId &targetServiceId, + const TActorId ¬ifyId, NKikimrBlobStorage::TSyncGuidInfo::EState state, TVDiskEternalGuid guid) : TVDiskGuidProxyBase(std::move(vctx), selfVDiskId, targetVDiskId, targetServiceId, notifyId) @@ -50,8 +50,8 @@ namespace NKikimr { IActor* CreateProxyForWritingVDiskGuid(TIntrusivePtr<TVDiskContext> vctx, const TVDiskID &selfVDiskId, const TVDiskID &targetVDiskId, - const TActorId &targetServiceId, - const TActorId ¬ifyId, + const TActorId &targetServiceId, + const TActorId ¬ifyId, NKikimrBlobStorage::TSyncGuidInfo::EState state, TVDiskEternalGuid guid) { Y_VERIFY(!(state == NKikimrBlobStorage::TSyncGuidInfo::Final && diff --git a/ydb/core/blobstorage/vdisk/syncer/guid_proxywrite.h b/ydb/core/blobstorage/vdisk/syncer/guid_proxywrite.h index e0d01262c8b..7396bcceb41 100644 --- a/ydb/core/blobstorage/vdisk/syncer/guid_proxywrite.h +++ b/ydb/core/blobstorage/vdisk/syncer/guid_proxywrite.h @@ -52,8 +52,8 @@ namespace NKikimr { IActor *CreateProxyForWritingVDiskGuid(TIntrusivePtr<TVDiskContext> vctx, const TVDiskID &selfVDiskId, const TVDiskID &targetVDiskId, - const TActorId &targetServiceId, - const TActorId ¬ifyId, + const TActorId &targetServiceId, + const TActorId ¬ifyId, NKikimrBlobStorage::TSyncGuidInfo::EState state, TVDiskEternalGuid guid); diff --git a/ydb/core/blobstorage/vdisk/syncer/guid_recovery.cpp b/ydb/core/blobstorage/vdisk/syncer/guid_recovery.cpp index 7e36defc79d..d4dc97f9aba 100644 --- a/ydb/core/blobstorage/vdisk/syncer/guid_recovery.cpp +++ b/ydb/core/blobstorage/vdisk/syncer/guid_recovery.cpp @@ -214,7 +214,7 @@ namespace NKikimr { // ActorID of a proxy that communicates with the given VDisk - TActorId ProxyId; + TActorId ProxyId; // guid TVDiskEternalGuid Guid = 0; // confidence @@ -224,7 +224,7 @@ namespace NKikimr { TNeighborVDiskState() = default; - void SetActorID(const TActorId &proxyId) { + void SetActorID(const TActorId &proxyId) { ProxyId = proxyId; } @@ -632,12 +632,12 @@ namespace NKikimr { TIntrusivePtr<TVDiskContext> VCtx; TIntrusivePtr<TBlobStorageGroupInfo> GInfo; - const TActorId CommitterId; - const TActorId NotifyId; + const TActorId CommitterId; + const TActorId NotifyId; TDecisionMaker DecisionMaker; std::unique_ptr<TDecision> Decision; EPhase Phase = PhaseNotSet; - TActorId FirstRunActorId; + TActorId FirstRunActorId; //////////////////////////////////////////////////////////////////////// @@ -884,8 +884,8 @@ namespace NKikimr { TVDiskGuidRecoveryActor(TIntrusivePtr<TVDiskContext> vctx, TIntrusivePtr<TBlobStorageGroupInfo> info, - const TActorId &committerId, - const TActorId ¬ifyId, + const TActorId &committerId, + const TActorId ¬ifyId, const TLocalSyncerState &locallyRecoveredState) : TActorBootstrapped<TVDiskGuidRecoveryActor>() , VCtx(std::move(vctx)) @@ -903,8 +903,8 @@ namespace NKikimr { IActor *CreateVDiskGuidRecoveryActor(TIntrusivePtr<TVDiskContext> vctx, TIntrusivePtr<TBlobStorageGroupInfo> info, - const TActorId &committerId, - const TActorId ¬ifyId, + const TActorId &committerId, + const TActorId ¬ifyId, const NSyncer::TLocalSyncerState &localState) { return new NSyncer::TVDiskGuidRecoveryActor(std::move(vctx), std::move(info), committerId, notifyId, localState); diff --git a/ydb/core/blobstorage/vdisk/syncer/guid_recovery.h b/ydb/core/blobstorage/vdisk/syncer/guid_recovery.h index 201d5d4d27d..1b3be5a03a9 100644 --- a/ydb/core/blobstorage/vdisk/syncer/guid_recovery.h +++ b/ydb/core/blobstorage/vdisk/syncer/guid_recovery.h @@ -142,8 +142,8 @@ namespace NKikimr { class TVDiskContext; IActor *CreateVDiskGuidRecoveryActor(TIntrusivePtr<TVDiskContext> vctx, TIntrusivePtr<TBlobStorageGroupInfo> info, - const TActorId &committerId, - const TActorId ¬ifyId, + const TActorId &committerId, + const TActorId ¬ifyId, const NSyncer::TLocalSyncerState &localState); } // NKikimr diff --git a/ydb/core/blobstorage/vdisk/syncer/syncer_context.h b/ydb/core/blobstorage/vdisk/syncer/syncer_context.h index c92e0eba72d..ba825dc2f61 100644 --- a/ydb/core/blobstorage/vdisk/syncer/syncer_context.h +++ b/ydb/core/blobstorage/vdisk/syncer/syncer_context.h @@ -13,22 +13,22 @@ namespace NKikimr { const TIntrusivePtr<TVDiskContext> VCtx; const TIntrusivePtr<TLsnMngr> LsnMngr; const TPDiskCtxPtr PDiskCtx; - const TActorId SkeletonId; - const TActorId AnubisRunnerId; - const TActorId LoggerId; - const TActorId LogCutterId; - const TActorId SyncLogId; + const TActorId SkeletonId; + const TActorId AnubisRunnerId; + const TActorId LoggerId; + const TActorId LogCutterId; + const TActorId SyncLogId; const TIntrusivePtr<TVDiskConfig> Config; NMonGroup::TSyncerGroup MonGroup; TSyncerContext(TIntrusivePtr<TVDiskContext> vctx, TIntrusivePtr<TLsnMngr> lsnMngr, TPDiskCtxPtr pdiskCtx, - const TActorId &skeletonId, - const TActorId &anubisRunnerId, - const TActorId &loggerId, - const TActorId &logCutterId, - const TActorId &syncLogId, + const TActorId &skeletonId, + const TActorId &anubisRunnerId, + const TActorId &loggerId, + const TActorId &logCutterId, + const TActorId &syncLogId, TIntrusivePtr<TVDiskConfig> config) : VCtx(std::move(vctx)) , LsnMngr(std::move(lsnMngr)) diff --git a/ydb/core/blobstorage/vdisk/syncer/syncer_job_actor.cpp b/ydb/core/blobstorage/vdisk/syncer/syncer_job_actor.cpp index 8c63980a88a..c1a1629bf11 100644 --- a/ydb/core/blobstorage/vdisk/syncer/syncer_job_actor.cpp +++ b/ydb/core/blobstorage/vdisk/syncer/syncer_job_actor.cpp @@ -26,7 +26,7 @@ namespace NKikimr { TIntrusivePtr<TSyncerContext> SyncerCtx; std::unique_ptr<TSyncerJobTask> Task; const ui32 NodeId; - const TActorId NotifyId; + const TActorId NotifyId; const ui64 JobId; // just unique job id for log readability friend class TActorBootstrapped<TSyncerJob>; @@ -191,7 +191,7 @@ namespace NKikimr { public: TSyncerJob(const TIntrusivePtr<TSyncerContext> &sc, std::unique_ptr<TSyncerJobTask> task, - const TActorId ¬ifyId) + const TActorId ¬ifyId) : TActorBootstrapped<TSyncerJob>() , SyncerCtx(sc) , Task(std::move(task)) @@ -206,7 +206,7 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// IActor* CreateSyncerJob(const TIntrusivePtr<TSyncerContext> &sc, std::unique_ptr<TSyncerJobTask> task, - const TActorId ¬ifyId) { + const TActorId ¬ifyId) { return new TSyncerJob(sc, std::move(task), notifyId); } diff --git a/ydb/core/blobstorage/vdisk/syncer/syncer_job_actor.h b/ydb/core/blobstorage/vdisk/syncer/syncer_job_actor.h index be65fa8b780..835c33cbebb 100644 --- a/ydb/core/blobstorage/vdisk/syncer/syncer_job_actor.h +++ b/ydb/core/blobstorage/vdisk/syncer/syncer_job_actor.h @@ -28,6 +28,6 @@ namespace NKikimr { //////////////////////////////////////////////////////////////////////////// IActor* CreateSyncerJob(const TIntrusivePtr<TSyncerContext> &sc, std::unique_ptr<NSyncer::TSyncerJobTask> task, - const TActorId ¬ifyId); + const TActorId ¬ifyId); } // NKikimr diff --git a/ydb/core/blobstorage/vdisk/syncer/syncer_job_task.cpp b/ydb/core/blobstorage/vdisk/syncer/syncer_job_task.cpp index 1d276277230..de98610b833 100644 --- a/ydb/core/blobstorage/vdisk/syncer/syncer_job_task.cpp +++ b/ydb/core/blobstorage/vdisk/syncer/syncer_job_task.cpp @@ -50,7 +50,7 @@ namespace NKikimr { TSyncerJobTask::TSyncerJobTask( EJobType type, const TVDiskID &vdisk, - const TActorId &service, + const TActorId &service, const NSyncer::TPeerSyncState &peerState, const std::shared_ptr<TSjCtx> &ctx) : VDiskId(vdisk) diff --git a/ydb/core/blobstorage/vdisk/syncer/syncer_job_task.h b/ydb/core/blobstorage/vdisk/syncer/syncer_job_task.h index 84924cce9d5..9d6d0870092 100644 --- a/ydb/core/blobstorage/vdisk/syncer/syncer_job_task.h +++ b/ydb/core/blobstorage/vdisk/syncer/syncer_job_task.h @@ -27,7 +27,7 @@ namespace NKikimr { struct TSjOutcome { // we can send a message as a result ... std::unique_ptr<IEventBase> Ev; - TActorId To; + TActorId To; // ... or run a actor std::unique_ptr<IActor> ActorActivity; bool RunInBatchPool = false; @@ -131,7 +131,7 @@ namespace NKikimr { TSyncerJobTask( EJobType type, const TVDiskID &vdisk, - const TActorId &service, + const TActorId &service, const NSyncer::TPeerSyncState &peerState, const std::shared_ptr<TSjCtx> &ctx); void Output(IOutputStream &str) const; @@ -180,7 +180,7 @@ namespace NKikimr { public: const TVDiskID VDiskId; - const TActorId ServiceId; + const TActorId ServiceId; private: // job type EJobType Type = EJustSync; diff --git a/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclog.cpp b/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclog.cpp index e94323e385b..907b3873f86 100644 --- a/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclog.cpp +++ b/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclog.cpp @@ -26,8 +26,8 @@ namespace NKikimr { { TIntrusivePtr<TSyncLogCtx> SlCtx; TEvLocalStatus::TPtr Ev; - const TActorId NotifyId; - const TActorId KeeperId; + const TActorId NotifyId; + const TActorId KeeperId; friend class TActorBootstrapped<TSyncLogGetLocalStatusActor>; @@ -70,8 +70,8 @@ namespace NKikimr { TSyncLogGetLocalStatusActor(TIntrusivePtr<TSyncLogCtx> &slCtx, TEvLocalStatus::TPtr &ev, - const TActorId ¬ifyId, - const TActorId &keeperId) + const TActorId ¬ifyId, + const TActorId &keeperId) : TActorBootstrapped<TSyncLogGetLocalStatusActor>() , SlCtx(slCtx) , Ev(ev) @@ -98,7 +98,7 @@ namespace NKikimr { std::unique_ptr<TSyncLogRepaired> Repaired; TSyncLogNeighborsPtr NeighborsPtr; const TVDiskIncarnationGuid VDiskIncarnationGuid; - TActorId KeeperId; + TActorId KeeperId; TMaybe<ui64> DbBirthLsn; TActiveActors ActiveActors; diff --git a/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclog_public_events.h b/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclog_public_events.h index 05be731908a..48616643a85 100644 --- a/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclog_public_events.h +++ b/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclog_public_events.h @@ -93,14 +93,14 @@ namespace NKikimr { TSyncState SyncState; TVDiskID SourceVDiskID; TVDiskID TargetVDiskID; - TActorId Recipient; + TActorId Recipient; const TInstant Now; ui32 Channel; TEvSyncLogRead(const TSyncState &syncState, const TVDiskID &sourceVDisk, const TVDiskID &targetVDisk, - const TActorId &recipient, + const TActorId &recipient, const TInstant &now, ui32 channel) : SyncState(syncState) @@ -140,8 +140,8 @@ namespace NKikimr { const TIntrusivePtr<TVDiskContext> VCtx; const TIntrusivePtr<TLsnMngr> LsnMngr; const TPDiskCtxPtr PDiskCtx; - const TActorId LoggerID; - const TActorId LogCutterID; + const TActorId LoggerID; + const TActorId LogCutterID; const ui64 SyncLogMaxDiskAmount; const ui64 SyncLogMaxEntryPointSize; @@ -155,8 +155,8 @@ namespace NKikimr { TSyncLogCtx(TIntrusivePtr<TVDiskContext> vctx, TIntrusivePtr<TLsnMngr> lsnMngr, TPDiskCtxPtr pdiskCtx, - const TActorId &loggerId, - const TActorId &logCutterId, + const TActorId &loggerId, + const TActorId &logCutterId, ui64 syncLogMaxDiskAmount, ui64 syncLogMaxEntryPointSize, ui64 syncLogMaxMemAmount, diff --git a/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogdsk.cpp b/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogdsk.cpp index 87740f55dbc..1a2fb95e1c5 100644 --- a/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogdsk.cpp +++ b/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogdsk.cpp @@ -2,7 +2,7 @@ #include "blobstorage_synclog_private_events.h" #include <library/cpp/actors/core/actor_bootstrapped.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <library/cpp/actors/core/mon.h> diff --git a/ydb/core/blobstorage/vdisk/synclog/blobstorage_syncloghttp.cpp b/ydb/core/blobstorage/vdisk/synclog/blobstorage_syncloghttp.cpp index ab6d5a2ad4f..cc10f0ceed6 100644 --- a/ydb/core/blobstorage/vdisk/synclog/blobstorage_syncloghttp.cpp +++ b/ydb/core/blobstorage/vdisk/synclog/blobstorage_syncloghttp.cpp @@ -5,7 +5,7 @@ #include <ydb/core/blobstorage/vdisk/common/vdisk_mon.h> #include <library/cpp/actors/interconnect/interconnect.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> using namespace NKikimrServices; @@ -21,8 +21,8 @@ namespace NKikimr { TIntrusivePtr<TVDiskContext> VCtx; TIntrusivePtr<TBlobStorageGroupInfo> GInfo; NMon::TEvHttpInfo::TPtr Ev; - const TActorId NotifyId; - const TActorId KeeperId; + const TActorId NotifyId; + const TActorId KeeperId; TSyncLogNeighborsPtr NeighborsPtr; // we must obtain SnapPtr and NodesInfoMsg before calling Finish TSyncLogSnapshotPtr SnapPtr; @@ -37,7 +37,7 @@ namespace NKikimr { const bool introspection = true; ctx.Send(KeeperId, new TEvSyncLogSnapshot(introspection)); // obtain nodes list - ctx.Send(GetNameserviceActorId(), new TEvInterconnect::TEvListNodes()); + ctx.Send(GetNameserviceActorId(), new TEvInterconnect::TEvListNodes()); } void Handle(TEvSyncLogSnapshotResult::TPtr &ev, const TActorContext &ctx) { @@ -107,8 +107,8 @@ namespace NKikimr { TSyncLogGetHttpInfoActor(const TIntrusivePtr<TVDiskContext> &vctx, const TIntrusivePtr<TBlobStorageGroupInfo> &ginfo, NMon::TEvHttpInfo::TPtr &ev, - const TActorId ¬ifyId, - const TActorId &keeperId, + const TActorId ¬ifyId, + const TActorId &keeperId, TSyncLogNeighborsPtr neighborsPtr) : TActorBootstrapped<TSyncLogGetHttpInfoActor>() , VCtx(vctx) @@ -125,8 +125,8 @@ namespace NKikimr { IActor* CreateGetHttpInfoActor(const TIntrusivePtr<TVDiskContext> &vctx, const TIntrusivePtr<TBlobStorageGroupInfo> &ginfo, NMon::TEvHttpInfo::TPtr &ev, - const TActorId ¬ifyId, - const TActorId &keeperId, + const TActorId ¬ifyId, + const TActorId &keeperId, TSyncLogNeighborsPtr neighborsPtr) { return new TSyncLogGetHttpInfoActor(vctx, ginfo, ev, notifyId, keeperId, neighborsPtr); } diff --git a/ydb/core/blobstorage/vdisk/synclog/blobstorage_syncloghttp.h b/ydb/core/blobstorage/vdisk/synclog/blobstorage_syncloghttp.h index 52e0deb1a08..cc8df3a8977 100644 --- a/ydb/core/blobstorage/vdisk/synclog/blobstorage_syncloghttp.h +++ b/ydb/core/blobstorage/vdisk/synclog/blobstorage_syncloghttp.h @@ -17,8 +17,8 @@ namespace NKikimr { IActor* CreateGetHttpInfoActor(const TIntrusivePtr<TVDiskContext> &vctx, const TIntrusivePtr<TBlobStorageGroupInfo> &ginfo, NMon::TEvHttpInfo::TPtr &ev, - const TActorId ¬ifyId, - const TActorId &keeperId, + const TActorId ¬ifyId, + const TActorId &keeperId, TSyncLogNeighborsPtr neighborsPtr); } // NSyncLog diff --git a/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogkeeper.cpp b/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogkeeper.cpp index ff56c755180..c7893daa0a6 100644 --- a/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogkeeper.cpp +++ b/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogkeeper.cpp @@ -25,7 +25,7 @@ namespace NKikimr { // Sublog with circle buffer TSublog<TCircleBufStringStream<4096>> Sublog; // committer actor id, or empty if we don't have commit activity at this moment - TActorId CommitterId = {}; + TActorId CommitterId = {}; // FirstLsnToKeep reported to LogCutter last time TMaybe<ui64> LastReportedFirstLsnToKeep; @@ -185,7 +185,7 @@ namespace NKikimr { << " Commit done: message# " << msg->ToString().data() << "\n"; // commit finished - CommitterId = TActorId(); + CommitterId = TActorId(); const ui64 firstLsnToKeep = KeepState.ApplyCommitResult(msg); // If we have done some work to free up recovery log, the layout diff --git a/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogkeeper_committer.cpp b/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogkeeper_committer.cpp index 236cb0483d3..e02b627a698 100644 --- a/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogkeeper_committer.cpp +++ b/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogkeeper_committer.cpp @@ -18,7 +18,7 @@ namespace NKikimr { class TSyncLogCommitterActor : public TActorBootstrapped<TSyncLogCommitterActor> { TIntrusivePtr<TSyncLogCtx> SlCtx; TSyncLogSnapshotPtr SyncLogSnap; - const TActorId NotifyID; + const TActorId NotifyID; NPDisk::TCommitRecord CommitRecord; TEntryPointSerializer EntryPointSerializer; TMemRecLogSnapshotPtr SwapSnap; @@ -179,7 +179,7 @@ namespace NKikimr { TSyncLogCommitterActor( TIntrusivePtr<TSyncLogCtx> slCtx, - const TActorId ¬ifyID, + const TActorId ¬ifyID, TSyncLogKeeperCommitData &&commitData) : TActorBootstrapped<TSyncLogCommitterActor>() , SlCtx(std::move(slCtx)) @@ -207,7 +207,7 @@ namespace NKikimr { IActor *CreateSyncLogCommitter( TIntrusivePtr<TSyncLogCtx> slCtx, - const TActorId ¬ifyID, + const TActorId ¬ifyID, TSyncLogKeeperCommitData &&commitData) { return new TSyncLogCommitterActor(std::move(slCtx), notifyID, std::move(commitData)); diff --git a/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogkeeper_committer.h b/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogkeeper_committer.h index b750dc5bcec..616901557cb 100644 --- a/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogkeeper_committer.h +++ b/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogkeeper_committer.h @@ -130,7 +130,7 @@ namespace NKikimr { class TSyncLogCtx; IActor *CreateSyncLogCommitter( TIntrusivePtr<TSyncLogCtx> slCtx, - const TActorId ¬ifyID, + const TActorId ¬ifyID, TSyncLogKeeperCommitData &&commitData); } // NSyncLog diff --git a/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogkeeper_ut.cpp b/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogkeeper_ut.cpp index 9d405702121..44808438bee 100644 --- a/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogkeeper_ut.cpp +++ b/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogkeeper_ut.cpp @@ -71,7 +71,7 @@ namespace NKikimr { void TSyncLogKeeperTest::CreateState(TEntryPointPair ep) { TBlobStorageGroupInfo groupInfo(TBlobStorageGroupType::ErasureMirror3, 2, 4); TIntrusivePtr<TVDiskContext> vctx = new TVDiskContext( - TActorId(), + TActorId(), groupInfo.PickTopology(), new NMonitoring::TDynamicCounters(), TVDiskID(), diff --git a/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogmem.cpp b/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogmem.cpp index 18492318ae5..75d6349b824 100644 --- a/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogmem.cpp +++ b/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogmem.cpp @@ -1,5 +1,5 @@ #include "blobstorage_synclogmem.h" -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> namespace NKikimr { namespace NSyncLog { diff --git a/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogneighbors.cpp b/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogneighbors.cpp index 774767f27f6..b9cf5c78ee6 100644 --- a/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogneighbors.cpp +++ b/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogneighbors.cpp @@ -29,7 +29,7 @@ namespace NKikimr { auto vd = GInfo.GetVDiskId(it->VDiskIdShort); str << "VDiskId: " << vd.ToStringWOGeneration() << "<br>"; // output node info - TActorId aid = GInfo.GetActorId(it->VDiskIdShort); + TActorId aid = GInfo.GetActorId(it->VDiskIdShort); ui32 nodeId = aid.NodeId(); using TNodeInfo = TEvInterconnect::TNodeInfo; const TNodeInfo *info = NodesInfo->Get()->GetNodeInfo(nodeId); diff --git a/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogreader.cpp b/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogreader.cpp index b1b93419eb4..2873bcb56c8 100644 --- a/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogreader.cpp +++ b/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogreader.cpp @@ -136,8 +136,8 @@ namespace NKikimr { TIntrusivePtr<TSyncLogCtx> SlCtx; const TVDiskIncarnationGuid VDiskIncarnationGuid; TEvBlobStorage::TEvVSync::TPtr Ev; - const TActorId ParentId; - const TActorId KeeperId; + const TActorId ParentId; + const TActorId KeeperId; const TVDiskID SelfVDiskId; const TVDiskID SourceVDisk; TSyncLogSnapshotPtr SnapPtr; @@ -409,8 +409,8 @@ namespace NKikimr { const TIntrusivePtr<TSyncLogCtx> &slCtx, const TVDiskIncarnationGuid vdiskIncarnationGuid, TEvBlobStorage::TEvVSync::TPtr &ev, - const TActorId &parentId, - const TActorId &keeperId, + const TActorId &parentId, + const TActorId &keeperId, const TVDiskID &selfVDiskId, const TVDiskID &sourceVDisk, ui64 dbBirthLsn, @@ -441,8 +441,8 @@ namespace NKikimr { const TIntrusivePtr<TSyncLogCtx> &slCtx, const TVDiskIncarnationGuid vdiskIncarnationGuid, TEvBlobStorage::TEvVSync::TPtr &ev, - const TActorId &parentId, - const TActorId &keeperId, + const TActorId &parentId, + const TActorId &keeperId, const TVDiskID &selfVDiskId, const TVDiskID &sourceVDisk, ui64 dbBirthLsn, diff --git a/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogreader.h b/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogreader.h index 9531311c40c..b04d98f8d90 100644 --- a/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogreader.h +++ b/ydb/core/blobstorage/vdisk/synclog/blobstorage_synclogreader.h @@ -59,8 +59,8 @@ namespace NKikimr { const TIntrusivePtr<TSyncLogCtx> &slCtx, TVDiskIncarnationGuid vdiskIncarnationGuid, TEvBlobStorage::TEvVSync::TPtr &ev, - const TActorId &parentId, - const TActorId &keeperId, + const TActorId &parentId, + const TActorId &keeperId, const TVDiskID &selfVDiskId, const TVDiskID &sourceVDisk, ui64 dbBirthLsn, diff --git a/ydb/core/blobstorage/vdisk/synclog/ya.make b/ydb/core/blobstorage/vdisk/synclog/ya.make index 25fcd746139..99fd4933874 100644 --- a/ydb/core/blobstorage/vdisk/synclog/ya.make +++ b/ydb/core/blobstorage/vdisk/synclog/ya.make @@ -7,7 +7,7 @@ PEERDIR( library/cpp/actors/interconnect library/cpp/blockcodecs library/cpp/codecs - library/cpp/monlib/service/pages + library/cpp/monlib/service/pages ydb/core/base ydb/core/blobstorage/base ydb/core/blobstorage/groupinfo diff --git a/ydb/core/blobstorage/ya.make b/ydb/core/blobstorage/ya.make index 232c5c2b337..17f379ebf17 100644 --- a/ydb/core/blobstorage/ya.make +++ b/ydb/core/blobstorage/ya.make @@ -36,11 +36,11 @@ IF (MSVC) ENDIF() END() - -RECURSE( + +RECURSE( backpressure base - crypto + crypto dsproxy groupinfo incrhuge @@ -51,7 +51,7 @@ RECURSE( storagepoolmon testload vdisk -) +) RECURSE_FOR_TESTS( ut_blobstorage diff --git a/ydb/core/client/client_ut.cpp b/ydb/core/client/client_ut.cpp index 8db8e881391..b94c722588b 100644 --- a/ydb/core/client/client_ut.cpp +++ b/ydb/core/client/client_ut.cpp @@ -274,7 +274,7 @@ void AlterTestTables(TClient& client) { struct TTxInfo { ui64 TabletId; - TActorId ActorId; + TActorId ActorId; std::pair<ui32, ui64> GenStep; bool IsFollower; ui64 TxId; @@ -311,7 +311,7 @@ void ExtractResultInfo(const NKikimrMiniKQL::TResult& result, TVector<TTxInfo>& TTxInfo info; info.TabletId = row["TabletId"]; - info.ActorId = TActorId(row["ActorIdRawX1"], row["ActorIdRawX2"]); + info.ActorId = TActorId(row["ActorIdRawX1"], row["ActorIdRawX2"]); info.GenStep = std::make_pair(row["Generation"], row["GenStep"]); info.IsFollower = row["IsFollower"]; info.TxId = row["TxId"]; @@ -1067,7 +1067,7 @@ Y_UNIT_TEST_SUITE(TClientTest) { ) )___"; - UNIT_ASSERT(client.FlatQuery(readQuery, readRes)); + UNIT_ASSERT(client.FlatQuery(readQuery, readRes)); { TValue value = TValue::Create(readRes.GetValue(), readRes.GetType()); @@ -1260,7 +1260,7 @@ Y_UNIT_TEST_SUITE(TClientTest) { ) )___"; - UNIT_ASSERT(client.FlatQuery(readQuery, readRes)); + UNIT_ASSERT(client.FlatQuery(readQuery, readRes)); { TValue value = TValue::Create(readRes.GetValue(), readRes.GetType()); @@ -1288,7 +1288,7 @@ Y_UNIT_TEST_SUITE(TClientTest) { ) )___"; - UNIT_ASSERT(client.FlatQuery(readQuery, readRes)); + UNIT_ASSERT(client.FlatQuery(readQuery, readRes)); { TValue value = TValue::Create(readRes.GetValue(), readRes.GetType()); @@ -1601,7 +1601,7 @@ Y_UNIT_TEST_SUITE(TClientTest) { NTabletPipe::TClientConfig pipeClientConfig; pipeClientConfig.ForceFollower = true; - server.GetRuntime()->SendToPipe(tabletId, TActorId(), new TEvents::TEvPoisonPill(), 0, pipeClientConfig); + server.GetRuntime()->SendToPipe(tabletId, TActorId(), new TEvents::TEvPoisonPill(), 0, pipeClientConfig); client.AlterTable(TablePlacement, Sprintf( R"___( @@ -2067,7 +2067,7 @@ Y_UNIT_TEST_SUITE(TClientTest) { TIntrusivePtr<TTabletStorageInfo> tabletInfo = CreateTestTabletInfo(tabletId, TTabletTypes::TX_DUMMY); TIntrusivePtr<TTabletSetupInfo> setupInfo = new TTabletSetupInfo(&CreateFlatDummyTablet, TMailboxType::Simple, 0, TMailboxType::Simple, 0); - const TActorId edge = runtime.AllocateEdgeActor(); + const TActorId edge = runtime.AllocateEdgeActor(); const TActorId leaderTablet = runtime.Register(CreateTablet(edge, tabletInfo.Get(), setupInfo.Get(), 0, nullptr, nullptr)); const TActorId leaderId = runtime.GrabEdgeEvent<TEvTablet::TEvRestored>(edge)->Get()->UserTabletActor; @@ -2147,7 +2147,7 @@ Y_UNIT_TEST_SUITE(TClientTest) { TIntrusivePtr<TTabletStorageInfo> tabletInfo = CreateTestTabletInfo(tabletId, TTabletTypes::TX_DUMMY); TIntrusivePtr<TTabletSetupInfo> setupInfo = new TTabletSetupInfo(&CreateFlatDummyTablet, TMailboxType::Simple, 0, TMailboxType::Simple, 0); - const TActorId edge = runtime.AllocateEdgeActor(); + const TActorId edge = runtime.AllocateEdgeActor(); { const TActorId leaderTablet = runtime.Register(CreateTablet(edge, tabletInfo.Get(), setupInfo.Get(), 0, nullptr, nullptr)); @@ -2198,7 +2198,7 @@ Y_UNIT_TEST_SUITE(TClientTest) { TIntrusivePtr<TTabletStorageInfo> tabletInfo = CreateTestTabletInfo(tabletId, TTabletTypes::TX_DUMMY); TIntrusivePtr<TTabletSetupInfo> setupInfo = new TTabletSetupInfo(&CreateFlatDummyTablet, TMailboxType::Simple, 0, TMailboxType::Simple, 0); - const TActorId edge = runtime.AllocateEdgeActor(); + const TActorId edge = runtime.AllocateEdgeActor(); const TActorId leaderTablet = runtime.Register(CreateTablet(edge, tabletInfo.Get(), setupInfo.Get(), 0, nullptr, nullptr)); const TActorId leaderId = runtime.GrabEdgeEvent<TEvTablet::TEvRestored>(edge)->Get()->UserTabletActor; @@ -2321,7 +2321,7 @@ Y_UNIT_TEST_SUITE(TClientTest) { TIntrusivePtr<TTabletStorageInfo> tabletInfo = CreateTestTabletInfo(tabletId, TTabletTypes::TX_DUMMY); TIntrusivePtr<TTabletSetupInfo> setupInfo = new TTabletSetupInfo(&CreateFlatDummyTablet, TMailboxType::Simple, 0, TMailboxType::Simple, 0); - const TActorId edge = runtime.AllocateEdgeActor(); + const TActorId edge = runtime.AllocateEdgeActor(); const TActorId leaderTablet = runtime.Register(CreateTablet(edge, tabletInfo.Get(), setupInfo.Get(), 0, nullptr, nullptr)); const TActorId leaderId = runtime.GrabEdgeEvent<TEvTablet::TEvRestored>(edge)->Get()->UserTabletActor; diff --git a/ydb/core/client/flat_ut.cpp b/ydb/core/client/flat_ut.cpp index d575519f8df..362c052aa11 100644 --- a/ydb/core/client/flat_ut.cpp +++ b/ydb/core/client/flat_ut.cpp @@ -2750,8 +2750,8 @@ Y_UNIT_TEST_SUITE(TFlatTest) { for (auto tabletId : xrange(72075186224037888, 72075186224037893)) { cleverServer.GetRuntime()->Send( - new IEventHandle(MakeTabletResolverID(), TActorId(), - new TEvTabletResolver::TEvTabletProblem(tabletId, TActorId()) + new IEventHandle(MakeTabletResolverID(), TActorId(), + new TEvTabletResolver::TEvTabletProblem(tabletId, TActorId()) )); } diff --git a/ydb/core/client/flat_ut_client.h b/ydb/core/client/flat_ut_client.h index 05f0258066d..07f5b30cc6b 100644 --- a/ydb/core/client/flat_ut_client.h +++ b/ydb/core/client/flat_ut_client.h @@ -61,8 +61,8 @@ public: void ResetSchemeCache(Tests::TServer &server, TTableId tableId) { TTestActorRuntime* runtime = server.GetRuntime(); - TActorId txProxy = MakeTxProxyID(); - TActorId sender = runtime->AllocateEdgeActor(); + TActorId txProxy = MakeTxProxyID(); + TActorId sender = runtime->AllocateEdgeActor(); TAutoPtr<TEvTxUserProxy::TEvInvalidateTable> ev(new TEvTxUserProxy::TEvInvalidateTable(tableId)); runtime->Send(new IEventHandle(txProxy, sender, ev.Release())); TAutoPtr<IEventHandle> handle; @@ -72,16 +72,16 @@ public: void KillTablet(Tests::TServer &server, ui64 tabletId) { TTestActorRuntime* runtime = server.GetRuntime(); - TActorId sender = runtime->AllocateEdgeActor(); + TActorId sender = runtime->AllocateEdgeActor(); - runtime->Send(new IEventHandle(MakeTabletResolverID(), sender, new TEvTabletResolver::TEvTabletProblem(tabletId, TActorId()))); + runtime->Send(new IEventHandle(MakeTabletResolverID(), sender, new TEvTabletResolver::TEvTabletProblem(tabletId, TActorId()))); runtime->Send(new IEventHandle(MakeTabletResolverID(), sender, new TEvTabletResolver::TEvForward(tabletId, nullptr))); TAutoPtr<IEventHandle> handle; auto forwardResult = runtime->GrabEdgeEventRethrow<TEvTabletResolver::TEvForwardResult>(handle); UNIT_ASSERT(forwardResult && forwardResult->Tablet); runtime->Send(new IEventHandle(forwardResult->Tablet, sender, new TEvents::TEvPoisonPill())); - runtime->Send(new IEventHandle(MakeTabletResolverID(), sender, new TEvTabletResolver::TEvTabletProblem(tabletId, TActorId()))); + runtime->Send(new IEventHandle(MakeTabletResolverID(), sender, new TEvTabletResolver::TEvTabletProblem(tabletId, TActorId()))); } TVector<ui64> GetTablePartitions(const TString& tablePath) { diff --git a/ydb/core/client/locks_ut.cpp b/ydb/core/client/locks_ut.cpp index c9388b78b29..47b7c3dfdd5 100644 --- a/ydb/core/client/locks_ut.cpp +++ b/ydb/core/client/locks_ut.cpp @@ -49,8 +49,8 @@ public: void ResetSchemeCache(TServer server, TTableId tableId) { TTestActorRuntime* runtime = server.GetRuntime(); - TActorId txProxy = MakeTxProxyID(); - TActorId sender = runtime->AllocateEdgeActor(); + TActorId txProxy = MakeTxProxyID(); + TActorId sender = runtime->AllocateEdgeActor(); TAutoPtr<TEvTxUserProxy::TEvInvalidateTable> ev(new TEvTxUserProxy::TEvInvalidateTable(tableId)); runtime->Send(new IEventHandle(txProxy, sender, ev.Release())); TAutoPtr<IEventHandle> handle; diff --git a/ydb/core/client/metadata/functions_metadata.cpp b/ydb/core/client/metadata/functions_metadata.cpp index e3769866301..6f8a0423745 100644 --- a/ydb/core/client/metadata/functions_metadata.cpp +++ b/ydb/core/client/metadata/functions_metadata.cpp @@ -1,70 +1,70 @@ -#include "functions_metadata.h" - +#include "functions_metadata.h" + #include <ydb/library/yql/minikql/mkql_function_metadata.h> #include <ydb/core/protos/scheme_type_operation.pb.h> - - -namespace NKikimr { -using namespace NMiniKQL; - + + +namespace NKikimr { +using namespace NMiniKQL; + void SerializeMetadata(const IBuiltinFunctionRegistry& funcRegistry, TString* out) -{ - NKikimrSchemeTypeOperation::TMetadata metadata; +{ + NKikimrSchemeTypeOperation::TMetadata metadata; for (const auto& op : funcRegistry.GetFunctions()) { - auto protoOp = metadata.AddOperation(); - protoOp->SetName(op.first); - for (const auto& desc : op.second) { - auto protoDesc = protoOp->AddDescription(); + auto protoOp = metadata.AddOperation(); + protoOp->SetName(op.first); + for (const auto& desc : op.second) { + auto protoDesc = protoOp->AddDescription(); for (const auto* arg = desc.ResultAndArgs; arg->SchemeType; ++arg) { - auto protoArg = protoDesc->AddArg(); - protoArg->SetSchemeType(arg->SchemeType); - protoArg->SetFlags(arg->Flags); - } - } - } - + auto protoArg = protoDesc->AddArg(); + protoArg->SetSchemeType(arg->SchemeType); + protoArg->SetFlags(arg->Flags); + } + } + } + Y_PROTOBUF_SUPPRESS_NODISCARD metadata.SerializeToString(out); -} - +} + void DeserializeMetadata(TStringBuf buffer, IBuiltinFunctionRegistry& funcRegistry) -{ - NKikimrSchemeTypeOperation::TMetadata metadata; +{ + NKikimrSchemeTypeOperation::TMetadata metadata; Y_VERIFY(metadata.ParseFromArray(buffer.data(), buffer.size())); - size_t totalArgsToAllocate = 0; - for (const auto& protoOp : metadata.GetOperation()) { - for (const auto& protoDesc : protoOp.GetDescription()) { - totalArgsToAllocate += protoDesc.ArgSize() + 1; - } - } - + size_t totalArgsToAllocate = 0; + for (const auto& protoOp : metadata.GetOperation()) { + for (const auto& protoDesc : protoOp.GetDescription()) { + totalArgsToAllocate += protoDesc.ArgSize() + 1; + } + } + TFunctionParamMetadataList arguments; - arguments.resize(totalArgsToAllocate); - - TFunctionsMap functions; - functions.reserve(metadata.OperationSize()); - - size_t argPosition = 0; - for (const auto& protoOp : metadata.GetOperation()) { - auto& desc = functions[protoOp.GetName()]; - Y_VERIFY(desc.empty()); - desc.reserve(protoOp.DescriptionSize()); - for (const auto& protoDesc : protoOp.GetDescription()) { - const size_t firstArg = argPosition; - for (const auto& protoArg : protoDesc.GetArg()) { - auto& arg = arguments[argPosition++]; - arg.SchemeType = protoArg.GetSchemeType(); - arg.Flags = protoArg.GetFlags(); - } - - ++argPosition; // terminating arg - + arguments.resize(totalArgsToAllocate); + + TFunctionsMap functions; + functions.reserve(metadata.OperationSize()); + + size_t argPosition = 0; + for (const auto& protoOp : metadata.GetOperation()) { + auto& desc = functions[protoOp.GetName()]; + Y_VERIFY(desc.empty()); + desc.reserve(protoOp.DescriptionSize()); + for (const auto& protoDesc : protoOp.GetDescription()) { + const size_t firstArg = argPosition; + for (const auto& protoArg : protoDesc.GetArg()) { + auto& arg = arguments[argPosition++]; + arg.SchemeType = protoArg.GetSchemeType(); + arg.Flags = protoArg.GetFlags(); + } + + ++argPosition; // terminating arg + desc.push_back(TFunctionDescriptor(&arguments[firstArg], nullptr)); - } - } - - Y_VERIFY(argPosition == arguments.size()); - + } + } + + Y_VERIFY(argPosition == arguments.size()); + funcRegistry.RegisterAll(std::move(functions), std::move(arguments)); -} - -} // namespace NKikimr +} + +} // namespace NKikimr diff --git a/ydb/core/client/metadata/functions_metadata.h b/ydb/core/client/metadata/functions_metadata.h index 404df002a55..a9a17dd2eb4 100644 --- a/ydb/core/client/metadata/functions_metadata.h +++ b/ydb/core/client/metadata/functions_metadata.h @@ -1,14 +1,14 @@ -#pragma once - -#include <util/generic/fwd.h> - -namespace NKikimr { - +#pragma once + +#include <util/generic/fwd.h> + +namespace NKikimr { + namespace NMiniKQL { class IBuiltinFunctionRegistry; } void SerializeMetadata(const NMiniKQL::IBuiltinFunctionRegistry& funcRegistry, TString* out); void DeserializeMetadata(TStringBuf buffer, NMiniKQL::IBuiltinFunctionRegistry& funcRegistry); - -} // namespace NKikimr + +} // namespace NKikimr diff --git a/ydb/core/client/metadata/types_metadata.cpp b/ydb/core/client/metadata/types_metadata.cpp index 0828e4871bf..c1c2265734f 100644 --- a/ydb/core/client/metadata/types_metadata.cpp +++ b/ydb/core/client/metadata/types_metadata.cpp @@ -1,42 +1,42 @@ -#include "types_metadata.h" - +#include "types_metadata.h" + #include <ydb/core/protos/scheme_type_metadata.pb.h> #include <ydb/core/scheme_types/scheme_type_metadata.h> - + #include <util/generic/vector.h> - -namespace NKikimr { - + +namespace NKikimr { + void SerializeMetadata(const NScheme::TTypeMetadataRegistry& typesRegistry, TString* out) -{ - NKikimrSchemeTypeMetadata::TMetadata metadata; - for (const auto& type: typesRegistry) { - auto protoType = metadata.AddType(); - protoType->SetId(type.second->GetTypeId()); - protoType->SetName(type.second->GetName()); - } +{ + NKikimrSchemeTypeMetadata::TMetadata metadata; + for (const auto& type: typesRegistry) { + auto protoType = metadata.AddType(); + protoType->SetId(type.second->GetTypeId()); + protoType->SetName(type.second->GetName()); + } Y_PROTOBUF_SUPPRESS_NODISCARD metadata.SerializeToString(out); -} - -void DeserializeMetadata(TStringBuf buffer, NScheme::TTypeMetadataRegistry* registry) -{ - using TTypeMetadata = NScheme::TTypeMetadataRegistry::TTypeMetadata; - - NKikimrSchemeTypeMetadata::TMetadata metadata; +} + +void DeserializeMetadata(TStringBuf buffer, NScheme::TTypeMetadataRegistry* registry) +{ + using TTypeMetadata = NScheme::TTypeMetadataRegistry::TTypeMetadata; + + NKikimrSchemeTypeMetadata::TMetadata metadata; Y_VERIFY(metadata.ParseFromArray(buffer.data(), buffer.size())); - + TVector<TTypeMetadata> deserializedMetadata; - deserializedMetadata.reserve(metadata.TypeSize()); - for (const auto& protoType : metadata.GetType()) { - TTypeMetadata typeMetadata( + deserializedMetadata.reserve(metadata.TypeSize()); + for (const auto& protoType : metadata.GetType()) { + TTypeMetadata typeMetadata( protoType.GetId(), protoType.GetName()); - deserializedMetadata.push_back(typeMetadata); - } - - registry->Clear(); - for (auto& type : deserializedMetadata) { - registry->Register(&type); - } -} - -} // namespace NKikimr + deserializedMetadata.push_back(typeMetadata); + } + + registry->Clear(); + for (auto& type : deserializedMetadata) { + registry->Register(&type); + } +} + +} // namespace NKikimr diff --git a/ydb/core/client/metadata/types_metadata.h b/ydb/core/client/metadata/types_metadata.h index 7eba98968a3..ff366d513f3 100644 --- a/ydb/core/client/metadata/types_metadata.h +++ b/ydb/core/client/metadata/types_metadata.h @@ -1,14 +1,14 @@ -#pragma once - -#include <util/generic/fwd.h> - - -namespace NKikimr { -namespace NScheme { - class TTypeMetadataRegistry; -} // namespace NScheme - +#pragma once + +#include <util/generic/fwd.h> + + +namespace NKikimr { +namespace NScheme { + class TTypeMetadataRegistry; +} // namespace NScheme + void SerializeMetadata(const NScheme::TTypeMetadataRegistry& typesRegistry, TString* out); -void DeserializeMetadata(TStringBuf buffer, NScheme::TTypeMetadataRegistry* registry); - -} // namespace NKikimr +void DeserializeMetadata(TStringBuf buffer, NScheme::TTypeMetadataRegistry* registry); + +} // namespace NKikimr diff --git a/ydb/core/client/metadata/ut/functions_metadata_ut.cpp b/ydb/core/client/metadata/ut/functions_metadata_ut.cpp index 30f464cb175..0713c33e8dd 100644 --- a/ydb/core/client/metadata/ut/functions_metadata_ut.cpp +++ b/ydb/core/client/metadata/ut/functions_metadata_ut.cpp @@ -1,40 +1,40 @@ #include <ydb/core/client/metadata/functions_metadata.h> - + #include <ydb/core/scheme_types/scheme_type_registry.h> #include <ydb/public/lib/scheme_types/scheme_type_id.h> #include <ydb/library/yql/minikql/invoke_builtins/mkql_builtins.h> - + #include <library/cpp/testing/unittest/registar.h> - - -using namespace NKikimr; -using namespace NMiniKQL; - -static const NScheme::TTypeId Uint32Id = NScheme::NTypeIds::Uint32; - + + +using namespace NKikimr; +using namespace NMiniKQL; + +static const NScheme::TTypeId Uint32Id = NScheme::NTypeIds::Uint32; + Y_UNIT_TEST_SUITE(TFunctionsMetadataTest) -{ +{ Y_UNIT_TEST(Serialization) { - static TFunctionParamMetadata AddUi32Metadata[] = { - { Uint32Id, 0 }, // result - { Uint32Id, 0 }, // first arg - { Uint32Id, 0 }, // second arg - { 0, 0 } - }; - + static TFunctionParamMetadata AddUi32Metadata[] = { + { Uint32Id, 0 }, // result + { Uint32Id, 0 }, // first arg + { Uint32Id, 0 }, // second arg + { 0, 0 } + }; + const auto functionRegistry = CreateBuiltinRegistry(); functionRegistry->Register("MyAdd", TFunctionDescriptor(AddUi32Metadata, nullptr)); - + TString metadata; SerializeMetadata(*functionRegistry, &metadata); DeserializeMetadata(metadata, *functionRegistry); - - std::pair<NScheme::TTypeId, bool> argTypes[] = { - { Uint32Id, false}, + + std::pair<NScheme::TTypeId, bool> argTypes[] = { { Uint32Id, false}, - { Uint32Id, false} - }; - + { Uint32Id, false}, + { Uint32Id, false} + }; + const auto op = functionRegistry->GetBuiltin("MyAdd", argTypes, Y_ARRAY_SIZE(argTypes)); UNIT_ASSERT_EQUAL(op.Function, nullptr); UNIT_ASSERT_EQUAL(op.ResultAndArgs[0].SchemeType, Uint32Id); @@ -44,5 +44,5 @@ Y_UNIT_TEST_SUITE(TFunctionsMetadataTest) UNIT_ASSERT_EQUAL(op.ResultAndArgs[2].SchemeType, Uint32Id); UNIT_ASSERT_EQUAL(op.ResultAndArgs[2].Flags, 0); UNIT_ASSERT_EQUAL(op.ResultAndArgs[3].SchemeType, 0); - } -} + } +} diff --git a/ydb/core/client/metadata/ut/ya.make b/ydb/core/client/metadata/ut/ya.make index 1904a30fd65..c4d21862f14 100644 --- a/ydb/core/client/metadata/ut/ya.make +++ b/ydb/core/client/metadata/ut/ya.make @@ -1,15 +1,15 @@ UNITTEST_FOR(ydb/core/client/metadata) - + OWNER(g:kikimr) - -SRCS( - functions_metadata_ut.cpp -) - + +SRCS( + functions_metadata_ut.cpp +) + PEERDIR( ydb/library/yql/public/udf/service/stub ) YQL_LAST_ABI_VERSION() -END() +END() diff --git a/ydb/core/client/metadata/ya.make b/ydb/core/client/metadata/ya.make index 637c6a8d589..a4a9d732427 100644 --- a/ydb/core/client/metadata/ya.make +++ b/ydb/core/client/metadata/ya.make @@ -1,18 +1,18 @@ -LIBRARY() - +LIBRARY() + OWNER(g:kikimr) - -SRCS( - types_metadata.cpp - functions_metadata.cpp -) - -PEERDIR( + +SRCS( + types_metadata.cpp + functions_metadata.cpp +) + +PEERDIR( ydb/core/protos ydb/core/scheme_types ydb/library/yql/minikql/invoke_builtins -) - +) + YQL_LAST_ABI_VERSION() END() diff --git a/ydb/core/client/minikql_compile/compile_context.h b/ydb/core/client/minikql_compile/compile_context.h index 3d6361fd46c..22a9ebdf37e 100644 --- a/ydb/core/client/minikql_compile/compile_context.h +++ b/ydb/core/client/minikql_compile/compile_context.h @@ -21,7 +21,7 @@ using NKikimr::NMiniKQL::TParametersBuilder; using NKikimr::NMiniKQL::TTypeEnvironment; using NKikimr::NMiniKQL::TType; using NKikimr::NMiniKQL::TRuntimeNode; -using NKikimr::NMiniKQL::TKikimrProgramBuilder; +using NKikimr::NMiniKQL::TKikimrProgramBuilder; struct TContext : public TAtomicRefCount<TContext> { @@ -50,7 +50,7 @@ public: const IFunctionRegistry* FuncRegistry; const TTypeEnvironment* TypeEnv; NKikimr::NUdf::ITypeInfoHelper::TPtr TypeInfoHelper; - TAutoPtr<TKikimrProgramBuilder> PgmBuilder; + TAutoPtr<TKikimrProgramBuilder> PgmBuilder; TMaybe<TParametersBuilder> ParamsBuilder; private: diff --git a/ydb/core/client/minikql_compile/db_key_resolver.h b/ydb/core/client/minikql_compile/db_key_resolver.h index 69e886ee2af..e748525fa81 100644 --- a/ydb/core/client/minikql_compile/db_key_resolver.h +++ b/ydb/core/client/minikql_compile/db_key_resolver.h @@ -77,7 +77,7 @@ public: }; }; - virtual void ResolveTables(const TVector<TTable>& tables, NActors::TActorId responseTo) = 0; // TEvResolveTablesResult. + virtual void ResolveTables(const TVector<TTable>& tables, NActors::TActorId responseTo) = 0; // TEvResolveTablesResult. }; } // namespace NYql diff --git a/ydb/core/client/minikql_compile/mkql_compile_service.cpp b/ydb/core/client/minikql_compile/mkql_compile_service.cpp index 90c448685c4..e01d61a22cc 100644 --- a/ydb/core/client/minikql_compile/mkql_compile_service.cpp +++ b/ydb/core/client/minikql_compile/mkql_compile_service.cpp @@ -31,7 +31,7 @@ public: using TPtr = TIntrusivePtr<TCompileContext>; TCompileContext(const TString& pgm, - TActorId sender, + TActorId sender, const TAlignedPagePoolCounters& allocPoolCounters, const NMiniKQL::IFunctionRegistry* functionRegistry) : Program(pgm) @@ -50,7 +50,7 @@ public: } TString Program; - TActorId ResponseTo; + TActorId ResponseTo; NMiniKQL::TScopedAlloc Alloc; NMiniKQL::TTypeEnvironment TypeEnv; ui64 Cookie; @@ -131,7 +131,7 @@ private: c->Retried = true; auto *compileActor = NYql::CreateCompileActor(c->Program, &c->TypeEnv, DbSchemeResolver.Get(), ctx.SelfID, std::move(msg->CompileResolveCookies), false); - const TActorId actId = ctx.ExecutorThread.RegisterActor(compileActor, TMailboxType::HTSwap, appData->UserPoolId); + const TActorId actId = ctx.ExecutorThread.RegisterActor(compileActor, TMailboxType::HTSwap, appData->UserPoolId); Compiling.insert(TCompilingMap::value_type(actId, c)); } } @@ -161,23 +161,23 @@ private: private: const size_t COMPILE_INFLIGHT_LIMIT; TQueue<TCompileContext::TPtr> CompileQueue; - using TCompilingMap = THashMap<TActorId, TCompileContext::TPtr>; + using TCompilingMap = THashMap<TActorId, TCompileContext::TPtr>; TCompilingMap Compiling; TIntrusivePtr<NMonitoring::TDynamicCounters> Counters; TAlignedPagePoolCounters AllocPoolCounters; - TActorId SchemeCache; + TActorId SchemeCache; THolder<NYql::IDbSchemeResolver> DbSchemeResolver; }; -TActorId MakeMiniKQLCompileServiceID() { +TActorId MakeMiniKQLCompileServiceID() { const char x[12] = "MKQLCompile"; - return TActorId(0, TStringBuf(x, 12)); + return TActorId(0, TStringBuf(x, 12)); } -const TActorId& GetMiniKQLCompileServiceID() { - static TActorId miniKQLCompileServiceID = MakeMiniKQLCompileServiceID(); +const TActorId& GetMiniKQLCompileServiceID() { + static TActorId miniKQLCompileServiceID = MakeMiniKQLCompileServiceID(); return miniKQLCompileServiceID; } diff --git a/ydb/core/client/minikql_compile/mkql_compile_service.h b/ydb/core/client/minikql_compile/mkql_compile_service.h index be07924c629..4b0aca14917 100644 --- a/ydb/core/client/minikql_compile/mkql_compile_service.h +++ b/ydb/core/client/minikql_compile/mkql_compile_service.h @@ -40,8 +40,8 @@ struct TMiniKQLCompileServiceEvents { }; }; -NActors::TActorId MakeMiniKQLCompileServiceID(); -const NActors::TActorId& GetMiniKQLCompileServiceID(); +NActors::TActorId MakeMiniKQLCompileServiceID(); +const NActors::TActorId& GetMiniKQLCompileServiceID(); NActors::IActor* CreateMiniKQLCompileService(size_t compileInflightLimit); NActors::IActor* CreateMiniKQLCompileService(size_t compileInflightLimit, THolder<NYql::IDbSchemeResolver>&& dbSchemeResolver); diff --git a/ydb/core/client/minikql_compile/yql_expr_minikql.cpp b/ydb/core/client/minikql_compile/yql_expr_minikql.cpp index 026e1e59103..76f38bf35cb 100644 --- a/ydb/core/client/minikql_compile/yql_expr_minikql.cpp +++ b/ydb/core/client/minikql_compile/yql_expr_minikql.cpp @@ -986,7 +986,7 @@ TIntrusivePtr<NCommon::IMkqlCallableCompiler> CreateMkqlCompiler(TContext::TPtr } } - TTableRangeOptions options = mkqlContext->PgmBuilder->GetDefaultTableRangeOptions(); + TTableRangeOptions options = mkqlContext->PgmBuilder->GetDefaultTableRangeOptions(); TVector<ui32> keyTypes(Max(fromComponents, toComponents)); TVector<TRuntimeNode> from(fromComponents); TVector<TRuntimeNode> to(toComponents); @@ -1085,7 +1085,7 @@ TIntrusivePtr<NCommon::IMkqlCallableCompiler> CreateMkqlCompiler(TContext::TPtr keyTypes[i] = column->Type; } - auto update = mkqlContext->PgmBuilder->GetUpdateRowBuilder(); + auto update = mkqlContext->PgmBuilder->GetUpdateRowBuilder(); auto updateTuple = node.Child(2); for (ui32 i = 0; i < updateTuple->ChildrenSize(); ++i) { auto child = updateTuple->Child(i); @@ -1107,7 +1107,7 @@ TIntrusivePtr<NCommon::IMkqlCallableCompiler> CreateMkqlCompiler(TContext::TPtr } } - TRuntimeNode result = mkqlContext->PgmBuilder->UpdateRow(*lookup->TableId, keyTypes, row, update); + TRuntimeNode result = mkqlContext->PgmBuilder->UpdateRow(*lookup->TableId, keyTypes, row, update); return result; }); @@ -1136,7 +1136,7 @@ TIntrusivePtr<NCommon::IMkqlCallableCompiler> CreateMkqlCompiler(TContext::TPtr keyTypes[i] = column->Type; } - TRuntimeNode result = mkqlContext->PgmBuilder->EraseRow(*lookup->TableId, keyTypes, row); + TRuntimeNode result = mkqlContext->PgmBuilder->EraseRow(*lookup->TableId, keyTypes, row); return result; }); @@ -1324,7 +1324,7 @@ void CollectKeys(const TExprNode* root, TContext::TPtr ctx) { TFuture<TConvertResult> ConvertToMiniKQL(TExprContainer::TPtr expr, - const NKikimr::NMiniKQL::IFunctionRegistry* functionRegistry, + const NKikimr::NMiniKQL::IFunctionRegistry* functionRegistry, const NKikimr::NMiniKQL::TTypeEnvironment* typeEnv, IDbSchemeResolver* dbSchemeResolver) { @@ -1429,7 +1429,7 @@ public: TMiniKQLCompileActor(const TString& program, const NKikimr::NMiniKQL::TTypeEnvironment* typeEnv, IDbSchemeResolver* dbSchemeResolver, - TActorId responseTo, + TActorId responseTo, THashMap<TString, ui64> &&resolveRefreshCookies, bool forceCacheRefresh) : TypeEnv(typeEnv) @@ -1613,7 +1613,7 @@ private: TString Program; TContext::TPtr CompileCtx; IDbSchemeResolver* DbSchemeResolver; - TActorId ResponseTo; + TActorId ResponseTo; TExprContainer::TPtr Expr; TIntrusivePtr<NCommon::IMkqlCallableCompiler> Compiler; THashMap<TString, ui64> ResolveRefreshCookies; @@ -1623,7 +1623,7 @@ NActors::IActor* CreateCompileActor(const TString& program, const NKikimr::NMiniKQL::TTypeEnvironment* typeEnv, IDbSchemeResolver* dbSchemeResolver, - TActorId responseTo, + TActorId responseTo, THashMap<TString, ui64> &&resolveRefreshCookies, bool forceCacheRefresh) { diff --git a/ydb/core/client/minikql_compile/yql_expr_minikql.h b/ydb/core/client/minikql_compile/yql_expr_minikql.h index bf0647a3d53..ab6011c105b 100644 --- a/ydb/core/client/minikql_compile/yql_expr_minikql.h +++ b/ydb/core/client/minikql_compile/yql_expr_minikql.h @@ -17,7 +17,7 @@ namespace NKikimr { namespace NMiniKQL { class TProgramBuilder; -class IFunctionRegistry; +class IFunctionRegistry; } // namespace NMiniKQL } // namespace NKikimr @@ -33,7 +33,7 @@ struct TExprContainer : public TAtomicRefCount<TExprContainer> { NThreading::TFuture<TConvertResult> ConvertToMiniKQL(TExprContainer::TPtr expr, - const NKikimr::NMiniKQL::IFunctionRegistry* functionRegistry, + const NKikimr::NMiniKQL::IFunctionRegistry* functionRegistry, const NKikimr::NMiniKQL::TTypeEnvironment* typeEnv, IDbSchemeResolver* dbSchemeResolver); @@ -56,7 +56,7 @@ NActors::IActor* CreateCompileActor(const TString& program, const NKikimr::NMiniKQL::TTypeEnvironment* typeEnv, IDbSchemeResolver* dbSchemeResolver, - NActors::TActorId responseTo, + NActors::TActorId responseTo, THashMap<TString, ui64> &&resolveRefreshCookies, bool forceCacheRefresh); // TEvCompileResult. diff --git a/ydb/core/client/minikql_result_lib/data_funcs.inl b/ydb/core/client/minikql_result_lib/data_funcs.inl index a3b45615131..a152c054a1a 100644 --- a/ydb/core/client/minikql_result_lib/data_funcs.inl +++ b/ydb/core/client/minikql_result_lib/data_funcs.inl @@ -3,7 +3,7 @@ #define ENSURE_SCHEME_TYPE(expected, gotName) \ do { \ - Y_ENSURE(expected == NScheme::NTypeIds::gotName, "Data scheme type mismatch: expected " << expected << ", but got " #gotName << "."); \ + Y_ENSURE(expected == NScheme::NTypeIds::gotName, "Data scheme type mismatch: expected " << expected << ", but got " #gotName << "."); \ } while (0); diff --git a/ydb/core/client/minikql_result_lib/objects_ut.cpp b/ydb/core/client/minikql_result_lib/objects_ut.cpp index 4c5b929c606..49b3d300dcb 100644 --- a/ydb/core/client/minikql_result_lib/objects_ut.cpp +++ b/ydb/core/client/minikql_result_lib/objects_ut.cpp @@ -17,7 +17,7 @@ Y_UNIT_TEST(TOptionalTest) { TType type; type.SetKind(ETypeKind::Optional); type.MutableOptional()->MutableItem()->SetKind(ETypeKind::Data); - type.MutableOptional()->MutableItem()->MutableData()->SetScheme(NScheme::NTypeIds::Bool); + type.MutableOptional()->MutableItem()->MutableData()->SetScheme(NScheme::NTypeIds::Bool); TValue value; value.MutableOptional()->SetBool(true); TOptional opt(value, type); @@ -27,7 +27,7 @@ Y_UNIT_TEST(TOptionalTest) { TType type; type.SetKind(ETypeKind::Optional); type.MutableOptional()->MutableItem()->SetKind(ETypeKind::Data); - type.MutableOptional()->MutableItem()->MutableData()->SetScheme(NScheme::NTypeIds::Uint64); + type.MutableOptional()->MutableItem()->MutableData()->SetScheme(NScheme::NTypeIds::Uint64); TValue value; value.MutableOptional()->SetUint64(200100); TOptional opt(value, type); @@ -40,7 +40,7 @@ Y_UNIT_TEST(TListTest) { TType type; type.SetKind(ETypeKind::List); type.MutableList()->MutableItem()->SetKind(ETypeKind::Data); - type.MutableList()->MutableItem()->MutableData()->SetScheme(NScheme::NTypeIds::Int32); + type.MutableList()->MutableItem()->MutableData()->SetScheme(NScheme::NTypeIds::Int32); TValue value; value.AddList()->SetInt32(-100); value.AddList()->SetInt32(0); @@ -67,7 +67,7 @@ Y_UNIT_TEST(TListTest) { type.SetKind(ETypeKind::List); type.MutableList()->MutableItem()->SetKind(ETypeKind::Optional); type.MutableList()->MutableItem()->MutableOptional()->MutableItem()->SetKind(ETypeKind::Data); - type.MutableList()->MutableItem()->MutableOptional()->MutableItem()->MutableData()->SetScheme(NScheme::NTypeIds::Int32); + type.MutableList()->MutableItem()->MutableOptional()->MutableItem()->MutableData()->SetScheme(NScheme::NTypeIds::Int32); TValue value; value.AddList()->MutableOptional()->SetInt32(-100); value.AddList()->MutableOptional()->SetInt32(0); @@ -93,12 +93,12 @@ Y_UNIT_TEST(TTupleTest) { auto el0Type = type.MutableTuple()->AddElement(); el0Type->SetKind(ETypeKind::Optional); el0Type->MutableOptional()->MutableItem()->SetKind(ETypeKind::Data); - el0Type->MutableOptional()->MutableItem()->MutableData()->SetScheme(NScheme::NTypeIds::Int32); + el0Type->MutableOptional()->MutableItem()->MutableData()->SetScheme(NScheme::NTypeIds::Int32); auto el1Type = type.MutableTuple()->AddElement(); el1Type->SetKind(ETypeKind::List); el1Type->MutableList()->MutableItem()->SetKind(ETypeKind::Data); - el1Type->MutableList()->MutableItem()->MutableData()->SetScheme(NScheme::NTypeIds::Uint64); + el1Type->MutableList()->MutableItem()->MutableData()->SetScheme(NScheme::NTypeIds::Uint64); TValue value; value.AddTuple()->MutableOptional()->SetInt32(911); @@ -131,11 +131,11 @@ Y_UNIT_TEST(TStructTest) { auto mem1Type = type.MutableStruct()->AddMember(); mem1Type->SetName("a"); mem1Type->MutableType()->SetKind(ETypeKind::Data); - mem1Type->MutableType()->MutableData()->SetScheme(NScheme::NTypeIds::Int64); + mem1Type->MutableType()->MutableData()->SetScheme(NScheme::NTypeIds::Int64); auto mem2Type = type.MutableStruct()->AddMember(); mem2Type->SetName("b"); mem2Type->MutableType()->SetKind(ETypeKind::Data); - mem2Type->MutableType()->MutableData()->SetScheme(NScheme::NTypeIds::Int32); + mem2Type->MutableType()->MutableData()->SetScheme(NScheme::NTypeIds::Int32); TValue value; value.AddStruct()->SetInt64(-1000); diff --git a/ydb/core/client/scheme_cache_lib/yql_db_scheme_resolver.cpp b/ydb/core/client/scheme_cache_lib/yql_db_scheme_resolver.cpp index 4fdb0204f73..eb74fa36e94 100644 --- a/ydb/core/client/scheme_cache_lib/yql_db_scheme_resolver.cpp +++ b/ydb/core/client/scheme_cache_lib/yql_db_scheme_resolver.cpp @@ -19,8 +19,8 @@ class TTableProxyActor : public TActorBootstrapped<TTableProxyActor> { using TTableResult = NYql::IDbSchemeResolver::TTableResult; using TTableResults = NYql::IDbSchemeResolver::TTableResults; - const TActorId SchemeCache; - const TActorId ResponseTo; + const TActorId SchemeCache; + const TActorId ResponseTo; TVector<TTable> Tables; void Handle(TEvTxProxySchemeCache::TEvNavigateKeySetResult::TPtr &ev, const TActorContext &ctx) { @@ -101,7 +101,7 @@ public: return NKikimrServices::TActivity::TABLE_SCHEME_RESOLVER; } - TTableProxyActor(TActorId schemeCache, TActorId responseTo, const TVector<TTable> &tables) + TTableProxyActor(TActorId schemeCache, TActorId responseTo, const TVector<TTable> &tables) : SchemeCache(schemeCache) , ResponseTo(responseTo) , Tables(tables) @@ -139,7 +139,7 @@ public: { } - TDbSchemeResolver(TActorSystem *actorSystem, const TActorId &schemeCacheActor) + TDbSchemeResolver(TActorSystem *actorSystem, const TActorId &schemeCacheActor) : HostActorSystem(actorSystem) , SchemeCacheActor(schemeCacheActor) {} @@ -157,17 +157,17 @@ public: return NThreading::MakeFuture(results); } - virtual void ResolveTables(const TVector<TTable>& tables, NActors::TActorId responseTo) override { + virtual void ResolveTables(const TVector<TTable>& tables, NActors::TActorId responseTo) override { TAutoPtr<NActors::IActor> proxyActor(new TTableProxyActor(SchemeCacheActor, responseTo, tables)); HostActorSystem->Register(proxyActor.Release(), TMailboxType::HTSwap, HostActorSystem->AppData<TAppData>()->UserPoolId); } private: TActorSystem *HostActorSystem; - TActorId SchemeCacheActor; + TActorId SchemeCacheActor; }; -NYql::IDbSchemeResolver* CreateDbSchemeResolver(TActorSystem *actorSystem, const TActorId &schemeCacheActor) { +NYql::IDbSchemeResolver* CreateDbSchemeResolver(TActorSystem *actorSystem, const TActorId &schemeCacheActor) { TAutoPtr<NYql::IDbSchemeResolver> resolver(new TDbSchemeResolver(actorSystem, schemeCacheActor)); return resolver.Release(); } diff --git a/ydb/core/client/scheme_cache_lib/yql_db_scheme_resolver.h b/ydb/core/client/scheme_cache_lib/yql_db_scheme_resolver.h index 359a3f82cb0..b1d2eaae8ee 100644 --- a/ydb/core/client/scheme_cache_lib/yql_db_scheme_resolver.h +++ b/ydb/core/client/scheme_cache_lib/yql_db_scheme_resolver.h @@ -7,13 +7,13 @@ class IDbSchemeResolver; namespace NActors { class TActorSystem; - struct TActorId; + struct TActorId; } namespace NKikimr { namespace NSchCache { -NYql::IDbSchemeResolver* CreateDbSchemeResolver(TActorSystem *actorSystem, const TActorId &schemeCacheActor); +NYql::IDbSchemeResolver* CreateDbSchemeResolver(TActorSystem *actorSystem, const TActorId &schemeCacheActor); } // namespace NSchCache } // namespace NKikimr diff --git a/ydb/core/client/server/grpc_proxy_status.cpp b/ydb/core/client/server/grpc_proxy_status.cpp index d0a7804f3a4..827335ad408 100644 --- a/ydb/core/client/server/grpc_proxy_status.cpp +++ b/ydb/core/client/server/grpc_proxy_status.cpp @@ -13,9 +13,9 @@ //////////////////////////////////////////// namespace NKikimr { -TActorId MakeGRpcProxyStatusID(ui32 node) { +TActorId MakeGRpcProxyStatusID(ui32 node) { char x[12] = {'g','r','p','c','p','r','x','y','s','t','a','t'}; - return TActorId(node, TStringBuf(x, 12)); + return TActorId(node, TStringBuf(x, 12)); } @@ -31,7 +31,7 @@ const ui32 TIMEOUT_SECONDS = 10; class TChooseProxyActorImpl : public TActorBootstrapped<TChooseProxyActorImpl> { using TBase = TActorBootstrapped<TChooseProxyActorImpl>; - TActorId Sender; + TActorId Sender; ui32 NodesRequested; ui32 NodesReceived; THolder<NMsgBusProxy::TBusChooseProxy> Request; @@ -47,7 +47,7 @@ public: } // - TChooseProxyActorImpl(const TActorId& sender) + TChooseProxyActorImpl(const TActorId& sender) : Sender(sender) , NodesRequested(0) , NodesReceived(0) @@ -59,7 +59,7 @@ public: void SendRequest(ui32 nodeId, const TActorContext &ctx) { - TActorId proxyNodeServiceId = MakeGRpcProxyStatusID(nodeId); + TActorId proxyNodeServiceId = MakeGRpcProxyStatusID(nodeId); ctx.Send(proxyNodeServiceId, new TEvGRpcProxyStatus::TEvGetStatusRequest(), IEventHandle::FlagTrackDelivery | IEventHandle::FlagSubscribeOnSession, nodeId); Nodes.emplace_back(nodeId); @@ -73,7 +73,7 @@ public: } void Bootstrap(const TActorContext& ctx) { - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); ctx.Send(nameserviceId, new TEvInterconnect::TEvListNodes()); TBase::Become(&TThis::StateRequestedBrowse); ctx.Schedule(TDuration::Seconds(TIMEOUT_SECONDS), new TEvents::TEvWakeup()); @@ -196,8 +196,8 @@ private: ui32 WriteSessions; ui32 ReadSessions; - TActorId Worker; - std::deque<TActorId> Requests; + TActorId Worker; + std::deque<TActorId> Requests; }; @@ -225,7 +225,7 @@ TGRpcProxyStatusActor::Bootstrap(const TActorContext &ctx) { void TGRpcProxyStatusActor::Handle(TEvGRpcProxyStatus::TEvRequest::TPtr &ev, const TActorContext& ctx) { Requests.push_back(ev->Sender); - if (Worker == TActorId()) { + if (Worker == TActorId()) { Worker = ctx.Register(new TChooseProxyActorImpl(ctx.SelfID)); } } @@ -242,7 +242,7 @@ TGRpcProxyStatusActor::Handle(TEvGRpcProxyStatus::TEvResponse::TPtr &ev, const T ctx.Send(sender, response.Release()); } Requests.clear(); - Worker = TActorId(); + Worker = TActorId(); } diff --git a/ydb/core/client/server/grpc_proxy_status.h b/ydb/core/client/server/grpc_proxy_status.h index b37bdb6c8ae..13ea9302624 100644 --- a/ydb/core/client/server/grpc_proxy_status.h +++ b/ydb/core/client/server/grpc_proxy_status.h @@ -12,7 +12,7 @@ namespace NKikimr { //////////////////////////////////////////// -TActorId MakeGRpcProxyStatusID(ui32 node); +TActorId MakeGRpcProxyStatusID(ui32 node); //////////////////////////////////////////// struct TEvGRpcProxyStatus { diff --git a/ydb/core/client/server/grpc_server.cpp b/ydb/core/client/server/grpc_server.cpp index e02ccaab29c..aa122da36a1 100644 --- a/ydb/core/client/server/grpc_server.cpp +++ b/ydb/core/client/server/grpc_server.cpp @@ -3,9 +3,9 @@ #include <ydb/core/client/server/msgbus_server_persqueue.h> #include <ydb/core/grpc_services/grpc_helper.h> -#include <library/cpp/grpc/server/grpc_request.h> -#include <library/cpp/grpc/server/grpc_counters.h> -#include <library/cpp/grpc/server/grpc_async_ctx_base.h> +#include <library/cpp/grpc/server/grpc_request.h> +#include <library/cpp/grpc/server/grpc_counters.h> +#include <library/cpp/grpc/server/grpc_async_ctx_base.h> #include <library/cpp/json/json_writer.h> @@ -31,7 +31,7 @@ using grpc::CompletionQueue; using NKikimrClient::TResponse; using NKikimrClient::TPersQueueRequest; -using NGrpc::IQueueEvent; +using NGrpc::IQueueEvent; using namespace NActors; using namespace NThreading; @@ -40,7 +40,7 @@ namespace NKikimr { namespace NGRpcProxy { namespace { -using TGrpcBaseAsyncContext = NGrpc::TBaseAsyncContext<NGRpcProxy::TGRpcService>; +using TGrpcBaseAsyncContext = NGrpc::TBaseAsyncContext<NGRpcProxy::TGRpcService>; template <typename TIn, typename TOut = TResponse> class TSimpleRequest @@ -62,7 +62,7 @@ public: TRequestCallback requestCallback, TActorSystem& as, const char* name, - NGrpc::ICounterBlockPtr counters) + NGrpc::ICounterBlockPtr counters) : TGrpcBaseAsyncContext(service, cq) , Server(server) , Cb(cb) @@ -351,7 +351,7 @@ private: TRequestCallback RequestCallback; TActorSystem& ActorSystem; const char* const Name; - NGrpc::ICounterBlockPtr Counters; + NGrpc::ICounterBlockPtr Counters; THolder<ServerAsyncResponseWriter<TOut>> Writer; @@ -373,13 +373,13 @@ TGRpcService::TGRpcService() : ActorSystem(nullptr) {} -void TGRpcService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr) { +void TGRpcService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr) { CQ = cq; Y_ASSERT(InitCb_); InitCb_(); } -TFuture<void> TGRpcService::Prepare(TActorSystem* system, const TActorId& pqMeta, const TActorId& msgBusProxy, +TFuture<void> TGRpcService::Prepare(TActorSystem* system, const TActorId& pqMeta, const TActorId& msgBusProxy, TIntrusivePtr<NMonitoring::TDynamicCounters> counters) { auto promise = NewPromise<void>(); InitCb_ = [=]() mutable { @@ -397,7 +397,7 @@ TFuture<void> TGRpcService::Prepare(TActorSystem* system, const TActorId& pqMeta return promise.GetFuture(); } -void TGRpcService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter *limiter) { +void TGRpcService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter *limiter) { Limiter_ = limiter; } diff --git a/ydb/core/client/server/grpc_server.h b/ydb/core/client/server/grpc_server.h index ff4895fe568..2f2ae1da6b6 100644 --- a/ydb/core/client/server/grpc_server.h +++ b/ydb/core/client/server/grpc_server.h @@ -4,7 +4,7 @@ #include <library/cpp/actors/core/actorsystem.h> #include <library/cpp/actors/core/actor_bootstrapped.h> -#include <library/cpp/grpc/server/grpc_server.h> +#include <library/cpp/grpc/server/grpc_server.h> #include <ydb/public/lib/deprecated/client/grpc_client.h> #include <ydb/public/lib/base/defs.h> #include <ydb/public/lib/base/msgbus.h> @@ -52,15 +52,15 @@ public: //! Implements interaction Kikimr via gRPC protocol. class TGRpcService - : public NGrpc::TGrpcServiceBase<NKikimrClient::TGRpcServer> + : public NGrpc::TGrpcServiceBase<NKikimrClient::TGRpcServer> { public: TGRpcService(); - void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; - void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; + void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; + void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; - NThreading::TFuture<void> Prepare(NActors::TActorSystem* system, const NActors::TActorId& pqMeta, const NActors::TActorId& msgBusProxy, TIntrusivePtr<NMonitoring::TDynamicCounters> counters); + NThreading::TFuture<void> Prepare(NActors::TActorSystem* system, const NActors::TActorId& pqMeta, const NActors::TActorId& msgBusProxy, TIntrusivePtr<NMonitoring::TDynamicCounters> counters); void Start(); bool IncRequest(); @@ -78,8 +78,8 @@ private: NActors::TActorSystem* ActorSystem; - NActors::TActorId PQMeta; - NActors::TActorId MsgBusProxy; + NActors::TActorId PQMeta; + NActors::TActorId MsgBusProxy; grpc::ServerCompletionQueue* CQ = nullptr; diff --git a/ydb/core/client/server/http_ping.h b/ydb/core/client/server/http_ping.h index 8845cfb2b14..e07aba1eea2 100644 --- a/ydb/core/client/server/http_ping.h +++ b/ydb/core/client/server/http_ping.h @@ -1,7 +1,7 @@ #pragma once #include <ydb/public/lib/base/defs.h> -#include <library/cpp/monlib/service/monservice.h> -#include <library/cpp/monlib/service/pages/mon_page.h> +#include <library/cpp/monlib/service/monservice.h> +#include <library/cpp/monlib/service/pages/mon_page.h> #include <ydb/core/mon/mon.h> namespace NKikimr { diff --git a/ydb/core/client/server/msgbus_http_server.h b/ydb/core/client/server/msgbus_http_server.h index 56ce2c9f662..911978c422e 100644 --- a/ydb/core/client/server/msgbus_http_server.h +++ b/ydb/core/client/server/msgbus_http_server.h @@ -1,7 +1,7 @@ #pragma once #include <ydb/public/lib/base/defs.h> -#include <library/cpp/monlib/service/monservice.h> -#include <library/cpp/monlib/service/pages/mon_page.h> +#include <library/cpp/monlib/service/monservice.h> +#include <library/cpp/monlib/service/pages/mon_page.h> #include <ydb/core/mon/mon.h> #include <library/cpp/actors/helpers/mon_histogram_helper.h> #include <ydb/public/lib/base/msgbus.h> diff --git a/ydb/core/client/server/msgbus_server.cpp b/ydb/core/client/server/msgbus_server.cpp index 0c411be4b47..8a4d8a3bbd5 100644 --- a/ydb/core/client/server/msgbus_server.cpp +++ b/ydb/core/client/server/msgbus_server.cpp @@ -1,4 +1,4 @@ -#include <library/cpp/monlib/messagebus/mon_messagebus.h> +#include <library/cpp/monlib/messagebus/mon_messagebus.h> #include <ydb/core/base/appdata.h> #include <ydb/core/node_whiteboard/node_whiteboard.h> #include "msgbus_server.h" @@ -430,7 +430,7 @@ protected: } } - TActorId WhiteboardServiceId; + TActorId WhiteboardServiceId; NBus::TBusServerSessionPtr Session; const NBus::TBusServerSessionConfig &SessionConfig; }; @@ -451,7 +451,7 @@ TMessageBusServer::~TMessageBusServer() { } -void TMessageBusServer::InitSession(TActorSystem *actorSystem, const TActorId &proxy) { +void TMessageBusServer::InitSession(TActorSystem *actorSystem, const TActorId &proxy) { ActorSystem = actorSystem; Proxy = proxy; Session = NBus::TBusServerSession::Create(&Protocol, this, SessionConfig, BusQueue); diff --git a/ydb/core/client/server/msgbus_server.h b/ydb/core/client/server/msgbus_server.h index 06b5bbafbb6..3d40fc90017 100644 --- a/ydb/core/client/server/msgbus_server.h +++ b/ydb/core/client/server/msgbus_server.h @@ -134,8 +134,8 @@ class TMessageBusServer : public IMessageBusServer, public NBus::IBusServerHandl const NBus::TBusServerSessionConfig &SessionConfig; NBus::TBusMessageQueuePtr BusQueue; NBus::TBusServerSessionPtr Session; - TActorId Proxy; - TActorId Monitor; + TActorId Proxy; + TActorId Monitor; std::shared_ptr<IPersQueueGetReadSessionsInfoWorkerFactory> PQReadSessionsInfoWorkerFactory; protected: TProtocol Protocol; @@ -150,7 +150,7 @@ public: ); ~TMessageBusServer(); - void InitSession(TActorSystem *actorSystem, const TActorId &proxy); + void InitSession(TActorSystem *actorSystem, const TActorId &proxy); void ShutdownSession(); void RegisterMonPage(NMonitoring::TBusNgMonPage *busMonPage); diff --git a/ydb/core/client/server/msgbus_server_cms.cpp b/ydb/core/client/server/msgbus_server_cms.cpp index 783e3bf47fd..53ada2a3fad 100644 --- a/ydb/core/client/server/msgbus_server_cms.cpp +++ b/ydb/core/client/server/msgbus_server_cms.cpp @@ -291,7 +291,7 @@ private: NKikimrClient::TCmsRequest Request; NKikimrClient::TCmsResponse Response; ui32 StateStorageGroup = 0; - TActorId CmsPipe; + TActorId CmsPipe; }; } // namespace diff --git a/ydb/core/client/server/msgbus_server_console.cpp b/ydb/core/client/server/msgbus_server_console.cpp index af0064955a6..66bc0b02ebe 100644 --- a/ydb/core/client/server/msgbus_server_console.cpp +++ b/ydb/core/client/server/msgbus_server_console.cpp @@ -347,7 +347,7 @@ private: NKikimrClient::TConsoleRequest Request; NKikimrClient::TConsoleResponse Response; ui32 StateStorageGroup = 0; - TActorId ConsolePipe; + TActorId ConsolePipe; }; } // namespace diff --git a/ydb/core/client/server/msgbus_server_db.cpp b/ydb/core/client/server/msgbus_server_db.cpp index c7eb09681e5..08f3d717864 100644 --- a/ydb/core/client/server/msgbus_server_db.cpp +++ b/ydb/core/client/server/msgbus_server_db.cpp @@ -167,9 +167,9 @@ public: class TActorInterface { private: - TActorId HostActor; + TActorId HostActor; public: - TActorInterface(const TActorId& hostActor) + TActorInterface(const TActorId& hostActor) : HostActor(hostActor) {} @@ -191,8 +191,8 @@ protected: using TBase = TActorBootstrapped<TServerDbOperation<InterfaceBase>>; using TTabletId = ui64; - TActorId TxProxyId; - TActorId SchemeCache; + TActorId TxProxyId; + TActorId SchemeCache; TIntrusivePtr<TMessageBusDbOpsCounters> DbOperationsCounters; NMon::THistogramCounterHelper* OperationHistogram; @@ -296,14 +296,14 @@ protected: public: static constexpr NKikimrServices::TActivity::EType ActorActivityType() { return NKikimrServices::TActivity::MSGBUS_COMMON; } - TServerDbOperation(TBusMessageContext& msg, TActorId txProxyId, const TActorId& schemeCache, const TIntrusivePtr<TMessageBusDbOpsCounters>& dbOperationsCounters); + TServerDbOperation(TBusMessageContext& msg, TActorId txProxyId, const TActorId& schemeCache, const TIntrusivePtr<TMessageBusDbOpsCounters>& dbOperationsCounters); TServerDbOperation( - const TActorId& hostActor, + const TActorId& hostActor, NJson::TJsonValue&& jsonValue, const TString& SecurityToken, - TActorId txProxyId, - const TActorId& schemeCache, + TActorId txProxyId, + const TActorId& schemeCache, const TIntrusivePtr<TMessageBusDbOpsCounters>& dbOperationsCounters ); @@ -344,23 +344,23 @@ public: TBase::Become(&TThis::StateWaitResolve, ctx, Timeout, new TEvents::TEvWakeup()); } - static NMiniKQL::TRuntimeNode NewDataLiteral(NMiniKQL::TKikimrProgramBuilder& pgmBuilder, const NJson::TJsonValue& jsonValue, NScheme::TTypeId typeId) { + static NMiniKQL::TRuntimeNode NewDataLiteral(NMiniKQL::TKikimrProgramBuilder& pgmBuilder, const NJson::TJsonValue& jsonValue, NScheme::TTypeId typeId) { // TODO auto& builder = static_cast<NMiniKQL::TProgramBuilder&>(pgmBuilder); switch (typeId) { - case NScheme::NTypeIds::Bool: + case NScheme::NTypeIds::Bool: return builder.NewDataLiteral<bool>(jsonValue.GetBoolean()); - case NScheme::NTypeIds::Float: + case NScheme::NTypeIds::Float: return builder.NewDataLiteral<float>(jsonValue.GetDouble()); - case NScheme::NTypeIds::Double: + case NScheme::NTypeIds::Double: return builder.NewDataLiteral(jsonValue.GetDouble()); - case NScheme::NTypeIds::Int32: + case NScheme::NTypeIds::Int32: return builder.NewDataLiteral<i32>(jsonValue.GetInteger()); - case NScheme::NTypeIds::Int64: + case NScheme::NTypeIds::Int64: return builder.NewDataLiteral<i64>(jsonValue.GetInteger()); - case NScheme::NTypeIds::Uint32: + case NScheme::NTypeIds::Uint32: return builder.NewDataLiteral<ui32>(jsonValue.GetUInteger()); - case NScheme::NTypeIds::Uint64: + case NScheme::NTypeIds::Uint64: return builder.NewDataLiteral<ui64>(jsonValue.GetUInteger()); case NScheme::NTypeIds::Utf8: return builder.NewDataLiteral<NUdf::EDataSlot::Utf8>(jsonValue.GetString()); @@ -560,7 +560,7 @@ public: template <> TServerDbOperation<TMessageBusInterface<TBusDbOperation>>::TServerDbOperation( - TBusMessageContext& msg, TActorId txProxyId, const TActorId& schemeCache, const TIntrusivePtr<TMessageBusDbOpsCounters>& dbOperationsCounters) + TBusMessageContext& msg, TActorId txProxyId, const TActorId& schemeCache, const TIntrusivePtr<TMessageBusDbOpsCounters>& dbOperationsCounters) : TMessageBusInterface<TBusDbOperation>(msg) , TxProxyId(txProxyId) , SchemeCache(schemeCache) @@ -570,11 +570,11 @@ TServerDbOperation<TMessageBusInterface<TBusDbOperation>>::TServerDbOperation( template <> TServerDbOperation<TActorInterface>::TServerDbOperation( - const TActorId& hostActor, + const TActorId& hostActor, NJson::TJsonValue&& jsonValue, const TString& securityToken, - TActorId txProxyId, - const TActorId& schemeCache, + TActorId txProxyId, + const TActorId& schemeCache, const TIntrusivePtr<TMessageBusDbOpsCounters>& dbOperationsCounters ) : TActorInterface(hostActor) @@ -593,10 +593,10 @@ protected: using TBase = TActorBootstrapped<TServerDbSchema<InterfaceBase>>; using TTabletId = ui64; - TActorId TxProxyId; + TActorId TxProxyId; TIntrusivePtr<TMessageBusDbOpsCounters> DbOperationsCounters; NJson::TJsonValue JSON; - THashMap<TTabletId, TActorId> Pipes; + THashMap<TTabletId, TActorId> Pipes; TDeque<TAutoPtr<TEvTxUserProxy::TEvProposeTransaction>> Requests; THPTimer StartTime; TString SecurityToken; @@ -657,7 +657,7 @@ protected: const TEvTxUserProxy::TEvProposeTransactionStatus::EStatus status = static_cast<TEvTxUserProxy::TEvProposeTransactionStatus::EStatus>(msg->Record.GetStatus()); if (status == TEvTxUserProxy::TEvProposeTransactionStatus::EStatus::ExecInProgress) { TTabletId schemeShardId = msg->Record.GetSchemeShardTabletId(); - TActorId pipe; + TActorId pipe; { auto itPipe = Pipes.find(schemeShardId); if (itPipe == Pipes.end()) { @@ -731,13 +731,13 @@ public: return NKikimrServices::TActivity::MSGBUS_COMMON; } - TServerDbSchema(TBusMessageContext &msg, TActorId txProxyId, const TIntrusivePtr<TMessageBusDbOpsCounters>& dbOperationsCounters); + TServerDbSchema(TBusMessageContext &msg, TActorId txProxyId, const TIntrusivePtr<TMessageBusDbOpsCounters>& dbOperationsCounters); TServerDbSchema( - const TActorId& hostActor, + const TActorId& hostActor, NJson::TJsonValue&& jsonValue, const TString& securityToken, - TActorId txProxyId, + TActorId txProxyId, const TIntrusivePtr<TMessageBusDbOpsCounters>& dbOperationsCounters ); @@ -882,7 +882,7 @@ public: template <> TServerDbSchema<TMessageBusInterface<TBusDbSchema>>::TServerDbSchema( - TBusMessageContext &msg, TActorId txProxyId, const TIntrusivePtr<TMessageBusDbOpsCounters>& dbOperationsCounters) + TBusMessageContext &msg, TActorId txProxyId, const TIntrusivePtr<TMessageBusDbOpsCounters>& dbOperationsCounters) : TMessageBusInterface<TBusDbSchema>(msg) , TxProxyId(txProxyId) , DbOperationsCounters(dbOperationsCounters) @@ -890,10 +890,10 @@ TServerDbSchema<TMessageBusInterface<TBusDbSchema>>::TServerDbSchema( template <> TServerDbSchema<TActorInterface>::TServerDbSchema( - const TActorId& hostActor, + const TActorId& hostActor, NJson::TJsonValue&& jsonValue, const TString& securityToken, - TActorId txProxyId, + TActorId txProxyId, const TIntrusivePtr<TMessageBusDbOpsCounters>& dbOperationsCounters ) : TActorInterface(hostActor) @@ -910,8 +910,8 @@ protected: using TBase = TActorBootstrapped<TServerDbBatch<InterfaceBase>>; using TTabletId = ui64; - TActorId TxProxyId; - TActorId SchemeCache; + TActorId TxProxyId; + TActorId SchemeCache; TIntrusivePtr<TMessageBusDbOpsCounters> DbOperationsCounters; NJson::TJsonValue JSON; TDeque<TAutoPtr<IActor>> Operations; @@ -985,7 +985,7 @@ public: return NKikimrServices::TActivity::MSGBUS_COMMON; } - TServerDbBatch(TBusMessageContext &msg, const TActorId txProxyId, const TActorId& schemeCache, const TIntrusivePtr<TMessageBusDbOpsCounters>& dbOperationsCounters); + TServerDbBatch(TBusMessageContext &msg, const TActorId txProxyId, const TActorId& schemeCache, const TIntrusivePtr<TMessageBusDbOpsCounters>& dbOperationsCounters); void HandleTimeout(const TActorContext& ctx) { return ReplyWithError(MSTATUS_TIMEOUT, "Request timed out", ctx); @@ -1034,7 +1034,7 @@ public: template <> TServerDbBatch<TMessageBusInterface<TBusDbBatch>>::TServerDbBatch( - TBusMessageContext &msg, TActorId txProxyId, const TActorId& schemeCache, const TIntrusivePtr<TMessageBusDbOpsCounters>& dbOperationsCounters) + TBusMessageContext &msg, TActorId txProxyId, const TActorId& schemeCache, const TIntrusivePtr<TMessageBusDbOpsCounters>& dbOperationsCounters) : TMessageBusInterface<TBusDbBatch>(msg) , TxProxyId(txProxyId) , SchemeCache(schemeCache) diff --git a/ydb/core/client/server/msgbus_server_hive_create_tablet.cpp b/ydb/core/client/server/msgbus_server_hive_create_tablet.cpp index fcae9fc3250..0c41e53f090 100644 --- a/ydb/core/client/server/msgbus_server_hive_create_tablet.cpp +++ b/ydb/core/client/server/msgbus_server_hive_create_tablet.cpp @@ -57,7 +57,7 @@ using TBase = TActorBootstrapped<TMessageBusHiveCreateTablet<ResponseType>>; TDuration Timeout; bool WithRetry; - TActorId PipeClient; + TActorId PipeClient; TDeque<TRequest> Requests; NKikimrProto::EReplyStatus Status; @@ -204,7 +204,7 @@ public: void Handle(TEvTabletPipe::TEvClientConnected::TPtr &ev, const TActorContext &ctx) { TEvTabletPipe::TEvClientConnected *msg = ev->Get(); if (msg->Status != NKikimrProto::OK) { - PipeClient = TActorId(); + PipeClient = TActorId(); ErrorReason = Sprintf("Client pipe to Hive connection error, Status# %s, Marker# HC10", NKikimrProto::EReplyStatus_Name(msg->Status).data()); return SendReplyAndDie(CreateErrorReply(MSTATUS_ERROR, ctx), ctx); @@ -213,7 +213,7 @@ public: void Handle(TEvTabletPipe::TEvClientDestroyed::TPtr &ev, const TActorContext &ctx) { Y_UNUSED(ev); - PipeClient = TActorId(); + PipeClient = TActorId(); ErrorReason = Sprintf("Client pipe to Hive destroyed (connection lost), Marker# HC9"); SendReplyMove(CreateErrorReply(MSTATUS_ERROR, ctx)); return Die(ctx); @@ -227,7 +227,7 @@ public: void Die(const TActorContext &ctx) override { if (PipeClient) { NTabletPipe::CloseClient(ctx, PipeClient); - PipeClient = TActorId(); + PipeClient = TActorId(); } TActorBootstrapped<TMessageBusHiveCreateTablet>::Die(ctx); } diff --git a/ydb/core/client/server/msgbus_server_local_enumerate_tablets.cpp b/ydb/core/client/server/msgbus_server_local_enumerate_tablets.cpp index 64f1101e46a..9f867dddc7c 100644 --- a/ydb/core/client/server/msgbus_server_local_enumerate_tablets.cpp +++ b/ydb/core/client/server/msgbus_server_local_enumerate_tablets.cpp @@ -67,12 +67,12 @@ public: } } - TActorId MakeServiceID(const TActorContext &ctx) { + TActorId MakeServiceID(const TActorContext &ctx) { auto &domainsInfo = *AppData(ctx)->DomainsInfo; auto domainIt = domainsInfo.Domains.find(DomainUid); if (domainIt == domainsInfo.Domains.end()) { // Report details in CreateErrorReply - TActorId invalidId; + TActorId invalidId; return invalidId; } ui32 nodeId = IsNodeIdPresent ? NodeId : ctx.SelfID.NodeId(); diff --git a/ydb/core/client/server/msgbus_server_node_registration.cpp b/ydb/core/client/server/msgbus_server_node_registration.cpp index c4cb173e646..a3dda6092d0 100644 --- a/ydb/core/client/server/msgbus_server_node_registration.cpp +++ b/ydb/core/client/server/msgbus_server_node_registration.cpp @@ -97,7 +97,7 @@ public: Response.SetScopePathId(rec.GetScopePathId()); } - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); ctx.Send(nameserviceId, new TEvInterconnect::TEvListNodes()); } @@ -167,7 +167,7 @@ public: private: NKikimrClient::TNodeRegistrationRequest Request; NKikimrClient::TNodeRegistrationResponse Response; - TActorId NodeBrokerPipe; + TActorId NodeBrokerPipe; }; } // namespace diff --git a/ydb/core/client/server/msgbus_server_persqueue.cpp b/ydb/core/client/server/msgbus_server_persqueue.cpp index f141499d59e..78b83ef3a3b 100644 --- a/ydb/core/client/server/msgbus_server_persqueue.cpp +++ b/ydb/core/client/server/msgbus_server_persqueue.cpp @@ -157,7 +157,7 @@ struct TTopicInfo { struct TTabletInfo { ui32 NodeId = 0; TString Topic; - TActorId PipeClient; + TActorId PipeClient; bool BrokenPipe = false; bool IsBalancer = false; TVector<NKikimrPQ::TOffsetsResponse::TPartResult> OffsetResponses; @@ -195,7 +195,7 @@ void TPersQueueBaseRequestProcessor::Bootstrap(const TActorContext& ctx) { ctx.Send(PqMetaCache, new NPqMetaCacheV2::TEvPqNewMetaCache::TEvDescribeAllTopicsRequest(TopicPrefix(ctx))); if (ListNodes) { - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); ctx.Send(nameserviceId, new TEvInterconnect::TEvListNodes()); } @@ -360,7 +360,7 @@ bool TPersQueueBaseRequestProcessor::CreateChildrenIfNeeded(const TActorContext& const auto& name = perTopicInfo->TopicEntry.PQGroupInfo->Description.GetName(); THolder<IActor> childActor = CreateTopicSubactor(perTopicInfo->TopicEntry, name); if (childActor.Get() != nullptr) { - const TActorId actorId = ctx.Register(childActor.Release()); + const TActorId actorId = ctx.Register(childActor.Release()); perTopicInfo->ActorId = actorId; topics.emplace(name); Children.emplace(actorId, std::move(perTopicInfo)); @@ -447,8 +447,8 @@ protected: ui64 CurrentCookie; ui32 FetchRequestBytesLeft; NKikimrClient::TPersQueueFetchResponse FetchResponse; - TVector<TActorId> PQClient; - const TActorId SchemeCache; + TVector<TActorId> PQClient; + const TActorId SchemeCache; TAutoPtr<TEvInterconnect::TEvNodesInfo> NodesInfo; @@ -473,7 +473,7 @@ public: virtual void SendReplyAndDie(NKikimrClient::TResponse&& response, const TActorContext& ctx) = 0; - TMessageBusServerPersQueueImpl(const NKikimrClient::TPersQueueRequest& request, const TActorId& schemeCache) + TMessageBusServerPersQueueImpl(const NKikimrClient::TPersQueueRequest& request, const TActorId& schemeCache) : RequestProto(request) , RequestId(RequestProto.HasRequestId() ? RequestProto.GetRequestId() : "<none>") , IsMetaRequest(RequestProto.HasMetaRequest()) @@ -988,7 +988,7 @@ public: NTabletPipe::TClientConfig clientConfig; PQClient.push_back(ctx.RegisterWithSameMailbox(NTabletPipe::CreateClient(ctx.SelfID, tabletId, clientConfig))); - ActorIdToProto(PQClient.back(), RequestProto.MutablePartitionRequest()->MutablePipeClient()); + ActorIdToProto(PQClient.back(), RequestProto.MutablePartitionRequest()->MutablePipeClient()); TAutoPtr<TEvPersQueue::TEvRequest> req(new TEvPersQueue::TEvRequest); req->Record.Swap(&RequestProto); @@ -1317,7 +1317,7 @@ public: if (RequestProto.HasMetaRequest() && (RequestProto.GetMetaRequest().HasCmdGetPartitionLocations() || RequestProto.GetMetaRequest().HasCmdGetReadSessionsInfo())) { //only for this request NodeId-s and Nodes names are required - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); ctx.Send(nameserviceId, new TEvInterconnect::TEvListNodes()); } @@ -1369,7 +1369,7 @@ private: class TErrorReplier : public TActorBootstrapped<TErrorReplier> { public: - TErrorReplier(const NKikimrClient::TPersQueueRequest& request, const TActorId& /*schemeCache*/) + TErrorReplier(const NKikimrClient::TPersQueueRequest& request, const TActorId& /*schemeCache*/) : RequestId(request.HasRequestId() ? request.GetRequestId() : "<none>") { } diff --git a/ydb/core/client/server/msgbus_server_persqueue.h b/ydb/core/client/server/msgbus_server_persqueue.h index cfade578147..52c67832e4e 100644 --- a/ydb/core/client/server/msgbus_server_persqueue.h +++ b/ydb/core/client/server/msgbus_server_persqueue.h @@ -72,7 +72,7 @@ protected: { } - TActorId ActorId; + TActorId ActorId; TSchemeEntry TopicEntry; NKikimrClient::TResponse Response; bool ActorAnswered = false; @@ -143,7 +143,7 @@ protected: THashSet<TString> TopicsToRequest; // Topics that we need to request. If this set id empty, we are interested in all existing topics. const TActorId PqMetaCache; - THashMap<TActorId, THolder<TPerTopicInfo>> Children; + THashMap<TActorId, THolder<TPerTopicInfo>> Children; size_t ChildrenAnswered = 0; THolder<NSchemeCache::TSchemeCacheNavigate> SchemeCacheResponse; @@ -169,7 +169,7 @@ protected: STFUNC(StateFunc); protected: - TActorId SchemeCache; + TActorId SchemeCache; TSchemeEntry SchemeEntry; TString Name; TProcessingResult ProcessingResult; @@ -185,7 +185,7 @@ template <class TBase> class TReplierToParent : public TBase { public: template <class... T> - explicit TReplierToParent(const TActorId& parent, T&&... t) + explicit TReplierToParent(const TActorId& parent, T&&... t) : TBase(std::forward<T>(t)...) , Parent(parent) { @@ -209,7 +209,7 @@ protected: } protected: - const TActorId Parent; + const TActorId Parent; }; // Pipe client helpers @@ -222,7 +222,7 @@ protected: { } - TActorId CreatePipe(ui64 tabletId, const TActorContext& ctx) { + TActorId CreatePipe(ui64 tabletId, const TActorContext& ctx) { NTabletPipe::TClientConfig clientConfig; const TActorId pipe = ctx.RegisterWithSameMailbox(NTabletPipe::CreateClient(ctx.SelfID, tabletId, clientConfig)); Y_VERIFY(Pipes.emplace(tabletId, pipe).second); @@ -234,8 +234,8 @@ protected: } template <class TEventPtr> - TActorId CreatePipeAndSend(ui64 tabletId, const TActorContext& ctx, TEventPtr ev) { - const TActorId pipe = CreatePipe(tabletId, ctx); + TActorId CreatePipeAndSend(ui64 tabletId, const TActorContext& ctx, TEventPtr ev) { + const TActorId pipe = CreatePipe(tabletId, ctx); NTabletPipe::SendData(ctx, pipe, ev.Release()); return pipe; } @@ -364,7 +364,7 @@ protected: } protected: - THashMap<ui64, TActorId> Pipes; // Tablet id -> pipe + THashMap<ui64, TActorId> Pipes; // Tablet id -> pipe THashMap<ui64, typename TPipeEvent::TPtr> PipeAnswers; // Mapped by tablet id }; diff --git a/ydb/core/client/server/msgbus_server_pq_metacache.h b/ydb/core/client/server/msgbus_server_pq_metacache.h index f01410ef855..6c12b76b041 100644 --- a/ydb/core/client/server/msgbus_server_pq_metacache.h +++ b/ydb/core/client/server/msgbus_server_pq_metacache.h @@ -17,7 +17,7 @@ static const ui32 PQ_METACACHE_REFRESH_INTERVAL_SECONDS = 10; inline TActorId CreatePersQueueMetaCacheV2Id() { - return TActorId(0, "PQMetaCache"); + return TActorId(0, "PQMetaCache"); } namespace NPqMetaCacheV2 { diff --git a/ydb/core/client/server/msgbus_server_pq_metarequest.cpp b/ydb/core/client/server/msgbus_server_pq_metarequest.cpp index e48bb7a40ae..32f37251cc5 100644 --- a/ydb/core/client/server/msgbus_server_pq_metarequest.cpp +++ b/ydb/core/client/server/msgbus_server_pq_metarequest.cpp @@ -28,7 +28,7 @@ void SetErrorCode( // GetTopicMetadata command // -TPersQueueGetTopicMetadataProcessor::TPersQueueGetTopicMetadataProcessor(const NKikimrClient::TPersQueueRequest& request, const TActorId& schemeCache) +TPersQueueGetTopicMetadataProcessor::TPersQueueGetTopicMetadataProcessor(const NKikimrClient::TPersQueueRequest& request, const TActorId& schemeCache) : TPersQueueBaseRequestProcessor(request, schemeCache, false) { const auto& topics = RequestProto->GetMetaRequest().GetCmdGetTopicMetadata().GetTopic(); @@ -198,7 +198,7 @@ void TPersQueueGetPartitionOffsetsTopicWorker::Answer(const TActorContext& ctx, // GetPartitionStatus command // -TPersQueueGetPartitionStatusProcessor::TPersQueueGetPartitionStatusProcessor(const NKikimrClient::TPersQueueRequest& request, const TActorId& schemeCache) +TPersQueueGetPartitionStatusProcessor::TPersQueueGetPartitionStatusProcessor(const NKikimrClient::TPersQueueRequest& request, const TActorId& schemeCache) : TPersQueueBaseRequestProcessor(request, schemeCache, false) { GetTopicsListOrThrow(RequestProto->GetMetaRequest().GetCmdGetPartitionStatus().GetTopicRequest(), PartitionsToRequest); diff --git a/ydb/core/client/server/msgbus_server_pq_metarequest.h b/ydb/core/client/server/msgbus_server_pq_metarequest.h index 85491953fa2..1d56c28dc09 100644 --- a/ydb/core/client/server/msgbus_server_pq_metarequest.h +++ b/ydb/core/client/server/msgbus_server_pq_metarequest.h @@ -10,7 +10,7 @@ namespace NMsgBusProxy { // class TPersQueueGetTopicMetadataProcessor : public TPersQueueBaseRequestProcessor { public: - TPersQueueGetTopicMetadataProcessor(const NKikimrClient::TPersQueueRequest& request, const TActorId& schemeCache); + TPersQueueGetTopicMetadataProcessor(const NKikimrClient::TPersQueueRequest& request, const TActorId& schemeCache); private: THolder<IActor> CreateTopicSubactor(const TSchemeEntry& topicEntry, const TString& name) override; @@ -42,7 +42,7 @@ private: class TPersQueueGetPartitionOffsetsTopicWorker : public TReplierToParent<TPipesWaiterActor<TTopicInfoBasedActor, TEvPersQueue::TEvOffsetsResponse>> { public: - TPersQueueGetPartitionOffsetsTopicWorker(const TActorId& parent, + TPersQueueGetPartitionOffsetsTopicWorker(const TActorId& parent, const TSchemeEntry& topicEntry, const TString& name, const std::shared_ptr<THashSet<ui64>>& partitionsToRequest, const std::shared_ptr<const NKikimrClient::TPersQueueRequest>& requestProto); @@ -63,7 +63,7 @@ private: class TPersQueueGetPartitionStatusProcessor : public TPersQueueBaseRequestProcessor { public: - TPersQueueGetPartitionStatusProcessor(const NKikimrClient::TPersQueueRequest& request, const TActorId& schemeCache); + TPersQueueGetPartitionStatusProcessor(const NKikimrClient::TPersQueueRequest& request, const TActorId& schemeCache); private: THolder<IActor> CreateTopicSubactor(const TSchemeEntry& topicEntry, const TString& name) override; @@ -74,7 +74,7 @@ private: class TPersQueueGetPartitionStatusTopicWorker : public TReplierToParent<TPipesWaiterActor<TTopicInfoBasedActor, TEvPersQueue::TEvStatusResponse>> { public: - TPersQueueGetPartitionStatusTopicWorker(const TActorId& parent, + TPersQueueGetPartitionStatusTopicWorker(const TActorId& parent, const TTopicInfoBasedActor::TSchemeEntry& topicEntry, const TString& name, const std::shared_ptr<THashSet<ui64>>& partitionsToRequest, @@ -96,7 +96,7 @@ private: class TPersQueueGetPartitionLocationsProcessor : public TPersQueueBaseRequestProcessor { public: - TPersQueueGetPartitionLocationsProcessor(const NKikimrClient::TPersQueueRequest& request, const TActorId& schemeCache); + TPersQueueGetPartitionLocationsProcessor(const NKikimrClient::TPersQueueRequest& request, const TActorId& schemeCache); private: THolder<IActor> CreateTopicSubactor(const TSchemeEntry& topicEntry, const TString& name) override; @@ -107,7 +107,7 @@ private: class TPersQueueGetPartitionLocationsTopicWorker : public TReplierToParent<TPipesWaiterActor<TTopicInfoBasedActor, TEvTabletPipe::TEvClientConnected>> { public: - TPersQueueGetPartitionLocationsTopicWorker(const TActorId& parent, + TPersQueueGetPartitionLocationsTopicWorker(const TActorId& parent, const TTopicInfoBasedActor::TSchemeEntry& topicEntry, const TString& name, const std::shared_ptr<THashSet<ui64>>& partitionsToRequest, const std::shared_ptr<const NKikimrClient::TPersQueueRequest>& requestProto, @@ -152,7 +152,7 @@ public: void Handle(TEvPersQueue::TEvReadSessionsInfoResponse::TPtr& ev, const TActorContext&) { for (auto & s : ev->Get()->Record.GetReadSessions()) { if (!s.GetSession().empty()) { - TActorId actor = ActorIdFromProto(s.GetSessionActor()); + TActorId actor = ActorIdFromProto(s.GetSessionActor()); ReadSessions.insert(std::make_pair(s.GetSession(), actor)); } } @@ -170,17 +170,17 @@ public: private: THolder<IActor> CreateTopicSubactor(const TSchemeEntry& topicEntry, const TString& name) override; - THolder<IActor> CreateSessionsSubactor(const THashMap<TString, TActorId>&& readSessions); + THolder<IActor> CreateSessionsSubactor(const THashMap<TString, TActorId>&& readSessions); std::shared_ptr<IPersQueueGetReadSessionsInfoWorkerFactory> PQReadSessionsInfoWorkerFactory; mutable bool HasSessionsRequest = false; - THashMap<TString, TActorId> ReadSessions; + THashMap<TString, TActorId> ReadSessions; }; class TPersQueueGetReadSessionsInfoTopicWorker : public TReplierToParent<TPipesWaiterActor<TTopicInfoBasedActor, TEvPersQueue::TEvOffsetsResponse>> { public: - TPersQueueGetReadSessionsInfoTopicWorker(const TActorId& parent, + TPersQueueGetReadSessionsInfoTopicWorker(const TActorId& parent, const TTopicInfoBasedActor::TSchemeEntry& topicEntry, const TString& name, const std::shared_ptr<const NKikimrClient::TPersQueueRequest>& requestProto, std::shared_ptr<const TPersQueueBaseRequestProcessor::TNodesInfo> nodesInfo); @@ -206,7 +206,7 @@ public: private: std::shared_ptr<const NKikimrClient::TPersQueueRequest> RequestProto; - TActorId BalancerPipe; + TActorId BalancerPipe; TEvPersQueue::TEvReadSessionsInfoResponse::TPtr BalancerResponse; bool BalancerReplied = false; bool PipeEventsAreReady = false; diff --git a/ydb/core/client/server/msgbus_server_pq_metarequest_ut.cpp b/ydb/core/client/server/msgbus_server_pq_metarequest_ut.cpp index e7a8a09a1fc..237bba147b9 100644 --- a/ydb/core/client/server/msgbus_server_pq_metarequest_ut.cpp +++ b/ydb/core/client/server/msgbus_server_pq_metarequest_ut.cpp @@ -45,7 +45,7 @@ protected: Runtime->SetObserverFunc([this](TTestActorRuntimeBase&, TAutoPtr<IEventHandle>& event) { return EventsObserver(event); }); - Runtime->SetRegistrationObserverFunc([this](TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId) { + Runtime->SetRegistrationObserverFunc([this](TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId) { TTestActorRuntime::DefaultRegistrationObserver(runtime, parentId, actorId); return RegistrationObserver(parentId, actorId); }); @@ -72,8 +72,8 @@ protected: MockPQMetaCache = nullptr; Actor = nullptr; TestMainActorHasAnswered = false; - EdgeActorId = TActorId(); - TestMainActorId = TActorId(); + EdgeActorId = TActorId(); + TestMainActorId = TActorId(); EdgeEventHandle.Reset(); LoadedFakeSchemeShard = false; TestActors.clear(); @@ -121,7 +121,7 @@ protected: return request; } - TActorId StartBalancer(ui64 balancerTabletId) { + TActorId StartBalancer(ui64 balancerTabletId) { TActorId id = CreateTestBootstrapper(*Runtime, CreateTestTabletInfo(balancerTabletId, TTabletTypes::PERSQUEUE_READ_BALANCER, TErasureType::ErasureNone), &CreatePersQueueReadBalancer); @@ -132,9 +132,9 @@ protected: return id; } - TActorId PrepareBalancer(const TString& topic, ui64 balancerTabletId, const TVector<std::pair<ui32, ui64>>& partitionsToTablets, const ui64 schemeShardId = 123) { + TActorId PrepareBalancer(const TString& topic, ui64 balancerTabletId, const TVector<std::pair<ui32, ui64>>& partitionsToTablets, const ui64 schemeShardId = 123) { EnsureHasFakeSchemeShard(); - TActorId id = StartBalancer(balancerTabletId); + TActorId id = StartBalancer(balancerTabletId); THolder<TEvPersQueue::TEvUpdateBalancerConfig> request = MakeUpdateBalancerConfigRequest(topic, partitionsToTablets, schemeShardId); @@ -179,7 +179,7 @@ protected: return request; } - TActorId StartPQTablet(ui64 tabletId) { + TActorId StartPQTablet(ui64 tabletId) { TActorId id = CreateTestBootstrapper(*Runtime, CreateTestTabletInfo(tabletId, TTabletTypes::PERSQUEUE, TErasureType::ErasureNone), &CreatePersQueue); @@ -190,9 +190,9 @@ protected: return id; } - TActorId PreparePQTablet(const TString& topic, ui64 tabletId, const TVector<size_t>& partitions) { + TActorId PreparePQTablet(const TString& topic, ui64 tabletId, const TVector<size_t>& partitions) { EnsureHasFakeSchemeShard(); - TActorId id = StartPQTablet(tabletId); + TActorId id = StartPQTablet(tabletId); TAutoPtr<IEventHandle> handle; { @@ -364,7 +364,7 @@ protected: return TTestActorRuntime::EEventAction::PROCESS; } - void RegistrationObserver(const TActorId& parentId, const TActorId& actorId) { + void RegistrationObserver(const TActorId& parentId, const TActorId& actorId) { if (IsIn(TestActors, parentId)) { IActor* child = Runtime->FindActor(actorId); UNIT_ASSERT(child); @@ -375,13 +375,13 @@ protected: // TODO: move this code to test actor runtime void AssertTestActorsDestroyed() { auto events = Runtime->CaptureEvents(); - THashSet<TActorId> destroyedActors; + THashSet<TActorId> destroyedActors; for (const auto& event : events) { if (event->Type == TEvents::TSystem::PoisonPill) { destroyedActors.insert(event->Recipient); } } - for (const TActorId& actorId : TestActors) { + for (const TActorId& actorId : TestActors) { IActor* actor = Runtime->FindActor(actorId); if (actor != nullptr) { const bool isPipe = actor->ActivityType == NKikimrServices::TActivity::TABLET_PIPE_CLIENT; @@ -418,14 +418,14 @@ protected: } protected: - TActorId EdgeActorId; + TActorId EdgeActorId; IActor* Actor = nullptr; - TActorId TestMainActorId; + TActorId TestMainActorId; bool TestMainActorHasAnswered = false; TMockPQMetaCache* MockPQMetaCache = nullptr; TAutoPtr<IEventHandle> EdgeEventHandle; bool LoadedFakeSchemeShard = false; - THashSet<TActorId> TestActors; // Actor and its children + THashSet<TActorId> TestActors; // Actor and its children THashSet<ui64> PausedEventTypes; std::list<TAutoPtr<IEventHandle>> PausedEvents; THolder<TTestActorRuntime> Runtime; diff --git a/ydb/core/client/server/msgbus_server_proxy.h b/ydb/core/client/server/msgbus_server_proxy.h index cc175da2d6f..cb9f9c19d38 100644 --- a/ydb/core/client/server/msgbus_server_proxy.h +++ b/ydb/core/client/server/msgbus_server_proxy.h @@ -38,12 +38,12 @@ class TMessageBusServerProxy : public TActorBootstrapped<TMessageBusServerProxy> TIntrusivePtr<TMessageBusDbOpsCounters> DbOperationsCounters; - TActorId SchemeCache; - TActorId PqMetaCache; + TActorId SchemeCache; + TActorId PqMetaCache; public: - TActorId SelfID; - TActorId TxProxy; + TActorId SelfID; + TActorId TxProxy; private: void Handle(TEvBusProxy::TEvRequest::TPtr &ev, const TActorContext &ctx); diff --git a/ydb/core/client/server/msgbus_server_resolve_node.cpp b/ydb/core/client/server/msgbus_server_resolve_node.cpp index 11ff184ae84..762df383cb8 100644 --- a/ydb/core/client/server/msgbus_server_resolve_node.cpp +++ b/ydb/core/client/server/msgbus_server_resolve_node.cpp @@ -15,8 +15,8 @@ public: , ResolveRequest(static_cast<TBusResolveNode*>(msg.GetMessage())->Record) {} - static TActorId MakeServiceID(const TActorContext&) { - return GetNameserviceActorId(); + static TActorId MakeServiceID(const TActorContext&) { + return GetNameserviceActorId(); } TEvInterconnect::TEvListNodes* MakeReq(const TActorContext&) { diff --git a/ydb/core/client/server/msgbus_server_s3_listing.cpp b/ydb/core/client/server/msgbus_server_s3_listing.cpp index 9b0cae962dc..3626866322b 100644 --- a/ydb/core/client/server/msgbus_server_s3_listing.cpp +++ b/ydb/core/client/server/msgbus_server_s3_listing.cpp @@ -26,10 +26,10 @@ private: const NKikimrClient::TS3ListingRequest* Request; THolder<const NACLib::TUserToken> UserToken; ui32 MaxKeys; - TActorId SchemeCache; + TActorId SchemeCache; TActorId LeaderPipeCache; TDuration Timeout; - TActorId TimeoutTimerActorId; + TActorId TimeoutTimerActorId; TAutoPtr<TKeyDesc> KeyRange; bool WaitingResolveReply; bool Finished; @@ -51,7 +51,7 @@ public: return acitvityType; } - TS3ListingRequestBase(TActorId schemeCache, THolder<const NACLib::TUserToken>&& userToken) + TS3ListingRequestBase(TActorId schemeCache, THolder<const NACLib::TUserToken>&& userToken) : Request(nullptr) , UserToken(std::move(userToken)) , MaxKeys(DEFAULT_MAX_KEYS) @@ -427,7 +427,7 @@ private: void Handle(TEvPipeCache::TEvDeliveryProblem::TPtr &ev, const TActorContext &ctx) { Y_UNUSED(ev); // Invalidate scheme cache in case of partitioning change - ctx.Send(SchemeCache, new TEvTxProxySchemeCache::TEvInvalidateTable(KeyRange->TableId, TActorId())); + ctx.Send(SchemeCache, new TEvTxProxySchemeCache::TEvInvalidateTable(KeyRange->TableId, TActorId())); ReplyWithError(MSTATUS_NOTREADY, NTxProxy::TResultStatus::EStatus::ProxyShardNotAvailable, "Failed to connect to shard", ctx); } @@ -451,7 +451,7 @@ private: if (shardResponse.GetStatus() == NKikimrTxDataShard::TError::WRONG_SHARD_STATE) { // Invalidate scheme cache in case of partitioning change - ctx.Send(SchemeCache, new TEvTxProxySchemeCache::TEvInvalidateTable(KeyRange->TableId, TActorId())); + ctx.Send(SchemeCache, new TEvTxProxySchemeCache::TEvInvalidateTable(KeyRange->TableId, TActorId())); ReplyWithError(MSTATUS_NOTREADY, NTxProxy::TResultStatus::EStatus::ProxyShardNotAvailable, shardResponse.GetErrorDescription(), ctx); return; } @@ -573,7 +573,7 @@ private: TAutoPtr<TBusS3ListingRequest> RequestHolder; public: - TS3ListingRequestMsgbus(NMsgBusProxy::TBusMessageContext& msgCtx, TActorId schemeCache) + TS3ListingRequestMsgbus(NMsgBusProxy::TBusMessageContext& msgCtx, TActorId schemeCache) : TMessageBusSessionIdentHolder(msgCtx) , TS3ListingRequestBase(schemeCache, nullptr) , RequestHolder(static_cast<TBusS3ListingRequest*>(msgCtx.ReleaseMessage())) @@ -593,7 +593,7 @@ protected: }; IActor* CreateMessageBusS3ListingRequest(TBusMessageContext& msg) { - TActorId schemeCache = MakeSchemeCacheID(); + TActorId schemeCache = MakeSchemeCacheID(); return new TS3ListingRequestMsgbus(msg, schemeCache); } @@ -716,7 +716,7 @@ private: NKikimrClient::TS3ListingRequest MsgbusRequest; public: - TS3ListingRequestGrpc(TAutoPtr<TEvS3ListingRequest> request, TActorId schemeCache) + TS3ListingRequestGrpc(TAutoPtr<TEvS3ListingRequest> request, TActorId schemeCache) : TS3ListingRequestBase(schemeCache, THolder<const NACLib::TUserToken>(request->GetInternalToken() ? new NACLib::TUserToken(request->GetInternalToken()) : nullptr)) , GrpcRequest(request) @@ -754,7 +754,7 @@ protected: IActor* CreateGrpcS3ListingRequest(TAutoPtr<TEvS3ListingRequest> request) { - TActorId schemeCache = MakeSchemeCacheID(); + TActorId schemeCache = MakeSchemeCacheID(); return new TS3ListingRequestGrpc(request, schemeCache); } diff --git a/ydb/core/client/server/msgbus_server_scheme_initroot.cpp b/ydb/core/client/server/msgbus_server_scheme_initroot.cpp index 4b27803455b..a8538eea1cb 100644 --- a/ydb/core/client/server/msgbus_server_scheme_initroot.cpp +++ b/ydb/core/client/server/msgbus_server_scheme_initroot.cpp @@ -17,7 +17,7 @@ class TMessageBusSchemeInitRoot : public TMessageBusSecureRequest<TMessageBusSer using TBase = TMessageBusSecureRequest<TMessageBusServerRequestBase<TMessageBusSchemeInitRoot>>; THolder<TBusSchemeInitRoot> Request; const bool WithRetry = true; - TActorId PipeClient; + TActorId PipeClient; void ReplyWithResult(EResponseStatus status, TEvSchemeShard::TEvInitRootShardResult::EStatus ssStatus, const TActorContext &ctx) { TAutoPtr<TBusResponseStatus> response(new TBusResponseStatus(status)); @@ -42,7 +42,7 @@ class TMessageBusSchemeInitRoot : public TMessageBusSecureRequest<TMessageBusSer void Die(const TActorContext &ctx) override { if (PipeClient) { NTabletPipe::CloseClient(ctx, PipeClient); - PipeClient = TActorId(); + PipeClient = TActorId(); } TBase::Die(ctx); } diff --git a/ydb/core/client/server/msgbus_server_tablet_state.cpp b/ydb/core/client/server/msgbus_server_tablet_state.cpp index 17171a68621..c8582fc5501 100644 --- a/ydb/core/client/server/msgbus_server_tablet_state.cpp +++ b/ydb/core/client/server/msgbus_server_tablet_state.cpp @@ -41,7 +41,7 @@ public: {} void Bootstrap(const TActorContext &ctx) { - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); ctx.Schedule(TDuration::Seconds(60), new TEvents::TEvWakeup()); ctx.Send(nameserviceId, new TEvInterconnect::TEvListNodes()); Become(&TThis::StateRequestedBrowse); @@ -66,7 +66,7 @@ public: NodesInfo = ev->Release(); if (!NodesInfo->Nodes.empty()) { for (const auto& ni : NodesInfo->Nodes) { - TActorId tabletStateActorId = NNodeWhiteboard::MakeNodeWhiteboardServiceId(ni.NodeId); + TActorId tabletStateActorId = NNodeWhiteboard::MakeNodeWhiteboardServiceId(ni.NodeId); ctx.Send(tabletStateActorId, new TEvWhiteboard::TEvTabletStateRequest(), IEventHandle::FlagTrackDelivery, ni.NodeId); ++NodesRequested; } diff --git a/ydb/core/client/server/msgbus_server_tracer.cpp b/ydb/core/client/server/msgbus_server_tracer.cpp index 020b6793ed1..ff93f8d1cbe 100644 --- a/ydb/core/client/server/msgbus_server_tracer.cpp +++ b/ydb/core/client/server/msgbus_server_tracer.cpp @@ -166,7 +166,7 @@ public: this->SendReplyAndDie(response.Release(), ctx); } - TActorId MakeServiceID(const TActorContext &ctx) { + TActorId MakeServiceID(const TActorContext &ctx) { Y_UNUSED(ctx); return MakeMessageBusTraceServiceID(); } diff --git a/ydb/core/client/server/msgbus_server_tracer.h b/ydb/core/client/server/msgbus_server_tracer.h index c27c7f41ae3..57a3812f5d5 100644 --- a/ydb/core/client/server/msgbus_server_tracer.h +++ b/ydb/core/client/server/msgbus_server_tracer.h @@ -26,7 +26,7 @@ public: ); IActor* CreateMessageBusTraceService() override; protected: - TActorId MessageBusTracerActorID; + TActorId MessageBusTracerActorID; bool TraceActive; TString TracePath; void OnMessage(NBus::TOnMessageContext &msg) override; @@ -98,9 +98,9 @@ private: TString Path; }; -inline TActorId MakeMessageBusTraceServiceID(ui32 node = 0) { +inline TActorId MakeMessageBusTraceServiceID(ui32 node = 0) { char x[12] = {'m','s','g','b','u','s','t','r','a','c','e','r'}; - return TActorId(node, TStringBuf(x, 12)); + return TActorId(node, TStringBuf(x, 12)); } IActor* CreateMessageBusTracerStartTrace(NMsgBusProxy::TBusMessageContext &msg, const TString &path); diff --git a/ydb/core/client/server/msgbus_server_types.cpp b/ydb/core/client/server/msgbus_server_types.cpp index 4cc8d0e38cd..0495040e3cd 100644 --- a/ydb/core/client/server/msgbus_server_types.cpp +++ b/ydb/core/client/server/msgbus_server_types.cpp @@ -1,5 +1,5 @@ #include "msgbus_server.h" - + #include <ydb/core/base/appdata.h> #include <ydb/core/scheme/scheme_type_registry.h> #include <ydb/library/yql/minikql/mkql_function_registry.h> @@ -7,9 +7,9 @@ #include <ydb/core/client/metadata/types_metadata.h> #include <ydb/core/client/metadata/functions_metadata.h> -#include <util/digest/numeric.h> - - +#include <util/digest/numeric.h> + + namespace NKikimr { namespace NMsgBusProxy { @@ -25,19 +25,19 @@ public: {} void Bootstrap(const TActorContext& ctx) { - const auto* typeRegistry = AppData(ctx)->TypeRegistry; + const auto* typeRegistry = AppData(ctx)->TypeRegistry; const auto& functionRegistry = AppData(ctx)->FunctionRegistry->GetBuiltins(); - ui64 typeEtag = typeRegistry->GetMetadataEtag(); - ui64 functionEtag = functionRegistry->GetMetadataEtag(); + ui64 typeEtag = typeRegistry->GetMetadataEtag(); + ui64 functionEtag = functionRegistry->GetMetadataEtag(); + + ui64 currentEtag = CombineHashes(typeEtag, functionEtag); - ui64 currentEtag = CombineHashes(typeEtag, functionEtag); - auto reply = new TBusTypesResponse(); reply->Record.SetStatus(MSTATUS_OK); if (!Etag.Defined() || *Etag.Get() != currentEtag) { reply->Record.SetETag(currentEtag); - SerializeMetadata(typeRegistry->GetTypeMetadataRegistry(), reply->Record.MutableTypeMetadata()); - SerializeMetadata(*functionRegistry, reply->Record.MutableFunctionMetadata()); + SerializeMetadata(typeRegistry->GetTypeMetadataRegistry(), reply->Record.MutableTypeMetadata()); + SerializeMetadata(*functionRegistry, reply->Record.MutableFunctionMetadata()); } SendReplyMove(reply); diff --git a/ydb/core/client/server/msgbus_servicereq.h b/ydb/core/client/server/msgbus_servicereq.h index 9822b33a588..704a231dd1d 100644 --- a/ydb/core/client/server/msgbus_servicereq.h +++ b/ydb/core/client/server/msgbus_servicereq.h @@ -13,7 +13,7 @@ namespace NMsgBusProxy { template<typename TDerived, NKikimrServices::TActivity::EType Activity> class TMessageBusLocalServiceRequest : public TActorBootstrapped<TDerived>, public TMessageBusSessionIdentHolder { protected: - TActorId ServiceID; + TActorId ServiceID; const TDuration Timeout; void SendReplyAndDie(NBus::TBusMessage *reply, const TActorContext &ctx) { diff --git a/ydb/core/client/server/msgbus_tabletreq.h b/ydb/core/client/server/msgbus_tabletreq.h index 03d9b5b9e9b..f173ae90976 100644 --- a/ydb/core/client/server/msgbus_tabletreq.h +++ b/ydb/core/client/server/msgbus_tabletreq.h @@ -27,13 +27,13 @@ protected: const bool ConnectToFollower; private: - TActorId PipeClient; + TActorId PipeClient; ui64 TabletId = 0; void Handle(TEvTabletPipe::TEvClientConnected::TPtr &ev, const TActorContext &ctx) { TEvTabletPipe::TEvClientConnected *msg = ev->Get(); if (msg->Status != NKikimrProto::OK) { - PipeClient = TActorId(); + PipeClient = TActorId(); return SendReplyAndDie(CreateErrorReply(MSTATUS_ERROR, ctx, Sprintf("Tablet pipe client connected with status# %s for tablet %" PRIu64 " Marker# MBT3", NKikimrProto::EReplyStatus_Name(msg->Status).data(), msg->TabletId)), ctx); @@ -42,7 +42,7 @@ private: void Handle(TEvTabletPipe::TEvClientDestroyed::TPtr &ev, const TActorContext &ctx) { Y_UNUSED(ev); - PipeClient = TActorId(); + PipeClient = TActorId(); SendReplyMove(CreateErrorReply(MSTATUS_ERROR, ctx, "Tablet pipe client destroyed Marker# MBT2")); return Die(ctx); } @@ -54,7 +54,7 @@ protected: void Die(const TActorContext &ctx) override { if (PipeClient) { NTabletPipe::CloseClient(ctx, PipeClient); - PipeClient = TActorId(); + PipeClient = TActorId(); } TActorBootstrapped<TDerived>::Die(ctx); } diff --git a/ydb/core/client/server/ya.make b/ydb/core/client/server/ya.make index 6a58fe4b531..8cbeec21b77 100644 --- a/ydb/core/client/server/ya.make +++ b/ydb/core/client/server/ya.make @@ -69,7 +69,7 @@ PEERDIR( library/cpp/json library/cpp/messagebus library/cpp/messagebus/protobuf - library/cpp/monlib/messagebus + library/cpp/monlib/messagebus library/cpp/protobuf/json library/cpp/protobuf/util library/cpp/threading/future diff --git a/ydb/core/client/ya.make b/ydb/core/client/ya.make index 6bdf5f20bc5..cd39222ad17 100644 --- a/ydb/core/client/ya.make +++ b/ydb/core/client/ya.make @@ -6,7 +6,7 @@ OWNER( ) PEERDIR( - library/cpp/grpc/server + library/cpp/grpc/server ydb/core/base ydb/core/client/scheme_cache_lib ydb/core/client/server diff --git a/ydb/core/cms/cms.cpp b/ydb/core/cms/cms.cpp index 3c132a7cf80..0a205024e3c 100644 --- a/ydb/core/cms/cms.cpp +++ b/ydb/core/cms/cms.cpp @@ -1323,7 +1323,7 @@ void TCms::OnBSCPipeDestroyed(const TActorContext &ctx) if (State->BSControllerPipe) { NTabletPipe::CloseClient(ctx, State->BSControllerPipe); - State->BSControllerPipe = TActorId(); + State->BSControllerPipe = TActorId(); } if (State->Sentinel) @@ -1907,7 +1907,7 @@ void TCms::Handle(TEvTabletPipe::TEvClientConnected::TPtr &ev, OnBSCPipeDestroyed(ctx); } -IActor *CreateCms(const TActorId &tablet, TTabletStorageInfo *info) +IActor *CreateCms(const TActorId &tablet, TTabletStorageInfo *info) { return new TCms(tablet, info); } diff --git a/ydb/core/cms/cms.h b/ydb/core/cms/cms.h index c3df18d2196..25cb8ef024b 100644 --- a/ydb/core/cms/cms.h +++ b/ydb/core/cms/cms.h @@ -287,7 +287,7 @@ struct TEvCms { }; }; -IActor *CreateCms(const TActorId &tablet, TTabletStorageInfo *info); +IActor *CreateCms(const TActorId &tablet, TTabletStorageInfo *info); } // NCms } // NKikimr diff --git a/ydb/core/cms/cms_impl.h b/ydb/core/cms/cms_impl.h index c1830c4e77d..48138884907 100644 --- a/ydb/core/cms/cms_impl.h +++ b/ydb/core/cms/cms_impl.h @@ -427,7 +427,7 @@ private: TSchedulerCookieHolder LogCleanupTimerCookieHolder; public: - TCms(const TActorId &tablet, TTabletStorageInfo *info) + TCms(const TActorId &tablet, TTabletStorageInfo *info) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, new NMiniKQL::TMiniKQLFactory) , State(new TCmsState) diff --git a/ydb/core/cms/cms_state.h b/ydb/core/cms/cms_state.h index dffd3a9b3e2..5f0b6ff2e21 100644 --- a/ydb/core/cms/cms_state.h +++ b/ydb/core/cms/cms_state.h @@ -52,9 +52,9 @@ struct TCmsState : public TAtomicRefCount<TCmsState> { // Static info. ui64 CmsTabletId = 0; - TActorId CmsActorId; - TActorId BSControllerPipe; - TActorId Sentinel; + TActorId CmsActorId; + TActorId BSControllerPipe; + TActorId Sentinel; }; using TCmsStatePtr = TIntrusivePtr<TCmsState>; diff --git a/ydb/core/cms/cms_tx_update_config.cpp b/ydb/core/cms/cms_tx_update_config.cpp index dce6569c2bb..59f962b3e2c 100644 --- a/ydb/core/cms/cms_tx_update_config.cpp +++ b/ydb/core/cms/cms_tx_update_config.cpp @@ -60,7 +60,7 @@ public: } else { if (Self->State->Sentinel) { ctx.Send(Self->State->Sentinel, new TEvents::TEvPoisonPill()); - Self->State->Sentinel = TActorId(); + Self->State->Sentinel = TActorId(); } } } diff --git a/ydb/core/cms/cms_ut_common.h b/ydb/core/cms/cms_ut_common.h index 1fafe90457d..f1f9c1a74db 100644 --- a/ydb/core/cms/cms_ut_common.h +++ b/ydb/core/cms/cms_ut_common.h @@ -73,7 +73,7 @@ public: TCmsTestEnv(ui32 nodeCount, const TNodeTenantsMap &tenants); - TActorId GetSender() { return Sender; } + TActorId GetSender() { return Sender; } NCms::TPDiskID PDiskId(ui32 nodeIndex, ui32 pdiskIndex = 0); TString PDiskName(ui32 nodeIndex, ui32 pdiskIndex = 0); @@ -362,7 +362,7 @@ private: void CheckResetMarker(TAutoPtr<NCms::TEvCms::TEvResetMarkerRequest> req, NKikimrCms::TStatus::ECode code); - TActorId Sender; + TActorId Sender; }; } // NCmsTest diff --git a/ydb/core/cms/console/config_helpers.cpp b/ydb/core/cms/console/config_helpers.cpp index 3ebdff4f809..6d5a81beaf0 100644 --- a/ydb/core/cms/console/config_helpers.cpp +++ b/ydb/core/cms/console/config_helpers.cpp @@ -39,15 +39,15 @@ private: }; ui64 TabletId; - TActorId ServiceId; + TActorId ServiceId; TString Tenant; bool DetectTenant; TString NodeType; TVector<ui32> ConfigItemKinds; ui64 SubscriptionId; EAction Action; - TActorId OwnerId; - TActorId Pipe; + TActorId OwnerId; + TActorId Pipe; ui64 Cookie; public: @@ -58,10 +58,10 @@ public: // Add/replace subscription. TConfigHelper(ui64 tabletId, - TActorId serviceId, + TActorId serviceId, const TVector<ui32> &configItemKinds, const TString &tenant, - TActorId ownerId, + TActorId ownerId, bool replace, ui64 cookie) : TabletId(tabletId) @@ -78,9 +78,9 @@ public: // Add/replace subscription. TConfigHelper(ui64 tabletId, - TActorId serviceId, + TActorId serviceId, const TVector<ui32> &configItemKinds, - TActorId ownerId, + TActorId ownerId, bool replace, ui64 cookie) : TabletId(tabletId) @@ -96,7 +96,7 @@ public: // Remove subscription. TConfigHelper(ui64 subscriptionId, - TActorId ownerId, + TActorId ownerId, ui64 cookie) : TabletId(0) , DetectTenant(false) @@ -109,7 +109,7 @@ public: // Get node config. TConfigHelper(const TVector<ui32> &configItemKinds, - TActorId ownerId, + TActorId ownerId, ui64 cookie) : TabletId(0) , DetectTenant(true) @@ -124,7 +124,7 @@ public: // Get node config. TConfigHelper(const TVector<ui32> &configItemKinds, const TString &tenant, - TActorId ownerId, + TActorId ownerId, ui64 cookie) : TabletId(0) , Tenant(tenant) @@ -151,7 +151,7 @@ public: { if (Pipe) { NTabletPipe::CloseClient(ctx, Pipe); - Pipe = TActorId(); + Pipe = TActorId(); } SendSubscriptionRequest(ctx); @@ -184,7 +184,7 @@ public: if (TabletId) subscription.MutableSubscriber()->SetTabletId(TabletId); else - ActorIdToProto(ServiceId, + ActorIdToProto(ServiceId, subscription.MutableSubscriber()->MutableServiceId()); subscription.MutableOptions()->SetNodeId(SelfId().NodeId()); subscription.MutableOptions()->SetHost(FQDNHostName()); @@ -359,28 +359,28 @@ public: } // anonymous namespace -IActor *CreateNodeConfigCourier(TActorId owner, +IActor *CreateNodeConfigCourier(TActorId owner, ui64 cookie) { return new TConfigHelper(TVector<ui32>(), owner, cookie); } IActor *CreateNodeConfigCourier(ui32 configItemKind, - TActorId owner, + TActorId owner, ui64 cookie) { return new TConfigHelper(TVector<ui32>({configItemKind}), owner, cookie); } IActor *CreateNodeConfigCourier(const TVector<ui32> &configItemKinds, - TActorId owner, + TActorId owner, ui64 cookie) { return new TConfigHelper(configItemKinds, owner, cookie); } IActor *CreateNodeConfigCourier(const TString &tenant, - TActorId owner, + TActorId owner, ui64 cookie) { return new TConfigHelper(TVector<ui32>({}), tenant, owner, cookie); @@ -388,7 +388,7 @@ IActor *CreateNodeConfigCourier(const TString &tenant, IActor *CreateNodeConfigCourier(ui32 configItemKind, const TString &tenant, - TActorId owner, + TActorId owner, ui64 cookie) { return new TConfigHelper(TVector<ui32>({configItemKind}), tenant, owner, cookie); @@ -396,7 +396,7 @@ IActor *CreateNodeConfigCourier(ui32 configItemKind, IActor *CreateNodeConfigCourier(const TVector<ui32> &configItemKinds, const TString &tenant, - TActorId owner, + TActorId owner, ui64 cookie) { return new TConfigHelper(configItemKinds, tenant, owner, cookie); @@ -404,36 +404,36 @@ IActor *CreateNodeConfigCourier(const TVector<ui32> &configItemKinds, IActor *CreateConfigSubscriber(ui64 tabletId, const TVector<ui32> &configItemKinds, - TActorId owner, + TActorId owner, bool replace, ui64 cookie) { - return new TConfigHelper(tabletId, TActorId(), configItemKinds, owner, replace, cookie); + return new TConfigHelper(tabletId, TActorId(), configItemKinds, owner, replace, cookie); } IActor *CreateConfigSubscriber(ui64 tabletId, const TVector<ui32> &configItemKinds, const TString &tenant, - TActorId owner, + TActorId owner, bool replace, ui64 cookie) { - return new TConfigHelper(tabletId, TActorId(), configItemKinds, tenant, owner, replace, cookie); + return new TConfigHelper(tabletId, TActorId(), configItemKinds, tenant, owner, replace, cookie); } -IActor *CreateConfigSubscriber(TActorId serviceId, +IActor *CreateConfigSubscriber(TActorId serviceId, const TVector<ui32> &configItemKinds, - TActorId owner, + TActorId owner, bool replace, ui64 cookie) { return new TConfigHelper(0, serviceId, configItemKinds, owner, replace, cookie); } -IActor *CreateConfigSubscriber(TActorId serviceId, +IActor *CreateConfigSubscriber(TActorId serviceId, const TVector<ui32> &configItemKinds, const TString &tenant, - TActorId owner, + TActorId owner, bool replace, ui64 cookie) { @@ -441,7 +441,7 @@ IActor *CreateConfigSubscriber(TActorId serviceId, } IActor *CreateSubscriptionEraser(ui64 subscriptionId, - TActorId owner, + TActorId owner, ui64 cookie) { return new TConfigHelper(subscriptionId, owner, cookie); diff --git a/ydb/core/cms/console/config_helpers.h b/ydb/core/cms/console/config_helpers.h index 164f18c70c1..b45b4ed5df3 100644 --- a/ydb/core/cms/console/config_helpers.h +++ b/ydb/core/cms/console/config_helpers.h @@ -13,24 +13,24 @@ namespace NConsole { * Courier forwards TEvConsole::TEvGetNodeConfigResponse event to the * owner. */ -IActor *CreateNodeConfigCourier(TActorId owner, +IActor *CreateNodeConfigCourier(TActorId owner, ui64 cookie = 0); IActor *CreateNodeConfigCourier(ui32 configItemKind, - TActorId owner, + TActorId owner, ui64 cookie = 0); IActor *CreateNodeConfigCourier(const TVector<ui32> &configItemKinds, - TActorId owner, + TActorId owner, ui64 cookie = 0); IActor *CreateNodeConfigCourier(const TString &tenant, - TActorId owner, + TActorId owner, ui64 cookie = 0); IActor *CreateNodeConfigCourier(ui32 configItemKind, const TString &tenant, - TActorId owner, + TActorId owner, ui64 cookie = 0); IActor *CreateNodeConfigCourier(const TVector<ui32> &configItemKinds, const TString &tenant, - TActorId owner, + TActorId owner, ui64 cookie = 0); /* @@ -50,24 +50,24 @@ IActor *CreateNodeConfigCourier(const TVector<ui32> &configItemKinds, */ IActor *CreateConfigSubscriber(ui64 tabletId, const TVector<ui32> &configItemKinds, - TActorId owner, + TActorId owner, bool replace = true, ui64 cookie = 0); IActor *CreateConfigSubscriber(ui64 tabletId, const TVector<ui32> &configItemKinds, const TString &tenant, - TActorId owner, + TActorId owner, bool replace = true, ui64 cookie = 0); -IActor *CreateConfigSubscriber(TActorId serviceId, +IActor *CreateConfigSubscriber(TActorId serviceId, const TVector<ui32> &configItemKinds, - TActorId owner, + TActorId owner, bool replace = true, ui64 cookie = 0); -IActor *CreateConfigSubscriber(TActorId serviceId, +IActor *CreateConfigSubscriber(TActorId serviceId, const TVector<ui32> &configItemKinds, const TString &tenant, - TActorId owner, + TActorId owner, bool replace = true, ui64 cookie = 0); @@ -77,7 +77,7 @@ IActor *CreateConfigSubscriber(TActorId serviceId, * forwared to it. */ IActor *CreateSubscriptionEraser(ui64 subscriptionId, - TActorId owner = TActorId(), + TActorId owner = TActorId(), ui64 cookie = 0); } // namespace NConsole diff --git a/ydb/core/cms/console/config_index.cpp b/ydb/core/cms/console/config_index.cpp index decebdce08a..98ede70c6fe 100644 --- a/ydb/core/cms/console/config_index.cpp +++ b/ydb/core/cms/console/config_index.cpp @@ -599,7 +599,7 @@ void TSubscription::Load(const NKikimrConsole::TSubscription &subscription) if (subscription.GetSubscriber().HasTabletId()) Subscriber.TabletId = subscription.GetSubscriber().GetTabletId(); else if (subscription.GetSubscriber().HasServiceId()) - Subscriber.ServiceId = ActorIdFromProto(subscription.GetSubscriber().GetServiceId()); + Subscriber.ServiceId = ActorIdFromProto(subscription.GetSubscriber().GetServiceId()); NodeId = subscription.GetOptions().GetNodeId(); Host = subscription.GetOptions().GetHost(); Tenant = subscription.GetOptions().GetTenant(); diff --git a/ydb/core/cms/console/config_index.h b/ydb/core/cms/console/config_index.h index de41682fedb..b332b90b8ff 100644 --- a/ydb/core/cms/console/config_index.h +++ b/ydb/core/cms/console/config_index.h @@ -60,7 +60,7 @@ struct TSubscriberId { if (proto.HasTabletId()) TabletId = proto.GetTabletId(); else if (proto.HasServiceId()) - ServiceId = ActorIdFromProto(proto.GetServiceId()); + ServiceId = ActorIdFromProto(proto.GetServiceId()); } void Serialize(NKikimrConsole::TSubscriber &proto) const @@ -68,7 +68,7 @@ struct TSubscriberId { if (TabletId) proto.SetTabletId(TabletId); else - ActorIdToProto(ServiceId, proto.MutableServiceId()); + ActorIdToProto(ServiceId, proto.MutableServiceId()); } bool operator==(const TSubscriberId &other) const @@ -78,7 +78,7 @@ struct TSubscriberId { } ui64 TabletId = 0; - TActorId ServiceId; + TActorId ServiceId; }; } // namespace NConsole @@ -590,7 +590,7 @@ struct TSubscription : public TThrRefBase { TConfigId LastProvidedConfig; TConfigId CurrentConfigId; NKikimrConfig::TAppConfig CurrentConfig; - TActorId Worker; + TActorId Worker; // Cookie allows to identify whether TEvConfigNotificationResponse // still holds actual data. If Cookie doesn't match the one carried // in an event then this event is ignored. Cookie is defined when diff --git a/ydb/core/cms/console/configs_dispatcher.cpp b/ydb/core/cms/console/configs_dispatcher.cpp index 9ca886079d5..8e507542392 100644 --- a/ydb/core/cms/console/configs_dispatcher.cpp +++ b/ydb/core/cms/console/configs_dispatcher.cpp @@ -50,7 +50,7 @@ private: // we haven't received subscription confirmation from CMS yet. ui64 SubscriptionId = 0; TDynBitMap Kinds; - THashSet<TActorId> Subscribers; + THashSet<TActorId> Subscribers; // Set to true if there were no config update notifications // processed for this subscription. bool FirstUpdate = true; @@ -59,7 +59,7 @@ private: // Config update which is currently delivered to subscribers. TEvConsole::TEvConfigNotificationRequest::TPtr UpdateInProcess; // Subscribers who didn't respond yet to the latest config update. - THashSet<TActorId> SubscribersToUpdate; + THashSet<TActorId> SubscribersToUpdate; }; /** @@ -71,7 +71,7 @@ private: struct TSubscriber : public TThrRefBase { using TPtr = TIntrusivePtr<TSubscriber>; - TActorId Subscriber; + TActorId Subscriber; THashSet<TSubscription::TPtr> Subscriptions; TConfigId CurrentConfigId; }; @@ -104,7 +104,7 @@ public: TSubscription::TPtr FindSubscription(ui64 id); TSubscription::TPtr FindSubscription(const TDynBitMap &kinds); - TSubscriber::TPtr FindSubscriber(TActorId aid); + TSubscriber::TPtr FindSubscriber(TActorId aid); void SendNotificationResponse(TEvConsole::TEvConfigNotificationRequest::TPtr &ev); @@ -117,7 +117,7 @@ public: * having config update being processed. */ - void SendUpdateToSubscriber(TSubscription::TPtr subscription, TActorId subscriber); + void SendUpdateToSubscriber(TSubscription::TPtr subscription, TActorId subscriber); /** * Remove subscriber and all his subscriptions. */ @@ -136,7 +136,7 @@ public: * create a new one. If existing subscription has some config received * then deliver it to the new subscriber. */ - void AddSubscription(TActorId subscriber, const TDynBitMap &kinds, bool replace); + void AddSubscription(TActorId subscriber, const TDynBitMap &kinds, bool replace); /** * This is called on start and on tenant change to clean up old config @@ -229,7 +229,7 @@ private: THashSet<TSubscription::TPtr> Subscriptions; THashMap<ui64, TSubscription::TPtr> SubscriptionsById; THashMap<TDynBitMap, TSubscription::TPtr> SubscriptionsByKinds; - THashMap<TActorId, TSubscriber::TPtr> Subscribers; + THashMap<TActorId, TSubscriber::TPtr> Subscribers; // Messages that had an unknown subscription id at the time they are received THashMap<ui64, TEvConsole::TEvConfigNotificationRequest::TPtr> OutOfOrderConfigNotifications; @@ -349,7 +349,7 @@ TConfigsDispatcher::TSubscription::TPtr TConfigsDispatcher::FindSubscription(con return it->second; } -TConfigsDispatcher::TSubscriber::TPtr TConfigsDispatcher::FindSubscriber(TActorId aid) +TConfigsDispatcher::TSubscriber::TPtr TConfigsDispatcher::FindSubscriber(TActorId aid) { auto it = Subscribers.find(aid); if (it == Subscribers.end()) @@ -408,7 +408,7 @@ void TConfigsDispatcher::CreateSubscriberActor(const TDynBitMap &kinds, bool rep ++NextRequestCookie; } -void TConfigsDispatcher::SendUpdateToSubscriber(TSubscription::TPtr subscription, TActorId subscriber) +void TConfigsDispatcher::SendUpdateToSubscriber(TSubscription::TPtr subscription, TActorId subscriber) { Y_VERIFY(subscription->SubscriptionId); Y_VERIFY(subscription->UpdateInProcess); @@ -459,7 +459,7 @@ void TConfigsDispatcher::RemoveSubscription(TSubscription::TPtr subscription) Register(CreateSubscriptionEraser(subscription->SubscriptionId)); } -void TConfigsDispatcher::AddSubscription(TActorId subscriber, +void TConfigsDispatcher::AddSubscription(TActorId subscriber, const TDynBitMap &kinds, bool replace) { diff --git a/ydb/core/cms/console/configs_dispatcher.h b/ydb/core/cms/console/configs_dispatcher.h index a054e6279a7..160d1ebac9c 100644 --- a/ydb/core/cms/console/configs_dispatcher.h +++ b/ydb/core/cms/console/configs_dispatcher.h @@ -90,9 +90,9 @@ struct TEvConfigsDispatcher { */ IActor *CreateConfigsDispatcher(const NKikimrConfig::TAppConfig &config); -inline TActorId MakeConfigsDispatcherID(ui32 node = 0) { +inline TActorId MakeConfigsDispatcherID(ui32 node = 0) { char x[12] = { 'c', 'o', 'n', 'f', 'i', 'g', 's', 'd', 'i', 's', 'p' }; - return TActorId(node, TStringBuf(x, 12)); + return TActorId(node, TStringBuf(x, 12)); } } // namespace NConsole diff --git a/ydb/core/cms/console/configs_dispatcher_ut.cpp b/ydb/core/cms/console/configs_dispatcher_ut.cpp index 8cd09789f1f..997d1ba4514 100644 --- a/ydb/core/cms/console/configs_dispatcher_ut.cpp +++ b/ydb/core/cms/console/configs_dispatcher_ut.cpp @@ -56,7 +56,7 @@ TTenantTestConfig DefaultConsoleTestConfig() NKikimrConsole::TConfigItem ITEM_DOMAIN_LOG_1; NKikimrConsole::TConfigItem ITEM_DOMAIN_LOG_2; -TActorId InitConfigsDispatcher(TTenantTestRuntime &runtime) +TActorId InitConfigsDispatcher(TTenantTestRuntime &runtime) { ITEM_DOMAIN_LOG_1 = MakeConfigItem(NKikimrConsole::TConfigItem::LogConfigItem, @@ -153,7 +153,7 @@ struct TEvPrivate { class TTestSubscriber : public TActorBootstrapped<TTestSubscriber> { public: - TTestSubscriber(TActorId sink, TVector<ui32> kinds, bool hold) + TTestSubscriber(TActorId sink, TVector<ui32> kinds, bool hold) : Sink(sink) , Kinds(std::move(kinds)) , HoldResponse(hold) @@ -244,13 +244,13 @@ public: } private: - TActorId Sink; + TActorId Sink; TVector<ui32> Kinds; bool HoldResponse; TDeque<TAutoPtr<IEventHandle>> EventsQueue; }; -TActorId AddSubscriber(TTenantTestRuntime &runtime, TVector<ui32> kinds, bool hold = false) +TActorId AddSubscriber(TTenantTestRuntime &runtime, TVector<ui32> kinds, bool hold = false) { auto aid = runtime.Register(new TTestSubscriber(runtime.Sender, std::move(kinds), hold)); runtime.EnableScheduleForActor(aid, true); @@ -266,19 +266,19 @@ NKikimrConfig::TAppConfig GetConfig(TTenantTestRuntime &runtime, TVector<ui32> k return *runtime.GrabEdgeEventRethrow<TEvConfigsDispatcher::TEvGetConfigResponse>(handle)->Config; } -void HoldSubscriber(TTenantTestRuntime &runtime, TActorId aid) +void HoldSubscriber(TTenantTestRuntime &runtime, TActorId aid) { TAutoPtr<IEventHandle> handle; runtime.Send(new IEventHandle(aid, runtime.Sender, new TEvPrivate::TEvHoldResponse(true))); runtime.GrabEdgeEventRethrow<TEvPrivate::TEvComplete>(handle); } -void UnholdSubscriber(TTenantTestRuntime &runtime, TActorId aid) +void UnholdSubscriber(TTenantTestRuntime &runtime, TActorId aid) { runtime.Send(new IEventHandle(aid, runtime.Sender, new TEvPrivate::TEvHoldResponse(false))); } -void SetSubscriptions(TTenantTestRuntime &runtime, TActorId aid, TVector<ui32> kinds) +void SetSubscriptions(TTenantTestRuntime &runtime, TActorId aid, TVector<ui32> kinds) { runtime.Send(new IEventHandle(aid, runtime.Sender, new TEvPrivate::TEvSetSubscription(kinds))); } @@ -517,7 +517,7 @@ Y_UNIT_TEST_SUITE(TConfigsDispatcherTests) { InitConfigsDispatcher(runtime); ui64 notifications = 0; - TActorId subscriber; + TActorId subscriber; auto observer = [¬ifications,&subscriber,recipient=runtime.Sender](TTestActorRuntimeBase&, TAutoPtr<IEventHandle> &ev) -> TTenantTestRuntime::EEventAction { if (ev->Recipient == recipient && ev->Sender == subscriber) { switch (ev->GetTypeRewrite()) { diff --git a/ydb/core/cms/console/console.cpp b/ydb/core/cms/console/console.cpp index 57646e476a6..fe477202af5 100644 --- a/ydb/core/cms/console/console.cpp +++ b/ydb/core/cms/console/console.cpp @@ -8,7 +8,7 @@ #include <ydb/core/base/counters.h> #include <ydb/core/cms/console/validators/registry.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> namespace NKikimr { namespace NConsole { @@ -182,7 +182,7 @@ void TConsole::Handle(TEvents::TEvPoisonPill::TPtr &ev, ctx.Send(Tablet(), new TEvents::TEvPoisonPill); } -IActor *CreateConsole(const TActorId &tablet, TTabletStorageInfo *info) +IActor *CreateConsole(const TActorId &tablet, TTabletStorageInfo *info) { return new TConsole(tablet, info); } diff --git a/ydb/core/cms/console/console.h b/ydb/core/cms/console/console.h index 78e03eb5456..934d4e97e45 100644 --- a/ydb/core/cms/console/console.h +++ b/ydb/core/cms/console/console.h @@ -269,7 +269,7 @@ struct TEvConsole { struct TEvToggleConfigValidatorResponse : public TEventShortDebugPB<TEvToggleConfigValidatorResponse, NKikimrConsole::TToggleConfigValidatorResponse, EvToggleConfigValidatorResponse> {}; }; -IActor *CreateConsole(const TActorId &tablet, TTabletStorageInfo *info); +IActor *CreateConsole(const TActorId &tablet, TTabletStorageInfo *info); } // namespace NConsole } // namespace NKikimr diff --git a/ydb/core/cms/console/console__remove_tenant.cpp b/ydb/core/cms/console/console__remove_tenant.cpp index 01da02059bb..14bf4e19f36 100644 --- a/ydb/core/cms/console/console__remove_tenant.cpp +++ b/ydb/core/cms/console/console__remove_tenant.cpp @@ -104,7 +104,7 @@ public: Self->ChangeTenantState(Tenant, TTenant::REMOVING_SUBDOMAIN, ctx); Tenant->UserToken = Request->Get()->Record.GetUserToken(); - Tenant->Worker = TActorId(); + Tenant->Worker = TActorId(); Self->ProcessTenantActions(Tenant, ctx); } diff --git a/ydb/core/cms/console/console__update_confirmed_subdomain.cpp b/ydb/core/cms/console/console__update_confirmed_subdomain.cpp index 21f487c400d..bf7e4a49fc9 100644 --- a/ydb/core/cms/console/console__update_confirmed_subdomain.cpp +++ b/ydb/core/cms/console/console__update_confirmed_subdomain.cpp @@ -8,7 +8,7 @@ public: TTxUpdateConfirmedSubdomain(TTenantsManager *self, const TString &path, ui64 version, - TActorId worker) + TActorId worker) : TBase(self) , Path(path) , Version(version) @@ -44,7 +44,7 @@ public: if (Tenant) { if (Tenant->Worker == Worker) - Tenant->Worker = TActorId(); + Tenant->Worker = TActorId(); Tenant->ConfirmedSubdomain = Version; if (Tenant->State == TTenant::CONFIGURING_SUBDOMAIN) { Tenant->State = TTenant::RUNNING; @@ -61,12 +61,12 @@ private: TTenant::TPtr Tenant; TString Path; ui64 Version; - TActorId Worker; + TActorId Worker; }; ITransaction *TTenantsManager::CreateTxUpdateConfirmedSubdomain(const TString &path, ui64 version, - TActorId worker) + TActorId worker) { return new TTxUpdateConfirmedSubdomain(this, path, version, worker); } diff --git a/ydb/core/cms/console/console__update_pool_state.cpp b/ydb/core/cms/console/console__update_pool_state.cpp index 8305ede69ec..1f12ff0e0d7 100644 --- a/ydb/core/cms/console/console__update_pool_state.cpp +++ b/ydb/core/cms/console/console__update_pool_state.cpp @@ -8,7 +8,7 @@ public: TTxUpdatePoolState(TTenantsManager *self, TTenant::TPtr tenant, TStoragePool::TPtr pool, - TActorId worker, + TActorId worker, TStoragePool::EState state) : TBase(self) , Tenant(tenant) @@ -77,7 +77,7 @@ public: Self->Counters.Inc(Pool->Kind, COUNTER_ALLOCATED_STORAGE_UNITS, AllocatedNumGroups - Pool->AllocatedNumGroups); - Pool->Worker = TActorId(); + Pool->Worker = TActorId(); Pool->State = State; Pool->AllocatedNumGroups = AllocatedNumGroups; Tenant->SubdomainVersion = SubdomainVersion; @@ -96,7 +96,7 @@ public: private: TTenant::TPtr Tenant; TStoragePool::TPtr Pool; - TActorId Worker; + TActorId Worker; TStoragePool::EState State; ui64 SubdomainVersion; ui64 AllocatedNumGroups; @@ -105,7 +105,7 @@ private: ITransaction *TTenantsManager::CreateTxUpdatePoolState(TTenant::TPtr tenant, TStoragePool::TPtr pool, - TActorId worker, + TActorId worker, TStoragePool::EState state) { return new TTxUpdatePoolState(this, tenant, pool, worker, state); diff --git a/ydb/core/cms/console/console__update_subdomain_key.cpp b/ydb/core/cms/console/console__update_subdomain_key.cpp index 1b8f864cca2..6a9fcba8cb7 100644 --- a/ydb/core/cms/console/console__update_subdomain_key.cpp +++ b/ydb/core/cms/console/console__update_subdomain_key.cpp @@ -9,7 +9,7 @@ public: const TString &path, ui64 schemeShardId, ui64 pathId, - TActorId worker) + TActorId worker) : TBase(self) , Path(path) , SchemeShardId(schemeShardId) @@ -76,7 +76,7 @@ public: Self->TenantIdToName[Tenant->DomainId] = Tenant->Path; if (Tenant->Worker == Worker) - Tenant->Worker = TActorId(); + Tenant->Worker = TActorId(); for (auto &pr : Tenant->StoragePools) { pr.second->SetScopeId(SchemeShardId, PathId); @@ -98,13 +98,13 @@ private: TString Path; ui64 SchemeShardId; ui64 PathId; - TActorId Worker; + TActorId Worker; }; ITransaction *TTenantsManager::CreateTxUpdateSubDomainKey(const TString &path, ui64 schemeShardId, ui64 pathId, - TActorId worker) + TActorId worker) { return new TTxUpdateSubDomainKey(this, path, schemeShardId, pathId, worker); } diff --git a/ydb/core/cms/console/console__update_tenant_state.cpp b/ydb/core/cms/console/console__update_tenant_state.cpp index 517a7dd8e1a..ea916bcfd6d 100644 --- a/ydb/core/cms/console/console__update_tenant_state.cpp +++ b/ydb/core/cms/console/console__update_tenant_state.cpp @@ -8,7 +8,7 @@ public: TTxUpdateTenantState(TTenantsManager *self, const TString &path, TTenant::EState state, - TActorId worker) + TActorId worker) : TBase(self) , Path(path) , State(state) @@ -47,7 +47,7 @@ public: if (Tenant && PrevState != State) { if (Tenant->Worker == Worker) - Tenant->Worker = TActorId(); + Tenant->Worker = TActorId(); Self->ChangeTenantState(Tenant, State, ctx); Self->ProcessTenantActions(Tenant, ctx); } @@ -60,12 +60,12 @@ private: TString Path; TTenant::EState State; TTenant::EState PrevState; - TActorId Worker; + TActorId Worker; }; ITransaction *TTenantsManager::CreateTxUpdateTenantState(const TString &path, TTenant::EState state, - TActorId worker) + TActorId worker) { return new TTxUpdateTenantState(this, path, state, worker); } diff --git a/ydb/core/cms/console/console_configs_manager.cpp b/ydb/core/cms/console/console_configs_manager.cpp index ac1714dd3a4..a0415de947b 100644 --- a/ydb/core/cms/console/console_configs_manager.cpp +++ b/ydb/core/cms/console/console_configs_manager.cpp @@ -366,7 +366,7 @@ bool TConfigsManager::DbLoadState(TTransactionContext &txc, while (!subscriptionRowset.EndOfSet()) { ui64 id = subscriptionRowset.GetValue<Schema::ConfigSubscriptions::Id>(); ui64 tabletId = subscriptionRowset.GetValue<Schema::ConfigSubscriptions::TabletId>(); - TActorId serviceId = subscriptionRowset.GetValue<Schema::ConfigSubscriptions::ServiceId>(); + TActorId serviceId = subscriptionRowset.GetValue<Schema::ConfigSubscriptions::ServiceId>(); ui32 nodeId = subscriptionRowset.GetValue<Schema::ConfigSubscriptions::NodeId>(); TString host = subscriptionRowset.GetValue<Schema::ConfigSubscriptions::Host>(); TString tenant = subscriptionRowset.GetValue<Schema::ConfigSubscriptions::Tenant>(); @@ -580,12 +580,12 @@ void TConfigsManager::Handle(TEvPrivate::TEvStateLoaded::TPtr &/*ev*/, const TAc { ctx.Send(ConfigsProvider, new TConfigsProvider::TEvPrivate::TEvSetConfigs(ConfigIndex.GetConfigItems())); ctx.Send(ConfigsProvider, new TConfigsProvider::TEvPrivate::TEvSetSubscriptions(SubscriptionIndex.GetSubscriptions())); - ctx.Send(GetNameserviceActorId(), new TEvInterconnect::TEvListNodes()); + ctx.Send(GetNameserviceActorId(), new TEvInterconnect::TEvListNodes()); } void TConfigsManager::Handle(TEvPrivate::TEvCleanupSubscriptions::TPtr &/*ev*/, const TActorContext &ctx) { - ctx.Send(GetNameserviceActorId(), new TEvInterconnect::TEvListNodes()); + ctx.Send(GetNameserviceActorId(), new TEvInterconnect::TEvListNodes()); } void TConfigsManager::ForwardToConfigsProvider(TAutoPtr<IEventHandle> &ev, const TActorContext &ctx) diff --git a/ydb/core/cms/console/console_configs_manager.h b/ydb/core/cms/console/console_configs_manager.h index cb33c10ca54..2dca13be264 100644 --- a/ydb/core/cms/console/console_configs_manager.h +++ b/ydb/core/cms/console/console_configs_manager.h @@ -194,7 +194,7 @@ private: TSubscriptionModifications PendingSubscriptionModifications; TSchedulerCookieHolder SubscriptionsCleanupTimerCookieHolder; - TActorId ConfigsProvider; + TActorId ConfigsProvider; TTxProcessor::TPtr TxProcessor; }; diff --git a/ydb/core/cms/console/console_configs_provider.cpp b/ydb/core/cms/console/console_configs_provider.cpp index 087a9d46f59..4ba6f460338 100644 --- a/ydb/core/cms/console/console_configs_provider.cpp +++ b/ydb/core/cms/console/console_configs_provider.cpp @@ -17,8 +17,8 @@ private: using TBase = TActorBootstrapped<TTabletConfigSender>; TSubscription::TPtr Subscription; - TActorId OwnerId; - TActorId Pipe; + TActorId OwnerId; + TActorId Pipe; TSchedulerCookieHolder TimeoutTimerCookieHolder; public: @@ -27,7 +27,7 @@ public: return NKikimrServices::TActivity::CMS_CONFIGS_PROVIDER; } - TTabletConfigSender(TSubscription::TPtr subscription, TActorId ownerId) + TTabletConfigSender(TSubscription::TPtr subscription, TActorId ownerId) : Subscription(subscription) , OwnerId(ownerId) { @@ -47,7 +47,7 @@ public: { if (Pipe) { NTabletPipe::CloseClient(ctx, Pipe); - Pipe = TActorId(); + Pipe = TActorId(); } SendNotifyRequest(ctx); @@ -153,7 +153,7 @@ private: using TBase = TActorBootstrapped<TServiceConfigSender>; TSubscription::TPtr Subscription; - TActorId OwnerId; + TActorId OwnerId; TDuration RetryInterval; TSchedulerCookieHolder TimeoutTimerCookieHolder; bool ScheduledRetry; @@ -164,7 +164,7 @@ public: return NKikimrServices::TActivity::CMS_CONFIGS_PROVIDER; } - TServiceConfigSender(TSubscription::TPtr subscription, TActorId ownerId) + TServiceConfigSender(TSubscription::TPtr subscription, TActorId ownerId) : Subscription(subscription) , OwnerId(ownerId) , RetryInterval(TDuration::Seconds(5)) @@ -457,7 +457,7 @@ void TConfigsProvider::ApplySubscriptionModifications(const TSubscriptionModific "TConfigsProvider: remove subscription " << subscription->ToString()); if (subscription->Worker) { ctx.Send(subscription->Worker, new TEvents::TEvPoisonPill); - subscription->Worker = TActorId(); + subscription->Worker = TActorId(); } SubscriptionIndex.RemoveSubscription(id); } @@ -485,7 +485,7 @@ void TConfigsProvider::ApplySubscriptionModifications(const TSubscriptionModific subscription->Cookie = pr.second; if (subscription->Worker) { ctx.Send(subscription->Worker, new TEvents::TEvPoisonPill); - subscription->Worker = TActorId(); + subscription->Worker = TActorId(); } subscriptions.insert(subscription); } @@ -883,7 +883,7 @@ void TConfigsProvider::Handle(TEvConsole::TEvConfigNotificationResponse::TPtr &e // provided config anyway because cookie update always come with // last provided update. subscription->LastProvidedConfig.Load(rec.GetConfigId()); - subscription->Worker = TActorId(); + subscription->Worker = TActorId(); ctx.Send(ev->Forward(ConfigsManager)); } @@ -1012,7 +1012,7 @@ void TConfigsProvider::Handle(TEvPrivate::TEvNotificationTimeout::TPtr &ev, cons << " subscription id=" << subscription->Id); return; } - subscription->Worker = TActorId(); + subscription->Worker = TActorId(); CheckSubscription(subscription, ctx); } @@ -1033,7 +1033,7 @@ void TConfigsProvider::Handle(TEvPrivate::TEvSenderDied::TPtr &ev, const TActorC << " subscription id=" << subscription->Id); return; } - subscription->Worker = TActorId(); + subscription->Worker = TActorId(); CheckSubscription(subscription, ctx); } diff --git a/ydb/core/cms/console/console_configs_provider.h b/ydb/core/cms/console/console_configs_provider.h index 25dfd216890..ddb74d52d48 100644 --- a/ydb/core/cms/console/console_configs_provider.h +++ b/ydb/core/cms/console/console_configs_provider.h @@ -186,7 +186,7 @@ private: } public: - TConfigsProvider(TActorId ownerId) + TConfigsProvider(TActorId ownerId) : ConfigsManager(ownerId) { } @@ -205,7 +205,7 @@ public: void Die(const TActorContext &ctx) override; private: - TActorId ConfigsManager; + TActorId ConfigsManager; TConfigsConfig Config; TConfigIndex ConfigIndex; TSubscriptionIndex SubscriptionIndex; diff --git a/ydb/core/cms/console/console_impl.h b/ydb/core/cms/console/console_impl.h index 92123b1c50d..0894337c3ec 100644 --- a/ydb/core/cms/console/console_impl.h +++ b/ydb/core/cms/console/console_impl.h @@ -123,7 +123,7 @@ private: } public: - TConsole(const TActorId &tablet, TTabletStorageInfo *info) + TConsole(const TActorId &tablet, TTabletStorageInfo *info) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, new NMiniKQL::TMiniKQLFactory) , TxProcessor(new TTxProcessor(*this, "console", NKikimrServices::CMS)) @@ -165,7 +165,7 @@ private: TConfigsManager* ConfigsManager; TTenantsManager* TenantsManager; - TActorId NetClassifierUpdaterId; + TActorId NetClassifierUpdaterId; }; } // namespace NConsole diff --git a/ydb/core/cms/console/console_tenants_manager.cpp b/ydb/core/cms/console/console_tenants_manager.cpp index 733b2d59854..0db3f5dceda 100644 --- a/ydb/core/cms/console/console_tenants_manager.cpp +++ b/ydb/core/cms/console/console_tenants_manager.cpp @@ -38,11 +38,11 @@ public: private: using TBase = TActorBootstrapped<TPoolManip>; - TActorId OwnerId; + TActorId OwnerId; TDomainsInfo::TDomain::TPtr Domain; TTenantsManager::TTenant::TPtr Tenant; TTenantsManager::TStoragePool::TPtr Pool; - TActorId BSControllerPipe; + TActorId BSControllerPipe; bool PoolStateAcquired; EAction Action; ui64 PoolId; @@ -54,7 +54,7 @@ public: return NKikimrServices::TActivity::CMS_TENANTS_MANAGER; } - TPoolManip(TActorId ownerId, TDomainsInfo::TDomain::TPtr domain, + TPoolManip(TActorId ownerId, TDomainsInfo::TDomain::TPtr domain, TTenantsManager::TTenant::TPtr tenant, TTenantsManager::TStoragePool::TPtr pool, EAction action) : OwnerId(ownerId) @@ -83,7 +83,7 @@ public: if (BSControllerPipe) { NTabletPipe::CloseClient(ctx, BSControllerPipe); - BSControllerPipe = TActorId(); + BSControllerPipe = TActorId(); } DoWork(ctx); @@ -383,7 +383,7 @@ public: private: using TBase = TActorBootstrapped<TSubDomainManip>; - TActorId OwnerId; + TActorId OwnerId; TTenantsManager::TTenant::TPtr Tenant; TTenantsManager::TTenant::TPtr SharedTenant; EAction Action; @@ -392,7 +392,7 @@ private: ui64 TxId; ui64 TabletId; ui64 Version; - TActorId Pipe; + TActorId Pipe; // For CREATE/GET_KEY action SchemeshardId and PathId will hold subdomain key. ui64 SchemeshardId; ui64 PathId; @@ -405,7 +405,7 @@ public: return NKikimrServices::TActivity::CMS_TENANTS_MANAGER; } - TSubDomainManip(TActorId ownerId, TTenantsManager::TTenant::TPtr tenant, EAction action, + TSubDomainManip(TActorId ownerId, TTenantsManager::TTenant::TPtr tenant, EAction action, TTenantsManager::TTenant::TPtr sharedTenant = nullptr) : OwnerId(ownerId) , Tenant(tenant) @@ -671,7 +671,7 @@ public: { if (Pipe) { NTabletPipe::CloseClient(ctx, Pipe); - Pipe = TActorId(); + Pipe = TActorId(); } SendNotifyRequest(ctx); @@ -1739,7 +1739,7 @@ void TTenantsManager::OnTenantSlotBrokerPipeDestroyed(const TActorContext &ctx) { if (TenantSlotBrokerPipe) { NTabletPipe::CloseClient(ctx, TenantSlotBrokerPipe); - TenantSlotBrokerPipe = TActorId(); + TenantSlotBrokerPipe = TActorId(); } RetryResourcesRequests(ctx); @@ -3122,13 +3122,13 @@ void TTenantsManager::Handle(TEvPrivate::TEvPoolFailed::TPtr &ev, const TActorCo << " for tenant " << tenant->Path << ": " << issue); Counters.Inc(COUNTER_REMOVE_POOL_FAILED); - pool->Worker = TActorId(); + pool->Worker = TActorId(); pool->GroupFitErrors = 0; ctx.Schedule(TDuration::Seconds(10), new TEvPrivate::TEvRetryAllocateResources(tenant->Path)); } else if (tenant->IsCreating()) { Counters.Inc(COUNTER_ALLOC_POOL_FAILED); - pool->Worker = TActorId(); + pool->Worker = TActorId(); pool->GroupFitErrors = 0; tenant->Issue = issue; TxProcessor->ProcessTx(CreateTxUpdateTenantState(tenant->Path, TTenant::REMOVING_POOLS), ctx); @@ -3148,7 +3148,7 @@ void TTenantsManager::Handle(TEvPrivate::TEvPoolFailed::TPtr &ev, const TActorCo } Counters.Inc(COUNTER_ALLOC_POOL_FAILED); - pool->Worker = TActorId(); + pool->Worker = TActorId(); ctx.Schedule(TDuration::Seconds(10), new TEvPrivate::TEvRetryAllocateResources(tenant->Path)); } @@ -3229,7 +3229,7 @@ void TTenantsManager::Handle(TEvPrivate::TEvSubdomainFailed::TPtr &ev, const TAc << ": " << code << ": " << issue); tenant->Issue = issue; - tenant->Worker = TActorId(); + tenant->Worker = TActorId(); ctx.Schedule(TDuration::Seconds(10), new TEvPrivate::TEvRetryAllocateResources(tenant->Path)); } diff --git a/ydb/core/cms/console/console_tenants_manager.h b/ydb/core/cms/console/console_tenants_manager.h index f5eb777e160..a21615dd622 100644 --- a/ydb/core/cms/console/console_tenants_manager.h +++ b/ydb/core/cms/console/console_tenants_manager.h @@ -338,7 +338,7 @@ public: ui64 AllocatedNumGroups; EState State; TString Issue; - TActorId Worker; + TActorId Worker; size_t GroupFitErrors = 0; }; @@ -485,7 +485,7 @@ public: ui64 Mediators; ui64 PlanResolution; ui32 TimeCastBucketsPerMediator; - TActorId Worker; + TActorId Worker; // <type> -> pool THashMap<TString, TStoragePool::TPtr> StoragePools; // <kind, az> -> count @@ -498,7 +498,7 @@ public: TVector<TAutoPtr<IEventHandle>> StatusRequests; ui64 StorageUnitsQuota; ui64 ComputationalUnitsQuota; - TVector<TActorId> Subscribers; + TVector<TActorId> Subscribers; // Error code and issue are used to report the last // problem with tenant. Ydb::StatusIds::StatusCode ErrorCode; @@ -714,17 +714,17 @@ public: Ydb::StatusIds::StatusCode code); ITransaction *CreateTxUpdateConfirmedSubdomain(const TString &name, ui64 version, - TActorId worker); + TActorId worker); ITransaction *CreateTxUpdateTenantState(const TString &name, TTenant::EState state, - TActorId worker = TActorId()); + TActorId worker = TActorId()); ITransaction *CreateTxUpdateSubDomainKey(const TString &path, ui64 schemeShardId, ui64 pathId, - TActorId worker); + TActorId worker); ITransaction *CreateTxUpdatePoolState(TTenant::TPtr tenant, TStoragePool::TPtr pool, - TActorId worker, + TActorId worker, TStoragePool::EState state); ITransaction *CreateTxRevertPoolState(TTenant::TPtr tenant, TStoragePool::TPtr pool, @@ -984,7 +984,7 @@ public: private: TConsole &Self; TDomainsInfo::TDomain::TPtr Domain; - TActorId TenantSlotBrokerPipe; + TActorId TenantSlotBrokerPipe; THashMap<TString, TTenant::TPtr> Tenants; THashMap<TDomainId, TString> TenantIdToName; THashMap<TString, TRemovedTenant> RemovedTenants; diff --git a/ydb/core/cms/console/console_ut_configs.cpp b/ydb/core/cms/console/console_ut_configs.cpp index 5eb4736c14d..0cb1655281d 100644 --- a/ydb/core/cms/console/console_ut_configs.cpp +++ b/ydb/core/cms/console/console_ut_configs.cpp @@ -430,7 +430,7 @@ void IncGeneration(NKikimrConsole::TConfigItem &item, Ts&... args) ui64 CheckAddConfigSubscription(TTenantTestRuntime &runtime, Ydb::StatusIds::StatusCode code, ui32 nodeId, const TString &host, const TString &tenant, - const TString &nodeType, ui64 tabletId, TActorId serviceId, + const TString &nodeType, ui64 tabletId, TActorId serviceId, TVector<ui32> kinds, ui64 id = 0) { auto *event = new TEvConsole::TEvAddConfigSubscriptionRequest; @@ -438,7 +438,7 @@ ui64 CheckAddConfigSubscription(TTenantTestRuntime &runtime, Ydb::StatusIds::Sta if (tabletId) event->Record.MutableSubscription()->MutableSubscriber()->SetTabletId(tabletId); else if (serviceId) - ActorIdToProto(serviceId, event->Record.MutableSubscription()->MutableSubscriber()->MutableServiceId()); + ActorIdToProto(serviceId, event->Record.MutableSubscription()->MutableSubscriber()->MutableServiceId()); event->Record.MutableSubscription()->MutableOptions()->SetNodeId(nodeId); event->Record.MutableSubscription()->MutableOptions()->SetHost(host); event->Record.MutableSubscription()->MutableOptions()->SetTenant(tenant); @@ -458,7 +458,7 @@ ui64 CheckAddConfigSubscription(TTenantTestRuntime &runtime, Ydb::StatusIds::Sta ui64 CheckReplaceConfigSubscriptions(TTenantTestRuntime &runtime, Ydb::StatusIds::StatusCode code, ui32 nodeId, const TString &host, const TString &tenant, - const TString &nodeType, ui64 tabletId, TActorId serviceId, + const TString &nodeType, ui64 tabletId, TActorId serviceId, TVector<ui32> kinds, ui64 id = 0) { auto *event = new TEvConsole::TEvReplaceConfigSubscriptionsRequest; @@ -466,7 +466,7 @@ ui64 CheckReplaceConfigSubscriptions(TTenantTestRuntime &runtime, Ydb::StatusIds if (tabletId) event->Record.MutableSubscription()->MutableSubscriber()->SetTabletId(tabletId); else if (serviceId) - ActorIdToProto(serviceId, event->Record.MutableSubscription()->MutableSubscriber()->MutableServiceId()); + ActorIdToProto(serviceId, event->Record.MutableSubscription()->MutableSubscriber()->MutableServiceId()); event->Record.MutableSubscription()->MutableOptions()->SetNodeId(nodeId); event->Record.MutableSubscription()->MutableOptions()->SetHost(host); event->Record.MutableSubscription()->MutableOptions()->SetTenant(tenant); @@ -486,7 +486,7 @@ ui64 CheckReplaceConfigSubscriptions(TTenantTestRuntime &runtime, Ydb::StatusIds void CheckGetConfigSubscription(TTenantTestRuntime &runtime, Ydb::StatusIds::StatusCode code, ui64 id, ui32 nodeId = 0, const TString &host = "", const TString &tenant = "", - const TString &nodeType = "", ui64 tabletId = 0, TActorId serviceId = {}, + const TString &nodeType = "", ui64 tabletId = 0, TActorId serviceId = {}, TVector<ui32> kinds = {}) { auto *event = new TEvConsole::TEvGetConfigSubscriptionRequest; @@ -501,7 +501,7 @@ void CheckGetConfigSubscription(TTenantTestRuntime &runtime, Ydb::StatusIds::Sta THashSet<ui32> k(kinds.begin(), kinds.end()); UNIT_ASSERT_VALUES_EQUAL(reply->Record.GetSubscription().GetId(), id); UNIT_ASSERT_VALUES_EQUAL(reply->Record.GetSubscription().GetSubscriber().GetTabletId(), tabletId); - UNIT_ASSERT_VALUES_EQUAL(ActorIdFromProto(reply->Record.GetSubscription().GetSubscriber().GetServiceId()), serviceId); + UNIT_ASSERT_VALUES_EQUAL(ActorIdFromProto(reply->Record.GetSubscription().GetSubscriber().GetServiceId()), serviceId); UNIT_ASSERT_VALUES_EQUAL(reply->Record.GetSubscription().GetOptions().GetNodeId(), nodeId); UNIT_ASSERT_VALUES_EQUAL(reply->Record.GetSubscription().GetOptions().GetHost(), host); UNIT_ASSERT_VALUES_EQUAL(reply->Record.GetSubscription().GetOptions().GetTenant(), tenant); @@ -526,13 +526,13 @@ void CheckRemoveConfigSubscription(TTenantTestRuntime &runtime, Ydb::StatusIds:: } void CheckRemoveConfigSubscriptions(TTenantTestRuntime &runtime, Ydb::StatusIds::StatusCode code, - ui64 tabletId, TActorId serviceId) + ui64 tabletId, TActorId serviceId) { auto *event = new TEvConsole::TEvRemoveConfigSubscriptionsRequest; if (tabletId) event->Record.MutableSubscriber()->SetTabletId(tabletId); else if (serviceId) - ActorIdToProto(serviceId, event->Record.MutableSubscriber()->MutableServiceId()); + ActorIdToProto(serviceId, event->Record.MutableSubscriber()->MutableServiceId()); TAutoPtr<IEventHandle> handle; runtime.SendToConsole(event); @@ -2378,7 +2378,7 @@ class TConfigProxy : public TActor<TConfigProxy>, public TTabletExecutedFlat { } public: - TConfigProxy(const TActorId &tablet, TTabletStorageInfo *info, TActorId sink) + TConfigProxy(const TActorId &tablet, TTabletStorageInfo *info, TActorId sink) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, nullptr) , Sink(sink) @@ -2401,12 +2401,12 @@ public: } } private: - TActorId Sink; + TActorId Sink; }; class TConfigProxyService : public TActorBootstrapped<TConfigProxyService> { public: - TConfigProxyService(TActorId sink) + TConfigProxyService(TActorId sink) : Sink(sink) { } @@ -2429,7 +2429,7 @@ public: } private: - TActorId Sink; + TActorId Sink; }; const ui64 CONFIG_PROXY_TABLET_ID = 0x0000000000840103; @@ -2443,7 +2443,7 @@ void StartConfigProxy(TTenantTestRuntime &runtime) runtime.EnableScheduleForActor(actorId, true); auto aid = runtime.Register(new TConfigProxyService(runtime.Sender)); - runtime.RegisterService(TActorId(runtime.GetNodeId(0), "configproxy"), aid, 0); + runtime.RegisterService(TActorId(runtime.GetNodeId(0), "configproxy"), aid, 0); TAutoPtr<IEventHandle> handle; runtime.GrabEdgeEventRethrow<TEvents::TEvWakeup>(handle); @@ -2459,11 +2459,11 @@ Y_UNIT_TEST_SUITE(TConsoleConfigSubscriptionTests) { ui32 nodeId = runtime.GetNodeId(0); ui64 id1 = CheckAddConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, 1, "host1", "tenant-1", "type1", - 1, TActorId(), {{1, 2, 3}}); + 1, TActorId(), {{1, 2, 3}}); // OK subscription for service ui64 id2 = CheckAddConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, 2, "host2", "tenant-2", "type2", - 0, TActorId(nodeId, "service"), {{4, 5, 6}}); + 0, TActorId(nodeId, "service"), {{4, 5, 6}}); runtime.Register(CreateTabletKiller(MakeConsoleID(0))); TDispatchOptions options; @@ -2473,34 +2473,34 @@ Y_UNIT_TEST_SUITE(TConsoleConfigSubscriptionTests) { // OK subscription for service ui64 id3 = CheckAddConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, 3, "host3", "tenant-3", "type3", - 0, TActorId(nodeId, "service"), {{1, 1, 2}}); + 0, TActorId(nodeId, "service"), {{1, 1, 2}}); CheckGetConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, id1, 1, "host1", "tenant-1", "type1", - 1, TActorId(), {{1, 2, 3}}); + 1, TActorId(), {{1, 2, 3}}); CheckGetConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, id2, 2, "host2", "tenant-2", "type2", - 0, TActorId(nodeId, "service"), {{4, 5, 6}}); + 0, TActorId(nodeId, "service"), {{4, 5, 6}}); CheckGetConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, id3, 3, "host3", "tenant-3", "type3", - 0, TActorId(nodeId, "service"), {{1, 2}}); + 0, TActorId(nodeId, "service"), {{1, 2}}); // non-zero subscription id CheckAddConfigSubscription(runtime, Ydb::StatusIds::BAD_REQUEST, 1, "host1", "tenant-1", "type1", - 1, TActorId(), {{1, 2, 3}}, 1); + 1, TActorId(), {{1, 2, 3}}, 1); // no subscriber CheckAddConfigSubscription(runtime, Ydb::StatusIds::BAD_REQUEST, 1, "host1", "tenant-1", "type1", - 0, TActorId(), {{1, 2, 3}}); + 0, TActorId(), {{1, 2, 3}}); // wrong service id CheckAddConfigSubscription(runtime, Ydb::StatusIds::BAD_REQUEST, 1, "host1", "tenant-1", "type1", - 0, TActorId(1, 0, 0, 0), {{1, 2, 3}}); + 0, TActorId(1, 0, 0, 0), {{1, 2, 3}}); // no kinds CheckAddConfigSubscription(runtime, Ydb::StatusIds::BAD_REQUEST, 1, "host1", "tenant-1", "type1", - 1, TActorId(), {}); + 1, TActorId(), {}); } Y_UNIT_TEST(TestRemoveConfigSubscription) { @@ -2510,15 +2510,15 @@ Y_UNIT_TEST_SUITE(TConsoleConfigSubscriptionTests) { ui32 nodeId = runtime.GetNodeId(0); ui64 id1 = CheckAddConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, 1, "host1", "tenant-1", "type1", - 1, TActorId(), {{1, 2, 3}}); + 1, TActorId(), {{1, 2, 3}}); // OK subscription for tablet ui64 id2 = CheckAddConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, 2, "host1", "tenant-1", "type1", - 1, TActorId(), {{1, 2, 3}}); + 1, TActorId(), {{1, 2, 3}}); // OK subscription for service ui64 id3 = CheckAddConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, 2, "host2", "tenant-2", "type2", - 0, TActorId(nodeId, "service"), {{4, 5, 6}}); + 0, TActorId(nodeId, "service"), {{4, 5, 6}}); CheckRemoveConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, id2); CheckRemoveConfigSubscription(runtime, Ydb::StatusIds::NOT_FOUND, id2); @@ -2530,11 +2530,11 @@ Y_UNIT_TEST_SUITE(TConsoleConfigSubscriptionTests) { CheckGetConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, id1, 1, "host1", "tenant-1", "type1", - 1, TActorId(), {{1, 2, 3}}); + 1, TActorId(), {{1, 2, 3}}); CheckGetConfigSubscription(runtime, Ydb::StatusIds::NOT_FOUND, id2); CheckGetConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, id3, 2, "host2", "tenant-2", "type2", - 0, TActorId(nodeId, "service"), {{4, 5, 6}}); + 0, TActorId(nodeId, "service"), {{4, 5, 6}}); CheckRemoveConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, id1); CheckGetConfigSubscription(runtime, Ydb::StatusIds::NOT_FOUND, id1); @@ -2547,28 +2547,28 @@ Y_UNIT_TEST_SUITE(TConsoleConfigSubscriptionTests) { ui32 nodeId = runtime.GetNodeId(0); ui64 id1 = CheckAddConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, 1, "host1", "tenant-1", "type1", - 1, TActorId(), {{1, 2, 3}}); + 1, TActorId(), {{1, 2, 3}}); // OK subscription for tablet ui64 id2 = CheckAddConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, 2, "host1", "tenant-1", "type1", - 1, TActorId(), {{1, 2, 3}}); + 1, TActorId(), {{1, 2, 3}}); // OK subscription for service ui64 id3 = CheckAddConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, 2, "host2", "tenant-2", "type2", - 0, TActorId(nodeId, "service"), {{4, 5, 6}}); + 0, TActorId(nodeId, "service"), {{4, 5, 6}}); // OK subscription for service ui64 id4 = CheckAddConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, 3, "host2", "tenant-2", "type2", - 0, TActorId(nodeId, "service"), {{4, 5, 6}}); + 0, TActorId(nodeId, "service"), {{4, 5, 6}}); CheckRemoveConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, - 1, TActorId()); + 1, TActorId()); CheckRemoveConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, - 1, TActorId()); + 1, TActorId()); CheckRemoveConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, - 2, TActorId()); + 2, TActorId()); CheckRemoveConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, - 0, TActorId()); + 0, TActorId()); runtime.Register(CreateTabletKiller(MakeConsoleID(0))); TDispatchOptions options; @@ -2579,13 +2579,13 @@ Y_UNIT_TEST_SUITE(TConsoleConfigSubscriptionTests) { CheckGetConfigSubscription(runtime, Ydb::StatusIds::NOT_FOUND, id2); CheckGetConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, id3, 2, "host2", "tenant-2", "type2", - 0, TActorId(nodeId, "service"), {{4, 5, 6}}); + 0, TActorId(nodeId, "service"), {{4, 5, 6}}); CheckGetConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, id4, 3, "host2", "tenant-2", "type2", - 0, TActorId(nodeId, "service"), {{4, 5, 6}}); + 0, TActorId(nodeId, "service"), {{4, 5, 6}}); CheckRemoveConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, - 0, TActorId(nodeId, "service")); + 0, TActorId(nodeId, "service")); CheckGetConfigSubscription(runtime, Ydb::StatusIds::NOT_FOUND, id3); CheckGetConfigSubscription(runtime, Ydb::StatusIds::NOT_FOUND, id4); @@ -2598,65 +2598,65 @@ Y_UNIT_TEST_SUITE(TConsoleConfigSubscriptionTests) { ui32 nodeId = runtime.GetNodeId(0); ui64 id1 = CheckAddConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, 1, "host1", "tenant-1", "type1", - 1, TActorId(), {{1, 2}}); + 1, TActorId(), {{1, 2}}); // OK subscription for tablet ui64 id2 = CheckAddConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, 1, "host1", "tenant-1", "type1", - 1, TActorId(), {{3, 4}}); + 1, TActorId(), {{3, 4}}); // OK subscription for service ui64 id3 = CheckAddConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, 2, "host2", "tenant-2", "type2", - 0, TActorId(nodeId, "service"), {{4, 5}}); + 0, TActorId(nodeId, "service"), {{4, 5}}); // OK subscription for service ui64 id4 = CheckAddConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, 2, "host2", "tenant-2", "type2", - 0, TActorId(nodeId, "service"), {{6, 7}}); + 0, TActorId(nodeId, "service"), {{6, 7}}); // List 1 2 - CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 1, TActorId(), + CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 1, TActorId(), id1, 1, "host1", "tenant-1", "type1", - 1, TActorId(), TVector<ui32>({1, 2}), + 1, TActorId(), TVector<ui32>({1, 2}), id2, 1, "host1", "tenant-1", "type1", - 1, TActorId(), TVector<ui32>({3, 4})); + 1, TActorId(), TVector<ui32>({3, 4})); // List 3 4 - CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(nodeId, "service"), + CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(nodeId, "service"), id3, 2, "host2", "tenant-2", "type2", - 0, TActorId(nodeId, "service"), TVector<ui32>({4, 5}), + 0, TActorId(nodeId, "service"), TVector<ui32>({4, 5}), id4, 2, "host2", "tenant-2", "type2", - 0, TActorId(nodeId, "service"), TVector<ui32>({6, 7})); + 0, TActorId(nodeId, "service"), TVector<ui32>({6, 7})); // List 1 2 3 4 - CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(), + CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(), id1, 1, "host1", "tenant-1", "type1", - 1, TActorId(), TVector<ui32>({1, 2}), + 1, TActorId(), TVector<ui32>({1, 2}), id2, 1, "host1", "tenant-1", "type1", - 1, TActorId(), TVector<ui32>({3, 4}), + 1, TActorId(), TVector<ui32>({3, 4}), id3, 2, "host2", "tenant-2", "type2", - 0, TActorId(nodeId, "service"), TVector<ui32>({4, 5}), + 0, TActorId(nodeId, "service"), TVector<ui32>({4, 5}), id4, 2, "host2", "tenant-2", "type2", - 0, TActorId(nodeId, "service"), TVector<ui32>({6, 7})); + 0, TActorId(nodeId, "service"), TVector<ui32>({6, 7})); // Empty list - CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 2, TActorId()); + CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 2, TActorId()); // Empty list - CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(nodeId, "service1")); + CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(nodeId, "service1")); // Empty list - CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(nodeId + 1, "service")); + CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(nodeId + 1, "service")); // Remove 1 2 CheckRemoveConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, - 1, TActorId()); + 1, TActorId()); // Remove 3 CheckRemoveConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, id3); // Empty list - CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 1, TActorId()); + CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 1, TActorId()); // List 4 - CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(nodeId, "service"), + CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(nodeId, "service"), id4, 2, "host2", "tenant-2", "type2", - 0, TActorId(nodeId, "service"), TVector<ui32>({6, 7})); + 0, TActorId(nodeId, "service"), TVector<ui32>({6, 7})); // List 4 - CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(), + CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(), id4, 2, "host2", "tenant-2", "type2", - 0, TActorId(nodeId, "service"), TVector<ui32>({6, 7})); + 0, TActorId(nodeId, "service"), TVector<ui32>({6, 7})); } Y_UNIT_TEST(TestReplaceConfigSubscriptions) { @@ -2665,32 +2665,32 @@ Y_UNIT_TEST_SUITE(TConsoleConfigSubscriptionTests) { ui32 nodeId = runtime.GetNodeId(0); ui64 id1 = CheckReplaceConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 1, "host1", "tenant-1", "type1", - 1, TActorId(), {{1, 2, 3}}); + 1, TActorId(), {{1, 2, 3}}); ui64 id2 = CheckReplaceConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 2, "host2", "tenant-2", "type2", - 0, TActorId(nodeId, "service"), {{4, 5, 6}}); + 0, TActorId(nodeId, "service"), {{4, 5, 6}}); - CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 1, TActorId(), + CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 1, TActorId(), id1, 1, "host1", "tenant-1", "type1", - 1, TActorId(), TVector<ui32>({1, 2, 3})); - CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(nodeId, "service"), + 1, TActorId(), TVector<ui32>({1, 2, 3})); + CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(nodeId, "service"), id2, 2, "host2", "tenant-2", "type2", - 0, TActorId(nodeId, "service"), TVector<ui32>({4, 5, 6})); + 0, TActorId(nodeId, "service"), TVector<ui32>({4, 5, 6})); CheckAddConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, 1, "host1", "tenant-1", "type1", - 1, TActorId(), {{1, 2}}); + 1, TActorId(), {{1, 2}}); CheckAddConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, 2, "host2", "tenant-2", "type2", - 0, TActorId(nodeId, "service"), {{4, 5}}); + 0, TActorId(nodeId, "service"), {{4, 5}}); ui64 id5 = CheckReplaceConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 1, "host1", "tenant-1", "type1", - 1, TActorId(), {{1, 2, 3}}); + 1, TActorId(), {{1, 2, 3}}); UNIT_ASSERT_VALUES_EQUAL(id1, id5); ui64 id6 = CheckReplaceConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 2, "host2", "tenant-2", "type2", - 0, TActorId(nodeId, "service"), {{4, 5, 6}}); + 0, TActorId(nodeId, "service"), {{4, 5, 6}}); UNIT_ASSERT_VALUES_EQUAL(id2, id6); runtime.Register(CreateTabletKiller(MakeConsoleID(0))); @@ -2698,28 +2698,28 @@ Y_UNIT_TEST_SUITE(TConsoleConfigSubscriptionTests) { options.FinalEvents.emplace_back(&IsTabletActiveEvent, 1); runtime.DispatchEvents(options); - CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 1, TActorId(), + CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 1, TActorId(), id1, 1, "host1", "tenant-1", "type1", - 1, TActorId(), TVector<ui32>({1, 2, 3})); - CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(nodeId, "service"), + 1, TActorId(), TVector<ui32>({1, 2, 3})); + CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(nodeId, "service"), id2, 2, "host2", "tenant-2", "type2", - 0, TActorId(nodeId, "service"), TVector<ui32>({4, 5, 6})); + 0, TActorId(nodeId, "service"), TVector<ui32>({4, 5, 6})); ui64 id7 = CheckReplaceConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 1, "host1", "tenant-1", "type1", - 1, TActorId(), {{1, 2}}); + 1, TActorId(), {{1, 2}}); UNIT_ASSERT(id1 != id7); ui64 id8 = CheckReplaceConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 2, "host2", "tenant-2", "type2", - 0, TActorId(nodeId, "service"), {{4, 5}}); + 0, TActorId(nodeId, "service"), {{4, 5}}); UNIT_ASSERT(id2 != id8); - CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 1, TActorId(), + CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 1, TActorId(), id7, 1, "host1", "tenant-1", "type1", - 1, TActorId(), TVector<ui32>({1, 2})); - CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(nodeId, "service"), + 1, TActorId(), TVector<ui32>({1, 2})); + CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(nodeId, "service"), id8, 2, "host2", "tenant-2", "type2", - 0, TActorId(nodeId, "service"), TVector<ui32>({4, 5})); + 0, TActorId(nodeId, "service"), TVector<ui32>({4, 5})); } void TestNotificationForNewSubscriptionBase(bool useService) @@ -2728,11 +2728,11 @@ Y_UNIT_TEST_SUITE(TConsoleConfigSubscriptionTests) { InitializeTestConfigItems(); StartConfigProxy(runtime); TAutoPtr<IEventHandle> handle; - TActorId serviceId; + TActorId serviceId; ui64 tabletId = 0; if (useService) - serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); + serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); else tabletId = CONFIG_PROXY_TABLET_ID; @@ -2761,11 +2761,11 @@ Y_UNIT_TEST_SUITE(TConsoleConfigSubscriptionTests) { InitializeTestConfigItems(); StartConfigProxy(runtime); TAutoPtr<IEventHandle> handle; - TActorId serviceId; + TActorId serviceId; ui64 tabletId = 0; if (useService) - serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); + serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); else tabletId = CONFIG_PROXY_TABLET_ID; @@ -2854,11 +2854,11 @@ Y_UNIT_TEST_SUITE(TConsoleConfigSubscriptionTests) { InitializeTestConfigItems(); StartConfigProxy(runtime); TAutoPtr<IEventHandle> handle; - TActorId serviceId; + TActorId serviceId; ui64 tabletId = 0; if (useService) - serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); + serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); else tabletId = CONFIG_PROXY_TABLET_ID; @@ -2973,11 +2973,11 @@ Y_UNIT_TEST_SUITE(TConsoleConfigSubscriptionTests) { InitializeTestConfigItems(); StartConfigProxy(runtime); TAutoPtr<IEventHandle> handle; - TActorId serviceId; + TActorId serviceId; ui64 tabletId = 0; if (useService) - serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); + serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); else tabletId = CONFIG_PROXY_TABLET_ID; @@ -3135,11 +3135,11 @@ Y_UNIT_TEST_SUITE(TConsoleConfigSubscriptionTests) { InitializeTestConfigItems(); StartConfigProxy(runtime); TAutoPtr<IEventHandle> handle; - TActorId serviceId; + TActorId serviceId; ui64 tabletId = 0; if (useService) - serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); + serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); else tabletId = CONFIG_PROXY_TABLET_ID; @@ -3234,11 +3234,11 @@ Y_UNIT_TEST_SUITE(TConsoleConfigSubscriptionTests) { InitializeTestConfigItems(); StartConfigProxy(runtime); TAutoPtr<IEventHandle> handle; - TActorId serviceId; + TActorId serviceId; ui64 tabletId = 0; if (useService) - serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); + serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); else tabletId = CONFIG_PROXY_TABLET_ID; @@ -3293,11 +3293,11 @@ Y_UNIT_TEST_SUITE(TConsoleConfigSubscriptionTests) { InitializeTestConfigItems(); StartConfigProxy(runtime); TAutoPtr<IEventHandle> handle; - TActorId serviceId; + TActorId serviceId; ui64 tabletId = 0; if (useService) - serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); + serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); else tabletId = CONFIG_PROXY_TABLET_ID; @@ -3330,11 +3330,11 @@ Y_UNIT_TEST_SUITE(TConsoleConfigSubscriptionTests) { InitializeTestConfigItems(); StartConfigProxy(runtime); TAutoPtr<IEventHandle> handle; - TActorId serviceId; + TActorId serviceId; ui64 tabletId = 0; if (useService) - serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); + serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); else tabletId = CONFIG_PROXY_TABLET_ID; @@ -3377,11 +3377,11 @@ Y_UNIT_TEST_SUITE(TConsoleConfigSubscriptionTests) { InitializeTestConfigItems(); StartConfigProxy(runtime); TAutoPtr<IEventHandle> handle; - TActorId serviceId; + TActorId serviceId; ui64 tabletId = 0; if (useService) - serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); + serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); else tabletId = CONFIG_PROXY_TABLET_ID; @@ -3432,7 +3432,7 @@ Y_UNIT_TEST_SUITE(TConsoleConfigSubscriptionTests) { CheckAddConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, 100, "host100", "tenant-100", "type100", - 0, TActorId(100, "service"), + 0, TActorId(100, "service"), TVector<ui32>({(ui32)NKikimrConsole::TConfigItem::LogConfigItem})); ui32 undelivered = 0; @@ -3483,23 +3483,23 @@ Y_UNIT_TEST_SUITE(TConsoleConfigSubscriptionTests) { ui32 nodeId = runtime.GetNodeId(0); ui64 id1 = CheckAddConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, nodeId, "host1", "tenant-1", "type1", - 0, TActorId(nodeId, "service"), TVector<ui32>({1})); + 0, TActorId(nodeId, "service"), TVector<ui32>({1})); ui64 id2 = CheckAddConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, 100, "host100", "tenant-100", "type100", - 0, TActorId(100, "service"), TVector<ui32>({1})); + 0, TActorId(100, "service"), TVector<ui32>({1})); ui64 id3 = CheckAddConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, 100, "host100", "tenant-100", "type100", - 100, TActorId(), TVector<ui32>({1})); + 100, TActorId(), TVector<ui32>({1})); - CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(nodeId, "service"), + CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(nodeId, "service"), id1, nodeId, "host1", "tenant-1", "type1", - 0, TActorId(nodeId, "service"), TVector<ui32>({1})); - CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(100, "service"), + 0, TActorId(nodeId, "service"), TVector<ui32>({1})); + CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(100, "service"), id2, 100, "host100", "tenant-100", "type100", - 0, TActorId(100, "service"), TVector<ui32>({1})); - CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 100, TActorId(), + 0, TActorId(100, "service"), TVector<ui32>({1})); + CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 100, TActorId(), id3, 100, "host100", "tenant-100", "type100", - 100, TActorId(), TVector<ui32>({1})); + 100, TActorId(), TVector<ui32>({1})); TDispatchOptions options; options.FinalEvents.emplace_back([&](IEventHandle &ev) -> bool { @@ -3513,15 +3513,15 @@ Y_UNIT_TEST_SUITE(TConsoleConfigSubscriptionTests) { CheckAddConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, nodeId + 1, "host2", "tenant-2", "type2", - 0, TActorId(nodeId + 1, "service"), TVector<ui32>({1})); - CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(nodeId, "service"), + 0, TActorId(nodeId + 1, "service"), TVector<ui32>({1})); + CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(nodeId, "service"), id1, nodeId, "host1", "tenant-1", "type1", - 0, TActorId(nodeId, "service"), TVector<ui32>({1})); - CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(100, "service")); + 0, TActorId(nodeId, "service"), TVector<ui32>({1})); + CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 0, TActorId(100, "service")); - CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 100, TActorId(), + CheckListConfigSubscriptions(runtime, Ydb::StatusIds::SUCCESS, 100, TActorId(), id3, 100, "host100", "tenant-100", "type100", - 100, TActorId(), TVector<ui32>({1})); + 100, TActorId(), TVector<ui32>({1})); } } @@ -3805,12 +3805,12 @@ Y_UNIT_TEST_SUITE(TConsoleConfigHelpersTests) { InitializeTestConfigItems(); StartConfigProxy(runtime); TAutoPtr<IEventHandle> handle; - TActorId serviceId; + TActorId serviceId; ui64 tabletId = 0; IActor *subscriber; if (useService) - serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); + serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); else tabletId = CONFIG_PROXY_TABLET_ID; @@ -3886,12 +3886,12 @@ Y_UNIT_TEST_SUITE(TConsoleConfigHelpersTests) { InitializeTestConfigItems(); StartConfigProxy(runtime); TAutoPtr<IEventHandle> handle; - TActorId serviceId; + TActorId serviceId; ui64 tabletId = 0; IActor *subscriber; if (useService) { - serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); + serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); subscriber = CreateConfigSubscriber(serviceId, TVector<ui32>({1, 2}), runtime.Sender); @@ -3923,12 +3923,12 @@ Y_UNIT_TEST_SUITE(TConsoleConfigHelpersTests) { InitializeTestConfigItems(); StartConfigProxy(runtime); TAutoPtr<IEventHandle> handle; - TActorId serviceId; + TActorId serviceId; ui64 tabletId = 0; IActor *subscriber; if (useService) { - serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); + serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); subscriber = CreateConfigSubscriber(serviceId, TVector<ui32>({1, 2}), runtime.Sender); @@ -3960,12 +3960,12 @@ Y_UNIT_TEST_SUITE(TConsoleConfigHelpersTests) { InitializeTestConfigItems(); StartConfigProxy(runtime); TAutoPtr<IEventHandle> handle; - TActorId serviceId; + TActorId serviceId; ui64 tabletId = 0; IActor *subscriber; if (useService) { - serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); + serviceId = TActorId(runtime.GetNodeId(0), "configproxy"); subscriber = CreateConfigSubscriber(serviceId, TVector<ui32>({1, 2}), runtime.Sender); @@ -3999,7 +3999,7 @@ Y_UNIT_TEST_SUITE(TConsoleConfigHelpersTests) { ui64 id1 = CheckAddConfigSubscription(runtime, Ydb::StatusIds::SUCCESS, 1, "host1", "tenant-1", "type1", - CONFIG_PROXY_TABLET_ID, TActorId(), {{1, 2, 3}}); + CONFIG_PROXY_TABLET_ID, TActorId(), {{1, 2, 3}}); auto *eraser = CreateSubscriptionEraser(id1, runtime.Sender, 123); runtime.Register(eraser, 0); diff --git a/ydb/core/cms/console/console_ut_tenants.cpp b/ydb/core/cms/console/console_ut_tenants.cpp index e1ea4131009..57e4820af72 100644 --- a/ydb/core/cms/console/console_ut_tenants.cpp +++ b/ydb/core/cms/console/console_ut_tenants.cpp @@ -509,7 +509,7 @@ Y_UNIT_TEST_SUITE(TConsoleTxProcessorTests) { class TTxTestActor : public TActorBootstrapped<TTxTestActor> { public: - TTxTestActor(TActorId sink, std::function<void(const TActorContext &)> testFunction) + TTxTestActor(TActorId sink, std::function<void(const TActorContext &)> testFunction) : Sink(sink) , TestFunction(testFunction) { @@ -521,7 +521,7 @@ Y_UNIT_TEST_SUITE(TConsoleTxProcessorTests) { ctx.Send(Sink, new TEvents::TEvWakeup); } private: - TActorId Sink; + TActorId Sink; std::function<void(const TActorContext &)> TestFunction; }; diff --git a/ydb/core/cms/console/http.h b/ydb/core/cms/console/http.h index a4569cab0f4..f64ba795df1 100644 --- a/ydb/core/cms/console/http.h +++ b/ydb/core/cms/console/http.h @@ -3,7 +3,7 @@ #include <ydb/core/protos/config.pb.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/stream/output.h> diff --git a/ydb/core/cms/console/net_classifier_updater.cpp b/ydb/core/cms/console/net_classifier_updater.cpp index 530ae4fb77e..121a65efc37 100644 --- a/ydb/core/cms/console/net_classifier_updater.cpp +++ b/ydb/core/cms/console/net_classifier_updater.cpp @@ -65,7 +65,7 @@ private: using TBase = TActorBootstrapped<NetClassifierUpdater>; public: - NetClassifierUpdater(TActorId localConsole) + NetClassifierUpdater(TActorId localConsole) : LocalConsole(localConsole) { } @@ -326,16 +326,16 @@ private: } private: - TActorId LocalConsole; - TActorId HttpProxyId; - NMonitoring::TMetricRegistry HttpSensors; + TActorId LocalConsole; + TActorId HttpProxyId; + NMonitoring::TMetricRegistry HttpSensors; TString PackedNetData; TString LastUpdateDatetimeUTC; TInstant LastUpdateTimestamp; }; -IActor* MakeNetClassifierUpdaterActor(TActorId localConsole) { +IActor* MakeNetClassifierUpdaterActor(TActorId localConsole) { return new NetClassifierUpdater(localConsole); } diff --git a/ydb/core/cms/console/net_classifier_updater.h b/ydb/core/cms/console/net_classifier_updater.h index 328564c3879..b0ebcd63d95 100644 --- a/ydb/core/cms/console/net_classifier_updater.h +++ b/ydb/core/cms/console/net_classifier_updater.h @@ -5,7 +5,7 @@ namespace NKikimr::NNetClassifierUpdater { -IActor* MakeNetClassifierUpdaterActor(TActorId localConsole); +IActor* MakeNetClassifierUpdaterActor(TActorId localConsole); TString UnpackNetData(const TString& packedNetData); diff --git a/ydb/core/cms/console/net_classifier_updater_ut.cpp b/ydb/core/cms/console/net_classifier_updater_ut.cpp index a82fd9fbdef..da41ee51f26 100644 --- a/ydb/core/cms/console/net_classifier_updater_ut.cpp +++ b/ydb/core/cms/console/net_classifier_updater_ut.cpp @@ -10,7 +10,7 @@ #include <library/cpp/testing/unittest/tests_data.h> #include <library/cpp/testing/unittest/registar.h> -#include <library/cpp/json/json_writer.h> +#include <library/cpp/json/json_writer.h> #include <util/string/builder.h> @@ -133,7 +133,7 @@ Y_UNIT_TEST_SUITE(TNetClassifierUpdaterTest) { TNetClassifierUpdaterConfig::EFormat format = TNetClassifierUpdaterConfig::TSV, const TVector<TString>& netBoxTags = {} ) { - NMonitoring::TMetricRegistry sensors; + NMonitoring::TMetricRegistry sensors; TPortManager pm; const ui16 port = pm.GetPort(2134); @@ -145,13 +145,13 @@ Y_UNIT_TEST_SUITE(TNetClassifierUpdaterTest) { auto& actorSystem = *cleverServer.GetRuntime(); NActors::IActor* proxy = NHttp::CreateHttpProxy(sensors); - NActors::TActorId proxyId = actorSystem.Register(proxy); + NActors::TActorId proxyId = actorSystem.Register(proxy); actorSystem.Send( - new NActors::IEventHandle(proxyId, TActorId(), new NHttp::TEvHttpProxy::TEvAddListeningPort(netDataSourcePort)), 0, true + new NActors::IEventHandle(proxyId, TActorId(), new NHttp::TEvHttpProxy::TEvAddListeningPort(netDataSourcePort)), 0, true ); - NActors::TActorId serverId = actorSystem.AllocateEdgeActor(); + NActors::TActorId serverId = actorSystem.AllocateEdgeActor(); actorSystem.Send(new NActors::IEventHandle(proxyId, serverId, new NHttp::TEvHttpProxy::TEvRegisterHandler(NETWORKS_URI, serverId)), 0, true); @@ -160,7 +160,7 @@ Y_UNIT_TEST_SUITE(TNetClassifierUpdaterTest) { UNIT_ASSERT_EQUAL(request->Request->URL, NETWORKS_URI); actorSystem.Send(new NActors::IEventHandle(handle->Sender, serverId, MakeHttpResponse(request, sourceResponce)), 0, true); - const TActorId sender = actorSystem.AllocateEdgeActor(); + const TActorId sender = actorSystem.AllocateEdgeActor(); size_t iterations = 0; while (true) { diff --git a/ydb/core/cms/console/ut_helpers.h b/ydb/core/cms/console/ut_helpers.h index becb958054d..9faf6468fa2 100644 --- a/ydb/core/cms/console/ut_helpers.h +++ b/ydb/core/cms/console/ut_helpers.h @@ -175,7 +175,7 @@ inline void CollectSubscriptions(THashMap<ui64, TSubscription> &) inline void CollectSubscriptions(THashMap<ui64, TSubscription> &subscriptions, ui64 id, ui32 nodeId, const TString &host, const TString &tenant, - const TString &nodeType, ui64 tabletId, TActorId serviceId, + const TString &nodeType, ui64 tabletId, TActorId serviceId, TVector<ui32> kinds) { TSubscription subscription; @@ -194,7 +194,7 @@ inline void CollectSubscriptions(THashMap<ui64, TSubscription> &subscriptions, template <typename ...Ts> void CollectSubscriptions(THashMap<ui64, TSubscription> &subscriptions, ui64 id, ui32 nodeId, const TString &host, const TString &tenant, - const TString &nodeType, ui64 tabletId, TActorId serviceId, + const TString &nodeType, ui64 tabletId, TActorId serviceId, TVector<ui32> kinds, Ts ...args) { CollectSubscriptions(subscriptions, id, nodeId, host, tenant, nodeType, tabletId, serviceId, kinds); @@ -203,7 +203,7 @@ void CollectSubscriptions(THashMap<ui64, TSubscription> &subscriptions, template <typename ...Ts> void CheckListConfigSubscriptions(TTenantTestRuntime &runtime, Ydb::StatusIds::StatusCode code, - ui64 tabletId, TActorId serviceId, Ts ...args) + ui64 tabletId, TActorId serviceId, Ts ...args) { THashMap<ui64, TSubscription> subscriptions; CollectSubscriptions(subscriptions, args...); @@ -212,7 +212,7 @@ void CheckListConfigSubscriptions(TTenantTestRuntime &runtime, Ydb::StatusIds::S if (tabletId) event->Record.MutableSubscriber()->SetTabletId(tabletId); else if (serviceId) - ActorIdToProto(serviceId, event->Record.MutableSubscriber()->MutableServiceId()); + ActorIdToProto(serviceId, event->Record.MutableSubscriber()->MutableServiceId()); TAutoPtr<IEventHandle> handle; runtime.SendToConsole(event); diff --git a/ydb/core/cms/downtime_ut.cpp b/ydb/core/cms/downtime_ut.cpp index dc1130d6f45..12a5bdbdde8 100644 --- a/ydb/core/cms/downtime_ut.cpp +++ b/ydb/core/cms/downtime_ut.cpp @@ -160,20 +160,20 @@ Y_UNIT_TEST_SUITE(TDowntimeTest) { TInstant t7 = t1 + TDuration::Seconds(33); downtime1.AddDowntime(t2, t4); - UNIT_ASSERT(!downtime1.HasUpcomingDowntime(t1, TDuration::Seconds(0), TDuration::Seconds(5))); - UNIT_ASSERT(downtime1.HasUpcomingDowntime(t1, TDuration::Seconds(3), TDuration::Seconds(5))); - UNIT_ASSERT(!downtime1.HasUpcomingDowntime(t2, TDuration::Seconds(3), TDuration::Seconds(15))); - UNIT_ASSERT(downtime1.HasUpcomingDowntime(t2, TDuration::Seconds(3), TDuration::Seconds(5))); - UNIT_ASSERT(!downtime1.HasUpcomingDowntime(t3, TDuration::Seconds(3), TDuration::Seconds(15))); - UNIT_ASSERT(downtime1.HasUpcomingDowntime(t3, TDuration::Seconds(3), TDuration::Seconds(5))); - UNIT_ASSERT(downtime1.HasUpcomingDowntime(t4, TDuration::Seconds(3), TDuration::Seconds(5))); - UNIT_ASSERT(!downtime1.HasUpcomingDowntime(t5, TDuration::Seconds(3), TDuration::Seconds(5))); + UNIT_ASSERT(!downtime1.HasUpcomingDowntime(t1, TDuration::Seconds(0), TDuration::Seconds(5))); + UNIT_ASSERT(downtime1.HasUpcomingDowntime(t1, TDuration::Seconds(3), TDuration::Seconds(5))); + UNIT_ASSERT(!downtime1.HasUpcomingDowntime(t2, TDuration::Seconds(3), TDuration::Seconds(15))); + UNIT_ASSERT(downtime1.HasUpcomingDowntime(t2, TDuration::Seconds(3), TDuration::Seconds(5))); + UNIT_ASSERT(!downtime1.HasUpcomingDowntime(t3, TDuration::Seconds(3), TDuration::Seconds(15))); + UNIT_ASSERT(downtime1.HasUpcomingDowntime(t3, TDuration::Seconds(3), TDuration::Seconds(5))); + UNIT_ASSERT(downtime1.HasUpcomingDowntime(t4, TDuration::Seconds(3), TDuration::Seconds(5))); + UNIT_ASSERT(!downtime1.HasUpcomingDowntime(t5, TDuration::Seconds(3), TDuration::Seconds(5))); downtime1.AddDowntime(t5, t7); - UNIT_ASSERT(!downtime1.HasUpcomingDowntime(t1, TDuration::Seconds(0), TDuration::Seconds(10))); - UNIT_ASSERT(downtime1.HasUpcomingDowntime(t1, TDuration::Seconds(25), TDuration::Seconds(10))); - UNIT_ASSERT(!downtime1.HasUpcomingDowntime(t1, TDuration::Seconds(30), TDuration::Seconds(11))); - UNIT_ASSERT(!downtime1.HasUpcomingDowntime(t6, TDuration::Seconds(0), TDuration::Seconds(11))); + UNIT_ASSERT(!downtime1.HasUpcomingDowntime(t1, TDuration::Seconds(0), TDuration::Seconds(10))); + UNIT_ASSERT(downtime1.HasUpcomingDowntime(t1, TDuration::Seconds(25), TDuration::Seconds(10))); + UNIT_ASSERT(!downtime1.HasUpcomingDowntime(t1, TDuration::Seconds(30), TDuration::Seconds(11))); + UNIT_ASSERT(!downtime1.HasUpcomingDowntime(t6, TDuration::Seconds(0), TDuration::Seconds(11))); } Y_UNIT_TEST(SetIgnoredDowntimeGap) diff --git a/ydb/core/cms/json_proxy.h b/ydb/core/cms/json_proxy.h index fba87ee4b43..185d0ce01b6 100644 --- a/ydb/core/cms/json_proxy.h +++ b/ydb/core/cms/json_proxy.h @@ -182,7 +182,7 @@ protected: } NMon::TEvHttpInfo::TPtr RequestEvent; - TActorId Pipe; + TActorId Pipe; }; template <typename TRequestEvent, typename TResponseEvent> diff --git a/ydb/core/cms/json_proxy_proto.h b/ydb/core/cms/json_proxy_proto.h index 9c55a7f0a93..b1b87f30cda 100644 --- a/ydb/core/cms/json_proxy_proto.h +++ b/ydb/core/cms/json_proxy_proto.h @@ -12,7 +12,7 @@ #include <library/cpp/actors/core/mon.h> #include <library/cpp/json/json_value.h> #include <library/cpp/json/json_reader.h> -#include <library/cpp/json/json_writer.h> +#include <library/cpp/json/json_writer.h> #include <util/generic/serialized_enum.h> #include <iostream> diff --git a/ydb/core/cms/sentinel.cpp b/ydb/core/cms/sentinel.cpp index 8e369f276e2..638b78fe09b 100644 --- a/ydb/core/cms/sentinel.cpp +++ b/ydb/core/cms/sentinel.cpp @@ -303,7 +303,7 @@ public: } protected: - const TActorId Parent; + const TActorId Parent; TCmsStatePtr CmsState; const TCmsSentinelConfig& Config; @@ -485,7 +485,7 @@ class TStateUpdater: public TUpdaterBase<TEvSentinel::TEvStateUpdated, TStateUpd } void RequestPDiskState(ui32 nodeId) { - const TActorId wbId = NNodeWhiteboard::MakeNodeWhiteboardServiceId(nodeId); + const TActorId wbId = NNodeWhiteboard::MakeNodeWhiteboardServiceId(nodeId); const ui32 flags = IEventHandle::FlagTrackDelivery; LOG_D("Request pdisks state" @@ -688,7 +688,7 @@ public: } explicit TStatusChanger( - const TActorId& parent, + const TActorId& parent, TCmsStatePtr state, const TPDiskID& id, NKikimrBlobStorage::EDriveStatus status) @@ -749,7 +749,7 @@ class TSentinel: public TActorBootstrapped<TSentinel> { }; struct TUpdaterInfo { - TActorId Id; + TActorId Id; TInstant StartedAt; bool Delayed; @@ -757,14 +757,14 @@ class TSentinel: public TActorBootstrapped<TSentinel> { Clear(); } - void Start(const TActorId& id, const TInstant& now) { + void Start(const TActorId& id, const TInstant& now) { Id = id; StartedAt = now; Delayed = false; } void Clear() { - Id = TActorId(); + Id = TActorId(); StartedAt = TInstant::Zero(); Delayed = false; } @@ -979,32 +979,32 @@ class TSentinel: public TActorBootstrapped<TSentinel> { (*Counters->PDisksChanged)++; } - it->second.StatusChanger = TActorId(); + it->second.StatusChanger = TActorId(); } void OnPipeDisconnected() { - if (const TActorId& actor = ConfigUpdater.Id) { + if (const TActorId& actor = ConfigUpdater.Id) { Send(actor, new TEvSentinel::TEvBSCPipeDisconnected()); } for (const auto& pdisk : SentinelState->PDisks) { - if (const TActorId& actor = pdisk.second.StatusChanger) { + if (const TActorId& actor = pdisk.second.StatusChanger) { Send(actor, new TEvSentinel::TEvBSCPipeDisconnected()); } } } void PassAway() override { - if (const TActorId& actor = ConfigUpdater.Id) { + if (const TActorId& actor = ConfigUpdater.Id) { Send(actor, new TEvents::TEvPoisonPill()); } - if (const TActorId& actor = StateUpdater.Id) { + if (const TActorId& actor = StateUpdater.Id) { Send(actor, new TEvents::TEvPoisonPill()); } for (const auto& pdisk : SentinelState->PDisks) { - if (const TActorId& actor = pdisk.second.StatusChanger) { + if (const TActorId& actor = pdisk.second.StatusChanger) { Send(actor, new TEvents::TEvPoisonPill()); } } diff --git a/ydb/core/cms/sentinel_impl.h b/ydb/core/cms/sentinel_impl.h index f1975435b9a..8aa6934bafa 100644 --- a/ydb/core/cms/sentinel_impl.h +++ b/ydb/core/cms/sentinel_impl.h @@ -59,7 +59,7 @@ private: }; // TPDiskStatus struct TPDiskInfo: public TPDiskStatus { - TActorId StatusChanger; + TActorId StatusChanger; explicit TPDiskInfo(EPDiskStatus initialStatus, const ui32& defaultStateLimit, const TLimitsMap& stateLimits); diff --git a/ydb/core/cms/sentinel_ut.cpp b/ydb/core/cms/sentinel_ut.cpp index 12454dd3867..62c0fdf309d 100644 --- a/ydb/core/cms/sentinel_ut.cpp +++ b/ydb/core/cms/sentinel_ut.cpp @@ -343,7 +343,7 @@ Y_UNIT_TEST_SUITE(TSentinelTests) { pdisk.SetState(state); } - Send(new IEventHandle(Sentinel, TActorId(), new TEvSentinel::TEvUpdateState)); + Send(new IEventHandle(Sentinel, TActorId(), new TEvSentinel::TEvUpdateState)); } public: @@ -487,7 +487,7 @@ Y_UNIT_TEST_SUITE(TSentinelTests) { private: TCmsStatePtr State; - TActorId Sentinel; + TActorId Sentinel; }; // TTestEnv diff --git a/ydb/core/cms/walle.h b/ydb/core/cms/walle.h index d9f80f4fa28..9a08bb45758 100644 --- a/ydb/core/cms/walle.h +++ b/ydb/core/cms/walle.h @@ -14,10 +14,10 @@ namespace NCms { constexpr const char *WALLE_CMS_USER = "Wall-E"; constexpr const char *WALLE_API_URL_PREFIX = "/api/walle/v11/"; -IActor *CreateWalleAdapter(TEvCms::TEvWalleCreateTaskRequest::TPtr &ev, TActorId cms); +IActor *CreateWalleAdapter(TEvCms::TEvWalleCreateTaskRequest::TPtr &ev, TActorId cms); IActor *CreateWalleAdapter(TEvCms::TEvWalleListTasksRequest::TPtr &ev, const TCmsStatePtr state); -IActor *CreateWalleAdapter(TEvCms::TEvWalleCheckTaskRequest::TPtr &ev, const TCmsStatePtr state, TActorId cms); -IActor *CreateWalleAdapter(TEvCms::TEvWalleRemoveTaskRequest::TPtr &ev, const TCmsStatePtr state, TActorId cms); +IActor *CreateWalleAdapter(TEvCms::TEvWalleCheckTaskRequest::TPtr &ev, const TCmsStatePtr state, TActorId cms); +IActor *CreateWalleAdapter(TEvCms::TEvWalleRemoveTaskRequest::TPtr &ev, const TCmsStatePtr state, TActorId cms); IActor *CreateWalleApiHandler(NMon::TEvHttpInfo::TPtr &event); diff --git a/ydb/core/cms/walle_api_handler.cpp b/ydb/core/cms/walle_api_handler.cpp index d6daebcc713..7b62f3275a9 100644 --- a/ydb/core/cms/walle_api_handler.cpp +++ b/ydb/core/cms/walle_api_handler.cpp @@ -372,7 +372,7 @@ private: } NMon::TEvHttpInfo::TPtr RequestEvent; - TActorId CmsPipe; + TActorId CmsPipe; }; diff --git a/ydb/core/cms/walle_check_task_adapter.cpp b/ydb/core/cms/walle_check_task_adapter.cpp index 529c9fcc42f..e5570c9af91 100644 --- a/ydb/core/cms/walle_check_task_adapter.cpp +++ b/ydb/core/cms/walle_check_task_adapter.cpp @@ -16,7 +16,7 @@ public: } TWalleCheckTaskAdapter(TEvCms::TEvWalleCheckTaskRequest::TPtr &event, - const TCmsStatePtr state, TActorId cms) + const TCmsStatePtr state, TActorId cms) : RequestEvent(event) , State(state) , Cms(cms) @@ -114,12 +114,12 @@ private: TEvCms::TEvWalleCheckTaskRequest::TPtr RequestEvent; TAutoPtr<TEvCms::TEvWalleCheckTaskResponse> Response; const TCmsStatePtr State; - TActorId Cms; + TActorId Cms; }; IActor *CreateWalleAdapter(TEvCms::TEvWalleCheckTaskRequest::TPtr &ev, - const TCmsStatePtr state, TActorId cms) + const TCmsStatePtr state, TActorId cms) { return new TWalleCheckTaskAdapter(ev, state, cms); } diff --git a/ydb/core/cms/walle_create_task_adapter.cpp b/ydb/core/cms/walle_create_task_adapter.cpp index 02e99306278..56989e2db03 100644 --- a/ydb/core/cms/walle_create_task_adapter.cpp +++ b/ydb/core/cms/walle_create_task_adapter.cpp @@ -16,7 +16,7 @@ public: return NKikimrServices::TActivity::CMS_WALLE_REQ; } - TWalleCreateTaskAdapter(TEvCms::TEvWalleCreateTaskRequest::TPtr &event, TActorId cms) + TWalleCreateTaskAdapter(TEvCms::TEvWalleCreateTaskRequest::TPtr &event, TActorId cms) : RequestEvent(event) , Cms(cms) { @@ -208,11 +208,11 @@ private: TEvCms::TEvWalleCreateTaskRequest::TPtr RequestEvent; TAutoPtr<TEvCms::TEvWalleCreateTaskResponse> Response; - TActorId Cms; + TActorId Cms; }; -IActor *CreateWalleAdapter(TEvCms::TEvWalleCreateTaskRequest::TPtr &ev, TActorId cms) +IActor *CreateWalleAdapter(TEvCms::TEvWalleCreateTaskRequest::TPtr &ev, TActorId cms) { return new TWalleCreateTaskAdapter(ev, cms); } diff --git a/ydb/core/cms/walle_remove_task_adapter.cpp b/ydb/core/cms/walle_remove_task_adapter.cpp index 256c3409d76..899d4848250 100644 --- a/ydb/core/cms/walle_remove_task_adapter.cpp +++ b/ydb/core/cms/walle_remove_task_adapter.cpp @@ -15,7 +15,7 @@ public: return NKikimrServices::TActivity::CMS_WALLE_REQ; } - TWalleRemoveTaskAdapter(TEvCms::TEvWalleRemoveTaskRequest::TPtr &event, const TCmsStatePtr state, TActorId cms) + TWalleRemoveTaskAdapter(TEvCms::TEvWalleRemoveTaskRequest::TPtr &event, const TCmsStatePtr state, TActorId cms) : RequestEvent(event) , State(state) , Cms(cms) @@ -83,11 +83,11 @@ private: TEvCms::TEvWalleRemoveTaskRequest::TPtr RequestEvent; const TCmsStatePtr State; - TActorId Cms; + TActorId Cms; }; -IActor *CreateWalleAdapter(TEvCms::TEvWalleRemoveTaskRequest::TPtr &ev, const TCmsStatePtr state, TActorId cms) +IActor *CreateWalleAdapter(TEvCms::TEvWalleRemoveTaskRequest::TPtr &ev, const TCmsStatePtr state, TActorId cms) { return new TWalleRemoveTaskAdapter(ev, state, cms); } diff --git a/ydb/core/control/immediate_control_board_actor.cpp b/ydb/core/control/immediate_control_board_actor.cpp index 9fe19f8f776..bcb4a3344c3 100644 --- a/ydb/core/control/immediate_control_board_actor.cpp +++ b/ydb/core/control/immediate_control_board_actor.cpp @@ -3,8 +3,8 @@ #include <ydb/core/mon/mon.h> #include <ydb/core/base/appdata.h> #include <ydb/core/base/counters.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/service/pages/templates.h> namespace NKikimr { diff --git a/ydb/core/control/immediate_control_board_actor.h b/ydb/core/control/immediate_control_board_actor.h index b6585c16958..0b2cff4cf66 100644 --- a/ydb/core/control/immediate_control_board_actor.h +++ b/ydb/core/control/immediate_control_board_actor.h @@ -2,15 +2,15 @@ #include "defs.h" #include "immediate_control_board_impl.h" -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <library/cpp/actors/core/actor.h> #include <library/cpp/actors/core/actor_bootstrapped.h> namespace NKikimr { -inline NActors::TActorId MakeIcbId(ui32 node) { +inline NActors::TActorId MakeIcbId(ui32 node) { char x[12] = {'i','c','b','_','a','c','t','o','r'}; - return NActors::TActorId(node, TStringBuf(x, 12)); + return NActors::TActorId(node, TStringBuf(x, 12)); } class TImmediateControlActor; diff --git a/ydb/core/control/immediate_control_board_actor_ut.cpp b/ydb/core/control/immediate_control_board_actor_ut.cpp index eb4f1febb7e..2753077d936 100644 --- a/ydb/core/control/immediate_control_board_actor_ut.cpp +++ b/ydb/core/control/immediate_control_board_actor_ut.cpp @@ -61,10 +61,10 @@ static void SignalDoneEvent() { } struct TTestConfig { - TActorId IcbActorId; + TActorId IcbActorId; TControlBoard *Icb; - TTestConfig(TActorId icbActorId, TControlBoard *icb) + TTestConfig(TActorId icbActorId, TControlBoard *icb) : IcbActorId(icbActorId) , Icb(icb) {} @@ -72,7 +72,7 @@ struct TTestConfig { template <class T> static void Run(i64 instances = 1) { - TVector<TActorId> testIds; + TVector<TActorId> testIds; TAppData appData(0, 0, 0, 0, TMap<TString, ui32>(), nullptr, nullptr, nullptr, nullptr); @@ -95,21 +95,21 @@ static void Run(i64 instances = 1) { setup->Executors[2].Reset(new TIOExecutorPool(2, 10)); setup->Scheduler.Reset(new TBasicSchedulerThread(TSchedulerConfig(512, 100))); - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); TActorSetupCmd nameserviceSetup(CreateNameserverTable(nameserverTable), TMailboxType::Simple, 0); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(nameserviceId, nameserviceSetup)); + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(nameserviceId, nameserviceSetup)); // ICB Actor creation - TActorId IcbActorId = MakeIcbId(setup->NodeId); + TActorId IcbActorId = MakeIcbId(setup->NodeId); TActorSetupCmd testSetup(CreateImmediateControlActor(appData.Icb, Counters), TMailboxType::Revolving, 0); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(IcbActorId, testSetup)); + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(IcbActorId, testSetup)); THolder<TTestConfig> testConfig(new TTestConfig(IcbActorId, appData.Icb.Get())); for (ui32 i = 0; i < instances; ++i) { testIds[i] = MakeBlobStorageProxyID(1 + i); TActorSetupCmd testSetup(new T(testConfig.Get()), TMailboxType::Revolving, 0); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(testIds[i], testSetup)); + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(testIds[i], testSetup)); } AtomicSet(DoneCounter, 0); @@ -117,7 +117,7 @@ static void Run(i64 instances = 1) { /////////////////////// LOGGER /////////////////////////////////////////////// - NActors::TActorId loggerActorId = NActors::TActorId(1, "logger"); + NActors::TActorId loggerActorId = NActors::TActorId(1, "logger"); TIntrusivePtr<NActors::NLog::TSettings> logSettings( new NActors::NLog::TSettings(loggerActorId, NKikimrServices::LOGGER, NActors::NLog::PRI_ERROR, NActors::NLog::PRI_ERROR, 0)); //logSettings->Append( @@ -137,7 +137,7 @@ static void Run(i64 instances = 1) { NActors::TLoggerActor *loggerActor = new NActors::TLoggerActor(logSettings, NActors::CreateStderrBackend(), GetServiceCounters(Counters, "utils")); NActors::TActorSetupCmd loggerActorCmd(loggerActor, NActors::TMailboxType::Simple, 2); - std::pair<NActors::TActorId, NActors::TActorSetupCmd> loggerActorPair(loggerActorId, loggerActorCmd); + std::pair<NActors::TActorId, NActors::TActorSetupCmd> loggerActorPair(loggerActorId, loggerActorCmd); setup->LocalServices.push_back(loggerActorPair); ////////////////////////////////////////////////////////////////////////////// @@ -147,7 +147,7 @@ static void Run(i64 instances = 1) { VERBOSE_COUT("Sending TEvBoot to test"); for (ui32 i = 0; i < instances; ++i) { - ActorSystem->Send(testIds[i], new TEvTablet::TEvBoot(MakeTabletID(0, 0, 1), 0, nullptr, TActorId(), nullptr)); + ActorSystem->Send(testIds[i], new TEvTablet::TEvBoot(MakeTabletID(0, 0, 1), 0, nullptr, TActorId(), nullptr)); } TAtomicBase doneCount = 0; @@ -203,7 +203,7 @@ protected: TResponseData LastResponse; - const TActorId IcbActor; + const TActorId IcbActor; TControlBoard *Icb; int TestStep; diff --git a/ydb/core/control/immediate_control_board_impl.cpp b/ydb/core/control/immediate_control_board_impl.cpp index d5a5871aa34..fa26926fae5 100644 --- a/ydb/core/control/immediate_control_board_impl.cpp +++ b/ydb/core/control/immediate_control_board_impl.cpp @@ -2,7 +2,7 @@ #include <util/generic/string.h> #include <util/stream/str.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> namespace NKikimr { diff --git a/ydb/core/control/ya.make b/ydb/core/control/ya.make index a8725fa7520..bb1de545ae7 100644 --- a/ydb/core/control/ya.make +++ b/ydb/core/control/ya.make @@ -7,7 +7,7 @@ OWNER( PEERDIR( library/cpp/actors/core - library/cpp/monlib/dynamic_counters + library/cpp/monlib/dynamic_counters util ydb/core/base ydb/core/mon diff --git a/ydb/core/driver_lib/cli_utils/cli_cmds_server.cpp b/ydb/core/driver_lib/cli_utils/cli_cmds_server.cpp index 1610ea7353d..6766dd31719 100644 --- a/ydb/core/driver_lib/cli_utils/cli_cmds_server.cpp +++ b/ydb/core/driver_lib/cli_utils/cli_cmds_server.cpp @@ -228,8 +228,8 @@ protected: config.Opts->AddLongOption('r', "restarts-count-file", "State for restarts monitoring counter,\nuse empty string to disable\n") .OptionalArgument("PATH").DefaultValue(RestartsCountFile).StoreResult(&RestartsCountFile); config.Opts->AddLongOption("compile-inflight-limit", "Limit on parallel programs compilation").OptionalArgument("NUM").StoreResult(&CompileInflightLimit); - config.Opts->AddLongOption("udf", "Load shared library with UDF by given path").AppendTo(&UDFsPaths); - config.Opts->AddLongOption("udfs-dir", "Load all shared libraries with UDFs found in given directory").StoreResult(&UDFsDir); + config.Opts->AddLongOption("udf", "Load shared library with UDF by given path").AppendTo(&UDFsPaths); + config.Opts->AddLongOption("udfs-dir", "Load all shared libraries with UDFs found in given directory").StoreResult(&UDFsDir); config.Opts->AddLongOption("node-type", "Type of the node") .RequiredArgument("NAME").StoreResult(&NodeType); config.Opts->AddLongOption("ignore-cms-configs", "Don't load configs from CMS") @@ -513,7 +513,7 @@ protected: AppConfig.AddUDFsPaths(path); } } - + if (!AppConfig.HasMonitoringConfig()) AppConfig.MutableMonitoringConfig()->SetMonitoringThreads(MonitoringThreads); if (!AppConfig.HasRestartsCountConfig() && RestartsCountFile) @@ -530,7 +530,7 @@ protected: auto& conf = *AppConfig.MutableGRpcConfig(); conf.SetStartGRpcProxy(true); conf.SetPort(GRpcPort); - } + } if (GRpcsPort) { auto& conf = *AppConfig.MutableGRpcConfig(); conf.SetStartGRpcProxy(true); diff --git a/ydb/core/driver_lib/run/config_parser.cpp b/ydb/core/driver_lib/run/config_parser.cpp index d52a8f53f19..bb686562912 100644 --- a/ydb/core/driver_lib/run/config_parser.cpp +++ b/ydb/core/driver_lib/run/config_parser.cpp @@ -223,8 +223,8 @@ void TRunCommandConfigParser::SetupGlobalOpts(NLastGetopt::TOpts& opts) { .DefaultValue("full").OptionalArgument("full|short|json").StoreResult(&GlobalOpts.LogFormat); opts.AddLongOption("syslog", "send to syslog instead of stderr").NoArgument(); opts.AddLongOption("tcp", "start tcp interconnect").NoArgument(); - opts.AddLongOption("udf", "Load shared library with UDF by given path").AppendTo(&GlobalOpts.UDFsPaths); - opts.AddLongOption("udfs-dir", "Load all shared libraries with UDFs found in given directory").StoreResult(&GlobalOpts.UDFsDir); + opts.AddLongOption("udf", "Load shared library with UDF by given path").AppendTo(&GlobalOpts.UDFsPaths); + opts.AddLongOption("udfs-dir", "Load all shared libraries with UDFs found in given directory").StoreResult(&GlobalOpts.UDFsDir); } @@ -241,7 +241,7 @@ void TRunCommandConfigParser::ParseRunOpts(int argc, char **argv) { TOpts opts = TOpts::Default(); opts.AddLongOption('n', "node", "Node ID") - .RequiredArgument(Sprintf("NUM (0,%d]", NActors::TActorId::MaxNodeId)).Required().StoreResult(&RunOpts.NodeId); + .RequiredArgument(Sprintf("NUM (0,%d]", NActors::TActorId::MaxNodeId)).Required().StoreResult(&RunOpts.NodeId); opts.AddLongOption("proxy", "Bind to proxy(-ies)").RequiredArgument("ADDR").AppendTo(&RunOpts.ProxyBindToProxy); opts.AddLongOption("mon-port", "Monitoring port").OptionalArgument("NUM").StoreResult(&RunOpts.MonitoringPort); opts.AddLongOption("mon-address", "Monitoring address").OptionalArgument("ADDR").StoreResult(&RunOpts.MonitoringAddress); @@ -279,7 +279,7 @@ void TRunCommandConfigParser::ParseRunOpts(int argc, char **argv) { RunOpts.StartBusProxy = true; } - Y_VERIFY(RunOpts.NodeId > 0 && RunOpts.NodeId <= NActors::TActorId::MaxNodeId); + Y_VERIFY(RunOpts.NodeId > 0 && RunOpts.NodeId <= NActors::TActorId::MaxNodeId); } void TRunCommandConfigParser::ApplyParsedOptions() { @@ -293,12 +293,12 @@ void TRunCommandConfigParser::ApplyParsedOptions() { logConfig->SetFormat(GlobalOpts.LogFormat); logConfig->SetClusterName(GlobalOpts.ClusterName); - // apply options affecting UDF paths - Config.AppConfig.SetUDFsDir(GlobalOpts.UDFsDir); - for (const auto& path : GlobalOpts.UDFsPaths) { - Config.AppConfig.AddUDFsPaths(path); - } - + // apply options affecting UDF paths + Config.AppConfig.SetUDFsDir(GlobalOpts.UDFsDir); + for (const auto& path : GlobalOpts.UDFsPaths) { + Config.AppConfig.AddUDFsPaths(path); + } + // apply run options Config.NodeId = RunOpts.NodeId; auto messageBusConfig = Config.AppConfig.MutableMessageBusConfig(); diff --git a/ydb/core/driver_lib/run/kikimr_services_initializers.cpp b/ydb/core/driver_lib/run/kikimr_services_initializers.cpp index e1490f68d2d..819c1478d19 100644 --- a/ydb/core/driver_lib/run/kikimr_services_initializers.cpp +++ b/ydb/core/driver_lib/run/kikimr_services_initializers.cpp @@ -522,7 +522,7 @@ void TBasicServicesInitializer::InitializeServices(NActors::TActorSystemSetup* s const auto &execConfig = systemConfig.GetExecutor(poolId); if (execConfig.HasInjectMadSquirrels()) { for (ui32 i = execConfig.GetInjectMadSquirrels(); i > 0; --i) { - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(TActorId(), TActorSetupCmd(CreateMadSquirrel(), TMailboxType::HTSwap, poolId))); + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(TActorId(), TActorSetupCmd(CreateMadSquirrel(), TMailboxType::HTSwap, poolId))); } } } @@ -538,7 +538,7 @@ void TBasicServicesInitializer::InitializeServices(NActors::TActorSystemSetup* s if (Config.HasNameserviceConfig()) { const auto& nsConfig = Config.GetNameserviceConfig(); const TActorId resolverId = NDnsResolver::MakeDnsResolverActorId(); - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); ui32 numNodes = 0; TSet<TString> dataCenters; @@ -675,7 +675,7 @@ void TBasicServicesInitializer::InitializeServices(NActors::TActorSystemSetup* s } if (const auto& mon = appData->Mon) { - icCommon->RegisterMonPage = [mon](const TString& path, const TString& title, TActorSystem *actorSystem, const TActorId& actorId) { + icCommon->RegisterMonPage = [mon](const TString& path, const TString& title, TActorSystem *actorSystem, const TActorId& actorId) { NMonitoring::TIndexMonPage *page = mon->RegisterIndexPage("actors", "Actors")->RegisterIndexPage("interconnect", "Interconnect"); mon->RegisterActorPage(page, path, title, false, actorSystem, actorId); }; @@ -770,12 +770,12 @@ TImmediateControlBoardInitializer::TImmediateControlBoardInitializer(const TKiki void TImmediateControlBoardInitializer::InitializeServices(NActors::TActorSystemSetup* setup, const NKikimr::TAppData* appData) { - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>( + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>( MakeIcbId(NodeId), TActorSetupCmd(CreateImmediateControlActor(appData->Icb, appData->Counters), TMailboxType::ReadAsFilled, appData->UserPoolId) )); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>( - TActorId(), + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>( + TActorId(), TActorSetupCmd(NConsole::CreateImmediateControlsConfigurator(appData->Icb, Config.GetImmediateControlsConfig()), TMailboxType::ReadAsFilled, appData->UserPoolId) @@ -815,7 +815,7 @@ void TBSNodeWardenInitializer::InitializeServices(NActors::TActorSystemSetup* se ObtainStaticKey(&nodeWardenConfig->StaticKey); ObtainPDiskKey(&nodeWardenConfig->PDiskKey, Config.GetPDiskKeyConfig()); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeBlobStorageNodeWardenID(NodeId), + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeBlobStorageNodeWardenID(NodeId), TActorSetupCmd(CreateBSNodeWarden(nodeWardenConfig.Release()), TMailboxType::ReadAsFilled, appData->SystemPoolId))); @@ -829,7 +829,7 @@ template<typename TCreateFunc> void StartLocalStateStorageReplicas(TCreateFunc createFunc, TStateStorageInfo *info, ui32 poolId, TActorSystemSetup &setup) { ui32 index = 0; for (auto &ring : info->Rings) { - for (TActorId replica : ring.Replicas) { + for (TActorId replica : ring.Replicas) { if (replica.NodeId() == setup.NodeId) { setup.LocalServices.emplace_back( replica, @@ -864,21 +864,21 @@ void TStateStorageServiceInitializer::InitializeServices(NActors::TActorSystemSe StartLocalStateStorageReplicas(CreateStateStorageBoardReplica, ssbInfo.Get(), appData->SystemPoolId, *setup); StartLocalStateStorageReplicas(CreateSchemeBoardReplica, sbrInfo.Get(), appData->SystemPoolId, *setup); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeStateStorageProxyID(ssid), + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeStateStorageProxyID(ssid), TActorSetupCmd(CreateStateStorageProxy(ssrInfo.Get(), ssbInfo.Get(), sbrInfo.Get()), TMailboxType::ReadAsFilled, appData->SystemPoolId))); } for (ui32 ssid = 0; ssid <= maxssid; ++ssid) { if (!knownss[ssid]) - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeStateStorageProxyID(ssid), + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeStateStorageProxyID(ssid), TActorSetupCmd(CreateStateStorageProxyStub(), TMailboxType::HTSwap, appData->SystemPoolId))); } setup->LocalServices.emplace_back( - TActorId(), + TActorId(), TActorSetupCmd(CreateTenantNodeEnumerationPublisher(), TMailboxType::HTSwap, appData->SystemPoolId) ); } @@ -946,7 +946,7 @@ void TLocalServiceInitializer::InitializeServices( TActorSetupCmd(CreateTenantPool(tenantPoolConfig), TMailboxType::ReadAsFilled, 0))); setup->LocalServices.push_back(std::make_pair( - TActorId(), + TActorId(), TActorSetupCmd(CreateLabelsMaintainer(Config.GetMonitoringConfig()), TMailboxType::ReadAsFilled, 0))); @@ -991,7 +991,7 @@ void TSharedCacheInitializer::InitializeServices( TActorSetupCmd(CreateSharedPageCache(config.Get()), TMailboxType::ReadAsFilled, appData->UserPoolId))); auto *configurator = NConsole::CreateSharedCacheConfigurator(); - setup->LocalServices.emplace_back(TActorId(), + setup->LocalServices.emplace_back(TActorId(), TActorSetupCmd(configurator, TMailboxType::HTSwap, appData->UserPoolId)); } @@ -1033,7 +1033,7 @@ void TLoggerInitializer::InitializeServices( // log settings must be initialized before calling this method NActors::TLoggerActor *loggerActor = new NActors::TLoggerActor(LogSettings, LogBackend, utilsCounters); NActors::TActorSetupCmd loggerActorCmd(loggerActor, NActors::TMailboxType::HTSwap, appData->IOPoolId); - std::pair<NActors::TActorId, NActors::TActorSetupCmd> loggerActorPair(LogSettings->LoggerActorId, loggerActorCmd); + std::pair<NActors::TActorId, NActors::TActorSetupCmd> loggerActorPair(LogSettings->LoggerActorId, loggerActorCmd); setup->LocalServices.push_back(loggerActorPair); IActor *configurator; @@ -1043,7 +1043,7 @@ void TLoggerInitializer::InitializeServices( configurator = NConsole::CreateLogSettingsConfigurator(); } - setup->LocalServices.emplace_back(TActorId(), + setup->LocalServices.emplace_back(TActorId(), TActorSetupCmd(configurator, TMailboxType::HTSwap, appData->UserPoolId)); } @@ -1060,7 +1060,7 @@ void TSchedulerActorInitializer::InitializeServices( NActors::IActor *schedulerActor = CreateSchedulerActor(CreateSchedulerConfig(systemConfig.GetScheduler())); if (schedulerActor) { NActors::TActorSetupCmd schedulerActorCmd(schedulerActor, NActors::TMailboxType::ReadAsFilled, appData->SystemPoolId); - setup->LocalServices.emplace_back(MakeSchedulerActorId(), schedulerActorCmd); + setup->LocalServices.emplace_back(MakeSchedulerActorId(), schedulerActorCmd); } } @@ -1076,7 +1076,7 @@ void TProfilerInitializer::InitializeServices( const TIntrusivePtr<NMonitoring::TDynamicCounters> utilsCounters = GetServiceCounters(appData->Counters, "utils"); TActorSetupCmd profilerSetup(CreateProfilerActor(utilsCounters, "/var/tmp"), TMailboxType::HTSwap, 0); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeProfilerID(NodeId), profilerSetup)); + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeProfilerID(NodeId), profilerSetup)); } // TResourceBrokerInitializer @@ -1153,7 +1153,7 @@ void TTabletResolverInitializer::InitializeServices( NActors::TActorSystemSetup* setup, const NKikimr::TAppData* appData) { TIntrusivePtr<TTabletResolverConfig> tabletResolverConfig(new TTabletResolverConfig()); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeTabletResolverID(), TActorSetupCmd(CreateTabletResolver(tabletResolverConfig), TMailboxType::ReadAsFilled, appData->SystemPoolId))); + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeTabletResolverID(), TActorSetupCmd(CreateTabletResolver(tabletResolverConfig), TMailboxType::ReadAsFilled, appData->SystemPoolId))); } @@ -1201,7 +1201,7 @@ void TTabletMonitoringProxyInitializer::InitializeServices( proxyConfig.SetRetryLimitCount(Config.GetMonitoringConfig().GetTabletMonitoringRetries()); TActorSetupCmd tabletMonitoringProxySetup(NTabletMonitoringProxy::CreateTabletMonitoringProxy(std::move(proxyConfig)), TMailboxType::ReadAsFilled, appData->UserPoolId); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(NTabletMonitoringProxy::MakeTabletMonitoringProxyID(), tabletMonitoringProxySetup)); + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(NTabletMonitoringProxy::MakeTabletMonitoringProxyID(), tabletMonitoringProxySetup)); } @@ -1216,11 +1216,11 @@ void TTabletCountersAggregatorInitializer::InitializeServices( const NKikimr::TAppData* appData) { { TActorSetupCmd tabletCountersAggregatorSetup(CreateTabletCountersAggregator(false), TMailboxType::ReadAsFilled, appData->UserPoolId); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeTabletCountersAggregatorID(NodeId, false), tabletCountersAggregatorSetup)); + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeTabletCountersAggregatorID(NodeId, false), tabletCountersAggregatorSetup)); } { TActorSetupCmd tabletCountersAggregatorSetup(CreateTabletCountersAggregator(true), TMailboxType::ReadAsFilled, appData->UserPoolId); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeTabletCountersAggregatorID(NodeId, true), tabletCountersAggregatorSetup)); + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeTabletCountersAggregatorID(NodeId, true), tabletCountersAggregatorSetup)); } } @@ -1234,7 +1234,7 @@ void TGRpcProxyStatusInitializer::InitializeServices( NActors::TActorSystemSetup* setup, const NKikimr::TAppData* appData) { TActorSetupCmd gRpcProxyStatusSetup(CreateGRpcProxyStatus(), TMailboxType::ReadAsFilled, appData->UserPoolId); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeGRpcProxyStatusID(NodeId), gRpcProxyStatusSetup)); + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeGRpcProxyStatusID(NodeId), gRpcProxyStatusSetup)); } @@ -1292,8 +1292,8 @@ void TBootstrapperInitializer::InitializeServices( if (Config.HasBootstrapConfig()) { for (const auto &boot : Config.GetBootstrapConfig().GetTablet()) { if (boot.GetAllowDynamicConfiguration()) { - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>( - TActorId(), + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>( + TActorId(), TActorSetupCmd(CreateConfiguredTabletBootstrapper(boot), TMailboxType::HTSwap, appData->SystemPoolId))); } else { const bool standby = boot.HasStandBy() && boot.GetStandBy(); @@ -1322,7 +1322,7 @@ void TBootstrapperInitializer::InitializeServices( bi->StartFollowers = boot.GetStartFollowers(); } - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeBootstrapperID(info->TabletID, bootstrapperNode), TActorSetupCmd(CreateBootstrapper(info.Get(), bi.Get(), standby), TMailboxType::HTSwap, appData->SystemPoolId))); + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeBootstrapperID(info->TabletID, bootstrapperNode), TActorSetupCmd(CreateBootstrapper(info.Get(), bi.Get(), standby), TMailboxType::HTSwap, appData->SystemPoolId))); } } } @@ -1383,7 +1383,7 @@ TMediatorTimeCastProxyInitializer::TMediatorTimeCastProxyInitializer(const TKiki void TMediatorTimeCastProxyInitializer::InitializeServices( NActors::TActorSystemSetup* setup, const NKikimr::TAppData* appData) { - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>( + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>( MakeMediatorTimecastProxyID(), TActorSetupCmd(CreateMediatorTimecastProxy(), TMailboxType::ReadAsFilled, appData->SystemPoolId))); } @@ -1398,13 +1398,13 @@ void TMiniKQLCompileServiceInitializer::InitializeServices(NActors::TActorSystem const NKikimr::TAppData* appData) { const auto compileInFlight = Config.GetBootstrapConfig().GetCompileServiceConfig().GetInflightLimit(); IActor* compileService = CreateMiniKQLCompileService(compileInFlight); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeMiniKQLCompileServiceID(), + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeMiniKQLCompileServiceID(), TActorSetupCmd(compileService, TMailboxType::ReadAsFilled, appData->UserPoolId))); } -static bool IsServiceInitialized(NActors::TActorSystemSetup* setup, TActorId service) +static bool IsServiceInitialized(NActors::TActorSystemSetup* setup, TActorId service) { for (auto &pr : setup->LocalServices) if (pr.first == service) @@ -1439,7 +1439,7 @@ void TMessageBusServicesInitializer::InitializeServices(NActors::TActorSystemSet if (IActor* traceService = BusServer.CreateMessageBusTraceService()) { TActorSetupCmd messageBusTraceServiceSetup(traceService, TMailboxType::HTSwap, appData->IOPoolId); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(NMessageBusTracer::MakeMessageBusTraceServiceID(), + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(NMessageBusTracer::MakeMessageBusTraceServiceID(), messageBusTraceServiceSetup)); } } @@ -1463,7 +1463,7 @@ void TSecurityServicesInitializer::InitializeServices(NActors::TActorSystemSetup ticketParser = CreateTicketParser(Config.GetAuthConfig()); } if (ticketParser) { - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeTicketParserID(), + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeTicketParserID(), TActorSetupCmd(ticketParser, TMailboxType::HTSwap, appData->UserPoolId))); } } @@ -1508,7 +1508,7 @@ void TGRpcServicesInitializer::InitializeServices(NActors::TActorSystemSetup* se if (!IsServiceInitialized(setup, NGRpcService::CreateGRpcRequestProxyId())) { auto grpcReqProxy = NGRpcService::CreateGRpcRequestProxy(Config); - setup->LocalServices.push_back(std::pair<TActorId, + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(NGRpcService::CreateGRpcRequestProxyId(), TActorSetupCmd(grpcReqProxy, TMailboxType::ReadAsFilled, appData->UserPoolId))); @@ -1639,7 +1639,7 @@ void TStatsCollectorInitializer::InitializeServices( *setup, appData->Counters); setup->LocalServices.emplace_back( - TActorId(), + TActorId(), TActorSetupCmd( statsCollector, TMailboxType::HTSwap, @@ -1649,7 +1649,7 @@ void TStatsCollectorInitializer::InitializeServices( 1, // seconds appData->Counters); setup->LocalServices.emplace_back( - TActorId(), + TActorId(), TActorSetupCmd( memStatsCollector, TMailboxType::HTSwap, @@ -1659,7 +1659,7 @@ void TStatsCollectorInitializer::InitializeServices( 5, // seconds appData->Counters); setup->LocalServices.emplace_back( - TActorId(), + TActorId(), TActorSetupCmd( procStatCollector, TMailboxType::HTSwap, @@ -1690,7 +1690,7 @@ void TSelfPingInitializer::InitializeServices( auto counter = poolGroup->GetCounter("SelfPingMaxUs", false); auto cpuTimeCounter = poolGroup->GetCounter("CpuMatBenchNs", false); IActor* selfPingActor = CreateSelfPingActor(selfPingInterval, counter, cpuTimeCounter); - setup->LocalServices.push_back(std::make_pair(TActorId(), + setup->LocalServices.push_back(std::make_pair(TActorId(), TActorSetupCmd(selfPingActor, TMailboxType::HTSwap, poolId))); @@ -1706,7 +1706,7 @@ TWhiteBoardServiceInitializer::TWhiteBoardServiceInitializer(const TKikimrRunCon void TWhiteBoardServiceInitializer::InitializeServices(NActors::TActorSystemSetup* setup, const NKikimr::TAppData* appData) { IActor* tabletStateService = NNodeWhiteboard::CreateNodeWhiteboardService(); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(NNodeWhiteboard::MakeNodeWhiteboardServiceId(NodeId), + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(NNodeWhiteboard::MakeNodeWhiteboardServiceId(NodeId), TActorSetupCmd(tabletStateService, TMailboxType::HTSwap, appData->SystemPoolId))); @@ -1721,7 +1721,7 @@ TNodeIdentifierInitializer::TNodeIdentifierInitializer(const TKikimrRunConfig& r void TNodeIdentifierInitializer::InitializeServices(NActors::TActorSystemSetup* setup, const NKikimr::TAppData* appData) { IActor* nodeIdentifier = CreateNodeIdentifier(); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeNodeIdentifierServiceId(), + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeNodeIdentifierServiceId(), TActorSetupCmd(nodeIdentifier, TMailboxType::Simple, appData->IOPoolId))); @@ -1741,7 +1741,7 @@ TTabletMonitorInitializer::TTabletMonitorInitializer( void TTabletMonitorInitializer::InitializeServices(NActors::TActorSystemSetup* setup, const NKikimr::TAppData* appData) { IActor* nodeTabletMonitor = NNodeTabletMonitor::CreateNodeTabletMonitor(TabletStateClassifier, TabletListRenderer); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(NNodeTabletMonitor::MakeNodeTabletMonitorID(NodeId), + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(NNodeTabletMonitor::MakeNodeTabletMonitorID(NodeId), TActorSetupCmd(nodeTabletMonitor, TMailboxType::HTSwap, appData->UserPoolId))); @@ -1760,7 +1760,7 @@ void TViewerInitializer::InitializeServices(NActors::TActorSystemSetup* setup, IActor* viewer = CreateViewer(KikimrRunConfig); SetupPQVirtualHandlers(dynamic_cast<IViewer*>(viewer)); SetupDBVirtualHandlers(dynamic_cast<IViewer*>(viewer)); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeViewerID(NodeId), + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeViewerID(NodeId), TActorSetupCmd(viewer, TMailboxType::HTSwap, appData->BatchPoolId))); @@ -1828,7 +1828,7 @@ void TPersQueueL2CacheInitializer::InitializeServices(NActors::TActorSystemSetup TIntrusivePtr<NMonitoring::TDynamicCounters> pqCacheGroup = tabletGroup->GetSubgroup("type", "PQ_CACHE"); IActor* actor = NPQ::CreateNodePersQueueL2Cache(params, pqCacheGroup); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>( + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>( NPQ::MakePersQueueL2CacheID(), TActorSetupCmd(actor, TMailboxType::HTSwap, appData->UserPoolId))); } @@ -1842,7 +1842,7 @@ TNetClassifierInitializer::TNetClassifierInitializer(const TKikimrRunConfig& run void TNetClassifierInitializer::InitializeServices(NActors::TActorSystemSetup* setup, const NKikimr::TAppData* appData) { IActor* actor = NNetClassifier::CreateNetClassifier(); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>( + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>( NNetClassifier::MakeNetClassifierID(), TActorSetupCmd(actor, TMailboxType::HTSwap, appData->UserPoolId))); } @@ -1855,7 +1855,7 @@ TPersQueueClusterTrackerInitializer::TPersQueueClusterTrackerInitializer(const T void TPersQueueClusterTrackerInitializer::InitializeServices(NActors::TActorSystemSetup* setup, const NKikimr::TAppData* appData) { IActor* actor = NPQ::NClusterTracker::CreateClusterTracker(); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>( + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>( NPQ::NClusterTracker::MakeClusterTrackerID(), TActorSetupCmd(actor, TMailboxType::HTSwap, appData->UserPoolId))); } @@ -2011,7 +2011,7 @@ void TCmsServiceInitializer::InitializeServices(NActors::TActorSystemSetup* setu const NKikimr::TAppData* appData) { auto http = NCms::CreateCmsHttp(); - setup->LocalServices.emplace_back(TActorId(), + setup->LocalServices.emplace_back(TActorId(), TActorSetupCmd(http, TMailboxType::HTSwap, appData->UserPoolId)); } @@ -2034,7 +2034,7 @@ TVector<ui64> TTxProxyInitializer::CollectAllAllocatorsFromAllDomains(const TApp void TTxProxyInitializer::InitializeServices(TActorSystemSetup *setup, const TAppData *appData) { - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>( + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>( MakeTxProxyID(), TActorSetupCmd(CreateTxProxy(CollectAllAllocatorsFromAllDomains(appData)), TMailboxType::ReadAsFilled, appData->UserPoolId))); } @@ -2079,7 +2079,7 @@ void TLeaseHolderInitializer::InitializeServices(NActors::TActorSystemSetup* set if (Config.GetDynamicNodeConfig().HasNodeInfo()) { TInstant expire = TInstant::MicroSeconds(Config.GetDynamicNodeConfig().GetNodeInfo().GetExpire()); auto holder = NNodeBroker::CreateLeaseHolder(expire); - setup->LocalServices.emplace_back(TActorId(), + setup->LocalServices.emplace_back(TActorId(), TActorSetupCmd(holder, TMailboxType::HTSwap, appData->UserPoolId)); } } @@ -2121,7 +2121,7 @@ TConfigsDispatcherInitializer::TConfigsDispatcherInitializer(const TKikimrRunCon void TConfigsDispatcherInitializer::InitializeServices(NActors::TActorSystemSetup* setup, const NKikimr::TAppData* appData) { IActor* actor = NConsole::CreateConfigsDispatcher(Config); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>( + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>( NConsole::MakeConfigsDispatcherID(NodeId), TActorSetupCmd(actor, TMailboxType::HTSwap, appData->UserPoolId))); } @@ -2151,7 +2151,7 @@ void TTabletInfoInitializer::InitializeServices( NActors::TActorSystemSetup* setup, const NKikimr::TAppData* appData) { TActorSetupCmd tabletInfoSetup(NTabletInfo::CreateTabletInfo(), TMailboxType::ReadAsFilled, appData->UserPoolId); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(NTabletInfo::MakeTabletInfoID(), tabletInfoSetup)); + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(NTabletInfo::MakeTabletInfoID(), tabletInfoSetup)); } TConfigValidatorsInitializer::TConfigValidatorsInitializer(const TKikimrRunConfig& runConfig) @@ -2183,7 +2183,7 @@ void TSysViewServiceInitializer::InitializeServices(NActors::TActorSystemSetup* auto actor = NSysView::CreateSysViewService(std::move(config), hasExternalCounters); - setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>( + setup->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>( NSysView::MakeSysViewServiceID(NodeId), TActorSetupCmd(actor.Release(), TMailboxType::HTSwap, appData->UserPoolId))); } diff --git a/ydb/core/driver_lib/run/kikimr_services_initializers.h b/ydb/core/driver_lib/run/kikimr_services_initializers.h index 34431e73a83..407ce1bb7ba 100644 --- a/ydb/core/driver_lib/run/kikimr_services_initializers.h +++ b/ydb/core/driver_lib/run/kikimr_services_initializers.h @@ -22,7 +22,7 @@ #include <library/cpp/actors/core/scheduler_actor.h> #include <library/cpp/actors/core/scheduler_basic.h> #include <library/cpp/actors/interconnect/poller_tcp.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <util/generic/vector.h> diff --git a/ydb/core/driver_lib/run/run.cpp b/ydb/core/driver_lib/run/run.cpp index affb4755542..a4f74aa4e0f 100644 --- a/ydb/core/driver_lib/run/run.cpp +++ b/ydb/core/driver_lib/run/run.cpp @@ -75,8 +75,8 @@ #include <ydb/core/client/server/msgbus_server_tracer.h> #include <ydb/core/client/server/http_ping.h> -#include <library/cpp/grpc/server/actors/logger.h> - +#include <library/cpp/grpc/server/actors/logger.h> + #include <ydb/services/yq/private_grpc.h> #include <ydb/services/cms/grpc_service.h> #include <ydb/services/datastreams/grpc_service.h> @@ -104,7 +104,7 @@ #include <ydb/core/yq/libs/init/init.h> #include <library/cpp/logger/global/global.h> -#include <library/cpp/monlib/messagebus/mon_messagebus.h> +#include <library/cpp/monlib/messagebus/mon_messagebus.h> #include <library/cpp/sighandler/async_signals_handler.h> #include <library/cpp/svnversion/svnversion.h> #include <library/cpp/malloc/api/malloc.h> @@ -113,7 +113,7 @@ #include <ydb/core/node_whiteboard/node_whiteboard.h> #include <ydb/core/tablet/node_tablet_monitor.h> - + #include <library/cpp/actors/core/memory_track.h> #include <library/cpp/actors/prof/tag.h> #include <ydb/library/yql/minikql/invoke_builtins/mkql_builtins.h> @@ -733,15 +733,15 @@ void TKikimrRunner::InitializeGRpc(const TKikimrRunConfig& runConfig) { const auto& grpcConfig = appConfig.GetGRpcConfig(); EnabledGrpcService = true; - NGrpc::TServerOptions opts; + NGrpc::TServerOptions opts; opts.SetHost(grpcConfig.GetHost()); opts.SetPort(grpcConfig.GetPort()); opts.SetWorkerThreads(grpcConfig.GetWorkerThreads()); opts.SetGRpcMemoryQuotaBytes(grpcConfig.GetGRpcMemoryQuotaBytes()); opts.SetMaxMessageSize(grpcConfig.HasMaxMessageSize() ? grpcConfig.GetMaxMessageSize() : DEFAULT_GRPC_MESSAGE_SIZE_LIMIT); opts.SetMaxGlobalRequestInFlight(grpcConfig.GetMaxInFlight()); - opts.SetLogger(NGrpc::CreateActorSystemLogger(*ActorSystem.Get(), NKikimrServices::GRPC_SERVER)); - + opts.SetLogger(NGrpc::CreateActorSystemLogger(*ActorSystem.Get(), NKikimrServices::GRPC_SERVER)); + if (appConfig.HasDomainsConfig() && appConfig.GetDomainsConfig().HasSecurityConfig() && appConfig.GetDomainsConfig().GetSecurityConfig().HasEnforceUserTokenRequirement()) { @@ -763,25 +763,25 @@ void TKikimrRunner::InitializeGRpc(const TKikimrRunConfig& runConfig) { } } - NGrpc::TServerOptions sslOpts = opts; + NGrpc::TServerOptions sslOpts = opts; if (grpcConfig.HasSslPort() && grpcConfig.GetSslPort()) { Y_VERIFY(grpcConfig.HasCA(), "CA not set"); Y_VERIFY(grpcConfig.HasCert(), "Cert not set"); Y_VERIFY(grpcConfig.HasKey(), "Key not set"); sslOpts.SetPort(grpcConfig.GetSslPort()); - NGrpc::TSslData sslData; + NGrpc::TSslData sslData; sslData.Root = ReadFile(grpcConfig.GetCA()); sslData.Cert = ReadFile(grpcConfig.GetCert()); sslData.Key = ReadFile(grpcConfig.GetKey()); sslOpts.SetSslData(sslData); - GRpcServers.push_back({ "grpcs", new NGrpc::TGRpcServer(sslOpts) }); + GRpcServers.push_back({ "grpcs", new NGrpc::TGRpcServer(sslOpts) }); fillFn(grpcConfig, *GRpcServers.back().second, sslOpts); } if (grpcConfig.GetPort()) { - GRpcServers.push_back({ "grpc", new NGrpc::TGRpcServer(opts) }); + GRpcServers.push_back({ "grpc", new NGrpc::TGRpcServer(opts) }); fillFn(grpcConfig, *GRpcServers.back().second, opts); } @@ -789,22 +789,22 @@ void TKikimrRunner::InitializeGRpc(const TKikimrRunConfig& runConfig) { for (auto &ex : grpcConfig.GetExtEndpoints()) { // todo: check uniq if (ex.GetPort()) { - NGrpc::TServerOptions xopts = opts; + NGrpc::TServerOptions xopts = opts; xopts.SetPort(ex.GetPort()); if (ex.GetHost()) xopts.SetHost(ex.GetHost()); - GRpcServers.push_back({ "grpc", new NGrpc::TGRpcServer(xopts) }); + GRpcServers.push_back({ "grpc", new NGrpc::TGRpcServer(xopts) }); fillFn(ex, *GRpcServers.back().second, xopts); } if (ex.HasSslPort() && ex.GetSslPort()) { - NGrpc::TServerOptions xopts = opts; + NGrpc::TServerOptions xopts = opts; xopts.SetPort(ex.GetSslPort()); - NGrpc::TSslData sslData; + NGrpc::TSslData sslData; sslData.Root = ex.HasCA() ? ReadFile(ex.GetCA()) : ReadFile(grpcConfig.GetCA()); sslData.Cert = ex.HasCert() ? ReadFile(ex.GetCert()) : ReadFile(grpcConfig.GetCert()); sslData.Key = ex.HasKey() ? ReadFile(ex.GetKey()) : ReadFile(grpcConfig.GetKey()); @@ -814,7 +814,7 @@ void TKikimrRunner::InitializeGRpc(const TKikimrRunConfig& runConfig) { Y_VERIFY(xopts.SslData->Cert, "Cert not set"); Y_VERIFY(xopts.SslData->Key, "Key not set"); - GRpcServers.push_back({ "grpcs", new NGrpc::TGRpcServer(xopts) }); + GRpcServers.push_back({ "grpcs", new NGrpc::TGRpcServer(xopts) }); fillFn(ex, *GRpcServers.back().second, xopts); } } @@ -951,7 +951,7 @@ void TKikimrRunner::InitializeLogSettings(const TKikimrRunConfig& runConfig) return; auto logConfig = runConfig.AppConfig.GetLogConfig(); - LogSettings.Reset(new NActors::NLog::TSettings(NActors::TActorId(runConfig.NodeId, "logger"), + LogSettings.Reset(new NActors::NLog::TSettings(NActors::TActorId(runConfig.NodeId, "logger"), NKikimrServices::LOGGER, (NActors::NLog::EPriority)logConfig.GetDefaultLevel(), (NActors::NLog::EPriority)logConfig.GetDefaultSamplingLevel(), @@ -1426,7 +1426,7 @@ void TKikimrRunner::KikimrStop(bool graceful) { } if (ActorSystem) { - ActorSystem->BroadcastToProxies([](const TActorId& proxyId) { + ActorSystem->BroadcastToProxies([](const TActorId& proxyId) { return new IEventHandle(proxyId, {}, new TEvInterconnect::TEvTerminate); }); ActorSystem->Send(new IEventHandle(MakeInterconnectListenerActorId(false), {}, new TEvents::TEvPoisonPill)); @@ -1466,8 +1466,8 @@ void TKikimrRunner::KikimrStop(bool graceful) { if (ActorSystem) { ActorSystem->Stop(); - } - + } + for (auto& server : GRpcServers) { server.second.Destroy(); } @@ -1523,7 +1523,7 @@ void TKikimrRunner::SetSignalHandlers() { } void TKikimrRunner::InitializeRegistries(const TKikimrRunConfig& runConfig) { - TypeRegistry.Reset(new NScheme::TKikimrTypeRegistry()); + TypeRegistry.Reset(new NScheme::TKikimrTypeRegistry()); TypeRegistry->CalculateMetadataEtag(); FunctionRegistry.Reset(NMiniKQL::CreateFunctionRegistry(NMiniKQL::CreateBuiltinRegistry())->Clone()); @@ -1531,7 +1531,7 @@ void TKikimrRunner::InitializeRegistries(const TKikimrRunConfig& runConfig) { #ifdef KIKIMR_UDF_DYNAMIC_LINK const TString& udfsDir = runConfig.AppConfig.GetUDFsDir(); - + TVector<TString> udfsPaths; if (!udfsDir.empty()) { if (NFs::Exists(udfsDir) && IsDir(udfsDir)) { diff --git a/ydb/core/driver_lib/run/run.h b/ydb/core/driver_lib/run/run.h index 0d2ebbb8b1a..dfd93a24e71 100644 --- a/ydb/core/driver_lib/run/run.h +++ b/ydb/core/driver_lib/run/run.h @@ -7,7 +7,7 @@ #include <library/cpp/actors/core/log_settings.h> #include <library/cpp/actors/interconnect/poller_tcp.h> #include <library/cpp/actors/util/should_continue.h> -#include <library/cpp/grpc/server/grpc_server.h> +#include <library/cpp/grpc/server/grpc_server.h> #include <ydb/core/base/appdata.h> #include <ydb/core/base/statestorage.h> #include <ydb/core/scheme_types/scheme_types.h> @@ -20,7 +20,7 @@ #include <ydb/core/ymq/http/http.h> #include <ydb/core/yq/libs/shared_resources/interface/shared_resources.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> namespace NKikimr { @@ -57,7 +57,7 @@ protected: TAutoPtr<NMsgBusProxy::IMessageBusServer> BusServer; TIntrusivePtr<NMonitoring::TBusNgMonPage> BusMonPage; - TVector<std::pair<TString, TAutoPtr<NGrpc::TGRpcServer>>> GRpcServers; + TVector<std::pair<TString, TAutoPtr<NGrpc::TGRpcServer>>> GRpcServers; TIntrusivePtr<NActors::NLog::TSettings> LogSettings; std::shared_ptr<TLogBackend> LogBackend; diff --git a/ydb/core/driver_lib/run/ya.make b/ydb/core/driver_lib/run/ya.make index b37af4736ec..f410ef42f20 100644 --- a/ydb/core/driver_lib/run/ya.make +++ b/ydb/core/driver_lib/run/ya.make @@ -53,12 +53,12 @@ PEERDIR( library/cpp/getopt/small library/cpp/grpc/client library/cpp/grpc/server - library/cpp/grpc/server/actors + library/cpp/grpc/server/actors library/cpp/logger library/cpp/malloc/api library/cpp/messagebus - library/cpp/monlib/dynamic_counters - library/cpp/monlib/messagebus + library/cpp/monlib/dynamic_counters + library/cpp/monlib/messagebus library/cpp/sighandler library/cpp/string_utils/parse_size library/cpp/svnversion diff --git a/ydb/core/engine/kikimr_program_builder.cpp b/ydb/core/engine/kikimr_program_builder.cpp index 8fc65b3b83a..fd32a772ee9 100644 --- a/ydb/core/engine/kikimr_program_builder.cpp +++ b/ydb/core/engine/kikimr_program_builder.cpp @@ -1,29 +1,29 @@ -#include "kikimr_program_builder.h" - +#include "kikimr_program_builder.h" + #include <ydb/public/lib/scheme_types/scheme_type_id.h> #include <ydb/library/yql/minikql/mkql_node_cast.h> #include <ydb/library/yql/minikql/mkql_node_visitor.h> #include <ydb/library/yql/minikql/mkql_node_printer.h> #include <ydb/library/yql/minikql/mkql_opt_literal.h> - -#include <array> - -namespace NKikimr { -namespace NMiniKQL { -namespace { - -TType* ValidateColumns( - const TArrayRef<const TSelectColumn>& columns, - TRuntimeNode& tags, + +#include <array> + +namespace NKikimr { +namespace NMiniKQL { +namespace { + +TType* ValidateColumns( + const TArrayRef<const TSelectColumn>& columns, + TRuntimeNode& tags, const TProgramBuilder* builder) -{ +{ TStructTypeBuilder rowTypeBuilder(builder->GetTypeEnvironment()); TStructLiteralBuilder tagsBuilder(builder->GetTypeEnvironment()); rowTypeBuilder.Reserve(columns.size()); tagsBuilder.Reserve(columns.size()); - for (auto& col : columns) { - MKQL_ENSURE(col.SchemeType != 0, "Null type is not allowed"); + for (auto& col : columns) { + MKQL_ENSURE(col.SchemeType != 0, "Null type is not allowed"); TDataType *dataType; if (col.SchemeType == NYql::NProto::TypeIds::Decimal) dataType = TDataDecimalType::Create( @@ -33,48 +33,48 @@ TType* ValidateColumns( else dataType = TDataType::Create(col.SchemeType, builder->GetTypeEnvironment()); auto optType = TOptionalType::Create(dataType, builder->GetTypeEnvironment()); - rowTypeBuilder.Add(col.Label, optType); + rowTypeBuilder.Add(col.Label, optType); tagsBuilder.Add(col.Label, builder->NewDataLiteral<ui32>(col.ColumnId)); - } - - tags = TRuntimeNode(tagsBuilder.Build(), true); - return rowTypeBuilder.Build(); -} - -} // namespace - -TReadRangeOptions::TReadRangeOptions(ui32 valueType, const TTypeEnvironment& env) + } + + tags = TRuntimeNode(tagsBuilder.Build(), true); + return rowTypeBuilder.Build(); +} + +} // namespace + +TReadRangeOptions::TReadRangeOptions(ui32 valueType, const TTypeEnvironment& env) : ItemsLimit(BuildDataLiteral(NUdf::TUnboxedValuePod((ui64)0), NUdf::EDataSlot::Uint64, env), true) , BytesLimit(BuildDataLiteral(NUdf::TUnboxedValuePod((ui64)0), NUdf::EDataSlot::Uint64, env), true) - , InitValue(BuildEmptyOptionalDataLiteral(valueType, env), true) - , TermValue(BuildEmptyOptionalDataLiteral(valueType, env), true) - , PayloadStruct(env.GetEmptyStruct(), true) + , InitValue(BuildEmptyOptionalDataLiteral(valueType, env), true) + , TermValue(BuildEmptyOptionalDataLiteral(valueType, env), true) + , PayloadStruct(env.GetEmptyStruct(), true) , Flags(BuildDataLiteral(NUdf::TUnboxedValuePod((ui32)TFlags::Default), NUdf::EDataSlot::Uint32, env), true) -{ -} - -TTableRangeOptions::TTableRangeOptions(const TTypeEnvironment& env) +{ +} + +TTableRangeOptions::TTableRangeOptions(const TTypeEnvironment& env) : ItemsLimit(BuildDataLiteral(NUdf::TUnboxedValuePod((ui64)0), NUdf::EDataSlot::Uint64, env), true) , BytesLimit(BuildDataLiteral(NUdf::TUnboxedValuePod((ui64)0), NUdf::EDataSlot::Uint64, env), true) , Flags(BuildDataLiteral(NUdf::TUnboxedValuePod((ui32)TReadRangeOptions::TFlags::Default), NUdf::EDataSlot::Uint32, env), true) , Reverse(BuildDataLiteral(NUdf::TUnboxedValuePod(false), NUdf::EDataSlot::Bool, env), true) -{ -} - -TParametersBuilder::TParametersBuilder(const TTypeEnvironment& env) - : StructBuilder(env) -{ -} - -TParametersBuilder& TParametersBuilder::Add(const TStringBuf& name, TRuntimeNode value) { - StructBuilder.Add(name, value); - return *this; -} - -TRuntimeNode TParametersBuilder::Build() { - return TRuntimeNode(StructBuilder.Build(), true); -} - +{ +} + +TParametersBuilder::TParametersBuilder(const TTypeEnvironment& env) + : StructBuilder(env) +{ +} + +TParametersBuilder& TParametersBuilder::Add(const TStringBuf& name, TRuntimeNode value) { + StructBuilder.Add(name, value); + return *this; +} + +TRuntimeNode TParametersBuilder::Build() { + return TRuntimeNode(StructBuilder.Build(), true); +} + static TRuntimeNode DoRewriteNullType( TRuntimeNode value, NUdf::TDataTypeId expectedType, @@ -91,92 +91,92 @@ static TRuntimeNode DoRewriteNullType( return value; } -TUpdateRowBuilder::TUpdateRowBuilder(const TTypeEnvironment& env) - : Builder(env) - , Env(env) -{ +TUpdateRowBuilder::TUpdateRowBuilder(const TTypeEnvironment& env) + : Builder(env) + , Env(env) +{ NullInternName = Env.InternName(TStringBuf("Null")); -} - -void TUpdateRowBuilder::SetColumn( - ui32 columnId, ui32 expectedType, - TRuntimeNode value) -{ - bool isOptional; +} + +void TUpdateRowBuilder::SetColumn( + ui32 columnId, ui32 expectedType, + TRuntimeNode value) +{ + bool isOptional; value = DoRewriteNullType(value, expectedType, NullInternName, Env); - TDataType* dataType = UnpackOptionalData(value, isOptional); - MKQL_ENSURE(expectedType == dataType->GetSchemeType(), - "Mismatch of column type expectedType = " << expectedType - << " actual type = " << dataType->GetSchemeType()); - Builder.Add(ToString(columnId), value); -} - -void TUpdateRowBuilder::InplaceUpdateColumn( - ui32 columnId, ui32 expectedType, - TRuntimeNode value, - EInplaceUpdateMode mode) -{ - MKQL_ENSURE(AS_TYPE(TDataType, value)->GetSchemeType() == expectedType, - "Mismatch of column type"); + TDataType* dataType = UnpackOptionalData(value, isOptional); + MKQL_ENSURE(expectedType == dataType->GetSchemeType(), + "Mismatch of column type expectedType = " << expectedType + << " actual type = " << dataType->GetSchemeType()); + Builder.Add(ToString(columnId), value); +} + +void TUpdateRowBuilder::InplaceUpdateColumn( + ui32 columnId, ui32 expectedType, + TRuntimeNode value, + EInplaceUpdateMode mode) +{ + MKQL_ENSURE(AS_TYPE(TDataType, value)->GetSchemeType() == expectedType, + "Mismatch of column type"); TDataType* byteType = TDataType::Create(NUdf::TDataType<ui8>::Id, Env); - std::array<TType*, 2> elements; - elements[0] = byteType; - elements[1] = value.GetStaticType(); - TTupleType* tupleType = TTupleType::Create(elements.size(), elements.data(), Env); - std::array<TRuntimeNode, 2> items; + std::array<TType*, 2> elements; + elements[0] = byteType; + elements[1] = value.GetStaticType(); + TTupleType* tupleType = TTupleType::Create(elements.size(), elements.data(), Env); + std::array<TRuntimeNode, 2> items; items[0] = TRuntimeNode(BuildDataLiteral(NUdf::TUnboxedValuePod((ui8)mode), NUdf::TDataType<ui8>::Id, Env), true); - items[1] = value; - TTupleLiteral* tuple = TTupleLiteral::Create( - items.size(), items.data(), tupleType, Env); - Builder.Add(ToString(columnId), TRuntimeNode(tuple, true)); -} - -void TUpdateRowBuilder::EraseColumn(ui32 columnId) -{ - Builder.Add(ToString(columnId), TRuntimeNode(Env.GetVoid(), true)); -} - -TRuntimeNode TUpdateRowBuilder::Build() -{ - return TRuntimeNode(Builder.Build(), true); -} - -const char* InplaceUpdateModeToCString(EInplaceUpdateMode mode) { - switch (mode) { - case EInplaceUpdateMode::Sum: - return "Sum"; - case EInplaceUpdateMode::Min: - return "Min"; - case EInplaceUpdateMode::Max: - return "Max"; - case EInplaceUpdateMode::IfNotExistOrEmpty: - return "IfNotExistOrEmpty"; - case EInplaceUpdateMode::Unknown: - case EInplaceUpdateMode::LastMode: - default: - return "Unknown"; - } -} - -EInplaceUpdateMode InplaceUpdateModeFromCString(const char* str) { - if (strcmp(str, "Sum") == 0) { - return EInplaceUpdateMode::Sum; - } else if (strcmp(str, "Min") == 0) { - return EInplaceUpdateMode::Min; - } else if (strcmp(str, "Max") == 0) { - return EInplaceUpdateMode::Max; - } else if (strcmp(str, "IfNotExistOrEmpty") == 0) { - return EInplaceUpdateMode::IfNotExistOrEmpty; - } else { - return EInplaceUpdateMode::Unknown; - } -} - -TKikimrProgramBuilder::TKikimrProgramBuilder( - const TTypeEnvironment& env, - const IFunctionRegistry& functionRegistry) + items[1] = value; + TTupleLiteral* tuple = TTupleLiteral::Create( + items.size(), items.data(), tupleType, Env); + Builder.Add(ToString(columnId), TRuntimeNode(tuple, true)); +} + +void TUpdateRowBuilder::EraseColumn(ui32 columnId) +{ + Builder.Add(ToString(columnId), TRuntimeNode(Env.GetVoid(), true)); +} + +TRuntimeNode TUpdateRowBuilder::Build() +{ + return TRuntimeNode(Builder.Build(), true); +} + +const char* InplaceUpdateModeToCString(EInplaceUpdateMode mode) { + switch (mode) { + case EInplaceUpdateMode::Sum: + return "Sum"; + case EInplaceUpdateMode::Min: + return "Min"; + case EInplaceUpdateMode::Max: + return "Max"; + case EInplaceUpdateMode::IfNotExistOrEmpty: + return "IfNotExistOrEmpty"; + case EInplaceUpdateMode::Unknown: + case EInplaceUpdateMode::LastMode: + default: + return "Unknown"; + } +} + +EInplaceUpdateMode InplaceUpdateModeFromCString(const char* str) { + if (strcmp(str, "Sum") == 0) { + return EInplaceUpdateMode::Sum; + } else if (strcmp(str, "Min") == 0) { + return EInplaceUpdateMode::Min; + } else if (strcmp(str, "Max") == 0) { + return EInplaceUpdateMode::Max; + } else if (strcmp(str, "IfNotExistOrEmpty") == 0) { + return EInplaceUpdateMode::IfNotExistOrEmpty; + } else { + return EInplaceUpdateMode::Unknown; + } +} + +TKikimrProgramBuilder::TKikimrProgramBuilder( + const TTypeEnvironment& env, + const IFunctionRegistry& functionRegistry) : TProgramBuilder(env, functionRegistry, true) -{ +{ UseNullType = false; NullInternName = Env.InternName(TStringBuf("Null")); @@ -185,8 +185,8 @@ TKikimrProgramBuilder::TKikimrProgramBuilder( tupleTypes[1] = NewDataType(NUdf::TDataType<ui64>::Id); tupleTypes[2] = NewDataType(NUdf::TDataType<ui64>::Id); TableIdType = TTupleType::Create(tupleTypes.size(), tupleTypes.data(), Env); -} - +} + TRuntimeNode TKikimrProgramBuilder::RewriteNullType( TRuntimeNode value, NUdf::TDataTypeId expectedType) const @@ -195,23 +195,23 @@ TRuntimeNode TKikimrProgramBuilder::RewriteNullType( } TVector<TRuntimeNode> TKikimrProgramBuilder::FixKeysType( - const TArrayRef<const ui32>& keyTypes, + const TArrayRef<const ui32>& keyTypes, const TKeyColumnValues& row) const -{ - MKQL_ENSURE(!row.empty(), "Expected at least 1 key column"); - MKQL_ENSURE(keyTypes.size() == row.size(), "Mismatch of key types count"); +{ + MKQL_ENSURE(!row.empty(), "Expected at least 1 key column"); + MKQL_ENSURE(keyTypes.size() == row.size(), "Mismatch of key types count"); TVector<TRuntimeNode> tmp(row.size()); - for (ui32 i = 0; i < row.size(); ++i) { + for (ui32 i = 0; i < row.size(); ++i) { tmp[i] = RewriteNullType(row[i], keyTypes[i]); - bool isOptional; + bool isOptional; TDataType* dataType = UnpackOptionalData(tmp[i], isOptional); - MKQL_ENSURE(keyTypes[i] == dataType->GetSchemeType(), + MKQL_ENSURE(keyTypes[i] == dataType->GetSchemeType(), "Mismatch of key column type, expected: " << keyTypes[i] << ", but got: " << dataType->GetSchemeType()); - } + } return tmp; } - + TRuntimeNode TKikimrProgramBuilder::ReadTarget(const TReadTarget& target) { MKQL_ENSURE(!target.HasSnapshotTime(), "Snapshots are not supported"); return NewDataLiteral((ui32)target.GetMode()); @@ -237,28 +237,28 @@ TRuntimeNode TKikimrProgramBuilder::SelectRow( auto rows = FixKeysType(keyTypes, row); - TRuntimeNode tags; + TRuntimeNode tags; auto rowType = ValidateColumns(columns, tags, this); - TType* optType = NewOptionalType(rowType); - - TCallableBuilder builder(Env, "SelectRow", optType); + TType* optType = NewOptionalType(rowType); + + TCallableBuilder builder(Env, "SelectRow", optType); builder.Add(BuildTableId(tableId)); - builder.Add(TRuntimeNode(rowType, true)); - builder.Add(tags); + builder.Add(TRuntimeNode(rowType, true)); + builder.Add(tags); builder.Add(NewTuple(TKeyColumnValues(rows))); builder.Add(readTarget); - - return TRuntimeNode(builder.Build(), false); -} - -TRuntimeNode TKikimrProgramBuilder::SelectRange( - const TTableId& tableId, - const TArrayRef<const ui32>& keyTypes, - const TArrayRef<const TSelectColumn>& columns, - const TTableRangeOptions& options, - const TReadTarget& target) -{ + + return TRuntimeNode(builder.Build(), false); +} + +TRuntimeNode TKikimrProgramBuilder::SelectRange( + const TTableId& tableId, + const TArrayRef<const ui32>& keyTypes, + const TArrayRef<const TSelectColumn>& columns, + const TTableRangeOptions& options, + const TReadTarget& target) +{ return SelectRange(tableId, keyTypes, columns, options, ReadTarget(target)); } @@ -289,71 +289,71 @@ TRuntimeNode TKikimrProgramBuilder::SelectRange( MKQL_ENSURE(AS_TYPE(TDataType, options.Flags)->GetSchemeType() == NUdf::TDataType<ui32>::Id, "Flags must be ui32"); MKQL_ENSURE(AS_TYPE(TDataType, options.ItemsLimit)->GetSchemeType() == NUdf::TDataType<ui64>::Id, "ItemsLimit must be ui64"); MKQL_ENSURE(AS_TYPE(TDataType, options.BytesLimit)->GetSchemeType() == NUdf::TDataType<ui64>::Id, "BytesLimit must be ui64"); - + if (options.Reverse) { MKQL_ENSURE(AS_TYPE(TDataType, options.Reverse)->GetSchemeType() == NUdf::TDataType<bool>::Id, "Reverse must be bool"); } - MKQL_ENSURE(options.FromColumns.size() > 0, "Expected at least one key component in the 'from' section of the range"); - const ui32 maxKeyColumnsCount = ::Max<ui32>(options.FromColumns.size(), options.ToColumns.size()); - MKQL_ENSURE(keyTypes.size() == maxKeyColumnsCount, "Mismatch of key types count"); + MKQL_ENSURE(options.FromColumns.size() > 0, "Expected at least one key component in the 'from' section of the range"); + const ui32 maxKeyColumnsCount = ::Max<ui32>(options.FromColumns.size(), options.ToColumns.size()); + MKQL_ENSURE(keyTypes.size() == maxKeyColumnsCount, "Mismatch of key types count"); TVector<TRuntimeNode> from(options.FromColumns.size()); TVector<TRuntimeNode> to(options.ToColumns.size()); - for (ui32 i = 0; i < maxKeyColumnsCount; ++i) { - TDataType* dataFrom = nullptr; - TDataType* dataTo = nullptr; - if (i < options.FromColumns.size()) { + for (ui32 i = 0; i < maxKeyColumnsCount; ++i) { + TDataType* dataFrom = nullptr; + TDataType* dataTo = nullptr; + if (i < options.FromColumns.size()) { from[i] = RewriteNullType(options.FromColumns[i], keyTypes[i]); - bool isOptionalFrom; + bool isOptionalFrom; dataFrom = UnpackOptionalData(from[i], isOptionalFrom); - } - - if (i < options.ToColumns.size()) { + } + + if (i < options.ToColumns.size()) { to[i] = RewriteNullType(options.ToColumns[i], keyTypes[i]); - bool isOptionalTo; + bool isOptionalTo; dataTo = UnpackOptionalData(to[i], isOptionalTo); - } - + } + MKQL_ENSURE(dataFrom || dataTo, "Invalid key tuple values"); - if (dataFrom && dataTo) { - MKQL_ENSURE(dataFrom->IsSameType(*dataTo), "Data types for key column " << i + if (dataFrom && dataTo) { + MKQL_ENSURE(dataFrom->IsSameType(*dataTo), "Data types for key column " << i << " don't match From: " << NUdf::GetDataTypeInfo(*dataFrom->GetDataSlot()).Name << " To: " << NUdf::GetDataTypeInfo(*dataTo->GetDataSlot()).Name); - } - - ui32 dataType = dataFrom ? dataFrom->GetSchemeType() : dataTo->GetSchemeType(); - MKQL_ENSURE(keyTypes[i] == dataType, "Mismatch of key column " << i + } + + ui32 dataType = dataFrom ? dataFrom->GetSchemeType() : dataTo->GetSchemeType(); + MKQL_ENSURE(keyTypes[i] == dataType, "Mismatch of key column " << i << " type, expected: " << NUdf::GetDataTypeInfo(NUdf::GetDataSlot(keyTypes[i])).Name << ", but got: " << NUdf::GetDataTypeInfo(NUdf::GetDataSlot(dataType)).Name); - } - - TRuntimeNode tags; + } + + TRuntimeNode tags; auto rowType = ValidateColumns(columns, tags, this); - TType* listType = NewListType(rowType); + TType* listType = NewListType(rowType); TDataType* boolType = TDataType::Create(NUdf::TDataType<bool>::Id, Env); - + TRuntimeNode skipNullKeys = BuildOptionList(options.SkipNullKeys, *this); TRuntimeNode forbidNullArgsFrom = BuildOptionList(options.ForbidNullArgsFrom, *this); TRuntimeNode forbidNullArgsTo = BuildOptionList(options.ForbidNullArgsTo, *this); - TStructTypeBuilder returnTypeBuilder(Env); - returnTypeBuilder.Reserve(2); + TStructTypeBuilder returnTypeBuilder(Env); + returnTypeBuilder.Reserve(2); returnTypeBuilder.Add(TStringBuf("List"), listType); returnTypeBuilder.Add(TStringBuf("Truncated"), boolType); - auto returnType = returnTypeBuilder.Build(); - TCallableBuilder builder(Env, "SelectRange", returnType); + auto returnType = returnTypeBuilder.Build(); + TCallableBuilder builder(Env, "SelectRange", returnType); builder.Add(BuildTableId(tableId)); - builder.Add(TRuntimeNode(rowType, true)); - builder.Add(tags); + builder.Add(TRuntimeNode(rowType, true)); + builder.Add(tags); builder.Add(NewTuple(TKeyColumnValues(from))); builder.Add(NewTuple(TKeyColumnValues(to))); - builder.Add(options.Flags); - builder.Add(options.ItemsLimit); - builder.Add(options.BytesLimit); + builder.Add(options.Flags); + builder.Add(options.ItemsLimit); + builder.Add(options.BytesLimit); builder.Add(readTarget); builder.Add(skipNullKeys); builder.Add(options.Reverse); @@ -366,112 +366,112 @@ TRuntimeNode TKikimrProgramBuilder::SelectRange( builder.Add(forbidNullArgsTo); } - return TRuntimeNode(builder.Build(), false); -} - -TRuntimeNode TKikimrProgramBuilder::UpdateRow( - const TTableId& tableId, - const TArrayRef<const ui32>& keyTypes, - const TKeyColumnValues& row, - TUpdateRowBuilder& update) -{ + return TRuntimeNode(builder.Build(), false); +} + +TRuntimeNode TKikimrProgramBuilder::UpdateRow( + const TTableId& tableId, + const TArrayRef<const ui32>& keyTypes, + const TKeyColumnValues& row, + TUpdateRowBuilder& update) +{ auto rows = FixKeysType(keyTypes, row); - - TCallableBuilder builder(Env, "UpdateRow", Env.GetTypeOfVoid()); + + TCallableBuilder builder(Env, "UpdateRow", Env.GetTypeOfVoid()); builder.Add(BuildTableId(tableId)); builder.Add(NewTuple(TKeyColumnValues(rows))); - builder.Add(update.Build()); - return TRuntimeNode(builder.Build(), false); -} - -TRuntimeNode TKikimrProgramBuilder::EraseRow( - const TTableId& tableId, - const TArrayRef<const ui32>& keyTypes, - const TKeyColumnValues& row) -{ + builder.Add(update.Build()); + return TRuntimeNode(builder.Build(), false); +} + +TRuntimeNode TKikimrProgramBuilder::EraseRow( + const TTableId& tableId, + const TArrayRef<const ui32>& keyTypes, + const TKeyColumnValues& row) +{ auto rows = FixKeysType(keyTypes, row); - - TCallableBuilder builder(Env, "EraseRow", Env.GetTypeOfVoid()); + + TCallableBuilder builder(Env, "EraseRow", Env.GetTypeOfVoid()); builder.Add(BuildTableId(tableId)); builder.Add(NewTuple(TKeyColumnValues(rows))); - return TRuntimeNode(builder.Build(), false); -} - -TRuntimeNode TKikimrProgramBuilder::Prepare(TRuntimeNode listOfVoid) -{ - auto listType = listOfVoid.GetStaticType(); + return TRuntimeNode(builder.Build(), false); +} + +TRuntimeNode TKikimrProgramBuilder::Prepare(TRuntimeNode listOfVoid) +{ + auto listType = listOfVoid.GetStaticType(); AS_TYPE(TListType, listType); - - const auto& listDetailedType = static_cast<const TListType&>(*listType); - MKQL_ENSURE(listDetailedType.GetItemType()->IsVoid(), "Expected list of void"); - - return listOfVoid; -} - -TRuntimeNode TKikimrProgramBuilder::Parameter( - const TStringBuf& name, TType* type) -{ + + const auto& listDetailedType = static_cast<const TListType&>(*listType); + MKQL_ENSURE(listDetailedType.GetItemType()->IsVoid(), "Expected list of void"); + + return listOfVoid; +} + +TRuntimeNode TKikimrProgramBuilder::Parameter( + const TStringBuf& name, TType* type) +{ MKQL_ENSURE(!name.empty(), "Empty parameter name is not allowed"); - - TCallableBuilder callableBuilder(Env, "Parameter", type, true); - callableBuilder.SetTypePayload(name); - return TRuntimeNode(callableBuilder.Build(), false); -} - -TRuntimeNode TKikimrProgramBuilder::MapParameter( - TRuntimeNode list, - std::function<TRuntimeNode(TRuntimeNode item)> handler) -{ - MKQL_ENSURE(!list.IsImmediate() && list.GetNode()->GetType()->IsCallable(), "Expected Parameter callable"); - auto callable = static_cast<TCallable*>(list.GetNode()); - MKQL_ENSURE(callable->GetType()->GetName() == "Parameter", "Expected Parameter callable"); - - auto listType = list.GetStaticType(); + + TCallableBuilder callableBuilder(Env, "Parameter", type, true); + callableBuilder.SetTypePayload(name); + return TRuntimeNode(callableBuilder.Build(), false); +} + +TRuntimeNode TKikimrProgramBuilder::MapParameter( + TRuntimeNode list, + std::function<TRuntimeNode(TRuntimeNode item)> handler) +{ + MKQL_ENSURE(!list.IsImmediate() && list.GetNode()->GetType()->IsCallable(), "Expected Parameter callable"); + auto callable = static_cast<TCallable*>(list.GetNode()); + MKQL_ENSURE(callable->GetType()->GetName() == "Parameter", "Expected Parameter callable"); + + auto listType = list.GetStaticType(); AS_TYPE(TListType, listType); - - const auto& listDetailedType = static_cast<const TListType&>(*listType); - auto itemType = listDetailedType.GetItemType(); - ThrowIfListOfVoid(itemType); - + + const auto& listDetailedType = static_cast<const TListType&>(*listType); + auto itemType = listDetailedType.GetItemType(); + ThrowIfListOfVoid(itemType); + TRuntimeNode itemArg = Arg(itemType); - auto newItem = handler(itemArg); - - auto resultListType = TListType::Create(newItem.GetStaticType(), Env); - TCallableBuilder callableBuilder(Env, "MapParameter", resultListType); - callableBuilder.Add(list); - callableBuilder.Add(itemArg); - callableBuilder.Add(newItem); - return TRuntimeNode(callableBuilder.Build(), false); -} - -TRuntimeNode TKikimrProgramBuilder::FlatMapParameter( - TRuntimeNode list, - std::function<TRuntimeNode(TRuntimeNode item)> handler) -{ - MKQL_ENSURE(!list.IsImmediate() && list.GetNode()->GetType()->IsCallable(), "Expected Parameter callable"); - auto callable = static_cast<TCallable*>(list.GetNode()); - MKQL_ENSURE(callable->GetType()->GetName() == "Parameter", "Expected Parameter callable"); - - auto listType = list.GetStaticType(); + auto newItem = handler(itemArg); + + auto resultListType = TListType::Create(newItem.GetStaticType(), Env); + TCallableBuilder callableBuilder(Env, "MapParameter", resultListType); + callableBuilder.Add(list); + callableBuilder.Add(itemArg); + callableBuilder.Add(newItem); + return TRuntimeNode(callableBuilder.Build(), false); +} + +TRuntimeNode TKikimrProgramBuilder::FlatMapParameter( + TRuntimeNode list, + std::function<TRuntimeNode(TRuntimeNode item)> handler) +{ + MKQL_ENSURE(!list.IsImmediate() && list.GetNode()->GetType()->IsCallable(), "Expected Parameter callable"); + auto callable = static_cast<TCallable*>(list.GetNode()); + MKQL_ENSURE(callable->GetType()->GetName() == "Parameter", "Expected Parameter callable"); + + auto listType = list.GetStaticType(); AS_TYPE(TListType, listType); - - const auto& listDetailedType = static_cast<const TListType&>(*listType); - auto itemType = listDetailedType.GetItemType(); - ThrowIfListOfVoid(itemType); - + + const auto& listDetailedType = static_cast<const TListType&>(*listType); + auto itemType = listDetailedType.GetItemType(); + ThrowIfListOfVoid(itemType); + TRuntimeNode itemArg = Arg(itemType); - auto newItem = handler(itemArg); - auto outputListType = AS_TYPE(TListType, newItem); - - TCallableBuilder callableBuilder(Env, "FlatMapParameter", outputListType); - callableBuilder.Add(list); - callableBuilder.Add(itemArg); - callableBuilder.Add(newItem); - return TRuntimeNode(callableBuilder.Build(), false); -} - + auto newItem = handler(itemArg); + auto outputListType = AS_TYPE(TListType, newItem); + + TCallableBuilder callableBuilder(Env, "FlatMapParameter", outputListType); + callableBuilder.Add(list); + callableBuilder.Add(itemArg); + callableBuilder.Add(newItem); + return TRuntimeNode(callableBuilder.Build(), false); +} + TRuntimeNode TKikimrProgramBuilder::AcquireLocks(TRuntimeNode lockTxId) { MKQL_ENSURE(AS_TYPE(TDataType, lockTxId)->GetSchemeType() == NUdf::TDataType<ui64>::Id, "LockTxId must be ui64"); @@ -504,97 +504,97 @@ TRuntimeNode TKikimrProgramBuilder::PartialTake(TRuntimeNode list, TRuntimeNode return BuildTake("PartialTake", list, count); } -TRuntimeNode TKikimrProgramBuilder::Bind(TRuntimeNode program, TRuntimeNode parameters, ui32 bindFlags) { - auto listType = program.GetStaticType(); +TRuntimeNode TKikimrProgramBuilder::Bind(TRuntimeNode program, TRuntimeNode parameters, ui32 bindFlags) { + auto listType = program.GetStaticType(); AS_TYPE(TListType, listType); - - const auto& listDetailedType = static_cast<const TListType&>(*listType); - MKQL_ENSURE(listDetailedType.GetItemType()->IsVoid(), "Expected list of void"); - - MKQL_ENSURE(parameters.IsImmediate() && parameters.GetStaticType()->IsStruct(), "Expected struct"); - - auto& parametersStruct = static_cast<TStructLiteral&>(*parameters.GetNode()); - - { - TExploringNodeVisitor explorer; - explorer.Walk(program.GetNode(), Env); + + const auto& listDetailedType = static_cast<const TListType&>(*listType); + MKQL_ENSURE(listDetailedType.GetItemType()->IsVoid(), "Expected list of void"); + + MKQL_ENSURE(parameters.IsImmediate() && parameters.GetStaticType()->IsStruct(), "Expected struct"); + + auto& parametersStruct = static_cast<TStructLiteral&>(*parameters.GetNode()); + + { + TExploringNodeVisitor explorer; + explorer.Walk(program.GetNode(), Env); auto parameterFunc = Env.InternName(TStringBuf("Parameter")); auto mapParameterFunc = Env.InternName(TStringBuf("MapParameter")); auto flatMapParameterFunc = Env.InternName(TStringBuf("FlatMapParameter")); auto arg = Env.InternName(TStringBuf("Arg")); - for (auto node : explorer.GetNodes()) { - if (node->GetType()->GetKind() != TType::EKind::Callable) - continue; - - auto& callable = static_cast<TCallable&>(*node); - if (callable.HasResult()) - continue; - - auto callableName = callable.GetType()->GetNameStr(); - if (callableName == parameterFunc) { - MKQL_ENSURE(callable.GetType()->GetPayload(), "Expected payload"); - auto structObj = AS_VALUE(NMiniKQL::TStructLiteral, NMiniKQL::TRuntimeNode(callable.GetType()->GetPayload(), true)); - auto payloadIndex = 1; // fields: Args, Payload + for (auto node : explorer.GetNodes()) { + if (node->GetType()->GetKind() != TType::EKind::Callable) + continue; + + auto& callable = static_cast<TCallable&>(*node); + if (callable.HasResult()) + continue; + + auto callableName = callable.GetType()->GetNameStr(); + if (callableName == parameterFunc) { + MKQL_ENSURE(callable.GetType()->GetPayload(), "Expected payload"); + auto structObj = AS_VALUE(NMiniKQL::TStructLiteral, NMiniKQL::TRuntimeNode(callable.GetType()->GetPayload(), true)); + auto payloadIndex = 1; // fields: Args, Payload const TStringBuf parameterName(AS_VALUE(NMiniKQL::TDataLiteral, structObj->GetValue(payloadIndex))->AsValue().AsStringRef()); auto parameterIndex = parametersStruct.GetType()->FindMemberIndex(parameterName); MKQL_ENSURE(parameterIndex, "Missing value for parameter: " << parameterName); auto value = parametersStruct.GetValue(*parameterIndex); - MKQL_ENSURE(value.IsImmediate(), "Parameter value must be immediate"); + MKQL_ENSURE(value.IsImmediate(), "Parameter value must be immediate"); MKQL_ENSURE(value.GetStaticType()->IsSameType(*callable.GetType()->GetReturnType()), "Incorrect type for parameter " << parameterName << ", expected: " << PrintNode(callable.GetType()->GetReturnType(), true) << ", actual: " << PrintNode(value.GetStaticType(), true)); - callable.SetResult(value, Env); - } else if (callableName == mapParameterFunc || callableName == flatMapParameterFunc) { - MKQL_ENSURE(callable.GetInput(0).HasValue(), "Expected parameter value"); - auto value = callable.GetInput(0).GetValue(); - auto list = AS_VALUE(TListLiteral, TRuntimeNode(value, true)); - - // lambda item type - auto retItemType = callable.GetInput(2).GetStaticType(); - if (callableName == flatMapParameterFunc) { - retItemType = AS_TYPE(TListType, retItemType)->GetItemType(); - } - + callable.SetResult(value, Env); + } else if (callableName == mapParameterFunc || callableName == flatMapParameterFunc) { + MKQL_ENSURE(callable.GetInput(0).HasValue(), "Expected parameter value"); + auto value = callable.GetInput(0).GetValue(); + auto list = AS_VALUE(TListLiteral, TRuntimeNode(value, true)); + + // lambda item type + auto retItemType = callable.GetInput(2).GetStaticType(); + if (callableName == flatMapParameterFunc) { + retItemType = AS_TYPE(TListType, retItemType)->GetItemType(); + } + TVector<TRuntimeNode> mapResults; mapResults.reserve(list->GetItemsCount()); - - TExploringNodeVisitor explorer; - auto lambdaArgNode = callable.GetInput(1).GetNode(); - auto lambdaRootNode = callable.GetInput(2); - explorer.Walk(lambdaRootNode.GetNode(), Env); + + TExploringNodeVisitor explorer; + auto lambdaArgNode = callable.GetInput(1).GetNode(); + auto lambdaRootNode = callable.GetInput(2); + explorer.Walk(lambdaRootNode.GetNode(), Env); for (ui32 i = 0; i < list->GetItemsCount(); ++i) { auto itemValue = list->GetItems()[i]; - bool wereChanges; - auto newValue = SinglePassVisitCallables(lambdaRootNode, explorer, + bool wereChanges; + auto newValue = SinglePassVisitCallables(lambdaRootNode, explorer, [&](TInternName name) { - Y_UNUSED(name); - return [&](TCallable& callable, const TTypeEnvironment& env) { - Y_UNUSED(env); - if (&callable == lambdaArgNode) { - return itemValue; - } - - if (callable.GetType()->GetNameStr() == arg) { + Y_UNUSED(name); + return [&](TCallable& callable, const TTypeEnvironment& env) { + Y_UNUSED(env); + if (&callable == lambdaArgNode) { + return itemValue; + } + + if (callable.GetType()->GetNameStr() == arg) { TCallableBuilder itemCallableBuilder(Env, arg.Str(), callable.GetType()->GetReturnType(), true); - TRuntimeNode itemArg(itemCallableBuilder.Build(), false); - return itemArg; - } - - return TRuntimeNode(&callable, false); - }; - }, Env, false, wereChanges); - + TRuntimeNode itemArg(itemCallableBuilder.Build(), false); + return itemArg; + } + + return TRuntimeNode(&callable, false); + }; + }, Env, false, wereChanges); + mapResults.push_back(newValue); - } - + } + TRuntimeNode res; TListLiteralBuilder listBuilder(Env, retItemType); - if (callableName == mapParameterFunc) { + if (callableName == mapParameterFunc) { for (auto& node : mapResults) { listBuilder.Add(node); } @@ -603,32 +603,32 @@ TRuntimeNode TKikimrProgramBuilder::Bind(TRuntimeNode program, TRuntimeNode para res = mapResults.empty() ? TRuntimeNode(listBuilder.Build(), true) : Extend(mapResults); - } - - callable.SetResult(res, Env); - } - } - - for (auto node : explorer.GetNodes()) { - node->Freeze(Env); - } - } - - program.Freeze(); - if (bindFlags & TBindFlags::OptimizeLiterals) { - program = LiteralPropagationOptimization(program, Env, true); - } - - return program; -} - -TRuntimeNode TKikimrProgramBuilder::Build(TRuntimeNode listOfVoid, ui32 bindFlags) -{ - auto program = Prepare(listOfVoid); - TParametersBuilder parametersBuilder(Env); - return Bind(program, parametersBuilder.Build(), bindFlags); -} - + } + + callable.SetResult(res, Env); + } + } + + for (auto node : explorer.GetNodes()) { + node->Freeze(Env); + } + } + + program.Freeze(); + if (bindFlags & TBindFlags::OptimizeLiterals) { + program = LiteralPropagationOptimization(program, Env, true); + } + + return program; +} + +TRuntimeNode TKikimrProgramBuilder::Build(TRuntimeNode listOfVoid, ui32 bindFlags) +{ + auto program = Prepare(listOfVoid); + TParametersBuilder parametersBuilder(Env); + return Bind(program, parametersBuilder.Build(), bindFlags); +} + TRuntimeNode TKikimrProgramBuilder::Abort() { TCallableBuilder callableBuilder(Env, "Abort", Env.GetVoid()->GetGenericType()); return TRuntimeNode(callableBuilder.Build(), false); @@ -671,5 +671,5 @@ TRuntimeNode TKikimrProgramBuilder::BuildTableId(const TTableId& tableId) const } } -} // namespace NMiniKQL -} // namespace NKikimr +} // namespace NMiniKQL +} // namespace NKikimr diff --git a/ydb/core/engine/kikimr_program_builder.h b/ydb/core/engine/kikimr_program_builder.h index 170cfd3aeaf..717132c0b96 100644 --- a/ydb/core/engine/kikimr_program_builder.h +++ b/ydb/core/engine/kikimr_program_builder.h @@ -1,13 +1,13 @@ -#pragma once - +#pragma once + #include <ydb/library/yql/minikql/mkql_node.h> #include <ydb/core/scheme/scheme_tabledefs.h> #include <ydb/library/yql/minikql/mkql_program_builder.h> - - -namespace NKikimr { -namespace NMiniKQL { - + + +namespace NKikimr { +namespace NMiniKQL { + static constexpr NUdf::TDataTypeId LegacyPairUi64Ui64 = 0x101; struct TSelectColumn { @@ -25,123 +25,123 @@ struct TSelectColumn { {} }; -struct TReadRangeOptions -{ - struct TFlags { - enum : ui32 { - IncludeInitValue = 0x00, - ExcludeInitValue = 0x01, - - IncludeTermValue = 0x00, - ExcludeTermValue = 0x02, - - IncludeBoth = IncludeInitValue | IncludeTermValue, - ExcludeBoth = ExcludeInitValue | ExcludeTermValue, - Default = IncludeBoth - }; - }; - - TReadRangeOptions(ui32 valueType, const TTypeEnvironment& env); - - TRuntimeNode ItemsLimit; - TRuntimeNode BytesLimit; - TRuntimeNode InitValue; - TRuntimeNode TermValue; - TRuntimeNode PayloadStruct; - TRuntimeNode Flags; -}; - -// Supported types - Data or Optional of Data for each column -using TKeyColumnValues = TArrayRef<const TRuntimeNode>; - -struct TTableRangeOptions -{ - TTableRangeOptions(const TTypeEnvironment& env); - - TRuntimeNode ItemsLimit; // default = ui64 0 - TRuntimeNode BytesLimit; // default = ui64 0 - TRuntimeNode Flags; // default = ui64 IncludeInitValue | IncludeTermValue - TKeyColumnValues FromColumns; - TKeyColumnValues ToColumns; +struct TReadRangeOptions +{ + struct TFlags { + enum : ui32 { + IncludeInitValue = 0x00, + ExcludeInitValue = 0x01, + + IncludeTermValue = 0x00, + ExcludeTermValue = 0x02, + + IncludeBoth = IncludeInitValue | IncludeTermValue, + ExcludeBoth = ExcludeInitValue | ExcludeTermValue, + Default = IncludeBoth + }; + }; + + TReadRangeOptions(ui32 valueType, const TTypeEnvironment& env); + + TRuntimeNode ItemsLimit; + TRuntimeNode BytesLimit; + TRuntimeNode InitValue; + TRuntimeNode TermValue; + TRuntimeNode PayloadStruct; + TRuntimeNode Flags; +}; + +// Supported types - Data or Optional of Data for each column +using TKeyColumnValues = TArrayRef<const TRuntimeNode>; + +struct TTableRangeOptions +{ + TTableRangeOptions(const TTypeEnvironment& env); + + TRuntimeNode ItemsLimit; // default = ui64 0 + TRuntimeNode BytesLimit; // default = ui64 0 + TRuntimeNode Flags; // default = ui64 IncludeInitValue | IncludeTermValue + TKeyColumnValues FromColumns; + TKeyColumnValues ToColumns; TArrayRef<bool> SkipNullKeys; TArrayRef<bool> ForbidNullArgsFrom; TArrayRef<bool> ForbidNullArgsTo; TRuntimeNode Reverse; // default = <unset> -}; - -enum class EInplaceUpdateMode -{ - Unknown = 0, - FirstMode = 1, - Sum = FirstMode, - Min = 2, - Max = 3, - IfNotExistOrEmpty = 4, - LastMode -}; - -const char* InplaceUpdateModeToCString(EInplaceUpdateMode mode); -EInplaceUpdateMode InplaceUpdateModeFromCString(const char* str); - -class TUpdateRowBuilder -{ -public: - TUpdateRowBuilder(const TTypeEnvironment& env); - TUpdateRowBuilder(const TUpdateRowBuilder&) = default; - TUpdateRowBuilder& operator=(const TUpdateRowBuilder&) = default; - // Supports Data or Optional of Data - void SetColumn(ui32 columnId, ui32 expectedType, TRuntimeNode value); - // Supports Data - void InplaceUpdateColumn( - ui32 columnId, ui32 expectedType, - TRuntimeNode value, EInplaceUpdateMode mode); - void EraseColumn(ui32 columnId); - TRuntimeNode Build(); - -private: - TStructLiteralBuilder Builder; - const TTypeEnvironment& Env; +}; + +enum class EInplaceUpdateMode +{ + Unknown = 0, + FirstMode = 1, + Sum = FirstMode, + Min = 2, + Max = 3, + IfNotExistOrEmpty = 4, + LastMode +}; + +const char* InplaceUpdateModeToCString(EInplaceUpdateMode mode); +EInplaceUpdateMode InplaceUpdateModeFromCString(const char* str); + +class TUpdateRowBuilder +{ +public: + TUpdateRowBuilder(const TTypeEnvironment& env); + TUpdateRowBuilder(const TUpdateRowBuilder&) = default; + TUpdateRowBuilder& operator=(const TUpdateRowBuilder&) = default; + // Supports Data or Optional of Data + void SetColumn(ui32 columnId, ui32 expectedType, TRuntimeNode value); + // Supports Data + void InplaceUpdateColumn( + ui32 columnId, ui32 expectedType, + TRuntimeNode value, EInplaceUpdateMode mode); + void EraseColumn(ui32 columnId); + TRuntimeNode Build(); + +private: + TStructLiteralBuilder Builder; + const TTypeEnvironment& Env; TInternName NullInternName; -}; - -class TParametersBuilder -{ -public: - TParametersBuilder(const TTypeEnvironment& env); - TParametersBuilder(const TParametersBuilder&) = default; - TParametersBuilder& operator=(const TParametersBuilder&) = default; - TParametersBuilder& Add(const TStringBuf& name, TRuntimeNode value); - TRuntimeNode Build(); - -private: - TStructLiteralBuilder StructBuilder; -}; - -class TKikimrProgramBuilder: public TProgramBuilder -{ -public: - struct TBindFlags { - enum { +}; + +class TParametersBuilder +{ +public: + TParametersBuilder(const TTypeEnvironment& env); + TParametersBuilder(const TParametersBuilder&) = default; + TParametersBuilder& operator=(const TParametersBuilder&) = default; + TParametersBuilder& Add(const TStringBuf& name, TRuntimeNode value); + TRuntimeNode Build(); + +private: + TStructLiteralBuilder StructBuilder; +}; + +class TKikimrProgramBuilder: public TProgramBuilder +{ +public: + struct TBindFlags { + enum { OptimizeLiterals = 0x1, Default = OptimizeLiterals, - DisableOptimization = 0 - }; - }; - - TKikimrProgramBuilder( - const TTypeEnvironment& env, - const IFunctionRegistry& functionRegistry); - - //-- table db functions - // row should contain only static nodes, columns - structType - // returns Optional of structType - TRuntimeNode SelectRow( - const TTableId& tableId, - const TArrayRef<const ui32>& keyTypes, - const TArrayRef<const TSelectColumn>& columns, - const TKeyColumnValues& row, - const TReadTarget& target = TReadTarget()); - + DisableOptimization = 0 + }; + }; + + TKikimrProgramBuilder( + const TTypeEnvironment& env, + const IFunctionRegistry& functionRegistry); + + //-- table db functions + // row should contain only static nodes, columns - structType + // returns Optional of structType + TRuntimeNode SelectRow( + const TTableId& tableId, + const TArrayRef<const ui32>& keyTypes, + const TArrayRef<const TSelectColumn>& columns, + const TKeyColumnValues& row, + const TReadTarget& target = TReadTarget()); + TRuntimeNode SelectRow( const TTableId& tableId, const TArrayRef<const ui32>& keyTypes, @@ -149,19 +149,19 @@ public: const TKeyColumnValues& row, TRuntimeNode readTarget); - TTableRangeOptions GetDefaultTableRangeOptions() const { - return TTableRangeOptions(Env); - } - - // from/to should contain only static nodes, columns - structType - // returns Struct with fields { List: List of structType, Truncated: Bool } - TRuntimeNode SelectRange( - const TTableId& tableId, - const TArrayRef<const ui32>& keyTypes, - const TArrayRef<const TSelectColumn>& columns, - const TTableRangeOptions& options, - const TReadTarget& target = TReadTarget()); - + TTableRangeOptions GetDefaultTableRangeOptions() const { + return TTableRangeOptions(Env); + } + + // from/to should contain only static nodes, columns - structType + // returns Struct with fields { List: List of structType, Truncated: Bool } + TRuntimeNode SelectRange( + const TTableId& tableId, + const TArrayRef<const ui32>& keyTypes, + const TArrayRef<const TSelectColumn>& columns, + const TTableRangeOptions& options, + const TReadTarget& target = TReadTarget()); + TRuntimeNode SelectRange( const TTableId& tableId, const TArrayRef<const ui32>& keyTypes, @@ -169,36 +169,36 @@ public: const TTableRangeOptions& options, TRuntimeNode readTarget); - TUpdateRowBuilder GetUpdateRowBuilder() const { - return TUpdateRowBuilder(Env); - } - - // returns Void - TRuntimeNode UpdateRow( - const TTableId& tableId, - const TArrayRef<const ui32>& keyTypes, - const TKeyColumnValues& row, - TUpdateRowBuilder& update); - - // returns Void - TRuntimeNode EraseRow( - const TTableId& tableId, - const TArrayRef<const ui32>& keyTypes, - const TKeyColumnValues& row); - - //-- parameters functions - TRuntimeNode Prepare(TRuntimeNode listOfVoid); - - TRuntimeNode Parameter(const TStringBuf& name, TType* type); - - TRuntimeNode MapParameter( - TRuntimeNode list, // must be a parameter - std::function<TRuntimeNode(TRuntimeNode item)> handler); - - TRuntimeNode FlatMapParameter( - TRuntimeNode list, - std::function<TRuntimeNode(TRuntimeNode item)> handler); - + TUpdateRowBuilder GetUpdateRowBuilder() const { + return TUpdateRowBuilder(Env); + } + + // returns Void + TRuntimeNode UpdateRow( + const TTableId& tableId, + const TArrayRef<const ui32>& keyTypes, + const TKeyColumnValues& row, + TUpdateRowBuilder& update); + + // returns Void + TRuntimeNode EraseRow( + const TTableId& tableId, + const TArrayRef<const ui32>& keyTypes, + const TKeyColumnValues& row); + + //-- parameters functions + TRuntimeNode Prepare(TRuntimeNode listOfVoid); + + TRuntimeNode Parameter(const TStringBuf& name, TType* type); + + TRuntimeNode MapParameter( + TRuntimeNode list, // must be a parameter + std::function<TRuntimeNode(TRuntimeNode item)> handler); + + TRuntimeNode FlatMapParameter( + TRuntimeNode list, + std::function<TRuntimeNode(TRuntimeNode item)> handler); + TRuntimeNode AcquireLocks(TRuntimeNode lockTxId); TRuntimeNode ReadTarget(const TReadTarget& target); @@ -212,18 +212,18 @@ public: TRuntimeNode PartialTake(TRuntimeNode list, TRuntimeNode count); - TRuntimeNode Bind( - TRuntimeNode program, - TRuntimeNode parameters, - ui32 bindFlags = TBindFlags::Default); - - TRuntimeNode Build( - TRuntimeNode listOfVoid, - ui32 bindFlags = TBindFlags::Default); - - TParametersBuilder GetParametersBuilder() const { - return TParametersBuilder(Env); - } + TRuntimeNode Bind( + TRuntimeNode program, + TRuntimeNode parameters, + ui32 bindFlags = TBindFlags::Default); + + TRuntimeNode Build( + TRuntimeNode listOfVoid, + ui32 bindFlags = TBindFlags::Default); + + TParametersBuilder GetParametersBuilder() const { + return TParametersBuilder(Env); + } //-- special functions TRuntimeNode Abort(); @@ -242,8 +242,8 @@ private: NUdf::TDataTypeId expectedType) const; TInternName NullInternName; TTupleType* TableIdType; -}; - -} // namespace NMiniKQL -} // namespace NKikimr - +}; + +} // namespace NMiniKQL +} // namespace NKikimr + diff --git a/ydb/core/engine/kikimr_program_builder_ut.cpp b/ydb/core/engine/kikimr_program_builder_ut.cpp index ba333b6ee3d..efa073ae2f8 100644 --- a/ydb/core/engine/kikimr_program_builder_ut.cpp +++ b/ydb/core/engine/kikimr_program_builder_ut.cpp @@ -1,5 +1,5 @@ -#include "kikimr_program_builder.h" - +#include "kikimr_program_builder.h" + #include <ydb/library/yql/minikql/mkql_node_printer.h> #include <ydb/library/yql/minikql/mkql_node_serialization.h> #include <ydb/library/yql/minikql/mkql_node_visitor.h> @@ -11,15 +11,15 @@ #include <ydb/library/yql/minikql/comp_nodes/mkql_factories.h> #include <ydb/core/engine/mkql_keys.h> #include <ydb/core/scheme_types/scheme_types_defs.h> - + #include <library/cpp/testing/unittest/registar.h> - -namespace NKikimr { -namespace NMiniKQL { + +namespace NKikimr { +namespace NMiniKQL { TIntrusivePtr<IRandomProvider> CreateRandomProvider() { return CreateDeterministicRandomProvider(1); } - + TIntrusivePtr<ITimeProvider> CreateTimeProvider() { return CreateDeterministicTimeProvider(1); } @@ -66,622 +66,622 @@ namespace NMiniKQL { }; Y_UNIT_TEST_SUITE(TMiniKQLProgramBuilderTest) { - - void VerifySerialization(TNode* pgm, const TTypeEnvironment& env) { + + void VerifySerialization(TNode* pgm, const TTypeEnvironment& env) { TString s = PrintNode(pgm); TString serialized = SerializeNode(pgm, env); Cout << "Serialized as " << serialized.size() << " bytes" << Endl; - TNode* pgm2 = DeserializeNode(serialized, env); + TNode* pgm2 = DeserializeNode(serialized, env); TString s2 = PrintNode(pgm2); - UNIT_ASSERT_EQUAL(s, s2); - } - - void VerifyProgram(TNode* pgm, const TTypeEnvironment& env) { - VerifySerialization(pgm, env); - } - + UNIT_ASSERT_EQUAL(s, s2); + } + + void VerifyProgram(TNode* pgm, const TTypeEnvironment& env) { + VerifySerialization(pgm, env); + } + Y_UNIT_TEST(TestEraseRowStaticKey) { TScopedAlloc alloc; TTypeEnvironment env(alloc); auto functionRegistry = CreateFunctionRegistry(CreateBuiltinRegistry()); TKikimrProgramBuilder pgmBuilder(env, *functionRegistry); - auto pgmReturn = pgmBuilder.NewEmptyListOfVoid(); + auto pgmReturn = pgmBuilder.NewEmptyListOfVoid(); TVector<TRuntimeNode> keyColumns; keyColumns.push_back(pgmBuilder.TProgramBuilder::NewDataLiteral<ui32>(42)); keyColumns.push_back(pgmBuilder.NewEmptyOptionalDataLiteral(NUdf::TDataType<ui64>::Id)); - keyColumns.push_back(pgmBuilder.NewOptional( + keyColumns.push_back(pgmBuilder.NewOptional( pgmBuilder.NewDataLiteral<NUdf::EDataSlot::String>("qwe"))); TVector<ui32> keyTypes({ NUdf::TDataType<ui32>::Id, NUdf::TDataType<ui64>::Id, NUdf::TDataType<char*>::Id }); - pgmReturn = pgmBuilder.Append(pgmReturn, pgmBuilder.EraseRow(TTableId(1, 2), - keyTypes, - keyColumns)); - auto pgm = pgmBuilder.Build(pgmReturn, TKikimrProgramBuilder::TBindFlags::DisableOptimization).GetNode(); - //Cout << PrintNode(*pgm) << Endl; - VerifyProgram(pgm, env); - - TExploringNodeVisitor explorer; - explorer.Walk(pgm, env); + pgmReturn = pgmBuilder.Append(pgmReturn, pgmBuilder.EraseRow(TTableId(1, 2), + keyTypes, + keyColumns)); + auto pgm = pgmBuilder.Build(pgmReturn, TKikimrProgramBuilder::TBindFlags::DisableOptimization).GetNode(); + //Cout << PrintNode(*pgm) << Endl; + VerifyProgram(pgm, env); + + TExploringNodeVisitor explorer; + explorer.Walk(pgm, env); TVector<THolder<TKeyDesc>> tableKeys = ExtractTableKeys(explorer, env); - UNIT_ASSERT_VALUES_EQUAL(tableKeys.size(), 1); + UNIT_ASSERT_VALUES_EQUAL(tableKeys.size(), 1); UNIT_ASSERT(tableKeys[0]->TableId.HasSamePath(TTableId(1, 2))); - UNIT_ASSERT(tableKeys[0]->RowOperation == TKeyDesc::ERowOperation::Erase); - UNIT_ASSERT(tableKeys[0]->Range.InclusiveFrom); - UNIT_ASSERT(tableKeys[0]->Range.InclusiveTo); - UNIT_ASSERT(tableKeys[0]->Range.Point); + UNIT_ASSERT(tableKeys[0]->RowOperation == TKeyDesc::ERowOperation::Erase); + UNIT_ASSERT(tableKeys[0]->Range.InclusiveFrom); + UNIT_ASSERT(tableKeys[0]->Range.InclusiveTo); + UNIT_ASSERT(tableKeys[0]->Range.Point); UNIT_ASSERT(tableKeys[0]->Range.From.size() == 3); - UNIT_ASSERT(tableKeys[0]->Range.From[0].Size() == 4); - UNIT_ASSERT(tableKeys[0]->Range.From[1].IsNull()); - UNIT_ASSERT(tableKeys[0]->Range.From[2].Size() == 3); + UNIT_ASSERT(tableKeys[0]->Range.From[0].Size() == 4); + UNIT_ASSERT(tableKeys[0]->Range.From[1].IsNull()); + UNIT_ASSERT(tableKeys[0]->Range.From[2].Size() == 3); UNIT_ASSERT(tableKeys[0]->Range.To.size() == 3); - UNIT_ASSERT(tableKeys[0]->Range.To[0].Size() == 4); - UNIT_ASSERT(tableKeys[0]->Range.To[1].IsNull()); - UNIT_ASSERT(tableKeys[0]->Range.To[2].Size() == 3); - UNIT_ASSERT(tableKeys[0]->Columns.size() == 0); - UNIT_ASSERT(tableKeys[0]->KeyColumnTypes.size() == 3); + UNIT_ASSERT(tableKeys[0]->Range.To[0].Size() == 4); + UNIT_ASSERT(tableKeys[0]->Range.To[1].IsNull()); + UNIT_ASSERT(tableKeys[0]->Range.To[2].Size() == 3); + UNIT_ASSERT(tableKeys[0]->Columns.size() == 0); + UNIT_ASSERT(tableKeys[0]->KeyColumnTypes.size() == 3); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[0] == NUdf::TDataType<ui32>::Id); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[1] == NUdf::TDataType<ui64>::Id); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[2] == NUdf::TDataType<char*>::Id); - } - + } + Y_UNIT_TEST(TestEraseRowPartialDynamicKey) { TScopedAlloc alloc; TTypeEnvironment env(alloc); auto functionRegistry = CreateFunctionRegistry(CreateBuiltinRegistry()); TKikimrProgramBuilder pgmBuilder(env, *functionRegistry); - auto pgmReturn = pgmBuilder.NewEmptyListOfVoid(); + auto pgmReturn = pgmBuilder.NewEmptyListOfVoid(); TVector<TRuntimeNode> keyColumns; keyColumns.push_back(pgmBuilder.NewEmptyOptionalDataLiteral(NUdf::TDataType<ui64>::Id)); - keyColumns.push_back(pgmBuilder.Add( + keyColumns.push_back(pgmBuilder.Add( pgmBuilder.TProgramBuilder::NewDataLiteral<ui32>(34), pgmBuilder.TProgramBuilder::NewDataLiteral<ui32>(12))); keyColumns.push_back(pgmBuilder.NewDataLiteral<NUdf::EDataSlot::String>("qwe")); TVector<ui32> keyTypes({ NUdf::TDataType<ui64>::Id, NUdf::TDataType<ui32>::Id, NUdf::TDataType<char*>::Id }); - pgmReturn = pgmBuilder.Append(pgmReturn, pgmBuilder.EraseRow(TTableId(1, 2), - keyTypes, - keyColumns)); - auto pgm = pgmBuilder.Build(pgmReturn, TKikimrProgramBuilder::TBindFlags::DisableOptimization).GetNode(); - //Cout << PrintNode(*pgm) << Endl; - VerifyProgram(pgm, env); - - TExploringNodeVisitor explorer; - explorer.Walk(pgm, env); + pgmReturn = pgmBuilder.Append(pgmReturn, pgmBuilder.EraseRow(TTableId(1, 2), + keyTypes, + keyColumns)); + auto pgm = pgmBuilder.Build(pgmReturn, TKikimrProgramBuilder::TBindFlags::DisableOptimization).GetNode(); + //Cout << PrintNode(*pgm) << Endl; + VerifyProgram(pgm, env); + + TExploringNodeVisitor explorer; + explorer.Walk(pgm, env); TVector<THolder<TKeyDesc>> tableKeys = ExtractTableKeys(explorer, env); - UNIT_ASSERT_VALUES_EQUAL(tableKeys.size(), 1); + UNIT_ASSERT_VALUES_EQUAL(tableKeys.size(), 1); UNIT_ASSERT(tableKeys[0]->TableId.HasSamePath(TTableId(1, 2))); - UNIT_ASSERT(tableKeys[0]->RowOperation == TKeyDesc::ERowOperation::Erase); - UNIT_ASSERT(tableKeys[0]->Range.InclusiveFrom); - UNIT_ASSERT(!tableKeys[0]->Range.InclusiveTo); - UNIT_ASSERT(!tableKeys[0]->Range.Point); + UNIT_ASSERT(tableKeys[0]->RowOperation == TKeyDesc::ERowOperation::Erase); + UNIT_ASSERT(tableKeys[0]->Range.InclusiveFrom); + UNIT_ASSERT(!tableKeys[0]->Range.InclusiveTo); + UNIT_ASSERT(!tableKeys[0]->Range.Point); UNIT_ASSERT(tableKeys[0]->Range.From.size() == 3); - UNIT_ASSERT(tableKeys[0]->Range.From[0].IsNull()); - UNIT_ASSERT(tableKeys[0]->Range.From[1].IsNull()); - UNIT_ASSERT(tableKeys[0]->Range.From[2].IsNull()); + UNIT_ASSERT(tableKeys[0]->Range.From[0].IsNull()); + UNIT_ASSERT(tableKeys[0]->Range.From[1].IsNull()); + UNIT_ASSERT(tableKeys[0]->Range.From[2].IsNull()); UNIT_ASSERT(tableKeys[0]->Range.To.size() == 1); - UNIT_ASSERT(tableKeys[0]->Range.To[0].IsNull()); - UNIT_ASSERT(tableKeys[0]->Columns.size() == 0); - UNIT_ASSERT(tableKeys[0]->KeyColumnTypes.size() == 3); + UNIT_ASSERT(tableKeys[0]->Range.To[0].IsNull()); + UNIT_ASSERT(tableKeys[0]->Columns.size() == 0); + UNIT_ASSERT(tableKeys[0]->KeyColumnTypes.size() == 3); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[0] == NUdf::TDataType<ui64>::Id); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[1] == NUdf::TDataType<ui32>::Id); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[2] == NUdf::TDataType<char*>::Id); - } - + } + Y_UNIT_TEST(TestEraseRowDynamicKey) { TScopedAlloc alloc; TTypeEnvironment env(alloc); auto functionRegistry = CreateFunctionRegistry(CreateBuiltinRegistry()); TKikimrProgramBuilder pgmBuilder(env, *functionRegistry); - auto pgmReturn = pgmBuilder.NewEmptyListOfVoid(); + auto pgmReturn = pgmBuilder.NewEmptyListOfVoid(); TVector<TRuntimeNode> keyColumns; - keyColumns.push_back(pgmBuilder.Add( + keyColumns.push_back(pgmBuilder.Add( pgmBuilder.TProgramBuilder::NewDataLiteral<ui32>(34), pgmBuilder.TProgramBuilder::NewDataLiteral<ui32>(12))); keyColumns.push_back(pgmBuilder.NewDataLiteral<NUdf::EDataSlot::String>("qwe")); - + TVector<ui32> keyTypes({ NUdf::TDataType<ui32>::Id, NUdf::TDataType<char*>::Id }); - pgmReturn = pgmBuilder.Append(pgmReturn, pgmBuilder.EraseRow(TTableId(1, 2), - keyTypes, - keyColumns)); - auto pgm = pgmBuilder.Build(pgmReturn, TKikimrProgramBuilder::TBindFlags::DisableOptimization).GetNode(); - //Cout << PrintNode(*pgm) << Endl; - VerifyProgram(pgm, env); - - TExploringNodeVisitor explorer; - explorer.Walk(pgm, env); + pgmReturn = pgmBuilder.Append(pgmReturn, pgmBuilder.EraseRow(TTableId(1, 2), + keyTypes, + keyColumns)); + auto pgm = pgmBuilder.Build(pgmReturn, TKikimrProgramBuilder::TBindFlags::DisableOptimization).GetNode(); + //Cout << PrintNode(*pgm) << Endl; + VerifyProgram(pgm, env); + + TExploringNodeVisitor explorer; + explorer.Walk(pgm, env); TVector<THolder<TKeyDesc>> tableKeys = ExtractTableKeys(explorer, env); - UNIT_ASSERT_VALUES_EQUAL(tableKeys.size(), 1); + UNIT_ASSERT_VALUES_EQUAL(tableKeys.size(), 1); UNIT_ASSERT(tableKeys[0]->TableId.HasSamePath(TTableId(1, 2))); - UNIT_ASSERT(tableKeys[0]->RowOperation == TKeyDesc::ERowOperation::Erase); - UNIT_ASSERT(tableKeys[0]->Range.InclusiveFrom); - UNIT_ASSERT(!tableKeys[0]->Range.InclusiveTo); - UNIT_ASSERT(!tableKeys[0]->Range.Point); + UNIT_ASSERT(tableKeys[0]->RowOperation == TKeyDesc::ERowOperation::Erase); + UNIT_ASSERT(tableKeys[0]->Range.InclusiveFrom); + UNIT_ASSERT(!tableKeys[0]->Range.InclusiveTo); + UNIT_ASSERT(!tableKeys[0]->Range.Point); UNIT_ASSERT(tableKeys[0]->Range.From.size() == 2); - UNIT_ASSERT(tableKeys[0]->Range.From[0].IsNull()); - UNIT_ASSERT(tableKeys[0]->Range.From[1].IsNull()); + UNIT_ASSERT(tableKeys[0]->Range.From[0].IsNull()); + UNIT_ASSERT(tableKeys[0]->Range.From[1].IsNull()); UNIT_ASSERT(tableKeys[0]->Range.To.size() == 0); - UNIT_ASSERT(tableKeys[0]->Columns.size() == 0); - UNIT_ASSERT(tableKeys[0]->KeyColumnTypes.size() == 2); + UNIT_ASSERT(tableKeys[0]->Columns.size() == 0); + UNIT_ASSERT(tableKeys[0]->KeyColumnTypes.size() == 2); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[0] == NUdf::TDataType<ui32>::Id); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[1] == NUdf::TDataType<char*>::Id); - } - + } + Y_UNIT_TEST(TestSelectRow) { TScopedAlloc alloc; TTypeEnvironment env(alloc); auto functionRegistry = CreateFunctionRegistry(CreateBuiltinRegistry()); TKikimrProgramBuilder pgmBuilder(env, *functionRegistry); - auto pgmReturn = pgmBuilder.NewEmptyListOfVoid(); + auto pgmReturn = pgmBuilder.NewEmptyListOfVoid(); TVector<TRuntimeNode> keyColumns; keyColumns.push_back(pgmBuilder.TProgramBuilder::NewDataLiteral<ui32>(42)); keyColumns.push_back(pgmBuilder.NewEmptyOptionalDataLiteral(NUdf::TDataType<ui64>::Id)); - keyColumns.push_back(pgmBuilder.NewOptional( + keyColumns.push_back(pgmBuilder.NewOptional( pgmBuilder.NewDataLiteral<NUdf::EDataSlot::String>("qwe"))); TVector<TSelectColumn> columnsToRead; columnsToRead.emplace_back("column1", 34, (ui32)NUdf::TDataType<ui32>::Id); columnsToRead.emplace_back("column2", 56, (ui32)NUdf::TDataType<ui64>::Id); TVector<ui32> keyTypes({ NUdf::TDataType<ui32>::Id, NUdf::TDataType<ui64>::Id, NUdf::TDataType<char*>::Id }); - pgmReturn = pgmBuilder.Append(pgmReturn, pgmBuilder.SetResult("myRes", - pgmBuilder.SelectRow(TTableId(1, 2), - keyTypes, - columnsToRead, keyColumns))); - auto pgm = pgmBuilder.Build(pgmReturn, TKikimrProgramBuilder::TBindFlags::DisableOptimization).GetNode(); - //Cout << PrintNode(*pgm) << Endl; - VerifyProgram(pgm, env); - - TExploringNodeVisitor explorer; - explorer.Walk(pgm, env); + pgmReturn = pgmBuilder.Append(pgmReturn, pgmBuilder.SetResult("myRes", + pgmBuilder.SelectRow(TTableId(1, 2), + keyTypes, + columnsToRead, keyColumns))); + auto pgm = pgmBuilder.Build(pgmReturn, TKikimrProgramBuilder::TBindFlags::DisableOptimization).GetNode(); + //Cout << PrintNode(*pgm) << Endl; + VerifyProgram(pgm, env); + + TExploringNodeVisitor explorer; + explorer.Walk(pgm, env); TVector<THolder<TKeyDesc>> tableKeys = ExtractTableKeys(explorer, env); - UNIT_ASSERT_VALUES_EQUAL(tableKeys.size(), 1); + UNIT_ASSERT_VALUES_EQUAL(tableKeys.size(), 1); UNIT_ASSERT(tableKeys[0]->TableId.HasSamePath(TTableId(1, 2))); - UNIT_ASSERT(tableKeys[0]->RowOperation == TKeyDesc::ERowOperation::Read); - UNIT_ASSERT(tableKeys[0]->Range.InclusiveFrom); - UNIT_ASSERT(tableKeys[0]->Range.InclusiveTo); - UNIT_ASSERT(tableKeys[0]->Range.Point); + UNIT_ASSERT(tableKeys[0]->RowOperation == TKeyDesc::ERowOperation::Read); + UNIT_ASSERT(tableKeys[0]->Range.InclusiveFrom); + UNIT_ASSERT(tableKeys[0]->Range.InclusiveTo); + UNIT_ASSERT(tableKeys[0]->Range.Point); UNIT_ASSERT(tableKeys[0]->Range.From.size() == 3); - UNIT_ASSERT(tableKeys[0]->Range.From[0].Size() == 4); - UNIT_ASSERT(tableKeys[0]->Range.From[1].IsNull()); - UNIT_ASSERT(tableKeys[0]->Range.From[2].Size() == 3); + UNIT_ASSERT(tableKeys[0]->Range.From[0].Size() == 4); + UNIT_ASSERT(tableKeys[0]->Range.From[1].IsNull()); + UNIT_ASSERT(tableKeys[0]->Range.From[2].Size() == 3); UNIT_ASSERT(tableKeys[0]->Range.To.size() == 3); - UNIT_ASSERT(tableKeys[0]->Range.To[0].Size() == 4); - UNIT_ASSERT(tableKeys[0]->Range.To[1].IsNull()); - UNIT_ASSERT(tableKeys[0]->Range.To[2].Size() == 3); - UNIT_ASSERT(tableKeys[0]->Columns.size() == 2); - UNIT_ASSERT(tableKeys[0]->Columns[0].Column == 34); - UNIT_ASSERT(tableKeys[0]->Columns[0].Operation == TKeyDesc::EColumnOperation::Read); + UNIT_ASSERT(tableKeys[0]->Range.To[0].Size() == 4); + UNIT_ASSERT(tableKeys[0]->Range.To[1].IsNull()); + UNIT_ASSERT(tableKeys[0]->Range.To[2].Size() == 3); + UNIT_ASSERT(tableKeys[0]->Columns.size() == 2); + UNIT_ASSERT(tableKeys[0]->Columns[0].Column == 34); + UNIT_ASSERT(tableKeys[0]->Columns[0].Operation == TKeyDesc::EColumnOperation::Read); UNIT_ASSERT(tableKeys[0]->Columns[0].ExpectedType == NUdf::TDataType<ui32>::Id); - UNIT_ASSERT(tableKeys[0]->Columns[1].Column == 56); - UNIT_ASSERT(tableKeys[0]->Columns[1].Operation == TKeyDesc::EColumnOperation::Read); + UNIT_ASSERT(tableKeys[0]->Columns[1].Column == 56); + UNIT_ASSERT(tableKeys[0]->Columns[1].Operation == TKeyDesc::EColumnOperation::Read); UNIT_ASSERT(tableKeys[0]->Columns[1].ExpectedType == NUdf::TDataType<ui64>::Id); - UNIT_ASSERT(tableKeys[0]->KeyColumnTypes.size() == 3); + UNIT_ASSERT(tableKeys[0]->KeyColumnTypes.size() == 3); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[0] == NUdf::TDataType<ui32>::Id); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[1] == NUdf::TDataType<ui64>::Id); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[2] == NUdf::TDataType<char*>::Id); - } - + } + Y_UNIT_TEST(TestUpdateRowStaticKey) { TScopedAlloc alloc; TTypeEnvironment env(alloc); auto functionRegistry = CreateFunctionRegistry(CreateBuiltinRegistry()); TKikimrProgramBuilder pgmBuilder(env, *functionRegistry); - auto pgmReturn = pgmBuilder.NewEmptyListOfVoid(); + auto pgmReturn = pgmBuilder.NewEmptyListOfVoid(); TVector<TRuntimeNode> keyColumns; keyColumns.push_back(pgmBuilder.TProgramBuilder::NewDataLiteral<ui32>(42)); keyColumns.push_back(pgmBuilder.NewEmptyOptionalDataLiteral(NUdf::TDataType<ui64>::Id)); - keyColumns.push_back(pgmBuilder.NewOptional( + keyColumns.push_back(pgmBuilder.NewOptional( pgmBuilder.NewDataLiteral<NUdf::EDataSlot::String>("qwe"))); - auto update = pgmBuilder.GetUpdateRowBuilder(); + auto update = pgmBuilder.GetUpdateRowBuilder(); update.SetColumn(34, NUdf::TDataType<ui32>::Id, pgmBuilder.NewOptional(pgmBuilder.TProgramBuilder::NewDataLiteral<ui32>(12))); - update.EraseColumn(56); + update.EraseColumn(56); update.InplaceUpdateColumn(78, NUdf::TDataType<ui64>::Id, pgmBuilder.TProgramBuilder::NewDataLiteral<ui64>(1), EInplaceUpdateMode::Sum); TVector<ui32> keyTypes({ NUdf::TDataType<ui32>::Id, NUdf::TDataType<ui64>::Id, NUdf::TDataType<char*>::Id }); - pgmReturn = pgmBuilder.Append(pgmReturn, pgmBuilder.UpdateRow(TTableId(1, 2), - keyTypes, - keyColumns, update)); - auto pgm = pgmBuilder.Build(pgmReturn, TKikimrProgramBuilder::TBindFlags::DisableOptimization).GetNode(); - //Cout << PrintNode(*pgm) << Endl; - VerifyProgram(pgm, env); - - TExploringNodeVisitor explorer; - explorer.Walk(pgm, env); + pgmReturn = pgmBuilder.Append(pgmReturn, pgmBuilder.UpdateRow(TTableId(1, 2), + keyTypes, + keyColumns, update)); + auto pgm = pgmBuilder.Build(pgmReturn, TKikimrProgramBuilder::TBindFlags::DisableOptimization).GetNode(); + //Cout << PrintNode(*pgm) << Endl; + VerifyProgram(pgm, env); + + TExploringNodeVisitor explorer; + explorer.Walk(pgm, env); TVector<THolder<TKeyDesc>> tableKeys = ExtractTableKeys(explorer, env); - UNIT_ASSERT_VALUES_EQUAL(tableKeys.size(), 1); + UNIT_ASSERT_VALUES_EQUAL(tableKeys.size(), 1); UNIT_ASSERT(tableKeys[0]->TableId.HasSamePath(TTableId(1, 2))); - UNIT_ASSERT(tableKeys[0]->RowOperation == TKeyDesc::ERowOperation::Update); - UNIT_ASSERT(tableKeys[0]->Range.InclusiveFrom); - UNIT_ASSERT(tableKeys[0]->Range.InclusiveTo); - UNIT_ASSERT(tableKeys[0]->Range.Point); + UNIT_ASSERT(tableKeys[0]->RowOperation == TKeyDesc::ERowOperation::Update); + UNIT_ASSERT(tableKeys[0]->Range.InclusiveFrom); + UNIT_ASSERT(tableKeys[0]->Range.InclusiveTo); + UNIT_ASSERT(tableKeys[0]->Range.Point); UNIT_ASSERT(tableKeys[0]->Range.From.size() == 3); - UNIT_ASSERT(tableKeys[0]->Range.From[0].Size() == 4); - UNIT_ASSERT(tableKeys[0]->Range.From[1].IsNull()); - UNIT_ASSERT(tableKeys[0]->Range.From[2].Size() == 3); + UNIT_ASSERT(tableKeys[0]->Range.From[0].Size() == 4); + UNIT_ASSERT(tableKeys[0]->Range.From[1].IsNull()); + UNIT_ASSERT(tableKeys[0]->Range.From[2].Size() == 3); UNIT_ASSERT(tableKeys[0]->Range.To.size() == 3); - UNIT_ASSERT(tableKeys[0]->Range.To[0].Size() == 4); - UNIT_ASSERT(tableKeys[0]->Range.To[1].IsNull()); - UNIT_ASSERT(tableKeys[0]->Range.To[2].Size() == 3); - UNIT_ASSERT(tableKeys[0]->Columns.size() == 3); - UNIT_ASSERT(tableKeys[0]->Columns[0].Column == 34); - UNIT_ASSERT(tableKeys[0]->Columns[0].Operation == TKeyDesc::EColumnOperation::Set); + UNIT_ASSERT(tableKeys[0]->Range.To[0].Size() == 4); + UNIT_ASSERT(tableKeys[0]->Range.To[1].IsNull()); + UNIT_ASSERT(tableKeys[0]->Range.To[2].Size() == 3); + UNIT_ASSERT(tableKeys[0]->Columns.size() == 3); + UNIT_ASSERT(tableKeys[0]->Columns[0].Column == 34); + UNIT_ASSERT(tableKeys[0]->Columns[0].Operation == TKeyDesc::EColumnOperation::Set); UNIT_ASSERT(tableKeys[0]->Columns[0].ExpectedType == NUdf::TDataType<ui32>::Id); - UNIT_ASSERT(tableKeys[0]->Columns[1].Column == 56); - UNIT_ASSERT(tableKeys[0]->Columns[1].Operation == TKeyDesc::EColumnOperation::Set); - UNIT_ASSERT(tableKeys[0]->Columns[1].ExpectedType == 0); - UNIT_ASSERT(tableKeys[0]->Columns[2].Column == 78); - UNIT_ASSERT(tableKeys[0]->Columns[2].Operation == TKeyDesc::EColumnOperation::InplaceUpdate); + UNIT_ASSERT(tableKeys[0]->Columns[1].Column == 56); + UNIT_ASSERT(tableKeys[0]->Columns[1].Operation == TKeyDesc::EColumnOperation::Set); + UNIT_ASSERT(tableKeys[0]->Columns[1].ExpectedType == 0); + UNIT_ASSERT(tableKeys[0]->Columns[2].Column == 78); + UNIT_ASSERT(tableKeys[0]->Columns[2].Operation == TKeyDesc::EColumnOperation::InplaceUpdate); UNIT_ASSERT(tableKeys[0]->Columns[2].ExpectedType == NUdf::TDataType<ui64>::Id); - UNIT_ASSERT(tableKeys[0]->KeyColumnTypes.size() == 3); + UNIT_ASSERT(tableKeys[0]->KeyColumnTypes.size() == 3); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[0] == NUdf::TDataType<ui32>::Id); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[1] == NUdf::TDataType<ui64>::Id); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[2] == NUdf::TDataType<char*>::Id); - } - + } + Y_UNIT_TEST(TestUpdateRowDynamicKey) { TScopedAlloc alloc; TTypeEnvironment env(alloc); auto functionRegistry = CreateFunctionRegistry(CreateBuiltinRegistry()); TKikimrProgramBuilder pgmBuilder(env, *functionRegistry); - auto pgmReturn = pgmBuilder.NewEmptyListOfVoid(); + auto pgmReturn = pgmBuilder.NewEmptyListOfVoid(); TVector<TRuntimeNode> keyColumns; - keyColumns.push_back(pgmBuilder.Add( + keyColumns.push_back(pgmBuilder.Add( pgmBuilder.TProgramBuilder::NewDataLiteral<ui32>(34), pgmBuilder.TProgramBuilder::NewDataLiteral<ui32>(12))); keyColumns.push_back(pgmBuilder.NewEmptyOptionalDataLiteral(NUdf::TDataType<ui64>::Id)); - keyColumns.push_back(pgmBuilder.NewOptional( + keyColumns.push_back(pgmBuilder.NewOptional( pgmBuilder.NewDataLiteral<NUdf::EDataSlot::String>("qwe"))); - auto update = pgmBuilder.GetUpdateRowBuilder(); + auto update = pgmBuilder.GetUpdateRowBuilder(); update.SetColumn(34, NUdf::TDataType<ui32>::Id, pgmBuilder.NewOptional(pgmBuilder.TProgramBuilder::NewDataLiteral<ui32>(12))); - update.EraseColumn(56); + update.EraseColumn(56); update.InplaceUpdateColumn(78, NUdf::TDataType<ui64>::Id, pgmBuilder.TProgramBuilder::NewDataLiteral<ui64>(1), EInplaceUpdateMode::Sum); TVector<ui32> keyTypes({ NUdf::TDataType<ui32>::Id, NUdf::TDataType<ui64>::Id, NUdf::TDataType<char*>::Id }); - pgmReturn = pgmBuilder.Append(pgmReturn, pgmBuilder.UpdateRow(TTableId(1, 2), - keyTypes, - keyColumns, update)); - auto pgm = pgmBuilder.Build(pgmReturn, TKikimrProgramBuilder::TBindFlags::DisableOptimization).GetNode(); - //Cout << PrintNode(*pgm) << Endl; - VerifyProgram(pgm, env); - - TExploringNodeVisitor explorer; - explorer.Walk(pgm, env); + pgmReturn = pgmBuilder.Append(pgmReturn, pgmBuilder.UpdateRow(TTableId(1, 2), + keyTypes, + keyColumns, update)); + auto pgm = pgmBuilder.Build(pgmReturn, TKikimrProgramBuilder::TBindFlags::DisableOptimization).GetNode(); + //Cout << PrintNode(*pgm) << Endl; + VerifyProgram(pgm, env); + + TExploringNodeVisitor explorer; + explorer.Walk(pgm, env); TVector<THolder<TKeyDesc>> tableKeys = ExtractTableKeys(explorer, env); - UNIT_ASSERT_VALUES_EQUAL(tableKeys.size(), 1); + UNIT_ASSERT_VALUES_EQUAL(tableKeys.size(), 1); UNIT_ASSERT(tableKeys[0]->TableId.HasSamePath(TTableId(1, 2))); - UNIT_ASSERT(tableKeys[0]->RowOperation == TKeyDesc::ERowOperation::Update); - UNIT_ASSERT(tableKeys[0]->Range.InclusiveFrom); - UNIT_ASSERT(!tableKeys[0]->Range.InclusiveTo); - UNIT_ASSERT(!tableKeys[0]->Range.Point); + UNIT_ASSERT(tableKeys[0]->RowOperation == TKeyDesc::ERowOperation::Update); + UNIT_ASSERT(tableKeys[0]->Range.InclusiveFrom); + UNIT_ASSERT(!tableKeys[0]->Range.InclusiveTo); + UNIT_ASSERT(!tableKeys[0]->Range.Point); UNIT_ASSERT(tableKeys[0]->Range.From.size() == 3); - UNIT_ASSERT(tableKeys[0]->Range.From[0].IsNull()); - UNIT_ASSERT(tableKeys[0]->Range.From[1].IsNull()); - UNIT_ASSERT(tableKeys[0]->Range.From[2].IsNull()); + UNIT_ASSERT(tableKeys[0]->Range.From[0].IsNull()); + UNIT_ASSERT(tableKeys[0]->Range.From[1].IsNull()); + UNIT_ASSERT(tableKeys[0]->Range.From[2].IsNull()); UNIT_ASSERT(tableKeys[0]->Range.To.size() == 0); - UNIT_ASSERT(tableKeys[0]->Columns.size() == 3); - UNIT_ASSERT(tableKeys[0]->Columns[0].Column == 34); - UNIT_ASSERT(tableKeys[0]->Columns[0].Operation == TKeyDesc::EColumnOperation::Set); + UNIT_ASSERT(tableKeys[0]->Columns.size() == 3); + UNIT_ASSERT(tableKeys[0]->Columns[0].Column == 34); + UNIT_ASSERT(tableKeys[0]->Columns[0].Operation == TKeyDesc::EColumnOperation::Set); UNIT_ASSERT(tableKeys[0]->Columns[0].ExpectedType == NUdf::TDataType<ui32>::Id); - UNIT_ASSERT(tableKeys[0]->Columns[1].Column == 56); - UNIT_ASSERT(tableKeys[0]->Columns[1].Operation == TKeyDesc::EColumnOperation::Set); - UNIT_ASSERT(tableKeys[0]->Columns[1].ExpectedType == 0); - UNIT_ASSERT(tableKeys[0]->Columns[2].Column == 78); - UNIT_ASSERT(tableKeys[0]->Columns[2].Operation == TKeyDesc::EColumnOperation::InplaceUpdate); + UNIT_ASSERT(tableKeys[0]->Columns[1].Column == 56); + UNIT_ASSERT(tableKeys[0]->Columns[1].Operation == TKeyDesc::EColumnOperation::Set); + UNIT_ASSERT(tableKeys[0]->Columns[1].ExpectedType == 0); + UNIT_ASSERT(tableKeys[0]->Columns[2].Column == 78); + UNIT_ASSERT(tableKeys[0]->Columns[2].Operation == TKeyDesc::EColumnOperation::InplaceUpdate); UNIT_ASSERT(tableKeys[0]->Columns[2].ExpectedType == NUdf::TDataType<ui64>::Id); - UNIT_ASSERT(tableKeys[0]->Columns[2].InplaceUpdateMode == (ui32)EInplaceUpdateMode::Sum); - UNIT_ASSERT(tableKeys[0]->KeyColumnTypes.size() == 3); + UNIT_ASSERT(tableKeys[0]->Columns[2].InplaceUpdateMode == (ui32)EInplaceUpdateMode::Sum); + UNIT_ASSERT(tableKeys[0]->KeyColumnTypes.size() == 3); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[0] == NUdf::TDataType<ui32>::Id); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[1] == NUdf::TDataType<ui64>::Id); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[2] == NUdf::TDataType<char*>::Id); - } - + } + Y_UNIT_TEST(TestSelectFromInclusiveRange) { TScopedAlloc alloc; TTypeEnvironment env(alloc); auto functionRegistry = CreateFunctionRegistry(CreateBuiltinRegistry()); TKikimrProgramBuilder pgmBuilder(env, *functionRegistry); - auto pgmReturn = pgmBuilder.NewEmptyListOfVoid(); - TTableRangeOptions options(pgmBuilder.GetDefaultTableRangeOptions()); + auto pgmReturn = pgmBuilder.NewEmptyListOfVoid(); + TTableRangeOptions options(pgmBuilder.GetDefaultTableRangeOptions()); TVector<TRuntimeNode> from; from.push_back(pgmBuilder.TProgramBuilder::NewDataLiteral<ui32>(42)); from.push_back(pgmBuilder.NewEmptyOptionalDataLiteral(NUdf::TDataType<ui64>::Id)); - options.FromColumns = from; - + options.FromColumns = from; + TVector<TSelectColumn> columnsToRead; columnsToRead.emplace_back("column1", 34, (ui32)NUdf::TDataType<ui32>::Id); columnsToRead.emplace_back("column2", 56, (ui32)NUdf::TDataType<ui64>::Id); TVector<ui32> keyTypes({ NUdf::TDataType<ui32>::Id, NUdf::TDataType<ui64>::Id }); - pgmReturn = pgmBuilder.Append(pgmReturn, pgmBuilder.SetResult("myRes", - pgmBuilder.SelectRange(TTableId(1, 2), - keyTypes, - columnsToRead, options))); - auto pgm = pgmBuilder.Build(pgmReturn, TKikimrProgramBuilder::TBindFlags::DisableOptimization).GetNode(); - //Cout << PrintNode(*pgm) << Endl; - VerifyProgram(pgm, env); - - TExploringNodeVisitor explorer; - explorer.Walk(pgm, env); + pgmReturn = pgmBuilder.Append(pgmReturn, pgmBuilder.SetResult("myRes", + pgmBuilder.SelectRange(TTableId(1, 2), + keyTypes, + columnsToRead, options))); + auto pgm = pgmBuilder.Build(pgmReturn, TKikimrProgramBuilder::TBindFlags::DisableOptimization).GetNode(); + //Cout << PrintNode(*pgm) << Endl; + VerifyProgram(pgm, env); + + TExploringNodeVisitor explorer; + explorer.Walk(pgm, env); TVector<THolder<TKeyDesc>> tableKeys = ExtractTableKeys(explorer, env); - UNIT_ASSERT_VALUES_EQUAL(tableKeys.size(), 1); + UNIT_ASSERT_VALUES_EQUAL(tableKeys.size(), 1); UNIT_ASSERT(tableKeys[0]->TableId.HasSamePath(TTableId(1, 2))); - UNIT_ASSERT(tableKeys[0]->RowOperation == TKeyDesc::ERowOperation::Read); - UNIT_ASSERT(tableKeys[0]->Range.InclusiveFrom); - UNIT_ASSERT(tableKeys[0]->Range.InclusiveTo); - UNIT_ASSERT(!tableKeys[0]->Range.Point); + UNIT_ASSERT(tableKeys[0]->RowOperation == TKeyDesc::ERowOperation::Read); + UNIT_ASSERT(tableKeys[0]->Range.InclusiveFrom); + UNIT_ASSERT(tableKeys[0]->Range.InclusiveTo); + UNIT_ASSERT(!tableKeys[0]->Range.Point); UNIT_ASSERT(tableKeys[0]->Range.From.size() == 2); - UNIT_ASSERT(tableKeys[0]->Range.From[0].Size() == 4); - UNIT_ASSERT(tableKeys[0]->Range.From[1].IsNull()); + UNIT_ASSERT(tableKeys[0]->Range.From[0].Size() == 4); + UNIT_ASSERT(tableKeys[0]->Range.From[1].IsNull()); UNIT_ASSERT(tableKeys[0]->Range.To.size() == 0); - UNIT_ASSERT(tableKeys[0]->KeyColumnTypes.size() == 2); + UNIT_ASSERT(tableKeys[0]->KeyColumnTypes.size() == 2); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[0] == NUdf::TDataType<ui32>::Id); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[1] == NUdf::TDataType<ui64>::Id); - UNIT_ASSERT(tableKeys[0]->Columns.size() == 2); - UNIT_ASSERT(tableKeys[0]->Columns[0].Column == 34); - UNIT_ASSERT(tableKeys[0]->Columns[0].Operation == TKeyDesc::EColumnOperation::Read); + UNIT_ASSERT(tableKeys[0]->Columns.size() == 2); + UNIT_ASSERT(tableKeys[0]->Columns[0].Column == 34); + UNIT_ASSERT(tableKeys[0]->Columns[0].Operation == TKeyDesc::EColumnOperation::Read); UNIT_ASSERT(tableKeys[0]->Columns[0].ExpectedType == NUdf::TDataType<ui32>::Id); - UNIT_ASSERT(tableKeys[0]->Columns[1].Column == 56); - UNIT_ASSERT(tableKeys[0]->Columns[1].Operation == TKeyDesc::EColumnOperation::Read); + UNIT_ASSERT(tableKeys[0]->Columns[1].Column == 56); + UNIT_ASSERT(tableKeys[0]->Columns[1].Operation == TKeyDesc::EColumnOperation::Read); UNIT_ASSERT(tableKeys[0]->Columns[1].ExpectedType == NUdf::TDataType<ui64>::Id); - } - + } + Y_UNIT_TEST(TestSelectFromExclusiveRange) { TScopedAlloc alloc; TTypeEnvironment env(alloc); auto functionRegistry = CreateFunctionRegistry(CreateBuiltinRegistry()); TKikimrProgramBuilder pgmBuilder(env, *functionRegistry); - auto pgmReturn = pgmBuilder.NewEmptyListOfVoid(); - TTableRangeOptions options(pgmBuilder.GetDefaultTableRangeOptions()); + auto pgmReturn = pgmBuilder.NewEmptyListOfVoid(); + TTableRangeOptions options(pgmBuilder.GetDefaultTableRangeOptions()); TVector<TRuntimeNode> from; from.push_back(pgmBuilder.TProgramBuilder::NewDataLiteral<ui32>(42)); from.push_back(pgmBuilder.NewEmptyOptionalDataLiteral(NUdf::TDataType<ui64>::Id)); - options.FromColumns = from; + options.FromColumns = from; options.Flags = pgmBuilder.TProgramBuilder::NewDataLiteral<ui32>(TReadRangeOptions::TFlags::ExcludeInitValue); - + TVector<TSelectColumn> columnsToRead; columnsToRead.emplace_back("column1", 34, (ui32)NUdf::TDataType<ui32>::Id); columnsToRead.emplace_back("column2", 56, (ui32)NUdf::TDataType<ui64>::Id); TVector<ui32> keyTypes({ NUdf::TDataType<ui32>::Id, NUdf::TDataType<ui64>::Id }); - pgmReturn = pgmBuilder.Append(pgmReturn, pgmBuilder.SetResult("myRes", - pgmBuilder.SelectRange(TTableId(1, 2), - keyTypes, - columnsToRead, options))); - auto pgm = pgmBuilder.Build(pgmReturn, TKikimrProgramBuilder::TBindFlags::DisableOptimization).GetNode(); - //Cout << PrintNode(*pgm) << Endl; - VerifyProgram(pgm, env); - - TExploringNodeVisitor explorer; - explorer.Walk(pgm, env); + pgmReturn = pgmBuilder.Append(pgmReturn, pgmBuilder.SetResult("myRes", + pgmBuilder.SelectRange(TTableId(1, 2), + keyTypes, + columnsToRead, options))); + auto pgm = pgmBuilder.Build(pgmReturn, TKikimrProgramBuilder::TBindFlags::DisableOptimization).GetNode(); + //Cout << PrintNode(*pgm) << Endl; + VerifyProgram(pgm, env); + + TExploringNodeVisitor explorer; + explorer.Walk(pgm, env); TVector<THolder<TKeyDesc>> tableKeys = ExtractTableKeys(explorer, env); - UNIT_ASSERT_VALUES_EQUAL(tableKeys.size(), 1); + UNIT_ASSERT_VALUES_EQUAL(tableKeys.size(), 1); UNIT_ASSERT(tableKeys[0]->TableId.HasSamePath(TTableId(1, 2))); - UNIT_ASSERT(tableKeys[0]->RowOperation == TKeyDesc::ERowOperation::Read); - UNIT_ASSERT(!tableKeys[0]->Range.InclusiveFrom); - UNIT_ASSERT(tableKeys[0]->Range.InclusiveTo); - UNIT_ASSERT(!tableKeys[0]->Range.Point); + UNIT_ASSERT(tableKeys[0]->RowOperation == TKeyDesc::ERowOperation::Read); + UNIT_ASSERT(!tableKeys[0]->Range.InclusiveFrom); + UNIT_ASSERT(tableKeys[0]->Range.InclusiveTo); + UNIT_ASSERT(!tableKeys[0]->Range.Point); UNIT_ASSERT(tableKeys[0]->Range.From.size() == 2); - UNIT_ASSERT(tableKeys[0]->Range.From[0].Size() == 4); - UNIT_ASSERT(tableKeys[0]->Range.From[1].IsNull()); + UNIT_ASSERT(tableKeys[0]->Range.From[0].Size() == 4); + UNIT_ASSERT(tableKeys[0]->Range.From[1].IsNull()); UNIT_ASSERT(tableKeys[0]->Range.To.size() == 0); - UNIT_ASSERT(tableKeys[0]->KeyColumnTypes.size() == 2); + UNIT_ASSERT(tableKeys[0]->KeyColumnTypes.size() == 2); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[0] == NUdf::TDataType<ui32>::Id); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[1] == NUdf::TDataType<ui64>::Id); - UNIT_ASSERT(tableKeys[0]->Columns.size() == 2); - UNIT_ASSERT(tableKeys[0]->Columns[0].Column == 34); - UNIT_ASSERT(tableKeys[0]->Columns[0].Operation == TKeyDesc::EColumnOperation::Read); + UNIT_ASSERT(tableKeys[0]->Columns.size() == 2); + UNIT_ASSERT(tableKeys[0]->Columns[0].Column == 34); + UNIT_ASSERT(tableKeys[0]->Columns[0].Operation == TKeyDesc::EColumnOperation::Read); UNIT_ASSERT(tableKeys[0]->Columns[0].ExpectedType == NUdf::TDataType<ui32>::Id); - UNIT_ASSERT(tableKeys[0]->Columns[1].Column == 56); - UNIT_ASSERT(tableKeys[0]->Columns[1].Operation == TKeyDesc::EColumnOperation::Read); + UNIT_ASSERT(tableKeys[0]->Columns[1].Column == 56); + UNIT_ASSERT(tableKeys[0]->Columns[1].Operation == TKeyDesc::EColumnOperation::Read); UNIT_ASSERT(tableKeys[0]->Columns[1].ExpectedType == NUdf::TDataType<ui64>::Id); - } - + } + Y_UNIT_TEST(TestSelectToInclusiveRange) { TScopedAlloc alloc; TTypeEnvironment env(alloc); auto functionRegistry = CreateFunctionRegistry(CreateBuiltinRegistry()); TKikimrProgramBuilder pgmBuilder(env, *functionRegistry); - auto pgmReturn = pgmBuilder.NewEmptyListOfVoid(); - TTableRangeOptions options(pgmBuilder.GetDefaultTableRangeOptions()); + auto pgmReturn = pgmBuilder.NewEmptyListOfVoid(); + TTableRangeOptions options(pgmBuilder.GetDefaultTableRangeOptions()); TVector<TRuntimeNode> from; from.push_back(pgmBuilder.NewEmptyOptionalDataLiteral(NUdf::TDataType<ui32>::Id)); TVector<TRuntimeNode> to; to.push_back(pgmBuilder.TProgramBuilder::NewDataLiteral<ui32>(42)); to.push_back(pgmBuilder.NewEmptyOptionalDataLiteral(NUdf::TDataType<ui64>::Id)); - options.FromColumns = from; - options.ToColumns = to; - + options.FromColumns = from; + options.ToColumns = to; + TVector<TSelectColumn> columnsToRead; columnsToRead.emplace_back("column1", 34, (ui32)NUdf::TDataType<ui32>::Id); columnsToRead.emplace_back("column2", 56, (ui32)NUdf::TDataType<ui64>::Id); TVector<ui32> keyTypes({ NUdf::TDataType<ui32>::Id, NUdf::TDataType<ui64>::Id }); - pgmReturn = pgmBuilder.Append(pgmReturn, pgmBuilder.SetResult("myRes", - pgmBuilder.SelectRange(TTableId(1, 2), - keyTypes, - columnsToRead, options))); - auto pgm = pgmBuilder.Build(pgmReturn, TKikimrProgramBuilder::TBindFlags::DisableOptimization).GetNode(); - //Cout << PrintNode(*pgm) << Endl; - VerifyProgram(pgm, env); - - TExploringNodeVisitor explorer; - explorer.Walk(pgm, env); + pgmReturn = pgmBuilder.Append(pgmReturn, pgmBuilder.SetResult("myRes", + pgmBuilder.SelectRange(TTableId(1, 2), + keyTypes, + columnsToRead, options))); + auto pgm = pgmBuilder.Build(pgmReturn, TKikimrProgramBuilder::TBindFlags::DisableOptimization).GetNode(); + //Cout << PrintNode(*pgm) << Endl; + VerifyProgram(pgm, env); + + TExploringNodeVisitor explorer; + explorer.Walk(pgm, env); TVector<THolder<TKeyDesc>> tableKeys = ExtractTableKeys(explorer, env); - UNIT_ASSERT_VALUES_EQUAL(tableKeys.size(), 1); + UNIT_ASSERT_VALUES_EQUAL(tableKeys.size(), 1); UNIT_ASSERT(tableKeys[0]->TableId.HasSamePath(TTableId(1, 2))); - UNIT_ASSERT(tableKeys[0]->RowOperation == TKeyDesc::ERowOperation::Read); - UNIT_ASSERT(tableKeys[0]->Range.InclusiveFrom); - UNIT_ASSERT(tableKeys[0]->Range.InclusiveTo); - UNIT_ASSERT(!tableKeys[0]->Range.Point); + UNIT_ASSERT(tableKeys[0]->RowOperation == TKeyDesc::ERowOperation::Read); + UNIT_ASSERT(tableKeys[0]->Range.InclusiveFrom); + UNIT_ASSERT(tableKeys[0]->Range.InclusiveTo); + UNIT_ASSERT(!tableKeys[0]->Range.Point); UNIT_ASSERT(tableKeys[0]->Range.From.size() == 2); - UNIT_ASSERT(tableKeys[0]->Range.From[0].IsNull()); - UNIT_ASSERT(tableKeys[0]->Range.From[1].IsNull()); + UNIT_ASSERT(tableKeys[0]->Range.From[0].IsNull()); + UNIT_ASSERT(tableKeys[0]->Range.From[1].IsNull()); UNIT_ASSERT(tableKeys[0]->Range.To.size() == 2); - UNIT_ASSERT(tableKeys[0]->Range.To[0].Size() == 4); - UNIT_ASSERT(tableKeys[0]->Range.To[1].IsNull()); - UNIT_ASSERT(tableKeys[0]->KeyColumnTypes.size() == 2); + UNIT_ASSERT(tableKeys[0]->Range.To[0].Size() == 4); + UNIT_ASSERT(tableKeys[0]->Range.To[1].IsNull()); + UNIT_ASSERT(tableKeys[0]->KeyColumnTypes.size() == 2); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[0] == NUdf::TDataType<ui32>::Id); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[1] == NUdf::TDataType<ui64>::Id); - UNIT_ASSERT(tableKeys[0]->Columns.size() == 2); - UNIT_ASSERT(tableKeys[0]->Columns[0].Column == 34); - UNIT_ASSERT(tableKeys[0]->Columns[0].Operation == TKeyDesc::EColumnOperation::Read); + UNIT_ASSERT(tableKeys[0]->Columns.size() == 2); + UNIT_ASSERT(tableKeys[0]->Columns[0].Column == 34); + UNIT_ASSERT(tableKeys[0]->Columns[0].Operation == TKeyDesc::EColumnOperation::Read); UNIT_ASSERT(tableKeys[0]->Columns[0].ExpectedType == NUdf::TDataType<ui32>::Id); - UNIT_ASSERT(tableKeys[0]->Columns[1].Column == 56); - UNIT_ASSERT(tableKeys[0]->Columns[1].Operation == TKeyDesc::EColumnOperation::Read); + UNIT_ASSERT(tableKeys[0]->Columns[1].Column == 56); + UNIT_ASSERT(tableKeys[0]->Columns[1].Operation == TKeyDesc::EColumnOperation::Read); UNIT_ASSERT(tableKeys[0]->Columns[1].ExpectedType == NUdf::TDataType<ui64>::Id); - } - + } + Y_UNIT_TEST(TestSelectToExclusiveRange) { TScopedAlloc alloc; TTypeEnvironment env(alloc); auto functionRegistry = CreateFunctionRegistry(CreateBuiltinRegistry()); TKikimrProgramBuilder pgmBuilder(env, *functionRegistry); - auto pgmReturn = pgmBuilder.NewEmptyListOfVoid(); - TTableRangeOptions options(pgmBuilder.GetDefaultTableRangeOptions()); + auto pgmReturn = pgmBuilder.NewEmptyListOfVoid(); + TTableRangeOptions options(pgmBuilder.GetDefaultTableRangeOptions()); TVector<TRuntimeNode> from; from.push_back(pgmBuilder.NewEmptyOptionalDataLiteral(NUdf::TDataType<ui32>::Id)); TVector<TRuntimeNode> to; to.push_back(pgmBuilder.TProgramBuilder::NewDataLiteral<ui32>(42)); to.push_back(pgmBuilder.NewEmptyOptionalDataLiteral(NUdf::TDataType<ui64>::Id)); - options.FromColumns = from; - options.ToColumns = to; + options.FromColumns = from; + options.ToColumns = to; options.Flags = pgmBuilder.TProgramBuilder::NewDataLiteral<ui32>(TReadRangeOptions::TFlags::ExcludeTermValue); - + TVector<TSelectColumn> columnsToRead; columnsToRead.emplace_back("column1", 34, (ui32)NUdf::TDataType<ui32>::Id); columnsToRead.emplace_back("column2", 56, (ui32)NUdf::TDataType<ui64>::Id); TVector<ui32> keyTypes({ NUdf::TDataType<ui32>::Id, NUdf::TDataType<ui64>::Id }); - pgmReturn = pgmBuilder.Append(pgmReturn, pgmBuilder.SetResult("myRes", - pgmBuilder.SelectRange(TTableId(1, 2), - keyTypes, - columnsToRead, options))); - auto pgm = pgmBuilder.Build(pgmReturn, TKikimrProgramBuilder::TBindFlags::DisableOptimization).GetNode(); - //Cout << PrintNode(*pgm) << Endl; - VerifyProgram(pgm, env); - - TExploringNodeVisitor explorer; - explorer.Walk(pgm, env); + pgmReturn = pgmBuilder.Append(pgmReturn, pgmBuilder.SetResult("myRes", + pgmBuilder.SelectRange(TTableId(1, 2), + keyTypes, + columnsToRead, options))); + auto pgm = pgmBuilder.Build(pgmReturn, TKikimrProgramBuilder::TBindFlags::DisableOptimization).GetNode(); + //Cout << PrintNode(*pgm) << Endl; + VerifyProgram(pgm, env); + + TExploringNodeVisitor explorer; + explorer.Walk(pgm, env); TVector<THolder<TKeyDesc>> tableKeys = ExtractTableKeys(explorer, env); - UNIT_ASSERT_VALUES_EQUAL(tableKeys.size(), 1); + UNIT_ASSERT_VALUES_EQUAL(tableKeys.size(), 1); UNIT_ASSERT(tableKeys[0]->TableId.HasSamePath(TTableId(1, 2))); - UNIT_ASSERT(tableKeys[0]->RowOperation == TKeyDesc::ERowOperation::Read); - UNIT_ASSERT(tableKeys[0]->Range.InclusiveFrom); - UNIT_ASSERT(!tableKeys[0]->Range.InclusiveTo); - UNIT_ASSERT(!tableKeys[0]->Range.Point); + UNIT_ASSERT(tableKeys[0]->RowOperation == TKeyDesc::ERowOperation::Read); + UNIT_ASSERT(tableKeys[0]->Range.InclusiveFrom); + UNIT_ASSERT(!tableKeys[0]->Range.InclusiveTo); + UNIT_ASSERT(!tableKeys[0]->Range.Point); UNIT_ASSERT(tableKeys[0]->Range.From.size() == 2); - UNIT_ASSERT(tableKeys[0]->Range.From[0].IsNull()); - UNIT_ASSERT(tableKeys[0]->Range.From[1].IsNull()); + UNIT_ASSERT(tableKeys[0]->Range.From[0].IsNull()); + UNIT_ASSERT(tableKeys[0]->Range.From[1].IsNull()); UNIT_ASSERT(tableKeys[0]->Range.To.size() == 2); - UNIT_ASSERT(tableKeys[0]->Range.To[0].Size() == 4); - UNIT_ASSERT(tableKeys[0]->Range.To[1].IsNull()); - UNIT_ASSERT(tableKeys[0]->KeyColumnTypes.size() == 2); + UNIT_ASSERT(tableKeys[0]->Range.To[0].Size() == 4); + UNIT_ASSERT(tableKeys[0]->Range.To[1].IsNull()); + UNIT_ASSERT(tableKeys[0]->KeyColumnTypes.size() == 2); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[0] == NUdf::TDataType<ui32>::Id); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[1] == NUdf::TDataType<ui64>::Id); - UNIT_ASSERT(tableKeys[0]->Columns.size() == 2); - UNIT_ASSERT(tableKeys[0]->Columns[0].Column == 34); - UNIT_ASSERT(tableKeys[0]->Columns[0].Operation == TKeyDesc::EColumnOperation::Read); + UNIT_ASSERT(tableKeys[0]->Columns.size() == 2); + UNIT_ASSERT(tableKeys[0]->Columns[0].Column == 34); + UNIT_ASSERT(tableKeys[0]->Columns[0].Operation == TKeyDesc::EColumnOperation::Read); UNIT_ASSERT(tableKeys[0]->Columns[0].ExpectedType == NUdf::TDataType<ui32>::Id); - UNIT_ASSERT(tableKeys[0]->Columns[1].Column == 56); - UNIT_ASSERT(tableKeys[0]->Columns[1].Operation == TKeyDesc::EColumnOperation::Read); + UNIT_ASSERT(tableKeys[0]->Columns[1].Column == 56); + UNIT_ASSERT(tableKeys[0]->Columns[1].Operation == TKeyDesc::EColumnOperation::Read); UNIT_ASSERT(tableKeys[0]->Columns[1].ExpectedType == NUdf::TDataType<ui64>::Id); - } - + } + Y_UNIT_TEST(TestSelectBothFromInclusiveToInclusiveRange) { TScopedAlloc alloc; TTypeEnvironment env(alloc); auto functionRegistry = CreateFunctionRegistry(CreateBuiltinRegistry()); TKikimrProgramBuilder pgmBuilder(env, *functionRegistry); - auto pgmReturn = pgmBuilder.NewEmptyListOfVoid(); - TTableRangeOptions options(pgmBuilder.GetDefaultTableRangeOptions()); + auto pgmReturn = pgmBuilder.NewEmptyListOfVoid(); + TTableRangeOptions options(pgmBuilder.GetDefaultTableRangeOptions()); TVector<TRuntimeNode> from; TVector<TRuntimeNode> to; from.push_back(pgmBuilder.NewDataLiteral<NUdf::EDataSlot::String>("a")); to.push_back(pgmBuilder.NewDataLiteral<NUdf::EDataSlot::String>("bc")); - options.FromColumns = from; - options.ToColumns = to; - + options.FromColumns = from; + options.ToColumns = to; + TVector<TSelectColumn> columnsToRead; columnsToRead.emplace_back("column1", 34, (ui32)NUdf::TDataType<ui32>::Id); columnsToRead.emplace_back("column2", 56, (ui32)NUdf::TDataType<ui64>::Id); TVector<ui32> keyTypes({ NUdf::TDataType<char*>::Id }); - pgmReturn = pgmBuilder.Append(pgmReturn, pgmBuilder.SetResult("myRes", - pgmBuilder.SelectRange(TTableId(1, 2), - keyTypes, - columnsToRead, options))); - auto pgm = pgmBuilder.Build(pgmReturn, TKikimrProgramBuilder::TBindFlags::DisableOptimization).GetNode(); - //Cout << PrintNode(*pgm) << Endl; - VerifyProgram(pgm, env); - - TExploringNodeVisitor explorer; - explorer.Walk(pgm, env); + pgmReturn = pgmBuilder.Append(pgmReturn, pgmBuilder.SetResult("myRes", + pgmBuilder.SelectRange(TTableId(1, 2), + keyTypes, + columnsToRead, options))); + auto pgm = pgmBuilder.Build(pgmReturn, TKikimrProgramBuilder::TBindFlags::DisableOptimization).GetNode(); + //Cout << PrintNode(*pgm) << Endl; + VerifyProgram(pgm, env); + + TExploringNodeVisitor explorer; + explorer.Walk(pgm, env); TVector<THolder<TKeyDesc>> tableKeys = ExtractTableKeys(explorer, env); - UNIT_ASSERT_VALUES_EQUAL(tableKeys.size(), 1); + UNIT_ASSERT_VALUES_EQUAL(tableKeys.size(), 1); UNIT_ASSERT(tableKeys[0]->TableId.HasSamePath(TTableId(1, 2))); - UNIT_ASSERT(tableKeys[0]->RowOperation == TKeyDesc::ERowOperation::Read); - UNIT_ASSERT(tableKeys[0]->Range.InclusiveFrom); - UNIT_ASSERT(tableKeys[0]->Range.InclusiveTo); - UNIT_ASSERT(!tableKeys[0]->Range.Point); + UNIT_ASSERT(tableKeys[0]->RowOperation == TKeyDesc::ERowOperation::Read); + UNIT_ASSERT(tableKeys[0]->Range.InclusiveFrom); + UNIT_ASSERT(tableKeys[0]->Range.InclusiveTo); + UNIT_ASSERT(!tableKeys[0]->Range.Point); UNIT_ASSERT(tableKeys[0]->Range.From.size() == 1); - UNIT_ASSERT(tableKeys[0]->Range.From[0].Size() == 1); + UNIT_ASSERT(tableKeys[0]->Range.From[0].Size() == 1); UNIT_ASSERT(tableKeys[0]->Range.To.size() == 1); - UNIT_ASSERT(tableKeys[0]->Range.To[0].Size() == 2); - UNIT_ASSERT(tableKeys[0]->KeyColumnTypes.size() == 1); + UNIT_ASSERT(tableKeys[0]->Range.To[0].Size() == 2); + UNIT_ASSERT(tableKeys[0]->KeyColumnTypes.size() == 1); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[0] == NUdf::TDataType<char*>::Id); - UNIT_ASSERT(tableKeys[0]->Columns.size() == 2); - UNIT_ASSERT(tableKeys[0]->Columns[0].Column == 34); - UNIT_ASSERT(tableKeys[0]->Columns[0].Operation == TKeyDesc::EColumnOperation::Read); + UNIT_ASSERT(tableKeys[0]->Columns.size() == 2); + UNIT_ASSERT(tableKeys[0]->Columns[0].Column == 34); + UNIT_ASSERT(tableKeys[0]->Columns[0].Operation == TKeyDesc::EColumnOperation::Read); UNIT_ASSERT(tableKeys[0]->Columns[0].ExpectedType == NUdf::TDataType<ui32>::Id); - UNIT_ASSERT(tableKeys[0]->Columns[1].Column == 56); - UNIT_ASSERT(tableKeys[0]->Columns[1].Operation == TKeyDesc::EColumnOperation::Read); + UNIT_ASSERT(tableKeys[0]->Columns[1].Column == 56); + UNIT_ASSERT(tableKeys[0]->Columns[1].Operation == TKeyDesc::EColumnOperation::Read); UNIT_ASSERT(tableKeys[0]->Columns[1].ExpectedType == NUdf::TDataType<ui64>::Id); - } - + } + Y_UNIT_TEST(TestSelectBothFromExclusiveToExclusiveRange) { TScopedAlloc alloc; TTypeEnvironment env(alloc); auto functionRegistry = CreateFunctionRegistry(CreateBuiltinRegistry()); TKikimrProgramBuilder pgmBuilder(env, *functionRegistry); - auto pgmReturn = pgmBuilder.NewEmptyListOfVoid(); - TTableRangeOptions options(pgmBuilder.GetDefaultTableRangeOptions()); + auto pgmReturn = pgmBuilder.NewEmptyListOfVoid(); + TTableRangeOptions options(pgmBuilder.GetDefaultTableRangeOptions()); TVector<TRuntimeNode> from; TVector<TRuntimeNode> to; from.push_back(pgmBuilder.NewDataLiteral<NUdf::EDataSlot::String>("a")); to.push_back(pgmBuilder.NewDataLiteral<NUdf::EDataSlot::String>("bc")); - options.FromColumns = from; - options.ToColumns = to; + options.FromColumns = from; + options.ToColumns = to; options.Flags = pgmBuilder.TProgramBuilder::NewDataLiteral<ui32>( TReadRangeOptions::TFlags::ExcludeInitValue | TReadRangeOptions::TFlags::ExcludeTermValue); - + TVector<TSelectColumn> columnsToRead; columnsToRead.emplace_back("column1", 34, (ui32)NUdf::TDataType<ui32>::Id); columnsToRead.emplace_back("column2", 56, (ui32)NUdf::TDataType<ui64>::Id); TVector<ui32> keyTypes({ NUdf::TDataType<char*>::Id }); - pgmReturn = pgmBuilder.Append(pgmReturn, pgmBuilder.SetResult("myRes", - pgmBuilder.SelectRange(TTableId(1, 2), - keyTypes, - columnsToRead, options))); - auto pgm = pgmBuilder.Build(pgmReturn, TKikimrProgramBuilder::TBindFlags::DisableOptimization).GetNode(); - //Cout << PrintNode(*pgm) << Endl; - VerifyProgram(pgm, env); - - TExploringNodeVisitor explorer; - explorer.Walk(pgm, env); + pgmReturn = pgmBuilder.Append(pgmReturn, pgmBuilder.SetResult("myRes", + pgmBuilder.SelectRange(TTableId(1, 2), + keyTypes, + columnsToRead, options))); + auto pgm = pgmBuilder.Build(pgmReturn, TKikimrProgramBuilder::TBindFlags::DisableOptimization).GetNode(); + //Cout << PrintNode(*pgm) << Endl; + VerifyProgram(pgm, env); + + TExploringNodeVisitor explorer; + explorer.Walk(pgm, env); TVector<THolder<TKeyDesc>> tableKeys = ExtractTableKeys(explorer, env); - UNIT_ASSERT_VALUES_EQUAL(tableKeys.size(), 1); + UNIT_ASSERT_VALUES_EQUAL(tableKeys.size(), 1); UNIT_ASSERT(tableKeys[0]->TableId.HasSamePath(TTableId(1, 2))); - UNIT_ASSERT(tableKeys[0]->RowOperation == TKeyDesc::ERowOperation::Read); - UNIT_ASSERT(!tableKeys[0]->Range.InclusiveFrom); - UNIT_ASSERT(!tableKeys[0]->Range.InclusiveTo); - UNIT_ASSERT(!tableKeys[0]->Range.Point); + UNIT_ASSERT(tableKeys[0]->RowOperation == TKeyDesc::ERowOperation::Read); + UNIT_ASSERT(!tableKeys[0]->Range.InclusiveFrom); + UNIT_ASSERT(!tableKeys[0]->Range.InclusiveTo); + UNIT_ASSERT(!tableKeys[0]->Range.Point); UNIT_ASSERT(tableKeys[0]->Range.From.size() == 1); - UNIT_ASSERT(tableKeys[0]->Range.From[0].Size() == 1); + UNIT_ASSERT(tableKeys[0]->Range.From[0].Size() == 1); UNIT_ASSERT(tableKeys[0]->Range.To.size() == 1); - UNIT_ASSERT(tableKeys[0]->Range.To[0].Size() == 2); - UNIT_ASSERT(tableKeys[0]->KeyColumnTypes.size() == 1); + UNIT_ASSERT(tableKeys[0]->Range.To[0].Size() == 2); + UNIT_ASSERT(tableKeys[0]->KeyColumnTypes.size() == 1); UNIT_ASSERT(tableKeys[0]->KeyColumnTypes[0] == NUdf::TDataType<char*>::Id); - UNIT_ASSERT(tableKeys[0]->Columns.size() == 2); - UNIT_ASSERT(tableKeys[0]->Columns[0].Column == 34); - UNIT_ASSERT(tableKeys[0]->Columns[0].Operation == TKeyDesc::EColumnOperation::Read); + UNIT_ASSERT(tableKeys[0]->Columns.size() == 2); + UNIT_ASSERT(tableKeys[0]->Columns[0].Column == 34); + UNIT_ASSERT(tableKeys[0]->Columns[0].Operation == TKeyDesc::EColumnOperation::Read); UNIT_ASSERT(tableKeys[0]->Columns[0].ExpectedType == NUdf::TDataType<ui32>::Id); - UNIT_ASSERT(tableKeys[0]->Columns[1].Column == 56); - UNIT_ASSERT(tableKeys[0]->Columns[1].Operation == TKeyDesc::EColumnOperation::Read); + UNIT_ASSERT(tableKeys[0]->Columns[1].Column == 56); + UNIT_ASSERT(tableKeys[0]->Columns[1].Operation == TKeyDesc::EColumnOperation::Read); UNIT_ASSERT(tableKeys[0]->Columns[1].ExpectedType == NUdf::TDataType<ui64>::Id); - } + } Y_UNIT_TEST(TestAcquireLocks) { TScopedAlloc alloc; @@ -749,7 +749,7 @@ Y_UNIT_TEST_SUITE(TMiniKQLProgramBuilderTest) { UNIT_FAIL("Expected exception."); } -} - -} // namespace NMiniKQL -} // namespace NKikimr +} + +} // namespace NMiniKQL +} // namespace NKikimr diff --git a/ydb/core/engine/minikql/flat_local_tx_minikql.h b/ydb/core/engine/minikql/flat_local_tx_minikql.h index 161615dcf35..0feabf20e68 100644 --- a/ydb/core/engine/minikql/flat_local_tx_minikql.h +++ b/ydb/core/engine/minikql/flat_local_tx_minikql.h @@ -60,7 +60,7 @@ public: return NThreading::MakeFuture<TTableResults>(results); } - virtual void ResolveTables(const TVector<TTable>& tables, NActors::TActorId responseTo) override { + virtual void ResolveTables(const TVector<TTable>& tables, NActors::TActorId responseTo) override { Y_UNUSED(tables); Y_UNUSED(responseTo); Y_FAIL("Not implemented for local resolve."); @@ -72,7 +72,7 @@ private: class TFlatLocalMiniKQL : public NTabletFlatExecutor::ITransaction { ui64 TabletId = Max<ui64>(); - const TActorId Sender; + const TActorId Sender; const TLocalMiniKQLProgram SourceProgram; const TMiniKQLFactory* const Factory; @@ -357,7 +357,7 @@ class TFlatLocalMiniKQL : public NTabletFlatExecutor::ITransaction { public: TFlatLocalMiniKQL( - TActorId sender, + TActorId sender, const TLocalMiniKQLProgram &program, const TMiniKQLFactory* factory) : Sender(sender) diff --git a/ydb/core/engine/minikql/flat_local_tx_read_columns.h b/ydb/core/engine/minikql/flat_local_tx_read_columns.h index e68ccb3fa41..57d3c7bef9d 100644 --- a/ydb/core/engine/minikql/flat_local_tx_read_columns.h +++ b/ydb/core/engine/minikql/flat_local_tx_read_columns.h @@ -13,7 +13,7 @@ namespace NMiniKQL { class TFlatLocalReadColumns : public NTabletFlatExecutor::ITransaction { public: - TFlatLocalReadColumns(TActorId sender, TEvTablet::TEvLocalReadColumns::TPtr &ev) + TFlatLocalReadColumns(TActorId sender, TEvTablet::TEvLocalReadColumns::TPtr &ev) : Sender(sender) , Ev(ev) {} @@ -196,7 +196,7 @@ private: } private: - const TActorId Sender; + const TActorId Sender; TEvTablet::TEvLocalReadColumns::TPtr Ev; TAutoPtr<TEvTablet::TEvLocalReadColumnsResponse> Response; diff --git a/ydb/core/engine/minikql/flat_local_tx_scheme.h b/ydb/core/engine/minikql/flat_local_tx_scheme.h index 3bb2833ffc6..c70476a0cdb 100644 --- a/ydb/core/engine/minikql/flat_local_tx_scheme.h +++ b/ydb/core/engine/minikql/flat_local_tx_scheme.h @@ -16,7 +16,7 @@ namespace NMiniKQL { class TFlatLocalSchemeTx : public NTabletFlatExecutor::ITransaction { public: - TFlatLocalSchemeTx(TActorId sender, TEvTablet::TEvLocalSchemeTx::TPtr &ev) + TFlatLocalSchemeTx(TActorId sender, TEvTablet::TEvLocalSchemeTx::TPtr &ev) : Sender(sender) , Ev(ev) {} @@ -52,7 +52,7 @@ public: } private: - const TActorId Sender; + const TActorId Sender; TEvTablet::TEvLocalSchemeTx::TPtr Ev; TAutoPtr<TEvTablet::TEvLocalSchemeTxResponse> Response; }; diff --git a/ydb/core/engine/mkql_engine_flat.cpp b/ydb/core/engine/mkql_engine_flat.cpp index cd78b8ba7fb..c7fe388eef8 100644 --- a/ydb/core/engine/mkql_engine_flat.cpp +++ b/ydb/core/engine/mkql_engine_flat.cpp @@ -299,7 +299,7 @@ public: if (params) { TKikimrProgramBuilder builder(Env, *Settings.FunctionRegistry); TRuntimeNode paramsNode = DeserializeRuntimeNode(params, Env); - programNode = builder.Bind(programNode, paramsNode, TKikimrProgramBuilder::TBindFlags::OptimizeLiterals); + programNode = builder.Bind(programNode, paramsNode, TKikimrProgramBuilder::TBindFlags::OptimizeLiterals); } auto listType = programNode.GetStaticType(); diff --git a/ydb/core/engine/mkql_engine_flat.h b/ydb/core/engine/mkql_engine_flat.h index 51df5cfeeba..7bee67584b9 100644 --- a/ydb/core/engine/mkql_engine_flat.h +++ b/ydb/core/engine/mkql_engine_flat.h @@ -1,7 +1,7 @@ #pragma once - -#include "mkql_keys.h" - + +#include "mkql_keys.h" + #include <ydb/library/yql/minikql/defs.h> #include <ydb/library/yql/minikql/mkql_node.h> #include <ydb/library/yql/minikql/computation/mkql_computation_node_holders.h> diff --git a/ydb/core/engine/mkql_engine_flat_ut.cpp b/ydb/core/engine/mkql_engine_flat_ut.cpp index 8e09e7b2fcf..e2b12e2d7a5 100644 --- a/ydb/core/engine/mkql_engine_flat_ut.cpp +++ b/ydb/core/engine/mkql_engine_flat_ut.cpp @@ -111,7 +111,7 @@ namespace { TIntrusivePtr<ITimeProvider> TimeProvider; TScopedAlloc Alloc; TTypeEnvironment Env; - TKikimrProgramBuilder PgmBuilder; + TKikimrProgramBuilder PgmBuilder; TShardDbState ShardDbState; std::function<void(TEngineFlatSettings&)> SettingsConfigurer; std::function<void(ui64, const TString&)> ShardProgramInspector; diff --git a/ydb/core/engine/mkql_keys.cpp b/ydb/core/engine/mkql_keys.cpp index 3031ac64754..05afb89adc2 100644 --- a/ydb/core/engine/mkql_keys.cpp +++ b/ydb/core/engine/mkql_keys.cpp @@ -1,11 +1,11 @@ #include "mkql_keys.h" - + #include <ydb/library/yql/minikql/mkql_node_cast.h> #include <ydb/core/base/domain.h> #include <ydb/core/scheme_types/scheme_types_defs.h> #include <util/generic/maybe.h> -#include <util/generic/algorithm.h> +#include <util/generic/algorithm.h> #include <functional> namespace NKikimr { diff --git a/ydb/core/engine/mkql_keys.h b/ydb/core/engine/mkql_keys.h index 37301f709af..635ad3ce147 100644 --- a/ydb/core/engine/mkql_keys.h +++ b/ydb/core/engine/mkql_keys.h @@ -1,15 +1,15 @@ #pragma once - -#include "kikimr_program_builder.h" - + +#include "kikimr_program_builder.h" + #include <ydb/library/yql/minikql/mkql_node.h> #include <ydb/library/yql/minikql/mkql_node_visitor.h> #include <ydb/core/scheme_types/scheme_types.h> - + #include <util/stream/output.h> // for IOutputStream #include <library/cpp/deprecated/enum_codegen/enum_codegen.h> - + namespace NKikimr { namespace NMiniKQL { diff --git a/ydb/core/engine/ut/ya.make b/ydb/core/engine/ut/ya.make index a2dcade0418..02810813ab4 100644 --- a/ydb/core/engine/ut/ya.make +++ b/ydb/core/engine/ut/ya.make @@ -16,7 +16,7 @@ SIZE(MEDIUM) SRCS( mkql_engine_flat_host_ut.cpp mkql_engine_flat_ut.cpp - kikimr_program_builder_ut.cpp + kikimr_program_builder_ut.cpp mkql_proto_ut.cpp ) diff --git a/ydb/core/engine/ya.make b/ydb/core/engine/ya.make index 6a6679eb99f..df644f17dc4 100644 --- a/ydb/core/engine/ya.make +++ b/ydb/core/engine/ya.make @@ -7,11 +7,11 @@ OWNER( ) SRCS( - kikimr_program_builder.cpp + kikimr_program_builder.cpp mkql_engine_flat.cpp mkql_engine_flat_extfunc.cpp mkql_engine_flat_host.cpp - mkql_keys.cpp + mkql_keys.cpp mkql_proto.cpp mkql_proto.h ) @@ -34,7 +34,7 @@ PEERDIR( ydb/library/yql/public/decimal ) -YQL_LAST_ABI_VERSION() +YQL_LAST_ABI_VERSION() END() diff --git a/ydb/core/grpc_services/base/base.h b/ydb/core/grpc_services/base/base.h index 1d7bc42eda1..44b25c4a5f9 100644 --- a/ydb/core/grpc_services/base/base.h +++ b/ydb/core/grpc_services/base/base.h @@ -3,7 +3,7 @@ #include <grpc++/support/byte_buffer.h> #include <grpc++/support/slice.h> -#include <library/cpp/grpc/server/grpc_request_base.h> +#include <library/cpp/grpc/server/grpc_request_base.h> #include <library/cpp/string_utils/quote/quote.h> #include <ydb/public/api/protos/ydb_issue_message.pb.h> @@ -46,7 +46,7 @@ using TYdbIssueMessageType = Ydb::Issue::IssueMessage; std::pair<TString, TString> SplitPath(const TMaybe<TString>& database, const TString& path); std::pair<TString, TString> SplitPath(const TString& path); -void RefreshToken(const TString& token, const TString& database, const TActorContext& ctx, TActorId id); +void RefreshToken(const TString& token, const TString& database, const TActorContext& ctx, TActorId id); struct TRpcServices { enum EServiceId { @@ -288,7 +288,7 @@ public: } private: - TIntrusivePtr<NGrpc::IRequestContextBase> Ctx_; + TIntrusivePtr<NGrpc::IRequestContextBase> Ctx_; TMessage* RespData_; //Allocated on arena owned by implementation of IRequestContextBase const TString RequestName_; const ui64 Ru_; @@ -323,9 +323,9 @@ class IRequestProxyCtx : public virtual IRequestCtxBase { public: virtual ~IRequestProxyCtx() = default; virtual const TMaybe<TString> GetYdbToken() const = 0; - virtual void UpdateAuthState(NGrpc::TAuthState::EAuthState state) = 0; + virtual void UpdateAuthState(NGrpc::TAuthState::EAuthState state) = 0; virtual void SetInternalToken(const TString& token) = 0; - virtual const NGrpc::TAuthState& GetAuthState() const = 0; + virtual const NGrpc::TAuthState& GetAuthState() const = 0; virtual void ReplyUnauthenticated(const TString& msg = "") = 0; virtual void ReplyUnavaliable() = 0; @@ -411,7 +411,7 @@ class TRefreshTokenImpl : public IRequestProxyCtx , public TEventLocal<TRefreshTokenImpl, TRpcServices::EvRefreshTokenRequest> { public: - TRefreshTokenImpl(const TString& token, const TString& database, TActorId from) + TRefreshTokenImpl(const TString& token, const TString& database, TActorId from) : Token_(token) , Database_(database) , From_(from) @@ -422,7 +422,7 @@ public: return Token_; } - void UpdateAuthState(NGrpc::TAuthState::EAuthState state) override { + void UpdateAuthState(NGrpc::TAuthState::EAuthState state) override { State_.State = state; } @@ -434,7 +434,7 @@ public: return Database_; } - const NGrpc::TAuthState& GetAuthState() const override { + const NGrpc::TAuthState& GetAuthState() const override { return State_; } @@ -483,7 +483,7 @@ public: Y_UNUSED(database); } - TActorId GetFromId() const { + TActorId GetFromId() const { return From_; } @@ -525,8 +525,8 @@ public: private: const TString Token_; const TString Database_; - const TActorId From_; - NGrpc::TAuthState State_; + const TActorId From_; + NGrpc::TAuthState State_; TString InternalToken_; NYql::TIssueManager IssueManager_; }; @@ -563,7 +563,7 @@ public: if (res.empty()) { return {}; } - return TString{res[0]}; + return TString{res[0]}; } const TMaybe<TString> GetDatabaseName() const override { @@ -574,12 +574,12 @@ public: return CGIUnescapeRet(res[0]); } - void UpdateAuthState(NGrpc::TAuthState::EAuthState state) override { + void UpdateAuthState(NGrpc::TAuthState::EAuthState state) override { auto& s = Ctx_->GetAuthState(); s.State = state; } - const NGrpc::TAuthState& GetAuthState() const override { + const NGrpc::TAuthState& GetAuthState() const override { return Ctx_->GetAuthState(); } @@ -593,14 +593,14 @@ public: } void ReplyUnavaliable() override { - Ctx_->Attach(TActorId()); + Ctx_->Attach(TActorId()); TResponse resp; FillYdbStatus(resp, IssueManager_.GetIssues(), Ydb::StatusIds::UNAVAILABLE); Ctx_->WriteAndFinish(std::move(resp), grpc::Status()); } void ReplyWithYdbStatus(Ydb::StatusIds::StatusCode status) override { - Ctx_->Attach(TActorId()); + Ctx_->Attach(TActorId()); TResponse resp; FillYdbStatus(resp, IssueManager_.GetIssues(), status); Ctx_->WriteAndFinish(std::move(resp), grpc::Status()); @@ -674,7 +674,7 @@ public: return ToMaybe(Ctx_->GetPeerMetaValues(key)); } - void RefreshToken(const TString& token, const TActorContext& ctx, TActorId id) { + void RefreshToken(const TString& token, const TActorContext& ctx, TActorId id) { NGRpcService::RefreshToken(token, GetDatabaseName().GetOrElse(""), ctx, id); } @@ -800,7 +800,7 @@ public: using TRequest = TReq; using TResponse = TResp; - TGRpcRequestWrapperImpl(NGrpc::IRequestContextBase* ctx) + TGRpcRequestWrapperImpl(NGrpc::IRequestContextBase* ctx) : Ctx_(ctx) { } @@ -809,7 +809,7 @@ public: if (res.empty()) { return {}; } - return TString{res[0]}; + return TString{res[0]}; } const TMaybe<TString> GetDatabaseName() const override { @@ -820,12 +820,12 @@ public: return CGIUnescapeRet(res[0]); } - void UpdateAuthState(NGrpc::TAuthState::EAuthState state) override { + void UpdateAuthState(NGrpc::TAuthState::EAuthState state) override { auto& s = Ctx_->GetAuthState(); s.State = state; } - const NGrpc::TAuthState& GetAuthState() const override { + const NGrpc::TAuthState& GetAuthState() const override { return Ctx_->GetAuthState(); } @@ -960,9 +960,9 @@ public: } void SetClientLostAction(std::function<void()>&& cb) override { - auto shutdown = [cb{move(cb)}](const NGrpc::IRequestContextBase::TAsyncFinishResult& future) mutable { + auto shutdown = [cb{move(cb)}](const NGrpc::IRequestContextBase::TAsyncFinishResult& future) mutable { Y_ASSERT(future.HasValue()); - if (future.GetValue() == NGrpc::IRequestContextBase::EFinishStatus::CANCEL) { + if (future.GetValue() == NGrpc::IRequestContextBase::EFinishStatus::CANCEL) { cb(); } }; @@ -1015,7 +1015,7 @@ private: return google::protobuf::Arena::CreateMessage<TResponse>(Ctx_->GetArena()); } - TIntrusivePtr<NGrpc::IRequestContextBase> Ctx_; + TIntrusivePtr<NGrpc::IRequestContextBase> Ctx_; TString InternalToken_; NYql::TIssueManager IssueManager; Ydb::CostInfo* CostInfo = nullptr; @@ -1080,7 +1080,7 @@ public: static constexpr bool IsOp = IsOperation; static constexpr TRateLimiterMode RateLimitMode = RlMode; - TGRpcRequestWrapper(NGrpc::IRequestContextBase* ctx) + TGRpcRequestWrapper(NGrpc::IRequestContextBase* ctx) : TGRpcRequestWrapperImpl<TRpcId, TReq, TResp, IsOperation, TGRpcRequestWrapper<TRpcId, TReq, TResp, IsOperation, RlMode>>(ctx) { } @@ -1128,7 +1128,7 @@ public: static IActor* CreateRpcActor(typename std::conditional<IsOperation, IRequestOpCtx, IRequestNoOpCtx>::type* msg); static constexpr bool IsOp = IsOperation; static constexpr TRateLimiterMode RateLimitMode = RlMode; - TGRpcRequestValidationWrapper(NGrpc::IRequestContextBase* ctx) + TGRpcRequestValidationWrapper(NGrpc::IRequestContextBase* ctx) : TGRpcRequestWrapperImpl<TRpcId, TReq, TResp, IsOperation, TGRpcRequestValidationWrapper<TRpcId, TReq, TResp, IsOperation, RlMode>>(ctx) { } diff --git a/ydb/core/grpc_services/counters/counters.cpp b/ydb/core/grpc_services/counters/counters.cpp index 6c7ea619e00..8fc8ede9174 100644 --- a/ydb/core/grpc_services/counters/counters.cpp +++ b/ydb/core/grpc_services/counters/counters.cpp @@ -31,7 +31,7 @@ struct TYdbRpcCounters { THashMap<ui32, NMonitoring::TDynamicCounters::TCounterPtr> ResponseByStatus; }; -class TYdbCounterBlock : public NGrpc::ICounterBlock { +class TYdbCounterBlock : public NGrpc::ICounterBlock { protected: bool Streaming = false; bool Percentile = false; @@ -142,7 +142,7 @@ public: } } - NGrpc::ICounterBlockPtr Clone() override { + NGrpc::ICounterBlockPtr Clone() override { return this; } @@ -463,7 +463,7 @@ private: }; -class TYdbCounterBlockWrapper : public NGrpc::ICounterBlock { +class TYdbCounterBlockWrapper : public NGrpc::ICounterBlock { TYdbCounterBlockPtr Common; TString ServiceName; TString RequestName; @@ -542,7 +542,7 @@ public: Db->FinishProcessing(requestSize, responseSize, ok, status, requestDuration); } - NGrpc::ICounterBlockPtr Clone() override { + NGrpc::ICounterBlockPtr Clone() override { return new TYdbCounterBlockWrapper(Common, ServiceName, RequestName, Percentile, Streaming); } @@ -617,7 +617,7 @@ TServiceCounterCB::TServiceCounterCB(NMonitoring::TDynamicCounterPtr counters, T } } -NGrpc::ICounterBlockPtr TServiceCounterCB::operator()(const char* serviceName, +NGrpc::ICounterBlockPtr TServiceCounterCB::operator()(const char* serviceName, const char* requestName, bool percentile, bool streaming) const { auto block = MakeIntrusive<TYdbCounterBlock>(Counters, serviceName, requestName, percentile, streaming); @@ -626,7 +626,7 @@ NGrpc::ICounterBlockPtr TServiceCounterCB::operator()(const char* serviceName, ActorSystem->Send(ActorId, new TUpdaterActor::TEvRegisterItem(block)); } - NGrpc::ICounterBlockPtr res(block); + NGrpc::ICounterBlockPtr res(block); if (ActorSystem && AppData(ActorSystem)->FeatureFlags.GetEnableDbCounters()) { res = MakeIntrusive<TYdbCounterBlockWrapper>(block, serviceName, requestName, percentile, streaming); } diff --git a/ydb/core/grpc_services/counters/counters.h b/ydb/core/grpc_services/counters/counters.h index 9e22ae82155..da499dac9e0 100644 --- a/ydb/core/grpc_services/counters/counters.h +++ b/ydb/core/grpc_services/counters/counters.h @@ -2,7 +2,7 @@ #include <ydb/core/sys_view/common/events.h> -#include <library/cpp/grpc/server/grpc_counters.h> +#include <library/cpp/grpc/server/grpc_counters.h> namespace NKikimr { namespace NGRpcService { @@ -11,7 +11,7 @@ class TServiceCounterCB { public: TServiceCounterCB(NMonitoring::TDynamicCounterPtr counters, TActorSystem *actorSystem); - NGrpc::ICounterBlockPtr operator()(const char* serviceName, const char* requestName, + NGrpc::ICounterBlockPtr operator()(const char* serviceName, const char* requestName, bool percentile = true, bool streaming = false) const; private: diff --git a/ydb/core/grpc_services/grpc_helper.cpp b/ydb/core/grpc_services/grpc_helper.cpp index aec7e1a6128..b1ef484cc7d 100644 --- a/ydb/core/grpc_services/grpc_helper.cpp +++ b/ydb/core/grpc_services/grpc_helper.cpp @@ -5,15 +5,15 @@ namespace NGRpcService { //using namespace NActors; -NGrpc::IGRpcRequestLimiterPtr TCreateLimiterCB::operator()(const char* serviceName, const char* requestName, i64 limit) const { +NGrpc::IGRpcRequestLimiterPtr TCreateLimiterCB::operator()(const char* serviceName, const char* requestName, i64 limit) const { TString fullName = TString(serviceName) + "_" + requestName; return LimiterRegistry->RegisterRequestType(fullName, limit); } -class TRequestInFlightLimiter : public NGrpc::IGRpcRequestLimiter { +class TRequestInFlightLimiter : public NGrpc::IGRpcRequestLimiter { private: - NGrpc::TInFlightLimiterImpl<TControlWrapper> RequestLimiter; + NGrpc::TInFlightLimiterImpl<TControlWrapper> RequestLimiter; public: explicit TRequestInFlightLimiter(TControlWrapper limiter) @@ -30,7 +30,7 @@ public: }; -NGrpc::IGRpcRequestLimiterPtr TInFlightLimiterRegistry::RegisterRequestType(TString name, i64 limit) { +NGrpc::IGRpcRequestLimiterPtr TInFlightLimiterRegistry::RegisterRequestType(TString name, i64 limit) { TGuard<TMutex> g(Lock); if (!PerTypeLimiters.count(name)) { TControlWrapper control(limit, 0, 1000000); diff --git a/ydb/core/grpc_services/grpc_helper.h b/ydb/core/grpc_services/grpc_helper.h index 31a20ea4824..9a2a673c7b7 100644 --- a/ydb/core/grpc_services/grpc_helper.h +++ b/ydb/core/grpc_services/grpc_helper.h @@ -5,7 +5,7 @@ #include <ydb/core/control/immediate_control_board_impl.h> #include <ydb/core/grpc_services/counters/counters.h> -#include <library/cpp/grpc/server/grpc_request.h> +#include <library/cpp/grpc/server/grpc_request.h> namespace NKikimr { namespace NGRpcService { @@ -14,14 +14,14 @@ class TInFlightLimiterRegistry : public TThrRefBase { private: TIntrusivePtr<NKikimr::TControlBoard> Icb; TMutex Lock; - THashMap<TString, NGrpc::IGRpcRequestLimiterPtr> PerTypeLimiters; + THashMap<TString, NGrpc::IGRpcRequestLimiterPtr> PerTypeLimiters; public: explicit TInFlightLimiterRegistry(TIntrusivePtr<NKikimr::TControlBoard> icb) : Icb(icb) {} - NGrpc::IGRpcRequestLimiterPtr RegisterRequestType(TString name, i64 limit); + NGrpc::IGRpcRequestLimiterPtr RegisterRequestType(TString name, i64 limit); }; class TCreateLimiterCB { @@ -30,7 +30,7 @@ public: : LimiterRegistry(limiterRegistry) {} - NGrpc::IGRpcRequestLimiterPtr operator()(const char* serviceName, const char* requestName, i64 limit) const; + NGrpc::IGRpcRequestLimiterPtr operator()(const char* serviceName, const char* requestName, i64 limit) const; private: TIntrusivePtr<TInFlightLimiterRegistry> LimiterRegistry; diff --git a/ydb/core/grpc_services/grpc_mon.cpp b/ydb/core/grpc_services/grpc_mon.cpp index b0ed5b8267c..cd15157b052 100644 --- a/ydb/core/grpc_services/grpc_mon.cpp +++ b/ydb/core/grpc_services/grpc_mon.cpp @@ -3,7 +3,7 @@ #include <ydb/core/base/events.h> #include <library/cpp/cache/cache.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <library/cpp/actors/core/mon.h> #include <library/cpp/actors/core/hfunc.h> @@ -104,9 +104,9 @@ private: TLRUCache<TString, TPeerInfo> Peers; }; -TActorId GrpcMonServiceId() { +TActorId GrpcMonServiceId() { const char x[12] = "GrpcMonSvc!"; - return TActorId(0, TStringBuf(x, 12)); + return TActorId(0, TStringBuf(x, 12)); } IActor* CreateGrpcMonService() { diff --git a/ydb/core/grpc_services/grpc_mon.h b/ydb/core/grpc_services/grpc_mon.h index 43ca6222d1e..f5996769777 100644 --- a/ydb/core/grpc_services/grpc_mon.h +++ b/ydb/core/grpc_services/grpc_mon.h @@ -7,7 +7,7 @@ namespace NKikimr { namespace NGRpcService { -TActorId GrpcMonServiceId(); +TActorId GrpcMonServiceId(); IActor* CreateGrpcMonService(); void ReportGrpcReqToMon(NActors::TActorSystem&, const TString& fromAddress); diff --git a/ydb/core/grpc_services/grpc_request_proxy.cpp b/ydb/core/grpc_services/grpc_request_proxy.cpp index a4d5cb5f0b8..a1e7accb877 100644 --- a/ydb/core/grpc_services/grpc_request_proxy.cpp +++ b/ydb/core/grpc_services/grpc_request_proxy.cpp @@ -290,7 +290,7 @@ private: THashSet<TSubDomainKey> SubDomainKeys; bool AllowYdbRequestsWithoutDatabase = true; NKikimrConfig::TAppConfig AppConfig; - TActorId SchemeCache; + TActorId SchemeCache; bool DynamicNode = false; TString RootDatabase; TIntrusivePtr<TGrpcProxyCounters> Counters; @@ -381,9 +381,9 @@ void TGRpcRequestProxyImpl::HandlePoolStatus(TEvTenantPool::TEvTenantPoolStatus: void TGRpcRequestProxyImpl::HandleRefreshToken(TRefreshTokenImpl::TPtr& ev, const TActorContext& ctx) { const auto record = ev->Get(); ctx.Send(record->GetFromId(), new TGRpcRequestProxy::TEvRefreshTokenResponse { - record->GetAuthState().State == NGrpc::TAuthState::EAuthState::AS_OK, + record->GetAuthState().State == NGrpc::TAuthState::EAuthState::AS_OK, record->GetInternalToken(), - record->GetAuthState().State == NGrpc::TAuthState::EAuthState::AS_UNAVAILABLE, + record->GetAuthState().State == NGrpc::TAuthState::EAuthState::AS_UNAVAILABLE, NYql::TIssues()}); } @@ -426,8 +426,8 @@ void TGRpcRequestProxyImpl::HandleUndelivery(TEvents::TEvUndelivered::TPtr& ev) bool TGRpcRequestProxyImpl::IsAuthStateOK(const IRequestProxyCtx& ctx) { const auto& state = ctx.GetAuthState(); - return state.State == NGrpc::TAuthState::AS_OK || - state.State == NGrpc::TAuthState::AS_FAIL && state.NeedAuth == false || + return state.State == NGrpc::TAuthState::AS_OK || + state.State == NGrpc::TAuthState::AS_FAIL && state.NeedAuth == false || state.NeedAuth == false && !ctx.GetYdbToken(); } diff --git a/ydb/core/grpc_services/grpc_request_proxy.h b/ydb/core/grpc_services/grpc_request_proxy.h index f540c2b610e..97315f6e9fb 100644 --- a/ydb/core/grpc_services/grpc_request_proxy.h +++ b/ydb/core/grpc_services/grpc_request_proxy.h @@ -152,11 +152,11 @@ protected: void Handle(TEvDataStreamsStartStreamEncryptionRequest::TPtr& ev, const TActorContext& ctx); void Handle(TEvDataStreamsStopStreamEncryptionRequest::TPtr& ev, const TActorContext& ctx); - TActorId DiscoveryCacheActorID; + TActorId DiscoveryCacheActorID; }; -inline TActorId CreateGRpcRequestProxyId() { - const auto actorId = TActorId(0, "GRpcReqProxy"); +inline TActorId CreateGRpcRequestProxyId() { + const auto actorId = TActorId(0, "GRpcReqProxy"); return actorId; } diff --git a/ydb/core/grpc_services/operation_helpers.cpp b/ydb/core/grpc_services/operation_helpers.cpp index 5d0f4e704a8..66e95a319ab 100644 --- a/ydb/core/grpc_services/operation_helpers.cpp +++ b/ydb/core/grpc_services/operation_helpers.cpp @@ -44,7 +44,7 @@ IEventBase* CreateNavigateForPath(const TString& path) { return new TEvTxProxySchemeCache::TEvNavigateKeySet(request.Release()); } -TActorId CreatePipeClient(ui64 id, const TActorContext& ctx) { +TActorId CreatePipeClient(ui64 id, const TActorContext& ctx) { NTabletPipe::TClientConfig clientConfig; clientConfig.RetryPolicy = {.RetryLimitCount = 3}; return ctx.RegisterWithSameMailbox(NTabletPipe::CreateClient(ctx.SelfID, id, clientConfig)); diff --git a/ydb/core/grpc_services/operation_helpers.h b/ydb/core/grpc_services/operation_helpers.h index 06c6f1d586d..7e46b658879 100644 --- a/ydb/core/grpc_services/operation_helpers.h +++ b/ydb/core/grpc_services/operation_helpers.h @@ -19,7 +19,7 @@ namespace NGRpcService { class IRequestOpCtx; IEventBase* CreateNavigateForPath(const TString& path); -TActorId CreatePipeClient(ui64 id, const TActorContext& ctx); +TActorId CreatePipeClient(ui64 id, const TActorContext& ctx); Ydb::TOperationId ToOperationId(const NKikimrIndexBuilder::TIndexBuild& build); void ToOperation(const NKikimrIndexBuilder::TIndexBuild& build, Ydb::Operations::Operation* operation); bool TryGetId(const NOperationId::TOperationId& operationId, ui64& id); diff --git a/ydb/core/grpc_services/rpc_alter_table.cpp b/ydb/core/grpc_services/rpc_alter_table.cpp index 6bb30f25488..d1f2755b66d 100644 --- a/ydb/core/grpc_services/rpc_alter_table.cpp +++ b/ydb/core/grpc_services/rpc_alter_table.cpp @@ -83,7 +83,7 @@ class TAlterTableRPC : public TRpcSchemeRequestActor<TAlterTableRPC, TEvAlterTab void PassAway() override { if (SSPipeClient) { NTabletPipe::CloseClient(SelfId(), SSPipeClient); - SSPipeClient = TActorId(); + SSPipeClient = TActorId(); } IActor::PassAway(); } @@ -528,11 +528,11 @@ private: } ui64 TxId = 0; ui64 SchemeshardId = 0; - TActorId SchemeCache; + TActorId SchemeCache; TString DatabaseName; TIntrusivePtr<NTxProxy::TTxProxyMon> TxProxyMon; TString LogPrefix; - TActorId SSPipeClient; + TActorId SSPipeClient; THolder<const NACLib::TUserToken> UserToken; TTableProfiles Profiles; }; diff --git a/ydb/core/grpc_services/rpc_calls.cpp b/ydb/core/grpc_services/rpc_calls.cpp index d713d6ba5c8..b94d168147c 100644 --- a/ydb/core/grpc_services/rpc_calls.cpp +++ b/ydb/core/grpc_services/rpc_calls.cpp @@ -51,12 +51,12 @@ std::pair<TString, TString> SplitPath(const TString& path) { return {path.substr(0, splitPos), path.substr(splitPos + 1)}; } -void RefreshToken(const TString& token, const TString& database, const TActorContext& ctx, TActorId from) { +void RefreshToken(const TString& token, const TString& database, const TActorContext& ctx, TActorId from) { ctx.Send(CreateGRpcRequestProxyId(), new TRefreshTokenImpl(token, database, from)); } void TRefreshTokenImpl::ReplyUnauthenticated(const TString&) { - TActivationContext::Send(new IEventHandle(From_, TActorId(), + TActivationContext::Send(new IEventHandle(From_, TActorId(), new TGRpcRequestProxy::TEvRefreshTokenResponse { false, "", false, IssueManager_.GetIssues()})); } diff --git a/ydb/core/grpc_services/rpc_cms.cpp b/ydb/core/grpc_services/rpc_cms.cpp index 4e9550ffc7c..78450233069 100644 --- a/ydb/core/grpc_services/rpc_cms.cpp +++ b/ydb/core/grpc_services/rpc_cms.cpp @@ -19,7 +19,7 @@ class TCmsRPC : public TRpcOperationRequestActor<TCmsRPC<TRequest, TCmsRequest, using TThis = TCmsRPC<TRequest, TCmsRequest, TCmsResponse>; using TBase = TRpcOperationRequestActor<TThis, TRequest>; - TActorId CmsPipe; + TActorId CmsPipe; public: TCmsRPC(TRequest* msg) diff --git a/ydb/core/grpc_services/rpc_discovery.cpp b/ydb/core/grpc_services/rpc_discovery.cpp index 176c7df10a1..75550f92acf 100644 --- a/ydb/core/grpc_services/rpc_discovery.cpp +++ b/ydb/core/grpc_services/rpc_discovery.cpp @@ -48,7 +48,7 @@ namespace NDiscoveryPrivate { THashMap<TString, THolder<TEvStateStorage::TEvBoardInfo>> NewInfo; struct TWaiter { - TActorId ActorId; + TActorId ActorId; ui64 Cookie; }; @@ -128,7 +128,7 @@ namespace NDiscoveryPrivate { class TListEndpointsRPC : public TActorBootstrapped<TListEndpointsRPC> { THolder<TEvListEndpointsRequest> Request; - const TActorId CacheId; + const TActorId CacheId; const bool RequestScheme = true; @@ -144,7 +144,7 @@ public: return NKikimrServices::TActivity::GRPC_REQ; } - TListEndpointsRPC(TEvListEndpointsRequest::TPtr &msg, TActorId cacheId) + TListEndpointsRPC(TEvListEndpointsRequest::TPtr &msg, TActorId cacheId) : Request(msg->Release().Release()) , CacheId(cacheId) {} @@ -154,7 +154,7 @@ public: Lookup(Request->GetProtoRequest()->database()); // request self node info - Send(GetNameserviceActorId(), new TEvInterconnect::TEvGetNode(SelfId().NodeId())); + Send(GetNameserviceActorId(), new TEvInterconnect::TEvGetNode(SelfId().NodeId())); // request path info if (RequestScheme) { diff --git a/ydb/core/grpc_services/rpc_get_operation.cpp b/ydb/core/grpc_services/rpc_get_operation.cpp index 097a8901f4b..6f7a1406863 100644 --- a/ydb/core/grpc_services/rpc_get_operation.cpp +++ b/ydb/core/grpc_services/rpc_get_operation.cpp @@ -275,7 +275,7 @@ private: TOperationId OperationId_; ui64 RawOperationId_ = 0; - TActorId PipeActorId_; + TActorId PipeActorId_; }; void TGRpcRequestProxy::Handle(TEvGetOperationRequest::TPtr& ev, const TActorContext& ctx) { diff --git a/ydb/core/grpc_services/rpc_get_shard_locations.cpp b/ydb/core/grpc_services/rpc_get_shard_locations.cpp index 82f2880af95..f7088c8eb60 100644 --- a/ydb/core/grpc_services/rpc_get_shard_locations.cpp +++ b/ydb/core/grpc_services/rpc_get_shard_locations.cpp @@ -34,7 +34,7 @@ private: TAutoPtr<TEvGetShardLocationsRequest> Request; Ydb::ClickhouseInternal::GetShardLocationsResult Result; - THashMap<ui64, TActorId> ShardPipes; + THashMap<ui64, TActorId> ShardPipes; THashMap<ui64, ui32> ShardNodes; THashMap<ui32, TNodeInfo> NodeInfos; @@ -102,7 +102,7 @@ private: } // Get list of cluster nodes - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); ctx.Send(nameserviceId, new TEvInterconnect::TEvListNodes()); TBase::Become(&TThis::StateWaitResolve); diff --git a/ydb/core/grpc_services/rpc_kh_describe.cpp b/ydb/core/grpc_services/rpc_kh_describe.cpp index d5b3c308a9c..e2c48213ced 100644 --- a/ydb/core/grpc_services/rpc_kh_describe.cpp +++ b/ydb/core/grpc_services/rpc_kh_describe.cpp @@ -33,7 +33,7 @@ private: Ydb::ClickhouseInternal::DescribeTableResult Result; TDuration Timeout; - TActorId TimeoutTimerActorId; + TActorId TimeoutTimerActorId; bool WaitingResolveReply; bool Finished; diff --git a/ydb/core/grpc_services/rpc_read_columns.cpp b/ydb/core/grpc_services/rpc_read_columns.cpp index fa005e25781..4baae606a97 100644 --- a/ydb/core/grpc_services/rpc_read_columns.cpp +++ b/ydb/core/grpc_services/rpc_read_columns.cpp @@ -30,10 +30,10 @@ private: static constexpr ui32 DEFAULT_TIMEOUT_SEC = 5*60; TAutoPtr<TEvReadColumnsRequest> Request; - TActorId SchemeCache; + TActorId SchemeCache; TActorId LeaderPipeCache; TDuration Timeout; - TActorId TimeoutTimerActorId; + TActorId TimeoutTimerActorId; bool WaitingResolveReply; bool Finished; Ydb::ClickhouseInternal::ScanResult Result; @@ -59,7 +59,7 @@ private: ui32 ShardRequestCount; ui32 ShardReplyCount; - TActorId SysViewScanActor; + TActorId SysViewScanActor; std::unique_ptr<IBlockBuilder> BlockBuilder; TVector<NScheme::TTypeId> ValueColumnTypes; ui64 SysViewMaxRows; diff --git a/ydb/core/grpc_services/rpc_read_table.cpp b/ydb/core/grpc_services/rpc_read_table.cpp index 3446f70a790..bbb87907289 100644 --- a/ydb/core/grpc_services/rpc_read_table.cpp +++ b/ydb/core/grpc_services/rpc_read_table.cpp @@ -723,7 +723,7 @@ private: std::unique_ptr<TEvReadTableRequest> Request_; - TActorId ReadTableActor; + TActorId ReadTableActor; TList<TEvTxProcessing::TEvStreamQuotaRequest::TPtr> QuotaRequestQueue_; @@ -734,7 +734,7 @@ private: size_t LeftInGRpcAdaptorQueue_; - THashSet<TActorId> StreamSubscribers_; + THashSet<TActorId> StreamSubscribers_; TDuration InactiveClientTimeout_; TDuration InactiveServerTimeout_; diff --git a/ydb/core/grpc_services/rpc_scheme_base.h b/ydb/core/grpc_services/rpc_scheme_base.h index 64dca021ec0..0be68e802e1 100644 --- a/ydb/core/grpc_services/rpc_scheme_base.h +++ b/ydb/core/grpc_services/rpc_scheme_base.h @@ -140,7 +140,7 @@ protected: } private: - TActorId SchemePipeActorId_; + TActorId SchemePipeActorId_; }; } // namespace NGRpcService diff --git a/ydb/core/grpc_services/rpc_stream_execute_scan_query.cpp b/ydb/core/grpc_services/rpc_stream_execute_scan_query.cpp index 855a518f718..5f95f263c6f 100644 --- a/ydb/core/grpc_services/rpc_stream_execute_scan_query.cpp +++ b/ydb/core/grpc_services/rpc_stream_execute_scan_query.cpp @@ -315,7 +315,7 @@ private: ev->Record.SetTraceId(traceId.GetRef()); } - ActorIdToProto(this->SelfId(), ev->Record.MutableRequestActorId()); + ActorIdToProto(this->SelfId(), ev->Record.MutableRequestActorId()); TParseRequestError parseError; if (!FillKqpRequest(*req, ev->Record, parseError)) { @@ -498,7 +498,7 @@ private: } void Handle(NKqp::TEvKqpExecuter::TEvExecuterProgress::TPtr& ev, const TActorContext& ctx) { - ExecuterActorId_ = ActorIdFromProto(ev->Get()->Record.GetExecuterActorId()); + ExecuterActorId_ = ActorIdFromProto(ev->Get()->Record.GetExecuterActorId()); LOG_DEBUG_S(ctx, NKikimrServices::RPC_REQUEST, this->SelfId() << " ExecuterActorId: " << ExecuterActorId_); } @@ -618,7 +618,7 @@ private: TSchedulerCookieHolder TimeoutTimerCookieHolder_; TVector<std::unique_ptr<NYql::NDqProto::TDqExecutionStats>> ExecutionProfiles_; - TActorId ExecuterActorId_; + TActorId ExecuterActorId_; }; } // namespace diff --git a/ydb/core/grpc_streaming/grpc_streaming.h b/ydb/core/grpc_streaming/grpc_streaming.h index af2a56d6979..ecac6fcf171 100644 --- a/ydb/core/grpc_streaming/grpc_streaming.h +++ b/ydb/core/grpc_streaming/grpc_streaming.h @@ -1,10 +1,10 @@ #pragma once #include <ydb/core/base/events.h> -#include <library/cpp/grpc/server/event_callback.h> -#include <library/cpp/grpc/server/grpc_async_ctx_base.h> -#include <library/cpp/grpc/server/grpc_counters.h> -#include <library/cpp/grpc/server/grpc_request_base.h> +#include <library/cpp/grpc/server/event_callback.h> +#include <library/cpp/grpc/server/grpc_async_ctx_base.h> +#include <library/cpp/grpc/server/grpc_counters.h> +#include <library/cpp/grpc/server/grpc_request_base.h> #include <library/cpp/actors/core/actorsystem.h> #include <library/cpp/actors/core/log.h> @@ -69,7 +69,7 @@ public: * * This must be called before any Read or Write calls */ - virtual void Attach(TActorId actor) = 0; + virtual void Attach(TActorId actor) = 0; /** * Schedules the next message read @@ -112,9 +112,9 @@ public: virtual bool WriteAndFinish(TOut&& message, const grpc::WriteOptions& options, const grpc::Status& status) = 0; public: - virtual NGrpc::TAuthState& GetAuthState() const = 0; + virtual NGrpc::TAuthState& GetAuthState() const = 0; virtual TString GetPeerName() const = 0; - virtual TVector<TStringBuf> GetPeerMetaValues(TStringBuf key) const = 0; + virtual TVector<TStringBuf> GetPeerMetaValues(TStringBuf key) const = 0; virtual grpc_compression_level GetCompressionLevel() const = 0; virtual void UseDatabase(const TString& database) = 0; }; @@ -122,7 +122,7 @@ public: template<class TIn, class TOut, class TServer, int LoggerServiceId> class TGRpcStreamingRequest final : public TThrRefBase - , private NGrpc::TBaseAsyncContext<TServer> + , private NGrpc::TBaseAsyncContext<TServer> { using TSelf = TGRpcStreamingRequest<TIn, TOut, TServer, LoggerServiceId>; @@ -155,8 +155,8 @@ public: TAcceptCallback acceptCallback, NActors::TActorSystem& actorSystem, const char* name, - NGrpc::ICounterBlockPtr counters = nullptr, - NGrpc::IGRpcRequestLimiterPtr limiter = nullptr) + NGrpc::ICounterBlockPtr counters = nullptr, + NGrpc::IGRpcRequestLimiterPtr limiter = nullptr) { TIntrusivePtr<TSelf> self(new TSelf( server, @@ -180,9 +180,9 @@ private: TAcceptCallback acceptCallback, NActors::TActorSystem& as, const char* name, - NGrpc::ICounterBlockPtr counters, - NGrpc::IGRpcRequestLimiterPtr limiter) - : NGrpc::TBaseAsyncContext<TServer>(service, cq) + NGrpc::ICounterBlockPtr counters, + NGrpc::IGRpcRequestLimiterPtr limiter) + : NGrpc::TBaseAsyncContext<TServer>(service, cq) , Server(server) , AcceptRequest(acceptRequest) , AcceptCallback(acceptCallback) @@ -210,12 +210,12 @@ private: return false; } - void OnAccepted(NGrpc::EQueueEventStatus status) { + void OnAccepted(NGrpc::EQueueEventStatus status) { MaybeClone(); if (!this->Context.c_call()) { // Request dropped before it could start - Y_VERIFY(status == NGrpc::EQueueEventStatus::ERROR); + Y_VERIFY(status == NGrpc::EQueueEventStatus::ERROR); // Drop extra reference by OnDoneTag this->UnRef(); // We will be freed after return @@ -224,10 +224,10 @@ private: LOG_DEBUG(ActorSystem, LoggerServiceId, "[%p] stream accepted Name# %s ok# %s peer# %s", this, Name, - status == NGrpc::EQueueEventStatus::OK ? "true" : "false", + status == NGrpc::EQueueEventStatus::OK ? "true" : "false", this->Context.peer().c_str()); - if (status == NGrpc::EQueueEventStatus::ERROR) { + if (status == NGrpc::EQueueEventStatus::ERROR) { // Don't bother registering if accept failed if (Counters) { Counters->CountNotOkRequest(); @@ -262,13 +262,13 @@ private: } } - void OnDone(NGrpc::EQueueEventStatus status) { + void OnDone(NGrpc::EQueueEventStatus status) { LOG_DEBUG(ActorSystem, LoggerServiceId, "[%p] stream done notification Name# %s ok# %s peer# %s", this, Name, - status == NGrpc::EQueueEventStatus::OK ? "true" : "false", + status == NGrpc::EQueueEventStatus::OK ? "true" : "false", this->Context.peer().c_str()); - bool success = status == NGrpc::EQueueEventStatus::OK; + bool success = status == NGrpc::EQueueEventStatus::OK; auto flags = Flags.load(std::memory_order_acquire); auto added = FlagDoneCalled | (success ? FlagDoneSuccess : 0); @@ -295,7 +295,7 @@ private: FinishInternal(grpc::Status(grpc::StatusCode::CANCELLED, "Request abandoned")); } - void Attach(TActorId actor) { + void Attach(TActorId actor) { LOG_DEBUG(ActorSystem, LoggerServiceId, "[%p] facade attach Name# %s actor# %s peer# %s", this, Name, actor.ToString().c_str(), @@ -346,10 +346,10 @@ private: return true; } - void OnReadDone(NGrpc::EQueueEventStatus status) { + void OnReadDone(NGrpc::EQueueEventStatus status) { auto dumpResultText = [&] { TString text; - if (status == NGrpc::EQueueEventStatus::OK) { + if (status == NGrpc::EQueueEventStatus::OK) { google::protobuf::TextFormat::Printer printer; printer.SetSingleLineMode(true); printer.PrintToString(ReadInProgress->Record, &text); @@ -361,7 +361,7 @@ private: LOG_DEBUG(ActorSystem, LoggerServiceId, "[%p] read finished Name# %s ok# %s data# %s peer# %s", this, Name, - status == NGrpc::EQueueEventStatus::OK ? "true" : "false", + status == NGrpc::EQueueEventStatus::OK ? "true" : "false", dumpResultText().c_str(), this->Context.peer().c_str()); @@ -373,7 +373,7 @@ private: auto was = ReadQueue--; Y_VERIFY_DEBUG(was > 0); - read->Success = status == NGrpc::EQueueEventStatus::OK; + read->Success = status == NGrpc::EQueueEventStatus::OK; if (Counters && read->Success) { Counters->CountRequestBytes(read->Record.ByteSize()); } @@ -470,14 +470,14 @@ private: return true; } - void OnWriteDone(NGrpc::EQueueEventStatus status) { + void OnWriteDone(NGrpc::EQueueEventStatus status) { LOG_DEBUG(ActorSystem, LoggerServiceId, "[%p] write finished Name# %s ok# %s peer# %s", this, Name, - status == NGrpc::EQueueEventStatus::OK ? "true" : "false", + status == NGrpc::EQueueEventStatus::OK ? "true" : "false", this->Context.peer().c_str()); auto event = MakeHolder<typename IContext::TEvWriteFinished>(); - event->Success = status == NGrpc::EQueueEventStatus::OK; + event->Success = status == NGrpc::EQueueEventStatus::OK; ActorSystem.Send(Actor, event.Release()); THolder<TWriteItem> next; @@ -559,10 +559,10 @@ private: return true; } - void OnFinishDone(NGrpc::EQueueEventStatus status) { + void OnFinishDone(NGrpc::EQueueEventStatus status) { LOG_DEBUG(ActorSystem, LoggerServiceId, "[%p] stream finished Name# %s ok# %s peer# %s grpc status# (%d) message# %s", this, Name, - status == NGrpc::EQueueEventStatus::OK ? "true" : "false", + status == NGrpc::EQueueEventStatus::OK ? "true" : "false", this->Context.peer().c_str(), static_cast<int>(Status->error_code()), Status->error_message().c_str()); @@ -638,7 +638,7 @@ private: return Self->Cancel(); } - void Attach(TActorId actor) override { + void Attach(TActorId actor) override { Self->Attach(actor); } @@ -662,7 +662,7 @@ private: return Self->Finish(status); } - NGrpc::TAuthState& GetAuthState() const override { + NGrpc::TAuthState& GetAuthState() const override { return Self->AuthState; } @@ -670,7 +670,7 @@ private: return Self->GetPeerName(); } - TVector<TStringBuf> GetPeerMetaValues(TStringBuf key) const override { + TVector<TStringBuf> GetPeerMetaValues(TStringBuf key) const override { return Self->GetPeerMetaValues(key); } @@ -752,17 +752,17 @@ private: TAcceptCallback const AcceptCallback; NActors::TActorSystem& ActorSystem; const char* const Name; - NGrpc::ICounterBlockPtr const Counters; - NGrpc::IGRpcRequestLimiterPtr Limiter; + NGrpc::ICounterBlockPtr const Counters; + NGrpc::IGRpcRequestLimiterPtr Limiter; - NGrpc::TAuthState AuthState; + NGrpc::TAuthState AuthState; grpc::ServerAsyncReaderWriter<TOut, TIn> Stream; TSingleThreaded SingleThreaded; THPTimer RequestTimer; std::atomic<intptr_t> Flags{ 0 }; - TActorId Actor; + TActorId Actor; std::atomic<size_t> ReadQueue{ 0 }; THolder<typename IContext::TEvReadFinished> ReadInProgress; @@ -772,7 +772,7 @@ private: TMaybe<grpc::Status> Status; - using TFixedEvent = NGrpc::TQueueFixedEvent<TSelf>; + using TFixedEvent = NGrpc::TQueueFixedEvent<TSelf>; TFixedEvent OnDoneTag = { this, &TSelf::OnDone }; TFixedEvent OnAcceptedTag = { this, &TSelf::OnAccepted }; TFixedEvent OnReadDoneTag = { this, &TSelf::OnReadDone }; diff --git a/ydb/core/grpc_streaming/grpc_streaming_ut.cpp b/ydb/core/grpc_streaming/grpc_streaming_ut.cpp index 6823317ddde..fe0b6aec6a6 100644 --- a/ydb/core/grpc_streaming/grpc_streaming_ut.cpp +++ b/ydb/core/grpc_streaming/grpc_streaming_ut.cpp @@ -31,7 +31,7 @@ namespace { } template<class TImplActor> -class TStreamingService : public NGrpc::TGrpcServiceBase<NStreamingTest::TStreamingService> { +class TStreamingService : public NGrpc::TGrpcServiceBase<NStreamingTest::TStreamingService> { public: using TSelf = TStreamingService<TImplActor>; @@ -46,7 +46,7 @@ public: , Counters(std::move(counters)) { } - void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr) override { + void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr) override { CQ = cq; auto getCounterBlock = NGRpcService::CreateCounterCb(Counters, &ActorSystem); @@ -64,7 +64,7 @@ public: getCounterBlock("streaming", "Session")); } - void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override { + void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override { Limiter = limiter; } @@ -81,7 +81,7 @@ private: TIntrusivePtr<NMonitoring::TDynamicCounters> const Counters; grpc::ServerCompletionQueue* CQ = nullptr; - NGrpc::TGlobalLimiter* Limiter = nullptr; + NGrpc::TGlobalLimiter* Limiter = nullptr; }; template<class TImplActor> @@ -96,9 +96,9 @@ public: Server->GetRuntime()->SetLogPriority(NKikimrServices::GRPC_SERVER, NActors::NLog::PRI_DEBUG); - NGrpc::TServerOptions options; + NGrpc::TServerOptions options; options.SetPort(grpc); - GRpcServer.Reset(new NGrpc::TGRpcServer(options)); + GRpcServer.Reset(new NGrpc::TGRpcServer(options)); auto* as = Server->GetRuntime()->GetAnyNodeActorSystem(); TIntrusivePtr<NMonitoring::TDynamicCounters> counters(MakeIntrusive<NMonitoring::TDynamicCounters>()); @@ -118,7 +118,7 @@ public: public: TServerSettings::TPtr ServerSettings; TServer::TPtr Server; - THolder<NGrpc::TGRpcServer> GRpcServer; + THolder<NGrpc::TGRpcServer> GRpcServer; TString GRpcEndpoint; private: diff --git a/ydb/core/grpc_streaming/ya.make b/ydb/core/grpc_streaming/ya.make index 8042f7be0d8..19498f0cf0d 100644 --- a/ydb/core/grpc_streaming/ya.make +++ b/ydb/core/grpc_streaming/ya.make @@ -8,7 +8,7 @@ OWNER( PEERDIR( contrib/libs/grpc library/cpp/actors/core - library/cpp/grpc/server + library/cpp/grpc/server ydb/core/base ) diff --git a/ydb/core/kesus/proxy/events.h b/ydb/core/kesus/proxy/events.h index 93d9f33cb35..2062661bf63 100644 --- a/ydb/core/kesus/proxy/events.h +++ b/ydb/core/kesus/proxy/events.h @@ -34,10 +34,10 @@ struct TEvKesusProxy { }; struct TEvAttachProxyActor : public TEventLocal<TEvAttachProxyActor, EvAttachProxyActor> { - const TActorId ProxyActor; + const TActorId ProxyActor; const TIntrusivePtr<TSecurityObject> SecurityObject; - TEvAttachProxyActor(const TActorId& proxyActor, TIntrusivePtr<TSecurityObject> securityObject) + TEvAttachProxyActor(const TActorId& proxyActor, TIntrusivePtr<TSecurityObject> securityObject) : ProxyActor(proxyActor) , SecurityObject(std::move(securityObject)) {} diff --git a/ydb/core/kesus/proxy/proxy.cpp b/ydb/core/kesus/proxy/proxy.cpp index 905e93e3adc..9633816e60a 100644 --- a/ydb/core/kesus/proxy/proxy.cpp +++ b/ydb/core/kesus/proxy/proxy.cpp @@ -23,10 +23,10 @@ class TKesusProxyService : public TActor<TKesusProxyService> { }; struct TResolveReplyInfo { - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; - TResolveReplyInfo(const TActorId& sender, ui64 cookie) + TResolveReplyInfo(const TActorId& sender, ui64 cookie) : Sender(sender) , Cookie(cookie) {} @@ -37,7 +37,7 @@ class TKesusProxyService : public TActor<TKesusProxyService> { TVector<TString> KesusPath; ui64 TabletId = -1; TIntrusivePtr<TSecurityObject> SecurityObject; - TActorId ProxyActor; + TActorId ProxyActor; NKikimrKesus::TKesusError LastError; TVector<TResolveReplyInfo> ResolveSubscribers; }; @@ -220,11 +220,11 @@ private: class TKesusProxyService::TResolveActor : public TActorBootstrapped<TResolveActor> { private: - const TActorId Owner; + const TActorId Owner; const TString KesusPath; public: - TResolveActor(const TActorId& owner, const TString& kesusPath) + TResolveActor(const TActorId& owner, const TString& kesusPath) : Owner(owner) , KesusPath(kesusPath) {} @@ -268,8 +268,8 @@ IActor* TKesusProxyService::CreateResolveActor(const TString& kesusPath) { return new TResolveActor(SelfId(), kesusPath); } -TActorId MakeKesusProxyServiceId() { - return TActorId(0, "kesus-proxy"); +TActorId MakeKesusProxyServiceId() { + return TActorId(0, "kesus-proxy"); } IActor* CreateKesusProxyService() { diff --git a/ydb/core/kesus/proxy/proxy.h b/ydb/core/kesus/proxy/proxy.h index 705840dc299..6fff6289507 100644 --- a/ydb/core/kesus/proxy/proxy.h +++ b/ydb/core/kesus/proxy/proxy.h @@ -5,7 +5,7 @@ namespace NKikimr { namespace NKesus { -TActorId MakeKesusProxyServiceId(); +TActorId MakeKesusProxyServiceId(); IActor* CreateKesusProxyService(); diff --git a/ydb/core/kesus/proxy/proxy_actor.cpp b/ydb/core/kesus/proxy/proxy_actor.cpp index 3ff2aa99021..e93d90355fc 100644 --- a/ydb/core/kesus/proxy/proxy_actor.cpp +++ b/ydb/core/kesus/proxy/proxy_actor.cpp @@ -32,7 +32,7 @@ class TKesusProxyActor : public TActorBootstrapped<TKesusProxyActor> { }; struct TDirectRequest { - TActorId Sender; + TActorId Sender; ui64 Cookie; THolder<IEventBase> Event; }; @@ -47,7 +47,7 @@ class TKesusProxyActor : public TActorBootstrapped<TKesusProxyActor> { ui64 SeqNo; ui64 SessionId; ui64 ClientSeqNo; - TActorId Owner; + TActorId Owner; ui64 OwnerCookie; ESessionState State = ESessionState::ATTACHING; THolder<TEvKesus::TEvAttachSession> AttachEvent; @@ -58,30 +58,30 @@ class TKesusProxyActor : public TActorBootstrapped<TKesusProxyActor> { }; private: - const TActorId MetaProxy; + const TActorId MetaProxy; const ui64 TabletId; const TString KesusPath; EState State = STATE_IDLE; - TActorId TabletPipe; + TActorId TabletPipe; ui64 ProxyGeneration = 0; ui64 SeqNo = 0; // SeqNo -> direct request THashMap<ui64, TDirectRequest> DirectRequests; - THashMap<TActorId, ui64> DirectRequestBySender; + THashMap<TActorId, ui64> DirectRequestBySender; // SeqNo -> session data THashMap<ui64, TSessionData> Sessions; // SessionId -> SeqNo THashMap<ui64, TSessionData*> SessionById; // Owner -> SeqNo - THashMap<TActorId, TSessionData*> SessionByOwner; + THashMap<TActorId, TSessionData*> SessionByOwner; // Request SeqNo -> Session SeqNo THashMap<ui64, TSessionData*> SessionByRequest; public: - TKesusProxyActor(const TActorId& meta, ui64 tabletId, const TString& kesusPath) + TKesusProxyActor(const TActorId& meta, ui64 tabletId, const TString& kesusPath) : MetaProxy(meta) , TabletId(tabletId) , KesusPath(kesusPath) @@ -106,7 +106,7 @@ private: return SessionById.Value(sessionId, nullptr); } - TSessionData* FindSessionByOwner(const TActorId& owner) { + TSessionData* FindSessionByOwner(const TActorId& owner) { return SessionByOwner.Value(owner, nullptr); } @@ -221,7 +221,7 @@ private: CancelSessionRequest(ev->Sender); } - void CancelDirectRequest(const TActorId& owner) { + void CancelDirectRequest(const TActorId& owner) { auto it = DirectRequestBySender.find(owner); if (it != DirectRequestBySender.end()) { DirectRequests.erase(it->second); @@ -229,7 +229,7 @@ private: } } - void CancelSessionRequest(const TActorId& owner) { + void CancelSessionRequest(const TActorId& owner) { if (auto* data = FindSessionByOwner(owner)) { ClearSessionOwner(data); @@ -423,7 +423,7 @@ private: } } - void HandleDirectRequest(const TActorId& sender, ui64 cookie, THolder<IEventBase> event) { + void HandleDirectRequest(const TActorId& sender, ui64 cookie, THolder<IEventBase> event) { KPROXY_LOG_TRACE_S("Received " << event->ToStringHeader() << " from " << sender); Y_VERIFY(!DirectRequestBySender.contains(sender), "Only one outgoing request per sender is allowed"); const ui64 seqNo = ++SeqNo; @@ -748,7 +748,7 @@ private: } template<class TRequest> - void HandleSessionRequest(const TActorId& sender, ui64 cookie, TAutoPtr<TRequest> event) { + void HandleSessionRequest(const TActorId& sender, ui64 cookie, TAutoPtr<TRequest> event) { KPROXY_LOG_TRACE_S("Received " << event->ToStringHeader() << " from " << sender); Y_VERIFY(sender); auto& record = event->Record; @@ -842,7 +842,7 @@ private: } }; -IActor* CreateKesusProxyActor(const TActorId& meta, ui64 tabletId, const TString& kesusPath) { +IActor* CreateKesusProxyActor(const TActorId& meta, ui64 tabletId, const TString& kesusPath) { return new TKesusProxyActor(meta, tabletId, kesusPath); } diff --git a/ydb/core/kesus/proxy/proxy_actor.h b/ydb/core/kesus/proxy/proxy_actor.h index 316f4af9dfa..7c7d56468b8 100644 --- a/ydb/core/kesus/proxy/proxy_actor.h +++ b/ydb/core/kesus/proxy/proxy_actor.h @@ -5,7 +5,7 @@ namespace NKikimr { namespace NKesus { -IActor* CreateKesusProxyActor(const TActorId& meta, ui64 tabletId, const TString& kesusPath); +IActor* CreateKesusProxyActor(const TActorId& meta, ui64 tabletId, const TString& kesusPath); } } diff --git a/ydb/core/kesus/proxy/ut_helpers.cpp b/ydb/core/kesus/proxy/ut_helpers.cpp index 8a6e5010b27..74580d10f71 100644 --- a/ydb/core/kesus/proxy/ut_helpers.cpp +++ b/ydb/core/kesus/proxy/ut_helpers.cpp @@ -41,14 +41,14 @@ void TTestContext::SetupTabletServices() { } void TTestContext::Sleep(ui64 millis) { - TActorId sender = Runtime->AllocateEdgeActor(); + TActorId sender = Runtime->AllocateEdgeActor(); Runtime->Schedule(new IEventHandle(sender, sender, new TEvents::TEvWakeup()), TDuration::MilliSeconds(millis), 0); ExpectEdgeEvent<TEvents::TEvWakeup>(sender, 0); } void TTestContext::RebootTablet() { ui32 nodeIndex = 0; - TActorId sender = Runtime->AllocateEdgeActor(nodeIndex); + TActorId sender = Runtime->AllocateEdgeActor(nodeIndex); ForwardToTablet(*Runtime, TabletId, sender, new TEvents::TEvPoisonPill(), nodeIndex); { TDispatchOptions options; @@ -58,17 +58,17 @@ void TTestContext::RebootTablet() { InvalidateTabletResolverCache(*Runtime, TabletId, nodeIndex); } -TActorId TTestContext::GetProxy(ui64 proxyId, ui32 nodeIndex) { +TActorId TTestContext::GetProxy(ui64 proxyId, ui32 nodeIndex) { auto* pActor = ProxyActors.FindPtr(proxyId); if (!pActor) { pActor = &ProxyActors[proxyId]; - *pActor = Runtime->Register(CreateKesusProxyActor(TActorId(), TabletId, ""), nodeIndex); + *pActor = Runtime->Register(CreateKesusProxyActor(TActorId(), TabletId, ""), nodeIndex); Runtime->EnableScheduleForActor(*pActor); } return *pActor; } -void TTestContext::SendFromEdge(ui64 proxyId, const TActorId& edge, IEventBase* payload, ui64 cookie) { +void TTestContext::SendFromEdge(ui64 proxyId, const TActorId& edge, IEventBase* payload, ui64 cookie) { ui32 nodeIndex = edge.NodeId() - Runtime->GetNodeId(0); Runtime->Send( new IEventHandle(GetProxy(proxyId), edge, payload, 0, cookie), @@ -76,7 +76,7 @@ void TTestContext::SendFromEdge(ui64 proxyId, const TActorId& edge, IEventBase* true); } -void TTestContext::ExpectProxyError(const TActorId& edge, ui64 cookie, Ydb::StatusIds::StatusCode status) { +void TTestContext::ExpectProxyError(const TActorId& edge, ui64 cookie, Ydb::StatusIds::StatusCode status) { auto result = ExpectEdgeEvent<TEvKesusProxy::TEvProxyError>(edge, cookie); UNIT_ASSERT_VALUES_EQUAL(result->Error.GetStatus(), status); } @@ -100,14 +100,14 @@ void TTestContext::DeleteSemaphore(ui64 proxyId, const TString& name, Ydb::Statu } ui64 TTestContext::SendAttachSession( - ui64 proxyId, const TActorId& edge, ui64 sessionId, ui64 timeoutMillis, const TString& description, ui64 seqNo) + ui64 proxyId, const TActorId& edge, ui64 sessionId, ui64 timeoutMillis, const TString& description, ui64 seqNo) { ui64 cookie = RandomNumber<ui64>(); SendFromEdge(proxyId, edge, new TEvKesus::TEvAttachSession("", 0, sessionId, timeoutMillis, description, seqNo), cookie); return cookie; } -ui64 TTestContext::ExpectAttachSessionResult(const TActorId& edge, ui64 cookie, Ydb::StatusIds::StatusCode status) { +ui64 TTestContext::ExpectAttachSessionResult(const TActorId& edge, ui64 cookie, Ydb::StatusIds::StatusCode status) { auto result = ExpectEdgeEvent<TEvKesus::TEvAttachSessionResult>(edge, cookie); UNIT_ASSERT_VALUES_EQUAL(result->Record.GetError().GetStatus(), status); return result->Record.GetSessionId(); @@ -115,7 +115,7 @@ ui64 TTestContext::ExpectAttachSessionResult(const TActorId& edge, ui64 cookie, TTestContext::TSimpleSemaphoreDescription TTestContext::DescribeSemaphore(ui64 proxyId, const TString& name, bool includeWaiters) { ui64 cookie = RandomNumber<ui64>(); - TActorId edge = Runtime->AllocateEdgeActor(1); + TActorId edge = Runtime->AllocateEdgeActor(1); auto request = new TEvKesus::TEvDescribeSemaphore("", name); request->Record.SetIncludeOwners(true); request->Record.SetIncludeWaiters(includeWaiters); @@ -139,7 +139,7 @@ TTestContext::TSimpleSemaphoreDescription TTestContext::DescribeSemaphore(ui64 p void TTestContext::VerifySemaphoreNotFound(ui64 proxyId, const TString& name) { ui64 cookie = RandomNumber<ui64>(); - TActorId edge = Runtime->AllocateEdgeActor(1); + TActorId edge = Runtime->AllocateEdgeActor(1); SendFromEdge(proxyId, edge, new TEvKesus::TEvDescribeSemaphore("", name), cookie); auto event = ExpectEdgeEvent<TEvKesus::TEvDescribeSemaphoreResult>(edge, cookie); UNIT_ASSERT_VALUES_EQUAL(event->Record.GetError().GetStatus(), Ydb::StatusIds::NOT_FOUND); diff --git a/ydb/core/kesus/proxy/ut_helpers.h b/ydb/core/kesus/proxy/ut_helpers.h index 1ce440056d5..20bcbac73f6 100644 --- a/ydb/core/kesus/proxy/ut_helpers.h +++ b/ydb/core/kesus/proxy/ut_helpers.h @@ -20,7 +20,7 @@ struct TTestContext { TTabletTypes::EType TabletType; ui64 TabletId; THolder<TTestActorRuntime> Runtime; - THashMap<ui64, TActorId> ProxyActors; + THashMap<ui64, TActorId> ProxyActors; TTestContext(); @@ -38,26 +38,26 @@ struct TTestContext { // Extremely pedantic version of GrabEdgeEvent template<class TEvent> - THolder<TEvent> ExpectEdgeEvent(const TActorId& actor) { + THolder<TEvent> ExpectEdgeEvent(const TActorId& actor) { return Runtime->GrabEdgeEvent<TEvent>(actor)->Release(); } // Extremely pedantic version of GrabEdgeEvent template<class TEvent> - THolder<TEvent> ExpectEdgeEvent(const TActorId& actor, ui64 cookie) { + THolder<TEvent> ExpectEdgeEvent(const TActorId& actor, ui64 cookie) { return Runtime->GrabEdgeEventIf<TEvent>(actor, [=](const auto& ev) { return ev->Cookie == cookie; })->Release(); } // Dynamically allocates a proxy - TActorId GetProxy(ui64 proxyId, ui32 nodeIndex = 1); + TActorId GetProxy(ui64 proxyId, ui32 nodeIndex = 1); // Sends payload to proxy from edge - void SendFromEdge(ui64 proxyId, const TActorId& edge, IEventBase* payload, ui64 cookie = 0); + void SendFromEdge(ui64 proxyId, const TActorId& edge, IEventBase* payload, ui64 cookie = 0); // Expects TEvProxyError at the edge with the specified cookie and status - void ExpectProxyError(const TActorId& edge, ui64 cookie, Ydb::StatusIds::StatusCode status); + void ExpectProxyError(const TActorId& edge, ui64 cookie, Ydb::StatusIds::StatusCode status); void CreateSemaphore(ui64 proxyId, const TString& name, ui64 limit, const TString& data, Ydb::StatusIds::StatusCode status = Ydb::StatusIds::SUCCESS); @@ -70,11 +70,11 @@ struct TTestContext { // Returns a cookie used for the session ui64 SendAttachSession( - ui64 proxyId, const TActorId& edge, ui64 sessionId, ui64 timeoutMillis = 0, + ui64 proxyId, const TActorId& edge, ui64 sessionId, ui64 timeoutMillis = 0, const TString& description = TString(), ui64 seqNo = 0); // Returns sessionId from the result - ui64 ExpectAttachSessionResult(const TActorId& edge, ui64 cookie, Ydb::StatusIds::StatusCode status = Ydb::StatusIds::SUCCESS); + ui64 ExpectAttachSessionResult(const TActorId& edge, ui64 cookie, Ydb::StatusIds::StatusCode status = Ydb::StatusIds::SUCCESS); // Describes and verifies info about semaphores diff --git a/ydb/core/kesus/tablet/probes.h b/ydb/core/kesus/tablet/probes.h index bd1ad72640e..077375fb368 100644 --- a/ydb/core/kesus/tablet/probes.h +++ b/ydb/core/kesus/tablet/probes.h @@ -9,7 +9,7 @@ struct TActorIdParam { using TStoreType = TString; - using TFuncParam = typename TTypeTraits<NActors::TActorId>::TFuncParam; + using TFuncParam = typename TTypeTraits<NActors::TActorId>::TFuncParam; inline static void ToString(typename TTypeTraits<TStoreType>::TFuncParam stored, TString* out) { *out = stored; diff --git a/ydb/core/kesus/tablet/quoter_performance_test/session.h b/ydb/core/kesus/tablet/quoter_performance_test/session.h index 242e2d2d6ac..99457ca6346 100644 --- a/ydb/core/kesus/tablet/quoter_performance_test/session.h +++ b/ydb/core/kesus/tablet/quoter_performance_test/session.h @@ -36,7 +36,7 @@ public: void Subscribe() { auto req = MakeHolder<TEvKesus::TEvSubscribeOnResources>(); - ActorIdToProto(SelfId(), req->Record.MutableActorID()); + ActorIdToProto(SelfId(), req->Record.MutableActorID()); req->Record.MutableResources()->Reserve(State->Options.ResourcesCount); //const double amount = State->Options.MaxUnitsPerSecond * State->Options.TestTime.Seconds() * 2.0; @@ -92,6 +92,6 @@ public: private: TIntrusivePtr<TTestState> State; TInstant StartSessionTime; - TActorId TabletPipe; + TActorId TabletPipe; std::vector<TTestState::TResourceState> ResState; }; diff --git a/ydb/core/kesus/tablet/quoter_performance_test/test_state.h b/ydb/core/kesus/tablet/quoter_performance_test/test_state.h index 50043af69d4..e4d4766ef44 100644 --- a/ydb/core/kesus/tablet/quoter_performance_test/test_state.h +++ b/ydb/core/kesus/tablet/quoter_performance_test/test_state.h @@ -48,7 +48,7 @@ struct TTestState : public TAtomicRefCount<TTestState> { const TOptions& Options; ui64 TabletId = 0; std::vector<TResourceState> ResourcesState; - TActorId EdgeActorId; + TActorId EdgeActorId; THashMap<ui64, size_t> ResId2StateIndex; TMutex Mutex; }; diff --git a/ydb/core/kesus/tablet/quoter_resource_tree.cpp b/ydb/core/kesus/tablet/quoter_resource_tree.cpp index ad2112358bb..3f9d6e56530 100644 --- a/ydb/core/kesus/tablet/quoter_resource_tree.cpp +++ b/ydb/core/kesus/tablet/quoter_resource_tree.cpp @@ -5,7 +5,7 @@ #include <ydb/core/base/path.h> #include <util/string/builder.h> -#include <util/generic/maybe.h> +#include <util/generic/maybe.h> #include <algorithm> #include <cmath> @@ -162,7 +162,7 @@ public: void CalcParameters() override; void CalcParametersForAccounting(); - THolder<TQuoterSession> DoCreateSession(const NActors::TActorId& clientId) override; + THolder<TQuoterSession> DoCreateSession(const NActors::TActorId& clientId) override; void AddActiveChild(THierarchicalDRRResourceConsumer* child, TTickProcessorQueue& queue, TInstant now); void RemoveActiveChild(THierarchicalDRRResourceConsumer* child); @@ -246,7 +246,7 @@ THolder<TQuoterResourceTree> CreateResource(ui64 resourceId, ui64 parentId, NAct // Session in case of hierarchical DRR algorithm. class THierarhicalDRRQuoterSession : public TQuoterSession, public THierarchicalDRRResourceConsumer { public: - THierarhicalDRRQuoterSession(const NActors::TActorId& clientId, THierarhicalDRRQuoterResourceTree* resource) + THierarhicalDRRQuoterSession(const NActors::TActorId& clientId, THierarhicalDRRQuoterResourceTree* resource) : TQuoterSession(clientId, resource) { } @@ -431,7 +431,7 @@ void THierarhicalDRRQuoterSession::OnPropsChanged() { } // anonymous namespace -TQuoterSession::TQuoterSession(const NActors::TActorId& clientId, TQuoterResourceTree* resource) +TQuoterSession::TQuoterSession(const NActors::TActorId& clientId, TQuoterResourceTree* resource) : Resource(resource) , ClientId(clientId) { @@ -925,7 +925,7 @@ void THierarhicalDRRQuoterResourceTree::ScheduleNextTick(TTickProcessorQueue& qu Schedule(queue, NextTick(now, TickSize)); } -THolder<TQuoterSession> THierarhicalDRRQuoterResourceTree::DoCreateSession(const NActors::TActorId& clientId) { +THolder<TQuoterSession> THierarhicalDRRQuoterResourceTree::DoCreateSession(const NActors::TActorId& clientId) { return MakeHolder<THierarhicalDRRQuoterSession>(clientId, this); } @@ -1032,13 +1032,13 @@ bool TQuoterResources::DeleteResource(TQuoterResourceTree* resource, TString& er const auto sessions = resource->GetSessions(); TStringBuilder closeReason; closeReason << "Resource \"" << resource->GetPath() << "\" was deleted."; - for (const NActors::TActorId& clientId : sessions) { + for (const NActors::TActorId& clientId : sessions) { const auto sessionId = TQuoterSessionId{clientId, resource->GetResourceId()}; const auto sessionIt = Sessions.find(sessionId); Y_VERIFY(sessionIt != Sessions.end()); TQuoterSession* session = sessionIt->second.Get(); session->CloseSession(Ydb::StatusIds::NOT_FOUND, closeReason); - const NActors::TActorId pipeServerId = session->SetPipeServerId({}); + const NActors::TActorId pipeServerId = session->SetPipeServerId({}); SetPipeServerId(sessionId, pipeServerId, {}); // Erase pipeServerId from index. Sessions.erase(sessionIt); } @@ -1117,7 +1117,7 @@ void TQuoterResources::ProcessTick(const TTickProcessorTask& task, TTickProcesso } } -TQuoterSession* TQuoterResources::GetOrCreateSession(const NActors::TActorId& clientId, TQuoterResourceTree* resource) { +TQuoterSession* TQuoterResources::GetOrCreateSession(const NActors::TActorId& clientId, TQuoterResourceTree* resource) { const ui64 resourceId = resource->GetResourceId(); if (TQuoterSession* session = FindSession(clientId, resourceId)) { return session; @@ -1127,19 +1127,19 @@ TQuoterSession* TQuoterResources::GetOrCreateSession(const NActors::TActorId& cl } } -TQuoterSession* TQuoterResources::FindSession(const NActors::TActorId& clientId, ui64 resourceId) { +TQuoterSession* TQuoterResources::FindSession(const NActors::TActorId& clientId, ui64 resourceId) { const auto sessionIt = Sessions.find(TQuoterSessionId{clientId, resourceId}); return sessionIt != Sessions.end() ? sessionIt->second.Get() : nullptr; } -const TQuoterSession* TQuoterResources::FindSession(const NActors::TActorId& clientId, ui64 resourceId) const { +const TQuoterSession* TQuoterResources::FindSession(const NActors::TActorId& clientId, ui64 resourceId) const { const auto sessionIt = Sessions.find(TQuoterSessionId{clientId, resourceId}); return sessionIt != Sessions.end() ? sessionIt->second.Get() : nullptr; } void TQuoterResources::OnUpdateResourceProps(TQuoterResourceTree* rootResource) { const ui64 resId = rootResource->GetResourceId(); - for (const NActors::TActorId& sessionActor : rootResource->GetSessions()) { + for (const NActors::TActorId& sessionActor : rootResource->GetSessions()) { TQuoterSession* session = FindSession(sessionActor, resId); Y_VERIFY(session); session->OnPropsChanged(); @@ -1199,7 +1199,7 @@ void TQuoterResources::FillCounters(NKikimrKesus::TEvGetQuoterResourceCountersRe } } -void TQuoterResources::SetPipeServerId(TQuoterSessionId sessionId, const NActors::TActorId& prevId, const NActors::TActorId& id) { +void TQuoterResources::SetPipeServerId(TQuoterSessionId sessionId, const NActors::TActorId& prevId, const NActors::TActorId& id) { if (prevId) { auto [prevIt, prevItEnd] = PipeServerIdToSession.equal_range(prevId); for (; prevIt != prevItEnd; ++prevIt) { @@ -1214,11 +1214,11 @@ void TQuoterResources::SetPipeServerId(TQuoterSessionId sessionId, const NActors } } -void TQuoterResources::DisconnectSession(const NActors::TActorId& pipeServerId) { +void TQuoterResources::DisconnectSession(const NActors::TActorId& pipeServerId) { auto [pipeToSessionItBegin, pipeToSessionItEnd] = PipeServerIdToSession.equal_range(pipeServerId); for (auto pipeToSessionIt = pipeToSessionItBegin; pipeToSessionIt != pipeToSessionItEnd; ++pipeToSessionIt) { const TQuoterSessionId sessionId = pipeToSessionIt->second; - const NActors::TActorId sessionClientId = sessionId.first; + const NActors::TActorId sessionClientId = sessionId.first; { const auto sessionIter = Sessions.find(sessionId); diff --git a/ydb/core/kesus/tablet/quoter_resource_tree.h b/ydb/core/kesus/tablet/quoter_resource_tree.h index ad0a602ada9..bc28e518c3f 100644 --- a/ydb/core/kesus/tablet/quoter_resource_tree.h +++ b/ydb/core/kesus/tablet/quoter_resource_tree.h @@ -5,7 +5,7 @@ #include <ydb/core/protos/kesus.pb.h> #include <library/cpp/actors/core/actor.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <util/datetime/base.h> #include <util/generic/hash.h> @@ -20,8 +20,8 @@ namespace NKikimr { namespace NKesus { -using TQuoterSessionId = std::pair<NActors::TActorId, ui64>; // client id, resource id -using TTickProcessorId = std::pair<NActors::TActorId, ui64>; // == TQuoterSessionId for sessions. == ResourceId for resources (with empty actor id). +using TQuoterSessionId = std::pair<NActors::TActorId, ui64>; // client id, resource id +using TTickProcessorId = std::pair<NActors::TActorId, ui64>; // == TQuoterSessionId for sessions. == ResourceId for resources (with empty actor id). class TQuoterResourceTree; TString CanonizeQuoterResourcePath(const TVector<TString>& path); @@ -116,7 +116,7 @@ public: // Common interface for session to resource. class TQuoterSession : public TTickProcessor { public: - TQuoterSession(const NActors::TActorId& clientId, TQuoterResourceTree* resource); + TQuoterSession(const NActors::TActorId& clientId, TQuoterResourceTree* resource); virtual ~TQuoterSession() = default; @@ -129,7 +129,7 @@ public: } // Client actor id (remote client: quoter proxy). - const NActors::TActorId& GetClientId() const { + const NActors::TActorId& GetClientId() const { return ClientId; } @@ -167,13 +167,13 @@ public: Send(0); // Update props immediately // KIKIMR-8563 } - NActors::TActorId SetPipeServerId(const NActors::TActorId& pipeServerId) { - const NActors::TActorId prevId = PipeServerId; + NActors::TActorId SetPipeServerId(const NActors::TActorId& pipeServerId) { + const NActors::TActorId prevId = PipeServerId; PipeServerId = pipeServerId; return prevId; } - NActors::TActorId GetPipeServerId() const { + NActors::TActorId GetPipeServerId() const { return PipeServerId; } @@ -182,8 +182,8 @@ protected: protected: TQuoterResourceTree* Resource = nullptr; - NActors::TActorId ClientId; - NActors::TActorId PipeServerId; + NActors::TActorId ClientId; + NActors::TActorId PipeServerId; double AmountRequested = 0.0; double TotalConsumed = 0.0; // Only for session statistics. Accuracy of this variable will degrade in time. bool Active = false; @@ -245,9 +245,9 @@ public: // Runtime algorithm entry points. virtual void CalcParameters(); // Recursively calculates all parameters for runtime algorithm. - virtual THolder<TQuoterSession> DoCreateSession(const NActors::TActorId& clientId) = 0; + virtual THolder<TQuoterSession> DoCreateSession(const NActors::TActorId& clientId) = 0; - THolder<TQuoterSession> CreateSession(const NActors::TActorId& clientId) { + THolder<TQuoterSession> CreateSession(const NActors::TActorId& clientId) { THolder<TQuoterSession> session = DoCreateSession(clientId); if (session) { Sessions.insert(clientId); @@ -258,11 +258,11 @@ public: return session; } - const THashSet<NActors::TActorId>& GetSessions() const { + const THashSet<NActors::TActorId>& GetSessions() const { return Sessions; } - void OnSessionDisconnected(const NActors::TActorId& clientId) { + void OnSessionDisconnected(const NActors::TActorId& clientId) { Sessions.erase(clientId); } @@ -272,7 +272,7 @@ public: } TTickProcessorId GetTickProcessorId() const override { - return {NActors::TActorId(), ResourceId}; + return {NActors::TActorId(), ResourceId}; } class TCounters { @@ -319,7 +319,7 @@ protected: size_t ResourceLevel = 0; TQuoterResourceTree* Parent = nullptr; THashSet<TQuoterResourceTree*> Children; - THashSet<NActors::TActorId> Sessions; + THashSet<NActors::TActorId> Sessions; NKikimrKesus::TStreamingQuoterResource Props; NKikimrKesus::TStreamingQuoterResource EffectiveProps; // Props with actual values taken from Props or from parent's Props or from defaults. TCounters Counters; @@ -370,11 +370,11 @@ public: void ProcessTick(const TTickProcessorTask& task, TTickProcessorQueue& queue); - TQuoterSession* GetOrCreateSession(const NActors::TActorId& clientId, TQuoterResourceTree* resource); - TQuoterSession* FindSession(const NActors::TActorId& clientId, ui64 resourceId); - const TQuoterSession* FindSession(const NActors::TActorId& clientId, ui64 resourceId) const; - void DisconnectSession(const NActors::TActorId& pipeServerId); - void SetPipeServerId(TQuoterSessionId sessionId, const NActors::TActorId& prevId, const NActors::TActorId& id); + TQuoterSession* GetOrCreateSession(const NActors::TActorId& clientId, TQuoterResourceTree* resource); + TQuoterSession* FindSession(const NActors::TActorId& clientId, ui64 resourceId); + const TQuoterSession* FindSession(const NActors::TActorId& clientId, ui64 resourceId) const; + void DisconnectSession(const NActors::TActorId& pipeServerId); + void SetPipeServerId(TQuoterSessionId sessionId, const NActors::TActorId& prevId, const NActors::TActorId& id); void OnUpdateResourceProps(TQuoterResourceTree* rootResource); @@ -399,7 +399,7 @@ private: THashMap<ui64, THolder<TQuoterResourceTree>> ResourcesById; THashMap<TString, TQuoterResourceTree*> ResourcesByPath; THashMap<TQuoterSessionId, THolder<TQuoterSession>> Sessions; - THashMultiMap<NActors::TActorId, TQuoterSessionId> PipeServerIdToSession; + THashMultiMap<NActors::TActorId, TQuoterSessionId> PipeServerIdToSession; TCounters Counters; }; diff --git a/ydb/core/kesus/tablet/quoter_resource_tree_ut.cpp b/ydb/core/kesus/tablet/quoter_resource_tree_ut.cpp index 028c9bf7ba8..b280b4b29b8 100644 --- a/ydb/core/kesus/tablet/quoter_resource_tree_ut.cpp +++ b/ydb/core/kesus/tablet/quoter_resource_tree_ut.cpp @@ -165,7 +165,7 @@ public: return res; } - std::vector<TTestSession> CreateSession(std::vector<TQuoterResourceTree*> resources, bool consume, double amount, TIntrusivePtr<TTestResourceSink> sink = nullptr, NActors::TActorId clientId = {}, NActors::TActorId pipeServerId = {}) { + std::vector<TTestSession> CreateSession(std::vector<TQuoterResourceTree*> resources, bool consume, double amount, TIntrusivePtr<TTestResourceSink> sink = nullptr, NActors::TActorId clientId = {}, NActors::TActorId pipeServerId = {}) { UNIT_ASSERT(!resources.empty()); if (!sink) { @@ -185,7 +185,7 @@ public: TQuoterSession* session = Resources->GetOrCreateSession(clientId, resource); UNIT_ASSERT(session); session->SetResourceSink(sink); - const NActors::TActorId prevPipeServerId = session->SetPipeServerId(pipeServerId); + const NActors::TActorId prevPipeServerId = session->SetPipeServerId(pipeServerId); UNIT_ASSERT(!newClientId || !prevPipeServerId); Resources->SetPipeServerId(TQuoterSessionId(clientId, resource->GetResourceId()), prevPipeServerId, pipeServerId); session->UpdateConsumptionState(consume, amount, queue, Time); @@ -196,7 +196,7 @@ public: return result; } - TTestSession CreateSession(TQuoterResourceTree* resource, bool consume, double amount, TIntrusivePtr<TTestResourceSink> sink = nullptr, NActors::TActorId clientId = {}, NActors::TActorId pipeServerId = {}) { + TTestSession CreateSession(TQuoterResourceTree* resource, bool consume, double amount, TIntrusivePtr<TTestResourceSink> sink = nullptr, NActors::TActorId clientId = {}, NActors::TActorId pipeServerId = {}) { return CreateSession(std::vector<TQuoterResourceTree*>(1, resource), consume, amount, sink, clientId, pipeServerId)[0]; } @@ -204,10 +204,10 @@ public: Resources->DisconnectSession(session->GetPipeServerId()); } - NActors::TActorId NewActorID() { + NActors::TActorId NewActorID() { const ui64 x1 = NextActorId++; const ui64 x2 = NextActorId++; - return NActors::TActorId(x1, x2); + return NActors::TActorId(x1, x2); } // Tests @@ -575,8 +575,8 @@ public: AddResource("Root", 100500); - const NActors::TActorId clientId = NewActorID(); - const NActors::TActorId pipeServerId = NewActorID(); + const NActors::TActorId clientId = NewActorID(); + const NActors::TActorId pipeServerId = NewActorID(); std::vector<TQuoterResourceTree*> resources; resources.reserve(resourcesCount); for (size_t resourceIndex = 0; resourceIndex < resourcesCount; ++resourceIndex) { diff --git a/ydb/core/kesus/tablet/quoter_runtime.cpp b/ydb/core/kesus/tablet/quoter_runtime.cpp index bc5b57dfc50..d00c10739db 100644 --- a/ydb/core/kesus/tablet/quoter_runtime.cpp +++ b/ydb/core/kesus/tablet/quoter_runtime.cpp @@ -14,7 +14,7 @@ namespace NKesus { class TKesusTablet::TQuoterResourceSink : public IResourceSink { public: - TQuoterResourceSink(const TActorId& actor, TKesusTablet* kesus) + TQuoterResourceSink(const TActorId& actor, TKesusTablet* kesus) : Actor(actor) , Kesus(kesus) { @@ -34,11 +34,11 @@ public: } private: - TActorId Actor; + TActorId Actor; TKesusTablet* Kesus; }; -void TKesusTablet::TQuoterResourceSessionsAccumulator::Accumulate(const TActorId& recipient, ui64 resourceId, double amount, const NKikimrKesus::TStreamingQuoterResource* props) { +void TKesusTablet::TQuoterResourceSessionsAccumulator::Accumulate(const TActorId& recipient, ui64 resourceId, double amount, const NKikimrKesus::TStreamingQuoterResource* props) { TSendInfo& info = SendInfos[recipient]; if (!info.Event) { info.Event = MakeHolder<TEvKesus::TEvResourcesAllocated>(); @@ -63,7 +63,7 @@ void TKesusTablet::TQuoterResourceSessionsAccumulator::Accumulate(const TActorId void TKesusTablet::TQuoterResourceSessionsAccumulator::SendAll(const TActorContext& ctx, ui64 tabletId) { for (auto infoIter = SendInfos.begin(), infoEnd = SendInfos.end(); infoIter != infoEnd; ++infoIter) { - const TActorId& recipientId = infoIter->first; + const TActorId& recipientId = infoIter->first; auto& info = infoIter->second; TRACE_LOG_EVENT(tabletId, "TEvResourcesAllocated", info.Event->Record, recipientId, 0); ctx.Send(recipientId, std::move(info.Event)); @@ -73,8 +73,8 @@ void TKesusTablet::TQuoterResourceSessionsAccumulator::SendAll(const TActorConte void TKesusTablet::Handle(TEvKesus::TEvSubscribeOnResources::TPtr& ev) { THolder<TEvKesus::TEvSubscribeOnResourcesResult> reply = MakeHolder<TEvKesus::TEvSubscribeOnResourcesResult>(); - const TActorId clientId = ActorIdFromProto(ev->Get()->Record.GetActorID()); - const TActorId pipeServerId = ev->Recipient; + const TActorId clientId = ActorIdFromProto(ev->Get()->Record.GetActorID()); + const TActorId pipeServerId = ev->Recipient; reply->Record.MutableResults()->Reserve(ev->Get()->Record.ResourcesSize()); IResourceSink::TPtr sink = new TQuoterResourceSink(ev->Sender, this); const TInstant now = TActivationContext::Now(); @@ -88,7 +88,7 @@ void TKesusTablet::Handle(TEvKesus::TEvSubscribeOnResources::TPtr& ev) { ++subscriptions; TQuoterSession* session = QuoterResources.GetOrCreateSession(clientId, resourceTree); session->SetResourceSink(sink); - const NActors::TActorId prevPipeServerId = session->SetPipeServerId(pipeServerId); + const NActors::TActorId prevPipeServerId = session->SetPipeServerId(pipeServerId); QuoterResources.SetPipeServerId(TQuoterSessionId(clientId, resourceTree->GetResourceId()), prevPipeServerId, pipeServerId); session->UpdateConsumptionState(resource.GetStartConsuming(), resource.GetInitialAmount(), queue, now); @@ -119,7 +119,7 @@ void TKesusTablet::Handle(TEvKesus::TEvSubscribeOnResources::TPtr& ev) { void TKesusTablet::Handle(TEvKesus::TEvUpdateConsumptionState::TPtr& ev) { THolder<TEvKesus::TEvResourcesAllocated> errors; - const TActorId clientId = ActorIdFromProto(ev->Get()->Record.GetActorID()); + const TActorId clientId = ActorIdFromProto(ev->Get()->Record.GetActorID()); const TInstant now = TActivationContext::Now(); IResourceSink::TPtr sink = new TQuoterResourceSink(ev->Sender, this); TTickProcessorQueue queue; diff --git a/ydb/core/kesus/tablet/tablet.cpp b/ydb/core/kesus/tablet/tablet.cpp index 9e6fb682371..3641f4df981 100644 --- a/ydb/core/kesus/tablet/tablet.cpp +++ b/ydb/core/kesus/tablet/tablet.cpp @@ -8,7 +8,7 @@ namespace NKikimr { namespace NKesus { -IActor* CreateKesusTablet(const TActorId& tablet, TTabletStorageInfo* info) { +IActor* CreateKesusTablet(const TActorId& tablet, TTabletStorageInfo* info) { return new TKesusTablet(tablet, info); } diff --git a/ydb/core/kesus/tablet/tablet.h b/ydb/core/kesus/tablet/tablet.h index 1e1e7bd950c..55fd609333c 100644 --- a/ydb/core/kesus/tablet/tablet.h +++ b/ydb/core/kesus/tablet/tablet.h @@ -7,7 +7,7 @@ namespace NKikimr { namespace NKesus { -IActor* CreateKesusTablet(const TActorId& tablet, TTabletStorageInfo* info); +IActor* CreateKesusTablet(const TActorId& tablet, TTabletStorageInfo* info); void AddKesusProbesList(); diff --git a/ydb/core/kesus/tablet/tablet_html.cpp b/ydb/core/kesus/tablet/tablet_html.cpp index 0c86d818bbc..20d269bf65a 100644 --- a/ydb/core/kesus/tablet/tablet_html.cpp +++ b/ydb/core/kesus/tablet/tablet_html.cpp @@ -1,6 +1,6 @@ #include "tablet_impl.h" -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/string/escape.h> #include <util/string/subst.h> @@ -31,7 +31,7 @@ struct TKesusTablet::THtmlRenderer { } } - void RenderProxyLink(IOutputStream& out, const TActorId& actorId) { + void RenderProxyLink(IOutputStream& out, const TActorId& actorId) { TCgiParameters params; params.InsertEscaped("proxy", TStringBuilder() << actorId); out << "<a href=\"app?TabletID=" << Self->TabletID() << "&" << params() << "\">" << actorId << "</a>"; @@ -63,7 +63,7 @@ struct TKesusTablet::THtmlRenderer { } } - void RenderProxyDetails(IOutputStream& out, const TActorId& actorId) { + void RenderProxyDetails(IOutputStream& out, const TActorId& actorId) { const auto* proxy = Self->Proxies.FindPtr(actorId); if (!proxy) { RenderError(out, "Proxy not found"); @@ -97,7 +97,7 @@ struct TKesusTablet::THtmlRenderer { } void RenderProxyDetails(IOutputStream& out, const TString& actorIdText) { - TActorId actorId; + TActorId actorId; if (!actorId.Parse(actorIdText.data(), actorIdText.size())) { RenderError(out, "Invalid proxy id"); return; @@ -429,7 +429,7 @@ struct TKesusTablet::THtmlRenderer { } TABLEBODY() { const auto& clients = resource->GetSessions(); - for (const NActors::TActorId& clientId : clients) { + for (const NActors::TActorId& clientId : clients) { const TQuoterSession* session = Self->QuoterResources.FindSession(clientId, resource->GetResourceId()); Y_VERIFY(session); TABLER() { diff --git a/ydb/core/kesus/tablet/tablet_impl.cpp b/ydb/core/kesus/tablet/tablet_impl.cpp index e55b712e570..245c4b91b8c 100644 --- a/ydb/core/kesus/tablet/tablet_impl.cpp +++ b/ydb/core/kesus/tablet/tablet_impl.cpp @@ -9,7 +9,7 @@ namespace NKikimr { namespace NKesus { -TKesusTablet::TKesusTablet(const TActorId& tablet, TTabletStorageInfo* info) +TKesusTablet::TKesusTablet(const TActorId& tablet, TTabletStorageInfo* info) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, new NMiniKQL::TMiniKQLFactory) { @@ -163,7 +163,7 @@ void TKesusTablet::Handle(TEvKesus::TEvDescribeProxies::TPtr& ev) { for (const auto& kv : Proxies) { const auto* proxy = &kv.second; auto* proxyInfo = event->Record.AddProxies(); - ActorIdToProto(proxy->ActorID, proxyInfo->MutableActorID()); + ActorIdToProto(proxy->ActorID, proxyInfo->MutableActorID()); proxyInfo->SetGeneration(proxy->Generation); for (ui64 sessionId : proxy->AttachedSessions) { proxyInfo->AddAttachedSessions(sessionId); diff --git a/ydb/core/kesus/tablet/tablet_impl.h b/ydb/core/kesus/tablet/tablet_impl.h index 5fa72fa70a6..1607e352d05 100644 --- a/ydb/core/kesus/tablet/tablet_impl.h +++ b/ydb/core/kesus/tablet/tablet_impl.h @@ -77,11 +77,11 @@ private: struct THtmlRenderer; struct TDelayedEvent { - const TActorId Recipient; + const TActorId Recipient; const ui64 Cookie; THolder<IEventBase> Event; - TDelayedEvent(const TActorId& recipient, ui64 cookie, IEventBase* event) + TDelayedEvent(const TActorId& recipient, ui64 cookie, IEventBase* event) : Recipient(recipient) , Cookie(cookie) , Event(event) @@ -89,7 +89,7 @@ private: }; struct TProxyInfo { - TActorId ActorID; + TActorId ActorID; ui64 Generation = 0; THashSet<ui64> AttachedSessions; }; @@ -128,7 +128,7 @@ private: THashMap<TSemaphoreInfo*, ui64> SemaphoreWaitCookie; THashMap<TSemaphoreInfo*, ui64> SemaphoreWatchCookie; - TActorId LastOwnerProxy; + TActorId LastOwnerProxy; ui64 LastOwnerSeqNo = 0; void ClearWatchCookies(); @@ -217,7 +217,7 @@ private: }; struct TQuoterResourceSessionsAccumulator { - void Accumulate(const TActorId& recipient, ui64 resourceId, double amount, const NKikimrKesus::TStreamingQuoterResource* props); + void Accumulate(const TActorId& recipient, ui64 resourceId, double amount, const NKikimrKesus::TStreamingQuoterResource* props); void SendAll(const TActorContext& ctx, ui64 tabletId); struct TSendInfo { @@ -225,7 +225,7 @@ private: THashMap<ui64, size_t> ResIdIndex; }; - THashMap<TActorId, TSendInfo> SendInfos; + THashMap<TActorId, TSendInfo> SendInfos; }; struct TEvPrivate { @@ -293,7 +293,7 @@ private: THashMap<ui64, TSessionInfo> Sessions; THashMap<ui64, TSemaphoreInfo> Semaphores; THashMap<TString, TSemaphoreInfo*> SemaphoresByName; - THashMap<TActorId, TProxyInfo> Proxies; + THashMap<TActorId, TProxyInfo> Proxies; THashMap<ui32, THashSet<TProxyInfo*>> ProxiesByNode; THashMap<ui64, ui64> SessionsTxCount; @@ -317,7 +317,7 @@ private: ui64 StrictMarkerCounter = 0; public: - TKesusTablet(const TActorId& tablet, TTabletStorageInfo* info); + TKesusTablet(const TActorId& tablet, TTabletStorageInfo* info); virtual ~TKesusTablet(); static constexpr NKikimrServices::TActivity::EType ActorActivityType() { diff --git a/ydb/core/kesus/tablet/tablet_ut.cpp b/ydb/core/kesus/tablet/tablet_ut.cpp index 9bff9ec80d3..1cf1a80ca8e 100644 --- a/ydb/core/kesus/tablet/tablet_ut.cpp +++ b/ydb/core/kesus/tablet/tablet_ut.cpp @@ -123,7 +123,7 @@ Y_UNIT_TEST_SUITE(TKesusTest) { ctx.Runtime->Send( new IEventHandle( ctx.Runtime->GetInterconnectProxy(0, 1), - TActorId(), + TActorId(), new TEvInterconnect::TEvDisconnect()), 0, true); TDispatchOptions options; @@ -2130,7 +2130,7 @@ Y_UNIT_TEST_SUITE(TKesusTest) { const NKikimrKesus::TEvSubscribeOnResourcesResult subscribeResult = ctx.SubscribeOnResource(client, edge, "/Root1/Res", true, 300); UNIT_ASSERT(subscribeResult.GetResults(0).HasEffectiveProps()); - auto WaitAllocated = [&ctx](const TActorId edge, double amount) -> double { + auto WaitAllocated = [&ctx](const TActorId edge, double amount) -> double { double allocated = 0.0; do { auto result = ctx.ExpectEdgeEvent<TEvKesus::TEvResourcesAllocated>(edge); @@ -2190,11 +2190,11 @@ Y_UNIT_TEST_SUITE(TKesusTest) { cfg.SetMaxUnitsPerSecond(100.0); ctx.AddQuoterResource("Root", cfg); - auto CreateSession = [&]() -> std::pair<TActorId, TActorId> { - TActorId edge = ctx.Runtime->AllocateEdgeActor(); - const TActorId sessionPipe = ctx.Runtime->ConnectToPipe(ctx.TabletId, edge, 0, GetPipeConfigWithRetries()); + auto CreateSession = [&]() -> std::pair<TActorId, TActorId> { + TActorId edge = ctx.Runtime->AllocateEdgeActor(); + const TActorId sessionPipe = ctx.Runtime->ConnectToPipe(ctx.TabletId, edge, 0, GetPipeConfigWithRetries()); auto req = MakeHolder<TEvKesus::TEvSubscribeOnResources>(); - ActorIdToProto(edge, req->Record.MutableActorID()); + ActorIdToProto(edge, req->Record.MutableActorID()); auto* reqRes = req->Record.AddResources(); reqRes->SetResourcePath("Root"); reqRes->SetStartConsuming(true); @@ -2210,10 +2210,10 @@ Y_UNIT_TEST_SUITE(TKesusTest) { return std::make_pair(edge, sessionPipe); }; - const std::pair<TActorId, TActorId> edgeAndSession1 = CreateSession(); - const std::pair<TActorId, TActorId> edgeAndSession2 = CreateSession(); + const std::pair<TActorId, TActorId> edgeAndSession1 = CreateSession(); + const std::pair<TActorId, TActorId> edgeAndSession2 = CreateSession(); - auto WaitAllocation = [&](TActorId edge, double expectedAmount) { + auto WaitAllocation = [&](TActorId edge, double expectedAmount) { size_t attempts = 30; do { auto result = ctx.ExpectEdgeEvent<TEvKesus::TEvResourcesAllocated>(edge); diff --git a/ydb/core/kesus/tablet/tx_config_get.cpp b/ydb/core/kesus/tablet/tx_config_get.cpp index 068177a4ab9..06f81a1abe0 100644 --- a/ydb/core/kesus/tablet/tx_config_get.cpp +++ b/ydb/core/kesus/tablet/tx_config_get.cpp @@ -4,12 +4,12 @@ namespace NKikimr { namespace NKesus { struct TKesusTablet::TTxConfigGet : public TTxBase { - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; THolder<TEvKesus::TEvGetConfigResult> Reply; - TTxConfigGet(TSelf* self, const TActorId& sender, ui64 cookie) + TTxConfigGet(TSelf* self, const TActorId& sender, ui64 cookie) : TTxBase(self) , Sender(sender) , Cookie(cookie) diff --git a/ydb/core/kesus/tablet/tx_config_set.cpp b/ydb/core/kesus/tablet/tx_config_set.cpp index 72a146ab478..93eae8a9edf 100644 --- a/ydb/core/kesus/tablet/tx_config_set.cpp +++ b/ydb/core/kesus/tablet/tx_config_set.cpp @@ -7,13 +7,13 @@ namespace NKikimr { namespace NKesus { struct TKesusTablet::TTxConfigSet : public TTxBase { - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; const NKikimrKesus::TEvSetConfig Record; THolder<TEvKesus::TEvSetConfigResult> Reply; - TTxConfigSet(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvSetConfig& record) + TTxConfigSet(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvSetConfig& record) : TTxBase(self) , Sender(sender) , Cookie(cookie) diff --git a/ydb/core/kesus/tablet/tx_dummy.cpp b/ydb/core/kesus/tablet/tx_dummy.cpp index 8f4859ec634..f92c5eb9319 100644 --- a/ydb/core/kesus/tablet/tx_dummy.cpp +++ b/ydb/core/kesus/tablet/tx_dummy.cpp @@ -4,10 +4,10 @@ namespace NKikimr { namespace NKesus { struct TKesusTablet::TTxDummy : public TTxBase { - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; - TTxDummy(TSelf* self, const TActorId& sender, ui64 cookie) + TTxDummy(TSelf* self, const TActorId& sender, ui64 cookie) : TTxBase(self) , Sender(sender) , Cookie(cookie) diff --git a/ydb/core/kesus/tablet/tx_init.cpp b/ydb/core/kesus/tablet/tx_init.cpp index 063f6df44a3..1f3f96ff9f3 100644 --- a/ydb/core/kesus/tablet/tx_init.cpp +++ b/ydb/core/kesus/tablet/tx_init.cpp @@ -12,7 +12,7 @@ namespace NKikimr { namespace NKesus { struct TKesusTablet::TTxInit : public TTxBase { - TActorId PreviousTabletActorID; + TActorId PreviousTabletActorID; explicit TTxInit(TSelf* self) : TTxBase(self) diff --git a/ydb/core/kesus/tablet/tx_quoter_resource_add.cpp b/ydb/core/kesus/tablet/tx_quoter_resource_add.cpp index 3609630594b..fa5755eb6ee 100644 --- a/ydb/core/kesus/tablet/tx_quoter_resource_add.cpp +++ b/ydb/core/kesus/tablet/tx_quoter_resource_add.cpp @@ -6,13 +6,13 @@ namespace NKikimr { namespace NKesus { struct TKesusTablet::TTxQuoterResourceAdd : public TTxBase { - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; NKikimrKesus::TEvAddQuoterResource Record; THolder<TEvKesus::TEvAddQuoterResourceResult> Reply; - TTxQuoterResourceAdd(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvAddQuoterResource& record) + TTxQuoterResourceAdd(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvAddQuoterResource& record) : TTxBase(self) , Sender(sender) , Cookie(cookie) diff --git a/ydb/core/kesus/tablet/tx_quoter_resource_delete.cpp b/ydb/core/kesus/tablet/tx_quoter_resource_delete.cpp index f5d292021bb..33048ba7051 100644 --- a/ydb/core/kesus/tablet/tx_quoter_resource_delete.cpp +++ b/ydb/core/kesus/tablet/tx_quoter_resource_delete.cpp @@ -4,13 +4,13 @@ namespace NKikimr { namespace NKesus { struct TKesusTablet::TTxQuoterResourceDelete : public TTxBase { - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; NKikimrKesus::TEvDeleteQuoterResource Record; THolder<TEvKesus::TEvDeleteQuoterResourceResult> Reply; - TTxQuoterResourceDelete(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvDeleteQuoterResource& record) + TTxQuoterResourceDelete(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvDeleteQuoterResource& record) : TTxBase(self) , Sender(sender) , Cookie(cookie) diff --git a/ydb/core/kesus/tablet/tx_quoter_resource_describe.cpp b/ydb/core/kesus/tablet/tx_quoter_resource_describe.cpp index a0bb095fe3e..3e78d3c5102 100644 --- a/ydb/core/kesus/tablet/tx_quoter_resource_describe.cpp +++ b/ydb/core/kesus/tablet/tx_quoter_resource_describe.cpp @@ -6,13 +6,13 @@ namespace NKikimr { namespace NKesus { struct TKesusTablet::TTxQuoterResourceDescribe : public TTxBase { - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; const NKikimrKesus::TEvDescribeQuoterResources Record; THolder<TEvKesus::TEvDescribeQuoterResourcesResult> Reply; - TTxQuoterResourceDescribe(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvDescribeQuoterResources& record) + TTxQuoterResourceDescribe(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvDescribeQuoterResources& record) : TTxBase(self) , Sender(sender) , Cookie(cookie) diff --git a/ydb/core/kesus/tablet/tx_quoter_resource_update.cpp b/ydb/core/kesus/tablet/tx_quoter_resource_update.cpp index d30d09247bc..71896120613 100644 --- a/ydb/core/kesus/tablet/tx_quoter_resource_update.cpp +++ b/ydb/core/kesus/tablet/tx_quoter_resource_update.cpp @@ -4,13 +4,13 @@ namespace NKikimr { namespace NKesus { struct TKesusTablet::TTxQuoterResourceUpdate : public TTxBase { - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; NKikimrKesus::TEvUpdateQuoterResource Record; THolder<TEvKesus::TEvUpdateQuoterResourceResult> Reply; - TTxQuoterResourceUpdate(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvUpdateQuoterResource& record) + TTxQuoterResourceUpdate(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvUpdateQuoterResource& record) : TTxBase(self) , Sender(sender) , Cookie(cookie) diff --git a/ydb/core/kesus/tablet/tx_semaphore_acquire.cpp b/ydb/core/kesus/tablet/tx_semaphore_acquire.cpp index 48d532f746a..996dca18bf9 100644 --- a/ydb/core/kesus/tablet/tx_semaphore_acquire.cpp +++ b/ydb/core/kesus/tablet/tx_semaphore_acquire.cpp @@ -4,13 +4,13 @@ namespace NKikimr { namespace NKesus { struct TKesusTablet::TTxSemaphoreAcquire : public TTxBase { - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; const NKikimrKesus::TEvAcquireSemaphore Record; TVector<TDelayedEvent> Events; - TTxSemaphoreAcquire(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvAcquireSemaphore& record) + TTxSemaphoreAcquire(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvAcquireSemaphore& record) : TTxBase(self) , Sender(sender) , Cookie(cookie) diff --git a/ydb/core/kesus/tablet/tx_semaphore_create.cpp b/ydb/core/kesus/tablet/tx_semaphore_create.cpp index 0783f837c2b..100cf550b3b 100644 --- a/ydb/core/kesus/tablet/tx_semaphore_create.cpp +++ b/ydb/core/kesus/tablet/tx_semaphore_create.cpp @@ -4,13 +4,13 @@ namespace NKikimr { namespace NKesus { struct TKesusTablet::TTxSemaphoreCreate : public TTxBase { - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; const NKikimrKesus::TEvCreateSemaphore Record; THolder<TEvKesus::TEvCreateSemaphoreResult> Reply; - TTxSemaphoreCreate(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvCreateSemaphore& record) + TTxSemaphoreCreate(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvCreateSemaphore& record) : TTxBase(self) , Sender(sender) , Cookie(cookie) diff --git a/ydb/core/kesus/tablet/tx_semaphore_delete.cpp b/ydb/core/kesus/tablet/tx_semaphore_delete.cpp index 2f5e73bb6be..bdd94a6511f 100644 --- a/ydb/core/kesus/tablet/tx_semaphore_delete.cpp +++ b/ydb/core/kesus/tablet/tx_semaphore_delete.cpp @@ -4,13 +4,13 @@ namespace NKikimr { namespace NKesus { struct TKesusTablet::TTxSemaphoreDelete : public TTxBase { - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; const NKikimrKesus::TEvDeleteSemaphore Record; TVector<TDelayedEvent> Events; - TTxSemaphoreDelete(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvDeleteSemaphore& record) + TTxSemaphoreDelete(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvDeleteSemaphore& record) : TTxBase(self) , Sender(sender) , Cookie(cookie) diff --git a/ydb/core/kesus/tablet/tx_semaphore_describe.cpp b/ydb/core/kesus/tablet/tx_semaphore_describe.cpp index 10525e50b29..d6595232f16 100644 --- a/ydb/core/kesus/tablet/tx_semaphore_describe.cpp +++ b/ydb/core/kesus/tablet/tx_semaphore_describe.cpp @@ -4,7 +4,7 @@ namespace NKikimr { namespace NKesus { struct TKesusTablet::TTxSemaphoreDescribe : public TTxBase { - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; const NKikimrKesus::TEvDescribeSemaphore Record; @@ -12,7 +12,7 @@ struct TKesusTablet::TTxSemaphoreDescribe : public TTxBase { THolder<TEvKesus::TEvDescribeSemaphoreResult> Reply; ui64 NotificationCookie = 0; - TTxSemaphoreDescribe(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvDescribeSemaphore& record) + TTxSemaphoreDescribe(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvDescribeSemaphore& record) : TTxBase(self) , Sender(sender) , Cookie(cookie) diff --git a/ydb/core/kesus/tablet/tx_semaphore_release.cpp b/ydb/core/kesus/tablet/tx_semaphore_release.cpp index 03db78786e6..326a3318613 100644 --- a/ydb/core/kesus/tablet/tx_semaphore_release.cpp +++ b/ydb/core/kesus/tablet/tx_semaphore_release.cpp @@ -4,13 +4,13 @@ namespace NKikimr { namespace NKesus { struct TKesusTablet::TTxSemaphoreRelease : public TTxBase { - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; const NKikimrKesus::TEvReleaseSemaphore Record; TVector<TDelayedEvent> Events; - TTxSemaphoreRelease(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvReleaseSemaphore& record) + TTxSemaphoreRelease(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvReleaseSemaphore& record) : TTxBase(self) , Sender(sender) , Cookie(cookie) diff --git a/ydb/core/kesus/tablet/tx_semaphore_update.cpp b/ydb/core/kesus/tablet/tx_semaphore_update.cpp index 660d5ac7dab..08d595555e9 100644 --- a/ydb/core/kesus/tablet/tx_semaphore_update.cpp +++ b/ydb/core/kesus/tablet/tx_semaphore_update.cpp @@ -4,14 +4,14 @@ namespace NKikimr { namespace NKesus { struct TKesusTablet::TTxSemaphoreUpdate : public TTxBase { - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; const NKikimrKesus::TEvUpdateSemaphore Record; TVector<TDelayedEvent> Events; THolder<TEvKesus::TEvUpdateSemaphoreResult> Reply; - TTxSemaphoreUpdate(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvUpdateSemaphore& record) + TTxSemaphoreUpdate(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvUpdateSemaphore& record) : TTxBase(self) , Sender(sender) , Cookie(cookie) diff --git a/ydb/core/kesus/tablet/tx_session_attach.cpp b/ydb/core/kesus/tablet/tx_session_attach.cpp index 769cbc50877..ca5f4191f2a 100644 --- a/ydb/core/kesus/tablet/tx_session_attach.cpp +++ b/ydb/core/kesus/tablet/tx_session_attach.cpp @@ -4,17 +4,17 @@ namespace NKikimr { namespace NKesus { struct TKesusTablet::TTxSessionAttach : public TTxBase { - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; const NKikimrKesus::TEvAttachSession Record; ui64 SessionId; - TActorId PreviousOwner; + TActorId PreviousOwner; ui64 PreviousGeneration = 0; ui64 PreviousCookie = 0; THolder<TEvKesus::TEvAttachSessionResult> Reply; - explicit TTxSessionAttach(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvAttachSession& record) + explicit TTxSessionAttach(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvAttachSession& record) : TTxBase(self) , Sender(sender) , Cookie(cookie) diff --git a/ydb/core/kesus/tablet/tx_session_destroy.cpp b/ydb/core/kesus/tablet/tx_session_destroy.cpp index d3182d0e0ea..45c81456c88 100644 --- a/ydb/core/kesus/tablet/tx_session_destroy.cpp +++ b/ydb/core/kesus/tablet/tx_session_destroy.cpp @@ -4,13 +4,13 @@ namespace NKikimr { namespace NKesus { struct TKesusTablet::TTxSessionDestroy : public TTxBase { - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; const NKikimrKesus::TEvDestroySession Record; TVector<TDelayedEvent> Events; - TTxSessionDestroy(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvDestroySession& record) + TTxSessionDestroy(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvDestroySession& record) : TTxBase(self) , Sender(sender) , Cookie(cookie) diff --git a/ydb/core/kesus/tablet/tx_session_detach.cpp b/ydb/core/kesus/tablet/tx_session_detach.cpp index ce9146a3046..1fb85da73cc 100644 --- a/ydb/core/kesus/tablet/tx_session_detach.cpp +++ b/ydb/core/kesus/tablet/tx_session_detach.cpp @@ -4,13 +4,13 @@ namespace NKikimr { namespace NKesus { struct TKesusTablet::TTxSessionDetach : public TTxBase { - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; const NKikimrKesus::TEvDetachSession Record; THolder<TEvKesus::TEvDetachSessionResult> Reply; - explicit TTxSessionDetach(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvDetachSession& record) + explicit TTxSessionDetach(TSelf* self, const TActorId& sender, ui64 cookie, const NKikimrKesus::TEvDetachSession& record) : TTxBase(self) , Sender(sender) , Cookie(cookie) diff --git a/ydb/core/kesus/tablet/tx_sessions_describe.cpp b/ydb/core/kesus/tablet/tx_sessions_describe.cpp index fc8fe9174b3..7f2ea8a0eb6 100644 --- a/ydb/core/kesus/tablet/tx_sessions_describe.cpp +++ b/ydb/core/kesus/tablet/tx_sessions_describe.cpp @@ -4,12 +4,12 @@ namespace NKikimr { namespace NKesus { struct TKesusTablet::TTxSessionsDescribe : public TTxBase { - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; THolder<TEvKesus::TEvDescribeSessionsResult> Reply; - TTxSessionsDescribe(TSelf* self, const TActorId& sender, ui64 cookie) + TTxSessionsDescribe(TSelf* self, const TActorId& sender, ui64 cookie) : TTxBase(self) , Sender(sender) , Cookie(cookie) @@ -35,7 +35,7 @@ struct TKesusTablet::TTxSessionsDescribe : public TTxBase { sessionInfo->SetTimeoutMillis(session->TimeoutMillis); sessionInfo->SetDescription(session->Description); if (session->OwnerProxy) { - ActorIdToProto(session->OwnerProxy->ActorID, sessionInfo->MutableOwnerProxy()); + ActorIdToProto(session->OwnerProxy->ActorID, sessionInfo->MutableOwnerProxy()); } } diff --git a/ydb/core/kesus/tablet/ut_helpers.cpp b/ydb/core/kesus/tablet/ut_helpers.cpp index 4c58945bfd5..f559800a769 100644 --- a/ydb/core/kesus/tablet/ut_helpers.cpp +++ b/ydb/core/kesus/tablet/ut_helpers.cpp @@ -104,14 +104,14 @@ void TTestContext::SetupTabletServices() { } void TTestContext::Sleep(ui64 millis) { - TActorId sender = Runtime->AllocateEdgeActor(); + TActorId sender = Runtime->AllocateEdgeActor(); Runtime->Schedule(new IEventHandle(sender, sender, new TEvents::TEvWakeup()), TDuration::MilliSeconds(millis), 0); ExpectEdgeEvent<TEvents::TEvWakeup>(sender, 0); } void TTestContext::RebootTablet() { ui32 nodeIndex = 0; - TActorId sender = Runtime->AllocateEdgeActor(nodeIndex); + TActorId sender = Runtime->AllocateEdgeActor(nodeIndex); ForwardToTablet(*Runtime, TabletId, sender, new TEvents::TEvPoisonPill(), nodeIndex); { TDispatchOptions options; @@ -122,9 +122,9 @@ void TTestContext::RebootTablet() { ProxyClients.clear(); } -TActorId TTestContext::GetTabletActorId() { +TActorId TTestContext::GetTabletActorId() { ui64 cookie = RandomNumber<ui64>(); - TActorId edge = Runtime->AllocateEdgeActor(); + TActorId edge = Runtime->AllocateEdgeActor(); SendFromEdge(edge, new TEvKesus::TEvDummyRequest(), cookie); TAutoPtr<IEventHandle> handle; Runtime->GrabEdgeEvent<TEvKesus::TEvDummyResponse>(handle); @@ -134,7 +134,7 @@ TActorId TTestContext::GetTabletActorId() { return handle->Sender; } -void TTestContext::SendFromEdge(const TActorId& edge, IEventBase* payload, ui64 cookie) { +void TTestContext::SendFromEdge(const TActorId& edge, IEventBase* payload, ui64 cookie) { ui32 nodeIndex = edge.NodeId() - Runtime->GetNodeId(0); Runtime->SendToPipe( TabletId, @@ -142,13 +142,13 @@ void TTestContext::SendFromEdge(const TActorId& edge, IEventBase* payload, ui64 payload, nodeIndex, GetPipeConfigWithRetries(), - TActorId(), + TActorId(), cookie); } -void TTestContext::SendFromProxy(const TActorId& proxy, ui64 generation, IEventBase* payload, ui64 cookie) { +void TTestContext::SendFromProxy(const TActorId& proxy, ui64 generation, IEventBase* payload, ui64 cookie) { ui32 nodeIndex = proxy.NodeId() - Runtime->GetNodeId(0); - TActorId& clientId = ProxyClients[std::tie(proxy, generation)]; + TActorId& clientId = ProxyClients[std::tie(proxy, generation)]; if (!clientId) { clientId = Runtime->ConnectToPipe(TabletId, proxy, nodeIndex, GetPipeConfigWithRetries()); } @@ -184,13 +184,13 @@ NKikimrKesus::TEvSetConfigResult TTestContext::SetConfig(ui64 txId, const Ydb::C return result->Record; } -void TTestContext::SyncProxy(const TActorId& proxy, ui64 generation, bool useTransactions) { +void TTestContext::SyncProxy(const TActorId& proxy, ui64 generation, bool useTransactions) { ui64 cookie = RandomNumber<ui64>(); SendFromProxy(proxy, generation, new TEvKesus::TEvDummyRequest(useTransactions), cookie); ExpectEdgeEvent<TEvKesus::TEvDummyResponse>(proxy, cookie); } -NKikimrKesus::TEvRegisterProxyResult TTestContext::RegisterProxy(const TActorId& proxy, ui64 generation) { +NKikimrKesus::TEvRegisterProxyResult TTestContext::RegisterProxy(const TActorId& proxy, ui64 generation) { ui64 cookie = RandomNumber<ui64>(); SendFromProxy(proxy, generation, new TEvKesus::TEvRegisterProxy("", generation), cookie); auto result = ExpectEdgeEvent<TEvKesus::TEvRegisterProxyResult>(proxy, cookie); @@ -198,12 +198,12 @@ NKikimrKesus::TEvRegisterProxyResult TTestContext::RegisterProxy(const TActorId& return result->Record; } -void TTestContext::MustRegisterProxy(const TActorId& proxy, ui64 generation, Ydb::StatusIds::StatusCode status) { +void TTestContext::MustRegisterProxy(const TActorId& proxy, ui64 generation, Ydb::StatusIds::StatusCode status) { auto result = RegisterProxy(proxy, generation); UNIT_ASSERT_VALUES_EQUAL(result.GetError().GetStatus(), status); } -NKikimrKesus::TEvUnregisterProxyResult TTestContext::UnregisterProxy(const TActorId& proxy, ui64 generation) { +NKikimrKesus::TEvUnregisterProxyResult TTestContext::UnregisterProxy(const TActorId& proxy, ui64 generation) { ui64 cookie = RandomNumber<ui64>(); SendFromProxy(proxy, generation, new TEvKesus::TEvUnregisterProxy("", generation), cookie); auto result = ExpectEdgeEvent<TEvKesus::TEvUnregisterProxyResult>(proxy, cookie); @@ -211,13 +211,13 @@ NKikimrKesus::TEvUnregisterProxyResult TTestContext::UnregisterProxy(const TActo return result->Record; } -void TTestContext::MustUnregisterProxy(const TActorId& proxy, ui64 generation, Ydb::StatusIds::StatusCode status) { +void TTestContext::MustUnregisterProxy(const TActorId& proxy, ui64 generation, Ydb::StatusIds::StatusCode status) { auto result = UnregisterProxy(proxy, generation); UNIT_ASSERT_VALUES_EQUAL(result.GetError().GetStatus(), status); } void TTestContext::SendAttachSession( - ui64 cookie, const TActorId& proxy, ui64 generation, ui64 sessionId, + ui64 cookie, const TActorId& proxy, ui64 generation, ui64 sessionId, ui64 timeoutMillis, const TString& description, ui64 seqNo, const TString& key) { @@ -225,7 +225,7 @@ void TTestContext::SendAttachSession( } NKikimrKesus::TEvAttachSessionResult TTestContext::NextAttachSessionResult( - ui64 cookie, const TActorId& proxy, ui64 generation) + ui64 cookie, const TActorId& proxy, ui64 generation) { auto result = ExpectEdgeEvent<TEvKesus::TEvAttachSessionResult>(proxy, cookie); UNIT_ASSERT_VALUES_EQUAL(result->Record.GetProxyGeneration(), generation); @@ -233,7 +233,7 @@ NKikimrKesus::TEvAttachSessionResult TTestContext::NextAttachSessionResult( } ui64 TTestContext::ExpectAttachSessionResult( - ui64 cookie, const TActorId& proxy, ui64 generation, Ydb::StatusIds::StatusCode status) + ui64 cookie, const TActorId& proxy, ui64 generation, Ydb::StatusIds::StatusCode status) { auto result = NextAttachSessionResult(cookie, proxy, generation); UNIT_ASSERT_VALUES_EQUAL(result.GetError().GetStatus(), status); @@ -241,7 +241,7 @@ ui64 TTestContext::ExpectAttachSessionResult( } NKikimrKesus::TEvAttachSessionResult TTestContext::AttachSession( - const TActorId& proxy, ui64 generation, ui64 sessionId, + const TActorId& proxy, ui64 generation, ui64 sessionId, ui64 timeoutMillis, const TString& description, ui64 seqNo, const TString& key) { @@ -251,7 +251,7 @@ NKikimrKesus::TEvAttachSessionResult TTestContext::AttachSession( } ui64 TTestContext::MustAttachSession( - const TActorId& proxy, ui64 generation, ui64 sessionId, + const TActorId& proxy, ui64 generation, ui64 sessionId, ui64 timeoutMillis, const TString& description, ui64 seqNo, const TString& key) { @@ -260,7 +260,7 @@ ui64 TTestContext::MustAttachSession( return ExpectAttachSessionResult(cookie, proxy, generation); } -NKikimrKesus::TEvDetachSessionResult TTestContext::DetachSession(const TActorId& proxy, ui64 generation, ui64 sessionId) { +NKikimrKesus::TEvDetachSessionResult TTestContext::DetachSession(const TActorId& proxy, ui64 generation, ui64 sessionId) { ui64 cookie = RandomNumber<ui64>(); SendFromProxy(proxy, generation, new TEvKesus::TEvDetachSession("", generation, sessionId), cookie); auto result = ExpectEdgeEvent<TEvKesus::TEvDetachSessionResult>(proxy, cookie); @@ -268,12 +268,12 @@ NKikimrKesus::TEvDetachSessionResult TTestContext::DetachSession(const TActorId& return result->Record; } -void TTestContext::MustDetachSession(const TActorId& proxy, ui64 generation, ui64 sessionId, Ydb::StatusIds::StatusCode status) { +void TTestContext::MustDetachSession(const TActorId& proxy, ui64 generation, ui64 sessionId, Ydb::StatusIds::StatusCode status) { auto result = DetachSession(proxy, generation, sessionId); UNIT_ASSERT_VALUES_EQUAL(result.GetError().GetStatus(), status); } -NKikimrKesus::TEvDestroySessionResult TTestContext::DestroySession(const TActorId& proxy, ui64 generation, ui64 sessionId) { +NKikimrKesus::TEvDestroySessionResult TTestContext::DestroySession(const TActorId& proxy, ui64 generation, ui64 sessionId) { ui64 cookie = RandomNumber<ui64>(); SendFromProxy(proxy, generation, new TEvKesus::TEvDestroySession("", generation, sessionId), cookie); auto result = ExpectEdgeEvent<TEvKesus::TEvDestroySessionResult>(proxy, cookie); @@ -281,13 +281,13 @@ NKikimrKesus::TEvDestroySessionResult TTestContext::DestroySession(const TActorI return result->Record; } -void TTestContext::MustDestroySession(const TActorId& proxy, ui64 generation, ui64 sessionId, Ydb::StatusIds::StatusCode status) { +void TTestContext::MustDestroySession(const TActorId& proxy, ui64 generation, ui64 sessionId, Ydb::StatusIds::StatusCode status) { auto result = DestroySession(proxy, generation, sessionId); UNIT_ASSERT_VALUES_EQUAL(result.GetError().GetStatus(), status); } void TTestContext::SendAcquireLock( - ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, + ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& lockName, ELockMode mode, ui64 timeoutMillis, const TString& data) { @@ -305,18 +305,18 @@ void TTestContext::SendAcquireLock( SendFromProxy(proxy, generation, new TEvKesus::TEvAcquireSemaphore("", generation, sessionId, lockName, count, timeoutMillis, data, true), reqId); } -bool TTestContext::ExpectAcquireLockResult(ui64 reqId, const TActorId& proxy, ui64 generation, Ydb::StatusIds::StatusCode status) { +bool TTestContext::ExpectAcquireLockResult(ui64 reqId, const TActorId& proxy, ui64 generation, Ydb::StatusIds::StatusCode status) { auto result = ExpectEdgeEvent<TEvKesus::TEvAcquireSemaphoreResult>(proxy, reqId); UNIT_ASSERT_VALUES_EQUAL(result->Record.GetProxyGeneration(), generation); UNIT_ASSERT_VALUES_EQUAL(result->Record.GetError().GetStatus(), status); return result->Record.GetAcquired(); } -void TTestContext::ExpectAcquireLockResult(ui64 reqId, const TActorId& proxy, ui64 generation, bool acquired) { +void TTestContext::ExpectAcquireLockResult(ui64 reqId, const TActorId& proxy, ui64 generation, bool acquired) { UNIT_ASSERT_VALUES_EQUAL(ExpectAcquireLockResult(reqId, proxy, generation, Ydb::StatusIds::SUCCESS), acquired); } -bool TTestContext::MustReleaseLock(ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& lockName, Ydb::StatusIds::StatusCode status) { +bool TTestContext::MustReleaseLock(ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& lockName, Ydb::StatusIds::StatusCode status) { SendFromProxy(proxy, generation, new TEvKesus::TEvReleaseSemaphore("", generation, sessionId, lockName), reqId); auto result = ExpectEdgeEvent<TEvKesus::TEvReleaseSemaphoreResult>(proxy, reqId); UNIT_ASSERT_VALUES_EQUAL(result->Record.GetProxyGeneration(), generation); @@ -324,7 +324,7 @@ bool TTestContext::MustReleaseLock(ui64 reqId, const TActorId& proxy, ui64 gener return result->Record.GetReleased(); } -void TTestContext::MustReleaseLock(ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& lockName, bool released) { +void TTestContext::MustReleaseLock(ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& lockName, bool released) { UNIT_ASSERT_VALUES_EQUAL(MustReleaseLock(reqId, proxy, generation, sessionId, lockName, Ydb::StatusIds::SUCCESS), released); } @@ -337,7 +337,7 @@ void TTestContext::CreateSemaphore(const TString& name, ui64 limit, const TStrin } void TTestContext::SessionCreateSemaphore( - ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, + ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& name, ui64 limit, const TString& data, Ydb::StatusIds::StatusCode status) { auto event = new TEvKesus::TEvCreateSemaphore("", name, limit, data); @@ -358,7 +358,7 @@ void TTestContext::UpdateSemaphore(const TString& name, const TString& data, Ydb } void TTestContext::SessionUpdateSemaphore( - ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, + ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& name, const TString& data, Ydb::StatusIds::StatusCode status) { auto event = new TEvKesus::TEvUpdateSemaphore("", name, data); @@ -379,7 +379,7 @@ void TTestContext::DeleteSemaphore(const TString& name, bool force, Ydb::StatusI } void TTestContext::SessionDeleteSemaphore( - ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, + ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& name, Ydb::StatusIds::StatusCode status) { auto event = new TEvKesus::TEvDeleteSemaphore("", name, false); @@ -392,31 +392,31 @@ void TTestContext::SessionDeleteSemaphore( } void TTestContext::SendAcquireSemaphore( - ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, + ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& name, ui64 count, ui64 timeoutMillis, const TString& data) { SendFromProxy(proxy, generation, new TEvKesus::TEvAcquireSemaphore("", generation, sessionId, name, count, timeoutMillis, data), reqId); } -bool TTestContext::ExpectAcquireSemaphoreResult(ui64 reqId, const TActorId& proxy, ui64 generation, Ydb::StatusIds::StatusCode status) { +bool TTestContext::ExpectAcquireSemaphoreResult(ui64 reqId, const TActorId& proxy, ui64 generation, Ydb::StatusIds::StatusCode status) { auto result = ExpectEdgeEvent<TEvKesus::TEvAcquireSemaphoreResult>(proxy, reqId); UNIT_ASSERT_VALUES_EQUAL(result->Record.GetProxyGeneration(), generation); UNIT_ASSERT_VALUES_EQUAL(result->Record.GetError().GetStatus(), status); return result->Record.GetAcquired(); } -void TTestContext::ExpectAcquireSemaphoreResult(ui64 reqId, const TActorId& proxy, ui64 generation, bool acquired) { +void TTestContext::ExpectAcquireSemaphoreResult(ui64 reqId, const TActorId& proxy, ui64 generation, bool acquired) { UNIT_ASSERT_VALUES_EQUAL(ExpectAcquireSemaphoreResult(reqId, proxy, generation, Ydb::StatusIds::SUCCESS), acquired); } -void TTestContext::ExpectAcquireSemaphorePending(ui64 reqId, const TActorId& proxy, ui64 generation) { +void TTestContext::ExpectAcquireSemaphorePending(ui64 reqId, const TActorId& proxy, ui64 generation) { auto result = ExpectEdgeEvent<TEvKesus::TEvAcquireSemaphorePending>(proxy, reqId); UNIT_ASSERT_VALUES_EQUAL(result->Record.GetProxyGeneration(), generation); } bool TTestContext::MustReleaseSemaphore( - ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& name, + ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& name, Ydb::StatusIds::StatusCode status) { SendFromProxy(proxy, generation, new TEvKesus::TEvReleaseSemaphore("", generation, sessionId, name), reqId); @@ -427,19 +427,19 @@ bool TTestContext::MustReleaseSemaphore( } void TTestContext::MustReleaseSemaphore( - ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& name, bool released) + ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& name, bool released) { UNIT_ASSERT_VALUES_EQUAL(MustReleaseSemaphore(reqId, proxy, generation, sessionId, name, Ydb::StatusIds::SUCCESS), released); } -THashMap<TActorId, TTestContext::TSimpleProxyInfo> TTestContext::DescribeProxies() { +THashMap<TActorId, TTestContext::TSimpleProxyInfo> TTestContext::DescribeProxies() { ui64 cookie = RandomNumber<ui64>(); - TActorId edge = Runtime->AllocateEdgeActor(); + TActorId edge = Runtime->AllocateEdgeActor(); SendFromEdge(edge, new TEvKesus::TEvDescribeProxies(""), cookie); auto result = ExpectEdgeEvent<TEvKesus::TEvDescribeProxiesResult>(edge, cookie); - THashMap<TActorId, TSimpleProxyInfo> proxies; + THashMap<TActorId, TSimpleProxyInfo> proxies; for (const auto& proxyDesc : result->Record.GetProxies()) { - TActorId proxyId = ActorIdFromProto(proxyDesc.GetActorID()); + TActorId proxyId = ActorIdFromProto(proxyDesc.GetActorID()); auto& proxyInfo = proxies[proxyId]; proxyInfo.Generation = proxyDesc.GetGeneration(); for (ui64 sessionId : proxyDesc.GetAttachedSessions()) { @@ -449,19 +449,19 @@ THashMap<TActorId, TTestContext::TSimpleProxyInfo> TTestContext::DescribeProxies return proxies; } -void TTestContext::VerifyProxyRegistered(const TActorId& proxy, ui64 generation) { +void TTestContext::VerifyProxyRegistered(const TActorId& proxy, ui64 generation) { auto proxies = DescribeProxies(); UNIT_ASSERT_C(proxies.contains(proxy), "Proxy " << proxy << " is not registered"); const auto& proxyInfo = proxies.at(proxy); UNIT_ASSERT_VALUES_EQUAL(proxyInfo.Generation, generation); } -void TTestContext::VerifyProxyNotRegistered(const TActorId& proxy) { +void TTestContext::VerifyProxyNotRegistered(const TActorId& proxy) { auto proxies = DescribeProxies(); UNIT_ASSERT_C(!proxies.contains(proxy), "Proxy " << proxy << " is registered"); } -void TTestContext::VerifyProxyHasSessions(const TActorId& proxy, ui64 generation, const THashSet<ui64>& expectedSessions) { +void TTestContext::VerifyProxyHasSessions(const TActorId& proxy, ui64 generation, const THashSet<ui64>& expectedSessions) { auto proxies = DescribeProxies(); UNIT_ASSERT_C(proxies.contains(proxy), "Proxy " << proxy << " is not registered"); const auto& proxyInfo = proxies.at(proxy); @@ -476,7 +476,7 @@ void TTestContext::VerifyProxyHasSessions(const TActorId& proxy, ui64 generation THashMap<ui64, TTestContext::TSimpleSessionInfo> TTestContext::DescribeSessions() { ui64 cookie = RandomNumber<ui64>(); - TActorId edge = Runtime->AllocateEdgeActor(); + TActorId edge = Runtime->AllocateEdgeActor(); SendFromEdge(edge, new TEvKesus::TEvDescribeSessions(""), cookie); auto result = ExpectEdgeEvent<TEvKesus::TEvDescribeSessionsResult>(edge, cookie); THashMap<ui64, TSimpleSessionInfo> sessions; @@ -485,7 +485,7 @@ THashMap<ui64, TTestContext::TSimpleSessionInfo> TTestContext::DescribeSessions( auto& session = sessions[sessionInfo.GetSessionId()]; session.TimeoutMillis = sessionInfo.GetTimeoutMillis(); session.Description = sessionInfo.GetDescription(); - session.OwnerProxy = ActorIdFromProto(sessionInfo.GetOwnerProxy()); + session.OwnerProxy = ActorIdFromProto(sessionInfo.GetOwnerProxy()); } return sessions; } @@ -502,7 +502,7 @@ void TTestContext::VerifySessionExists(ui64 sessionId) { TTestContext::TSimpleLockDescription TTestContext::DescribeLock(const TString& lockName, bool includeWaiters) { ui64 cookie = RandomNumber<ui64>(); - TActorId edge = Runtime->AllocateEdgeActor(); + TActorId edge = Runtime->AllocateEdgeActor(); auto request = new TEvKesus::TEvDescribeSemaphore("", lockName); request->Record.SetIncludeOwners(true); request->Record.SetIncludeWaiters(includeWaiters); @@ -594,7 +594,7 @@ TTestContext::TSimpleSemaphoreDescription MakeSimpleSemaphoreDescription(const Y TTestContext::TSimpleSemaphoreDescription TTestContext::DescribeSemaphore(const TString& name, bool includeWaiters) { ui64 cookie = RandomNumber<ui64>(); - TActorId edge = Runtime->AllocateEdgeActor(); + TActorId edge = Runtime->AllocateEdgeActor(); auto request = new TEvKesus::TEvDescribeSemaphore("", name); request->Record.SetIncludeOwners(true); request->Record.SetIncludeWaiters(includeWaiters); @@ -607,7 +607,7 @@ TTestContext::TSimpleSemaphoreDescription TTestContext::DescribeSemaphore(const void TTestContext::VerifySemaphoreNotFound(const TString& name) { ui64 cookie = RandomNumber<ui64>(); - TActorId edge = Runtime->AllocateEdgeActor(); + TActorId edge = Runtime->AllocateEdgeActor(); SendFromEdge(edge, new TEvKesus::TEvDescribeSemaphore("", name), cookie); auto event = ExpectEdgeEvent<TEvKesus::TEvDescribeSemaphoreResult>(edge, cookie); UNIT_ASSERT_VALUES_EQUAL(event->Record.GetError().GetStatus(), Ydb::StatusIds::NOT_FOUND); @@ -638,7 +638,7 @@ void TTestContext::VerifySemaphoreWaiters(const TString& name, const THashSet<ui } void TTestContext::SendSessionDescribeSemaphore( - ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, + ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& name, bool watchData, bool watchOwners) { auto event = new TEvKesus::TEvDescribeSemaphore("", name); @@ -650,7 +650,7 @@ void TTestContext::SendSessionDescribeSemaphore( } TTestContext::TSimpleSemaphoreDescription TTestContext::ExpectDescribeSemaphoreResult( - ui64 reqId, const TActorId& proxy, ui64 generation, + ui64 reqId, const TActorId& proxy, ui64 generation, Ydb::StatusIds::StatusCode status) { auto event = ExpectEdgeEvent<TEvKesus::TEvDescribeSemaphoreResult>(proxy, reqId); @@ -662,7 +662,7 @@ TTestContext::TSimpleSemaphoreDescription TTestContext::ExpectDescribeSemaphoreR } TTestContext::TDescribeSemaphoreChanges TTestContext::ExpectDescribeSemaphoreChanged( - ui64 reqId, const TActorId& proxy, ui64 generation) + ui64 reqId, const TActorId& proxy, ui64 generation) { auto event = ExpectEdgeEvent<TEvKesus::TEvDescribeSemaphoreChanged>(proxy, reqId); UNIT_ASSERT_VALUES_EQUAL(event->Record.GetProxyGeneration(), generation); @@ -801,10 +801,10 @@ TTestContext::TResourceConsumingInfo::TResourceConsumingInfo(ui64 id, bool consu { } -NKikimrKesus::TEvSubscribeOnResourcesResult TTestContext::SubscribeOnResources(const TActorId& client, const TActorId& edge, const std::vector<TResourceConsumingInfo>& info) { +NKikimrKesus::TEvSubscribeOnResourcesResult TTestContext::SubscribeOnResources(const TActorId& client, const TActorId& edge, const std::vector<TResourceConsumingInfo>& info) { const ui64 cookie = RandomNumber<ui64>(); auto req = MakeHolder<TEvKesus::TEvSubscribeOnResources>(); - ActorIdToProto(client, req->Record.MutableActorID()); + ActorIdToProto(client, req->Record.MutableActorID()); req->Record.MutableResources()->Reserve(info.size()); for (const TResourceConsumingInfo& res : info) { auto* reqRes = req->Record.AddResources(); @@ -829,14 +829,14 @@ NKikimrKesus::TEvSubscribeOnResourcesResult TTestContext::SubscribeOnResources(c return result->Record; } -NKikimrKesus::TEvSubscribeOnResourcesResult TTestContext::SubscribeOnResource(const TActorId& client, const TActorId& edge, const TString& path, bool startConsuming, double amount, Ydb::StatusIds::StatusCode status) { +NKikimrKesus::TEvSubscribeOnResourcesResult TTestContext::SubscribeOnResource(const TActorId& client, const TActorId& edge, const TString& path, bool startConsuming, double amount, Ydb::StatusIds::StatusCode status) { return SubscribeOnResources(client, edge, {TResourceConsumingInfo(path, startConsuming, amount, status)}); } -void TTestContext::UpdateConsumptionState(const TActorId& client, const TActorId& edge, const std::vector<TResourceConsumingInfo>& info) { +void TTestContext::UpdateConsumptionState(const TActorId& client, const TActorId& edge, const std::vector<TResourceConsumingInfo>& info) { const ui64 cookie = RandomNumber<ui64>(); auto req = MakeHolder<TEvKesus::TEvUpdateConsumptionState>(); - ActorIdToProto(client, req->Record.MutableActorID()); + ActorIdToProto(client, req->Record.MutableActorID()); req->Record.MutableResourcesInfo()->Reserve(info.size()); for (const TResourceConsumingInfo& res : info) { auto* reqRes = req->Record.AddResourcesInfo(); @@ -850,7 +850,7 @@ void TTestContext::UpdateConsumptionState(const TActorId& client, const TActorId ExpectEdgeEvent<TEvKesus::TEvUpdateConsumptionStateAck>(edge, cookie); } -void TTestContext::UpdateConsumptionState(const TActorId& client, const TActorId& edge, ui64 id, bool consume, double amount, Ydb::StatusIds::StatusCode status) { +void TTestContext::UpdateConsumptionState(const TActorId& client, const TActorId& edge, ui64 id, bool consume, double amount, Ydb::StatusIds::StatusCode status) { UpdateConsumptionState(client, edge, {TResourceConsumingInfo(id, consume, amount, status)}); } diff --git a/ydb/core/kesus/tablet/ut_helpers.h b/ydb/core/kesus/tablet/ut_helpers.h index 2719a268418..8f487a30332 100644 --- a/ydb/core/kesus/tablet/ut_helpers.h +++ b/ydb/core/kesus/tablet/ut_helpers.h @@ -21,7 +21,7 @@ struct TTestContext { TTabletTypes::EType TabletType; ui64 TabletId; THolder<TTestActorRuntime> Runtime; - THashMap<std::tuple<TActorId, ui64>, TActorId> ProxyClients; + THashMap<std::tuple<TActorId, ui64>, TActorId> ProxyClients; TTestContext(); @@ -38,24 +38,24 @@ struct TTestContext { void RebootTablet(); // Returns tablet actor for direct manipulation - TActorId GetTabletActorId(); + TActorId GetTabletActorId(); // Extremely pedantic version of GrabEdgeEvent template<class TEvent> - THolder<TEvent> ExpectEdgeEvent(const TActorId& actor) { + THolder<TEvent> ExpectEdgeEvent(const TActorId& actor) { return Runtime->GrabEdgeEvent<TEvent>(actor)->Release(); } // Extremely pedantic version of GrabEdgeEvent template<class TEvent> - THolder<TEvent> ExpectEdgeEvent(const TActorId& actor, ui64 cookie) { + THolder<TEvent> ExpectEdgeEvent(const TActorId& actor, ui64 cookie) { auto ev = Runtime->GrabEdgeEvent<TEvent>(actor); UNIT_ASSERT_VALUES_EQUAL(ev->Cookie, cookie); return ev->Release(); } template<class TEvent> - void ExpectNoEdgeEvent(const TActorId& actor, TDuration duration) { + void ExpectNoEdgeEvent(const TActorId& actor, TDuration duration) { auto edge = Runtime->AllocateEdgeActor(); Runtime->EnableScheduleForActor(edge); Runtime->Schedule(new IEventHandle(edge, edge, new TEvent()), duration); @@ -64,72 +64,72 @@ struct TTestContext { } // Sends payload to tablet from edge, fresh pipe every time - void SendFromEdge(const TActorId& edge, IEventBase* payload, ui64 cookie = 0); + void SendFromEdge(const TActorId& edge, IEventBase* payload, ui64 cookie = 0); template <class TEvent> - void SendFromEdge(const TActorId& edge, THolder<TEvent> payload, ui64 cookie = 0) { + void SendFromEdge(const TActorId& edge, THolder<TEvent> payload, ui64 cookie = 0) { SendFromEdge(edge, payload.Release(), cookie); } // Sends payload to tablet from proxy (caches pipe per proxy/generation pair) - void SendFromProxy(const TActorId& proxy, ui64 generation, IEventBase* payload, ui64 cookie = 0); + void SendFromProxy(const TActorId& proxy, ui64 generation, IEventBase* payload, ui64 cookie = 0); // set/get config requests NKikimrKesus::TEvGetConfigResult GetConfig(); NKikimrKesus::TEvSetConfigResult SetConfig(ui64 txId, const Ydb::Coordination::Config& config, ui64 version, Ydb::StatusIds::StatusCode status = Ydb::StatusIds::SUCCESS); // Makes a dummy request using this proxy/generation pair - void SyncProxy(const TActorId& proxy, ui64 generation, bool useTransactions = false); + void SyncProxy(const TActorId& proxy, ui64 generation, bool useTransactions = false); // Registers a proxy/generation pair with the tablet - NKikimrKesus::TEvRegisterProxyResult RegisterProxy(const TActorId& proxy, ui64 generation); - void MustRegisterProxy(const TActorId& proxy, ui64 generation, Ydb::StatusIds::StatusCode status = Ydb::StatusIds::SUCCESS); + NKikimrKesus::TEvRegisterProxyResult RegisterProxy(const TActorId& proxy, ui64 generation); + void MustRegisterProxy(const TActorId& proxy, ui64 generation, Ydb::StatusIds::StatusCode status = Ydb::StatusIds::SUCCESS); // Unregisters a proxy/generation pair from the tablet - NKikimrKesus::TEvUnregisterProxyResult UnregisterProxy(const TActorId& proxy, ui64 generation); - void MustUnregisterProxy(const TActorId& proxy, ui64 generation, Ydb::StatusIds::StatusCode status = Ydb::StatusIds::SUCCESS); + NKikimrKesus::TEvUnregisterProxyResult UnregisterProxy(const TActorId& proxy, ui64 generation); + void MustUnregisterProxy(const TActorId& proxy, ui64 generation, Ydb::StatusIds::StatusCode status = Ydb::StatusIds::SUCCESS); // Session attachment helpers void SendAttachSession( - ui64 cookie, const TActorId& proxy, ui64 generation, ui64 sessionId, + ui64 cookie, const TActorId& proxy, ui64 generation, ui64 sessionId, ui64 timeoutMillis = 0, const TString& description = TString(), ui64 seqNo = 0, const TString& key = TString()); NKikimrKesus::TEvAttachSessionResult NextAttachSessionResult( - ui64 cookie, const TActorId& proxy, ui64 generation); + ui64 cookie, const TActorId& proxy, ui64 generation); ui64 ExpectAttachSessionResult( - ui64 cookie, const TActorId& proxy, ui64 generation, Ydb::StatusIds::StatusCode status = Ydb::StatusIds::SUCCESS); + ui64 cookie, const TActorId& proxy, ui64 generation, Ydb::StatusIds::StatusCode status = Ydb::StatusIds::SUCCESS); NKikimrKesus::TEvAttachSessionResult AttachSession( - const TActorId& proxy, ui64 generation, ui64 sessionId, + const TActorId& proxy, ui64 generation, ui64 sessionId, ui64 timeoutMillis = 0, const TString& description = TString(), ui64 seqNo = 0, const TString& key = TString()); ui64 MustAttachSession( - const TActorId& proxy, ui64 generation, ui64 sessionId, + const TActorId& proxy, ui64 generation, ui64 sessionId, ui64 timeoutMillis = 0, const TString& description = TString(), ui64 seqNo = 0, const TString& key = TString()); // Session detachment helpers - NKikimrKesus::TEvDetachSessionResult DetachSession(const TActorId& proxy, ui64 generation, ui64 sessionId); - void MustDetachSession(const TActorId& proxy, ui64 generation, ui64 sessionId, Ydb::StatusIds::StatusCode status = Ydb::StatusIds::SUCCESS); + NKikimrKesus::TEvDetachSessionResult DetachSession(const TActorId& proxy, ui64 generation, ui64 sessionId); + void MustDetachSession(const TActorId& proxy, ui64 generation, ui64 sessionId, Ydb::StatusIds::StatusCode status = Ydb::StatusIds::SUCCESS); // Session destruction helpers - NKikimrKesus::TEvDestroySessionResult DestroySession(const TActorId& proxy, ui64 generation, ui64 sessionId); - void MustDestroySession(const TActorId& proxy, ui64 generation, ui64 sessionId, Ydb::StatusIds::StatusCode status = Ydb::StatusIds::SUCCESS); + NKikimrKesus::TEvDestroySessionResult DestroySession(const TActorId& proxy, ui64 generation, ui64 sessionId); + void MustDestroySession(const TActorId& proxy, ui64 generation, ui64 sessionId, Ydb::StatusIds::StatusCode status = Ydb::StatusIds::SUCCESS); // Lock acquire helpers void SendAcquireLock( - ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, + ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& lockName, ELockMode mode, ui64 timeoutMillis = 0, const TString& data = TString()); - bool ExpectAcquireLockResult(ui64 reqId, const TActorId& proxy, ui64 generation, + bool ExpectAcquireLockResult(ui64 reqId, const TActorId& proxy, ui64 generation, Ydb::StatusIds::StatusCode status); - void ExpectAcquireLockResult(ui64 reqId, const TActorId& proxy, ui64 generation, bool acquired = true); + void ExpectAcquireLockResult(ui64 reqId, const TActorId& proxy, ui64 generation, bool acquired = true); // Lock release helpers bool MustReleaseLock( - ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& lockName, + ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& lockName, Ydb::StatusIds::StatusCode status); void MustReleaseLock( - ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& lockName, bool released = true); + ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& lockName, bool released = true); // Semaphore helpers void CreateSemaphore( @@ -147,33 +147,33 @@ struct TTestContext { } void SessionCreateSemaphore( - ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, + ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& name, ui64 limit, const TString& data, Ydb::StatusIds::StatusCode status = Ydb::StatusIds::SUCCESS); void SessionUpdateSemaphore( - ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, + ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& name, const TString& data, Ydb::StatusIds::StatusCode status = Ydb::StatusIds::SUCCESS); void SessionDeleteSemaphore( - ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, + ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& name, Ydb::StatusIds::StatusCode status = Ydb::StatusIds::SUCCESS); // Semaphore acquire helpers void SendAcquireSemaphore( - ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, + ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& name, ui64 count, ui64 timeoutMillis = 0, const TString& data = TString()); - bool ExpectAcquireSemaphoreResult(ui64 reqId, const TActorId& proxy, ui64 generation, + bool ExpectAcquireSemaphoreResult(ui64 reqId, const TActorId& proxy, ui64 generation, Ydb::StatusIds::StatusCode status); - void ExpectAcquireSemaphoreResult(ui64 reqId, const TActorId& proxy, ui64 generation, bool acquired = true); - void ExpectAcquireSemaphorePending(ui64 reqId, const TActorId& proxy, ui64 generation); + void ExpectAcquireSemaphoreResult(ui64 reqId, const TActorId& proxy, ui64 generation, bool acquired = true); + void ExpectAcquireSemaphorePending(ui64 reqId, const TActorId& proxy, ui64 generation); // Semaphore release helpers bool MustReleaseSemaphore( - ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& name, + ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& name, Ydb::StatusIds::StatusCode status); void MustReleaseSemaphore( - ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& name, bool released = true); + ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& name, bool released = true); // Describes and verifies info about proxies from the tablet @@ -182,17 +182,17 @@ struct TTestContext { THashSet<ui64> AttachedSessions; }; - THashMap<TActorId, TSimpleProxyInfo> DescribeProxies(); - void VerifyProxyRegistered(const TActorId& proxy, ui64 generation); - void VerifyProxyNotRegistered(const TActorId& proxy); - void VerifyProxyHasSessions(const TActorId& proxy, ui64 generation, const THashSet<ui64>& expectedSessions); + THashMap<TActorId, TSimpleProxyInfo> DescribeProxies(); + void VerifyProxyRegistered(const TActorId& proxy, ui64 generation); + void VerifyProxyNotRegistered(const TActorId& proxy); + void VerifyProxyHasSessions(const TActorId& proxy, ui64 generation, const THashSet<ui64>& expectedSessions); // Describes and verifies info about sessions on the tablet struct TSimpleSessionInfo { ui64 TimeoutMillis; TString Description; - TActorId OwnerProxy; + TActorId OwnerProxy; }; THashMap<ui64, TSimpleSessionInfo> DescribeSessions(); @@ -235,12 +235,12 @@ struct TTestContext { }; void SendSessionDescribeSemaphore( - ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, + ui64 reqId, const TActorId& proxy, ui64 generation, ui64 sessionId, const TString& name, bool watchData = false, bool watchOwners = false); TSimpleSemaphoreDescription ExpectDescribeSemaphoreResult( - ui64 reqId, const TActorId& proxy, ui64 generation, + ui64 reqId, const TActorId& proxy, ui64 generation, Ydb::StatusIds::StatusCode status = Ydb::StatusIds::SUCCESS); - TDescribeSemaphoreChanges ExpectDescribeSemaphoreChanged(ui64 reqId, const TActorId& proxy, ui64 generation); + TDescribeSemaphoreChanges ExpectDescribeSemaphoreChanged(ui64 reqId, const TActorId& proxy, ui64 generation); // Quoter THolder<TEvKesus::TEvDescribeQuoterResourcesResult> VerifyDescribeQuoterResources( @@ -305,11 +305,11 @@ struct TTestContext { {} }; - NKikimrKesus::TEvSubscribeOnResourcesResult SubscribeOnResources(const TActorId& client, const TActorId& edge, const std::vector<TResourceConsumingInfo>& info); - NKikimrKesus::TEvSubscribeOnResourcesResult SubscribeOnResource(const TActorId& client, const TActorId& edge, const TString& path, bool startConsuming, double amount = 0.0, Ydb::StatusIds::StatusCode status = Ydb::StatusIds::SUCCESS); + NKikimrKesus::TEvSubscribeOnResourcesResult SubscribeOnResources(const TActorId& client, const TActorId& edge, const std::vector<TResourceConsumingInfo>& info); + NKikimrKesus::TEvSubscribeOnResourcesResult SubscribeOnResource(const TActorId& client, const TActorId& edge, const TString& path, bool startConsuming, double amount = 0.0, Ydb::StatusIds::StatusCode status = Ydb::StatusIds::SUCCESS); - void UpdateConsumptionState(const TActorId& client, const TActorId& edge, const std::vector<TResourceConsumingInfo>& info); - void UpdateConsumptionState(const TActorId& client, const TActorId& edge, ui64 id, bool consume, double amount = 0.0, Ydb::StatusIds::StatusCode status = Ydb::StatusIds::SUCCESS); + void UpdateConsumptionState(const TActorId& client, const TActorId& edge, const std::vector<TResourceConsumingInfo>& info); + void UpdateConsumptionState(const TActorId& client, const TActorId& edge, ui64 id, bool consume, double amount = 0.0, Ydb::StatusIds::StatusCode status = Ydb::StatusIds::SUCCESS); void AccountResources(const TActorId& client, const TActorId& edge, const std::vector<TResourceAccountInfo>& info); void AccountResources(const TActorId& client, const TActorId& edge, ui64 id, TInstant start, TDuration interval, std::vector<double>&& amount); diff --git a/ydb/core/keyvalue/channel_balancer.h b/ydb/core/keyvalue/channel_balancer.h index a8a39ebda73..36c3bea5ab2 100644 --- a/ydb/core/keyvalue/channel_balancer.h +++ b/ydb/core/keyvalue/channel_balancer.h @@ -132,14 +132,14 @@ namespace NKikimr::NKeyValue { }; TVector<TChannelInfo> ChannelInfo; - const TActorId ActorId; + const TActorId ActorId; public: static constexpr NKikimrServices::TActivity::EType ActorActivityType() { return NKikimrServices::TActivity::KEYVALUE_ACTOR; } - TChannelBalancer(ui8 numChannels, TActorId actorId) + TChannelBalancer(ui8 numChannels, TActorId actorId) : ChannelInfo(numChannels) , ActorId(actorId) {} diff --git a/ydb/core/keyvalue/keyvalue.cpp b/ydb/core/keyvalue/keyvalue.cpp index 4812ed4eaed..4fd3f98d3a5 100644 --- a/ydb/core/keyvalue/keyvalue.cpp +++ b/ydb/core/keyvalue/keyvalue.cpp @@ -2,7 +2,7 @@ namespace NKikimr { -IActor* CreateKeyValueFlat(const TActorId &tablet, TTabletStorageInfo *info) { +IActor* CreateKeyValueFlat(const TActorId &tablet, TTabletStorageInfo *info) { return new NKeyValue::TKeyValueFlat(tablet, info); } diff --git a/ydb/core/keyvalue/keyvalue.h b/ydb/core/keyvalue/keyvalue.h index cb2ed96cde3..fa3bc719178 100644 --- a/ydb/core/keyvalue/keyvalue.h +++ b/ydb/core/keyvalue/keyvalue.h @@ -8,6 +8,6 @@ namespace NKikimr { -IActor* CreateKeyValueFlat(const TActorId &tablet, TTabletStorageInfo *info); +IActor* CreateKeyValueFlat(const TActorId &tablet, TTabletStorageInfo *info); } //NKikimr diff --git a/ydb/core/keyvalue/keyvalue_collector.cpp b/ydb/core/keyvalue/keyvalue_collector.cpp index 0b97f364d1c..8b7696bcb25 100644 --- a/ydb/core/keyvalue/keyvalue_collector.cpp +++ b/ydb/core/keyvalue/keyvalue_collector.cpp @@ -15,7 +15,7 @@ struct TGroupCollector { }; class TKeyValueCollector : public TActorBootstrapped<TKeyValueCollector> { - TActorId KeyValueActorId; + TActorId KeyValueActorId; TIntrusivePtr<TCollectOperation> CollectOperation; TIntrusivePtr<TTabletStorageInfo> TabletInfo; ui32 RecordGeneration; @@ -30,7 +30,7 @@ public: return NKikimrServices::TActivity::KEYVALUE_ACTOR; } - TKeyValueCollector(const TActorId &keyValueActorId, TIntrusivePtr<TCollectOperation> &collectOperation, + TKeyValueCollector(const TActorId &keyValueActorId, TIntrusivePtr<TCollectOperation> &collectOperation, const TTabletStorageInfo *tabletInfo, ui32 recordGeneration, ui32 perGenerationCounter, bool isSpringCleanup) : KeyValueActorId(keyValueActorId) @@ -191,7 +191,7 @@ public: } }; -IActor* CreateKeyValueCollector(const TActorId &keyValueActorId, TIntrusivePtr<TCollectOperation> &collectOperation, +IActor* CreateKeyValueCollector(const TActorId &keyValueActorId, TIntrusivePtr<TCollectOperation> &collectOperation, const TTabletStorageInfo *tabletInfo, ui32 recordGeneration, ui32 perGenerationCounter, bool isSpringCleanup) { return new TKeyValueCollector(keyValueActorId, collectOperation, tabletInfo, recordGeneration, perGenerationCounter, isSpringCleanup); diff --git a/ydb/core/keyvalue/keyvalue_collector.h b/ydb/core/keyvalue/keyvalue_collector.h index ad5c2d6bd6c..9e4d848ab7d 100644 --- a/ydb/core/keyvalue/keyvalue_collector.h +++ b/ydb/core/keyvalue/keyvalue_collector.h @@ -5,7 +5,7 @@ namespace NKikimr { namespace NKeyValue { -IActor* CreateKeyValueCollector(const TActorId &keyValueActorId, TIntrusivePtr<TCollectOperation> &collectOperation, +IActor* CreateKeyValueCollector(const TActorId &keyValueActorId, TIntrusivePtr<TCollectOperation> &collectOperation, const TTabletStorageInfo *tabletInfo, ui32 recordGeneration, ui32 perGenerationCounter, bool isSpringCleanup); } // NKeyValue diff --git a/ydb/core/keyvalue/keyvalue_collector_ut.cpp b/ydb/core/keyvalue/keyvalue_collector_ut.cpp index a752d5ce149..4456a823803 100644 --- a/ydb/core/keyvalue/keyvalue_collector_ut.cpp +++ b/ydb/core/keyvalue/keyvalue_collector_ut.cpp @@ -21,9 +21,9 @@ class TContext { const ui32 NodeIndex = 0; THolder<TTestActorRuntime> Runtime; TIntrusivePtr<TTabletStorageInfo> TabletInfo; - TActorId CollectorId; - TActorId TabletActorId; - TActorId Sender; + TActorId CollectorId; + TActorId TabletActorId; + TActorId Sender; public: void SetActor(IActor *actor) { @@ -39,8 +39,8 @@ public: Sender = Runtime->AllocateEdgeActor(NodeIndex); TabletActorId = Runtime->AllocateEdgeActor(NodeIndex); for (ui32 groupId = 0; groupId < 6; ++groupId) { - const TActorId actorId = Runtime->AllocateEdgeActor(NodeIndex); - const TActorId proxyId = MakeBlobStorageProxyID(groupId); + const TActorId actorId = Runtime->AllocateEdgeActor(NodeIndex); + const TActorId proxyId = MakeBlobStorageProxyID(groupId); Runtime->RegisterService(proxyId, actorId, NodeIndex); } } @@ -79,7 +79,7 @@ public: return (channel == 3 ? 3 : 2); } - TActorId GetProxyActorId(ui32 channel, ui32 generation) { + TActorId GetProxyActorId(ui32 channel, ui32 generation) { ui32 groupId = GetGroupId(channel, generation); return MakeBlobStorageProxyID(groupId); } @@ -88,7 +88,7 @@ public: Runtime->Send(new IEventHandle(CollectorId, Sender, ev)); } - TActorId GetTabletActorId() { + TActorId GetTabletActorId() { return TabletActorId; } diff --git a/ydb/core/keyvalue/keyvalue_flat_impl.h b/ydb/core/keyvalue/keyvalue_flat_impl.h index 0eb3e0eb55c..292f2feff93 100644 --- a/ydb/core/keyvalue/keyvalue_flat_impl.h +++ b/ydb/core/keyvalue/keyvalue_flat_impl.h @@ -16,7 +16,7 @@ #include <library/cpp/actors/core/hfunc.h> #include <library/cpp/actors/core/log.h> -#include <library/cpp/json/json_writer.h> +#include <library/cpp/json/json_writer.h> #include <ydb/core/base/appdata.h> #include <ydb/core/base/blobstorage.h> #include <ydb/core/base/tablet_resolver.h> @@ -40,10 +40,10 @@ constexpr ui64 PeriodicRefreshMs = 15000; class TKeyValueFlat : public TActor<TKeyValueFlat>, public NTabletFlatExecutor::TTabletExecutedFlat { protected: struct TTxInit : public NTabletFlatExecutor::ITransaction { - TActorId KeyValueActorId; + TActorId KeyValueActorId; TKeyValueFlat &Self; - TTxInit(TActorId keyValueActorId, TKeyValueFlat &keyValueFlat) + TTxInit(TActorId keyValueActorId, TKeyValueFlat &keyValueFlat) : KeyValueActorId(keyValueActorId) , Self(keyValueFlat) {} @@ -167,10 +167,10 @@ protected: struct TTxMonitoring : public NTabletFlatExecutor::ITransaction { const THolder<NMon::TEvRemoteHttpInfo> Event; - const TActorId RespondTo; + const TActorId RespondTo; TKeyValueFlat *Self; - TTxMonitoring(THolder<NMon::TEvRemoteHttpInfo> event, const TActorId &respondTo, TKeyValueFlat *keyValue) + TTxMonitoring(THolder<NMon::TEvRemoteHttpInfo> event, const TActorId &respondTo, TKeyValueFlat *keyValue) : Event(std::move(event)) , RespondTo(respondTo) , Self(keyValue) @@ -248,7 +248,7 @@ protected: TKeyValueState State; TDeque<TAutoPtr<IEventHandle>> InitialEventsQueue; - TActorId CollectorActorId; + TActorId CollectorActorId; void OnDetach(const TActorContext &ctx) override { LOG_DEBUG_S(ctx, NKikimrServices::KEYVALUE, "KeyValue# " << TabletID() << " OnDetach"); @@ -420,7 +420,7 @@ public: return NKikimrServices::TActivity::KEYVALUE_ACTOR; } - TKeyValueFlat(const TActorId &tablet, TTabletStorageInfo *info) + TKeyValueFlat(const TActorId &tablet, TTabletStorageInfo *info) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, new NMiniKQL::TMiniKQLFactory) { diff --git a/ydb/core/keyvalue/keyvalue_intermediate.cpp b/ydb/core/keyvalue/keyvalue_intermediate.cpp index 4b67ef6dc76..82a78d9fa29 100644 --- a/ydb/core/keyvalue/keyvalue_intermediate.cpp +++ b/ydb/core/keyvalue/keyvalue_intermediate.cpp @@ -51,7 +51,7 @@ NKikimrProto::EReplyStatus TIntermediate::TRead::CumulativeStatus() const { } } -TIntermediate::TIntermediate(TActorId respondTo, TActorId keyValueActorId, ui64 channelGeneration, ui64 channelStep, +TIntermediate::TIntermediate(TActorId respondTo, TActorId keyValueActorId, ui64 channelGeneration, ui64 channelStep, TRequestType::EType requestType) : Cookie(0) , Generation(0) diff --git a/ydb/core/keyvalue/keyvalue_intermediate.h b/ydb/core/keyvalue/keyvalue_intermediate.h index bc6f97d4e82..c28c14b9c37 100644 --- a/ydb/core/keyvalue/keyvalue_intermediate.h +++ b/ydb/core/keyvalue/keyvalue_intermediate.h @@ -132,8 +132,8 @@ struct TIntermediate { bool HasGeneration; bool HasIncrementGeneration; - TActorId RespondTo; - TActorId KeyValueActorId; + TActorId RespondTo; + TActorId KeyValueActorId; ui64 TotalSize; ui64 TotalSizeLimit; @@ -157,7 +157,7 @@ struct TIntermediate { ui32 EvType = 0; - TIntermediate(TActorId respondTo, TActorId keyValueActorId, ui64 channelGeneration, ui64 channelStep, + TIntermediate(TActorId respondTo, TActorId keyValueActorId, ui64 channelGeneration, ui64 channelStep, TRequestType::EType requestType); void UpdateStat(); diff --git a/ydb/core/keyvalue/keyvalue_state.cpp b/ydb/core/keyvalue/keyvalue_state.cpp index 38945a8bcf9..6b31c463f83 100644 --- a/ydb/core/keyvalue/keyvalue_state.cpp +++ b/ydb/core/keyvalue/keyvalue_state.cpp @@ -11,8 +11,8 @@ #include <ydb/core/tablet/tablet_counters_protobuf.h> #include <ydb/core/tablet/tablet_metrics.h> #include <ydb/core/util/stlog.h> -#include <library/cpp/monlib/service/pages/templates.h> -#include <library/cpp/json/writer/json_value.h> +#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/json/writer/json_value.h> #include <util/string/escape.h> #include <util/charset/utf8.h> @@ -90,7 +90,7 @@ void TKeyValueState::Clear() { UsedChannels.reset(); TabletId = 0; - KeyValueActorId = TActorId(); + KeyValueActorId = TActorId(); ExecutorGeneration = 0; Queue.clear(); @@ -480,7 +480,7 @@ void TKeyValueState::Load(const TString &key, const TString& value) { } } -void TKeyValueState::InitExecute(ui64 tabletId, TActorId keyValueActorId, ui32 executorGeneration, +void TKeyValueState::InitExecute(ui64 tabletId, TActorId keyValueActorId, ui32 executorGeneration, ISimpleDb &db, const TActorContext &ctx, const TTabletStorageInfo *info) { Y_UNUSED(info); Y_VERIFY(IsEmptyDbStart || IsStatePresent); @@ -560,9 +560,9 @@ void TKeyValueState::InitExecute(ui64 tabletId, TActorId keyValueActorId, ui32 e PerGenerationCounter, channel, generation, step, TInstant::Max()); ++PerGenerationCounter; - const TActorId nodeWarden = MakeBlobStorageNodeWardenID(ctx.SelfID.NodeId()); - const TActorId proxy = MakeBlobStorageProxyID(group); - ctx.ExecutorThread.Send(new IEventHandle(proxy, TActorId(), ev.Release(), + const TActorId nodeWarden = MakeBlobStorageNodeWardenID(ctx.SelfID.NodeId()); + const TActorId proxy = MakeBlobStorageProxyID(group); + ctx.ExecutorThread.Send(new IEventHandle(proxy, TActorId(), ev.Release(), IEventHandle::FlagForwardOnNondelivery, 0, &nodeWarden)); } } @@ -625,8 +625,8 @@ void TKeyValueState::InitExecute(ui64 tabletId, TActorId keyValueActorId, ui32 e nullptr /*doNotKeep*/, TInstant::Max(), false /*isMultiCollectAllowed*/, false /*hard*/); ++PerGenerationCounter; - const TActorId nodeWarden = MakeBlobStorageNodeWardenID(ctx.SelfID.NodeId()); - const TActorId proxy = MakeBlobStorageProxyID(group); + const TActorId nodeWarden = MakeBlobStorageNodeWardenID(ctx.SelfID.NodeId()); + const TActorId proxy = MakeBlobStorageProxyID(group); ctx.ExecutorThread.Send(new IEventHandle(proxy, KeyValueActorId, ev.Release(), IEventHandle::FlagForwardOnNondelivery, 0, &nodeWarden)); } @@ -733,7 +733,7 @@ void TKeyValueState::SendCutHistory(const TActorContext &ctx) { record.SetTabletID(TabletId); record.SetChannel(channel); record.SetFromGeneration(fromGeneration); - TActorId localActorId = MakeLocalID(ctx.SelfID.NodeId()); + TActorId localActorId = MakeLocalID(ctx.SelfID.NodeId()); ctx.Send(localActorId, ev.Release()); } } diff --git a/ydb/core/keyvalue/keyvalue_state.h b/ydb/core/keyvalue/keyvalue_state.h index 7a339a1c1e0..f57a69d35d3 100644 --- a/ydb/core/keyvalue/keyvalue_state.h +++ b/ydb/core/keyvalue/keyvalue_state.h @@ -258,14 +258,14 @@ protected: THolder<TChannelBalancer::TWeightManager> WeightManager; ui64 TabletId; - TActorId KeyValueActorId; + TActorId KeyValueActorId; ui32 ExecutorGeneration; bool IsStatePresent; bool IsEmptyDbStart; bool IsDamaged; bool IsTabletYellowMove; bool IsTabletYellowStop; - TActorId ChannelBalancerActorId; + TActorId ChannelBalancerActorId; ui64 InitialCollectsSent = 0; TDeque<TAutoPtr<TIntermediate>> Queue; @@ -310,7 +310,7 @@ public: void Terminate(const TActorContext& ctx); void Load(const TString &key, const TString& value); - void InitExecute(ui64 tabletId, TActorId keyValueActorId, ui32 executorGeneration, ISimpleDb &db, + void InitExecute(ui64 tabletId, TActorId keyValueActorId, ui32 executorGeneration, ISimpleDb &db, const TActorContext &ctx, const TTabletStorageInfo *info); void RegisterInitialCollectResult(const TActorContext &ctx); void SendCutHistory(const TActorContext &ctx); diff --git a/ydb/core/keyvalue/keyvalue_storage_request.cpp b/ydb/core/keyvalue/keyvalue_storage_request.cpp index cadeda623b7..e9032d2ed3d 100644 --- a/ydb/core/keyvalue/keyvalue_storage_request.cpp +++ b/ydb/core/keyvalue/keyvalue_storage_request.cpp @@ -376,7 +376,7 @@ public: IntermediateResults->Stat.YellowMoveChannels.reserve(YellowMoveChannels.size()); IntermediateResults->Stat.YellowMoveChannels.insert(IntermediateResults->Stat.YellowMoveChannels.end(), YellowMoveChannels.begin(), YellowMoveChannels.end()); - TActorId keyValueActorId = IntermediateResults->KeyValueActorId; + TActorId keyValueActorId = IntermediateResults->KeyValueActorId; ctx.Send(keyValueActorId, new TEvKeyValue::TEvIntermediate(std::move(IntermediateResults))); Die(ctx); return true; @@ -479,7 +479,7 @@ public: YellowMoveChannels.begin(), YellowMoveChannels.end()); IntermediateResults->UpdateStat(); - TActorId keyValueActorId = IntermediateResults->KeyValueActorId; + TActorId keyValueActorId = IntermediateResults->KeyValueActorId; ctx.Send(keyValueActorId, new TEvKeyValue::TEvNotify( IntermediateResults->RequestUid, IntermediateResults->CreatedAtGeneration, IntermediateResults->CreatedAtStep, diff --git a/ydb/core/keyvalue/keyvalue_ut.cpp b/ydb/core/keyvalue/keyvalue_ut.cpp index 1e78c7b4890..4ae1fcf56a6 100644 --- a/ydb/core/keyvalue/keyvalue_ut.cpp +++ b/ydb/core/keyvalue/keyvalue_ut.cpp @@ -65,7 +65,7 @@ struct TTestContext { TInitialEventsFilter InitialEventsFilter; TVector<ui64> TabletIds; THolder<TTestActorRuntime> Runtime; - TActorId Edge; + TActorId Edge; TTestContext() { TabletType = TTabletTypes::KEYVALUEFLAT; diff --git a/ydb/core/kqp/common/kqp_gateway.h b/ydb/core/kqp/common/kqp_gateway.h index 383dde225c0..1e06beca393 100644 --- a/ydb/core/kqp/common/kqp_gateway.h +++ b/ydb/core/kqp/common/kqp_gateway.h @@ -153,10 +153,10 @@ public: /* Physical */ virtual NThreading::TFuture<TExecPhysicalResult> ExecutePhysical(TExecPhysicalRequest&& request, - const NActors::TActorId& target) = 0; + const NActors::TActorId& target) = 0; virtual NThreading::TFuture<TExecPhysicalResult> ExecuteScanQuery(TExecPhysicalRequest&& request, - const NActors::TActorId& target) = 0; + const NActors::TActorId& target) = 0; virtual NThreading::TFuture<TExecPhysicalResult> ExecutePure(TExecPhysicalRequest&& request, const NActors::TActorId& target) = 0; diff --git a/ydb/core/kqp/executer/kqp_planner.h b/ydb/core/kqp/executer/kqp_planner.h index 6c38963c7af..dc5e993a74d 100644 --- a/ydb/core/kqp/executer/kqp_planner.h +++ b/ydb/core/kqp/executer/kqp_planner.h @@ -56,7 +56,7 @@ private: private: const ui64 TxId; - const TActorId ExecuterId; + const TActorId ExecuterId; TVector<NYql::NDqProto::TDqTask> Tasks; THashMap<ui64, TVector<NYql::NDqProto::TDqTask>> ScanTasks; const IKqpGateway::TKqpSnapshot Snapshot; @@ -71,7 +71,7 @@ private: THashSet<ui32> TrackingNodes; }; -IActor* CreateKqpPlanner(ui64 txId, const TActorId& executer, TVector<NYql::NDqProto::TDqTask>&& tasks, +IActor* CreateKqpPlanner(ui64 txId, const TActorId& executer, TVector<NYql::NDqProto::TDqTask>&& tasks, THashMap<ui64, TVector<NYql::NDqProto::TDqTask>>&& scanTasks, const IKqpGateway::TKqpSnapshot& snapshot, const TString& database, const TMaybe<TString>& userToken, TInstant deadline, const NYql::NDqProto::EDqStatsMode& statsMode, bool disableLlvmForUdfStages, bool enableLlvm, bool withSpilling, const TMaybe<NKikimrKqp::TRlPath>& rlPath); diff --git a/ydb/core/kqp/executer/kqp_planner_strategy.cpp b/ydb/core/kqp/executer/kqp_planner_strategy.cpp index 47dc9fc2a06..b0f323dbc24 100644 --- a/ydb/core/kqp/executer/kqp_planner_strategy.cpp +++ b/ydb/core/kqp/executer/kqp_planner_strategy.cpp @@ -28,7 +28,7 @@ public: struct TNodeDesc { ui32 NodeId = std::numeric_limits<ui32>::max(); - TActorId ResourceManagerId; + TActorId ResourceManagerId; ui64 RemainsMemory = 0; ui32 RemainsComputeActors = 0; TVector<ui64> Tasks; @@ -45,7 +45,7 @@ public: for (auto& node : nodeResources) { nodes.emplace(TNodeDesc{ node.GetNodeId(), - ActorIdFromProto(node.GetResourceManagerActorId()), + ActorIdFromProto(node.GetResourceManagerActorId()), node.GetTotalMemory() - node.GetUsedMemory(), node.GetAvailableComputeActors(), {} diff --git a/ydb/core/kqp/executer/kqp_planner_strategy.h b/ydb/core/kqp/executer/kqp_planner_strategy.h index 1dbb821b485..42df6703ab0 100644 --- a/ydb/core/kqp/executer/kqp_planner_strategy.h +++ b/ydb/core/kqp/executer/kqp_planner_strategy.h @@ -19,7 +19,7 @@ public: struct TResult { ui32 NodeId; - NActors::TActorId ResourceManager; + NActors::TActorId ResourceManager; TVector<ui64> TaskIds; }; diff --git a/ydb/core/kqp/executer/kqp_result_channel.cpp b/ydb/core/kqp/executer/kqp_result_channel.cpp index ca3258f2b41..0fe235323e3 100644 --- a/ydb/core/kqp/executer/kqp_result_channel.cpp +++ b/ydb/core/kqp/executer/kqp_result_channel.cpp @@ -113,8 +113,8 @@ private: const ui64 TxId; const ui64 ChannelId; TQueryExecutionStats* Stats; // owned by KqpExecuter - const NActors::TActorId Executer; - NActors::TActorId ComputeActor; + const NActors::TActorId Executer; + NActors::TActorId ComputeActor; }; class TResultStreamChannelProxy : public TResultCommonChannelProxy { diff --git a/ydb/core/kqp/executer/kqp_result_channel.h b/ydb/core/kqp/executer/kqp_result_channel.h index 28fd7579d9b..299b0f686bf 100644 --- a/ydb/core/kqp/executer/kqp_result_channel.h +++ b/ydb/core/kqp/executer/kqp_result_channel.h @@ -25,7 +25,7 @@ struct TQueryExecutionStats; struct TKqpExecuterTxResult; NActors::IActor* CreateResultStreamChannelProxy(ui64 txId, ui64 channelId, const NKikimrMiniKQL::TType& itemType, - const NKikimrMiniKQL::TType* resultItemType, NActors::TActorId target, TQueryExecutionStats* stats, + const NKikimrMiniKQL::TType* resultItemType, NActors::TActorId target, TQueryExecutionStats* stats, NActors::TActorId executer); NActors::IActor* CreateResultDataChannelProxy(ui64 txId, ui64 channelId, TQueryExecutionStats* stats, diff --git a/ydb/core/kqp/executer/kqp_scan_executer.cpp b/ydb/core/kqp/executer/kqp_scan_executer.cpp index 1d5f9ab8cba..07c29f64e21 100644 --- a/ydb/core/kqp/executer/kqp_scan_executer.cpp +++ b/ydb/core/kqp/executer/kqp_scan_executer.cpp @@ -172,7 +172,7 @@ private: } void HandleExecute(TEvDqCompute::TEvState::TPtr& ev) { - TActorId computeActor = ev->Sender; + TActorId computeActor = ev->Sender; auto& state = ev->Get()->Record; ui64 taskId = state.GetTaskId(); @@ -206,7 +206,7 @@ private: auto& task = TasksGraph.GetTask(taskId); task.ComputeActorId = computeActor; - THashMap<TActorId, THashSet<ui64>> updates; + THashMap<TActorId, THashSet<ui64>> updates; CollectTaskChannelsUpdates(task, updates); PropagateChannelsUpdates(updates); } else { @@ -768,7 +768,7 @@ private: NYql::NDqProto::TDqTask taskDesc; taskDesc.SetId(task.Id); - ActorIdToProto(SelfId(), taskDesc.MutableExecuter()->MutableActorId()); + ActorIdToProto(SelfId(), taskDesc.MutableExecuter()->MutableActorId()); for (auto& input : task.Inputs) { FillInputDesc(*taskDesc.AddInputs(), input); @@ -1032,7 +1032,7 @@ private: public: void FillEndpointDesc(NYql::NDqProto::TEndpoint& endpoint, const TTask& task) { if (task.ComputeActorId) { - ActorIdToProto(task.ComputeActorId, endpoint.MutableActorId()); + ActorIdToProto(task.ComputeActorId, endpoint.MutableActorId()); } } diff --git a/ydb/core/kqp/host/kqp_host.cpp b/ydb/core/kqp/host/kqp_host.cpp index 4ff3a454282..44af6d11682 100644 --- a/ydb/core/kqp/host/kqp_host.cpp +++ b/ydb/core/kqp/host/kqp_host.cpp @@ -83,7 +83,7 @@ private: struct TExecuteContext : TThrRefBase { TVector<IKqpHost::TQueryResult> QueryResults; IKikimrQueryExecutor::TExecuteSettings Settings; - NActors::TActorId ReplyTarget; + NActors::TActorId ReplyTarget; void Reset(const IKikimrQueryExecutor::TExecuteSettings& settings) { Settings = settings; @@ -1370,7 +1370,7 @@ public: } IAsyncQueryResultPtr ExecuteScanQuery(const TString& query, bool isSql, NKikimrMiniKQL::TParams&& parameters, - const NActors::TActorId& target, const IKikimrQueryExecutor::TExecuteSettings& settings) override + const NActors::TActorId& target, const IKikimrQueryExecutor::TExecuteSettings& settings) override { return CheckedProcessQuery(*ExprCtx, [this, &query, isSql, parameters = std::move(parameters), target, settings] (TExprContext& ctx) mutable { @@ -2063,7 +2063,7 @@ private: } IAsyncQueryResultPtr ExecuteScanQueryInternal(const TString& query, bool isSql, - NKikimrMiniKQL::TParams&& parameters, const NActors::TActorId& target, + NKikimrMiniKQL::TParams&& parameters, const NActors::TActorId& target, const IKikimrQueryExecutor::TExecuteSettings& settings, TExprContext& ctx) { auto prepareResult = isSql @@ -2096,7 +2096,7 @@ private: } IAsyncQueryResultPtr ExecuteScanQueryInternal(std::shared_ptr<const NKikimrKqp::TPreparedQuery>& query, - NKikimrMiniKQL::TParams&& parameters, const NActors::TActorId& target, + NKikimrMiniKQL::TParams&& parameters, const NActors::TActorId& target, const IKikimrQueryExecutor::TExecuteSettings& settings, const TIssues& issues, const TMaybe<TSqlVersion>& sqlVersion, TExprContext& ctx) { diff --git a/ydb/core/kqp/host/kqp_host.h b/ydb/core/kqp/host/kqp_host.h index d46c2869b44..11282d0ebe1 100644 --- a/ydb/core/kqp/host/kqp_host.h +++ b/ydb/core/kqp/host/kqp_host.h @@ -95,7 +95,7 @@ public: /* Scan queries */ virtual IAsyncQueryResultPtr ExecuteScanQuery(const TString& query, bool isSql, - NKikimrMiniKQL::TParams&& parameters, const NActors::TActorId& target, + NKikimrMiniKQL::TParams&& parameters, const NActors::TActorId& target, const NYql::IKikimrQueryExecutor::TExecuteSettings& settings) = 0; virtual IAsyncQueryResultPtr ExplainScanQuery(const TString& query, bool isSql) = 0; diff --git a/ydb/core/kqp/host/kqp_host_impl.h b/ydb/core/kqp/host/kqp_host_impl.h index 671fcd8550d..fb1fb139cd5 100644 --- a/ydb/core/kqp/host/kqp_host_impl.h +++ b/ydb/core/kqp/host/kqp_host_impl.h @@ -267,7 +267,7 @@ public: virtual TIntrusivePtr<TAsyncQueryResult> ExecutePreparedScanQuery(const TString& cluster, const NYql::TExprNode::TPtr& world, const NKqpProto::TKqpPhyQuery& phyQuery, NYql::TExprContext& ctx, - const NActors::TActorId& target) = 0; + const NActors::TActorId& target) = 0; }; TIntrusivePtr<IKqpRunner> CreateKqpRunner(TIntrusivePtr<IKqpGateway> gateway, const TString& cluster, diff --git a/ydb/core/kqp/host/kqp_runner.cpp b/ydb/core/kqp/host/kqp_runner.cpp index ee6ffba777a..42b4eca7c66 100644 --- a/ydb/core/kqp/host/kqp_runner.cpp +++ b/ydb/core/kqp/host/kqp_runner.cpp @@ -329,7 +329,7 @@ public: TIntrusivePtr<TAsyncQueryResult> ExecutePreparedScanQuery(const TString& cluster, const NYql::TExprNode::TPtr& world, const NKqpProto::TKqpPhyQuery& phyQuery, TExprContext& ctx, - const NActors::TActorId& target) override + const NActors::TActorId& target) override { YQL_ENSURE(cluster == Cluster); YQL_ENSURE(phyQuery.GetType() == NKqpProto::TKqpPhyQuery::TYPE_SCAN); @@ -590,7 +590,7 @@ private: } TIntrusivePtr<TAsyncQueryResult> ExecutePhysicalScanQuery(const TExprNode::TPtr& world, - const NKqpProto::TKqpPhyQuery& phyQuery, TExprContext& ctx, const NActors::TActorId& target) + const NKqpProto::TKqpPhyQuery& phyQuery, TExprContext& ctx, const NActors::TActorId& target) { ScanRunQueryTransformer->Rewind(); diff --git a/ydb/core/kqp/kqp.h b/ydb/core/kqp/kqp.h index e964c830d40..09f794591d3 100644 --- a/ydb/core/kqp/kqp.h +++ b/ydb/core/kqp/kqp.h @@ -18,19 +18,19 @@ namespace NKqp { const TStringBuf DefaultKikimrClusterName = "kikimr"; const TStringBuf DefaultKikimrPublicClusterName = "db"; -inline NActors::TActorId MakeKqpProxyID(ui32 nodeId) { +inline NActors::TActorId MakeKqpProxyID(ui32 nodeId) { const char name[12] = "kqp_proxy"; - return NActors::TActorId(nodeId, TStringBuf(name, 12)); + return NActors::TActorId(nodeId, TStringBuf(name, 12)); } -inline NActors::TActorId MakeKqpCompileServiceID(ui32 nodeId) { +inline NActors::TActorId MakeKqpCompileServiceID(ui32 nodeId) { const char name[12] = "kqp_compile"; - return NActors::TActorId(nodeId, TStringBuf(name, 12)); + return NActors::TActorId(nodeId, TStringBuf(name, 12)); } -inline NActors::TActorId MakeKqpResourceManagerServiceID(ui32 nodeId) { +inline NActors::TActorId MakeKqpResourceManagerServiceID(ui32 nodeId) { const char name[12] = "kqp_resman"; - return NActors::TActorId(nodeId, TStringBuf(name, 12)); + return NActors::TActorId(nodeId, TStringBuf(name, 12)); } inline NActors::TActorId MakeKqpRmServiceID(ui32 nodeId) { @@ -43,9 +43,9 @@ inline NActors::TActorId MakeKqpNodeServiceID(ui32 nodeId) { return NActors::TActorId(nodeId, TStringBuf(name, 12)); } -inline NActors::TActorId MakeKqpLocalFileSpillingServiceID(ui32 nodeId) { +inline NActors::TActorId MakeKqpLocalFileSpillingServiceID(ui32 nodeId) { const char name[12] = "kqp_lfspill"; - return NActors::TActorId(nodeId, TStringBuf(name, 12)); + return NActors::TActorId(nodeId, TStringBuf(name, 12)); } class TKqpShutdownController; diff --git a/ydb/core/kqp/kqp_compile_actor.cpp b/ydb/core/kqp/kqp_compile_actor.cpp index c8bf8119815..27cf296a03d 100644 --- a/ydb/core/kqp/kqp_compile_actor.cpp +++ b/ydb/core/kqp/kqp_compile_actor.cpp @@ -43,7 +43,7 @@ public: return NKikimrServices::TActivity::KQP_COMPILE_ACTOR; } - TKqpCompileActor(const TActorId& owner, const TKqpSettings::TConstPtr& kqpSettings, + TKqpCompileActor(const TActorId& owner, const TKqpSettings::TConstPtr& kqpSettings, const TTableServiceConfig& serviceConfig, TIntrusivePtr<TModuleResolverState> moduleResolverState, TIntrusivePtr<TKqpCounters> counters, const TString& uid, const TKqpQueryId& query, const TString& userToken, TKqpDbCountersPtr dbCounters, bool recompileWithNewEngine) @@ -159,7 +159,7 @@ private: private: void Continue(const TActorContext &ctx) { TActorSystem* actorSystem = ctx.ExecutorThread.ActorSystem; - TActorId selfId = ctx.SelfID; + TActorId selfId = ctx.SelfID; auto callback = [actorSystem, selfId](const TFuture<bool>& future) { bool finished = future.GetValue(); @@ -369,7 +369,7 @@ private: } private: - TActorId Owner; + TActorId Owner; TIntrusivePtr<TModuleResolverState> ModuleResolverState; TIntrusivePtr<TKqpCounters> Counters; TString Uid; @@ -382,7 +382,7 @@ private: TInstant StartTime; TDuration CompileCpuTime; TInstant RecompileStartTime; - TActorId TimeoutTimerActorId; + TActorId TimeoutTimerActorId; TIntrusivePtr<IKqpGateway> Gateway; TIntrusivePtr<IKqpHost> KqpHost; TIntrusivePtr<IKqpHost::IAsyncQueryResult> AsyncCompileResult; @@ -400,7 +400,7 @@ void ApplyServiceConfig(TKikimrConfiguration& kqpConfig, const TTableServiceConf } } -IActor* CreateKqpCompileActor(const TActorId& owner, const TKqpSettings::TConstPtr& kqpSettings, +IActor* CreateKqpCompileActor(const TActorId& owner, const TKqpSettings::TConstPtr& kqpSettings, const TTableServiceConfig& serviceConfig, TIntrusivePtr<TModuleResolverState> moduleResolverState, TIntrusivePtr<TKqpCounters> counters, const TString& uid, const TKqpQueryId& query, const TString& userToken, TKqpDbCountersPtr dbCounters, bool recompileWithNewEngine) diff --git a/ydb/core/kqp/kqp_compile_service.cpp b/ydb/core/kqp/kqp_compile_service.cpp index 1dccb8ae072..da886224a2a 100644 --- a/ydb/core/kqp/kqp_compile_service.cpp +++ b/ydb/core/kqp/kqp_compile_service.cpp @@ -189,14 +189,14 @@ struct TKqpCompileRequest { , Deadline(deadline) , DbCounters(dbCounters) {} - TActorId Sender; + TActorId Sender; TKqpQueryId Query; TString Uid; bool KeepInCache = false; TString UserToken; TInstant Deadline; TKqpDbCountersPtr DbCounters; - TActorId CompileActor; + TActorId CompileActor; }; class TKqpRequestsQueue { @@ -734,7 +734,7 @@ private: ReplyError(sender, uid, Ydb::StatusIds::INTERNAL_ERROR, {issue}, ctx); } - static void LogException(const TString& scope, const TActorId& sender, const std::exception& e, + static void LogException(const TString& scope, const TActorId& sender, const std::exception& e, const TActorContext& ctx) { LOG_CRIT_S(ctx, NKikimrServices::KQP_COMPILE_SERVICE, "Exception" diff --git a/ydb/core/kqp/kqp_ic_gateway.cpp b/ydb/core/kqp/kqp_ic_gateway.cpp index aec398fcba7..47489a9bd54 100644 --- a/ydb/core/kqp/kqp_ic_gateway.cpp +++ b/ydb/core/kqp/kqp_ic_gateway.cpp @@ -89,7 +89,7 @@ public: : TBase(request, promise, callback) {} void Bootstrap(const TActorContext& ctx) { - TActorId txproxy = MakeTxProxyID(); + TActorId txproxy = MakeTxProxyID(); ctx.Send(txproxy, this->Request.Release()); this->Become(&TProxyRequestHandler::AwaitState); @@ -125,7 +125,7 @@ public: : TBase(request, promise, callback) {} void Bootstrap(const TActorContext& ctx) { - TActorId kqpProxy = MakeKqpProxyID(ctx.SelfID.NodeId()); + TActorId kqpProxy = MakeKqpProxyID(ctx.SelfID.NodeId()); ctx.Send(kqpProxy, this->Request.Release()); this->Become(&TKqpRequestHandler::AwaitState); @@ -173,9 +173,9 @@ public: , RowsLimit(rowsLimit) {} void Bootstrap(const TActorContext& ctx) { - ActorIdToProto(SelfId(), this->Request->Record.MutableRequestActorId()); + ActorIdToProto(SelfId(), this->Request->Record.MutableRequestActorId()); - TActorId kqpProxy = MakeKqpProxyID(ctx.SelfID.NodeId()); + TActorId kqpProxy = MakeKqpProxyID(ctx.SelfID.NodeId()); ctx.Send(kqpProxy, this->Request.Release()); this->Become(&TKqpScanQueryRequestHandler::AwaitState); @@ -220,7 +220,7 @@ public: } void Handle(NKqp::TEvKqpExecuter::TEvExecuterProgress::TPtr& ev, const TActorContext& ctx) { - ExecuterActorId = ActorIdFromProto(ev->Get()->Record.GetExecuterActorId()); + ExecuterActorId = ActorIdFromProto(ev->Get()->Record.GetExecuterActorId()); LOG_DEBUG_S(ctx, NKikimrServices::KQP_GATEWAY, SelfId() << "Received executer progress for scan query, id: " << ExecuterActorId); } @@ -271,7 +271,7 @@ public: private: ui64 RowsLimit = 0; - TActorId ExecuterActorId; + TActorId ExecuterActorId; bool HasMeta = false; Ydb::ResultSet ResultSet; TVector<NYql::NDqProto::TDqExecutionStats> Executions; @@ -642,7 +642,7 @@ private: TAutoPtr<TRequest> ev = new TRequest(); ev->Record.CopyFrom(this->Request->Record); - TActorId txproxy = MakeTxProxyID(); + TActorId txproxy = MakeTxProxyID(); ctx.Send(txproxy, ev.Release()); } } @@ -673,7 +673,7 @@ public: : TBase(request, promise, {}) {} void Bootstrap(const TActorContext& ctx) { - TActorId txproxy = MakeTxProxyID(); + TActorId txproxy = MakeTxProxyID(); ctx.Send(txproxy, this->Request.Release()); this->Become(&TSchemeOpRequestHandler::AwaitState); @@ -819,7 +819,7 @@ public: } private: - TActorId ShemePipeActorId; + TActorId ShemePipeActorId; }; class TKqpExecPhysicalRequestHandler: public TActorBootstrapped<TKqpExecPhysicalRequestHandler> { @@ -831,7 +831,7 @@ public: return NKikimrServices::TActivity::KQP_EXEC_PHYSICAL_REQUEST_HANDLER; } - TKqpExecPhysicalRequestHandler(TRequest* request, bool streaming, const TActorId& target, TPromise<TResult> promise) + TKqpExecPhysicalRequestHandler(TRequest* request, bool streaming, const TActorId& target, TPromise<TResult> promise) : Request(request) , Streaming(streaming) , Executer(request->ExecuterId) @@ -929,8 +929,8 @@ private: private: THolder<TRequest> Request; bool Streaming; - TActorId Executer; - TActorId Target; + TActorId Executer; + TActorId Target; TPromise<TResult> Promise; }; @@ -1610,7 +1610,7 @@ public: return RunInternal(cluster, program, TKqpParamsMap(), true, false, TMkqlSettings()); } - TFuture<TExecPhysicalResult> ExecutePhysical(TExecPhysicalRequest&& request, const NActors::TActorId& target) override { + TFuture<TExecPhysicalResult> ExecutePhysical(TExecPhysicalRequest&& request, const NActors::TActorId& target) override { return ExecutePhysicalQueryInternal(std::move(request), target, false); } @@ -1846,7 +1846,7 @@ public: }); } - TFuture<TExecPhysicalResult> ExecuteScanQuery(TExecPhysicalRequest&& request, const TActorId& target) override { + TFuture<TExecPhysicalResult> ExecuteScanQuery(TExecPhysicalRequest&& request, const TActorId& target) override { return ExecutePhysicalQueryInternal(std::move(request), target, true); } @@ -1915,7 +1915,7 @@ private: using TTransactionResponse = TEvTxUserProxy::TEvProposeTransactionStatus; private: - TActorId RegisterActor(IActor* actor) { + TActorId RegisterActor(IActor* actor) { return ActorSystem->Register(actor, TMailboxType::HTSwap, ActorSystem->AppData<TAppData>()->UserPoolId); } @@ -1976,7 +1976,7 @@ private: } template<typename TRequest, typename TResponse, typename TResult> - TFuture<TResult> SendActorRequest(const TActorId& actorId, TRequest* request, + TFuture<TResult> SendActorRequest(const TActorId& actorId, TRequest* request, typename TActorRequestHandler<TRequest, TResponse, TResult>::TCallbackFunc callback) { auto promise = NewPromise<TResult>(); @@ -2120,7 +2120,7 @@ private: } } - TFuture<TExecPhysicalResult> ExecutePhysicalQueryInternal(TExecPhysicalRequest&& request, const TActorId& target, + TFuture<TExecPhysicalResult> ExecutePhysicalQueryInternal(TExecPhysicalRequest&& request, const TActorId& target, bool streaming) { auto executerActor = CreateKqpExecuter(std::move(request), Database, diff --git a/ydb/core/kqp/kqp_impl.h b/ydb/core/kqp/kqp_impl.h index 32f34c7ec5d..7d6f172899b 100644 --- a/ydb/core/kqp/kqp_impl.h +++ b/ydb/core/kqp/kqp_impl.h @@ -49,7 +49,7 @@ IActor* CreateKqpCompileService(const NKikimrConfig::TTableServiceConfig& servic const TKqpSettings::TConstPtr& kqpSettings, TIntrusivePtr<TModuleResolverState> moduleResolverState, TIntrusivePtr<TKqpCounters> counters, std::shared_ptr<IQueryReplayBackendFactory> queryReplayFactory); -IActor* CreateKqpCompileActor(const TActorId& owner, const TKqpSettings::TConstPtr& kqpSettings, +IActor* CreateKqpCompileActor(const TActorId& owner, const TKqpSettings::TConstPtr& kqpSettings, const NKikimrConfig::TTableServiceConfig& serviceConfig, TIntrusivePtr<TModuleResolverState> moduleResolverState, TIntrusivePtr<TKqpCounters> counters, const TString& uid, const TKqpQueryId& query, const TString& userToken, TKqpDbCountersPtr dbCounters, bool recompileWithNewEngine); @@ -74,7 +74,7 @@ struct TKqpWorkerSettings { , DbCounters(dbCounters) {} }; -IActor* CreateKqpWorkerActor(const TActorId& owner, const TString& sessionId, +IActor* CreateKqpWorkerActor(const TActorId& owner, const TString& sessionId, const TKqpSettings::TConstPtr& kqpSettings, const TKqpWorkerSettings& workerSettings, TIntrusivePtr<TModuleResolverState> moduleResolverState, TIntrusivePtr<TKqpCounters> counters); diff --git a/ydb/core/kqp/kqp_worker_actor.cpp b/ydb/core/kqp/kqp_worker_actor.cpp index 0ca28f7e24a..cccb8a51e8e 100644 --- a/ydb/core/kqp/kqp_worker_actor.cpp +++ b/ydb/core/kqp/kqp_worker_actor.cpp @@ -46,7 +46,7 @@ constexpr std::string_view DocumentApiRequestType = "_document_api_request"sv; using TQueryResult = IKqpHost::TQueryResult; struct TKqpQueryState { - TActorId Sender; + TActorId Sender; ui64 ProxyRequestId = 0; NKikimrKqp::TQueryRequest Request; TIntrusivePtr<IKqpHost::IAsyncQueryResult> AsyncQueryResult; @@ -56,7 +56,7 @@ struct TKqpQueryState { TString RequestType; ui64 ParametersSize = 0; TString UserToken; - TActorId RequestActorId; + TActorId RequestActorId; TInstant StartTime; TDuration CpuTime; NYql::TKikimrQueryDeadlines QueryDeadlines; @@ -144,7 +144,7 @@ public: return NKikimrServices::TActivity::KQP_WORKER_ACTOR; } - TKqpWorkerActor(const TActorId& owner, const TString& sessionId, const TKqpSettings::TConstPtr& kqpSettings, + TKqpWorkerActor(const TActorId& owner, const TString& sessionId, const TKqpSettings::TConstPtr& kqpSettings, const TKqpWorkerSettings& workerSettings, TIntrusivePtr<TModuleResolverState> moduleResolverState, TIntrusivePtr<TKqpCounters> counters) : Owner(owner) @@ -281,7 +281,7 @@ public: QueryState->StartTime = now; QueryState->ReplyFlags = queryRequest.GetReplyFlags(); QueryState->UserToken = event.GetUserToken(); - QueryState->RequestActorId = ActorIdFromProto(event.GetRequestActorId()); + QueryState->RequestActorId = ActorIdFromProto(event.GetRequestActorId()); if (GetStatsMode(queryRequest, EKikimrStatsMode::None) > EKikimrStatsMode::Basic) { QueryState->ReplyFlags |= NKikimrKqp::QUERY_REPLY_FLAG_AST; @@ -1271,7 +1271,7 @@ private: } bool ExecuteQuery(NKikimrKqp::TQueryRequest& queryRequest, NKikimrKqp::EQueryType type, bool commit, - const TActorId& requestActorId) + const TActorId& requestActorId) { const auto& query = queryRequest.GetQuery(); auto* parameters = queryRequest.MutableParameters(); @@ -1470,7 +1470,7 @@ private: Y_VERIFY(QueryState); TActorSystem* actorSystem = ctx.ExecutorThread.ActorSystem; - TActorId selfId = ctx.SelfID; + TActorId selfId = ctx.SelfID; ui32 queryId = QueryId; auto callback = [actorSystem, selfId, queryId](const TFuture<bool>& future) { @@ -1486,7 +1486,7 @@ private: Y_VERIFY(CleanupState); TActorSystem* actorSystem = ctx.ExecutorThread.ActorSystem; - TActorId selfId = ctx.SelfID; + TActorId selfId = ctx.SelfID; ui32 queryId = QueryId; auto callback = [actorSystem, selfId, queryId](const TFuture<bool>& future) { @@ -1744,7 +1744,7 @@ private: } } - bool ReplyPingStatus(const TActorId& sender, ui64 proxyRequestId, bool ready, const TActorContext& ctx) { + bool ReplyPingStatus(const TActorId& sender, ui64 proxyRequestId, bool ready, const TActorContext& ctx) { auto ev = MakeHolder<TEvKqp::TEvPingSessionResponse>(); auto& record = ev->Record; record.SetStatus(Ydb::StatusIds::SUCCESS); @@ -1756,7 +1756,7 @@ private: return ctx.Send(sender, ev.Release(), 0, proxyRequestId); } - bool ReplyProcessError(const TActorId& sender, ui64 proxyRequestId, const TKqpRequestInfo& requestInfo, + bool ReplyProcessError(const TActorId& sender, ui64 proxyRequestId, const TKqpRequestInfo& requestInfo, Ydb::StatusIds::StatusCode ydbStatus, const TString& message, const TActorContext& ctx) { LOG_WARN_S(ctx, NKikimrServices::KQP_WORKER, requestInfo << message); @@ -1767,7 +1767,7 @@ private: return ctx.Send(sender, response.Release(), 0, proxyRequestId); } - bool CheckRequest(const TKqpRequestInfo& requestInfo, const TActorId& sender, ui64 proxyRequestId, + bool CheckRequest(const TKqpRequestInfo& requestInfo, const TActorId& sender, ui64 proxyRequestId, const TActorContext& ctx) { if (requestInfo.GetSessionId() != SessionId) { @@ -2106,7 +2106,7 @@ private: << IdleTimerActorId); ctx.Send(IdleTimerActorId, new TEvents::TEvPoisonPill()); } - IdleTimerActorId = TActorId(); + IdleTimerActorId = TActorId(); } IKikimrQueryExecutor::TExecuteSettings CreateRollbackSettings() { @@ -2162,7 +2162,7 @@ private: } private: - TActorId Owner; + TActorId Owner; TString SessionId; TKqpWorkerSettings Settings; TIntrusivePtr<TModuleResolverState> ModuleResolverState; @@ -2176,13 +2176,13 @@ private: THolder<TKqpQueryState> QueryState; THolder<TKqpCleanupState> CleanupState; ui32 IdleTimerId; - TActorId IdleTimerActorId; + TActorId IdleTimerActorId; std::optional<TSessionShutdownState> ShutdownState; }; } // namespace -IActor* CreateKqpWorkerActor(const TActorId& owner, const TString& sessionId, +IActor* CreateKqpWorkerActor(const TActorId& owner, const TString& sessionId, const TKqpSettings::TConstPtr& kqpSettings, const TKqpWorkerSettings& workerSettings, TIntrusivePtr<TModuleResolverState> moduleResolverState, TIntrusivePtr<TKqpCounters> counters) { diff --git a/ydb/core/kqp/prepare/kqp_prepare.h b/ydb/core/kqp/prepare/kqp_prepare.h index 2ef8d2c8706..d114a449458 100644 --- a/ydb/core/kqp/prepare/kqp_prepare.h +++ b/ydb/core/kqp/prepare/kqp_prepare.h @@ -85,7 +85,7 @@ struct TKqlTransformContext : TThrRefBase { TIntrusivePtr<NYql::TKikimrQueryContext> QueryCtx; TIntrusivePtr<NYql::TKikimrTablesData> Tables; NKikimrKqp::TKqlSettings Settings; - NActors::TActorId ReplyTarget; + NActors::TActorId ReplyTarget; TKqpAnalyzeResults AnalyzeResults; NKikimrKqp::TPreparedKql* PreparingKql = nullptr; diff --git a/ydb/core/kqp/prepare/kqp_query_analyze.cpp b/ydb/core/kqp/prepare/kqp_query_analyze.cpp index 189293111f5..b37e4eb0582 100644 --- a/ydb/core/kqp/prepare/kqp_query_analyze.cpp +++ b/ydb/core/kqp/prepare/kqp_query_analyze.cpp @@ -3,7 +3,7 @@ #include <ydb/library/yql/core/yql_expr_optimize.h> #include <ydb/library/yql/core/yql_expr_type_annotation.h> #include <ydb/library/yql/utils/log/log.h> - + #include <util/generic/queue.h> namespace NKikimr { diff --git a/ydb/core/kqp/prepare/kqp_query_exec.cpp b/ydb/core/kqp/prepare/kqp_query_exec.cpp index 3111e3fcd8f..bc2d4e94ce9 100644 --- a/ydb/core/kqp/prepare/kqp_query_exec.cpp +++ b/ydb/core/kqp/prepare/kqp_query_exec.cpp @@ -802,8 +802,8 @@ void LogMkqlResult(const NKikimrMiniKQL::TResult& result, TExprContext& ctx) { NProtoBuf::TextFormat::PrintToString(result.GetValue(), &resultValue); YQL_CLOG(TRACE, ProviderKqp) << "MiniKQL results\n" - << "Type:\n" << resultType - << "Value:\n" << resultValue; + << "Type:\n" << resultType + << "Value:\n" << resultValue; } } diff --git a/ydb/core/kqp/prepare/kqp_query_finalize.cpp b/ydb/core/kqp/prepare/kqp_query_finalize.cpp index 1e3db6c56bd..5e442780f7a 100644 --- a/ydb/core/kqp/prepare/kqp_query_finalize.cpp +++ b/ydb/core/kqp/prepare/kqp_query_finalize.cpp @@ -130,7 +130,7 @@ public: YQL_CLOG(INFO, ProviderKqp) << "Rollback Tx" << ", deferred effects count: " << TxState->Tx().DeferredEffects.Size() << ", locks count: " << TxState->Tx().Locks.Size(); - + effectsNode = GetRollbackEffects(input->Pos(), ctx); State = EFinalizeState::RollbackInProgress; } diff --git a/ydb/core/kqp/provider/yql_kikimr_gateway.h b/ydb/core/kqp/provider/yql_kikimr_gateway.h index 7570609c862..7598494a2b9 100644 --- a/ydb/core/kqp/provider/yql_kikimr_gateway.h +++ b/ydb/core/kqp/provider/yql_kikimr_gateway.h @@ -30,7 +30,7 @@ namespace NYql { using NUdf::EDataSlot; -class TKikimrGatewayConfig; +class TKikimrGatewayConfig; class TKikimrClusterConfig; using TKikimrParamsMap = TMap<TString, NKikimrMiniKQL::TParams>; diff --git a/ydb/core/kqp/provider/yql_kikimr_provider.cpp b/ydb/core/kqp/provider/yql_kikimr_provider.cpp index fe5ed851add..635d1648279 100644 --- a/ydb/core/kqp/provider/yql_kikimr_provider.cpp +++ b/ydb/core/kqp/provider/yql_kikimr_provider.cpp @@ -254,7 +254,7 @@ void TKikimrTableDescription::ToYson(NYson::TYsonWriter& writer) const { writer.OnKeyedItem(TStringBuf("IsRealData")); writer.OnBooleanScalar(true); writer.OnKeyedItem(TStringBuf("YqlCompatibleSchema")); - writer.OnBooleanScalar(true); + writer.OnBooleanScalar(true); writer.OnKeyedItem(TStringBuf("RecordsCount")); writer.OnInt64Scalar(meta.RecordsCount); diff --git a/ydb/core/kqp/proxy/kqp_proxy_service.cpp b/ydb/core/kqp/proxy/kqp_proxy_service.cpp index 9f37d5a4c9e..decd2cb3011 100644 --- a/ydb/core/kqp/proxy/kqp_proxy_service.cpp +++ b/ydb/core/kqp/proxy/kqp_proxy_service.cpp @@ -576,9 +576,9 @@ public: } } - TActorId targetId; + TActorId targetId; if (!request.GetSessionId().empty()) { - TProcessResult<TActorId> result; + TProcessResult<TActorId> result; if (!TryGetSessionTargetActor(request.GetSessionId(), requestInfo, result)) { if (!dbCounters) { dbCounters = Counters->GetDbCounters(request.GetDatabase()); @@ -638,7 +638,7 @@ public: LogRequest(request, requestInfo, ev->Sender, dbCounters); if (!sessionId.empty()) { - TProcessResult<TActorId> result; + TProcessResult<TActorId> result; if (TryGetSessionTargetActor(sessionId, requestInfo, result)) { Send(result.Value, ev->Release().Release()); } @@ -658,7 +658,7 @@ public: LogRequest(request, requestInfo, ev->Sender, requestId, dbCounters); - TProcessResult<TActorId> result; + TProcessResult<TActorId> result; if (!TryGetSessionTargetActor(sessionId, requestInfo, result)) { ReplyProcessError(result.YdbStatus, result.Error, requestId); return; @@ -954,7 +954,7 @@ public: const auto &event = ev->Get()->Record; if (event.GetStatus() == Ydb::StatusIds::SUCCESS && event.GetResponse().GetClosed()) { auto sessionId = event.GetResponse().GetSessionId(); - TActorId workerId = ev->Sender; + TActorId workerId = ev->Sender; RemoveSession(sessionId, workerId); @@ -1290,9 +1290,9 @@ private: std::optional<TInstant> LastPublishResourcesAt; TActorId BoardLookupActor; TActorId BoardPublishActor; - TActorId CompileService; + TActorId CompileService; TActorId KqpNodeService; - TActorId SpillingService; + TActorId SpillingService; TActorId WhiteBoardService; NKikimrKqp::TKqpProxyNodeResources NodeResources; }; diff --git a/ydb/core/kqp/runtime/kqp_channel_storage.cpp b/ydb/core/kqp/runtime/kqp_channel_storage.cpp index 7f2e3263f56..d39d0090905 100644 --- a/ydb/core/kqp/runtime/kqp_channel_storage.cpp +++ b/ydb/core/kqp/runtime/kqp_channel_storage.cpp @@ -186,7 +186,7 @@ private: const ui64 TxId; const ui64 ChannelId; IDqChannelStorage::TWakeUpCallback WakeUp; - TActorId SpillingActorId; + TActorId SpillingActorId; TMap<ui64, ui64> WritingBlobs; // blobId -> blobSize ui64 WritingBlobsSize = 0; diff --git a/ydb/core/kqp/runtime/kqp_spilling_file.cpp b/ydb/core/kqp/runtime/kqp_spilling_file.cpp index d8e55947c6e..0dd0a4f7a3d 100644 --- a/ydb/core/kqp/runtime/kqp_spilling_file.cpp +++ b/ydb/core/kqp/runtime/kqp_spilling_file.cpp @@ -68,7 +68,7 @@ struct TEvKqpSpillingLocalFile { // it provides human-readable description for logs and viewer. class TKqpLocalFileSpillingActor : public TActorBootstrapped<TKqpLocalFileSpillingActor> { public: - TKqpLocalFileSpillingActor(ui64 txId, const TString& details, const TActorId& client, bool removeBlobsAfterRead) + TKqpLocalFileSpillingActor(ui64 txId, const TString& details, const TActorId& client, bool removeBlobsAfterRead) : TxId(txId) , Details(details) , ClientActorId(client) @@ -132,7 +132,7 @@ private: } private: - void ValidateSender(const TActorId& sender) { + void ValidateSender(const TActorId& sender) { YQL_ENSURE(ClientActorId == sender, "" << ClientActorId << " != " << sender); } @@ -144,9 +144,9 @@ private: private: const ui64 TxId; const TString Details; - const TActorId ClientActorId; + const TActorId ClientActorId; const bool RemoveBlobsAfterRead; - TActorId ServiceActorId; + TActorId ServiceActorId; }; class TKqpLocalFileSpillingService : public TActorBootstrapped<TKqpLocalFileSpillingService> { @@ -163,13 +163,13 @@ private: static_assert(EEv::LastEvent - EventSpaceBegin(TKikimrEvents::ES_PRIVATE) < 16); struct TEvCloseFileResponse : public TEventLocal<TEvCloseFileResponse, EvCloseFileResponse> { - TActorId Client; + TActorId Client; TDuration WaitTime; TDuration WorkTime; }; struct TEvWriteFileResponse : public TEventLocal<TEvWriteFileResponse, EvWriteFileResponse> { - TActorId Client; + TActorId Client; TDuration WaitTime; TDuration WorkTime; ui64 BlobId = 0; @@ -178,7 +178,7 @@ private: }; struct TEvReadFileResponse : public TEventLocal<TEvReadFileResponse, EvReadFileResponse> { - TActorId Client; + TActorId Client; TDuration WaitTime; TDuration WorkTime; ui64 BlobId = 0; @@ -189,7 +189,7 @@ private: }; struct TFileDesc; - using TFilesIt = __yhashtable_iterator<std::pair<const TActorId, TFileDesc>>; + using TFilesIt = __yhashtable_iterator<std::pair<const TActorId, TFileDesc>>; public: TKqpLocalFileSpillingService(const NKikimrConfig::TTableServiceConfig::TSpillingServiceConfig::TLocalFileConfig& config, @@ -746,8 +746,8 @@ private: private: struct TCloseFileOp : public IObjectInQueue { - TActorId Client; - TActorId Service; + TActorId Client; + TActorId Service; TActorSystem* ActorSystem; TVector<THolder<TFileHandle>> FileHandles; TVector<TString> FileNames; @@ -779,8 +779,8 @@ private: }; struct TWriteFileOp : public IObjectInQueue { - TActorId Client; - TActorId Service; + TActorId Client; + TActorId Service; TActorSystem* ActorSystem; TString FileName; bool CreateFile = false; @@ -818,8 +818,8 @@ private: }; struct TReadFileOp : public IObjectInQueue { - TActorId Client; - TActorId Service; + TActorId Client; + TActorId Service; TActorSystem* ActorSystem; TString FileName; ui64 BlobId; @@ -942,14 +942,14 @@ private: }; THolder<IThreadPool> IoThreadPool; - THashMap<TActorId, TFileDesc> Files; + THashMap<TActorId, TFileDesc> Files; TList<const TClosedFileDesc> ClosedFiles; ui64 TotalSize = 0; }; } // anonymous namespace -IActor* CreateKqpLocalFileSpillingActor(ui64 txId, const TString& details, const TActorId& client, +IActor* CreateKqpLocalFileSpillingActor(ui64 txId, const TString& details, const TActorId& client, bool removeBlobsAfterRead) { return new TKqpLocalFileSpillingActor(txId, details, client, removeBlobsAfterRead); diff --git a/ydb/core/kqp/runtime/kqp_spilling_file.h b/ydb/core/kqp/runtime/kqp_spilling_file.h index 4355c04b141..14196a8498e 100644 --- a/ydb/core/kqp/runtime/kqp_spilling_file.h +++ b/ydb/core/kqp/runtime/kqp_spilling_file.h @@ -8,7 +8,7 @@ namespace NKikimr::NKqp { -NActors::IActor* CreateKqpLocalFileSpillingActor(ui64 txId, const TString& details, const NActors::TActorId& client, +NActors::IActor* CreateKqpLocalFileSpillingActor(ui64 txId, const TString& details, const NActors::TActorId& client, bool removeBlobsAfterRead); NActors::IActor* CreateKqpLocalFileSpillingService( diff --git a/ydb/core/kqp/runtime/kqp_spilling_file_ut.cpp b/ydb/core/kqp/runtime/kqp_spilling_file_ut.cpp index a553513e4cc..c3efc85cbcf 100644 --- a/ydb/core/kqp/runtime/kqp_spilling_file_ut.cpp +++ b/ydb/core/kqp/runtime/kqp_spilling_file_ut.cpp @@ -36,7 +36,7 @@ TIntrusivePtr<NMonitoring::TDynamicCounters> Counters() { return counters; } -TActorId StartSpillingService(TTestBasicRuntime& runtime, ui64 maxTotalSize = 1000, ui64 maxFileSize = 500, +TActorId StartSpillingService(TTestBasicRuntime& runtime, ui64 maxTotalSize = 1000, ui64 maxFileSize = 500, ui64 maxFilePartSize = 100, const TString& root = "./kqp_spilling/") { Cerr << "cwd: " << NFs::CurrentWorkingDirectory() << Endl; @@ -60,7 +60,7 @@ TActorId StartSpillingService(TTestBasicRuntime& runtime, ui64 maxTotalSize = 10 return spillingServiceActorId; } -TActorId StartSpillingActor(TTestBasicRuntime& runtime, const TActorId& client, bool removeBlobsAfterRead = true) { +TActorId StartSpillingActor(TTestBasicRuntime& runtime, const TActorId& client, bool removeBlobsAfterRead = true) { auto *spillingActor = CreateKqpLocalFileSpillingActor(1, "test", client, removeBlobsAfterRead); auto spillingActorId = runtime.Register(spillingActor); runtime.EnableScheduleForActor(spillingActorId); diff --git a/ydb/core/kqp/ut/kqp_indexes_ut.cpp b/ydb/core/kqp/ut/kqp_indexes_ut.cpp index ca132f627cb..505cf09e6b8 100644 --- a/ydb/core/kqp/ut/kqp_indexes_ut.cpp +++ b/ydb/core/kqp/ut/kqp_indexes_ut.cpp @@ -4234,7 +4234,7 @@ R"([[#;#;["Primary1"];[41u]];[["Secondary2"];[2u];["Primary2"];[42u]];[["Seconda query, TTxControl::BeginTx(TTxSettings::SerializableRW()).CommitTx()) .ExtractValueSync(); - UNIT_ASSERT(result.GetIssues().ToString().Contains("Unexpected token")); + UNIT_ASSERT(result.GetIssues().ToString().Contains("Unexpected token")); UNIT_ASSERT_VALUES_EQUAL(result.GetStatus(), NYdb::EStatus::GENERIC_ERROR); } @@ -4247,7 +4247,7 @@ R"([[#;#;["Primary1"];[41u]];[["Secondary2"];[2u];["Primary2"];[42u]];[["Seconda query, TTxControl::BeginTx(TTxSettings::SerializableRW()).CommitTx()) .ExtractValueSync(); - UNIT_ASSERT(result.GetIssues().ToString().Contains("Unexpected token")); + UNIT_ASSERT(result.GetIssues().ToString().Contains("Unexpected token")); UNIT_ASSERT_VALUES_EQUAL(result.GetStatus(), NYdb::EStatus::GENERIC_ERROR); } @@ -4260,7 +4260,7 @@ R"([[#;#;["Primary1"];[41u]];[["Secondary2"];[2u];["Primary2"];[42u]];[["Seconda query, TTxControl::BeginTx(TTxSettings::SerializableRW()).CommitTx()) .ExtractValueSync(); - UNIT_ASSERT(result.GetIssues().ToString().Contains("Unexpected token")); + UNIT_ASSERT(result.GetIssues().ToString().Contains("Unexpected token")); UNIT_ASSERT_VALUES_EQUAL(result.GetStatus(), NYdb::EStatus::GENERIC_ERROR); } @@ -4273,7 +4273,7 @@ R"([[#;#;["Primary1"];[41u]];[["Secondary2"];[2u];["Primary2"];[42u]];[["Seconda query, TTxControl::BeginTx(TTxSettings::SerializableRW()).CommitTx()) .ExtractValueSync(); - UNIT_ASSERT(result.GetIssues().ToString().Contains("Unexpected token")); + UNIT_ASSERT(result.GetIssues().ToString().Contains("Unexpected token")); UNIT_ASSERT_VALUES_EQUAL(result.GetStatus(), NYdb::EStatus::GENERIC_ERROR); } diff --git a/ydb/core/mind/address_classification/counters.h b/ydb/core/mind/address_classification/counters.h index 9b8d6cf6fda..0ca6e924680 100644 --- a/ydb/core/mind/address_classification/counters.h +++ b/ydb/core/mind/address_classification/counters.h @@ -1,6 +1,6 @@ #pragma once -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <util/generic/ptr.h> #include <util/generic/noncopyable.h> diff --git a/ydb/core/mind/address_classification/net_classifier.cpp b/ydb/core/mind/address_classification/net_classifier.cpp index 56d0fc8a953..11c7e6750c7 100644 --- a/ydb/core/mind/address_classification/net_classifier.cpp +++ b/ydb/core/mind/address_classification/net_classifier.cpp @@ -58,7 +58,7 @@ private: return AppData(Ctx())->NetClassifierConfig; } - void AddSubscriber(const TActorId subscriberId) { + void AddSubscriber(const TActorId subscriberId) { Subscribers.insert(subscriberId); *Counters->SubscribersCount = Subscribers.size(); @@ -286,7 +286,7 @@ private: SendClassifierUpdate(ev->Sender); } - void SendClassifierUpdate(const TActorId recipient) { + void SendClassifierUpdate(const TActorId recipient) { auto ev = MakeHolder<TEvNetClassifier::TEvClassifierUpdate>(); ev->Classifier = LabeledAddressClassifier; ev->NetDataUpdateTimestamp = NetDataUpdateTimestamp; @@ -398,7 +398,7 @@ private: TLabeledAddressClassifier::TConstPtr LabeledAddressClassifier; TMaybe<TInstant> NetDataUpdateTimestamp; - THashSet<TActorId> Subscribers; + THashSet<TActorId> Subscribers; ui64 ClassifierUpdatesSent = 0; ui64 GoodConfigNotifications = 0; diff --git a/ydb/core/mind/address_classification/net_classifier.h b/ydb/core/mind/address_classification/net_classifier.h index 0c84f140975..6a298cd4c4d 100644 --- a/ydb/core/mind/address_classification/net_classifier.h +++ b/ydb/core/mind/address_classification/net_classifier.h @@ -8,7 +8,7 @@ #include <library/cpp/actors/core/defs.h> #include <library/cpp/actors/core/event_local.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <util/generic/ptr.h> @@ -16,11 +16,11 @@ namespace NKikimr::NNetClassifier { -using NActors::TActorId; +using NActors::TActorId; -inline TActorId MakeNetClassifierID() { +inline TActorId MakeNetClassifierID() { static const char x[12] = "net_classvc"; - return TActorId(0, TStringBuf(x, 12)); + return TActorId(0, TStringBuf(x, 12)); } NActors::IActor* CreateNetClassifier(); diff --git a/ydb/core/mind/address_classification/net_classifier_ut.cpp b/ydb/core/mind/address_classification/net_classifier_ut.cpp index 9c0bc407e54..088d25b9c76 100644 --- a/ydb/core/mind/address_classification/net_classifier_ut.cpp +++ b/ydb/core/mind/address_classification/net_classifier_ut.cpp @@ -43,7 +43,7 @@ static TString FormNetData() { "2a02:6b8:fc00::/48\tSAS"; } -static TAutoPtr<IEventHandle> GetClassifierUpdate(TServer& server, const TActorId sender) { +static TAutoPtr<IEventHandle> GetClassifierUpdate(TServer& server, const TActorId sender) { auto& actorSystem = *server.GetRuntime(); actorSystem.Send( @@ -81,7 +81,7 @@ Y_UNIT_TEST_SUITE(TNetClassifierTest) { TServer cleverServer = TServer(settings); { - const TActorId sender = cleverServer.GetRuntime()->AllocateEdgeActor(); + const TActorId sender = cleverServer.GetRuntime()->AllocateEdgeActor(); auto handle = GetClassifierUpdate(cleverServer, sender); auto* event = handle->Get<TEvNetClassifier::TEvClassifierUpdate>(); @@ -118,7 +118,7 @@ Y_UNIT_TEST_SUITE(TNetClassifierTest) { TServer cleverServer = TServer(settings); { - const TActorId sender = cleverServer.GetRuntime()->AllocateEdgeActor(); + const TActorId sender = cleverServer.GetRuntime()->AllocateEdgeActor(); auto handle = GetClassifierUpdate(cleverServer, sender); auto* event = handle->Get<TEvNetClassifier::TEvClassifierUpdate>(); @@ -144,7 +144,7 @@ Y_UNIT_TEST_SUITE(TNetClassifierTest) { } Y_UNIT_TEST(TestInitFromRemoteSource) { - NMonitoring::TMetricRegistry sensors; + NMonitoring::TMetricRegistry sensors; TPortManager pm; const ui16 port = pm.GetPort(2134); @@ -165,16 +165,16 @@ Y_UNIT_TEST_SUITE(TNetClassifierTest) { auto& actorSystem = *cleverServer.GetRuntime(); NActors::IActor* proxy = NHttp::CreateHttpProxy(sensors); - NActors::TActorId proxyId = actorSystem.Register(proxy); + NActors::TActorId proxyId = actorSystem.Register(proxy); actorSystem.Send( - new NActors::IEventHandle(proxyId, TActorId(), new NHttp::TEvHttpProxy::TEvAddListeningPort(netDataSourcePort)), 0, true + new NActors::IEventHandle(proxyId, TActorId(), new NHttp::TEvHttpProxy::TEvAddListeningPort(netDataSourcePort)), 0, true ); - NActors::TActorId serverId = actorSystem.AllocateEdgeActor(); + NActors::TActorId serverId = actorSystem.AllocateEdgeActor(); actorSystem.Send(new NActors::IEventHandle(proxyId, serverId, new NHttp::TEvHttpProxy::TEvRegisterHandler(uri, serverId)), 0, true); - const TActorId sender = cleverServer.GetRuntime()->AllocateEdgeActor(); + const TActorId sender = cleverServer.GetRuntime()->AllocateEdgeActor(); // initially classifier's data should be empty auto handle = GetClassifierUpdate(cleverServer, sender); diff --git a/ydb/core/mind/address_classification/ya.make b/ydb/core/mind/address_classification/ya.make index 82a15e52c0f..73301851da9 100644 --- a/ydb/core/mind/address_classification/ya.make +++ b/ydb/core/mind/address_classification/ya.make @@ -14,7 +14,7 @@ GENERATE_ENUM_SERIALIZATION(net_classifier.h) PEERDIR( library/cpp/actors/core - library/cpp/monlib/dynamic_counters + library/cpp/monlib/dynamic_counters ydb/core/base ydb/core/cms/console ydb/core/mon diff --git a/ydb/core/mind/bscontroller/bsc.cpp b/ydb/core/mind/bscontroller/bsc.cpp index c03bad81d5a..779381fd513 100644 --- a/ydb/core/mind/bscontroller/bsc.cpp +++ b/ydb/core/mind/bscontroller/bsc.cpp @@ -25,7 +25,7 @@ TString TPDiskID::ToString() const { return str.Str(); } -IActor* CreateFlatBsController(const TActorId &tablet, TTabletStorageInfo *info) { +IActor* CreateFlatBsController(const TActorId &tablet, TTabletStorageInfo *info) { return new NBsController::TBlobStorageController(tablet, info); } diff --git a/ydb/core/mind/bscontroller/bsc.h b/ydb/core/mind/bscontroller/bsc.h index 48018887836..afa641dc094 100644 --- a/ydb/core/mind/bscontroller/bsc.h +++ b/ydb/core/mind/bscontroller/bsc.h @@ -3,6 +3,6 @@ namespace NKikimr { -IActor* CreateFlatBsController(const TActorId &tablet, TTabletStorageInfo *info); +IActor* CreateFlatBsController(const TActorId &tablet, TTabletStorageInfo *info); } //NKikimr diff --git a/ydb/core/mind/bscontroller/config_cmd.cpp b/ydb/core/mind/bscontroller/config_cmd.cpp index 30257b421fe..2fc925a9cd7 100644 --- a/ydb/core/mind/bscontroller/config_cmd.cpp +++ b/ydb/core/mind/bscontroller/config_cmd.cpp @@ -7,7 +7,7 @@ namespace NKikimr::NBsController { class TBlobStorageController::TBlobStorageController::TTxConfigCmd : public TTransactionBase<TBlobStorageController> { - const TActorId NotifyId; + const TActorId NotifyId; const ui64 Cookie; const NKikimrBlobStorage::TConfigRequest Cmd; const bool SelfHeal; diff --git a/ydb/core/mind/bscontroller/defs.h b/ydb/core/mind/bscontroller/defs.h index a1596cb5998..162907fe8e3 100644 --- a/ydb/core/mind/bscontroller/defs.h +++ b/ydb/core/mind/bscontroller/defs.h @@ -40,7 +40,7 @@ #include <library/cpp/actors/core/interconnect.h> #include <library/cpp/actors/core/log.h> #include <library/cpp/actors/interconnect/interconnect.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <type_traits> #include <util/generic/algorithm.h> #include <util/generic/intrlist.h> diff --git a/ydb/core/mind/bscontroller/impl.h b/ydb/core/mind/bscontroller/impl.h index d07651e842b..ba311a254c3 100644 --- a/ydb/core/mind/bscontroller/impl.h +++ b/ydb/core/mind/bscontroller/impl.h @@ -1600,7 +1600,7 @@ private: // Metric collection struct TSelectGroupsQueueItem { - TActorId RespondTo; + TActorId RespondTo; ui64 Cookie; THolder<TEvBlobStorage::TEvControllerSelectGroupsResult> Event; TSet<TPDiskId> BlockedPDisks; @@ -1659,7 +1659,7 @@ public: return NKikimrServices::TActivity::BS_CONTROLLER_ACTOR; } - TBlobStorageController(const TActorId &tablet, TTabletStorageInfo *info) + TBlobStorageController(const TActorId &tablet, TTabletStorageInfo *info) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, new NMiniKQL::TMiniKQLFactory) , ResponsivenessPinger(nullptr) @@ -1965,7 +1965,7 @@ public: //////////////////////////////////////////////////////////////////////////////////////////////////////////////////// // NODE WARDEN PIPE LIFETIME MANAGEMENT - THashMap<TActorId, std::optional<TNodeId>> PipeServerToNode; + THashMap<TActorId, std::optional<TNodeId>> PipeServerToNode; void Handle(TEvTabletPipe::TEvServerConnected::TPtr& ev); void Handle(TEvTabletPipe::TEvServerDisconnected::TPtr& ev); diff --git a/ydb/core/mind/bscontroller/monitoring.cpp b/ydb/core/mind/bscontroller/monitoring.cpp index b87b01549a0..245c782cea6 100644 --- a/ydb/core/mind/bscontroller/monitoring.cpp +++ b/ydb/core/mind/bscontroller/monitoring.cpp @@ -1,7 +1,7 @@ #include "impl.h" -#include <library/cpp/json/json_writer.h> - +#include <library/cpp/json/json_writer.h> + namespace NKikimr { namespace NBsController { @@ -25,7 +25,7 @@ static TString PrintMaybe(const TMaybe<T>& m) { } class TBlobStorageController::TTxMonEvent_OperationLog : public TTransactionBase<TBlobStorageController> { - const TActorId RespondTo; + const TActorId RespondTo; const TCgiParameters Params; private: @@ -43,7 +43,7 @@ private: ui64 NumRows = 0; public: - TTxMonEvent_OperationLog(const TActorId& sender, TCgiParameters params, TSelf *controller) + TTxMonEvent_OperationLog(const TActorId& sender, TCgiParameters params, TSelf *controller) : TBase(controller) , RespondTo(sender) , Params(std::move(params)) @@ -173,7 +173,7 @@ public: class TBlobStorageController::TTxMonEvent_HealthEvents : public TTransactionBase<TBlobStorageController> { - const TActorId RespondTo; + const TActorId RespondTo; const bool Json; const ui64 Offset = 0; const ui64 NumRows = 1000; @@ -493,13 +493,13 @@ public: class TBlobStorageController::TTxMonEvent_SetDown : public TTransactionBase<TBlobStorageController> { public: - const TActorId Source; + const TActorId Source; const TGroupId GroupId; const bool Down; const bool Persist; TString Response; - TTxMonEvent_SetDown(const TActorId& source, TGroupId groupId, bool down, bool persist, TSelf* bsc) + TTxMonEvent_SetDown(const TActorId& source, TGroupId groupId, bool down, bool persist, TSelf* bsc) : TBase(bsc) , Source(source) , GroupId(groupId) @@ -534,11 +534,11 @@ public: class TBlobStorageController::TTxMonEvent_GetDown : public TTransactionBase<TBlobStorageController> { public: - const TActorId Source; + const TActorId Source; const TGroupId GroupId; TString Response; - TTxMonEvent_GetDown(const TActorId& source, TGroupId groupId, TSelf* bsc) + TTxMonEvent_GetDown(const TActorId& source, TGroupId groupId, TSelf* bsc) : TBase(bsc) , Source(source) , GroupId(groupId) @@ -584,16 +584,16 @@ public: }; class TDisableSelfHealActor : public TActorBootstrapped<TDisableSelfHealActor> { - const TActorId MonProxy; + const TActorId MonProxy; const TString Url; public: - TDisableSelfHealActor(TActorId monProxy, TString url) + TDisableSelfHealActor(TActorId monProxy, TString url) : MonProxy(monProxy) , Url(std::move(url)) {} - void Bootstrap(const TActorId& parent) { + void Bootstrap(const TActorId& parent) { auto ev = MakeHolder<TEvBlobStorage::TEvControllerConfigRequest>(); ev->Record.MutableRequest()->AddCommand()->MutableEnableSelfHeal()->SetEnable(false); Send(parent, ev.Release()); diff --git a/ydb/core/mind/bscontroller/self_heal.cpp b/ydb/core/mind/bscontroller/self_heal.cpp index af452f8b105..b732d98e937 100644 --- a/ydb/core/mind/bscontroller/self_heal.cpp +++ b/ydb/core/mind/bscontroller/self_heal.cpp @@ -20,19 +20,19 @@ namespace NKikimr::NBsController { }; class TReassignerActor : public TActorBootstrapped<TReassignerActor> { - const TActorId ControllerId; - TActorId SelfHealId; // filled on bootstrap + const TActorId ControllerId; + TActorId SelfHealId; // filled on bootstrap const TGroupId GroupId; const TEvControllerUpdateSelfHealInfo::TGroupContent Group; const TVDiskID VDiskToReplace; TBlobStorageGroupInfo::TTopology Topology; THolder<TBlobStorageGroupInfo::TGroupVDisks> FailedGroupDisks; THashSet<TVDiskID> PendingVDisks; - THashMap<TActorId, TVDiskID> ActorToDiskMap; + THashMap<TActorId, TVDiskID> ActorToDiskMap; THashMap<TNodeId, TVector<TVDiskID>> NodeToDiskMap; public: - TReassignerActor(TActorId controllerId, TGroupId groupId, TEvControllerUpdateSelfHealInfo::TGroupContent group, + TReassignerActor(TActorId controllerId, TGroupId groupId, TEvControllerUpdateSelfHealInfo::TGroupContent group, TVDiskID vdiskToReplace) : ControllerId(controllerId) , GroupId(groupId) @@ -41,7 +41,7 @@ namespace NKikimr::NBsController { , Topology(Group.Type) {} - void Bootstrap(const TActorId& parent) { + void Bootstrap(const TActorId& parent) { SelfHealId = parent; Become(&TThis::StateFunc, TDuration::Seconds(60), new TEvents::TEvWakeup); @@ -83,7 +83,7 @@ namespace NKikimr::NBsController { // send TEvVStatus message to disk const auto& l = vdisk.Location; - const TActorId& vdiskActorId = MakeBlobStorageVDiskID(l.NodeId, l.PDiskId, l.VSlotId); + const TActorId& vdiskActorId = MakeBlobStorageVDiskID(l.NodeId, l.PDiskId, l.VSlotId); Send(vdiskActorId, new TEvBlobStorage::TEvVStatus(vdiskId), IEventHandle::MakeFlags(0, IEventHandle::FlagTrackDelivery | IEventHandle::FlagSubscribeOnSession)); ActorToDiskMap.emplace(vdiskActorId, vdiskId); @@ -210,14 +210,14 @@ namespace NKikimr::NBsController { struct TGroupRecord { TEvControllerUpdateSelfHealInfo::TGroupContent Content; - TActorId ReassignerActorId; // reassigner in flight + TActorId ReassignerActorId; // reassigner in flight TDuration RetryTimeout = MinRetryTimeout; TInstant NextRetryTimestamp = TInstant::Zero(); THashMap<TVDiskID, TVDiskStatusTracker> VDiskStatus; }; const ui64 TabletId; - TActorId ControllerId; + TActorId ControllerId; THashMap<TGroupId, TGroupRecord> Groups; TSet<TGroupId> GroupsWithFaultyDisks; std::shared_ptr<std::atomic_uint64_t> UnreassignableGroups; @@ -228,7 +228,7 @@ namespace NKikimr::NBsController { , UnreassignableGroups(std::move(unreassignableGroups)) {} - void Bootstrap(const TActorId& parentId) { + void Bootstrap(const TActorId& parentId) { ControllerId = parentId; Become(&TThis::StateFunc); HandleWakeup(); diff --git a/ydb/core/mind/bscontroller/stat_processor.cpp b/ydb/core/mind/bscontroller/stat_processor.cpp index 1e7854933ee..8cfe681dd42 100644 --- a/ydb/core/mind/bscontroller/stat_processor.cpp +++ b/ydb/core/mind/bscontroller/stat_processor.cpp @@ -3,7 +3,7 @@ namespace NKikimr::NBsController { class TStatProcessorActor : public TActorBootstrapped<TStatProcessorActor> { - using TGroupCleanupSchedule = TMultiMap<TInstant, std::pair<TGroupId, TActorId>>; + using TGroupCleanupSchedule = TMultiMap<TInstant, std::pair<TGroupId, TActorId>>; struct TPerGroupRecord { struct TPerAggregatorInfo { @@ -12,13 +12,13 @@ namespace NKikimr::NBsController { }; TGroupStat Accum; - THashMap<TActorId, TPerAggregatorInfo> PerAggegatorInfo; + THashMap<TActorId, TPerAggregatorInfo> PerAggegatorInfo; TGroupLatencyStats Stats; void Update(const NKikimrBlobStorage::TEvGroupStatReport& report, TInstant now, TGroupId groupId, TGroupCleanupSchedule& schedule) { TGroupStat stat; if (stat.Deserialize(report)) { - const TActorId vdiskServiceId = ActorIdFromProto(report.GetVDiskServiceId()); + const TActorId vdiskServiceId = ActorIdFromProto(report.GetVDiskServiceId()); auto& item = PerAggegatorInfo[vdiskServiceId]; Accum.Replace(stat, item.Stat); item.Stat = std::move(stat); @@ -31,7 +31,7 @@ namespace NKikimr::NBsController { } } - void Cleanup(const TActorId& vdiskServiceId) { + void Cleanup(const TActorId& vdiskServiceId) { auto it = PerAggegatorInfo.find(vdiskServiceId); Y_VERIFY(it != PerAggegatorInfo.end()); auto& item = it->second; @@ -48,7 +48,7 @@ namespace NKikimr::NBsController { static constexpr TDuration UpdatePeriod = TDuration::Seconds(10); - TActorId ParentActorId; + TActorId ParentActorId; TMap<TGroupId, TPerGroupRecord> Groups; TSet<TGroupId> UpdatedGroupIds; TGroupCleanupSchedule GroupCleanupSchedule; @@ -89,7 +89,7 @@ namespace NKikimr::NBsController { TGroupCleanupSchedule::iterator it; for (it = GroupCleanupSchedule.begin(); it != GroupCleanupSchedule.end() && it->first <= now; ++it) { TGroupId groupId; - TActorId vdiskServiceId; + TActorId vdiskServiceId; std::tie(groupId, vdiskServiceId) = it->second; auto groupIt = Groups.find(groupId); if (groupIt != Groups.end()) { diff --git a/ydb/core/mind/bscontroller/ut_bscontroller/main.cpp b/ydb/core/mind/bscontroller/ut_bscontroller/main.cpp index 9ff59046707..87f7b18fb35 100644 --- a/ydb/core/mind/bscontroller/ut_bscontroller/main.cpp +++ b/ydb/core/mind/bscontroller/ut_bscontroller/main.cpp @@ -91,7 +91,7 @@ struct TEnvironmentSetup { } NKikimrBlobStorage::TConfigResponse Invoke(const NKikimrBlobStorage::TConfigRequest& request) { - const TActorId self = Runtime->AllocateEdgeActor(); + const TActorId self = Runtime->AllocateEdgeActor(); auto ev = MakeHolder<TEvBlobStorage::TEvControllerConfigRequest>(); ev->Record.MutableRequest()->CopyFrom(request); Runtime->SendToPipe(TabletId, self, ev.Release(), NodeId, GetPipeConfigWithRetries()); @@ -110,7 +110,7 @@ struct TEnvironmentSetup { NKikimrBlobStorage::TEvControllerSelectGroupsResult SelectGroups(const NKikimrBlobStorage::TEvControllerSelectGroups& request) { - const TActorId self = Runtime->AllocateEdgeActor(); + const TActorId self = Runtime->AllocateEdgeActor(); auto ev = MakeHolder<TEvBlobStorage::TEvControllerSelectGroups>(); ev->Record.MergeFrom(request); Runtime->SendToPipe(TabletId, self, ev.Release(), NodeId, GetPipeConfigWithRetries()); @@ -119,8 +119,8 @@ struct TEnvironmentSetup { } TVector<std::tuple<TString, i32, ui32>> GetNodes() { - const TActorId edge = Runtime->AllocateEdgeActor(); - Runtime->Send(new IEventHandle(GetNameserviceActorId(), edge, new TEvInterconnect::TEvListNodes)); + const TActorId edge = Runtime->AllocateEdgeActor(); + Runtime->Send(new IEventHandle(GetNameserviceActorId(), edge, new TEvInterconnect::TEvListNodes)); auto response = Runtime->GrabEdgeEventRethrow<TEvInterconnect::TEvNodesInfo>(edge); TVector<std::tuple<TString, i32, ui32>> res; for (const auto& nodeInfo : response->Get()->Nodes) { @@ -224,7 +224,7 @@ struct TEnvironmentSetup { } void SetupStorage() { - const TActorId proxyId = MakeBlobStorageProxyID(GroupId); + const TActorId proxyId = MakeBlobStorageProxyID(GroupId); Runtime->RegisterService(proxyId, Runtime->Register(CreateBlobStorageGroupProxyMockActor(), NodeId), NodeId); } diff --git a/ydb/core/mind/bscontroller/ut_selfheal/env.h b/ydb/core/mind/bscontroller/ut_selfheal/env.h index 18e0c3293f3..e79bf914a17 100644 --- a/ydb/core/mind/bscontroller/ut_selfheal/env.h +++ b/ydb/core/mind/bscontroller/ut_selfheal/env.h @@ -15,7 +15,7 @@ struct TEnvironmentSetup { const ui32 GroupId = 0; const ui32 NodeId = 1; ui64 NextHostConfigId = 1; - TActorId TimerActor; + TActorId TimerActor; TEnvironmentSetup(ui32 nodeCount) : NodeCount(nodeCount) @@ -168,7 +168,7 @@ struct TEnvironmentSetup { } void SetupStorage() { - const TActorId proxyId = MakeBlobStorageProxyID(GroupId); + const TActorId proxyId = MakeBlobStorageProxyID(GroupId); Runtime->RegisterService(proxyId, Runtime->Register(CreateBlobStorageGroupProxyMockActor(), NodeId)); for (ui32 nodeId : Runtime->GetNodes()) { @@ -190,7 +190,7 @@ struct TEnvironmentSetup { void WaitForNodeWardensToConnect() { std::vector<TActorId> edges; for (ui32 nodeId : Runtime->GetNodes()) { - const TActorId wardenId = MakeBlobStorageNodeWardenID(nodeId); + const TActorId wardenId = MakeBlobStorageNodeWardenID(nodeId); const TActorId edge = Runtime->AllocateEdgeActor(nodeId); Runtime->Send(new IEventHandle(wardenId, edge, new TEvCheckState(EState::CONNECTED)), nodeId); edges.push_back(edge); diff --git a/ydb/core/mind/bscontroller/ut_selfheal/node_warden_mock.h b/ydb/core/mind/bscontroller/ut_selfheal/node_warden_mock.h index 7a3c9f49737..5c993ba0fb3 100644 --- a/ydb/core/mind/bscontroller/ut_selfheal/node_warden_mock.h +++ b/ydb/core/mind/bscontroller/ut_selfheal/node_warden_mock.h @@ -24,10 +24,10 @@ struct TEvUpdateDriveStatus : TEventLocal<TEvUpdateDriveStatus, EvUpdateDriveSta class TNodeWardenMock : public TActorBootstrapped<TNodeWardenMock> { const ui32 NodeId; const ui64 TabletId; - TActorId PipeClient; + TActorId PipeClient; bool Connected = false; EState CurrentState = EState::INITIAL; - std::multimap<EState, TActorId> Queue; + std::multimap<EState, TActorId> Queue; std::map<ui32, ui32> Groups; std::map<std::tuple<ui32, ui32>, std::unique_ptr<TVDisk>> VDisks; @@ -145,7 +145,7 @@ public: } else { VDisks.emplace_hint(it, id, std::make_unique<TVDisk>(vdiskId, location.GetNodeID(), location.GetPDiskID(), location.GetVDiskSlotID(), location.GetPDiskGuid())); - } + } } else if (const auto it = VDisks.find(id); it != VDisks.end()) { it->second->StopActor(); VDisks.erase(it); diff --git a/ydb/core/mind/bscontroller/ut_selfheal/timer_actor.h b/ydb/core/mind/bscontroller/ut_selfheal/timer_actor.h index 3fcdd888674..2bbf215c7a1 100644 --- a/ydb/core/mind/bscontroller/ut_selfheal/timer_actor.h +++ b/ydb/core/mind/bscontroller/ut_selfheal/timer_actor.h @@ -16,7 +16,7 @@ struct TEvTimer : TEventLocal<TEvTimer, EvTimer> { std::unique_ptr<IEventHandle> HitEvent; TEvTimer(const IEventHandle& ev) - : HitEvent(new IEventHandle(TEvents::TSystem::Wakeup, 0, ev.Sender, TActorId(), {}, ev.Cookie)) + : HitEvent(new IEventHandle(TEvents::TSystem::Wakeup, 0, ev.Sender, TActorId(), {}, ev.Cookie)) {} void Hit() { diff --git a/ydb/core/mind/bscontroller/ut_selfheal/vdisk_mock.h b/ydb/core/mind/bscontroller/ut_selfheal/vdisk_mock.h index 35290198470..dddd4728ac1 100644 --- a/ydb/core/mind/bscontroller/ut_selfheal/vdisk_mock.h +++ b/ydb/core/mind/bscontroller/ut_selfheal/vdisk_mock.h @@ -68,7 +68,7 @@ public: } } - TActorId GetActorId() const { + TActorId GetActorId() const { return MakeBlobStorageVDiskID(NodeId, PDiskId, VSlotId); } diff --git a/ydb/core/mind/configured_tablet_bootstrapper.cpp b/ydb/core/mind/configured_tablet_bootstrapper.cpp index 007d5e3a290..008acd2b400 100644 --- a/ydb/core/mind/configured_tablet_bootstrapper.cpp +++ b/ydb/core/mind/configured_tablet_bootstrapper.cpp @@ -30,7 +30,7 @@ namespace NKikimr { class TConfiguredTabletBootstrapper : public TActorBootstrapped<TConfiguredTabletBootstrapper> { const ui64 TabletId; const ::NKikimrConfig::TBootstrap::TTablet DefaultConfig; - TActorId BootstrapperInstance; + TActorId BootstrapperInstance; TString CurrentConfig; void Handle(NConsole::TEvConsole::TEvConfigNotificationRequest::TPtr &ev) { @@ -58,8 +58,8 @@ class TConfiguredTabletBootstrapper : public TActorBootstrapped<TConfiguredTable if (BootstrapperInstance) { Send(BootstrapperInstance, new TEvents::TEvPoisonPill()); - TlsActivationContext->ExecutorThread.ActorSystem->RegisterLocalService(MakeBootstrapperID(TabletId, SelfId().NodeId()), TActorId()); - BootstrapperInstance = TActorId(); + TlsActivationContext->ExecutorThread.ActorSystem->RegisterLocalService(MakeBootstrapperID(TabletId, SelfId().NodeId()), TActorId()); + BootstrapperInstance = TActorId(); } CurrentConfig = x; diff --git a/ydb/core/mind/dynamic_nameserver.cpp b/ydb/core/mind/dynamic_nameserver.cpp index 1015379104d..6f4f147982a 100644 --- a/ydb/core/mind/dynamic_nameserver.cpp +++ b/ydb/core/mind/dynamic_nameserver.cpp @@ -301,7 +301,7 @@ void TDynamicNameserver::UpdateState(const NKikimrNodeBroker::TNodesInfo &rec, void TDynamicNameserver::OnPipeDestroyed(ui32 domain, const TActorContext &ctx) { - NodeBrokerPipes[domain] = TActorId(); + NodeBrokerPipes[domain] = TActorId(); PendingRequestAnswered(domain, ctx); if (EpochUpdates.contains(domain)) { diff --git a/ydb/core/mind/dynamic_nameserver_impl.h b/ydb/core/mind/dynamic_nameserver_impl.h index bda36bd9603..ae0e1889511 100644 --- a/ydb/core/mind/dynamic_nameserver_impl.h +++ b/ydb/core/mind/dynamic_nameserver_impl.h @@ -73,7 +73,7 @@ public: return NKikimrServices::TActivity::NAMESERVICE; } - TDynamicNodeResolverBase(TActorId owner, ui32 nodeId, TDynamicConfigPtr config, + TDynamicNodeResolverBase(TActorId owner, ui32 nodeId, TDynamicConfigPtr config, TAutoPtr<IEventHandle> origRequest, TInstant deadline) : Owner(owner) , NodeId(nodeId) @@ -111,19 +111,19 @@ private: void Handle(TEvTabletPipe::TEvClientConnected::TPtr &ev, const TActorContext &ctx); protected: - TActorId Owner; + TActorId Owner; ui32 NodeId; TDynamicConfigPtr Config; TAutoPtr<IEventHandle> OrigRequest; const TInstant Deadline; private: - TActorId NodeBrokerPipe; + TActorId NodeBrokerPipe; }; class TDynamicNodeResolver : public TDynamicNodeResolverBase { public: - TDynamicNodeResolver(TActorId owner, ui32 nodeId, TDynamicConfigPtr config, + TDynamicNodeResolver(TActorId owner, ui32 nodeId, TDynamicConfigPtr config, TAutoPtr<IEventHandle> origRequest, TInstant deadline) : TDynamicNodeResolverBase(owner, nodeId, config, origRequest, deadline) { @@ -135,7 +135,7 @@ public: class TDynamicNodeSearcher : public TDynamicNodeResolverBase { public: - TDynamicNodeSearcher(TActorId owner, ui32 nodeId, TDynamicConfigPtr config, + TDynamicNodeSearcher(TActorId owner, ui32 nodeId, TDynamicConfigPtr config, TAutoPtr<IEventHandle> origRequest, TInstant deadline) : TDynamicNodeResolverBase(owner, nodeId, config, origRequest, deadline) { @@ -241,8 +241,8 @@ private: private: TIntrusivePtr<TTableNameserverSetup> StaticConfig; std::array<TDynamicConfigPtr, DOMAINS_COUNT> DynamicConfigs; - TVector<TActorId> ListNodesQueue; - std::array<TActorId, DOMAINS_COUNT> NodeBrokerPipes; + TVector<TActorId> ListNodesQueue; + std::array<TActorId, DOMAINS_COUNT> NodeBrokerPipes; // When ListNodes requests are sent to NodeBroker tablets this // bitmap indicates domains which didn't answer yet. TBitMap<DOMAINS_COUNT> PendingRequests; diff --git a/ydb/core/mind/dynamic_nameserver_mon.cpp b/ydb/core/mind/dynamic_nameserver_mon.cpp index 2a2e0f23d60..65a3b9664e4 100644 --- a/ydb/core/mind/dynamic_nameserver_mon.cpp +++ b/ydb/core/mind/dynamic_nameserver_mon.cpp @@ -3,7 +3,7 @@ #include <ydb/core/base/appdata.h> #include <ydb/core/base/location.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/generic/set.h> diff --git a/ydb/core/mind/hive/fill.cpp b/ydb/core/mind/hive/fill.cpp index 90d78966f78..f90fa56f422 100644 --- a/ydb/core/mind/hive/fill.cpp +++ b/ydb/core/mind/hive/fill.cpp @@ -15,7 +15,7 @@ protected: ui32 KickInFlight; ui32 Movements; TNodeId NodeId; - TActorId Initiator; + TActorId Initiator; TString GetLogPrefix() const { return Hive->GetLogPrefix(); @@ -83,7 +83,7 @@ public: return NKikimrServices::TActivity::HIVE_BALANCER_ACTOR; } - THiveFill(THive* hive, TNodeId nodeId, const TActorId& initiator) + THiveFill(THive* hive, TNodeId nodeId, const TActorId& initiator) : Hive(hive) , NextKick(Tablets.end()) , KickInFlight(0) @@ -128,7 +128,7 @@ public: } }; -void THive::StartHiveFill(TNodeId nodeId, const TActorId& initiator) { +void THive::StartHiveFill(TNodeId nodeId, const TActorId& initiator) { if (BalancerNodes.emplace(nodeId).second) { auto* balancer = new THiveFill(this, nodeId, initiator); SubActors.emplace_back(balancer); diff --git a/ydb/core/mind/hive/hive_impl.cpp b/ydb/core/mind/hive/hive_impl.cpp index 34be947bd1b..58ca2ac3e43 100644 --- a/ydb/core/mind/hive/hive_impl.cpp +++ b/ydb/core/mind/hive/hive_impl.cpp @@ -119,9 +119,9 @@ void THive::Handle(TEvTabletPipe::TEvServerDisconnected::TPtr& ev) { void THive::Handle(TEvLocal::TEvRegisterNode::TPtr& ev) { NKikimrLocal::TEvRegisterNode& record = ev->Get()->Record; if (record.GetHiveId() == TabletID()) { - const TActorId &local = ev->Sender; + const TActorId &local = ev->Sender; BLOG_D("Handle TEvLocal::TEvRegisterNode from " << ev->Sender << " " << record.ShortDebugString()); - Send(GetNameserviceActorId(), new TEvInterconnect::TEvGetNode(ev->Sender.NodeId())); + Send(GetNameserviceActorId(), new TEvInterconnect::TEvGetNode(ev->Sender.NodeId())); Execute(CreateRegisterNode(local, std::move(record))); } else { BLOG_W("Handle incorrect TEvLocal::TEvRegisterNode from " << ev->Sender << " " << record.ShortDebugString()); @@ -142,7 +142,7 @@ bool THive::OnRenderAppHtmlPage(NMon::TEvRemoteHttpInfo::TPtr ev, const TActorCo void THive::Handle(TEvHive::TEvStopTablet::TPtr& ev) { BLOG_D("Handle StopTablet"); NKikimrHive::TEvStopTablet& rec = ev->Get()->Record; - const TActorId actorToNotify = rec.HasActorToNotify() ? ActorIdFromProto(rec.GetActorToNotify()) : ev->Sender; + const TActorId actorToNotify = rec.HasActorToNotify() ? ActorIdFromProto(rec.GetActorToNotify()) : ev->Sender; if (rec.HasTabletID()) { } else { @@ -544,12 +544,12 @@ void THive::Cleanup() { if (BSControllerPipeClient) { NTabletPipe::CloseClient(SelfId(), BSControllerPipeClient); - BSControllerPipeClient = TActorId(); + BSControllerPipeClient = TActorId(); } if (RootHivePipeClient) { NTabletPipe::CloseClient(SelfId(), RootHivePipeClient); - RootHivePipeClient = TActorId(); + RootHivePipeClient = TActorId(); } if (ResponsivenessPinger) { @@ -607,7 +607,7 @@ void THive::Handle(TEvHive::TEvTabletMetrics::TPtr& ev) { void THive::Handle(TEvInterconnect::TEvNodeConnected::TPtr &ev) { TNodeId nodeId = ev->Get()->NodeId; BLOG_W("Handle TEvInterconnect::TEvNodeConnected, NodeId " << nodeId); - Send(GetNameserviceActorId(), new TEvInterconnect::TEvGetNode(nodeId)); + Send(GetNameserviceActorId(), new TEvInterconnect::TEvGetNode(nodeId)); } void THive::Handle(TEvInterconnect::TEvNodeDisconnected::TPtr &ev) { @@ -927,7 +927,7 @@ void THive::RestartBSControllerPipe() { BLOG_D("THive::RestartBSControllerPipe"); if (BSControllerPipeClient) { NTabletPipe::CloseClient(SelfId(), BSControllerPipeClient); - BSControllerPipeClient = TActorId(); + BSControllerPipeClient = TActorId(); } RequestPoolsInformation(); for (auto it = Tablets.begin(); it != Tablets.end(); ++it) { @@ -943,7 +943,7 @@ void THive::RestartRootHivePipe() { BLOG_D("THive::RestartRootHivePipe"); if (RootHivePipeClient) { NTabletPipe::CloseClient(SelfId(), RootHivePipeClient); - RootHivePipeClient = TActorId(); + RootHivePipeClient = TActorId(); } // trying to retry for free sequence request if (RequestingSequenceNow) { @@ -2148,11 +2148,11 @@ THolder<NKikimrBlobStorage::TEvControllerSelectGroups::TGroupParameters> THive:: return groupParameters; } -void THive::ExecuteStartTablet(TFullTabletId tabletId, const TActorId& local, ui64 cookie, bool external) { +void THive::ExecuteStartTablet(TFullTabletId tabletId, const TActorId& local, ui64 cookie, bool external) { Execute(CreateStartTablet(tabletId, local, cookie, external)); } -void THive::SendPing(const TActorId& local, TNodeId id) { +void THive::SendPing(const TActorId& local, TNodeId id) { Send(local, new TEvLocal::TEvPing(HiveId, HiveGeneration, @@ -2162,7 +2162,7 @@ void THive::SendPing(const TActorId& local, TNodeId id) { id); } -void THive::SendReconnect(const TActorId& local) { +void THive::SendReconnect(const TActorId& local) { Send(local, new TEvLocal::TEvReconnect(HiveId, HiveGeneration)); } @@ -2184,7 +2184,7 @@ void THive::UpdateRegisteredDataCenters(TDataCenterId dataCenterId) { } } -THive::THive(TTabletStorageInfo *info, const TActorId &tablet) +THive::THive(TTabletStorageInfo *info, const TActorId &tablet) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, new NMiniKQL::TMiniKQLFactory) , HiveUid(Max<ui32>()) @@ -2337,12 +2337,12 @@ void THive::KickTablet(const TTabletInfo& tablet) { Send(SelfId(), new TEvPrivate::TEvKickTablet(tablet)); } -void THive::StopTablet(const TActorId& local, const TTabletInfo& tablet) { +void THive::StopTablet(const TActorId& local, const TTabletInfo& tablet) { BLOG_D("Sending TEvStopTablet(" << tablet.ToString() << ") to node " << local.NodeId()); Send(local, new TEvLocal::TEvStopTablet(tablet.GetFullTabletId())); } -void THive::StopTablet(const TActorId& local, TFullTabletId tabletId) { +void THive::StopTablet(const TActorId& local, TFullTabletId tabletId) { BLOG_D("Sending TEvStopTablet(" << tabletId << ") to node " << local.NodeId()); Send(local, new TEvLocal::TEvStopTablet(tabletId)); } @@ -2624,7 +2624,7 @@ TString THive::GetLogPrefix() const { } // NHive -IActor* CreateDefaultHive(const TActorId &tablet, TTabletStorageInfo *info) { +IActor* CreateDefaultHive(const TActorId &tablet, TTabletStorageInfo *info) { return new NHive::THive(info, tablet); } diff --git a/ydb/core/mind/hive/hive_impl.h b/ydb/core/mind/hive/hive_impl.h index 627aeb64458..70c4530fee0 100644 --- a/ydb/core/mind/hive/hive_impl.h +++ b/ydb/core/mind/hive/hive_impl.h @@ -118,14 +118,14 @@ namespace NHive { struct TCompleteNotifications { TVector<THolder<IEventHandle>> Notifications; - TActorId SelfID; + TActorId SelfID; - void Reset(const TActorId &selfId) { + void Reset(const TActorId &selfId) { Notifications.clear(); SelfID = selfId; } - void Send(const TActorId &recipient, IEventBase *ev, ui32 flags = 0, ui64 cookie = 0) { + void Send(const TActorId &recipient, IEventBase *ev, ui32 flags = 0, ui64 cookie = 0) { Notifications.emplace_back(new IEventHandle(recipient, SelfID, ev, flags, cookie)); } @@ -239,7 +239,7 @@ protected: void StartHiveBalancer(int maxMovements = 0, bool recheckOnFinish = false, const std::vector<TNodeId>& filterNodeIds = {}); void StartHiveDrain(TNodeId nodeId, TDrainSettings settings); - void StartHiveFill(TNodeId nodeId, const TActorId& initiator); + void StartHiveFill(TNodeId nodeId, const TActorId& initiator); void CreateEvMonitoring(NMon::TEvRemoteHttpInfo::TPtr& ev, const TActorContext& ctx); ITransaction* CreateDeleteTablet(TEvHive::TEvDeleteTablet::TPtr& ev); ITransaction* CreateDeleteOwnerTablets(TEvHive::TEvDeleteOwnerTablets::TPtr& ev); @@ -249,11 +249,11 @@ protected: ITransaction* CreateRestartTablet(TFullTabletId tabletId); ITransaction* CreateRestartTablet(TFullTabletId tabletId, TNodeId preferredNodeId); ITransaction* CreateInitScheme(); - ITransaction* CreateAdoptTablet(NKikimrHive::TEvAdoptTablet &rec, const TActorId &sender, const ui64 cookie); - ITransaction* CreateCreateTablet(NKikimrHive::TEvCreateTablet rec, const TActorId& sender, const ui64 cookie); + ITransaction* CreateAdoptTablet(NKikimrHive::TEvAdoptTablet &rec, const TActorId &sender, const ui64 cookie); + ITransaction* CreateCreateTablet(NKikimrHive::TEvCreateTablet rec, const TActorId& sender, const ui64 cookie); ITransaction* CreateLoadEverything(); - ITransaction* CreateRegisterNode(const TActorId& local, NKikimrLocal::TEvRegisterNode rec); - ITransaction* CreateStatus(const TActorId& local, NKikimrLocal::TEvStatus rec); + ITransaction* CreateRegisterNode(const TActorId& local, NKikimrLocal::TEvRegisterNode rec); + ITransaction* CreateStatus(const TActorId& local, NKikimrLocal::TEvStatus rec); ITransaction* CreateUpdateTabletStatus(TTabletId tabletId, const TActorId &local, ui32 generation, @@ -264,14 +264,14 @@ protected: ITransaction* CreateKillNode(TNodeId nodeId, const TActorId& local); ITransaction* CreateUpdateTabletGroups(TTabletId tabletId, TVector<NKikimrBlobStorage::TEvControllerSelectGroupsResult::TGroupParameters> groups = {}); ITransaction* CreateCheckTablets(); - ITransaction* CreateSyncTablets(const TActorId &local, NKikimrLocal::TEvSyncTablets& rec); - ITransaction* CreateStopTablet(TTabletId tabletId, const TActorId& actorToNotify); + ITransaction* CreateSyncTablets(const TActorId &local, NKikimrLocal::TEvSyncTablets& rec); + ITransaction* CreateStopTablet(TTabletId tabletId, const TActorId& actorToNotify); ITransaction* CreateResumeTablet(TTabletId tabletId, const TActorId& actorToNotify); - ITransaction* CreateStartTablet(TFullTabletId tabletId, const TActorId& local, ui64 cookie, bool external = false); + ITransaction* CreateStartTablet(TFullTabletId tabletId, const TActorId& local, ui64 cookie, bool external = false); ITransaction* CreateUpdateTabletMetrics(TEvHive::TEvTabletMetrics::TPtr& ev); - ITransaction* CreateReassignGroups(TTabletId tabletId, const TActorId& actorToNotify, const std::bitset<MAX_TABLET_CHANNELS>& channelProfileNewGroup); - ITransaction* CreateLockTabletExecution(const NKikimrHive::TEvLockTabletExecution& rec, const TActorId& sender, const ui64 cookie); - ITransaction* CreateUnlockTabletExecution(const NKikimrHive::TEvUnlockTabletExecution& rec, const TActorId& sender, const ui64 cookie); + ITransaction* CreateReassignGroups(TTabletId tabletId, const TActorId& actorToNotify, const std::bitset<MAX_TABLET_CHANNELS>& channelProfileNewGroup); + ITransaction* CreateLockTabletExecution(const NKikimrHive::TEvLockTabletExecution& rec, const TActorId& sender, const ui64 cookie); + ITransaction* CreateUnlockTabletExecution(const NKikimrHive::TEvUnlockTabletExecution& rec, const TActorId& sender, const ui64 cookie); ITransaction* CreateUnlockTabletExecution(ui64 tabletId, ui64 seqNo); ITransaction* CreateRequestTabletSequence(TEvHive::TEvRequestTabletIdSequence::TPtr event); ITransaction* CreateResponseTabletSequence(TEvHive::TEvResponseTabletIdSequence::TPtr event); @@ -291,8 +291,8 @@ public: TDomainsView DomainsView; protected: - TActorId BSControllerPipeClient; - TActorId RootHivePipeClient; + TActorId BSControllerPipeClient; + TActorId RootHivePipeClient; ui64 HiveUid; // Hive Personal Identifier - identifies a unique individual hive ui32 HiveDomain; TTabletId RootHiveId; @@ -367,7 +367,7 @@ protected: i32 MigrationProgress = 0; NKikimrHive::TEvSeizeTablets MigrationFilter; - TActorId ResponsivenessActorID; + TActorId ResponsivenessActorID; TTabletResponsivenessPinger *ResponsivenessPinger; // remove after upgrade to sub hives ui64 NextTabletId = 0x10000; @@ -391,7 +391,7 @@ protected: struct TPendingCreateTablet { NKikimrHive::TEvCreateTablet CreateTablet; - TActorId Sender; + TActorId Sender; ui64 Cookie; }; @@ -507,7 +507,7 @@ public: return NKikimrServices::TActivity::HIVE_ACTOR; } - THive(TTabletStorageInfo *info, const TActorId &tablet); + THive(TTabletStorageInfo *info, const TActorId &tablet); protected: STATEFN(StateInit); @@ -600,16 +600,16 @@ public: NKikimr::NHive::TResourceRawValues deltaRaw, NKikimr::NHive::TResourceNormalizedValues deltaNormalized); static void FillTabletInfo(NKikimrHive::TEvResponseHiveInfo& response, ui64 tabletId, const TLeaderTabletInfo* info, const NKikimrHive::TEvRequestHiveInfo& req); - void ExecuteStartTablet(TFullTabletId tabletId, const TActorId& local, ui64 cookie, bool external); + void ExecuteStartTablet(TFullTabletId tabletId, const TActorId& local, ui64 cookie, bool external); ui32 GetDataCenters(); ui32 GetRegisteredDataCenters(); void UpdateRegisteredDataCenters(TDataCenterId dataCenterId); - void SendPing(const TActorId& local, TNodeId id); - void SendReconnect(const TActorId& local); + void SendPing(const TActorId& local, TNodeId id); + void SendReconnect(const TActorId& local); static THolder<NKikimrBlobStorage::TEvControllerSelectGroups::TGroupParameters> BuildGroupParametersForChannel(const TLeaderTabletInfo& tablet, ui32 channelId); void KickTablet(const TTabletInfo& tablet); - void StopTablet(const TActorId& local, const TTabletInfo& tablet); - void StopTablet(const TActorId& local, TFullTabletId tabletId); + void StopTablet(const TActorId& local, const TTabletInfo& tablet); + void StopTablet(const TActorId& local, TFullTabletId tabletId); void RunProcessBootQueue(); TTabletMetricsAggregates DefaultResourceMetricsAggregates; diff --git a/ydb/core/mind/hive/hive_impl_ut.cpp b/ydb/core/mind/hive/hive_impl_ut.cpp index 963cbd6275f..077c3eab24c 100644 --- a/ydb/core/mind/hive/hive_impl_ut.cpp +++ b/ydb/core/mind/hive/hive_impl_ut.cpp @@ -30,7 +30,7 @@ Y_UNIT_TEST_SUITE(THiveImplTest) { TIntrusivePtr<TTabletStorageInfo> hiveStorage = new TTabletStorageInfo; hiveStorage->TabletType = TTabletTypes::Hive; - THive hive(hiveStorage.Get(), TActorId()); + THive hive(hiveStorage.Get(), TActorId()); std::unordered_map<ui64, TLeaderTabletInfo> tablets; THPTimer timer; diff --git a/ydb/core/mind/hive/hive_schema.h b/ydb/core/mind/hive/hive_schema.h index a4dfa473d93..6289499cc31 100644 --- a/ydb/core/mind/hive/hive_schema.h +++ b/ydb/core/mind/hive/hive_schema.h @@ -74,10 +74,10 @@ struct Schema : NIceDb::Schema { struct AllowedDataCenters : Column<17, NScheme::NTypeIds::String> { using Type = TVector<ui32>; }; struct TabletStorageVersion : Column<18, NScheme::NTypeIds::Uint32> { static constexpr ui32 Default = 0; }; struct ObjectID : Column<19, NScheme::NTypeIds::Uint64> { using Type = TObjectId; }; - struct ActorsToNotify : Column<111, NScheme::NTypeIds::String> { using Type = TVector<TActorId>; }; + struct ActorsToNotify : Column<111, NScheme::NTypeIds::String> { using Type = TVector<TActorId>; }; struct AllowedDomains : Column<112, NScheme::NTypeIds::String> { using Type = TVector<TSubDomainKey>; }; //order sets priority struct BootMode : Column<113, NScheme::NTypeIds::Uint64> { using Type = NKikimrHive::ETabletBootMode; static constexpr NKikimrHive::ETabletBootMode Default = NKikimrHive::TABLET_BOOT_MODE_DEFAULT; }; - struct LockedToActor : Column<114, NScheme::NTypeIds::String> { using Type = TActorId; }; + struct LockedToActor : Column<114, NScheme::NTypeIds::String> { using Type = TActorId; }; struct LockedReconnectTimeout : Column<115, NScheme::NTypeIds::Uint64> { static constexpr ui64 Default = 0; }; struct ObjectDomain : Column<116, NScheme::NTypeIds::String> { using Type = NKikimrSubDomains::TDomainKey; }; diff --git a/ydb/core/mind/hive/hive_statics.cpp b/ydb/core/mind/hive/hive_statics.cpp index 9aab1aeb837..42036789565 100644 --- a/ydb/core/mind/hive/hive_statics.cpp +++ b/ydb/core/mind/hive/hive_statics.cpp @@ -1,7 +1,7 @@ #include "hive_impl.h" -#include <library/cpp/json/json_value.h> - +#include <library/cpp/json/json_value.h> + namespace NKikimr { namespace NHive { diff --git a/ydb/core/mind/hive/hive_ut.cpp b/ydb/core/mind/hive/hive_ut.cpp index 772eef498c9..0383d49a123 100644 --- a/ydb/core/mind/hive/hive_ut.cpp +++ b/ydb/core/mind/hive/hive_ut.cpp @@ -236,7 +236,7 @@ namespace { void EnableSchedule(TTestActorRuntime &runtime, bool isLocalEnabled) { for (ui32 nodeIndex = 0; nodeIndex < runtime.GetNodeCount(); ++nodeIndex) { if (isLocalEnabled) { - TActorId localActor = runtime.GetLocalServiceId(MakeLocalID(runtime.GetNodeId(nodeIndex)), nodeIndex); + TActorId localActor = runtime.GetLocalServiceId(MakeLocalID(runtime.GetNodeId(nodeIndex)), nodeIndex); runtime.EnableScheduleForActor(localActor, true); } runtime.EnableScheduleForActor(runtime.GetLocalServiceId(MakeBlobStorageNodeWardenID(runtime.GetNodeId(nodeIndex)), nodeIndex), true); @@ -290,7 +290,7 @@ namespace { } void SetupBoxAndStoragePool(TTestActorRuntime &runtime, ui32 numGroups = 1, const TString& storagePoolNamePrefix = STORAGE_POOL) { - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); ui32 domainId = 0; ui32 nodeIndex = 0; TString pDiskPath; @@ -303,7 +303,7 @@ namespace { NTabletPipe::TClientConfig pipeConfig; pipeConfig.RetryPolicy = NTabletPipe::TClientRetryPolicy::WithRetries(); - runtime.Send(new IEventHandle(GetNameserviceActorId(), sender, new TEvInterconnect::TEvListNodes)); + runtime.Send(new IEventHandle(GetNameserviceActorId(), sender, new TEvInterconnect::TEvListNodes)); TAutoPtr<IEventHandle> handleNodesInfo; auto nodesInfo = runtime.GrabEdgeEventRethrow<TEvInterconnect::TEvNodesInfo>(handleNodesInfo); @@ -490,8 +490,8 @@ Y_UNIT_TEST_SUITE(THiveTest) { } void SendToLocal(TTestActorRuntime &runtime, ui32 nodeIndex, IEventBase* event) { - TActorId local = MakeLocalID(runtime.GetNodeId(nodeIndex)); - runtime.Send(new IEventHandle(local, TActorId(), event), nodeIndex); + TActorId local = MakeLocalID(runtime.GetNodeId(nodeIndex)); + runtime.Send(new IEventHandle(local, TActorId(), event), nodeIndex); } void SendKillLocal(TTestActorRuntime &runtime, ui32 nodeIndex) { @@ -508,7 +508,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { ui64 SendCreateTestTablet(TTestActorRuntime &runtime, ui64 hiveTablet, ui64 testerTablet, THolder<TEvHive::TEvCreateTablet> ev, ui32 nodeIndex, bool doWaitForResult, NKikimrProto::EReplyStatus expectedStatus = NKikimrProto::OK) { - TActorId senderB = runtime.AllocateEdgeActor(nodeIndex); + TActorId senderB = runtime.AllocateEdgeActor(nodeIndex); runtime.SendToPipe(hiveTablet, senderB, ev.Release(), 0, GetPipeConfigWithRetries()); TAutoPtr<IEventHandle> handle; auto createTabletReply = runtime.GrabEdgeEventRethrow<TEvHive::TEvCreateTabletReply>(handle); @@ -541,7 +541,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { } return prevObserverFunc(runtime, event); }); - TActorId senderB = runtime.AllocateEdgeActor(nodeIndex); + TActorId senderB = runtime.AllocateEdgeActor(nodeIndex); runtime.SendToPipe(hiveTablet, senderB, ev.Release(), 0, GetPipeConfigWithRetries()); TAutoPtr<IEventHandle> handle; auto deleteTabletReply = runtime.GrabEdgeEventRethrow<TEvHive::TEvDeleteTabletReply>(handle); @@ -566,7 +566,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { } return prevObserverFunc(runtime, event); }); - TActorId senderB = runtime.AllocateEdgeActor(nodeIndex); + TActorId senderB = runtime.AllocateEdgeActor(nodeIndex); runtime.SendToPipe(hiveTablet, senderB, ev.Release(), 0, GetPipeConfigWithRetries()); TAutoPtr<IEventHandle> handle; auto deleteTabletReply = runtime.GrabEdgeEventRethrow<TEvHive::TEvDeleteOwnerTabletsReply>(handle); @@ -588,7 +588,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { } void SendStopTablet(TTestActorRuntime &runtime, ui64 hiveTablet, ui64 tabletId, ui32 nodeIndex) { - TActorId senderB = runtime.AllocateEdgeActor(nodeIndex); + TActorId senderB = runtime.AllocateEdgeActor(nodeIndex); runtime.SendToPipe(hiveTablet, senderB, new TEvHive::TEvStopTablet(tabletId), 0, GetPipeConfigWithRetries()); TAutoPtr<IEventHandle> handle; auto stopTabletResult = runtime.GrabEdgeEventRethrow<TEvHive::TEvStopTabletResult>(handle); @@ -604,7 +604,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { ui64 tabletId, const TVector<ui32>& channels, ui32 nodeIndex) { - TActorId senderB = runtime.AllocateEdgeActor(nodeIndex); + TActorId senderB = runtime.AllocateEdgeActor(nodeIndex); runtime.SendToPipe(hiveTablet, senderB, new TEvHive::TEvReassignTablet(tabletId, channels), 0, GetPipeConfigWithRetries()); } @@ -618,7 +618,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { } void MakeSureTabletIsDown(TTestActorRuntime &runtime, ui64 tabletId, ui32 nodeIndex) { - TActorId sender = runtime.AllocateEdgeActor(nodeIndex); + TActorId sender = runtime.AllocateEdgeActor(nodeIndex); runtime.ConnectToPipe(tabletId, sender, nodeIndex, NTabletPipe::TClientConfig()); bool isException = false; TEvTabletPipe::TEvClientConnected* clientConnectedResult; @@ -644,7 +644,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { TTenantPoolConfig::TPtr tenantPoolConfig = new TTenantPoolConfig(localConfig); tenantPoolConfig->AddStaticSlot(DOMAIN_NAME); - TActorId actorId = runtime.Register( + TActorId actorId = runtime.Register( CreateTenantPool(tenantPoolConfig), nodeIndex, 0, TMailboxType::Revolving, 0); runtime.EnableScheduleForActor(actorId, true); runtime.RegisterService(MakeTenantPoolRootID(), actorId, nodeIndex); @@ -670,7 +670,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { } void MakeSureTabletIsUp(TTestActorRuntime &runtime, ui64 tabletId, ui32 nodeIndex, NTabletPipe::TClientConfig* pipeConfig = nullptr, bool* roleConnected = nullptr) { - TActorId sender = runtime.AllocateEdgeActor(nodeIndex); + TActorId sender = runtime.AllocateEdgeActor(nodeIndex); runtime.ConnectToPipe(tabletId, sender, nodeIndex, pipeConfig ? *pipeConfig : GetPipeConfigWithRetries()); for(;;) { TAutoPtr<IEventHandle> handle; @@ -686,7 +686,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { } void MakeSureTheTabletIsDeleted(TTestActorRuntime &runtime, ui64 hiveTablet, ui64 tabletId) { - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); runtime.SendToPipe(hiveTablet, sender, new TEvHive::TEvRequestHiveInfo(true)); TAutoPtr<IEventHandle> handle; TEvHive::TEvResponseHiveInfo* response = runtime.GrabEdgeEventRethrow<TEvHive::TEvResponseHiveInfo>(handle); @@ -702,7 +702,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { NTabletPipe::TClientConfig* pipeConfig = nullptr, bool* roleConnected = nullptr, ui32 maxAttempts = 10) { - TActorId sender = runtime.AllocateEdgeActor(nodeIndex); + TActorId sender = runtime.AllocateEdgeActor(nodeIndex); ui32 attempts = 0; runtime.ConnectToPipe(tabletId, sender, nodeIndex, pipeConfig ? *pipeConfig : GetPipeConfigWithRetries()); for(;;) { @@ -733,7 +733,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { NTabletPipe::TClientConfig* pipeConfig = nullptr, bool* roleConnected = nullptr, ui32 maxAttempts = 10) { - TActorId sender = runtime.AllocateEdgeActor(nodeIndex); + TActorId sender = runtime.AllocateEdgeActor(nodeIndex); ui32 attempts = 0; runtime.ConnectToPipe(tabletId, sender, nodeIndex, pipeConfig ? *pipeConfig : GetPipeConfigWithRetries()); for(;;) { @@ -817,7 +817,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { ui64 tabletId = SendCreateTestTablet(runtime, hiveTablet, testerTablet, MakeHolder<TEvHive::TEvCreateTablet>(testerTablet, 0, tabletType, BINDED_CHANNELS), 0, true); MakeSureTabletIsUp(runtime, tabletId, 0); THolder<TEvHive::TEvDeleteOwnerTablets> deleteOwner = MakeHolder<TEvHive::TEvDeleteOwnerTablets>(testerTablet, 1); - TActorId senderB = runtime.AllocateEdgeActor(0); + TActorId senderB = runtime.AllocateEdgeActor(0); runtime.SendToPipe(hiveTablet, senderB, deleteOwner.Release(), 0, GetPipeConfigWithRetries()); TAutoPtr<IEventHandle> handle; auto deleteTabletsReply = runtime.GrabEdgeEventRethrow<TEvHive::TEvDeleteOwnerTabletsReply>(handle); @@ -838,7 +838,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { MakeSureTabletIsUp(runtime, hiveTablet, 0); TTabletTypes::EType tabletType = TTabletTypes::Dummy; TVector<TTabletId> tablets; - TActorId senderB = runtime.AllocateEdgeActor(0); + TActorId senderB = runtime.AllocateEdgeActor(0); for (int i = 0; i < 100; ++i) { runtime.SendToPipe(hiveTablet, senderB, new TEvHive::TEvCreateTablet(testerTablet, i, tabletType, BINDED_CHANNELS), 0, GetPipeConfigWithRetries()); } @@ -931,7 +931,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { MakeSureTabletIsUp(runtime, hiveTablet, 0); // root hive good MakeSureTabletIsUp(runtime, TTestTxConfig::SchemeShard, 0); // root ss good - TActorId sender = runtime.AllocateEdgeActor(0); + TActorId sender = runtime.AllocateEdgeActor(0); InitSchemeRoot(runtime, sender); TSubDomainKey subdomainKey; @@ -994,7 +994,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { MakeSureTabletIsUp(runtime, TTestTxConfig::SchemeShard, 0); // root ss good - TActorId sender = runtime.AllocateEdgeActor(0); + TActorId sender = runtime.AllocateEdgeActor(0); InitSchemeRoot(runtime, sender); TSubDomainKey subdomainKey; @@ -1744,7 +1744,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { } { - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); runtime.SendToPipe(hiveTablet, sender, new TEvHive::TEvRequestHiveInfo(true)); TAutoPtr<IEventHandle> handle; TEvHive::TEvResponseHiveInfo* response = runtime.GrabEdgeEventRethrow<TEvHive::TEvResponseHiveInfo>(handle); @@ -1852,7 +1852,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { TTabletTypes::EType tabletType = TTabletTypes::Dummy; TVector<ui64> tabletIds; - TActorId senderB = runtime.AllocateEdgeActor(0); + TActorId senderB = runtime.AllocateEdgeActor(0); for (int i = 0; i < 2; ++i) { TChannelsBindings bindings; for (int n = 0; n <= i + 1; ++n) { @@ -1885,7 +1885,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { Y_UNIT_TEST(TestDeleteTablet) { TTestBasicRuntime runtime(1, false); Setup(runtime, true); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); const ui64 hiveTablet = MakeDefaultHiveID(0); const ui64 testerTablet = MakeDefaultHiveID(1); CreateTestBootstrapper(runtime, CreateTestTabletInfo(hiveTablet, TTabletTypes::FLAT_HIVE), &CreateDefaultHive); @@ -1912,7 +1912,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { Y_UNIT_TEST(TestDeleteOwnerTablets) { TTestBasicRuntime runtime(1, false); Setup(runtime, true); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); const ui64 hiveTablet = MakeDefaultHiveID(0); const ui64 testerTablet = MakeDefaultHiveID(1); CreateTestBootstrapper(runtime, CreateTestTabletInfo(hiveTablet, TTabletTypes::FLAT_HIVE), &CreateDefaultHive); @@ -1941,7 +1941,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { Y_UNIT_TEST(TestDeleteOwnerTabletsMany) { TTestBasicRuntime runtime(1, false); Setup(runtime, true); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); const ui64 hiveTablet = MakeDefaultHiveID(0); const ui64 testerTablet = MakeDefaultHiveID(1); CreateTestBootstrapper(runtime, CreateTestTabletInfo(hiveTablet, TTabletTypes::FLAT_HIVE), &CreateDefaultHive); @@ -2002,7 +2002,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { Y_UNIT_TEST(PipeAlivenessOfDeadTablet) { TTestBasicRuntime runtime(1, false); Setup(runtime, true); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); const ui64 hiveTablet = MakeDefaultHiveID(0); const ui64 testerTablet = 1; CreateTestBootstrapper(runtime, CreateTestTabletInfo(hiveTablet, TTabletTypes::FLAT_HIVE), &CreateDefaultHive); @@ -2027,7 +2027,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { Y_UNIT_TEST(TestCreateTabletBeforeLocal) { TTestBasicRuntime runtime(1, false); Setup(runtime, false); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); const ui64 hiveTablet = MakeDefaultHiveID(0); const ui64 testerTablet = MakeDefaultHiveID(1); CreateTestBootstrapper(runtime, CreateTestTabletInfo(hiveTablet, TTabletTypes::FLAT_HIVE), &CreateDefaultHive); @@ -2058,7 +2058,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { TTenantPoolConfig::TPtr tenantPoolConfig = new TTenantPoolConfig(localConfig); tenantPoolConfig->AddStaticSlot(DOMAIN_NAME); - TActorId actorId = runtime.Register(CreateTenantPool(tenantPoolConfig)); + TActorId actorId = runtime.Register(CreateTenantPool(tenantPoolConfig)); runtime.EnableScheduleForActor(actorId, true); runtime.RegisterService(MakeTenantPoolRootID(), actorId); } @@ -2069,7 +2069,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { Y_UNIT_TEST(TestReCreateTablet) { TTestBasicRuntime runtime; Setup(runtime, true); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); const ui64 hiveTablet = MakeDefaultHiveID(0); const ui64 testerTablet = MakeDefaultHiveID(1); CreateTestBootstrapper(runtime, CreateTestTabletInfo(hiveTablet, TTabletTypes::FLAT_HIVE), &CreateDefaultHive); @@ -2108,7 +2108,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { Y_UNIT_TEST(TestReCreateTabletError) { TTestBasicRuntime runtime; Setup(runtime, true); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); const ui64 hiveTablet = MakeDefaultHiveID(0); const ui64 testerTablet = MakeDefaultHiveID(1); CreateTestBootstrapper(runtime, CreateTestTabletInfo(hiveTablet, TTabletTypes::FLAT_HIVE), &CreateDefaultHive); @@ -2150,7 +2150,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { TTestBasicRuntime runtime; Setup(runtime, true); setup(runtime); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); CreateTestBootstrapper(runtime, CreateTestTabletInfo(hiveTablet, TTabletTypes::FLAT_HIVE), &CreateDefaultHive); TDispatchOptions options; @@ -2208,7 +2208,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { TTestBasicRuntime runtime(1, false); Setup(runtime, true); TVector<ui64> tabletIds; - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); //TAutoPtr<ITabletScheduledEventsGuard> guard = CreateTabletScheduledEventsGuard(tabletIds, runtime, sender); const ui64 hiveTablet = MakeDefaultHiveID(0); const ui64 testerTablet = MakeDefaultHiveID(1); @@ -2226,8 +2226,8 @@ Y_UNIT_TEST_SUITE(THiveTest) { WaitForTabletIsUp(runtime, tabletId, 0, &pipeConfig); runtime.SendToPipe(hiveTablet, sender, new TEvInterconnect::TEvNodeDisconnected(runtime.GetNodeId(0))); - //TActorId local = MakeLocalID(runtime.GetNodeId(0)); - //runtime.Send(new IEventHandle(local, sender, new TEvTabletPipe::TEvClientDestroyed(hiveTablet, TActorId(), TActorId())), 0); + //TActorId local = MakeLocalID(runtime.GetNodeId(0)); + //runtime.Send(new IEventHandle(local, sender, new TEvTabletPipe::TEvClientDestroyed(hiveTablet, TActorId(), TActorId())), 0); SendKillLocal(runtime, 0); runtime.Register(CreateTabletKiller(hiveTablet)); { @@ -2281,7 +2281,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { MakeHolder<TEvHive::TEvCreateTablet>(testerTablet, 100500, tabletType, BINDED_CHANNELS), nodeIndex, true); MakeSureTabletIsUp(runtime, tabletId, nodeIndex); - TActorId senderA = runtime.AllocateEdgeActor(); + TActorId senderA = runtime.AllocateEdgeActor(); // first check, aquiring generation runtime.SendToPipe(hiveTablet, senderA, new TEvHive::TEvRequestHiveInfo(tabletId)); @@ -2444,7 +2444,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { Y_UNIT_TEST(TestReassignGroupsWithRecreateTablet) { TTestBasicRuntime runtime(1, false); Setup(runtime, true, 3); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); const ui64 hiveTablet = MakeDefaultHiveID(0); const ui64 testerTablet = MakeDefaultHiveID(1); CreateTestBootstrapper(runtime, CreateTestTabletInfo(hiveTablet, TTabletTypes::FLAT_HIVE), &CreateDefaultHive); @@ -2514,7 +2514,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { TTestBasicRuntime runtime; Setup(runtime, true); setup(runtime); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); CreateTestBootstrapper(runtime, CreateTestTabletInfo(hiveTablet, TTabletTypes::FLAT_HIVE), &CreateDefaultHive); TDispatchOptions options; @@ -2560,7 +2560,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { // Y_UNIT_TEST(TestCreateTabletAndChangeProfiles) { // TTestBasicRuntime runtime(1, false); // Setup(runtime, true); -// TActorId sender = runtime.AllocateEdgeActor(); +// TActorId sender = runtime.AllocateEdgeActor(); // CreatePDiskAndGroup(runtime, sender); // const ui64 hiveTablet = MakeDefaultHiveID(0); // const ui64 testerTablet = MakeDefaultHiveID(1); @@ -2592,7 +2592,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { Y_UNIT_TEST(topTablet) { TTestBasicRuntime runtime(1, false); Setup(runtime, true); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); const ui64 hiveTablet = MakeDefaultHiveID(0); const ui64 testerTablet = MakeDefaultHiveID(1); CreateTestBootstrapper(runtime, CreateTestTabletInfo(hiveTablet, TTabletTypes::FLAT_HIVE), &CreateDefaultHive); @@ -2612,7 +2612,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { Y_UNIT_TEST(TestStopAndRestartTablet) { TTestBasicRuntime runtime(1, false); Setup(runtime, true); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); TVector<ui64> tabletIds; TAutoPtr<ITabletScheduledEventsGuard> guard = CreateTabletScheduledEventsGuard(tabletIds, runtime, sender); const ui64 hiveTablet = MakeDefaultHiveID(0); @@ -2637,7 +2637,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { TTestBasicRuntime runtime(3, false); Setup(runtime, true); TVector<ui64> tabletIds; - TActorId senderA = runtime.AllocateEdgeActor(); + TActorId senderA = runtime.AllocateEdgeActor(); TAutoPtr<ITabletScheduledEventsGuard> guard = CreateTabletScheduledEventsGuard(tabletIds, runtime, senderA); const ui64 hiveTablet = MakeDefaultHiveID(0); const ui64 testerTablet = MakeDefaultHiveID(1); @@ -2690,7 +2690,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { TTestBasicRuntime runtime(3, false); Setup(runtime, true); TVector<ui64> tabletIds; - TActorId senderA = runtime.AllocateEdgeActor(); + TActorId senderA = runtime.AllocateEdgeActor(); //TAutoPtr<ITabletScheduledEventsGuard> guard = CreateTabletScheduledEventsGuard(tabletIds, runtime, senderA); const ui64 hiveTablet = MakeDefaultHiveID(0); const ui64 testerTablet = MakeDefaultHiveID(1); @@ -2809,7 +2809,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { Setup(runtime, true); const int nodeBase = runtime.GetNodeId(0); TVector<ui64> tabletIds; - TActorId senderA = runtime.AllocateEdgeActor(); + TActorId senderA = runtime.AllocateEdgeActor(); const ui64 hiveTablet = MakeDefaultHiveID(0); const ui64 testerTablet = MakeDefaultHiveID(1); CreateTestBootstrapper(runtime, CreateTestTabletInfo(hiveTablet, TTabletTypes::FLAT_HIVE), &CreateDefaultHive); @@ -2879,7 +2879,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { TTestBasicRuntime runtime(NUM_NODES, false); Setup(runtime, true); const int nodeBase = runtime.GetNodeId(0); - TActorId senderA = runtime.AllocateEdgeActor(); + TActorId senderA = runtime.AllocateEdgeActor(); const ui64 hiveTablet = MakeDefaultHiveID(0); const ui64 testerTablet = MakeDefaultHiveID(1); CreateTestBootstrapper(runtime, CreateTestTabletInfo(hiveTablet, TTabletTypes::FLAT_HIVE), &CreateDefaultHive); @@ -2889,7 +2889,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { runtime.DispatchEvents(options); } for (int nodeIdx = 0; nodeIdx < NUM_NODES; ++nodeIdx) { - TActorId senderLocal = runtime.AllocateEdgeActor(nodeIdx); + TActorId senderLocal = runtime.AllocateEdgeActor(nodeIdx); THolder<TEvHive::TEvTabletMetrics> ev = MakeHolder<TEvHive::TEvTabletMetrics>(); ev->Record.MutableTotalResourceUsage()->SetCPU(999); // KIKIMR-9870 runtime.SendToPipe(hiveTablet, senderLocal, ev.Release(), nodeIdx, GetPipeConfigWithRetries()); @@ -3199,7 +3199,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { TTestBasicRuntime runtime(NUM_NODES, false); Setup(runtime, true); const int nodeBase = runtime.GetNodeId(0); - TActorId senderA = runtime.AllocateEdgeActor(); + TActorId senderA = runtime.AllocateEdgeActor(); const ui64 hiveTablet = MakeDefaultHiveID(0); const ui64 testerTablet = MakeDefaultHiveID(1); CreateTestBootstrapper(runtime, CreateTestTabletInfo(hiveTablet, TTabletTypes::FLAT_HIVE), &CreateDefaultHive); @@ -3877,7 +3877,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { } void SendGetTabletStorageInfo(TTestActorRuntime& runtime, ui64 hiveTablet, ui64 tabletId, ui32 nodeIndex) { - TActorId senderB = runtime.AllocateEdgeActor(nodeIndex); + TActorId senderB = runtime.AllocateEdgeActor(nodeIndex); runtime.SendToPipe(hiveTablet, senderB, new TEvHive::TEvGetTabletStorageInfo(tabletId), nodeIndex, GetPipeConfigWithRetries()); } @@ -3947,12 +3947,12 @@ Y_UNIT_TEST_SUITE(THiveTest) { void SendLockTabletExecution(TTestActorRuntime& runtime, ui64 hiveTablet, ui64 tabletId, ui32 nodeIndex, NKikimrProto::EReplyStatus expectedStatus = NKikimrProto::OK, - const TActorId& owner = TActorId(), ui64 maxTimeout = 0, + const TActorId& owner = TActorId(), ui64 maxTimeout = 0, bool reconnect = false) { THolder<TEvHive::TEvLockTabletExecution> event(new TEvHive::TEvLockTabletExecution(tabletId)); if (owner) { - ActorIdToProto(owner, event->Record.MutableOwnerActor()); + ActorIdToProto(owner, event->Record.MutableOwnerActor()); } if (maxTimeout > 0) { event->Record.SetMaxReconnectTimeout(maxTimeout); @@ -3960,7 +3960,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { if (reconnect) { event->Record.SetReconnect(true); } - TActorId senderB = runtime.AllocateEdgeActor(nodeIndex); + TActorId senderB = runtime.AllocateEdgeActor(nodeIndex); runtime.SendToPipe(hiveTablet, senderB, event.Release(), nodeIndex, GetPipeConfigWithRetries()); TAutoPtr<IEventHandle> handle; @@ -3970,7 +3970,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { UNIT_ASSERT_VALUES_EQUAL(result->Record.GetStatus(), expectedStatus); } - void VerifyLockTabletExecutionLost(TTestActorRuntime& runtime, ui64 tabletId, const TActorId& owner) { + void VerifyLockTabletExecutionLost(TTestActorRuntime& runtime, ui64 tabletId, const TActorId& owner) { TAutoPtr<IEventHandle> handle; auto result = runtime.GrabEdgeEventRethrow<TEvHive::TEvLockTabletExecutionLost>(handle); UNIT_ASSERT(result); @@ -3994,8 +3994,8 @@ Y_UNIT_TEST_SUITE(THiveTest) { SendLockTabletExecution(runtime, hiveTablet, tabletId, 1); MakeSureTabletIsDown(runtime, tabletId, 0); - TActorId disconnecter = runtime.AllocateEdgeActor(0); - TActorId proxy = runtime.GetInterconnectProxy(0, 1); + TActorId disconnecter = runtime.AllocateEdgeActor(0); + TActorId proxy = runtime.GetInterconnectProxy(0, 1); runtime.Send(new IEventHandle(proxy, disconnecter, new TEvInterconnect::TEvDisconnect()), 0); // Tablet should boot when the locking node disconnects @@ -4016,7 +4016,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { MakeSureTabletIsUp(runtime, tabletId, 0); // Owner cannot be on a different node - TActorId owner = runtime.AllocateEdgeActor(0); + TActorId owner = runtime.AllocateEdgeActor(0); SendLockTabletExecution(runtime, hiveTablet, tabletId, 1, NKikimrProto::ERROR, owner); MakeSureTabletIsUp(runtime, tabletId, 0); } @@ -4035,12 +4035,12 @@ Y_UNIT_TEST_SUITE(THiveTest) { ui64 tabletId = SendCreateTestTablet(runtime, hiveTablet, testerTablet, std::move(ev), 0, true); MakeSureTabletIsUp(runtime, tabletId, 0); - TActorId owner = runtime.AllocateEdgeActor(1); + TActorId owner = runtime.AllocateEdgeActor(1); SendLockTabletExecution(runtime, hiveTablet, tabletId, 1, NKikimrProto::OK, owner, 1000); MakeSureTabletIsDown(runtime, tabletId, 0); - TActorId disconnecter = runtime.AllocateEdgeActor(0); - TActorId proxy = runtime.GetInterconnectProxy(0, 1); + TActorId disconnecter = runtime.AllocateEdgeActor(0); + TActorId proxy = runtime.GetInterconnectProxy(0, 1); runtime.Send(new IEventHandle(proxy, disconnecter, new TEvInterconnect::TEvDisconnect()), 0); // Tablet should boot when timeout expires @@ -4065,7 +4065,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { MakeSureTabletIsUp(runtime, tabletId, 0); // Lock with a 40 second timeout (test reboots take 30 seconds) - TActorId owner = runtime.AllocateEdgeActor(1); + TActorId owner = runtime.AllocateEdgeActor(1); SendLockTabletExecution(runtime, hiveTablet, tabletId, 1, NKikimrProto::OK, owner, 40000); MakeSureTabletIsDown(runtime, tabletId, 0); @@ -4083,7 +4083,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { TTestBasicRuntime runtime(2, false); Setup(runtime, false); CreateLocal(runtime, 0); // only the 1st node has local running - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); const ui64 hiveTablet = MakeDefaultHiveID(0); const ui64 testerTablet = MakeDefaultHiveID(1); const TActorId hiveActor = CreateTestBootstrapper(runtime, CreateTestTabletInfo(hiveTablet, TTabletTypes::FLAT_HIVE), &CreateDefaultHive); @@ -4094,7 +4094,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { ui64 tabletId = SendCreateTestTablet(runtime, hiveTablet, testerTablet, std::move(ev), 0, true); MakeSureTabletIsUp(runtime, tabletId, 0); - TActorId owner = runtime.AllocateEdgeActor(1); + TActorId owner = runtime.AllocateEdgeActor(1); SendLockTabletExecution(runtime, hiveTablet, tabletId, 1, NKikimrProto::OK, owner, 1000); MakeSureTabletIsDown(runtime, tabletId, 0); @@ -4129,7 +4129,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { ui64 tabletId = SendCreateTestTablet(runtime, hiveTablet, testerTablet, std::move(ev), 0, true); MakeSureTabletIsUp(runtime, tabletId, 0); - TActorId owner = runtime.AllocateEdgeActor(1); + TActorId owner = runtime.AllocateEdgeActor(1); SendLockTabletExecution(runtime, hiveTablet, tabletId, 1, NKikimrProto::OK, owner, 1000); MakeSureTabletIsDown(runtime, tabletId, 0); @@ -4157,11 +4157,11 @@ Y_UNIT_TEST_SUITE(THiveTest) { } void MakeSureTabletStaysDown(TTestActorRuntime& runtime, ui64 tabletId, const TDuration& timeout) { - TActorId edge = runtime.AllocateEdgeActor(); + TActorId edge = runtime.AllocateEdgeActor(); NTabletPipe::TClientConfig clientConfig; clientConfig.AllowFollower = true; clientConfig.RetryPolicy = NTabletPipe::TClientRetryPolicy::WithRetries(); - TActorId pipeClient = runtime.Register(NTabletPipe::CreateClient(edge, tabletId, clientConfig)); + TActorId pipeClient = runtime.Register(NTabletPipe::CreateClient(edge, tabletId, clientConfig)); TAutoPtr<IEventHandle> handle; TInstant deadline = TInstant::Now() + timeout; bool res = false; @@ -4179,7 +4179,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { } } while (TInstant::Now() <= deadline); - runtime.Send(new IEventHandle(pipeClient, TActorId(), new TEvents::TEvPoisonPill())); + runtime.Send(new IEventHandle(pipeClient, TActorId(), new TEvents::TEvPoisonPill())); UNIT_ASSERT_C(!res, "Unexpected successful tablet connection"); } @@ -4198,13 +4198,13 @@ Y_UNIT_TEST_SUITE(THiveTest) { MakeSureTabletIsUp(runtime, tabletId, 0); // lock with a 500ms timeout - TActorId owner = runtime.AllocateEdgeActor(1); + TActorId owner = runtime.AllocateEdgeActor(1); SendLockTabletExecution(runtime, hiveTablet, tabletId, 1, NKikimrProto::OK, owner, 500); MakeSureTabletIsDown(runtime, tabletId, 0); // disconnect the node - TActorId disconnecter = runtime.AllocateEdgeActor(0); - TActorId proxy = runtime.GetInterconnectProxy(0, 1); + TActorId disconnecter = runtime.AllocateEdgeActor(0); + TActorId proxy = runtime.GetInterconnectProxy(0, 1); runtime.Send(new IEventHandle(proxy, disconnecter, new TEvInterconnect::TEvDisconnect()), 0); // reconnect the lock @@ -4227,7 +4227,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { MakeSureTabletIsUp(runtime, tabletId, 0); // Lock with a 40 second timeout (test reboots take 30 seconds) - TActorId owner = runtime.AllocateEdgeActor(1); + TActorId owner = runtime.AllocateEdgeActor(1); SendLockTabletExecution(runtime, hiveTablet, tabletId, 1, NKikimrProto::OK, owner, 40000); MakeSureTabletIsDown(runtime, tabletId, 0); @@ -4253,13 +4253,13 @@ Y_UNIT_TEST_SUITE(THiveTest) { MakeSureTabletIsUp(runtime, tabletId, 0); // lock with a 500ms timeout - TActorId owner = runtime.AllocateEdgeActor(1); + TActorId owner = runtime.AllocateEdgeActor(1); SendLockTabletExecution(runtime, hiveTablet, tabletId, 1, NKikimrProto::OK, owner, 500); MakeSureTabletIsDown(runtime, tabletId, 0); // disconnect the node - TActorId disconnecter = runtime.AllocateEdgeActor(0); - TActorId proxy = runtime.GetInterconnectProxy(0, 1); + TActorId disconnecter = runtime.AllocateEdgeActor(0); + TActorId proxy = runtime.GetInterconnectProxy(0, 1); runtime.Send(new IEventHandle(proxy, disconnecter, new TEvInterconnect::TEvDisconnect()), 0); // wait for the lost lock notification @@ -4271,13 +4271,13 @@ Y_UNIT_TEST_SUITE(THiveTest) { void SendUnlockTabletExecution(TTestActorRuntime& runtime, ui64 hiveTablet, ui64 tabletId, ui32 nodeIndex, NKikimrProto::EReplyStatus expectedStatus = NKikimrProto::OK, - const TActorId& owner = TActorId()) + const TActorId& owner = TActorId()) { THolder<TEvHive::TEvUnlockTabletExecution> event(new TEvHive::TEvUnlockTabletExecution(tabletId)); if (owner) { - ActorIdToProto(owner, event->Record.MutableOwnerActor()); + ActorIdToProto(owner, event->Record.MutableOwnerActor()); } - TActorId senderB = runtime.AllocateEdgeActor(nodeIndex); + TActorId senderB = runtime.AllocateEdgeActor(nodeIndex); runtime.SendToPipe(hiveTablet, senderB, event.Release(), nodeIndex, GetPipeConfigWithRetries()); TAutoPtr<IEventHandle> handle; @@ -4321,7 +4321,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { ui64 tabletId = SendCreateTestTablet(runtime, hiveTablet, testerTablet, std::move(ev), 0, true); MakeSureTabletIsUp(runtime, tabletId, 0); - TActorId owner = runtime.AllocateEdgeActor(1); + TActorId owner = runtime.AllocateEdgeActor(1); SendLockTabletExecution(runtime, hiveTablet, tabletId, 1, NKikimrProto::OK, owner); MakeSureTabletIsDown(runtime, tabletId, 0); @@ -4346,12 +4346,12 @@ Y_UNIT_TEST_SUITE(THiveTest) { ui64 tabletId = SendCreateTestTablet(runtime, hiveTablet, testerTablet, std::move(ev), 0, true); MakeSureTabletIsUp(runtime, tabletId, 0); - TActorId owner = runtime.AllocateEdgeActor(1); + TActorId owner = runtime.AllocateEdgeActor(1); SendLockTabletExecution(runtime, hiveTablet, tabletId, 1, NKikimrProto::OK, owner); MakeSureTabletIsDown(runtime, tabletId, 0); // Lock to a different owner - TActorId owner2 = runtime.AllocateEdgeActor(1); + TActorId owner2 = runtime.AllocateEdgeActor(1); SendLockTabletExecution(runtime, hiveTablet, tabletId, 1, NKikimrProto::OK, owner2); // Hive should notify the old owner on unlocking @@ -4370,7 +4370,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { ev->Record.SetTabletBootMode(NKikimrHive::ETabletBootMode::TABLET_BOOT_MODE_EXTERNAL); ui64 tabletId = SendCreateTestTablet(runtime, hiveTablet, testerTablet, std::move(ev), 0, true); - TActorId owner1 = runtime.AllocateEdgeActor(0); + TActorId owner1 = runtime.AllocateEdgeActor(0); runtime.SendToPipe(hiveTablet, owner1, new TEvHive::TEvInitiateTabletExternalBoot(tabletId), 0, GetPipeConfigWithRetries()); TAutoPtr<IEventHandle> handle; @@ -4396,7 +4396,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { ui64 tabletId = SendCreateTestTablet(runtime, hiveTablet, testerTablet, std::move(ev), 0, true); MakeSureTabletIsUp(runtime, tabletId, 0); - TActorId bootOwner = runtime.AllocateEdgeActor(1); + TActorId bootOwner = runtime.AllocateEdgeActor(1); // cannot use external boot for normal tablets { @@ -4407,7 +4407,7 @@ Y_UNIT_TEST_SUITE(THiveTest) { UNIT_ASSERT_EQUAL(msg->Record.GetStatus(), NKikimrProto::EReplyStatus::ERROR); } - TActorId lockOwner = runtime.AllocateEdgeActor(1); + TActorId lockOwner = runtime.AllocateEdgeActor(1); SendLockTabletExecution(runtime, hiveTablet, tabletId, 1, NKikimrProto::OK, lockOwner); MakeSureTabletIsDown(runtime, tabletId, 0); diff --git a/ydb/core/mind/hive/leader_tablet_info.cpp b/ydb/core/mind/hive/leader_tablet_info.cpp index 8082c20a37d..13e1d5b2ee2 100644 --- a/ydb/core/mind/hive/leader_tablet_info.cpp +++ b/ydb/core/mind/hive/leader_tablet_info.cpp @@ -141,7 +141,7 @@ TFollowerGroup& TLeaderTabletInfo::AddFollowerGroup(TFollowerGroupId followerGro } TActorId TLeaderTabletInfo::SetLockedToActor(const TActorId& actor, const TDuration& timeout) { - TActorId previousOwner = LockedToActor; + TActorId previousOwner = LockedToActor; if (LockedToActor != actor) { if (LockedToActor.NodeId() != actor.NodeId()) { if (LockedToActor) { diff --git a/ydb/core/mind/hive/leader_tablet_info.h b/ydb/core/mind/hive/leader_tablet_info.h index 7cf6996c48a..88425b3caed 100644 --- a/ydb/core/mind/hive/leader_tablet_info.h +++ b/ydb/core/mind/hive/leader_tablet_info.h @@ -44,8 +44,8 @@ public: TOwnerIdxType::TValueType Owner; TVector<TSubDomainKey> EffectiveAllowedDomains; // AllowedDomains | ObjectDomain NKikimrHive::ETabletBootMode BootMode; - TVector<TActorId> StorageInfoSubscribers; - TActorId LockedToActor; + TVector<TActorId> StorageInfoSubscribers; + TActorId LockedToActor; TDuration LockedReconnectTimeout; ui64 PendingUnlockSeqNo; @@ -101,7 +101,7 @@ public: return IsReadyToWork() && TTabletInfo::IsAlive(); } - bool IsAliveOnLocal(const TActorId& local) const { + bool IsAliveOnLocal(const TActorId& local) const { return IsReadyToWork() && TTabletInfo::IsAliveOnLocal(local); } @@ -280,17 +280,17 @@ public: } void NotifyStorageInfo(const TActorContext& ctx) { - TVector<TActorId> targets; + TVector<TActorId> targets; targets.swap(StorageInfoSubscribers); - for (TActorId target : targets) { + for (TActorId target : targets) { ctx.Send(target, new TEvHive::TEvGetTabletStorageInfoResult(Id, *TabletStorageInfo)); } } - TActorId SetLockedToActor(const TActorId& actor, const TDuration& timeout); + TActorId SetLockedToActor(const TActorId& actor, const TDuration& timeout); - TActorId ClearLockedToActor() { - return SetLockedToActor(TActorId(), TDuration()); + TActorId ClearLockedToActor() { + return SetLockedToActor(TActorId(), TDuration()); } void ActualizeTabletStatistics(TInstant now); diff --git a/ydb/core/mind/hive/monitoring.cpp b/ydb/core/mind/hive/monitoring.cpp index c2d81f410b4..7e9a83b354e 100644 --- a/ydb/core/mind/hive/monitoring.cpp +++ b/ydb/core/mind/hive/monitoring.cpp @@ -1,4 +1,4 @@ -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <library/cpp/json/json_writer.h> #include <library/cpp/protobuf/json/proto2json.h> #include <util/string/vector.h> @@ -22,7 +22,7 @@ public: }; struct TNodeInfo { - TActorId Local; + TActorId Local; ui64 TabletsOn; TNodeInfo() @@ -30,12 +30,12 @@ public: {} }; - const TActorId Source; + const TActorId Source; TMap<ui64, TTabletInfo> TabletInfo; TMap<ui32, TNodeInfo> NodeInfo; - TTxMonEvent_DbState(const TActorId &source, TSelf *hive) + TTxMonEvent_DbState(const TActorId &source, TSelf *hive) : TBase(hive) , Source(source) {} @@ -72,7 +72,7 @@ public: return false; while (rowset.IsValid()) { const ui32 nodeId = rowset.GetValue<Schema::Node::ID>(); - const TActorId local = rowset.GetValue<Schema::Node::Local>(); + const TActorId local = rowset.GetValue<Schema::Node::Local>(); NodeInfo[nodeId].Local = local; if (!rowset.Next()) @@ -177,14 +177,14 @@ public: class TTxMonEvent_MemStateTablets : public TTransactionBase<THive> { public: - const TActorId Source; + const TActorId Source; THolder<NMon::TEvRemoteHttpInfo> Event; bool BadOnly = false; bool WaitingOnly = false; ui64 MaxCount = 0; TString Sort; - TTxMonEvent_MemStateTablets(const TActorId &source, NMon::TEvRemoteHttpInfo::TPtr& ev, TSelf *hive) + TTxMonEvent_MemStateTablets(const TActorId &source, NMon::TEvRemoteHttpInfo::TPtr& ev, TSelf *hive) : TBase(hive) , Source(source) , Event(ev->Release()) @@ -303,12 +303,12 @@ public: class TTxMonEvent_MemStateNodes : public TTransactionBase<THive> { public: - const TActorId Source; + const TActorId Source; THolder<NMon::TEvRemoteHttpInfo> Event; bool BadOnly = false; ui64 MaxCount = 0; - TTxMonEvent_MemStateNodes(const TActorId &source, NMon::TEvRemoteHttpInfo::TPtr& ev, TSelf *hive) + TTxMonEvent_MemStateNodes(const TActorId &source, NMon::TEvRemoteHttpInfo::TPtr& ev, TSelf *hive) : TBase(hive) , Source(source) , Event(ev->Release()) @@ -440,10 +440,10 @@ TString GetDurationString(TDuration duration) { class TTxMonEvent_Resources : public TTransactionBase<THive> { public: - const TActorId Source; + const TActorId Source; TAutoPtr<NMon::TEvRemoteHttpInfo> Event; - TTxMonEvent_Resources(const TActorId &source, NMon::TEvRemoteHttpInfo::TPtr& ev, TSelf *hive) + TTxMonEvent_Resources(const TActorId &source, NMon::TEvRemoteHttpInfo::TPtr& ev, TSelf *hive) : TBase(hive) , Source(source) , Event(ev->Release()) @@ -590,11 +590,11 @@ public: class TTxMonEvent_Settings : public TTransactionBase<THive> { public: - const TActorId Source; + const TActorId Source; TAutoPtr<NMon::TEvRemoteHttpInfo> Event; bool ChangeRequest = false; - TTxMonEvent_Settings(const TActorId &source, NMon::TEvRemoteHttpInfo::TPtr& ev, TSelf *hive) + TTxMonEvent_Settings(const TActorId &source, NMon::TEvRemoteHttpInfo::TPtr& ev, TSelf *hive) : TBase(hive) , Source(source) , Event(ev->Release()) @@ -1032,11 +1032,11 @@ public: class TTxMonEvent_Landing : public TTransactionBase<THive> { public: - const TActorId Source; + const TActorId Source; TAutoPtr<NMon::TEvRemoteHttpInfo> Event; TCgiParameters Cgi; - TTxMonEvent_Landing(const TActorId &source, NMon::TEvRemoteHttpInfo::TPtr& ev, TSelf *hive) + TTxMonEvent_Landing(const TActorId &source, NMon::TEvRemoteHttpInfo::TPtr& ev, TSelf *hive) : TBase(hive) , Source(source) , Event(ev->Release()) @@ -1718,11 +1718,11 @@ public: class TTxMonEvent_LandingData : public TTransactionBase<THive> { public: - const TActorId Source; + const TActorId Source; TAutoPtr<NMon::TEvRemoteHttpInfo> Event; TCgiParameters Cgi; - TTxMonEvent_LandingData(const TActorId &source, NMon::TEvRemoteHttpInfo::TPtr& ev, TSelf *hive) + TTxMonEvent_LandingData(const TActorId &source, NMon::TEvRemoteHttpInfo::TPtr& ev, TSelf *hive) : TBase(hive) , Source(source) , Event(ev->Release()) @@ -1861,12 +1861,12 @@ public: class TTxMonEvent_SetDown : public TTransactionBase<THive> { public: - const TActorId Source; + const TActorId Source; const TNodeId NodeId; const bool Down; TString Response; - TTxMonEvent_SetDown(const TActorId& source, TNodeId nodeId, bool down, TSelf* hive) + TTxMonEvent_SetDown(const TActorId& source, TNodeId nodeId, bool down, TSelf* hive) : TBase(hive) , Source(source) , NodeId(nodeId) @@ -1896,12 +1896,12 @@ public: class TTxMonEvent_SetFreeze : public TTransactionBase<THive> { public: - const TActorId Source; + const TActorId Source; const TNodeId NodeId; const bool Freeze; TString Response; - TTxMonEvent_SetFreeze(const TActorId& source, TNodeId nodeId, bool freeze, TSelf* hive) + TTxMonEvent_SetFreeze(const TActorId& source, TNodeId nodeId, bool freeze, TSelf* hive) : TBase(hive) , Source(source) , NodeId(nodeId) @@ -1931,11 +1931,11 @@ public: class TTxMonEvent_KickNode : public TTransactionBase<THive> { public: - const TActorId Source; + const TActorId Source; const TNodeId NodeId; TString Response; - TTxMonEvent_KickNode(const TActorId& source, TNodeId nodeId, TSelf* hive) + TTxMonEvent_KickNode(const TActorId& source, TNodeId nodeId, TSelf* hive) : TBase(hive) , Source(source) , NodeId(nodeId) @@ -2045,7 +2045,7 @@ public: class TTxMonEvent_Rebalance : public TTransactionBase<THive> { public: - const TActorId Source; + const TActorId Source; int MaxMovements = 1000; TTxMonEvent_Rebalance(const TActorId& source, NMon::TEvRemoteHttpInfo::TPtr& ev, TSelf* hive) @@ -2069,7 +2069,7 @@ public: class TReassignTabletWaitActor : public TActor<TReassignTabletWaitActor>, public ISubActor { public: - TActorId Source; + TActorId Source; ui32 TabletsTotal = std::numeric_limits<ui32>::max(); ui32 TabletsDone = 0; THive* Hive; @@ -2078,7 +2078,7 @@ public: return NKikimrServices::TActivity::HIVE_MON_REQUEST; } - TReassignTabletWaitActor(const TActorId& source, THive* hive) + TReassignTabletWaitActor(const TActorId& source, THive* hive) : TActor(&TReassignTabletWaitActor::StateWork) , Source(source) , Hive(hive) @@ -2112,7 +2112,7 @@ public: class TTxMonEvent_ReassignTablet : public TTransactionBase<THive> { public: TAutoPtr<NMon::TEvRemoteHttpInfo> Event; - const TActorId Source; + const TActorId Source; TTabletId TabletId = 0; TTabletTypes::EType TabletType = TTabletTypes::TYPE_INVALID; TVector<ui32> TabletChannels; @@ -2122,7 +2122,7 @@ public: TString Error; bool Wait = true; - TTxMonEvent_ReassignTablet(const TActorId& source, NMon::TEvRemoteHttpInfo::TPtr& ev, TSelf* hive) + TTxMonEvent_ReassignTablet(const TActorId& source, NMon::TEvRemoteHttpInfo::TPtr& ev, TSelf* hive) : TBase(hive) , Event(ev->Release()) , Source(source) @@ -2186,7 +2186,7 @@ public: tablets.resize(tablets.size() * TabletPercent / 100); } TVector<THolder<TEvHive::TEvReassignTablet>> operations; - TActorId waitActorId; + TActorId waitActorId; TReassignTabletWaitActor* waitActor = nullptr; if (Wait) { waitActor = new TReassignTabletWaitActor(Source, Self); @@ -2551,7 +2551,7 @@ public: class TTxMonEvent_FindTablet : public TTransactionBase<THive> { public: THolder<NMon::TEvRemoteHttpInfo> Event; - const TActorId Source; + const TActorId Source; TTabletId TabletId = 0; TTabletTypes::EType TabletType = TTabletTypes::TYPE_INVALID; ui32 ChannelFrom = 0; @@ -2562,7 +2562,7 @@ public: int TabletPercent = 100; NJson::TJsonValue Result; - TTxMonEvent_FindTablet(const TActorId& source, NMon::TEvRemoteHttpInfo::TPtr& ev, TSelf* hive) + TTxMonEvent_FindTablet(const TActorId& source, NMon::TEvRemoteHttpInfo::TPtr& ev, TSelf* hive) : TBase(hive) , Event(ev->Release()) , Source(source) @@ -2870,7 +2870,7 @@ public: class TTxMonEvent_ResetTablet : public TTransactionBase<THive> { class TResetter : public TActorBootstrapped<TResetter> { TIntrusivePtr<TTabletStorageInfo> Info; - const TActorId Source; + const TActorId Source; const ui32 KnownGeneration; public: @@ -2878,7 +2878,7 @@ class TTxMonEvent_ResetTablet : public TTransactionBase<THive> { return NKikimrServices::TActivity::HIVE_MON_REQUEST; } - TResetter(TIntrusivePtr<TTabletStorageInfo> info, TActorId source, ui32 knownGeneration) + TResetter(TIntrusivePtr<TTabletStorageInfo> info, TActorId source, ui32 knownGeneration) : Info(std::move(info)) , Source(source) , KnownGeneration(knownGeneration) @@ -2900,13 +2900,13 @@ class TTxMonEvent_ResetTablet : public TTransactionBase<THive> { }; public: - const TActorId Source; + const TActorId Source; TTabletId TabletId = 0; TString Error; TIntrusivePtr<TTabletStorageInfo> Info; ui32 KnownGeneration = 0; - TTxMonEvent_ResetTablet(const TActorId& source, NMon::TEvRemoteHttpInfo::TPtr& ev, TSelf* hive) + TTxMonEvent_ResetTablet(const TActorId& source, NMon::TEvRemoteHttpInfo::TPtr& ev, TSelf* hive) : TBase(hive) , Source(source) { @@ -2942,15 +2942,15 @@ public: class TUpdateResourcesActor : public TActorBootstrapped<TUpdateResourcesActor> { public: - TActorId Source; - TActorId Hive; + TActorId Source; + TActorId Hive; NKikimrHive::TTabletMetrics Metrics; static constexpr NKikimrServices::TActivity::EType ActorActivityType() { return NKikimrServices::TActivity::HIVE_MON_REQUEST; } - TUpdateResourcesActor(const TActorId& source, const TActorId& hive, const NKikimrHive::TTabletMetrics& metrics) + TUpdateResourcesActor(const TActorId& source, const TActorId& hive, const NKikimrHive::TTabletMetrics& metrics) : Source(source) , Hive(hive) , Metrics(metrics) @@ -2985,7 +2985,7 @@ public: class TCreateTabletActor : public TActorBootstrapped<TCreateTabletActor> { public: - TActorId Source; + TActorId Source; TAutoPtr<TEvHive::TEvCreateTablet> Event; THive* Hive; @@ -3043,11 +3043,11 @@ private: ui64 FAKE_TXID = -1; public: - TActorId Source; + TActorId Source; TAutoPtr<TEvHive::TEvDeleteTablet> Event; THive* Hive; - TDeleteTabletActor(const TActorId& source, ui64 owner, ui64 ownerIdx, THive* hive) + TDeleteTabletActor(const TActorId& source, ui64 owner, ui64 ownerIdx, THive* hive) : Source(source) , Event(new TEvHive::TEvDeleteTablet()) , Hive(hive) @@ -3061,7 +3061,7 @@ public: return NKikimrServices::TActivity::HIVE_MON_REQUEST; } - TDeleteTabletActor(const TActorId& source, ui64 tabletId, THive* hive) + TDeleteTabletActor(const TActorId& source, ui64 tabletId, THive* hive) : Source(source) , Event(new TEvHive::TEvDeleteTablet()) , Hive(hive) @@ -3110,10 +3110,10 @@ public: class TTxMonEvent_Groups : public TTransactionBase<THive> { public: - const TActorId Source; + const TActorId Source; TAutoPtr<NMon::TEvRemoteHttpInfo> Event; - TTxMonEvent_Groups(const TActorId &source, NMon::TEvRemoteHttpInfo::TPtr& ev, TSelf *hive) + TTxMonEvent_Groups(const TActorId &source, NMon::TEvRemoteHttpInfo::TPtr& ev, TSelf *hive) : TBase(hive) , Source(source) , Event(ev->Release()) @@ -3192,9 +3192,9 @@ public: class TTxMonEvent_NotReady : public TTransactionBase<THive> { public: - const TActorId Source; + const TActorId Source; - TTxMonEvent_NotReady(const TActorId& source, TSelf* hive) + TTxMonEvent_NotReady(const TActorId& source, TSelf* hive) : TBase(hive) , Source(source) {} @@ -3211,11 +3211,11 @@ public: class TTxMonEvent_Storage : public TTransactionBase<THive> { public: - const TActorId Source; + const TActorId Source; THolder<NMon::TEvRemoteHttpInfo> Event; bool Kinds = true; - TTxMonEvent_Storage(const TActorId &source, NMon::TEvRemoteHttpInfo::TPtr& ev, TSelf *hive) + TTxMonEvent_Storage(const TActorId &source, NMon::TEvRemoteHttpInfo::TPtr& ev, TSelf *hive) : TBase(hive) , Source(source) , Event(ev->Release()) diff --git a/ydb/core/mind/hive/node_info.cpp b/ydb/core/mind/hive/node_info.cpp index 7145251f650..8b6913c9819 100644 --- a/ydb/core/mind/hive/node_info.cpp +++ b/ydb/core/mind/hive/node_info.cpp @@ -310,7 +310,7 @@ void TNodeInfo::Ping() { Hive.SendPing(Local, Id); } -void TNodeInfo::SendReconnect(const TActorId& local) { +void TNodeInfo::SendReconnect(const TActorId& local) { BLOG_D("Node(" << Id << ") Reconnect(" << local << ")"); Hive.SendReconnect(local); } diff --git a/ydb/core/mind/hive/node_info.h b/ydb/core/mind/hive/node_info.h index 5971a37ffa2..fa7400fd25b 100644 --- a/ydb/core/mind/hive/node_info.h +++ b/ydb/core/mind/hive/node_info.h @@ -34,7 +34,7 @@ protected: public: THive& Hive; TNodeId Id; - TActorId Local; + TActorId Local; bool Down; bool Freeze; bool Drain; @@ -187,7 +187,7 @@ public: t->BecomeStopped(); } Y_VERIFY(GetTabletsTotal() == 0, "%s", DumpTablets().data()); - Local = TActorId(); + Local = TActorId(); ChangeVolatileState(EVolatileState::Disconnected); for (TTabletInfo* tablet : TabletsToRestart) { if (tablet->IsReadyToBoot()) { @@ -212,7 +212,7 @@ public: void RegisterInDomains(); void DeregisterInDomains(); void Ping(); - void SendReconnect(const TActorId& local); + void SendReconnect(const TActorId& local); void SetDown(bool down); void SetFreeze(bool freeze); void UpdateResourceMaximum(const NKikimrTabletBase::TMetrics& metrics); diff --git a/ydb/core/mind/hive/tablet_info.cpp b/ydb/core/mind/hive/tablet_info.cpp index ca79772a4b1..9fddd10801e 100644 --- a/ydb/core/mind/hive/tablet_info.cpp +++ b/ydb/core/mind/hive/tablet_info.cpp @@ -168,7 +168,7 @@ bool TTabletInfo::CanBeAlive() const { || VolatileState == EVolatileState::TABLET_VOLATILE_STATE_UNKNOWN); // KIKIMR-12558 } -bool TTabletInfo::IsAliveOnLocal(const TActorId& local) const { +bool TTabletInfo::IsAliveOnLocal(const TActorId& local) const { return Node != nullptr && Node->Local == local && (VolatileState == EVolatileState::TABLET_VOLATILE_STATE_STARTING @@ -196,7 +196,7 @@ bool TTabletInfo::InitiateBoot() { } bool TTabletInfo::InitiateStop() { - TActorId local; + TActorId local; TNodeInfo* node = Node; if (node == nullptr && NodeId != 0) { node = Hive.FindNode(NodeId); @@ -293,7 +293,7 @@ bool TTabletInfo::Kick() { } void TTabletInfo::Kill() { - TActorId local; + TActorId local; TNodeInfo* node = Node; if (node == nullptr && NodeId != 0) { node = Hive.FindNode(NodeId); @@ -458,7 +458,7 @@ bool TTabletInfo::InitiateStart(TNodeInfo* node) { return false; } -void TTabletInfo::SendStopTablet(const TActorId& local, TFullTabletId tabletId) { +void TTabletInfo::SendStopTablet(const TActorId& local, TFullTabletId tabletId) { if (local) { Hive.StopTablet(local, tabletId); } diff --git a/ydb/core/mind/hive/tablet_info.h b/ydb/core/mind/hive/tablet_info.h index 1ed03711ced..dc360a30e48 100644 --- a/ydb/core/mind/hive/tablet_info.h +++ b/ydb/core/mind/hive/tablet_info.h @@ -207,14 +207,14 @@ public: bool IsAlive() const; bool CanBeAlive() const; // IsAlive() + <Unknown> - bool IsAliveOnLocal(const TActorId& local) const; + bool IsAliveOnLocal(const TActorId& local) const; bool IsStopped() const; bool InitiateBoot(); bool BecomeStarting(TNodeId nodeId); bool BecomeRunning(TNodeId nodeId); bool BecomeStopped(); - void SendStopTablet(const TActorId& local, TFullTabletId tabletId); + void SendStopTablet(const TActorId& local, TFullTabletId tabletId); bool InitiateStop(); diff --git a/ydb/core/mind/hive/tx__adopt_tablet.cpp b/ydb/core/mind/hive/tx__adopt_tablet.cpp index dbb5eac41ee..45203af0d1f 100644 --- a/ydb/core/mind/hive/tx__adopt_tablet.cpp +++ b/ydb/core/mind/hive/tx__adopt_tablet.cpp @@ -13,14 +13,14 @@ class TTxAdoptTablet : public TTransactionBase<THive> { const ui64 Owner; const ui64 OwnerIdx; - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; TString Explain; NKikimrProto::EReplyStatus Status; public: - TTxAdoptTablet(NKikimrHive::TEvAdoptTablet &rec, const TActorId &sender, const ui64 cookie, THive *hive) + TTxAdoptTablet(NKikimrHive::TEvAdoptTablet &rec, const TActorId &sender, const ui64 cookie, THive *hive) : TBase(hive) , TabletId(rec.GetTabletID()) , PrevOwner(rec.GetPrevOwner()) @@ -110,7 +110,7 @@ public: }; -ITransaction* THive::CreateAdoptTablet(NKikimrHive::TEvAdoptTablet &rec, const TActorId &sender, const ui64 cookie) { +ITransaction* THive::CreateAdoptTablet(NKikimrHive::TEvAdoptTablet &rec, const TActorId &sender, const ui64 cookie) { return new TTxAdoptTablet(rec, sender, cookie, this); } diff --git a/ydb/core/mind/hive/tx__create_tablet.cpp b/ydb/core/mind/hive/tx__create_tablet.cpp index 9db3929f75a..f3b27bfd9ac 100644 --- a/ydb/core/mind/hive/tx__create_tablet.cpp +++ b/ydb/core/mind/hive/tx__create_tablet.cpp @@ -11,7 +11,7 @@ class TTxCreateTablet : public TTransactionBase<THive> { const TTabletTypes::EType TabletType; const ui32 AssignStateStorage; - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; NKikimrProto::EReplyStatus Status; @@ -32,7 +32,7 @@ class TTxCreateTablet : public TTransactionBase<THive> { NKikimrHive::TForwardRequest ForwardRequest; public: - TTxCreateTablet(NKikimrHive::TEvCreateTablet record, const TActorId& sender, const ui64 cookie, THive* hive) + TTxCreateTablet(NKikimrHive::TEvCreateTablet record, const TActorId& sender, const ui64 cookie, THive* hive) : TBase(hive) , RequestData(std::move(record)) , OwnerId(RequestData.GetOwner()) @@ -364,7 +364,7 @@ public: NIceDb::TUpdate<Schema::Tablet::TabletType>(tablet.Type), NIceDb::TUpdate<Schema::Tablet::KnownGeneration>(tablet.KnownGeneration), NIceDb::TUpdate<Schema::Tablet::State>(tablet.State), - NIceDb::TUpdate<Schema::Tablet::ActorsToNotify>(TVector<TActorId>(1, Sender)), + NIceDb::TUpdate<Schema::Tablet::ActorsToNotify>(TVector<TActorId>(1, Sender)), NIceDb::TUpdate<Schema::Tablet::AllowedNodes>(tablet.AllowedNodes), NIceDb::TUpdate<Schema::Tablet::AllowedDataCenters>(allowedDataCenters), NIceDb::TUpdate<Schema::Tablet::AllowedDataCenterIds>(tablet.AllowedDataCenters), @@ -485,7 +485,7 @@ public: tablet->InitiateAssignTabletGroups(); } else if (Status == NKikimrProto::OK && tablet->IsBootingSuppressed()) { // Tablet will never boot, so notify about creation right now - for (const TActorId& actor : tablet->ActorsToNotify) { + for (const TActorId& actor : tablet->ActorsToNotify) { ctx.Send(actor, new TEvHive::TEvTabletCreationResult(NKikimrProto::OK, TabletId)); } tablet->ActorsToNotify.clear(); @@ -509,7 +509,7 @@ public: } }; -ITransaction* THive::CreateCreateTablet(NKikimrHive::TEvCreateTablet rec, const TActorId& sender, const ui64 cookie) { +ITransaction* THive::CreateCreateTablet(NKikimrHive::TEvCreateTablet rec, const TActorId& sender, const ui64 cookie) { return new TTxCreateTablet(std::move(rec), sender, cookie, this); } diff --git a/ydb/core/mind/hive/tx__delete_tablet_result.cpp b/ydb/core/mind/hive/tx__delete_tablet_result.cpp index ad0328923e0..bbe89acae48 100644 --- a/ydb/core/mind/hive/tx__delete_tablet_result.cpp +++ b/ydb/core/mind/hive/tx__delete_tablet_result.cpp @@ -8,8 +8,8 @@ class TTxDeleteTabletResult : public TTransactionBase<THive> { TEvTabletBase::TEvDeleteTabletResult::TPtr Result; TTabletId TabletId; TLeaderTabletInfo* Tablet = nullptr; - TVector<TActorId> StorageInfoSubscribers; - TActorId UnlockedFromActor; + TVector<TActorId> StorageInfoSubscribers; + TActorId UnlockedFromActor; public: TTxDeleteTabletResult(TEvTabletBase::TEvDeleteTabletResult::TPtr& ev, THive* hive) @@ -65,7 +65,7 @@ public: ctx.Schedule(TDuration::MilliSeconds(1000), new TEvHive::TEvInitiateDeleteStorage(Tablet->Id)); } } - for (const TActorId& subscriber : StorageInfoSubscribers) { + for (const TActorId& subscriber : StorageInfoSubscribers) { ctx.Send( subscriber, new TEvHive::TEvGetTabletStorageInfoResult(TabletId, NKikimrProto::ERROR, "Tablet deleted")); diff --git a/ydb/core/mind/hive/tx__init_scheme.cpp b/ydb/core/mind/hive/tx__init_scheme.cpp index 1c8ed32c560..b65451f94a7 100644 --- a/ydb/core/mind/hive/tx__init_scheme.cpp +++ b/ydb/core/mind/hive/tx__init_scheme.cpp @@ -94,7 +94,7 @@ public: void Complete(const TActorContext& ctx) override { BLOG_D("THive::TTxInitScheme::Complete"); - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); ctx.Send(nameserviceId, new TEvInterconnect::TEvListNodes()); } }; diff --git a/ydb/core/mind/hive/tx__kill_node.cpp b/ydb/core/mind/hive/tx__kill_node.cpp index ce79a4cbe09..268327a0ddd 100644 --- a/ydb/core/mind/hive/tx__kill_node.cpp +++ b/ydb/core/mind/hive/tx__kill_node.cpp @@ -7,7 +7,7 @@ namespace NHive { class TTxKillNode : public TTransactionBase<THive> { protected: TNodeId NodeId; - TActorId Local; + TActorId Local; public: TTxKillNode(TNodeId nodeId, const TActorId& local, THive *hive) : TBase(hive) @@ -50,7 +50,7 @@ public: db.Table<Schema::Node>().Key(NodeId).Delete(); Self->DeleteNode(NodeId); } else { - db.Table<Schema::Node>().Key(NodeId).Update<Schema::Node::Local>(TActorId()); + db.Table<Schema::Node>().Key(NodeId).Update<Schema::Node::Local>(TActorId()); } } return true; diff --git a/ydb/core/mind/hive/tx__lock_tablet.cpp b/ydb/core/mind/hive/tx__lock_tablet.cpp index c1e847abcbf..6365dd09886 100644 --- a/ydb/core/mind/hive/tx__lock_tablet.cpp +++ b/ydb/core/mind/hive/tx__lock_tablet.cpp @@ -7,19 +7,19 @@ namespace NHive { class TTxLockTabletExecution : public TTransactionBase<THive> { private: const ui64 TabletId; - const TActorId OwnerActor; + const TActorId OwnerActor; const TDuration ReconnectTimeout; const bool IsReconnect; - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; NKikimrProto::EReplyStatus Status; TString StatusMessage; - TActorId PreviousOwner; + TActorId PreviousOwner; public: - TTxLockTabletExecution(const NKikimrHive::TEvLockTabletExecution& rec, const TActorId& sender, const ui64 cookie, THive* hive) + TTxLockTabletExecution(const NKikimrHive::TEvLockTabletExecution& rec, const TActorId& sender, const ui64 cookie, THive* hive) : TBase(hive) , TabletId(rec.GetTabletID()) , OwnerActor(GetOwnerActor(rec, sender)) @@ -109,16 +109,16 @@ public: } private: - static TActorId GetOwnerActor(const NKikimrHive::TEvLockTabletExecution& rec, const TActorId& sender) { - TActorId owner = sender; + static TActorId GetOwnerActor(const NKikimrHive::TEvLockTabletExecution& rec, const TActorId& sender) { + TActorId owner = sender; if (rec.HasOwnerActor()) { - owner = ActorIdFromProto(rec.GetOwnerActor()); + owner = ActorIdFromProto(rec.GetOwnerActor()); } return owner; } }; -ITransaction* THive::CreateLockTabletExecution(const NKikimrHive::TEvLockTabletExecution& rec, const TActorId& sender, const ui64 cookie) { +ITransaction* THive::CreateLockTabletExecution(const NKikimrHive::TEvLockTabletExecution& rec, const TActorId& sender, const ui64 cookie) { return new TTxLockTabletExecution(rec, sender, cookie, this); } diff --git a/ydb/core/mind/hive/tx__reassign_groups.cpp b/ydb/core/mind/hive/tx__reassign_groups.cpp index 7571855e776..ec9f4bfdd23 100644 --- a/ydb/core/mind/hive/tx__reassign_groups.cpp +++ b/ydb/core/mind/hive/tx__reassign_groups.cpp @@ -7,13 +7,13 @@ namespace NHive { class TTxReassignGroups : public TTransactionBase<THive> { protected: TTabletId TabletId; - TActorId Sender; + TActorId Sender; std::bitset<MAX_TABLET_CHANNELS> ChannelProfileNewGroup; bool InitiatedReassignTablet = false; public: TTxReassignGroups(TTabletId tabletId, - const TActorId& sender, + const TActorId& sender, const std::bitset<MAX_TABLET_CHANNELS>& channelProfileNewGroup, THive *hive) : TBase(hive) @@ -69,7 +69,7 @@ public: }; ITransaction* THive::CreateReassignGroups(TTabletId tabletId, - const TActorId& actorToNotify, + const TActorId& actorToNotify, const std::bitset<MAX_TABLET_CHANNELS>& channelProfileNewGroup) { return new TTxReassignGroups(tabletId, actorToNotify, channelProfileNewGroup, this); } diff --git a/ydb/core/mind/hive/tx__register_node.cpp b/ydb/core/mind/hive/tx__register_node.cpp index 4154e7d07fd..941c0dce658 100644 --- a/ydb/core/mind/hive/tx__register_node.cpp +++ b/ydb/core/mind/hive/tx__register_node.cpp @@ -5,11 +5,11 @@ namespace NKikimr { namespace NHive { class TTxRegisterNode : public TTransactionBase<THive> { - TActorId Local; + TActorId Local; NKikimrLocal::TEvRegisterNode Record; public: - TTxRegisterNode(const TActorId& local, NKikimrLocal::TEvRegisterNode record, THive *hive) + TTxRegisterNode(const TActorId& local, NKikimrLocal::TEvRegisterNode record, THive *hive) : TBase(hive) , Local(local) , Record(std::move(record)) @@ -74,7 +74,7 @@ public: } }; -ITransaction* THive::CreateRegisterNode(const TActorId& local, NKikimrLocal::TEvRegisterNode rec) { +ITransaction* THive::CreateRegisterNode(const TActorId& local, NKikimrLocal::TEvRegisterNode rec) { return new TTxRegisterNode(local, std::move(rec), this); } diff --git a/ydb/core/mind/hive/tx__start_tablet.cpp b/ydb/core/mind/hive/tx__start_tablet.cpp index 35bf5c7eeda..4cef3a0e069 100644 --- a/ydb/core/mind/hive/tx__start_tablet.cpp +++ b/ydb/core/mind/hive/tx__start_tablet.cpp @@ -6,13 +6,13 @@ namespace NHive { class TTxStartTablet : public TTransactionBase<THive> { TFullTabletId TabletId; - TActorId Local; + TActorId Local; ui64 Cookie; bool External; ui32 KnownGeneration = -1; public: - TTxStartTablet(TFullTabletId tabletId, const TActorId& local, ui64 cookie, bool external, THive *hive) + TTxStartTablet(TFullTabletId tabletId, const TActorId& local, ui64 cookie, bool external, THive *hive) : TBase(hive) , TabletId(tabletId) , Local(local) @@ -106,7 +106,7 @@ public: } }; -ITransaction* THive::CreateStartTablet(TFullTabletId tabletId, const TActorId& local, ui64 cookie, bool external) { +ITransaction* THive::CreateStartTablet(TFullTabletId tabletId, const TActorId& local, ui64 cookie, bool external) { return new TTxStartTablet(tabletId, local, cookie, external, this); } diff --git a/ydb/core/mind/hive/tx__status.cpp b/ydb/core/mind/hive/tx__status.cpp index 304af25c124..e45fa5055a1 100644 --- a/ydb/core/mind/hive/tx__status.cpp +++ b/ydb/core/mind/hive/tx__status.cpp @@ -5,11 +5,11 @@ namespace NKikimr { namespace NHive { class TTxStatus : public TTransactionBase<THive> { - TActorId Local; + TActorId Local; NKikimrLocal::TEvStatus Record; public: - TTxStatus(const TActorId& local, NKikimrLocal::TEvStatus record, THive* hive) + TTxStatus(const TActorId& local, NKikimrLocal::TEvStatus record, THive* hive) : TBase(hive) , Local(local) , Record(std::move(record)) @@ -57,7 +57,7 @@ public: } }; -ITransaction* THive::CreateStatus(const TActorId& local, NKikimrLocal::TEvStatus rec) { +ITransaction* THive::CreateStatus(const TActorId& local, NKikimrLocal::TEvStatus rec) { return new TTxStatus(local, std::move(rec), this); } diff --git a/ydb/core/mind/hive/tx__stop_tablet.cpp b/ydb/core/mind/hive/tx__stop_tablet.cpp index 0b99adcd9a7..c05103b258d 100644 --- a/ydb/core/mind/hive/tx__stop_tablet.cpp +++ b/ydb/core/mind/hive/tx__stop_tablet.cpp @@ -6,11 +6,11 @@ namespace NHive { class TTxStopTablet : public TTransactionBase<THive> { const TTabletId TabletId; - const TActorId ActorToNotify; + const TActorId ActorToNotify; NKikimrProto::EReplyStatus Status; public: - TTxStopTablet(ui64 tabletId, const TActorId &actorToNotify, THive *hive) + TTxStopTablet(ui64 tabletId, const TActorId &actorToNotify, THive *hive) : TBase(hive) , TabletId(tabletId) , ActorToNotify(actorToNotify) @@ -87,7 +87,7 @@ public: } }; -ITransaction* THive::CreateStopTablet(TTabletId tabletId, const TActorId &actorToNotify) { +ITransaction* THive::CreateStopTablet(TTabletId tabletId, const TActorId &actorToNotify) { return new TTxStopTablet(tabletId, actorToNotify, this); } diff --git a/ydb/core/mind/hive/tx__sync_tablets.cpp b/ydb/core/mind/hive/tx__sync_tablets.cpp index f14ffe43d36..29eb4cfba1c 100644 --- a/ydb/core/mind/hive/tx__sync_tablets.cpp +++ b/ydb/core/mind/hive/tx__sync_tablets.cpp @@ -5,12 +5,12 @@ namespace NKikimr { namespace NHive { class TTxSyncTablets : public TTransactionBase<THive> { - TActorId Local; + TActorId Local; NKikimrLocal::TEvSyncTablets SyncTablets; THashSet<std::pair<TTabletId, TFollowerId>> TabletsToStop; THashSet<std::pair<TTabletId, TFollowerId>> TabletsToBoot; public: - TTxSyncTablets(const TActorId &local, NKikimrLocal::TEvSyncTablets& rec, THive* hive) + TTxSyncTablets(const TActorId &local, NKikimrLocal::TEvSyncTablets& rec, THive* hive) : TBase(hive) , Local(local) { @@ -126,7 +126,7 @@ public: } }; -ITransaction* THive::CreateSyncTablets(const TActorId &local, NKikimrLocal::TEvSyncTablets& rec) { +ITransaction* THive::CreateSyncTablets(const TActorId &local, NKikimrLocal::TEvSyncTablets& rec) { return new TTxSyncTablets(local, rec, this); } diff --git a/ydb/core/mind/hive/tx__unlock_tablet.cpp b/ydb/core/mind/hive/tx__unlock_tablet.cpp index b28ef729b38..66786b43b2a 100644 --- a/ydb/core/mind/hive/tx__unlock_tablet.cpp +++ b/ydb/core/mind/hive/tx__unlock_tablet.cpp @@ -6,18 +6,18 @@ namespace NHive { class TTxUnlockTabletExecution : public TTransactionBase<THive> { const ui64 TabletId; - const TActorId OwnerActor; + const TActorId OwnerActor; const ui64 SeqNo; - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; NKikimrProto::EReplyStatus Status; TString StatusMessage; - TActorId PreviousOwner; + TActorId PreviousOwner; public: - TTxUnlockTabletExecution(const NKikimrHive::TEvUnlockTabletExecution& rec, const TActorId& sender, const ui64 cookie, THive* hive) + TTxUnlockTabletExecution(const NKikimrHive::TEvUnlockTabletExecution& rec, const TActorId& sender, const ui64 cookie, THive* hive) : TBase(hive) , TabletId(rec.GetTabletID()) , OwnerActor(GetOwnerActor(rec, sender)) @@ -100,16 +100,16 @@ public: } private: - static TActorId GetOwnerActor(const NKikimrHive::TEvUnlockTabletExecution& rec, const TActorId& sender) { - TActorId owner = sender; + static TActorId GetOwnerActor(const NKikimrHive::TEvUnlockTabletExecution& rec, const TActorId& sender) { + TActorId owner = sender; if (rec.HasOwnerActor()) { - owner = ActorIdFromProto(rec.GetOwnerActor()); + owner = ActorIdFromProto(rec.GetOwnerActor()); } return owner; } }; -ITransaction* THive::CreateUnlockTabletExecution(const NKikimrHive::TEvUnlockTabletExecution& rec, const TActorId& sender, const ui64 cookie) { +ITransaction* THive::CreateUnlockTabletExecution(const NKikimrHive::TEvUnlockTabletExecution& rec, const TActorId& sender, const ui64 cookie) { return new TTxUnlockTabletExecution(rec, sender, cookie, this); } diff --git a/ydb/core/mind/hive/tx__update_tablet_groups.cpp b/ydb/core/mind/hive/tx__update_tablet_groups.cpp index c672db24d23..2d9a11bc3b8 100644 --- a/ydb/core/mind/hive/tx__update_tablet_groups.cpp +++ b/ydb/core/mind/hive/tx__update_tablet_groups.cpp @@ -213,7 +213,7 @@ public: if (tablet->IsBootingSuppressed()) { // Tablet will never boot, so will notify about creation right after commit - for (const TActorId& actor : tablet->ActorsToNotify) { + for (const TActorId& actor : tablet->ActorsToNotify) { Notifications.Send(actor, new TEvHive::TEvTabletCreationResult(NKikimrProto::OK, TabletId)); } tablet->ActorsToNotify.clear(); diff --git a/ydb/core/mind/hive/tx__update_tablet_status.cpp b/ydb/core/mind/hive/tx__update_tablet_status.cpp index 5b6180644e5..7e682e857d0 100644 --- a/ydb/core/mind/hive/tx__update_tablet_status.cpp +++ b/ydb/core/mind/hive/tx__update_tablet_status.cpp @@ -6,7 +6,7 @@ namespace NHive { class TTxUpdateTabletStatus : public TTransactionBase<THive> { const TTabletId TabletId; - const TActorId Local; + const TActorId Local; const TEvLocal::TEvTabletStatus::EStatus Status; const TEvTablet::TEvTabletDead::EReason Reason; ui32 Generation; @@ -16,7 +16,7 @@ class TTxUpdateTabletStatus : public TTransactionBase<THive> { public: TTxUpdateTabletStatus( TTabletId tabletId, - const TActorId &local, + const TActorId &local, ui32 generation, TFollowerId followerId, TEvLocal::TEvTabletStatus::EStatus status, @@ -117,7 +117,7 @@ public: NIceDb::TUpdate<Schema::TabletFollowerTablet::FollowerNode>(Local.NodeId()), NIceDb::TUpdate<Schema::TabletFollowerTablet::Statistics>(tablet->Statistics)); } - for (const TActorId& actor : tablet->ActorsToNotify) { + for (const TActorId& actor : tablet->ActorsToNotify) { Notifications.Send(actor, new TEvHive::TEvTabletCreationResult(NKikimrProto::OK, TabletId)); } tablet->ActorsToNotify.clear(); @@ -198,7 +198,7 @@ public: ITransaction* THive::CreateUpdateTabletStatus( TTabletId tabletId, - const TActorId &local, + const TActorId &local, ui32 generation, TFollowerId followerId, TEvLocal::TEvTabletStatus::EStatus status, diff --git a/ydb/core/mind/hive/ya.make b/ydb/core/mind/hive/ya.make index 30f65682aa8..171f26f1b5f 100644 --- a/ydb/core/mind/hive/ya.make +++ b/ydb/core/mind/hive/ya.make @@ -82,7 +82,7 @@ PEERDIR( library/cpp/actors/core library/cpp/actors/interconnect library/cpp/json - library/cpp/monlib/dynamic_counters + library/cpp/monlib/dynamic_counters ydb/core/base ydb/core/blobstorage/base ydb/core/blobstorage/crypto diff --git a/ydb/core/mind/lease_holder.cpp b/ydb/core/mind/lease_holder.cpp index 09d483b0a94..5d47d04c71f 100644 --- a/ydb/core/mind/lease_holder.cpp +++ b/ydb/core/mind/lease_holder.cpp @@ -8,7 +8,7 @@ #include <ydb/core/base/appdata.h> #include <ydb/core/base/tablet_pipe.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/random/random.h> @@ -101,7 +101,7 @@ private: void OnPipeDestroyedIdle(const TActorContext &ctx) { NTabletPipe::CloseClient(ctx, NodeBrokerPipe); - NodeBrokerPipe = TActorId(); + NodeBrokerPipe = TActorId(); } void HandleIdle(TEvTabletPipe::TEvClientConnected::TPtr &ev, const TActorContext &ctx) { @@ -122,7 +122,7 @@ private: void OnPipeDestroyed(const TActorContext &ctx) { NTabletPipe::CloseClient(ctx, NodeBrokerPipe); - NodeBrokerPipe = TActorId(); + NodeBrokerPipe = TActorId(); Ping(ctx); } @@ -244,7 +244,7 @@ private: } private: - TActorId NodeBrokerPipe; + TActorId NodeBrokerPipe; ui64 LastPingEpoch; TInstant EpochEnd; TInstant Expire; diff --git a/ydb/core/mind/local.cpp b/ydb/core/mind/local.cpp index 75e5740bdee..09fc15f76dd 100644 --- a/ydb/core/mind/local.cpp +++ b/ydb/core/mind/local.cpp @@ -44,7 +44,7 @@ class TLocalNodeRegistrar : public TActorBootstrapped<TLocalNodeRegistrar> { }; struct TTablet { - TActorId Tablet; + TActorId Tablet; ui32 Generation; TTabletTypes::EType TabletType; NKikimrLocal::EBootMode BootMode; @@ -76,16 +76,16 @@ class TLocalNodeRegistrar : public TActorBootstrapped<TLocalNodeRegistrar> { {} }; - const TActorId Owner; + const TActorId Owner; const ui64 HiveId; TVector<TSubDomainKey> ServicedDomains; - TActorId HivePipeClient; + TActorId HivePipeClient; bool Connected; TIntrusivePtr<TLocalConfig> Config; - TActorId BootQueue; + TActorId BootQueue; ui32 HiveGeneration; TActorId KnownHiveLeader; @@ -142,7 +142,7 @@ class TLocalNodeRegistrar : public TActorBootstrapped<TLocalNodeRegistrar> { } NTabletPipe::CloseClient(ctx, HivePipeClient); } - HivePipeClient = TActorId(); + HivePipeClient = TActorId(); for (const auto &xpair : OnlineTablets) { ctx.Send(xpair.second.Tablet, new TEvents::TEvPoisonPill()); @@ -201,7 +201,7 @@ class TLocalNodeRegistrar : public TActorBootstrapped<TLocalNodeRegistrar> { void HandlePipeDestroyed(const TActorContext &ctx) { LOG_DEBUG_S(ctx, NKikimrServices::LOCAL, "TLocalNodeRegistrar HandlePipeDestroyed - DISCONNECTED"); - HivePipeClient = TActorId(); + HivePipeClient = TActorId(); Connected = false; TryToRegister(ctx); if (SentDrainNode && !DrainResultReceived) { @@ -291,7 +291,7 @@ class TLocalNodeRegistrar : public TActorBootstrapped<TLocalNodeRegistrar> { void Handle(TEvLocal::TEvReconnect::TPtr& ev, const TActorContext& ctx) { LOG_DEBUG(ctx, NKikimrServices::LOCAL, "TLocalNodeRegistrar::Handle TEvLocal::TEvReconnect"); - const TActorId& sender = ev->Sender; + const TActorId& sender = ev->Sender; const NKikimrLocal::TEvReconnect& record = ev->Get()->Record; Y_VERIFY(HiveId == record.GetHiveId()); const ui32 hiveGen = record.GetHiveGeneration(); @@ -309,7 +309,7 @@ class TLocalNodeRegistrar : public TActorBootstrapped<TLocalNodeRegistrar> { void Handle(TEvLocal::TEvPing::TPtr &ev, const TActorContext &ctx) { LOG_DEBUG(ctx, NKikimrServices::LOCAL, "TLocalNodeRegistrar::Handle TEvLocal::TEvPing"); - const TActorId &sender = ev->Sender; + const TActorId &sender = ev->Sender; const NKikimrLocal::TEvPing &record = ev->Get()->Record; Y_VERIFY(HiveId == record.GetHiveId()); @@ -861,7 +861,7 @@ public: return NKikimrServices::TActivity::LOCAL_ACTOR; } - TLocalNodeRegistrar(const TActorId &owner, ui64 hiveId, TVector<TSubDomainKey> servicedDomains, + TLocalNodeRegistrar(const TActorId &owner, ui64 hiveId, TVector<TSubDomainKey> servicedDomains, const NKikimrTabletBase::TMetrics &resourceLimit, TIntrusivePtr<TLocalConfig> config, NMonitoring::TDynamicCounterPtr counters) : Owner(owner) @@ -941,7 +941,7 @@ public: class TDomainLocal : public TActorBootstrapped<TDomainLocal> { struct TResolveTask { TRegistrationInfo Info; - TVector<TActorId> Senders; + TVector<TActorId> Senders; }; struct TTenantInfo { @@ -954,8 +954,8 @@ class TDomainLocal : public TActorBootstrapped<TDomainLocal> { TTenantInfo(TTenantInfo &&other) = default; TRegistrationInfo Info; - TVector<TActorId> Locals; - TActorId Subscriber; + TVector<TActorId> Locals; + TActorId Subscriber; TVector<TTabletId> HiveIds; THashMap<TString, TString> Attributes; TSubDomainKey DomainKey; @@ -964,7 +964,7 @@ class TDomainLocal : public TActorBootstrapped<TDomainLocal> { TString Domain; ui64 SchemeRoot; TVector<ui64> HiveIds; - TActorId SchemeShardPipe; + TActorId SchemeShardPipe; NTabletPipe::TClientConfig PipeConfig; THashMap<TString, TResolveTask> ResolveTasks; TIntrusivePtr<TLocalConfig> Config; @@ -981,7 +981,7 @@ class TDomainLocal : public TActorBootstrapped<TDomainLocal> { { if (SchemeShardPipe) { NTabletPipe::CloseClient(ctx, SchemeShardPipe); - SchemeShardPipe = TActorId(); + SchemeShardPipe = TActorId(); } } @@ -996,7 +996,7 @@ class TDomainLocal : public TActorBootstrapped<TDomainLocal> { } void SendStatus(const TString &tenant, - TActorId recipient, + TActorId recipient, const TActorContext &ctx) { TAutoPtr<TEvLocal::TEvTenantStatus> ev; @@ -1019,7 +1019,7 @@ class TDomainLocal : public TActorBootstrapped<TDomainLocal> { void SendStatus(const TString &tenant, TEvLocal::TEvTenantStatus::EStatus status, const TString &msg, - TActorId recipient, + TActorId recipient, const TActorContext &ctx) { TAutoPtr<TEvLocal::TEvTenantStatus> ev = new TEvLocal::TEvTenantStatus(tenant, status, msg); @@ -1027,7 +1027,7 @@ class TDomainLocal : public TActorBootstrapped<TDomainLocal> { } void SendStatus(const TString &tenant, - const TVector<TActorId> &recipients, + const TVector<TActorId> &recipients, const TActorContext &ctx) { for (auto &r : recipients) @@ -1037,7 +1037,7 @@ class TDomainLocal : public TActorBootstrapped<TDomainLocal> { void SendStatus(const TString &tenant, TEvLocal::TEvTenantStatus::EStatus status, const TString &msg, - const TVector<TActorId> &recipients, + const TVector<TActorId> &recipients, const TActorContext &ctx) { for (auto &r : recipients) @@ -1052,11 +1052,11 @@ class TDomainLocal : public TActorBootstrapped<TDomainLocal> { { auto counters = GetServiceCounters(AppData(ctx)->Counters, "tablets"); auto actor = new TLocalNodeRegistrar(SelfId(), hiveId, servicedDomains, resourceLimit, Config, counters); - TActorId actorId = ctx.Register(actor); + TActorId actorId = ctx.Register(actor); RunningTenants.at(tenant).Locals.push_back(actorId); - TActorId localRegistrarServiceId = MakeLocalRegistrarID(ctx.SelfID.NodeId(), hiveId); + TActorId localRegistrarServiceId = MakeLocalRegistrarID(ctx.SelfID.NodeId(), hiveId); ctx.ExecutorThread.ActorSystem->RegisterLocalService(localRegistrarServiceId, actorId); } @@ -1089,7 +1089,7 @@ class TDomainLocal : public TActorBootstrapped<TDomainLocal> { for (auto &attr : rec.GetPathDescription().GetUserAttributes()) info.Attributes.emplace(std::make_pair(attr.GetKey(), attr.GetValue())); RunningTenants.emplace(std::make_pair(task.Info.TenantName, info)); - const TActorId whiteboardServiceId(NNodeWhiteboard::MakeNodeWhiteboardServiceId(SelfId().NodeId())); + const TActorId whiteboardServiceId(NNodeWhiteboard::MakeNodeWhiteboardServiceId(SelfId().NodeId())); Send(whiteboardServiceId, new NNodeWhiteboard::TEvWhiteboard::TEvSystemStateAddRole("Tenant")); Send(whiteboardServiceId, new NNodeWhiteboard::TEvWhiteboard::TEvSystemStateSetTenant(task.Info.TenantName)); for (TTabletId hId : hiveIds) { @@ -1115,7 +1115,7 @@ class TDomainLocal : public TActorBootstrapped<TDomainLocal> { LOG_DEBUG_S(ctx, NKikimrServices::LOCAL, LogPrefix << "Send poison pill to scheme subscriber of " << pr.second.Info.TenantName); ctx.Send(pr.second.Subscriber, new TEvents::TEvPoisonPill); - pr.second.Subscriber = TActorId(); + pr.second.Subscriber = TActorId(); } ctx.Send(NNodeWhiteboard::MakeNodeWhiteboardServiceId(ctx.SelfID.NodeId()), new NNodeWhiteboard::TEvWhiteboard::TEvSystemStateRemoveTenant(pr.first)); @@ -1306,7 +1306,7 @@ class TDomainLocal : public TActorBootstrapped<TDomainLocal> { ctx.Send(aid, new TEvents::TEvPoisonPill()); if (it->second.Subscriber) { ctx.Send(it->second.Subscriber, new TEvents::TEvPoisonPill()); - it->second.Subscriber = TActorId(); + it->second.Subscriber = TActorId(); } ctx.Send(NNodeWhiteboard::MakeNodeWhiteboardServiceId(ctx.SelfID.NodeId()), new NNodeWhiteboard::TEvWhiteboard::TEvSystemStateRemoveTenant(it->first)); @@ -1395,7 +1395,7 @@ public: class TLocal : public TActorBootstrapped<TLocal> { TIntrusivePtr<TLocalConfig> Config; - THashMap<TString, TActorId> DomainLocals; + THashMap<TString, TActorId> DomainLocals; public: static constexpr NKikimrServices::TActivity::EType ActorActivityType() { @@ -1436,7 +1436,7 @@ public: return true; } - void SendUnknownDomain(const TString &tenant, TActorId sender, const TActorContext &ctx) + void SendUnknownDomain(const TString &tenant, TActorId sender, const TActorContext &ctx) { TAutoPtr<TEvLocal::TEvTenantStatus> ev = new TEvLocal::TEvTenantStatus(tenant, TEvLocal::TEvTenantStatus::UNKNOWN_TENANT, diff --git a/ydb/core/mind/local.h b/ydb/core/mind/local.h index 49610505802..05c9bdc48eb 100644 --- a/ydb/core/mind/local.h +++ b/ydb/core/mind/local.h @@ -369,7 +369,7 @@ struct TLocalConfig : public TThrRefBase { IActor* CreateLocal(TLocalConfig *config); -inline TActorId MakeLocalID(ui32 node) { +inline TActorId MakeLocalID(ui32 node) { char x[12] = { 'l', 'o', 'c', 'l'}; x[4] = (char)(node & 0xFF); x[5] = (char)((node >> 8) & 0xFF); @@ -379,10 +379,10 @@ inline TActorId MakeLocalID(ui32 node) { x[9] = 0; x[10] = 0; x[11] = 0; - return TActorId(node, TStringBuf(x, 12)); + return TActorId(node, TStringBuf(x, 12)); } -inline TActorId MakeLocalRegistrarID(ui32 node, ui64 hiveId) { +inline TActorId MakeLocalRegistrarID(ui32 node, ui64 hiveId) { char x[12] = { 'l', 'o', 'c', 'l'}; x[4] = (char)(node & 0xFF); x[5] = (char)((node >> 8) & 0xFF); @@ -392,7 +392,7 @@ inline TActorId MakeLocalRegistrarID(ui32 node, ui64 hiveId) { x[9] = (char)(((hiveId >> 8) & 0xFF) ^ ((hiveId >> 40) & 0xFF)); x[10] = (char)(((hiveId >> 16) & 0xFF) ^ ((hiveId >> 48) & 0xFF)); x[11] = (char)(((hiveId >> 24) & 0xFF) ^ ((hiveId >> 56) & 0xFF)); - return TActorId(node, TStringBuf(x, 12)); + return TActorId(node, TStringBuf(x, 12)); } } diff --git a/ydb/core/mind/node_broker.cpp b/ydb/core/mind/node_broker.cpp index 2a98ff59541..a2ef9383691 100644 --- a/ydb/core/mind/node_broker.cpp +++ b/ydb/core/mind/node_broker.cpp @@ -9,7 +9,7 @@ #include <ydb/core/tablet_flat/tablet_flat_executed.h> #include <ydb/core/tx/scheme_cache/scheme_cache.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/generic/set.h> @@ -940,7 +940,7 @@ void TNodeBroker::Handle(TEvPrivate::TEvUpdateEpoch::TPtr &ev, ProcessTx(CreateTxUpdateEpoch(), ctx); } -IActor *CreateNodeBroker(const TActorId &tablet, +IActor *CreateNodeBroker(const TActorId &tablet, TTabletStorageInfo *info) { return new TNodeBroker(tablet, info); diff --git a/ydb/core/mind/node_broker.h b/ydb/core/mind/node_broker.h index 66762553dec..c86edfe0e7a 100644 --- a/ydb/core/mind/node_broker.h +++ b/ydb/core/mind/node_broker.h @@ -195,7 +195,7 @@ inline ui32 NodeIdToDomain(ui32 nodeId, const TDomainsInfo& domains) { return NodeIdToDomain(nodeId); } -IActor *CreateNodeBroker(const TActorId &tablet, TTabletStorageInfo *info); +IActor *CreateNodeBroker(const TActorId &tablet, TTabletStorageInfo *info); } // NNodeBroker } // NKikimr diff --git a/ydb/core/mind/node_broker_impl.h b/ydb/core/mind/node_broker_impl.h index 4afbce47d94..e3b68b0587b 100644 --- a/ydb/core/mind/node_broker_impl.h +++ b/ydb/core/mind/node_broker_impl.h @@ -141,7 +141,7 @@ private: void Die(const TActorContext &ctx) override; template<typename TResponseEvent> - void ReplyWithError(TActorId sender, + void ReplyWithError(TActorId sender, NKikimrNodeBroker::TStatus::ECode code, const TString &reason, const TActorContext &ctx) @@ -317,7 +317,7 @@ private: bool SingleDomainAlloc = false; public: - TNodeBroker(const TActorId &tablet, TTabletStorageInfo *info) + TNodeBroker(const TActorId &tablet, TTabletStorageInfo *info) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, new NMiniKQL::TMiniKQLFactory) , EpochDuration(TDuration::Hours(1)) diff --git a/ydb/core/mind/node_broker_ut.cpp b/ydb/core/mind/node_broker_ut.cpp index a9a4fa75e22..0d9498ddcdd 100644 --- a/ydb/core/mind/node_broker_ut.cpp +++ b/ydb/core/mind/node_broker_ut.cpp @@ -161,7 +161,7 @@ void SetupServices(TTestActorRuntime &runtime, } void SetConfig(TTestActorRuntime& runtime, - TActorId sender, + TActorId sender, const NKikimrNodeBroker::TConfig &config) { auto event = MakeHolder<TEvNodeBroker::TEvSetConfigRequest>(); @@ -174,7 +174,7 @@ void SetConfig(TTestActorRuntime& runtime, } void SetEpochDuration(TTestActorRuntime& runtime, - TActorId sender, + TActorId sender, TDuration lease) { NKikimrNodeBroker::TConfig config; @@ -183,7 +183,7 @@ void SetEpochDuration(TTestActorRuntime& runtime, } void SetBannedIds(TTestActorRuntime& runtime, - TActorId sender, + TActorId sender, const TVector<std::pair<ui32, ui32>> ids) { NKikimrNodeBroker::TConfig config; @@ -252,7 +252,7 @@ MakeRegistrationRequest(const TString &host, } void CheckRegistration(TTestActorRuntime &runtime, - TActorId sender, + TActorId sender, const TString &host, ui16 port, const TString &resolveHost, @@ -292,7 +292,7 @@ void CheckRegistration(TTestActorRuntime &runtime, } NKikimrNodeBroker::TEpoch GetEpoch(TTestActorRuntime &runtime, - TActorId sender) + TActorId sender) { TAutoPtr<TEvNodeBroker::TEvListNodes> event = new TEvNodeBroker::TEvListNodes; runtime.SendToPipe(MakeNodeBrokerID(0), sender, event.Release(), 0, GetPipeConfigWithRetries()); @@ -303,7 +303,7 @@ NKikimrNodeBroker::TEpoch GetEpoch(TTestActorRuntime &runtime, } NKikimrNodeBroker::TEpoch WaitForEpochUpdate(TTestActorRuntime &runtime, - TActorId sender) + TActorId sender) { auto epoch = GetEpoch(runtime, sender); @@ -352,7 +352,7 @@ void CheckNodesListResponse(const NKikimrNodeBroker::TNodesInfo &rec, } NKikimrNodeBroker::TEpoch CheckFilteredNodesList(TTestActorRuntime &runtime, - TActorId sender, + TActorId sender, TSet<ui64> ids, TSet<ui64> expiredIds, ui64 minEpoch, @@ -395,7 +395,7 @@ NKikimrNodeBroker::TEpoch CheckFilteredNodesList(TTestActorRuntime &runtime, } NKikimrNodeBroker::TEpoch CheckNodesList(TTestActorRuntime &runtime, - TActorId sender, + TActorId sender, TSet<ui64> ids, TSet<ui64> expiredIds, ui64 epoch) @@ -414,7 +414,7 @@ NKikimrNodeBroker::TEpoch CheckNodesList(TTestActorRuntime &runtime, } void CheckNodeInfo(TTestActorRuntime &runtime, - TActorId sender, + TActorId sender, ui32 nodeId, TStatus::ECode code) { @@ -431,7 +431,7 @@ void CheckNodeInfo(TTestActorRuntime &runtime, } void CheckNodeInfo(TTestActorRuntime &runtime, - TActorId sender, + TActorId sender, ui32 nodeId, const TString &host, ui16 port, @@ -466,7 +466,7 @@ void CheckNodeInfo(TTestActorRuntime &runtime, } void CheckLeaseExtension(TTestActorRuntime &runtime, - TActorId sender, + TActorId sender, ui32 nodeId, TStatus::ECode code, const NKikimrNodeBroker::TEpoch &epoch = {}, @@ -509,12 +509,12 @@ TString AddrToString(const struct sockaddr *sa) } void CheckResolveNode(TTestActorRuntime &runtime, - TActorId sender, + TActorId sender, ui32 nodeId, const TString &addr) { TAutoPtr<TEvInterconnect::TEvResolveNode> event = new TEvInterconnect::TEvResolveNode(nodeId); - runtime.Send(new IEventHandle(GetNameserviceActorId(), sender, event.Release())); + runtime.Send(new IEventHandle(GetNameserviceActorId(), sender, event.Release())); TAutoPtr<IEventHandle> handle; auto reply = runtime.GrabEdgeEventRethrow<TEvLocalNodeInfo>(handle); @@ -525,11 +525,11 @@ void CheckResolveNode(TTestActorRuntime &runtime, } void CheckResolveUnknownNode(TTestActorRuntime &runtime, - TActorId sender, + TActorId sender, ui32 nodeId) { TAutoPtr<TEvInterconnect::TEvResolveNode> event = new TEvInterconnect::TEvResolveNode(nodeId); - runtime.Send(new IEventHandle(GetNameserviceActorId(), sender, event.Release())); + runtime.Send(new IEventHandle(GetNameserviceActorId(), sender, event.Release())); TAutoPtr<IEventHandle> handle; auto reply = runtime.GrabEdgeEventRethrow<TEvLocalNodeInfo>(handle); @@ -540,13 +540,13 @@ void CheckResolveUnknownNode(TTestActorRuntime &runtime, } void GetNameserverNodesList(TTestActorRuntime &runtime, - TActorId sender, + TActorId sender, THashMap<ui32, TEvInterconnect::TNodeInfo> &nodes, bool includeStatic) { ui32 maxStaticNodeId = runtime.GetAppData().DynamicNameserviceConfig->MaxStaticNodeId; TAutoPtr<TEvInterconnect::TEvListNodes> event = new TEvInterconnect::TEvListNodes; - runtime.Send(new IEventHandle(GetNameserviceActorId(), sender, event.Release())); + runtime.Send(new IEventHandle(GetNameserviceActorId(), sender, event.Release())); TAutoPtr<IEventHandle> handle; auto reply = runtime.GrabEdgeEventRethrow<TEvInterconnect::TEvNodesInfo>(handle); @@ -558,7 +558,7 @@ void GetNameserverNodesList(TTestActorRuntime &runtime, } void CheckNameserverNodesList(TTestActorRuntime &runtime, - TActorId sender, + TActorId sender, size_t count) { THashMap<ui32, TEvInterconnect::TNodeInfo> nodes; @@ -607,7 +607,7 @@ void CheckNameserverDynamicNodesList(const THashMap<ui32, TEvInterconnect::TNode template<typename... Ts> void CheckNameserverDynamicNodesList(TTestActorRuntime &runtime, - TActorId sender, + TActorId sender, size_t count, Ts... args) { @@ -618,12 +618,12 @@ void CheckNameserverDynamicNodesList(TTestActorRuntime &runtime, } void CheckGetNode(TTestActorRuntime &runtime, - TActorId sender, + TActorId sender, ui32 nodeId, bool exists) { TAutoPtr<TEvInterconnect::TEvGetNode> event = new TEvInterconnect::TEvGetNode(nodeId); - runtime.Send(new IEventHandle(GetNameserviceActorId(), sender, event.Release())); + runtime.Send(new IEventHandle(GetNameserviceActorId(), sender, event.Release())); TAutoPtr<IEventHandle> handle; auto reply = runtime.GrabEdgeEventRethrow<TEvInterconnect::TEvNodeInfo>(handle); @@ -647,7 +647,7 @@ Y_UNIT_TEST_SUITE(TNodeBrokerTest) { { TTestBasicRuntime runtime(8, false); Setup(runtime, 4); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); // There should be no dynamic nodes initially. auto epoch = GetEpoch(runtime, sender); @@ -723,7 +723,7 @@ Y_UNIT_TEST_SUITE(TNodeBrokerTest) { { TTestBasicRuntime runtime(8, false); Setup(runtime); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); auto epoch = GetEpoch(runtime, sender); // Register node 1024 with fixed ID. @@ -765,7 +765,7 @@ Y_UNIT_TEST_SUITE(TNodeBrokerTest) { { TTestBasicRuntime runtime(8, false); Setup(runtime, 10); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); WaitForEpochUpdate(runtime, sender); WaitForEpochUpdate(runtime, sender); @@ -791,7 +791,7 @@ Y_UNIT_TEST_SUITE(TNodeBrokerTest) { { TTestBasicRuntime runtime(8, false); Setup(runtime, 10); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); TVector<TString> hosts = { "host1", "host2", "host3", "host4", "host5", @@ -965,7 +965,7 @@ Y_UNIT_TEST_SUITE(TNodeBrokerTest) { { TTestBasicRuntime runtime(8, false); Setup(runtime, 10); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); SetBannedIds(runtime, sender, {{1025, 1280}}); @@ -1081,7 +1081,7 @@ Y_UNIT_TEST_SUITE(TDynamicNameserverTest) { { TTestBasicRuntime runtime(8, false); Setup(runtime); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); // Register node 1024. SetEpochDuration(runtime, sender, TDuration::Seconds(10)); @@ -1125,7 +1125,7 @@ Y_UNIT_TEST_SUITE(TDynamicNameserverTest) { { TTestBasicRuntime runtime(1, false); Setup(runtime); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); TVector<NKikimrNodeBroker::TListNodes> listRequests; TVector<NKikimrNodeBroker::TResolveNode> resolveRequests; diff --git a/ydb/core/mind/tenant_node_enumeration.cpp b/ydb/core/mind/tenant_node_enumeration.cpp index 1bc0a34eb7c..15ecddceb2a 100644 --- a/ydb/core/mind/tenant_node_enumeration.cpp +++ b/ydb/core/mind/tenant_node_enumeration.cpp @@ -24,12 +24,12 @@ static ui32 ExtractDefaultGroupForPath(const TString &path) { } class TTenantNodeEnumerationPublisher : public TActorBootstrapped<TTenantNodeEnumerationPublisher> { - TMap<TString, TActorId> PublishActors; + TMap<TString, TActorId> PublishActors; void Handle(TEvTenantPool::TEvTenantPoolStatus::TPtr &ev) { const auto &record = ev->Get()->Record; - TMap<TString, TActorId> toRemove; + TMap<TString, TActorId> toRemove; toRemove.swap(PublishActors); for (auto &x : record.GetSlots()) { @@ -44,7 +44,7 @@ class TTenantNodeEnumerationPublisher : public TActorBootstrapped<TTenantNodeEnu if (statestorageGroupId == Max<ui32>()) continue; - const TActorId publishActor = Register(CreateBoardPublishActor(assignedPath, TString(), SelfId(), statestorageGroupId, 0, true)); + const TActorId publishActor = Register(CreateBoardPublishActor(assignedPath, TString(), SelfId(), statestorageGroupId, 0, true)); PublishActors.emplace(assigned, publishActor); } @@ -76,14 +76,14 @@ public: }; class TTenantNodeEnumerationLookup : public TActorBootstrapped<TTenantNodeEnumerationLookup> { - const TActorId ReplyTo; + const TActorId ReplyTo; const TString TenantName; - TActorId LookupActor; + TActorId LookupActor; void PassAway() override { if (LookupActor) { Send(LookupActor, new TEvents::TEvPoisonPill()); - LookupActor = TActorId(); + LookupActor = TActorId(); } IActor::PassAway(); @@ -95,7 +95,7 @@ class TTenantNodeEnumerationLookup : public TActorBootstrapped<TTenantNodeEnumer } void Handle(TEvStateStorage::TEvBoardInfo::TPtr &ev) { - LookupActor = TActorId(); + LookupActor = TActorId(); auto *msg = ev->Get(); @@ -116,7 +116,7 @@ public: return NKikimrServices::TActivity::TENANT_NODES_ENUMERATION; } - TTenantNodeEnumerationLookup(TActorId replyTo, const TString &tenantName) + TTenantNodeEnumerationLookup(TActorId replyTo, const TString &tenantName) : ReplyTo(replyTo) , TenantName(tenantName) {} @@ -143,7 +143,7 @@ IActor* CreateTenantNodeEnumerationPublisher() { return new TTenantNodeEnumerationPublisher(); } -IActor* CreateTenantNodeEnumerationLookup(TActorId replyTo, const TString &tenantName) { +IActor* CreateTenantNodeEnumerationLookup(TActorId replyTo, const TString &tenantName) { return new TTenantNodeEnumerationLookup(replyTo, tenantName); } diff --git a/ydb/core/mind/tenant_node_enumeration.h b/ydb/core/mind/tenant_node_enumeration.h index 486cc1f803b..f7ee7e937ec 100644 --- a/ydb/core/mind/tenant_node_enumeration.h +++ b/ydb/core/mind/tenant_node_enumeration.h @@ -35,5 +35,5 @@ namespace NKikimr { }; IActor* CreateTenantNodeEnumerationPublisher(); - IActor* CreateTenantNodeEnumerationLookup(TActorId replyTo, const TString &tenantName); + IActor* CreateTenantNodeEnumerationLookup(TActorId replyTo, const TString &tenantName); } // namespace NKikimr diff --git a/ydb/core/mind/tenant_node_enumeration_ut.cpp b/ydb/core/mind/tenant_node_enumeration_ut.cpp index b65ebb686fc..f8c7a505c29 100644 --- a/ydb/core/mind/tenant_node_enumeration_ut.cpp +++ b/ydb/core/mind/tenant_node_enumeration_ut.cpp @@ -58,7 +58,7 @@ Y_UNIT_TEST_SUITE(TEnumerationTest) { runtime.Register(CreateTenantNodeEnumerationPublisher(), 0); runtime.DispatchEvents(TDispatchOptions(), TDuration::MilliSeconds(1000)); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); runtime.Register(CreateTenantNodeEnumerationLookup(sender, "/" + DOMAIN1_NAME)); TAutoPtr<IEventHandle> handle; diff --git a/ydb/core/mind/tenant_pool.cpp b/ydb/core/mind/tenant_pool.cpp index 43e7cfb53d4..80236570b9e 100644 --- a/ydb/core/mind/tenant_pool.cpp +++ b/ydb/core/mind/tenant_pool.cpp @@ -16,7 +16,7 @@ #include <library/cpp/actors/core/hfunc.h> #include <library/cpp/actors/core/log.h> #include <library/cpp/actors/core/mon.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/generic/queue.h> #include <util/generic/hash.h> @@ -112,8 +112,8 @@ struct TTenantSlotBrokerInfo { ui64 TabletId = 0; ui64 Generation = 0; ui64 SeqNo = 0; - TActorId ActorId; - TActorId Pipe; + TActorId ActorId; + TActorId Pipe; }; class TDomainTenantPool : public TActorBootstrapped<TDomainTenantPool> { @@ -121,12 +121,12 @@ class TDomainTenantPool : public TActorBootstrapped<TDomainTenantPool> { TString DomainName; TString LogPrefix; - TActorId LocalID; + TActorId LocalID; TTenantSlotBrokerInfo TenantSlotBroker; TTenantPoolConfig::TPtr Config; THashMap<TString, TDynamicSlotInfo::TPtr> DynamicSlots; THashMap<TString, TTenantInfo::TPtr> Tenants; - THashSet<TActorId> StatusSubscribers; + THashSet<TActorId> StatusSubscribers; public: static constexpr NKikimrServices::TActivity::EType ActorActivityType() @@ -134,7 +134,7 @@ public: return NKikimrServices::TActivity::TENANT_POOL_ACTOR; } - TDomainTenantPool(const TString &domain, TActorId localID, TTenantPoolConfig::TPtr config) + TDomainTenantPool(const TString &domain, TActorId localID, TTenantPoolConfig::TPtr config) : DomainName(domain) , LocalID(localID) , Config(config) @@ -171,7 +171,7 @@ public: TenantSlotBroker.Pipe = ctx.ExecutorThread.RegisterActor(pipe); auto request = MakeHolder<TEvTenantSlotBroker::TEvRegisterPool>(); - ActorIdToProto(TenantSlotBroker.Pipe, request->Record.MutableClientId()); + ActorIdToProto(TenantSlotBroker.Pipe, request->Record.MutableClientId()); request->Record.SetSeqNo(++TenantSlotBroker.SeqNo); NTabletPipe::SendData(ctx, TenantSlotBroker.Pipe, request.Release()); @@ -788,8 +788,8 @@ public: class TTenantPool : public TActorBootstrapped<TTenantPool> { TTenantPoolConfig::TPtr Config; - TActorId LocalID; - THashMap<TString, TActorId> DomainTenantPools; + TActorId LocalID; + THashMap<TString, TActorId> DomainTenantPools; const TString DomainPrefix; public: diff --git a/ydb/core/mind/tenant_pool.h b/ydb/core/mind/tenant_pool.h index a277e6cffa6..0202ba78634 100644 --- a/ydb/core/mind/tenant_pool.h +++ b/ydb/core/mind/tenant_pool.h @@ -88,16 +88,16 @@ public: IActor* CreateTenantPool(TTenantPoolConfig::TPtr config); -inline TActorId MakeTenantPoolID(ui32 node = 0, ui32 domain = 0) { +inline TActorId MakeTenantPoolID(ui32 node = 0, ui32 domain = 0) { Y_VERIFY(domain < 32); char x[12] = { 't', 'e', 'n', 'a', 'n', 't', 'p', 'o', 'o', 'l' }; x[10] = static_cast<char>(domain); - return TActorId(node, TStringBuf(x, 12)); + return TActorId(node, TStringBuf(x, 12)); } -inline TActorId MakeTenantPoolRootID() { +inline TActorId MakeTenantPoolRootID() { char x[12] = { 't', 'e', 'n', 'a', 'n', 't', 'p', 'o', 'o', 'l', 'r', 't' }; - return TActorId(0, TStringBuf(x, 12)); + return TActorId(0, TStringBuf(x, 12)); } } // namespace NKikimr diff --git a/ydb/core/mind/tenant_slot_broker.cpp b/ydb/core/mind/tenant_slot_broker.cpp index c028c58a37a..65f7a86328b 100644 --- a/ydb/core/mind/tenant_slot_broker.cpp +++ b/ydb/core/mind/tenant_slot_broker.cpp @@ -3,7 +3,7 @@ #include <ydb/core/base/counters.h> #include <ydb/core/cms/console/config_helpers.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> namespace NKikimr { namespace NTenantSlotBroker { @@ -1458,7 +1458,7 @@ bool TTenantSlotBroker::AssignFreeSlotsForGroup(TTenant::TPtr tenant, return false; } -void TTenantSlotBroker::OnClientDisconnected(TActorId clientId, +void TTenantSlotBroker::OnClientDisconnected(TActorId clientId, const TActorContext &ctx) { if (KnownPoolPipes.contains(clientId)) { @@ -1772,14 +1772,14 @@ void TTenantSlotBroker::Handle(TEvTenantSlotBroker::TEvRegisterPool::TPtr &ev, const auto &record = ev->Get()->Record; auto nodeId = ev->Sender.NodeId(); - KnownPoolPipes.insert(ActorIdFromProto(record.GetClientId())); + KnownPoolPipes.insert(ActorIdFromProto(record.GetClientId())); *Counters->ConnectedPools = KnownPoolPipes.size(); DisconnectNodeSlots(nodeId, ctx); ctx.Send(ev->Sender, new TEvTenantPool::TEvTakeOwnership(Generation(), record.GetSeqNo())); } -IActor *CreateTenantSlotBroker(const TActorId &tablet, +IActor *CreateTenantSlotBroker(const TActorId &tablet, TTabletStorageInfo *info) { return new TTenantSlotBroker(tablet, info); diff --git a/ydb/core/mind/tenant_slot_broker.h b/ydb/core/mind/tenant_slot_broker.h index 0df3a9142ed..922924fb14d 100644 --- a/ydb/core/mind/tenant_slot_broker.h +++ b/ydb/core/mind/tenant_slot_broker.h @@ -53,7 +53,7 @@ struct TEvTenantSlotBroker { }; -IActor *CreateTenantSlotBroker(const TActorId &tablet, TTabletStorageInfo *info); +IActor *CreateTenantSlotBroker(const TActorId &tablet, TTabletStorageInfo *info); } // NTenantSlotBroker } // namespace NKikimr diff --git a/ydb/core/mind/tenant_slot_broker__load_state.cpp b/ydb/core/mind/tenant_slot_broker__load_state.cpp index 02663e75538..345b131ea3c 100644 --- a/ydb/core/mind/tenant_slot_broker__load_state.cpp +++ b/ydb/core/mind/tenant_slot_broker__load_state.cpp @@ -172,7 +172,7 @@ public: "Taking ownership of tenant pool on node " << pr.first); ctx.Send(MakeTenantPoolID(pr.first, Self->DomainId), new TEvTenantPool::TEvTakeOwnership(Self->Generation())); - ctx.Send(GetNameserviceActorId(), new TEvInterconnect::TEvGetNode(pr.first)); + ctx.Send(GetNameserviceActorId(), new TEvInterconnect::TEvGetNode(pr.first)); for (auto& slot : pr.second) { const ui64 randomDelay = TAppData::RandomProvider->GenRand64() % Self->PendingTimeout.GetValue(); diff --git a/ydb/core/mind/tenant_slot_broker__update_pool_status.cpp b/ydb/core/mind/tenant_slot_broker__update_pool_status.cpp index a92dac1e0f5..1e43062762c 100644 --- a/ydb/core/mind/tenant_slot_broker__update_pool_status.cpp +++ b/ydb/core/mind/tenant_slot_broker__update_pool_status.cpp @@ -75,7 +75,7 @@ public: } if (!Self->NodeIdToDataCenter.contains(nodeId)) - ctx.Send(GetNameserviceActorId(), new TEvInterconnect::TEvGetNode(nodeId)); + ctx.Send(GetNameserviceActorId(), new TEvInterconnect::TEvGetNode(nodeId)); return true; } diff --git a/ydb/core/mind/tenant_slot_broker_impl.h b/ydb/core/mind/tenant_slot_broker_impl.h index f19ed30a5c1..b5e3b107ab5 100644 --- a/ydb/core/mind/tenant_slot_broker_impl.h +++ b/ydb/core/mind/tenant_slot_broker_impl.h @@ -1043,7 +1043,7 @@ private: TCollocationGroup::TPtr group, TTransactionContext &txc, const TActorContext &ctx); - void OnClientDisconnected(TActorId clientId, + void OnClientDisconnected(TActorId clientId, const TActorContext &ctx); void DisconnectNodeSlots(ui32 nodeId, const TActorContext &ctx); @@ -1131,7 +1131,7 @@ private: } public: - TTenantSlotBroker(const TActorId &tablet, TTabletStorageInfo *info) + TTenantSlotBroker(const TActorId &tablet, TTabletStorageInfo *info) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, new NMiniKQL::TMiniKQLFactory) , PendingAssignFreeSlots(false) @@ -1172,7 +1172,7 @@ private: TSet<TTenant::TPtr, TTenantMoreMisplaced> MisplacedTenants; TSet<TTenant::TPtr, TTenantMoreSplit> SplitTenants; bool PendingAssignFreeSlots; - THashSet<TActorId> KnownPoolPipes; + THashSet<TActorId> KnownPoolPipes; TCounters::TPtr Counters; ITransaction *ActiveTx = nullptr; TDeque<THolder<ITransaction>> TxQueue; diff --git a/ydb/core/mind/tenant_ut_pool.cpp b/ydb/core/mind/tenant_ut_pool.cpp index 96cc6e7958d..9ac6b6843c3 100644 --- a/ydb/core/mind/tenant_ut_pool.cpp +++ b/ydb/core/mind/tenant_ut_pool.cpp @@ -17,7 +17,7 @@ struct TAction { }; class TActionChecker : public TActorBootstrapped<TActionChecker> { - TActorId EdgeActor; + TActorId EdgeActor; ui32 NodeId; ui32 Domain; TVector<TAction> Actions; @@ -25,7 +25,7 @@ class TActionChecker : public TActorBootstrapped<TActionChecker> { bool Finished; public: - TActionChecker(TActorId edge, ui32 nodeId, ui32 domain, TVector<TAction> actions) + TActionChecker(TActorId edge, ui32 nodeId, ui32 domain, TVector<TAction> actions) : EdgeActor(edge) , NodeId(nodeId) , Domain(domain) @@ -400,7 +400,7 @@ Y_UNIT_TEST_SUITE(TTenantPoolTests) { TTenantTestRuntime runtime(DefaultTenantTestConfig); TAutoPtr<IEventHandle> handle; - TActorId sender2 = runtime.AllocateEdgeActor(); + TActorId sender2 = runtime.AllocateEdgeActor(); // sender cannot configure slot because he is not owner runtime.Send(new IEventHandle(MakeTenantPoolID(runtime.GetNodeId(0), 0), @@ -1221,9 +1221,9 @@ Y_UNIT_TEST_SUITE(TTenantPoolTests) { TTenantTestRuntime runtime(config, {}, false); - const TActorId& sender = runtime.Sender; - const TActorId tenantPoolRoot = MakeTenantPoolRootID(); - const TActorId tenantPool = MakeTenantPoolID(runtime.GetNodeId(0), 0); + const TActorId& sender = runtime.Sender; + const TActorId tenantPoolRoot = MakeTenantPoolRootID(); + const TActorId tenantPool = MakeTenantPoolID(runtime.GetNodeId(0), 0); using TEvStatus = TEvTenantPool::TEvTenantPoolStatus; using EState = NKikimrTenantPool::EState; diff --git a/ydb/core/mind/ut_fat/blobstorage_node_warden_ut_fat.cpp b/ydb/core/mind/ut_fat/blobstorage_node_warden_ut_fat.cpp index c3776846342..011f03d93cd 100644 --- a/ydb/core/mind/ut_fat/blobstorage_node_warden_ut_fat.cpp +++ b/ydb/core/mind/ut_fat/blobstorage_node_warden_ut_fat.cpp @@ -216,7 +216,7 @@ void SetupServices(TTestActorRuntime &runtime) { runtime.Initialize(app.Unwrap()); for (ui32 nodeIndex = 0; nodeIndex < runtime.GetNodeCount(); ++nodeIndex) { - TActorId localActor = runtime.GetLocalServiceId( + TActorId localActor = runtime.GetLocalServiceId( MakeBlobStorageNodeWardenID(runtime.GetNodeId(nodeIndex)), nodeIndex); runtime.EnableScheduleForActor(localActor, true); } @@ -252,7 +252,7 @@ Y_UNIT_TEST_SUITE(TBlobStorageWardenTest) { return bsController; } - void CreatePDiskInBox(TTestActorRuntime& runtime, const TActorId& sender, ui32 nodeId, ui64 boxId, TString pdiskPath, + void CreatePDiskInBox(TTestActorRuntime& runtime, const TActorId& sender, ui32 nodeId, ui64 boxId, TString pdiskPath, std::optional<ui64> size) { if (size) { TFile file(pdiskPath, CreateAlways | RdWr | ARW); @@ -311,11 +311,11 @@ Y_UNIT_TEST_SUITE(TBlobStorageWardenTest) { return 0; } - void Put(TTestActorRuntime &runtime, TActorId &sender, ui32 groupId, TLogoBlobID logoBlobId, TString data) { + void Put(TTestActorRuntime &runtime, TActorId &sender, ui32 groupId, TLogoBlobID logoBlobId, TString data) { VERBOSE_COUT(" Sending TEvPut"); - TActorId proxy = MakeBlobStorageProxyID(groupId); + TActorId proxy = MakeBlobStorageProxyID(groupId); ui32 nodeId = sender.NodeId(); - TActorId nodeWarden = MakeBlobStorageNodeWardenID(nodeId); + TActorId nodeWarden = MakeBlobStorageNodeWardenID(nodeId); ui64 cookie = 6543210; runtime.Send(new IEventHandle(proxy, sender, new TEvBlobStorage::TEvPut(logoBlobId, data, TInstant::Max()), @@ -389,8 +389,8 @@ Y_UNIT_TEST_SUITE(TBlobStorageWardenTest) { TTempDir tempDir; TTestBasicRuntime runtime(2, false); Setup(runtime); - TActorId sender0 = runtime.AllocateEdgeActor(0); - TActorId sender1 = runtime.AllocateEdgeActor(1); + TActorId sender0 = runtime.AllocateEdgeActor(0); + TActorId sender1 = runtime.AllocateEdgeActor(1); CreatePDiskInBox(runtime, sender0, runtime.GetNodeId(0), 1, TFsPath(tempDir()) / "new_pdisk.dat", 16ULL << 30); const ui32 groupId = CreateGroupInBox(runtime, sender0, 1, 1, 0); @@ -400,7 +400,7 @@ Y_UNIT_TEST_SUITE(TBlobStorageWardenTest) { VGet(runtime, sender0, groupId, runtime.GetNodeId(0), TLogoBlobID(100, 0, 0, 0, 3, 0), "xxx", EExpectedEqualData); // TODO: the proxy just should not be there, check that instead! - TActorId proxy = MakeBlobStorageProxyID(groupId); + TActorId proxy = MakeBlobStorageProxyID(groupId); runtime.Send(new IEventHandle(proxy, sender1, new TEvents::TEvPoisonPill()), 1); Put(runtime, sender1, groupId, TLogoBlobID(100, 0, 1, 0, 3, 0), "yyy"); diff --git a/ydb/core/mon/mon.cpp b/ydb/core/mon/mon.cpp index 7c3d41f9014..d4722266f34 100644 --- a/ydb/core/mon/mon.cpp +++ b/ydb/core/mon/mon.cpp @@ -6,10 +6,10 @@ #include <library/cpp/actors/core/probes.h> #include <library/cpp/lwtrace/mon/mon_lwtrace.h> #include <library/cpp/mime/types/mime.h> -#include <library/cpp/monlib/service/pages/version_mon_page.h> -#include <library/cpp/monlib/service/pages/mon_page.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> -#include <library/cpp/monlib/dynamic_counters/page.h> +#include <library/cpp/monlib/service/pages/version_mon_page.h> +#include <library/cpp/monlib/service/pages/mon_page.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/page.h> #include <library/cpp/threading/future/future.h> #include <library/cpp/string_utils/url/url.h> #include <util/system/event.h> @@ -32,7 +32,7 @@ namespace NActors { return NKikimrServices::TActivity::ACTORLIB_COMMON; } - TMonRequest(const TActorId &targetActorId, IMonHttpRequest& request, + TMonRequest(const TActorId &targetActorId, IMonHttpRequest& request, NThreading::TPromise<THttpResponsePtr> result, const TVector<TString> &sids, TMon::TRequestAuthorizer authorizer) : TActor(&TMonRequest::StateFunc) , TargetActorId(targetActorId) @@ -210,13 +210,13 @@ namespace NActors { Die(ctx); } - virtual TAutoPtr<NActors::IEventHandle> AfterRegister(const NActors::TActorId &self, const TActorId& parentId) override { + virtual TAutoPtr<NActors::IEventHandle> AfterRegister(const NActors::TActorId &self, const TActorId& parentId) override { Y_UNUSED(parentId); return new NActors::IEventHandle(self, self, new TEvents::TEvBootstrap(), 0); } protected: - TActorId TargetActorId; + TActorId TargetActorId; IMonHttpRequest& Request; NThreading::TPromise<THttpResponsePtr> Result; const TVector<TString> &AllowedSIDs; @@ -260,8 +260,8 @@ namespace NActors { out << "<script language='javascript' type='text/javascript' src='https://yastatic.net/bootstrap/3.3.1/js/bootstrap.min.js'></script>\n"; if (OutputTableSorterJsCss) { - out << "<link rel='stylesheet' href='/jquery.tablesorter.css'>\n"; - out << "<script language='javascript' type='text/javascript' src='/jquery.tablesorter.js'></script>\n"; + out << "<link rel='stylesheet' href='/jquery.tablesorter.css'>\n"; + out << "<script language='javascript' type='text/javascript' src='/jquery.tablesorter.js'></script>\n"; } out << "<style type=\"text/css\">\n"; @@ -275,7 +275,7 @@ namespace NActors { out << "</head>"; out << "<body>"; - OutputNavBar(out); + OutputNavBar(out); out << "<div class='container'>"; if (Title) { @@ -330,7 +330,7 @@ namespace NActors { class TActorMonPage: public IMonPage { public: TActorMonPage(const TString &path, const TString &title, const TString &host, bool preTag, - TActorSystem *actorSystem, const TActorId &actorId, const TVector<TString> &sids, + TActorSystem *actorSystem, const TActorId &actorId, const TVector<TString> &sids, TMon::TRequestAuthorizer authorizer) : IMonPage(path, title) , Host(host) @@ -375,7 +375,7 @@ namespace NActors { TString Host; bool PreTag; TActorSystem *ActorSystem; - TActorId TargetActorId; + TActorId TargetActorId; const TVector<TString> AllowedSIDs; TMon::TRequestAuthorizer Authorizer; }; @@ -423,7 +423,7 @@ namespace NActors { } IMonPage *TMon::RegisterActorPage(TIndexMonPage *index, const TString &relPath, - const TString &title, bool preTag, TActorSystem *actorSystem, const TActorId &actorId, bool useAuth) { + const TString &title, bool preTag, TActorSystem *actorSystem, const TActorId &actorId, bool useAuth) { return RegisterActorPage({ .Title = title, .RelPath = relPath, diff --git a/ydb/core/mon/mon.h b/ydb/core/mon/mon.h index 118b4b93e99..1cb5dbecd0c 100644 --- a/ydb/core/mon/mon.h +++ b/ydb/core/mon/mon.h @@ -1,16 +1,16 @@ #pragma once -#include <library/cpp/monlib/service/monservice.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> -#include <library/cpp/monlib/service/pages/tablesorter/css_mon_page.h> -#include <library/cpp/monlib/service/pages/tablesorter/js_mon_page.h> +#include <library/cpp/monlib/service/monservice.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/service/pages/tablesorter/css_mon_page.h> +#include <library/cpp/monlib/service/pages/tablesorter/js_mon_page.h> #include <library/cpp/actors/core/mon.h> namespace NActors { class TActorSystem; - struct TActorId; + struct TActorId; class TMon : public NMonitoring::TMonService2 { public: @@ -48,7 +48,7 @@ namespace NActors { NMonitoring::IMonPage* RegisterActorPage(TRegisterActorPageFields fields); NMonitoring::IMonPage *RegisterActorPage(NMonitoring::TIndexMonPage *index, const TString &relPath, - const TString &title, bool preTag, TActorSystem *actorSystem, const TActorId &actorId, bool useAuth = true); + const TString &title, bool preTag, TActorSystem *actorSystem, const TActorId &actorId, bool useAuth = true); NMonitoring::IMonPage *RegisterCountersPage(const TString &path, const TString &title, TIntrusivePtr<NMonitoring::TDynamicCounters> counters); NMonitoring::IMonPage *FindPage(const TString &relPath); NMonitoring::TIndexMonPage *FindIndexPage(const TString &relPath); diff --git a/ydb/core/mon_alloc/monitor.cpp b/ydb/core/mon_alloc/monitor.cpp index 5372ac722cf..8afc7852c97 100644 --- a/ydb/core/mon_alloc/monitor.cpp +++ b/ydb/core/mon_alloc/monitor.cpp @@ -11,8 +11,8 @@ #include <library/cpp/html/pcdata/pcdata.h> #include <library/cpp/lfalloc/dbg_info/dbg_info.h> #include <library/cpp/malloc/api/malloc.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <library/cpp/ytalloc/api/ytalloc.h> #include <util/datetime/base.h> diff --git a/ydb/core/mon_alloc/monitor.h b/ydb/core/mon_alloc/monitor.h index 9dba0f51d6f..f6a8404b65d 100644 --- a/ydb/core/mon_alloc/monitor.h +++ b/ydb/core/mon_alloc/monitor.h @@ -5,16 +5,16 @@ #include <library/cpp/actors/core/defs.h> #include <library/cpp/actors/core/actor.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> namespace NActors { class TMon; } namespace NKikimr { - inline NActors::TActorId MakeMemProfMonitorID(ui32 node = 0) { + inline NActors::TActorId MakeMemProfMonitorID(ui32 node = 0) { char x[12] = {'m', 'e', 'm', 'p', 'r', 'o', 'f', 'm', 'o', 'n', 'i', 't'}; - return NActors::TActorId(node, TStringBuf(x, 12)); + return NActors::TActorId(node, TStringBuf(x, 12)); } struct IAllocMonitor { diff --git a/ydb/core/mon_alloc/profiler.cpp b/ydb/core/mon_alloc/profiler.cpp index 68b3e36ef4a..bfff45c1804 100644 --- a/ydb/core/mon_alloc/profiler.cpp +++ b/ydb/core/mon_alloc/profiler.cpp @@ -8,7 +8,7 @@ #include <library/cpp/actors/prof/tag.h> #include <library/cpp/html/pcdata/pcdata.h> #include <library/cpp/malloc/api/malloc.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #if defined(PROFILE_MEMORY_ALLOCATIONS) #include <library/cpp/lfalloc/alloc_profiler/profiler.h> diff --git a/ydb/core/mon_alloc/profiler.h b/ydb/core/mon_alloc/profiler.h index f7c7483a588..4fba8941da9 100644 --- a/ydb/core/mon_alloc/profiler.h +++ b/ydb/core/mon_alloc/profiler.h @@ -7,7 +7,7 @@ #include <library/cpp/actors/core/event_pb.h> #include <library/cpp/actors/core/events.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <util/generic/string.h> @@ -110,9 +110,9 @@ namespace NActors { virtual void Stop(IOutputStream& out, size_t limit, bool forLog) = 0; }; - inline TActorId MakeProfilerID(ui32 nodeId) { + inline TActorId MakeProfilerID(ui32 nodeId) { char x[12] = {'p', 'r', 'o', 'f', 'i', 'l', 'e', 'r', 's', 'e', 'r', 'v'}; - return TActorId(nodeId, TStringBuf(x, 12)); + return TActorId(nodeId, TStringBuf(x, 12)); } IActor* CreateProfilerActor( diff --git a/ydb/core/mon_alloc/stats.cpp b/ydb/core/mon_alloc/stats.cpp index 9f43566c619..050822241c6 100644 --- a/ydb/core/mon_alloc/stats.cpp +++ b/ydb/core/mon_alloc/stats.cpp @@ -8,7 +8,7 @@ #include <library/cpp/actors/core/process_stats.h> #include <library/cpp/lfalloc/dbg_info/dbg_info.h> #include <library/cpp/malloc/api/malloc.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <library/cpp/ytalloc/api/ytalloc.h> #include <util/datetime/base.h> diff --git a/ydb/core/mon_alloc/stats.h b/ydb/core/mon_alloc/stats.h index 5717af84dae..a5528e5afd3 100644 --- a/ydb/core/mon_alloc/stats.h +++ b/ydb/core/mon_alloc/stats.h @@ -3,7 +3,7 @@ #include <library/cpp/actors/core/defs.h> #include <library/cpp/actors/core/actor.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> namespace NKikimr { struct IAllocStats { @@ -12,8 +12,8 @@ namespace NKikimr { }; NActors::IActor* CreateMemStatsCollector( - ui32 intervalSec, - TIntrusivePtr<NMonitoring::TDynamicCounters> counters); + ui32 intervalSec, + TIntrusivePtr<NMonitoring::TDynamicCounters> counters); struct IAllocState { diff --git a/ydb/core/mon_alloc/ya.make b/ydb/core/mon_alloc/ya.make index 58c0d092589..972ea082f34 100644 --- a/ydb/core/mon_alloc/ya.make +++ b/ydb/core/mon_alloc/ya.make @@ -21,7 +21,7 @@ PEERDIR( library/cpp/lfalloc/alloc_profiler library/cpp/lfalloc/dbg_info library/cpp/malloc/api - library/cpp/monlib/service/pages + library/cpp/monlib/service/pages library/cpp/ytalloc/api ydb/core/base ydb/core/control diff --git a/ydb/core/node_whiteboard/node_whiteboard.h b/ydb/core/node_whiteboard/node_whiteboard.h index 2773d040015..cdf7601807c 100644 --- a/ydb/core/node_whiteboard/node_whiteboard.h +++ b/ydb/core/node_whiteboard/node_whiteboard.h @@ -400,9 +400,9 @@ struct TEvWhiteboard{ struct TEvSignalBodyResponse : TEventPB<TEvSignalBodyResponse, NKikimrWhiteboard::TEvSignalBodyResponse, EvSignalBodyResponse> {}; }; -inline TActorId MakeNodeWhiteboardServiceId(ui32 node) { +inline TActorId MakeNodeWhiteboardServiceId(ui32 node) { char x[12] = {'n','o','d','e','w','h','i','t','e','b','o','a'}; - return TActorId(node, TStringBuf(x, 12)); + return TActorId(node, TStringBuf(x, 12)); } IActor* CreateNodeWhiteboardService(); diff --git a/ydb/core/persqueue/cache_eviction.h b/ydb/core/persqueue/cache_eviction.h index 85bbc3d97c4..07ae210f852 100644 --- a/ydb/core/persqueue/cache_eviction.h +++ b/ydb/core/persqueue/cache_eviction.h @@ -52,13 +52,13 @@ namespace NPQ { }; ERequestType Type; - TActorId Sender; + TActorId Sender; ui64 CookiePQ; ui32 Partition; ui32 MetadataWritesCount; TVector<TRequestedBlob> Blobs; - TKvRequest(ERequestType type, TActorId sender, ui64 cookie, ui32 partition) + TKvRequest(ERequestType type, TActorId sender, ui64 cookie, ui32 partition) : Type(type) , Sender(sender) , CookiePQ(cookie) diff --git a/ydb/core/persqueue/cluster_tracker.cpp b/ydb/core/persqueue/cluster_tracker.cpp index e4296b58096..59fdea5c2b4 100644 --- a/ydb/core/persqueue/cluster_tracker.cpp +++ b/ydb/core/persqueue/cluster_tracker.cpp @@ -44,7 +44,7 @@ public: } private: - void AddSubscriber(const TActorId subscriberId) { + void AddSubscriber(const TActorId subscriberId) { Subscribers.insert(subscriberId); } @@ -89,7 +89,7 @@ private: } } - void SendClustersList(TActorId subscriberId) { + void SendClustersList(TActorId subscriberId) { auto ev = MakeHolder<TEvClusterTracker::TEvClustersUpdate>(); ev->ClustersList = ClustersList; @@ -188,7 +188,7 @@ private: TString PreparedQueryId; TClustersList::TConstPtr ClustersList = nullptr; TMaybe<TInstant> ClustersListUpdateTimestamp; - THashSet<TActorId> Subscribers; + THashSet<TActorId> Subscribers; TString Database; }; diff --git a/ydb/core/persqueue/cluster_tracker.h b/ydb/core/persqueue/cluster_tracker.h index bd88f142441..2ae06cf915a 100644 --- a/ydb/core/persqueue/cluster_tracker.h +++ b/ydb/core/persqueue/cluster_tracker.h @@ -7,7 +7,7 @@ #include <library/cpp/actors/core/defs.h> #include <library/cpp/actors/core/event_local.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <util/generic/maybe.h> #include <util/generic/ptr.h> @@ -17,11 +17,11 @@ namespace NKikimr::NPQ::NClusterTracker { -using NActors::TActorId; +using NActors::TActorId; -inline TActorId MakeClusterTrackerID() { +inline TActorId MakeClusterTrackerID() { static const char x[12] = "clstr_trckr"; - return TActorId(0, TStringBuf(x, 12)); + return TActorId(0, TStringBuf(x, 12)); } NActors::IActor* CreateClusterTracker(); diff --git a/ydb/core/persqueue/events/internal.h b/ydb/core/persqueue/events/internal.h index 8cde3a093d8..bc892988b47 100644 --- a/ydb/core/persqueue/events/internal.h +++ b/ydb/core/persqueue/events/internal.h @@ -194,12 +194,12 @@ struct TEvPQ { }; struct TEvMonRequest : public TEventLocal<TEvMonRequest, EvMonRequest> { - TEvMonRequest(const TActorId& sender, const TString& query) + TEvMonRequest(const TActorId& sender, const TString& query) : Sender(sender) , Query(query) {} - TActorId Sender; + TActorId Sender; TString Query; }; @@ -281,12 +281,12 @@ struct TEvPQ { struct TEvPartitionOffsets : public TEventLocal<TEvPartitionOffsets, EvPartitionOffsets> { - TEvPartitionOffsets(const TActorId& sender, const TString& clientId) + TEvPartitionOffsets(const TActorId& sender, const TString& clientId) : Sender(sender) , ClientId(clientId) {} - TActorId Sender; + TActorId Sender; TString ClientId; }; @@ -299,12 +299,12 @@ struct TEvPQ { }; struct TEvPartitionStatus : public TEventLocal<TEvPartitionStatus, EvPartitionStatus> { - explicit TEvPartitionStatus(const TActorId& sender, const TString& clientId) + explicit TEvPartitionStatus(const TActorId& sender, const TString& clientId) : Sender(sender) , ClientId(clientId) {} - TActorId Sender; + TActorId Sender; TString ClientId; }; @@ -395,7 +395,7 @@ struct TEvPQ { }; struct TEvChangeOwner : public TEventLocal<TEvChangeOwner, EvChangeOwner> { - explicit TEvChangeOwner(const ui64 cookie, const TString& owner, const TActorId& pipeClient, const TActorId& sender, const bool force) + explicit TEvChangeOwner(const ui64 cookie, const TString& owner, const TActorId& pipeClient, const TActorId& sender, const bool force) : Cookie(cookie) , Owner(owner) , PipeClient(pipeClient) @@ -405,19 +405,19 @@ struct TEvPQ { ui64 Cookie; TString Owner; - TActorId PipeClient; - TActorId Sender; + TActorId PipeClient; + TActorId Sender; bool Force; }; struct TEvPipeDisconnected : public TEventLocal<TEvPipeDisconnected, EvPipeDisconnected> { - explicit TEvPipeDisconnected(const TString& owner, const TActorId& pipeClient) + explicit TEvPipeDisconnected(const TString& owner, const TActorId& pipeClient) : Owner(owner) , PipeClient(pipeClient) {} TString Owner; - TActorId PipeClient; + TActorId PipeClient; }; struct TEvReserveBytes : public TEventLocal<TEvReserveBytes, EvReserveBytes> { @@ -504,11 +504,11 @@ struct TEvPQ { }; struct TEvGetPartitionClientInfo : TEventLocal<TEvGetPartitionClientInfo, EvGetPartitionClientInfo> { - TEvGetPartitionClientInfo(const TActorId& sender) + TEvGetPartitionClientInfo(const TActorId& sender) : Sender(sender) {} - TActorId Sender; + TActorId Sender; }; struct TEvUpdateAvailableSize : TEventLocal<TEvUpdateAvailableSize, EvUpdateAvailableSize> { diff --git a/ydb/core/persqueue/mirrorer.cpp b/ydb/core/persqueue/mirrorer.cpp index 9a10c44acd9..37aea95ad14 100644 --- a/ydb/core/persqueue/mirrorer.cpp +++ b/ydb/core/persqueue/mirrorer.cpp @@ -25,7 +25,7 @@ constexpr NKikimrServices::TActivity::EType TMirrorer::ActorActivityType() { TMirrorer::TMirrorer( TActorId tabletActor, - TActorId partitionActor, + TActorId partitionActor, const TString& topicName, ui32 partition, bool localDC, diff --git a/ydb/core/persqueue/mirrorer.h b/ydb/core/persqueue/mirrorer.h index 70555774ff6..6819891f6c7 100644 --- a/ydb/core/persqueue/mirrorer.h +++ b/ydb/core/persqueue/mirrorer.h @@ -98,7 +98,7 @@ private: private: template<class TEvent> - void ScheduleWithIncreasingTimeout(const TActorId& recipient, TDuration& timeout, const TDuration& maxTimeout, const TActorContext &ctx) { + void ScheduleWithIncreasingTimeout(const TActorId& recipient, TDuration& timeout, const TDuration& maxTimeout, const TActorContext &ctx) { ctx.ExecutorThread.ActorSystem->Schedule(timeout, new IEventHandle(recipient, SelfId(), new TEvent())); timeout = Min(timeout * 2, maxTimeout); } @@ -158,7 +158,7 @@ public: private: TActorId TabletActor; - TActorId PartitionActor; + TActorId PartitionActor; TString TopicName; ui32 Partition; bool LocalDC; diff --git a/ydb/core/persqueue/partition.cpp b/ydb/core/persqueue/partition.cpp index 615a01fea92..cc8e2419cf2 100644 --- a/ydb/core/persqueue/partition.cpp +++ b/ydb/core/persqueue/partition.cpp @@ -47,7 +47,7 @@ static const TString WRITE_QUOTA_ROOT_PATH = "write-quota"; struct TPartition::THasDataReq { ui64 Num; ui64 Offset; - TActorId Sender; + TActorId Sender; TMaybe<ui64> Cookie; TString ClientId; @@ -322,7 +322,7 @@ void TPartition::ReplyGetClientOffsetOk(const TActorContext& ctx, const ui64 dst } -static void RequestRange(const TActorContext& ctx, const TActorId& dst, ui32 partition, +static void RequestRange(const TActorContext& ctx, const TActorId& dst, ui32 partition, TKeyPrefix::EType c, bool includeData = false, const TString& key = "", bool dropTmp = false) { THolder<TEvKeyValue::TEvRequest> request(new TEvKeyValue::TEvRequest); @@ -369,7 +369,7 @@ void AddCheckDiskRequest(TEvKeyValue::TEvRequest *request, ui32 numChannels) { } -static void RequestDiskStatus(const TActorContext& ctx, const TActorId& dst, ui32 numChannels) +static void RequestDiskStatus(const TActorContext& ctx, const TActorId& dst, ui32 numChannels) { THolder<TEvKeyValue::TEvRequest> request(new TEvKeyValue::TEvRequest); @@ -379,7 +379,7 @@ static void RequestDiskStatus(const TActorContext& ctx, const TActorId& dst, ui3 } -void RequestInfoRange(const TActorContext& ctx, const TActorId& dst, ui32 partition, const TString& key) +void RequestInfoRange(const TActorContext& ctx, const TActorId& dst, ui32 partition, const TString& key) { RequestRange(ctx, dst, partition, TKeyPrefix::TypeInfo, true, key, key == ""); } @@ -393,7 +393,7 @@ void RequestMetaRead(const TActorContext& ctx, const TActorId& dst, ui32 partiti ctx.Send(dst, request.Release()); } -void RequestData(const TActorContext& ctx, const TActorId& dst, const TVector<TString>& keys) +void RequestData(const TActorContext& ctx, const TActorId& dst, const TVector<TString>& keys) { THolder<TEvKeyValue::TEvRequest> request(new TEvKeyValue::TEvRequest); for (auto& key: keys) { @@ -403,7 +403,7 @@ void RequestData(const TActorContext& ctx, const TActorId& dst, const TVector<TS ctx.Send(dst, request.Release()); } -void RequestDataRange(const TActorContext& ctx, const TActorId& dst, ui32 partition, const TString& key) +void RequestDataRange(const TActorContext& ctx, const TActorId& dst, ui32 partition, const TString& key) { RequestRange(ctx, dst, partition, TKeyPrefix::TypeData, false, key); } @@ -452,7 +452,7 @@ void TPartition::FillReadFromTimestamps(const NKikimrPQ::TPQTabletConfig& config } } -TPartition::TPartition(ui64 tabletId, ui32 partition, const TActorId& tablet, const TActorId& blobCache, +TPartition::TPartition(ui64 tabletId, ui32 partition, const TActorId& tablet, const TActorId& blobCache, const TString& topicName, const TString& topicPath, const bool localDC, TString dcId, const NKikimrPQ::TPQTabletConfig& config, const TTabletCountersBase& counters, const TActorContext &ctx, bool newPartition) @@ -1179,7 +1179,7 @@ void TPartition::Handle(TEvPersQueue::TEvHasDataInfo::TPtr& ev, const TActorCont auto& record = ev->Get()->Record; Y_VERIFY(record.HasSender()); - TActorId sender = ActorIdFromProto(record.GetSender()); + TActorId sender = ActorIdFromProto(record.GetSender()); if (InitDone && EndOffset > (ui64)record.GetOffset()) { //already has data, answer right now TAutoPtr<TEvPersQueue::TEvHasDataInfoResponse> res(new TEvPersQueue::TEvHasDataInfoResponse()); res->Record.SetEndOffset(EndOffset); @@ -1833,7 +1833,7 @@ void TPartition::Handle(TEvPQ::TEvChangeOwner::TPtr& ev, const TActorContext& ct void TPartition::Handle(TEvPQ::TEvPipeDisconnected::TPtr& ev, const TActorContext& ctx) { const TString& owner = ev->Get()->Owner; - const TActorId& pipeClient = ev->Get()->PipeClient; + const TActorId& pipeClient = ev->Get()->PipeClient; OwnerPipes.erase(pipeClient); diff --git a/ydb/core/persqueue/partition.h b/ydb/core/persqueue/partition.h index 1db3e456d7c..2eec26f2516 100644 --- a/ydb/core/persqueue/partition.h +++ b/ydb/core/persqueue/partition.h @@ -218,7 +218,7 @@ public: return NKikimrServices::TActivity::PERSQUEUE_PARTITION_ACTOR; } - TPartition(ui64 tabletId, ui32 partition, const TActorId& tablet, const TActorId& blobCache, + TPartition(ui64 tabletId, ui32 partition, const TActorId& tablet, const TActorId& blobCache, const TString& topicName, const TString& topicPath, const bool localDC, TString dcId, const NKikimrPQ::TPQTabletConfig& config, const TTabletCountersBase& counters, const TActorContext& ctx, bool newPartition = false); @@ -435,8 +435,8 @@ private: ui64 EndOffset; ui64 WriteInflightSize; - TActorId Tablet; - TActorId BlobCache; + TActorId Tablet; + TActorId BlobCache; EInitState InitState; @@ -587,7 +587,7 @@ private: const bool NewPartition; THashMap<TString, NKikimr::NPQ::TOwnerInfo> Owners; - THashSet<TActorId> OwnerPipes; + THashSet<TActorId> OwnerPipes; TSourceIdStorage SourceIdStorage; diff --git a/ydb/core/persqueue/percentile_counter.h b/ydb/core/persqueue/percentile_counter.h index 40634b1df78..74ee43d2872 100644 --- a/ydb/core/persqueue/percentile_counter.h +++ b/ydb/core/persqueue/percentile_counter.h @@ -1,6 +1,6 @@ #pragma once -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> namespace NKikimr { namespace NPQ { diff --git a/ydb/core/persqueue/pq.cpp b/ydb/core/persqueue/pq.cpp index d124d221a05..d0ab706a344 100644 --- a/ydb/core/persqueue/pq.cpp +++ b/ydb/core/persqueue/pq.cpp @@ -3,11 +3,11 @@ namespace NKikimr { -IActor* CreatePersQueue(const TActorId& tablet, TTabletStorageInfo *info) { +IActor* CreatePersQueue(const TActorId& tablet, TTabletStorageInfo *info) { return new NPQ::TPersQueue(tablet, info); } -IActor* CreatePersQueueReadBalancer(const TActorId& tablet, TTabletStorageInfo *info) { +IActor* CreatePersQueueReadBalancer(const TActorId& tablet, TTabletStorageInfo *info) { return new NPQ::TPersQueueReadBalancer(tablet, info); } diff --git a/ydb/core/persqueue/pq.h b/ydb/core/persqueue/pq.h index 83f0c391b26..e5f3309ba9f 100644 --- a/ydb/core/persqueue/pq.h +++ b/ydb/core/persqueue/pq.h @@ -6,7 +6,7 @@ namespace NKikimr { -IActor* CreatePersQueue(const TActorId& tablet, TTabletStorageInfo *info); -IActor* CreatePersQueueReadBalancer(const TActorId& tablet, TTabletStorageInfo *info); +IActor* CreatePersQueue(const TActorId& tablet, TTabletStorageInfo *info); +IActor* CreatePersQueueReadBalancer(const TActorId& tablet, TTabletStorageInfo *info); } //NKikimr diff --git a/ydb/core/persqueue/pq_impl.cpp b/ydb/core/persqueue/pq_impl.cpp index eef373abd4f..9937dbe561c 100644 --- a/ydb/core/persqueue/pq_impl.cpp +++ b/ydb/core/persqueue/pq_impl.cpp @@ -16,7 +16,7 @@ //TODO: move this code to vieiwer #include <ydb/core/tablet/tablet_counters_aggregator.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/string/escape.h> namespace NKikimr { @@ -46,7 +46,7 @@ struct TPartitionInfo { Baseline.Populate(info.Baseline); } - TActorId Actor; + TActorId Actor; TMaybe<TPartitionKeyRange> KeyRange; bool InitDone; TTabletCountersBase Baseline; @@ -54,13 +54,13 @@ struct TPartitionInfo { }; struct TChangeNotification { - TChangeNotification(const TActorId& actor, const ui64 txId) + TChangeNotification(const TActorId& actor, const ui64 txId) : Actor(actor) , TxId(txId) {} operator size_t() const { - return THash<TActorId>()(Actor) + TxId; + return THash<TActorId>()(Actor) + TxId; } bool operator==(const TChangeNotification& b) { @@ -71,7 +71,7 @@ struct TChangeNotification { return (Actor < req.Actor) || (Actor == req.Actor && TxId < req.TxId); } - TActorId Actor; + TActorId Actor; ui64 TxId; }; @@ -90,7 +90,7 @@ public: return NKikimrServices::TActivity::PERSQUEUE_ANS_ACTOR; } - TReadProxy(const TActorId& sender, const TActorId& tablet, const NKikimrClient::TPersQueueRequest& request) + TReadProxy(const TActorId& sender, const TActorId& tablet, const NKikimrClient::TPersQueueRequest& request) : Sender(sender) , Tablet(tablet) , Request(request) @@ -197,14 +197,14 @@ private: }; } - const TActorId Sender; - const TActorId Tablet; + const TActorId Sender; + const TActorId Tablet; NKikimrClient::TPersQueueRequest Request; THolder<TEvPersQueue::TEvResponse> Response; }; -TActorId CreateReadProxy(const TActorId& sender, const TActorId& tablet, const NKikimrClient::TPersQueueRequest& request, const TActorContext&ctx) +TActorId CreateReadProxy(const TActorId& sender, const TActorId& tablet, const NKikimrClient::TPersQueueRequest& request, const TActorContext&ctx) { return ctx.Register(new TReadProxy(sender, tablet, request)); } @@ -213,7 +213,7 @@ TActorId CreateReadProxy(const TActorId& sender, const TActorId& tablet, const N class TResponseBuilder { public: - TResponseBuilder(const TActorId& sender, const TActorId& tablet, const TString& topicName, const ui32 partition, const ui64 messageNo, + TResponseBuilder(const TActorId& sender, const TActorId& tablet, const TString& topicName, const ui32 partition, const ui64 messageNo, const TString& reqId, const TMaybe<ui64> cookie, NMetrics::TResourceMetrics* resourceMetrics, const TActorContext& ctx) : Sender(sender) , Tablet(tablet) @@ -291,8 +291,8 @@ public: return true; } - const TActorId Sender; - const TActorId Tablet; + const TActorId Sender; + const TActorId Tablet; const TString TopicName; const ui32 Partition; const ui64 MessageNo; @@ -307,7 +307,7 @@ public: }; -TAutoPtr<TResponseBuilder> CreateResponseProxy(const TActorId& sender, const TActorId& tablet, const TString& topicName, +TAutoPtr<TResponseBuilder> CreateResponseProxy(const TActorId& sender, const TActorId& tablet, const TString& topicName, const ui32 partition, const ui64 messageNo, const TString& reqId, const TMaybe<ui64> cookie, NMetrics::TResourceMetrics *resourceMetrics, const TActorContext& ctx) { @@ -329,7 +329,7 @@ public: return NKikimrServices::TActivity::PERSQUEUE_ANS_ACTOR; } - TBuilderProxy(const ui64 tabletId, const TActorId& sender, const ui32 count) + TBuilderProxy(const ui64 tabletId, const TActorId& sender, const ui32 count) : TabletId(tabletId) , Sender(sender) , Waiting(count) @@ -385,13 +385,13 @@ private: } ui64 TabletId; - TActorId Sender; + TActorId Sender; ui32 Waiting; TVector<typename T2::TPartResult> Result; }; -TActorId CreateOffsetsProxyActor(const ui64 tabletId, const TActorId& sender, const ui32 count, const TActorContext& ctx) +TActorId CreateOffsetsProxyActor(const ui64 tabletId, const TActorId& sender, const ui32 count, const TActorContext& ctx) { return ctx.Register(new TBuilderProxy<TEvPQ::TEvPartitionOffsetsResponse, NKikimrPQ::TOffsetsResponse, @@ -401,7 +401,7 @@ TActorId CreateOffsetsProxyActor(const ui64 tabletId, const TActorId& sender, co /******************************************************* StatusProxy *********************************************************/ -TActorId CreateStatusProxyActor(const ui64 tabletId, const TActorId& sender, const ui32 count, const TActorContext& ctx) +TActorId CreateStatusProxyActor(const ui64 tabletId, const TActorId& sender, const ui32 count, const TActorContext& ctx) { return ctx.Register(new TBuilderProxy<TEvPQ::TEvPartitionStatusResponse, NKikimrPQ::TStatusResponse, @@ -417,7 +417,7 @@ public: return NKikimrServices::TActivity::PERSQUEUE_MON_ACTOR; } - TMonitoringProxy(const TActorId& sender, const TString& query, const TMap<ui32, TActorId>& partitions, const TActorId& cache, + TMonitoringProxy(const TActorId& sender, const TString& query, const TMap<ui32, TActorId>& partitions, const TActorId& cache, const TString& topicName, ui64 tabletId, ui32 inflight) : Sender(sender) , Query(query) @@ -520,12 +520,12 @@ private: }; } - TActorId Sender; + TActorId Sender; TString Query; TMap<ui32, TVector<TString>> Results; TVector<TString> Str; - TMap<ui32, TActorId> Partitions; - TActorId Cache; + TMap<ui32, TActorId> Partitions; + TActorId Cache; ui32 TotalRequests; ui32 TotalResponses; TString TopicName; @@ -896,7 +896,7 @@ void TPersQueue::AggregateAndSendLabeledCountersFor(const TString& group, const Y_VERIFY(aggr->HasCounters()); - TActorId countersAggregator = MakeTabletCountersAggregatorID(ctx.SelfID.NodeId()); + TActorId countersAggregator = MakeTabletCountersAggregatorID(ctx.SelfID.NodeId()); ctx.Send(countersAggregator, new TEvTabletCounters::TEvTabletAddLabeledCounters( CounterEventsInflight[group], TabletID(), TTabletTypes::PERSQUEUE, aggr)); } @@ -999,7 +999,7 @@ void TPersQueue::Handle(TEvPersQueue::TEvUpdateConfig::TPtr& ev, const TActorCon } -void TPersQueue::ProcessUpdateConfigRequest(TAutoPtr<TEvPersQueue::TEvUpdateConfig> ev, const TActorId& sender, const TActorContext& ctx) +void TPersQueue::ProcessUpdateConfigRequest(TAutoPtr<TEvPersQueue::TEvUpdateConfig> ev, const TActorId& sender, const TActorContext& ctx) { auto& record = ev->Record; @@ -1219,7 +1219,7 @@ void TPersQueue::Handle(TEvPersQueue::TEvOffsets::TPtr& ev, const TActorContext& for (auto& p : Partitions) { cnt += p.second.InitDone; } - TActorId ans = CreateOffsetsProxyActor(TabletID(), ev->Sender, cnt, ctx); + TActorId ans = CreateOffsetsProxyActor(TabletID(), ev->Sender, cnt, ctx); for (auto& p : Partitions) { if (!p.second.InitDone) @@ -1233,7 +1233,7 @@ void TPersQueue::Handle(TEvPersQueue::TEvOffsets::TPtr& ev, const TActorContext& void TPersQueue::Handle(TEvPersQueue::TEvHasDataInfo::TPtr& ev, const TActorContext& ctx) { auto& record = ev->Get()->Record; - ActorIdToProto(ev->Sender, record.MutableSender()); + ActorIdToProto(ev->Sender, record.MutableSender()); if (!ConfigInited) { HasDataRequests.push_back(ev->Release()); } else { @@ -1275,7 +1275,7 @@ void TPersQueue::Handle(TEvPersQueue::TEvStatus::TPtr& ev, const TActorContext& cnt += p.second.InitDone; } - TActorId ans = CreateStatusProxyActor(TabletID(), ev->Sender, cnt, ctx); + TActorId ans = CreateStatusProxyActor(TabletID(), ev->Sender, cnt, ctx); for (auto& p : Partitions) { if (!p.second.InitDone) continue; @@ -1293,7 +1293,7 @@ void TPersQueue::InitResponseBuilder(const ui64 responseCookie, const ui32 count it->second->SetCounterId(counterId); } -void TPersQueue::HandleGetMaxSeqNoRequest(const ui64 responseCookie, const TActorId& partActor, +void TPersQueue::HandleGetMaxSeqNoRequest(const ui64 responseCookie, const TActorId& partActor, const NKikimrClient::TPersQueuePartitionRequest& req, const TActorContext& ctx) { Y_VERIFY(req.HasCmdGetMaxSeqNo()); @@ -1307,7 +1307,7 @@ void TPersQueue::HandleGetMaxSeqNoRequest(const ui64 responseCookie, const TActo ctx.Send(partActor, event.Release()); } -void TPersQueue::HandleDeleteSessionRequest(const ui64 responseCookie, const TActorId& partActor, +void TPersQueue::HandleDeleteSessionRequest(const ui64 responseCookie, const TActorId& partActor, const NKikimrClient::TPersQueuePartitionRequest& req, const TActorContext& ctx) { Y_VERIFY(req.HasCmdDeleteSession()); @@ -1327,7 +1327,7 @@ void TPersQueue::HandleDeleteSessionRequest(const ui64 responseCookie, const TAc } } -void TPersQueue::HandleCreateSessionRequest(const ui64 responseCookie, const TActorId& partActor, +void TPersQueue::HandleCreateSessionRequest(const ui64 responseCookie, const TActorId& partActor, const NKikimrClient::TPersQueuePartitionRequest& req, const TActorContext& ctx) { Y_VERIFY(req.HasCmdCreateSession()); @@ -1353,7 +1353,7 @@ void TPersQueue::HandleCreateSessionRequest(const ui64 responseCookie, const TAc } } -void TPersQueue::HandleSetClientOffsetRequest(const ui64 responseCookie, const TActorId& partActor, +void TPersQueue::HandleSetClientOffsetRequest(const ui64 responseCookie, const TActorId& partActor, const NKikimrClient::TPersQueuePartitionRequest& req, const TActorContext& ctx) { Y_VERIFY(req.HasCmdSetClientOffset()); @@ -1377,7 +1377,7 @@ void TPersQueue::HandleSetClientOffsetRequest(const ui64 responseCookie, const T } } -void TPersQueue::HandleGetClientOffsetRequest(const ui64 responseCookie, const TActorId& partActor, +void TPersQueue::HandleGetClientOffsetRequest(const ui64 responseCookie, const TActorId& partActor, const NKikimrClient::TPersQueuePartitionRequest& req, const TActorContext& ctx) { Y_VERIFY(req.HasCmdGetClientOffset()); @@ -1392,7 +1392,7 @@ void TPersQueue::HandleGetClientOffsetRequest(const ui64 responseCookie, const T } } -void TPersQueue::HandleUpdateWriteTimestampRequest(const ui64 responseCookie, const TActorId& partActor, +void TPersQueue::HandleUpdateWriteTimestampRequest(const ui64 responseCookie, const TActorId& partActor, const NKikimrClient::TPersQueuePartitionRequest& req, const TActorContext& ctx) { Y_VERIFY(req.HasCmdUpdateWriteTimestamp()); @@ -1402,7 +1402,7 @@ void TPersQueue::HandleUpdateWriteTimestampRequest(const ui64 responseCookie, co ctx.Send(partActor, event.Release()); } -void TPersQueue::HandleWriteRequest(const ui64 responseCookie, const TActorId& partActor, +void TPersQueue::HandleWriteRequest(const ui64 responseCookie, const TActorId& partActor, const NKikimrClient::TPersQueuePartitionRequest& req, const TActorContext& ctx) { Y_VERIFY(req.CmdWriteSize()); @@ -1563,9 +1563,9 @@ void TPersQueue::HandleWriteRequest(const ui64 responseCookie, const TActorId& p } -void TPersQueue::HandleReserveBytesRequest(const ui64 responseCookie, const TActorId& partActor, +void TPersQueue::HandleReserveBytesRequest(const ui64 responseCookie, const TActorId& partActor, const NKikimrClient::TPersQueuePartitionRequest& req, const TActorContext& ctx, - const TActorId& pipeClient, const TActorId&) + const TActorId& pipeClient, const TActorId&) { Y_VERIFY(req.HasCmdReserveBytes()); @@ -1592,9 +1592,9 @@ void TPersQueue::HandleReserveBytesRequest(const ui64 responseCookie, const TAct } -void TPersQueue::HandleGetOwnershipRequest(const ui64 responseCookie, const TActorId& partActor, +void TPersQueue::HandleGetOwnershipRequest(const ui64 responseCookie, const TActorId& partActor, const NKikimrClient::TPersQueuePartitionRequest& req, const TActorContext& ctx, - const TActorId& pipeClient, const TActorId& sender) + const TActorId& pipeClient, const TActorId& sender) { Y_VERIFY(req.HasCmdGetOwnership()); @@ -1604,7 +1604,7 @@ void TPersQueue::HandleGetOwnershipRequest(const ui64 responseCookie, const TAct TStringBuilder() << "empty owner in CmdGetOwnership request"); return; } - Y_VERIFY(pipeClient != TActorId()); + Y_VERIFY(pipeClient != TActorId()); auto it = PipesInfo.find(pipeClient); if (it == PipesInfo.end()) { //do nothing. this could not be happen, just in tests ReplyError(ctx, responseCookie, NPersQueue::NErrorCode::BAD_REQUEST, @@ -1620,7 +1620,7 @@ void TPersQueue::HandleGetOwnershipRequest(const ui64 responseCookie, const TAct } -void TPersQueue::HandleReadRequest(const ui64 responseCookie, const TActorId& partActor, +void TPersQueue::HandleReadRequest(const ui64 responseCookie, const TActorId& partActor, const NKikimrClient::TPersQueuePartitionRequest& req, const TActorContext& ctx) { Y_VERIFY(req.HasCmdRead()); @@ -1798,7 +1798,7 @@ void TPersQueue::Handle(TEvPersQueue::TEvRequest::TPtr& ev, const TActorContext& c = request.GetPartitionRequest().GetCookie(); TAutoPtr<TResponseBuilder> ans; if (request.HasPartitionRequest() && request.GetPartitionRequest().HasCmdRead() && s != TMP_REQUEST_MARKER) { - TActorId rr = CreateReadProxy(ev->Sender, ctx.SelfID, request, ctx); + TActorId rr = CreateReadProxy(ev->Sender, ctx.SelfID, request, ctx); ans = CreateResponseProxy(rr, ctx.SelfID, TopicName, p, m, s, c, ResourceMetrics, ctx); } else { ans = CreateResponseProxy(ev->Sender, ctx.SelfID, TopicName, p, m, s, c, ResourceMetrics, ctx); @@ -1865,9 +1865,9 @@ void TPersQueue::Handle(TEvPersQueue::TEvRequest::TPtr& ev, const TActorContext& return; } - const TActorId& partActor = it->second.Actor; + const TActorId& partActor = it->second.Actor; - TActorId pipeClient = ActorIdFromProto(req.GetPipeClient()); + TActorId pipeClient = ActorIdFromProto(req.GetPipeClient()); if (req.HasCmdGetMaxSeqNo()) { HandleGetMaxSeqNoRequest(responseCookie, partActor, req, ctx); @@ -1904,7 +1904,7 @@ void TPersQueue::Handle(TEvTabletPipe::TEvServerConnected::TPtr& ev, const TActo auto it = PipesInfo.find(ev->Get()->ClientId); if (it == PipesInfo.end()) { - PipesInfo.insert({ev->Get()->ClientId, {TActorId(), "", 1}}); + PipesInfo.insert({ev->Get()->ClientId, {TActorId(), "", 1}}); } else { it->second.ServerActors++; } @@ -1921,7 +1921,7 @@ void TPersQueue::Handle(TEvTabletPipe::TEvServerDisconnected::TPtr& ev, const TA if(--(it->second.ServerActors) > 0) { return; } - if (it->second.PartActor != TActorId()) { + if (it->second.PartActor != TActorId()) { ctx.Send(it->second.PartActor, new TEvPQ::TEvPipeDisconnected(it->second.Owner, it->first)); } PipesInfo.erase(it); @@ -1938,7 +1938,7 @@ bool TPersQueue::OnRenderAppHtmlPage(NMon::TEvRemoteHttpInfo::TPtr ev, const TAc return TKeyValueFlat::OnRenderAppHtmlPage(ev, ctx); } LOG_INFO_S(ctx, NKikimrServices::PERSQUEUE, "Handle TEvRemoteHttpInfo: " << ev->Get()->Query); - TMap<ui32, TActorId> res; + TMap<ui32, TActorId> res; for (auto& p : Partitions) { res.insert({p.first, p.second.Actor}); } @@ -1967,7 +1967,7 @@ void TPersQueue::HandleDie(const TActorContext& ctx) } -TPersQueue::TPersQueue(const TActorId& tablet, TTabletStorageInfo *info) +TPersQueue::TPersQueue(const TActorId& tablet, TTabletStorageInfo *info) : TKeyValueFlat(tablet, info) , ConfigInited(false) , PartitionsInited(0) @@ -1996,7 +1996,7 @@ TPersQueue::TPersQueue(const TActorId& tablet, TTabletStorageInfo *info) void TPersQueue::CreatedHook(const TActorContext& ctx) { - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); ctx.Send(nameserviceId, new TEvInterconnect::TEvGetNode(ctx.SelfID.NodeId())); auto& pqConfig = AppData(ctx)->PQConfig; diff --git a/ydb/core/persqueue/pq_impl.h b/ydb/core/persqueue/pq_impl.h index 2d899ea9b63..bc3bfb10ba5 100644 --- a/ydb/core/persqueue/pq_impl.h +++ b/ydb/core/persqueue/pq_impl.h @@ -28,7 +28,7 @@ class TPersQueue : public NKeyValue::TKeyValueFlat { void CreatedHook(const TActorContext& ctx) override; bool HandleHook(STFUNC_SIG) override; -// void ReplyError(const TActorContext& ctx, const TActorId& dst, NPersQueue::NErrorCode::EErrorCode errorCode, const TString& error); +// void ReplyError(const TActorContext& ctx, const TActorId& dst, NPersQueue::NErrorCode::EErrorCode errorCode, const TString& error); void ReplyError(const TActorContext& ctx, const ui64 responseCookie, NPersQueue::NErrorCode::EErrorCode errorCode, const TString& error); void HandleWakeup(const TActorContext&); @@ -58,7 +58,7 @@ class TPersQueue : public NKeyValue::TKeyValueFlat { //client requests void Handle(TEvPersQueue::TEvUpdateConfig::TPtr& ev, const TActorContext& ctx); - void ProcessUpdateConfigRequest(TAutoPtr<TEvPersQueue::TEvUpdateConfig> ev, const TActorId& sender, const TActorContext& ctx); + void ProcessUpdateConfigRequest(TAutoPtr<TEvPersQueue::TEvUpdateConfig> ev, const TActorId& sender, const TActorContext& ctx); void Handle(TEvPersQueue::TEvOffsets::TPtr& ev, const TActorContext& ctx); void Handle(TEvPersQueue::TEvStatus::TPtr& ev, const TActorContext& ctx); void Handle(TEvPersQueue::TEvDropTablet::TPtr& ev, const TActorContext& ctx); @@ -98,7 +98,7 @@ class TPersQueue : public NKeyValue::TKeyValueFlat { //client request void Handle(TEvPersQueue::TEvRequest::TPtr& ev, const TActorContext& ctx); -#define DESCRIBE_HANDLE(A) void A(const ui64 responseCookie, const TActorId& partActor, \ +#define DESCRIBE_HANDLE(A) void A(const ui64 responseCookie, const TActorId& partActor, \ const NKikimrClient::TPersQueuePartitionRequest& req, const TActorContext& ctx); DESCRIBE_HANDLE(HandleGetMaxSeqNoRequest) DESCRIBE_HANDLE(HandleDeleteSessionRequest) @@ -112,9 +112,9 @@ class TPersQueue : public NKeyValue::TKeyValueFlat { DESCRIBE_HANDLE(HandleDeregisterMessageGroupRequest) DESCRIBE_HANDLE(HandleSplitMessageGroupRequest) #undef DESCRIBE_HANDLE -#define DESCRIBE_HANDLE_WITH_SENDER(A) void A(const ui64 responseCookie, const TActorId& partActor, \ +#define DESCRIBE_HANDLE_WITH_SENDER(A) void A(const ui64 responseCookie, const TActorId& partActor, \ const NKikimrClient::TPersQueuePartitionRequest& req, const TActorContext& ctx,\ - const TActorId& pipeClient, const TActorId& sender); + const TActorId& pipeClient, const TActorId& sender); DESCRIBE_HANDLE_WITH_SENDER(HandleGetOwnershipRequest) DESCRIBE_HANDLE_WITH_SENDER(HandleReserveBytesRequest) #undef DESCRIBE_HANDLE_WITH_SENDER @@ -130,7 +130,7 @@ public: return NKikimrServices::TActivity::PERSQUEUE_ACTOR; } - TPersQueue(const TActorId& tablet, TTabletStorageInfo *info); + TPersQueue(const TActorId& tablet, TTabletStorageInfo *info); private: bool ConfigInited; @@ -138,7 +138,7 @@ private: THashMap<ui32, TPartitionInfo> Partitions; THashMap<TString, TIntrusivePtr<TEvTabletCounters::TInFlightCookie>> CounterEventsInflight; - TActorId CacheActor; + TActorId CacheActor; TSet<TChangeNotification> ChangeConfigNotification; NKikimrPQ::TPQTabletConfig NewConfig; @@ -162,15 +162,15 @@ private: THashMap<TString, TTabletLabeledCountersBase> LabeledCounters; TVector<TAutoPtr<TEvPersQueue::TEvHasDataInfo>> HasDataRequests; - TVector<std::pair<TAutoPtr<TEvPersQueue::TEvUpdateConfig>, TActorId> > UpdateConfigRequests; + TVector<std::pair<TAutoPtr<TEvPersQueue::TEvUpdateConfig>, TActorId> > UpdateConfigRequests; struct TPipeInfo { - TActorId PartActor; + TActorId PartActor; TString Owner; ui32 ServerActors; }; - THashMap<TActorId, TPipeInfo> PipesInfo; + THashMap<TActorId, TPipeInfo> PipesInfo; ui64 NextResponseCookie; THashMap<ui64, TAutoPtr<TResponseBuilder>> ResponseProxy; diff --git a/ydb/core/persqueue/pq_l2_cache.cpp b/ydb/core/persqueue/pq_l2_cache.cpp index e570b8cd8f6..e56642c39ae 100644 --- a/ydb/core/persqueue/pq_l2_cache.cpp +++ b/ydb/core/persqueue/pq_l2_cache.cpp @@ -44,7 +44,7 @@ void TPersQueueCacheL2::Handle(TEvPqCache::TEvCacheL2Request::TPtr& ev, const TA void TPersQueueCacheL2::SendResponses(const TActorContext& ctx, const THashMap<TKey, TCacheValue::TPtr>& evictedBlobs) { TInstant now = TAppData::TimeProvider->Now(); - THashMap<TActorId, THolder<TCacheL2Response>> responses; + THashMap<TActorId, THolder<TCacheL2Response>> responses; for (auto rm : evictedBlobs) { const TKey& key = rm.first; diff --git a/ydb/core/persqueue/pq_l2_cache.h b/ydb/core/persqueue/pq_l2_cache.h index 6088c3f4191..f9fcccbc8ed 100644 --- a/ydb/core/persqueue/pq_l2_cache.h +++ b/ydb/core/persqueue/pq_l2_cache.h @@ -4,7 +4,7 @@ #include "pq_l2_service.h" #include <ydb/core/protos/pqconfig.pb.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <library/cpp/cache/cache.h> #include <util/generic/hash.h> diff --git a/ydb/core/persqueue/pq_l2_service.h b/ydb/core/persqueue/pq_l2_service.h index ab544ce7a2f..a9c5eefd164 100644 --- a/ydb/core/persqueue/pq_l2_service.h +++ b/ydb/core/persqueue/pq_l2_service.h @@ -8,10 +8,10 @@ namespace NKikimr { namespace NPQ { -inline TActorId MakePersQueueL2CacheID() { - static_assert(TActorId::MaxServiceIDLength == 12, "Unexpected actor id length"); +inline TActorId MakePersQueueL2CacheID() { + static_assert(TActorId::MaxServiceIDLength == 12, "Unexpected actor id length"); const char x[12] = "pq_l2_cache"; - return TActorId(0, TStringBuf(x, 12)); + return TActorId(0, TStringBuf(x, 12)); } struct TCacheL2Parameters { @@ -27,7 +27,7 @@ struct TCacheValue : TNonCopyable { using TPtr = std::shared_ptr<TCacheValue>; using TWeakPtr = std::weak_ptr<TCacheValue>; - TCacheValue(TString value, TActorId owner, TInstant accessTime) + TCacheValue(TString value, TActorId owner, TInstant accessTime) : Value(value) , Owner(owner) , AccessTime(accessTime.TimeT()) @@ -55,13 +55,13 @@ struct TCacheValue : TNonCopyable { return Value.size(); } - const TActorId& GetOwner() const { + const TActorId& GetOwner() const { return Owner; } private: const TString Value; - const TActorId Owner; + const TActorId Owner; std::atomic<ui64> AccessTime; std::atomic<ui32> AccessCount; }; diff --git a/ydb/core/persqueue/pq_ut.cpp b/ydb/core/persqueue/pq_ut.cpp index 5876dcdc1aa..13a61229105 100644 --- a/ydb/core/persqueue/pq_ut.cpp +++ b/ydb/core/persqueue/pq_ut.cpp @@ -42,36 +42,36 @@ Y_UNIT_TEST(TestGroupsBalancer) { BalancerPrepare("topic", {{0,{1, 1}}, {11,{1, 1}}, {1,{1, 2}}, {2,{1, 2}}}, ssId, tc); - TActorId pipe = RegisterReadSession("session1", tc); + TActorId pipe = RegisterReadSession("session1", tc); Y_UNUSED(pipe); - WaitPartition("session1", tc, 0, "", "", TActorId()); - WaitPartition("session1", tc, 0, "", "", TActorId()); - WaitPartition("session1", tc, 0, "", "", TActorId()); - WaitPartition("session1", tc, 0, "", "", TActorId()); - WaitPartition("", tc, 0, "", "", TActorId(), false);//no partitions - return error + WaitPartition("session1", tc, 0, "", "", TActorId()); + WaitPartition("session1", tc, 0, "", "", TActorId()); + WaitPartition("session1", tc, 0, "", "", TActorId()); + WaitPartition("session1", tc, 0, "", "", TActorId()); + WaitPartition("", tc, 0, "", "", TActorId(), false);//no partitions - return error - TActorId pipe2 = RegisterReadSession("session2", tc, {1}); + TActorId pipe2 = RegisterReadSession("session2", tc, {1}); - WaitPartition("session2", tc, 0, "", "", TActorId()); - WaitPartition("session2", tc, 0, "", "", TActorId()); - WaitPartition("", tc, 0, "", "", TActorId(), false);//no partitions to balance + WaitPartition("session2", tc, 0, "", "", TActorId()); + WaitPartition("session2", tc, 0, "", "", TActorId()); + WaitPartition("", tc, 0, "", "", TActorId(), false);//no partitions to balance - TActorId pipe4 = RegisterReadSession("session8", tc, {1}); + TActorId pipe4 = RegisterReadSession("session8", tc, {1}); Y_UNUSED(pipe4); WaitPartition("session8", tc, 0, "session2", "topic1", pipe2); - WaitPartition("", tc, 0, "", "", TActorId(), false);//no partitions to balance + WaitPartition("", tc, 0, "", "", TActorId(), false);//no partitions to balance tc.Runtime->Send(new IEventHandle(pipe2, tc.Edge, new TEvents::TEvPoisonPill()), 0, true); //will cause dying of pipe and first session - WaitPartition("session8", tc, 0, "", "", TActorId()); - WaitPartition("", tc, 0, "", "", TActorId(), false);//no partitions to balance + WaitPartition("session8", tc, 0, "", "", TActorId()); + WaitPartition("", tc, 0, "", "", TActorId(), false);//no partitions to balance RegisterReadSession("session3", tc); - WaitPartition("session3", tc, 0, "", "", TActorId()); - WaitPartition("session3", tc, 0, "", "", TActorId()); + WaitPartition("session3", tc, 0, "", "", TActorId()); + WaitPartition("session3", tc, 0, "", "", TActorId()); WaitPartition("session3", tc, 0, "session8", "topic1", pipe4); - WaitPartition("", tc, 0, "", "", TActorId(), false);//no partitions to balance + WaitPartition("", tc, 0, "", "", TActorId(), false);//no partitions to balance } @@ -90,18 +90,18 @@ Y_UNIT_TEST(TestGroupsBalancer2) { BalancerPrepare("topic", {{0, {1, 1}}, {1, {1, 2}}, {2, {1, 3}}, {3, {1, 4}}}, ssId, tc); - TActorId pipe = RegisterReadSession("session1", tc, {1,2}); + TActorId pipe = RegisterReadSession("session1", tc, {1,2}); Y_UNUSED(pipe); - WaitPartition("session1", tc, 0, "", "", TActorId()); - WaitPartition("session1", tc, 0, "", "", TActorId()); - WaitPartition("", tc, 0, "", "", TActorId(), false);//no partitions - return error - TActorId pipe2 = RegisterReadSession("session2", tc, {3,4}); + WaitPartition("session1", tc, 0, "", "", TActorId()); + WaitPartition("session1", tc, 0, "", "", TActorId()); + WaitPartition("", tc, 0, "", "", TActorId(), false);//no partitions - return error + TActorId pipe2 = RegisterReadSession("session2", tc, {3,4}); Y_UNUSED(pipe2); - WaitPartition("session2", tc, 0, "", "", TActorId()); - WaitPartition("session2", tc, 0, "", "", TActorId()); - WaitPartition("", tc, 0, "", "", TActorId(), false);//no partitions - return error + WaitPartition("session2", tc, 0, "", "", TActorId()); + WaitPartition("session2", tc, 0, "", "", TActorId()); + WaitPartition("", tc, 0, "", "", TActorId(), false);//no partitions - return error } Y_UNIT_TEST(TestGroupsBalancer3) { @@ -240,7 +240,7 @@ Y_UNIT_TEST(TestCreateBalancer) { BalancerPrepare("topic", {{1,{1,2}}}, ssId, tc); - TActorId pipe1 = RegisterReadSession("session0", tc, {1}); + TActorId pipe1 = RegisterReadSession("session0", tc, {1}); BalancerPrepare("topic", {{1,{1,2}}, {2,{1,3}}}, ssId, tc); @@ -252,23 +252,23 @@ Y_UNIT_TEST(TestCreateBalancer) { BalancerPrepare("topic", {{1,{1, 2}}, {2,{1, 3}}, {3,{1, 4}}}, ssId, tc); activeZone = false; - TActorId pipe = RegisterReadSession("session1", tc); - WaitPartition("session1", tc, 0, "", "", TActorId()); - WaitPartition("session1", tc, 0, "", "", TActorId()); - WaitPartition("session1", tc, 0, "", "", TActorId()); - WaitPartition("", tc, 0, "", "", TActorId(), false);//no partitions - return error - TActorId pipe2 = RegisterReadSession("session2", tc); + TActorId pipe = RegisterReadSession("session1", tc); + WaitPartition("session1", tc, 0, "", "", TActorId()); + WaitPartition("session1", tc, 0, "", "", TActorId()); + WaitPartition("session1", tc, 0, "", "", TActorId()); + WaitPartition("", tc, 0, "", "", TActorId(), false);//no partitions - return error + TActorId pipe2 = RegisterReadSession("session2", tc); Y_UNUSED(pipe2); WaitPartition("session2", tc, 1, "session1", "topic1", pipe); - WaitPartition("", tc, 0, "", "", TActorId(), false);//no partitions to balance + WaitPartition("", tc, 0, "", "", TActorId(), false);//no partitions to balance tc.Runtime->Send(new IEventHandle(pipe, tc.Edge, new TEvents::TEvPoisonPill()), 0, true); //will cause dying of pipe and first session TDispatchOptions options; options.FinalEvents.push_back(TDispatchOptions::TFinalEventCondition(TEvTabletPipe::EvServerDisconnected)); tc.Runtime->DispatchEvents(options); - WaitPartition("session2", tc, 0, "", "", TActorId()); - WaitPartition("session2", tc, 0, "", "", TActorId()); - WaitPartition("", tc, 0, "", "", TActorId(), false);//no partitions to balance + WaitPartition("session2", tc, 0, "", "", TActorId()); + WaitPartition("session2", tc, 0, "", "", TActorId()); + WaitPartition("", tc, 0, "", "", TActorId(), false);//no partitions to balance }); } @@ -314,7 +314,7 @@ Y_UNIT_TEST(TestCheckACL) { ui64 ssId = 9876; BootFakeSchemeShard(*tc.Runtime, ssId, state); IActor* ticketParser = NKikimr::CreateTicketParser(tc.Runtime->GetAppData().AuthConfig); - TActorId ticketParserId = tc.Runtime->Register(ticketParser); + TActorId ticketParserId = tc.Runtime->Register(ticketParser); tc.Runtime->RegisterService(NKikimr::MakeTicketParserID(), ticketParserId); TAutoPtr<IEventHandle> handle; @@ -605,7 +605,7 @@ Y_UNIT_TEST(TestWaitInOwners) { CmdSetOwner(0, tc, "owner", false); CmdSetOwner(0, tc, "owner", true); //will break last owner - TActorId newPipe = SetOwner(0, tc, "owner", false); //this owner will wait + TActorId newPipe = SetOwner(0, tc, "owner", false); //this owner will wait auto p = CmdSetOwner(0, tc, "owner", true); //will break last owner diff --git a/ydb/core/persqueue/pq_ut.h b/ydb/core/persqueue/pq_ut.h index 00de97fbbd4..3db93bc1047 100644 --- a/ydb/core/persqueue/pq_ut.h +++ b/ydb/core/persqueue/pq_ut.h @@ -94,7 +94,7 @@ struct TTestContext { TInitialEventsFilter InitialEventsFilter; TVector<ui64> TabletIds; THolder<TTestActorRuntime> Runtime; - TActorId Edge; + TActorId Edge; THashMap<ui32, ui32> MsgSeqNoMap; @@ -109,7 +109,7 @@ struct TTestContext { static bool RequestTimeoutFilter(TTestActorRuntimeBase& runtime, TAutoPtr<IEventHandle>& event, TDuration duration, TInstant& deadline) { if (event->GetTypeRewrite() == TEvents::TSystem::Wakeup) { - TActorId actorId = event->GetRecipientRewrite(); + TActorId actorId = event->GetRecipientRewrite(); IActor *actor = runtime.FindActor(actorId); if (actor && actor->GetActivityType() == NKikimrServices::TActivity::PERSQUEUE_ANS_ACTOR) { return true; @@ -382,8 +382,8 @@ void RestartTablet(TTestContext& tc) { } -TActorId SetOwner(const ui32 partition, TTestContext& tc, const TString& owner, bool force) { - TActorId pipeClient = tc.Runtime->ConnectToPipe(tc.TabletId, tc.Edge, 0, GetPipeConfigWithRetries()); +TActorId SetOwner(const ui32 partition, TTestContext& tc, const TString& owner, bool force) { + TActorId pipeClient = tc.Runtime->ConnectToPipe(tc.TabletId, tc.Edge, 0, GetPipeConfigWithRetries()); THolder<TEvPersQueue::TEvRequest> request; @@ -392,21 +392,21 @@ TActorId SetOwner(const ui32 partition, TTestContext& tc, const TString& owner, req->SetPartition(partition); req->MutableCmdGetOwnership()->SetOwner(owner); req->MutableCmdGetOwnership()->SetForce(force); - ActorIdToProto(pipeClient, req->MutablePipeClient()); + ActorIdToProto(pipeClient, req->MutablePipeClient()); tc.Runtime->SendToPipe(tc.TabletId, tc.Edge, request.Release(), 0, GetPipeConfigWithRetries(), pipeClient); return pipeClient; } -TActorId RegisterReadSession(const TString& session, TTestContext& tc, const TVector<ui32>& groups = {}) { - TActorId pipeClient = tc.Runtime->ConnectToPipe(tc.BalancerTabletId, tc.Edge, 0, GetPipeConfigWithRetries()); +TActorId RegisterReadSession(const TString& session, TTestContext& tc, const TVector<ui32>& groups = {}) { + TActorId pipeClient = tc.Runtime->ConnectToPipe(tc.BalancerTabletId, tc.Edge, 0, GetPipeConfigWithRetries()); THolder<TEvPersQueue::TEvRegisterReadSession> request; request.Reset(new TEvPersQueue::TEvRegisterReadSession); auto& req = request->Record; req.SetSession(session); - ActorIdToProto(pipeClient, req.MutablePipeClient()); + ActorIdToProto(pipeClient, req.MutablePipeClient()); req.SetClientId("user"); for (const auto& g : groups) { req.AddGroups(g); @@ -429,7 +429,7 @@ void WaitSessionKill(TTestContext& tc) { } -void WaitPartition(const TString &session, TTestContext& tc, ui32 partition, const TString& sessionToRelease, const TString& topic, const TActorId& pipe, bool ok = true) { +void WaitPartition(const TString &session, TTestContext& tc, ui32 partition, const TString& sessionToRelease, const TString& topic, const TActorId& pipe, bool ok = true) { TAutoPtr<IEventHandle> handle; tc.Runtime->ResetScheduledCount(); @@ -465,7 +465,7 @@ void WaitPartition(const TString &session, TTestContext& tc, ui32 partition, con req.SetPartition(partition); req.SetTopic(topic); req.SetClientId("user"); - ActorIdToProto(pipe, req.MutablePipeClient()); + ActorIdToProto(pipe, req.MutablePipeClient()); tc.Runtime->SendToPipe(tc.BalancerTabletId, tc.Edge, request.Release(), 0, GetPipeConfigWithRetries(), pipe); } @@ -478,11 +478,11 @@ void WaitPartition(const TString &session, TTestContext& tc, ui32 partition, con } -std::pair<TString, TActorId> CmdSetOwner(const ui32 partition, TTestContext& tc, const TString& owner = "default", bool force = true) { +std::pair<TString, TActorId> CmdSetOwner(const ui32 partition, TTestContext& tc, const TString& owner = "default", bool force = true) { TAutoPtr<IEventHandle> handle; TEvPersQueue::TEvResponse *result; TString cookie; - TActorId pipeClient; + TActorId pipeClient; for (i32 retriesLeft = 2; retriesLeft > 0; --retriesLeft) { try { tc.Runtime->ResetScheduledCount(); @@ -681,7 +681,7 @@ void CmdWrite(const ui32 partition, const TString& sourceId, const TVector<std:: void ReserveBytes(const ui32 partition, TTestContext& tc, - const TString& cookie, i32 msgSeqNo, i64 size, const TActorId& pipeClient, bool lastRequest) + const TString& cookie, i32 msgSeqNo, i64 size, const TActorId& pipeClient, bool lastRequest) { THolder<TEvPersQueue::TEvRequest> request; tc.Runtime->ResetScheduledCount(); @@ -690,7 +690,7 @@ void ReserveBytes(const ui32 partition, TTestContext& tc, req->SetPartition(partition); req->SetOwnerCookie(cookie); req->SetMessageNo(msgSeqNo); - ActorIdToProto(pipeClient, req->MutablePipeClient()); + ActorIdToProto(pipeClient, req->MutablePipeClient()); req->MutableCmdReserveBytes()->SetSize(size); req->MutableCmdReserveBytes()->SetLastRequest(lastRequest); tc.Runtime->SendToPipe(tc.TabletId, tc.Edge, request.Release(), 0, GetPipeConfigWithRetries()); @@ -699,7 +699,7 @@ void ReserveBytes(const ui32 partition, TTestContext& tc, } -void CmdReserveBytes(const ui32 partition, TTestContext& tc, const TString& ownerCookie, i32 msn, i64 size, TActorId pipeClient, bool noAnswer = false, bool lastRequest = false) { +void CmdReserveBytes(const ui32 partition, TTestContext& tc, const TString& ownerCookie, i32 msn, i64 size, TActorId pipeClient, bool noAnswer = false, bool lastRequest = false) { TAutoPtr<IEventHandle> handle; TEvPersQueue::TEvResponse *result; diff --git a/ydb/core/persqueue/read.h b/ydb/core/persqueue/read.h index ac2b9d27c5a..d0bc79c6929 100644 --- a/ydb/core/persqueue/read.h +++ b/ydb/core/persqueue/read.h @@ -21,7 +21,7 @@ namespace NPQ { return NKikimrServices::TActivity::PERSQUEUE_CACHE_ACTOR; } - TPQCacheProxy(const TActorId& tablet, TString topicName, ui32 size) + TPQCacheProxy(const TActorId& tablet, TString topicName, ui32 size) : Tablet(tablet) , TopicName(topicName) , Cookie(0) @@ -378,7 +378,7 @@ namespace NPQ { }; } - TActorId Tablet; + TActorId Tablet; TString TopicName; ui64 Cookie; // any TKvRequest would be placed into KvRequests or into BlockedReads depending on ReadsInProgress content diff --git a/ydb/core/persqueue/read_balancer.cpp b/ydb/core/persqueue/read_balancer.cpp index 4dbac2bfc00..4eb9c7206dd 100644 --- a/ydb/core/persqueue/read_balancer.cpp +++ b/ydb/core/persqueue/read_balancer.cpp @@ -1,6 +1,6 @@ #include "read_balancer.h" #include <ydb/core/tablet/tablet_exception.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <library/cpp/string_utils/base64/base64.h> namespace NKikimr { @@ -65,7 +65,7 @@ bool TPersQueueReadBalancer::TTxInit::Execute(TTransactionContext& txc, const TA ++Self->NumActiveParts; ui32 part = partsRowset.GetValue<Schema::Partitions::Partition>(); ui64 tabletId = partsRowset.GetValue<Schema::Partitions::TabletId>(); - Self->PartitionsInfo[part] = {tabletId, EPartitionState::EPS_FREE, TActorId(), part + 1}; + Self->PartitionsInfo[part] = {tabletId, EPartitionState::EPS_FREE, TActorId(), part + 1}; if (!partsRowset.Next()) return false; } @@ -506,7 +506,7 @@ void TPersQueueReadBalancer::Handle(TEvPersQueue::TEvUpdateBalancerConfig::TPtr if (it == PartitionsInfo.end()) { Y_VERIFY(group <= TotalGroups && group > prevGroups || TotalGroups == 0); Y_VERIFY(p.GetPartition() >= prevNextPartitionId && p.GetPartition() < NextPartitionId || NextPartitionId == 0); - partitionsInfo[p.GetPartition()] = {p.GetTabletId(), EPS_FREE, TActorId(), group}; + partitionsInfo[p.GetPartition()] = {p.GetTabletId(), EPS_FREE, TActorId(), group}; newPartitions.push_back(std::make_pair(p.GetPartition(), TPartInfo{p.GetTabletId(), group})); if (!NoGroupsInBase) newGroups.push_back(std::make_pair(group, p.GetPartition())); @@ -543,7 +543,7 @@ void TPersQueueReadBalancer::Handle(TEvPersQueue::TEvUpdateBalancerConfig::TPtr it = p.second.ClientGroupsInfo.find(group); } it->second.FreePartitions.push_back(part.first); - it->second.PartitionsInfo[part.first] = {part.second.TabletId, EPS_FREE, TActorId(), group}; + it->second.PartitionsInfo[part.first] = {part.second.TabletId, EPS_FREE, TActorId(), group}; it->second.ScheduleBalance(ctx); } } @@ -617,7 +617,7 @@ void TPersQueueReadBalancer::RequestTabletIfNeeded(const ui64 tabletId, const TA return; auto it = TabletPipes.find(tabletId); - TActorId pipeClient; + TActorId pipeClient; if (it == TabletPipes.end()) { NTabletPipe::TClientConfig clientConfig; pipeClient = ctx.RegisterWithSameMailbox(NTabletPipe::CreateClient(ctx.SelfID, tabletId, clientConfig)); @@ -724,10 +724,10 @@ void TPersQueueReadBalancer::GetACL(const TActorContext& ctx) { void TPersQueueReadBalancer::Handle(TEvTabletPipe::TEvServerConnected::TPtr& ev, const TActorContext& ctx) { - const TActorId& sender = ev->Get()->ClientId; + const TActorId& sender = ev->Get()->ClientId; auto it = PipesInfo.find(sender); if (it == PipesInfo.end()) { - PipesInfo.insert({sender, {"", "", TActorId(), false, 1}}); + PipesInfo.insert({sender, {"", "", TActorId(), false, 1}}); } else { it->second.ServerActors++; } @@ -760,9 +760,9 @@ void TPersQueueReadBalancer::TClientInfo::FillEmptyGroup(const ui32 group, const } void TPersQueueReadBalancer::TClientInfo::AddSession(const ui32 group, const THashMap<ui32, TPartitionInfo>& partitionsInfo, - const TActorId& sender, const NKikimrPQ::TRegisterReadSession& record) { + const TActorId& sender, const NKikimrPQ::TRegisterReadSession& record) { - TActorId pipe = ActorIdFromProto(record.GetPipeClient()); + TActorId pipe = ActorIdFromProto(record.GetPipeClient()); Y_VERIFY(pipe); @@ -793,7 +793,7 @@ void TPersQueueReadBalancer::Handle(TEvPersQueue::TEvRegisterReadSession::TPtr& { const auto& record = ev->Get()->Record; - TActorId pipe = ActorIdFromProto(record.GetPipeClient()); + TActorId pipe = ActorIdFromProto(record.GetPipeClient()); LOG_NOTICE_S(ctx, NKikimrServices::PERSQUEUE_READ_BALANCER, "client " << record.GetClientId() << " register session for pipe " << pipe << " session " << record.GetSession()); Y_VERIFY(!record.GetSession().empty()); @@ -900,7 +900,7 @@ void TPersQueueReadBalancer::Handle(TEvPersQueue::TEvGetReadSessionsInfo::TPtr& auto si = response->Record.AddReadSessions(); si->SetSession(s.second.Session); - ActorIdToProto(s.second.Sender, si->MutableSessionActor()); + ActorIdToProto(s.second.Sender, si->MutableSessionActor()); } } } @@ -968,7 +968,7 @@ void TPersQueueReadBalancer::TClientInfo::MergeGroups(const TActorContext& ctx) void TPersQueueReadBalancer::Handle(TEvPersQueue::TEvPartitionReleased::TPtr& ev, const TActorContext& ctx) { const auto& record = ev->Get()->Record; - TActorId sender = ActorIdFromProto(record.GetPipeClient()); + TActorId sender = ActorIdFromProto(record.GetPipeClient()); const TString& clientId = record.GetClientId(); auto pit = PartitionsInfo.find(record.GetPartition()); @@ -1007,7 +1007,7 @@ void TPersQueueReadBalancer::Handle(TEvPersQueue::TEvPartitionReleased::TPtr& ev } Y_VERIFY(kt != cit->second.SessionsInfo.end()); Y_VERIFY(jt != cit->second.PartitionsInfo.end()); - jt->second.Session = TActorId(); + jt->second.Session = TActorId(); jt->second.State = EPS_FREE; cit->second.FreePartitions.push_back(jt->first); @@ -1024,7 +1024,7 @@ void TPersQueueReadBalancer::RebuildStructs() { //TODO : track session with smallest and biggest number of (active but not suspended partitions } -void TPersQueueReadBalancer::RegisterSession(const TActorId& pipe, const TActorContext& ctx) +void TPersQueueReadBalancer::RegisterSession(const TActorId& pipe, const TActorContext& ctx) { //TODO : change structs for only this session, not all client auto it = PipesInfo.find(pipe); @@ -1036,7 +1036,7 @@ void TPersQueueReadBalancer::RegisterSession(const TActorId& pipe, const TActorC } } -void TPersQueueReadBalancer::UnregisterSession(const TActorId& pipe, const TActorContext& ctx) +void TPersQueueReadBalancer::UnregisterSession(const TActorId& pipe, const TActorContext& ctx) { //TODO : change structs for only this session auto it = PipesInfo.find(pipe); @@ -1047,7 +1047,7 @@ void TPersQueueReadBalancer::UnregisterSession(const TActorId& pipe, const TActo for (auto& c : jt->second.ClientGroupsInfo) { for (auto& p : c.second.PartitionsInfo) { //TODO: reverse map if (p.second.Session == pipe) { - p.second.Session = TActorId(); + p.second.Session = TActorId(); p.second.State = EPS_FREE; c.second.FreePartitions.push_back(p.first); } @@ -1124,7 +1124,7 @@ void TPersQueueReadBalancer::TClientGroupInfo::Balance(const TActorContext& ctx) } -void TPersQueueReadBalancer::TClientGroupInfo::LockPartition(const TActorId pipe, ui32 partition, const TActorContext& ctx) { +void TPersQueueReadBalancer::TClientGroupInfo::LockPartition(const TActorId pipe, ui32 partition, const TActorContext& ctx) { auto jt = SessionsInfo.find(std::make_pair(pipe, RandomNumber)); Y_VERIFY(jt != SessionsInfo.end()); @@ -1146,7 +1146,7 @@ void TPersQueueReadBalancer::TClientGroupInfo::LockPartition(const TActorId pipe res->Record.SetGeneration(Generation); res->Record.SetStep(++(*Step)); res->Record.SetClientId(ClientId); - ActorIdToProto(pipe, res->Record.MutablePipeClient()); + ActorIdToProto(pipe, res->Record.MutablePipeClient()); res->Record.SetTabletId(PartitionsInfo[partition].TabletId); LOG_INFO_S(ctx, NKikimrServices::PERSQUEUE_READ_BALANCER, GetPrefix() << "client " << ClientId << " lock partition for pipe " @@ -1155,7 +1155,7 @@ void TPersQueueReadBalancer::TClientGroupInfo::LockPartition(const TActorId pipe ctx.Send(pipeInfo.Sender, res.Release()); } -void TPersQueueReadBalancer::TClientGroupInfo::ReleasePartition(const TActorId pipe, const ui32 group, const ui32 count, const TActorContext& ctx) { +void TPersQueueReadBalancer::TClientGroupInfo::ReleasePartition(const TActorId pipe, const ui32 group, const ui32 count, const TActorContext& ctx) { auto it = SessionsInfo.find(std::make_pair(pipe, RandomNumber)); Y_VERIFY(it != SessionsInfo.end()); @@ -1171,7 +1171,7 @@ void TPersQueueReadBalancer::TClientGroupInfo::ReleasePartition(const TActorId p res->Record.SetClientId(ClientId); res->Record.SetCount(count); res->Record.SetGroup(group); - ActorIdToProto(pipe, res->Record.MutablePipeClient()); + ActorIdToProto(pipe, res->Record.MutablePipeClient()); LOG_INFO_S(ctx, NKikimrServices::PERSQUEUE_READ_BALANCER, GetPrefix() << "client " << ClientId << " release partition group " << group << " for pipe " << pipe << " session " << sessionInfo.Session); diff --git a/ydb/core/persqueue/read_balancer.h b/ydb/core/persqueue/read_balancer.h index 96edde883c7..6a07edcb784 100644 --- a/ydb/core/persqueue/read_balancer.h +++ b/ydb/core/persqueue/read_balancer.h @@ -286,9 +286,9 @@ class TPersQueueReadBalancer : public TActor<TPersQueueReadBalancer>, public TTa void Handle(TEvPersQueue::TEvStatusResponse::TPtr& ev, const TActorContext& ctx); void Handle(NSchemeShard::TEvSchemeShard::TEvDescribeSchemeResult::TPtr& ev, const TActorContext& ctx); - void RegisterSession(const TActorId& pipe, const TActorContext& ctx); + void RegisterSession(const TActorId& pipe, const TActorContext& ctx); struct TPipeInfo; - void UnregisterSession(const TActorId& pipe, const TActorContext& ctx); + void UnregisterSession(const TActorId& pipe, const TActorContext& ctx); void RebuildStructs(); bool Inited; @@ -308,14 +308,14 @@ class TPersQueueReadBalancer : public TActor<TPersQueueReadBalancer>, public TTa ui64 TxId; ui32 NumActiveParts; - TVector<TActorId> WaitingResponse; + TVector<TActorId> WaitingResponse; TVector<TEvPersQueue::TEvCheckACL::TPtr> WaitingACLRequests; TVector<TEvPersQueue::TEvDescribe::TPtr> WaitingDescribeRequests; struct TPipeInfo { TString ClientId; TString Session; - TActorId Sender; + TActorId Sender; bool WithGroups; ui32 ServerActors; }; @@ -328,7 +328,7 @@ class TPersQueueReadBalancer : public TActor<TPersQueueReadBalancer>, public TTa struct TPartitionInfo { ui64 TabletId; EPartitionState State; - TActorId Session; + TActorId Session; ui32 GroupId; }; @@ -345,7 +345,7 @@ class TPersQueueReadBalancer : public TActor<TPersQueueReadBalancer>, public TTa {} TString Session; - TActorId Sender; + TActorId Sender; ui32 NumSuspended; ui32 NumActive; @@ -366,12 +366,12 @@ class TPersQueueReadBalancer : public TActor<TPersQueueReadBalancer>, public TTa THashMap<ui32, TPartitionInfo> PartitionsInfo; // partitionId -> info std::deque<ui32> FreePartitions; - THashMap<std::pair<TActorId, ui64>, TSessionInfo> SessionsInfo; //map from ActorID and random value - need for reordering sessions in different topics + THashMap<std::pair<TActorId, ui64>, TSessionInfo> SessionsInfo; //map from ActorID and random value - need for reordering sessions in different topics void ScheduleBalance(const TActorContext& ctx); void Balance(const TActorContext& ctx); - void LockPartition(const TActorId pipe, ui32 partition, const TActorContext& ctx); - void ReleasePartition(const TActorId pipe, const ui32 group, const ui32 count, const TActorContext& ctx); + void LockPartition(const TActorId pipe, ui32 partition, const TActorContext& ctx); + void ReleasePartition(const TActorId pipe, const ui32 group, const ui32 count, const TActorContext& ctx); TStringBuilder GetPrefix() const; bool WakeupScheduled = false; @@ -405,18 +405,18 @@ class TPersQueueReadBalancer : public TActor<TPersQueueReadBalancer>, public TTa TClientGroupInfo& AddGroup(const ui32 group); void FillEmptyGroup(const ui32 group, const THashMap<ui32, TPartitionInfo>& partitionsInfo); void AddSession(const ui32 group, const THashMap<ui32, TPartitionInfo>& partitionsInfo, - const TActorId& sender, const NKikimrPQ::TRegisterReadSession& record); + const TActorId& sender, const NKikimrPQ::TRegisterReadSession& record); TStringBuilder GetPrefix() const; }; THashMap<TString, TClientInfo> ClientsInfo; //map from userId -> to info - THashMap<TActorId, TPipeInfo> PipesInfo; + THashMap<TActorId, TPipeInfo> PipesInfo; NMetrics::TResourceMetrics *ResourceMetrics; - THashMap<ui64, TActorId> TabletPipes; + THashMap<ui64, TActorId> TabletPipes; THashSet<ui64> WaitingForStat; bool WaitingForACL; @@ -436,7 +436,7 @@ public: return NKikimrServices::TActivity::PERSQUEUE_READ_BALANCER_ACTOR; } - TPersQueueReadBalancer(const TActorId &tablet, TTabletStorageInfo *info) + TPersQueueReadBalancer(const TActorId &tablet, TTabletStorageInfo *info) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, new NMiniKQL::TMiniKQLFactory) , Inited(false) diff --git a/ydb/core/persqueue/subscriber.cpp b/ydb/core/persqueue/subscriber.cpp index 27b1d38cacc..641c566a1c9 100644 --- a/ydb/core/persqueue/subscriber.cpp +++ b/ydb/core/persqueue/subscriber.cpp @@ -48,7 +48,7 @@ TVector<std::pair<TReadInfo, ui64>> TSubscriberLogic::CompleteSubscriptions(cons } -TSubscriber::TSubscriber(const ui32 partition, TTabletCountersBase& counters, const TActorId& tablet) +TSubscriber::TSubscriber(const ui32 partition, TTabletCountersBase& counters, const TActorId& tablet) : Subscriber() , Partition(partition) , Counters(counters) diff --git a/ydb/core/persqueue/subscriber.h b/ydb/core/persqueue/subscriber.h index e21ba9c93eb..7bbfe41e1f2 100644 --- a/ydb/core/persqueue/subscriber.h +++ b/ydb/core/persqueue/subscriber.h @@ -110,7 +110,7 @@ private: class TSubscriber : public TNonCopyable { public: - TSubscriber(const ui32 partition, TTabletCountersBase& counters, const TActorId& tablet); + TSubscriber(const ui32 partition, TTabletCountersBase& counters, const TActorId& tablet); //will wait for new data or timeout for this read and set timer for timeout ms void AddSubscription(TReadInfo&& info, const ui32 timeout, const ui64 cookie, const TActorContext& ctx); @@ -125,7 +125,7 @@ private: TSubscriberLogic Subscriber; const ui32 Partition; TTabletCountersBase& Counters; - TActorId Tablet; + TActorId Tablet; }; diff --git a/ydb/core/persqueue/type_codecs.h b/ydb/core/persqueue/type_codecs.h index 1eb74c44b54..788e0fce0e1 100644 --- a/ydb/core/persqueue/type_codecs.h +++ b/ydb/core/persqueue/type_codecs.h @@ -4,8 +4,8 @@ #include "type_coders.h" #include "type_decoders.h" -namespace NKikimr { -namespace NScheme { +namespace NKikimr { +namespace NScheme { template <typename TCoder, typename TDecoder> class TCodecImpl : public ICodec { @@ -64,6 +64,6 @@ class TDeltaZigZagCodec : public TCodecImpl<TDeltaZigZagCoder<TIntType, IsNullab template <bool IsNullable> class TBoolCodec : public TCodecImpl<TBoolCoder<IsNullable>, TBoolDecoder<IsNullable>> { }; -} // namespace NScheme -} // namespace NKikimr +} // namespace NScheme +} // namespace NKikimr diff --git a/ydb/core/persqueue/type_codecs_defs.cpp b/ydb/core/persqueue/type_codecs_defs.cpp index 03f61f667f1..f333be5cfc6 100644 --- a/ydb/core/persqueue/type_codecs_defs.cpp +++ b/ydb/core/persqueue/type_codecs_defs.cpp @@ -3,8 +3,8 @@ #include <ydb/core/scheme/scheme_types_defs.h> #include <util/generic/typetraits.h> -namespace NKikimr { -namespace NScheme { +namespace NKikimr { +namespace NScheme { void InitDefaults(TTypeCodecs* codecs, TCodecType type) { auto nullable = TCodecSig(type, true); @@ -63,47 +63,47 @@ void AddIntCodecs(TTypeCodecs* codecs) { } } - -TTypeCodecs::TTypeCodecs(TTypeId typeId) { + +TTypeCodecs::TTypeCodecs(TTypeId typeId) { using namespace NScheme; AddCodec<TVarLenCodec<true>>(); AddCodec<TVarLenCodec<false>>(); switch (typeId) { - case NTypeIds::Int32: + case NTypeIds::Int32: AddFixedLen<TInt32>(this); AddIntCodecs<TInt32>(this); break; - case NTypeIds::Uint32: + case NTypeIds::Uint32: AddFixedLen<TUint32>(this); AddIntCodecs<TUint32>(this); break; - case NTypeIds::Int64: + case NTypeIds::Int64: AddFixedLen<TInt64>(this); AddIntCodecs<TInt64>(this); break; - case NTypeIds::Uint64: + case NTypeIds::Uint64: AddFixedLen<TUint64>(this); AddIntCodecs<TUint64>(this); break; - case NTypeIds::Byte: + case NTypeIds::Byte: AddFixedLen<TUint8>(this); break; - case NTypeIds::Bool: + case NTypeIds::Bool: AddFixedLen<TBool>(this, false); AddCodec<TBoolCodec<true>>(); AddCodec<TBoolCodec<false>>(); InitDefaults(this, TCodecType::Bool); break; - case NTypeIds::Double: + case NTypeIds::Double: AddFixedLen<TDouble>(this); break; - case NTypeIds::Float: + case NTypeIds::Float: AddFixedLen<TFloat>(this); break; - case NTypeIds::PairUi64Ui64: + case NTypeIds::PairUi64Ui64: AddFixedLen<TPairUi64Ui64>(this); break; @@ -117,10 +117,10 @@ TTypeCodecs::TTypeCodecs(TTypeId typeId) { InitDefaults(this, TCodecType::VarLen); break; - case NTypeIds::ActorId: - AddFixedLen<TActorId>(this); + case NTypeIds::ActorId: + AddFixedLen<TActorId>(this); break; - case NTypeIds::StepOrderId: + case NTypeIds::StepOrderId: AddFixedLen<TStepOrderId>(this); break; case NTypeIds::Decimal: @@ -142,5 +142,5 @@ TTypeCodecs::TTypeCodecs(TTypeId typeId) { } } -} // namespace NScheme -} // namespace NKikimr +} // namespace NScheme +} // namespace NKikimr diff --git a/ydb/core/persqueue/type_codecs_defs.h b/ydb/core/persqueue/type_codecs_defs.h index f63f82d19ad..90c55d4b892 100644 --- a/ydb/core/persqueue/type_codecs_defs.h +++ b/ydb/core/persqueue/type_codecs_defs.h @@ -11,8 +11,8 @@ #include <string.h> -namespace NKikimr { -namespace NScheme { +namespace NKikimr { +namespace NScheme { class TDataRef; class TTypeCodecs; @@ -320,7 +320,7 @@ private: ******************************************************************************/ class TTypeCodecs { public: - TTypeCodecs(TTypeId typeId = 0); + TTypeCodecs(TTypeId typeId = 0); template <bool IsNullable> const ICodec* GetDefaultCodec() const { @@ -387,5 +387,5 @@ inline IChunkDecoder::TPtr IChunkDecoder::ReadChunk(const TDataRef& data, const return codec->ReadChunk(data); } -} // namespace NScheme -} // namespace NKikimr +} // namespace NScheme +} // namespace NKikimr diff --git a/ydb/core/persqueue/type_codecs_ut.cpp b/ydb/core/persqueue/type_codecs_ut.cpp index e5954c4a013..d9f0b647557 100644 --- a/ydb/core/persqueue/type_codecs_ut.cpp +++ b/ydb/core/persqueue/type_codecs_ut.cpp @@ -8,7 +8,7 @@ #include <util/random/fast.h> #include <util/datetime/base.h> -namespace NKikimr { +namespace NKikimr { using ICodec = NScheme::ICodec; using TTypeCodecs = NScheme::TTypeCodecs; @@ -256,5 +256,5 @@ Y_UNIT_TEST_SUITE(TTypeCodecsTest) { } -} // namespace NKikimr +} // namespace NKikimr diff --git a/ydb/core/persqueue/type_coders.h b/ydb/core/persqueue/type_coders.h index 38bf523019e..26c2566d32e 100644 --- a/ydb/core/persqueue/type_coders.h +++ b/ydb/core/persqueue/type_coders.h @@ -8,8 +8,8 @@ #include <util/generic/typetraits.h> -namespace NKikimr { -namespace NScheme { +namespace NKikimr { +namespace NScheme { //////////////////////////////////////////////////////////////////////////////// @@ -306,6 +306,6 @@ protected: //////////////////////////////////////////////////////////////////////////////// -} // namespace NScheme -} // namespace NKikimr +} // namespace NScheme +} // namespace NKikimr diff --git a/ydb/core/persqueue/type_decoders.h b/ydb/core/persqueue/type_decoders.h index 009950cfebf..30a8846923c 100644 --- a/ydb/core/persqueue/type_decoders.h +++ b/ydb/core/persqueue/type_decoders.h @@ -12,8 +12,8 @@ #include <util/generic/strbuf.h> #include <util/system/unaligned_mem.h> -namespace NKikimr { -namespace NScheme { +namespace NKikimr { +namespace NScheme { using TMaskIterator = TBitIterator<ui8>; @@ -559,6 +559,6 @@ const bool TBoolDecoder<IsNullable>::Values[2] = {false, true}; //////////////////////////////////////////////////////////////////////////////// -} // namespace NScheme -} // namespace NKikimr +} // namespace NScheme +} // namespace NKikimr diff --git a/ydb/core/protos/blobstorage.proto b/ydb/core/protos/blobstorage.proto index 2a3c44982a3..a08ccbf865c 100644 --- a/ydb/core/protos/blobstorage.proto +++ b/ydb/core/protos/blobstorage.proto @@ -1147,7 +1147,7 @@ message TEvGroupStatReport { repeated uint32 Buckets = 1; } optional uint32 GroupId = 1; - optional NActorsProto.TActorId VDiskServiceId = 2; + optional NActorsProto.TActorId VDiskServiceId = 2; optional TLatencyHistogram PutTabletLog = 10; optional TLatencyHistogram PutUserData = 11; optional TLatencyHistogram GetFast = 12; diff --git a/ydb/core/protos/config.proto b/ydb/core/protos/config.proto index c5c16c97d5d..d64169d4fc0 100644 --- a/ydb/core/protos/config.proto +++ b/ydb/core/protos/config.proto @@ -1346,8 +1346,8 @@ message TAppConfig { optional NKikimrBlobStorage.TAllVDiskKinds VDiskConfig = 14; optional NKikimrBlobStorage.TDriveModelList DriveModelConfig = 31; optional NKikimrBlobStorage.TIncrHugeConfig IncrHugeConfig = 18; - optional string UDFsDir = 15; - repeated string UDFsPaths = 16; + optional string UDFsDir = 15; + repeated string UDFsPaths = 16; optional TKQPConfig KQPConfig = 17; optional TMemoryLogConfig MemoryLogConfig = 19; optional TGRpcConfig GRpcConfig = 20; diff --git a/ydb/core/protos/console_config.proto b/ydb/core/protos/console_config.proto index 8a7e1d702cc..d5a32d05c66 100644 --- a/ydb/core/protos/console_config.proto +++ b/ydb/core/protos/console_config.proto @@ -289,7 +289,7 @@ message TSubscriber { uint64 TabletId = 1; // Subscriber is an actor, it should be registered as // a service and use its service id. - NActorsProto.TActorId ServiceId = 2; + NActorsProto.TActorId ServiceId = 2; } } diff --git a/ydb/core/protos/flat_tx_scheme.proto b/ydb/core/protos/flat_tx_scheme.proto index cdf27034874..dc06523e5cc 100644 --- a/ydb/core/protos/flat_tx_scheme.proto +++ b/ydb/core/protos/flat_tx_scheme.proto @@ -92,7 +92,7 @@ message TEvCancelTxResult { } message TEvUpdateConfig { - optional NActorsProto.TActorId Source = 1; + optional NActorsProto.TActorId Source = 1; optional TConfig Config = 2; } diff --git a/ydb/core/protos/hive.proto b/ydb/core/protos/hive.proto index dd59009570a..0059db81de8 100644 --- a/ydb/core/protos/hive.proto +++ b/ydb/core/protos/hive.proto @@ -129,7 +129,7 @@ message TEvTabletCreationResult { message TEvStopTablet { optional fixed64 TabletID = 1; - optional NActorsProto.TActorId ActorToNotify = 2; + optional NActorsProto.TActorId ActorToNotify = 2; } message TEvStopTabletResult { @@ -372,7 +372,7 @@ message TEvGetTabletStorageInfoRegistered { message TEvLockTabletExecution { optional fixed64 TabletID = 1; - optional NActorsProto.TActorId OwnerActor = 2; + optional NActorsProto.TActorId OwnerActor = 2; optional bool Reconnect = 3; optional uint64 MaxReconnectTimeout = 4; // ms } @@ -389,7 +389,7 @@ message TEvLockTabletExecutionResult { message TEvUnlockTabletExecution { optional fixed64 TabletID = 1; - optional NActorsProto.TActorId OwnerActor = 2; + optional NActorsProto.TActorId OwnerActor = 2; } message TEvUnlockTabletExecutionResult { diff --git a/ydb/core/protos/kesus.proto b/ydb/core/protos/kesus.proto index 658b74aebce..7c03a3ccd95 100644 --- a/ydb/core/protos/kesus.proto +++ b/ydb/core/protos/kesus.proto @@ -52,7 +52,7 @@ message TEvDescribeProxies { message TEvDescribeProxiesResult { message TProxyDescription { - NActorsProto.TActorId ActorID = 1; + NActorsProto.TActorId ActorID = 1; uint64 Generation = 2; repeated uint64 AttachedSessions = 3; } @@ -69,7 +69,7 @@ message TEvDescribeSessionsResult { uint64 SessionId = 1; uint64 TimeoutMillis = 2; string Description = 3; - NActorsProto.TActorId OwnerProxy = 4; + NActorsProto.TActorId OwnerProxy = 4; } repeated TSessionDescription Sessions = 1; @@ -421,7 +421,7 @@ message TEvSubscribeOnResources { double InitialAmount = 3; } repeated TResourceSubscribeInfo Resources = 1; - NActorsProto.TActorId ActorID = 2; + NActorsProto.TActorId ActorID = 2; } message TEvSubscribeOnResourcesResult { @@ -443,7 +443,7 @@ message TEvUpdateConsumptionState { double Amount = 3; } repeated TResourceInfo ResourcesInfo = 1; - NActorsProto.TActorId ActorID = 2; + NActorsProto.TActorId ActorID = 2; } // Ack for TEvUpdateConsumptionState event. diff --git a/ydb/core/protos/kqp.proto b/ydb/core/protos/kqp.proto index 4e3cd2a4adc..9b46b720e11 100644 --- a/ydb/core/protos/kqp.proto +++ b/ydb/core/protos/kqp.proto @@ -148,7 +148,7 @@ message TEvQueryRequest { optional TQueryRequest Request = 1; optional string TraceId = 2; optional string UserToken = 3; - optional NActorsProto.TActorId RequestActorId = 4; + optional NActorsProto.TActorId RequestActorId = 4; optional string RequestType = 5; optional TRlPath RlPath = 6; } @@ -356,7 +356,7 @@ message TExecuterTxRequest { message TEvExecuterTxRequest { optional TExecuterTxRequest Request = 1; - optional NActorsProto.TActorId Target = 2; + optional NActorsProto.TActorId Target = 2; optional string TraceId = 3; optional string UserToken = 4; }; @@ -395,7 +395,7 @@ message TEvExecuterStreamProfile { }; message TEvExecuterProgress { - optional NActorsProto.TActorId ExecuterActorId = 1; + optional NActorsProto.TActorId ExecuterActorId = 1; }; message TKqpProxyNodeResources { diff --git a/ydb/core/protos/msgbus.proto b/ydb/core/protos/msgbus.proto index e4f3403cbf7..df7cda59800 100644 --- a/ydb/core/protos/msgbus.proto +++ b/ydb/core/protos/msgbus.proto @@ -397,7 +397,7 @@ message TTypeMetadataResponse { optional uint64 ETag = 2; optional bytes TypeMetadata = 3; - optional bytes FunctionMetadata = 4; + optional bytes FunctionMetadata = 4; } message TMessageBusTraceRequest { diff --git a/ydb/core/protos/msgbus_pq.proto b/ydb/core/protos/msgbus_pq.proto index cd2a0a30c16..849b292383a 100644 --- a/ydb/core/protos/msgbus_pq.proto +++ b/ydb/core/protos/msgbus_pq.proto @@ -130,7 +130,7 @@ message TPersQueuePartitionRequest { optional TCmdDeregisterMessageGroup CmdDeregisterMessageGroup = 21; optional TCmdSplitMessageGroup CmdSplitMessageGroup = 22; - optional NActorsProto.TActorId PipeClient = 15; + optional NActorsProto.TActorId PipeClient = 15; optional uint64 Cookie = 16; //client cookie to be returned in response diff --git a/ydb/core/protos/pqconfig.proto b/ydb/core/protos/pqconfig.proto index 5a3f39bf9d2..7c859274496 100644 --- a/ydb/core/protos/pqconfig.proto +++ b/ydb/core/protos/pqconfig.proto @@ -390,7 +390,7 @@ message TGetPartitionIdForWriteResponse { message TRegisterReadSession { optional string Session = 1; - optional NActorsProto.TActorId PipeClient = 2; + optional NActorsProto.TActorId PipeClient = 2; optional string ClientId = 3; optional string ClientNode = 4; @@ -441,7 +441,7 @@ message TReadSessionsInfoResponse { optional uint64 TabletId = 2; message TReadSessionInfo { - optional NActorsProto.TActorId SessionActor = 1; + optional NActorsProto.TActorId SessionActor = 1; optional string Session = 2; } @@ -456,7 +456,7 @@ message TLockPartition { optional uint32 Step = 5; optional string Session = 6; optional string ClientId = 7; - optional NActorsProto.TActorId PipeClient = 8; + optional NActorsProto.TActorId PipeClient = 8; optional string Path = 9; } @@ -466,7 +466,7 @@ message TReleasePartition { optional string Session = 3; optional string ClientId = 4; optional uint32 Count = 5; - optional NActorsProto.TActorId PipeClient = 6; + optional NActorsProto.TActorId PipeClient = 6; optional uint32 Group = 7; optional string Path = 8; } @@ -475,7 +475,7 @@ message TPartitionReleased { optional uint32 Partition = 1; optional string Topic = 2; optional string Session = 3; - optional NActorsProto.TActorId PipeClient = 4; + optional NActorsProto.TActorId PipeClient = 4; optional string ClientId = 5; } @@ -627,7 +627,7 @@ message THasDataInfo { optional int32 Partition = 1; optional uint64 Offset = 2; optional uint64 Deadline = 3; //in ms - optional NActorsProto.TActorId Sender = 4; + optional NActorsProto.TActorId Sender = 4; optional uint64 Cookie = 5; optional string ClientId = 6; diff --git a/ydb/core/protos/scheme_type_operation.proto b/ydb/core/protos/scheme_type_operation.proto index e273a5fbfc4..e41db39a7d5 100644 --- a/ydb/core/protos/scheme_type_operation.proto +++ b/ydb/core/protos/scheme_type_operation.proto @@ -11,7 +11,7 @@ message TMetadata { message TDescription { repeated TArg Arg = 1; - optional bool Deterministic = 2; + optional bool Deterministic = 2; } message TOperation { diff --git a/ydb/core/protos/statestorage.proto b/ydb/core/protos/statestorage.proto index 92a71344b27..4165af89ba0 100644 --- a/ydb/core/protos/statestorage.proto +++ b/ydb/core/protos/statestorage.proto @@ -88,7 +88,7 @@ message TEvReplicaBoardPublish { optional bytes Payload = 2; optional uint64 TtlMs = 3; optional bool Register = 4; - optional NActorsProto.TActorId Owner = 5; + optional NActorsProto.TActorId Owner = 5; }; message TEvReplicaBoardPublishAck { @@ -96,7 +96,7 @@ message TEvReplicaBoardPublishAck { message TEvReplicaBoardLookup { optional string Path = 1; - optional NActorsProto.TActorId Owner = 2; + optional NActorsProto.TActorId Owner = 2; optional bool Subscribe = 3; }; @@ -104,7 +104,7 @@ message TEvReplicaBoardCleanup { }; message TBoardEntryInfo { - optional NActorsProto.TActorId Owner = 1; + optional NActorsProto.TActorId Owner = 1; optional bytes Payload = 2; }; diff --git a/ydb/core/protos/tablet_pipe.proto b/ydb/core/protos/tablet_pipe.proto index 8f4e03a8fe4..e1fd3769f84 100644 --- a/ydb/core/protos/tablet_pipe.proto +++ b/ydb/core/protos/tablet_pipe.proto @@ -10,22 +10,22 @@ enum EFeature { message TEvConnect { optional uint64 TabletId = 1; - optional NActorsProto.TActorId ClientId = 2; + optional NActorsProto.TActorId ClientId = 2; optional uint32 Features = 3; }; message TEvConnectResult { optional NKikimrProto.EReplyStatus Status = 1; optional uint64 TabletId = 2; - optional NActorsProto.TActorId ClientId = 3; - optional NActorsProto.TActorId ServerId = 4; + optional NActorsProto.TActorId ClientId = 3; + optional NActorsProto.TActorId ServerId = 4; optional bool Leader = 5 [default = true]; }; message TEvPush { optional uint64 TabletId = 1; optional uint32 Type = 2; - optional NActorsProto.TActorId Sender = 3; + optional NActorsProto.TActorId Sender = 3; optional bytes Buffer = 4; optional uint64 Cookie = 5; optional bool ExtendedFormat = 6; @@ -34,8 +34,8 @@ message TEvPush { message TEvPeerClosed { optional uint64 TabletId = 1; - optional NActorsProto.TActorId ClientId = 2; - optional NActorsProto.TActorId ServerId = 3; + optional NActorsProto.TActorId ClientId = 2; + optional NActorsProto.TActorId ServerId = 3; }; message TEvPeerShutdown { diff --git a/ydb/core/protos/tablet_tracing_signals.proto b/ydb/core/protos/tablet_tracing_signals.proto index 8aefaca7e29..51c1c7dd4b4 100644 --- a/ydb/core/protos/tablet_tracing_signals.proto +++ b/ydb/core/protos/tablet_tracing_signals.proto @@ -19,7 +19,7 @@ message TOnTabletBootstrap { optional fixed64 TimeStamp = 1; optional bool Leader = 2; optional uint32 SuggestedGeneration = 3; - optional NActorsProto.TActorId StateStorageProxyID = 4; + optional NActorsProto.TActorId StateStorageProxyID = 4; } message TOnHandleStateStorageInfoResolve { @@ -50,13 +50,13 @@ message TOnPromoteToCandidate { message TOnTabletBlockBlobStorage { optional fixed64 TimeStamp = 1; - optional NActorsProto.TActorId ReqBlockBlobStorageID = 2; + optional NActorsProto.TActorId ReqBlockBlobStorageID = 2; optional uint32 KnownGeneration = 3; } message TOnTabletRebuildGraph { optional fixed64 TimeStamp = 1; - optional NActorsProto.TActorId TabletReqRebuildGraphID = 2; + optional NActorsProto.TActorId TabletReqRebuildGraphID = 2; optional TTraceID RebuildGraphTraceID = 3; } diff --git a/ydb/core/protos/tenant_slot_broker.proto b/ydb/core/protos/tenant_slot_broker.proto index 36efcccbe55..bccc7f7dc7b 100644 --- a/ydb/core/protos/tenant_slot_broker.proto +++ b/ydb/core/protos/tenant_slot_broker.proto @@ -54,7 +54,7 @@ message TAlterTenant { } message TRegisterPool { - optional NActorsProto.TActorId ClientId = 1; + optional NActorsProto.TActorId ClientId = 1; optional uint64 SeqNo = 2; } diff --git a/ydb/core/protos/tx.proto b/ydb/core/protos/tx.proto index 525f678e428..1f18e7eca6e 100644 --- a/ydb/core/protos/tx.proto +++ b/ydb/core/protos/tx.proto @@ -167,7 +167,7 @@ message TMediatorTransaction { optional bytes Body = 4; - optional NActorsProto.TActorId AckTo = 10; // must be set to source coordinator-mediator queue + optional NActorsProto.TActorId AckTo = 10; // must be set to source coordinator-mediator queue } message TEvMediatorPlanStep { diff --git a/ydb/core/protos/tx_datashard.proto b/ydb/core/protos/tx_datashard.proto index 1e980a09f38..89863b25400 100644 --- a/ydb/core/protos/tx_datashard.proto +++ b/ydb/core/protos/tx_datashard.proto @@ -43,7 +43,7 @@ enum EDatashardState { } message TEvGetShardState { - optional NActorsProto.TActorId Source = 1; + optional NActorsProto.TActorId Source = 1; } message TEvGetShardStateResult { @@ -60,7 +60,7 @@ message TShardOpResult { } message TEvSchemaChanged { - optional NActorsProto.TActorId Source = 1; + optional NActorsProto.TActorId Source = 1; optional uint64 Origin = 2; optional uint32 State = 3; // EDatashardState optional uint64 TxId = 4; @@ -76,7 +76,7 @@ message TEvSchemaChangedResult { } message TEvStateChanged { - optional NActorsProto.TActorId Source = 1; + optional NActorsProto.TActorId Source = 1; optional uint64 TabletId = 2; optional uint32 State = 3; // EDatashardState } @@ -227,7 +227,7 @@ message TKqpTransaction { message TKqpTaskInfo { optional uint64 TaskId = 1; - optional NActorsProto.TActorId ComputeActor = 2; + optional NActorsProto.TActorId ComputeActor = 2; } message TKqpReply { @@ -250,7 +250,7 @@ message TDataTransaction { optional bool ReadOnly = 7; optional TReadTableTransaction ReadTableTransaction = 8; optional bool StreamResponse = 9; - optional NActorsProto.TActorId Sink = 10; + optional NActorsProto.TActorId Sink = 10; optional bool LlvmRuntime = 11; optional uint64 PerShardKeysSizeLimitBytes = 12; optional TKqpTransaction KqpTransaction = 13; @@ -471,7 +471,7 @@ message TMvccSnapshot { message TEvProposeTransaction { optional ETransactionKind TxKind = 1; - optional NActorsProto.TActorId Source = 2; + optional NActorsProto.TActorId Source = 2; optional bytes TxBody = 3; optional uint64 TxId = 4; optional uint32 ExecLevel = 5; @@ -528,7 +528,7 @@ message TTabletInfo { optional fixed64 TabletId = 1; optional uint32 Generation = 2; optional uint64 Step = 3; - optional NActorsProto.TActorId ActorId = 4; + optional NActorsProto.TActorId ActorId = 4; optional bool IsFollower = 5; } @@ -698,7 +698,7 @@ message TEvSplitTransferSnapshotAck { // Schemeshard -> Src datashard message TEvSplitPartitioningChanged { - optional NActorsProto.TActorId Sender = 2; + optional NActorsProto.TActorId Sender = 2; optional uint64 OperationCookie = 1; } diff --git a/ydb/core/protos/tx_scheme.proto b/ydb/core/protos/tx_scheme.proto index ea0547ea904..037db6c4972 100644 --- a/ydb/core/protos/tx_scheme.proto +++ b/ydb/core/protos/tx_scheme.proto @@ -28,7 +28,7 @@ message TEvNavigateSchemePart { optional bytes Path = 1; optional bytes TextPath = 2; optional uint32 ReadMaterializedFamily = 3; - optional NActorsProto.TActorId Source = 4; + optional NActorsProto.TActorId Source = 4; optional bool ShowFamilies = 5 [default = true]; optional bool ShowChildrenTags = 6 [default = true]; } @@ -41,7 +41,7 @@ message TEvNavigateSchemePartResult { } message TEvInitRootShard { - optional NActorsProto.TActorId Source = 1; + optional NActorsProto.TActorId Source = 1; optional uint32 RootTag = 2; optional bytes RootTagName = 3; optional TConfig Config = 4; @@ -55,7 +55,7 @@ message TEvInitRootShardResult { } message TEvUpdateConfig { - optional NActorsProto.TActorId Source = 1; + optional NActorsProto.TActorId Source = 1; optional TConfig Config = 2; } diff --git a/ydb/core/quoter/kesus_quoter_proxy.cpp b/ydb/core/quoter/kesus_quoter_proxy.cpp index 69bd919c30e..4d2d20691e9 100644 --- a/ydb/core/quoter/kesus_quoter_proxy.cpp +++ b/ydb/core/quoter/kesus_quoter_proxy.cpp @@ -241,13 +241,13 @@ class TKesusQuoterProxy : public TActorBootstrapped<TKesusQuoterProxy> { }; }; - const TActorId QuoterServiceId; + const TActorId QuoterServiceId; const ui64 QuoterId; const TVector<TString> Path; const TString LogPrefix; TIntrusiveConstPtr<NSchemeCache::TSchemeCacheNavigate::TKesusInfo> KesusInfo; THolder<ITabletPipeFactory> TabletPipeFactory; - TActorId KesusPipeClient; + TActorId KesusPipeClient; bool Connected = false; TInstant DisconnectTime; @@ -396,7 +396,7 @@ private: if (Connected) { KESUS_PROXY_LOG_DEBUG("Subscribe on resource \"" << resourcePath << "\""); NKikimrKesus::TEvSubscribeOnResources req; - ActorIdToProto(SelfId(), req.MutableActorID()); + ActorIdToProto(SelfId(), req.MutableActorID()); auto* res = req.AddResources(); res->SetResourcePath(resourcePath); NTabletPipe::SendData(SelfId(), KesusPipeClient, new TEvKesus::TEvSubscribeOnResources(std::move(req)), NewCookieForRequest(resourcePath)); @@ -411,7 +411,7 @@ private: std::vector<TString> resourcePaths; resourcePaths.reserve(Resources.size()); NKikimrKesus::TEvSubscribeOnResources req; - ActorIdToProto(SelfId(), req.MutableActorID()); + ActorIdToProto(SelfId(), req.MutableActorID()); for (auto&& [resourcePath, resInfo] : Resources) { auto* res = req.AddResources(); res->SetResourcePath(resourcePath); @@ -434,7 +434,7 @@ private: void InitUpdateEv() { if (!UpdateEv) { UpdateEv = MakeHolder<NKesus::TEvKesus::TEvUpdateConsumptionState>(); - ActorIdToProto(SelfId(), UpdateEv->Record.MutableActorID()); + ActorIdToProto(SelfId(), UpdateEv->Record.MutableActorID()); } } @@ -860,7 +860,7 @@ public: return NKikimrServices::TActivity::QUOTER_PROXY_ACTOR; } - TKesusQuoterProxy(ui64 quoterId, const NSchemeCache::TSchemeCacheNavigate::TEntry& navEntry, const TActorId& quoterServiceId, THolder<ITabletPipeFactory> tabletPipeFactory) + TKesusQuoterProxy(ui64 quoterId, const NSchemeCache::TSchemeCacheNavigate::TEntry& navEntry, const TActorId& quoterServiceId, THolder<ITabletPipeFactory> tabletPipeFactory) : QuoterServiceId(quoterServiceId) , QuoterId(quoterId) , Path(navEntry.Path) @@ -961,7 +961,7 @@ public: }; struct TDefaultTabletPipeFactory : public ITabletPipeFactory { - IActor* CreateTabletPipe(const NActors::TActorId& owner, ui64 tabletId, const NKikimr::NTabletPipe::TClientConfig& config) override { + IActor* CreateTabletPipe(const NActors::TActorId& owner, ui64 tabletId, const NKikimr::NTabletPipe::TClientConfig& config) override { return NTabletPipe::CreateClient(owner, tabletId, config); } }; @@ -970,7 +970,7 @@ THolder<ITabletPipeFactory> ITabletPipeFactory::GetDefaultFactory() { return MakeHolder<TDefaultTabletPipeFactory>(); } -IActor* CreateKesusQuoterProxy(ui64 quoterId, const NSchemeCache::TSchemeCacheNavigate::TEntry& navEntry, const TActorId& quoterServiceId, THolder<ITabletPipeFactory> tabletPipeFactory) { +IActor* CreateKesusQuoterProxy(ui64 quoterId, const NSchemeCache::TSchemeCacheNavigate::TEntry& navEntry, const TActorId& quoterServiceId, THolder<ITabletPipeFactory> tabletPipeFactory) { return new TKesusQuoterProxy(quoterId, navEntry, quoterServiceId, std::move(tabletPipeFactory)); } diff --git a/ydb/core/quoter/kesus_quoter_proxy.h b/ydb/core/quoter/kesus_quoter_proxy.h index 6c29cdb5f66..ed79edfc9fd 100644 --- a/ydb/core/quoter/kesus_quoter_proxy.h +++ b/ydb/core/quoter/kesus_quoter_proxy.h @@ -15,14 +15,14 @@ namespace NKikimr { namespace NQuoter { struct ITabletPipeFactory { - virtual NActors::IActor* CreateTabletPipe(const NActors::TActorId& owner, ui64 tabletId, const NKikimr::NTabletPipe::TClientConfig& config = NKikimr::NTabletPipe::TClientConfig()) = 0; + virtual NActors::IActor* CreateTabletPipe(const NActors::TActorId& owner, ui64 tabletId, const NKikimr::NTabletPipe::TClientConfig& config = NKikimr::NTabletPipe::TClientConfig()) = 0; virtual ~ITabletPipeFactory() = default; static THolder<ITabletPipeFactory> GetDefaultFactory(); }; -NActors::IActor* CreateKesusQuoterProxy(ui64 quoterId, const NSchemeCache::TSchemeCacheNavigate::TEntry& navEntry, const NActors::TActorId& quoterServiceId, THolder<ITabletPipeFactory> tabletPipeFactory = ITabletPipeFactory::GetDefaultFactory()); +NActors::IActor* CreateKesusQuoterProxy(ui64 quoterId, const NSchemeCache::TSchemeCacheNavigate::TEntry& navEntry, const NActors::TActorId& quoterServiceId, THolder<ITabletPipeFactory> tabletPipeFactory = ITabletPipeFactory::GetDefaultFactory()); class TKesusResourceAllocationStatistics { public: diff --git a/ydb/core/quoter/quoter_service.cpp b/ydb/core/quoter/quoter_service.cpp index e5f4f979fb7..b9a1b629f51 100644 --- a/ydb/core/quoter/quoter_service.cpp +++ b/ydb/core/quoter/quoter_service.cpp @@ -60,14 +60,14 @@ ui32 TReqState::Idx(TRequest &request) { return idx; } -ui32 TReqState::HeadByOwner(TActorId ownerId) { +ui32 TReqState::HeadByOwner(TActorId ownerId) { if (ui32 *x = ByOwner.FindPtr(ownerId)) return *x; else return Max<ui32>(); } -ui32 TReqState::Allocate(TActorId source, ui64 eventCookie) { +ui32 TReqState::Allocate(TActorId source, ui64 eventCookie) { ui32 idx; if (Unused) { idx = Unused.back(); @@ -136,7 +136,7 @@ void TReqState::Free(ui32 idx) { x.PrevDeadlineRequest = Max<ui32>(); x.NextDeadlineRequest = Max<ui32>(); - x.Source = TActorId(); + x.Source = TActorId(); x.Orbit.Reset(); Unused.push_back(idx); @@ -798,7 +798,7 @@ void TQuoterService::InitialRequestProcessing(TEvQuota::TEvRequest::TPtr &ev, co auto deadlineIt = ScheduleDeadline.find(request.Deadline); if (deadlineIt == ScheduleDeadline.end()) { TInstant deadline = request.Deadline; // allocate could invalidate request& - deadlineIt = ScheduleDeadline.emplace(deadline, ReqState.Allocate(TActorId(0, "placeholder"), 0)).first; + deadlineIt = ScheduleDeadline.emplace(deadline, ReqState.Allocate(TActorId(0, "placeholder"), 0)).first; } const ui32 placeholderIdx = deadlineIt->second; @@ -1108,7 +1108,7 @@ void TQuoterService::BreakQuoter(decltype(QuotersIndex)::iterator indexIt, declt if (quoter.ProxyId) { Send(quoter.ProxyId, new TEvents::TEvPoisonPill()); - quoter.ProxyId = TActorId(); + quoter.ProxyId = TActorId(); } TSet<ui32> waitingQueueResolve(std::move(quoter.WaitingQueueResolve)); diff --git a/ydb/core/quoter/quoter_service_bandwidth_test/quota_requester.cpp b/ydb/core/quoter/quoter_service_bandwidth_test/quota_requester.cpp index 24ffd8c6f3e..1a48d8211a2 100644 --- a/ydb/core/quoter/quoter_service_bandwidth_test/quota_requester.cpp +++ b/ydb/core/quoter/quoter_service_bandwidth_test/quota_requester.cpp @@ -20,7 +20,7 @@ TDuration TRequestDistribution::GetNextRequestTimeDelta() { return TDuration::MicroSeconds(static_cast<ui64>(LambdaCoefficient * std::log(rand))); } -TBaseQuotaRequester::TBaseQuotaRequester(const TOptions& opts, TRequestStats& stats, TActorId parent) +TBaseQuotaRequester::TBaseQuotaRequester(const TOptions& opts, TRequestStats& stats, TActorId parent) : Opts(opts) , Stats(stats) , Parent(parent) @@ -88,7 +88,7 @@ void TBaseQuotaRequester::SleepUntilNextRequest(TDuration duration) { Schedule(duration, new TEvents::TEvWakeup()); } -TKesusQuotaRequester::TKesusQuotaRequester(const NKikimr::TOptions& opts, NKikimr::TRequestStats& stats, TActorId parent, size_t kesusIndex, size_t resourceIndex) +TKesusQuotaRequester::TKesusQuotaRequester(const NKikimr::TOptions& opts, NKikimr::TRequestStats& stats, TActorId parent, size_t kesusIndex, size_t resourceIndex) : TBaseQuotaRequester(opts, stats, parent) , KesusPath(TTestServer::GetKesusPath(kesusIndex)) , ResourcePath(TTestServer::GetKesusResource(resourceIndex)) @@ -102,7 +102,7 @@ THolder<TEvQuota::TEvRequest> TKesusQuotaRequester::MakeQuoterRequest() { return MakeHolder<TEvQuota::TEvRequest>(TEvQuota::EResourceOperator::And, std::move(reqs), Opts.QuotaRequestDeadline); } -TLocalResourceQuotaRequester::TLocalResourceQuotaRequester(const NKikimr::TOptions& opts, NKikimr::TRequestStats& stats, TActorId parent, size_t resourceIndex) +TLocalResourceQuotaRequester::TLocalResourceQuotaRequester(const NKikimr::TOptions& opts, NKikimr::TRequestStats& stats, TActorId parent, size_t resourceIndex) : TBaseQuotaRequester(opts, stats, parent) , ResourceId(TEvQuota::TResourceLeaf::MakeTaggedRateRes(resourceIndex, Opts.LocalResourceQuotaRate)) { diff --git a/ydb/core/quoter/quoter_service_bandwidth_test/quota_requester.h b/ydb/core/quoter/quoter_service_bandwidth_test/quota_requester.h index 91dc1467664..6f21fa652ad 100644 --- a/ydb/core/quoter/quoter_service_bandwidth_test/quota_requester.h +++ b/ydb/core/quoter/quoter_service_bandwidth_test/quota_requester.h @@ -24,7 +24,7 @@ private: class TBaseQuotaRequester : public NActors::TActorBootstrapped<TBaseQuotaRequester> { public: - TBaseQuotaRequester(const TOptions& opts, TRequestStats& stats, TActorId parent); + TBaseQuotaRequester(const TOptions& opts, TRequestStats& stats, TActorId parent); void Bootstrap(const NActors::TActorContext& ctx); @@ -49,7 +49,7 @@ protected: protected: const TOptions& Opts; TRequestStats& Stats; - const TActorId Parent; + const TActorId Parent; TInstant StartTime; TRequestDistribution Distribution; @@ -60,7 +60,7 @@ protected: class TKesusQuotaRequester : public TBaseQuotaRequester { public: - TKesusQuotaRequester(const TOptions& opts, TRequestStats& stats, TActorId parent, size_t kesusIndex, size_t resourceIndex); + TKesusQuotaRequester(const TOptions& opts, TRequestStats& stats, TActorId parent, size_t kesusIndex, size_t resourceIndex); THolder<TEvQuota::TEvRequest> MakeQuoterRequest() override; @@ -71,7 +71,7 @@ private: class TLocalResourceQuotaRequester : public TBaseQuotaRequester { public: - TLocalResourceQuotaRequester(const TOptions& opts, TRequestStats& stats, TActorId parent, size_t resourceIndex); + TLocalResourceQuotaRequester(const TOptions& opts, TRequestStats& stats, TActorId parent, size_t resourceIndex); THolder<TEvQuota::TEvRequest> MakeQuoterRequest() override; diff --git a/ydb/core/quoter/quoter_service_bandwidth_test/server.cpp b/ydb/core/quoter/quoter_service_bandwidth_test/server.cpp index 48ccd2572f3..4b301ea3edf 100644 --- a/ydb/core/quoter/quoter_service_bandwidth_test/server.cpp +++ b/ydb/core/quoter/quoter_service_bandwidth_test/server.cpp @@ -77,7 +77,7 @@ void TTestServer::RegisterQuoterService() { const ui32 systemPoolId = runtime->GetAppData().SystemPoolId; //Cerr << "System pool: " << systemPoolId << Endl; const ui32 nodeIndex = 0; - const TActorId quoterServiceActorId = runtime->Register(CreateQuoterService(), nodeIndex, systemPoolId); + const TActorId quoterServiceActorId = runtime->Register(CreateQuoterService(), nodeIndex, systemPoolId); runtime->RegisterService(MakeQuoterServiceID(), quoterServiceActorId); } @@ -116,7 +116,7 @@ void TTestServer::CreateKesusResource(ui64 kesusTabletId, const TString& resourc hdrrConfig->SetMaxUnitsPerSecond(*maxUnitsPerSecond); } - TActorId sender = GetEdgeActor(); + TActorId sender = GetEdgeActor(); ForwardToTablet(*runtime, kesusTabletId, sender, request.Release(), 0); TAutoPtr<IEventHandle> handle; @@ -129,7 +129,7 @@ void TTestServer::CreateKesusResource(const TString& kesusPath, const TString& r CreateKesusResource(GetKesusTabletId(kesusPath), resourcePath, maxUnitsPerSecond); } -TActorId TTestServer::GetEdgeActor() { +TActorId TTestServer::GetEdgeActor() { if (!EdgeActor) { EdgeActor = Server->GetRuntime()->AllocateEdgeActor(0); } diff --git a/ydb/core/quoter/quoter_service_bandwidth_test/server.h b/ydb/core/quoter/quoter_service_bandwidth_test/server.h index 75f7b71e850..59522eb5bfa 100644 --- a/ydb/core/quoter/quoter_service_bandwidth_test/server.h +++ b/ydb/core/quoter/quoter_service_bandwidth_test/server.h @@ -37,7 +37,7 @@ private: void CreateKesusResource(const TString& kesusPath, const TString& resourcePath, TMaybe<double> maxUnitsPerSecond = Nothing()); void CreateKesusResource(ui64 kesusTabletId, const TString& resourcePath, TMaybe<double> maxUnitsPerSecond = Nothing()); - TActorId GetEdgeActor(); + TActorId GetEdgeActor(); void SetupSettings(); @@ -49,7 +49,7 @@ private: Tests::TServer::TPtr Server; THolder <Tests::TClient> Client; - TActorId EdgeActor; + TActorId EdgeActor; }; } // namespace NKikimr diff --git a/ydb/core/quoter/quoter_service_impl.h b/ydb/core/quoter/quoter_service_impl.h index 95789ed7304..d27ea134e19 100644 --- a/ydb/core/quoter/quoter_service_impl.h +++ b/ydb/core/quoter/quoter_service_impl.h @@ -44,7 +44,7 @@ struct TResource; NMonitoring::IHistogramCollectorPtr GetLatencyHistogramBuckets(); struct TRequest { - TActorId Source = TActorId(); + TActorId Source = TActorId(); ui64 EventCookie = 0; TInstant StartTime; @@ -65,13 +65,13 @@ struct TRequest { class TReqState { private: TVector<TRequest> Requests; - THashMap<TActorId, ui32> ByOwner; + THashMap<TActorId, ui32> ByOwner; TVector<ui32> Unused; public: ui32 Idx(TRequest &request); TRequest& Get(ui32 idx); - ui32 HeadByOwner(TActorId ownerId); - ui32 Allocate(TActorId source, ui64 eventCookie); + ui32 HeadByOwner(TActorId ownerId); + ui32 Allocate(TActorId source, ui64 eventCookie); void Free(ui32 idx); }; @@ -190,7 +190,7 @@ struct TScheduleTick { struct TQuoterState { const TString QuoterName; - TActorId ProxyId; + TActorId ProxyId; THashMap<ui64, THolder<TResource>> Resources; THashMap<TString, ui64> ResourcesIndex; diff --git a/ydb/core/quoter/quoter_service_ut.cpp b/ydb/core/quoter/quoter_service_ut.cpp index 312fe890108..88c74ca3fb1 100644 --- a/ydb/core/quoter/quoter_service_ut.cpp +++ b/ydb/core/quoter/quoter_service_ut.cpp @@ -20,11 +20,11 @@ Y_UNIT_TEST_SUITE(TQuoterServiceTest) { TServer server = TServer(serverSettings, true); TTestActorRuntime *runtime = server.GetRuntime(); - const TActorId serviceId = MakeQuoterServiceID(); - const TActorId serviceActorId = runtime->Register(CreateQuoterService()); + const TActorId serviceId = MakeQuoterServiceID(); + const TActorId serviceActorId = runtime->Register(CreateQuoterService()); runtime->RegisterService(serviceId, serviceActorId); - TActorId sender = runtime->AllocateEdgeActor(); + TActorId sender = runtime->AllocateEdgeActor(); { runtime->Send(new IEventHandle(MakeQuoterServiceID(), sender, new TEvQuota::TEvRequest(TEvQuota::EResourceOperator::And, { @@ -156,7 +156,7 @@ Y_UNIT_TEST_SUITE(TQuoterServiceTest) { UNIT_ASSERT(pathDesc.HasKesus()); const ui64 tabletId = pathDesc.GetKesus().GetKesusTabletId(); - TActorId sender = runtime->AllocateEdgeActor(); + TActorId sender = runtime->AllocateEdgeActor(); ForwardToTablet(*runtime, tabletId, sender, request.Release(), 0); TAutoPtr<IEventHandle> handle; @@ -173,11 +173,11 @@ Y_UNIT_TEST_SUITE(TQuoterServiceTest) { TTestActorRuntime* runtime = server.GetRuntime(); - const TActorId serviceId = MakeQuoterServiceID(); - const TActorId serviceActorId = runtime->Register(CreateQuoterService()); + const TActorId serviceId = MakeQuoterServiceID(); + const TActorId serviceActorId = runtime->Register(CreateQuoterService()); runtime->RegisterService(serviceId, serviceActorId); - const TActorId sender = runtime->AllocateEdgeActor(); + const TActorId sender = runtime->AllocateEdgeActor(); constexpr TDuration testDuration = TDuration::Seconds(2); constexpr TDuration waitDuration = TDuration::MilliSeconds(150); @@ -261,11 +261,11 @@ Y_UNIT_TEST_SUITE(TQuoterServiceTest) { TServer server = TServer(serverSettings, true); TTestActorRuntime *runtime = server.GetRuntime(); - const TActorId serviceId = MakeQuoterServiceID(); - const TActorId serviceActorId = runtime->Register(CreateQuoterService()); + const TActorId serviceId = MakeQuoterServiceID(); + const TActorId serviceActorId = runtime->Register(CreateQuoterService()); runtime->RegisterService(serviceId, serviceActorId); - TActorId sender = runtime->AllocateEdgeActor(); + TActorId sender = runtime->AllocateEdgeActor(); { runtime->Send(new IEventHandle(MakeQuoterServiceID(), sender, new TEvQuota::TEvRequest(TEvQuota::EResourceOperator::And, { @@ -304,11 +304,11 @@ Y_UNIT_TEST_SUITE(TQuoterServiceTest) { TServer server = TServer(serverSettings, true); TTestActorRuntime *runtime = server.GetRuntime(); - const TActorId serviceId = MakeQuoterServiceID(); - const TActorId serviceActorId = runtime->Register(CreateQuoterService()); + const TActorId serviceId = MakeQuoterServiceID(); + const TActorId serviceActorId = runtime->Register(CreateQuoterService()); runtime->RegisterService(serviceId, serviceActorId); - TActorId sender = runtime->AllocateEdgeActor(); + TActorId sender = runtime->AllocateEdgeActor(); { runtime->Send(new IEventHandle(MakeQuoterServiceID(), sender, new TEvQuota::TEvRequest(TEvQuota::EResourceOperator::And, { diff --git a/ydb/core/quoter/ut_helpers.cpp b/ydb/core/quoter/ut_helpers.cpp index 2b2fd923b54..e57a073d74a 100644 --- a/ydb/core/quoter/ut_helpers.cpp +++ b/ydb/core/quoter/ut_helpers.cpp @@ -45,7 +45,7 @@ void TKesusQuoterTestSetup::SetupLogging() { void TKesusQuoterTestSetup::RegisterQuoterService() { TTestActorRuntime* const runtime = GetServer().GetRuntime(); - const TActorId quoterServiceActorId = runtime->Register(CreateQuoterService()); + const TActorId quoterServiceActorId = runtime->Register(CreateQuoterService()); runtime->RegisterService(MakeQuoterServiceID(), quoterServiceActorId); } @@ -77,7 +77,7 @@ void TKesusQuoterTestSetup::CreateKesusResource(const TString& kesusPath, const request->Record.MutableResource()->SetResourcePath(resourcePath); *request->Record.MutableResource()->MutableHierarhicalDRRResourceConfig() = cfg; - TActorId sender = GetEdgeActor(); + TActorId sender = GetEdgeActor(); Cerr << "AddQuoterResource: " << request->Record << Endl; ForwardToTablet(*runtime, GetKesusTabletId(kesusPath), sender, request.Release(), 0); @@ -98,7 +98,7 @@ void TKesusQuoterTestSetup::DeleteKesusResource(const TString& kesusPath, const TAutoPtr<NKesus::TEvKesus::TEvDeleteQuoterResource> request(new NKesus::TEvKesus::TEvDeleteQuoterResource()); request->Record.SetResourcePath(resourcePath); - TActorId sender = GetEdgeActor(); + TActorId sender = GetEdgeActor(); Cerr << "DeleteQuoterResource: " << request->Record << Endl; ForwardToTablet(*runtime, GetKesusTabletId(kesusPath), sender, request.Release(), 0); @@ -108,7 +108,7 @@ void TKesusQuoterTestSetup::DeleteKesusResource(const TString& kesusPath, const UNIT_ASSERT_VALUES_EQUAL(record.GetError().GetStatus(), Ydb::StatusIds::SUCCESS); } -TActorId TKesusQuoterTestSetup::GetEdgeActor() { +TActorId TKesusQuoterTestSetup::GetEdgeActor() { if (!EdgeActor) { EdgeActor = GetServer().GetRuntime()->AllocateEdgeActor(0); } @@ -153,7 +153,7 @@ THolder<TEvQuota::TEvClearance> TKesusQuoterTestSetup::WaitGetQuotaAnswer() { void TKesusQuoterTestSetup::KillKesusTablet(const TString& kesusPath) { TTestActorRuntime* const runtime = Server->GetRuntime(); - TActorId sender = GetEdgeActor(); + TActorId sender = GetEdgeActor(); Cerr << "Kill kesus tablet: " << kesusPath << Endl; ForwardToTablet(*runtime, GetKesusTabletId(kesusPath), sender, new TEvents::TEvPoisonPill(), 0); } @@ -215,14 +215,14 @@ void TKesusProxyTestSetup::SetupLogging() { Runtime->SetLogPriority(NKikimrServices::QUOTER_PROXY, NActors::NLog::PRI_TRACE); } -TActorId TKesusProxyTestSetup::GetEdgeActor() { +TActorId TKesusProxyTestSetup::GetEdgeActor() { if (!EdgeActor) { EdgeActor = Runtime->AllocateEdgeActor(0); } return EdgeActor; } -TActorId TKesusProxyTestSetup::GetPipeEdgeActor() { +TActorId TKesusProxyTestSetup::GetPipeEdgeActor() { if (!PipeEdgeActor) { PipeEdgeActor = Runtime->AllocateEdgeActor(0); } @@ -237,7 +237,7 @@ void TKesusProxyTestSetup::WaitProxyStart() { void TKesusProxyTestSetup::SendNotConnected(TTestTabletPipeFactory::TTestTabletPipe* pipe) { WaitProxyStart(); - Runtime->Send(new IEventHandle(KesusProxyId, pipe->GetSelfID(), new TEvTabletPipe::TEvClientConnected(KESUS_TABLET_ID, NKikimrProto::ERROR, pipe->GetSelfID(), TActorId(), true, false)), 0, true); + Runtime->Send(new IEventHandle(KesusProxyId, pipe->GetSelfID(), new TEvTabletPipe::TEvClientConnected(KESUS_TABLET_ID, NKikimrProto::ERROR, pipe->GetSelfID(), TActorId(), true, false)), 0, true); } void TKesusProxyTestSetup::SendConnected(TTestTabletPipeFactory::TTestTabletPipe* pipe) { @@ -395,7 +395,7 @@ TKesusProxyTestSetup::TTestTabletPipeFactory::~TTestTabletPipeFactory() { PipesExpectedToCreate.clear(); } -IActor* TKesusProxyTestSetup::TTestTabletPipeFactory::CreateTabletPipe(const NActors::TActorId& owner, ui64 tabletId, const NKikimr::NTabletPipe::TClientConfig&) { +IActor* TKesusProxyTestSetup::TTestTabletPipeFactory::CreateTabletPipe(const NActors::TActorId& owner, ui64 tabletId, const NKikimr::NTabletPipe::TClientConfig&) { UNIT_ASSERT(owner); UNIT_ASSERT_VALUES_EQUAL(owner, Parent->KesusProxyId); UNIT_ASSERT(tabletId); @@ -463,7 +463,7 @@ void TKesusProxyTestSetup::TTestTabletPipeFactory::TTestTabletPipe::HandleSend(T } void TKesusProxyTestSetup::TTestTabletPipeFactory::TTestTabletPipe::HandleSubscribeOnResources(NKesus::TEvKesus::TEvSubscribeOnResources::TPtr& ev) { - const TActorId proxy = ActorIdFromProto(ev->Get()->Record.GetActorID()); + const TActorId proxy = ActorIdFromProto(ev->Get()->Record.GetActorID()); UNIT_ASSERT_VALUES_EQUAL(proxy, Parent->Parent->KesusProxyId); OnSubscribeOnResources(ev->Get()->Record, ev->Cookie); @@ -478,7 +478,7 @@ void TKesusProxyTestSetup::TTestTabletPipeFactory::TTestTabletPipe::HandleResour } void TKesusProxyTestSetup::TTestTabletPipeFactory::TTestTabletPipe::SendNotConnected() { - Send(Parent->Parent->KesusProxyId, new TEvTabletPipe::TEvClientConnected(KESUS_TABLET_ID, NKikimrProto::ERROR, SelfID, TActorId(), true, false)); + Send(Parent->Parent->KesusProxyId, new TEvTabletPipe::TEvClientConnected(KESUS_TABLET_ID, NKikimrProto::ERROR, SelfID, TActorId(), true, false)); } void TKesusProxyTestSetup::TTestTabletPipeFactory::TTestTabletPipe::SendConnected() { diff --git a/ydb/core/quoter/ut_helpers.h b/ydb/core/quoter/ut_helpers.h index 842e53e8ef7..ea76fdc0ae1 100644 --- a/ydb/core/quoter/ut_helpers.h +++ b/ydb/core/quoter/ut_helpers.h @@ -70,7 +70,7 @@ public: THolder<TEvQuota::TEvClearance> WaitGetQuotaAnswer(); - TActorId GetEdgeActor(); + TActorId GetEdgeActor(); private: void SetupLogging(); @@ -84,7 +84,7 @@ private: Tests::TServerSettings::TPtr ServerSettings; Tests::TServer::TPtr Server; THolder<Tests::TClient> Client; - TActorId EdgeActor; + TActorId EdgeActor; }; class TKesusProxyTestSetup { @@ -104,7 +104,7 @@ public: ~TTestTabletPipeFactory(); - IActor* CreateTabletPipe(const NActors::TActorId& owner, ui64 tabletId, const NKikimr::NTabletPipe::TClientConfig& config) override; + IActor* CreateTabletPipe(const NActors::TActorId& owner, ui64 tabletId, const NKikimr::NTabletPipe::TClientConfig& config) override; TTestTabletPipe* ExpectTabletPipeCreation(bool wait = false); // Set expectation to creation of a new pipe. TTestTabletPipe* ExpectTabletPipeConnection(); // Set expectation of creation and connecting to new pipe. If no expectation for a pipe is set this is set by default. @@ -136,7 +136,7 @@ public: THolder<IEventHandle> GetDestroyedEventHandle(); - const TActorId& GetSelfID() const { + const TActorId& GetSelfID() const { return SelfID; } @@ -155,7 +155,7 @@ public: private: TTestTabletPipeFactory* Parent; - TActorId SelfID; + TActorId SelfID; bool IsDead = false; }; @@ -177,14 +177,14 @@ public: return *Runtime; } - const TActorId& GetKesusProxyId() const { + const TActorId& GetKesusProxyId() const { return KesusProxyId; } void WaitProxyStart(); - TActorId GetEdgeActor(); - TActorId GetPipeEdgeActor(); + TActorId GetEdgeActor(); + TActorId GetPipeEdgeActor(); void SendProxyRequest(const TString& resourceName); THolder<TEventHandle<TEvQuota::TEvProxySession>> ProxyRequest(const TString& resourceName, TEvQuota::TEvProxySession::EResult = TEvQuota::TEvProxySession::Success); @@ -236,10 +236,10 @@ private: private: THolder<TTestActorRuntime> Runtime; - TActorId KesusProxyId; + TActorId KesusProxyId; TTestTabletPipeFactory* PipeFactory = nullptr; - TActorId EdgeActor; - TActorId PipeEdgeActor; + TActorId EdgeActor; + TActorId PipeEdgeActor; }; } // namespace NKikimr diff --git a/ydb/core/scheme/defs.h b/ydb/core/scheme/defs.h index 45944bb5d10..c42f4b9f603 100644 --- a/ydb/core/scheme/defs.h +++ b/ydb/core/scheme/defs.h @@ -1,12 +1,12 @@ -#pragma once - +#pragma once + #include <ydb/core/base/defs.h> - - -namespace NKikimr { -namespace NScheme { - -class TTypeRegistry; - -} // namspace NScheme -} // namspace NKikimr + + +namespace NKikimr { +namespace NScheme { + +class TTypeRegistry; + +} // namspace NScheme +} // namspace NKikimr diff --git a/ydb/core/scheme/scheme_tablecell.cpp b/ydb/core/scheme/scheme_tablecell.cpp index bbe89097d8c..c9ffaccc8e9 100644 --- a/ydb/core/scheme/scheme_tablecell.cpp +++ b/ydb/core/scheme/scheme_tablecell.cpp @@ -104,7 +104,7 @@ void DbgPrintValue(TString &res, const TCell &r, ui32 type) { res += ToString(r.AsValue<double>()); break; case NScheme::NTypeIds::ActorId: - res += ToString(r.AsValue<TActorId>()); + res += ToString(r.AsValue<TActorId>()); break; default: res += EscapeC(r.Data(), r.Size()); diff --git a/ydb/core/scheme/scheme_tablecell.h b/ydb/core/scheme/scheme_tablecell.h index 115198a7833..bccfdca97db 100644 --- a/ydb/core/scheme/scheme_tablecell.h +++ b/ydb/core/scheme/scheme_tablecell.h @@ -1,9 +1,9 @@ #pragma once - + #include "defs.h" -#include "scheme_type_id.h" +#include "scheme_type_id.h" #include "scheme_type_order.h" -#include "scheme_types_defs.h" +#include "scheme_types_defs.h" #include <util/generic/hash.h> #include <util/system/unaligned_mem.h> @@ -129,7 +129,7 @@ inline int CompareTypedCells(const TCell& a, const TCell& b, NScheme::TTypeIdOrd switch (type.GetTypeId()) { #define SIMPLE_TYPE_SWITCH(typeEnum, castType) \ - case NKikimr::NScheme::NTypeIds::typeEnum: \ + case NKikimr::NScheme::NTypeIds::typeEnum: \ { \ Y_VERIFY_DEBUG(a.IsInline()); \ Y_VERIFY_DEBUG(b.IsInline()); \ diff --git a/ydb/core/scheme/scheme_tabledefs.h b/ydb/core/scheme/scheme_tabledefs.h index 8fdbb4bc1e2..190233fdeee 100644 --- a/ydb/core/scheme/scheme_tabledefs.h +++ b/ydb/core/scheme/scheme_tabledefs.h @@ -2,12 +2,12 @@ #include "defs.h" #include "scheme_tablecell.h" - + #include <ydb/core/base/pathid.h> #include <ydb/core/protos/tx.pb.h> #include <ydb/core/scheme_types/scheme_types.h> #include <ydb/library/aclib/aclib.h> - + #include <library/cpp/deprecated/enum_codegen/enum_codegen.h> #include <util/generic/maybe.h> diff --git a/ydb/core/scheme/scheme_type_id.h b/ydb/core/scheme/scheme_type_id.h index 881e55cd54a..8aabe61bdc7 100644 --- a/ydb/core/scheme/scheme_type_id.h +++ b/ydb/core/scheme/scheme_type_id.h @@ -1,15 +1,15 @@ -#pragma once - +#pragma once + #include <ydb/public/lib/scheme_types/scheme_type_id.h> - - -namespace NKikimr { -namespace NScheme { -namespace NTypeIds { - -static constexpr TTypeId ActorId = 0x2001; -static constexpr TTypeId StepOrderId = 0x2002; - -} // namspace NTypeIds -} // namspace NScheme -} // namspace NKikimr + + +namespace NKikimr { +namespace NScheme { +namespace NTypeIds { + +static constexpr TTypeId ActorId = 0x2001; +static constexpr TTypeId StepOrderId = 0x2002; + +} // namspace NTypeIds +} // namspace NScheme +} // namspace NKikimr diff --git a/ydb/core/scheme/scheme_type_registry.h b/ydb/core/scheme/scheme_type_registry.h index c407acf45c4..c2eec5d1e64 100644 --- a/ydb/core/scheme/scheme_type_registry.h +++ b/ydb/core/scheme/scheme_type_registry.h @@ -1,21 +1,21 @@ -#pragma once - -#include "scheme_types_defs.h" - +#pragma once + +#include "scheme_types_defs.h" + #include <ydb/core/scheme_types/scheme_type_registry.h> - - -namespace NKikimr { -namespace NScheme { - -class TKikimrTypeRegistry: public TTypeRegistry -{ -public: - TKikimrTypeRegistry() { - RegisterType<TActorId>(); - RegisterType<TStepOrderId>(); - } -}; - -} // namespace NScheme -} // namespace NKikimr + + +namespace NKikimr { +namespace NScheme { + +class TKikimrTypeRegistry: public TTypeRegistry +{ +public: + TKikimrTypeRegistry() { + RegisterType<TActorId>(); + RegisterType<TStepOrderId>(); + } +}; + +} // namespace NScheme +} // namespace NKikimr diff --git a/ydb/core/scheme/scheme_types_defs.cpp b/ydb/core/scheme/scheme_types_defs.cpp index 859c65cc656..d27dc6aa10b 100644 --- a/ydb/core/scheme/scheme_types_defs.cpp +++ b/ydb/core/scheme/scheme_types_defs.cpp @@ -1,13 +1,13 @@ -#include "scheme_types_defs.h" - - -namespace NKikimr { -namespace NScheme { -namespace NNames { - - DECLARE_TYPED_TYPE_NAME(ActorID); - DECLARE_TYPED_TYPE_NAME(StepOrderId); - -} // namespace NNames -} // namespace NKikimr -} // namespace NScheme +#include "scheme_types_defs.h" + + +namespace NKikimr { +namespace NScheme { +namespace NNames { + + DECLARE_TYPED_TYPE_NAME(ActorID); + DECLARE_TYPED_TYPE_NAME(StepOrderId); + +} // namespace NNames +} // namespace NKikimr +} // namespace NScheme diff --git a/ydb/core/scheme/scheme_types_defs.h b/ydb/core/scheme/scheme_types_defs.h index 750cbd02eaa..32a3a2b8726 100644 --- a/ydb/core/scheme/scheme_types_defs.h +++ b/ydb/core/scheme/scheme_types_defs.h @@ -1,64 +1,64 @@ -#pragma once - -#include "scheme_type_id.h" - +#pragma once + +#include "scheme_type_id.h" + #include <library/cpp/actors/core/actorid.h> #include <ydb/core/scheme_types/scheme_types_defs.h> - -#include <util/stream/output.h> - - -#define KIKIMR_FOREACH_TYPE(xx, ...) \ + +#include <util/stream/output.h> + + +#define KIKIMR_FOREACH_TYPE(xx, ...) \ KIKIMR_FOREACH_MINIKQL_TYPE(xx, __VA_ARGS__) \ - xx(ActorId, TActorId, __VA_ARGS__) \ - xx(StepOrderId, TStepOrderId, __VA_ARGS__) \ - /**/ - -namespace NKikimr { -namespace NScheme { - -//////////////////////////////////////////////////////// -/// other internal types -/// 0x2001 - 0x8000 -/// DO NOT FORGET TO REGISTER THE TYPES in Library::OpenLibrary() / file tablet_library.h -namespace NNames { + xx(ActorId, TActorId, __VA_ARGS__) \ + xx(StepOrderId, TStepOrderId, __VA_ARGS__) \ + /**/ + +namespace NKikimr { +namespace NScheme { + +//////////////////////////////////////////////////////// +/// other internal types +/// 0x2001 - 0x8000 +/// DO NOT FORGET TO REGISTER THE TYPES in Library::OpenLibrary() / file tablet_library.h +namespace NNames { extern const char ActorID[8]; -} - -class TActorId : public TTypedType<NActors::TActorId, TActorId, NTypeIds::ActorId, NNames::ActorID> -{ -public: -}; - -namespace NNames { +} + +class TActorId : public TTypedType<NActors::TActorId, TActorId, NTypeIds::ActorId, NNames::ActorID> +{ +public: +}; + +namespace NNames { extern const char StepOrderId[12]; -} - -class TStepOrderId : public IIntegerPair<ui64, ui64, NTypeIds::StepOrderId, NNames::StepOrderId> {}; - -//////////////////////////////////////////////////////// -/// user types -/// 0x8001 - 0xFFFF -/// DO NOT FORGET TO REGISTER THE TYPES in Library::OpenLibrary() / file tablet_library.h - -// todo: range enum - -//////////////////////////////////////////////////////// -/// 0x10000 - 0xFFFFFFFF reserved -/// DO NOT FORGET TO REGISTER THE TYPES in Library::OpenLibrary() / file tablet_library.h - - -//////////////////////////////////////////////////////// - -inline ui32 GetFixedSize(NKikimr::NScheme::TTypeId typeId) { - switch (typeId) { -#define KIKIMR_TYPE_MACRO(typeEnum, typeType, ...) case NTypeIds::typeEnum: return typeType::GetFixedSize(); - KIKIMR_FOREACH_TYPE(KIKIMR_TYPE_MACRO) -#undef KIKIMR_TYPE_MACRO - default: - return 0; - } -} - -} // namespace NScheme -} // namespace NKikimr +} + +class TStepOrderId : public IIntegerPair<ui64, ui64, NTypeIds::StepOrderId, NNames::StepOrderId> {}; + +//////////////////////////////////////////////////////// +/// user types +/// 0x8001 - 0xFFFF +/// DO NOT FORGET TO REGISTER THE TYPES in Library::OpenLibrary() / file tablet_library.h + +// todo: range enum + +//////////////////////////////////////////////////////// +/// 0x10000 - 0xFFFFFFFF reserved +/// DO NOT FORGET TO REGISTER THE TYPES in Library::OpenLibrary() / file tablet_library.h + + +//////////////////////////////////////////////////////// + +inline ui32 GetFixedSize(NKikimr::NScheme::TTypeId typeId) { + switch (typeId) { +#define KIKIMR_TYPE_MACRO(typeEnum, typeType, ...) case NTypeIds::typeEnum: return typeType::GetFixedSize(); + KIKIMR_FOREACH_TYPE(KIKIMR_TYPE_MACRO) +#undef KIKIMR_TYPE_MACRO + default: + return 0; + } +} + +} // namespace NScheme +} // namespace NKikimr diff --git a/ydb/core/scheme/ya.make b/ydb/core/scheme/ya.make index 6a3cbef14b5..128ef015bfe 100644 --- a/ydb/core/scheme/ya.make +++ b/ydb/core/scheme/ya.make @@ -1,19 +1,19 @@ -LIBRARY() - -OWNER( - ddoarn - vvvv +LIBRARY() + +OWNER( + ddoarn + vvvv g:kikimr -) - -SRCS( +) + +SRCS( scheme_borders.cpp scheme_tablecell.cpp scheme_tabledefs.cpp - scheme_types_defs.cpp -) - -PEERDIR( + scheme_types_defs.cpp +) + +PEERDIR( library/cpp/actors/core library/cpp/charset library/cpp/containers/bitseq @@ -24,9 +24,9 @@ PEERDIR( ydb/core/util ydb/library/aclib ydb/public/lib/scheme_types -) - -END() +) + +END() RECURSE_FOR_TESTS( ut diff --git a/ydb/core/scheme_types/scheme_raw_type_value.h b/ydb/core/scheme_types/scheme_raw_type_value.h index 1375a4623d1..1cb0946b6d7 100644 --- a/ydb/core/scheme_types/scheme_raw_type_value.h +++ b/ydb/core/scheme_types/scheme_raw_type_value.h @@ -1,70 +1,70 @@ -#pragma once - +#pragma once + #include <ydb/public/lib/scheme_types/scheme_type_id.h> - + #include <util/generic/array_ref.h> -#include <util/string/builder.h> - -namespace NKikimr { - -class TRawTypeValue -{ -public: - // - TRawTypeValue() - : Buffer(nullptr) - , BufferSize(0) - , ValueType(0) - {} - +#include <util/string/builder.h> + +namespace NKikimr { + +class TRawTypeValue +{ +public: + // + TRawTypeValue() + : Buffer(nullptr) + , BufferSize(0) + , ValueType(0) + {} + TRawTypeValue(TArrayRef<const char> ref, NScheme::TTypeId vtype) : TRawTypeValue((void*)ref.data(), ref.size(), vtype) { } - TRawTypeValue(const void* buf, ui32 bufSize, NScheme::TTypeId vtype) - : Buffer(buf) - , BufferSize(bufSize) - , ValueType(vtype) - { + TRawTypeValue(const void* buf, ui32 bufSize, NScheme::TTypeId vtype) + : Buffer(buf) + , BufferSize(bufSize) + , ValueType(vtype) + { Y_VERIFY_DEBUG(!buf || vtype); - } - - const void* Data() const { return Buffer; } - ui32 Size() const { return BufferSize; } - NScheme::TTypeId Type() const { return ValueType; } - - // we must distinguish empty raw type value (nothing, buffer == nullptr) - // and zero-length string (value exists, but zero-length) - bool IsEmpty() const { return Buffer == nullptr; } + } + + const void* Data() const { return Buffer; } + ui32 Size() const { return BufferSize; } + NScheme::TTypeId Type() const { return ValueType; } + + // we must distinguish empty raw type value (nothing, buffer == nullptr) + // and zero-length string (value exists, but zero-length) + bool IsEmpty() const { return Buffer == nullptr; } explicit operator bool() const noexcept { return !IsEmpty(); } - + TString ToString() const { - TStringBuilder builder; + TStringBuilder builder; builder << "(type:" << ValueType; - if (!IsEmpty()) { + if (!IsEmpty()) { builder << ", value:" << TString((const char*)Buffer, BufferSize).Quote(); - } - builder << ")"; + } + builder << ")"; return std::move(builder); - } - - TStringBuf ToStringBuf() const { - return TStringBuf((const char*)Buffer, BufferSize); - } - + } + + TStringBuf ToStringBuf() const { + return TStringBuf((const char*)Buffer, BufferSize); + } + TArrayRef<const char> AsRef() const noexcept { return { static_cast<const char*>(Data()), Size() }; } -private: - const void* Buffer; - ui32 BufferSize; - NScheme::TTypeId ValueType; -}; - -} // namspace NKikimr +private: + const void* Buffer; + ui32 BufferSize; + NScheme::TTypeId ValueType; +}; + +} // namspace NKikimr inline IOutputStream& operator << (IOutputStream& out, const NKikimr::TRawTypeValue& v) { out << v.ToString(); diff --git a/ydb/core/scheme_types/scheme_type_metadata.cpp b/ydb/core/scheme_types/scheme_type_metadata.cpp index 8ceb5419065..d26b449d144 100644 --- a/ydb/core/scheme_types/scheme_type_metadata.cpp +++ b/ydb/core/scheme_types/scheme_type_metadata.cpp @@ -1,8 +1,8 @@ #include "scheme_type_metadata.h" - + namespace NKikimr { namespace NScheme { -} // namespace NScheme -} // namespace NKikimr +} // namespace NScheme +} // namespace NKikimr diff --git a/ydb/core/scheme_types/scheme_type_metadata.h b/ydb/core/scheme_types/scheme_type_metadata.h index f7fad52f47e..9c55fcc0a27 100644 --- a/ydb/core/scheme_types/scheme_type_metadata.h +++ b/ydb/core/scheme_types/scheme_type_metadata.h @@ -1,17 +1,17 @@ #pragma once -#include "scheme_raw_type_value.h" +#include "scheme_raw_type_value.h" #include "scheme_types.h" -#include <util/generic/hash.h> - - +#include <util/generic/hash.h> + + namespace NKikimr { namespace NScheme { -class TTypeMetadataRegistry: private TNonCopyable { +class TTypeMetadataRegistry: private TNonCopyable { public: - struct TTypeMetadata: public ITypeMetadata { + struct TTypeMetadata: public ITypeMetadata { public: TTypeMetadata() : TypeId(0) @@ -22,7 +22,7 @@ public: , Name(name) {} - TTypeId GetTypeId() const override { + TTypeId GetTypeId() const override { return TypeId; } @@ -31,47 +31,47 @@ public: } private: - TTypeId TypeId; + TTypeId TypeId; ::TString Name; }; typedef THashMap<TTypeId, const ITypeMetadata*> TMapById; typedef THashMap<::TString, const ITypeMetadata*> TMapByName; - void Register(const ITypeMetadata* metadata) { + void Register(const ITypeMetadata* metadata) { Y_VERIFY(MapById.insert({ metadata->GetTypeId(), metadata }).second); Y_VERIFY(MapByName.insert({ metadata->GetName(), metadata }).second); } - const ITypeMetadata* GetType(TTypeId typeId) const { + const ITypeMetadata* GetType(TTypeId typeId) const { auto it = MapById.find(typeId); if (it == MapById.end()) return nullptr; return it->second; } - const ITypeMetadata* GetKnownType(TTypeId typeId) const { - if (!typeId) - ythrow yexception() << "Type id must be non zero"; - - auto typeMetadata = GetType(typeId); - if (typeMetadata) return typeMetadata; - - ythrow yexception() << "Unknown type: " << typeId; - } - - const ITypeMetadata* GetType(const TStringBuf& name) const { + const ITypeMetadata* GetKnownType(TTypeId typeId) const { + if (!typeId) + ythrow yexception() << "Type id must be non zero"; + + auto typeMetadata = GetType(typeId); + if (typeMetadata) return typeMetadata; + + ythrow yexception() << "Unknown type: " << typeId; + } + + const ITypeMetadata* GetType(const TStringBuf& name) const { auto it = MapByName.find(name); if (it == MapByName.end()) return nullptr; return it->second; } - TMapById::const_iterator begin() const { + TMapById::const_iterator begin() const { return MapById.begin(); } - TMapById::const_iterator end() const { + TMapById::const_iterator end() const { return MapById.end(); } @@ -85,5 +85,5 @@ private: TMapByName MapByName; }; -} // namespace NScheme -} // namespace NKikimr +} // namespace NScheme +} // namespace NKikimr diff --git a/ydb/core/scheme_types/scheme_type_registry.cpp b/ydb/core/scheme_types/scheme_type_registry.cpp index eebec6ddab3..d737254361e 100644 --- a/ydb/core/scheme_types/scheme_type_registry.cpp +++ b/ydb/core/scheme_types/scheme_type_registry.cpp @@ -1,16 +1,16 @@ #include "scheme_type_registry.h" #include "scheme_types.h" #include "scheme_types_defs.h" - + #include <util/digest/murmur.h> -#include <util/generic/algorithm.h> +#include <util/generic/algorithm.h> #define REGISTER_TYPE(name, size, ...) RegisterType<T##name>(); -namespace NKikimr { -namespace NScheme { - +namespace NKikimr { +namespace NScheme { + TTypeRegistry::TTypeRegistry() { // move to 'init defaults?' @@ -40,6 +40,6 @@ TTypeRegistry::TTypeRegistry() void TTypeRegistry::CalculateMetadataEtag() { } - -} // namespace NScheme -} // namespace NKikimr + +} // namespace NScheme +} // namespace NKikimr diff --git a/ydb/core/scheme_types/scheme_type_registry.h b/ydb/core/scheme_types/scheme_type_registry.h index b285f0c8a28..a82af192675 100644 --- a/ydb/core/scheme_types/scheme_type_registry.h +++ b/ydb/core/scheme_types/scheme_type_registry.h @@ -2,17 +2,17 @@ #include "scheme_types.h" #include "scheme_type_metadata.h" - + #include <util/generic/map.h> -#include <util/generic/maybe.h> -#include <util/generic/singleton.h> +#include <util/generic/maybe.h> +#include <util/generic/singleton.h> #include <util/generic/vector.h> #include <util/string/builder.h> -namespace NKikimr { -namespace NScheme { - +namespace NKikimr { +namespace NScheme { + //////////////////////////////////////////////////////// /// The TTypeRegistry class /// this class is _not_ threadsafe!!!! @@ -28,18 +28,18 @@ public: RegisterType(Singleton<T>()); } - void RegisterType(const IType *type) { - const TTypeId typeId = type->GetTypeId(); + void RegisterType(const IType *type) { + const TTypeId typeId = type->GetTypeId(); Y_VERIFY(typeId <= Max<TTypeId>()); - + Y_VERIFY(TypeByIdMap.insert({ typeId, type }).second); Y_VERIFY(TypeByNameMap.insert({ type->GetName(), type }).second); - + TypeMetadataRegistry.Register(type); } // - ITypeSP GetType(TTypeId typeId) const { + ITypeSP GetType(TTypeId typeId) const { if (typeId) { auto iter = TypeByIdMap.find(typeId); if (iter != TypeByIdMap.end()) { @@ -58,22 +58,22 @@ public: return type.IsKnownType() ? ::TString(type->GetName()) : (TStringBuilder() << "Unknown(" << typeId << ")"); } - ITypeSP GetKnownType(TTypeId typeId) const { - if (!typeId) - ythrow yexception() << "Type id must be non zero"; - + ITypeSP GetKnownType(TTypeId typeId) const { + if (!typeId) + ythrow yexception() << "Type id must be non zero"; + auto type = GetType(typeId); if (Y_LIKELY(type)) return type; - ythrow yexception() << "Unknown type: " << typeId; - } - - const IType* GetType(const TStringBuf& name) const { + ythrow yexception() << "Unknown type: " << typeId; + } + + const IType* GetType(const TStringBuf& name) const { auto iter = TypeByNameMap.find(name); return iter != TypeByNameMap.end() ? iter->second : nullptr; } - const IType* GetKnownType(const TStringBuf& name) const { + const IType* GetKnownType(const TStringBuf& name) const { auto type = GetType(name); if (Y_LIKELY(type)) return type; @@ -97,7 +97,7 @@ public: } void CalculateMetadataEtag(); - + ui64 GetMetadataEtag() const { return MetadataEtag; } @@ -113,6 +113,6 @@ private: ui64 MetadataEtag = 0; }; -} // namespace NScheme -} // namespace NKikimr +} // namespace NScheme +} // namespace NKikimr diff --git a/ydb/core/scheme_types/scheme_types.h b/ydb/core/scheme_types/scheme_types.h index a0a7fcf4fc3..3034843a245 100644 --- a/ydb/core/scheme_types/scheme_types.h +++ b/ydb/core/scheme_types/scheme_types.h @@ -4,19 +4,19 @@ #include <util/string/hex.h> #include <util/string/cast.h> - + #include <typeinfo> namespace NKikimr { -namespace NScheme { +namespace NScheme { //////////////////////////////////////////////////////// -class ITypeMetadata { +class ITypeMetadata { public: - enum class EFlags { - CanBeValueInKey = 0x01, - CanCompare = 0x02, + enum class EFlags { + CanBeValueInKey = 0x01, + CanCompare = 0x02, CanEquate = 0x04, CanHash = 0x08, HasDeterministicCompare = 0x10, @@ -24,29 +24,29 @@ public: HasDeterministicToString = 0x40, HasDeterministicHash = 0x80, HasDeterministicBytes = 0x100, - }; - + }; + virtual ~ITypeMetadata() {} - virtual TTypeId GetTypeId() const = 0; + virtual TTypeId GetTypeId() const = 0; virtual const char* GetName() const = 0; }; -class IType : public ITypeMetadata { +class IType : public ITypeMetadata { friend class ITypeSP; -friend class TTypeRegistry; +friend class TTypeRegistry; }; //////////////////////////////////////////////////////// class ITypeSP { public: // - ITypeSP(const IType* t = nullptr) + ITypeSP(const IType* t = nullptr) : Type(t) , TypeId(t ? t->GetTypeId() : 0) {} - ITypeSP(TTypeId typeId) + ITypeSP(TTypeId typeId) : Type(nullptr) , TypeId(typeId) {} @@ -61,9 +61,9 @@ public: const IType* GetType() const noexcept { return Type; } private: - const IType* Type; - TTypeId TypeId; + const IType* Type; + TTypeId TypeId; }; -} // namspace NScheme -} // namespace NKikimr +} // namspace NScheme +} // namespace NKikimr diff --git a/ydb/core/scheme_types/scheme_types_defs.cpp b/ydb/core/scheme_types/scheme_types_defs.cpp index e45dcde15cd..eeaba2e9af0 100644 --- a/ydb/core/scheme_types/scheme_types_defs.cpp +++ b/ydb/core/scheme_types/scheme_types_defs.cpp @@ -1,11 +1,11 @@ #include "scheme_types_defs.h" -#include <util/stream/output.h> +#include <util/stream/output.h> + + +namespace NKikimr { +namespace NScheme { - -namespace NKikimr { -namespace NScheme { - namespace NNames { DECLARE_TYPED_TYPE_NAME(Int32); DECLARE_TYPED_TYPE_NAME(Uint32); @@ -33,26 +33,26 @@ namespace NNames { DECLARE_TYPED_TYPE_NAME(Interval); DECLARE_TYPED_TYPE_NAME(DyNumber); -} +} void WriteEscapedValue(IOutputStream &out, const char *data, size_t size) { - static const size_t BUFFER_SIZE = 32; - char buffer[BUFFER_SIZE]; - size_t bufPos = 0; - - for (const char *end = data + size; data != end; ++data) { - if (bufPos >= BUFFER_SIZE - 1) { - out.Write(buffer, bufPos); - bufPos = 0; - } - const char c = *data; - if (c == '#' || c == '/') - buffer[bufPos++] = '\\'; - buffer[bufPos++] = c; - } - if (bufPos) - out.Write(buffer, bufPos); + static const size_t BUFFER_SIZE = 32; + char buffer[BUFFER_SIZE]; + size_t bufPos = 0; + + for (const char *end = data + size; data != end; ++data) { + if (bufPos >= BUFFER_SIZE - 1) { + out.Write(buffer, bufPos); + bufPos = 0; + } + const char c = *data; + if (c == '#' || c == '/') + buffer[bufPos++] = '\\'; + buffer[bufPos++] = c; + } + if (bufPos) + out.Write(buffer, bufPos); } -} // namespace NScheme -} // namespace NKikimr +} // namespace NScheme +} // namespace NKikimr diff --git a/ydb/core/scheme_types/scheme_types_defs.h b/ydb/core/scheme_types/scheme_types_defs.h index 49eb3beee62..bb06b097023 100644 --- a/ydb/core/scheme_types/scheme_types_defs.h +++ b/ydb/core/scheme_types/scheme_types_defs.h @@ -11,18 +11,18 @@ #define DECLARE_TYPED_TYPE_NAME(x)\ const char x[] = #x; -namespace NKikimr { -namespace NScheme { +namespace NKikimr { +namespace NScheme { const ui32 MaxKeyValueSize = 4096; //////////////////////////////////////////////////////// template<typename T, typename TDerived, TTypeId TypeId_, const char* Name_> -class TTypedType : public IType { +class TTypedType : public IType { public: // typedef T TValueType; - static constexpr TTypeId TypeId = TypeId_; + static constexpr TTypeId TypeId = TypeId_; // class TInstance { @@ -37,9 +37,9 @@ public: } }; - TTypedType() = default; + TTypedType() = default; - // IType interface + // IType interface const char* GetName() const override { return Name_; } @@ -47,7 +47,7 @@ public: static constexpr ui32 GetFixedSize() { return sizeof(T); } - TTypeId GetTypeId() const override { return TypeId; } + TTypeId GetTypeId() const override { return TypeId; } static TRawTypeValue ToRawTypeValue(const T& value) { return TRawTypeValue((void*)&value, sizeof(T), TypeId); } @@ -76,10 +76,10 @@ namespace NNames { extern const char Uint64[7]; } -class TInt32 : public IIntegerTypeWithKeyString<i32, NTypeIds::Int32, NNames::Int32> {}; -class TUint32 : public IIntegerTypeWithKeyString<ui32, NTypeIds::Uint32, NNames::Uint32> {}; -class TInt64 : public IIntegerTypeWithKeyString<i64, NTypeIds::Int64, NNames::Int64> {}; -class TUint64 : public IIntegerTypeWithKeyString<ui64, NTypeIds::Uint64, NNames::Uint64> {}; +class TInt32 : public IIntegerTypeWithKeyString<i32, NTypeIds::Int32, NNames::Int32> {}; +class TUint32 : public IIntegerTypeWithKeyString<ui32, NTypeIds::Uint32, NNames::Uint32> {}; +class TInt64 : public IIntegerTypeWithKeyString<i64, NTypeIds::Int64, NNames::Int64> {}; +class TUint64 : public IIntegerTypeWithKeyString<ui64, NTypeIds::Uint64, NNames::Uint64> {}; // upyachka to get around undefined tryfromstring for chars @@ -109,8 +109,8 @@ class TRealBase : public TTypedType<T, TDerived, TypeId, Name> { public: }; -class TDouble : public TRealBase<double, TDouble, NTypeIds::Double, NNames::Double> {}; -class TFloat : public TRealBase<float, TFloat, NTypeIds::Float, NNames::Float> {}; +class TDouble : public TRealBase<double, TDouble, NTypeIds::Double, NNames::Double> {}; +class TFloat : public TRealBase<float, TFloat, NTypeIds::Float, NNames::Float> {}; //////////////////////////////////////////////////////// template<typename TFirst, typename TSecond, ui32 TypeId, const char* Name> @@ -130,7 +130,7 @@ namespace NNames { extern const char PairUi64Ui64[13]; } -class TPairUi64Ui64 : public IIntegerPair<ui64, ui64, NTypeIds::PairUi64Ui64, NNames::PairUi64Ui64> {}; +class TPairUi64Ui64 : public IIntegerPair<ui64, ui64, NTypeIds::PairUi64Ui64, NNames::PairUi64Ui64> {}; //////////////////////////////////////////////////////// @@ -142,7 +142,7 @@ class TStringImpl { public: }; -template <typename TDerived, TTypeId TypeId, const char* Name> +template <typename TDerived, TTypeId TypeId, const char* Name> class TStringBase : public TTypedType<::TString, TDerived, TypeId, Name> { public: @@ -165,7 +165,7 @@ namespace NNames { } void WriteEscapedValue(IOutputStream &out, const char *data, size_t size); - + class TString : public TStringBase<TString, NTypeIds::String, NNames::String> {}; class TUtf8 : public TStringBase<TUtf8, NTypeIds::Utf8, NNames::Utf8> { @@ -188,7 +188,7 @@ class TDyNumber : public TStringBase<TDyNumber, NTypeIds::DyNumber, NNames::DyNu public: }; -template <ui32 TMaxSize, TTypeId TypeId, const char* Name> +template <ui32 TMaxSize, TTypeId TypeId, const char* Name> class TBoundedString : public TStringBase<TBoundedString<TMaxSize, TypeId, Name>, TypeId, Name> { public: static constexpr ui32 MaxSize = TMaxSize; @@ -279,5 +279,5 @@ static inline bool TryGetTypeName(TTypeId typeId, ::TString& typeName) { return false; } -} // namespace NScheme -} // namespace NKikimr +} // namespace NScheme +} // namespace NKikimr diff --git a/ydb/core/scheme_types/ya.make b/ydb/core/scheme_types/ya.make index 52f51528d24..9ab58a3009c 100644 --- a/ydb/core/scheme_types/ya.make +++ b/ydb/core/scheme_types/ya.make @@ -1,19 +1,19 @@ -LIBRARY() - -OWNER( - ddoarn - vvvv +LIBRARY() + +OWNER( + ddoarn + vvvv g:kikimr -) - -SRCS( - scheme_type_metadata.cpp - scheme_type_registry.cpp - scheme_types_defs.cpp -) - -PEERDIR( +) + +SRCS( + scheme_type_metadata.cpp + scheme_type_registry.cpp + scheme_types_defs.cpp +) + +PEERDIR( ydb/public/lib/scheme_types -) - -END() +) + +END() diff --git a/ydb/core/security/ticket_parser.cpp b/ydb/core/security/ticket_parser.cpp index 1243e25df56..c7d024d03dc 100644 --- a/ydb/core/security/ticket_parser.cpp +++ b/ydb/core/security/ticket_parser.cpp @@ -313,7 +313,7 @@ class TTicketParser : public TActorBootstrapped<TTicketParser> { CrackTicket(ev->Get()->Ticket, ticket, ticketType); TString key = GetKey(ev->Get()); - TActorId sender = ev->Sender; + TActorId sender = ev->Sender; ui64 cookie = ev->Cookie; CounterTicketsReceived->Inc(); diff --git a/ydb/core/sys_view/common/scan_actor_base_impl.h b/ydb/core/sys_view/common/scan_actor_base_impl.h index d15c3faa6d2..298a8efbc03 100644 --- a/ydb/core/sys_view/common/scan_actor_base_impl.h +++ b/ydb/core/sys_view/common/scan_actor_base_impl.h @@ -26,7 +26,7 @@ class TScanActorBase : public TActorBootstrapped<TDerived> { public: using TBase = TActorBootstrapped<TDerived>; - TScanActorBase(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, + TScanActorBase(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns) : OwnerActorId(ownerId) , ScanId(scanId) @@ -318,7 +318,7 @@ private: protected: static constexpr TDuration Timeout = TDuration::Seconds(60); - const TActorId OwnerActorId; + const TActorId OwnerActorId; const ui32 ScanId; const TTableId TableId; TSerializedTableRange TableRange; diff --git a/ydb/core/sys_view/nodes/nodes.cpp b/ydb/core/sys_view/nodes/nodes.cpp index ad613ede99f..a2322de9f96 100644 --- a/ydb/core/sys_view/nodes/nodes.cpp +++ b/ydb/core/sys_view/nodes/nodes.cpp @@ -25,7 +25,7 @@ public: return NKikimrServices::TActivity::KQP_SYSTEM_VIEW_SCAN; } - TNodesScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, + TNodesScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns) : TBase(ownerId, scanId, tableId, tableRange, columns) { @@ -85,7 +85,7 @@ private: return; } - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); Send(nameserviceId, new TEvInterconnect::TEvListNodes()); } @@ -103,7 +103,7 @@ private: nodeId >= NodeIdFrom && nodeId <= NodeIdTo) { - TActorId whiteboardId = MakeNodeWhiteboardServiceId(nodeId); + TActorId whiteboardId = MakeNodeWhiteboardServiceId(nodeId); auto request = MakeHolder<TEvWhiteboard::TEvSystemStateRequest>(); Send(whiteboardId, request.Release(), @@ -233,7 +233,7 @@ private: THashMap<TNodeId, THolder<TEvWhiteboard::TEvSystemStateResponse>> WBSystemInfo; }; -THolder<IActor> CreateNodesScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, +THolder<IActor> CreateNodesScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns) { return MakeHolder<TNodesScan>(ownerId, scanId, tableId, tableRange, columns); diff --git a/ydb/core/sys_view/nodes/nodes.h b/ydb/core/sys_view/nodes/nodes.h index 249343b9e49..8f1192250bd 100644 --- a/ydb/core/sys_view/nodes/nodes.h +++ b/ydb/core/sys_view/nodes/nodes.h @@ -5,7 +5,7 @@ namespace NKikimr { namespace NSysView { -THolder<IActor> CreateNodesScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, +THolder<IActor> CreateNodesScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns); } // NSysView diff --git a/ydb/core/sys_view/partition_stats/partition_stats.cpp b/ydb/core/sys_view/partition_stats/partition_stats.cpp index d331cc31634..d6ea9ff73bd 100644 --- a/ydb/core/sys_view/partition_stats/partition_stats.cpp +++ b/ydb/core/sys_view/partition_stats/partition_stats.cpp @@ -319,7 +319,7 @@ public: return NKikimrServices::TActivity::KQP_SYSTEM_VIEW_SCAN; } - TPartitionStatsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, + TPartitionStatsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns) : TBase(ownerId, scanId, tableId, tableRange, columns) { @@ -547,7 +547,7 @@ private: bool IncludePathColumn = false; }; -THolder<IActor> CreatePartitionStatsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, +THolder<IActor> CreatePartitionStatsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns) { return MakeHolder<TPartitionStatsScan>(ownerId, scanId, tableId, tableRange, columns); diff --git a/ydb/core/sys_view/partition_stats/partition_stats.h b/ydb/core/sys_view/partition_stats/partition_stats.h index 18942f7e74e..dedf62906fb 100644 --- a/ydb/core/sys_view/partition_stats/partition_stats.h +++ b/ydb/core/sys_view/partition_stats/partition_stats.h @@ -12,7 +12,7 @@ THolder<IActor> CreatePartitionStatsCollector( size_t batchSize = STATS_COLLECTOR_BATCH_SIZE, size_t pendingRequestsCount = STATS_COLLECTOR_QUEUE_SIZE_LIMIT); -THolder<IActor> CreatePartitionStatsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, +THolder<IActor> CreatePartitionStatsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns); } // NSysView diff --git a/ydb/core/sys_view/partition_stats/partition_stats_ut.cpp b/ydb/core/sys_view/partition_stats/partition_stats_ut.cpp index 367c084fc31..c1230129482 100644 --- a/ydb/core/sys_view/partition_stats/partition_stats_ut.cpp +++ b/ydb/core/sys_view/partition_stats/partition_stats_ut.cpp @@ -32,7 +32,7 @@ Y_UNIT_TEST_SUITE(PartitionStats) { auto setPartitioning = MakeHolder<TEvSysView::TEvSetPartitioning>(domainKey, id, ""); setPartitioning->ShardIndices.push_back(TShardIdx(ownerId * 3 + pathId, 0)); setPartitioning->ShardIndices.push_back(TShardIdx(ownerId * 3 + pathId, 1)); - runtime.Send(new IEventHandle(collectorId, TActorId(), setPartitioning.Release())); + runtime.Send(new IEventHandle(collectorId, TActorId(), setPartitioning.Release())); } } diff --git a/ydb/core/sys_view/processor/processor.cpp b/ydb/core/sys_view/processor/processor.cpp index da954ca428b..d0c09f8e7b7 100644 --- a/ydb/core/sys_view/processor/processor.cpp +++ b/ydb/core/sys_view/processor/processor.cpp @@ -5,11 +5,11 @@ namespace NKikimr { namespace NSysView { -IActor* CreateSysViewProcessor(const TActorId& tablet, TTabletStorageInfo* info) { +IActor* CreateSysViewProcessor(const TActorId& tablet, TTabletStorageInfo* info) { return new TSysViewProcessor(tablet, info, EProcessorMode::MINUTE); } -IActor* CreateSysViewProcessorForTests(const TActorId& tablet, TTabletStorageInfo* info) { +IActor* CreateSysViewProcessorForTests(const TActorId& tablet, TTabletStorageInfo* info) { return new TSysViewProcessor(tablet, info, EProcessorMode::FAST); } diff --git a/ydb/core/sys_view/processor/processor.h b/ydb/core/sys_view/processor/processor.h index 18da9cee315..ad086965974 100644 --- a/ydb/core/sys_view/processor/processor.h +++ b/ydb/core/sys_view/processor/processor.h @@ -6,8 +6,8 @@ namespace NKikimr { namespace NSysView { -IActor* CreateSysViewProcessor(const TActorId& tablet, TTabletStorageInfo* info); -IActor* CreateSysViewProcessorForTests(const TActorId& tablet, TTabletStorageInfo* info); +IActor* CreateSysViewProcessor(const TActorId& tablet, TTabletStorageInfo* info); +IActor* CreateSysViewProcessorForTests(const TActorId& tablet, TTabletStorageInfo* info); } // NSysView } // NKikimr diff --git a/ydb/core/sys_view/processor/processor_impl.cpp b/ydb/core/sys_view/processor/processor_impl.cpp index bc491a730fb..86ddbfd4238 100644 --- a/ydb/core/sys_view/processor/processor_impl.cpp +++ b/ydb/core/sys_view/processor/processor_impl.cpp @@ -8,7 +8,7 @@ namespace NKikimr { namespace NSysView { -TSysViewProcessor::TSysViewProcessor(const TActorId& tablet, TTabletStorageInfo* info, EProcessorMode processorMode) +TSysViewProcessor::TSysViewProcessor(const TActorId& tablet, TTabletStorageInfo* info, EProcessorMode processorMode) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, new NMiniKQL::TMiniKQLFactory) , TotalInterval(TDuration::Seconds(processorMode == EProcessorMode::FAST ? 6 : 60)) diff --git a/ydb/core/sys_view/processor/processor_impl.h b/ydb/core/sys_view/processor/processor_impl.h index f9292b1849e..4b49ecbb93c 100644 --- a/ydb/core/sys_view/processor/processor_impl.h +++ b/ydb/core/sys_view/processor/processor_impl.h @@ -28,7 +28,7 @@ public: return NKikimrServices::TActivity::SYSTEM_VIEW_PROCESSOR; } - TSysViewProcessor(const TActorId& tablet, TTabletStorageInfo* info, EProcessorMode processorMode); + TSysViewProcessor(const TActorId& tablet, TTabletStorageInfo* info, EProcessorMode processorMode); private: using Schema = TProcessorSchema; diff --git a/ydb/core/sys_view/processor/tx_configure.cpp b/ydb/core/sys_view/processor/tx_configure.cpp index 2186948a3a5..68a2a133e55 100644 --- a/ydb/core/sys_view/processor/tx_configure.cpp +++ b/ydb/core/sys_view/processor/tx_configure.cpp @@ -5,9 +5,9 @@ namespace NSysView { struct TSysViewProcessor::TTxConfigure : public TTxBase { NKikimrSysView::TEvConfigureProcessor Record; - TActorId Sender; + TActorId Sender; - TTxConfigure(TSelf* self, NKikimrSysView::TEvConfigureProcessor&& record, const TActorId& sender) + TTxConfigure(TSelf* self, NKikimrSysView::TEvConfigureProcessor&& record, const TActorId& sender) : TTxBase(self) , Record(std::move(record)) , Sender(sender) diff --git a/ydb/core/sys_view/query_stats/query_metrics.cpp b/ydb/core/sys_view/query_stats/query_metrics.cpp index 4e124498e49..5f2acb5c23e 100644 --- a/ydb/core/sys_view/query_stats/query_metrics.cpp +++ b/ydb/core/sys_view/query_stats/query_metrics.cpp @@ -33,7 +33,7 @@ public: return NKikimrServices::TActivity::KQP_SYSTEM_VIEW_SCAN; } - TQueryMetricsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, + TQueryMetricsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns) : TBase(ownerId, scanId, tableId, tableRange, columns) { @@ -163,7 +163,7 @@ private: NKikimrSysView::TEvGetQueryMetricsRequest Request; }; -THolder<IActor> CreateQueryMetricsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, +THolder<IActor> CreateQueryMetricsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns) { return MakeHolder<TQueryMetricsScan>(ownerId, scanId, tableId, tableRange, columns); diff --git a/ydb/core/sys_view/query_stats/query_metrics.h b/ydb/core/sys_view/query_stats/query_metrics.h index 912fcb5ac7c..e3740811265 100644 --- a/ydb/core/sys_view/query_stats/query_metrics.h +++ b/ydb/core/sys_view/query_stats/query_metrics.h @@ -5,7 +5,7 @@ namespace NKikimr { namespace NSysView { -THolder<IActor> CreateQueryMetricsScan(const TActorId& ownerId, ui32 scanId, +THolder<IActor> CreateQueryMetricsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns); diff --git a/ydb/core/sys_view/query_stats/query_stats.cpp b/ydb/core/sys_view/query_stats/query_stats.cpp index d01e4733a8a..918c9b781e6 100644 --- a/ydb/core/sys_view/query_stats/query_stats.cpp +++ b/ydb/core/sys_view/query_stats/query_stats.cpp @@ -95,7 +95,7 @@ public: return NKikimrServices::TActivity::KQP_SYSTEM_VIEW_SCAN; } - TQueryStatsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, + TQueryStatsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns, NKikimrSysView::EStatsType statsType, ui64 bucketCount, const TDuration& bucketSize) @@ -506,7 +506,7 @@ private: NKikimrSysView::TEvGetQueryMetricsRequest Request; }; -THolder<IActor> CreateQueryStatsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TStringBuf viewName, +THolder<IActor> CreateQueryStatsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TStringBuf viewName, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns) { if (viewName == TopQueriesByDuration1MinuteName) { diff --git a/ydb/core/sys_view/query_stats/query_stats.h b/ydb/core/sys_view/query_stats/query_stats.h index af874fdb761..b9fe4b7ecc6 100644 --- a/ydb/core/sys_view/query_stats/query_stats.h +++ b/ydb/core/sys_view/query_stats/query_stats.h @@ -17,7 +17,7 @@ struct TQueryStatsBucketRange { explicit TQueryStatsBucketRange(const TSerializedTableRange& range, const TDuration& bucketSize); }; -THolder<IActor> CreateQueryStatsScan(const TActorId& ownerId, ui32 scanId, +THolder<IActor> CreateQueryStatsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TStringBuf viewName, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns); diff --git a/ydb/core/sys_view/scan.cpp b/ydb/core/sys_view/scan.cpp index 11a4684529d..f337cb6ba77 100644 --- a/ydb/core/sys_view/scan.cpp +++ b/ydb/core/sys_view/scan.cpp @@ -15,7 +15,7 @@ namespace NKikimr { namespace NSysView { -THolder<IActor> CreateSystemViewScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, +THolder<IActor> CreateSystemViewScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns) { if (tableId.SysViewInfo == PartitionStatsName) { diff --git a/ydb/core/sys_view/scan.h b/ydb/core/sys_view/scan.h index cd59ec111ac..bfac578bfae 100644 --- a/ydb/core/sys_view/scan.h +++ b/ydb/core/sys_view/scan.h @@ -5,7 +5,7 @@ namespace NKikimr { namespace NSysView { -THolder<IActor> CreateSystemViewScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, +THolder<IActor> CreateSystemViewScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns); } // NSysView diff --git a/ydb/core/sys_view/service/sysview_service.h b/ydb/core/sys_view/service/sysview_service.h index 16837595f6f..430dfb0261a 100644 --- a/ydb/core/sys_view/service/sysview_service.h +++ b/ydb/core/sys_view/service/sysview_service.h @@ -9,9 +9,9 @@ namespace NKikimr { namespace NSysView { -inline TActorId MakeSysViewServiceID(ui32 node) { +inline TActorId MakeSysViewServiceID(ui32 node) { const char x[12] = "SysViewSvc!"; - return TActorId(node, TStringBuf(x, 12)); + return TActorId(node, TStringBuf(x, 12)); } void CollectQueryStats(const TActorContext& ctx, const NKqpProto::TKqpStatsQuery* queryStats, diff --git a/ydb/core/sys_view/storage/groups.cpp b/ydb/core/sys_view/storage/groups.cpp index 9c2df75b31a..dabd46f9dc7 100644 --- a/ydb/core/sys_view/storage/groups.cpp +++ b/ydb/core/sys_view/storage/groups.cpp @@ -41,7 +41,7 @@ public: } }; -THolder<IActor> CreateGroupsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, +THolder<IActor> CreateGroupsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns) { return MakeHolder<TGroupsScan>(ownerId, scanId, tableId, tableRange, columns); diff --git a/ydb/core/sys_view/storage/groups.h b/ydb/core/sys_view/storage/groups.h index 24f756d50f8..ab861d94a84 100644 --- a/ydb/core/sys_view/storage/groups.h +++ b/ydb/core/sys_view/storage/groups.h @@ -5,7 +5,7 @@ namespace NKikimr { namespace NSysView { -THolder<IActor> CreateGroupsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, +THolder<IActor> CreateGroupsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns); } // NSysView diff --git a/ydb/core/sys_view/storage/pdisks.cpp b/ydb/core/sys_view/storage/pdisks.cpp index 90cee06b61d..47f84ae9aa5 100644 --- a/ydb/core/sys_view/storage/pdisks.cpp +++ b/ydb/core/sys_view/storage/pdisks.cpp @@ -44,7 +44,7 @@ public: } }; -THolder<IActor> CreatePDisksScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, +THolder<IActor> CreatePDisksScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns) { return MakeHolder<TPDisksScan>(ownerId, scanId, tableId, tableRange, columns); diff --git a/ydb/core/sys_view/storage/pdisks.h b/ydb/core/sys_view/storage/pdisks.h index 0883858e8e8..4172cb333fd 100644 --- a/ydb/core/sys_view/storage/pdisks.h +++ b/ydb/core/sys_view/storage/pdisks.h @@ -5,7 +5,7 @@ namespace NKikimr { namespace NSysView { -THolder<IActor> CreatePDisksScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, +THolder<IActor> CreatePDisksScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns); } // NSysView diff --git a/ydb/core/sys_view/storage/storage_pools.cpp b/ydb/core/sys_view/storage/storage_pools.cpp index 2e05bd96325..66896a69995 100644 --- a/ydb/core/sys_view/storage/storage_pools.cpp +++ b/ydb/core/sys_view/storage/storage_pools.cpp @@ -40,7 +40,7 @@ public: } }; -THolder<IActor> CreateStoragePoolsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, +THolder<IActor> CreateStoragePoolsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns) { return MakeHolder<TStoragePoolsScan>(ownerId, scanId, tableId, tableRange, columns); diff --git a/ydb/core/sys_view/storage/storage_pools.h b/ydb/core/sys_view/storage/storage_pools.h index d1f1baab9d3..7ced359d00b 100644 --- a/ydb/core/sys_view/storage/storage_pools.h +++ b/ydb/core/sys_view/storage/storage_pools.h @@ -5,7 +5,7 @@ namespace NKikimr { namespace NSysView { -THolder<IActor> CreateStoragePoolsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, +THolder<IActor> CreateStoragePoolsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns); } // NSysView diff --git a/ydb/core/sys_view/storage/vslots.cpp b/ydb/core/sys_view/storage/vslots.cpp index c89df50fa6c..987eb2750de 100644 --- a/ydb/core/sys_view/storage/vslots.cpp +++ b/ydb/core/sys_view/storage/vslots.cpp @@ -42,7 +42,7 @@ public: } }; -THolder<IActor> CreateVSlotsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, +THolder<IActor> CreateVSlotsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns) { return MakeHolder<TVSlotsScan>(ownerId, scanId, tableId, tableRange, columns); diff --git a/ydb/core/sys_view/storage/vslots.h b/ydb/core/sys_view/storage/vslots.h index 80aaf070c6d..34f38f9e0f8 100644 --- a/ydb/core/sys_view/storage/vslots.h +++ b/ydb/core/sys_view/storage/vslots.h @@ -5,7 +5,7 @@ namespace NKikimr { namespace NSysView { -THolder<IActor> CreateVSlotsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, +THolder<IActor> CreateVSlotsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns); } // NSysView diff --git a/ydb/core/sys_view/tablets/tablets.cpp b/ydb/core/sys_view/tablets/tablets.cpp index 5478e71d366..88627748d8c 100644 --- a/ydb/core/sys_view/tablets/tablets.cpp +++ b/ydb/core/sys_view/tablets/tablets.cpp @@ -23,7 +23,7 @@ public: return NKikimrServices::TActivity::KQP_SYSTEM_VIEW_SCAN; } - TTabletsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, + TTabletsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns) : TBase(ownerId, scanId, tableId, tableRange, columns) { @@ -357,7 +357,7 @@ private: bool BatchRequested = false; }; -THolder<IActor> CreateTabletsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, +THolder<IActor> CreateTabletsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns) { return MakeHolder<TTabletsScan>(ownerId, scanId, tableId, tableRange, columns); diff --git a/ydb/core/sys_view/tablets/tablets.h b/ydb/core/sys_view/tablets/tablets.h index e14e77c561e..ca54b08d3e1 100644 --- a/ydb/core/sys_view/tablets/tablets.h +++ b/ydb/core/sys_view/tablets/tablets.h @@ -5,7 +5,7 @@ namespace NKikimr { namespace NSysView { -THolder<IActor> CreateTabletsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, +THolder<IActor> CreateTabletsScan(const TActorId& ownerId, ui32 scanId, const TTableId& tableId, const TTableRange& tableRange, const TArrayRef<NMiniKQL::TKqpComputeContextBase::TColumn>& columns); } // NSysView diff --git a/ydb/core/sys_view/ut_kqp.cpp b/ydb/core/sys_view/ut_kqp.cpp index 9c165124f8e..5e90eb59927 100644 --- a/ydb/core/sys_view/ut_kqp.cpp +++ b/ydb/core/sys_view/ut_kqp.cpp @@ -411,7 +411,7 @@ Y_UNIT_TEST_SUITE(SystemView) { stats->QueryStats.SetEndTimeMs(endTimeUs / 1000); auto serviceId = MakeSysViewServiceID(runtime->GetNodeId(nodeIdx)); - runtime->Send(new IEventHandle(serviceId, TActorId(), stats.Release()), nodeIdx); + runtime->Send(new IEventHandle(serviceId, TActorId(), stats.Release()), nodeIdx); }; makeQueryEvent(tenant1Node0, buckets[0], "a", 100); @@ -816,7 +816,7 @@ Y_UNIT_TEST_SUITE(SystemView) { ).GetValueSync()); auto serviceToKill = MakeSysViewServiceID(env.GetServer().GetRuntime()->GetNodeId(2)); - env.GetServer().GetRuntime()->Send(new IEventHandle(serviceToKill, TActorId(), new TEvents::TEvPoison())); + env.GetServer().GetRuntime()->Send(new IEventHandle(serviceToKill, TActorId(), new TEvents::TEvPoison())); auto it = client.StreamExecuteScanQuery(R"( SELECT diff --git a/ydb/core/tablet/bootstrapper.cpp b/ydb/core/tablet/bootstrapper.cpp index 6413edba535..d2e5dd8b7f3 100644 --- a/ydb/core/tablet/bootstrapper.cpp +++ b/ydb/core/tablet/bootstrapper.cpp @@ -53,7 +53,7 @@ class TBootstrapper : public TActor<TBootstrapper> { TIntrusivePtr<TTabletStorageInfo> TabletInfo; TIntrusivePtr<TBootstrapperInfo> BootstrapperInfo; - TActorId LookOnActorID; + TActorId LookOnActorID; TActorId FollowerActorID; ui64 RoundCounter; @@ -64,13 +64,13 @@ class TBootstrapper : public TActor<TBootstrapper> { // we watch someone and do not act w/o shutdown struct TWatch { struct TWatched { - TActorId ActorID; + TActorId ActorID; bool Owner; TWatched() : Owner(false) {} - TWatched(const TActorId &actorID, bool owner) + TWatched(const TActorId &actorID, bool owner) : ActorID(actorID) , Owner(owner) {} @@ -94,7 +94,7 @@ class TBootstrapper : public TActor<TBootstrapper> { return Watched.empty(); } - bool RemoveAlien(const TActorId &alien) { + bool RemoveAlien(const TActorId &alien) { for (ui32 i = 0, e = Watched.size(); i != e; ++i) { if (Watched[i].ActorID == alien) return RemoveAlienEntry(i); @@ -116,20 +116,20 @@ class TBootstrapper : public TActor<TBootstrapper> { // we are under watch, must notify on error struct TWatched { struct TWatcher { - TActorId ActorID; + TActorId ActorID; ui64 Round; TWatcher() : ActorID() , Round() {} - TWatcher(const TActorId &actorID, ui64 round) + TWatcher(const TActorId &actorID, ui64 round) : ActorID(actorID) , Round(round) {} }; - void AddWatcher(const TActorId &actorId, ui64 round) { + void AddWatcher(const TActorId &actorId, ui64 round) { for (TWatcher &x : Watchers) { if (actorId.NodeId() == x.ActorID.NodeId()) { x.ActorID = actorId; @@ -236,7 +236,7 @@ class TBootstrapper : public TActor<TBootstrapper> { LOG_INFO(ctx, NKikimrServices::BOOTSTRAPPER, "tablet: %" PRIu64 ", type: %s, tablet dead", TabletInfo->TabletID, GetTabletTypeName()); - LookOnActorID = TActorId(); + LookOnActorID = TActorId(); NotifyAndRound(ctx); } } @@ -244,7 +244,7 @@ class TBootstrapper : public TActor<TBootstrapper> { void Stop() { if (LookOnActorID) { Send(LookOnActorID, new TEvents::TEvPoisonPill()); - LookOnActorID = TActorId(); + LookOnActorID = TActorId(); } if (FollowerActorID) { @@ -268,7 +268,7 @@ class TBootstrapper : public TActor<TBootstrapper> { Become(&TThis::StateStandBy); } - void BecomeWatch(const TActorId &watchOn, bool owner, const TActorContext &ctx) { + void BecomeWatch(const TActorId &watchOn, bool owner, const TActorContext &ctx) { Y_UNUSED(ctx); BootDelayedUntil = { }; @@ -294,7 +294,7 @@ class TBootstrapper : public TActor<TBootstrapper> { Become(&TThis::StateWatch); } - bool ApplyAlienState(const TActorId &alien, NKikimrBootstrapper::TEvWatchResult::EState state, ui64 seed, const TActorContext &ctx) { + bool ApplyAlienState(const TActorId &alien, NKikimrBootstrapper::TEvWatchResult::EState state, ui64 seed, const TActorContext &ctx) { const ui32 alienNodeIdx = AlienIndex(alien.NodeId()); if (alienNodeIdx == Max<ui32>()) return true; @@ -366,10 +366,10 @@ class TBootstrapper : public TActor<TBootstrapper> { TabletInfo->TabletID, GetTabletTypeName(), online, total, quorum); const ui64 wx = BootstrapperInfo->WatchThreshold.MicroSeconds(); - const auto sleepDuration = TDuration::MicroSeconds(wx / 2 + wx * (SelfSeed % 0x10000) / 0x20000); + const auto sleepDuration = TDuration::MicroSeconds(wx / 2 + wx * (SelfSeed % 0x10000) / 0x20000); ctx.ExecutorThread.ActorSystem->Schedule( - Min(sleepDuration, BootDelayedUntil - now), + Min(sleepDuration, BootDelayedUntil - now), new IEventHandle(ctx.SelfID, ctx.SelfID, new TEvents::TEvWakeup(), 0, RoundCounter)); Become(&TThis::StateSleep); return false; @@ -410,7 +410,7 @@ class TBootstrapper : public TActor<TBootstrapper> { TabletInfo->TabletID, GetTabletTypeName()); const ui64 wx = BootstrapperInfo->WatchThreshold.MicroSeconds(); - const auto sleepDuration = TDuration::MicroSeconds(wx / 2 + wx * (SelfSeed % 0x10000) / 0x20000); + const auto sleepDuration = TDuration::MicroSeconds(wx / 2 + wx * (SelfSeed % 0x10000) / 0x20000); Become(&TThis::StateSleep); ctx.ExecutorThread.ActorSystem->Schedule(sleepDuration, new IEventHandle(ctx.SelfID, ctx.SelfID, new TEvents::TEvWakeup(), 0, RoundCounter)); @@ -469,7 +469,7 @@ class TBootstrapper : public TActor<TBootstrapper> { LOG_DEBUG(ctx, NKikimrServices::BOOTSTRAPPER, "tablet: %" PRIu64 ", type: %s, disconnected from %" PRIu32, TabletInfo->TabletID, GetTabletTypeName(), node); - if (ApplyAlienState(TActorId(node, 0, 0, 0), NKikimrBootstrapper::TEvWatchResult::DISCONNECTED, Max<ui64>(), ctx)) + if (ApplyAlienState(TActorId(node, 0, 0, 0), NKikimrBootstrapper::TEvWatchResult::DISCONNECTED, Max<ui64>(), ctx)) return; CheckRoundCompletion(ctx); @@ -483,7 +483,7 @@ class TBootstrapper : public TActor<TBootstrapper> { } void HandleWatch(TEvBootstrapper::TEvNotify::TPtr &ev, const TActorContext &ctx) { - const TActorId alien = ev->Sender; + const TActorId alien = ev->Sender; if (Watches->RemoveAlien(alien)) { NotifyAndRound(ctx); } @@ -566,7 +566,7 @@ public: Y_VERIFY(TTabletTypes::TYPE_INVALID != TabletInfo->TabletType); } - TAutoPtr<IEventHandle> AfterRegister(const TActorId &selfId, const TActorId &parentId) override { + TAutoPtr<IEventHandle> AfterRegister(const TActorId &selfId, const TActorId &parentId) override { Y_UNUSED(parentId); return new IEventHandle(selfId, selfId, new TEvents::TEvBootstrap()); } @@ -629,10 +629,10 @@ IActor* CreateBootstrapper(TTabletStorageInfo *tabletInfo, TBootstrapperInfo *bo return new TBootstrapper(tabletInfo, bootstrapperInfo, standby); } -TActorId MakeBootstrapperID(ui64 tablet, ui32 node) { +TActorId MakeBootstrapperID(ui64 tablet, ui32 node) { char x[12] ={'b', 'o', 'o', 't'}; memcpy(x + 4, &tablet, sizeof(ui64)); - return TActorId(node, TStringBuf(x, x + 12)); + return TActorId(node, TStringBuf(x, x + 12)); } } diff --git a/ydb/core/tablet/bootstrapper.h b/ydb/core/tablet/bootstrapper.h index abe7b0d09f2..2e246ea2b07 100644 --- a/ydb/core/tablet/bootstrapper.h +++ b/ydb/core/tablet/bootstrapper.h @@ -48,6 +48,6 @@ struct TBootstrapperInfo : public TThrRefBase { }; IActor* CreateBootstrapper(TTabletStorageInfo *tabletInfo, TBootstrapperInfo *bootstrapperInfo, bool standby = false); -TActorId MakeBootstrapperID(ui64 tablet, ui32 node); +TActorId MakeBootstrapperID(ui64 tablet, ui32 node); } diff --git a/ydb/core/tablet/node_tablet_monitor.cpp b/ydb/core/tablet/node_tablet_monitor.cpp index 489a8c82e8b..8bb34e7ea0b 100644 --- a/ydb/core/tablet/node_tablet_monitor.cpp +++ b/ydb/core/tablet/node_tablet_monitor.cpp @@ -5,7 +5,7 @@ #include <ydb/core/mon/mon.h> #include <library/cpp/actors/core/mon.h> #include <ydb/core/base/appdata.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <library/cpp/actors/core/interconnect.h> #include <util/generic/algorithm.h> #include <ydb/core/base/tablet_types.h> @@ -24,12 +24,12 @@ public: return NKikimrServices::TActivity::TABLET_FORWARDING_ACTOR; } - TNodeList(const TActorId &sender) + TNodeList(const TActorId &sender) : Sender(sender) {} void Bootstrap(const TActorContext& ctx) { - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); ctx.Send(nameserviceId, new TEvInterconnect::TEvListNodes()); ctx.Schedule(TDuration::Seconds(60), new TEvents::TEvWakeup()); Become(&TThis::StateRequestedBrowse); @@ -126,7 +126,7 @@ public: } protected: - TActorId Sender; + TActorId Sender; TAutoPtr<TEvInterconnect::TEvNodesInfo> NodesInfo; }; @@ -136,7 +136,7 @@ public: return NKikimrServices::TActivity::TABLET_FORWARDING_ACTOR; } - TTabletList(const TActorId &sender, ui32 filterNodeId, + TTabletList(const TActorId &sender, ui32 filterNodeId, const TIntrusivePtr<ITabletStateClassifier>& stateClassifier, const TIntrusivePtr<ITabletListRenderer>& renderer) : Sender(sender) @@ -148,7 +148,7 @@ public: {} void Bootstrap(const TActorContext& ctx) { - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); ctx.Send(nameserviceId, new TEvInterconnect::TEvListNodes()); ctx.Schedule(TDuration::Seconds(60), new TEvents::TEvWakeup()); Become(&TThis::StateRequestedBrowse); @@ -173,12 +173,12 @@ public: NodesInfo = ev->Release(); if (!NodesInfo->Nodes.empty()) { if (FilterNodeId) { - TActorId tabletStateActorId = NNodeWhiteboard::MakeNodeWhiteboardServiceId(FilterNodeId); + TActorId tabletStateActorId = NNodeWhiteboard::MakeNodeWhiteboardServiceId(FilterNodeId); ctx.Send(tabletStateActorId, new TEvWhiteboard::TEvTabletStateRequest(), IEventHandle::FlagTrackDelivery, FilterNodeId); ++NodesRequested; } else { for (const auto& ni : NodesInfo->Nodes) { - TActorId tabletStateActorId = NNodeWhiteboard::MakeNodeWhiteboardServiceId(ni.NodeId); + TActorId tabletStateActorId = NNodeWhiteboard::MakeNodeWhiteboardServiceId(ni.NodeId); ctx.Send(tabletStateActorId, new TEvWhiteboard::TEvTabletStateRequest(), IEventHandle::FlagTrackDelivery, ni.NodeId); ++NodesRequested; } @@ -284,7 +284,7 @@ public: } protected: - TActorId Sender; + TActorId Sender; TAutoPtr<TEvInterconnect::TEvNodesInfo> NodesInfo; TMap<ui64, TAutoPtr<TEvWhiteboard::TEvTabletStateResponse>> PerNodeTabletInfo; size_t NodesRequested; @@ -301,13 +301,13 @@ public: return NKikimrServices::TActivity::TABLET_FORWARDING_ACTOR; } - TStateStorageTabletList(const TActorId &sender, ui32 stateStorageId) + TStateStorageTabletList(const TActorId &sender, ui32 stateStorageId) : Sender(sender) , StateStorageId(stateStorageId) {} void Bootstrap(const TActorContext& ctx) { - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); ctx.Send(nameserviceId, new TEvInterconnect::TEvListNodes()); ctx.Schedule(TDuration::Seconds(60), new TEvents::TEvWakeup()); Become(&TThis::StateRequestedBrowse); @@ -323,7 +323,7 @@ public: void Handle(TEvInterconnect::TEvNodesInfo::TPtr &ev, const TActorContext &ctx) { NodesInfo = ev->Release(); - const TActorId proxyActorID = MakeStateStorageProxyID(StateStorageId); + const TActorId proxyActorID = MakeStateStorageProxyID(StateStorageId); ctx.Send(proxyActorID, new TEvStateStorage::TEvRequestReplicasDumps()); } @@ -404,7 +404,7 @@ public: } protected: - TActorId Sender; + TActorId Sender; ui32 StateStorageId; TAutoPtr<TEvInterconnect::TEvNodesInfo> NodesInfo; }; diff --git a/ydb/core/tablet/node_tablet_monitor.h b/ydb/core/tablet/node_tablet_monitor.h index 8e3711b07a3..0fe88912003 100644 --- a/ydb/core/tablet/node_tablet_monitor.h +++ b/ydb/core/tablet/node_tablet_monitor.h @@ -13,9 +13,9 @@ namespace NKikimr { namespace NNodeTabletMonitor { -inline TActorId MakeNodeTabletMonitorID(ui32 node = 0) { +inline TActorId MakeNodeTabletMonitorID(ui32 node = 0) { char x[12] = {'n','o','d','e','t','a','b','l','e','t','m','o'}; - return TActorId(node, TStringBuf(x, 12)); + return TActorId(node, TStringBuf(x, 12)); } diff --git a/ydb/core/tablet/resource_broker.cpp b/ydb/core/tablet/resource_broker.cpp index bfa41b1bdc0..fff30b26010 100644 --- a/ydb/core/tablet/resource_broker.cpp +++ b/ydb/core/tablet/resource_broker.cpp @@ -51,7 +51,7 @@ void TResourceLimit::ReleaseResources(const TResourceValues &values) } TTask::TTask(const TEvResourceBroker::TTask &task, - const TActorId &client, + const TActorId &client, TInstant timestamp, TTaskCountersPtr counters) : TEvResourceBroker::TTask(task) @@ -380,7 +380,7 @@ const TTask* TScheduler::FindTask(ui64 taskId, const TActorId &client) const { } bool TScheduler::SubmitTask(const TEvResourceBroker::TTask &task, - const TActorId &client, + const TActorId &client, const TActorSystem &as) { auto &config = TaskConfig(task.Type); @@ -418,7 +418,7 @@ bool TScheduler::SubmitTask(const TEvResourceBroker::TTask &task, } bool TScheduler::UpdateTask(ui64 taskId, - const TActorId &client, + const TActorId &client, const TResourceValues &requiredResources, ui64 priority, const TString &type, @@ -454,7 +454,7 @@ bool TScheduler::UpdateTask(ui64 taskId, } bool TScheduler::UpdateTaskCookie(ui64 taskId, - const TActorId &client, + const TActorId &client, TIntrusivePtr<TThrRefBase> cookie, const TActorSystem &as) { diff --git a/ydb/core/tablet/resource_broker.h b/ydb/core/tablet/resource_broker.h index 2811a027088..095702cc19a 100644 --- a/ydb/core/tablet/resource_broker.h +++ b/ydb/core/tablet/resource_broker.h @@ -7,7 +7,7 @@ #include <library/cpp/actors/core/defs.h> #include <library/cpp/actors/core/actor.h> #include <library/cpp/actors/core/event.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <array> @@ -17,9 +17,9 @@ namespace NResourceBroker { constexpr size_t LAST_RESOURCE = NKikimrResourceBroker::MEMORY; constexpr size_t RESOURCE_COUNT = LAST_RESOURCE + 1; -inline TActorId MakeResourceBrokerID(ui32 node = 0) { +inline TActorId MakeResourceBrokerID(ui32 node = 0) { char x[12] = {'r','e','s','o','u','r','c','e','b','r','o','k'}; - return TActorId(node, TStringBuf(x, 12)); + return TActorId(node, TStringBuf(x, 12)); } using TResourceValues = std::array<ui64, RESOURCE_COUNT>; diff --git a/ydb/core/tablet/resource_broker_impl.h b/ydb/core/tablet/resource_broker_impl.h index 4ad988fa2ba..2c781692f60 100644 --- a/ydb/core/tablet/resource_broker_impl.h +++ b/ydb/core/tablet/resource_broker_impl.h @@ -46,7 +46,7 @@ using TResourceLimitConstPtr = TIntrusiveConstPtr<TResourceLimit>; class TTask : public TThrRefBase, public TEvResourceBroker::TTask { public: - TTask(const TEvResourceBroker::TTask &task, const TActorId &client, + TTask(const TEvResourceBroker::TTask &task, const TActorId &client, TInstant timestamp, TTaskCountersPtr counters); // Get string with task's name, ID and client info. @@ -68,7 +68,7 @@ public: public: // Actor which submitted task. - TActorId Client; + TActorId Client; // Task queue task is currently attached to. TTaskQueuePtr Queue; // True if task has allocated resources. @@ -315,7 +315,7 @@ public: /** * Update queued task. Return true on success and false if task not found. */ - bool UpdateTask(ui64 taskId, const TActorId &client, const TResourceValues &requiredResources, + bool UpdateTask(ui64 taskId, const TActorId &client, const TResourceValues &requiredResources, ui64 priority, const TString &type, bool resubmit, const TActorSystem &as); /** * Update cookie for submitted task. Return true on success and false if task not found. @@ -391,7 +391,7 @@ private: THashMap<TString, TTaskQueuePtr> Queues; THashMap<TString, TTaskConfig> TaskConfigs; TResourceLimitPtr ResourceLimit; - THashMap<std::pair<TActorId, ui64>, TTaskPtr> Tasks; + THashMap<std::pair<TActorId, ui64>, TTaskPtr> Tasks; const NMonitoring::TDynamicCounterPtr Counters; TQueueCountersPtr TotalCounters; NMonitoring::TDynamicCounters::TCounterPtr MissingTaskTypeCounter; diff --git a/ydb/core/tablet/resource_broker_ut.cpp b/ydb/core/tablet/resource_broker_ut.cpp index de3fb829b42..5948509025b 100644 --- a/ydb/core/tablet/resource_broker_ut.cpp +++ b/ydb/core/tablet/resource_broker_ut.cpp @@ -80,7 +80,7 @@ WaitForBootstrap(TTestActorRuntime &runtime) } static void -SubmitTask(TTestActorRuntime &runtime, TActorId broker, TActorId sender, +SubmitTask(TTestActorRuntime &runtime, TActorId broker, TActorId sender, ui64 id, ui64 cpu, ui64 memory, const TString &type, ui32 priority, TIntrusivePtr<TThrRefBase> cookie = nullptr) { @@ -95,7 +95,7 @@ SubmitTask(TTestActorRuntime &runtime, TActorId broker, TActorId sender, } static void -UpdateTask(TTestActorRuntime &runtime, TActorId broker, TActorId sender, +UpdateTask(TTestActorRuntime &runtime, TActorId broker, TActorId sender, ui64 id, ui64 cpu, ui64 memory, ui32 priority, const TString &type, bool resubmit = false) { @@ -108,7 +108,7 @@ UpdateTask(TTestActorRuntime &runtime, TActorId broker, TActorId sender, } static void -UpdateTaskCookie(TTestActorRuntime &runtime, TActorId broker, TActorId sender, +UpdateTaskCookie(TTestActorRuntime &runtime, TActorId broker, TActorId sender, ui64 id, TIntrusivePtr<TThrRefBase> cookie) { TAutoPtr<TEvResourceBroker::TEvUpdateTaskCookie> event @@ -118,7 +118,7 @@ UpdateTaskCookie(TTestActorRuntime &runtime, TActorId broker, TActorId sender, } static void -RemoveTask(TTestActorRuntime &runtime, TActorId broker, TActorId sender, ui64 id) +RemoveTask(TTestActorRuntime &runtime, TActorId broker, TActorId sender, ui64 id) { TAutoPtr<TEvResourceBroker::TEvRemoveTask> event = new TEvResourceBroker::TEvRemoveTask(id); @@ -127,7 +127,7 @@ RemoveTask(TTestActorRuntime &runtime, TActorId broker, TActorId sender, ui64 id } static void -FinishTask(TTestActorRuntime &runtime, TActorId broker, TActorId sender, ui64 id) +FinishTask(TTestActorRuntime &runtime, TActorId broker, TActorId sender, ui64 id) { TAutoPtr<TEvResourceBroker::TEvFinishTask> event = new TEvResourceBroker::TEvFinishTask(id); @@ -170,7 +170,7 @@ CheckCounters(NMonitoring::TDynamicCounterPtr counters, const TString &group, co UNIT_ASSERT_VALUES_EQUAL(g->GetCounter("InFlyTasks")->Val(), infly); } -static void CheckConfigure(TTestActorRuntime &runtime, TActorId broker, TActorId sender, +static void CheckConfigure(TTestActorRuntime &runtime, TActorId broker, TActorId sender, const NKikimrResourceBroker::TResourceBrokerConfig &config, bool success) { @@ -200,8 +200,8 @@ Y_UNIT_TEST_SUITE(TResourceBroker) { auto now = Now(); runtime.UpdateCurrentTime(now); - TActorId sender1 = runtime.AllocateEdgeActor(); - TActorId sender2 = runtime.AllocateEdgeActor(); + TActorId sender1 = runtime.AllocateEdgeActor(); + TActorId sender2 = runtime.AllocateEdgeActor(); NMonitoring::TDynamicCounterPtr counters = MakeIntrusive<NMonitoring::TDynamicCounters>(); auto config = MakeTestConfig(); @@ -281,7 +281,7 @@ Y_UNIT_TEST_SUITE(TResourceBroker) { auto now = Now(); runtime.UpdateCurrentTime(now); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); NMonitoring::TDynamicCounterPtr counters = MakeIntrusive<NMonitoring::TDynamicCounters>(); auto config = MakeTestConfig(); @@ -319,7 +319,7 @@ Y_UNIT_TEST_SUITE(TResourceBroker) { auto now = Now(); runtime.UpdateCurrentTime(now); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); NMonitoring::TDynamicCounterPtr counters = MakeIntrusive<NMonitoring::TDynamicCounters>(); auto config = MakeTestConfig(); @@ -409,7 +409,7 @@ Y_UNIT_TEST_SUITE(TResourceBroker) { auto now = Now(); runtime.UpdateCurrentTime(now); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); NMonitoring::TDynamicCounterPtr counters = MakeIntrusive<NMonitoring::TDynamicCounters>(); auto config = MakeTestConfig(); @@ -458,7 +458,7 @@ Y_UNIT_TEST_SUITE(TResourceBroker) { auto now = Now(); runtime.UpdateCurrentTime(now); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); NMonitoring::TDynamicCounterPtr counters = MakeIntrusive<NMonitoring::TDynamicCounters>(); auto config = MakeTestConfig(); @@ -553,7 +553,7 @@ Y_UNIT_TEST_SUITE(TResourceBroker) { auto now = Now(); runtime.UpdateCurrentTime(now); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); NMonitoring::TDynamicCounterPtr counters = MakeIntrusive<NMonitoring::TDynamicCounters>(); auto config = MakeTestConfig(); @@ -611,7 +611,7 @@ Y_UNIT_TEST_SUITE(TResourceBroker) { auto now = Now(); runtime.UpdateCurrentTime(now); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); NMonitoring::TDynamicCounterPtr counters = MakeIntrusive<NMonitoring::TDynamicCounters>(); auto config = MakeTestConfig(); @@ -647,8 +647,8 @@ Y_UNIT_TEST_SUITE(TResourceBroker) { auto now = Now(); runtime.UpdateCurrentTime(now); - TActorId sender1 = runtime.AllocateEdgeActor(); - TActorId sender2 = runtime.AllocateEdgeActor(); + TActorId sender1 = runtime.AllocateEdgeActor(); + TActorId sender2 = runtime.AllocateEdgeActor(); NMonitoring::TDynamicCounterPtr counters = MakeIntrusive<NMonitoring::TDynamicCounters>(); auto config = MakeTestConfig(); @@ -686,7 +686,7 @@ Y_UNIT_TEST_SUITE(TResourceBroker) { auto now = Now(); runtime.UpdateCurrentTime(now); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); NMonitoring::TDynamicCounterPtr counters = MakeIntrusive<NMonitoring::TDynamicCounters>(); auto config = MakeTestConfig(); @@ -723,7 +723,7 @@ Y_UNIT_TEST_SUITE(TResourceBroker) { auto now = Now(); runtime.UpdateCurrentTime(now); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); NMonitoring::TDynamicCounterPtr counters = MakeIntrusive<NMonitoring::TDynamicCounters>(); auto config = MakeTestConfig(); @@ -760,7 +760,7 @@ Y_UNIT_TEST_SUITE(TResourceBroker) { auto now = Now(); runtime.UpdateCurrentTime(now); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); NMonitoring::TDynamicCounterPtr counters = MakeIntrusive<NMonitoring::TDynamicCounters>(); auto config = MakeTestConfig(); @@ -806,7 +806,7 @@ Y_UNIT_TEST_SUITE(TResourceBroker) { auto now = Now(); runtime.UpdateCurrentTime(now); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); NMonitoring::TDynamicCounterPtr counters = MakeIntrusive<NMonitoring::TDynamicCounters>(); auto config = MakeTestConfig(); diff --git a/ydb/core/tablet/tablet_counters.h b/ydb/core/tablet/tablet_counters.h index 508b0b833a1..a8cc27e9eda 100644 --- a/ydb/core/tablet/tablet_counters.h +++ b/ydb/core/tablet/tablet_counters.h @@ -3,7 +3,7 @@ #include <util/generic/singleton.h> #include <util/generic/vector.h> #include <library/cpp/deprecated/enum_codegen/enum_codegen.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <google/protobuf/descriptor.pb.h> #include <ydb/core/protos/counters.pb.h> #include <ydb/core/protos/tablet_counters.pb.h> diff --git a/ydb/core/tablet/tablet_counters_aggregator.cpp b/ydb/core/tablet/tablet_counters_aggregator.cpp index d061e44633a..1885b26e09c 100644 --- a/ydb/core/tablet/tablet_counters_aggregator.cpp +++ b/ydb/core/tablet/tablet_counters_aggregator.cpp @@ -16,8 +16,8 @@ #include <ydb/core/util/wildcard.h> #include <ydb/library/persqueue/topic_parser/topic_parser.h> -#include <library/cpp/monlib/service/pages/templates.h> -#include <library/cpp/monlib/dynamic_counters/encode.h> +#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/dynamic_counters/encode.h> #include <util/generic/xrange.h> #include <util/string/vector.h> @@ -33,10 +33,10 @@ namespace NKikimr { TActorId MakeTabletCountersAggregatorID(ui32 node, bool follower) { if (!follower) { char x[12] = {'t','a','b','l','c','o','u','n','t','a','g','g'}; - return TActorId(node, TStringBuf(x, 12)); + return TActorId(node, TStringBuf(x, 12)); } else { char x[12] ={'s','l','a','v','c','o','u','n','t','a','g','g'}; - return TActorId(node, TStringBuf(x, 12)); + return TActorId(node, TStringBuf(x, 12)); } } @@ -1587,7 +1587,7 @@ private: // TAutoPtr<TTabletMon> TabletMon; TActorId DbWatcherActorId; - THashMap<TActorId, std::pair<TActorId, TAutoPtr<NMon::TEvHttpInfo>>> HttpRequestHandlers; + THashMap<TActorId, std::pair<TActorId, TAutoPtr<NMon::TEvHttpInfo>>> HttpRequestHandlers; THashSet<ui32> TabletTypeOfReceivedLabeledCounters; bool Follower; }; @@ -1770,11 +1770,11 @@ TTabletCountersAggregatorActor::HandleWork(TEvTabletCounters::TEvTabletLabeledCo bool spack = params.Has("spack"); if (json) { oss << NMonitoring::HTTPOKJSON; - auto encoder = NMonitoring::CreateEncoder(&oss, NMonitoring::EFormat::JSON); - counters.Accept(TString(), TString(), *encoder); + auto encoder = NMonitoring::CreateEncoder(&oss, NMonitoring::EFormat::JSON); + counters.Accept(TString(), TString(), *encoder); } else if (spack) { oss.Write(NMonitoring::HTTPOKSPACK); - auto encoder = NMonitoring::CreateEncoder(&oss, NMonitoring::EFormat::SPACK); + auto encoder = NMonitoring::CreateEncoder(&oss, NMonitoring::EFormat::SPACK); counters.Accept(TString(), TString(), *encoder); } else { counters.OutputHtml(oss); @@ -1803,7 +1803,7 @@ TTabletCountersAggregatorActor::HandleWork(NMon::TEvHttpInfo::TPtr &ev, const TA continue; TString tabletTypeStr = TTabletTypes::TypeToStr((TTabletTypes::EType)tabletType); if (tabletTypeStr == reqTabletType) { - TActorId handler = CreateClusterLabeledCountersAggregator(ctx.SelfID, (TTabletTypes::EType)tabletType, ctx, 1, "", workers); + TActorId handler = CreateClusterLabeledCountersAggregator(ctx.SelfID, (TTabletTypes::EType)tabletType, ctx, 1, "", workers); HttpRequestHandlers.insert(std::make_pair(handler, std::make_pair(ev->Sender, ev->Release()))); return; } @@ -1894,7 +1894,7 @@ void PreProcessResponse(TEvTabletCounters::TEvTabletLabeledCountersResponse* res class TClusterLabeledCountersAggregatorActorV1 : public TActorBootstrapped<TClusterLabeledCountersAggregatorActorV1> { using TBase = TActorBootstrapped<TClusterLabeledCountersAggregatorActorV1>; - TActorId Initiator; + TActorId Initiator; TTabletTypes::EType TabletType; ui32 NodesRequested; ui32 NodesReceived; @@ -1910,7 +1910,7 @@ public: } // - TClusterLabeledCountersAggregatorActorV1(const TActorId& parentActor, const TTabletTypes::EType tabletType, ui32 numWorkers = 0, ui32 workerId = 0) + TClusterLabeledCountersAggregatorActorV1(const TActorId& parentActor, const TTabletTypes::EType tabletType, ui32 numWorkers = 0, ui32 workerId = 0) : Initiator(parentActor) , TabletType(tabletType) , NodesRequested(0) @@ -1921,7 +1921,7 @@ public: } void SendRequest(ui32 nodeId, const TActorContext &ctx) { - TActorId aggregatorServiceId = MakeTabletCountersAggregatorID(nodeId); + TActorId aggregatorServiceId = MakeTabletCountersAggregatorID(nodeId); TAutoPtr<TEvTabletCounters::TEvTabletLabeledCountersRequest> request(new TEvTabletCounters::TEvTabletLabeledCountersRequest()); request->Record.SetTabletType(TabletType); request->Record.SetVersion(1); @@ -1940,7 +1940,7 @@ public: void Bootstrap(const TActorContext& ctx) { if (NumWorkers > 0) { - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); ctx.Send(nameserviceId, new TEvInterconnect::TEvListNodes()); TBase::Become(&TThis::StateRequestedBrowse); ctx.Schedule(TDuration::Seconds(AGGREGATOR_TIMEOUT_SECONDS), new TEvents::TEvWakeup()); @@ -2163,7 +2163,7 @@ public: class TClusterLabeledCountersAggregatorActorV2 : public TActorBootstrapped<TClusterLabeledCountersAggregatorActorV2> { using TBase = TActorBootstrapped<TClusterLabeledCountersAggregatorActorV2>; - TActorId Initiator; + TActorId Initiator; TTabletTypes::EType TabletType; ui32 NodesRequested; ui32 NodesReceived; @@ -2184,7 +2184,7 @@ public: } // - TClusterLabeledCountersAggregatorActorV2(const TActorId& parentActor, const TTabletTypes::EType tabletType, const TString& group, ui32 numWorkers = 0, ui32 workerId = 0, bool newFormat = false) + TClusterLabeledCountersAggregatorActorV2(const TActorId& parentActor, const TTabletTypes::EType tabletType, const TString& group, ui32 numWorkers = 0, ui32 workerId = 0, bool newFormat = false) : Initiator(parentActor) , TabletType(tabletType) , NodesRequested(0) @@ -2198,7 +2198,7 @@ public: {} void SendRequest(ui32 nodeId, const TActorContext &ctx) { - TActorId aggregatorServiceId = MakeTabletCountersAggregatorID(nodeId); + TActorId aggregatorServiceId = MakeTabletCountersAggregatorID(nodeId); TAutoPtr<TEvTabletCounters::TEvTabletLabeledCountersRequest> request(new TEvTabletCounters::TEvTabletLabeledCountersRequest()); request->Record.SetVersion(2); request->Record.SetTabletType(TabletType); @@ -2304,7 +2304,7 @@ public: void Bootstrap(const TActorContext& ctx) { if (NumWorkers > 0) { - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); ctx.Send(nameserviceId, new TEvInterconnect::TEvListNodes()); TBase::Become(&TThis::StateRequestedBrowse); ctx.Schedule(TDuration::Seconds(AGGREGATOR_TIMEOUT_SECONDS), new TEvents::TEvWakeup()); @@ -2433,7 +2433,7 @@ public: } }; -IActor* CreateClusterLabeledCountersAggregatorActor(const TActorId& parentActor, TTabletTypes::EType tabletType, ui32 version, const TString& group, const ui32 totalWorkersCount) { +IActor* CreateClusterLabeledCountersAggregatorActor(const TActorId& parentActor, TTabletTypes::EType tabletType, ui32 version, const TString& group, const ui32 totalWorkersCount) { switch (version) { case 1: return new TClusterLabeledCountersAggregatorActorV1(parentActor, tabletType, totalWorkersCount == 0 ? 1 : 0, totalWorkersCount); @@ -2446,7 +2446,7 @@ IActor* CreateClusterLabeledCountersAggregatorActor(const TActorId& parentActor, } -TActorId CreateClusterLabeledCountersAggregator(const TActorId& parentActor, TTabletTypes::EType tabletType, const TActorContext& ctx, ui32 version, const TString& group, const ui32 totalWorkersCount) { +TActorId CreateClusterLabeledCountersAggregator(const TActorId& parentActor, TTabletTypes::EType tabletType, const TActorContext& ctx, ui32 version, const TString& group, const ui32 totalWorkersCount) { return ctx.Register(CreateClusterLabeledCountersAggregatorActor(parentActor, tabletType, version, group, totalWorkersCount), TMailboxType::HTSwap, AppData(ctx)->BatchPoolId); } diff --git a/ydb/core/tablet/tablet_counters_aggregator.h b/ydb/core/tablet/tablet_counters_aggregator.h index 37f1b00289d..db350d41cbf 100644 --- a/ydb/core/tablet/tablet_counters_aggregator.h +++ b/ydb/core/tablet/tablet_counters_aggregator.h @@ -132,15 +132,15 @@ IActor* CreateTabletCountersAggregator(bool follower); //////////////////////////////////////////// //will create actor that aggregate LabeledCounters from all nodes and reports them as TEvTabletLabeledCountersResponse to parentActor -TActorId CreateClusterLabeledCountersAggregator( - const TActorId& parentActor, +TActorId CreateClusterLabeledCountersAggregator( + const TActorId& parentActor, TTabletTypes::EType tabletType, const TActorContext& ctx, ui32 version = 1, const TString& group = TString(), const ui32 TotalWorkersCount = WORKERS_COUNT); IActor* CreateClusterLabeledCountersAggregatorActor( - const TActorId& parentActor, + const TActorId& parentActor, TTabletTypes::EType tabletType, ui32 version = 1, const TString& group = TString(), const ui32 TotalWorkersCount = WORKERS_COUNT); diff --git a/ydb/core/tablet/tablet_counters_aggregator_ut.cpp b/ydb/core/tablet/tablet_counters_aggregator_ut.cpp index a0c8e802916..707ffb977ee 100644 --- a/ydb/core/tablet/tablet_counters_aggregator_ut.cpp +++ b/ydb/core/tablet/tablet_counters_aggregator_ut.cpp @@ -12,15 +12,15 @@ void TestHeavy(const ui32 v, ui32 numWorkers) { TInstant t(Now()); - TVector<TActorId> cc; - TActorId aggregatorId; + TVector<TActorId> cc; + TActorId aggregatorId; TTestBasicRuntime runtime(1); constexpr int NODES = 10; constexpr int GROUPS = 1000; constexpr int VALUES = 20; runtime.Initialize(TAppPrepare().Unwrap()); - TActorId edge = runtime.AllocateEdgeActor(); + TActorId edge = runtime.AllocateEdgeActor(); runtime.SetLogPriority(NKikimrServices::TABLET_AGGREGATOR, NActors::NLog::PRI_DEBUG); @@ -32,7 +32,7 @@ void TestHeavy(const ui32 v, ui32 numWorkers) { ++numWorkers; } - runtime.SetRegistrationObserverFunc([&cc, &aggregatorId](TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId) { + runtime.SetRegistrationObserverFunc([&cc, &aggregatorId](TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId) { TTestActorRuntime::DefaultRegistrationObserver(runtime, parentId, actorId); if (parentId == aggregatorId) { cc.push_back(actorId); @@ -87,18 +87,18 @@ void TestHeavy(const ui32 v, ui32 numWorkers) { Y_UNIT_TEST_SUITE(TTabletCountersAggregator) { Y_UNIT_TEST(SimpleAggregation) { - TVector<TActorId> cc; - TActorId aggregatorId; + TVector<TActorId> cc; + TActorId aggregatorId; TTestBasicRuntime runtime(1); runtime.Initialize(TAppPrepare().Unwrap()); - TActorId edge = runtime.AllocateEdgeActor(); + TActorId edge = runtime.AllocateEdgeActor(); IActor* aggregator = CreateClusterLabeledCountersAggregatorActor(edge, TTabletTypes::PersQueue, 2, TString(), 3); aggregatorId = runtime.Register(aggregator); - runtime.SetRegistrationObserverFunc([&cc, &aggregatorId](TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId) { + runtime.SetRegistrationObserverFunc([&cc, &aggregatorId](TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId) { TTestActorRuntime::DefaultRegistrationObserver(runtime, parentId, actorId); if (parentId == aggregatorId) { cc.push_back(actorId); @@ -185,18 +185,18 @@ Y_UNIT_TEST_SUITE(TTabletCountersAggregator) { } Y_UNIT_TEST(Version3Aggregation) { - TVector<TActorId> cc; - TActorId aggregatorId; + TVector<TActorId> cc; + TActorId aggregatorId; TTestBasicRuntime runtime(1); runtime.Initialize(TAppPrepare().Unwrap()); - TActorId edge = runtime.AllocateEdgeActor(); + TActorId edge = runtime.AllocateEdgeActor(); IActor* aggregator = CreateClusterLabeledCountersAggregatorActor(edge, TTabletTypes::PersQueue, 3, "rt3.*--*,cons*/*/rt.*--*", 3); aggregatorId = runtime.Register(aggregator); - runtime.SetRegistrationObserverFunc([&cc, &aggregatorId](TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId) { + runtime.SetRegistrationObserverFunc([&cc, &aggregatorId](TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId) { TTestActorRuntime::DefaultRegistrationObserver(runtime, parentId, actorId); if (parentId == aggregatorId) { cc.push_back(actorId); diff --git a/ydb/core/tablet/tablet_impl.h b/ydb/core/tablet/tablet_impl.h index 00ae140b0c4..b47b1f10971 100644 --- a/ydb/core/tablet/tablet_impl.h +++ b/ydb/core/tablet/tablet_impl.h @@ -14,10 +14,10 @@ namespace NKikimr { IActor* CreateTabletReqRebuildHistoryGraph(const TActorId &owner, TTabletStorageInfo *info, ui32 blockedGen, NTracing::ITrace *trace, ui64 followerCookie); -IActor* CreateTabletFindLastEntry(const TActorId &owner, bool readBody, TTabletStorageInfo *info, ui32 blockedGen); -IActor* CreateTabletReqWriteLog(const TActorId &owner, const TLogoBlobID &entryId, NKikimrTabletBase::TTabletLogEntry *entry, TVector<TEvTablet::TLogEntryReference> &refs, TEvBlobStorage::TEvPut::ETactic commitTactic, TTabletStorageInfo *info); -IActor* CreateTabletReqBlockBlobStorage(const TActorId &owner, TTabletStorageInfo *info, ui32 generation, bool blockPrevEntry); -IActor* CreateTabletReqDelete(const TActorId &owner, const TIntrusivePtr<TTabletStorageInfo> &tabletStorageInfo, ui32 generation = std::numeric_limits<ui32>::max()); +IActor* CreateTabletFindLastEntry(const TActorId &owner, bool readBody, TTabletStorageInfo *info, ui32 blockedGen); +IActor* CreateTabletReqWriteLog(const TActorId &owner, const TLogoBlobID &entryId, NKikimrTabletBase::TTabletLogEntry *entry, TVector<TEvTablet::TLogEntryReference> &refs, TEvBlobStorage::TEvPut::ETactic commitTactic, TTabletStorageInfo *info); +IActor* CreateTabletReqBlockBlobStorage(const TActorId &owner, TTabletStorageInfo *info, ui32 generation, bool blockPrevEntry); +IActor* CreateTabletReqDelete(const TActorId &owner, const TIntrusivePtr<TTabletStorageInfo> &tabletStorageInfo, ui32 generation = std::numeric_limits<ui32>::max()); struct TEvTabletBase { enum EEv { diff --git a/ydb/core/tablet/tablet_list_renderer.cpp b/ydb/core/tablet/tablet_list_renderer.cpp index bd6a96ef870..ad36f12a291 100644 --- a/ydb/core/tablet/tablet_list_renderer.cpp +++ b/ydb/core/tablet/tablet_list_renderer.cpp @@ -3,7 +3,7 @@ #include <ydb/core/mon/mon.h> #include <library/cpp/actors/core/mon.h> #include <ydb/core/tablet/tablet_sys.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/generic/algorithm.h> diff --git a/ydb/core/tablet/tablet_metrics.h b/ydb/core/tablet/tablet_metrics.h index 7b4a7fd36d5..b426284d33f 100644 --- a/ydb/core/tablet/tablet_metrics.h +++ b/ydb/core/tablet/tablet_metrics.h @@ -68,7 +68,7 @@ protected: const ui64 TabletId; const ui32 FollowerId; - const TActorId Launcher; + const TActorId Launcher; ui32 LevelCPU = 0; ui32 LevelMemory = 0; ui32 LevelNetwork = 0; diff --git a/ydb/core/tablet/tablet_monitoring_proxy.cpp b/ydb/core/tablet/tablet_monitoring_proxy.cpp index ebf3828f638..a61a43b7e5c 100644 --- a/ydb/core/tablet/tablet_monitoring_proxy.cpp +++ b/ydb/core/tablet/tablet_monitoring_proxy.cpp @@ -10,7 +10,7 @@ #include <ydb/core/node_whiteboard/node_whiteboard.h> #include <ydb/core/base/tablet.h> #include <ydb/core/tx/tx.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/string/builder.h> //////////////////////////////////////////// @@ -113,9 +113,9 @@ private: const TTabletMonitoringProxyConfig Config; const ui64 TargetTablet; const bool ForceFollower; - const TActorId Sender; + const TActorId Sender; const TString Query; - TActorId PipeClient; + TActorId PipeClient; const HTTP_METHOD Method; }; diff --git a/ydb/core/tablet/tablet_monitoring_proxy.h b/ydb/core/tablet/tablet_monitoring_proxy.h index df9461fd3b8..07fd7888db3 100644 --- a/ydb/core/tablet/tablet_monitoring_proxy.h +++ b/ydb/core/tablet/tablet_monitoring_proxy.h @@ -23,9 +23,9 @@ struct TTabletMonitoringProxyConfig { }; // -inline TActorId MakeTabletMonitoringProxyID(ui32 node = 0) { +inline TActorId MakeTabletMonitoringProxyID(ui32 node = 0) { char x[12] = {'t','a','b','l','m','o','n','p','r','o','x','y'}; - return TActorId(node, TStringBuf(x, 12)); + return TActorId(node, TStringBuf(x, 12)); } // diff --git a/ydb/core/tablet/tablet_pipe_client.cpp b/ydb/core/tablet/tablet_pipe_client.cpp index 8abccc77cfb..2fa35af9a0e 100644 --- a/ydb/core/tablet/tablet_pipe_client.cpp +++ b/ydb/core/tablet/tablet_pipe_client.cpp @@ -32,7 +32,7 @@ namespace NTabletPipe { return NKikimrServices::TActivity::TABLET_PIPE_CLIENT; } - TClient(const TActorId& owner, ui64 tabletId, const TClientConfig& config) + TClient(const TActorId& owner, ui64 tabletId, const TClientConfig& config) : Owner(owner) , TabletId(tabletId) , Config(config) @@ -281,7 +281,7 @@ namespace NTabletPipe { const auto &record = ev->Get()->Record; Y_VERIFY(record.GetTabletId() == TabletId); - ServerId = ActorIdFromProto(record.GetServerId()); + ServerId = ActorIdFromProto(record.GetServerId()); Leader = record.GetLeader(); Y_VERIFY(!ServerId || record.GetStatus() == NKikimrProto::OK); @@ -662,7 +662,7 @@ namespace NTabletPipe { } private: - const TActorId Owner; + const TActorId Owner; const ui64 TabletId; const TClientConfig Config; bool IsShutdown; @@ -673,22 +673,22 @@ namespace NTabletPipe { ui64 InterconnectCookie = 0; ui64 ConnectCookie = 0; TActorId InterconnectProxyId; - TActorId InterconnectSessionId; - TActorId ServerId; + TActorId InterconnectSessionId; + TActorId ServerId; typedef TOneOneQueueInplace<IEventHandle*, 32> TPayloadQueue; TAutoPtr<TPayloadQueue, TPayloadQueue::TPtrCleanDestructor> PayloadQueue; TClientRetryState RetryState; bool Leader; - TActorId HiveClient; + TActorId HiveClient; ui32 CurrentHiveForwards = 0; static constexpr ui32 MAX_HIVE_FORWARDS = 10; }; - IActor* CreateClient(const TActorId& owner, ui64 tabletId, const TClientConfig& config) { + IActor* CreateClient(const TActorId& owner, ui64 tabletId, const TClientConfig& config) { return new TClient(owner, tabletId, config); } - void SendData(TActorId self, TActorId clientId, IEventBase *payload, ui64 cookie) { + void SendData(TActorId self, TActorId clientId, IEventBase *payload, ui64 cookie) { auto ev = new IEventHandle(clientId, self, payload, 0, cookie); ev->Rewrite(TEvTabletPipe::EvSend, clientId); TActivationContext::Send(ev); @@ -705,40 +705,40 @@ namespace NTabletPipe { TActivationContext::Send(ev.Release()); } - void SendData(TActorId self, TActorId clientId, ui32 eventType, TIntrusivePtr<TEventSerializedData> buffer, ui64 cookie) { + void SendData(TActorId self, TActorId clientId, ui32 eventType, TIntrusivePtr<TEventSerializedData> buffer, ui64 cookie) { auto ev = new IEventHandle(eventType, 0, clientId, self, buffer, cookie); ev->Rewrite(TEvTabletPipe::EvSend, clientId); TActivationContext::Send(ev); } - void SendData(const TActorContext& ctx, const TActorId& clientId, IEventBase* payload, ui64 cookie) { + void SendData(const TActorContext& ctx, const TActorId& clientId, IEventBase* payload, ui64 cookie) { auto ev = new IEventHandle(clientId, ctx.SelfID, payload, 0, cookie); ev->Rewrite(TEvTabletPipe::EvSend, clientId); ctx.ExecutorThread.Send(ev); } - void SendData(const TActorContext& ctx, const TActorId& clientId, ui32 eventType, TIntrusivePtr<TEventSerializedData> buffer, ui64 cookie) { + void SendData(const TActorContext& ctx, const TActorId& clientId, ui32 eventType, TIntrusivePtr<TEventSerializedData> buffer, ui64 cookie) { auto ev = new IEventHandle(eventType, 0, clientId, ctx.SelfID, buffer, cookie); ev->Rewrite(TEvTabletPipe::EvSend, clientId); ctx.ExecutorThread.Send(ev); } - void ShutdownClient(const TActorContext& ctx, const TActorId& clientId) { + void ShutdownClient(const TActorContext& ctx, const TActorId& clientId) { ctx.Send(clientId, new TEvTabletPipe::TEvShutdown); } - void CloseClient(const TActorContext& ctx, const TActorId& clientId) { + void CloseClient(const TActorContext& ctx, const TActorId& clientId) { ctx.Send(clientId, new TEvents::TEvPoisonPill); } - void CloseClient(TActorIdentity self, TActorId clientId) { + void CloseClient(TActorIdentity self, TActorId clientId) { self.Send(clientId, new TEvents::TEvPoisonPill()); } - void CloseAndForgetClient(TActorIdentity self, TActorId &clientId) { + void CloseAndForgetClient(TActorIdentity self, TActorId &clientId) { if (clientId) { CloseClient(self, clientId); - clientId = TActorId(); + clientId = TActorId(); } } diff --git a/ydb/core/tablet/tablet_pipe_client_cache.cpp b/ydb/core/tablet/tablet_pipe_client_cache.cpp index 68de157d25a..47b81e45681 100644 --- a/ydb/core/tablet/tablet_pipe_client_cache.cpp +++ b/ydb/core/tablet/tablet_pipe_client_cache.cpp @@ -16,7 +16,7 @@ namespace NTabletPipe { ShutdownRequested = 1 << 1, }; - TActorId Client; + TActorId Client; ui32 Flags; TClientCacheEntry() @@ -24,7 +24,7 @@ namespace NTabletPipe { { } - TClientCacheEntry(const TActorId& client, ui32 flags) + TClientCacheEntry(const TActorId& client, ui32 flags) : Client(client) , Flags(flags) { @@ -69,7 +69,7 @@ namespace NTabletPipe { ActorSystem->Send(value.Client, new TEvents::TEvPoisonPill); } - TActorId Prepare(const TActorContext& ctx, ui64 tabletId) override { + TActorId Prepare(const TActorContext& ctx, ui64 tabletId) override { TClientCacheEntry* currentClient; if (Container->Find(tabletId, currentClient)) { currentClient->Flags &= ~TClientCacheEntry::ShutdownRequested; @@ -80,7 +80,7 @@ namespace NTabletPipe { if (PoolContainer->Find(tabletId, currentClient)) { TClientCacheEntry* insertedClient; Container->Insert(tabletId, *currentClient, insertedClient); - currentClient->Client = TActorId(); + currentClient->Client = TActorId(); insertedClient->Flags &= ~TClientCacheEntry::ShutdownRequested; return insertedClient->Client; } @@ -98,13 +98,13 @@ namespace NTabletPipe { return clientId; } - TActorId Send(const TActorContext& ctx, ui64 tabletId, IEventBase* payload, ui64 cookie) override { + TActorId Send(const TActorContext& ctx, ui64 tabletId, IEventBase* payload, ui64 cookie) override { auto clientId = Prepare(ctx, tabletId); SendData(ctx, clientId, payload, cookie); return clientId; } - TActorId Send(const TActorContext& ctx, ui64 tabletId, ui32 eventType, const TIntrusivePtr<TEventSerializedData>& eventBuffer, ui64 cookie) override { + TActorId Send(const TActorContext& ctx, ui64 tabletId, ui32 eventType, const TIntrusivePtr<TEventSerializedData>& eventBuffer, ui64 cookie) override { auto clientId = Prepare(ctx, tabletId); SendData(ctx, clientId, eventType, eventBuffer, cookie); return clientId; @@ -152,7 +152,7 @@ namespace NTabletPipe { if (!Container->Find(tabletId, currentClient)) return; - TActorId client = currentClient->Client; + TActorId client = currentClient->Client; CloseClient(ctx, client); Container->Erase(tabletId); } @@ -162,7 +162,7 @@ namespace NTabletPipe { if (!Container->Find(tabletId, currentClient)) return; - TActorId client = currentClient->Client; + TActorId client = currentClient->Client; if (!PoolContainer) { CloseClient(ctx, client); Container->Erase(tabletId); @@ -191,7 +191,7 @@ namespace NTabletPipe { Container->Erase(tabletId); } - void Erase(ui64 tabletId, const TActorId& clientId) { + void Erase(ui64 tabletId, const TActorId& clientId) { TClientCacheEntry* currentClient; if (Container->Find(tabletId, currentClient)) { Y_VERIFY_DEBUG(!!currentClient->Client); diff --git a/ydb/core/tablet/tablet_pipe_client_cache.h b/ydb/core/tablet/tablet_pipe_client_cache.h index 8c44e1c0f74..a6548c20e95 100644 --- a/ydb/core/tablet/tablet_pipe_client_cache.h +++ b/ydb/core/tablet/tablet_pipe_client_cache.h @@ -21,9 +21,9 @@ namespace NTabletPipe { class IClientCache { public: virtual ~IClientCache() {} - virtual TActorId Prepare(const TActorContext& ctx, ui64 tabletId) = 0; - virtual TActorId Send(const TActorContext& ctx, ui64 tabletId, ui32 eventType, const TIntrusivePtr<TEventSerializedData>& eventBuffer, ui64 cookie = 0) = 0; - virtual TActorId Send(const TActorContext& ctx, ui64 tabletId, IEventBase* payload, ui64 cookie = 0) = 0; + virtual TActorId Prepare(const TActorContext& ctx, ui64 tabletId) = 0; + virtual TActorId Send(const TActorContext& ctx, ui64 tabletId, ui32 eventType, const TIntrusivePtr<TEventSerializedData>& eventBuffer, ui64 cookie = 0) = 0; + virtual TActorId Send(const TActorContext& ctx, ui64 tabletId, IEventBase* payload, ui64 cookie = 0) = 0; virtual void Detach(const TActorContext& ctx) = 0; // returns true if connect was successfull virtual bool OnConnect(TEvTabletPipe::TEvClientConnected::TPtr& ev) = 0; diff --git a/ydb/core/tablet/tablet_pipe_server.cpp b/ydb/core/tablet/tablet_pipe_server.cpp index cd7acad9af8..b7c669e72cc 100644 --- a/ydb/core/tablet/tablet_pipe_server.cpp +++ b/ydb/core/tablet/tablet_pipe_server.cpp @@ -64,7 +64,7 @@ namespace NTabletPipe { void Handle(TEvTabletPipe::TEvPush::TPtr& ev, const TActorContext& ctx) { Y_VERIFY(ev->Get()->Record.GetTabletId() == TabletId); const auto& record = ev->Get()->Record; - const TActorId sender = ActorIdFromProto(record.GetSender()); + const TActorId sender = ActorIdFromProto(record.GetSender()); LOG_DEBUG_S(ctx, NKikimrServices::PIPE_SERVER, "[" << TabletId << "]" << " Push Sender# " << sender << " EventType# " << record.GetType() << (HadShutdown ? " ignored after shutdown" : "")); @@ -241,10 +241,10 @@ namespace NTabletPipe { private: const ui64 TabletId; - const TActorId ClientId; - TActorId OwnerId; - TActorId RecipientId; - TActorId InterconnectSession; + const TActorId ClientId; + TActorId OwnerId; + TActorId RecipientId; + TActorId InterconnectSession; ui64 MaxForwardedSeqNo = 0; const ui64 ConnectCookie; const ui32 Features; @@ -265,9 +265,9 @@ namespace NTabletPipe { TActorId Accept(TEvTabletPipe::TEvConnect::TPtr &ev, TActorIdentity owner, TActorId recipientId, bool leader) override { Y_VERIFY(ev->Get()->Record.GetTabletId() == TabletId); - const TActorId clientId = ActorIdFromProto(ev->Get()->Record.GetClientId()); + const TActorId clientId = ActorIdFromProto(ev->Get()->Record.GetClientId()); IActor* server = CreateServer(TabletId, clientId, ev->InterconnectSession, ev->Get()->Record.GetFeatures(), ev->Cookie); - TActorId serverId = TActivationContext::Register(server); + TActorId serverId = TActivationContext::Register(server); LOG_DEBUG_S(*TlsActivationContext, NKikimrServices::PIPE_SERVER, "[" << TabletId << "]" << " Accept Connect Originator# " << ev->Sender); ServerIds.insert(serverId); @@ -277,7 +277,7 @@ namespace NTabletPipe { void Reject(TEvTabletPipe::TEvConnect::TPtr &ev, TActorIdentity owner, NKikimrProto::EReplyStatus status, bool leader) override { Y_VERIFY(ev->Get()->Record.GetTabletId() == TabletId); - const TActorId clientId = ActorIdFromProto(ev->Get()->Record.GetClientId()); + const TActorId clientId = ActorIdFromProto(ev->Get()->Record.GetClientId()); LOG_DEBUG_S(*TlsActivationContext, NKikimrServices::PIPE_SERVER, "[" << TabletId << "]" << " Reject Connect Originator# " << ev->Sender); owner.Send(clientId, new TEvTabletPipe::TEvConnectResult(status, TabletId, clientId, TActorId(), leader)); @@ -302,12 +302,12 @@ namespace NTabletPipe { ActivatePending.clear(); } - TActorId Enqueue(TEvTabletPipe::TEvConnect::TPtr &ev, TActorIdentity owner) override { + TActorId Enqueue(TEvTabletPipe::TEvConnect::TPtr &ev, TActorIdentity owner) override { Y_UNUSED(owner); Y_VERIFY(ev->Get()->Record.GetTabletId() == TabletId); - const TActorId clientId = ActorIdFromProto(ev->Get()->Record.GetClientId()); + const TActorId clientId = ActorIdFromProto(ev->Get()->Record.GetClientId()); IActor* server = CreateServer(TabletId, clientId, ev->InterconnectSession, ev->Get()->Record.GetFeatures(), ev->Cookie); - TActorId serverId = TActivationContext::Register(server); + TActorId serverId = TActivationContext::Register(server); LOG_DEBUG_S(*TlsActivationContext, NKikimrServices::PIPE_SERVER, "[" << TabletId << "]" << " Enqueue Connect Originator# " << ev->Sender); ServerIds.insert(serverId); @@ -342,8 +342,8 @@ namespace NTabletPipe { private: private: const ui64 TabletId; - THashSet<TActorId> ServerIds; - THashSet<TActorId> ActivatePending; + THashSet<TActorId> ServerIds; + THashSet<TActorId> ActivatePending; bool Active; bool Stopped; }; @@ -360,7 +360,7 @@ namespace NTabletPipe { owner.Send(serverId, new TEvTabletPipe::TEvActivate(tabletId, owner, recipientId, leader)); } - void CloseServer(TActorIdentity owner, TActorId serverId) { + void CloseServer(TActorIdentity owner, TActorId serverId) { owner.Send(serverId, new TEvents::TEvPoisonPill); } } diff --git a/ydb/core/tablet/tablet_pipe_ut.cpp b/ydb/core/tablet/tablet_pipe_ut.cpp index 666ecccfebe..2a528070fb6 100644 --- a/ydb/core/tablet/tablet_pipe_ut.cpp +++ b/ydb/core/tablet/tablet_pipe_ut.cpp @@ -56,7 +56,7 @@ namespace NKikimr { class TProducerTablet : public TActor<TProducerTablet>, public NTabletFlatExecutor::TTabletExecutedFlat { public: - TProducerTablet(const TActorId &tablet, TTabletStorageInfo *info) + TProducerTablet(const TActorId &tablet, TTabletStorageInfo *info) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, nullptr) , HasData(false) @@ -128,7 +128,7 @@ namespace NKikimr { Send(ctx); } } else if (ClientId == ev->Get()->ClientId) { - ClientId = TActorId(); + ClientId = TActorId(); } } @@ -177,7 +177,7 @@ namespace NKikimr { } private: - TActorId ClientId; + TActorId ClientId; bool HasData; bool IsOpened; bool IsShutdown; @@ -208,7 +208,7 @@ namespace NKikimr { class TConsumerTablet : public TActor<TConsumerTablet>, public NTabletFlatExecutor::TTabletExecutedFlat { public: - TConsumerTablet(const TActorId &tablet, TTabletStorageInfo *info) + TConsumerTablet(const TActorId &tablet, TTabletStorageInfo *info) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, nullptr) , PipeConnectAcceptor(NTabletPipe::CreateConnectAcceptor(TabletID())) @@ -328,14 +328,14 @@ namespace NKikimr { THolder<NTabletPipe::IConnectAcceptor> PipeConnectAcceptor; bool RejectAll; - TActorId LastServerId; + TActorId LastServerId; ui32 ServerPipesOpened; ui32 ServerPipesClosed; }; class TConsumerTabletWithoutAcceptor : public TActor<TConsumerTabletWithoutAcceptor>, public NTabletFlatExecutor::TTabletExecutedFlat { public: - TConsumerTabletWithoutAcceptor(const TActorId &tablet, TTabletStorageInfo *info) + TConsumerTabletWithoutAcceptor(const TActorId &tablet, TTabletStorageInfo *info) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, nullptr) { @@ -404,7 +404,7 @@ Y_UNIT_TEST_SUITE(TTabletPipeTest) { TTestBasicRuntime runtime; SetupTabletServices(runtime); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); CreateTestBootstrapper(runtime, CreateTestTabletInfo(TTestTxConfig::TxTablet0, TTabletTypes::TX_DUMMY), [](const TActorId & tablet, TTabletStorageInfo* info) { return new TProducerTablet(tablet, info); }); @@ -444,7 +444,7 @@ Y_UNIT_TEST_SUITE(TTabletPipeTest) { TTestBasicRuntime runtime; SetupTabletServices(runtime); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); CreateTestBootstrapper(runtime, CreateTestTabletInfo(TTestTxConfig::TxTablet0, TTabletTypes::TX_DUMMY), [](const TActorId & tablet, TTabletStorageInfo* info) { return new TProducerTablet(tablet, info); }); @@ -473,7 +473,7 @@ Y_UNIT_TEST_SUITE(TTabletPipeTest) { runtime.SetLogPriority(NKikimrServices::PIPE_SERVER, NActors::NLog::PRI_DEBUG); runtime.SetLogPriority(NKikimrServices::PIPE_CLIENT, NActors::NLog::PRI_DEBUG); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); CreateTestBootstrapper(runtime, CreateTestTabletInfo(TTestTxConfig::TxTablet1, TTabletTypes::TX_DUMMY), [](const TActorId & tablet, TTabletStorageInfo* info) { return new TConsumerTablet(tablet, info); }); @@ -490,7 +490,7 @@ Y_UNIT_TEST_SUITE(TTabletPipeTest) { i32 i = 3; while (i --> 0) { auto client = NTabletPipe::CreateClient(sender, TTestTxConfig::TxTablet1, config); - TActorId clientId = runtime.Register(client); + TActorId clientId = runtime.Register(client); // We want to close the client right after it has sent EvConnect to the target tablet but before // the client received the EvConnectResult @@ -515,7 +515,7 @@ Y_UNIT_TEST_SUITE(TTabletPipeTest) { TTestBasicRuntime runtime; SetupTabletServices(runtime); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); CreateTestBootstrapper(runtime, CreateTestTabletInfo(TTestTxConfig::TxTablet0, TTabletTypes::TX_DUMMY), [](const TActorId & tablet, TTabletStorageInfo* info) { return new TProducerTablet(tablet, info); }); @@ -542,7 +542,7 @@ Y_UNIT_TEST_SUITE(TTabletPipeTest) { TTestBasicRuntime runtime; SetupTabletServices(runtime); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); CreateTestBootstrapper(runtime, CreateTestTabletInfo(TTestTxConfig::TxTablet0, TTabletTypes::TX_DUMMY), [](const TActorId & tablet, TTabletStorageInfo* info) { return new TProducerTablet(tablet, info); }); @@ -575,7 +575,7 @@ Y_UNIT_TEST_SUITE(TTabletPipeTest) { TTestBasicRuntime runtime; SetupTabletServices(runtime); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); TVector<ui64> tabletIds; tabletIds.push_back((ui64)TTestTxConfig::TxTablet0); tabletIds.push_back((ui64)TTestTxConfig::TxTablet1); @@ -625,7 +625,7 @@ Y_UNIT_TEST_SUITE(TTabletPipeTest) { TTestBasicRuntime runtime; SetupTabletServices(runtime); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); CreateTestBootstrapper(runtime, CreateTestTabletInfo(TTestTxConfig::TxTablet0, TTabletTypes::TX_DUMMY), [](const TActorId & tablet, TTabletStorageInfo* info) { return new TProducerTablet(tablet, info); }); @@ -659,7 +659,7 @@ Y_UNIT_TEST_SUITE(TTabletPipeTest) { TTestBasicRuntime runtime; SetupTabletServices(runtime); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); CreateTestBootstrapper(runtime, CreateTestTabletInfo(TTestTxConfig::TxTablet0, TTabletTypes::TX_DUMMY), [](const TActorId & tablet, TTabletStorageInfo* info) { return new TProducerTablet(tablet, info); }); @@ -686,7 +686,7 @@ Y_UNIT_TEST_SUITE(TTabletPipeTest) { TTestBasicRuntime runtime; SetupTabletServices(runtime); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); CreateTestBootstrapper(runtime, CreateTestTabletInfo(TTestTxConfig::TxTablet0, TTabletTypes::TX_DUMMY), [](const TActorId & tablet, TTabletStorageInfo* info) { return new TProducerTablet(tablet, info); }); @@ -719,7 +719,7 @@ Y_UNIT_TEST_SUITE(TTabletPipeTest) { TTestBasicRuntime runtime; SetupTabletServices(runtime); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); TVector<ui64> tabletIds; tabletIds.push_back((ui64)TTestTxConfig::TxTablet0); @@ -770,7 +770,7 @@ Y_UNIT_TEST_SUITE(TTabletPipeTest) { TTestBasicRuntime runtime; SetupTabletServices(runtime); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); CreateTestBootstrapper(runtime, CreateTestTabletInfo(TTestTxConfig::TxTablet0, TTabletTypes::TX_DUMMY), [](const TActorId & tablet, TTabletStorageInfo* info) { return new TProducerTablet(tablet, info); }); @@ -797,7 +797,7 @@ Y_UNIT_TEST_SUITE(TTabletPipeTest) { TTestBasicRuntime runtime(2); SetupTabletServices(runtime); - TActorId sender1 = runtime.AllocateEdgeActor(0); + TActorId sender1 = runtime.AllocateEdgeActor(0); CreateTestBootstrapper(runtime, CreateTestTabletInfo(TTestTxConfig::TxTablet0, TTabletTypes::TX_DUMMY), [](const TActorId & tablet, TTabletStorageInfo* info) { return new TProducerTablet(tablet, info); }, 0); @@ -830,7 +830,7 @@ Y_UNIT_TEST_SUITE(TTabletPipeTest) { TTestBasicRuntime runtime(2); SetupTabletServices(runtime); - TActorId sender1 = runtime.AllocateEdgeActor(0); + TActorId sender1 = runtime.AllocateEdgeActor(0); CreateTestBootstrapper(runtime, CreateTestTabletInfo(TTestTxConfig::TxTablet0, TTabletTypes::TX_DUMMY), [](const TActorId & tablet, TTabletStorageInfo* info) { return new TProducerTablet(tablet, info); }, 0); @@ -851,7 +851,7 @@ Y_UNIT_TEST_SUITE(TTabletPipeTest) { runtime.DispatchEvents(options); } - TActorId proxy = runtime.GetInterconnectProxy(0, 1); + TActorId proxy = runtime.GetInterconnectProxy(0, 1); Y_VERIFY(proxy); runtime.Send(new IEventHandle(proxy, sender1, new TEvInterconnect::TEvConnectNode), 0); TAutoPtr<IEventHandle> handle; @@ -877,7 +877,7 @@ Y_UNIT_TEST_SUITE(TTabletPipeTest) { Y_UNIT_TEST(TestSendBeforeBootTarget) { TTestBasicRuntime runtime; SetupTabletServices(runtime); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); TVector<ui64> tabletIds; tabletIds.push_back((ui64)TTestTxConfig::TxTablet0); tabletIds.push_back((ui64)TTestTxConfig::TxTablet1); @@ -935,7 +935,7 @@ Y_UNIT_TEST_SUITE(TTabletPipeTest) { TTestBasicRuntime runtime(2); SetupTabletServices(runtime); - TActorId sender1 = runtime.AllocateEdgeActor(0); + TActorId sender1 = runtime.AllocateEdgeActor(0); CreateTestBootstrapper(runtime, CreateTestTabletInfo(TTestTxConfig::TxTablet0, TTabletTypes::TX_DUMMY), [](const TActorId & tablet, TTabletStorageInfo* info) { return new TProducerTablet(tablet, info); }, 0); @@ -985,8 +985,8 @@ Y_UNIT_TEST_SUITE(TTabletPipeTest) { SetupTabletServices(runtime); runtime.SetLogPriority(NActorsServices::INTERCONNECT, NActors::NLog::PRI_DEBUG); - TActorId sender1 = runtime.AllocateEdgeActor(0); - TActorId sender2 = runtime.AllocateEdgeActor(1); + TActorId sender1 = runtime.AllocateEdgeActor(0); + TActorId sender2 = runtime.AllocateEdgeActor(1); TVector<ui64> tabletIds; tabletIds.push_back((ui64)TTestTxConfig::TxTablet0); @@ -1041,8 +1041,8 @@ Y_UNIT_TEST_SUITE(TTabletPipeTest) { TTestBasicRuntime runtime(2); SetupTabletServices(runtime); - TActorId edge1 = runtime.AllocateEdgeActor(0); - TActorId edge2 = runtime.AllocateEdgeActor(1); + TActorId edge1 = runtime.AllocateEdgeActor(0); + TActorId edge2 = runtime.AllocateEdgeActor(1); // Make an event that is from edge1 to edge2, but rewritten for edge1 auto event = new IEventHandle(edge2, edge1, new TEvents::TEvWakeup()); diff --git a/ydb/core/tablet/tablet_pipecache.cpp b/ydb/core/tablet/tablet_pipecache.cpp index 39aaa1ab617..292571f661c 100644 --- a/ydb/core/tablet/tablet_pipecache.cpp +++ b/ydb/core/tablet/tablet_pipecache.cpp @@ -58,7 +58,7 @@ class TPipePeNodeCache : public TActor<TPipePeNodeCache> { }; struct TClientState { - TActorId Client; + TActorId Client; THashMap<TActorId, ui64> Peers; ui64 LastSentSeqNo = 0; ui64 MaxForwardedSeqNo = Max<ui64>(); @@ -118,7 +118,7 @@ class TPipePeNodeCache : public TActor<TPipePeNodeCache> { } } - void UnlinkOne(TActorId peer, ui64 tablet) { + void UnlinkOne(TActorId peer, ui64 tablet) { auto *tabletState = FindTablet(tablet); if (Y_UNLIKELY(!tabletState)) return; @@ -152,7 +152,7 @@ class TPipePeNodeCache : public TActor<TPipePeNodeCache> { } } - void DropClient(ui64 tablet, TActorId client, bool notDelivered) { + void DropClient(ui64 tablet, TActorId client, bool notDelivered) { auto *tabletState = FindTablet(tablet); if (Y_UNLIKELY(!tabletState)) return; @@ -318,7 +318,7 @@ class TPipePeNodeCache : public TActor<TPipePeNodeCache> { void Handle(TEvPipeCache::TEvUnlink::TPtr &ev) { TEvPipeCache::TEvUnlink *msg = ev->Get(); const ui64 tablet = msg->TabletId; - const TActorId peer = ev->Sender; + const TActorId peer = ev->Sender; auto byPeerIt = ByPeer.find(peer); if (byPeerIt == ByPeer.end()) @@ -464,7 +464,7 @@ IActor* CreatePipePeNodeCache(const TIntrusivePtr<TPipePeNodeCacheConfig> &confi TActorId MakePipePeNodeCacheID(bool allowFollower) { char x[12] = "PipeCache"; x[9] = allowFollower ? 'F' : 'A'; - return TActorId(0, TStringBuf(x, 12)); + return TActorId(0, TStringBuf(x, 12)); } } diff --git a/ydb/core/tablet/tablet_req_blockbs.cpp b/ydb/core/tablet/tablet_req_blockbs.cpp index 5e8a231ece7..7e2e94ec95d 100644 --- a/ydb/core/tablet/tablet_req_blockbs.cpp +++ b/ydb/core/tablet/tablet_req_blockbs.cpp @@ -8,7 +8,7 @@ namespace NKikimr { constexpr ui32 MAX_ATTEMPTS = 3; class TTabletReqBlockBlobStorageGroup : public TActorBootstrapped<TTabletReqBlockBlobStorageGroup> { - const TActorId Owner; + const TActorId Owner; const ui64 TabletId; const ui32 GroupId; const ui32 Generation; @@ -20,7 +20,7 @@ class TTabletReqBlockBlobStorageGroup : public TActorBootstrapped<TTabletReqBloc } void SendRequest() { - const TActorId proxy = MakeBlobStorageProxyID(GroupId); + const TActorId proxy = MakeBlobStorageProxyID(GroupId); THolder<TEvBlobStorage::TEvBlock> event(new TEvBlobStorage::TEvBlock(TabletId, Generation, TInstant::Max())); event->IsMonitored = false; SendToBSProxy(TlsActivationContext->AsActorContext(), proxy, event.Release()); @@ -69,7 +69,7 @@ public: Become(&TThis::StateWait); } - TTabletReqBlockBlobStorageGroup(const TActorId &owner, ui64 tabletId, ui32 groupId, ui32 gen) + TTabletReqBlockBlobStorageGroup(const TActorId &owner, ui64 tabletId, ui32 groupId, ui32 gen) : Owner(owner) , TabletId(tabletId) , GroupId(groupId) @@ -79,13 +79,13 @@ public: }; class TTabletReqBlockBlobStorage : public TActorBootstrapped<TTabletReqBlockBlobStorage> { - const TActorId Owner; + const TActorId Owner; TIntrusiveConstPtr<TTabletStorageInfo> Info; const ui32 Generation; const bool BlockPrevEntry; ui32 Replied = 0; - TVector<TActorId> ReqActors; + TVector<TActorId> ReqActors; void PassAway() override { for (auto &x : ReqActors) @@ -104,7 +104,7 @@ class TTabletReqBlockBlobStorage : public TActorBootstrapped<TTabletReqBlockBlob auto *msg = ev->Get(); auto it = Find(ReqActors, ev->Sender); Y_VERIFY(it != ReqActors.end(), "must not get response from unknown actor"); - *it = TActorId(); + *it = TActorId(); switch (msg->Status) { case NKikimrProto::OK: @@ -116,7 +116,7 @@ class TTabletReqBlockBlobStorage : public TActorBootstrapped<TTabletReqBlockBlob } } public: - TTabletReqBlockBlobStorage(TActorId owner, TTabletStorageInfo *info, ui32 generation, bool blockPrevEntry) + TTabletReqBlockBlobStorage(TActorId owner, TTabletStorageInfo *info, ui32 generation, bool blockPrevEntry) : Owner(owner) , Info(info) , Generation(generation) @@ -157,7 +157,7 @@ public: } }; -IActor* CreateTabletReqBlockBlobStorage(const TActorId &owner, TTabletStorageInfo *info, ui32 generation, bool blockPrevEntry) { +IActor* CreateTabletReqBlockBlobStorage(const TActorId &owner, TTabletStorageInfo *info, ui32 generation, bool blockPrevEntry) { return new TTabletReqBlockBlobStorage(owner, info, generation, blockPrevEntry); } diff --git a/ydb/core/tablet/tablet_req_delete.cpp b/ydb/core/tablet/tablet_req_delete.cpp index c89c812289a..754c79bb325 100644 --- a/ydb/core/tablet/tablet_req_delete.cpp +++ b/ydb/core/tablet/tablet_req_delete.cpp @@ -19,7 +19,7 @@ class TTabletReqDelete : public TActorBootstrapped<TTabletReqDelete> { {} }; - const TActorId Owner; + const TActorId Owner; TIntrusivePtr<TTabletStorageInfo> TabletStorageInfo; TVector<TRequestInfo> Requests; ui32 FinishedRequests; @@ -28,7 +28,7 @@ class TTabletReqDelete : public TActorBootstrapped<TTabletReqDelete> { void ReplyAndDie(NKikimrProto::EReplyStatus status, const TActorContext &ctx) { if (status == NKikimrProto::OK) { - const TActorId tabletStateServiceId = NNodeWhiteboard::MakeNodeWhiteboardServiceId(ctx.SelfID.NodeId()); + const TActorId tabletStateServiceId = NNodeWhiteboard::MakeNodeWhiteboardServiceId(ctx.SelfID.NodeId()); ctx.Send(tabletStateServiceId, new NNodeWhiteboard::TEvWhiteboard::TEvTabletStateUpdate( TabletStorageInfo->TabletID, 0, @@ -83,7 +83,7 @@ class TTabletReqDelete : public TActorBootstrapped<TTabletReqDelete> { if (FinishedRequests >= Requests.size()) { if (Generation == std::numeric_limits<ui32>::max()) { ui64 StateStorageId = StateStorageGroupFromTabletID(TabletStorageInfo->TabletID); - const TActorId proxyActorID = MakeStateStorageProxyID(StateStorageId); + const TActorId proxyActorID = MakeStateStorageProxyID(StateStorageId); ctx.Send(proxyActorID, new TEvStateStorage::TEvDelete(TabletStorageInfo->TabletID)); } @@ -117,7 +117,7 @@ public: return NKikimrServices::TActivity::TABLET_REQ_DELETE_TABLET; } - TTabletReqDelete(const TActorId &owner, const TIntrusivePtr<TTabletStorageInfo>& tabletStorageInfo, ui32 generation = std::numeric_limits<ui32>::max()) + TTabletReqDelete(const TActorId &owner, const TIntrusivePtr<TTabletStorageInfo>& tabletStorageInfo, ui32 generation = std::numeric_limits<ui32>::max()) : Owner(owner) , TabletStorageInfo(tabletStorageInfo) , FinishedRequests(0) @@ -134,7 +134,7 @@ public: } }; -IActor* CreateTabletReqDelete(const TActorId &owner, const TIntrusivePtr<TTabletStorageInfo>& tabletStorageInfo, ui32 generation) { +IActor* CreateTabletReqDelete(const TActorId &owner, const TIntrusivePtr<TTabletStorageInfo>& tabletStorageInfo, ui32 generation) { return new TTabletReqDelete(owner, tabletStorageInfo, generation); } diff --git a/ydb/core/tablet/tablet_req_findlatest.cpp b/ydb/core/tablet/tablet_req_findlatest.cpp index f54519f6dfa..ca3d09aa432 100644 --- a/ydb/core/tablet/tablet_req_findlatest.cpp +++ b/ydb/core/tablet/tablet_req_findlatest.cpp @@ -14,7 +14,7 @@ namespace NKikimr { class TTabletReqFindLatestLogEntry : public TActorBootstrapped<TTabletReqFindLatestLogEntry> { - const TActorId Owner; + const TActorId Owner; const bool ReadBody; const ui32 BlockedGeneration; TIntrusivePtr<TTabletStorageInfo> Info; @@ -72,7 +72,7 @@ public: return NKikimrServices::TActivity::TABLET_REQ_FIND_LATEST; } - TTabletReqFindLatestLogEntry(const TActorId &owner, bool readBody, TTabletStorageInfo *info, ui32 blockedGeneration) + TTabletReqFindLatestLogEntry(const TActorId &owner, bool readBody, TTabletStorageInfo *info, ui32 blockedGeneration) : Owner(owner) , ReadBody(readBody) , BlockedGeneration(blockedGeneration) @@ -96,7 +96,7 @@ public: } }; -IActor* CreateTabletFindLastEntry(const TActorId &owner, bool readBody, TTabletStorageInfo *info, ui32 blockedGeneration) { +IActor* CreateTabletFindLastEntry(const TActorId &owner, bool readBody, TTabletStorageInfo *info, ui32 blockedGeneration) { return new TTabletReqFindLatestLogEntry(owner, readBody, info, blockedGeneration); } diff --git a/ydb/core/tablet/tablet_req_rebuildhistory.cpp b/ydb/core/tablet/tablet_req_rebuildhistory.cpp index 5a373120e70..c0b4c0c2bc1 100644 --- a/ydb/core/tablet/tablet_req_rebuildhistory.cpp +++ b/ydb/core/tablet/tablet_req_rebuildhistory.cpp @@ -185,7 +185,7 @@ class TTabletReqRebuildHistoryGraph : public TActorBootstrapped<TTabletReqRebuil } }; - const TActorId Owner; + const TActorId Owner; TIntrusivePtr<TTabletStorageInfo> Info; const ui32 BlockedGen; diff --git a/ydb/core/tablet/tablet_req_reset.cpp b/ydb/core/tablet/tablet_req_reset.cpp index f8d7c30da9c..b73c68c120f 100644 --- a/ydb/core/tablet/tablet_req_reset.cpp +++ b/ydb/core/tablet/tablet_req_reset.cpp @@ -7,7 +7,7 @@ namespace NKikimr { class TTabletReqReset : public TActorBootstrapped<TTabletReqReset> { - const TActorId Owner; + const TActorId Owner; const TIntrusivePtr<TTabletStorageInfo> TabletStorageInfo; ui32 Generation = 0; TActorId CurrentLeader; @@ -109,7 +109,7 @@ public: return NKikimrServices::TActivity::TABLET_REQ_DELETE_TABLET; } - TTabletReqReset(const TActorId& owner, const TIntrusivePtr<TTabletStorageInfo>& tabletStorageInfo, ui32 knownGeneration) + TTabletReqReset(const TActorId& owner, const TIntrusivePtr<TTabletStorageInfo>& tabletStorageInfo, ui32 knownGeneration) : Owner(owner) , TabletStorageInfo(tabletStorageInfo) , Generation(knownGeneration) @@ -119,7 +119,7 @@ public: } void Bootstrap(const TActorContext& ctx) { - TActorId stateStorageProxyId = MakeStateStorageProxyID(StateStorageGroupFromTabletID(TabletStorageInfo->TabletID)); + TActorId stateStorageProxyId = MakeStateStorageProxyID(StateStorageGroupFromTabletID(TabletStorageInfo->TabletID)); ctx.Send(stateStorageProxyId, new TEvStateStorage::TEvLookup(TabletStorageInfo->TabletID, 0)); if (Generation == 0) { FindLatestLogEntry(ctx); @@ -130,7 +130,7 @@ public: } }; -IActor* CreateTabletReqReset(const TActorId& owner, const TIntrusivePtr<NKikimr::TTabletStorageInfo>& info, ui32 knownGeneration) { +IActor* CreateTabletReqReset(const TActorId& owner, const TIntrusivePtr<NKikimr::TTabletStorageInfo>& info, ui32 knownGeneration) { return new TTabletReqReset(owner, info, knownGeneration); } diff --git a/ydb/core/tablet/tablet_req_writelog.cpp b/ydb/core/tablet/tablet_req_writelog.cpp index 4daec422e4f..7c4d02c405c 100644 --- a/ydb/core/tablet/tablet_req_writelog.cpp +++ b/ydb/core/tablet/tablet_req_writelog.cpp @@ -9,7 +9,7 @@ namespace NKikimr { class TTabletReqWriteLog : public TActorBootstrapped<TTabletReqWriteLog> { - const TActorId Owner; + const TActorId Owner; const TLogoBlobID LogEntryID; TAutoPtr<NKikimrTabletBase::TTabletLogEntry> LogEntry; TVector<TEvTablet::TLogEntryReference> References; @@ -104,7 +104,7 @@ public: return NKikimrServices::TActivity::TABLET_REQ_WRITE_LOG; } - TTabletReqWriteLog(const TActorId &owner, const TLogoBlobID &logid, NKikimrTabletBase::TTabletLogEntry *entry, TVector<TEvTablet::TLogEntryReference> &refs, TEvBlobStorage::TEvPut::ETactic commitTactic, TTabletStorageInfo *info) + TTabletReqWriteLog(const TActorId &owner, const TLogoBlobID &logid, NKikimrTabletBase::TTabletLogEntry *entry, TVector<TEvTablet::TLogEntryReference> &refs, TEvBlobStorage::TEvPut::ETactic commitTactic, TTabletStorageInfo *info) : Owner(owner) , LogEntryID(logid) , LogEntry(entry) @@ -148,7 +148,7 @@ public: } }; -IActor* CreateTabletReqWriteLog(const TActorId &owner, const TLogoBlobID &entryId, NKikimrTabletBase::TTabletLogEntry *entry, TVector<TEvTablet::TLogEntryReference> &refs, TEvBlobStorage::TEvPut::ETactic commitTactic, TTabletStorageInfo *info) { +IActor* CreateTabletReqWriteLog(const TActorId &owner, const TLogoBlobID &entryId, NKikimrTabletBase::TTabletLogEntry *entry, TVector<TEvTablet::TLogEntryReference> &refs, TEvBlobStorage::TEvPut::ETactic commitTactic, TTabletStorageInfo *info) { return new TTabletReqWriteLog(owner, entryId, entry, refs, commitTactic, info); } diff --git a/ydb/core/tablet/tablet_resolver.cpp b/ydb/core/tablet/tablet_resolver.cpp index 86c28217d51..c0a39a2216b 100644 --- a/ydb/core/tablet/tablet_resolver.cpp +++ b/ydb/core/tablet/tablet_resolver.cpp @@ -191,7 +191,7 @@ class TTabletResolver : public TActorBootstrapped<TTabletResolver> { } void ResolveRequest(ui64 tabletId, const TActorContext &ctx) { - const TActorId ssproxy = MakeStateStorageProxyID(StateStorageGroupFromTabletID(tabletId)); + const TActorId ssproxy = MakeStateStorageProxyID(StateStorageGroupFromTabletID(tabletId)); ctx.Send(ssproxy, new TEvStateStorage::TEvLookup(tabletId, 0), IEventHandle::FlagTrackDelivery, tabletId); InFlyResolveCounter->Inc(); @@ -204,7 +204,7 @@ class TTabletResolver : public TActorBootstrapped<TTabletResolver> { return true; } - std::pair<TActorId, TActorId> SelectForward(const TActorContext& ctx, const TEntry& entry, TResolveInfo& info, ui64 tabletId) + std::pair<TActorId, TActorId> SelectForward(const TActorContext& ctx, const TEntry& entry, TResolveInfo& info, ui64 tabletId) { const ui32 selfNode = ctx.SelfID.NodeId(); const std::optional<TString> selfDc = FindNodeDc(selfNode); @@ -303,13 +303,13 @@ class TTabletResolver : public TActorBootstrapped<TTabletResolver> { SelectedNone->Inc(); - return std::make_pair(TActorId(), TActorId()); + return std::make_pair(TActorId(), TActorId()); } - bool SendForward(const TActorId &sender, const TEntry &entry, TEvTabletResolver::TEvForward *msg, + bool SendForward(const TActorId &sender, const TEntry &entry, TEvTabletResolver::TEvForward *msg, const TActorContext &ctx, bool * needFollowerUpdate = nullptr) { TResolveInfo info(*msg, ctx.Now() - entry.LastResolved, needFollowerUpdate); // fills needFollowerUpdate in dtor - const std::pair<TActorId, TActorId> endpoint = SelectForward(ctx, entry, info, msg->TabletID); + const std::pair<TActorId, TActorId> endpoint = SelectForward(ctx, entry, info, msg->TabletID); if (endpoint.first) { ctx.Send(sender, new TEvTabletResolver::TEvForwardResult(msg->TabletID, endpoint.second, endpoint.first, LastCacheEpoch)); if (!!msg->Ev) @@ -796,7 +796,7 @@ class TTabletResolver : public TActorBootstrapped<TTabletResolver> { } void RefreshNodes(const TActorContext &ctx) { - ctx.Send(GetNameserviceActorId(), new TEvInterconnect::TEvListNodes()); + ctx.Send(GetNameserviceActorId(), new TEvInterconnect::TEvListNodes()); } public: @@ -872,9 +872,9 @@ IActor* CreateTabletResolver(const TIntrusivePtr<TTabletResolverConfig> &config) return new TTabletResolver(config); } -TActorId MakeTabletResolverID() { +TActorId MakeTabletResolverID() { const char x[12] = "TabletProxy"; - return TActorId(0, TStringBuf(x, 12)); + return TActorId(0, TStringBuf(x, 12)); } } diff --git a/ydb/core/tablet/tablet_setup.h b/ydb/core/tablet/tablet_setup.h index 9d3263d5465..57ea7d6edba 100644 --- a/ydb/core/tablet/tablet_setup.h +++ b/ydb/core/tablet/tablet_setup.h @@ -11,7 +11,7 @@ class TTabletStorageInfo; class TTabletSetupInfo : public TThrRefBase { public: - typedef std::function<IActor* (const TActorId &, TTabletStorageInfo*)> TTabletCreationFunc; + typedef std::function<IActor* (const TActorId &, TTabletStorageInfo*)> TTabletCreationFunc; using TPtr = TIntrusivePtr<TTabletSetupInfo>; private: @@ -30,9 +30,9 @@ public: , TabletPoolId(tabletPoolId) {} - TActorId Apply(TTabletStorageInfo *info, TActorIdentity owner); - TActorId Apply(TTabletStorageInfo *info, const TActorContext &ctx); - TActorId Tablet(TTabletStorageInfo *info, const TActorId &launcher, const TActorContext &ctx, + TActorId Apply(TTabletStorageInfo *info, TActorIdentity owner); + TActorId Apply(TTabletStorageInfo *info, const TActorContext &ctx); + TActorId Tablet(TTabletStorageInfo *info, const TActorId &launcher, const TActorContext &ctx, ui32 suggestedGeneration, TResourceProfilesPtr profiles = nullptr, TSharedQuotaPtr txCacheQuota = nullptr); TActorId Follower(TTabletStorageInfo *info, const TActorId &launcher, const TActorContext &ctx, @@ -40,7 +40,7 @@ public: TSharedQuotaPtr txCacheQuota = nullptr); }; -IActor* CreateTablet(const TActorId &launcher, TTabletStorageInfo *info, TTabletSetupInfo *setupInfo, +IActor* CreateTablet(const TActorId &launcher, TTabletStorageInfo *info, TTabletSetupInfo *setupInfo, ui32 suggestedGeneration, TResourceProfilesPtr profiles = nullptr, TSharedQuotaPtr txCacheQuota = nullptr); IActor* CreateTabletFollower(const TActorId &launcher, TTabletStorageInfo *info, TTabletSetupInfo *setupInfo, diff --git a/ydb/core/tablet/tablet_sys.cpp b/ydb/core/tablet/tablet_sys.cpp index 377a3e9bd0e..44bfac7b2d6 100644 --- a/ydb/core/tablet/tablet_sys.cpp +++ b/ydb/core/tablet/tablet_sys.cpp @@ -47,14 +47,14 @@ ui64 TTablet::TabletID() const { void TTablet::NextFollowerAttempt() { const ui32 node = FollowerInfo.KnownLeaderID.NodeId(); if (node && node != SelfId().NodeId()) { - const TActorId proxy = TActivationContext::InterconnectProxy(node); + const TActorId proxy = TActivationContext::InterconnectProxy(node); Send(proxy, new TEvents::TEvUnsubscribe); } FollowerInfo.NextAttempt(); } void TTablet::ReportTabletStateChange(ETabletState state) { - const TActorId tabletStateServiceId = NNodeWhiteboard::MakeNodeWhiteboardServiceId(SelfId().NodeId()); + const TActorId tabletStateServiceId = NNodeWhiteboard::MakeNodeWhiteboardServiceId(SelfId().NodeId()); if (state == TTabletStateInfo::Created || state == TTabletStateInfo::ResolveLeader) { Send(tabletStateServiceId, new NNodeWhiteboard::TEvWhiteboard::TEvTabletStateUpdate(TabletID(), FollowerId, state, Info, StateStorageInfo.KnownGeneration, Leader)); } else { @@ -91,7 +91,7 @@ void TTablet::TabletBlockBlobStorage() { Y_VERIFY(Info); IActor * const x = CreateTabletReqBlockBlobStorage(SelfId(), Info.Get(), StateStorageInfo.KnownGeneration - 1, false); - TActorId newActorId = Register(x); + TActorId newActorId = Register(x); if (IntrospectionTrace) { IntrospectionTrace->Attach(MakeHolder<NTracing::TOnTabletBlockBlobStorage>(newActorId, StateStorageInfo.KnownGeneration)); @@ -356,7 +356,7 @@ void TTablet::HandleByFollower(TEvTabletBase::TEvRebuildGraphResult::TPtr &ev) { return; } - RebuildGraphRequest = TActorId(); // check consistency?? + RebuildGraphRequest = TActorId(); // check consistency?? TEvTabletBase::TEvRebuildGraphResult *msg = ev->Get(); BLOG_TRACE("Follower received rebuild history result Status# " << msg->Status); @@ -479,7 +479,7 @@ void TTablet::HandleByFollower(TEvTablet::TEvFollowerUpdate::TPtr &ev) { // Drop currently running graph rebuild request if (RebuildGraphRequest) { Send(RebuildGraphRequest, new TEvents::TEvPoisonPill()); - RebuildGraphRequest = TActorId(); + RebuildGraphRequest = TActorId(); } if (!UserTablet) { @@ -537,7 +537,7 @@ void TTablet::HandleByFollower(TEvTablet::TEvPromoteToLeader::TPtr &ev) { if (RebuildGraphRequest) { Send(RebuildGraphRequest, new TEvents::TEvPoisonPill()); - RebuildGraphRequest = TActorId(); + RebuildGraphRequest = TActorId(); } // setup start info @@ -939,7 +939,7 @@ void TTablet::HandleRebuildGraphResult(TEvTabletBase::TEvRebuildGraphResult::TPt if (ev->Cookie != 0) // remains from follower past return; - RebuildGraphRequest = TActorId(); // check consistency?? + RebuildGraphRequest = TActorId(); // check consistency?? TEvTabletBase::TEvRebuildGraphResult *msg = ev->Get(); if (IntrospectionTrace) { @@ -1028,7 +1028,7 @@ TTablet::TLogEntry* TTablet::MakeLogEntry(TEvTablet::TCommitInfo &commitInfo, NK Graph.Index[step] = entry; entry->IsSnapshot = commitInfo.IsSnapshot || commitInfo.IsTotalSnapshot; entry->IsTotalSnapshot = commitInfo.IsTotalSnapshot; - entry->Source = TActorId(); + entry->Source = TActorId(); for (ui32 dependsOn : commitInfo.DependsOn) { TGraph::TIndex::iterator it = Graph.Index.find(dependsOn); @@ -1840,7 +1840,7 @@ TTablet::TTablet(const TActorId &launcher, TTabletStorageInfo *info, TTabletSetu Y_VERIFY(TTabletTypes::TYPE_INVALID != info->TabletType); } -TAutoPtr<IEventHandle> TTablet::AfterRegister(const TActorId &self, const TActorId& parentId) { +TAutoPtr<IEventHandle> TTablet::AfterRegister(const TActorId &self, const TActorId& parentId) { Y_UNUSED(parentId); return new IEventHandle(self, self, new TEvents::TEvBootstrap()); } @@ -1914,15 +1914,15 @@ void TTablet::ExternalWriteZeroEntry(TTabletStorageInfo *info, ui32 gen, TActorI TActivationContext::Register(CreateTabletReqWriteLog(owner, logid, entry.Release(), refs, TEvBlobStorage::TEvPut::TacticDefault, info)); } -TActorId TTabletSetupInfo::Apply(TTabletStorageInfo *info, TActorIdentity owner) { +TActorId TTabletSetupInfo::Apply(TTabletStorageInfo *info, TActorIdentity owner) { return TActivationContext::Register(Op(owner, info), owner, MailboxType, PoolId); } -TActorId TTabletSetupInfo::Apply(TTabletStorageInfo *info, const TActorContext &ctx) { +TActorId TTabletSetupInfo::Apply(TTabletStorageInfo *info, const TActorContext &ctx) { return Apply(info, TActorIdentity(ctx.SelfID)); } -TActorId TTabletSetupInfo::Tablet(TTabletStorageInfo *info, const TActorId &launcher, const TActorContext &ctx, +TActorId TTabletSetupInfo::Tablet(TTabletStorageInfo *info, const TActorId &launcher, const TActorContext &ctx, ui32 suggestedGeneration, TResourceProfilesPtr profiles, TSharedQuotaPtr txCacheQuota) { return ctx.ExecutorThread.RegisterActor(CreateTablet(launcher, info, this, suggestedGeneration, profiles, txCacheQuota), TabletMailboxType, TabletPoolId); @@ -1934,7 +1934,7 @@ TActorId TTabletSetupInfo::Follower(TTabletStorageInfo *info, const TActorId &la TabletMailboxType, TabletPoolId); } -IActor* CreateTablet(const TActorId &launcher, TTabletStorageInfo *info, TTabletSetupInfo *setupInfo, +IActor* CreateTablet(const TActorId &launcher, TTabletStorageInfo *info, TTabletSetupInfo *setupInfo, ui32 suggestedGeneration, TResourceProfilesPtr profiles, TSharedQuotaPtr txCacheQuota) { return new TTablet(launcher, info, setupInfo, true, suggestedGeneration, 0, profiles, txCacheQuota); } @@ -1945,7 +1945,7 @@ IActor* CreateTabletFollower(const TActorId &launcher, TTabletStorageInfo *info, } void TTablet::SendIntrospectionData() { - const TActorId tabletStateServiceId = NNodeWhiteboard::MakeNodeWhiteboardServiceId(SelfId().NodeId()); + const TActorId tabletStateServiceId = NNodeWhiteboard::MakeNodeWhiteboardServiceId(SelfId().NodeId()); Send(tabletStateServiceId, new NNodeWhiteboard::TEvWhiteboard::TEvIntrospectionData(TabletID(), IntrospectionTrace.Release())); IntrospectionTrace.Reset(NTracing::CreateTrace(NTracing::ITrace::TypeSysTabletBootstrap)); } diff --git a/ydb/core/tablet/tablet_sys.h b/ydb/core/tablet/tablet_sys.h index 8d324cf8714..661d75a4e0a 100644 --- a/ydb/core/tablet/tablet_sys.h +++ b/ydb/core/tablet/tablet_sys.h @@ -16,7 +16,7 @@ class TTablet : public TActor<TTablet> { using ETabletState = TTabletStateInfo::ETabletState; struct TStateStorageInfo { - TActorId ProxyID; + TActorId ProxyID; ui32 KnownGeneration; ui32 KnownStep; @@ -80,8 +80,8 @@ class TTablet : public TActor<TTablet> { bool WaitFollowerGcAck; TInstant CommitedMoment; - TActorId Source; - TActorId Task; + TActorId Source; + TActorId Task; ui64 SourceCookie; @@ -204,9 +204,9 @@ class TTablet : public TActor<TTablet> { TDeque<std::pair<ui32, TInstant>> WaitingForGcAck; // step, commitMoment bool InitialFollowerSyncDone; - const TActorId Launcher; - TActorId UserTablet; - TActorId StateStorageGuardian; + const TActorId Launcher; + TActorId UserTablet; + TActorId StateStorageGuardian; TActorId FollowerStStGuardian; TIntrusivePtr<TTabletStorageInfo> Info; TIntrusivePtr<TTabletSetupInfo> SetupInfo; @@ -225,7 +225,7 @@ class TTablet : public TActor<TTablet> { TResourceProfilesPtr ResourceProfiles; TSharedQuotaPtr TxCacheQuota; THolder<NTracing::ITrace> IntrospectionTrace; - TActorId RebuildGraphRequest; + TActorId RebuildGraphRequest; // Delayed cancellation reason struct TDelayedCancelTablet { @@ -598,7 +598,7 @@ public: } TTablet( - const TActorId &launcher, + const TActorId &launcher, TTabletStorageInfo *info, TTabletSetupInfo *setupInfo, bool leader, @@ -608,7 +608,7 @@ public: TSharedQuotaPtr txCacheQuota = nullptr ); - TAutoPtr<IEventHandle> AfterRegister(const TActorId &self, const TActorId &parentId) override; + TAutoPtr<IEventHandle> AfterRegister(const TActorId &self, const TActorId &parentId) override; static void ExternalWriteZeroEntry(TTabletStorageInfo *info, ui32 gen, TActorIdentity owner); }; diff --git a/ydb/core/tablet/tablet_tracing_signals.cpp b/ydb/core/tablet/tablet_tracing_signals.cpp index 78efbc41031..9d1923f822d 100644 --- a/ydb/core/tablet/tablet_tracing_signals.cpp +++ b/ydb/core/tablet/tablet_tracing_signals.cpp @@ -68,7 +68,7 @@ static TSignalTypeRegistrator SignalTypeRegistrator; TOnTabletBootstrap::TOnTabletBootstrap(ui32 suggestedGeneration, bool leader, const TActorId& proxyID) { PbSignal.SetSuggestedGeneration(suggestedGeneration); PbSignal.SetLeader(leader); - NActors::ActorIdToProto(proxyID, PbSignal.MutableStateStorageProxyID()); + NActors::ActorIdToProto(proxyID, PbSignal.MutableStateStorageProxyID()); } TOnTabletBootstrap::TOnTabletBootstrap(const TString& serializedSignal) @@ -76,7 +76,7 @@ TOnTabletBootstrap::TOnTabletBootstrap(const TString& serializedSignal) {} void TOnTabletBootstrap::OutText(TStringStream& str, TTimestampData& tsData, const TString& prefix) const { - TActorId ID = NActors::ActorIdFromProto(PbSignal.GetStateStorageProxyID()); + TActorId ID = NActors::ActorIdFromProto(PbSignal.GetStateStorageProxyID()); str << prefix << TimeStamp(tsData) << " Bootstrapped. Sent TEvLookup to SS proxy " << ID.ToString() << Endl; } @@ -184,8 +184,8 @@ void TOnPromoteToCandidate::OutText(TStringStream& str, TTimestampData& tsData, << " Promoted to candidate (KnownGeneration=" << PbSignal.GetKnownGeneration() << ")" << Endl; } -TOnTabletBlockBlobStorage::TOnTabletBlockBlobStorage(const TActorId& reqBlockBlobStorageID, ui32 knownGeneration) { - NActors::ActorIdToProto(reqBlockBlobStorageID, PbSignal.MutableReqBlockBlobStorageID()); +TOnTabletBlockBlobStorage::TOnTabletBlockBlobStorage(const TActorId& reqBlockBlobStorageID, ui32 knownGeneration) { + NActors::ActorIdToProto(reqBlockBlobStorageID, PbSignal.MutableReqBlockBlobStorageID()); PbSignal.SetKnownGeneration(knownGeneration); } @@ -194,16 +194,16 @@ TOnTabletBlockBlobStorage::TOnTabletBlockBlobStorage(const TString& serializedSi {} void TOnTabletBlockBlobStorage::OutText(TStringStream& str, TTimestampData& tsData, const TString& prefix) const { - TActorId ID = NActors::ActorIdFromProto(PbSignal.GetReqBlockBlobStorageID()); + TActorId ID = NActors::ActorIdFromProto(PbSignal.GetReqBlockBlobStorageID()); str << prefix << TimeStamp(tsData) << " Created ReqBlockBlobStorage Actor (" << ID.ToString() << ") with KnownGeneration=" << PbSignal.GetKnownGeneration() << Endl; } TOnTabletRebuildGraph::TOnTabletRebuildGraph( - const TActorId& tabletReqRebuildGraphID + const TActorId& tabletReqRebuildGraphID , const TTraceID& rebuildGraphTraceID ) { - NActors::ActorIdToProto(tabletReqRebuildGraphID, PbSignal.MutableTabletReqRebuildGraphID()); + NActors::ActorIdToProto(tabletReqRebuildGraphID, PbSignal.MutableTabletReqRebuildGraphID()); TraceIDFromTraceID(rebuildGraphTraceID, PbSignal.MutableRebuildGraphTraceID()); } @@ -212,7 +212,7 @@ TOnTabletRebuildGraph::TOnTabletRebuildGraph(const TString& serializedSignal) {} void TOnTabletRebuildGraph::OutText(TStringStream& str, TTimestampData& tsData, const TString& prefix) const { - TActorId ID = NActors::ActorIdFromProto(PbSignal.GetTabletReqRebuildGraphID()); + TActorId ID = NActors::ActorIdFromProto(PbSignal.GetTabletReqRebuildGraphID()); str << prefix << TimeStamp(tsData) << " Created ReqRebuildHistoryGraph Actor (" << ID.ToString() << ")" << Endl; } diff --git a/ydb/core/tablet/tablet_tracing_signals.h b/ydb/core/tablet/tablet_tracing_signals.h index 226c80aa0f9..a8e73087e15 100644 --- a/ydb/core/tablet/tablet_tracing_signals.h +++ b/ydb/core/tablet/tablet_tracing_signals.h @@ -106,7 +106,7 @@ namespace NTracing { NKikimrTracing::TOnTabletBlockBlobStorage, NSignalTypes::TypeOnTabletBlockBlobStorage> { public: - TOnTabletBlockBlobStorage(const TActorId& reqBlockBlobStorageID, ui32 knownGeneration); + TOnTabletBlockBlobStorage(const TActorId& reqBlockBlobStorageID, ui32 knownGeneration); TOnTabletBlockBlobStorage(const TString& serializedSignal); void OutText(TStringStream& str, TTimestampData& tsData, const TString& prefix) const override; }; @@ -115,7 +115,7 @@ namespace NTracing { NKikimrTracing::TOnTabletRebuildGraph, NSignalTypes::TypeOnTabletRebuildGraph> { public: - TOnTabletRebuildGraph(const TActorId& tabletReqRebuildGraphID, const TTraceID& rebuildGraphTraceID); + TOnTabletRebuildGraph(const TActorId& tabletReqRebuildGraphID, const TTraceID& rebuildGraphTraceID); TOnTabletRebuildGraph(const TString& serializedSignal); void OutText(TStringStream& str, TTimestampData& tsData, const TString& prefix) const override; }; diff --git a/ydb/core/tablet_flat/datetime_ut.cpp b/ydb/core/tablet_flat/datetime_ut.cpp index 31ccbd32b0c..67c77235a8b 100644 --- a/ydb/core/tablet_flat/datetime_ut.cpp +++ b/ydb/core/tablet_flat/datetime_ut.cpp @@ -315,7 +315,7 @@ Y_UNIT_TEST_SUITE(TFlatTableDatetime) { } public: - TDateTestFlatTablet(const TActorId &sender, const TActorId &tablet, TTabletStorageInfo *info) + TDateTestFlatTablet(const TActorId &sender, const TActorId &tablet, TTabletStorageInfo *info) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, nullptr) , Sender(sender) @@ -323,7 +323,7 @@ Y_UNIT_TEST_SUITE(TFlatTableDatetime) { } private: - TActorId Sender; + TActorId Sender; }; struct TDateEnvProfiles : public NFake::TRunner { @@ -334,7 +334,7 @@ Y_UNIT_TEST_SUITE(TFlatTableDatetime) { void Run() { - FireTablet(Edge, Tablet, [this](const TActorId &tablet, TTabletStorageInfo *info) { + FireTablet(Edge, Tablet, [this](const TActorId &tablet, TTabletStorageInfo *info) { return new TDateTestFlatTablet(Edge, tablet, info); }); @@ -343,7 +343,7 @@ Y_UNIT_TEST_SUITE(TFlatTableDatetime) { } const ui32 Tablet = MakeTabletID(0, 0, 1); - const TActorId Edge; + const TActorId Edge; }; Y_UNIT_TEST(TestDate) { diff --git a/ydb/core/tablet_flat/decimal_ut.cpp b/ydb/core/tablet_flat/decimal_ut.cpp index ef155864ee9..685215aa5a9 100644 --- a/ydb/core/tablet_flat/decimal_ut.cpp +++ b/ydb/core/tablet_flat/decimal_ut.cpp @@ -387,7 +387,7 @@ Y_UNIT_TEST_SUITE(TFlatTableDecimals) { } public: - TDecimalTestFlatTablet(const TActorId &sender, const TActorId &tablet, TTabletStorageInfo *info) + TDecimalTestFlatTablet(const TActorId &sender, const TActorId &tablet, TTabletStorageInfo *info) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, nullptr) , Sender(sender) @@ -395,7 +395,7 @@ Y_UNIT_TEST_SUITE(TFlatTableDecimals) { } private: - TActorId Sender; + TActorId Sender; }; struct TDecimalEnvProfiles : public NFake::TRunner { @@ -406,7 +406,7 @@ Y_UNIT_TEST_SUITE(TFlatTableDecimals) { void Run() { - FireTablet(Edge, Tablet, [this](const TActorId &tablet, TTabletStorageInfo *info) { + FireTablet(Edge, Tablet, [this](const TActorId &tablet, TTabletStorageInfo *info) { return new TDecimalTestFlatTablet(Edge, tablet, info); }); @@ -415,7 +415,7 @@ Y_UNIT_TEST_SUITE(TFlatTableDecimals) { } const ui32 Tablet = MakeTabletID(0, 0, 1); - const TActorId Edge; + const TActorId Edge; }; Y_UNIT_TEST(TestDecimal) { diff --git a/ydb/core/tablet_flat/flat_bio_actor.cpp b/ydb/core/tablet_flat/flat_bio_actor.cpp index 163477dfae8..6a866fb3956 100644 --- a/ydb/core/tablet_flat/flat_bio_actor.cpp +++ b/ydb/core/tablet_flat/flat_bio_actor.cpp @@ -13,7 +13,7 @@ using TEvGet = TEvBlobStorage::TEvGet; struct TBlockIO::TLoaded : public TEvBlobStorage::TEvGetResult::TResponse{ }; -TBlockIO::TBlockIO(TActorId service, ui64 cookie) +TBlockIO::TBlockIO(TActorId service, ui64 cookie) : ::NActors::IActor(static_cast<TReceiveFunc>(&TBlockIO::Inbox), NKikimrServices::TActivity::SAUSAGE_BIO_A) , Service(service) , Cookie(cookie) @@ -25,7 +25,7 @@ TBlockIO::~TBlockIO() } -void TBlockIO::Registered(TActorSystem *sys, const TActorId&) +void TBlockIO::Registered(TActorSystem *sys, const TActorId&) { Logger = new NUtil::TLogger(sys, NKikimrServices::SAUSAGE_BIO); } diff --git a/ydb/core/tablet_flat/flat_bio_actor.h b/ydb/core/tablet_flat/flat_bio_actor.h index c1a24d745be..6de0926223a 100644 --- a/ydb/core/tablet_flat/flat_bio_actor.h +++ b/ydb/core/tablet_flat/flat_bio_actor.h @@ -18,11 +18,11 @@ namespace NBlockIO { struct TLoaded; /* hack for fwd decl of BS interface units */ public: - TBlockIO(TActorId service, ui64 cookie); + TBlockIO(TActorId service, ui64 cookie); ~TBlockIO(); private: - void Registered(TActorSystem*, const TActorId&) override; + void Registered(TActorSystem*, const TActorId&) override; void Inbox(TEventHandlePtr &eh, const ::NActors::TActorContext &ctx); void Bootstrap(EPriority priority, TAutoPtr<NPageCollection::TFetch>) noexcept; void Dispatch() noexcept; @@ -30,13 +30,13 @@ namespace NBlockIO { void Terminate(EStatus code) noexcept; private: - const TActorId Service; + const TActorId Service; const ui64 Cookie = Max<ui64>(); TAutoPtr<NUtil::ILogger> Logger; /*_ immutable request settings */ - TActorId Owner; + TActorId Owner; EPriority Priority = EPriority::None; TAutoPtr<NPageCollection::TFetch> Origin; @@ -60,7 +60,7 @@ namespace NBlockIO { }; template<typename ... TArgs> - inline void Once(NActors::IActorOps *ops, TActorId service, + inline void Once(NActors::IActorOps *ops, TActorId service, ui64 cookie, TArgs&& ... args) noexcept { auto self = ops->Register(new TBlockIO(service, cookie)); diff --git a/ydb/core/tablet_flat/flat_comp_gen.cpp b/ydb/core/tablet_flat/flat_comp_gen.cpp index fb4d4789b36..43bf95357bc 100644 --- a/ydb/core/tablet_flat/flat_comp_gen.cpp +++ b/ydb/core/tablet_flat/flat_comp_gen.cpp @@ -1,6 +1,6 @@ #include "flat_comp_gen.h" -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/generic/cast.h> namespace NKikimr { diff --git a/ydb/core/tablet_flat/flat_comp_shard.cpp b/ydb/core/tablet_flat/flat_comp_shard.cpp index 16f6f043bd5..a3a58f923e2 100644 --- a/ydb/core/tablet_flat/flat_comp_shard.cpp +++ b/ydb/core/tablet_flat/flat_comp_shard.cpp @@ -7,7 +7,7 @@ #include <ydb/core/tablet_flat/protos/flat_table_shard.pb.h> #include <ydb/core/util/pb.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/generic/cast.h> #include <optional> diff --git a/ydb/core/tablet_flat/flat_cxx_database.h b/ydb/core/tablet_flat/flat_cxx_database.h index 2bd404302f6..27a260bcb27 100644 --- a/ydb/core/tablet_flat/flat_cxx_database.h +++ b/ydb/core/tablet_flat/flat_cxx_database.h @@ -62,7 +62,7 @@ public: : TRawTypeValue(value.empty() ? (const ElementType*)0xDEADBEEFDEADBEEF : value.data(), value.size() * sizeof(ElementType), type) {} - TTypeValue(const TActorId& value, NScheme::TTypeId type = NScheme::NTypeIds::ActorId) + TTypeValue(const TActorId& value, NScheme::TTypeId type = NScheme::NTypeIds::ActorId) : TRawTypeValue(&value, sizeof(value), type) {} @@ -137,13 +137,13 @@ public: } operator ui8() const { - Y_VERIFY(Type() == NScheme::NTypeIds::Byte && Size() == sizeof(ui8), "Data=%" PRIxPTR ", Type=%" PRIi64 ", Size=%" PRIi64, (ui64)Data(), (i64)Type(), (i64)Size()); + Y_VERIFY(Type() == NScheme::NTypeIds::Byte && Size() == sizeof(ui8), "Data=%" PRIxPTR ", Type=%" PRIi64 ", Size=%" PRIi64, (ui64)Data(), (i64)Type(), (i64)Size()); ui8 value = *reinterpret_cast<const ui8*>(Data()); return value; } operator bool() const { - Y_VERIFY(Type() == NScheme::NTypeIds::Bool && Size() == sizeof(bool), "Data=%" PRIxPTR ", Type=%" PRIi64 ", Size=%" PRIi64, (ui64)Data(), (i64)Type(), (i64)Size()); + Y_VERIFY(Type() == NScheme::NTypeIds::Bool && Size() == sizeof(bool), "Data=%" PRIxPTR ", Type=%" PRIi64 ", Size=%" PRIi64, (ui64)Data(), (i64)Type(), (i64)Size()); bool value = *reinterpret_cast<const bool*>(Data()); return value; } @@ -154,12 +154,12 @@ public: return value; } - operator TActorId() const { + operator TActorId() const { Y_VERIFY((Type() == NScheme::NTypeIds::ActorId || Type() == NScheme::NTypeIds::String || Type() == NScheme::NTypeIds::String2m - || Type() == NScheme::NTypeIds::String4k) && Size() == sizeof(TActorId), "Data=%" PRIxPTR ", Type=%" PRIi64 ", Size=%" PRIi64, (ui64)Data(), (i64)Type(), (i64)Size()); - return *reinterpret_cast<const TActorId*>(Data()); + || Type() == NScheme::NTypeIds::String4k) && Size() == sizeof(TActorId), "Data=%" PRIxPTR ", Type=%" PRIi64 ", Size=%" PRIi64, (ui64)Data(), (i64)Type(), (i64)Size()); + return *reinterpret_cast<const TActorId*>(Data()); } operator TString() const { @@ -178,7 +178,7 @@ public: } operator std::pair<ui64, ui64>() const { - Y_VERIFY(Type() == NScheme::NTypeIds::PairUi64Ui64 && Size() == sizeof(std::pair<ui64, ui64>), "Data=%" PRIxPTR ", Type=%" PRIi64 ", Size=%" PRIi64, (ui64)Data(), (i64)Type(), (i64)Size()); + Y_VERIFY(Type() == NScheme::NTypeIds::PairUi64Ui64 && Size() == sizeof(std::pair<ui64, ui64>), "Data=%" PRIxPTR ", Type=%" PRIi64 ", Size=%" PRIi64, (ui64)Data(), (i64)Type(), (i64)Size()); return *reinterpret_cast<const std::pair<ui64, ui64>*>(Data()); } @@ -214,17 +214,17 @@ public: template <NScheme::TTypeId T> struct NSchemeTypeMapper; -template <> struct NSchemeTypeMapper<NScheme::NTypeIds::Bool> { typedef bool Type; }; -template <> struct NSchemeTypeMapper<NScheme::NTypeIds::Byte> { typedef ui8 Type; }; -template <> struct NSchemeTypeMapper<NScheme::NTypeIds::Uint32> { typedef ui32 Type; }; +template <> struct NSchemeTypeMapper<NScheme::NTypeIds::Bool> { typedef bool Type; }; +template <> struct NSchemeTypeMapper<NScheme::NTypeIds::Byte> { typedef ui8 Type; }; +template <> struct NSchemeTypeMapper<NScheme::NTypeIds::Uint32> { typedef ui32 Type; }; template <> struct NSchemeTypeMapper<NScheme::NTypeIds::Int32> { typedef i32 Type; }; -template <> struct NSchemeTypeMapper<NScheme::NTypeIds::Uint64> { typedef ui64 Type; }; +template <> struct NSchemeTypeMapper<NScheme::NTypeIds::Uint64> { typedef ui64 Type; }; template <> struct NSchemeTypeMapper<NScheme::NTypeIds::Int64> { typedef i64 Type; }; template <> struct NSchemeTypeMapper<NScheme::NTypeIds::String> { typedef TString Type; }; template <> struct NSchemeTypeMapper<NScheme::NTypeIds::String4k> : NSchemeTypeMapper<NScheme::NTypeIds::String> {}; template <> struct NSchemeTypeMapper<NScheme::NTypeIds::Utf8> { typedef TString Type; }; -template <> struct NSchemeTypeMapper<NScheme::NTypeIds::ActorId> { typedef TActorId Type; }; -template <> struct NSchemeTypeMapper<NScheme::NTypeIds::PairUi64Ui64> { typedef std::pair<ui64, ui64> Type; }; +template <> struct NSchemeTypeMapper<NScheme::NTypeIds::ActorId> { typedef TActorId Type; }; +template <> struct NSchemeTypeMapper<NScheme::NTypeIds::PairUi64Ui64> { typedef std::pair<ui64, ui64> Type; }; template <> struct NSchemeTypeMapper<NScheme::NTypeIds::Double> { typedef double Type; }; template <> struct NSchemeTypeMapper<NScheme::NTypeIds::Decimal> { typedef std::pair<ui64, i64> Type; }; template <> struct NSchemeTypeMapper<NScheme::NTypeIds::Date> { typedef ui16 Type; }; diff --git a/ydb/core/tablet_flat/flat_cxx_database_ut.cpp b/ydb/core/tablet_flat/flat_cxx_database_ut.cpp index cfa597499b9..0423d82e624 100644 --- a/ydb/core/tablet_flat/flat_cxx_database_ut.cpp +++ b/ydb/core/tablet_flat/flat_cxx_database_ut.cpp @@ -21,10 +21,10 @@ Y_UNIT_TEST_SUITE(TFlatCxxDatabaseTest) { struct Schema : NIceDb::Schema { struct TestTable : Table<1> { - struct ID : Column<1, NScheme::NTypeIds::Uint64> {}; - struct Value : Column<2, NScheme::NTypeIds::Uint64> {}; + struct ID : Column<1, NScheme::NTypeIds::Uint64> {}; + struct Value : Column<2, NScheme::NTypeIds::Uint64> {}; struct Name : Column<3, NScheme::NTypeIds::Utf8> {}; - struct BoolValue : Column<4, NScheme::NTypeIds::Bool> {}; + struct BoolValue : Column<4, NScheme::NTypeIds::Bool> {}; struct EmptyValue : Column<5, NScheme::NTypeIds::Uint64> { static constexpr ui64 Default = 13; }; struct ProtoValue : Column<6, NScheme::NTypeIds::String> { using Type = NKikimrMiniKQL::TValue; }; struct EnumValue : Column<7, NScheme::NTypeIds::Uint64> { using Type = ESomeEnum; }; @@ -35,8 +35,8 @@ Y_UNIT_TEST_SUITE(TFlatCxxDatabaseTest) { }; struct TestTable2 : Table<2> { - struct ID1 : Column<1, NScheme::NTypeIds::Uint64> {}; - struct ID2 : Column<2, NScheme::NTypeIds::Uint64> {}; + struct ID1 : Column<1, NScheme::NTypeIds::Uint64> {}; + struct ID2 : Column<2, NScheme::NTypeIds::Uint64> {}; struct Value : Column<3, NScheme::NTypeIds::Uint64> {}; using TKey = TableKey<ID1, ID2>; @@ -48,7 +48,7 @@ Y_UNIT_TEST_SUITE(TFlatCxxDatabaseTest) { struct TestTable3 : Table<3> { struct ID1 : Column<1, NScheme::NTypeIds::Utf8> {}; struct ID2 : Column<2, NScheme::NTypeIds::String> {}; - struct Value : Column<3, NScheme::NTypeIds::Uint64> {}; + struct Value : Column<3, NScheme::NTypeIds::Uint64> {}; using TKey = TableKey<ID1, ID2>; using TColumns = TableColumns<ID1, ID2, Value>; diff --git a/ydb/core/tablet_flat/flat_exec_commit_mgr.h b/ydb/core/tablet_flat/flat_exec_commit_mgr.h index 6a586e40f43..20f5d5ffe10 100644 --- a/ydb/core/tablet_flat/flat_exec_commit_mgr.h +++ b/ydb/core/tablet_flat/flat_exec_commit_mgr.h @@ -76,7 +76,7 @@ namespace NTabletFlatExecutor { << ", lvl " << Waste->Level << "b}"; } - void Start(IOps *ops, TActorId owner, ui32 *step0, TMonCo *monCo) + void Start(IOps *ops, TActorId owner, ui32 *step0, TMonCo *monCo) { Y_VERIFY(!std::exchange(Ops, ops), "Commit manager is already started"); @@ -185,7 +185,7 @@ namespace NTabletFlatExecutor { private: IOps * Ops = nullptr; - TActorId Owner; + TActorId Owner; ui32 Back = Max<ui32>();/* Commits confirmation edge step */ ui32 Tail = 0; /* Active detached lower Step */ ui32 Head = 0; /* Tablet current step, AKA Step0 */ diff --git a/ydb/core/tablet_flat/flat_exec_scans.h b/ydb/core/tablet_flat/flat_exec_scans.h index 61ea953de05..4ab70473513 100644 --- a/ydb/core/tablet_flat/flat_exec_scans.h +++ b/ydb/core/tablet_flat/flat_exec_scans.h @@ -82,7 +82,7 @@ namespace NTabletFlatExecutor { TScanOptions Options; THolder<TScanSnapshot> Snapshot; EState State = EState::None; - TActorId Actor; /* Valid just after EState::Scan*/ + TActorId Actor; /* Valid just after EState::Scan*/ TAutoPtr<IScan> Scan; /* Valid before EState::Scan */ ui64 Cookie = Max<ui64>(); ui64 TaskId = 0; /* Task number in res. broker */ diff --git a/ydb/core/tablet_flat/flat_executor.cpp b/ydb/core/tablet_flat/flat_executor.cpp index 6b81ed65deb..59afc462158 100644 --- a/ydb/core/tablet_flat/flat_executor.cpp +++ b/ydb/core/tablet_flat/flat_executor.cpp @@ -90,7 +90,7 @@ TActorContext TExecutor::OwnerCtx() const { return TActivationContext::ActorContextFor(OwnerActorId); } -void TExecutor::Registered(TActorSystem *sys, const TActorId&) +void TExecutor::Registered(TActorSystem *sys, const TActorId&) { Logger = new NUtil::TLogger(sys, NKikimrServices::TABLET_EXECUTOR); Broker = new TBroker(this, Emitter); diff --git a/ydb/core/tablet_flat/flat_executor.h b/ydb/core/tablet_flat/flat_executor.h index 202a1e8667f..df08b3a338a 100644 --- a/ydb/core/tablet_flat/flat_executor.h +++ b/ydb/core/tablet_flat/flat_executor.h @@ -30,7 +30,7 @@ #include <ydb/core/tablet/tablet_metrics.h> #include <ydb/core/util/queue_oneone_inplace.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <util/system/hp_timer.h> #include <util/thread/singleton.h> @@ -408,7 +408,7 @@ class TExecutor TTabletCountersWithTxTypes* AppTxCounters = nullptr; - TActorId Launcher; + TActorId Launcher; THashMap<TPrivatePageCacheWaitPad*, THolder<TTransactionWaitPad>> TransactionWaitPads; THashMap<TPrivatePageCacheWaitPad*, THolder<TCompactionReadWaitPad>> CompactionReadWaitPads; @@ -441,7 +441,7 @@ class TExecutor TControlWrapper LogFlushDelayOverrideUsec; ui64 Stamp() const noexcept; - void Registered(TActorSystem*, const TActorId&) override; + void Registered(TActorSystem*, const TActorId&) override; void PassAway() override; void Broken(); void Active(const TActorContext &ctx); @@ -630,7 +630,7 @@ public: float GetRejectProbability() const override; - TActorId GetLauncher() const { return Launcher; } + TActorId GetLauncher() const { return Launcher; } }; }} diff --git a/ydb/core/tablet_flat/flat_executor_bootlogic.cpp b/ydb/core/tablet_flat/flat_executor_bootlogic.cpp index a6ada3aaab0..cf72960f9c4 100644 --- a/ydb/core/tablet_flat/flat_executor_bootlogic.cpp +++ b/ydb/core/tablet_flat/flat_executor_bootlogic.cpp @@ -28,7 +28,7 @@ NBoot::TLoadBlobs::TLoadBlobs(IStep *owner, NPageCollection::TLargeGlobId largeG Logic->LoadEntry(this); } -TExecutorBootLogic::TExecutorBootLogic(IOps *ops, const TActorId &self, TTabletStorageInfo *info, ui64 maxBytesInFly) +TExecutorBootLogic::TExecutorBootLogic(IOps *ops, const TActorId &self, TTabletStorageInfo *info, ui64 maxBytesInFly) : Ops(ops) , SelfId(self) , Info(info) diff --git a/ydb/core/tablet_flat/flat_executor_bootlogic.h b/ydb/core/tablet_flat/flat_executor_bootlogic.h index 1efbae05492..34edb2d6b40 100644 --- a/ydb/core/tablet_flat/flat_executor_bootlogic.h +++ b/ydb/core/tablet_flat/flat_executor_bootlogic.h @@ -70,7 +70,7 @@ private: bool Restored = false; IOps * const Ops = nullptr; - const TActorId SelfId; + const TActorId SelfId; TAutoPtr<NBoot::TBack> State_; TAutoPtr<NBoot::TResult> Result_; TAutoPtr<NBoot::TRoot> Steps; @@ -102,7 +102,7 @@ private: inline NBoot::TBack& State() const noexcept { return *State_; } public: - TExecutorBootLogic(IOps*, const TActorId&, TTabletStorageInfo *info, ui64 maxBytesInFly); + TExecutorBootLogic(IOps*, const TActorId&, TTabletStorageInfo *info, ui64 maxBytesInFly); ~TExecutorBootLogic(); void Describe(IOutputStream&) const noexcept; diff --git a/ydb/core/tablet_flat/flat_executor_borrowlogic.cpp b/ydb/core/tablet_flat/flat_executor_borrowlogic.cpp index 28c4232f69c..bd52264b10e 100644 --- a/ydb/core/tablet_flat/flat_executor_borrowlogic.cpp +++ b/ydb/core/tablet_flat/flat_executor_borrowlogic.cpp @@ -1,7 +1,7 @@ #include "flat_executor_borrowlogic.h" #include "flat_bio_eggs.h" -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/generic/xrange.h> #include <util/string/builder.h> diff --git a/ydb/core/tablet_flat/flat_executor_compaction_logic.cpp b/ydb/core/tablet_flat/flat_executor_compaction_logic.cpp index 547406a99fa..3d699e8f87b 100644 --- a/ydb/core/tablet_flat/flat_executor_compaction_logic.cpp +++ b/ydb/core/tablet_flat/flat_executor_compaction_logic.cpp @@ -5,7 +5,7 @@ #include <ydb/core/base/appdata.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/generic/cast.h> namespace NKikimr { diff --git a/ydb/core/tablet_flat/flat_executor_db_mon.cpp b/ydb/core/tablet_flat/flat_executor_db_mon.cpp index 1aa834cee61..d5353747beb 100644 --- a/ydb/core/tablet_flat/flat_executor_db_mon.cpp +++ b/ydb/core/tablet_flat/flat_executor_db_mon.cpp @@ -205,7 +205,7 @@ public: str << EncodeHtmlPcdata(EscapeC(TStringBuf(static_cast<const char*>(data), Min(size, (ui32)1024)))); break; case NScheme::NTypeIds::ActorId: - str << *(TActorId*)data; + str << *(TActorId*)data; break; case NScheme::NTypeIds::Utf8: case NScheme::NTypeIds::Json: diff --git a/ydb/core/tablet_flat/flat_executor_txloglogic.cpp b/ydb/core/tablet_flat/flat_executor_txloglogic.cpp index 15b6c471fb5..df59ddc2455 100644 --- a/ydb/core/tablet_flat/flat_executor_txloglogic.cpp +++ b/ydb/core/tablet_flat/flat_executor_txloglogic.cpp @@ -214,7 +214,7 @@ void TLogicRedo::MakeLogEntry(TLogCommit &commit, TString redo, TArrayRef<const } } -ui64 TLogicRedo::Confirm(ui32 step, const TActorContext &ctx, const TActorId &ownerId) { +ui64 TLogicRedo::Confirm(ui32 step, const TActorContext &ctx, const TActorId &ownerId) { Y_VERIFY(!CompletionQueue.empty(), "t: %" PRIu64 " non-expected confirmation %" PRIu32 ", prev %" PRIu32, Cookies->Tablet, step, PrevConfirmedStep); diff --git a/ydb/core/tablet_flat/flat_executor_txloglogic.h b/ydb/core/tablet_flat/flat_executor_txloglogic.h index 166ba14ff3a..1272c4f947b 100644 --- a/ydb/core/tablet_flat/flat_executor_txloglogic.h +++ b/ydb/core/tablet_flat/flat_executor_txloglogic.h @@ -65,7 +65,7 @@ public: void MakeLogEntry(TLogCommit&, TString redo, TArrayRef<const ui32> affects, bool embed); void FlushBatchedLog(); - ui64 Confirm(ui32 step, const TActorContext &ctx, const TActorId &ownerId); + ui64 Confirm(ui32 step, const TActorContext &ctx, const TActorId &ownerId); void CutLog(ui32 table, NTable::TSnapEdge, TGCBlobDelta&); void SnapToLog(NKikimrExecutorFlat::TLogSnapshot&); diff --git a/ydb/core/tablet_flat/flat_executor_ut.cpp b/ydb/core/tablet_flat/flat_executor_ut.cpp index 35b914174db..c9cf5289711 100644 --- a/ydb/core/tablet_flat/flat_executor_ut.cpp +++ b/ydb/core/tablet_flat/flat_executor_ut.cpp @@ -248,7 +248,7 @@ struct TDummyResult: public IDestructable { class TDummyScan : public TActor<TDummyScan>, public NTable::IScan { public: - TDummyScan(TActorId tablet, bool postponed, EAbort abort, ui32 rows) + TDummyScan(TActorId tablet, bool postponed, EAbort abort, ui32 rows) : TActor(&TThis::StateWork) , Tablet(tablet) , ExpectedRows(rows) @@ -324,7 +324,7 @@ private: } private: - TActorId Tablet; + TActorId Tablet; IDriver *Driver = nullptr; TIntrusiveConstPtr<TScheme> Scheme; ui64 StoredRows = 0; @@ -409,7 +409,7 @@ class TTestFlatTablet : public TActor<TTestFlatTablet>, public TTabletExecutedFl ui64 ScanTaskId; ui64 ScanCookie; ui64 SnapshotId; - TActorId Sender; + TActorId Sender; void SnapshotComplete(TIntrusivePtr<TTableSnapshotContext> snapContext, const TActorContext&) override { Send(Sender, new TEvTestFlatTablet::TEvSnapshotComplete(std::move(snapContext))); @@ -498,7 +498,7 @@ class TTestFlatTablet : public TActor<TTestFlatTablet>, public TTabletExecutedFl } public: - TTestFlatTablet(const TActorId &sender, const TActorId &tablet, TTabletStorageInfo *info) + TTestFlatTablet(const TActorId &sender, const TActorId &tablet, TTabletStorageInfo *info) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, nullptr) , Scan(nullptr) @@ -553,7 +553,7 @@ Y_UNIT_TEST_SUITE(TFlatTableCompactionScan) { env->SetLogPriority(NKikimrServices::RESOURCE_BROKER, NActors::NLog::PRI_DEBUG); - env.FireTablet(env.Edge, env.Tablet, [&env](const TActorId &tablet, TTabletStorageInfo *info) { + env.FireTablet(env.Edge, env.Tablet, [&env](const TActorId &tablet, TTabletStorageInfo *info) { return new TTestFlatTablet(env.Edge, tablet, info); }); @@ -597,7 +597,7 @@ Y_UNIT_TEST_SUITE(TFlatTableCompactionScan) { Y_UNIT_TEST_SUITE(TFlatTableExecutorTxLimit) { struct TTxSchema : public ITransaction { - TTxSchema(TActorId owner) : Owner(owner) { } + TTxSchema(TActorId owner) : Owner(owner) { } bool Execute(TTransactionContext &txc, const TActorContext&) override { @@ -611,11 +611,11 @@ Y_UNIT_TEST_SUITE(TFlatTableExecutorTxLimit) { ctx.Send(Owner, new NFake::TEvResult); } - const TActorId Owner; + const TActorId Owner; }; struct TTxNoop : public ITransaction { - TTxNoop(TActorId owner) : Owner(owner) { } + TTxNoop(TActorId owner) : Owner(owner) { } bool Execute(TTransactionContext&, const TActorContext&) override { @@ -627,7 +627,7 @@ Y_UNIT_TEST_SUITE(TFlatTableExecutorTxLimit) { ctx.Send(Owner, new NFake::TEvResult); } - const TActorId Owner; + const TActorId Owner; }; Y_UNIT_TEST(TestExecutorTxLimit) { @@ -898,7 +898,7 @@ Y_UNIT_TEST_SUITE(TFlatTablePostponedScan) { Env.SetLogPriority(NKikimrServices::TABLET_EXECUTOR, NActors::NLog::PRI_INFO); Env.SetLogPriority(NKikimrServices::TABLET_OPS_HOST, NActors::NLog::PRI_INFO); - FireTablet(Edge, Tablet, [this](const TActorId &tablet, TTabletStorageInfo *info) { + FireTablet(Edge, Tablet, [this](const TActorId &tablet, TTabletStorageInfo *info) { return new TTestFlatTablet(Edge, tablet, info); }); @@ -3363,7 +3363,7 @@ Y_UNIT_TEST_SUITE(TFlatTableExecutorFollower) { WaitForSecondAttach, }; - TActorId Edge; + TActorId Edge; bool Debug = false; TTestActorRuntimeBase::TEventObserver PrevObserverFunc; diff --git a/ydb/core/tablet_flat/flat_ops_compact.h b/ydb/core/tablet_flat/flat_ops_compact.h index 61c9df49e7e..2ac81c4113a 100644 --- a/ydb/core/tablet_flat/flat_ops_compact.h +++ b/ydb/core/tablet_flat/flat_ops_compact.h @@ -89,7 +89,7 @@ namespace NTabletFlatExecutor { } private: - void Registered(TActorSystem *sys, const TActorId&) override + void Registered(TActorSystem *sys, const TActorId&) override { Logger = new NUtil::TLogger(sys, NKikimrServices::OPS_COMPACT); } @@ -511,7 +511,7 @@ namespace NTabletFlatExecutor { private: const TLogoBlobID Mask; - const TActorId Owner; + const TActorId Owner; TAutoPtr<NUtil::ILogger> Logger; IDriver * Driver = nullptr; THolder<TCompactCfg> Conf; diff --git a/ydb/core/tablet_flat/flat_scan_actor.h b/ydb/core/tablet_flat/flat_scan_actor.h index 2b4e30394d4..5cbf2797a03 100644 --- a/ydb/core/tablet_flat/flat_scan_actor.h +++ b/ydb/core/tablet_flat/flat_scan_actor.h @@ -342,7 +342,7 @@ namespace NOps { Y_FAIL("Scan actor got an unexpected EScan value"); } - void Registered(TActorSystem *sys, const TActorId &owner) override + void Registered(TActorSystem *sys, const TActorId &owner) override { Owner = owner; Logger = new NUtil::TLogger(sys, NKikimrServices::TABLET_OPS_HOST); @@ -726,7 +726,7 @@ namespace NOps { const ui64 Serial = 0; const NOps::TConf Args; TAutoPtr<NUtil::ILogger> Logger; - TActorId Owner; + TActorId Owner; THolder<TScanSnapshot> Snapshot; TAutoPtr<TEnv> Cache; /* NFwd scan read ahead cache */ diff --git a/ydb/core/tablet_flat/shared_cache_events.h b/ydb/core/tablet_flat/shared_cache_events.h index 2d4c4ee8906..e7ed55be964 100644 --- a/ydb/core/tablet_flat/shared_cache_events.h +++ b/ydb/core/tablet_flat/shared_cache_events.h @@ -55,7 +55,7 @@ namespace NSharedCache { struct TEvAttach : public TEventLocal<TEvAttach, EvAttach> { TIntrusiveConstPtr<NPageCollection::IPageCollection> PageCollection; - TActorId Owner; + TActorId Owner; TEvAttach(TIntrusiveConstPtr<NPageCollection::IPageCollection> pageCollection, TActorId owner) : PageCollection(std::move(pageCollection)) @@ -68,7 +68,7 @@ namespace NSharedCache { struct TEvRequest : public TEventLocal<TEvRequest, EvRequest> { const EPriority Priority; TAutoPtr<NPageCollection::TFetch> Fetch; - TActorId Owner; + TActorId Owner; TEvRequest(EPriority priority, TAutoPtr<NPageCollection::TFetch> fetch, TActorId owner) : Priority(priority) diff --git a/ydb/core/tablet_flat/shared_sausagecache.cpp b/ydb/core/tablet_flat/shared_sausagecache.cpp index 81117345c98..c60c30202ba 100644 --- a/ydb/core/tablet_flat/shared_sausagecache.cpp +++ b/ydb/core/tablet_flat/shared_sausagecache.cpp @@ -112,7 +112,7 @@ class TSharedPageCache : public TActor<TSharedPageCache> { } const TLogoBlobID Label; - TActorId Source; /* receiver of read results */ + TActorId Source; /* receiver of read results */ TActorId Owner; /* receiver of NBlockIO::TEvStat*/ NBlockIO::EPriority Priority; TIntrusiveConstPtr<NPageCollection::IPageCollection> PageCollection; @@ -129,7 +129,7 @@ class TSharedPageCache : public TActor<TSharedPageCache> { struct TCollection { TLogoBlobID MetaId; - TSet<TActorId> Owners; + TSet<TActorId> Owners; TPageMap<TIntrusivePtr<TPage>> PageMap; TMap<ui32, TExpectant> Expectants; TDeque<ui32> DroppedPages; @@ -149,20 +149,20 @@ class TSharedPageCache : public TActor<TSharedPageCache> { THashMap<TLogoBlobID, TDeque<TPagesToRequest>> Index; }; - TMap<TActorId, TByActorRequest> Requests; + TMap<TActorId, TByActorRequest> Requests; i64 Limit = 0; i64 InFly = 0; - TActorId NextToRequest; + TActorId NextToRequest; }; TIntrusivePtr<TSharedPageGCList> GCList = new TSharedPageGCList; - TActorId Owner; + TActorId Owner; TAutoPtr<NUtil::ILogger> Logger; THashMap<TLogoBlobID, TCollection> Collections; - THashMap<TActorId, TCollectionsOwner> CollectionsOwners; + THashMap<TActorId, TCollectionsOwner> CollectionsOwners; TRequestQueue AsyncRequests; TRequestQueue ScanRequests; @@ -179,7 +179,7 @@ class TSharedPageCache : public TActor<TSharedPageCache> { bool GCScheduled = false; - void Registered(TActorSystem *sys, const TActorId &owner) + void Registered(TActorSystem *sys, const TActorId &owner) { Owner = owner; @@ -426,7 +426,7 @@ class TSharedPageCache : public TActor<TSharedPageCache> { if (queue.Requests.empty()) return; - TMap<TActorId, TRequestQueue::TByActorRequest>::iterator it; + TMap<TActorId, TRequestQueue::TByActorRequest>::iterator it; if (queue.NextToRequest) { it = queue.Requests.find(queue.NextToRequest); if (it == queue.Requests.end()) @@ -521,7 +521,7 @@ class TSharedPageCache : public TActor<TSharedPageCache> { it = queue.Requests.begin(); if (it == queue.Requests.end()) { - queue.NextToRequest = TActorId(); + queue.NextToRequest = TActorId(); break; } @@ -749,7 +749,7 @@ class TSharedPageCache : public TActor<TSharedPageCache> { for (TCollection *collection : recheck) { if (collection->DroppedPages) { // N.B. usually there is a single owner - for (TActorId owner : collection->Owners) { + for (TActorId owner : collection->Owners) { auto& actions = toSend[owner][collection->MetaId]; for (ui32 pageId : collection->DroppedPages) { actions.Dropped.insert(pageId); @@ -793,7 +793,7 @@ class TSharedPageCache : public TActor<TSharedPageCache> { result->Loaded = std::move(wa.ReadyBlocks); Send(wa.Source, result.Release(), 0, wa.EventCookie); - wa.Source = TActorId(); + wa.Source = TActorId(); StatBioReqs += 1; } @@ -812,7 +812,7 @@ class TSharedPageCache : public TActor<TSharedPageCache> { continue; Send(x->Source, new NSharedCache::TEvResult(std::move(x->PageCollection), x->RequestCookie, blobStorageError), 0, x->EventCookie); - x->Source = TActorId(); + x->Source = TActorId(); } } collection.Expectants.clear(); @@ -866,7 +866,7 @@ class TSharedPageCache : public TActor<TSharedPageCache> { collection.PageMap.clear(); } - for (TActorId owner : collection.Owners) { + for (TActorId owner : collection.Owners) { DropRequestsFor(owner, pageCollectionId); } @@ -876,12 +876,12 @@ class TSharedPageCache : public TActor<TSharedPageCache> { } } - void DropRequestsFor(TActorId owner, const TLogoBlobID &metaId) { + void DropRequestsFor(TActorId owner, const TLogoBlobID &metaId) { DropFromQueue(ScanRequests, owner, metaId); DropFromQueue(AsyncRequests, owner, metaId); } - void DropFromQueue(TRequestQueue &queue, TActorId ownerId, const TLogoBlobID &metaId) { + void DropFromQueue(TRequestQueue &queue, TActorId ownerId, const TLogoBlobID &metaId) { auto ownerIt = queue.Requests.find(ownerId); if (ownerIt == queue.Requests.end()) return; @@ -899,7 +899,7 @@ class TSharedPageCache : public TActor<TSharedPageCache> { } } - void DropFromQueue(TRequestQueue &queue, TActorId ownerId) { + void DropFromQueue(TRequestQueue &queue, TActorId ownerId) { auto it = queue.Requests.find(ownerId); if (it != queue.Requests.end()) { if (auto logl = Logger->Log(ELnLev::Debug)) { diff --git a/ydb/core/tablet_flat/shared_sausagecache.h b/ydb/core/tablet_flat/shared_sausagecache.h index aa9e39583cb..f7afb2ee999 100644 --- a/ydb/core/tablet_flat/shared_sausagecache.h +++ b/ydb/core/tablet_flat/shared_sausagecache.h @@ -52,7 +52,7 @@ IActor* CreateSharedPageCache(TSharedPageCacheConfig *config); inline TActorId MakeSharedPageCacheId(ui64 id = 0) { char x[12] = { 's', 'h', 's', 'c' }; WriteUnaligned<ui64>((ui64*)(x+4), id); - return TActorId(0, TStringBuf(x, 12)); + return TActorId(0, TStringBuf(x, 12)); } } diff --git a/ydb/core/tablet_flat/tablet_flat_executed.cpp b/ydb/core/tablet_flat/tablet_flat_executed.cpp index 38a72147de1..4b3c15490c5 100644 --- a/ydb/core/tablet_flat/tablet_flat_executed.cpp +++ b/ydb/core/tablet_flat/tablet_flat_executed.cpp @@ -2,14 +2,14 @@ #include "flat_executor.h" #include "flat_executor_counters.h" #include <ydb/core/base/appdata.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> namespace NKikimr { namespace NTabletFlatExecutor { using IExecutor = NFlatExecutorSetup::IExecutor; -TTabletExecutedFlat::TTabletExecutedFlat(TTabletStorageInfo *info, const TActorId &tablet, IMiniKQLFactory *factory) +TTabletExecutedFlat::TTabletExecutedFlat(TTabletStorageInfo *info, const TActorId &tablet, IMiniKQLFactory *factory) : ITablet(info, tablet) , Factory(factory) , Executor0(nullptr) diff --git a/ydb/core/tablet_flat/tablet_flat_executed.h b/ydb/core/tablet_flat/tablet_flat_executed.h index cb12c009c76..117d7532bb3 100644 --- a/ydb/core/tablet_flat/tablet_flat_executed.h +++ b/ydb/core/tablet_flat/tablet_flat_executed.h @@ -19,7 +19,7 @@ class TTabletExecutedFlat : public NFlatExecutorSetup::ITablet { protected: using IExecutor = NFlatExecutorSetup::IExecutor; - TTabletExecutedFlat(TTabletStorageInfo *info, const TActorId &tablet, IMiniKQLFactory *factory); + TTabletExecutedFlat(TTabletStorageInfo *info, const TActorId &tablet, IMiniKQLFactory *factory); IExecutor* Executor() const { return Executor0; } const TInstant StartTime() const { return StartTime0; } diff --git a/ydb/core/tablet_flat/tablet_flat_executor.cpp b/ydb/core/tablet_flat/tablet_flat_executor.cpp index e8c15e41c64..f69991ece9c 100644 --- a/ydb/core/tablet_flat/tablet_flat_executor.cpp +++ b/ydb/core/tablet_flat/tablet_flat_executor.cpp @@ -37,7 +37,7 @@ namespace NFlatExecutorSetup { return true; } - void ITablet::UpdateTabletInfo(TIntrusivePtr<TTabletStorageInfo> info, const TActorId& launcherID) { + void ITablet::UpdateTabletInfo(TIntrusivePtr<TTabletStorageInfo> info, const TActorId& launcherID) { if (info) TabletInfo = info; diff --git a/ydb/core/tablet_flat/tablet_flat_executor.h b/ydb/core/tablet_flat/tablet_flat_executor.h index 00765470f0c..075ebce354b 100644 --- a/ydb/core/tablet_flat/tablet_flat_executor.h +++ b/ydb/core/tablet_flat/tablet_flat_executor.h @@ -435,9 +435,9 @@ namespace NFlatExecutorSetup { TTabletStorageInfo* Info() const { return TabletInfo.Get(); } ui64 TabletID() const { return TabletInfo->TabletID; } TTabletTypes::EType TabletType() const { return TabletInfo->TabletType; } - const TActorId& Tablet() const { return TabletActorID; } - const TActorId& ExecutorID() const { return ExecutorActorID; } - const TActorId& LauncherID() const { return LauncherActorID; } + const TActorId& Tablet() const { return TabletActorID; } + const TActorId& ExecutorID() const { return ExecutorActorID; } + const TActorId& LauncherID() const { return LauncherActorID; } virtual void SnapshotComplete(TIntrusivePtr<TTableSnapshotContext> snapContext, const TActorContext &ctx); // would be FAIL in default implementation virtual void CompletedLoansChanged(const TActorContext &ctx); // would be no-op in default implementation @@ -454,18 +454,18 @@ namespace NFlatExecutorSetup { // create transaction? protected: - ITablet(TTabletStorageInfo *info, const TActorId &tablet) + ITablet(TTabletStorageInfo *info, const TActorId &tablet) : TabletActorID(tablet) , TabletInfo(info) { Y_VERIFY(TTabletTypes::TYPE_INVALID != TabletInfo->TabletType); } - TActorId ExecutorActorID; - TActorId TabletActorID; - TActorId LauncherActorID; + TActorId ExecutorActorID; + TActorId TabletActorID; + TActorId LauncherActorID; - void UpdateTabletInfo(TIntrusivePtr<TTabletStorageInfo> info, const TActorId& launcherID = {}); + void UpdateTabletInfo(TIntrusivePtr<TTabletStorageInfo> info, const TActorId& launcherID = {}); private: TIntrusivePtr<TTabletStorageInfo> TabletInfo; }; diff --git a/ydb/core/tablet_flat/test/libs/exec/dummy.h b/ydb/core/tablet_flat/test/libs/exec/dummy.h index d768a2924af..5c6e2f5b4f1 100644 --- a/ydb/core/tablet_flat/test/libs/exec/dummy.h +++ b/ydb/core/tablet_flat/test/libs/exec/dummy.h @@ -32,7 +32,7 @@ namespace NFake { Comp = 0x01, }; - TDummy(const TActorId &tablet, TInfo *info, const TActorId& owner, + TDummy(const TActorId &tablet, TInfo *info, const TActorId& owner, ui32 flags = 0 /* ORed EFlg enum */) : ::NActors::IActor(static_cast<TReceiveFunc>(&TDummy::Inbox), NKikimrServices::TActivity::FAKE_ENV_A) , TTabletExecutedFlat(info, tablet, nullptr) @@ -128,7 +128,7 @@ namespace NFake { } private: - TActorId Owner; + TActorId Owner; const ui32 Flags = 0; EState State = EState::Boot; }; diff --git a/ydb/core/tablet_flat/test/libs/exec/events.h b/ydb/core/tablet_flat/test/libs/exec/events.h index eedeebe478d..b6aeb3e82f6 100644 --- a/ydb/core/tablet_flat/test/libs/exec/events.h +++ b/ydb/core/tablet_flat/test/libs/exec/events.h @@ -26,7 +26,7 @@ namespace NFake { struct TEvTerm : public TEventLocal<TEvTerm, EvTerm> { }; struct TEvFire : public TEventLocal<TEvFire, EvFire> { - TEvFire(ui32 level, const TActorId &alias, TActorSetupCmd cmd) + TEvFire(ui32 level, const TActorId &alias, TActorSetupCmd cmd) : Level(level) , Alias(alias) , Cmd(cmd) @@ -35,17 +35,17 @@ namespace NFake { } const ui32 Level = Max<ui32>(); - const TActorId Alias; + const TActorId Alias; TActorSetupCmd Cmd; }; struct TEvReady : public TEventLocal<TEvReady, EvReady> { - TEvReady(ui64 tabletId, const TActorId& tabletActorID) + TEvReady(ui64 tabletId, const TActorId& tabletActorID) : TabletID(tabletId) , ActorId(tabletActorID) {} ui64 TabletID; - TActorId ActorId; + TActorId ActorId; }; struct TEvExecute : public TEventLocal<TEvExecute, EvExecute> { diff --git a/ydb/core/tablet_flat/test/libs/exec/leader.h b/ydb/core/tablet_flat/test/libs/exec/leader.h index b4f2485d3c7..f964c4b33c3 100644 --- a/ydb/core/tablet_flat/test/libs/exec/leader.h +++ b/ydb/core/tablet_flat/test/libs/exec/leader.h @@ -32,7 +32,7 @@ namespace NFake { } private: - void Registered(TActorSystem *sys, const TActorId&) override + void Registered(TActorSystem *sys, const TActorId&) override { Sys = sys, Start = Time->Now(); @@ -74,7 +74,7 @@ namespace NFake { } } - void HandleGone(const TActorId &actor) noexcept + void HandleGone(const TActorId &actor) noexcept { auto it = Childs.find(actor); @@ -94,7 +94,7 @@ namespace NFake { } } - void DoFire(ui32 level, const TActorId &alias, TActorSetupCmd &cmd) + void DoFire(ui32 level, const TActorId &alias, TActorSetupCmd &cmd) { if (level <= Edge && Levels[level].Alive) { auto actor = Register(cmd.Actor, cmd.MailboxType, 0); @@ -180,7 +180,7 @@ namespace NFake { ui32 Head = 0; ui64 Total = 0; TAtomic &Stopped; - std::map<TActorId, ui32> Childs; + std::map<TActorId, ui32> Childs; std::array<TLevel, 9> Levels; }; } diff --git a/ydb/core/tablet_flat/test/libs/exec/nanny.h b/ydb/core/tablet_flat/test/libs/exec/nanny.h index 98536805967..9b00928c8e4 100644 --- a/ydb/core/tablet_flat/test/libs/exec/nanny.h +++ b/ydb/core/tablet_flat/test/libs/exec/nanny.h @@ -17,7 +17,7 @@ namespace NFake { using TContext = NTabletFlatExecutor::TTransactionContext; using TCall = std::function<bool(NTable::ITestDb&, TContext&)>; - TFuncTx(NTable::ITestDb &fake, const TActorId& owner, TCall func) + TFuncTx(NTable::ITestDb &fake, const TActorId& owner, TCall func) : Owner(owner) , Func(std::move(func)) , Fake(fake) @@ -57,7 +57,7 @@ namespace NFake { } private: - const TActorId Owner; + const TActorId Owner; TCall Func; bool Completed = false; NTable::ITestDb &Fake; @@ -109,7 +109,7 @@ namespace NFake { private: virtual EDo Run() = 0; - void Registered(TActorSystem *sys, const TActorId &owner) override + void Registered(TActorSystem *sys, const TActorId &owner) override { Logger = new NUtil::TLogger(sys, NKikimrServices::FAKE_ENV); @@ -176,7 +176,7 @@ namespace NFake { logl << "TNanny initiates TDummy tablet " << MyId << " birth"; } - auto make = [this](const TActorId &tablet, TStorageInfo *info) { + auto make = [this](const TActorId &tablet, TStorageInfo *info) { return new NFake::TDummy(tablet, info, SelfId()); }; @@ -196,8 +196,8 @@ namespace NFake { private: const ui64 MyId = MakeTabletID(0, 0, 1); TAutoPtr<NUtil::ILogger> Logger; - TActorId Owner; - TActorId Tablet; + TActorId Owner; + TActorId Tablet; ui64 TxInFlight = 0; EDo State = EDo::Born; TAutoPtr<NTable::ITestDb> Fake; diff --git a/ydb/core/tablet_flat/test/libs/exec/owner.h b/ydb/core/tablet_flat/test/libs/exec/owner.h index 2df7c9ec204..f3e3d7363c8 100644 --- a/ydb/core/tablet_flat/test/libs/exec/owner.h +++ b/ydb/core/tablet_flat/test/libs/exec/owner.h @@ -35,7 +35,7 @@ namespace NFake { } private: - void Registered(TActorSystem *sys, const TActorId &owner) override + void Registered(TActorSystem *sys, const TActorId &owner) override { Owner = owner; Logger = new NUtil::TLogger(sys, NKikimrServices::FAKE_ENV); @@ -117,9 +117,9 @@ namespace NFake { TIntrusivePtr<TSetup> Setup; TAutoPtr<NUtil::ILogger> Logger; - TActorId Owner; - TActorId Agent; - TActorId User; + TActorId Owner; + TActorId Agent; + TActorId User; ui32 Borns = 0; ui32 Limit = 1; const ui32 FollowerId; diff --git a/ydb/core/tablet_flat/test/libs/exec/runner.h b/ydb/core/tablet_flat/test/libs/exec/runner.h index b2e9d5a3af8..3b58ab814b4 100644 --- a/ydb/core/tablet_flat/test/libs/exec/runner.h +++ b/ydb/core/tablet_flat/test/libs/exec/runner.h @@ -39,7 +39,7 @@ namespace NFake { auto *logger = new NFake::TLogFwd(Sink); - AddService(TActorId(NodeId, "logger"), logger, EMail::Simple); + AddService(TActorId(NodeId, "logger"), logger, EMail::Simple); SetupStaticServices(); @@ -78,7 +78,7 @@ namespace NFake { FireTablet(user, tablet, make, followerId); } - void AddService(TActorId service, IActor *actor, EMail box) + void AddService(TActorId service, IActor *actor, EMail box) { Env.AddLocalService(service, TActorSetupCmd(actor, box, 0), 0); } @@ -88,7 +88,7 @@ namespace NFake { return RunOn(8, { }, actor.Release(), EMail::Simple); } - void RunOn(ui32 lvl, TActorId alias, IActor* actor, EMail box) + void RunOn(ui32 lvl, TActorId alias, IActor* actor, EMail box) { auto *event = new NFake::TEvFire{ lvl, alias, { actor, box, 0 } }; diff --git a/ydb/core/tablet_flat/test/libs/exec/storage.h b/ydb/core/tablet_flat/test/libs/exec/storage.h index 95958d98591..76a937c4f8b 100644 --- a/ydb/core/tablet_flat/test/libs/exec/storage.h +++ b/ydb/core/tablet_flat/test/libs/exec/storage.h @@ -22,7 +22,7 @@ namespace NFake { } private: - void Registered(TActorSystem *sys, const TActorId &owner) override + void Registered(TActorSystem *sys, const TActorId &owner) override { Owner = owner; @@ -96,7 +96,7 @@ namespace NFake { private: const ui32 Group = NPageCollection::TLargeGlobId::InvalidGroup; - TActorId Owner; + TActorId Owner; TAutoPtr<NUtil::ILogger> Logger; TAutoPtr<NFake::TProxyDS> Model; diff --git a/ydb/core/tablet_flat/test/libs/exec/warden.h b/ydb/core/tablet_flat/test/libs/exec/warden.h index beec60df981..cc2afbde883 100644 --- a/ydb/core/tablet_flat/test/libs/exec/warden.h +++ b/ydb/core/tablet_flat/test/libs/exec/warden.h @@ -37,7 +37,7 @@ namespace NFake { } private: - void Registered(TActorSystem *sys, const TActorId &owner) override + void Registered(TActorSystem *sys, const TActorId &owner) override { Sys = sys, Owner = owner; diff --git a/ydb/core/tablet_flat/test/libs/exec/world.h b/ydb/core/tablet_flat/test/libs/exec/world.h index 261fbfdb053..576ff162f26 100644 --- a/ydb/core/tablet_flat/test/libs/exec/world.h +++ b/ydb/core/tablet_flat/test/libs/exec/world.h @@ -15,14 +15,14 @@ namespace NFake { struct TWorld { - static NActors::TActorId Where(EPath path) noexcept + static NActors::TActorId Where(EPath path) noexcept { std::array<char,13> token{ "NFakeWorld??" }; token[10] = 0xff & ui16(path); token[11] = 0xff & (ui16(path) >> 8); - return NActors::TActorId(0, TStringBuf(token.begin(), 12)); + return NActors::TActorId(0, TStringBuf(token.begin(), 12)); } }; diff --git a/ydb/core/tablet_flat/ut/flat_test_db_helpers.h b/ydb/core/tablet_flat/ut/flat_test_db_helpers.h index 867fe02a463..ffc9a36daac 100644 --- a/ydb/core/tablet_flat/ut/flat_test_db_helpers.h +++ b/ydb/core/tablet_flat/ut/flat_test_db_helpers.h @@ -62,12 +62,12 @@ inline TFakeTableCell FromVal(NScheme::TTypeId t, i64 val) { TFakeTableCell c; ui32 sz = sizeof(val); switch (t) { - case NScheme::NTypeIds::Byte: - case NScheme::NTypeIds::Bool: + case NScheme::NTypeIds::Byte: + case NScheme::NTypeIds::Bool: sz = 1; break; - case NScheme::NTypeIds::Int32: - case NScheme::NTypeIds::Uint32: + case NScheme::NTypeIds::Int32: + case NScheme::NTypeIds::Uint32: sz = 4; break; } diff --git a/ydb/core/tablet_flat/util_fmt_logger.h b/ydb/core/tablet_flat/util_fmt_logger.h index 20933736ce3..784e1795fd5 100644 --- a/ydb/core/tablet_flat/util_fmt_logger.h +++ b/ydb/core/tablet_flat/util_fmt_logger.h @@ -15,7 +15,7 @@ namespace NUtil { using TSys = ::NActors::TActorSystem; using EComp = ::NActors::NLog::EComponent; using EPrio = ::NActors::NLog::EPrio; - using TPath = ::NActors::TActorId; + using TPath = ::NActors::TActorId; TLogger(TSys *sys, EComp comp) : Time(TAppData::TimeProvider.Get()) diff --git a/ydb/core/testlib/actor_helpers.h b/ydb/core/testlib/actor_helpers.h index 8c3ef2c0806..b9df571aca0 100644 --- a/ydb/core/testlib/actor_helpers.h +++ b/ydb/core/testlib/actor_helpers.h @@ -10,7 +10,7 @@ struct TActorSystemStub { THolder<NActors::TActorSystem> System; THolder<NActors::TMailboxHeader> Mailbox; THolder<NActors::TExecutorThread> ExecutorThread; - NActors::TActorId SelfID; + NActors::TActorId SelfID; THolder<NActors::TActorContext> Ctx; NActors::TActivationContext* PrevCtx; diff --git a/ydb/core/testlib/actors/test_runtime.cpp b/ydb/core/testlib/actors/test_runtime.cpp index 2019d1e2e45..63f0436bff7 100644 --- a/ydb/core/testlib/actors/test_runtime.cpp +++ b/ydb/core/testlib/actors/test_runtime.cpp @@ -103,7 +103,7 @@ namespace NActors { if (!SingleSysEnv) { const TIntrusivePtr<NMonitoring::TDynamicCounters> profilerCounters = NKikimr::GetServiceCounters(node->DynamicCounters, "utils"); TActorSetupCmd profilerSetup(CreateProfilerActor(profilerCounters, "."), TMailboxType::Simple, 0); - node->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeProfilerID(FirstNodeId + nodeIndex), profilerSetup)); + node->LocalServices.push_back(std::pair<TActorId, TActorSetupCmd>(MakeProfilerID(FirstNodeId + nodeIndex), profilerSetup)); } if (!UseRealThreads) { @@ -153,7 +153,7 @@ namespace NActors { node->Mon->RegisterCountersPage("counters", "Counters", node->DynamicCounters); auto actorsMonPage = node->Mon->RegisterIndexPage("actors", "Actors"); node->Mon->RegisterActorPage(actorsMonPage, "profiler", "Profiler", false, node->ActorSystem.Get(), MakeProfilerID(FirstNodeId + nodeIndex)); - const NActors::TActorId loggerActorId = NActors::TActorId(FirstNodeId + nodeIndex, "logger"); + const NActors::TActorId loggerActorId = NActors::TActorId(FirstNodeId + nodeIndex, "logger"); node->Mon->RegisterActorPage(actorsMonPage, "logger", "Logger", false, node->ActorSystem.Get(), loggerActorId); MonPorts.push_back(port); } @@ -209,8 +209,8 @@ namespace NActors { return true; } - void TTestActorRuntime::SendToPipe(ui64 tabletId, const TActorId& sender, IEventBase* payload, ui32 nodeIndex, const NKikimr::NTabletPipe::TClientConfig& pipeConfig, TActorId clientId, ui64 cookie) { - bool newPipe = (clientId == TActorId()); + void TTestActorRuntime::SendToPipe(ui64 tabletId, const TActorId& sender, IEventBase* payload, ui32 nodeIndex, const NKikimr::NTabletPipe::TClientConfig& pipeConfig, TActorId clientId, ui64 cookie) { + bool newPipe = (clientId == TActorId()); if (newPipe) clientId = Register(NKikimr::NTabletPipe::CreateClient(sender, tabletId, pipeConfig), nodeIndex); if (!UseRealThreads) { @@ -224,15 +224,15 @@ namespace NActors { Send(new IEventHandle(clientId, sender, new NKikimr::TEvTabletPipe::TEvShutdown()), nodeIndex, true); } - void TTestActorRuntime::SendToPipe(TActorId clientId, const TActorId& sender, IEventBase* payload, + void TTestActorRuntime::SendToPipe(TActorId clientId, const TActorId& sender, IEventBase* payload, ui32 nodeIndex, ui64 cookie) { auto pipeEv = new IEventHandle(clientId, sender, payload, 0, cookie); pipeEv->Rewrite(NKikimr::TEvTabletPipe::EvSend, clientId); Send(pipeEv, nodeIndex, true); } - TActorId TTestActorRuntime::ConnectToPipe(ui64 tabletId, const TActorId& sender, ui32 nodeIndex, const NKikimr::NTabletPipe::TClientConfig& pipeConfig) { - TActorId clientId = Register(NKikimr::NTabletPipe::CreateClient(sender, tabletId, pipeConfig), nodeIndex); + TActorId TTestActorRuntime::ConnectToPipe(ui64 tabletId, const TActorId& sender, ui32 nodeIndex, const NKikimr::NTabletPipe::TClientConfig& pipeConfig) { + TActorId clientId = Register(NKikimr::NTabletPipe::CreateClient(sender, tabletId, pipeConfig), nodeIndex); if (!UseRealThreads) { EnableScheduleForActor(clientId, true); } diff --git a/ydb/core/testlib/actors/test_runtime.h b/ydb/core/testlib/actors/test_runtime.h index d0f7151cc97..deb156876bf 100644 --- a/ydb/core/testlib/actors/test_runtime.h +++ b/ydb/core/testlib/actors/test_runtime.h @@ -56,11 +56,11 @@ namespace NActors { ui16 GetMonPort(ui32 nodeIndex = 0) const; - void SendToPipe(ui64 tabletId, const TActorId& sender, IEventBase* payload, ui32 nodeIndex = 0, - const NKikimr::NTabletPipe::TClientConfig& pipeConfig = NKikimr::NTabletPipe::TClientConfig(), TActorId clientId = TActorId(), ui64 cookie = 0); - void SendToPipe(TActorId clientId, const TActorId& sender, IEventBase* payload, + void SendToPipe(ui64 tabletId, const TActorId& sender, IEventBase* payload, ui32 nodeIndex = 0, + const NKikimr::NTabletPipe::TClientConfig& pipeConfig = NKikimr::NTabletPipe::TClientConfig(), TActorId clientId = TActorId(), ui64 cookie = 0); + void SendToPipe(TActorId clientId, const TActorId& sender, IEventBase* payload, ui32 nodeIndex = 0, ui64 cookie = 0); - TActorId ConnectToPipe(ui64 tabletId, const TActorId& sender, ui32 nodeIndex, const NKikimr::NTabletPipe::TClientConfig& pipeConfig); + TActorId ConnectToPipe(ui64 tabletId, const TActorId& sender, ui32 nodeIndex, const NKikimr::NTabletPipe::TClientConfig& pipeConfig); NKikimr::TAppData& GetAppData(ui32 nodeIndex = 0); TPortManager& GetPortManager() { diff --git a/ydb/core/testlib/actors/test_runtime_ut.cpp b/ydb/core/testlib/actors/test_runtime_ut.cpp index 7082de6c8cc..fce7f192934 100644 --- a/ydb/core/testlib/actors/test_runtime_ut.cpp +++ b/ydb/core/testlib/actors/test_runtime_ut.cpp @@ -18,7 +18,7 @@ Y_UNIT_TEST_SUITE(TActorTest) { Y_UNIT_TEST(TestHandleEvent) { class TMyActor : public TActor<TMyActor> { public: - TMyActor(const TActorId& sender) + TMyActor(const TActorId& sender) : TActor(&TMyActor::StateFunc) , Sender(sender) { @@ -32,13 +32,13 @@ Y_UNIT_TEST_SUITE(TActorTest) { } private: - const TActorId Sender; + const TActorId Sender; }; TTestActorRuntime runtime; runtime.Initialize(MakeEgg()); - TActorId sender = runtime.AllocateEdgeActor(); - TActorId actorId = runtime.Register(new TMyActor(sender)); + TActorId sender = runtime.AllocateEdgeActor(); + TActorId actorId = runtime.Register(new TMyActor(sender)); runtime.Send(new IEventHandle(actorId, sender, new TEvents::TEvWakeup)); } @@ -59,8 +59,8 @@ Y_UNIT_TEST_SUITE(TActorTest) { TTestActorRuntime runtime; runtime.Initialize(MakeEgg()); - TActorId sender = runtime.AllocateEdgeActor(); - TActorId actorId = runtime.Register(new TMyActor); + TActorId sender = runtime.AllocateEdgeActor(); + TActorId actorId = runtime.Register(new TMyActor); runtime.Send(new IEventHandle(actorId, sender, new TEvents::TEvPoisonPill)); } @@ -97,9 +97,9 @@ Y_UNIT_TEST_SUITE(TActorTest) { TTestActorRuntime runtime; runtime.Initialize(MakeEgg()); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); auto actor = new TMyActor; - TActorId actorId = runtime.Register(actor); + TActorId actorId = runtime.Register(actor); runtime.Send(new IEventHandle(actorId, sender, new TEvents::TEvPing)); UNIT_ASSERT_EQUAL(actor->IsStateChanged(), false); runtime.Send(new IEventHandle(actorId, sender, new TEvents::TEvPong)); @@ -122,8 +122,8 @@ Y_UNIT_TEST_SUITE(TActorTest) { TTestActorRuntime runtime; runtime.Initialize(MakeEgg()); - TActorId sender = runtime.AllocateEdgeActor(); - TActorId actorId = runtime.Register(new TMyActor); + TActorId sender = runtime.AllocateEdgeActor(); + TActorId actorId = runtime.Register(new TMyActor); runtime.Send(new IEventHandle(actorId, sender, new TEvents::TEvPing)); auto events = runtime.CaptureEvents(); bool passed = false; @@ -161,8 +161,8 @@ Y_UNIT_TEST_SUITE(TActorTest) { return false; }); - TActorId sender = runtime.AllocateEdgeActor(); - TActorId actorId = runtime.Register(new TMyActor); + TActorId sender = runtime.AllocateEdgeActor(); + TActorId actorId = runtime.Register(new TMyActor); runtime.Send(new IEventHandle(actorId, sender, new TEvents::TEvPing)); auto scheduledEvents = runtime.CaptureScheduledEvents(); UNIT_ASSERT_EQUAL_C(scheduledEvents.size(), 1, "check scheduled count"); @@ -199,19 +199,19 @@ Y_UNIT_TEST_SUITE(TActorTest) { ctx.Send(ChildId, new TEvents::TEvPing()); } - TActorId GetChildId() const { + TActorId GetChildId() const { return ChildId; } private: - TActorId ChildId; + TActorId ChildId; }; TTestActorRuntime runtime; runtime.Initialize(MakeEgg()); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); auto actor = new TMyActor; - TActorId actorId = runtime.Register(actor); + TActorId actorId = runtime.Register(actor); runtime.Send(new IEventHandle(actorId, sender, new TEvents::TEvWakeup)); auto events = runtime.CaptureEvents(); bool passed = false; @@ -270,9 +270,9 @@ Y_UNIT_TEST_SUITE(TActorTest) { TTestActorRuntime runtime; runtime.Initialize(MakeEgg()); try { - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); auto myActor = new TMyActor(&syncMutex); - TActorId actorId = runtime.Register(myActor); + TActorId actorId = runtime.Register(myActor); runtime.Send(new IEventHandle(actorId, sender, new TEvents::TEvPing)); runtime.DispatchEvents(); auto events = runtime.CaptureEvents(); @@ -318,7 +318,7 @@ Y_UNIT_TEST_SUITE(TActorTest) { } private: - TActorId Sender; + TActorId Sender; TInstant CurrentTime; }; @@ -333,9 +333,9 @@ Y_UNIT_TEST_SUITE(TActorTest) { }); runtime.SetScheduledEventsSelectorFunc(&TTestActorRuntimeBase::CollapsedTimeScheduledEventsSelector); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); auto myActor = new TMyActor; - TActorId actorId = runtime.Register(myActor); + TActorId actorId = runtime.Register(myActor); runtime.Send(new IEventHandle(actorId, sender, new TEvents::TEvPing)); runtime.DispatchEvents(); auto events = runtime.CaptureEvents(); @@ -391,7 +391,7 @@ Y_UNIT_TEST_SUITE(TActorTest) { class TProducerActor : public TActor<TProducerActor> { public: - TProducerActor(ui32 count, const TVector<TActorId>& recipents) + TProducerActor(ui32 count, const TVector<TActorId>& recipents) : TActor(&TProducerActor::StateFunc) , Count(count) , Recipents(recipents) @@ -428,30 +428,30 @@ Y_UNIT_TEST_SUITE(TActorTest) { private: const ui32 Count; - const TVector<TActorId> Recipents; + const TVector<TActorId> Recipents; TAutoPtr<TThread> Thread; TActorSystem* ActorSystem = nullptr; - TActorId SelfId; + TActorId SelfId; }; TTestActorRuntime runtime; runtime.Initialize(MakeEgg()); try { - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); ui32 count = 1000; ui32 consumersCount = 4; TVector<TConsumerActor*> consumers; - TVector<TActorId> consumerIds; + TVector<TActorId> consumerIds; for (ui32 i = 0; i < consumersCount; ++i) { auto consumerActor = new TConsumerActor(count); - TActorId consumerId = runtime.Register(consumerActor); + TActorId consumerId = runtime.Register(consumerActor); consumers.push_back(consumerActor); consumerIds.push_back(consumerId); } auto producerActor = new TProducerActor(count, consumerIds); - TActorId producerId = runtime.Register(producerActor); + TActorId producerId = runtime.Register(producerActor); runtime.Send(new IEventHandle(producerId, sender, new TEvents::TEvPing)); runtime.SetObserverFunc([](TTestActorRuntimeBase& runtime, TAutoPtr<IEventHandle>& event) { Y_UNUSED(runtime); @@ -517,15 +517,15 @@ Y_UNIT_TEST_SUITE(TActorTest) { TTestActorRuntime runtime; runtime.Initialize(MakeEgg()); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); TMyActor* myActor = new TMyActor; - TActorId actorId = runtime.Register(myActor); + TActorId actorId = runtime.Register(myActor); runtime.EnableScheduleForActor(actorId); runtime.Send(new IEventHandle(actorId, sender, new TEvents::TEvBootstrap())); TAutoPtr<IEventHandle> handle; UNIT_ASSERT(runtime.GrabEdgeEventRethrow<TEvents::TEvCompleted>(handle)); - runtime.Send(new IEventHandle(actorId, TActorId(), new TEvents::TEvPing())); - runtime.Schedule(new IEventHandle(sender, TActorId(), new TEvents::TEvWakeup()), + runtime.Send(new IEventHandle(actorId, TActorId(), new TEvents::TEvPing())); + runtime.Schedule(new IEventHandle(sender, TActorId(), new TEvents::TEvWakeup()), TDuration::MilliSeconds(1000000)); do { runtime.GrabEdgeEventRethrow<TEvents::TEvWakeup>(handle); @@ -548,11 +548,11 @@ Y_UNIT_TEST_SUITE(TActorTest) { class TCountingActor : public TActorBootstrapped<TCountingActor> { private: - TVector<TActorId> Targets; + TVector<TActorId> Targets; ui32 Counter = 0; public: - TCountingActor(TVector<TActorId> targets) + TCountingActor(TVector<TActorId> targets) : Targets(targets) {} @@ -578,10 +578,10 @@ Y_UNIT_TEST_SUITE(TActorTest) { TTestActorRuntime runtime; runtime.Initialize(MakeEgg()); - TActorId edge1 = runtime.AllocateEdgeActor(); - TActorId edge2 = runtime.AllocateEdgeActor(); - TActorId edge3 = runtime.AllocateEdgeActor(); - TActorId countingActor = runtime.Register(new TCountingActor({edge1, edge2})); + TActorId edge1 = runtime.AllocateEdgeActor(); + TActorId edge2 = runtime.AllocateEdgeActor(); + TActorId edge3 = runtime.AllocateEdgeActor(); + TActorId countingActor = runtime.Register(new TCountingActor({edge1, edge2})); runtime.EnableScheduleForActor(countingActor); // Ignores edge1 event diff --git a/ydb/core/testlib/basics/helpers.cpp b/ydb/core/testlib/basics/helpers.cpp index 47ca7dfd141..259e4f9da94 100644 --- a/ydb/core/testlib/basics/helpers.cpp +++ b/ydb/core/testlib/basics/helpers.cpp @@ -25,7 +25,7 @@ namespace NKikimr { } TActorId CreateTestBootstrapper(TTestActorRuntime &runtime, TTabletStorageInfo *info, - std::function<IActor* (const TActorId &, TTabletStorageInfo*)> op, ui32 nodeIndex) + std::function<IActor* (const TActorId &, TTabletStorageInfo*)> op, ui32 nodeIndex) { TIntrusivePtr<TBootstrapperInfo> bi(new TBootstrapperInfo(new TTabletSetupInfo(op, TMailboxType::Simple, 0, TMailboxType::Simple, 0))); return runtime.Register(CreateBootstrapper(info, bi.Get()), nodeIndex); @@ -73,16 +73,16 @@ namespace NKikimr { Y_VERIFY(!Runtime.IsRealThreads()); Y_VERIFY(nodeId >= Runtime.GetNodeId(0) && nodeId < Runtime.GetNodeId(0) + Runtime.GetNodeCount()); ui32 nodeIndex = nodeId - Runtime.GetNodeId(0); - Runtime.BlockOutputForActor(TActorId(nodeId, "actorsystem")); + Runtime.BlockOutputForActor(TActorId(nodeId, "actorsystem")); TActorId actorId = Runtime.Register(CreatePDisk(cfg, mainKey, Runtime.GetAppData(0).Counters), nodeIndex, poolId, TMailboxType::Revolving); - TActorId pDiskServiceId = MakeBlobStoragePDiskID(nodeId, pDiskID); + TActorId pDiskServiceId = MakeBlobStoragePDiskID(nodeId, pDiskID); Runtime.BlockOutputForActor(pDiskServiceId); Runtime.BlockOutputForActor(actorId); auto factory = CreateStrandingDecoratorFactory(&Runtime, []{ return MakeHolder<TPDiskReplyChecker>(); }); - IActor* wrappedActor = factory->Wrap(actorId, true, TVector<TActorId>()); - TActorId wrappedActorId = Runtime.Register(wrappedActor, nodeIndex, poolId, TMailboxType::Revolving); + IActor* wrappedActor = factory->Wrap(actorId, true, TVector<TActorId>()); + TActorId wrappedActorId = Runtime.Register(wrappedActor, nodeIndex, poolId, TMailboxType::Revolving); Runtime.RegisterService(pDiskServiceId, wrappedActorId, nodeIndex); } } diff --git a/ydb/core/testlib/basics/helpers.h b/ydb/core/testlib/basics/helpers.h index b3ac3b3af57..7a70063f7a3 100644 --- a/ydb/core/testlib/basics/helpers.h +++ b/ydb/core/testlib/basics/helpers.h @@ -35,7 +35,7 @@ namespace NFake { TTabletStorageInfo* CreateTestTabletInfo(ui64 tabletId, TTabletTypes::EType tabletType, TBlobStorageGroupType::EErasureSpecies erasure = BootGroupErasure, ui32 groupId = 0); TActorId CreateTestBootstrapper(TTestActorRuntime &runtime, TTabletStorageInfo *info, - std::function<IActor* (const TActorId &, TTabletStorageInfo*)> op, ui32 nodeIndex = 0); + std::function<IActor* (const TActorId &, TTabletStorageInfo*)> op, ui32 nodeIndex = 0); NTabletPipe::TClientConfig GetPipeConfigWithRetries(); void SetupStateStorage(TTestActorRuntime& runtime, ui32 nodeIndex, diff --git a/ydb/core/testlib/basics/runtime.cpp b/ydb/core/testlib/basics/runtime.cpp index 561bfadb287..cd62ce82bfc 100644 --- a/ydb/core/testlib/basics/runtime.cpp +++ b/ydb/core/testlib/basics/runtime.cpp @@ -41,7 +41,7 @@ namespace NActors { } const TActorId dnsId = NDnsResolver::MakeDnsResolverActorId(); - const TActorId namesId = GetNameserviceActorId(); + const TActorId namesId = GetNameserviceActorId(); for (auto num : xrange(GetNodeCount())) { auto* node = GetRawNode(num); @@ -66,7 +66,7 @@ namespace NActors { common->AcceptUUID = {ClusterUUID}; if (UseRealInterconnect) { - auto listener = new TInterconnectListenerTCP(nameNode.first, nameNode.second, common); + auto listener = new TInterconnectListenerTCP(nameNode.first, nameNode.second, common); AddLocalService({}, TActorSetupCmd(listener, TMailboxType::Simple, InterconnectPoolId()), num); AddLocalService(MakePollerActorId(), TActorSetupCmd(CreatePollerActor(), TMailboxType::Simple, 0), num); } diff --git a/ydb/core/testlib/basics/services.cpp b/ydb/core/testlib/basics/services.cpp index 690840d3cce..b4585ad96c6 100644 --- a/ydb/core/testlib/basics/services.cpp +++ b/ydb/core/testlib/basics/services.cpp @@ -151,7 +151,7 @@ namespace NPDisk { } template<size_t N> - static TIntrusivePtr<TStateStorageInfo> GenerateStateStorageInfo(const TActorId (&replicas)[N], ui64 stateStorageGroup) + static TIntrusivePtr<TStateStorageInfo> GenerateStateStorageInfo(const TActorId (&replicas)[N], ui64 stateStorageGroup) { TIntrusivePtr<TStateStorageInfo> info(new TStateStorageInfo()); info->StateStorageGroup = stateStorageGroup; @@ -164,7 +164,7 @@ namespace NPDisk { return info; } - static TActorId MakeBoardReplicaID( + static TActorId MakeBoardReplicaID( const ui32 node, const ui64 stateStorageGroup, const ui32 replicaIndex @@ -172,30 +172,30 @@ namespace NPDisk { char x[12] = { 's', 's', 'b' }; x[3] = (char)stateStorageGroup; memcpy(x + 5, &replicaIndex, sizeof(ui32)); - return TActorId(node, TStringBuf(x, 12)); + return TActorId(node, TStringBuf(x, 12)); } void SetupStateStorage(TTestActorRuntime& runtime, ui32 nodeIndex, ui64 stateStorageGroup, bool firstNode) { - const TActorId ssreplicas[3] = { + const TActorId ssreplicas[3] = { MakeStateStorageReplicaID(runtime.GetNodeId(0), stateStorageGroup, 0), MakeStateStorageReplicaID(runtime.GetNodeId(0), stateStorageGroup, 1), MakeStateStorageReplicaID(runtime.GetNodeId(0), stateStorageGroup, 2), }; - const TActorId breplicas[3] = { + const TActorId breplicas[3] = { MakeBoardReplicaID(runtime.GetNodeId(0), stateStorageGroup, 0), MakeBoardReplicaID(runtime.GetNodeId(0), stateStorageGroup, 1), MakeBoardReplicaID(runtime.GetNodeId(0), stateStorageGroup, 2), }; - const TActorId sbreplicas[3] = { + const TActorId sbreplicas[3] = { MakeSchemeBoardReplicaID(runtime.GetNodeId(0), stateStorageGroup, 0), MakeSchemeBoardReplicaID(runtime.GetNodeId(0), stateStorageGroup, 1), MakeSchemeBoardReplicaID(runtime.GetNodeId(0), stateStorageGroup, 2), }; - const TActorId ssproxy = MakeStateStorageProxyID(stateStorageGroup); + const TActorId ssproxy = MakeStateStorageProxyID(stateStorageGroup); auto ssInfo = GenerateStateStorageInfo(ssreplicas, stateStorageGroup); auto sbInfo = GenerateStateStorageInfo(sbreplicas, stateStorageGroup); diff --git a/ydb/core/testlib/fake_coordinator.cpp b/ydb/core/testlib/fake_coordinator.cpp index 7c4fc6a8183..0be35b3bcfe 100644 --- a/ydb/core/testlib/fake_coordinator.cpp +++ b/ydb/core/testlib/fake_coordinator.cpp @@ -5,7 +5,7 @@ namespace NKikimr { void BootFakeCoordinator(TTestActorRuntime& runtime, ui64 coordinatorId, TFakeCoordinator::TState::TPtr state) { CreateTestBootstrapper(runtime, CreateTestTabletInfo(coordinatorId, TTabletTypes::FLAT_TX_COORDINATOR), - [=](const TActorId & tablet, TTabletStorageInfo* info) { + [=](const TActorId & tablet, TTabletStorageInfo* info) { return new TFakeCoordinator(tablet, info, state); }); diff --git a/ydb/core/testlib/fake_coordinator.h b/ydb/core/testlib/fake_coordinator.h index c98bbf1a670..b3098d60a59 100644 --- a/ydb/core/testlib/fake_coordinator.h +++ b/ydb/core/testlib/fake_coordinator.h @@ -24,7 +24,7 @@ namespace NKikimr { return NKikimrServices::TActivity::TX_COORDINATOR_ACTOR; } - TFakeCoordinator(const TActorId &tablet, TTabletStorageInfo *info, TState::TPtr state) + TFakeCoordinator(const TActorId &tablet, TTabletStorageInfo *info, TState::TPtr state) : TActor<TFakeCoordinator>(&TFakeCoordinator::StateInit) , TTabletExecutedFlat(info, tablet, new NMiniKQL::TMiniKQLFactory) , State(state) @@ -101,7 +101,7 @@ namespace NKikimr { auto planTx = evPlan->Record.AddTransactions(); planTx->SetCoordinator(TabletID()); planTx->SetTxId(tx.GetTxId()); - ActorIdToProto(ctx.SelfID, planTx->MutableAckTo()); + ActorIdToProto(ctx.SelfID, planTx->MutableAckTo()); State->QueuedPlans[std::make_pair(shard, State->CurrentStep)].push_back(evPlan); } diff --git a/ydb/core/testlib/fake_scheme_shard.h b/ydb/core/testlib/fake_scheme_shard.h index 491db925bc8..8c388d49dd0 100644 --- a/ydb/core/testlib/fake_scheme_shard.h +++ b/ydb/core/testlib/fake_scheme_shard.h @@ -34,7 +34,7 @@ public: return NKikimrServices::TActivity::FLAT_SCHEMESHARD_ACTOR; } - TFakeSchemeShard(const TActorId &tablet, TTabletStorageInfo *info, TState::TPtr state) + TFakeSchemeShard(const TActorId &tablet, TTabletStorageInfo *info, TState::TPtr state) : TActor<TFakeSchemeShard>(&TFakeSchemeShard::StateInit) , NTabletFlatExecutor::TTabletExecutedFlat(info, tablet, new NMiniKQL::TMiniKQLFactory) , State(state) diff --git a/ydb/core/testlib/minikql_compile.h b/ydb/core/testlib/minikql_compile.h index a75faa638c8..4d76ca8d735 100644 --- a/ydb/core/testlib/minikql_compile.h +++ b/ydb/core/testlib/minikql_compile.h @@ -56,7 +56,7 @@ public: }, MtpQueue); } - virtual void ResolveTables(const TVector<TTable>& tables, NActors::TActorId responseTo) override { + virtual void ResolveTables(const TVector<TTable>& tables, NActors::TActorId responseTo) override { Y_UNUSED(tables); Y_UNUSED(responseTo); ythrow yexception() << "Not implemented"; diff --git a/ydb/core/testlib/tablet_flat_dummy.cpp b/ydb/core/testlib/tablet_flat_dummy.cpp index 3b90342ab8b..1a9dca04a52 100644 --- a/ydb/core/testlib/tablet_flat_dummy.cpp +++ b/ydb/core/testlib/tablet_flat_dummy.cpp @@ -114,7 +114,7 @@ public: return NKikimrServices::TActivity::TEST_ACTOR_RUNTIME; } - TDummyFlatTablet(const TActorId &tablet, TTabletStorageInfo *info) + TDummyFlatTablet(const TActorId &tablet, TTabletStorageInfo *info) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, new NMiniKQL::TMiniKQLFactory) {} @@ -141,7 +141,7 @@ public: } // namespace -IActor* CreateFlatDummyTablet(const TActorId &tablet, TTabletStorageInfo *info) { +IActor* CreateFlatDummyTablet(const TActorId &tablet, TTabletStorageInfo *info) { return new TDummyFlatTablet(tablet, info); }} diff --git a/ydb/core/testlib/tablet_helpers.cpp b/ydb/core/testlib/tablet_helpers.cpp index 622b9e5ddee..a853040b932 100644 --- a/ydb/core/testlib/tablet_helpers.cpp +++ b/ydb/core/testlib/tablet_helpers.cpp @@ -191,7 +191,7 @@ namespace NKikimr { } } - void OnRegistration(TTestActorRuntime& runtime, const TActorId& parentId, const TActorId& actorId) { + void OnRegistration(TTestActorRuntime& runtime, const TActorId& parentId, const TActorId& actorId) { Y_UNUSED(runtime); auto it = TabletRelatedActors.find(parentId); if (it != TabletRelatedActors.end()) { @@ -256,7 +256,7 @@ namespace NKikimr { protected: TMap<ui64, TActorId> TabletLeaders; - TMap<TActorId, ui64> TabletRelatedActors; + TMap<TActorId, ui64> TabletRelatedActors; TSet<ui64> DeletedTablets; bool& TracingActive; const TVector<ui64> TabletIds; @@ -279,7 +279,7 @@ namespace NKikimr { TTestActorRuntime::EEventAction OnEvent(TTestActorRuntime& runtime, TAutoPtr<IEventHandle>& event) { TTabletTracer::OnEvent(runtime, event); - TActorId actor = event->Recipient; + TActorId actor = event->Recipient; if (KillOnCommit && IsCommitResult(event) && HideCommitsFrom.contains(actor)) { // We dropped one of the previous TEvCommitResult coming to this Executore actor // after that we must drop all TEvCommitResult until this Executor dies @@ -315,7 +315,7 @@ namespace NKikimr { TActorId targetActorId = TabletLeaders[TabletId]; - if (targetActorId == TActorId()) { + if (targetActorId == TActorId()) { if (ENABLE_REBOOT_DISPATCH_LOG) Cerr << "!IGNORE " << TabletId << " event " << eventType << " becouse actor is null!\n"; @@ -326,7 +326,7 @@ namespace NKikimr { Cerr << "!Reboot " << TabletId << " (actor " << targetActorId << ") on event " << eventType << " !\n"; // Wait for the tablet to boot or to become deleted - runtime.Send(new IEventHandle(targetActorId, TActorId(), new TEvents::TEvPoisonPill())); + runtime.Send(new IEventHandle(targetActorId, TActorId(), new TEvents::TEvPoisonPill())); TDispatchOptions rebootOptions; rebootOptions.FinalEvents.push_back(TDispatchOptions::TFinalEventCondition(TEvTablet::EvRestored, 2)); rebootOptions.CustomFinalCondition = [this]() -> bool { @@ -359,7 +359,7 @@ namespace NKikimr { const bool KillOnCommit; // Kill tablet after log is committed but before Complete() is called for Tx's ui32 CurrentEventCount; bool HasReboot0; - TSet<TActorId> HideCommitsFrom; + TSet<TActorId> HideCommitsFrom; }; // Breaks pipe after the specified number of events @@ -396,8 +396,8 @@ namespace NKikimr { Cerr << "!Reset pipe\n"; // Replace the event with PoisonPill in order to kill PipeClient or PipeServer - TActorId targetActorId = event->GetRecipientRewrite(); - runtime.Send(new IEventHandle(targetActorId, TActorId(), new TEvents::TEvPoisonPill())); + TActorId targetActorId = event->GetRecipientRewrite(); + runtime.Send(new IEventHandle(targetActorId, TActorId(), new TEvents::TEvPoisonPill())); return TTestActorRuntime::EEventAction::DROP; } @@ -592,8 +592,8 @@ namespace NKikimr { } void InvalidateTabletResolverCache(TTestActorRuntime &runtime, ui64 tabletId, ui32 nodeIndex) { - runtime.Send(new IEventHandle(MakeTabletResolverID(), TActorId(), - new TEvTabletResolver::TEvTabletProblem(tabletId, TActorId())), nodeIndex); + runtime.Send(new IEventHandle(MakeTabletResolverID(), TActorId(), + new TEvTabletResolver::TEvTabletProblem(tabletId, TActorId())), nodeIndex); } void RebootTablet(TTestActorRuntime &runtime, ui64 tabletId, const TActorId& sender, ui32 nodeIndex, bool sysTablet) { @@ -703,12 +703,12 @@ namespace NKikimr { app.SetChannels(std::move(channelProfiles)); } - void SetupBoxAndStoragePool(TTestActorRuntime &runtime, const TActorId& sender, ui32 domainId, ui32 nGroups) { + void SetupBoxAndStoragePool(TTestActorRuntime &runtime, const TActorId& sender, ui32 domainId, ui32 nGroups) { NTabletPipe::TClientConfig pipeConfig; pipeConfig.RetryPolicy = NTabletPipe::TClientRetryPolicy::WithRetries(); //get NodesInfo, nodes hostname and port are interested - runtime.Send(new IEventHandle(GetNameserviceActorId(), sender, new TEvInterconnect::TEvListNodes)); + runtime.Send(new IEventHandle(GetNameserviceActorId(), sender, new TEvInterconnect::TEvListNodes)); TAutoPtr<IEventHandle> handleNodesInfo; auto nodesInfo = runtime.GrabEdgeEventRethrow<TEvInterconnect::TEvNodesInfo>(handleNodesInfo); auto bsConfigureRequest = MakeHolder<TEvBlobStorage::TEvControllerConfigRequest>(); @@ -765,7 +765,7 @@ namespace NKikimr { return TTestActorRuntime::EEventAction::PROCESS; }); - runtime.SetRegistrationObserverFunc([&](TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId) { + runtime.SetRegistrationObserverFunc([&](TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId) { tabletTracer.OnRegistration(AsKikimrRuntime(runtime), parentId, actorId); }); @@ -821,7 +821,7 @@ namespace NKikimr { return rebootingObserver.OnEvent(AsKikimrRuntime(runtime), event); }); - runtime.SetRegistrationObserverFunc([&](TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId) { + runtime.SetRegistrationObserverFunc([&](TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId) { rebootingObserver.OnRegistration(AsKikimrRuntime(runtime), parentId, actorId); }); @@ -865,7 +865,7 @@ namespace NKikimr { return TTestActorRuntime::EEventAction::PROCESS; }); - runtime.SetRegistrationObserverFunc([&](TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId) { + runtime.SetRegistrationObserverFunc([&](TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId) { tabletTracer.OnRegistration(AsKikimrRuntime(runtime), parentId, actorId); }); @@ -912,7 +912,7 @@ namespace NKikimr { return pipeResetingObserver.OnEvent(AsKikimrRuntime(runtime), event); }); - runtime.SetRegistrationObserverFunc([&](TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId) { + runtime.SetRegistrationObserverFunc([&](TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId) { pipeResetingObserver.OnRegistration(AsKikimrRuntime(runtime), parentId, actorId); }); @@ -961,7 +961,7 @@ namespace NKikimr { return delayingObserver.OnEvent(AsKikimrRuntime(runtime), event); }); - runtime.SetRegistrationObserverFunc([&](TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId) { + runtime.SetRegistrationObserverFunc([&](TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId) { delayingObserver.OnRegistration(AsKikimrRuntime(runtime), parentId, actorId); }); @@ -988,7 +988,7 @@ namespace NKikimr { class TTabletScheduledEventsGuard : public ITabletScheduledEventsGuard { public: - TTabletScheduledEventsGuard(const TVector<ui64>& tabletIds, TTestActorRuntime& runtime, const TActorId& sender) + TTabletScheduledEventsGuard(const TVector<ui64>& tabletIds, TTestActorRuntime& runtime, const TActorId& sender) : Runtime(runtime) , Sender(sender) , TracingActive(true) @@ -1001,7 +1001,7 @@ namespace NKikimr { }); PrevRegistrationObserverFunc = Runtime.SetRegistrationObserverFunc( - [&](TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId) { + [&](TTestActorRuntimeBase& runtime, const TActorId& parentId, const TActorId& actorId) { TabletTracer.OnRegistration(AsKikimrRuntime(runtime), parentId, actorId); }); @@ -1027,7 +1027,7 @@ namespace NKikimr { private: TTestActorRuntime& Runtime; - const TActorId Sender; + const TActorId Sender; bool TracingActive; TTabletTracer TabletTracer; TTabletScheduledFilter ScheduledFilter; @@ -1038,12 +1038,12 @@ namespace NKikimr { TTestActorRuntime::TRegistrationObserver PrevRegistrationObserverFunc; }; - TAutoPtr<ITabletScheduledEventsGuard> CreateTabletScheduledEventsGuard(const TVector<ui64>& tabletIds, TTestActorRuntime& runtime, const TActorId& sender) { + TAutoPtr<ITabletScheduledEventsGuard> CreateTabletScheduledEventsGuard(const TVector<ui64>& tabletIds, TTestActorRuntime& runtime, const TActorId& sender) { return TAutoPtr<ITabletScheduledEventsGuard>(new TTabletScheduledEventsGuard(tabletIds, runtime, sender)); } - ui64 GetFreePDiskSize(TTestActorRuntime& runtime, const TActorId& sender) { - TActorId pdiskServiceId = MakeBlobStoragePDiskID(runtime.GetNodeId(0), 0); + ui64 GetFreePDiskSize(TTestActorRuntime& runtime, const TActorId& sender) { + TActorId pdiskServiceId = MakeBlobStoragePDiskID(runtime.GetNodeId(0), 0); runtime.Send(new IEventHandle(pdiskServiceId, sender, nullptr)); TAutoPtr<IEventHandle> handle; auto event = runtime.GrabEdgeEvent<NMon::TEvHttpInfoRes>(handle); @@ -1069,7 +1069,7 @@ namespace NKikimr { return pipeConfig; } - void WaitScheduledEvents(TTestActorRuntime &runtime, TDuration delay, const TActorId &sender, ui32 nodeIndex) { + void WaitScheduledEvents(TTestActorRuntime &runtime, TDuration delay, const TActorId &sender, ui32 nodeIndex) { runtime.Schedule(new IEventHandle(sender, sender, new TEvents::TEvWakeup()), delay, nodeIndex); TAutoPtr<IEventHandle> handle; runtime.GrabEdgeEvent<TEvents::TEvWakeup>(handle); @@ -1077,7 +1077,7 @@ namespace NKikimr { class TFakeHive : public TActor<TFakeHive>, public NTabletFlatExecutor::TTabletExecutedFlat { public: - static std::function<IActor* (const TActorId &, TTabletStorageInfo*)> DefaultGetTabletCreationFunc(ui32 type) { + static std::function<IActor* (const TActorId &, TTabletStorageInfo*)> DefaultGetTabletCreationFunc(ui32 type) { Y_UNUSED(type); return nullptr; } @@ -1089,7 +1089,7 @@ namespace NKikimr { return NKikimrServices::TActivity::HIVE_ACTOR; } - TFakeHive(const TActorId &tablet, TTabletStorageInfo *info, TState::TPtr state, + TFakeHive(const TActorId &tablet, TTabletStorageInfo *info, TState::TPtr state, TGetTabletCreationFunc getTabletCreationFunc) : TActor<TFakeHive>(&TFakeHive::StateInit) , NTabletFlatExecutor::TTabletExecutedFlat(info, tablet, new NMiniKQL::TMiniKQLFactory) @@ -1154,7 +1154,7 @@ namespace NKikimr { const auto bootMode = ev->Get()->Record.GetTabletBootMode(); auto it = State->Tablets.find(key); const auto& defaultTabletTypes = AppData(ctx)->DefaultTabletTypes; - TActorId bootstrapperActorId; + TActorId bootstrapperActorId; if (it == State->Tablets.end()) { if (bootMode == NKikimrHive::TABLET_BOOT_MODE_EXTERNAL) { // don't boot anything @@ -1180,7 +1180,7 @@ namespace NKikimr { bootstrapperActorId = Boot(ctx, type, &NKesus::CreateKesusTablet, DataGroupErasure); } else if (type == defaultTabletTypes.Hive) { TFakeHiveState::TPtr state = State->AllocateSubHive(); - bootstrapperActorId = Boot(ctx, type, [=](const TActorId& tablet, TTabletStorageInfo* info) { + bootstrapperActorId = Boot(ctx, type, [=](const TActorId& tablet, TTabletStorageInfo* info) { return new TFakeHive(tablet, info, state, &TFakeHive::DefaultGetTabletCreationFunc); }, DataGroupErasure); } else if (type == defaultTabletTypes.SysViewProcessor) { @@ -1293,10 +1293,10 @@ namespace NKikimr { ctx.Send(ctx.SelfID, new TEvFakeHive::TEvNotifyTabletDeleted(tabletInfo.TabletId)); // Kill the tablet and don't restart it - TActorId bootstrapperActorId = tabletInfo.BootstrapperActorId; + TActorId bootstrapperActorId = tabletInfo.BootstrapperActorId; ctx.Send(bootstrapperActorId, new TEvBootstrapper::TEvStandBy()); - for (TActorId waiter : tabletInfo.DeletionWaiters) { + for (TActorId waiter : tabletInfo.DeletionWaiters) { SendDeletionNotification(it->second.TabletId, waiter, ctx); } State->TabletIdToOwner.erase(it->second.TabletId); @@ -1393,7 +1393,7 @@ namespace NKikimr { } } - void SendDeletionNotification(ui64 tabletId, TActorId waiter, const TActorContext& ctx) { + void SendDeletionNotification(ui64 tabletId, TActorId waiter, const TActorContext& ctx) { TAutoPtr<TEvHive::TEvResponseHiveInfo> response = new TEvHive::TEvResponseHiveInfo(); FillTabletInfo(response->Record, tabletId, nullptr); ctx.Send(waiter, response.Release()); @@ -1406,7 +1406,7 @@ namespace NKikimr { } private: - TActorId Boot(const TActorContext& ctx, TTabletTypes::EType tabletType, std::function<IActor* (const TActorId &, TTabletStorageInfo *)> op, + TActorId Boot(const TActorContext& ctx, TTabletTypes::EType tabletType, std::function<IActor* (const TActorId &, TTabletStorageInfo *)> op, TBlobStorageGroupType::EErasureSpecies erasure) { TIntrusivePtr<TBootstrapperInfo> bi(new TBootstrapperInfo(new TTabletSetupInfo(op, TMailboxType::Simple, 0, TMailboxType::Simple, 0))); diff --git a/ydb/core/testlib/tablet_helpers.h b/ydb/core/testlib/tablet_helpers.h index b13df8186f0..81a7c614aaa 100644 --- a/ydb/core/testlib/tablet_helpers.h +++ b/ydb/core/testlib/tablet_helpers.h @@ -19,7 +19,7 @@ namespace NKikimr { const TBlobStorageGroupType::EErasureSpecies DataGroupErasure = TBlobStorageGroupType::ErasureNone; TActorId FollowerTablet(TTestActorRuntime &runtime, const TActorId &launcher, TTabletStorageInfo *info, - std::function<IActor* (const TActorId &, TTabletStorageInfo*)> op); + std::function<IActor* (const TActorId &, TTabletStorageInfo*)> op); TActorId ResolveTablet(TTestActorRuntime& runtime, ui64 tabletId, ui32 nodeIndex = 0, bool sysTablet = false); void ForwardToTablet(TTestActorRuntime& runtime, ui64 tabletId, const TActorId& sender, IEventBase *ev, ui32 nodeIndex = 0, bool sysTablet = false); void InvalidateTabletResolverCache(TTestActorRuntime& runtime, ui64 tabletId, ui32 nodeIndex = 0); @@ -37,7 +37,7 @@ namespace NKikimr { } const TChannelsBindings DEFAULT_BINDED_CHANNELS = {GetDefaultChannelBind(), GetDefaultChannelBind(), GetDefaultChannelBind()}; - void SetupBoxAndStoragePool(TTestActorRuntime &runtime, const TActorId& sender, ui32 domainId = 0, ui32 nGroups = 1); + void SetupBoxAndStoragePool(TTestActorRuntime &runtime, const TActorId& sender, ui32 domainId = 0, ui32 nGroups = 1); void SetupChannelProfiles(TAppPrepare &app, ui32 domainId = 0, ui32 nchannels = 3); TDomainsInfo::TDomain::TStoragePoolKinds DefaultPoolKinds(ui32 count = 1); @@ -75,15 +75,15 @@ namespace NKikimr { virtual ~ITabletScheduledEventsGuard() {} }; - TAutoPtr<ITabletScheduledEventsGuard> CreateTabletScheduledEventsGuard(const TVector<ui64>& tabletIds, TTestActorRuntime& runtime, const TActorId& sender); - ui64 GetFreePDiskSize(TTestActorRuntime& runtime, const TActorId& sender); - void PrintTabletDb(TTestActorRuntime& runtime, ui64 tabletId, const TActorId& sender); + TAutoPtr<ITabletScheduledEventsGuard> CreateTabletScheduledEventsGuard(const TVector<ui64>& tabletIds, TTestActorRuntime& runtime, const TActorId& sender); + ui64 GetFreePDiskSize(TTestActorRuntime& runtime, const TActorId& sender); + void PrintTabletDb(TTestActorRuntime& runtime, ui64 tabletId, const TActorId& sender); NTabletPipe::TClientConfig GetPipeConfigWithRetriesAndFollowers(); IActor* CreateFlatDummyTablet(const TActorId &tablet, TTabletStorageInfo *info); - void WaitScheduledEvents(TTestActorRuntime &runtime, TDuration delay, const TActorId &sender, ui32 nodeIndex = 0); + void WaitScheduledEvents(TTestActorRuntime &runtime, TDuration delay, const TActorId &sender, ui32 nodeIndex = 0); struct TEvFakeHive { @@ -112,14 +112,14 @@ namespace NKikimr { struct TFakeHiveTabletInfo { const TTabletTypes::EType Type; const ui64 TabletId; - TActorId BootstrapperActorId; + TActorId BootstrapperActorId; TChannelsBindings BoundChannels; ui32 ChannelsProfile; - THashSet<TActorId> DeletionWaiters; + THashSet<TActorId> DeletionWaiters; - TFakeHiveTabletInfo(TTabletTypes::EType type, ui64 tabletId, TActorId bootstrapperActorId) + TFakeHiveTabletInfo(TTabletTypes::EType type, ui64 tabletId, TActorId bootstrapperActorId) : Type(type) , TabletId(tabletId) , BootstrapperActorId(bootstrapperActorId) @@ -162,7 +162,7 @@ namespace NKikimr { } }; - typedef std::function<std::function<IActor* (const TActorId &, TTabletStorageInfo*)>(ui32 type)> TGetTabletCreationFunc; + typedef std::function<std::function<IActor* (const TActorId &, TTabletStorageInfo*)>(ui32 type)> TGetTabletCreationFunc; void BootFakeHive(TTestActorRuntime& runtime, ui64 tabletId, TFakeHiveState::TPtr state, TGetTabletCreationFunc getTabletCreationFunc = nullptr); diff --git a/ydb/core/testlib/tenant_runtime.cpp b/ydb/core/testlib/tenant_runtime.cpp index b49ad314528..974cc166d0d 100644 --- a/ydb/core/testlib/tenant_runtime.cpp +++ b/ydb/core/testlib/tenant_runtime.cpp @@ -197,8 +197,8 @@ public: return NKikimrServices::TActivity::FLAT_SCHEMESHARD_ACTOR; } - TFakeSchemeShard(const TActorId &tablet, TTabletStorageInfo *info, - TActorId sender, const TVector<std::pair<TString, ui64>> &subDomains) + TFakeSchemeShard(const TActorId &tablet, TTabletStorageInfo *info, + TActorId sender, const TVector<std::pair<TString, ui64>> &subDomains) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, nullptr) , Sender(sender) @@ -227,7 +227,7 @@ public: THashMap<TString, ui64> SubDomains; THashMap<ui64, TString> Paths; - TActorId Sender; + TActorId Sender; TVector<TAutoPtr<IEventHandle>> Queue; bool HoldResolve; }; @@ -249,7 +249,7 @@ class TFakeBSController : public TActor<TFakeBSController>, public TTabletExecut } public: - TFakeBSController(const TActorId &tablet, TTabletStorageInfo *info) + TFakeBSController(const TActorId &tablet, TTabletStorageInfo *info) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, nullptr) { @@ -283,7 +283,7 @@ class TFakeTenantSlotBroker : public TActor<TFakeTenantSlotBroker>, public TTabl Die(ctx); } - void SendState(const TString &name, TActorId sender, const TActorContext &ctx) + void SendState(const TString &name, TActorId sender, const TActorContext &ctx) { auto *resp = new TEvTenantSlotBroker::TEvTenantState; resp->Record.SetTenantName(name); @@ -304,7 +304,7 @@ class TFakeTenantSlotBroker : public TActor<TFakeTenantSlotBroker>, public TTabl } public: - TFakeTenantSlotBroker(const TActorId &tablet, TTabletStorageInfo *info) + TFakeTenantSlotBroker(const TActorId &tablet, TTabletStorageInfo *info) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, nullptr) { @@ -350,7 +350,7 @@ class TFakeHive : public TActor<TFakeHive>, public TTabletExecutedFlat { void ResolveKey(TSubDomainKey key, const TActorContext &ctx) { - TActorId clientId = ctx.Register(NKikimr::NTabletPipe::CreateClient(ctx.SelfID, key.GetSchemeShard())); + TActorId clientId = ctx.Register(NKikimr::NTabletPipe::CreateClient(ctx.SelfID, key.GetSchemeShard())); auto *request = new TEvSchemeShard::TEvDescribeScheme(key.GetSchemeShard(), key.GetPathId()); NTabletPipe::SendData(ctx, clientId, request); ctx.Send(clientId, new NKikimr::TEvTabletPipe::TEvShutdown); @@ -373,7 +373,7 @@ class TFakeHive : public TActor<TFakeHive>, public TTabletExecutedFlat { CheckState(ctx); } - TActorId Boot(const TActorContext& ctx, TTabletTypes::EType tabletType, std::function<IActor* (const TActorId &, TTabletStorageInfo *)> op, + TActorId Boot(const TActorContext& ctx, TTabletTypes::EType tabletType, std::function<IActor* (const TActorId &, TTabletStorageInfo *)> op, TBlobStorageGroupType::EErasureSpecies erasure) { TIntrusivePtr<TBootstrapperInfo> bi(new TBootstrapperInfo(new TTabletSetupInfo(op, TMailboxType::Simple, 0, @@ -381,7 +381,7 @@ class TFakeHive : public TActor<TFakeHive>, public TTabletExecutedFlat { return ctx.ExecutorThread.RegisterActor(CreateBootstrapper(CreateTestTabletInfo(State.NextTabletId, tabletType, erasure), bi.Get())); } - void SendDeletionNotification(ui64 tabletId, TActorId waiter, const TActorContext& ctx) + void SendDeletionNotification(ui64 tabletId, TActorId waiter, const TActorContext& ctx) { TAutoPtr<TEvHive::TEvResponseHiveInfo> response = new TEvHive::TEvResponseHiveInfo(); FillTabletInfo(response->Record, tabletId, nullptr); @@ -406,7 +406,7 @@ class TFakeHive : public TActor<TFakeHive>, public TTabletExecutedFlat { const auto bootMode = ev->Get()->Record.GetTabletBootMode(); auto it = State.Tablets.find(key); const auto& defaultTabletTypes = AppData(ctx)->DefaultTabletTypes; - TActorId bootstrapperActorId; + TActorId bootstrapperActorId; if (it == State.Tablets.end()) { if (ev->Get()->Record.AllowedDomainsSize()) { bool found = false; @@ -476,10 +476,10 @@ class TFakeHive : public TActor<TFakeHive>, public TTabletExecutedFlat { if (it != State.Tablets.end()) { ctx.Send(ctx.SelfID, new TEvFakeHive::TEvNotifyTabletDeleted(it->second.TabletId)); - TActorId bootstrapperActorId = it->second.BootstrapperActorId; + TActorId bootstrapperActorId = it->second.BootstrapperActorId; ctx.Send(bootstrapperActorId, new TEvBootstrapper::TEvStandBy()); - for (TActorId waiter : it->second.DeletionWaiters) { + for (TActorId waiter : it->second.DeletionWaiters) { SendDeletionNotification(it->second.TabletId, waiter, ctx); } State.TabletIdToOwner.erase(it->second.TabletId); @@ -504,10 +504,10 @@ class TFakeHive : public TActor<TFakeHive>, public TTabletExecutedFlat { if (it != State.Tablets.end()) { ctx.Send(ctx.SelfID, new TEvFakeHive::TEvNotifyTabletDeleted(it->second.TabletId)); - TActorId bootstrapperActorId = it->second.BootstrapperActorId; + TActorId bootstrapperActorId = it->second.BootstrapperActorId; ctx.Send(bootstrapperActorId, new TEvBootstrapper::TEvStandBy()); - for (TActorId waiter : it->second.DeletionWaiters) { + for (TActorId waiter : it->second.DeletionWaiters) { SendDeletionNotification(it->second.TabletId, waiter, ctx); } State.TabletIdToOwner.erase(it->second.TabletId); @@ -640,7 +640,7 @@ public: return NKikimrServices::TActivity::HIVE_ACTOR; } - TFakeHive(const TActorId &tablet, TTabletStorageInfo *info, TActorId sender, + TFakeHive(const TActorId &tablet, TTabletStorageInfo *info, TActorId sender, ui64 hiveId, const THashMap<TSubDomainKey, TString> &subDomainKeys) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, nullptr) @@ -680,8 +680,8 @@ public: } } - THashMap<TActorId, TClientInfo> Clients; - TActorId Sender; + THashMap<TActorId, TClientInfo> Clients; + TActorId Sender; THashMap<TString, TClientInfo> ExpectedState; bool WaitForState; ui64 HiveId; @@ -763,7 +763,7 @@ void TTenantTestRuntime::CreateTenantPool(ui32 nodeIndex, const TTenantTestConfi tenantPoolConfig->DynamicSlotLabel = Extension.GetMonitoringConfig().GetDatabaseLabels() .GetDynamicSlotLabelValue(); - TActorId actorId = Register(NKikimr::CreateTenantPool(tenantPoolConfig), nodeIndex, 0, TMailboxType::Revolving, 0); + TActorId actorId = Register(NKikimr::CreateTenantPool(tenantPoolConfig), nodeIndex, 0, TMailboxType::Revolving, 0); EnableScheduleForActor(actorId, true); RegisterService(MakeTenantPoolRootID(), actorId, nodeIndex); } @@ -932,7 +932,7 @@ void TTenantTestRuntime::Setup(bool createTenantPools) // Create TxProxy services for (size_t i = 0; i< Config.Nodes.size(); ++i) { IActor* txProxy = CreateTxProxy(GetTxAllocatorTabletIds()); - TActorId txProxyId = Register(txProxy, i); + TActorId txProxyId = Register(txProxy, i); RegisterService(MakeTxProxyID(), txProxyId, i); } @@ -971,7 +971,7 @@ void TTenantTestRuntime::Setup(bool createTenantPools) // Get list of nodes to reveal IC ports. - Send(new IEventHandle(GetNameserviceActorId(), Sender, new TEvInterconnect::TEvListNodes)); + Send(new IEventHandle(GetNameserviceActorId(), Sender, new TEvInterconnect::TEvListNodes)); TAutoPtr<IEventHandle> handle; auto reply1 = GrabEdgeEventRethrow<TEvInterconnect::TEvNodesInfo>(handle); diff --git a/ydb/core/testlib/tenant_runtime.h b/ydb/core/testlib/tenant_runtime.h index f5ec9cd4c48..f98c1441af3 100644 --- a/ydb/core/testlib/tenant_runtime.h +++ b/ydb/core/testlib/tenant_runtime.h @@ -205,7 +205,7 @@ public: void SendToBroker(IEventBase* event); void SendToConsole(IEventBase* event); - TActorId Sender; + TActorId Sender; TTenantTestConfig Config; NKikimrConfig::TAppConfig Extension; THashMap<TSubDomainKey, TString> SubDomainKeys; diff --git a/ydb/core/testlib/test_client.cpp b/ydb/core/testlib/test_client.cpp index 6e96d550bd0..d4907c26f1e 100644 --- a/ydb/core/testlib/test_client.cpp +++ b/ydb/core/testlib/test_client.cpp @@ -1,5 +1,5 @@ #include "test_client.h" - + #include <ydb/core/testlib/basics/runtime.h> #include <ydb/core/base/appdata.h> #include <ydb/core/base/hive.h> @@ -73,7 +73,7 @@ #include <ydb/library/yql/minikql/invoke_builtins/mkql_builtins.h> #include <ydb/library/yql/public/issue/yql_issue_message.h> #include <ydb/core/engine/mkql_engine_flat.h> - + #include <library/cpp/testing/unittest/registar.h> #include <ydb/core/kesus/proxy/proxy.h> #include <ydb/core/kesus/tablet/tablet.h> @@ -88,10 +88,10 @@ #include <ydb/library/folder_service/mock/mock_folder_service.h> #include <ydb/core/client/server/msgbus_server_tracer.h> - + #include <library/cpp/actors/interconnect/interconnect.h> -#include <library/cpp/grpc/server/actors/logger.h> +#include <library/cpp/grpc/server/actors/logger.h> #include <util/system/sanitizers.h> #include <util/system/valgrind.h> @@ -169,7 +169,7 @@ namespace Tests { Runtime = MakeHolder<TTestBasicRuntime>(StaticNodes() + DynamicNodes(), Settings->UseRealThreads); if (!Settings->UseRealThreads) - Runtime->SetRegistrationObserverFunc([](TTestActorRuntimeBase& runtime, const TActorId&, const TActorId& actorId) { + Runtime->SetRegistrationObserverFunc([](TTestActorRuntimeBase& runtime, const TActorId&, const TActorId& actorId) { runtime.EnableScheduleForActor(actorId); }); @@ -244,8 +244,8 @@ namespace Tests { } } - void TServer::EnableGRpc(const NGrpc::TServerOptions& options) { - GRpcServer.reset(new NGrpc::TGRpcServer(options)); + void TServer::EnableGRpc(const NGrpc::TServerOptions& options) { + GRpcServer.reset(new NGrpc::TGRpcServer(options)); auto grpcService = new NGRpcProxy::TGRpcService(); auto system(Runtime->GetAnyNodeActorSystem()); @@ -337,7 +337,7 @@ namespace Tests { EnableGRpc(NGrpc::TServerOptions() .SetHost("localhost") .SetPort(port) - .SetLogger(NGrpc::CreateActorSystemLogger(*Runtime->GetAnyNodeActorSystem(), NKikimrServices::GRPC_SERVER)) + .SetLogger(NGrpc::CreateActorSystemLogger(*Runtime->GetAnyNodeActorSystem(), NKikimrServices::GRPC_SERVER)) ); } @@ -375,13 +375,13 @@ namespace Tests { } void TServer::SetupStorage() { - TActorId sender = Runtime->AllocateEdgeActor(); + TActorId sender = Runtime->AllocateEdgeActor(); NTabletPipe::TClientConfig pipeConfig; pipeConfig.RetryPolicy = NTabletPipe::TClientRetryPolicy::WithRetries(); //get NodesInfo, nodes hostname and port are interested - Runtime->Send(new IEventHandle(GetNameserviceActorId(), sender, new TEvInterconnect::TEvListNodes)); + Runtime->Send(new IEventHandle(GetNameserviceActorId(), sender, new TEvInterconnect::TEvListNodes)); TAutoPtr<IEventHandle> handleNodesInfo; auto nodesInfo = Runtime->GrabEdgeEventRethrow<TEvInterconnect::TEvNodesInfo>(handleNodesInfo); @@ -488,8 +488,8 @@ namespace Tests { void TServer::DestroyDynamicLocalService(ui32 nodeIdx) { Y_VERIFY(nodeIdx >= StaticNodes()); - TActorId local = MakeLocalID(Runtime->GetNodeId(nodeIdx)); // MakeTenantPoolRootID? - Runtime->Send(new IEventHandle(local, TActorId(), new TEvents::TEvPoisonPill())); + TActorId local = MakeLocalID(Runtime->GetNodeId(nodeIdx)); // MakeTenantPoolRootID? + Runtime->Send(new IEventHandle(local, TActorId(), new TEvents::TEvPoisonPill())); } void TServer::SetupLocalConfig(TLocalConfig &localConfig, const NKikimr::TAppData &appData) { @@ -570,7 +570,7 @@ namespace Tests { Runtime->Register(CreateLabelsMaintainer({}), nodeIdx, appData.SystemPoolId, TMailboxType::Revolving, 0); auto sysViewService = NSysView::CreateSysViewServiceForTests(); - TActorId sysViewServiceId = Runtime->Register(sysViewService.Release(), nodeIdx); + TActorId sysViewServiceId = Runtime->Register(sysViewService.Release(), nodeIdx); Runtime->RegisterService(NSysView::MakeSysViewServiceID(Runtime->GetNodeId(nodeIdx)), sysViewServiceId, nodeIdx); auto tenantPublisher = CreateTenantNodeEnumerationPublisher(); @@ -587,7 +587,7 @@ namespace Tests { Runtime->SetTxAllocatorTabletIds({ChangeStateStorage(TxAllocator, Settings->Domain)}); { IActor* ticketParser = Settings->CreateTicketParser(Settings->AuthConfig); - TActorId ticketParserId = Runtime->Register(ticketParser, nodeIdx); + TActorId ticketParserId = Runtime->Register(ticketParser, nodeIdx); Runtime->RegisterService(MakeTicketParserID(), ticketParserId, nodeIdx); } @@ -608,19 +608,19 @@ namespace Tests { Settings->AppConfig.GetTableServiceConfig(), TVector<NKikimrKqp::TKqpSetting>(Settings->KqpSettings), nullptr); - TActorId kqpProxyServiceId = Runtime->Register(kqpProxyService, nodeIdx); + TActorId kqpProxyServiceId = Runtime->Register(kqpProxyService, nodeIdx); Runtime->RegisterService(NKqp::MakeKqpProxyID(Runtime->GetNodeId(nodeIdx)), kqpProxyServiceId, nodeIdx); } { IActor* txProxy = CreateTxProxy(Runtime->GetTxAllocatorTabletIds()); - TActorId txProxyId = Runtime->Register(txProxy, nodeIdx); + TActorId txProxyId = Runtime->Register(txProxy, nodeIdx); Runtime->RegisterService(MakeTxProxyID(), txProxyId, nodeIdx); } { IActor* compileService = CreateMiniKQLCompileService(100000); - TActorId compileServiceId = Runtime->Register(compileService, nodeIdx, Runtime->GetAppData(nodeIdx).SystemPoolId, TMailboxType::Revolving, 0); + TActorId compileServiceId = Runtime->Register(compileService, nodeIdx, Runtime->GetAppData(nodeIdx).SystemPoolId, TMailboxType::Revolving, 0); Runtime->RegisterService(MakeMiniKQLCompileServiceID(), compileServiceId, nodeIdx); } @@ -639,14 +639,14 @@ namespace Tests { if (BusServer && nodeIdx == 0) { // MsgBus and GRPC are run now only on first node { IActor* proxy = BusServer->CreateProxy(); - TActorId proxyId = Runtime->Register(proxy, nodeIdx, Runtime->GetAppData(nodeIdx).SystemPoolId, TMailboxType::Revolving, 0); + TActorId proxyId = Runtime->Register(proxy, nodeIdx, Runtime->GetAppData(nodeIdx).SystemPoolId, TMailboxType::Revolving, 0); Runtime->RegisterService(NMsgBusProxy::CreateMsgBusProxyId(), proxyId, nodeIdx); } { IActor* traceService = BusServer->CreateMessageBusTraceService(); if (traceService) { - TActorId traceServiceId = Runtime->Register(traceService, nodeIdx, Runtime->GetAppData(nodeIdx).IOPoolId, TMailboxType::Simple, 0); + TActorId traceServiceId = Runtime->Register(traceService, nodeIdx, Runtime->GetAppData(nodeIdx).IOPoolId, TMailboxType::Simple, 0); Runtime->RegisterService(NMessageBusTracer::MakeMessageBusTraceServiceID(), traceServiceId, nodeIdx); } } @@ -689,19 +689,19 @@ namespace Tests { { IActor* kesusService = NKesus::CreateKesusProxyService(); - TActorId kesusServiceId = Runtime->Register(kesusService, nodeIdx); + TActorId kesusServiceId = Runtime->Register(kesusService, nodeIdx); Runtime->RegisterService(NKesus::MakeKesusProxyServiceId(), kesusServiceId, nodeIdx); } { IActor* pqClusterTracker = NPQ::NClusterTracker::CreateClusterTracker(); - TActorId pqClusterTrackerId = Runtime->Register(pqClusterTracker, nodeIdx); + TActorId pqClusterTrackerId = Runtime->Register(pqClusterTracker, nodeIdx); Runtime->RegisterService(NPQ::NClusterTracker::MakeClusterTrackerID(), pqClusterTrackerId, nodeIdx); } { IActor* netClassifier = NNetClassifier::CreateNetClassifier(); - TActorId netClassifierId = Runtime->Register(netClassifier, nodeIdx); + TActorId netClassifierId = Runtime->Register(netClassifier, nodeIdx); Runtime->RegisterService(NNetClassifier::MakeNetClassifierID(), netClassifierId, nodeIdx); } @@ -861,15 +861,15 @@ namespace Tests { TTestActorRuntime* TServer::GetRuntime() const { return Runtime.Get(); } - + const TServerSettings &TServer::GetSettings() const { return *Settings; } const NScheme::TTypeRegistry* TServer::GetTypeRegistry() { return Runtime->GetAppData().TypeRegistry; - } - + } + const NMiniKQL::IFunctionRegistry* TServer::GetFunctionRegistry() { return Runtime->GetAppData().FunctionRegistry; } @@ -950,8 +950,8 @@ namespace Tests { return false; } - UNIT_ASSERT(response.HasTypeMetadata() && response.HasFunctionMetadata()); - DeserializeMetadata(response.GetTypeMetadata(), &LoadedTypeMetadataRegistry); + UNIT_ASSERT(response.HasTypeMetadata() && response.HasFunctionMetadata()); + DeserializeMetadata(response.GetTypeMetadata(), &LoadedTypeMetadataRegistry); DeserializeMetadata(response.GetFunctionMetadata(), *LoadedFunctionRegistry->GetBuiltins()); TypesEtag = response.GetETag(); return true; @@ -964,11 +964,11 @@ namespace Tests { return TypeRegistry.GetTypeMetadataRegistry(); } - const NMiniKQL::IFunctionRegistry& TClient::GetFunctionRegistry() const { + const NMiniKQL::IFunctionRegistry& TClient::GetFunctionRegistry() const { if (TypesEtag.Defined()) - return *LoadedFunctionRegistry; + return *LoadedFunctionRegistry; - return *FunctionRegistry; + return *FunctionRegistry; } ui64 TClient::GetPatchedSchemeRoot(ui64 schemeRoot, ui32 domain, bool supportsRedirect) { @@ -1587,7 +1587,7 @@ namespace Tests { TAutoPtr<NSchemeShard::TEvSchemeShard::TEvDescribeScheme> request(new NSchemeShard::TEvSchemeShard::TEvDescribeScheme()); request->Record.SetPath(path); const ui64 schemeRoot = GetPatchedSchemeRoot(SchemeRoot, Domain, SupportsRedirect); - TActorId sender = runtime->AllocateEdgeActor(0); + TActorId sender = runtime->AllocateEdgeActor(0); ForwardToTablet(*runtime, schemeRoot, sender, request.Release(), 0); TAutoPtr<IEventHandle> handle; @@ -1658,7 +1658,7 @@ namespace Tests { } void TClient::RefreshPathCache(TTestActorRuntime* runtime, const TString& path, ui32 nodeIdx) { - TActorId sender = runtime->AllocateEdgeActor(nodeIdx); + TActorId sender = runtime->AllocateEdgeActor(nodeIdx); auto request = MakeHolder<NSchemeCache::TSchemeCacheNavigate>(); auto& entry = request->ResultSet.emplace_back(); entry.Path = SplitPath(path); @@ -2006,7 +2006,7 @@ namespace Tests { } TString TClient::SendTabletMonQuery(TTestActorRuntime* runtime, ui64 tabletId, TString query) { - TActorId sender = runtime->AllocateEdgeActor(0); + TActorId sender = runtime->AllocateEdgeActor(0); ForwardToTablet(*runtime, tabletId, sender, new NActors::NMon::TEvRemoteHttpInfo(query), 0); TAutoPtr<IEventHandle> handle; // Timeout for DEBUG purposes only @@ -2039,12 +2039,12 @@ namespace Tests { } bool TClient::WaitForTabletAlive(TTestActorRuntime* runtime, ui64 tabletId, bool leader, TDuration timeout) { - TActorId edge = runtime->AllocateEdgeActor(); + TActorId edge = runtime->AllocateEdgeActor(); NTabletPipe::TClientConfig clientConfig; clientConfig.AllowFollower = !leader; clientConfig.ForceFollower = !leader; clientConfig.RetryPolicy = NTabletPipe::TClientRetryPolicy::WithRetries(); - TActorId pipeClient = runtime->Register(NTabletPipe::CreateClient(edge, tabletId, clientConfig)); + TActorId pipeClient = runtime->Register(NTabletPipe::CreateClient(edge, tabletId, clientConfig)); TAutoPtr<IEventHandle> handle; const TInstant deadline = TInstant::Now() + timeout; bool res = false; @@ -2062,12 +2062,12 @@ namespace Tests { } } catch (TEmptyEventQueueException &) {} - runtime->Send(new IEventHandle(pipeClient, TActorId(), new TEvents::TEvPoisonPill())); + runtime->Send(new IEventHandle(pipeClient, TActorId(), new TEvents::TEvPoisonPill())); return res; } bool TClient::WaitForTabletDown(TTestActorRuntime* runtime, ui64 tabletId, bool leader, TDuration timeout) { - TActorId edge = runtime->AllocateEdgeActor(); + TActorId edge = runtime->AllocateEdgeActor(); NTabletPipe::TClientConfig clientConfig; clientConfig.AllowFollower = !leader; clientConfig.ForceFollower = !leader; @@ -2077,7 +2077,7 @@ namespace Tests { .MaxRetryTime = TDuration::Seconds(1), .BackoffMultiplier = 2, }; - TActorId pipeClient = runtime->Register(NTabletPipe::CreateClient(edge, tabletId, clientConfig)); + TActorId pipeClient = runtime->Register(NTabletPipe::CreateClient(edge, tabletId, clientConfig)); TInstant deadline = TInstant::Now() + timeout; bool res = false; @@ -2117,7 +2117,7 @@ namespace Tests { ev->Record.SetReturnFollowers(returnFollowers); ui64 hive = ChangeStateStorage(Tests::Hive, Domain); - TActorId edge = runtime->AllocateEdgeActor(); + TActorId edge = runtime->AllocateEdgeActor(); runtime->SendToPipe(hive, edge, ev.Release()); TAutoPtr<IEventHandle> handle; TEvHive::TEvResponseHiveInfo* response = runtime->GrabEdgeEventRethrow<TEvHive::TEvResponseHiveInfo>(handle); @@ -2156,7 +2156,7 @@ namespace Tests { TAutoPtr<TEvHive::TEvGetTabletStorageInfo> ev(new TEvHive::TEvGetTabletStorageInfo(tabletId)); ui64 hive = ChangeStateStorage(Tests::Hive, Domain); - TActorId edge = runtime->AllocateEdgeActor(); + TActorId edge = runtime->AllocateEdgeActor(); runtime->SendToPipe(hive, edge, ev.Release()); TAutoPtr<IEventHandle> handle; TEvHive::TEvGetTabletStorageInfoResult* response = runtime->GrabEdgeEventRethrow<TEvHive::TEvGetTabletStorageInfoResult>(handle); @@ -2242,7 +2242,7 @@ namespace Tests { request->Record.MutableResource()->SetResourcePath(resourcePath); *request->Record.MutableResource()->MutableHierarhicalDRRResourceConfig() = props; - TActorId sender = runtime->AllocateEdgeActor(0); + TActorId sender = runtime->AllocateEdgeActor(0); ForwardToTablet(*runtime, GetKesusTabletId(kesusPath), sender, request.Release(), 0); TAutoPtr<IEventHandle> handle; @@ -2254,7 +2254,7 @@ namespace Tests { THolder<NKesus::TEvKesus::TEvGetConfigResult> TClient::GetKesusConfig(TTestActorRuntime* runtime, const TString& kesusPath) { THolder<NKesus::TEvKesus::TEvGetConfig> request = MakeHolder<NKesus::TEvKesus::TEvGetConfig>(); - TActorId sender = runtime->AllocateEdgeActor(0); + TActorId sender = runtime->AllocateEdgeActor(0); ForwardToTablet(*runtime, GetKesusTabletId(kesusPath), sender, request.Release(), 0); TAutoPtr<IEventHandle> handle; diff --git a/ydb/core/testlib/test_client.h b/ydb/core/testlib/test_client.h index 1a7549c3add..2064752ab45 100644 --- a/ydb/core/testlib/test_client.h +++ b/ydb/core/testlib/test_client.h @@ -17,7 +17,7 @@ #include <ydb/library/yql/minikql/mkql_function_registry.h> #include <ydb/library/mkql_proto/protos/minikql.pb.h> #include <ydb/core/protos/flat_scheme_op.pb.h> -#include <library/cpp/grpc/server/grpc_server.h> +#include <library/cpp/grpc/server/grpc_server.h> #include <ydb/core/testlib/basics/runtime.h> #include <ydb/core/testlib/basics/appdata.h> #include <ydb/core/protos/kesus.pb.h> @@ -217,7 +217,7 @@ namespace Tests { TServer& operator =(TServer&& server) = default; virtual ~TServer(); - void EnableGRpc(const NGrpc::TServerOptions& options); + void EnableGRpc(const NGrpc::TServerOptions& options); void EnableGRpc(ui16 port); void SetupDefaultProfiles(); @@ -256,7 +256,7 @@ namespace Tests { TIntrusivePtr<NBus::TBusMessageQueue> Bus; const NBus::TBusServerSessionConfig BusServerSessionConfig; //BusServer hold const & on config TAutoPtr<NMsgBusProxy::IMessageBusServer> BusServer; - std::unique_ptr<NGrpc::TGRpcServer> GRpcServer; + std::unique_ptr<NGrpc::TGRpcServer> GRpcServer; TIntrusivePtr<NMonitoring::TDynamicCounters> GRpcServerRootCounters; }; @@ -283,7 +283,7 @@ namespace Tests { bool LoadTypes(); const NScheme::TTypeRegistry& GetTypeRegistry() const; const NScheme::TTypeMetadataRegistry& GetTypeMetadataRegistry() const; - const NMiniKQL::IFunctionRegistry& GetFunctionRegistry() const; + const NMiniKQL::IFunctionRegistry& GetFunctionRegistry() const; template <typename T> void PrepareRequest(TAutoPtr<T>&) {} @@ -511,13 +511,13 @@ namespace Tests { const TString DomainName; const bool SupportsRedirect; const TStoragePoolKinds StoragePoolTypes; - NScheme::TKikimrTypeRegistry TypeRegistry; - TIntrusivePtr<NMiniKQL::IFunctionRegistry> FunctionRegistry; + NScheme::TKikimrTypeRegistry TypeRegistry; + TIntrusivePtr<NMiniKQL::IFunctionRegistry> FunctionRegistry; NMsgBusProxy::TMsgBusClientConfig ClientConfig; std::shared_ptr<NMsgBusProxy::TMsgBusClient> Client; TMaybe<ui64> TypesEtag; NScheme::TTypeMetadataRegistry LoadedTypeMetadataRegistry; - TIntrusivePtr<NMiniKQL::IFunctionRegistry> LoadedFunctionRegistry; + TIntrusivePtr<NMiniKQL::IFunctionRegistry> LoadedFunctionRegistry; TString SecurityToken; }; diff --git a/ydb/core/testlib/test_pq_client.h b/ydb/core/testlib/test_pq_client.h index 9e07edcc531..f0599232508 100644 --- a/ydb/core/testlib/test_pq_client.h +++ b/ydb/core/testlib/test_pq_client.h @@ -722,7 +722,7 @@ public: } void RestartSchemeshard(TTestActorRuntime* runtime) { - TActorId sender = runtime->AllocateEdgeActor(); + TActorId sender = runtime->AllocateEdgeActor(); const ui64 schemeRoot = GetPatchedSchemeRoot(Tests::SchemeRoot, Settings.Domain, Settings.SupportsRedirect); ForwardToTablet(*runtime, schemeRoot, sender, new TEvents::TEvPoisonPill(), 0); TDispatchOptions options; @@ -774,7 +774,7 @@ public: TAutoPtr<NMsgBusProxy::TBusResponse> res = Ls("/Root/PQ/" + topic); Cerr << res->Record << "\n"; const ui64 tablet = res->Record.GetPathDescription().GetPersQueueGroup().GetBalancerTabletID(); - TActorId sender = runtime->AllocateEdgeActor(); + TActorId sender = runtime->AllocateEdgeActor(); ForwardToTablet(*runtime, tablet, sender, new TEvents::TEvPoisonPill(), 0); TDispatchOptions options; runtime->DispatchEvents(options); @@ -789,7 +789,7 @@ public: for (ui32 i = 0; i < pq.PartitionsSize(); ++i) { tablets.insert(pq.GetPartitions(i).GetTabletId()); } - TActorId sender = runtime->AllocateEdgeActor(); + TActorId sender = runtime->AllocateEdgeActor(); for (auto & tablet : tablets) { ForwardToTablet(*runtime, tablet, sender, new TEvents::TEvPoisonPill(), 0); TDispatchOptions options; diff --git a/ydb/core/testlib/ya.make b/ydb/core/testlib/ya.make index 87e28141843..137fa829f3a 100644 --- a/ydb/core/testlib/ya.make +++ b/ydb/core/testlib/ya.make @@ -35,8 +35,8 @@ PEERDIR( library/cpp/actors/core library/cpp/actors/interconnect library/cpp/grpc/client - library/cpp/grpc/server - library/cpp/grpc/server/actors + library/cpp/grpc/server + library/cpp/grpc/server/actors library/cpp/regex/pcre library/cpp/testing/gmock_in_unittest library/cpp/testing/unittest diff --git a/ydb/core/tracing/http.h b/ydb/core/tracing/http.h index 03c59d8eca5..25d04e1088a 100644 --- a/ydb/core/tracing/http.h +++ b/ydb/core/tracing/http.h @@ -1,5 +1,5 @@ #pragma once -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <ydb/core/base/tracing.h> #define COLLAPSED_REF_CONTENT(target, text) \ diff --git a/ydb/core/tracing/tablet_info.cpp b/ydb/core/tracing/tablet_info.cpp index b3447d55146..6eb99ffc60e 100644 --- a/ydb/core/tracing/tablet_info.cpp +++ b/ydb/core/tracing/tablet_info.cpp @@ -89,7 +89,7 @@ public: return NKikimrServices::TActivity::TABLET_LOOKUP_ACTOR; } - TTabletLookupActor(ui32 nodeId, const TActorId& sender, ui32 timeout) + TTabletLookupActor(ui32 nodeId, const TActorId& sender, ui32 timeout) : NodeId(nodeId) , Sender(sender) , Timeout(timeout) @@ -102,7 +102,7 @@ public: Become(&TThis::StateTabletLookupRequested); } else { // Broadcast request - const TActorId nameserviceId = NActors::GetNameserviceActorId(); + const TActorId nameserviceId = NActors::GetNameserviceActorId(); ctx.Send(nameserviceId, new TEvInterconnect::TEvListNodes()); Become(&TThis::StateNodeListRequested); } @@ -133,7 +133,7 @@ public: } void SendRequest(ui32 nodeId, const TActorContext& ctx) { - TActorId whiteboardServiceId = MakeNodeWhiteboardServiceId(nodeId); + TActorId whiteboardServiceId = MakeNodeWhiteboardServiceId(nodeId); ctx.Send(whiteboardServiceId , new TEvWhiteboard::TEvTabletLookupRequest() , IEventHandle::FlagTrackDelivery, nodeId); @@ -260,7 +260,7 @@ public: private: const ui32 NodeId; - const TActorId Sender; + const TActorId Sender; ui32 Timeout; ui32 Requested = 0; ui32 Received = 0; @@ -275,7 +275,7 @@ public: return NKikimrServices::TActivity::TRACE_LOOKUP_ACTOR; } - TTraceLookupActor(ui32 nodeId, ui64 tabletId, const TActorId& sender, ui32 timeout) + TTraceLookupActor(ui32 nodeId, ui64 tabletId, const TActorId& sender, ui32 timeout) : NodeId(nodeId) , TabletId(tabletId) , Sender(sender) @@ -289,7 +289,7 @@ public: Become(&TThis::StateTraceLookupRequested); } else { // Broadcast request - const TActorId nameserviceId = NActors::GetNameserviceActorId(); + const TActorId nameserviceId = NActors::GetNameserviceActorId(); ctx.Send(nameserviceId, new TEvInterconnect::TEvListNodes()); Become(&TThis::StateNodeListRequested); ctx.Schedule(TDuration::MilliSeconds(Timeout), new TEvents::TEvWakeup()); @@ -315,7 +315,7 @@ public: } void SendRequest(ui32 nodeId, const TActorContext& ctx) { - TActorId whiteboardServiceId = MakeNodeWhiteboardServiceId(nodeId); + TActorId whiteboardServiceId = MakeNodeWhiteboardServiceId(nodeId); THolder<TEvWhiteboard::TEvTraceLookupRequest> request = MakeHolder<TEvWhiteboard::TEvTraceLookupRequest>(); auto& record = request->Record; record.SetTabletID(TabletId); @@ -471,7 +471,7 @@ public: private: const ui32 NodeId; const ui64 TabletId; - const TActorId Sender; + const TActorId Sender; ui32 Timeout; ui32 Requested = 0; ui32 Received = 0; @@ -484,13 +484,13 @@ public: return NKikimrServices::TActivity::TRACE_REQUEST_ACTOR; } - TTraceRequestActor(const NTracing::TTraceInfo& traceInfo, const TActorId& sender) + TTraceRequestActor(const NTracing::TTraceInfo& traceInfo, const TActorId& sender) : TraceInfo(traceInfo) , Sender(sender) {} void Bootstrap(const TActorContext& ctx) { - TActorId whiteboardServiceId = MakeNodeWhiteboardServiceId(TraceInfo.NodeId); + TActorId whiteboardServiceId = MakeNodeWhiteboardServiceId(TraceInfo.NodeId); THolder<TEvWhiteboard::TEvTraceRequest> request = MakeHolder<TEvWhiteboard::TEvTraceRequest>(); auto& record = request->Record; record.SetTabletID(TraceInfo.TabletId); @@ -551,7 +551,7 @@ public: private: const NTracing::TTraceInfo TraceInfo; - const TActorId Sender; + const TActorId Sender; }; class TSignalBodyRequestActor : public TActorBootstrapped<TSignalBodyRequestActor> { @@ -560,14 +560,14 @@ public: return NKikimrServices::TActivity::SIGNAL_BODY_REQUEST_ACTOR; } - TSignalBodyRequestActor(const NTracing::TTraceInfo& traceInfo, const TString& signalId, const TActorId& sender) + TSignalBodyRequestActor(const NTracing::TTraceInfo& traceInfo, const TString& signalId, const TActorId& sender) : TraceInfo(traceInfo) , SignalId(signalId) , Sender(sender) {} void Bootstrap(const TActorContext& ctx) { - TActorId whiteboardServiceId = MakeNodeWhiteboardServiceId(TraceInfo.NodeId); + TActorId whiteboardServiceId = MakeNodeWhiteboardServiceId(TraceInfo.NodeId); THolder<TEvWhiteboard::TEvSignalBodyRequest> request = MakeHolder<TEvWhiteboard::TEvSignalBodyRequest>(); auto& record = request->Record; record.SetTabletID(TraceInfo.TabletId); @@ -618,7 +618,7 @@ public: private: const NTracing::TTraceInfo TraceInfo; const TString SignalId; - const TActorId Sender; + const TActorId Sender; }; ui64 TryParseTabletId(TStringBuf tabletIdParam) { diff --git a/ydb/core/tracing/tablet_info.h b/ydb/core/tracing/tablet_info.h index 95491d4aa41..c3ebf102250 100644 --- a/ydb/core/tracing/tablet_info.h +++ b/ydb/core/tracing/tablet_info.h @@ -7,9 +7,9 @@ namespace NKikimr { namespace NTabletInfo { -inline TActorId MakeTabletInfoID(ui32 node = 0) { +inline TActorId MakeTabletInfoID(ui32 node = 0) { char x[12] = {'t','a','b','l','e','t','i','n','f','o','r','m'}; - return TActorId(node, TStringBuf(x, 12)); + return TActorId(node, TStringBuf(x, 12)); } IActor* CreateTabletInfo(); diff --git a/ydb/core/tracing/trace.cpp b/ydb/core/tracing/trace.cpp index ee9e21371dd..e243c047d74 100644 --- a/ydb/core/tracing/trace.cpp +++ b/ydb/core/tracing/trace.cpp @@ -2,9 +2,9 @@ #include "http.h" #include <ydb/core/protos/tracing.pb.h> - -#include <util/string/cast.h> - + +#include <util/string/cast.h> + namespace NKikimr { namespace NTracing { diff --git a/ydb/core/tx/coordinator/coordinator.h b/ydb/core/tx/coordinator/coordinator.h index f345ff38ea5..e7a76df3cee 100644 --- a/ydb/core/tx/coordinator/coordinator.h +++ b/ydb/core/tx/coordinator/coordinator.h @@ -18,7 +18,7 @@ namespace NFlatTxCoordinator { namespace NKikimr { -IActor* CreateFlatTxCoordinator(const TActorId &tablet, TTabletStorageInfo *info); +IActor* CreateFlatTxCoordinator(const TActorId &tablet, TTabletStorageInfo *info); struct TEvTxCoordinator { enum EEv { diff --git a/ydb/core/tx/coordinator/coordinator__configure.cpp b/ydb/core/tx/coordinator/coordinator__configure.cpp index c9e480a2e70..338432dbc35 100644 --- a/ydb/core/tx/coordinator/coordinator__configure.cpp +++ b/ydb/core/tx/coordinator/coordinator__configure.cpp @@ -4,7 +4,7 @@ namespace NKikimr { namespace NFlatTxCoordinator { struct TTxCoordinator::TTxConfigure : public TTransactionBase<TTxCoordinator> { - TActorId AckTo; + TActorId AckTo; ui64 Version; ui64 Resolution; TVector<TTabletId> Mediators; @@ -12,7 +12,7 @@ struct TTxCoordinator::TTxConfigure : public TTransactionBase<TTxCoordinator> { TAutoPtr<TEvSubDomain::TEvConfigureStatus> Respond; bool ConfigurationApplied; - TTxConfigure(TSelf *coordinator, TActorId ackTo, + TTxConfigure(TSelf *coordinator, TActorId ackTo, ui64 version, ui64 resolution, const TVector<TTabletId>& mediators, const NKikimrSubDomains::TProcessingParams& config) : TBase(coordinator) diff --git a/ydb/core/tx/coordinator/coordinator__monitoring.cpp b/ydb/core/tx/coordinator/coordinator__monitoring.cpp index 1336769c4e5..550c8ef2815 100644 --- a/ydb/core/tx/coordinator/coordinator__monitoring.cpp +++ b/ydb/core/tx/coordinator/coordinator__monitoring.cpp @@ -5,10 +5,10 @@ namespace NKikimr { namespace NFlatTxCoordinator { struct TTxCoordinator::TTxMonitoring : public TTxCoordinator::TTxConsistencyCheck { - TActorId ActorToRespond; + TActorId ActorToRespond; TString CheckResult; - TTxMonitoring(TSelf* self, const TActorId& actorToRespond) + TTxMonitoring(TSelf* self, const TActorId& actorToRespond) : TTxCoordinator::TTxConsistencyCheck(self) , ActorToRespond(actorToRespond) {} diff --git a/ydb/core/tx/coordinator/coordinator_impl.cpp b/ydb/core/tx/coordinator/coordinator_impl.cpp index 5d92fc17c5a..b4eeb184e29 100644 --- a/ydb/core/tx/coordinator/coordinator_impl.cpp +++ b/ydb/core/tx/coordinator/coordinator_impl.cpp @@ -13,7 +13,7 @@ namespace NFlatTxCoordinator { static constexpr TDuration MaxEmptyPlanDelay = TDuration::Seconds(1); -static void SendTransactionStatus(const TActorId &proxy, TEvTxProxy::TEvProposeTransactionStatus::EStatus status, +static void SendTransactionStatus(const TActorId &proxy, TEvTxProxy::TEvProposeTransactionStatus::EStatus status, ui64 txid, ui64 stepId, const TActorContext &ctx, ui64 tabletId) { LOG_DEBUG_S(ctx, NKikimrServices::TX_COORDINATOR, "tablet# " << tabletId << " txid# " << txid << " step# " << stepId << " Status# " << status << " SEND to# " << proxy.ToString() << " Proxy marker# C1"); @@ -21,7 +21,7 @@ static void SendTransactionStatus(const TActorId &proxy, TEvTxProxy::TEvProposeT } static TAutoPtr<TTransactionProposal> MakeTransactionProposal(TEvTxProxy::TEvProposeTransaction::TPtr &ev, NMonitoring::TDynamicCounters::TCounterPtr &counter) { - const TActorId &sender = ev->Sender; + const TActorId &sender = ev->Sender; const NKikimrTx::TEvProposeTransaction &record = ev->Get()->Record; const NKikimrTx::TProxyTransaction &txrec = record.GetTransaction(); @@ -47,7 +47,7 @@ static TAutoPtr<TTransactionProposal> MakeTransactionProposal(TEvTxProxy::TEvPro const ui32 TTxCoordinator::Schema::CurrentVersion = 1; -TTxCoordinator::TTxCoordinator(TTabletStorageInfo *info, const TActorId &tablet) +TTxCoordinator::TTxCoordinator(TTabletStorageInfo *info, const TActorId &tablet) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, new NMiniKQL::TMiniKQLFactory) #ifdef COORDINATOR_LOG_TO_FILE @@ -268,7 +268,7 @@ void TTxCoordinator::Handle(TEvTxCoordinator::TEvCoordinatorConfirmPlan::TPtr &e } } -void TTxCoordinator::DoConfiguration(const TEvSubDomain::TEvConfigure &ev, const TActorContext &ctx, const TActorId &ackTo) { +void TTxCoordinator::DoConfiguration(const TEvSubDomain::TEvConfigure &ev, const TActorContext &ctx, const TActorId &ackTo) { const TEvSubDomain::TEvConfigure::ProtoRecordType &record = ev.Record; if(0 == record.MediatorsSize()) { @@ -432,7 +432,7 @@ void TTxCoordinator::OnStopGuardComplete(const TActorContext &ctx) { } -IActor* CreateFlatTxCoordinator(const TActorId &tablet, TTabletStorageInfo *info) { +IActor* CreateFlatTxCoordinator(const TActorId &tablet, TTabletStorageInfo *info) { return new NFlatTxCoordinator::TTxCoordinator(info, tablet); } diff --git a/ydb/core/tx/coordinator/coordinator_impl.h b/ydb/core/tx/coordinator/coordinator_impl.h index b32add61498..39018aff1e7 100644 --- a/ydb/core/tx/coordinator/coordinator_impl.h +++ b/ydb/core/tx/coordinator/coordinator_impl.h @@ -32,7 +32,7 @@ typedef ui64 TTabletId; typedef ui64 TTxId; struct TTransactionProposal { - TActorId Proxy; + TActorId Proxy; TTxId TxId; // could be zero, then transaction is not saved a-priori and must be idempotent, with body provided TStepId MinStep; // not plan before (tablet would be not ready), could be zero @@ -54,7 +54,7 @@ struct TTransactionProposal { TInstant AcceptMoment; bool IgnoreLowDiskSpace; - TTransactionProposal(const TActorId &proxy, TTxId txId, TStepId minStep, TStepId maxStep, bool ignoreLowDiskSpace) + TTransactionProposal(const TActorId &proxy, TTxId txId, TStepId minStep, TStepId maxStep, bool ignoreLowDiskSpace) : Proxy(proxy) , TxId(txId) , MinStep(minStep) @@ -66,11 +66,11 @@ struct TTransactionProposal { struct TCoordinatorStepConfirmations { struct TEntry { TTxId TxId; - TActorId ProxyId; + TActorId ProxyId; TEvTxProxy::TEvProposeTransactionStatus::EStatus Status; TStepId Step; - TEntry(TTxId txid, const TActorId &proxyId, TEvTxProxy::TEvProposeTransactionStatus::EStatus status, TStepId step) + TEntry(TTxId txid, const TActorId &proxyId, TEvTxProxy::TEvProposeTransactionStatus::EStatus status, TStepId step) : TxId(txid) , ProxyId(proxyId) , Status(status) @@ -135,7 +135,7 @@ struct TMediatorConfirmations { {} }; -IActor* CreateTxCoordinatorMediatorQueue(const TActorId &owner, ui64 coordinator, ui64 mediator, ui64 coordinatorGeneration); +IActor* CreateTxCoordinatorMediatorQueue(const TActorId &owner, ui64 coordinator, ui64 mediator, ui64 coordinatorGeneration); using NTabletFlatExecutor::TTabletExecutedFlat; using NTabletFlatExecutor::ITransaction; @@ -277,7 +277,7 @@ class TTxCoordinator : public TActor<TTxCoordinator>, public TTabletExecutedFlat struct TMediator { typedef TOneOneQueueInplace<TMediatorStep *, 32> TStepQueue; - TActorId QueueActor; + TActorId QueueActor; ui64 GenCookie; bool PushUpdates; @@ -331,8 +331,8 @@ public: static const ui32 CurrentVersion; struct Transaction : Table<0> { - struct ID : Column<0, NScheme::NTypeIds::Uint64> {}; // PK - struct Plan : Column<1, NScheme::NTypeIds::Uint64> {}; + struct ID : Column<0, NScheme::NTypeIds::Uint64> {}; // PK + struct Plan : Column<1, NScheme::NTypeIds::Uint64> {}; struct AffectedSet : Column<2, NScheme::NTypeIds::String> { using Type = TVector<TTabletId>; }; using TKey = TableKey<ID>; @@ -340,9 +340,9 @@ public: }; struct AffectedSet : Table<4> { - struct MediatorID : Column<1, NScheme::NTypeIds::Uint64> {}; + struct MediatorID : Column<1, NScheme::NTypeIds::Uint64> {}; struct TransactionID : Column<2, Transaction::ID::ColumnType> {}; - struct DataShardID : Column<3, NScheme::NTypeIds::Uint64> {}; + struct DataShardID : Column<3, NScheme::NTypeIds::Uint64> {}; using TKey = TableKey<MediatorID, TransactionID, DataShardID>; using TColumns = TableColumns<MediatorID, TransactionID, DataShardID>; @@ -355,8 +355,8 @@ public: AcquireReadStepLast, }; - struct StateKey : Column<0, NScheme::NTypeIds::Uint64> { using Type = EKeyType; }; // PK - struct StateValue : Column<1, NScheme::NTypeIds::Uint64> {}; + struct StateKey : Column<0, NScheme::NTypeIds::Uint64> { using Type = EKeyType; }; // PK + struct StateValue : Column<1, NScheme::NTypeIds::Uint64> {}; using TKey = TableKey<StateKey>; using TColumns = TableColumns<StateKey, StateValue>; @@ -477,7 +477,7 @@ private: void Handle(TEvents::TEvPoisonPill::TPtr& ev, const TActorContext& ctx); - void DoConfiguration(const TEvSubDomain::TEvConfigure &ev, const TActorContext &ctx, const TActorId &ackTo = TActorId()); + void DoConfiguration(const TEvSubDomain::TEvConfigure &ev, const TActorContext &ctx, const TActorId &ackTo = TActorId()); void Sync(ui64 mediator, const TActorContext &ctx); void Sync(const TActorContext &ctx); @@ -505,7 +505,7 @@ public: return NKikimrServices::TActivity::TX_COORDINATOR_ACTOR; } - TTxCoordinator(TTabletStorageInfo *info, const TActorId &tablet); + TTxCoordinator(TTabletStorageInfo *info, const TActorId &tablet); // no incomming pipes is allowed in StateInit STFUNC_TABLET_INIT(StateInit, diff --git a/ydb/core/tx/coordinator/mediator_queue.cpp b/ydb/core/tx/coordinator/mediator_queue.cpp index 920595c9c74..9b3955fa1bf 100644 --- a/ydb/core/tx/coordinator/mediator_queue.cpp +++ b/ydb/core/tx/coordinator/mediator_queue.cpp @@ -10,13 +10,13 @@ namespace NKikimr { namespace NFlatTxCoordinator { class TTxCoordinatorMediatorQueue : public TActorBootstrapped<TTxCoordinatorMediatorQueue> { - const TActorId Owner; + const TActorId Owner; const ui64 Coordinator; const ui64 Mediator; const ui64 CoordinatorGeneration; - TActorId PipeClient; + TActorId PipeClient; ui64 GenCookie; ui64 PrevStep; @@ -26,7 +26,7 @@ class TTxCoordinatorMediatorQueue : public TActorBootstrapped<TTxCoordinatorMedi void Die(const TActorContext &ctx) override { if (PipeClient) { NTabletPipe::CloseClient(ctx, PipeClient); - PipeClient = TActorId(); + PipeClient = TActorId(); } return IActor::Die(ctx); } @@ -36,7 +36,7 @@ class TTxCoordinatorMediatorQueue : public TActorBootstrapped<TTxCoordinatorMedi if (PipeClient) { NTabletPipe::CloseClient(ctx, PipeClient); - PipeClient = TActorId(); + PipeClient = TActorId(); } PipeClient = ctx.RegisterWithSameMailbox(NTabletPipe::CreateClient(ctx.SelfID, Mediator)); @@ -138,7 +138,7 @@ public: return NKikimrServices::TActivity::TX_COORDINATOR_MEDIATORQ_ACTOR; } - TTxCoordinatorMediatorQueue(const TActorId &owner, ui64 coordinator, ui64 mediator, ui64 coordinatorGeneration) + TTxCoordinatorMediatorQueue(const TActorId &owner, ui64 coordinator, ui64 mediator, ui64 coordinatorGeneration) : Owner(owner) , Coordinator(coordinator) , Mediator(mediator) @@ -172,7 +172,7 @@ public: } }; -IActor* CreateTxCoordinatorMediatorQueue(const TActorId &owner, ui64 coordinator, ui64 mediator, ui64 coordinatorGeneration) { +IActor* CreateTxCoordinatorMediatorQueue(const TActorId &owner, ui64 coordinator, ui64 mediator, ui64 coordinatorGeneration) { return new NFlatTxCoordinator::TTxCoordinatorMediatorQueue(owner, coordinator, mediator, coordinatorGeneration); } diff --git a/ydb/core/tx/datashard/datashard.cpp b/ydb/core/tx/datashard/datashard.cpp index f54d279bcee..d86568b4a15 100644 --- a/ydb/core/tx/datashard/datashard.cpp +++ b/ydb/core/tx/datashard/datashard.cpp @@ -6,7 +6,7 @@ #include <ydb/core/scheme/scheme_tablecell.h> #include <ydb/core/tablet/tablet_counters_protobuf.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <contrib/libs/apache/arrow/cpp/src/arrow/api.h> @@ -410,7 +410,7 @@ void TDataShard::SendDelayedAcks(const TActorContext& ctx, TVector<THolder<IEven void TDataShard::SendResult(const TActorContext &ctx, TOutputOpData::TResultPtr &res, - const TActorId &target, + const TActorId &target, ui64 step, ui64 txId) { @@ -1682,7 +1682,7 @@ void TDataShard::Handle(TEvPrivate::TEvDelayedProposeTransaction::TPtr &ev, cons return; } - TActorId target = item.Event->Get()->GetSource(); + TActorId target = item.Event->Get()->GetSource(); ui64 cookie = item.Event->Cookie; auto kind = item.Event->Get()->GetTxKind(); auto txId = item.Event->Get()->GetTxId(); @@ -1747,7 +1747,7 @@ void TDataShard::Handle(TEvTabletPipe::TEvClientConnected::TPtr &ev, const TActo } TEvTabletPipe::TEvClientConnected *msg = ev->Get(); if (msg->Status != NKikimrProto::OK) { - SchemeShardPipe = TActorId(); + SchemeShardPipe = TActorId(); NotifySchemeshard(ctx); } return; @@ -1755,18 +1755,18 @@ void TDataShard::Handle(TEvTabletPipe::TEvClientConnected::TPtr &ev, const TActo if (ev->Get()->Status != NKikimrProto::OK) { if (ev->Get()->ClientId == StateReportPipe) { - StateReportPipe = TActorId(); + StateReportPipe = TActorId(); ReportState(ctx, State); return; } if (ev->Get()->ClientId == DbStatsReportPipe) { - DbStatsReportPipe = TActorId(); + DbStatsReportPipe = TActorId(); return; } if (ev->Get()->ClientId == TableResolvePipe) { - TableResolvePipe = TActorId(); + TableResolvePipe = TActorId(); ResolveTablePath(ctx); return; } @@ -1821,19 +1821,19 @@ void TDataShard::Handle(TEvTabletPipe::TEvClientDestroyed::TPtr &ev, const TActo LOG_ERROR(ctx, NKikimrServices::TX_DATASHARD, "Datashard's schemeshard pipe destroyed while no messages to sent at %" PRIu64, TabletID()); } - SchemeShardPipe = TActorId(); + SchemeShardPipe = TActorId(); NotifySchemeshard(ctx); return; } if (ev->Get()->ClientId == StateReportPipe) { - StateReportPipe = TActorId(); + StateReportPipe = TActorId(); ReportState(ctx, State); return; } if (ev->Get()->ClientId == DbStatsReportPipe) { - DbStatsReportPipe = TActorId(); + DbStatsReportPipe = TActorId(); return; } diff --git a/ydb/core/tx/datashard/datashard.h b/ydb/core/tx/datashard/datashard.h index a4f19f40610..5a93c035026 100644 --- a/ydb/core/tx/datashard/datashard.h +++ b/ydb/core/tx/datashard/datashard.h @@ -308,13 +308,13 @@ struct TEvDataShard { { } - TEvGetShardState(const TActorId& source) + TEvGetShardState(const TActorId& source) { - ActorIdToProto(source, Record.MutableSource()); + ActorIdToProto(source, Record.MutableSource()); } - TActorId GetSource() const { - return ActorIdFromProto(Record.GetSource()); + TActorId GetSource() const { + return ActorIdFromProto(Record.GetSource()); } }; @@ -344,8 +344,8 @@ struct TEvDataShard { TEvSchemaChanged() {} - TEvSchemaChanged(const TActorId& source, ui64 origin, ui32 state, ui64 txId, ui64 step, ui32 generation) { - ActorIdToProto(source, Record.MutableSource()); + TEvSchemaChanged(const TActorId& source, ui64 origin, ui32 state, ui64 txId, ui64 step, ui32 generation) { + ActorIdToProto(source, Record.MutableSource()); Record.SetOrigin(origin); Record.SetState(state); Record.SetTxId(txId); @@ -353,8 +353,8 @@ struct TEvDataShard { Record.SetGeneration(generation); } - TActorId GetSource() const { - return ActorIdFromProto(Record.GetSource()); + TActorId GetSource() const { + return ActorIdFromProto(Record.GetSource()); } ui32 GetGeneration() const { @@ -377,14 +377,14 @@ struct TEvDataShard { TEvStateChanged() {} - TEvStateChanged(const TActorId& source, ui64 tabletId, ui32 state) { - ActorIdToProto(source, Record.MutableSource()); + TEvStateChanged(const TActorId& source, ui64 tabletId, ui32 state) { + ActorIdToProto(source, Record.MutableSource()); Record.SetTabletId(tabletId); Record.SetState(state); } - TActorId GetSource() const { - return ActorIdFromProto(Record.GetSource()); + TActorId GetSource() const { + return ActorIdFromProto(Record.GetSource()); } }; @@ -405,11 +405,11 @@ struct TEvDataShard { { } - TEvProposeTransaction(NKikimrTxDataShard::ETransactionKind txKind, const TActorId& source, ui64 txId, + TEvProposeTransaction(NKikimrTxDataShard::ETransactionKind txKind, const TActorId& source, ui64 txId, const TStringBuf& txBody, ui32 flags = NDataShard::TTxFlags::Default) { Record.SetTxKind(txKind); - ActorIdToProto(source, Record.MutableSource()); + ActorIdToProto(source, Record.MutableSource()); Record.SetTxId(txId); Record.SetExecLevel(0); Record.SetTxBody(txBody.data(), txBody.size()); @@ -431,7 +431,7 @@ struct TEvDataShard { { } - TEvProposeTransaction(NKikimrTxDataShard::ETransactionKind txKind, ui64 ssId, const TActorId& source, ui64 txId, + TEvProposeTransaction(NKikimrTxDataShard::ETransactionKind txKind, ui64 ssId, const TActorId& source, ui64 txId, const TStringBuf& txBody, const NKikimrSubDomains::TProcessingParams &processingParams, ui32 flags = NDataShard::TTxFlags::Default) : TEvProposeTransaction(txKind, source, txId, txBody, flags) { @@ -453,8 +453,8 @@ struct TEvDataShard { return Record.GetTxKind(); } - TActorId GetSource() const { - return ActorIdFromProto(Record.GetSource()); + TActorId GetSource() const { + return ActorIdFromProto(Record.GetSource()); } ui64 GetTxId() const { @@ -1152,7 +1152,7 @@ struct TEvDataShard { struct TEvGetS3Upload : public TEventLocal<TEvGetS3Upload, TEvDataShard::EvGetS3Upload> { - TActorId ReplyTo; + TActorId ReplyTo; ui64 TxId; explicit TEvGetS3Upload(const TActorId& replyTo, ui64 txId) @@ -1165,11 +1165,11 @@ struct TEvDataShard { struct TEvStoreS3UploadId : public TEventLocal<TEvStoreS3UploadId, TEvDataShard::EvStoreS3UploadId> { - TActorId ReplyTo; + TActorId ReplyTo; ui64 TxId; TString UploadId; - explicit TEvStoreS3UploadId(const TActorId& replyTo, ui64 txId, const TString& uploadId) + explicit TEvStoreS3UploadId(const TActorId& replyTo, ui64 txId, const TString& uploadId) : ReplyTo(replyTo) , TxId(txId) , UploadId(uploadId) diff --git a/ydb/core/tx/datashard/datashard__build_index.cpp b/ydb/core/tx/datashard/datashard__build_index.cpp index 389a14b6ef9..5aebf94dc51 100644 --- a/ydb/core/tx/datashard/datashard__build_index.cpp +++ b/ydb/core/tx/datashard/datashard__build_index.cpp @@ -193,8 +193,8 @@ class TBuildIndexScan : public TActor<TBuildIndexScan>, public NTable::IScan { const TBuildIndexRecord::TSeqNo SeqNo; const ui64 DataShardId; - const TActorId DatashardActorId; - const TActorId SchemeShardActorID; + const TActorId DatashardActorId; + const TActorId SchemeShardActorID; const TTags ScanTags; // first: columns we scan, order as in IndexTable const std::shared_ptr<TTypes> UploadColumnsTypes; // columns types we upload to indexTable @@ -212,7 +212,7 @@ class TBuildIndexScan : public TActor<TBuildIndexScan>, public NTable::IScan { TBufferData WriteBuf; TSerializedCellVec LastUploadedKey; - TActorId Uploader; + TActorId Uploader; ui64 RetryCount = 0; TUploadMonStats Stats = TUploadMonStats("tablets", "build_index_upload"); @@ -227,8 +227,8 @@ public: const TString& target, const TBuildIndexRecord::TSeqNo& seqNo, ui64 dataShardId, - const TActorId& datashardActorId, - const TActorId& schemeshardActorId, + const TActorId& datashardActorId, + const TActorId& schemeshardActorId, const TSerializedTableRange& range, const TVector<TString> targetIndexColumns, const TVector<TString> targetDataColumns, @@ -532,8 +532,8 @@ TAutoPtr<NTable::IScan> CreateBuildIndexScan( TString target, const TBuildIndexRecord::TSeqNo& seqNo, ui64 dataShardId, - const TActorId& datashardActorId, - const TActorId& schemeshardActorId, + const TActorId& datashardActorId, + const TActorId& schemeshardActorId, const TSerializedTableRange& range, const TVector<TString>& targetIndexColumns, const TVector<TString>& targetDataColumns, diff --git a/ydb/core/tx/datashard/datashard__cleanup_borrowed.cpp b/ydb/core/tx/datashard/datashard__cleanup_borrowed.cpp index 01e91154fa2..c53faa1c603 100644 --- a/ydb/core/tx/datashard/datashard__cleanup_borrowed.cpp +++ b/ydb/core/tx/datashard/datashard__cleanup_borrowed.cpp @@ -17,8 +17,8 @@ class TDataShard::TTxMonitoringCleanupBorrowedPartsActor { public: TTxMonitoringCleanupBorrowedPartsActor( - const TActorId& owner, - const TActorId& replyTo, + const TActorId& owner, + const TActorId& replyTo, THashMap<TLogoBlobID, TVector<ui64>> borrowedParts, bool dryRun) : Owner(owner) @@ -184,8 +184,8 @@ public: } private: - const TActorId Owner; - const TActorId ReplyTo; + const TActorId Owner; + const TActorId ReplyTo; const THashMap<TLogoBlobID, TVector<ui64>> BorrowedParts; const bool DryRun; diff --git a/ydb/core/tx/datashard/datashard__conditional_erase_rows.cpp b/ydb/core/tx/datashard/datashard__conditional_erase_rows.cpp index 3838fb02371..026b14e461f 100644 --- a/ydb/core/tx/datashard/datashard__conditional_erase_rows.cpp +++ b/ydb/core/tx/datashard/datashard__conditional_erase_rows.cpp @@ -34,7 +34,7 @@ protected: class TCondEraseScan: public IActor, public IScan, public IEraserOps { struct TDataShardId { - TActorId ActorId; + TActorId ActorId; ui64 TabletId; }; @@ -354,7 +354,7 @@ protected: private: const TDataShardId DataShard; - TActorId ReplyTo; + TActorId ReplyTo; const ui64 TxId; THolder<IEraseRowsCondition> Condition; diff --git a/ydb/core/tx/datashard/datashard__plan_step.cpp b/ydb/core/tx/datashard/datashard__plan_step.cpp index 151186d057a..6399598865a 100644 --- a/ydb/core/tx/datashard/datashard__plan_step.cpp +++ b/ydb/core/tx/datashard/datashard__plan_step.cpp @@ -36,7 +36,7 @@ bool TDataShard::TTxPlanStep::Execute(TTransactionContext &txc, const TActorCont txIds.push_back(tx.GetTxId()); - TActorId txOwner = ActorIdFromProto(tx.GetAckTo()); + TActorId txOwner = ActorIdFromProto(tx.GetAckTo()); TxByAck[txOwner].push_back(tx.GetTxId()); } diff --git a/ydb/core/tx/datashard/datashard__propose_tx_base.cpp b/ydb/core/tx/datashard/datashard__propose_tx_base.cpp index 9ca58b70836..7dfb1a294e6 100644 --- a/ydb/core/tx/datashard/datashard__propose_tx_base.cpp +++ b/ydb/core/tx/datashard/datashard__propose_tx_base.cpp @@ -64,7 +64,7 @@ bool TDataShard::TTxProposeTransactionBase::Execute(NTabletFlatExecutor::TTransa << result->GetStatus() << " errors: " << errors); } - TActorId target = Op ? Op->GetTarget() : Ev->Get()->GetSource(); + TActorId target = Op ? Op->GetTarget() : Ev->Get()->GetSource(); ui64 cookie = Op ? Op->GetCookie() : Ev->Cookie; ctx.Send(target, result.Release(), 0, cookie); diff --git a/ydb/core/tx/datashard/datashard__read_columns.cpp b/ydb/core/tx/datashard/datashard__read_columns.cpp index d6231dd59bc..2b56a192d10 100644 --- a/ydb/core/tx/datashard/datashard__read_columns.cpp +++ b/ydb/core/tx/datashard/datashard__read_columns.cpp @@ -35,8 +35,8 @@ struct TKeyBoundary { }; class TReadColumnsScan : public INoTxScan { - const TActorId ReplyTo; - const TActorId DatashardActorId; + const TActorId ReplyTo; + const TActorId DatashardActorId; const TString TableName; const ui64 TabletId; const TKeyBoundary From; @@ -66,8 +66,8 @@ public: std::unique_ptr<IBlockBuilder>&& blockBuilder, ui64 rowsLimit, ui64 bytesLimit, const TKeyBoundary& shardEnd, - const TActorId& replyTo, - const TActorId& datashardActorId, + const TActorId& replyTo, + const TActorId& datashardActorId, TMaybe<TSnapshotKey> snapshotKey, const TString& tableName, ui64 tabletId) @@ -152,7 +152,7 @@ public: Result->Record.SetErrorDescription("Scan aborted"); } - TlsActivationContext->Send(new IEventHandle(ReplyTo, TActorId(), Result.Release())); + TlsActivationContext->Send(new IEventHandle(ReplyTo, TActorId(), Result.Release())); TlsActivationContext->Send(new IEventHandle(DatashardActorId, TActorId(), new TDataShard::TEvPrivate::TEvScanStats(Rows, Bytes))); return this; diff --git a/ydb/core/tx/datashard/datashard__stats.cpp b/ydb/core/tx/datashard/datashard__stats.cpp index ecfd288d270..129a14dca5f 100644 --- a/ydb/core/tx/datashard/datashard__stats.cpp +++ b/ydb/core/tx/datashard/datashard__stats.cpp @@ -49,7 +49,7 @@ public: } private: - TActorId ReplyTo; + TActorId ReplyTo; ui64 TableId; ui64 IndexSize; TInstant StatsUpdateTime; diff --git a/ydb/core/tx/datashard/datashard_active_transaction.cpp b/ydb/core/tx/datashard/datashard_active_transaction.cpp index d640f8d15ce..0226bd42e25 100644 --- a/ydb/core/tx/datashard/datashard_active_transaction.cpp +++ b/ydb/core/tx/datashard/datashard_active_transaction.cpp @@ -343,7 +343,7 @@ TActiveTransaction::TActiveTransaction(TDataShard *self, TTransactionContext &txc, const TActorContext &ctx, const TBasicOpInfo &op, - const TActorId &target, + const TActorId &target, const TString &txBody, const TVector<TSysTables::TLocksTable::TLock> &locks, ui64 artifactFlags) @@ -373,7 +373,7 @@ void TActiveTransaction::FillTxData(TValidatedDataTx::TPtr dataTx) void TActiveTransaction::FillTxData(TDataShard *self, TTransactionContext &txc, const TActorContext &ctx, - const TActorId &target, + const TActorId &target, const TString &txBody, const TVector<TSysTables::TLocksTable::TLock> &locks, ui64 artifactFlags) diff --git a/ydb/core/tx/datashard/datashard_active_transaction.h b/ydb/core/tx/datashard/datashard_active_transaction.h index 6e8d9679443..43abd525e15 100644 --- a/ydb/core/tx/datashard/datashard_active_transaction.h +++ b/ydb/core/tx/datashard/datashard_active_transaction.h @@ -58,7 +58,7 @@ struct TSchemaOperation { ui64 TxId; EType Type; - TActorId Source; + TActorId Source; ui64 TabletId; ui64 MinStep; ui64 MaxStep; @@ -178,10 +178,10 @@ public: TVector<NMiniKQL::IChangeCollector::TChange> GetCollectedChanges() const { return EngineBay.GetCollectedChanges(); } - TActorId Source() const { return Source_; } - void SetSource(const TActorId& actorId) { Source_ = actorId; } + TActorId Source() const { return Source_; } + void SetSource(const TActorId& actorId) { Source_ = actorId; } void SetStep(ui64 step) { StepTxId_.Step = step; } - bool IsProposed() const { return Source_ != TActorId(); } + bool IsProposed() const { return Source_ != TActorId(); } bool IsTableRead() const { return Tx.HasReadTableTransaction(); } @@ -201,7 +201,7 @@ public: NMiniKQL::TKqpDatashardComputeContext& GetKqpComputeCtx() { Y_VERIFY(IsKqpDataTx()); return EngineBay.GetKqpComputeCtx(); } bool HasStreamResponse() const { return Tx.GetStreamResponse(); } - TActorId GetSink() const { return ActorIdFromProto(Tx.GetSink()); } + TActorId GetSink() const { return ActorIdFromProto(Tx.GetSink()); } const NKikimrTxDataShard::TReadTableTransaction &GetReadTableTransaction() const { return Tx.GetReadTableTransaction(); } ui32 ExtractKeys(bool allowErrors); @@ -230,7 +230,7 @@ private: TStepOrder StepTxId_; ui64 TabletId_; TString TxBody; - TActorId Source_; + TActorId Source_; TEngineBay EngineBay; NKikimrTxDataShard::TDataTransaction Tx; NKikimrTxDataShard::TError::EKind ErrCode; @@ -331,7 +331,7 @@ public: TTransactionContext &txc, const TActorContext &ctx, const TBasicOpInfo &op, - const TActorId &target, + const TActorId &target, const TString &txBody, const TVector<TSysTables::TLocksTable::TLock> &locks, ui64 artifactFlags); @@ -342,7 +342,7 @@ public: void FillTxData(TDataShard *self, TTransactionContext &txc, const TActorContext &ctx, - const TActorId &target, + const TActorId &target, const TString &txBody, const TVector<TSysTables::TLocksTable::TLock> &locks, ui64 artifactFlags); @@ -523,11 +523,11 @@ public: TActorId GetAsyncJobActor() const { return AsyncJobActor; } void KillAsyncJobActor(const TActorContext& ctx); - void SetStreamSink(TActorId sink) { StreamSink = sink; } - TActorId GetStreamSink() const { return StreamSink; } + void SetStreamSink(TActorId sink) { StreamSink = sink; } + TActorId GetStreamSink() const { return StreamSink; } - void SetScanActor(TActorId aid) { ScanActor = aid; } - TActorId GetScanActor() const { return ScanActor; } + void SetScanActor(TActorId aid) { ScanActor = aid; } + TActorId GetScanActor() const { return ScanActor; } ui64 IncrementPageFaultCount() { return ++PageFaultCount; @@ -556,8 +556,8 @@ private: ui64 ScanSnapshotId; ui64 ScanTask; TActorId AsyncJobActor; - TActorId StreamSink; - TActorId ScanActor; + TActorId StreamSink; + TActorId ScanActor; ui64 PageFaultCount = 0; }; diff --git a/ydb/core/tx/datashard/datashard_impl.h b/ydb/core/tx/datashard/datashard_impl.h index 29fedb41fb1..11313474f80 100644 --- a/ydb/core/tx/datashard/datashard_impl.h +++ b/ydb/core/tx/datashard/datashard_impl.h @@ -386,14 +386,14 @@ class TDataShard struct TEvPersistScanStateAck : public TEventLocal<TEvPersistScanStateAck, EvPersistScanStateAck> {}; struct TEvConditionalEraseRowsRegistered : public TEventLocal<TEvConditionalEraseRowsRegistered, EvConditionalEraseRowsRegistered> { - explicit TEvConditionalEraseRowsRegistered(ui64 txId, const TActorId& actorId) + explicit TEvConditionalEraseRowsRegistered(ui64 txId, const TActorId& actorId) : TxId(txId) , ActorId(actorId) { } const ui64 TxId; - const TActorId ActorId; + const TActorId ActorId; }; struct TEvAsyncJobComplete : public TEventLocal<TEvAsyncJobComplete, EvAsyncJobComplete> { @@ -440,9 +440,9 @@ class TDataShard struct Schema : NIceDb::Schema { struct Sys : Table<1> { - struct Id : Column<1, NScheme::NTypeIds::Uint64> {}; + struct Id : Column<1, NScheme::NTypeIds::Uint64> {}; struct Bytes : Column<2, NScheme::NTypeIds::String> {}; - struct Uint64 : Column<3, NScheme::NTypeIds::Uint64> {}; + struct Uint64 : Column<3, NScheme::NTypeIds::Uint64> {}; using TKey = TableKey<Id>; using TColumns = TableColumns<Id, Bytes, Uint64>; @@ -450,8 +450,8 @@ class TDataShard // Note that table UserTablesStats must be always updated with this one struct UserTables : Table<2> { - struct Tid : Column<1, NScheme::NTypeIds::Uint64> {}; - struct LocalTid : Column<2, NScheme::NTypeIds::Uint32> {}; + struct Tid : Column<1, NScheme::NTypeIds::Uint64> {}; + struct LocalTid : Column<2, NScheme::NTypeIds::Uint32> {}; struct Path : Column<3, NScheme::NTypeIds::String> {}; struct Name : Column<4, NScheme::NTypeIds::String> {}; struct Schema : Column<5, NScheme::NTypeIds::String> { using Type = TString; }; @@ -462,12 +462,12 @@ class TDataShard }; struct TxMain : Table<3> { - struct TxId : Column<1, NScheme::NTypeIds::Uint64> {}; + struct TxId : Column<1, NScheme::NTypeIds::Uint64> {}; struct Kind : Column<2, NScheme::NTypeIds::Uint32> { using Type = EOperationKind; }; - struct Flags : Column<3, NScheme::NTypeIds::Uint32> {}; - struct State : Column<4, NScheme::NTypeIds::Uint32> {}; - struct InRSRemain : Column<5, NScheme::NTypeIds::Uint64> {}; - struct MaxStep : Column<6, NScheme::NTypeIds::Uint64> {}; + struct Flags : Column<3, NScheme::NTypeIds::Uint32> {}; + struct State : Column<4, NScheme::NTypeIds::Uint32> {}; + struct InRSRemain : Column<5, NScheme::NTypeIds::Uint64> {}; + struct MaxStep : Column<6, NScheme::NTypeIds::Uint64> {}; struct ReceivedAt : Column<7, NScheme::NTypeIds::Uint64> {}; struct Flags64 : Column<8, NScheme::NTypeIds::Uint64> {}; struct Source : Column<9, NScheme::NTypeIds::ActorId> {}; @@ -478,21 +478,21 @@ class TDataShard }; struct TxDetails : Table<4> { - struct TxId : Column<1, NScheme::NTypeIds::Uint64> {}; - struct Origin : Column<2, NScheme::NTypeIds::Uint64> {}; - struct InReadSetState : Column<3, NScheme::NTypeIds::Uint64> {}; // Not used + struct TxId : Column<1, NScheme::NTypeIds::Uint64> {}; + struct Origin : Column<2, NScheme::NTypeIds::Uint64> {}; + struct InReadSetState : Column<3, NScheme::NTypeIds::Uint64> {}; // Not used struct Body : Column<4, NScheme::NTypeIds::String> { using Type = TString; }; - struct Source : Column<5, NScheme::NTypeIds::ActorId> {}; + struct Source : Column<5, NScheme::NTypeIds::ActorId> {}; using TKey = TableKey<TxId, Origin>; using TColumns = TableColumns<TxId, Origin, InReadSetState, Body, Source>; }; struct InReadSets : Table<5> { - struct TxId : Column<1, NScheme::NTypeIds::Uint64> {}; - struct Origin : Column<2, NScheme::NTypeIds::Uint64> {}; - struct From : Column<3, NScheme::NTypeIds::Uint64> {}; - struct To : Column<4, NScheme::NTypeIds::Uint64> {}; + struct TxId : Column<1, NScheme::NTypeIds::Uint64> {}; + struct Origin : Column<2, NScheme::NTypeIds::Uint64> {}; + struct From : Column<3, NScheme::NTypeIds::Uint64> {}; + struct To : Column<4, NScheme::NTypeIds::Uint64> {}; struct Body : Column<5, NScheme::NTypeIds::String> { using Type = TString; }; struct BalanceTrackList : Column<6, NScheme::NTypeIds::String> { using Type = TString; }; @@ -501,12 +501,12 @@ class TDataShard }; struct OutReadSets : Table<6> { - struct Seqno : Column<1, NScheme::NTypeIds::Uint64> {}; - struct Step : Column<2, NScheme::NTypeIds::Uint64> {}; - struct TxId : Column<3, NScheme::NTypeIds::Uint64> {}; - struct Origin : Column<4, NScheme::NTypeIds::Uint64> {}; - struct From : Column<5, NScheme::NTypeIds::Uint64> {}; - struct To : Column<6, NScheme::NTypeIds::Uint64> {}; + struct Seqno : Column<1, NScheme::NTypeIds::Uint64> {}; + struct Step : Column<2, NScheme::NTypeIds::Uint64> {}; + struct TxId : Column<3, NScheme::NTypeIds::Uint64> {}; + struct Origin : Column<4, NScheme::NTypeIds::Uint64> {}; + struct From : Column<5, NScheme::NTypeIds::Uint64> {}; + struct To : Column<6, NScheme::NTypeIds::Uint64> {}; struct Body : Column<7, NScheme::NTypeIds::String> { using Type = TString; }; struct SplitTraj : Column<8, NScheme::NTypeIds::String> { using Type = TString; }; @@ -515,16 +515,16 @@ class TDataShard }; struct PlanQueue : Table<7> { - struct Step : Column<1, NScheme::NTypeIds::Uint64> {}; - struct TxId : Column<2, NScheme::NTypeIds::Uint64> {}; + struct Step : Column<1, NScheme::NTypeIds::Uint64> {}; + struct TxId : Column<2, NScheme::NTypeIds::Uint64> {}; using TKey = TableKey<Step, TxId>; using TColumns = TableColumns<Step, TxId>; }; struct DeadlineQueue : Table<8> { - struct MaxStep : Column<1, NScheme::NTypeIds::Uint64> {}; - struct TxId : Column<2, NScheme::NTypeIds::Uint64> {}; + struct MaxStep : Column<1, NScheme::NTypeIds::Uint64> {}; + struct TxId : Column<2, NScheme::NTypeIds::Uint64> {}; using TKey = TableKey<MaxStep, TxId>; using TColumns = TableColumns<MaxStep, TxId>; @@ -1080,7 +1080,7 @@ public: void SendDelayedAcks(const TActorContext& ctx, TVector<THolder<IEventHandle>>& delayedAcks) const; void SendResult(const TActorContext &ctx, TOutputOpData::TResultPtr &result, - const TActorId &target, + const TActorId &target, ui64 step, ui64 txId); void FillSplitTrajectory(ui64 origin, NKikimrTx::TBalanceTrackList& tracks); @@ -1447,7 +1447,7 @@ private: /// class TLoanReturnTracker { struct TLoanReturnInfo { - TActorId PipeToOwner; + TActorId PipeToOwner; THashSet<TLogoBlobID> PartMeta; }; @@ -1545,7 +1545,7 @@ private: } } - bool Has(ui64 ownerTabletId, TActorId pipeClientActorId) const { + bool Has(ui64 ownerTabletId, TActorId pipeClientActorId) const { return LoanReturns.contains(ownerTabletId) && LoanReturns.FindPtr(ownerTabletId)->PipeToOwner == pipeClientActorId; } @@ -1624,7 +1624,7 @@ private: return !DataToSend.contains(dstTabletId); } - bool Has(ui64 dstTabletId, TActorId pipeClientActorId) const { + bool Has(ui64 dstTabletId, TActorId pipeClientActorId) const { return PipesToDstShards.contains(dstTabletId) && *PipesToDstShards.FindPtr(dstTabletId) == pipeClientActorId; } @@ -1638,7 +1638,7 @@ private: TDataShard* Self; THashSet<ui64> Dst; THashMap<ui64, TAutoPtr<NKikimrTxDataShard::TEvSplitTransferSnapshot>> DataToSend; - THashMap<ui64, TActorId> PipesToDstShards; + THashMap<ui64, TActorId> PipesToDstShards; }; /// @@ -1890,8 +1890,8 @@ private: THolder<NTabletPipe::IClientCache> PipeClientCache; TPipeTracker ResendReadSetPipeTracker; NTabletPipe::TClientRetryPolicy SchemeShardPipeRetryPolicy; - TActorId SchemeShardPipe; // For notifications about schema changes - TActorId StateReportPipe; // For notifications about shard state changes + TActorId SchemeShardPipe; // For notifications about schema changes + TActorId StateReportPipe; // For notifications about shard state changes ui64 PathOwnerId; // TabletID of the schmemeshard that allocated the TPathId(ownerId,localId) ui64 CurrentSchemeShardId; // TabletID of SchemeShard wich manages the path right now ui64 LastKnownMediator; @@ -1902,8 +1902,8 @@ private: TInstant LastDbStatsReportTime; TInstant LastCpuWarnTime; TInstant LastDataSizeWarnTime; - TActorId DbStatsReportPipe; - TActorId TableResolvePipe; + TActorId DbStatsReportPipe; + TActorId TableResolvePipe; ui64 StatsReportRound = 0; TActorId FindSubDomainPathIdActor; @@ -1912,7 +1912,7 @@ private: std::optional<TPathId> WatchingSubDomainPathId; bool SubDomainOutOfSpace = false; - THashSet<TActorId> Actors; + THashSet<TActorId> Actors; TLoanReturnTracker LoanReturnTracker; TFollowerState FollowerState; @@ -1926,7 +1926,7 @@ private: bool DstSplitSchemaInitialized = false; std::shared_ptr<NKikimrTxDataShard::TSplitMergeDescription> DstSplitDescription; std::shared_ptr<NKikimrTxDataShard::TSplitMergeDescription> SrcSplitDescription; - THashSet<TActorId> SrcAckSplitTo; + THashSet<TActorId> SrcAckSplitTo; THashMap<TActorId, THashSet<ui64>> SrcAckPartitioningChangedTo; const ui32 SysTablesToTransferAtSplit[4] = { Schema::TxMain::TableId, @@ -2002,7 +2002,7 @@ private: struct TInFlightCondErase { ui64 TxId; ui64 ScanId; - TActorId ActorId; + TActorId ActorId; ui32 Condition; TInFlightCondErase() { @@ -2012,7 +2012,7 @@ private: void Clear() { TxId = 0; ScanId = 0; - ActorId = TActorId(); + ActorId = TActorId(); Condition = 0; } @@ -2282,7 +2282,7 @@ protected: InvokeOtherActor(*ReplicationSourceOffsetsServer, &TReplicationSourceOffsetsServer::PassAway); } - for (const TActorId& actorId : Actors) { + for (const TActorId& actorId : Actors) { Send(actorId, new TEvents::TEvPoison); } Actors.clear(); diff --git a/ydb/core/tx/datashard/datashard_loans.cpp b/ydb/core/tx/datashard/datashard_loans.cpp index 4eb13d52e02..adbfc4769db 100644 --- a/ydb/core/tx/datashard/datashard_loans.cpp +++ b/ydb/core/tx/datashard/datashard_loans.cpp @@ -86,7 +86,7 @@ public: void Complete(const TActorContext &ctx) override { // Send Ack - TActorId ackTo = Ev->Sender; + TActorId ackTo = Ev->Sender; LOG_DEBUG_S(ctx, NKikimrServices::TX_DATASHARD, Self->TabletID() << " ack parts " << PartMetaVec << " return to tablet " << FromTabletId); ctx.Send(ackTo, new TEvDataShard::TEvReturnBorrowedPartAck(PartMetaVec), 0, Ev->Cookie); diff --git a/ydb/core/tx/datashard/datashard_pipeline.cpp b/ydb/core/tx/datashard/datashard_pipeline.cpp index 1fbdff50b5c..3498162c17e 100644 --- a/ydb/core/tx/datashard/datashard_pipeline.cpp +++ b/ydb/core/tx/datashard/datashard_pipeline.cpp @@ -520,7 +520,7 @@ bool TPipeline::LoadTxDetails(TTransactionContext &txc, << tx->GetStep() << ":" << tx->GetTxId()); } else { NIceDb::TNiceDb db(txc.DB); - TActorId target; + TActorId target; TString txBody; TVector<TSysTables::TLocksTable::TLock> locks; ui64 artifactFlags = 0; @@ -1097,7 +1097,7 @@ void TPipeline::SaveForPropose(TValidatedDataTx::TPtr tx) { } } -void TPipeline::SetProposed(ui64 txId, const TActorId& actorId) { +void TPipeline::SetProposed(ui64 txId, const TActorId& actorId) { auto it = DataTxCache.find(txId); if (it != DataTxCache.end()) { it->second->SetSource(actorId); diff --git a/ydb/core/tx/datashard/datashard_pipeline.h b/ydb/core/tx/datashard/datashard_pipeline.h index d1d36cf663c..67c35260b28 100644 --- a/ydb/core/tx/datashard/datashard_pipeline.h +++ b/ydb/core/tx/datashard/datashard_pipeline.h @@ -107,7 +107,7 @@ public: // tx propose void SaveForPropose(TValidatedDataTx::TPtr tx); - void SetProposed(ui64 txId, const TActorId& actorId); + void SetProposed(ui64 txId, const TActorId& actorId); void ForgetUnproposedTx(ui64 txId); void ForgetTx(ui64 txId); diff --git a/ydb/core/tx/datashard/datashard_split_dst.cpp b/ydb/core/tx/datashard/datashard_split_dst.cpp index d464731064d..65750810013 100644 --- a/ydb/core/tx/datashard/datashard_split_dst.cpp +++ b/ydb/core/tx/datashard/datashard_split_dst.cpp @@ -96,7 +96,7 @@ public: void Complete(const TActorContext &ctx) override { // Send Ack - TActorId ackTo = Ev->Sender; + TActorId ackTo = Ev->Sender; ui64 opId = Ev->Get()->Record.GetOperationCookie(); LOG_DEBUG_S(ctx, NKikimrServices::TX_DATASHARD, Self->TabletID() << " ack init split/merge destination OpId " << opId); @@ -279,7 +279,7 @@ public: } void Complete(const TActorContext &ctx) override { - TActorId ackTo = Ev->Sender; + TActorId ackTo = Ev->Sender; ui64 opId = Ev->Get()->Record.GetOperationCookie(); LOG_DEBUG_S(ctx, NKikimrServices::TX_DATASHARD, Self->TabletID() << " ack snapshot OpId " << opId); diff --git a/ydb/core/tx/datashard/datashard_split_src.cpp b/ydb/core/tx/datashard/datashard_split_src.cpp index a69047dab5e..a7349fc9cf3 100644 --- a/ydb/core/tx/datashard/datashard_split_src.cpp +++ b/ydb/core/tx/datashard/datashard_split_src.cpp @@ -82,7 +82,7 @@ public: void Complete(const TActorContext &ctx) override { if (SplitAlreadyFinished) { // Send the Ack - for (const TActorId& ackTo : Self->SrcAckSplitTo) { + for (const TActorId& ackTo : Self->SrcAckSplitTo) { LOG_DEBUG_S(ctx, NKikimrServices::TX_DATASHARD, Self->TabletID() << " ack split to schemeshard " << Self->SrcSplitOpId); ctx.Send(ackTo, new TEvDataShard::TEvSplitAck(Self->SrcSplitOpId, Self->TabletID())); } @@ -440,7 +440,7 @@ public: void Complete(const TActorContext &ctx) override { if (AllDstAcksReceived) { - for (const TActorId& ackTo : Self->SrcAckSplitTo) { + for (const TActorId& ackTo : Self->SrcAckSplitTo) { ui64 opId = Self->SrcSplitOpId; LOG_DEBUG_S(ctx, NKikimrServices::TX_DATASHARD, Self->TabletID() << " ack split to schemeshard " << opId); ctx.Send(ackTo, new TEvDataShard::TEvSplitAck(opId, Self->TabletID())); @@ -504,7 +504,7 @@ public: } void Complete(const TActorContext &ctx) override { - TActorId ackTo = Ev->Sender; + TActorId ackTo = Ev->Sender; ui64 opId = Ev->Get()->Record.GetOperationCookie(); if (DelayPartitioningChangedAck) { diff --git a/ydb/core/tx/datashard/datashard_trans_queue.cpp b/ydb/core/tx/datashard/datashard_trans_queue.cpp index 83545e9fbf3..d7096c25b32 100644 --- a/ydb/core/tx/datashard/datashard_trans_queue.cpp +++ b/ydb/core/tx/datashard/datashard_trans_queue.cpp @@ -124,7 +124,7 @@ bool TTransQueue::Load(NIceDb::TNiceDb& db) { return false; while (!rowset.EndOfSet()) { ui64 txId = rowset.GetValue<Schema::SchemaOperations::TxId>(); - TActorId source = rowset.GetValue<Schema::SchemaOperations::Source>(); + TActorId source = rowset.GetValue<Schema::SchemaOperations::Source>(); ui64 tabletId = rowset.GetValueOrDefault<Schema::SchemaOperations::SourceTablet>(0); ui32 opType = rowset.GetValue<Schema::SchemaOperations::Operation>(); ui64 minStep = rowset.GetValue<Schema::SchemaOperations::MinStep>(); @@ -219,7 +219,7 @@ void TTransQueue::ProposeSchemaTx(NIceDb::TNiceDb& db, const TSchemaOperation& o db.NoMoreReadsForTx(); } -void TTransQueue::ProposeTx(NIceDb::TNiceDb& db, TOperation::TPtr op, TActorId source, const TStringBuf& txBody) { +void TTransQueue::ProposeTx(NIceDb::TNiceDb& db, TOperation::TPtr op, TActorId source, const TStringBuf& txBody) { using Schema = TDataShard::Schema; const ui64 preserveFlagsMask = TTxFlags::PublicFlagsMask | TTxFlags::PreservedPrivateFlagsMask; @@ -315,7 +315,7 @@ bool TTransQueue::GetNextPlannedTxId(ui64& step, ui64& txId) const { bool TTransQueue::LoadTxDetails(NIceDb::TNiceDb &db, ui64 txId, - TActorId &target, + TActorId &target, TString &txBody, TVector<TSysTables::TLocksTable::TLock> &locks, ui64 &artifactFlags) { diff --git a/ydb/core/tx/datashard/datashard_trans_queue.h b/ydb/core/tx/datashard/datashard_trans_queue.h index ebc1afa1fc5..1d392676da3 100644 --- a/ydb/core/tx/datashard/datashard_trans_queue.h +++ b/ydb/core/tx/datashard/datashard_trans_queue.h @@ -81,7 +81,7 @@ private: // for pipeline only // Propose - void ProposeTx(NIceDb::TNiceDb& db, TOperation::TPtr op, TActorId source, const TStringBuf& txBody); + void ProposeTx(NIceDb::TNiceDb& db, TOperation::TPtr op, TActorId source, const TStringBuf& txBody); void UpdateTxFlags(NIceDb::TNiceDb& db, ui64 txId, ui64 flags); void UpdateTxBody(NIceDb::TNiceDb& db, ui64 txId, const TStringBuf& txBody); void ProposeSchemaTx(NIceDb::TNiceDb& db, const TSchemaOperation& op); @@ -102,7 +102,7 @@ private: // for pipeline only bool GetNextPlannedTxId(ui64& step, ui64& txId) const; bool LoadTxDetails(NIceDb::TNiceDb &db, ui64 txId, - TActorId &targets, + TActorId &targets, TString &txBody, TVector<TSysTables::TLocksTable::TLock> &locks, ui64 &artifactFlags); diff --git a/ydb/core/tx/datashard/datashard_txs.h b/ydb/core/tx/datashard/datashard_txs.h index 145b117f27e..4a9ab008789 100644 --- a/ydb/core/tx/datashard/datashard_txs.h +++ b/ydb/core/tx/datashard/datashard_txs.h @@ -61,7 +61,7 @@ private: TEvTxProcessing::TEvPlanStep::TPtr Ev; bool IsAccepted; TInstant RequestStartTime; - TMap<TActorId, TVector<ui64>> TxByAck; + TMap<TActorId, TVector<ui64>> TxByAck; }; class TDataShard::TTxProgressTransaction : public NTabletFlatExecutor::TTransactionBase<TDataShard> { @@ -174,7 +174,7 @@ public: TTxType GetTxType() const override { return TXTYPE_STORE_SCAN_STATE; } private: TEvPrivate::TEvPersistScanState::TPtr Ev; - TActorId Sender; + TActorId Sender; }; class TDataShard::TTxRefreshVolatileSnapshot : public NTabletFlatExecutor::TTransactionBase<TDataShard> { diff --git a/ydb/core/tx/datashard/datashard_ut_build_index.cpp b/ydb/core/tx/datashard/datashard_ut_build_index.cpp index 2069104338c..2b9e3f0a3fc 100644 --- a/ydb/core/tx/datashard/datashard_ut_build_index.cpp +++ b/ydb/core/tx/datashard/datashard_ut_build_index.cpp @@ -27,7 +27,7 @@ using namespace Tests; Y_UNIT_TEST_SUITE(TTxDataShardBuildIndexScan) { - static void DoBuildIndex(Tests::TServer::TPtr server, TActorId sender, + static void DoBuildIndex(Tests::TServer::TPtr server, TActorId sender, const TString& tableFrom, const TString& tableTo, const TRowVersion& snapshot, const NKikimrTxDataShard::TEvBuildIndexProgressResponse::EStatus& expected) { @@ -78,7 +78,7 @@ Y_UNIT_TEST_SUITE(TTxDataShardBuildIndexScan) { } static void CreateShardedTableForIndex( - Tests::TServer::TPtr server, TActorId sender, + Tests::TServer::TPtr server, TActorId sender, const TString &root, const TString &name, ui64 shards, bool enableOutOfOrder) { diff --git a/ydb/core/tx/datashard/datashard_ut_common.cpp b/ydb/core/tx/datashard/datashard_ut_common.cpp index 87b49fa96d0..d754f7ad0e2 100644 --- a/ydb/core/tx/datashard/datashard_ut_common.cpp +++ b/ydb/core/tx/datashard/datashard_ut_common.cpp @@ -203,7 +203,7 @@ void TTester::CreateSchema(ESchema schema, const TOptions& opts) { "Name: \"table1\"\n" "Id_Deprecated: 13\n" "Path: \"/Root/table1\"\n" - "Columns { Id: 34 Name: \"key\" TypeId: " + ToString<int>(NScheme::NTypeIds::Uint32) + " }\n" + "Columns { Id: 34 Name: \"key\" TypeId: " + ToString<int>(NScheme::NTypeIds::Uint32) + " }\n" "Columns { Id: 56 Name: \"value\" TypeId: " + ToString<int>(NScheme::NTypeIds::Utf8) + " }\n" "Columns { Id: 57 Name: \"uint\" TypeId: " + ToString<int>(NScheme::NTypeIds::Uint32) + " }\n" "KeyColumnIds: [ 34 ]\n" @@ -214,7 +214,7 @@ void TTester::CreateSchema(ESchema schema, const TOptions& opts) { "PathId { OwnerId: " + ToString(FAKE_SCHEMESHARD_TABLET_ID) + " LocalId: 14 }\n" "Path: \"/Root/table2\"\n" - "Columns { Id: 34 Name: \"key1\" TypeId: " + ToString<int>(NScheme::NTypeIds::Uint32) + " }\n" + "Columns { Id: 34 Name: \"key1\" TypeId: " + ToString<int>(NScheme::NTypeIds::Uint32) + " }\n" "Columns { Id: 35 Name: \"key2\" TypeId: " + ToString<int>(NScheme::NTypeIds::Utf8) + " }\n" "Columns { Id: 56 Name: \"value\" TypeId: " + ToString<int>(NScheme::NTypeIds::Utf8) + " }\n" "KeyColumnIds: [ 34, 35 ]\n" @@ -422,7 +422,7 @@ NKikimrMiniKQL::TResult TFakeProxyTx::GetResult() const { ui32 TFakeScanTx::SetProgram(TTester& tester) { NKikimrTxDataShard::TDataTransaction dataTransaction; Y_PROTOBUF_SUPPRESS_NODISCARD dataTransaction.ParseFromArray(TxBody_.data(), TxBody_.size()); - ActorIdToProto(tester.Sender, dataTransaction.MutableSink()); + ActorIdToProto(tester.Sender, dataTransaction.MutableSink()); TxBody_ = dataTransaction.SerializeAsString(); const char * rangePattern = R"( @@ -728,7 +728,7 @@ ui64 TFakeMiniKQLProxy::Plan(ui64 stepId, const TMap<ui64, TFakeProxyTx::TPtr>& if (!tx->Immediate()) { auto plannedTx = planStep->Record.MutableTransactions()->Add(); plannedTx->SetTxId(txId); - ActorIdToProto(Tester.Sender, plannedTx->MutableAckTo()); + ActorIdToProto(Tester.Sender, plannedTx->MutableAckTo()); } } Tester.Runtime.SendToPipe(shard, Tester.Sender, planStep); @@ -924,7 +924,7 @@ ui64 TFakeMiniKQLProxy::Plan(ui64 stepId, const TMap<ui64, TFakeProxyTx::TPtr>& results.insert(std::make_pair(shard, txId)); auto plannedTx = planStep->Record.MutableTransactions()->Add(); plannedTx->SetTxId(txId); - ActorIdToProto(Tester.Sender, plannedTx->MutableAckTo()); + ActorIdToProto(Tester.Sender, plannedTx->MutableAckTo()); } Tester.Runtime.SendToPipe(shard, Tester.Sender, planStep); } @@ -1043,7 +1043,7 @@ THolder<NKqp::TEvKqp::TEvQueryRequest> MakeSQLRequest(const TString &sql, } void InitRoot(Tests::TServer::TPtr server, - TActorId sender) + TActorId sender) { if (server->GetSettings().StoragePoolTypes.empty()) { return; @@ -1088,7 +1088,7 @@ void InitRoot(Tests::TServer::TPtr server, } void CreateShardedTable(Tests::TServer::TPtr server, - TActorId sender, + TActorId sender, const TString &root, const TString &name, const TShardedTableOptions &opts) @@ -1183,7 +1183,7 @@ void CreateShardedTable(Tests::TServer::TPtr server, } void CreateShardedTable(Tests::TServer::TPtr server, - TActorId sender, + TActorId sender, const TString &root, const TString &name, ui64 shards, @@ -1253,7 +1253,7 @@ std::pair<TTableInfoMap, ui64> GetTables( TTableId ResolveTableId( Tests::TServer::TPtr server, - TActorId sender, + TActorId sender, const TString& path) { auto& runtime = *server->GetRuntime(); @@ -1298,7 +1298,7 @@ TRowVersion CreateVolatileSnapshot( { auto& runtime = *server->GetRuntime(); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); { auto request = MakeHolder<TEvTxUserProxy::TEvProposeTransaction>(); @@ -1331,7 +1331,7 @@ bool RefreshVolatileSnapshot( { auto& runtime = *server->GetRuntime(); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); { auto request = MakeHolder<TEvTxUserProxy::TEvProposeTransaction>(); @@ -1357,7 +1357,7 @@ bool DiscardVolatileSnapshot( { auto& runtime = *server->GetRuntime(); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); { auto request = MakeHolder<TEvTxUserProxy::TEvProposeTransaction>(); @@ -1445,7 +1445,7 @@ TRowVersion CommitWrites( ui64 AsyncSplitTable( Tests::TServer::TPtr server, - TActorId sender, + TActorId sender, const TString& path, ui64 sourceTablet, ui32 splitKey) @@ -1470,7 +1470,7 @@ ui64 AsyncSplitTable( ui64 AsyncMergeTable( Tests::TServer::TPtr server, - TActorId sender, + TActorId sender, const TString& path, const TVector<ui64>& sourceTabletIds) { @@ -1688,7 +1688,7 @@ ui64 AsyncAlterDropStream( return ev->Get()->Record.GetTxId(); } -void WaitTxNotification(Tests::TServer::TPtr server, TActorId sender, ui64 txId) { +void WaitTxNotification(Tests::TServer::TPtr server, TActorId sender, ui64 txId) { auto &runtime = *server->GetRuntime(); auto &settings = server->GetSettings(); @@ -1713,7 +1713,7 @@ void SimulateSleep(Tests::TServer::TPtr server, TDuration duration) { } void SendSQL(Tests::TServer::TPtr server, - TActorId sender, + TActorId sender, const TString &sql, bool dml) { @@ -1723,7 +1723,7 @@ void SendSQL(Tests::TServer::TPtr server, } void ExecSQL(Tests::TServer::TPtr server, - TActorId sender, + TActorId sender, const TString &sql, bool dml, Ydb::StatusIds::StatusCode code) @@ -1954,13 +1954,13 @@ namespace { private: struct TPendingRequest { - TActorId Sender; + TActorId Sender; ui64 Cookie; ui64 TxId; }; private: - const TActorId Owner; + const TActorId Owner; const TString Path; const TRowVersion Snapshot; const bool Ordered; @@ -1995,7 +1995,7 @@ void ResumeReadShardedTable( TReadShardedTableState& state) { auto& runtime = *server->GetRuntime(); - runtime.Send(new IEventHandle(state.Worker, TActorId(), new TReadTableImpl::TEvResume()), 0, true); + runtime.Send(new IEventHandle(state.Worker, TActorId(), new TReadTableImpl::TEvResume()), 0, true); auto ev = runtime.GrabEdgeEventRethrow<TReadTableImpl::TEvResult>(state.Sender); state.Result = ev->Get()->Result; } diff --git a/ydb/core/tx/datashard/datashard_ut_common.h b/ydb/core/tx/datashard/datashard_ut_common.h index 1ed2b6518b5..4b5d7fe4ea0 100644 --- a/ydb/core/tx/datashard/datashard_ut_common.h +++ b/ydb/core/tx/datashard/datashard_ut_common.h @@ -102,7 +102,7 @@ public: private: ESchema Schema; - TActorId Sender; + TActorId Sender; ui64 LastTxId; ui64 LastStep; TMockDbSchemeResolver DbSchemeResolver; @@ -307,7 +307,7 @@ private: ui64& LastTxId_; ui64& LastStep_; TVector<TFakeProxyTx::TPtr> TxQueue; - TMap<ui64, TActorId> ShardActors; + TMap<ui64, TActorId> ShardActors; // TVector<TExpectedReadSet> DelayedReadSets; TVector<TExpectedReadSet::TWaitFor> DelayedData; @@ -360,7 +360,7 @@ THolder<NKqp::TEvKqp::TEvQueryRequest> MakeSQLRequest(const TString &sql, bool dml = true); void InitRoot(Tests::TServer::TPtr server, - TActorId sender); + TActorId sender); enum class EShadowDataMode { Default, @@ -452,13 +452,13 @@ struct TShardedTableOptions { void N(NUnitTest::TTestContext&) void CreateShardedTable(Tests::TServer::TPtr server, - TActorId sender, + TActorId sender, const TString &root, const TString &name, const TShardedTableOptions &opts = TShardedTableOptions()); void CreateShardedTable(Tests::TServer::TPtr server, - TActorId sender, + TActorId sender, const TString &root, const TString &name, ui64 shards, @@ -467,7 +467,7 @@ void CreateShardedTable(Tests::TServer::TPtr server, EShadowDataMode shadowData = EShadowDataMode::Default); TVector<ui64> GetTableShards(Tests::TServer::TPtr server, - TActorId sender, + TActorId sender, const TString &path); using TTableInfoMap = THashMap<TString, NKikimrTxDataShard::TEvGetInfoResponse::TUserTable>; @@ -477,7 +477,7 @@ std::pair<TTableInfoMap, ui64> GetTables(Tests::TServer::TPtr server, TTableId ResolveTableId( Tests::TServer::TPtr server, - TActorId sender, + TActorId sender, const TString& path); NTable::TRowVersionRanges GetRemovedRowVersions( @@ -522,14 +522,14 @@ TRowVersion CommitWrites( ui64 AsyncSplitTable( Tests::TServer::TPtr server, - TActorId sender, + TActorId sender, const TString& path, ui64 sourceTablet, ui32 splitKey); ui64 AsyncMergeTable( Tests::TServer::TPtr server, - TActorId sender, + TActorId sender, const TString& path, const TVector<ui64>& sourceTabletIds); @@ -575,8 +575,8 @@ ui64 AsyncAlterDropStream( const TString& streamName); struct TReadShardedTableState { - TActorId Sender; - TActorId Worker; + TActorId Sender; + TActorId Worker; TString Result; }; @@ -596,17 +596,17 @@ TString ReadShardedTable( const TString& path, TRowVersion snapshot = TRowVersion::Max()); -void WaitTxNotification(Tests::TServer::TPtr server, TActorId sender, ui64 txId); +void WaitTxNotification(Tests::TServer::TPtr server, TActorId sender, ui64 txId); void WaitTxNotification(Tests::TServer::TPtr server, ui64 txId); void SimulateSleep(Tests::TServer::TPtr server, TDuration duration); void SendSQL(Tests::TServer::TPtr server, - TActorId sender, + TActorId sender, const TString &sql, bool dml = true); void ExecSQL(Tests::TServer::TPtr server, - TActorId sender, + TActorId sender, const TString &sql, bool dml = true, Ydb::StatusIds::StatusCode code = Ydb::StatusIds::SUCCESS); diff --git a/ydb/core/tx/datashard/datashard_ut_common_kqp.h b/ydb/core/tx/datashard/datashard_ut_common_kqp.h index 2c50428ad20..b8a78736526 100644 --- a/ydb/core/tx/datashard/datashard_ut_common_kqp.h +++ b/ydb/core/tx/datashard/datashard_ut_common_kqp.h @@ -94,7 +94,7 @@ namespace NKqpHelpers { inline void SendRequest( TTestActorRuntime& runtime, - TActorId sender, + TActorId sender, THolder<NKqp::TEvKqp::TEvQueryRequest> request) { runtime.Send( @@ -104,7 +104,7 @@ namespace NKqpHelpers { inline NKqp::TEvKqp::TEvQueryResponse::TPtr ExecRequest( TTestActorRuntime& runtime, - TActorId sender, + TActorId sender, THolder<NKqp::TEvKqp::TEvQueryRequest> request) { SendRequest(runtime, sender, std::move(request)); @@ -125,7 +125,7 @@ namespace NKqpHelpers { } inline THolder<NKqp::TEvKqp::TEvQueryRequest> MakeStreamRequest( - const TActorId sender, + const TActorId sender, const TString& sql, const bool collectStats = false) { @@ -135,7 +135,7 @@ namespace NKqpHelpers { request->Record.MutableRequest()->SetKeepSession(false); request->Record.MutableRequest()->SetQuery(sql); request->Record.MutableRequest()->SetProfile(collectStats); - ActorIdToProto(sender, request->Record.MutableRequestActorId()); + ActorIdToProto(sender, request->Record.MutableRequestActorId()); return request; } diff --git a/ydb/core/tx/datashard/datashard_ut_erase_rows.cpp b/ydb/core/tx/datashard/datashard_ut_erase_rows.cpp index 3defe26d2f8..87a51ef80ab 100644 --- a/ydb/core/tx/datashard/datashard_ut_erase_rows.cpp +++ b/ydb/core/tx/datashard/datashard_ut_erase_rows.cpp @@ -143,7 +143,7 @@ protected: virtual IEventBase* MakeRequest() const = 0; public: - explicit TRequestRunner(const TActorId& replyTo, ui64 tabletID) + explicit TRequestRunner(const TActorId& replyTo, ui64 tabletID) : ReplyTo(replyTo) , TabletID(tabletID) { @@ -171,14 +171,14 @@ public: using TBase = TRequestRunner<TEvResponse, TDerived>; private: - const TActorId ReplyTo; + const TActorId ReplyTo; const ui64 TabletID; - TActorId Pipe; + TActorId Pipe; }; void EraseRows( - TServer::TPtr server, const TActorId& sender, const TString& path, + TServer::TPtr server, const TActorId& sender, const TString& path, const TTableId& tableId, TVector<ui32> keyTags, TVector<TString> keys, ui32 status = TProto::TEvEraseResponse::OK, const TString& error = "") { using TEvRequest = TEvDataShard::TEvEraseRowsRequest; @@ -220,7 +220,7 @@ void EraseRows( } void ConditionalEraseRows( - TServer::TPtr server, const TActorId& sender, const TString& path, + TServer::TPtr server, const TActorId& sender, const TString& path, const TTableId& tableId, ui32 columnId, ui64 threshold, EUnit unit = TUnit::AUTO, const NDataShard::TIndexes& indexes = {}, const TProto::TLimits& limits = {}, ui32 status = TProto::TEvCondEraseResponse::ACCEPTED, const TString& error = "") { @@ -365,7 +365,7 @@ Y_UNIT_TEST_SUITE(EraseRowsTests) { TServer::TPtr server = new TServer(serverSettings); auto& runtime = *server->GetRuntime(); - const TActorId sender = runtime.AllocateEdgeActor(); + const TActorId sender = runtime.AllocateEdgeActor(); runtime.SetLogPriority(NKikimrServices::TX_DATASHARD, NLog::PRI_DEBUG); InitRoot(server, sender); @@ -400,7 +400,7 @@ Y_UNIT_TEST_SUITE(EraseRowsTests) { TServer::TPtr server = new TServer(serverSettings); auto& runtime = *server->GetRuntime(); - const TActorId sender = runtime.AllocateEdgeActor(); + const TActorId sender = runtime.AllocateEdgeActor(); runtime.SetLogPriority(NKikimrServices::TX_DATASHARD, NLog::PRI_DEBUG); InitRoot(server, sender); @@ -439,7 +439,7 @@ Y_UNIT_TEST_SUITE(EraseRowsTests) { TServer::TPtr server = new TServer(serverSettings); auto& runtime = *server->GetRuntime(); - const TActorId sender = runtime.AllocateEdgeActor(); + const TActorId sender = runtime.AllocateEdgeActor(); runtime.SetLogPriority(NKikimrServices::TX_DATASHARD, NLog::PRI_DEBUG); InitRoot(server, sender); @@ -627,7 +627,7 @@ key = 4, value = (empty maybe) TServer::TPtr server = new TServer(serverSettings); auto& runtime = *server->GetRuntime(); - const TActorId sender = runtime.AllocateEdgeActor(); + const TActorId sender = runtime.AllocateEdgeActor(); runtime.SetLogPriority(NKikimrServices::TX_DATASHARD, NLog::PRI_DEBUG); InitRoot(server, sender); @@ -673,7 +673,7 @@ key = 4, value = (empty maybe) TServer::TPtr server = new TServer(serverSettings); auto& runtime = *server->GetRuntime(); - const TActorId sender = runtime.AllocateEdgeActor(); + const TActorId sender = runtime.AllocateEdgeActor(); runtime.SetLogPriority(NKikimrServices::TX_DATASHARD, NLog::PRI_DEBUG); InitRoot(server, sender); diff --git a/ydb/core/tx/datashard/datashard_ut_init.cpp b/ydb/core/tx/datashard/datashard_ut_init.cpp index 4003a005fa2..818e30ab0cb 100644 --- a/ydb/core/tx/datashard/datashard_ut_init.cpp +++ b/ydb/core/tx/datashard/datashard_ut_init.cpp @@ -18,7 +18,7 @@ using NClient::TValue; namespace { TString GetTablePath(TTestActorRuntime &runtime, - TActorId sender, + TActorId sender, ui64 tableId, ui64 shard) { @@ -98,7 +98,7 @@ Y_UNIT_TEST_SUITE(TTxDataShardTestInit) { TTestBasicRuntime runtime; TTester::Setup(runtime); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); CreateTestBootstrapper(runtime, CreateTestTabletInfo(TTestTxConfig::TxTablet0, TTabletTypes::FLAT_DATASHARD), &CreateDataShard); TDispatchOptions options; diff --git a/ydb/core/tx/datashard/datashard_ut_kqp.cpp b/ydb/core/tx/datashard/datashard_ut_kqp.cpp index 4347363fdfc..d3145951a4e 100644 --- a/ydb/core/tx/datashard/datashard_ut_kqp.cpp +++ b/ydb/core/tx/datashard/datashard_ut_kqp.cpp @@ -127,7 +127,7 @@ private: if (Reason) { auto evUndelivered = new TEvents::TEvUndelivered(event->GetTypeRewrite(), *Reason); - auto handle = new IEventHandle(event->Sender, TActorId(), evUndelivered, 0, event->Cookie); + auto handle = new IEventHandle(event->Sender, TActorId(), evUndelivered, 0, event->Cookie); rt.Send(handle, 0, true); ActionDone = true; } else { diff --git a/ydb/core/tx/datashard/datashard_ut_locks.cpp b/ydb/core/tx/datashard/datashard_ut_locks.cpp index b1f7f6f3f61..1d25c39f99d 100644 --- a/ydb/core/tx/datashard/datashard_ut_locks.cpp +++ b/ydb/core/tx/datashard/datashard_ut_locks.cpp @@ -698,7 +698,7 @@ void CheckLocksCacheUsage(bool waitForLocksStore) { return false; } - TActorId Recipient; + TActorId Recipient; }; TDispatchOptions options; diff --git a/ydb/core/tx/datashard/datashard_ut_minikql.cpp b/ydb/core/tx/datashard/datashard_ut_minikql.cpp index 05339b51278..7974e474861 100644 --- a/ydb/core/tx/datashard/datashard_ut_minikql.cpp +++ b/ydb/core/tx/datashard/datashard_ut_minikql.cpp @@ -722,7 +722,7 @@ Y_UNIT_TEST(WriteAndReadMultipleShards) { void GetTableStats(TTestActorRuntime &runtime, ui64 tabletId, ui64 tableId, NKikimrTableStats::TTableStats& stats) { TAutoPtr<TEvDataShard::TEvGetTableStats> ev(new TEvDataShard::TEvGetTableStats(tableId)); - TActorId edge = runtime.AllocateEdgeActor(); + TActorId edge = runtime.AllocateEdgeActor(); runtime.SendToPipe(tabletId, edge, ev.Release()); TAutoPtr<IEventHandle> handle; TEvDataShard::TEvGetTableStatsResult* response = runtime.GrabEdgeEventRethrow<TEvDataShard::TEvGetTableStatsResult>(handle); diff --git a/ydb/core/tx/datashard/datashard_ut_minstep.cpp b/ydb/core/tx/datashard/datashard_ut_minstep.cpp index d2491385ade..99d72b2e457 100644 --- a/ydb/core/tx/datashard/datashard_ut_minstep.cpp +++ b/ydb/core/tx/datashard/datashard_ut_minstep.cpp @@ -356,7 +356,7 @@ Y_UNIT_TEST_SUITE(TDataShardMinStepTest) { { auto event = new TEvDataShard::TEvProposeTransaction; event->Record = proposeRecord; - ActorIdToProto(sender, event->Record.MutableSource()); + ActorIdToProto(sender, event->Record.MutableSource()); runtime.Send(new IEventHandle(shardActorId, sender, event), 0, true); } diff --git a/ydb/core/tx/datashard/datashard_ut_order.cpp b/ydb/core/tx/datashard/datashard_ut_order.cpp index 9979ccf39ec..32472dce59e 100644 --- a/ydb/core/tx/datashard/datashard_ut_order.cpp +++ b/ydb/core/tx/datashard/datashard_ut_order.cpp @@ -2522,7 +2522,7 @@ Y_UNIT_TEST_QUAD(TestOutOfOrderNoBarrierRestartImmediateLongTail, UseMvcc, UseNe namespace { ui64 AsyncCreateCopyTable( Tests::TServer::TPtr server, - TActorId sender, + TActorId sender, const TString &root, const TString &name, const TString &from) @@ -2815,9 +2815,9 @@ Y_UNIT_TEST_NEW_ENGINE(TestPlannedCancelSplit) { captureTxProposeResult = false; // Remember which senders claim to be which shards - TVector<TActorId> actors(2); + TVector<TActorId> actors(2); for (auto& event : txProposeResults) { - TActorId actor = event->Sender; + TActorId actor = event->Sender; const auto* msg = event->Get<TEvDataShard::TEvProposeTransactionResult>(); ui64 shard = msg->Record.GetOrigin(); for (size_t i = 0; i < tablets.size(); ++i) { @@ -2860,7 +2860,7 @@ Y_UNIT_TEST_NEW_ENGINE(TestPlannedCancelSplit) { if (event && event->GetRecipientRewrite() == actors[1]) { Cerr << "---- found propose for table-2 ----" << Endl; const auto* msg = event->Get<TEvDataShard::TEvProposeTransaction>(); - TActorId target = msg->GetSource(); + TActorId target = msg->GetSource(); auto* result = new TEvDataShard::TEvProposeTransactionResult( msg->GetTxKind(), tablets[1], @@ -3182,7 +3182,7 @@ namespace { void AsyncReadTable( Tests::TServer::TPtr server, - TActorId sender, + TActorId sender, const TString& path) { auto &runtime = *server->GetRuntime(); @@ -3496,7 +3496,7 @@ Y_UNIT_TEST_NEW_ENGINE(TestImmediateQueueThenSplit) { bool captureSplit = true; bool captureSplitChanged = true; bool capturePropose = true; - THashSet<TActorId> captureDelayedProposeFrom; + THashSet<TActorId> captureDelayedProposeFrom; TVector<THolder<IEventHandle>> eventsSplit; TVector<THolder<IEventHandle>> eventsSplitChanged; TVector<THolder<IEventHandle>> eventsPropose; @@ -3540,7 +3540,7 @@ Y_UNIT_TEST_NEW_ENGINE(TestImmediateQueueThenSplit) { auto prevObserverFunc = runtime.SetObserverFunc(captureEvents); const int totalWrites = 10; - TVector<TActorId> writeSenders; + TVector<TActorId> writeSenders; // Send a lot of write requests in parallel (so there's a large propose queue) for (int i = 0; i < totalWrites; ++i) { diff --git a/ydb/core/tx/datashard/datashard_ut_rs.cpp b/ydb/core/tx/datashard/datashard_ut_rs.cpp index b3daf821c71..edbe2968da2 100644 --- a/ydb/core/tx/datashard/datashard_ut_rs.cpp +++ b/ydb/core/tx/datashard/datashard_ut_rs.cpp @@ -13,7 +13,7 @@ namespace NKikimr { using namespace NSchemeShard; using namespace Tests; -ui64 GetRSCount(TTestActorRuntime &runtime, TActorId sender, ui64 shard) +ui64 GetRSCount(TTestActorRuntime &runtime, TActorId sender, ui64 shard) { auto request = MakeHolder<TEvTablet::TEvLocalMKQL>(); TString miniKQL = R"___(( diff --git a/ydb/core/tx/datashard/export_s3_base_uploader.h b/ydb/core/tx/datashard/export_s3_base_uploader.h index 897507a8f7b..8c72a8c3936 100644 --- a/ydb/core/tx/datashard/export_s3_base_uploader.h +++ b/ydb/core/tx/datashard/export_s3_base_uploader.h @@ -452,7 +452,7 @@ private: bool MultiPart; bool Last; - TActorId Scanner; + TActorId Scanner; TString Buffer; TMaybe<TString> UploadId; diff --git a/ydb/core/tx/datashard/finish_propose_unit.cpp b/ydb/core/tx/datashard/finish_propose_unit.cpp index 69cc032e5f3..de0e9e4ad2e 100644 --- a/ydb/core/tx/datashard/finish_propose_unit.cpp +++ b/ydb/core/tx/datashard/finish_propose_unit.cpp @@ -166,7 +166,7 @@ void TFinishProposeUnit::CompleteRequest(TOperation::TPtr op, void TFinishProposeUnit::AddDiagnosticsResult(TOutputOpData::TResultPtr &res) { auto &tabletInfo = *res->Record.MutableTabletInfo(); - ActorIdToProto(DataShard.SelfId(), tabletInfo.MutableActorId()); + ActorIdToProto(DataShard.SelfId(), tabletInfo.MutableActorId()); tabletInfo.SetTabletId(DataShard.TabletID()); tabletInfo.SetGeneration(DataShard.Generation()); diff --git a/ydb/core/tx/datashard/operation.h b/ydb/core/tx/datashard/operation.h index 9dc406c0607..0f4b713e91c 100644 --- a/ydb/core/tx/datashard/operation.h +++ b/ydb/core/tx/datashard/operation.h @@ -508,8 +508,8 @@ public: { } - TActorId GetTarget() const { return Target; } - void SetTarget(TActorId target) { Target = target; } + TActorId GetTarget() const { return Target; } + void SetTarget(TActorId target) { Target = target; } ui64 GetCookie() const { return Cookie; } void SetCookie(ui64 cookie) { Cookie = cookie; } @@ -793,7 +793,7 @@ protected: } void ClearInputData() { InputData = nullptr; } - TActorId Target; + TActorId Target; private: THPTimer TotalTimer; diff --git a/ydb/core/tx/datashard/read_table_scan.cpp b/ydb/core/tx/datashard/read_table_scan.cpp index 75516032bc6..63e94fde967 100644 --- a/ydb/core/tx/datashard/read_table_scan.cpp +++ b/ydb/core/tx/datashard/read_table_scan.cpp @@ -271,8 +271,8 @@ public: } TReadTableScan(ui64 txId, ui64 shardId, TUserTable::TCPtr tableInfo, - const NKikimrTxDataShard::TReadTableTransaction &tx, TActorId sink, - TActorId dataShard) + const NKikimrTxDataShard::TReadTableTransaction &tx, TActorId sink, + TActorId dataShard) : TActor(&TThis::StateWork) , Sink(sink) , DataShard(dataShard) @@ -594,8 +594,8 @@ private: private: TVector<ui32> Tags; - TActorId Sink; - TActorId DataShard; + TActorId Sink; + TActorId DataShard; ui64 TxId; ui64 ShardId; IDriver *Driver; @@ -618,8 +618,8 @@ TAutoPtr<NTable::IScan> CreateReadTableScan(ui64 txId, ui64 shardId, TUserTable::TCPtr tableInfo, const NKikimrTxDataShard::TReadTableTransaction &tx, - TActorId sink, - TActorId dataShard) + TActorId sink, + TActorId dataShard) { return new TReadTableScan(txId, shardId, tableInfo, tx, sink, dataShard); } diff --git a/ydb/core/tx/datashard/read_table_scan.h b/ydb/core/tx/datashard/read_table_scan.h index f5a92352156..0650c6ef67a 100644 --- a/ydb/core/tx/datashard/read_table_scan.h +++ b/ydb/core/tx/datashard/read_table_scan.h @@ -21,8 +21,8 @@ TAutoPtr<NTable::IScan> CreateReadTableScan(ui64 txId, ui64 shardId, TUserTable::TCPtr tableInfo, const NKikimrTxDataShard::TReadTableTransaction &tx, - TActorId sink, - TActorId dataShard); + TActorId sink, + TActorId dataShard); } // namespace NDataShard } // namespace NKikimr diff --git a/ydb/core/tx/mediator/execute_queue.cpp b/ydb/core/tx/mediator/execute_queue.cpp index d4dc4f955c3..07740e9f826 100644 --- a/ydb/core/tx/mediator/execute_queue.cpp +++ b/ydb/core/tx/mediator/execute_queue.cpp @@ -13,10 +13,10 @@ namespace NTxMediator { class TTxMediatorExecQueue : public TActor<TTxMediatorExecQueue> { struct TBucket { - TActorId ActiveActor; + TActorId ActiveActor; }; - const TActorId Owner; + const TActorId Owner; const ui64 MediatorId; const ui64 HashRange; @@ -124,7 +124,7 @@ namespace NTxMediator { void Handle(TEvTxMediator::TEvRequestLostAcks::TPtr &ev, const TActorContext &ctx) { TEvTxMediator::TEvRequestLostAcks *msg = ev->Get(); TCoordinatorStep *step = msg->CoordinatorStep.Get(); - const TActorId &ackTo = msg->AckTo; + const TActorId &ackTo = msg->AckTo; LOG_DEBUG_S(ctx, NKikimrServices::TX_MEDIATOR_EXEC_QUEUE, "Actor# " << ctx.SelfID.ToString() << " MediatorId# " << MediatorId << " HANDLE TEvRequestLostAcks " << step->ToString() << " AckTo# " << ackTo.ToString()); @@ -153,7 +153,7 @@ namespace NTxMediator { void Handle(TEvMediatorTimecast::TEvWatch::TPtr &ev, const TActorContext &ctx) { const NKikimrTxMediatorTimecast::TEvWatch &record = ev->Get()->Record; // todo: check config coherence - const TActorId &sender = ev->Sender; + const TActorId &sender = ev->Sender; LOG_DEBUG_S(ctx, NKikimrServices::TX_MEDIATOR_EXEC_QUEUE, "Actor# " << ctx.SelfID.ToString() << " MediatorId# " << MediatorId << " HANDLE TEvWatch"); @@ -176,7 +176,7 @@ namespace NTxMediator { return NKikimrServices::TActivity::TX_MEDIATOR_ACTOR; } - TTxMediatorExecQueue(const TActorId &owner, ui64 mediator, ui64 hashRange, ui32 timecastBuckets) + TTxMediatorExecQueue(const TActorId &owner, ui64 mediator, ui64 hashRange, ui32 timecastBuckets) : TActor(&TThis::StateWork) , Owner(owner) , MediatorId(mediator) @@ -186,7 +186,7 @@ namespace NTxMediator { Y_UNUSED(HashRange); } - TAutoPtr<IEventHandle> AfterRegister(const TActorId &self, const TActorId& parentId) override { + TAutoPtr<IEventHandle> AfterRegister(const TActorId &self, const TActorId& parentId) override { Y_UNUSED(parentId); return new IEventHandle(self, self, new TEvents::TEvBootstrap()); } @@ -203,7 +203,7 @@ namespace NTxMediator { }; } -IActor* CreateTxMediatorExecQueue(const TActorId &owner, ui64 mediator, ui64 hashRange, ui32 timecastBuckets) { +IActor* CreateTxMediatorExecQueue(const TActorId &owner, ui64 mediator, ui64 hashRange, ui32 timecastBuckets) { return new NTxMediator::TTxMediatorExecQueue(owner, mediator, hashRange, timecastBuckets); } diff --git a/ydb/core/tx/mediator/mediator.cpp b/ydb/core/tx/mediator/mediator.cpp index d5f20b092e5..0fbd44403c0 100644 --- a/ydb/core/tx/mediator/mediator.cpp +++ b/ydb/core/tx/mediator/mediator.cpp @@ -34,7 +34,7 @@ namespace NTxMediator { } -IActor* CreateTxMediator(const TActorId &tablet, TTabletStorageInfo *info) { +IActor* CreateTxMediator(const TActorId &tablet, TTabletStorageInfo *info) { return new NTxMediator::TTxMediator(info, tablet); } diff --git a/ydb/core/tx/mediator/mediator.h b/ydb/core/tx/mediator/mediator.h index a12758bbb76..93a66ef307a 100644 --- a/ydb/core/tx/mediator/mediator.h +++ b/ydb/core/tx/mediator/mediator.h @@ -5,6 +5,6 @@ namespace NKikimr { -IActor* CreateTxMediator(const TActorId &tablet, TTabletStorageInfo *info); +IActor* CreateTxMediator(const TActorId &tablet, TTabletStorageInfo *info); } diff --git a/ydb/core/tx/mediator/mediator__configure.cpp b/ydb/core/tx/mediator/mediator__configure.cpp index 9c2fe5c9bb6..4a368e321f1 100644 --- a/ydb/core/tx/mediator/mediator__configure.cpp +++ b/ydb/core/tx/mediator/mediator__configure.cpp @@ -7,14 +7,14 @@ using NTabletFlatExecutor::TTransactionBase; using NTabletFlatExecutor::TTransactionContext; struct TTxMediator::TTxConfigure : public TTransactionBase<TTxMediator> { - TActorId AckTo; + TActorId AckTo; ui64 Version; TVector<TCoordinatorId> Coordinators; ui32 TimeCastBuketsPerMediator; TAutoPtr<TEvSubDomain::TEvConfigureStatus> Respond; bool ConfigurationApplied; - TTxConfigure(TSelf *mediator, TActorId ackTo, ui64 version, const TVector<TCoordinatorId>& coordinators, ui32 timeCastBuckets) + TTxConfigure(TSelf *mediator, TActorId ackTo, ui64 version, const TVector<TCoordinatorId>& coordinators, ui32 timeCastBuckets) : TBase(mediator) , AckTo(ackTo) , Version(version) @@ -83,7 +83,7 @@ struct TTxMediator::TTxConfigure : public TTransactionBase<TTxMediator> { } }; -ITransaction* TTxMediator::CreateTxConfigure(TActorId ackTo, ui64 version, const TVector<TCoordinatorId> &coordinators, ui32 timeCastBuckets) { +ITransaction* TTxMediator::CreateTxConfigure(TActorId ackTo, ui64 version, const TVector<TCoordinatorId> &coordinators, ui32 timeCastBuckets) { return new TTxMediator::TTxConfigure(this, ackTo, version, coordinators, timeCastBuckets); } diff --git a/ydb/core/tx/mediator/mediator_impl.cpp b/ydb/core/tx/mediator/mediator_impl.cpp index 202eb0bd3a1..c4b202b8221 100644 --- a/ydb/core/tx/mediator/mediator_impl.cpp +++ b/ydb/core/tx/mediator/mediator_impl.cpp @@ -10,7 +10,7 @@ const ui32 TTxMediator::Schema::CurrentVersion = 0; void TTxMediator::Die(const TActorContext &ctx) { if (!!ExecQueue) { ctx.Send(ExecQueue, new TEvents::TEvPoisonPill()); - ExecQueue = TActorId(); + ExecQueue = TActorId(); } return TActor::Die(ctx); @@ -75,7 +75,7 @@ void TTxMediator::ReplyEnqueuedWatch(const TActorContext &ctx) { WatchEnqueued.clear(); } -void TTxMediator::ReplySync(const TActorId &sender, const NKikimrTx::TEvCoordinatorSync &record, const TActorContext &ctx) { +void TTxMediator::ReplySync(const TActorId &sender, const NKikimrTx::TEvCoordinatorSync &record, const TActorContext &ctx) { Y_VERIFY(record.GetMediatorID() == TabletID()); LOG_DEBUG_S(ctx, NKikimrServices::TX_MEDIATOR, "tablet# " << TabletID() @@ -106,7 +106,7 @@ void TTxMediator::HandleEnqueue(TEvMediatorTimecast::TEvWatch::TPtr &ev, const T WatchEnqueued.push_back(ev); } -void TTxMediator::DoConfigure(const TEvSubDomain::TEvConfigure &ev, const TActorContext &ctx, const TActorId &ackTo) { +void TTxMediator::DoConfigure(const TEvSubDomain::TEvConfigure &ev, const TActorContext &ctx, const TActorId &ackTo) { const TEvSubDomain::TEvConfigure::ProtoRecordType &record = ev.Record; if (0 == record.CoordinatorsSize() || 0 == record.GetTimeCastBucketsPerMediator()) { LOG_ERROR_S(ctx, NKikimrServices::TX_MEDIATOR @@ -143,7 +143,7 @@ void TTxMediator::Handle(TEvTxCoordinator::TEvCoordinatorSync::TPtr &ev, const T ReplySync(ev->Sender, record, ctx); } -void TTxMediator::ReplyStep(const TActorId &sender, NKikimrTx::TEvCoordinatorStepResult::EStatus status, const NKikimrTx::TEvCoordinatorStep &request, const TActorContext &ctx) { +void TTxMediator::ReplyStep(const TActorId &sender, NKikimrTx::TEvCoordinatorStepResult::EStatus status, const NKikimrTx::TEvCoordinatorStep &request, const TActorContext &ctx) { // LOG_DEBUG_S(ctx, NKikimrServices::TX_MEDIATOR, "tablet# " << TabletID() << " SEND EvCoordinatorStepResult" // << " to# " << sender.ToString() << " sender status# " << status << " step# " << request.GetStep() // << " CompleteStep# " << VolatileState.CompleteStep @@ -219,14 +219,14 @@ void TTxMediator::CheckProgress(const TActorContext &ctx) { } } -void TTxMediator::RequestLostAcks(const TActorId &sender, const NKikimrTx::TEvCoordinatorStep &request, const TActorContext &ctx) { +void TTxMediator::RequestLostAcks(const TActorId &sender, const NKikimrTx::TEvCoordinatorStep &request, const TActorContext &ctx) { TAutoPtr<TCoordinatorStep> step(new TCoordinatorStep(request)); LOG_DEBUG_S(ctx, NKikimrServices::TX_MEDIATOR, "tablet# " << TabletID() << " SEND EvRequestLostAcks to# " << ExecQueue.ToString() << " ExecQueue step " << step->ToString()); ctx.Send(ExecQueue, new TEvTxMediator::TEvRequestLostAcks(step, sender)); } -void TTxMediator::ProcessDomainStep(const TActorId &sender, const NKikimrTx::TEvCoordinatorStep &request, ui64 coordinator, TCoordinatorInfo &info, const TActorContext &ctx) { +void TTxMediator::ProcessDomainStep(const TActorId &sender, const NKikimrTx::TEvCoordinatorStep &request, ui64 coordinator, TCoordinatorInfo &info, const TActorContext &ctx) { Y_UNUSED(coordinator); if (request.GetActiveCoordinatorGeneration() != info.ActiveCoordinatorGeneration) { if (request.GetActiveCoordinatorGeneration() < info.ActiveCoordinatorGeneration) // outdated request @@ -259,7 +259,7 @@ void TTxMediator::ProcessDomainStep(const TActorId &sender, const NKikimrTx::TEv CheckProgress(ctx); } -void TTxMediator::ProcessForeignStep(const TActorId &sender, const NKikimrTx::TEvCoordinatorStep &request, ui64 coordinator, TCoordinatorInfo &info, const TActorContext &ctx) { +void TTxMediator::ProcessForeignStep(const TActorId &sender, const NKikimrTx::TEvCoordinatorStep &request, ui64 coordinator, TCoordinatorInfo &info, const TActorContext &ctx) { Y_UNUSED(sender); Y_UNUSED(request); Y_UNUSED(coordinator); @@ -312,7 +312,7 @@ void TTxMediator::Handle(TEvents::TEvPoisonPill::TPtr &ev, const TActorContext & ctx.Send(Tablet(), new TEvents::TEvPoisonPill); } -TTxMediator::TTxMediator(TTabletStorageInfo *info, const TActorId &tablet) +TTxMediator::TTxMediator(TTabletStorageInfo *info, const TActorId &tablet) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, new NMiniKQL::TMiniKQLFactory) {} diff --git a/ydb/core/tx/mediator/mediator_impl.h b/ydb/core/tx/mediator/mediator_impl.h index 59e4edb6dda..c98bd4246b0 100644 --- a/ydb/core/tx/mediator/mediator_impl.h +++ b/ydb/core/tx/mediator/mediator_impl.h @@ -28,7 +28,7 @@ namespace NTxMediator { // transaction body ui64 Moderator; TTxId TxId; - TActorId AckTo; + TActorId AckTo; TTx(ui64 moderator, TTxId txid) : Moderator(moderator) @@ -159,9 +159,9 @@ struct TEvTxMediator { struct TEvRequestLostAcks : public TEventLocal<TEvRequestLostAcks, EvRequestLostAcks> { TAutoPtr<NTxMediator::TCoordinatorStep> CoordinatorStep; - const TActorId AckTo; + const TActorId AckTo; - TEvRequestLostAcks(TAutoPtr<NTxMediator::TCoordinatorStep> &cds, const TActorId &ackTo) + TEvRequestLostAcks(TAutoPtr<NTxMediator::TCoordinatorStep> &cds, const TActorId &ackTo) : CoordinatorStep(cds) , AckTo(ackTo) {} @@ -240,9 +240,9 @@ struct TEvTxMediator { }; struct TEvWatchBucket : public TEventLocal<TEvWatchBucket, EvWatchBucket> { - const TActorId Source; + const TActorId Source; - TEvWatchBucket(const TActorId &source) + TEvWatchBucket(const TActorId &source) : Source(source) {} @@ -282,7 +282,7 @@ class TTxMediator : public TActor<TTxMediator>, public NTabletFlatExecutor::TTab TQueueType Queue; ui64 ActiveCoordinatorGeneration; - TActorId AckTo; + TActorId AckTo; TCoordinatorInfo() : KnownPrevStep(0) @@ -309,7 +309,7 @@ class TTxMediator : public TActor<TTxMediator>, public NTabletFlatExecutor::TTab struct TTxUpgrade; ITransaction* CreateTxInit(); - ITransaction* CreateTxConfigure(TActorId ackTo, ui64 version, const TVector<TCoordinatorId> &coordinators, ui32 timeCastBuckets); + ITransaction* CreateTxConfigure(TActorId ackTo, ui64 version, const TVector<TCoordinatorId> &coordinators, ui32 timeCastBuckets); ITransaction* CreateTxSchema(); ITransaction* CreateTxUpgrade(); @@ -317,9 +317,9 @@ class TTxMediator : public TActor<TTxMediator>, public NTabletFlatExecutor::TTab TConfig Config; TVolatileState VolatileState; - TActorId ExecQueue; + TActorId ExecQueue; - THashMap<TActorId, NKikimrTx::TEvCoordinatorSync> CoordinatorsSyncEnqueued; + THashMap<TActorId, NKikimrTx::TEvCoordinatorSync> CoordinatorsSyncEnqueued; TVector<TEvMediatorTimecast::TEvWatch::TPtr> WatchEnqueued; void Die(const TActorContext &ctx) override; @@ -336,7 +336,7 @@ class TTxMediator : public TActor<TTxMediator>, public NTabletFlatExecutor::TTab void Handle(TEvMediatorTimecast::TEvWatch::TPtr &ev, const TActorContext &ctx); void Handle(TEvents::TEvPoisonPill::TPtr &ev, const TActorContext &ctx); - void DoConfigure(const TEvSubDomain::TEvConfigure &ev, const TActorContext &ctx, const TActorId &ackTo = TActorId()); + void DoConfigure(const TEvSubDomain::TEvConfigure &ev, const TActorContext &ctx, const TActorId &ackTo = TActorId()); static ui64 SubjectiveTime(); void InitSelfState(const TActorContext &ctx); @@ -344,15 +344,15 @@ class TTxMediator : public TActor<TTxMediator>, public NTabletFlatExecutor::TTab void ReplyEnqueuedWatch(const TActorContext &ctx); - void ReplySync(const TActorId &sender, const NKikimrTx::TEvCoordinatorSync &record, const TActorContext &ctx); - void ReplyStep(const TActorId &sender, NKikimrTx::TEvCoordinatorStepResult::EStatus status, const NKikimrTx::TEvCoordinatorStep &request, const TActorContext &ctx); + void ReplySync(const TActorId &sender, const NKikimrTx::TEvCoordinatorSync &record, const TActorContext &ctx); + void ReplyStep(const TActorId &sender, NKikimrTx::TEvCoordinatorStepResult::EStatus status, const NKikimrTx::TEvCoordinatorStep &request, const TActorContext &ctx); ui64 FindProgressCandidate(); void Progress(ui64 to, const TActorContext &ctx); void CheckProgress(const TActorContext &ctx); - void RequestLostAcks(const TActorId &sender, const NKikimrTx::TEvCoordinatorStep &request, const TActorContext &ctx); + void RequestLostAcks(const TActorId &sender, const NKikimrTx::TEvCoordinatorStep &request, const TActorContext &ctx); - void ProcessDomainStep(const TActorId &sender, const NKikimrTx::TEvCoordinatorStep &request, ui64 coordinator, TCoordinatorInfo &info, const TActorContext &ctx); - void ProcessForeignStep(const TActorId &sender, const NKikimrTx::TEvCoordinatorStep &request, ui64 coordinator, TCoordinatorInfo &info, const TActorContext &ctx); + void ProcessDomainStep(const TActorId &sender, const NKikimrTx::TEvCoordinatorStep &request, ui64 coordinator, TCoordinatorInfo &info, const TActorContext &ctx); + void ProcessForeignStep(const TActorId &sender, const NKikimrTx::TEvCoordinatorStep &request, ui64 coordinator, TCoordinatorInfo &info, const TActorContext &ctx); public: struct Schema : NIceDb::Schema { @@ -386,7 +386,7 @@ public: return NKikimrServices::TActivity::TX_MEDIATOR_ACTOR; } - TTxMediator(TTabletStorageInfo *info, const TActorId &tablet); + TTxMediator(TTabletStorageInfo *info, const TActorId &tablet); // no incomming pipes is allowed in StateInit STFUNC_TABLET_INIT(StateInit, @@ -414,7 +414,7 @@ public: }; } -IActor* CreateTxMediatorTabletQueue(const TActorId &owner, ui64 mediator, ui64 hashRange, ui64 hashBucket); -IActor* CreateTxMediatorExecQueue(const TActorId &owner, ui64 mediator, ui64 hashRange, ui32 timecastBuckets); +IActor* CreateTxMediatorTabletQueue(const TActorId &owner, ui64 mediator, ui64 hashRange, ui64 hashBucket); +IActor* CreateTxMediatorExecQueue(const TActorId &owner, ui64 mediator, ui64 hashRange, ui32 timecastBuckets); } diff --git a/ydb/core/tx/mediator/tablet_queue.cpp b/ydb/core/tx/mediator/tablet_queue.cpp index 3669e4587ea..b28179517ca 100644 --- a/ydb/core/tx/mediator/tablet_queue.cpp +++ b/ydb/core/tx/mediator/tablet_queue.cpp @@ -60,7 +60,7 @@ class TTxMediatorTabletQueue : public TActor<TTxMediatorTabletQueue> { void MergeToOutOfOrder(TStepId step, TVector<TTx> &update); }; - const TActorId Owner; + const TActorId Owner; const ui64 Mediator; const ui64 HashRange; const ui64 HashBucket; @@ -76,7 +76,7 @@ class TTxMediatorTabletQueue : public TActor<TTxMediatorTabletQueue> { TStepId CommitedStep; TStepEntry *ActiveStep; - THashSet<TActorId> TimecastWatches; + THashSet<TActorId> TimecastWatches; NMonitoring::TDynamicCounters::TCounterPtr TimecastLagCounter; void SendToTablet(TTabletEntry::TStep *tabletStep, ui64 tablet, const TActorContext &ctx) { @@ -87,7 +87,7 @@ class TTxMediatorTabletQueue : public TActor<TTxMediatorTabletQueue> { x->SetTxId(tx.TxId); if (tx.Moderator) x->SetModerator(tx.Moderator); - ActorIdToProto(tx.AckTo, x->MutableAckTo()); + ActorIdToProto(tx.AckTo, x->MutableAckTo()); LOG_DEBUG(ctx, NKikimrServices::TX_MEDIATOR_PRIVATE, "Send from %" PRIu64 " to tablet %" PRIu64 ", step# %" PRIu64 ", txid# %" PRIu64 ", marker M5" PRIu64, Mediator, tablet, tabletStep->StepRef->Step, tx.TxId); } @@ -125,7 +125,7 @@ class TTxMediatorTabletQueue : public TActor<TTxMediatorTabletQueue> { // todo: we must throttle delivery const ui32 sendFlags = IEventHandle::FlagTrackDelivery; - for (const TActorId &x : TimecastWatches) { + for (const TActorId &x : TimecastWatches) { LOG_DEBUG_S(ctx, NKikimrServices::TX_MEDIATOR_TABLETQUEUE, "Actor# " << ctx.SelfID.ToString() << " Mediator# " << Mediator << " SEND to# " << x.ToString() << " " << evx.ToString()); ctx.ExecutorThread.Send(new IEventHandle(TEvMediatorTimecast::TEvUpdate::EventType, sendFlags, x, ctx.SelfID, data, 0)); @@ -289,7 +289,7 @@ class TTxMediatorTabletQueue : public TActor<TTxMediatorTabletQueue> { void Handle(TEvTxMediator::TEvWatchBucket::TPtr &ev, const TActorContext &ctx) { LOG_DEBUG_S(ctx, NKikimrServices::TX_MEDIATOR_TABLETQUEUE, "Actor# " << ctx.SelfID.ToString() << " Mediator# " << Mediator << " HANDLE " << ev->Get()->ToString()); - const TActorId &source = ev->Get()->Source; + const TActorId &source = ev->Get()->Source; TimecastWatches.insert(source); } @@ -301,7 +301,7 @@ class TTxMediatorTabletQueue : public TActor<TTxMediatorTabletQueue> { } void AckOoO(TTabletId tablet, TStepId step, const TVector<TTx> &transactions, const TActorContext &ctx) { - TMap<TActorId, TAutoPtr<TEvTxProcessing::TEvPlanStepAck>> acks; + TMap<TActorId, TAutoPtr<TEvTxProcessing::TEvPlanStepAck>> acks; for (const TTx &tx : transactions) { TAutoPtr<TEvTxProcessing::TEvPlanStepAck> &ack = acks[tx.AckTo]; if (!ack) @@ -340,7 +340,7 @@ public: return NKikimrServices::TActivity::TX_MEDIATOR_ACTOR; } - TTxMediatorTabletQueue(const TActorId &owner, ui64 mediator, ui64 hashRange, ui64 hashBucket) + TTxMediatorTabletQueue(const TActorId &owner, ui64 mediator, ui64 hashRange, ui64 hashBucket) : TActor(&TThis::StateFunc) , Owner(owner) , Mediator(mediator) @@ -455,7 +455,7 @@ void TTxMediatorTabletQueue::TTabletEntry::MergeToOutOfOrder(TStepId step, TVect } -IActor* CreateTxMediatorTabletQueue(const TActorId &owner, ui64 mediator, ui64 hashRange, ui64 hashBucket) { +IActor* CreateTxMediatorTabletQueue(const TActorId &owner, ui64 mediator, ui64 hashRange, ui64 hashBucket) { return new NTxMediator::TTxMediatorTabletQueue(owner, mediator, hashRange, hashBucket); } diff --git a/ydb/core/tx/scheme_board/cache.cpp b/ydb/core/tx/scheme_board/cache.cpp index 480c0c714a8..8e778a4a1d8 100644 --- a/ydb/core/tx/scheme_board/cache.cpp +++ b/ydb/core/tx/scheme_board/cache.cpp @@ -104,7 +104,7 @@ namespace { return NKikimrServices::TActivity::SCHEME_BOARD_DB_RESOLVER; } - TDbResolver(const TActorId& cache, const TActorId& sender, THolder<TRequest> request, ui64 domainOwnerId) + TDbResolver(const TActorId& cache, const TActorId& sender, THolder<TRequest> request, ui64 domainOwnerId) : Cache(cache) , Sender(sender) , Request(std::move(request)) @@ -135,8 +135,8 @@ namespace { using TBase = TDbResolver<TRequest, TEvRequest, TDerived>; private: - const TActorId Cache; - const TActorId Sender; + const TActorId Cache; + const TActorId Sender; THolder<TRequest> Request; const ui64 DomainOwnerId; @@ -152,11 +152,11 @@ namespace { using TBase::TBase; }; - IActor* CreateDbResolver(const TActorId& cache, const TActorId& sender, THolder<TNavigate> request, ui64 domainOwnerId) { + IActor* CreateDbResolver(const TActorId& cache, const TActorId& sender, THolder<TNavigate> request, ui64 domainOwnerId) { return new TDbResolverNavigate(cache, sender, std::move(request), domainOwnerId); } - IActor* CreateDbResolver(const TActorId& cache, const TActorId& sender, THolder<TResolve> request, ui64 domainOwnerId) { + IActor* CreateDbResolver(const TActorId& cache, const TActorId& sender, THolder<TResolve> request, ui64 domainOwnerId) { return new TDbResolverResolve(cache, sender, std::move(request), domainOwnerId); } @@ -577,7 +577,7 @@ class TSchemeCache: public TMonitorableActor<TSchemeCache> { ByPath, }; - TActorId Subscriber; + TActorId Subscriber; ui64 DomainOwnerId; EType Type; mutable ui64 SyncCookie; @@ -589,7 +589,7 @@ class TSchemeCache: public TMonitorableActor<TSchemeCache> { { } - explicit TSubscriber(const TActorId& subscriber, const ui64 domainOwnerId, const TPathId&) + explicit TSubscriber(const TActorId& subscriber, const ui64 domainOwnerId, const TPathId&) : Subscriber(subscriber) , DomainOwnerId(domainOwnerId) , Type(EType::ByPathId) @@ -597,7 +597,7 @@ class TSchemeCache: public TMonitorableActor<TSchemeCache> { { } - explicit TSubscriber(const TActorId& subscriber, const ui64 domainOwnerId, const TString&) + explicit TSubscriber(const TActorId& subscriber, const ui64 domainOwnerId, const TString&) : Subscriber(subscriber) , DomainOwnerId(domainOwnerId) , Type(EType::ByPath) diff --git a/ydb/core/tx/scheme_board/cache_ut.cpp b/ydb/core/tx/scheme_board/cache_ut.cpp index d3c6c0d4390..c27b3517b34 100644 --- a/ydb/core/tx/scheme_board/cache_ut.cpp +++ b/ydb/core/tx/scheme_board/cache_ut.cpp @@ -100,7 +100,7 @@ protected: void CreateAndMigrateWithoutDecision(ui64& txId); private: - TActorId SchemeCache; + TActorId SchemeCache; }; // TCacheTest @@ -156,7 +156,7 @@ void TCacheTest::List() { } void TCacheTest::Recreate() { - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); ui64 txId = 100; TestMkDir(*Context, ++txId, "/Root", "DirA"); @@ -378,7 +378,7 @@ TNavigate::TEntry TCacheTest::TestNavigateImpl(THolder<TNavigate> request, TNavi request->UserToken = new NACLib::TUserToken(sid, {}); } - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); Context->Send(SchemeCache, edge, new TEvTxProxySchemeCache::TEvNavigateKeySet(request.Release()), 0, 0, 0, true); auto ev = Context->GrabEdgeEvent<TEvTxProxySchemeCache::TEvNavigateKeySetResult>(edge); @@ -434,7 +434,7 @@ TResolve::TEntry TCacheTest::TestResolve(const TTableId& tableId, TResolve::ESta request->UserToken = new NACLib::TUserToken(sid, {}); } - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); Context->Send(SchemeCache, edge, new TEvTxProxySchemeCache::TEvResolveKeySet(request.Release()), 0, 0, 0, true); auto ev = Context->GrabEdgeEvent<TEvTxProxySchemeCache::TEvResolveKeySetResult>(edge); diff --git a/ydb/core/tx/scheme_board/events.h b/ydb/core/tx/scheme_board/events.h index e6b29fae7a3..018ebbff7db 100644 --- a/ydb/core/tx/scheme_board/events.h +++ b/ydb/core/tx/scheme_board/events.h @@ -53,11 +53,11 @@ struct TSchemeBoardEvents { // populator events struct TEvRequestDescribe: public TEventLocal<TEvRequestDescribe, EvRequestDescribe> { const TPathId PathId; - const TActorId Replica; + const TActorId Replica; TEvRequestDescribe() = default; - explicit TEvRequestDescribe(const TPathId pathId, const TActorId& replica) + explicit TEvRequestDescribe(const TPathId pathId, const TActorId& replica) : PathId(pathId) , Replica(replica) { diff --git a/ydb/core/tx/scheme_board/helpers.cpp b/ydb/core/tx/scheme_board/helpers.cpp index ba36261caed..3b53323ad3f 100644 --- a/ydb/core/tx/scheme_board/helpers.cpp +++ b/ydb/core/tx/scheme_board/helpers.cpp @@ -12,7 +12,7 @@ namespace NKikimr { namespace NSchemeBoard { -TActorId MakeInterconnectProxyId(const ui32 nodeId) { +TActorId MakeInterconnectProxyId(const ui32 nodeId) { return TActivationContext::InterconnectProxy(nodeId); } @@ -87,9 +87,9 @@ TIntrusivePtr<TEventSerializedData> SerializeEvent(IEventBase* ev) { return serializer.Release(ev->IsExtendedFormat()); } -void MultiSend(const TVector<const TActorId*>& recipients, const TActorId& sender, TAutoPtr<IEventBase> ev, ui32 flags, ui64 cookie) { +void MultiSend(const TVector<const TActorId*>& recipients, const TActorId& sender, TAutoPtr<IEventBase> ev, ui32 flags, ui64 cookie) { auto buffer = SerializeEvent(ev.Get()); - for (const TActorId* recipient : recipients) { + for (const TActorId* recipient : recipients) { TlsActivationContext->Send(new IEventHandle( ev->Type(), flags, *recipient, sender, buffer, cookie )); diff --git a/ydb/core/tx/scheme_board/helpers.h b/ydb/core/tx/scheme_board/helpers.h index b528eb57f17..3b07df3e065 100644 --- a/ydb/core/tx/scheme_board/helpers.h +++ b/ydb/core/tx/scheme_board/helpers.h @@ -33,7 +33,7 @@ namespace NSchemeBoard { using TDomainId = TPathId; -TActorId MakeInterconnectProxyId(const ui32 nodeId); +TActorId MakeInterconnectProxyId(const ui32 nodeId); ui64 GetPathVersion(const NKikimrScheme::TEvDescribeSchemeResult& record); TPathId GetPathId(const NKikimrScheme::TEvDescribeSchemeResult& record); TDomainId GetDomainId(const NKikimrScheme::TEvDescribeSchemeResult& record); @@ -41,10 +41,10 @@ TSet<ui64> GetAbandonedSchemeShardIds(const NKikimrScheme::TEvDescribeSchemeResu TIntrusivePtr<TEventSerializedData> SerializeEvent(IEventBase* ev); -void MultiSend(const TVector<const TActorId*>& recipients, const TActorId& sender, TAutoPtr<IEventBase> ev, ui32 flags = 0, ui64 cookie = 0); +void MultiSend(const TVector<const TActorId*>& recipients, const TActorId& sender, TAutoPtr<IEventBase> ev, ui32 flags = 0, ui64 cookie = 0); template <typename TEvent> -void MultiSend(const TVector<const TActorId*>& recipients, const TActorId& sender, THolder<TEvent> ev, ui32 flags = 0, ui64 cookie = 0) { +void MultiSend(const TVector<const TActorId*>& recipients, const TActorId& sender, THolder<TEvent> ev, ui32 flags = 0, ui64 cookie = 0) { MultiSend(recipients, sender, static_cast<IEventBase*>(ev.Release()), flags, cookie); } diff --git a/ydb/core/tx/scheme_board/load_test.cpp b/ydb/core/tx/scheme_board/load_test.cpp index a7da83660e5..d993f1e5ed2 100644 --- a/ydb/core/tx/scheme_board/load_test.cpp +++ b/ydb/core/tx/scheme_board/load_test.cpp @@ -183,7 +183,7 @@ class TLoadProducer: public TActorBootstrapped<TLoadProducer> { } void Boot() { - const TActorId proxy = MakeStateStorageProxyID(StateStorageGroupFromTabletID(Owner)); + const TActorId proxy = MakeStateStorageProxyID(StateStorageGroupFromTabletID(Owner)); Send(proxy, new TEvStateStorage::TEvListSchemeBoard(), IEventHandle::FlagTrackDelivery); Become(&TThis::StateBoot); @@ -234,7 +234,7 @@ class TLoadProducer: public TActorBootstrapped<TLoadProducer> { Y_VERIFY(msg->PathId.LocalPathId == NextPathId - 1); Send(Subscriber, new TEvents::TEvPoisonPill()); - Subscriber = TActorId(); + Subscriber = TActorId(); const TInstant ts = TInstant::FromValue(GetPathVersion(msg->DescribeSchemeResult)); *SyncDuration = (TlsActivationContext->Now() - ts).MilliSeconds(); @@ -317,8 +317,8 @@ private: TDescriptions Descriptions; ui64 NextPathId; - TActorId Populator; - TActorId Subscriber; + TActorId Populator; + TActorId Subscriber; }; // TLoadProducer @@ -326,7 +326,7 @@ class TLoadConsumer: public TActorBootstrapped<TLoadConsumer> { void Subscribe(const TPathId& pathId) { const ui32 ssId = StateStorageGroupFromTabletID(Owner); for (ui32 i = 0; i < Config.SubscriberMulti; ++i) { - const TActorId subscriber = Register(CreateSchemeBoardSubscriber( + const TActorId subscriber = Register(CreateSchemeBoardSubscriber( SelfId(), pathId, ssId, ESchemeBoardSubscriberDeletionPolicy::Majority )); @@ -407,7 +407,7 @@ private: NMonitoring::TDynamicCounters::TCounterPtr SubscribersCount; ui64 MaxPathId; - TVector<TActorId> Subscribers; + TVector<TActorId> Subscribers; }; // TLoadConsumer diff --git a/ydb/core/tx/scheme_board/load_test.h b/ydb/core/tx/scheme_board/load_test.h index 8010167b4bb..271e96bf3fc 100644 --- a/ydb/core/tx/scheme_board/load_test.h +++ b/ydb/core/tx/scheme_board/load_test.h @@ -2,7 +2,7 @@ #include "defs.h" -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> namespace NKikimr { namespace NSchemeBoard { diff --git a/ydb/core/tx/scheme_board/populator.cpp b/ydb/core/tx/scheme_board/populator.cpp index 2411494bf6c..f860bb88cb1 100644 --- a/ydb/core/tx/scheme_board/populator.cpp +++ b/ydb/core/tx/scheme_board/populator.cpp @@ -408,8 +408,8 @@ public: } explicit TReplicaPopulator( - const TActorId& parent, - const TActorId& replica, + const TActorId& parent, + const TActorId& replica, const ui64 owner, const ui64 generation) : Parent(parent) @@ -478,8 +478,8 @@ public: } private: - const TActorId Parent; - const TActorId Replica; + const TActorId Parent; + const TActorId Replica; const ui64 Owner; const ui64 Generation; @@ -497,7 +497,7 @@ private: }; // TReplicaPopulator class TPopulator: public TMonitorableActor<TPopulator> { - TConstArrayRef<TActorId> SelectReplicas(TPathId pathId, TStringBuf path) { + TConstArrayRef<TActorId> SelectReplicas(TPathId pathId, TStringBuf path) { SelectionReplicaCache.clear(); const ui64 pathHash = CityHash64(path); @@ -509,16 +509,16 @@ class TPopulator: public TMonitorableActor<TPopulator> { SelectionReplicaCache.insert(SelectionReplicaCache.end(), selection.begin(), selection.end()); GroupInfo->SelectReplicas(idHash, &selection); - for (const TActorId& replica : selection) { + for (const TActorId& replica : selection) { if (Find(SelectionReplicaCache, replica) == SelectionReplicaCache.end()) { SelectionReplicaCache.emplace_back(replica); } } if (SelectionReplicaCache) { - return TConstArrayRef<TActorId>(&SelectionReplicaCache.front(), SelectionReplicaCache.size()); + return TConstArrayRef<TActorId>(&SelectionReplicaCache.front(), SelectionReplicaCache.size()); } else { - return TConstArrayRef<TActorId>(); + return TConstArrayRef<TActorId>(); } } @@ -528,9 +528,9 @@ class TPopulator: public TMonitorableActor<TPopulator> { const auto& record = it->second.Record; - TConstArrayRef<TActorId> replicas = SelectReplicas(pathId, record.GetPath()); + TConstArrayRef<TActorId> replicas = SelectReplicas(pathId, record.GetPath()); for (const auto& replica : replicas) { - const TActorId* replicaPopulator = ReplicaToReplicaPopulator.FindPtr(replica); + const TActorId* replicaPopulator = ReplicaToReplicaPopulator.FindPtr(replica); Y_VERIFY(replicaPopulator != nullptr); auto update = MakeHolder<TSchemeBoardEvents::TEvUpdateBuilder>(Owner, Generation, record, isDeletion); @@ -549,8 +549,8 @@ class TPopulator: public TMonitorableActor<TPopulator> { << ", sender# " << ev->Sender << ", pathId# " << ev->Get()->PathId); - const TActorId replicaPopulator = ev->Sender; - const TActorId replica = ev->Get()->Replica; + const TActorId replicaPopulator = ev->Sender; + const TActorId replica = ev->Get()->Replica; if (ReplicaToReplicaPopulator[replica] != replicaPopulator) { SBP_LOG_CRIT("Inconsistent replica populator" @@ -589,7 +589,7 @@ class TPopulator: public TMonitorableActor<TPopulator> { } TPathId pathId(it->second.Record.GetPathOwnerId(), it->second.Record.GetPathId()); - TConstArrayRef<TActorId> replicas = SelectReplicas(pathId, it->second.Record.GetPath()); + TConstArrayRef<TActorId> replicas = SelectReplicas(pathId, it->second.Record.GetPath()); if (Find(replicas, replica) != replicas.end()) { break; } @@ -922,7 +922,7 @@ public: void Bootstrap() { TMonitorableActor::Bootstrap(); - const TActorId proxy = MakeStateStorageProxyID(StateStorageGroup); + const TActorId proxy = MakeStateStorageProxyID(StateStorageGroup); Send(proxy, new TEvStateStorage::TEvListSchemeBoard(), IEventHandle::FlagTrackDelivery); Become(&TThis::StateResolve); } @@ -976,12 +976,12 @@ private: TDelayedUpdates DelayedUpdates; TIntrusiveConstPtr<TStateStorageInfo> GroupInfo; - THashMap<TActorId, TActorId> ReplicaToReplicaPopulator; + THashMap<TActorId, TActorId> ReplicaToReplicaPopulator; - TVector<TActorId> SelectionReplicaCache; + TVector<TActorId> SelectionReplicaCache; struct TUpdateAckInfo { - TActorId AckTo; + TActorId AckTo; TMap<std::pair<TPathId, ui64>, ui32> PathAcks; }; diff --git a/ydb/core/tx/scheme_board/populator_ut.cpp b/ydb/core/tx/scheme_board/populator_ut.cpp index 8c77fb276ed..c0eb1cb62fa 100644 --- a/ydb/core/tx/scheme_board/populator_ut.cpp +++ b/ydb/core/tx/scheme_board/populator_ut.cpp @@ -30,7 +30,7 @@ public: UNIT_TEST_SUITE_END(); void Boot() { - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); NKikimr::TPathId rootPathId(TTestTxConfig::SchemeShard, RootPathId); @@ -43,7 +43,7 @@ public: } void MakeDir() { - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); TestMkDir(*Context, 100, "/Root", "DirA"); auto describe = DescribePath(*Context, "/Root/DirA"); @@ -59,7 +59,7 @@ public: } void RemoveDir() { - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); TestMkDir(*Context, 100, "/Root", "DirB"); auto describe = DescribePath(*Context, "/Root/DirB"); @@ -118,7 +118,7 @@ public: } private: - TMap<TActorId, bool> ReplicaPopulators; + TMap<TActorId, bool> ReplicaPopulators; bool DropFirstAcks = false; }; // TPopulatorTestWithResets diff --git a/ydb/core/tx/scheme_board/replica.cpp b/ydb/core/tx/scheme_board/replica.cpp index d82d6b1fa8f..638656e91db 100644 --- a/ydb/core/tx/scheme_board/replica.cpp +++ b/ydb/core/tx/scheme_board/replica.cpp @@ -153,7 +153,7 @@ public: } auto notify = BuildNotify(); - TVector<const TActorId*> subscribers(Reserve(Subscribers.size())); + TVector<const TActorId*> subscribers(Reserve(Subscribers.size())); for (auto& [subscriber, info] : Subscribers) { if (!info.EnqueueVersion(notify.Get())) { @@ -435,26 +435,26 @@ public: return notify; } - void Subscribe(const TActorId& subscriber, const TString&, ui64 domainOwnerId, const TCapabilities& capabilities) { + void Subscribe(const TActorId& subscriber, const TString&, ui64 domainOwnerId, const TCapabilities& capabilities) { Subscribers.emplace(subscriber, TSubscriberInfo(SUBSCRIPTION_BY_PATH, domainOwnerId, capabilities)); } - void Subscribe(const TActorId& subscriber, const TPathId&, ui64 domainOwnerId, const TCapabilities& capabilities) { + void Subscribe(const TActorId& subscriber, const TPathId&, ui64 domainOwnerId, const TCapabilities& capabilities) { Subscribers.emplace(subscriber, TSubscriberInfo(SUBSCRIPTION_BY_PATH_ID, domainOwnerId, capabilities)); } - void Unsubscribe(const TActorId& subscriber) { + void Unsubscribe(const TActorId& subscriber) { Subscribers.erase(subscriber); } - TSubscriberInfo& GetSubscriberInfo(const TActorId& subscriber) { + TSubscriberInfo& GetSubscriberInfo(const TActorId& subscriber) { auto it = Subscribers.find(subscriber); Y_VERIFY(it != Subscribers.end()); return it->second; } - THashMap<TActorId, TSubscriberInfo> GetSubscribers(const ESubscriptionType type = SUBSCRIPTION_UNSPECIFIED) const { - THashMap<TActorId, TSubscriberInfo> result; + THashMap<TActorId, TSubscriberInfo> GetSubscribers(const ESubscriptionType type = SUBSCRIPTION_UNSPECIFIED) const { + THashMap<TActorId, TSubscriberInfo> result; for (const auto& [subscriber, info] : Subscribers) { if (type == SUBSCRIPTION_UNSPECIFIED || type == info.GetType()) { @@ -477,7 +477,7 @@ public: bool ExplicitlyDeleted = false; // subscribers - THashMap<TActorId, TSubscriberInfo> Subscribers; + THashMap<TActorId, TSubscriberInfo> Subscribers; // memory tracking size_t ResultSize = 0; @@ -495,7 +495,7 @@ private: struct TPopulatorInfo { ui64 Generation = 0; ui64 PendingGeneration = 0; - TActorId PopulatorActor; + TActorId PopulatorActor; bool IsCommited() const { return Generation && Generation == PendingGeneration; @@ -627,16 +627,16 @@ private: } // call it _after_ Subscribe() & _before_ Unsubscribe() - bool IsSingleSubscriberOnNode(const TActorId& subscriber) const { + bool IsSingleSubscriberOnNode(const TActorId& subscriber) const { const ui32 nodeId = subscriber.NodeId(); - auto it = Subscribers.lower_bound(TActorId(nodeId, 0, 0, 0)); + auto it = Subscribers.lower_bound(TActorId(nodeId, 0, 0, 0)); Y_VERIFY(it != Subscribers.end()); return ++it == Subscribers.end() || it->first.NodeId() != nodeId; } template <typename TPath> - void Subscribe(const TActorId& subscriber, const TPath& path, ui64 domainOwnerId, const TCapabilities& capabilities) { + void Subscribe(const TActorId& subscriber, const TPath& path, ui64 domainOwnerId, const TCapabilities& capabilities) { TDescription* desc = Descriptions.FindPtr(path); Y_VERIFY(desc); @@ -655,7 +655,7 @@ private: } template <typename TPath> - void Unsubscribe(const TActorId& subscriber, const TPath& path) { + void Unsubscribe(const TActorId& subscriber, const TPath& path) { TDescription* desc = Descriptions.FindPtr(path); Y_VERIFY(desc); @@ -669,7 +669,7 @@ private: } template <typename TPath> - void SubscribeBy(const TActorId& subscriber, const TPath& path, ui64 domainOwnerId, const TCapabilities& capabilities, + void SubscribeBy(const TActorId& subscriber, const TPath& path, ui64 domainOwnerId, const TCapabilities& capabilities, bool needNotify = true) { TDescription* desc = Descriptions.FindPtr(path); if (!desc) { @@ -698,7 +698,7 @@ private: } template <typename TPath> - void UnsubscribeBy(const TActorId& subscriber, const TPath& path) { + void UnsubscribeBy(const TActorId& subscriber, const TPath& path) { if (!Descriptions.FindPtr(path) || !Subscribers.contains(subscriber)) { return; } @@ -1268,9 +1268,9 @@ private: << ": self# " << SelfId() << ", nodeId# " << nodeId); - auto it = Subscribers.lower_bound(TActorId(nodeId, 0, 0, 0)); + auto it = Subscribers.lower_bound(TActorId(nodeId, 0, 0, 0)); while (it != Subscribers.end() && it->first.NodeId() == nodeId) { - const TActorId subscriber = it->first; + const TActorId subscriber = it->first; const auto id = it->second; ++it; @@ -1286,7 +1286,7 @@ private: void PassAway() override { for (auto &xpair : Populators) { - if (const TActorId populator = xpair.second.PopulatorActor) { + if (const TActorId populator = xpair.second.PopulatorActor) { Send(populator, new TEvStateStorage::TEvReplicaShutdown()); } } diff --git a/ydb/core/tx/scheme_board/replica.h b/ydb/core/tx/scheme_board/replica.h index 4d0e8b6cd48..a626fea2a1e 100644 --- a/ydb/core/tx/scheme_board/replica.h +++ b/ydb/core/tx/scheme_board/replica.h @@ -7,7 +7,7 @@ namespace NKikimr { // same as MakeStateStorageReplicaID -inline TActorId MakeSchemeBoardReplicaID( +inline TActorId MakeSchemeBoardReplicaID( const ui32 node, const ui64 stateStorageGroup, const ui32 replicaIndex @@ -15,7 +15,7 @@ inline TActorId MakeSchemeBoardReplicaID( char x[12] = { 's', 'b', 's' }; x[3] = (char)stateStorageGroup; memcpy(x + 5, &replicaIndex, sizeof(ui32)); - return TActorId(node, TStringBuf(x, 12)); + return TActorId(node, TStringBuf(x, 12)); } } // NKikimr diff --git a/ydb/core/tx/scheme_board/replica_ut.cpp b/ydb/core/tx/scheme_board/replica_ut.cpp index c130747ea1d..10113c7bcce 100644 --- a/ydb/core/tx/scheme_board/replica_ut.cpp +++ b/ydb/core/tx/scheme_board/replica_ut.cpp @@ -99,7 +99,7 @@ public: private: THolder<TTestContext> Context; - TActorId Replica; + TActorId Replica; }; // TReplicaTest @@ -114,7 +114,7 @@ void TReplicaTest::Handshake() { } void TReplicaTest::HandshakeWithStaleGeneration() { - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); Context->HandshakeReplica(Replica, edge, 1, 2); Context->HandshakeReplica(Replica, edge, 1, 1, false); @@ -124,7 +124,7 @@ void TReplicaTest::HandshakeWithStaleGeneration() { } void TReplicaTest::Commit() { - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); Context->HandshakeReplica(Replica, edge, 1, 1); Context->CommitReplica(Replica, edge, 1, 1); @@ -137,7 +137,7 @@ void TReplicaTest::Commit() { } void TReplicaTest::CommitWithoutHandshake() { - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); Context->CommitReplica(Replica, edge, 1, 1); auto ev = Context->HandshakeReplica(Replica, edge, 1, 1); @@ -148,8 +148,8 @@ void TReplicaTest::CommitWithoutHandshake() { } void TReplicaTest::CommitWithStaleGeneration() { - const TActorId edgeA = Context->AllocateEdgeActor(); - const TActorId edgeB = Context->AllocateEdgeActor(); + const TActorId edgeA = Context->AllocateEdgeActor(); + const TActorId edgeB = Context->AllocateEdgeActor(); Context->HandshakeReplica(Replica, edgeA, 1, 0); Context->HandshakeReplica(Replica, edgeB, 1, 1); @@ -165,7 +165,7 @@ void TReplicaTest::CommitWithStaleGeneration() { } void TReplicaTest::Update() { - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); Context->HandshakeReplica(Replica, edge, 1, 1); Context->Send(Replica, edge, GenerateUpdate(GenerateDescribe("path", TPathId(1, 1)), 1, 1)); @@ -194,7 +194,7 @@ void TReplicaTest::Update() { void TReplicaTest::Delete() { auto describe = GenerateDescribe("path", TPathId(42, 1)); - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); Context->HandshakeReplica(Replica, edge, 1, 1); Context->Send(Replica, edge, GenerateUpdate(describe, 1, 1)); @@ -237,7 +237,7 @@ void TReplicaTest::Delete() { } void TReplicaTest::UpdateWithoutHandshake() { - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); Context->Send(Replica, edge, GenerateUpdate(GenerateDescribe("path", TPathId(1, 1)), 1, 1)); @@ -255,7 +255,7 @@ void TReplicaTest::UpdateWithoutHandshake() { } void TReplicaTest::UpdateWithStaleGeneration() { - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); Context->HandshakeReplica(Replica, edge, 1, 1); Context->Send(Replica, edge, GenerateUpdate(GenerateDescribe("path", TPathId(1, 1)), 1, 0)); @@ -276,7 +276,7 @@ void TReplicaTest::UpdateWithStaleGeneration() { void TReplicaTest::Subscribe() { auto describe = GenerateDescribe("path", TPathId(1, 1)); - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); Context->HandshakeReplica(Replica, edge, 1, 1); Context->Send(Replica, edge, GenerateUpdate(describe, 1, 1)); @@ -297,7 +297,7 @@ void TReplicaTest::Subscribe() { } void TReplicaTest::SubscribeUnknownPath() { - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); Context->SubscribeReplica(Replica, edge, "path", false); { @@ -310,10 +310,10 @@ void TReplicaTest::SubscribeUnknownPath() { void TReplicaTest::Unsubscribe() { auto describe = GenerateDescribe("path", TPathId(1, 1)); - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); - const TActorId subscriberA = Context->AllocateEdgeActor(); - const TActorId subscriberB = Context->AllocateEdgeActor(); + const TActorId subscriberA = Context->AllocateEdgeActor(); + const TActorId subscriberB = Context->AllocateEdgeActor(); Context->HandshakeReplica(Replica, edge, 1, 1); @@ -332,7 +332,7 @@ void TReplicaTest::Unsubscribe() { } void TReplicaTest::UnsubscribeUnknownPath() { - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); Context->UnsubscribeReplica(Replica, edge, "path"); } @@ -340,7 +340,7 @@ void TReplicaTest::UnsubscribeUnknownPath() { void TReplicaTest::DoubleUnsubscribe() { auto describe = GenerateDescribe("path", TPathId(1, 1)); - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); Context->HandshakeReplica(Replica, edge, 1, 1); Context->Send(Replica, edge, GenerateUpdate(describe, 1, 1)); @@ -356,7 +356,7 @@ void TReplicaTest::DoubleUnsubscribe() { void TReplicaTest::UnsubscribeWithoutSubscribe() { auto describe = GenerateDescribe("path", TPathId(1, 1)); - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); Context->HandshakeReplica(Replica, edge, 1, 1); Context->Send(Replica, edge, GenerateUpdate(describe, 1, 1)); @@ -367,9 +367,9 @@ void TReplicaTest::UnsubscribeWithoutSubscribe() { void TReplicaTest::Merge() { auto describe = GenerateDescribe("path", TPathId(1, 1)); - const TActorId populator = Context->AllocateEdgeActor(); - const TActorId subscriberA = Context->AllocateEdgeActor(); - const TActorId subscriberB = Context->AllocateEdgeActor(); + const TActorId populator = Context->AllocateEdgeActor(); + const TActorId subscriberA = Context->AllocateEdgeActor(); + const TActorId subscriberB = Context->AllocateEdgeActor(); Context->SubscribeReplica(Replica, subscriberA, "path"); Context->SubscribeReplica(Replica, subscriberB, TPathId(1, 1)); @@ -388,10 +388,10 @@ void TReplicaTest::Merge() { void TReplicaTest::DoubleDelete() { auto describe = GenerateDescribe("path", TPathId(1, 1)); - const TActorId populator = Context->AllocateEdgeActor(); + const TActorId populator = Context->AllocateEdgeActor(); Context->HandshakeReplica(Replica, populator); - const TActorId subscriberA = Context->AllocateEdgeActor(); + const TActorId subscriberA = Context->AllocateEdgeActor(); Context->SubscribeReplica(Replica, subscriberA, "path"); Context->Send(Replica, populator, GenerateUpdate(describe)); @@ -400,14 +400,14 @@ void TReplicaTest::DoubleDelete() { Context->Send(Replica, populator, GenerateUpdate(describe, 1, 1, true)); Context->GrabEdgeEvent<TSchemeBoardEvents::TEvNotify>(subscriberA); - const TActorId subscriberB = Context->AllocateEdgeActor(); + const TActorId subscriberB = Context->AllocateEdgeActor(); Context->SubscribeReplica(Replica, subscriberB, "path"); Context->Send(Replica, populator, GenerateUpdate(describe, 1, 1, true)); } void TReplicaTest::SyncVersion() { - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); const ui64 version = 100500; Context->HandshakeReplica(Replica, edge); @@ -425,10 +425,10 @@ void TReplicaTest::IdempotencyUpdates(bool aliveSubscriber) { auto describeA = GenerateDescribe("path", TPathId(1, 1)); auto describeB = GenerateDescribe("path", TPathId(1, 2)); // same path but another path id - const TActorId populator = Context->AllocateEdgeActor(); + const TActorId populator = Context->AllocateEdgeActor(); Context->HandshakeReplica(Replica, populator); - const TActorId subscriberA = Context->AllocateEdgeActor(); + const TActorId subscriberA = Context->AllocateEdgeActor(); Context->SubscribeReplica(Replica, subscriberA, TPathId(1, 1)); Context->Send(Replica, populator, GenerateUpdate(describeA)); @@ -441,7 +441,7 @@ void TReplicaTest::IdempotencyUpdates(bool aliveSubscriber) { Context->Send(Replica, populator, GenerateUpdate(describeA)); Context->Send(Replica, populator, GenerateUpdate(describeB)); - const TActorId subscriberB = Context->AllocateEdgeActor(); + const TActorId subscriberB = Context->AllocateEdgeActor(); Context->SubscribeReplica(Replica, subscriberB, TPathId(1, 2)); } @@ -457,7 +457,7 @@ void TReplicaTest::IdempotencyUpdatesVariant2() { auto describeA = GenerateDescribe("path", TPathId(1, 1)); auto describeB = GenerateDescribe("path", TPathId(1, 2)); // same path but another path id - const TActorId populator = Context->AllocateEdgeActor(); + const TActorId populator = Context->AllocateEdgeActor(); Context->HandshakeReplica(Replica, populator); Context->Send(Replica, populator, GenerateUpdate(describeA)); @@ -470,8 +470,8 @@ void TReplicaTest::IdempotencyUpdatesVariant2() { void TReplicaTest::AckNotifications() { auto describe = GenerateDescribe("path", TPathId(1, 1)); - const TActorId populator = Context->AllocateEdgeActor(); - const TActorId subscriber = Context->AllocateEdgeActor(); + const TActorId populator = Context->AllocateEdgeActor(); + const TActorId subscriber = Context->AllocateEdgeActor(); NKikimrSchemeBoard::TEvSubscribe::TCapabilities capabilities; capabilities.SetAckNotifications(true); @@ -530,8 +530,8 @@ void TReplicaTest::AckNotificationsUponPathRecreation() { } void TReplicaTest::StrongNotificationAfterCommit() { - const TActorId populator = Context->AllocateEdgeActor(); - const TActorId subscriber = Context->AllocateEdgeActor(); + const TActorId populator = Context->AllocateEdgeActor(); + const TActorId subscriber = Context->AllocateEdgeActor(); Context->SubscribeReplica(Replica, subscriber, "path", false, 1); { @@ -558,7 +558,7 @@ public: Context->SetLogPriority(NKikimrServices::SCHEME_BOARD_SUBSCRIBER, NLog::PRI_DEBUG); } - TActorId GetReplica() { + TActorId GetReplica() { return CreateReplica(*Context); } @@ -590,13 +590,13 @@ void TReplicaCombinationTest::UpdatesCombinationsDomainRoot() { //make all the variants for (const auto& argsLeft: combinations) { for (const auto& argsRight: combinations) { - const TActorId replica = GetReplica(); + const TActorId replica = GetReplica(); - const TActorId populatorLeft = Context->AllocateEdgeActor(); + const TActorId populatorLeft = Context->AllocateEdgeActor(); Context->HandshakeReplica(replica, populatorLeft, argsLeft.OwnerId, argsLeft.Generation); Context->CommitReplica(replica, populatorLeft, argsLeft.OwnerId, argsLeft.Generation); - const TActorId populatorRight = Context->AllocateEdgeActor(); + const TActorId populatorRight = Context->AllocateEdgeActor(); Context->HandshakeReplica(replica, populatorRight, argsRight.OwnerId, argsRight.Generation); Context->CommitReplica(replica, populatorRight, argsRight.OwnerId, argsRight.Generation); diff --git a/ydb/core/tx/scheme_board/subscriber.cpp b/ydb/core/tx/scheme_board/subscriber.cpp index 63123ba3df6..97173d81ad5 100644 --- a/ydb/core/tx/scheme_board/subscriber.cpp +++ b/ydb/core/tx/scheme_board/subscriber.cpp @@ -563,8 +563,8 @@ public: using TBase = TSubscriberProxy<TPath, TDerived, TReplicaDerived>; private: - const TActorId Parent; - const TActorId Replica; + const TActorId Parent; + const TActorId Replica; const TPath Path; const ui64 DomainOwnerId; @@ -617,7 +617,7 @@ class TSubscriber: public TMonitorableActor<TDerived> { return notify; } - TMap<TActorId, TState>::const_iterator SelectStateImpl() const { + TMap<TActorId, TState>::const_iterator SelectStateImpl() const { Y_VERIFY(!States.empty()); auto it = States.begin(); @@ -924,7 +924,7 @@ public: } explicit TSubscriber( - const TActorId& owner, + const TActorId& owner, const TPath& path, const ui64 stateStorageGroup, const ui64 domainOwnerId) @@ -940,7 +940,7 @@ public: void Bootstrap(const TActorContext&) { TMonitorableActor<TDerived>::Bootstrap(); - const TActorId proxy = MakeStateStorageProxyID(StateStorageGroup); + const TActorId proxy = MakeStateStorageProxyID(StateStorageGroup); this->Send(proxy, new TEvStateStorage::TEvResolveSchemeBoard(Path), IEventHandle::FlagTrackDelivery); this->Become(&TDerived::StateResolve); } @@ -981,14 +981,14 @@ public: using TBase = TSubscriber<TPath, TDerived, TProxyDerived>; private: - const TActorId Owner; + const TActorId Owner; const TPath Path; const ui64 StateStorageGroup; const ui64 DomainOwnerId; TSet<TActorId> Proxies; TMap<TActorId, TState> States; - TMap<TActorId, NKikimrSchemeBoard::TEvNotify> InitialResponses; + TMap<TActorId, NKikimrSchemeBoard::TEvNotify> InitialResponses; TMaybe<TState> State; ui64 DelayedSyncRequest; @@ -1011,7 +1011,7 @@ public: } // NSchemeBoard IActor* CreateSchemeBoardSubscriber( - const TActorId& owner, + const TActorId& owner, const TString& path, const ui64 stateStorageGroup, const ui64 domainOwnerId @@ -1020,7 +1020,7 @@ IActor* CreateSchemeBoardSubscriber( } IActor* CreateSchemeBoardSubscriber( - const TActorId& owner, + const TActorId& owner, const TPathId& pathId, const ui64 stateStorageGroup, const ui64 domainOwnerId @@ -1029,7 +1029,7 @@ IActor* CreateSchemeBoardSubscriber( } IActor* CreateSchemeBoardSubscriber( - const TActorId& owner, + const TActorId& owner, const TString& path, const ui64 stateStorageGroup, const EDeletionPolicy deletionPolicy @@ -1039,7 +1039,7 @@ IActor* CreateSchemeBoardSubscriber( } IActor* CreateSchemeBoardSubscriber( - const TActorId& owner, + const TActorId& owner, const TPathId& pathId, const ui64 stateStorageGroup, const EDeletionPolicy deletionPolicy diff --git a/ydb/core/tx/scheme_board/subscriber.h b/ydb/core/tx/scheme_board/subscriber.h index 53d58e4bc15..f415e4e9d57 100644 --- a/ydb/core/tx/scheme_board/subscriber.h +++ b/ydb/core/tx/scheme_board/subscriber.h @@ -9,14 +9,14 @@ namespace NKikimr { IActor* CreateSchemeBoardSubscriber( - const TActorId& owner, + const TActorId& owner, const TString& path, const ui64 stateStorageGroup, const ui64 domainOwnerId ); IActor* CreateSchemeBoardSubscriber( - const TActorId& owner, + const TActorId& owner, const TPathId& pathId, const ui64 stateStorageGroup, const ui64 domainOwnerId @@ -30,14 +30,14 @@ enum class ESchemeBoardSubscriberDeletionPolicy { }; IActor* CreateSchemeBoardSubscriber( - const TActorId& owner, + const TActorId& owner, const TString& path, const ui64 stateStorageGroup, const ESchemeBoardSubscriberDeletionPolicy deletionPolicy ); IActor* CreateSchemeBoardSubscriber( - const TActorId& owner, + const TActorId& owner, const TPathId& pathId, const ui64 stateStorageGroup, const ESchemeBoardSubscriberDeletionPolicy deletionPolicy diff --git a/ydb/core/tx/scheme_board/subscriber_ut.cpp b/ydb/core/tx/scheme_board/subscriber_ut.cpp index 4dd16bd0a09..7e1d580305f 100644 --- a/ydb/core/tx/scheme_board/subscriber_ut.cpp +++ b/ydb/core/tx/scheme_board/subscriber_ut.cpp @@ -18,16 +18,16 @@ namespace NKikimr { namespace NSchemeBoard { class TSubscriberTest: public NUnitTest::TTestBase { - TVector<TActorId> ResolveReplicas() { - const TActorId proxy = MakeStateStorageProxyID(0); - const TActorId edge = Context->AllocateEdgeActor(); + TVector<TActorId> ResolveReplicas() { + const TActorId proxy = MakeStateStorageProxyID(0); + const TActorId edge = Context->AllocateEdgeActor(); Context->Send(proxy, edge, new TEvStateStorage::TEvListSchemeBoard()); auto ev = Context->GrabEdgeEvent<TEvStateStorage::TEvListSchemeBoardResult>(edge); Y_VERIFY(ev->Get()->Info); auto allReplicas = ev->Get()->Info->SelectAllReplicas(); - return TVector<TActorId>(allReplicas.begin(), allReplicas.end()); + return TVector<TActorId>(allReplicas.begin(), allReplicas.end()); } public: @@ -76,7 +76,7 @@ private: UNIT_TEST_SUITE_REGISTRATION(TSubscriberTest); void TSubscriberTest::Boot() { - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); Context->CreateSubscriber(edge, "path"); auto ev = Context->GrabEdgeEvent<TSchemeBoardEvents::TEvNotifyDelete>(edge); @@ -86,7 +86,7 @@ void TSubscriberTest::Boot() { } void TSubscriberTest::NotifyUpdate() { - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); Context->CreateSubscriber<TSchemeBoardEvents::TEvNotifyDelete>(edge, "path"); @@ -101,7 +101,7 @@ void TSubscriberTest::NotifyUpdate() { } void TSubscriberTest::NotifyDelete() { - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); auto replicas = ResolveReplicas(); Y_VERIFY(replicas.size() > 2); @@ -124,7 +124,7 @@ void TSubscriberTest::NotifyDelete() { } void TSubscriberTest::StrongNotificationAfterCommit() { - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); Context->CreateSubscriber<TSchemeBoardEvents::TEvNotifyDelete>(edge, "path", 0, 1, false); { @@ -146,9 +146,9 @@ void TSubscriberTest::StrongNotificationAfterCommit() { } void TSubscriberTest::InvalidNotification() { - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); - const TActorId subscriber = Context->CreateSubscriber<TSchemeBoardEvents::TEvNotifyDelete>(edge, "path"); + const TActorId subscriber = Context->CreateSubscriber<TSchemeBoardEvents::TEvNotifyDelete>(edge, "path"); // send notification directly to subscriber auto* notify = new TSchemeBoardEvents::TEvNotifyBuilder(TPathId(1, 1)); @@ -160,7 +160,7 @@ void TSubscriberTest::InvalidNotification() { } void TSubscriberTest::ReconnectOnFailure() { - const TActorId edge = Context->AllocateEdgeActor(1); + const TActorId edge = Context->AllocateEdgeActor(1); Context->CreateSubscriber<TSchemeBoardEvents::TEvNotifyDelete>(edge, "path", 0, 1, true, 1); @@ -178,7 +178,7 @@ void TSubscriberTest::ReconnectOnFailure() { } void TSubscriberTest::Sync() { - const TActorId edge = Context->AllocateEdgeActor(); + const TActorId edge = Context->AllocateEdgeActor(); auto replicas = ResolveReplicas(); for (const auto& replica : replicas) { @@ -186,7 +186,7 @@ void TSubscriberTest::Sync() { Context->Send(replica, edge, GenerateUpdate(GenerateDescribe("path", TPathId(1, 1)))); } - const TActorId subscriber = Context->CreateSubscriber<TSchemeBoardEvents::TEvNotifyUpdate>(edge, "path"); + const TActorId subscriber = Context->CreateSubscriber<TSchemeBoardEvents::TEvNotifyUpdate>(edge, "path"); Context->Send(subscriber, edge, new TSchemeBoardEvents::TEvSyncRequest(), 0, 1); auto ev = Context->GrabEdgeEvent<TSchemeBoardEvents::TEvSyncResponse>(edge); @@ -196,8 +196,8 @@ void TSubscriberTest::Sync() { } void TSubscriberTest::SyncPartial() { - const TActorId edge = Context->AllocateEdgeActor(); - const TActorId subscriber = Context->CreateSubscriber<TSchemeBoardEvents::TEvNotifyDelete>(edge, "path"); + const TActorId edge = Context->AllocateEdgeActor(); + const TActorId subscriber = Context->CreateSubscriber<TSchemeBoardEvents::TEvNotifyDelete>(edge, "path"); ui64 syncCookie = 0; auto replicas = ResolveReplicas(); @@ -250,16 +250,16 @@ void TSubscriberTest::SyncWithOutdatedReplica() { } class TSubscriberCombinationsTest: public NUnitTest::TTestBase { - TVector<TActorId> ResolveReplicas(TTestContext& context) { - const TActorId proxy = MakeStateStorageProxyID(0); - const TActorId edge = context.AllocateEdgeActor(); + TVector<TActorId> ResolveReplicas(TTestContext& context) { + const TActorId proxy = MakeStateStorageProxyID(0); + const TActorId edge = context.AllocateEdgeActor(); context.Send(proxy, edge, new TEvStateStorage::TEvListSchemeBoard()); auto ev = context.GrabEdgeEvent<TEvStateStorage::TEvListSchemeBoardResult>(edge); Y_VERIFY(ev->Get()->Info); auto allReplicas = ev->Get()->Info->SelectAllReplicas(); - return TVector<TActorId>(allReplicas.begin(), allReplicas.end()); + return TVector<TActorId>(allReplicas.begin(), allReplicas.end()); } THolder<TTestContext> CreateContext() { @@ -303,18 +303,18 @@ void TSubscriberCombinationsTest::CombinationsRootDomain() { auto context = CreateContext(); - TVector<TActorId> replicas = ResolveReplicas(*context); + TVector<TActorId> replicas = ResolveReplicas(*context); Y_ASSERT(replicas.size() >= 2); - const TActorId populatorLeft = context->AllocateEdgeActor(); + const TActorId populatorLeft = context->AllocateEdgeActor(); context->HandshakeReplica(replicas[0], populatorLeft, argsLeft.OwnerId, argsLeft.Generation); context->CommitReplica(replicas[0], populatorLeft, argsLeft.OwnerId, argsLeft.Generation); - const TActorId populatorRight = context->AllocateEdgeActor(); + const TActorId populatorRight = context->AllocateEdgeActor(); context->HandshakeReplica(replicas[1], populatorRight, argsRight.OwnerId, argsRight.Generation); context->CommitReplica(replicas[1], populatorRight, argsRight.OwnerId, argsRight.Generation); - const TActorId edge = context->AllocateEdgeActor(); + const TActorId edge = context->AllocateEdgeActor(); context->CreateSubscriber<TSchemeBoardEvents::TEvNotifyDelete>(edge, path); context->Send(replicas[0], populatorLeft, argsLeft.GenerateUpdate()); diff --git a/ydb/core/tx/scheme_board/ut_helpers.h b/ydb/core/tx/scheme_board/ut_helpers.h index 7cb73d51446..1f2e4d5e9b8 100644 --- a/ydb/core/tx/scheme_board/ut_helpers.h +++ b/ydb/core/tx/scheme_board/ut_helpers.h @@ -28,8 +28,8 @@ public: using TTestBasicRuntime::TTestBasicRuntime; void Send( - const TActorId& recipient, - const TActorId& sender, + const TActorId& recipient, + const TActorId& sender, IEventBase* ev, ui32 flags = 0, ui64 cookie = 0, @@ -46,16 +46,16 @@ public: } void Connect(ui32 nodeIndexFrom, ui32 nodeIndexTo) { - const TActorId proxy = GetInterconnectProxy(nodeIndexFrom, nodeIndexTo); + const TActorId proxy = GetInterconnectProxy(nodeIndexFrom, nodeIndexTo); - Send(proxy, TActorId(), new TEvInterconnect::TEvConnectNode(), 0, 0, nodeIndexFrom, true); + Send(proxy, TActorId(), new TEvInterconnect::TEvConnectNode(), 0, 0, nodeIndexFrom, true); WaitForEvent(TEvInterconnect::EvNodeConnected); } void Disconnect(ui32 nodeIndexFrom, ui32 nodeIndexTo) { - const TActorId proxy = GetInterconnectProxy(nodeIndexFrom, nodeIndexTo); + const TActorId proxy = GetInterconnectProxy(nodeIndexFrom, nodeIndexTo); - Send(proxy, TActorId(), new TEvInterconnect::TEvDisconnect(), 0, 0, nodeIndexFrom, true); + Send(proxy, TActorId(), new TEvInterconnect::TEvDisconnect(), 0, 0, nodeIndexFrom, true); WaitForEvent(TEvInterconnect::EvNodeDisconnected); } @@ -78,8 +78,8 @@ public: } TSchemeBoardEvents::TEvHandshakeResponse::TPtr HandshakeReplica( - const TActorId& replica, - const TActorId& sender, + const TActorId& replica, + const TActorId& sender, ui64 owner = 1, ui64 generation = 1, bool grabResponse = true @@ -94,8 +94,8 @@ public: } void CommitReplica( - const TActorId& replica, - const TActorId& sender, + const TActorId& replica, + const TActorId& sender, ui64 owner = 1, ui64 generation = 1 ) { @@ -104,8 +104,8 @@ public: template <typename TPath> TSchemeBoardEvents::TEvNotify::TPtr SubscribeReplica( - const TActorId& replica, - const TActorId& sender, + const TActorId& replica, + const TActorId& sender, const TPath& path, bool grabResponse = true, const ui64 domainOwnerId = 0, @@ -124,20 +124,20 @@ public: } template <typename TPath> - void UnsubscribeReplica(const TActorId& replica, const TActorId& sender, const TPath& path) { + void UnsubscribeReplica(const TActorId& replica, const TActorId& sender, const TPath& path) { Send(replica, sender, new TSchemeBoardEvents::TEvUnsubscribe(path)); } template <typename TEvent, typename TPath> - TActorId CreateSubscriber( - const TActorId& owner, + TActorId CreateSubscriber( + const TActorId& owner, const TPath& path, ui64 stateStorageGroup = 0, ui64 domainOwnerId = 1, bool grabResponse = true, ui32 nodeIndex = 0 ) { - const TActorId subscriber = Register( + const TActorId subscriber = Register( CreateSchemeBoardSubscriber(owner, path, stateStorageGroup, domainOwnerId), nodeIndex ); EnableScheduleForActor(subscriber, true); @@ -150,8 +150,8 @@ public: } template <typename TPath> - TActorId CreateSubscriber( - const TActorId& owner, + TActorId CreateSubscriber( + const TActorId& owner, const TPath& path, ui64 stateStorageGroup = 0, ui64 domainOwnerId = 1, @@ -211,7 +211,7 @@ class TTestWithSchemeshard: public NUnitTest::TTestBase { CreateTestBootstrapper(runtime, CreateTestTabletInfo(tabletId, TTabletTypes::FLAT_SCHEMESHARD), &CreateFlatTxSchemeShard); - const TActorId edge = runtime.AllocateEdgeActor(); + const TActorId edge = runtime.AllocateEdgeActor(); auto init = new TEvSchemeShard::TEvInitRootShard(edge, 32, "Root"); runtime.SendToPipe(tabletId, edge, init, 0, GetPipeConfigWithRetries()); @@ -258,7 +258,7 @@ public: return; } - TActorId sender = Context->AllocateEdgeActor(); + TActorId sender = Context->AllocateEdgeActor(); TVector<ui64> tabletIds; tabletIds.push_back((ui64)TTestTxConfig::SchemeShard); for (auto x: xrange(TTestTxConfig::FakeHiveTablets, TTestTxConfig::FakeHiveTablets + 10)) { diff --git a/ydb/core/tx/scheme_cache/scheme_cache.h b/ydb/core/tx/scheme_cache/scheme_cache.h index 1d76b585571..26a07897a11 100644 --- a/ydb/core/tx/scheme_cache/scheme_cache.h +++ b/ydb/core/tx/scheme_cache/scheme_cache.h @@ -334,7 +334,7 @@ struct TSchemeCacheRequest { }; // TSchemeCacheRequest struct TSchemeCacheRequestContext : TAtomicRefCount<TSchemeCacheRequestContext>, TNonCopyable { - TActorId Sender; + TActorId Sender; ui64 WaitCounter; TAutoPtr<TSchemeCacheRequest> Request; const TInstant CreatedAt; @@ -348,7 +348,7 @@ struct TSchemeCacheRequestContext : TAtomicRefCount<TSchemeCacheRequestContext>, }; struct TSchemeCacheNavigateContext : TAtomicRefCount<TSchemeCacheNavigateContext>, TNonCopyable { - TActorId Sender; + TActorId Sender; ui64 WaitCounter; TAutoPtr<TSchemeCacheNavigate> Request; const TInstant CreatedAt; @@ -448,18 +448,18 @@ public: struct TEvInvalidateTable : public TEventLocal<TEvInvalidateTable, EvInvalidateTable> { const TTableId TableId; - const TActorId Sender; + const TActorId Sender; - TEvInvalidateTable(const TTableId& tableId, const TActorId& sender) + TEvInvalidateTable(const TTableId& tableId, const TActorId& sender) : TableId(tableId) , Sender(sender) {} }; struct TEvInvalidateTableResult : public TEventLocal<TEvInvalidateTableResult, EvInvalidateTableResult> { - const TActorId Sender; + const TActorId Sender; - TEvInvalidateTableResult(const TActorId& sender) + TEvInvalidateTableResult(const TActorId& sender) : Sender(sender) {} }; diff --git a/ydb/core/tx/schemeshard/schemeshard.cpp b/ydb/core/tx/schemeshard/schemeshard.cpp index 890e0e3b1d7..24b73f590cc 100644 --- a/ydb/core/tx/schemeshard/schemeshard.cpp +++ b/ydb/core/tx/schemeshard/schemeshard.cpp @@ -45,7 +45,7 @@ namespace NSchemeShard { } } -IActor* CreateFlatTxSchemeShard(const TActorId &tablet, TTabletStorageInfo *info) { +IActor* CreateFlatTxSchemeShard(const TActorId &tablet, TTabletStorageInfo *info) { return new NSchemeShard::TSchemeShard(tablet, info); } diff --git a/ydb/core/tx/schemeshard/schemeshard.h b/ydb/core/tx/schemeshard/schemeshard.h index fa9032fc758..d33173508ce 100644 --- a/ydb/core/tx/schemeshard/schemeshard.h +++ b/ydb/core/tx/schemeshard/schemeshard.h @@ -184,9 +184,9 @@ struct TEvSchemeShard { TEvInitRootShard() {} - TEvInitRootShard(const TActorId& source, ui32 rootTag, const TString& rootTagName) + TEvInitRootShard(const TActorId& source, ui32 rootTag, const TString& rootTagName) { - ActorIdToProto(source, Record.MutableSource()); + ActorIdToProto(source, Record.MutableSource()); Record.SetRootTag(rootTag); Record.SetRootTagName(rootTagName); } @@ -540,7 +540,7 @@ struct TEvSchemeShard { } -IActor* CreateFlatTxSchemeShard(const TActorId &tablet, TTabletStorageInfo *info); +IActor* CreateFlatTxSchemeShard(const TActorId &tablet, TTabletStorageInfo *info); bool PartitionConfigHasExternalBlobsEnabled(const NKikimrSchemeOp::TPartitionConfig &partitionConfig); } diff --git a/ydb/core/tx/schemeshard/schemeshard__describe_scheme.cpp b/ydb/core/tx/schemeshard/schemeshard__describe_scheme.cpp index e5085e5c5fd..e6d96356527 100644 --- a/ydb/core/tx/schemeshard/schemeshard__describe_scheme.cpp +++ b/ydb/core/tx/schemeshard/schemeshard__describe_scheme.cpp @@ -9,7 +9,7 @@ namespace NSchemeShard { using namespace NTabletFlatExecutor; struct TSchemeShard::TTxDescribeScheme : public TSchemeShard::TRwTxBase { - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; TPathDescriber PathDescriber; diff --git a/ydb/core/tx/schemeshard/schemeshard__operation.cpp b/ydb/core/tx/schemeshard/schemeshard__operation.cpp index 4df460c83b7..daf1218bcf7 100644 --- a/ydb/core/tx/schemeshard/schemeshard__operation.cpp +++ b/ydb/core/tx/schemeshard/schemeshard__operation.cpp @@ -256,7 +256,7 @@ struct TSchemeShard::TTxOperationPropose: public NTabletFlatExecutor::TTransacti << ", response: " << Response->Record.ShortDebugString() << ", at schemeshard: " << Self->TabletID()); - const TActorId sender = Request->Sender; + const TActorId sender = Request->Sender; const ui64 cookie = Request->Cookie; ctx.Send(sender, Response.Release(), 0, cookie); @@ -405,7 +405,7 @@ struct TSchemeShard::TTxOperationPlanStep: public NTabletFlatExecutor::TTransact for (size_t i = 0; i < txCount; ++i) { const auto txId = TTxId(record.GetTransactions(i).GetTxId()); - const auto coordinator = ActorIdFromProto(record.GetTransactions(i).GetAckTo()); + const auto coordinator = ActorIdFromProto(record.GetTransactions(i).GetAckTo()); const auto coordinatorId = TTabletId(record.GetTransactions(i).GetCoordinator()); if (!Self->Operations.contains(txId)) { @@ -443,7 +443,7 @@ struct TSchemeShard::TTxOperationPlanStep: public NTabletFlatExecutor::TTransact OnComplete.UnbindMsgFromPipe(TOperationId(txId, InvalidSubTxId), coordinatorId, TPipeMessageId(0, txId)); } - const TActorId mediator = Ev->Sender; + const TActorId mediator = Ev->Sender; OnComplete.MediatorAck(mediator, step); OnComplete.ApplyOnExecute(Self, txc, ctx); @@ -1162,7 +1162,7 @@ bool TOperation::IsReadyToNotify() const { return IsPublished() && ReadyToNotifyParts.size() == Parts.size(); } -void TOperation::AddNotifySubscriber(const TActorId& actorId) { +void TOperation::AddNotifySubscriber(const TActorId& actorId) { Y_VERIFY(!IsReadyToNotify()); Subscribers.insert(actorId); } diff --git a/ydb/core/tx/schemeshard/schemeshard__operation.h b/ydb/core/tx/schemeshard/schemeshard__operation.h index 613985eb4d6..52f1848aa12 100644 --- a/ydb/core/tx/schemeshard/schemeshard__operation.h +++ b/ydb/core/tx/schemeshard/schemeshard__operation.h @@ -14,7 +14,7 @@ struct TOperation: TSimpleRefCount<TOperation> { TTxId TxId; TVector<ISubOperationBase::TPtr> Parts; - THashSet<TActorId> Subscribers; + THashSet<TActorId> Subscribers; THashSet<TTxId> DependentOperations; THashSet<TTxId> WaitOperations; @@ -80,7 +80,7 @@ struct TOperation: TSimpleRefCount<TOperation> { void ReadyToNotifyPart(TSubTxId partId); bool IsReadyToNotify(const TActorContext& ctx) const; bool IsReadyToNotify() const; - void AddNotifySubscriber(const TActorId& actorId); + void AddNotifySubscriber(const TActorId& actorId); void DoNotify(TSchemeShard* ss, TSideEffects& sideEffects, const TActorContext& ctx); bool IsReadyToDone(const TActorContext& ctx) const; diff --git a/ydb/core/tx/schemeshard/schemeshard__operation_common.cpp b/ydb/core/tx/schemeshard/schemeshard__operation_common.cpp index 446630112e7..5a88636ffae 100644 --- a/ydb/core/tx/schemeshard/schemeshard__operation_common.cpp +++ b/ydb/core/tx/schemeshard/schemeshard__operation_common.cpp @@ -118,7 +118,7 @@ bool NTableState::CollectSchemaChanged( auto ssId = context.SS->SelfTabletId(); const auto& evRecord = ev->Get()->Record; - const TActorId ackTo = ev->Get()->GetSource(); + const TActorId ackTo = ev->Get()->GetSource(); auto datashardId = TTabletId(evRecord.GetOrigin()); @@ -193,7 +193,7 @@ void NTableState::AckAllSchemaChanges(const TOperationId &operationId, TTxState // Ack to all participating datashards for (const auto& items : txState.SchemeChangeNotificationReceived) { - const TActorId ackTo = items.second.first; + const TActorId ackTo = items.second.first; const auto shardIdx = items.first; const auto tabletId = context.SS->ShardInfos[shardIdx].TabletID; diff --git a/ydb/core/tx/schemeshard/schemeshard__operation_common.h b/ydb/core/tx/schemeshard/schemeshard__operation_common.h index 20fb8632d0b..5a3da32cade 100644 --- a/ydb/core/tx/schemeshard/schemeshard__operation_common.h +++ b/ydb/core/tx/schemeshard/schemeshard__operation_common.h @@ -18,7 +18,7 @@ bool CollectProposeTransactionResults(const TOperationId& operationId, const TEv bool CollectProposeTransactionResults(const TOperationId& operationId, const TEvColumnShard::TEvProposeTransactionResult::TPtr& ev, TOperationContext& context); bool CollectSchemaChanged(const TOperationId& operationId, const TEvDataShard::TEvSchemaChanged::TPtr& ev, TOperationContext& context); -void SendSchemaChangedNotificationAck(const TOperationId& operationId, TActorId ackTo, TShardIdx shardIdx, TOperationContext& context); +void SendSchemaChangedNotificationAck(const TOperationId& operationId, TActorId ackTo, TShardIdx shardIdx, TOperationContext& context); void AckAllSchemaChanges(const TOperationId& operationId, TTxState& txState, TOperationContext& context); bool CheckPartitioningChangedForTableModification(TTxState& txState, TOperationContext& context); diff --git a/ydb/core/tx/schemeshard/schemeshard__operation_drop_table.cpp b/ydb/core/tx/schemeshard/schemeshard__operation_drop_table.cpp index 88e163e8adc..45730be4095 100644 --- a/ydb/core/tx/schemeshard/schemeshard__operation_drop_table.cpp +++ b/ydb/core/tx/schemeshard/schemeshard__operation_drop_table.cpp @@ -406,7 +406,7 @@ public: bool HandleReply(TEvDataShard::TEvSchemaChanged::TPtr& ev, TOperationContext& context) override { TTabletId ssId = context.SS->SelfTabletId(); - const TActorId& ackTo = ev->Sender; + const TActorId& ackTo = ev->Sender; LOG_INFO_S(context.Ctx, NKikimrServices::FLAT_TX_SCHEMESHARD, DebugHint() << " HandleReply TProposedDeletePart" diff --git a/ydb/core/tx/schemeshard/schemeshard__publish_to_scheme_board.cpp b/ydb/core/tx/schemeshard/schemeshard__publish_to_scheme_board.cpp index d6ef6e4d0ca..cb9a270d256 100644 --- a/ydb/core/tx/schemeshard/schemeshard__publish_to_scheme_board.cpp +++ b/ydb/core/tx/schemeshard/schemeshard__publish_to_scheme_board.cpp @@ -201,7 +201,7 @@ private: return paths.empty(); } - void Notify(TTxId txId, const THashSet<TActorId>& subscribers, const TActorContext& ctx) { + void Notify(TTxId txId, const THashSet<TActorId>& subscribers, const TActorContext& ctx) { for (const auto& subscriber : subscribers) { LOG_DEBUG_S(ctx, NKikimrServices::FLAT_TX_SCHEMESHARD, "TTxAckPublishToSchemeBoard Notify" diff --git a/ydb/core/tx/schemeshard/schemeshard__upgrade_access_database.cpp b/ydb/core/tx/schemeshard/schemeshard__upgrade_access_database.cpp index 4a5e5f1ea26..4864dc9db36 100644 --- a/ydb/core/tx/schemeshard/schemeshard__upgrade_access_database.cpp +++ b/ydb/core/tx/schemeshard/schemeshard__upgrade_access_database.cpp @@ -7,13 +7,13 @@ using namespace NTabletFlatExecutor; struct TSchemeShard::TTxUpgradeAccessDatabaseRights : public TTransactionBase<TSchemeShard> { bool IsDryRun; - TActorId AnswerTo; + TActorId AnswerTo; std::function< NActors::IEventBase* (const TMap<TPathId, TSet<TString>>&) > AnswerFunc; TSideEffects SideEffects; TTxUpgradeAccessDatabaseRights(TSelf* self, const bool isDryRun, - const TActorId& answerTo, + const TActorId& answerTo, std::function< NActors::IEventBase* (const TMap<TPathId, TSet<TString>>&) > answerFunc) : TTransactionBase<TSchemeShard>(self) , IsDryRun(isDryRun) @@ -145,7 +145,7 @@ struct TSchemeShard::TTxUpgradeAccessDatabaseRights : public TTransactionBase<TS }; NTabletFlatExecutor::ITransaction* TSchemeShard::CreateTxUpgradeAccessDatabaseRights( - const TActorId& answerTo, bool isDryRun, std::function< NActors::IEventBase*(const TMap<TPathId, TSet<TString>>&) > func) { + const TActorId& answerTo, bool isDryRun, std::function< NActors::IEventBase*(const TMap<TPathId, TSet<TString>>&) > func) { return new TTxUpgradeAccessDatabaseRights(this, isDryRun, answerTo, func); } diff --git a/ydb/core/tx/schemeshard/schemeshard_build_index_tx_base.h b/ydb/core/tx/schemeshard/schemeshard_build_index_tx_base.h index 6740c4f3494..f2e4d9e56bd 100644 --- a/ydb/core/tx/schemeshard/schemeshard_build_index_tx_base.h +++ b/ydb/core/tx/schemeshard/schemeshard_build_index_tx_base.h @@ -311,7 +311,7 @@ protected: << ": id# " << indexInfo->Id << ", subscribers count# " << indexInfo->Subscribers.size()); - TSet<TActorId> toAnswer; + TSet<TActorId> toAnswer; toAnswer.swap(indexInfo->Subscribers); for (auto& actorId: toAnswer) { Send(actorId, MakeHolder<TEvSchemeShard::TEvNotifyTxCompletionResult>(ui64(indexInfo->Id))); diff --git a/ydb/core/tx/schemeshard/schemeshard_domain_links.cpp b/ydb/core/tx/schemeshard/schemeshard_domain_links.cpp index 15b3425a443..b50aaa509a3 100644 --- a/ydb/core/tx/schemeshard/schemeshard_domain_links.cpp +++ b/ydb/core/tx/schemeshard/schemeshard_domain_links.cpp @@ -53,20 +53,20 @@ void TParentDomainLink::SendSync(const TActorContext &ctx) { void TParentDomainLink::AtPipeError(const TActorContext &ctx) { if (Pipe) { NTabletPipe::CloseClient(ctx, Pipe); - Pipe = TActorId(); + Pipe = TActorId(); } SendSync(ctx); } -bool TParentDomainLink::HasPipeTo(TTabletId tabletId, TActorId clientId) { +bool TParentDomainLink::HasPipeTo(TTabletId tabletId, TActorId clientId) { return TTabletId(Self->ParentDomainId.OwnerId) == tabletId && Pipe == clientId; } void TParentDomainLink::Shutdown(const NActors::TActorContext &ctx) { if (Pipe) { NTabletPipe::CloseClient(ctx, Pipe); - Pipe = TActorId(); + Pipe = TActorId(); } } @@ -74,7 +74,7 @@ bool TSubDomainsLinks::Sync(TEvSchemeShard::TEvSyncTenantSchemeShard::TPtr &ev, Y_VERIFY(Self->IsDomainSchemeShard); const auto& record = ev->Get()->Record; - const TActorId actorId = ev->Sender; + const TActorId actorId = ev->Sender; const TPathId pathId = Self->MakeLocalId(record.GetDomainPathId()); const ui64 generation = record.GetGeneration(); diff --git a/ydb/core/tx/schemeshard/schemeshard_domain_links.h b/ydb/core/tx/schemeshard/schemeshard_domain_links.h index 594ecc8cbe8..d9cb6133c42 100644 --- a/ydb/core/tx/schemeshard/schemeshard_domain_links.h +++ b/ydb/core/tx/schemeshard/schemeshard_domain_links.h @@ -16,7 +16,7 @@ class TSchemeShard; class TParentDomainLink { TSchemeShard* Self; - TActorId Pipe; + TActorId Pipe; NTabletPipe::TClientConfig PipeClientConfig; public: @@ -25,7 +25,7 @@ public: THolder<TEvSchemeShard::TEvSyncTenantSchemeShard> MakeSyncMsg() const; void SendSync(const TActorContext& ctx); void AtPipeError(const TActorContext& ctx); - bool HasPipeTo(TTabletId tabletId, TActorId clientId); + bool HasPipeTo(TTabletId tabletId, TActorId clientId); }; @@ -34,7 +34,7 @@ public: struct TLink { TPathId DomainKey = InvalidPathId; ui64 Generation = 0; - TActorId ActorId; + TActorId ActorId; ui64 EffectiveACLVersion = 0; ui64 SubdomainVersion = 0; diff --git a/ydb/core/tx/schemeshard/schemeshard_impl.cpp b/ydb/core/tx/schemeshard/schemeshard_impl.cpp index 5c9f0149985..0715543a229 100644 --- a/ydb/core/tx/schemeshard/schemeshard_impl.cpp +++ b/ydb/core/tx/schemeshard/schemeshard_impl.cpp @@ -4400,7 +4400,7 @@ void TSchemeShard::Handle(TEvDataShard::TEvSchemaChanged::TPtr& ev, const TActor const auto txId = TTxId(ev->Get()->Record.GetTxId()); const auto tableId = TTabletId(ev->Get()->Record.GetOrigin()); - TActorId ackTo = ev->Get()->GetSource(); + TActorId ackTo = ev->Get()->GetSource(); if (!Operations.contains(txId)) { LOG_WARN_S(ctx, NKikimrServices::FLAT_TX_SCHEMESHARD, @@ -4533,7 +4533,7 @@ void TSchemeShard::Handle(TEvPrivate::TEvProgressOperation::TPtr &ev, const TAct void TSchemeShard::Handle(TEvTabletPipe::TEvClientConnected::TPtr &ev, const TActorContext &ctx) { const auto tabletId = TTabletId(ev->Get()->TabletId); - const TActorId clientId = ev->Get()->ClientId; + const TActorId clientId = ev->Get()->ClientId; LOG_TRACE_S(ctx, NKikimrServices::FLAT_TX_SCHEMESHARD, "Handle TEvClientConnected" @@ -4580,7 +4580,7 @@ void TSchemeShard::Handle(TEvTabletPipe::TEvServerConnected::TPtr &ev, const TAc void TSchemeShard::Handle(TEvTabletPipe::TEvClientDestroyed::TPtr &ev, const TActorContext &ctx) { const auto tabletId = TTabletId(ev->Get()->TabletId); - const TActorId clientId = ev->Get()->ClientId; + const TActorId clientId = ev->Get()->ClientId; LOG_TRACE_S(ctx, NKikimrServices::FLAT_TX_SCHEMESHARD, "Client pipe" @@ -5856,7 +5856,7 @@ TSchemeShard::TDedicatedPipePool::TDedicatedPipePool() { void TSchemeShard::TDedicatedPipePool::Create(TIndexBuildId ownerTxId, TTabletId dst, THolder<IEventBase> message, const TActorContext &ctx) { Y_VERIFY(!Pipes[ownerTxId].contains(dst)); - TActorId clientId = ctx.ExecutorThread.RegisterActor(NTabletPipe::CreateClient(ctx.SelfID, ui64(dst), PipeCfg)); + TActorId clientId = ctx.ExecutorThread.RegisterActor(NTabletPipe::CreateClient(ctx.SelfID, ui64(dst), PipeCfg)); Pipes[ownerTxId][dst] = clientId; Owners[clientId] = TOwnerRec(ownerTxId, dst); @@ -5873,7 +5873,7 @@ void TSchemeShard::TDedicatedPipePool::Close(TIndexBuildId ownerTxId, TTabletId return; } - TActorId actorId = Pipes.at(ownerTxId).at(dst); + TActorId actorId = Pipes.at(ownerTxId).at(dst); NTabletPipe::CloseClient(ctx, actorId); Owners.erase(actorId); diff --git a/ydb/core/tx/schemeshard/schemeshard_impl.h b/ydb/core/tx/schemeshard/schemeshard_impl.h index 9efc6bf40de..e1a1d08b9a6 100644 --- a/ydb/core/tx/schemeshard/schemeshard_impl.h +++ b/ydb/core/tx/schemeshard/schemeshard_impl.h @@ -186,11 +186,11 @@ public: THashMap<TTabletId, TShardIdx> TabletIdToShardIdx; THashMap<TShardIdx, TVector<TActorId>> ShardDeletionSubscribers; // for tests - TActorId SchemeBoardPopulator; + TActorId SchemeBoardPopulator; static constexpr ui32 InitiateCachedTxIdsCount = 100; TDeque<TTxId> CachedTxIds; - TActorId TxAllocatorClient; + TActorId TxAllocatorClient; TAutoPtr<NTabletPipe::IClientCache> PipeClientCache; TPipeTracker PipeTracker; @@ -209,7 +209,7 @@ public: TAutoPtr<TSelfPinger> SelfPinger; - TActorId SysPartitionStatsCollector; + TActorId SysPartitionStatsCollector; TSet<TPathId> CleanDroppedPathsCandidates; TSet<TPathId> CleanDroppedSubDomainsCandidates; @@ -689,7 +689,7 @@ public: NTabletFlatExecutor::ITransaction* CreateTxOperationPlanStep(TEvTxProcessing::TEvPlanStep::TPtr& ev); struct TTxUpgradeAccessDatabaseRights; - NTabletFlatExecutor::ITransaction* CreateTxUpgradeAccessDatabaseRights(const TActorId& answerTo, bool isDryRun, std::function< NActors::IEventBase* (const TMap<TPathId, TSet<TString>>&) >); + NTabletFlatExecutor::ITransaction* CreateTxUpgradeAccessDatabaseRights(const TActorId& answerTo, bool isDryRun, std::function< NActors::IEventBase* (const TMap<TPathId, TSet<TString>>&) >); struct TTxMakeAccessDatabaseNoInheritable; NTabletFlatExecutor::ITransaction* CreateTxMakeAccessDatabaseNoInheritable(const TActorId& answerTo, bool isDryRun, std::function< NActors::IEventBase* (const TMap<TPathId, TSet<TString>>&) >); @@ -962,7 +962,7 @@ public: NTabletPipe::TClientConfig PipeCfg; TMap<TIndexBuildId, TMap<TTabletId, TActorId>> Pipes; - TMap<TActorId, TOwnerRec> Owners; + TMap<TActorId, TOwnerRec> Owners; TDedicatedPipePool(); @@ -970,7 +970,7 @@ public: void Close(TIndexBuildId ownerTxId, TTabletId dst, const TActorContext& ctx); ui64 CloseAll(TIndexBuildId ownerTxId, const TActorContext& ctx); - bool Has(TActorId actorId) const; + bool Has(TActorId actorId) const; TTabletId GetTabletId(TActorId actorId) const; TIndexBuildId GetOwnerId(TActorId actorId) const; }; diff --git a/ydb/core/tx/schemeshard/schemeshard_info_types.cpp b/ydb/core/tx/schemeshard/schemeshard_info_types.cpp index 136d1c34be5..fcb564b6065 100644 --- a/ydb/core/tx/schemeshard/schemeshard_info_types.cpp +++ b/ydb/core/tx/schemeshard/schemeshard_info_types.cpp @@ -1680,7 +1680,7 @@ bool TExportInfo::AllItemsAreDropped() const { return AllOf(Items, &TExportInfo::TItem::IsDropped); } -void TExportInfo::AddNotifySubscriber(const TActorId &actorId) { +void TExportInfo::AddNotifySubscriber(const TActorId &actorId) { Y_VERIFY(!IsFinished()); Subscribers.insert(actorId); } diff --git a/ydb/core/tx/schemeshard/schemeshard_info_types.h b/ydb/core/tx/schemeshard/schemeshard_info_types.h index 7b6bff34873..2e5710c3d3e 100644 --- a/ydb/core/tx/schemeshard/schemeshard_info_types.h +++ b/ydb/core/tx/schemeshard/schemeshard_info_types.h @@ -2212,7 +2212,7 @@ struct TReplicationInfo : public TSimpleRefCount<TReplicationInfo> { struct TPublicationInfo { TSet<std::pair<TPathId, ui64>> Paths; - THashSet<TActorId> Subscribers; + THashSet<TActorId> Subscribers; }; // namespace NExport { @@ -2282,7 +2282,7 @@ struct TExportInfo: public TSimpleRefCount<TExportInfo> { TDeque<ui32> PendingItems; TDeque<ui32> PendingDropItems; - TSet<TActorId> Subscribers; + TSet<TActorId> Subscribers; explicit TExportInfo( const ui64 id, @@ -2346,7 +2346,7 @@ struct TExportInfo: public TSimpleRefCount<TExportInfo> { } bool AllItemsAreDropped() const; - void AddNotifySubscriber(const TActorId& actorId); + void AddNotifySubscriber(const TActorId& actorId); TString ToString() const; @@ -2512,7 +2512,7 @@ struct TIndexBuildInfo: public TSimpleRefCount<TIndexBuildInfo> { Rejected = 550 }; - TActorId CreateSender; + TActorId CreateSender; ui64 SenderCookie = 0; TIndexBuildId Id; @@ -2532,7 +2532,7 @@ struct TIndexBuildInfo: public TSimpleRefCount<TIndexBuildInfo> { EState State = EState::Invalid; TString Issue; - TSet<TActorId> Subscribers; + TSet<TActorId> Subscribers; bool CancelRequested = false; @@ -2625,7 +2625,7 @@ struct TIndexBuildInfo: public TSimpleRefCount<TIndexBuildInfo> { return IsDone() || IsCancelled(); } - void AddNotifySubscriber(const TActorId& actorID) { + void AddNotifySubscriber(const TActorId& actorID) { Y_VERIFY(!IsFinished()); Subscribers.insert(actorID); } diff --git a/ydb/core/tx/schemeshard/schemeshard_tx_infly.h b/ydb/core/tx/schemeshard/schemeshard_tx_infly.h index 71c44de7899..41a8045155a 100644 --- a/ydb/core/tx/schemeshard/schemeshard_tx_infly.h +++ b/ydb/core/tx/schemeshard/schemeshard_tx_infly.h @@ -16,9 +16,9 @@ namespace NSchemeShard { struct TNotifyes { TTxId TxId = InvalidTxId; - THashSet<TActorId> Actors; + THashSet<TActorId> Actors; - void Add(const TActorId& actor, TTxId txId) { + void Add(const TActorId& actor, TTxId txId) { Y_VERIFY(!TxId || TxId == txId); TxId = txId; Actors.insert(actor); @@ -235,7 +235,7 @@ struct TTxState { TVector<TShardOperation> Shards; // shards + operations on them // not persist: THashSet<TShardIdx> ShardsInProgress; // indexes of datashards or pqs that operation waits for - THashMap<TShardIdx, std::pair<TActorId, ui32>> SchemeChangeNotificationReceived; + THashMap<TShardIdx, std::pair<TActorId, ui32>> SchemeChangeNotificationReceived; bool ReadyForNotifications = false; std::shared_ptr<NKikimrTxDataShard::TSplitMergeDescription> SplitDescription; bool TxShardsListFinalized = false; diff --git a/ydb/core/tx/schemeshard/schemeshard_utils.cpp b/ydb/core/tx/schemeshard/schemeshard_utils.cpp index 17816979a46..4ee28e35435 100644 --- a/ydb/core/tx/schemeshard/schemeshard_utils.cpp +++ b/ydb/core/tx/schemeshard/schemeshard_utils.cpp @@ -133,7 +133,7 @@ void TShardDeleter::ShardDeleted(TShardIdx shardIdx, const NActors::TActorContex } } -bool TShardDeleter::Has(TTabletId hiveTabletId, TActorId pipeClientActorId) const { +bool TShardDeleter::Has(TTabletId hiveTabletId, TActorId pipeClientActorId) const { return PerHiveDeletions.contains(hiveTabletId) && PerHiveDeletions.at(hiveTabletId).PipeToHive == pipeClientActorId; } diff --git a/ydb/core/tx/schemeshard/schemeshard_utils.h b/ydb/core/tx/schemeshard/schemeshard_utils.h index 609b7d4d895..de7d17f316f 100644 --- a/ydb/core/tx/schemeshard/schemeshard_utils.h +++ b/ydb/core/tx/schemeshard/schemeshard_utils.h @@ -56,7 +56,7 @@ TSerializedCellVec ChooseSplitKeyByHistogram(const NKikimrTableStats::THistogram class TShardDeleter { struct TPerHiveDeletions { - TActorId PipeToHive; + TActorId PipeToHive; THashSet<TShardIdx> ShardsToDelete; }; @@ -86,7 +86,7 @@ public: void RedirectDeleteRequest(TTabletId hiveFromTabletId, TTabletId hiveToTabletId, TShardIdx shardIdx, const THashMap<TShardIdx, TShardInfo>& shardsInfos, const TActorContext& ctx); void ShardDeleted(TShardIdx shardIdx, const TActorContext& ctx); - bool Has(TTabletId hiveTabletId, TActorId pipeClientActorId) const; + bool Has(TTabletId hiveTabletId, TActorId pipeClientActorId) const; bool Has(TShardIdx shardIdx) const; bool Empty() const; }; diff --git a/ydb/core/tx/schemeshard/schemeshard_xxport__tx_base.h b/ydb/core/tx/schemeshard/schemeshard_xxport__tx_base.h index 1303a4952b4..84b46815c2c 100644 --- a/ydb/core/tx/schemeshard/schemeshard_xxport__tx_base.h +++ b/ydb/core/tx/schemeshard/schemeshard_xxport__tx_base.h @@ -19,12 +19,12 @@ protected: virtual ~TTxBase() = default; - void Send(const TActorId& recipient, IEventBase* ev, ui32 flags = 0, ui64 cookie = 0) { + void Send(const TActorId& recipient, IEventBase* ev, ui32 flags = 0, ui64 cookie = 0) { SendOnComplete.emplace_back(new IEventHandle(recipient, Self->SelfId(), ev, flags, cookie)); } template <typename TEvent> - void Send(const TActorId& recipient, THolder<TEvent> ev, ui32 flags = 0, ui64 cookie = 0) { + void Send(const TActorId& recipient, THolder<TEvent> ev, ui32 flags = 0, ui64 cookie = 0) { return Send(recipient, static_cast<IEventBase*>(ev.Release()), flags, cookie); } @@ -62,7 +62,7 @@ protected: << ": id# " << info->Id << ", subscribers count# " << info->Subscribers.size()); - TSet<TActorId> toAnswer; + TSet<TActorId> toAnswer; toAnswer.swap(info->Subscribers); for (auto& actorId: toAnswer) { Send(actorId, new TEvSchemeShard::TEvNotifyTxCompletionResult(info->Id)); diff --git a/ydb/core/tx/schemeshard/ut_base.cpp b/ydb/core/tx/schemeshard/ut_base.cpp index c9f40a36415..a76dcf9d5bb 100644 --- a/ydb/core/tx/schemeshard/ut_base.cpp +++ b/ydb/core/tx/schemeshard/ut_base.cpp @@ -2583,7 +2583,7 @@ Y_UNIT_TEST_SUITE(TSchemeShardTest) { auto t1 = DescribePath(runtime, "/MyRoot/Table1"); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); RebootTablet(runtime, TTestTxConfig::SchemeShard, sender); auto t2 = DescribePath(runtime, "/MyRoot/Table1"); @@ -2609,7 +2609,7 @@ Y_UNIT_TEST_SUITE(TSchemeShardTest) { auto t1 = DescribePath(runtime, "/MyRoot/Table2"); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); RebootTablet(runtime, TTestTxConfig::SchemeShard, sender); auto t2 = DescribePath(runtime, "/MyRoot/Table2"); @@ -5966,7 +5966,7 @@ Y_UNIT_TEST_SUITE(TSchemeShardTest) { NLs::PathVersionEqual(1), NLs::NotFinished}); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); RebootTablet(runtime, TTestTxConfig::SchemeShard, sender); env.TestWaitNotification(runtime, txId); @@ -6078,7 +6078,7 @@ Y_UNIT_TEST_SUITE(TSchemeShardTest) { NLs::ShardsInsideDomain(41)}); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); RebootTablet(runtime, TTestTxConfig::SchemeShard, sender); env.TestWaitNotification(runtime, txId-1); @@ -6427,7 +6427,7 @@ Y_UNIT_TEST_SUITE(TSchemeShardTest) { TestModificationResult(runtime, txId-1, NKikimrScheme::StatusMultipleModifications); TestModificationResult(runtime, txId, NKikimrScheme::StatusAccepted); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); RebootTablet(runtime, TTestTxConfig::SchemeShard, sender); env.TestWaitNotification(runtime, {txId, txId-1, txId-2}); @@ -6824,7 +6824,7 @@ Y_UNIT_TEST_SUITE(TSchemeShardTest) { AsyncMkDir(runtime, ++txId, "/MyRoot/DirA/SubDirA", "BBB"); AsyncMkDir(runtime, ++txId, "/MyRoot/DirA/SubDirA", "CCC"); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); RebootTablet(runtime, TTestTxConfig::SchemeShard, sender); TestDescribeResult(DescribePath(runtime, "/MyRoot"), @@ -6862,7 +6862,7 @@ Y_UNIT_TEST_SUITE(TSchemeShardTest) { "KeyColumnNames: [\"RowId\"]"); // Set ReadOnly SetSchemeshardReadOnlyMode(runtime, true); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); RebootTablet(runtime, TTestTxConfig::SchemeShard, sender); // Verify that table creation successfully finished diff --git a/ydb/core/tx/schemeshard/ut_helpers/helpers.cpp b/ydb/core/tx/schemeshard/ut_helpers/helpers.cpp index da4821da7a6..6f999b268bb 100644 --- a/ydb/core/tx/schemeshard/ut_helpers/helpers.cpp +++ b/ydb/core/tx/schemeshard/ut_helpers/helpers.cpp @@ -26,7 +26,7 @@ namespace NSchemeShardUT_Private { NActors::TTestActorRuntime& runtime, const TActorId& proposer, TEvRequest* evRequest, const TVector<TStatus>& expectedStatuses) { - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); ui64 txId = evRequest->Record.GetTxId(); runtime.Send(new IEventHandle(proposer, sender, evRequest)); @@ -195,7 +195,7 @@ namespace NSchemeShardUT_Private { // NKikimrScheme::TEvDescribeSchemeResult DescribePath(TTestActorRuntime& runtime, ui64 schemeShard, const TString& path, const NKikimrSchemeOp::TDescribeOptions& opts) { - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); auto evLs = new TEvSchemeShard::TEvDescribeScheme(path); evLs->Record.MutableOptions()->CopyFrom(opts); ForwardToTablet(runtime, schemeShard, sender, evLs); @@ -310,7 +310,7 @@ namespace NSchemeShardUT_Private { void AsyncCopyTable(TTestActorRuntime& runtime, ui64 schemeShardId, ui64 txId, const TString& dstPath, const TString& dstName, const TString& srcFullName) { - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); ForwardToTablet(runtime, schemeShardId, sender, CopyTableRequest(txId, dstPath, dstName, srcFullName)); } @@ -376,7 +376,7 @@ namespace NSchemeShardUT_Private { } void AsyncLock(TTestActorRuntime& runtime, ui64 schemeShard, ui64 txId, const TString& parentPath, const TString& name) { - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); ForwardToTablet(runtime, schemeShard, sender, LockRequest(txId, parentPath, name)); } @@ -408,7 +408,7 @@ namespace NSchemeShardUT_Private { } void AsyncUnlock(TTestActorRuntime& runtime, ui64 schemeShard, ui64 txId, ui64 lockId, const TString& parentPath, const TString& name) { - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); ForwardToTablet(runtime, schemeShard, sender, UnlockRequest(txId, lockId, parentPath, name)); } @@ -810,7 +810,7 @@ namespace NSchemeShardUT_Private { void AsyncAssignBlockStoreVolume(TTestActorRuntime& runtime, ui64 txId, const TString& parentPath, const TString& name, const TString& mountToken, ui64 tokenVersion) { - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); auto evTx = new TEvSchemeShard::TEvModifySchemeTransaction(txId, TTestTxConfig::SchemeShard); auto transaction = evTx->Record.AddTransaction(); transaction->SetWorkingDir(parentPath); @@ -838,7 +838,7 @@ namespace NSchemeShardUT_Private { } void AsyncCancelTxTable(TTestActorRuntime& runtime, ui64 txId, ui64 targetTxId) { - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); ForwardToTablet(runtime, TTestTxConfig::SchemeShard, sender, CancelTxRequest(txId, targetTxId)); } @@ -1092,7 +1092,7 @@ namespace NSchemeShardUT_Private { } NKikimrProto::EReplyStatus LocalMiniKQL(TTestActorRuntime& runtime, ui64 tabletId, const TString& query, NKikimrMiniKQL::TResult& result, TString& err) { - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); auto evTx = new TEvTablet::TEvLocalMKQL; auto *mkql = evTx->Record.MutableProgram(); @@ -1147,7 +1147,7 @@ namespace NSchemeShardUT_Private { NKikimrProto::EReplyStatus LocalSchemeTx(TTestActorRuntime& runtime, ui64 tabletId, const TString& schemeChangesStr, bool dryRun, NTabletFlatScheme::TSchemeChanges& scheme, TString& err) { - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); auto evTx = new TEvTablet::TEvLocalSchemeTx; evTx->Record.SetDryRun(dryRun); @@ -1389,7 +1389,7 @@ namespace NSchemeShardUT_Private { Cdbg << result << "\n"; UNIT_ASSERT_VALUES_EQUAL(status, NKikimrProto::EReplyStatus::OK); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); RebootTablet(runtime, schemeShard, sender); } @@ -1453,7 +1453,7 @@ namespace NSchemeShardUT_Private { } void AsyncUpgradeSubDomain(TTestActorRuntime &runtime, ui64 txId, const TString &parentPath, const TString &name) { - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); auto evTx = UpgradeSubDomainRequest(txId, parentPath, name); ForwardToTablet(runtime, TTestTxConfig::SchemeShard, sender, evTx); } @@ -1479,7 +1479,7 @@ namespace NSchemeShardUT_Private { } void AsyncUpgradeSubDomainDecision(TTestActorRuntime &runtime, ui64 txId, const TString &parentPath, const TString &name, NKikimrSchemeOp::TUpgradeSubDomain::EDecision decision) { - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); auto evTx = UpgradeSubDomainDecisionRequest(txId, parentPath, name, decision); ForwardToTablet(runtime, TTestTxConfig::SchemeShard, sender, evTx); } @@ -1499,7 +1499,7 @@ namespace NSchemeShardUT_Private { const TVector<TString>& tables, TDuration timeout) { - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); { auto request = MakeHolder<TEvTxUserProxy::TEvProposeTransaction>(); diff --git a/ydb/core/tx/schemeshard/ut_helpers/test_env.cpp b/ydb/core/tx/schemeshard/ut_helpers/test_env.cpp index fedf31de53f..9cfd8037b7e 100644 --- a/ydb/core/tx/schemeshard/ut_helpers/test_env.cpp +++ b/ydb/core/tx/schemeshard/ut_helpers/test_env.cpp @@ -27,7 +27,7 @@ using namespace NSchemeShard; // BlockStoreVolume mock for testing schemeshard class TFakeBlockStoreVolume : public TActor<TFakeBlockStoreVolume>, public NTabletFlatExecutor::TTabletExecutedFlat { public: - TFakeBlockStoreVolume(const TActorId& tablet, TTabletStorageInfo* info) + TFakeBlockStoreVolume(const TActorId& tablet, TTabletStorageInfo* info) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, new NMiniKQL::TMiniKQLFactory) {} @@ -195,9 +195,9 @@ private: DoHandleDisconnect(ev->Get()->ClientId, ctx); } - void DoHandleDisconnect(TActorId pipeClient, const TActorContext &ctx) { + void DoHandleDisconnect(TActorId pipeClient, const TActorContext &ctx) { if (pipeClient == SchemeShardPipe) { - SchemeShardPipe = TActorId(); + SchemeShardPipe = TActorId(); // Resend all for (const auto& w : SchemeTxWaiters) { SendToSchemeshard(w.first, ctx); @@ -231,7 +231,7 @@ private: return; // Notifify all waiters and forget TxId - for (TActorId waiter : SchemeTxWaiters[txId]) { + for (TActorId waiter : SchemeTxWaiters[txId]) { LOG_DEBUG_S(ctx, NKikimrServices::FLAT_TX_SCHEMESHARD, "tests -- TTxNotificationSubscriber satisfy subsriber" << ", waiter: " << waiter @@ -257,8 +257,8 @@ private: private: ui64 SchemeshardTabletId; - TActorId SchemeShardPipe; - THashMap<ui64, THashSet<TActorId>> SchemeTxWaiters; + TActorId SchemeShardPipe; + THashMap<ui64, THashSet<TActorId>> SchemeTxWaiters; }; @@ -361,9 +361,9 @@ private: DoHandleDisconnect(ev->Get()->ClientId, ctx); } - void DoHandleDisconnect(TActorId pipeClient, const TActorContext &ctx) { + void DoHandleDisconnect(TActorId pipeClient, const TActorContext &ctx) { if (pipeClient == SchemeShardPipe) { - SchemeShardPipe = TActorId(); + SchemeShardPipe = TActorId(); // Resend all for (const auto& w : SchemeTxWaiters) { SendToSchemeshard(w.first, ctx); @@ -461,9 +461,9 @@ private: private: ui64 SchemeshardTabletId; - TActorId SchemeShardPipe; + TActorId SchemeShardPipe; THashMap<ui64, TPreSerialisedMessage> OnlineRequests; - THashMap<ui64, TActorId> SchemeTxWaiters; + THashMap<ui64, TActorId> SchemeTxWaiters; }; @@ -525,7 +525,7 @@ NSchemeShardUT_Private::TTestEnv::TTestEnv(TTestActorRuntime& runtime, const TTe EnableSchemeshardPipeRetriesGuard = EnableSchemeshardPipeRetries(runtime); } - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); //CreateTestBootstrapper(runtime, CreateTestTabletInfo(MakeBSControllerID(TTestTxConfig::DomainUid), TTabletTypes::FLAT_BS_CONTROLLER), &CreateFlatBsController); BootSchemeShard(runtime, schemeRoot); BootTxAllocator(runtime, txAllocator); @@ -536,7 +536,7 @@ NSchemeShardUT_Private::TTestEnv::TTestEnv(TTestActorRuntime& runtime, const TTe for (ui32 node = 0; node < runtime.GetNodeCount(); ++node) { IActor* txProxy = CreateTxProxy(runtime.GetTxAllocatorTabletIds()); - TActorId txProxyId = runtime.Register(txProxy, node); + TActorId txProxyId = runtime.Register(txProxy, node); runtime.RegisterService(MakeTxProxyID(), txProxyId, node); } @@ -645,14 +645,14 @@ TFakeHiveState::TPtr NSchemeShardUT_Private::TTestEnv::GetHiveState() const { } TAutoPtr<ITabletScheduledEventsGuard> NSchemeShardUT_Private::TTestEnv::EnableSchemeshardPipeRetries(TTestActorRuntime &runtime) { - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); TVector<ui64> tabletIds; // Add schemeshard tabletId to white list tabletIds.push_back((ui64)TTestTxConfig::SchemeShard); return CreateTabletScheduledEventsGuard(tabletIds, runtime, sender); } -NActors::TActorId NSchemeShardUT_Private::CreateNotificationSubscriber(NActors::TTestActorRuntime &runtime, ui64 schemeshardId) { +NActors::TActorId NSchemeShardUT_Private::CreateNotificationSubscriber(NActors::TTestActorRuntime &runtime, ui64 schemeshardId) { return runtime.Register(new TTxNotificationSubscriber(schemeshardId)); } @@ -662,9 +662,9 @@ NActors::TActorId NSchemeShardUT_Private::CreateFakeMetering(NActors::TTestActor return NMetering::MakeMeteringServiceID(); } -void NSchemeShardUT_Private::TestWaitNotification(NActors::TTestActorRuntime &runtime, TSet<ui64> txIds, TActorId subscriberActorId) { +void NSchemeShardUT_Private::TestWaitNotification(NActors::TTestActorRuntime &runtime, TSet<ui64> txIds, TActorId subscriberActorId) { - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); for (ui64 txId : txIds) { Cerr << Endl << "TestWaitNotification wait txId: " << txId << Endl; @@ -702,7 +702,7 @@ void NSchemeShardUT_Private::TTestEnv::TestWaitNotification(NActors::TTestActorR } void NSchemeShardUT_Private::TTestEnv::TestWaitTabletDeletion(NActors::TTestActorRuntime &runtime, TSet<ui64> tabletIds) { - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); for (ui64 tabletId : tabletIds) { Cerr << "wait until " << tabletId << " is deleted" << Endl; @@ -762,10 +762,10 @@ void NSchemeShardUT_Private::TTestEnv::SimulateSleep(NActors::TTestActorRuntime runtime.GrabEdgeEventRethrow<TEvents::TEvWakeup>(sender); } -std::function<NActors::IActor *(const NActors::TActorId &, NKikimr::TTabletStorageInfo *)> NSchemeShardUT_Private::TTestEnv::GetTabletCreationFunc(ui32 type) { +std::function<NActors::IActor *(const NActors::TActorId &, NKikimr::TTabletStorageInfo *)> NSchemeShardUT_Private::TTestEnv::GetTabletCreationFunc(ui32 type) { switch (type) { case TTabletTypes::BlockStoreVolume: - return [](const TActorId& tablet, TTabletStorageInfo* info) { + return [](const TActorId& tablet, TTabletStorageInfo* info) { return new TFakeBlockStoreVolume(tablet, info); }; case TTabletTypes::FileStore: diff --git a/ydb/core/tx/schemeshard/ut_helpers/test_env.h b/ydb/core/tx/schemeshard/ut_helpers/test_env.h index 1b234d7112b..72b934c570d 100644 --- a/ydb/core/tx/schemeshard/ut_helpers/test_env.h +++ b/ydb/core/tx/schemeshard/ut_helpers/test_env.h @@ -16,8 +16,8 @@ namespace NSchemeShardUT_Private { using namespace NKikimr; using namespace NSchemeShard; - void TestWaitNotification(NActors::TTestActorRuntime &runtime, TSet<ui64> txIds, TActorId subscriberActorId); - NActors::TActorId CreateNotificationSubscriber(NActors::TTestActorRuntime &runtime, ui64 schemeshardId); + void TestWaitNotification(NActors::TTestActorRuntime &runtime, TSet<ui64> txIds, TActorId subscriberActorId); + NActors::TActorId CreateNotificationSubscriber(NActors::TTestActorRuntime &runtime, ui64 schemeshardId); NActors::TActorId CreateFakeMetering(NActors::TTestActorRuntime &runtime); struct TTestEnvOptions { @@ -49,15 +49,15 @@ namespace NSchemeShardUT_Private { class TTestEnv { public: - using TSchemeShardFactory = std::function<IActor* (const TActorId &, TTabletStorageInfo *)>; + using TSchemeShardFactory = std::function<IActor* (const TActorId &, TTabletStorageInfo *)>; private: TSchemeShardFactory SchemeShardFactory; TFakeHiveState::TPtr HiveState; TFakeCoordinator::TState::TPtr CoordinatorState; TAutoPtr<ITabletScheduledEventsGuard> EnableSchemeshardPipeRetriesGuard; - TMap<ui64, TActorId> TxNotificationSubcribers; - TActorId TxReliablePropose; + TMap<ui64, TActorId> TxNotificationSubcribers; + TActorId TxReliablePropose; ui32 ChannelsCount; TActorId MeteringFake; @@ -103,7 +103,7 @@ namespace NSchemeShardUT_Private { void SetupLogging(TTestActorRuntime& runtime); private: - static std::function<IActor*(const TActorId&, TTabletStorageInfo*)> GetTabletCreationFunc(ui32 type); + static std::function<IActor*(const TActorId&, TTabletStorageInfo*)> GetTabletCreationFunc(ui32 type); void AddDomain(TTestActorRuntime& runtime, TAppPrepare& app, ui32 domainUid, ui32 ssId, ui64 hive, ui64 schemeRoot); void BootSchemeShard(TTestActorRuntime& runtime, ui64 schemeRoot); diff --git a/ydb/core/tx/schemeshard/ut_subdomain.cpp b/ydb/core/tx/schemeshard/ut_subdomain.cpp index 0761d1586e1..d0d30bf7da8 100644 --- a/ydb/core/tx/schemeshard/ut_subdomain.cpp +++ b/ydb/core/tx/schemeshard/ut_subdomain.cpp @@ -1351,7 +1351,7 @@ Y_UNIT_TEST_SUITE(TSchemeShardSubDomainTest) { NLs::PathsInsideDomain(1), NLs::ShardsInsideDomain(0)}); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); RebootTablet(runtime, TTestTxConfig::SchemeShard, sender); TestDescribeResult(DescribePath(runtime, "/MyRoot/USER_0"), @@ -1378,7 +1378,7 @@ Y_UNIT_TEST_SUITE(TSchemeShardSubDomainTest) { "TimeCastBucketsPerMediator: 2 " "Name: \"USER_0\""); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); RebootTablet(runtime, TTestTxConfig::SchemeShard, sender); env.TestWaitNotification(runtime, 100); @@ -1493,7 +1493,7 @@ Y_UNIT_TEST_SUITE(TSchemeShardSubDomainTest) { TestDropSubDomain(runtime, txId++, "/MyRoot", "USER_0"); { - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); RebootTablet(runtime, TTestTxConfig::SchemeShard, sender); } @@ -1501,7 +1501,7 @@ Y_UNIT_TEST_SUITE(TSchemeShardSubDomainTest) { TestLs(runtime, "/MyRoot/USER_0", false, NLs::PathNotExist); { - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); RebootTablet(runtime, TTestTxConfig::SchemeShard, sender); } diff --git a/ydb/core/tx/schemeshard/ut_upgradesubdomain.cpp b/ydb/core/tx/schemeshard/ut_upgradesubdomain.cpp index 9e2e40d0559..1ee6456acb1 100644 --- a/ydb/core/tx/schemeshard/ut_upgradesubdomain.cpp +++ b/ydb/core/tx/schemeshard/ut_upgradesubdomain.cpp @@ -882,7 +882,7 @@ Y_UNIT_TEST_SUITE(TSchemeShardUpgradeSubDomainTest) { env.TestWaitTabletDeletion(runtime, { TTestTxConfig::FakeHiveTablets + 2, TTestTxConfig::FakeHiveTablets + 3, TTestTxConfig::FakeHiveTablets + 4}); - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); RebootTablet(runtime, TTestTxConfig::SchemeShard, sender); TestDescribeResult(DescribePath(runtime, "/MyRoot/USER_0"), diff --git a/ydb/core/tx/time_cast/time_cast.cpp b/ydb/core/tx/time_cast/time_cast.cpp index 2c6486085d4..08042194fc5 100644 --- a/ydb/core/tx/time_cast/time_cast.cpp +++ b/ydb/core/tx/time_cast/time_cast.cpp @@ -57,7 +57,7 @@ class TMediatorTimecastProxy : public TActor<TMediatorTimecastProxy> { const ui32 BucketsSz; TArrayHolder<TMediatorBucket> Buckets; - TActorId PipeClient; + TActorId PipeClient; TMediator(ui32 bucketsSz) : BucketsSz(bucketsSz) @@ -117,7 +117,7 @@ class TMediatorTimecastProxy : public TActor<TMediatorTimecastProxy> { } } - void TryResync(const TActorId &pipeClient, ui64 tabletId, const TActorContext &ctx) { + void TryResync(const TActorId &pipeClient, ui64 tabletId, const TActorContext &ctx) { for (auto &xpair : Mediators) { const ui64 mediatorTabletId = xpair.first; TMediator &mediator = xpair.second; diff --git a/ydb/core/tx/time_cast/time_cast.h b/ydb/core/tx/time_cast/time_cast.h index e55483992c5..1202d5a7991 100644 --- a/ydb/core/tx/time_cast/time_cast.h +++ b/ydb/core/tx/time_cast/time_cast.h @@ -182,7 +182,7 @@ struct TEvMediatorTimecast { IActor* CreateMediatorTimecastProxy(); -inline TActorId MakeMediatorTimecastProxyID() { +inline TActorId MakeMediatorTimecastProxyID() { return TActorId(0, TStringBuf("txmdtimecast")); } diff --git a/ydb/core/tx/tx_allocator/txallocator.cpp b/ydb/core/tx/tx_allocator/txallocator.cpp index b1dbe553a0b..b9aa9606fb0 100644 --- a/ydb/core/tx/tx_allocator/txallocator.cpp +++ b/ydb/core/tx/tx_allocator/txallocator.cpp @@ -3,7 +3,7 @@ namespace NKikimr { -IActor* CreateTxAllocator(const TActorId &tablet, TTabletStorageInfo *info) { +IActor* CreateTxAllocator(const TActorId &tablet, TTabletStorageInfo *info) { return new NTxAllocator::TTxAllocator(tablet, info); } diff --git a/ydb/core/tx/tx_allocator/txallocator.h b/ydb/core/tx/tx_allocator/txallocator.h index 8dd2c536eef..3080bd0c719 100644 --- a/ydb/core/tx/tx_allocator/txallocator.h +++ b/ydb/core/tx/tx_allocator/txallocator.h @@ -5,7 +5,7 @@ #include <ydb/core/protos/tx.pb.h> namespace NKikimr { -IActor* CreateTxAllocator(const TActorId &tablet, TTabletStorageInfo *info); +IActor* CreateTxAllocator(const TActorId &tablet, TTabletStorageInfo *info); } namespace NKikimr { diff --git a/ydb/core/tx/tx_allocator/txallocator_impl.cpp b/ydb/core/tx/tx_allocator/txallocator_impl.cpp index 33d2f445868..a4940b8f354 100644 --- a/ydb/core/tx/tx_allocator/txallocator_impl.cpp +++ b/ydb/core/tx/tx_allocator/txallocator_impl.cpp @@ -6,7 +6,7 @@ namespace NKikimr { namespace NTxAllocator { -TTxAllocator::TTxAllocator(const TActorId &tablet, TTabletStorageInfo *info) +TTxAllocator::TTxAllocator(const TActorId &tablet, TTabletStorageInfo *info) : TActor(&TThis::StateInit) , TTabletExecutedFlat(info, tablet, new NMiniKQL::TMiniKQLFactory) , PrivateMarker((TabletID() & 0xFFFF) << (64 - 16)) diff --git a/ydb/core/tx/tx_allocator/txallocator_impl.h b/ydb/core/tx/tx_allocator/txallocator_impl.h index e57c4f216b2..5a539f9b7bc 100644 --- a/ydb/core/tx/tx_allocator/txallocator_impl.h +++ b/ydb/core/tx/tx_allocator/txallocator_impl.h @@ -79,7 +79,7 @@ public: return NKikimrServices::TActivity::TX_ALLOCATOR_ACTOR; } - TTxAllocator(const TActorId &tablet, TTabletStorageInfo *info); + TTxAllocator(const TActorId &tablet, TTabletStorageInfo *info); STFUNC(StateInit) { StateInitImpl(ev, ctx); diff --git a/ydb/core/tx/tx_allocator/txallocator_ut_helpers.cpp b/ydb/core/tx/tx_allocator/txallocator_ut_helpers.cpp index 1e71e74c002..95566493b6f 100644 --- a/ydb/core/tx/tx_allocator/txallocator_ut_helpers.cpp +++ b/ydb/core/tx/tx_allocator/txallocator_ut_helpers.cpp @@ -39,9 +39,9 @@ TAnswerWithCookie GrabAnswer(NActors::TTestActorRuntime &runtime) { } void AsyncAllocate(NActors::TTestActorRuntime &runtime, ui64 size) { - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); TEvTxAllocator::TEvAllocate *ev = new TEvTxAllocator::TEvAllocate(size); - runtime.SendToPipe(TTestEnv::TxAllocatorTablet, sender, ev, 0, NKikimr::NTabletPipe::TClientConfig(), TActorId(), SomeCockie(size)); + runtime.SendToPipe(TTestEnv::TxAllocatorTablet, sender, ev, 0, NKikimr::NTabletPipe::TClientConfig(), TActorId(), SomeCockie(size)); } void AllocateAndCheck(NActors::TTestActorRuntime &runtime, ui64 size, const TVector<TResultStatus> &expected) { @@ -90,7 +90,7 @@ void TTestEnv::Setup(TTestActorRuntime &runtime) { } void TTestEnv::Reboot(TTestActorRuntime &runtime) { - TActorId sender = runtime.AllocateEdgeActor(); + TActorId sender = runtime.AllocateEdgeActor(); RebootTablet(runtime, TxAllocatorTablet, sender); } diff --git a/ydb/core/tx/tx_allocator_client/actor_client.cpp b/ydb/core/tx/tx_allocator_client/actor_client.cpp index 00d8f40c694..20909a8401f 100644 --- a/ydb/core/tx/tx_allocator_client/actor_client.cpp +++ b/ydb/core/tx/tx_allocator_client/actor_client.cpp @@ -12,7 +12,7 @@ namespace NKikimr { class TTxAllocatorClientActor: public TActorBootstrapped<TTxAllocatorClientActor> { struct TDelayedRequest { - TActorId Sender; + TActorId Sender; ui64 Cookie; ui64 Count; }; diff --git a/ydb/core/tx/tx_proxy/datareq.cpp b/ydb/core/tx/tx_proxy/datareq.cpp index bb546289af5..af48b42d60f 100644 --- a/ydb/core/tx/tx_proxy/datareq.cpp +++ b/ydb/core/tx/tx_proxy/datareq.cpp @@ -135,7 +135,7 @@ private: struct TReadTableRequest : public TThrRefBase { struct TQuotaRequest { - TActorId Sender; + TActorId Sender; ui64 ShardId; }; @@ -148,8 +148,8 @@ struct TReadTableRequest : public TThrRefBase { TString ResponseData; ui64 ResponseDataFrom; TKeySpace KeySpace; - THashMap<ui64, TActorId> ClearanceSenders; - THashMap<ui64, TActorId> StreamingShards; + THashMap<ui64, TActorId> ClearanceSenders; + THashMap<ui64, TActorId> StreamingShards; TSerializedCellVec FromValues; TSerializedCellVec ToValues; THolder<TKeyDesc> KeyDesc; @@ -303,7 +303,7 @@ private: TDuration CancelAfter; TSchedulerCookieHolder ExecTimeoutCookieHolder; - TActorId RequestSource; + TActorId RequestSource; ui32 TxFlags; bool CanUseFollower; bool StreamResponse; @@ -1087,7 +1087,7 @@ void TDataReq::ProcessReadTableResolve(NSchemeCache::TSchemeCacheRequest *cacheR dataTransaction.SetStreamResponse(StreamResponse); dataTransaction.SetImmediate(immediate); dataTransaction.SetReadOnly(true); - ActorIdToProto(SelfId(), dataTransaction.MutableSink()); + ActorIdToProto(SelfId(), dataTransaction.MutableSink()); auto &tx = *dataTransaction.MutableReadTableTransaction(); tx.MutableTableId()->SetOwnerId(ReadTableRequest->KeyDesc->TableId.PathId.OwnerId); tx.MutableTableId()->SetTableId(ReadTableRequest->KeyDesc->TableId.PathId.LocalPathId); @@ -1173,7 +1173,7 @@ TAutoPtr<TEvTxProxySchemeCache::TEvResolveKeySet> TDataReq::PrepareFlatMKQLReque void TDataReq::TryToInvalidateTable(TTableId tableId, const TActorContext &ctx) { const bool notYetInvalidated = InvalidatedTables.insert(tableId).second; if (notYetInvalidated) - ctx.Send(Services.SchemeCache, new TEvTxProxySchemeCache::TEvInvalidateTable(tableId, TActorId())); + ctx.Send(Services.SchemeCache, new TEvTxProxySchemeCache::TEvInvalidateTable(tableId, TActorId())); } void TDataReq::MarkShardError(ui64 shardId, TDataReq::TPerTablet &perTablet, bool invalidateDistCache, const TActorContext &ctx) { @@ -1276,9 +1276,9 @@ void TDataReq::Handle(TEvTxProxyReq::TEvMakeRequest::TPtr &ev, const TActorConte if (txbody.HasMiniKQLTransaction()) { const auto& mkqlTxBody = txbody.GetMiniKQLTransaction(); - const TAppData* appData = AppData(ctx); - const auto functionRegistry = appData->FunctionRegistry; - + const TAppData* appData = AppData(ctx); + const auto functionRegistry = appData->FunctionRegistry; + if (mkqlTxBody.GetFlatMKQL()) { FlatMKQLRequest = new TFlatMKQLRequest; FlatMKQLRequest->LlvmRuntime = mkqlTxBody.GetLlvmRuntime(); diff --git a/ydb/core/tx/tx_proxy/describe.cpp b/ydb/core/tx/tx_proxy/describe.cpp index 06dd5a348f6..deb38ca2b7d 100644 --- a/ydb/core/tx/tx_proxy/describe.cpp +++ b/ydb/core/tx/tx_proxy/describe.cpp @@ -22,7 +22,7 @@ class TDescribeReq : public TActor<TDescribeReq> { TInstant WallClockStarted; - TActorId Source; + TActorId Source; ui64 SourceCookie; TAutoPtr<const NACLib::TUserToken> UserToken; diff --git a/ydb/core/tx/tx_proxy/proxy.cpp b/ydb/core/tx/tx_proxy/proxy.cpp index c51546e3f59..55962e7d06f 100644 --- a/ydb/core/tx/tx_proxy/proxy.cpp +++ b/ydb/core/tx/tx_proxy/proxy.cpp @@ -2,7 +2,7 @@ namespace NKikimr { -TActorId MakeTxProxyID() { +TActorId MakeTxProxyID() { return TActorId(0, TStringBuf("TxProxyServ")); } diff --git a/ydb/core/tx/tx_proxy/proxy.h b/ydb/core/tx/tx_proxy/proxy.h index ab82b8969e7..86d4f1ac79e 100644 --- a/ydb/core/tx/tx_proxy/proxy.h +++ b/ydb/core/tx/tx_proxy/proxy.h @@ -25,8 +25,8 @@ namespace NMiniKQL { namespace NTxProxy { struct TTxProxyServices { - TActorId Proxy; - TActorId SchemeCache; + TActorId Proxy; + TActorId SchemeCache; TActorId LeaderPipeCache; TActorId FollowerPipeCache; }; @@ -172,9 +172,9 @@ struct TEvTxUserProxy { struct TEvInvalidateTableResult : public TEventSimple<TEvInvalidateTableResult, EvInvalidateTableResult> {}; struct TEvProposeKqpTransaction : public TEventLocal<TEvProposeKqpTransaction, EvProposeKqpTransaction> { - TActorId ExecuterId; + TActorId ExecuterId; - TEvProposeKqpTransaction(const TActorId& executerId) + TEvProposeKqpTransaction(const TActorId& executerId) : ExecuterId(executerId) {} }; @@ -311,6 +311,6 @@ namespace NTxProxy { } IActor* CreateTxProxy(const TVector<ui64> &allocators); -TActorId MakeTxProxyID(); +TActorId MakeTxProxyID(); } diff --git a/ydb/core/tx/tx_proxy/proxy_impl.cpp b/ydb/core/tx/tx_proxy/proxy_impl.cpp index 5a5f8a4dfde..20dc8cce176 100644 --- a/ydb/core/tx/tx_proxy/proxy_impl.cpp +++ b/ydb/core/tx/tx_proxy/proxy_impl.cpp @@ -247,7 +247,7 @@ class TTxProxy : public TActorBootstrapped<TTxProxy> { ui64 cookie = ev->Cookie; const TString userRequestId = tx.GetUserRequestId(); TAutoPtr<TEvTxProxyReq::TEvSchemeRequest> request = new TEvTxProxyReq::TEvSchemeRequest(ev); - const TActorId reqId = ctx.ExecutorThread.RegisterActor(CreateTxProxyFlatSchemeReq(Services, txid, request, TxProxyMon)); + const TActorId reqId = ctx.ExecutorThread.RegisterActor(CreateTxProxyFlatSchemeReq(Services, txid, request, TxProxyMon)); TxProxyMon->SchemeRequest->Inc(); LOG_DEBUG_S(ctx, NKikimrServices::TX_PROXY, "actor# " << SelfId() << @@ -278,7 +278,7 @@ class TTxProxy : public TActorBootstrapped<TTxProxy> { if (ev->Get()->HasSnapshotProposal()) { auto cookie = ev->Cookie; auto userReqId = tx.GetUserRequestId(); - const TActorId reqId = ctx.ExecutorThread.RegisterActor(CreateTxProxySnapshotReq(Services, txid, std::move(ev), TxProxyMon)); + const TActorId reqId = ctx.ExecutorThread.RegisterActor(CreateTxProxySnapshotReq(Services, txid, std::move(ev), TxProxyMon)); TxProxyMon->SnapshotRequest->Inc(); LOG_DEBUG_S(ctx, NKikimrServices::TX_PROXY, "actor# " << SelfId() << @@ -342,7 +342,7 @@ class TTxProxy : public TActorBootstrapped<TTxProxy> { " ProcessProposeKqpTransaction"); auto executerEv = MakeHolder<NKqp::TEvKqpExecuter::TEvTxRequest>(); - ActorIdToProto(ev->Sender, executerEv->Record.MutableTarget()); + ActorIdToProto(ev->Sender, executerEv->Record.MutableTarget()); executerEv->Record.MutableRequest()->SetTxId(txid); ctx.Send(ev->Get()->ExecuterId, executerEv.Release()); } @@ -403,7 +403,7 @@ class TTxProxy : public TActorBootstrapped<TTxProxy> { " describe path " << path); TxProxyMon->Navigate->Inc(); - TActorId reqId = ctx.ExecutorThread.RegisterActor(CreateTxProxyDescribeFlatSchemeReq(Services, TxProxyMon)); + TActorId reqId = ctx.ExecutorThread.RegisterActor(CreateTxProxyDescribeFlatSchemeReq(Services, TxProxyMon)); ctx.Send(reqId, new TEvTxProxyReq::TEvNavigateScheme(ev)); } diff --git a/ydb/core/tx/tx_proxy/read_table.h b/ydb/core/tx/tx_proxy/read_table.h index f8ad742c602..a48d1d29a40 100644 --- a/ydb/core/tx/tx_proxy/read_table.h +++ b/ydb/core/tx/tx_proxy/read_table.h @@ -14,7 +14,7 @@ namespace NTxProxy { }; struct TReadTableSettings { - TActorId Owner; + TActorId Owner; ui64 Cookie = 0; ui64 ProxyFlags = 0; TString DatabaseName; diff --git a/ydb/core/tx/tx_proxy/read_table_impl.cpp b/ydb/core/tx/tx_proxy/read_table_impl.cpp index 410c46a9033..6c30975fd9b 100644 --- a/ydb/core/tx/tx_proxy/read_table_impl.cpp +++ b/ydb/core/tx/tx_proxy/read_table_impl.cpp @@ -231,23 +231,23 @@ private: ui64 ReadTxId = 0; ui64 ExpectedSeqNo = 1; - TActorId ClearanceActor; + TActorId ClearanceActor; ui64 ClearanceCookie = 0; - TActorId QuotaActor; + TActorId QuotaActor; size_t QuotaRequests = 0; ui64 QuotaReserved = 0; ui64 Retries = 0; ui64 RetrySeqNo = 0; - TActorId RetryTimer; + TActorId RetryTimer; TInstant RetryingSince; TDuration LastRetryDelay; bool AllowInstantRetry = true; ESnapshotState SnapshotState = ESnapshotState::Unknown; ui64 RefreshSeqNo = 0; - TActorId RefreshTimer; + TActorId RefreshTimer; TDuration LastRefreshDelay; explicit TShardState(ui64 shardId) @@ -1402,7 +1402,7 @@ private: dataTransaction.SetStreamResponse(true); dataTransaction.SetImmediate(true); dataTransaction.SetReadOnly(true); - ActorIdToProto(SelfId(), dataTransaction.MutableSink()); + ActorIdToProto(SelfId(), dataTransaction.MutableSink()); auto &tx = *dataTransaction.MutableReadTableTransaction(); tx.MutableTableId()->SetOwnerId(TableId.PathId.OwnerId); @@ -2344,7 +2344,7 @@ private: // Avoid setting DomainOwnerId to reduce possible races with schemeshard migration request->DatabaseName = Settings.DatabaseName; request->ResultSet.emplace_back(std::move(updatedKeyDesc)); - ctx.Send(Services.SchemeCache, new TEvTxProxySchemeCache::TEvInvalidateTable(TableId, TActorId())); + ctx.Send(Services.SchemeCache, new TEvTxProxySchemeCache::TEvInvalidateTable(TableId, TActorId())); ctx.Send(Services.SchemeCache, new TEvTxProxySchemeCache::TEvResolveKeySet(request)); ResolveInProgress = true; } @@ -2690,7 +2690,7 @@ private: void TryToInvalidateTable(TTableId tableId, const TActorContext& ctx) { const bool notYetInvalidated = InvalidatedTables.insert(tableId).second; if (notYetInvalidated) { - ctx.Send(Services.SchemeCache, new TEvTxProxySchemeCache::TEvInvalidateTable(tableId, TActorId())); + ctx.Send(Services.SchemeCache, new TEvTxProxySchemeCache::TEvInvalidateTable(tableId, TActorId())); } } diff --git a/ydb/core/tx/tx_proxy/schemereq.cpp b/ydb/core/tx/tx_proxy/schemereq.cpp index a8f1b952da6..923fe381ae6 100644 --- a/ydb/core/tx/tx_proxy/schemereq.cpp +++ b/ydb/core/tx/tx_proxy/schemereq.cpp @@ -32,8 +32,8 @@ struct TBaseSchemeReq: public TActorBootstrapped<TDerived> { TInstant WallClockStarted; - TActorId Source; - TActorId PipeClient; + TActorId Source; + TActorId PipeClient; struct TPathToResolve { NKikimrSchemeOp::EOperationType OperationRelated; diff --git a/ydb/core/tx/tx_proxy/snapshotreq.cpp b/ydb/core/tx/tx_proxy/snapshotreq.cpp index 07e96b68abd..4b2d7fe2756 100644 --- a/ydb/core/tx/tx_proxy/snapshotreq.cpp +++ b/ydb/core/tx/tx_proxy/snapshotreq.cpp @@ -229,7 +229,7 @@ public: void TryToInvalidateTable(TTableId tableId, const TActorContext& ctx) { const bool notYetInvalidated = InvalidatedTables.insert(tableId).second; if (notYetInvalidated) { - ctx.Send(Services.SchemeCache, new TEvTxProxySchemeCache::TEvInvalidateTable(tableId, TActorId())); + ctx.Send(Services.SchemeCache, new TEvTxProxySchemeCache::TEvInvalidateTable(tableId, TActorId())); } } @@ -1035,7 +1035,7 @@ public: private: const TTxProxyServices& Services; const ui64 TxId; - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; THolder<TEvTxUserProxy::TEvProposeTransaction> Request; const TIntrusivePtr<TTxProxyMon> TxProxyMon; @@ -1063,7 +1063,7 @@ private: ui64 SelectedCoordinator = 0; THolder<const NACLib::TUserToken> UserToken; - TActorId ResolveActorID; + TActorId ResolveActorID; TTablePathHashSet InvalidatedTables; TMap<ui64, TPerShardState> PerShardStates; size_t TabletsToPrepare = 0; @@ -1270,7 +1270,7 @@ public: void TryToInvalidateTable(TTableId tableId, const TActorContext& ctx) { const bool notYetInvalidated = InvalidatedTables.insert(tableId).second; if (notYetInvalidated) { - ctx.Send(Services.SchemeCache, new TEvTxProxySchemeCache::TEvInvalidateTable(tableId, TActorId())); + ctx.Send(Services.SchemeCache, new TEvTxProxySchemeCache::TEvInvalidateTable(tableId, TActorId())); } } @@ -1620,7 +1620,7 @@ public: private: const TTxProxyServices& Services; - const TActorId Sender; + const TActorId Sender; const ui64 Cookie; THolder<TEvTxUserProxy::TEvProposeTransaction> Request; const TIntrusivePtr<TTxProxyMon> TxProxyMon; @@ -1643,7 +1643,7 @@ private: THolder<const NACLib::TUserToken> UserToken; - TActorId ResolveActorID; + TActorId ResolveActorID; TTablePathHashSet InvalidatedTables; TMap<ui64, TPerShardState> PerShardStates; size_t TabletsToWait = 0; diff --git a/ydb/core/tx/tx_proxy/storage_tenant_ut.cpp b/ydb/core/tx/tx_proxy/storage_tenant_ut.cpp index 72472692352..c3b8e9921a8 100644 --- a/ydb/core/tx/tx_proxy/storage_tenant_ut.cpp +++ b/ydb/core/tx/tx_proxy/storage_tenant_ut.cpp @@ -332,7 +332,7 @@ Y_UNIT_TEST_SUITE(TStorageTenantTest) { ui32 nodeIdx = 0) { TIntrusivePtr<TTabletStorageInfo> info = TabletStorageInfoFromProto(storageInfo.GetInfo()); - TActorId edge = runtime->AllocateEdgeActor(nodeIdx); + TActorId edge = runtime->AllocateEdgeActor(nodeIdx); IActor* x = CreateTabletReqBlockBlobStorage(edge, info.Get(), Max<ui32>(), false); runtime->Register(x, nodeIdx); @@ -350,7 +350,7 @@ Y_UNIT_TEST_SUITE(TStorageTenantTest) { auto& lastEntry = *channel.GetHistory().rbegin(); ui32 group = lastEntry.GetGroupID(); - TActorId edge = runtime->AllocateEdgeActor(nodeIdx); + TActorId edge = runtime->AllocateEdgeActor(nodeIdx); const ui32 generation = Max<ui32>(); auto event = TEvBlobStorage::TEvCollectGarbage::CreateHardBarrier( @@ -362,7 +362,7 @@ Y_UNIT_TEST_SUITE(TStorageTenantTest) { std::numeric_limits<ui32>::max(), // collectStep TInstant::Max()); // deadline - TActorId nodeWarden = MakeBlobStorageNodeWardenID(runtime->GetNodeId(nodeIdx)); + TActorId nodeWarden = MakeBlobStorageNodeWardenID(runtime->GetNodeId(nodeIdx)); runtime->Send(new IEventHandle(MakeBlobStorageProxyID(group), edge, event.Release(), IEventHandle::FlagForwardOnNondelivery, 0, &nodeWarden), nodeIdx); diff --git a/ydb/core/tx/tx_proxy/upload_rows.cpp b/ydb/core/tx/tx_proxy/upload_rows.cpp index 8c91d6d6772..3c4580a8a86 100644 --- a/ydb/core/tx/tx_proxy/upload_rows.cpp +++ b/ydb/core/tx/tx_proxy/upload_rows.cpp @@ -9,7 +9,7 @@ namespace NTxProxy { class TUploadRowsInternal : public TUploadRowsBase<NKikimrServices::TActivity::UPLOAD_ROWS_INTERNAL> { public: TUploadRowsInternal( - TActorId sender, + TActorId sender, const TString& table, std::shared_ptr<TVector<std::pair<TString, Ydb::Type>>> types, std::shared_ptr<TVector<std::pair<TSerializedCellVec, TString>>> rows, @@ -73,7 +73,7 @@ private: } private: - const TActorId Sender; + const TActorId Sender; const TString Table; const std::shared_ptr<TVector<std::pair<TString, Ydb::Type>>> ColumnTypes; const std::shared_ptr<TVector<std::pair<TSerializedCellVec, TString>>> Rows; @@ -81,7 +81,7 @@ private: NYql::TIssues Issues; }; -IActor* CreateUploadRowsInternal(const TActorId& sender, +IActor* CreateUploadRowsInternal(const TActorId& sender, const TString& table, std::shared_ptr<TVector<std::pair<TString, Ydb::Type>>> types, std::shared_ptr<TVector<std::pair<TSerializedCellVec, TString>>> rows, diff --git a/ydb/core/tx/tx_proxy/upload_rows.h b/ydb/core/tx/tx_proxy/upload_rows.h index 5eb5e6f3b64..93d116b04ec 100644 --- a/ydb/core/tx/tx_proxy/upload_rows.h +++ b/ydb/core/tx/tx_proxy/upload_rows.h @@ -16,7 +16,7 @@ enum class EUploadRowsMode { WriteToTableShadow, }; -IActor* CreateUploadRowsInternal(const TActorId& sender, +IActor* CreateUploadRowsInternal(const TActorId& sender, const TString& table, std::shared_ptr<TVector<std::pair<TString, Ydb::Type> > > types, std::shared_ptr<TVector<std::pair<TSerializedCellVec, TString> > > rows, diff --git a/ydb/core/tx/tx_proxy/upload_rows_common_impl.h b/ydb/core/tx/tx_proxy/upload_rows_common_impl.h index 0a9f6690f5e..0aaee7e7f59 100644 --- a/ydb/core/tx/tx_proxy/upload_rows_common_impl.h +++ b/ydb/core/tx/tx_proxy/upload_rows_common_impl.h @@ -101,11 +101,11 @@ private: static constexpr TDuration DEFAULT_TIMEOUT = TDuration::Seconds(5*60); - TActorId SchemeCache; + TActorId SchemeCache; TActorId LeaderPipeCache; TDuration Timeout; TInstant Deadline; - TActorId TimeoutTimerActorId; + TActorId TimeoutTimerActorId; bool WaitingResolveReply; bool Finished; diff --git a/ydb/core/util/cache.h b/ydb/core/util/cache.h index 18063502334..916701ebc71 100644 --- a/ydb/core/util/cache.h +++ b/ydb/core/util/cache.h @@ -365,7 +365,7 @@ public: auto it = Items.begin(); ui64 size = TSelf::Measure(it->first, it->second); - this->OnEvict(it->first, const_cast<TValue&>(it->second), size); + this->OnEvict(it->first, const_cast<TValue&>(it->second), size); Items.erase(it); return true; } @@ -380,7 +380,7 @@ public: ui64 size = TSelf::Measure(key, value); it = Items.insert(std::make_pair(key, value)).first; currentValue = &it->second; - this->OnInsert(key, value, size); + this->OnInsert(key, value, size); return true; } @@ -390,7 +390,7 @@ public: return false; ui64 size = TSelf::Measure(key, it->second); - this->OnEvict(it->first, const_cast<TValue&>(it->second), size); + this->OnEvict(it->first, const_cast<TValue&>(it->second), size); Items.erase(it); return true; @@ -523,7 +523,7 @@ public: auto item = List.PopBack(); ui64 size = TSelf::Measure(item->Key, item->Value); - this->OnEvict(item->Key, item->Value, size); + this->OnEvict(item->Key, item->Value, size); auto itemIt = Index.find(*item); Y_VERIFY_DEBUG(itemIt != Index.end()); Index.erase(itemIt); @@ -544,7 +544,7 @@ public: indexIt = Index.insert(tmpItem).first; List.PushFront(const_cast<TItem*>(&*indexIt)); currentValue = const_cast<TValue*>(&indexIt->Value); - this->OnInsert(key, value, size); + this->OnInsert(key, value, size); return true; } @@ -556,7 +556,7 @@ public: auto item = const_cast<TItem*>(&*indexIt); ui64 size = TSelf::Measure(item->Key, item->Value); - this->OnEvict(item->Key, item->Value, size); + this->OnEvict(item->Key, item->Value, size); item->Unlink(); Index.erase(indexIt); return true; @@ -681,7 +681,7 @@ public: OutList.PushFront(const_cast<TItemKey*>(&*outInsertedIt)); ui64 size = TSelf::Measure(inItem->Key, inItem->Value); - this->OnEvict(inItem->Key, inItem->Value, size); + this->OnEvict(inItem->Key, inItem->Value, size); auto inItemIt = InIndex.find(*inItem); Y_VERIFY_DEBUG(inItemIt != InIndex.end()); InIndex.erase(inItemIt); @@ -691,7 +691,7 @@ public: auto outItem = OutList.PopBack(); auto outItemIt = OutIndex.find(*outItem); Y_VERIFY_DEBUG(outItemIt != OutIndex.end()); - this->OnKeyEvict(outItemIt->Key); + this->OnKeyEvict(outItemIt->Key); OutIndex.erase(outItemIt); } @@ -701,7 +701,7 @@ public: if (!MainIndex.empty()) { auto item = MainList.PopBack(); ui64 size = TSelf::Measure(item->Key, item->Value); - this->OnEvict(item->Key, item->Value, size); + this->OnEvict(item->Key, item->Value, size); auto mainIndexIt = MainIndex.find(*item); Y_VERIFY_DEBUG(mainIndexIt != MainIndex.end()); MainIndex.erase(mainIndexIt); @@ -744,7 +744,7 @@ public: currentValue = const_cast<TValue*>(&insertedIt->Value); } - this->OnInsert(key, value, size); + this->OnInsert(key, value, size); return true; } @@ -755,7 +755,7 @@ public: auto item = const_cast<TItem*>(&*mainIndexIt); EraseFromOut(item->Key); ui64 size = TSelf::Measure(item->Key, item->Value); - this->OnEvict(item->Key, item->Value, size); + this->OnEvict(item->Key, item->Value, size); item->Unlink(); MainIndex.erase(mainIndexIt); return true; @@ -766,7 +766,7 @@ public: auto item = const_cast<TItem*>(&*inIndexIt); EraseFromOut(item->Key); ui64 size = TSelf::Measure(item->Key, item->Value); - this->OnEvict(item->Key, item->Value, size); + this->OnEvict(item->Key, item->Value, size); item->Unlink(); InIndex.erase(inIndexIt); InSize -= size; @@ -787,7 +787,7 @@ private: TItemKey tmpItemKey(key); auto outIndexIt = OutIndex.find(tmpItemKey); if (outIndexIt != OutIndex.end()) { - this->OnKeyEvict(key); + this->OnKeyEvict(key); auto item = const_cast<TItemKey*>(&*outIndexIt); item->Unlink(); OutIndex.erase(outIndexIt); diff --git a/ydb/core/util/cache_cache.h b/ydb/core/util/cache_cache.h index 113ddb4c473..a8ab10977dd 100644 --- a/ydb/core/util/cache_cache.h +++ b/ydb/core/util/cache_cache.h @@ -1,8 +1,8 @@ #pragma once #include "defs.h" #include <ydb/core/util/queue_oneone_inplace.h> -#include <library/cpp/monlib/counters/counters.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <util/generic/ptr.h> #include <util/generic/intrlist.h> diff --git a/ydb/core/util/failure_injection.cpp b/ydb/core/util/failure_injection.cpp index e7528b4bd57..9eeaaa968ba 100644 --- a/ydb/core/util/failure_injection.cpp +++ b/ydb/core/util/failure_injection.cpp @@ -2,7 +2,7 @@ #include <ydb/core/protos/services.pb.h> #include <util/system/mutex.h> #include <util/generic/queue.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <library/cpp/lwtrace/all.h> #include <library/cpp/actors/core/event_local.h> #include <library/cpp/actors/core/actor_bootstrapped.h> diff --git a/ydb/core/util/throughput_meter.h b/ydb/core/util/throughput_meter.h index e727ef1a7bc..935dae21783 100644 --- a/ydb/core/util/throughput_meter.h +++ b/ydb/core/util/throughput_meter.h @@ -1,7 +1,7 @@ #pragma once #include "defs.h" -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> namespace NKikimr { diff --git a/ydb/core/util/ya.make b/ydb/core/util/ya.make index a46589ad00c..4f325d8fcbd 100644 --- a/ydb/core/util/ya.make +++ b/ydb/core/util/ya.make @@ -68,7 +68,7 @@ PEERDIR( library/cpp/html/escape library/cpp/ipmath library/cpp/lwtrace - library/cpp/monlib/dynamic_counters + library/cpp/monlib/dynamic_counters library/cpp/random_provider ydb/core/base ydb/core/protos diff --git a/ydb/core/viewer/browse.h b/ydb/core/viewer/browse.h index 4dbeb2e8cea..c5716f46a76 100644 --- a/ydb/core/viewer/browse.h +++ b/ydb/core/viewer/browse.h @@ -22,7 +22,7 @@ class TBrowse : public TActorBootstrapped<TBrowse> { static const bool WithRetry = false; using TBase = TActorBootstrapped<TBrowse>; const IViewer* Viewer; - TActorId Owner; + TActorId Owner; ui32 Requests = 0; ui32 Responses = 0; bool Final = false; @@ -32,8 +32,8 @@ class TBrowse : public TActorBootstrapped<TBrowse> { TString CurrentPath; // TStringBuf? NKikimrViewer::EObjectType CurrentType; THolder<NSchemeShard::TEvSchemeShard::TEvDescribeSchemeResult> DescribeResult; - THashSet<TActorId> Handlers; - TActorId TxProxy = MakeTxProxyID(); + THashSet<TActorId> Handlers; + TActorId TxProxy = MakeTxProxyID(); public: IViewer::TBrowseContext BrowseContext; @@ -42,7 +42,7 @@ public: return NKikimrServices::TActivity::VIEWER_HANDLER; } - TBrowse(const IViewer* viewer, const TActorId& owner, const TString& path, const TString& userToken) + TBrowse(const IViewer* viewer, const TActorId& owner, const TString& path, const TString& userToken) : Viewer(viewer) , Owner(owner) , Path(path) @@ -327,7 +327,7 @@ public: } void Die(const TActorContext& ctx) override { - for (const TActorId& actor : Handlers) { + for (const TActorId& actor : Handlers) { ctx.Send(actor, new TEvents::TEvPoisonPill()); } TBase::Die(ctx); @@ -361,7 +361,7 @@ public: if (ev->Get()->SourceType == TEvTxUserProxy::EvNavigate) { ++Responses; ctx.Send(BrowseContext.Owner, new NViewerEvents::TEvBrowseRequestCompleted(TxProxy, TEvTxUserProxy::EvNavigate)); - TxProxy = TActorId(); // do not query this TxProxy any more + TxProxy = TActorId(); // do not query this TxProxy any more if (Responses == Requests) { NextStep(ctx); } @@ -373,8 +373,8 @@ class TBrowseTabletsCommon : public TActorBootstrapped<TBrowseTabletsCommon> { protected: static const bool WithRetry = false; using TBase = TActorBootstrapped<TBrowseTabletsCommon>; - TActorId Owner; - THashMap<TTabletId, TActorId> PipeClients; + TActorId Owner; + THashMap<TTabletId, TActorId> PipeClients; TSet<TString> Consumers; TVector<ui64> Tablets; ui32 Requests = 0; @@ -395,7 +395,7 @@ public: return NKikimrServices::TActivity::VIEWER_HANDLER; } - TBrowseTabletsCommon(const TActorId& owner, const IViewer::TBrowseContext& browseContext) + TBrowseTabletsCommon(const TActorId& owner, const IViewer::TBrowseContext& browseContext) : Owner(owner) , Path(browseContext.Path) , BrowseContext(browseContext) @@ -414,7 +414,7 @@ public: return clientConfig; } - const TActorId& GetTabletPipe(TTabletId tabletId, const TActorContext& ctx) { + const TActorId& GetTabletPipe(TTabletId tabletId, const TActorContext& ctx) { auto it = PipeClients.find(tabletId); if (it != PipeClients.end()) { return it->second; @@ -448,7 +448,7 @@ public: ui32 groupId = historyInfo.GetGroupID(); if (GroupInfo.emplace(groupId, nullptr).second) { TTabletId bscTabletId = GetBscTabletId(lookupResult->Record.GetTabletID(), ctx); - TActorId pipeClient = GetTabletPipe(bscTabletId, ctx); + TActorId pipeClient = GetTabletPipe(bscTabletId, ctx); NTabletPipe::SendData(ctx, pipeClient, new TEvBlobStorage::TEvRequestControllerInfo(groupId)); ++Requests; ctx.Send(BrowseContext.Owner, new NViewerEvents::TEvBrowseRequestSent(pipeClient, TEvBlobStorage::EvRequestControllerInfo)); diff --git a/ydb/core/viewer/browse_db.h b/ydb/core/viewer/browse_db.h index 914f3512774..fe7b5d31408 100644 --- a/ydb/core/viewer/browse_db.h +++ b/ydb/core/viewer/browse_db.h @@ -23,10 +23,10 @@ protected: static const bool WithRetry = false; using TThis = TBrowseTable; using TBase = TBrowseTabletsCommon; - TActorId TxProxy = MakeTxProxyID(); + TActorId TxProxy = MakeTxProxyID(); public: - TBrowseTable(const TActorId& owner, const IViewer::TBrowseContext& browseContext) + TBrowseTable(const TActorId& owner, const IViewer::TBrowseContext& browseContext) : TBrowseTabletsCommon(owner, browseContext) {} @@ -66,7 +66,7 @@ public: void SendTabletRequests(const TVector<TTabletId>& tablets, const TActorContext& ctx) { TDomainsInfo* domainsInfo = AppData(ctx)->DomainsInfo.Get(); for (auto tabletId : tablets) { - TActorId pipeClient = GetTabletPipe(tabletId, ctx); + TActorId pipeClient = GetTabletPipe(tabletId, ctx); NTabletPipe::SendData(ctx, pipeClient, new TEvTablet::TEvGetCounters(), tabletId); ++Requests; ctx.Send(BrowseContext.Owner, new NViewerEvents::TEvBrowseRequestSent(TxProxy, tabletId, TEvTablet::EvGetCounters)); diff --git a/ydb/core/viewer/browse_events.h b/ydb/core/viewer/browse_events.h index f98f03a2b9e..96ced773938 100644 --- a/ydb/core/viewer/browse_events.h +++ b/ydb/core/viewer/browse_events.h @@ -40,17 +40,17 @@ namespace NViewerEvents { }; struct TEvBrowseRequestSent : TEventLocal<TEvBrowseRequestSent, EvBrowseRequestSent> { - TActorId Actor; + TActorId Actor; TTabletId Tablet; ui32 Event; - TEvBrowseRequestSent(const TActorId& actor, TTabletId tablet, ui32 event) + TEvBrowseRequestSent(const TActorId& actor, TTabletId tablet, ui32 event) : Actor(actor) , Tablet(tablet) , Event(event) {} - TEvBrowseRequestSent(const TActorId& actor, ui32 event) + TEvBrowseRequestSent(const TActorId& actor, ui32 event) : Actor(actor) , Tablet(0) , Event(event) @@ -58,17 +58,17 @@ namespace NViewerEvents { }; struct TEvBrowseRequestCompleted : TEventLocal<TEvBrowseRequestCompleted, EvBrowseRequestCompleted> { - TActorId Actor; + TActorId Actor; TTabletId Tablet; ui32 Event; - TEvBrowseRequestCompleted(const TActorId& actor, TTabletId tablet, ui32 event) + TEvBrowseRequestCompleted(const TActorId& actor, TTabletId tablet, ui32 event) : Actor(actor) , Tablet(tablet) , Event(event) {} - TEvBrowseRequestCompleted(const TActorId& actor, ui32 event) + TEvBrowseRequestCompleted(const TActorId& actor, ui32 event) : Actor(actor) , Tablet(0) , Event(event) diff --git a/ydb/core/viewer/browse_pq.h b/ydb/core/viewer/browse_pq.h index 7612913e038..3e3b32a21e3 100644 --- a/ydb/core/viewer/browse_pq.h +++ b/ydb/core/viewer/browse_pq.h @@ -21,14 +21,14 @@ using namespace NActors; class TBrowseRoot : public TActorBootstrapped<TBrowseRoot> { using TBase = TActorBootstrapped<TBrowseRoot>; - TActorId Owner; + TActorId Owner; public: static constexpr NKikimrServices::TActivity::EType ActorActivityType() { return NKikimrServices::TActivity::VIEWER_HANDLER; } - TBrowseRoot(const TActorId& owner, const IViewer::TBrowseContext&) + TBrowseRoot(const TActorId& owner, const IViewer::TBrowseContext&) : Owner(owner) {} @@ -59,10 +59,10 @@ protected: THashMap<TString, NKikimrViewer::TMetaTopicConsumerInfo> ConsumerInfo; TString FilterTopic; TString FilterConsumer; - TActorId TxProxy = MakeTxProxyID(); + TActorId TxProxy = MakeTxProxyID(); public: - TBrowseCommon(const TActorId& owner, const IViewer::TBrowseContext& browseContext) + TBrowseCommon(const TActorId& owner, const IViewer::TBrowseContext& browseContext) : TBrowseTabletsCommon(owner, browseContext) , PQ_ROOT_PATH("/Root/PQ") // TODO(xenoxeno) {} @@ -130,7 +130,7 @@ public: void SendTabletRequests(const TVector<TTabletId>& tablets, const TActorContext& ctx) { TDomainsInfo* domainsInfo = AppData(ctx)->DomainsInfo.Get(); for (auto tabletId : tablets) { - TActorId pipeClient = GetTabletPipe(tabletId, ctx); + TActorId pipeClient = GetTabletPipe(tabletId, ctx); const auto& range = TabletPartitions.equal_range(tabletId); if (range.first != range.second) { TAutoPtr<TEvPersQueue::TEvPartitionClientInfo> request = new TEvPersQueue::TEvPartitionClientInfo(); @@ -287,7 +287,7 @@ public: return NKikimrServices::TActivity::VIEWER_HANDLER; } - TBrowseConsumers(const TActorId& owner, const IViewer::TBrowseContext& browseContext) + TBrowseConsumers(const TActorId& owner, const IViewer::TBrowseContext& browseContext) : TBrowseCommon(owner, browseContext) {} @@ -308,7 +308,7 @@ public: class TBrowseConsumer : public TBrowseCommon { public: - TBrowseConsumer(const TActorId& owner, const IViewer::TBrowseContext& browseContext) + TBrowseConsumer(const TActorId& owner, const IViewer::TBrowseContext& browseContext) : TBrowseCommon(owner, browseContext) { FilterConsumer = BrowseContext.GetMyName(); @@ -356,7 +356,7 @@ public: class TBrowseTopic : public TBrowseCommon { public: - TBrowseTopic(const TActorId& owner, const IViewer::TBrowseContext& browseContext) + TBrowseTopic(const TActorId& owner, const IViewer::TBrowseContext& browseContext) : TBrowseCommon(owner, browseContext) { FilterTopic = BrowseContext.GetMyName(); diff --git a/ydb/core/viewer/counters_hosts.h b/ydb/core/viewer/counters_hosts.h index e5622ae6a3d..6a7bdcfb3de 100644 --- a/ydb/core/viewer/counters_hosts.h +++ b/ydb/core/viewer/counters_hosts.h @@ -36,7 +36,7 @@ public: const auto& params(Event->Get()->Request.GetParams()); StaticNodesOnly = FromStringWithDefault<bool>(params.Get("static_only"), StaticNodesOnly); DynamicNodesOnly = FromStringWithDefault<bool>(params.Get("dynamic_only"), DynamicNodesOnly); - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); ctx.Send(nameserviceId, new TEvInterconnect::TEvListNodes()); ctx.Schedule(TDuration::Seconds(10), new TEvents::TEvWakeup()); Become(&TThis::StateRequestedList); @@ -59,7 +59,7 @@ public: } void SendRequest(ui32 nodeId, const TActorContext& ctx) { - TActorId whiteboardServiceId = MakeNodeWhiteboardServiceId(nodeId); + TActorId whiteboardServiceId = MakeNodeWhiteboardServiceId(nodeId); THolder<TEvWhiteboard::TEvSystemStateRequest> request = MakeHolder<TEvWhiteboard::TEvSystemStateRequest>(); ctx.Send(whiteboardServiceId, request.Release(), IEventHandle::FlagTrackDelivery | IEventHandle::FlagSubscribeOnSession, nodeId); ++NodesRequested; diff --git a/ydb/core/viewer/json_browse.h b/ydb/core/viewer/json_browse.h index c57d9632b3d..cba672826ee 100644 --- a/ydb/core/viewer/json_browse.h +++ b/ydb/core/viewer/json_browse.h @@ -17,11 +17,11 @@ namespace std { template <> -struct hash<NActors::TActorId> { - size_t operator ()(const NActors::TActorId& actorId) const { - return actorId.Hash(); - } -}; +struct hash<NActors::TActorId> { + size_t operator ()(const NActors::TActorId& actorId) const { + return actorId.Hash(); + } +}; } @@ -41,10 +41,10 @@ class TJsonBrowse : public TActorBootstrapped<TJsonBrowse> { struct TPathStateInfo { TString Name; TString Path; - TActorId BrowseActorId; + TActorId BrowseActorId; NKikimrViewer::TBrowseInfo BrowseInfo; - TPathStateInfo(const TString& name, const TString& path, const TActorId& browseActorId) + TPathStateInfo(const TString& name, const TString& path, const TActorId& browseActorId) : Name(name) , Path(path) , BrowseActorId(browseActorId) @@ -61,7 +61,7 @@ class TJsonBrowse : public TActorBootstrapped<TJsonBrowse> { TVector<TPathStateInfo> Paths; - using TBrowseRequestKey = std::tuple<TActorId, TTabletId, ui32>; + using TBrowseRequestKey = std::tuple<TActorId, TTabletId, ui32>; std::unordered_multiset<TBrowseRequestKey> BrowseRequestsInFlight; ui32 Responses = 0; @@ -134,7 +134,7 @@ public: auto it = std::find(Paths.begin(), Paths.end(), event.BrowseInfo.GetPath()); if (it != Paths.end()) { it->BrowseInfo.MergeFrom(event.BrowseInfo); - it->BrowseActorId = TActorId(); + it->BrowseActorId = TActorId(); } // TODO: error handling? ++Responses; diff --git a/ydb/core/viewer/json_cluster.h b/ydb/core/viewer/json_cluster.h index e47a9727677..36b5362a452 100644 --- a/ydb/core/viewer/json_cluster.h +++ b/ydb/core/viewer/json_cluster.h @@ -19,7 +19,7 @@ class TJsonCluster : public TActorBootstrapped<TJsonCluster> { using TThis = TJsonCluster; using TBase = TActorBootstrapped<TJsonCluster>; IViewer* Viewer; - TActorId Initiator; + TActorId Initiator; ui32 Requested; ui32 Received; NMon::TEvHttpInfo::TPtr Event; @@ -55,7 +55,7 @@ public: } void Bootstrap(const TActorContext& ctx) { - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); ctx.Send(nameserviceId, new TEvInterconnect::TEvListNodes()); TBase::Become(&TThis::StateRequestedBrowse); ctx.Schedule(TDuration::MilliSeconds(Timeout), new TEvents::TEvWakeup()); @@ -71,7 +71,7 @@ public: } void SendRequest(ui32 nodeId, const TActorContext& ctx) { - TActorId whiteboardServiceId = MakeNodeWhiteboardServiceId(nodeId); + TActorId whiteboardServiceId = MakeNodeWhiteboardServiceId(nodeId); ctx.Send(whiteboardServiceId, new TEvWhiteboard::TEvSystemStateRequest(), IEventHandle::FlagTrackDelivery | IEventHandle::FlagSubscribeOnSession, nodeId); ++Requested; ctx.Send(whiteboardServiceId, new TEvWhiteboard::TEvVDiskStateRequest(), IEventHandle::FlagTrackDelivery | IEventHandle::FlagSubscribeOnSession, nodeId); @@ -100,7 +100,7 @@ public: record->MutableOptions()->SetReturnPartitioningInfo(false); record->MutableOptions()->SetReturnPartitionConfig(false); record->MutableOptions()->SetReturnChildren(false); - TActorId txproxy = MakeTxProxyID(); + TActorId txproxy = MakeTxProxyID(); ctx.Send(txproxy, request.Release()); ++Requested; } diff --git a/ydb/core/viewer/json_content.h b/ydb/core/viewer/json_content.h index e3fdb25bdaa..79b230f1288 100644 --- a/ydb/core/viewer/json_content.h +++ b/ydb/core/viewer/json_content.h @@ -18,7 +18,7 @@ class TJsonContent : public TActorBootstrapped<TJsonContent> { using TBase = TActorBootstrapped<TJsonContent>; IViewer* Viewer; - TActorId Initiator; + TActorId Initiator; NMon::TEvHttpInfo::TPtr Event; IViewer::TContentRequestContext ContentRequestContext; diff --git a/ydb/core/viewer/json_counters.h b/ydb/core/viewer/json_counters.h index 0646c58fa84..cac1e0885c4 100644 --- a/ydb/core/viewer/json_counters.h +++ b/ydb/core/viewer/json_counters.h @@ -17,7 +17,7 @@ class TJsonCounters : public TActorBootstrapped<TJsonCounters> { using TThis = TJsonCounters; using TBase = TActorBootstrapped<TJsonCounters>; IViewer* Viewer; - TActorId Initiator; + TActorId Initiator; ui32 Requested; ui32 Received; THolder<TEvInterconnect::TEvNodesInfo> NodesInfo; @@ -39,7 +39,7 @@ public: {} void Bootstrap(const TActorContext& ctx) { - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); ctx.Send(nameserviceId, new TEvInterconnect::TEvListNodes()); TBase::Become(&TThis::StateRequestedBrowse); ctx.Schedule(TDuration::Seconds(60), new TEvents::TEvWakeup()); @@ -55,7 +55,7 @@ public: } void SendRequest(ui32 nodeId, const TActorContext& ctx) { - TActorId whiteboardServiceId = MakeNodeWhiteboardServiceId(nodeId); + TActorId whiteboardServiceId = MakeNodeWhiteboardServiceId(nodeId); ctx.Send(whiteboardServiceId, new TEvWhiteboard::TEvVDiskStateRequest(), IEventHandle::FlagTrackDelivery | IEventHandle::FlagSubscribeOnSession, nodeId); ++Requested; ctx.Send(whiteboardServiceId, new TEvWhiteboard::TEvPDiskStateRequest(), IEventHandle::FlagTrackDelivery | IEventHandle::FlagSubscribeOnSession, nodeId); diff --git a/ydb/core/viewer/json_metainfo.h b/ydb/core/viewer/json_metainfo.h index 4055d220d85..54064d74037 100644 --- a/ydb/core/viewer/json_metainfo.h +++ b/ydb/core/viewer/json_metainfo.h @@ -32,8 +32,8 @@ class TJsonMetaInfo : public TActorBootstrapped<TJsonMetaInfo> { ui32 Timeout = 0; bool Counters = false; NKikimrViewer::TMetaInfo MetaInfo; - TActorId BrowseActorID; - using TBrowseRequestKey = std::tuple<TActorId, TTabletId, ui32>; + TActorId BrowseActorID; + using TBrowseRequestKey = std::tuple<TActorId, TTabletId, ui32>; std::unordered_multiset<TBrowseRequestKey> BrowseRequestsInFlight; public: diff --git a/ydb/core/viewer/json_netinfo.h b/ydb/core/viewer/json_netinfo.h index 289b32f8f2d..3d7c3835a0a 100644 --- a/ydb/core/viewer/json_netinfo.h +++ b/ydb/core/viewer/json_netinfo.h @@ -136,7 +136,7 @@ public: } } for (TNodeId nodeId : NodeIds) { - TActorId whiteboardServiceId = MakeNodeWhiteboardServiceId(nodeId); + TActorId whiteboardServiceId = MakeNodeWhiteboardServiceId(nodeId); SendRequest( whiteboardServiceId, new NNodeWhiteboard::TEvWhiteboard::TEvSystemStateRequest(), diff --git a/ydb/core/viewer/json_nodelist.h b/ydb/core/viewer/json_nodelist.h index 34f47623be7..504ed152d8d 100644 --- a/ydb/core/viewer/json_nodelist.h +++ b/ydb/core/viewer/json_nodelist.h @@ -28,7 +28,7 @@ public: {} void Bootstrap(const TActorContext& ctx) { - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); ctx.Send(nameserviceId, new TEvInterconnect::TEvListNodes()); ctx.Schedule(TDuration::Seconds(10), new TEvents::TEvWakeup()); Become(&TThis::StateRequestedBrowse); diff --git a/ydb/core/viewer/json_query.h b/ydb/core/viewer/json_query.h index 6dfbb23db4a..600d8088e10 100644 --- a/ydb/core/viewer/json_query.h +++ b/ydb/core/viewer/json_query.h @@ -24,7 +24,7 @@ class TJsonQuery : public TActorBootstrapped<TJsonQuery> { using TBase = TActorBootstrapped<TJsonQuery>; IViewer* Viewer; TJsonSettings JsonSettings; - TActorId Initiator; + TActorId Initiator; NMon::TEvHttpInfo::TPtr Event; ui32 Timeout = 0; TVector<Ydb::ResultSet> ResultSets; diff --git a/ydb/core/viewer/json_storage.h b/ydb/core/viewer/json_storage.h index a85e59b546b..76e5d2f5e8b 100644 --- a/ydb/core/viewer/json_storage.h +++ b/ydb/core/viewer/json_storage.h @@ -25,7 +25,7 @@ class TJsonStorage : public TViewerPipeClient<TJsonStorage> { using TBase = TViewerPipeClient<TJsonStorage>; using TNodeId = ui32; IViewer* Viewer; - TActorId Initiator; + TActorId Initiator; NMon::TEvHttpInfo::TPtr Event; THolder<TEvInterconnect::TEvNodesInfo> NodesInfo; TMap<ui32, THolder<TEvWhiteboard::TEvVDiskStateResponse>> VDiskInfo; diff --git a/ydb/core/viewer/json_tabletcounters.h b/ydb/core/viewer/json_tabletcounters.h index e0f422be191..7f399104410 100644 --- a/ydb/core/viewer/json_tabletcounters.h +++ b/ydb/core/viewer/json_tabletcounters.h @@ -20,7 +20,7 @@ class TJsonTabletCounters : public TActorBootstrapped<TJsonTabletCounters> { using TBase = TActorBootstrapped<TJsonTabletCounters>; IViewer* Viewer; NMon::TEvHttpInfo::TPtr Event; - TVector<TActorId> PipeClients; + TVector<TActorId> PipeClients; TVector<ui64> Tablets; TMap<TTabletId, THolder<TEvTablet::TEvGetCountersResponse>> Results; THolder<NSchemeShard::TEvSchemeShard::TEvDescribeSchemeResult> DescribeResult; @@ -65,7 +65,7 @@ public: NKikimrSchemeOp::TDescribePath* record = request->Record.MutableDescribePath(); record->SetPath(params.Get("path")); - TActorId txproxy = MakeTxProxyID(); + TActorId txproxy = MakeTxProxyID(); ctx.Send(txproxy, request.Release()); Become(&TThis::StateRequestedDescribe, ctx, TDuration::MilliSeconds(Timeout), new TEvents::TEvWakeup()); } else if (params.Has("tablet_id")) { @@ -85,7 +85,7 @@ public: } void Die(const TActorContext& ctx) override { - for (const TActorId& pipeClient : PipeClients) { + for (const TActorId& pipeClient : PipeClients) { NTabletPipe::CloseClient(ctx, pipeClient); } TBase::Die(ctx); diff --git a/ydb/core/viewer/json_tabletinfo.h b/ydb/core/viewer/json_tabletinfo.h index 0bc60d9c4f6..32546865a15 100644 --- a/ydb/core/viewer/json_tabletinfo.h +++ b/ydb/core/viewer/json_tabletinfo.h @@ -83,7 +83,7 @@ public: NKikimrSchemeOp::TDescribePath* record = request->Record.MutableDescribePath(); record->SetPath(params.Get("path")); - TActorId txproxy = MakeTxProxyID(); + TActorId txproxy = MakeTxProxyID(); TBase::Send(txproxy, request.Release()); Become(&TThis::StateRequestedDescribe, TDuration::MilliSeconds(Timeout), new TEvents::TEvWakeup()); } else { diff --git a/ydb/core/viewer/json_wb_req.h b/ydb/core/viewer/json_wb_req.h index 4782578e00e..f04c85688f4 100644 --- a/ydb/core/viewer/json_wb_req.h +++ b/ydb/core/viewer/json_wb_req.h @@ -40,7 +40,7 @@ protected: using TThis = TJsonWhiteboardRequest<RequestType, ResponseType>; using TBase = TViewerPipeClient<TThis>; IViewer* Viewer; - TActorId Initiator; + TActorId Initiator; NMon::TEvHttpInfo::TPtr Event; std::vector<TNodeId> FilterNodeIds; ui64 ChangedSince; @@ -81,7 +81,7 @@ public: } void SendNodeRequest(TNodeId nodeId) { - TActorId whiteboardServiceId = MakeNodeWhiteboardServiceId(nodeId); + TActorId whiteboardServiceId = MakeNodeWhiteboardServiceId(nodeId); THolder<RequestType> request = BuildRequest(nodeId); TBase::SendRequest(whiteboardServiceId, request.Release(), IEventHandle::FlagTrackDelivery | IEventHandle::FlagSubscribeOnSession, nodeId); NodeIds.insert(nodeId); @@ -132,7 +132,7 @@ public: static const TActorId whiteboardServiceId = MakeNodeWhiteboardServiceId(TBase::SelfId().NodeId()); TBase::SendRequest(whiteboardServiceId, new TEvWhiteboard::TEvNodeStateRequest()); } else { - const TActorId nameserviceId = GetNameserviceActorId(); + const TActorId nameserviceId = GetNameserviceActorId(); TBase::Send(nameserviceId, new TEvInterconnect::TEvListNodes()); } TBase::Become(&TThis::StateRequestedBrowse); diff --git a/ydb/core/viewer/viewer.cpp b/ydb/core/viewer/viewer.cpp index da84fedf6b0..f1124115bf7 100644 --- a/ydb/core/viewer/viewer.cpp +++ b/ydb/core/viewer/viewer.cpp @@ -3,7 +3,7 @@ #include <ydb/core/mon/mon.h> #include <library/cpp/actors/core/mon.h> #include <ydb/core/base/appdata.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <library/cpp/actors/core/interconnect.h> #include <util/generic/algorithm.h> #include <ydb/core/base/path.h> @@ -95,22 +95,22 @@ public: void SetupPQVirtualHandlers(IViewer* viewer) { viewer->RegisterVirtualHandler( NKikimrViewer::EObjectType::Root, - [] (const TActorId& owner, const IViewer::TBrowseContext& browseContext) -> IActor* { + [] (const TActorId& owner, const IViewer::TBrowseContext& browseContext) -> IActor* { return new NViewerPQ::TBrowseRoot(owner, browseContext); }); viewer->RegisterVirtualHandler( NKikimrViewer::EObjectType::Consumers, - [] (const TActorId& owner, const IViewer::TBrowseContext& browseContext) -> IActor* { + [] (const TActorId& owner, const IViewer::TBrowseContext& browseContext) -> IActor* { return new NViewerPQ::TBrowseConsumers(owner, browseContext); }); viewer->RegisterVirtualHandler( NKikimrViewer::EObjectType::Consumer, - [] (const TActorId& owner, const IViewer::TBrowseContext& browseContext) -> IActor* { + [] (const TActorId& owner, const IViewer::TBrowseContext& browseContext) -> IActor* { return new NViewerPQ::TBrowseConsumer(owner, browseContext); }); viewer->RegisterVirtualHandler( NKikimrViewer::EObjectType::Topic, - [] (const TActorId& owner, const IViewer::TBrowseContext& browseContext) -> IActor* { + [] (const TActorId& owner, const IViewer::TBrowseContext& browseContext) -> IActor* { return new NViewerPQ::TBrowseTopic(owner, browseContext); }); } @@ -118,7 +118,7 @@ void SetupPQVirtualHandlers(IViewer* viewer) { void SetupDBVirtualHandlers(IViewer* viewer) { viewer->RegisterVirtualHandler( NKikimrViewer::EObjectType::Table, - [] (const TActorId& owner, const IViewer::TBrowseContext& browseContext) -> IActor* { + [] (const TActorId& owner, const IViewer::TBrowseContext& browseContext) -> IActor* { return new NViewerDB::TBrowseTable(owner, browseContext); }); } diff --git a/ydb/core/viewer/viewer.h b/ydb/core/viewer/viewer.h index c9b7d0d1da9..ab34762e3ed 100644 --- a/ydb/core/viewer/viewer.h +++ b/ydb/core/viewer/viewer.h @@ -13,9 +13,9 @@ namespace NKikimr { namespace NViewer { -inline TActorId MakeViewerID(ui32 node = 0) { +inline TActorId MakeViewerID(ui32 node = 0) { char x[12] = {'v','i','e','w','e','r'}; - return TActorId(node, TStringBuf(x, 12)); + return TActorId(node, TStringBuf(x, 12)); } IActor* CreateViewer(const TKikimrRunConfig &kikimrRunConfig); @@ -30,10 +30,10 @@ public: TString Path; TVector<TPathEntry> Paths; - TActorId Owner; + TActorId Owner; TString UserToken; - TBrowseContext(const TActorId owner, const TString& userToken) + TBrowseContext(const TActorId owner, const TString& userToken) : Owner(owner) , UserToken(userToken) {} @@ -69,7 +69,7 @@ public: } }; - using TVirtualHandlerType = std::function<IActor*(const TActorId& owner, const TBrowseContext& context)>; + using TVirtualHandlerType = std::function<IActor*(const TActorId& owner, const TBrowseContext& context)>; struct TVirtualHandler { IViewer::TVirtualHandlerType BrowseHandler = nullptr; @@ -100,7 +100,7 @@ public: TString Dump() const; }; - using TContentHandler = std::function<IActor*(const TActorId&, const TContentRequestContext&)>; + using TContentHandler = std::function<IActor*(const TActorId&, const TContentRequestContext&)>; virtual const TKikimrRunConfig& GetKikimrRunConfig() const = 0; virtual TVector<const TVirtualHandler*> GetVirtualHandlers(NKikimrViewer::EObjectType type, const TString& path) const = 0; diff --git a/ydb/core/ymq/actor/action.h b/ydb/core/ymq/actor/action.h index 517bd902fff..1801c98d68f 100644 --- a/ydb/core/ymq/actor/action.h +++ b/ydb/core/ymq/actor/action.h @@ -761,7 +761,7 @@ protected: TIntrusivePtr<TQueueCounters> QueueCounters_; TMaybe<TSqsEvents::TQueueAttributes> QueueAttributes_; NKikimrClient::TSqsResponse Response_; - TActorId SchemeCache_; + TActorId SchemeCache_; TActorId QueueLeader_; bool StartRequestWasCalled_ = false; bool FinishRequestWasCalled_ = false; diff --git a/ydb/core/ymq/actor/common_batch_actor.h b/ydb/core/ymq/actor/common_batch_actor.h index e128f0e5f21..1d9cc6916d9 100644 --- a/ydb/core/ymq/actor/common_batch_actor.h +++ b/ydb/core/ymq/actor/common_batch_actor.h @@ -9,7 +9,7 @@ namespace NKikimr::NSQS { class TBatchRequestReplyCallback : public IReplyCallback { public: - TBatchRequestReplyCallback(const TActorId& bacthActor, ui64 cookie) + TBatchRequestReplyCallback(const TActorId& bacthActor, ui64 cookie) : BatchActor_(bacthActor) , Cookie_(cookie) { @@ -17,12 +17,12 @@ public: private: void DoSendReply(const NKikimrClient::TSqsResponse& resp) override { - const TActorId sender = TActivationContext::AsActorContext().SelfID; + const TActorId sender = TActivationContext::AsActorContext().SelfID; TActivationContext::Send(new IEventHandle(BatchActor_, sender, new TSqsEvents::TEvSqsResponse(resp), 0, Cookie_)); } private: - const TActorId BatchActor_; + const TActorId BatchActor_; const ui64 Cookie_; }; diff --git a/ydb/core/ymq/actor/create_queue.cpp b/ydb/core/ymq/actor/create_queue.cpp index 81941d55501..2d4b3af214e 100644 --- a/ydb/core/ymq/actor/create_queue.cpp +++ b/ydb/core/ymq/actor/create_queue.cpp @@ -152,7 +152,7 @@ private: } void HandleQueueCreated(TSqsEvents::TEvQueueCreated::TPtr& ev) { - SchemaActor_ = TActorId(); + SchemaActor_ = TActorId(); auto event = ev->Get(); auto* result = Response_.MutableCreateQueue(); @@ -192,7 +192,7 @@ private: void PassAway() override { if (SchemaActor_) { Send(SchemaActor_, new TEvPoisonPill()); - SchemaActor_ = TActorId(); + SchemaActor_ = TActorId(); } TActionActor<TCreateQueueActor>::PassAway(); } @@ -203,7 +203,7 @@ private: private: TString ResourceId_; - TActorId SchemaActor_; + TActorId SchemaActor_; }; IActor* CreateCreateQueueActor(const NKikimrClient::TSqsRequest& sourceSqsRequest, THolder<IReplyCallback> cb) { diff --git a/ydb/core/ymq/actor/create_user.cpp b/ydb/core/ymq/actor/create_user.cpp index c2998083dfd..deebfc54e8a 100644 --- a/ydb/core/ymq/actor/create_user.cpp +++ b/ydb/core/ymq/actor/create_user.cpp @@ -63,7 +63,7 @@ private: } void HandleUserCreated(TSqsEvents::TEvUserCreated::TPtr& ev) { - SchemaActor = TActorId(); + SchemaActor = TActorId(); if (ev->Get()->Success) { } else { MakeError(Response_.MutableCreateUser(), NErrors::INTERNAL_FAILURE); @@ -75,7 +75,7 @@ private: void PassAway() override { if (SchemaActor) { Send(SchemaActor, new TEvPoisonPill()); - SchemaActor = TActorId(); + SchemaActor = TActorId(); } TActionActor<TCreateUserActor>::PassAway(); } @@ -85,7 +85,7 @@ private: } private: - TActorId SchemaActor; + TActorId SchemaActor; }; IActor* CreateCreateUserActor(const NKikimrClient::TSqsRequest& sourceSqsRequest, THolder<IReplyCallback> cb) { diff --git a/ydb/core/ymq/actor/defs.h b/ydb/core/ymq/actor/defs.h index 072279bb1fa..2ca60599036 100644 --- a/ydb/core/ymq/actor/defs.h +++ b/ydb/core/ymq/actor/defs.h @@ -4,7 +4,7 @@ namespace NKikimr::NSQS { -using NActors::TActorId; +using NActors::TActorId; using NActors::IActor; template <typename TDerived> diff --git a/ydb/core/ymq/actor/delete_queue.cpp b/ydb/core/ymq/actor/delete_queue.cpp index 35759c788ba..9711b917340 100644 --- a/ydb/core/ymq/actor/delete_queue.cpp +++ b/ydb/core/ymq/actor/delete_queue.cpp @@ -56,7 +56,7 @@ private: } void HandleQueueDeleted(TSqsEvents::TEvQueueDeleted::TPtr& ev) { - SchemaActor_ = TActorId(); + SchemaActor_ = TActorId(); if (!ev->Get()->Success) { MakeError(Response_.MutableDeleteQueue(), NErrors::INTERNAL_FAILURE, ev->Get()->Message); } @@ -67,7 +67,7 @@ private: void PassAway() override { if (SchemaActor_) { Send(SchemaActor_, new TEvPoisonPill()); - SchemaActor_ = TActorId(); + SchemaActor_ = TActorId(); } TActionActor<TDeleteQueueActor>::PassAway(); } @@ -77,7 +77,7 @@ private: } private: - TActorId SchemaActor_; + TActorId SchemaActor_; }; class TDeleteQueueBatchActor diff --git a/ydb/core/ymq/actor/events.h b/ydb/core/ymq/actor/events.h index ab034717a4e..9fff62c7cf8 100644 --- a/ydb/core/ymq/actor/events.h +++ b/ydb/core/ymq/actor/events.h @@ -18,7 +18,7 @@ #include <library/cpp/actors/core/event_pb.h> #include <library/cpp/actors/core/event_local.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <util/generic/hash.h> #include <util/generic/maybe.h> @@ -198,7 +198,7 @@ struct TSqsEvents { // Common info TString RootUrl; - TActorId SchemeCache; + TActorId SchemeCache; TActorId QueueLeader; }; @@ -207,7 +207,7 @@ struct TSqsEvents { struct TEvExecute : public NActors::TEventLocal<TEvExecute, EvExecute> { /// Query sender - TActorId Sender; + TActorId Sender; // User request id this query belongs to TString RequestId; /// Queue path in the catalog @@ -229,7 +229,7 @@ struct TSqsEvents { TEvExecute(const TEvExecute& other) = default; - TEvExecute(const TActorId& sender, TString requestId, const TQueuePath& path, const EQueryId idx, const ui64 shard = 0) + TEvExecute(const TActorId& sender, TString requestId, const TQueuePath& path, const EQueryId idx, const ui64 shard = 0) : Sender(sender) , RequestId(std::move(requestId)) , QueuePath(path) diff --git a/ydb/core/ymq/actor/executor.cpp b/ydb/core/ymq/actor/executor.cpp index e871b06f4d1..e17dc68fe95 100644 --- a/ydb/core/ymq/actor/executor.cpp +++ b/ydb/core/ymq/actor/executor.cpp @@ -37,7 +37,7 @@ static TString MiniKQLParamsToString(const NKikimrMiniKQL::TParams& params) { return val.GetValueText<NClient::TFormatJSON>(); } -TExecutorBuilder::TExecutorBuilder(TActorId parent, const TString& requestId) +TExecutorBuilder::TExecutorBuilder(TActorId parent, const TString& requestId) : Parent_(parent) , RequestId_(requestId) , ProposeTransactionRequest_(MakeHolder<TEvTxUserProxy::TEvProposeTransaction>()) @@ -116,7 +116,7 @@ const char* TExecutorBuilder::GetQueryById(size_t idx) { } TMiniKqlExecutionActor::TMiniKqlExecutionActor( - const TActorId sender, + const TActorId sender, TString requestId, THolder<TRequest> req, bool retryOnTimeout, @@ -435,7 +435,7 @@ TString TMiniKqlExecutionActor::GetRequestType() const { void TMiniKqlExecutionActor::PassAway() { if (TabletPipeClient_) { NTabletPipe::CloseClient(SelfId(), TabletPipeClient_); - TabletPipeClient_ = TActorId(); + TabletPipeClient_ = TActorId(); } TActorBootstrapped<TMiniKqlExecutionActor>::PassAway(); } diff --git a/ydb/core/ymq/actor/executor.h b/ydb/core/ymq/actor/executor.h index 914ac0dc6cd..5dd51c6da23 100644 --- a/ydb/core/ymq/actor/executor.h +++ b/ydb/core/ymq/actor/executor.h @@ -11,7 +11,7 @@ #include <library/cpp/actors/core/actor_bootstrapped.h> #include <library/cpp/actors/core/actor.h> #include <library/cpp/actors/core/hfunc.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <util/generic/hash.h> #include <util/generic/maybe.h> @@ -22,7 +22,7 @@ namespace NKikimr::NSQS { // Can either send TEvExecute to queue leader or create execution actor. class TExecutorBuilder { public: - explicit TExecutorBuilder(TActorId parent, const TString& requestId); + explicit TExecutorBuilder(TActorId parent, const TString& requestId); TExecutorBuilder& User(const TString& user) { UserName_ = user; @@ -129,7 +129,7 @@ private: const char* GetQueryById(size_t idx); private: - const TActorId Parent_; + const TActorId Parent_; TString RequestId_; THolder<TEvTxUserProxy::TEvProposeTransaction> ProposeTransactionRequest_; TMaybe<TParameters> Parameters_; @@ -153,7 +153,7 @@ class TMiniKqlExecutionActor public: TMiniKqlExecutionActor( - const TActorId sender, + const TActorId sender, TString requestId, THolder<TRequest> req, bool retryOnTimeout, @@ -221,7 +221,7 @@ private: }; private: - const TActorId Sender_; + const TActorId Sender_; const TString RequestId_; const TSqsEvents::TExecutedCallback Cb_; THolder<TRequest> Request_; @@ -242,7 +242,7 @@ private: // Waiting for transaction to complete TResponse::TPtr ResponseEvent_; - TActorId TabletPipeClient_; + TActorId TabletPipeClient_; }; } // namespace NKikimr::NSQS diff --git a/ydb/core/ymq/actor/garbage_collector.cpp b/ydb/core/ymq/actor/garbage_collector.cpp index 64c03ebe395..e82284c020a 100644 --- a/ydb/core/ymq/actor/garbage_collector.cpp +++ b/ydb/core/ymq/actor/garbage_collector.cpp @@ -14,7 +14,7 @@ #include <library/cpp/actors/core/actor_bootstrapped.h> #include <library/cpp/actors/core/hfunc.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <util/string/join.h> @@ -58,8 +58,8 @@ static TSchemeNode& FindOrAllocateSchemeNode(TSchemeNode* root, const TSchemePat class TSchemeTraversalActor : public TActorBootstrapped<TSchemeTraversalActor> { public: - TSchemeTraversalActor(const TActorId parentId, - const TActorId schemeCacheId, + TSchemeTraversalActor(const TActorId parentId, + const TActorId schemeCacheId, const TVector<TSchemePath>& pathsToTraverse, const ui64 maxDepth = Max()) : ParentId(parentId) @@ -157,8 +157,8 @@ private: } private: - const TActorId ParentId; - const TActorId SchemeCacheId; + const TActorId ParentId; + const TActorId SchemeCacheId; const TVector<TSchemePath> PathsToTraverse; const ui64 MaxDepth; @@ -169,9 +169,9 @@ private: class TGarbageSearcher : public TActorBootstrapped<TGarbageSearcher> { public: - TGarbageSearcher(const TActorId parentId, - const TActorId schemeCacheId, - const TActorId queuesListReaderId, + TGarbageSearcher(const TActorId parentId, + const TActorId schemeCacheId, + const TActorId queuesListReaderId, const ui64 minimumItemAgeSeconds) : ParentId(parentId) , SchemeCacheId(schemeCacheId) @@ -341,9 +341,9 @@ private: } private: - const TActorId ParentId; - const TActorId SchemeCacheId; - const TActorId QueuesListReaderId; + const TActorId ParentId; + const TActorId SchemeCacheId; + const TActorId QueuesListReaderId; const TDuration MinimumItemAge; TSchemePath RootPath; @@ -361,8 +361,8 @@ private: class TGarbageCleaner : public TActorBootstrapped<TGarbageCleaner> { public: - TGarbageCleaner(const TActorId parentId, - const TActorId schemeCacheId, + TGarbageCleaner(const TActorId parentId, + const TActorId schemeCacheId, TGarbageHint&& garbageHint) : ParentId(parentId) , SchemeCacheId(schemeCacheId) @@ -532,8 +532,8 @@ private: } private: - const TActorId ParentId; - const TActorId SchemeCacheId; + const TActorId ParentId; + const TActorId SchemeCacheId; const TGarbageHint GarbageHint; std::queue<TSchemeNode> NodesToRemove; TSchemeNode CurrentNode; @@ -543,7 +543,7 @@ private: class TGarbageCollector : public TActorBootstrapped<TGarbageCollector> { public: - TGarbageCollector(const TActorId schemeCacheId, const TActorId queuesListReaderId) + TGarbageCollector(const TActorId schemeCacheId, const TActorId queuesListReaderId) : SchemeCacheId(schemeCacheId) , QueuesListReaderId(queuesListReaderId) { @@ -782,14 +782,14 @@ public: } private: - const TActorId SchemeCacheId; - const TActorId QueuesListReaderId; + const TActorId SchemeCacheId; + const TActorId QueuesListReaderId; bool Scanning = false; THashMap<TString, TGarbageHint> CurrentGarbageHints; TVector<TCleaningResult> CleaningHistory; }; -IActor* CreateGarbageCollector(const TActorId schemeCacheId, const TActorId queuesListReaderId) { +IActor* CreateGarbageCollector(const TActorId schemeCacheId, const TActorId queuesListReaderId) { return new TGarbageCollector(schemeCacheId, queuesListReaderId); } diff --git a/ydb/core/ymq/actor/garbage_collector.h b/ydb/core/ymq/actor/garbage_collector.h index 6fd94a029fb..713b31cc1de 100644 --- a/ydb/core/ymq/actor/garbage_collector.h +++ b/ydb/core/ymq/actor/garbage_collector.h @@ -6,5 +6,5 @@ #include <ydb/core/ymq/base/action.h> namespace NKikimr::NSQS { -IActor* CreateGarbageCollector(const TActorId schemeCacheId, const TActorId queuesListReaderId); +IActor* CreateGarbageCollector(const TActorId schemeCacheId, const TActorId queuesListReaderId); } diff --git a/ydb/core/ymq/actor/migration.cpp b/ydb/core/ymq/actor/migration.cpp index f34b09d4914..e547e975581 100644 --- a/ydb/core/ymq/actor/migration.cpp +++ b/ydb/core/ymq/actor/migration.cpp @@ -18,8 +18,8 @@ class TAddColumnActor : public TActorBootstrapped<TAddColumnActor> { public: TAddColumnActor(const TString& userName, const TString& queueName, - const TActorId& parent, - const TActorId& schemeCache, + const TActorId& parent, + const TActorId& schemeCache, TIntrusivePtr<TQueueCounters> counters, const TString& tablePath, const TString& columnName, @@ -52,15 +52,15 @@ private: const TString TablePath; const TString ColumnName; const NScheme::TTypeId ColumnType; - const TActorId Parent; - const TActorId SchemeCache; + const TActorId Parent; + const TActorId SchemeCache; TIntrusivePtr<TQueueCounters> Counters; }; TAddColumnActor::TAddColumnActor(const TString& userName, const TString& queueName, - const TActorId& parent, - const TActorId& schemeCache, + const TActorId& parent, + const TActorId& schemeCache, TIntrusivePtr<TQueueCounters> counters, const TString& tablePath, const TString& columnName, diff --git a/ydb/core/ymq/actor/migration.h b/ydb/core/ymq/actor/migration.h index 29f06cadf43..501ec050f3f 100644 --- a/ydb/core/ymq/actor/migration.h +++ b/ydb/core/ymq/actor/migration.h @@ -44,7 +44,7 @@ private: const TString UserName; const TString QueueName; const TActorId QueueLeader; - const TActorId SchemeCache; + const TActorId SchemeCache; TIntrusivePtr<TQueueCounters> Counters; TDuration WaitBeforeMigration; bool IsFifoQueue = false; diff --git a/ydb/core/ymq/actor/proxy_actor.h b/ydb/core/ymq/actor/proxy_actor.h index 7415d12d5e2..b553716bf06 100644 --- a/ydb/core/ymq/actor/proxy_actor.h +++ b/ydb/core/ymq/actor/proxy_actor.h @@ -10,7 +10,7 @@ #include <ydb/core/ymq/base/debug_info.h> #include <library/cpp/actors/core/actor_bootstrapped.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> namespace NKikimr::NSQS { diff --git a/ydb/core/ymq/actor/proxy_service.cpp b/ydb/core/ymq/actor/proxy_service.cpp index 7e48e11cb61..1eecd851031 100644 --- a/ydb/core/ymq/actor/proxy_service.cpp +++ b/ydb/core/ymq/actor/proxy_service.cpp @@ -44,7 +44,7 @@ struct TSqsProxyService::TProxyRequestInfo : public TAtomicRefCount<TProxyReques } TString RequestId; - TActorId ProxyActorId; + TActorId ProxyActorId; TSqsEvents::TEvProxySqsRequest::TPtr ProxyRequest; }; diff --git a/ydb/core/ymq/actor/proxy_service.h b/ydb/core/ymq/actor/proxy_service.h index edec0b76a5e..a3c991105fe 100644 --- a/ydb/core/ymq/actor/proxy_service.h +++ b/ydb/core/ymq/actor/proxy_service.h @@ -33,7 +33,7 @@ struct TReplierToSenderActorCallback : public IReplyCallback { response.SetRequestId(Request->Get()->Record.GetRequestId()); RLOG_SQS_REQ_TRACE(Request->Get()->Record.GetRequestId(), "Sending sqs response: " << response); - const TActorId selfId = TActivationContext::AsActorContext().SelfID; + const TActorId selfId = TActivationContext::AsActorContext().SelfID; TActivationContext::Send( new IEventHandle( Request->Sender, diff --git a/ydb/core/ymq/actor/queue_leader.h b/ydb/core/ymq/actor/queue_leader.h index 8fc87490581..3504e4ddb1a 100644 --- a/ydb/core/ymq/actor/queue_leader.h +++ b/ydb/core/ymq/actor/queue_leader.h @@ -12,7 +12,7 @@ #include <library/cpp/actors/core/actorid.h> #include <library/cpp/actors/core/actor_bootstrapped.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <util/generic/guid.h> #include <util/generic/ptr.h> @@ -160,7 +160,7 @@ private: bool IsFifoQueue_ = false; TString QueueId_; ui64 QueueVersion_ = 0; - TActorId SchemeCache_; + TActorId SchemeCache_; TIntrusivePtr<TSqsEvents::TQuoterResourcesForActions> QuoterResources_; TLocalRateLimiterResource SendMessageQuoterResource_; TLocalRateLimiterResource ReceiveMessageQuoterResource_; @@ -414,10 +414,10 @@ private: TMessageDelayStatistics DelayStatistics_; // background actors - TActorId DeduplicationCleanupActor_; - TActorId ReadsCleanupActor_; - TActorId RetentionActor_; - TActorId PurgeActor_; + TActorId DeduplicationCleanupActor_; + TActorId ReadsCleanupActor_; + TActorId RetentionActor_; + TActorId PurgeActor_; struct TSendMessageBatchRequestProcessing { TSendMessageBatchRequestProcessing(TSqsEvents::TEvSendMessageBatch::TPtr&& ev); diff --git a/ydb/core/ymq/actor/queue_schema.cpp b/ydb/core/ymq/actor/queue_schema.cpp index cd19e997830..91d3c5d3df4 100644 --- a/ydb/core/ymq/actor/queue_schema.cpp +++ b/ydb/core/ymq/actor/queue_schema.cpp @@ -37,7 +37,7 @@ static bool IsGoodStatusCode(ui32 code) { TCreateQueueSchemaActorV2::TCreateQueueSchemaActorV2(const TQueuePath& path, const TCreateQueueRequest& req, - const TActorId& sender, + const TActorId& sender, const TString& requestId, const TString& customQueueName, const TString& folderId, @@ -490,7 +490,7 @@ void TCreateQueueSchemaActorV2::CreateComponents() { auto* cmd = ev->Record.MutableTransaction()->MutableModifyScheme()->MutableCreateTable(); cmd->MutablePartitionConfig()->MutablePipelineConfig()->SetEnableOutOfOrder(Request_.GetEnableOutOfOrderTransactionsExecution()); - const TActorId actorId = Register(new TMiniKqlExecutionActor( + const TActorId actorId = Register(new TMiniKqlExecutionActor( SelfId(), RequestId_, std::move(ev), false, QueuePath_, GetTransactionCounters(UserCounters_))); if (table.HasLeaderTablet && !CreateTableWithLeaderTabletActorId_) { @@ -700,7 +700,7 @@ void TCreateQueueSchemaActorV2::AddRPSQuota() { } void TCreateQueueSchemaActorV2::HandleAddQuoterResource(NKesus::TEvKesus::TEvAddQuoterResourceResult::TPtr& ev) { - AddQuoterResourceActor_ = TActorId(); + AddQuoterResourceActor_ = TActorId(); auto status = ev->Get()->Record.GetError().GetStatus(); if (status == Ydb::StatusIds::SUCCESS || status == Ydb::StatusIds::ALREADY_EXISTS) { RLOG_SQS_DEBUG("Successfully added quoter resource. Id: " << ev->Get()->Record.GetResourceId()); @@ -1233,13 +1233,13 @@ void TCreateQueueSchemaActorV2::OnAttributesMatch(TSqsEvents::TEvExecuted::TPtr& void TCreateQueueSchemaActorV2::PassAway() { if (AddQuoterResourceActor_) { Send(AddQuoterResourceActor_, new TEvPoisonPill()); - AddQuoterResourceActor_ = TActorId(); + AddQuoterResourceActor_ = TActorId(); } TActorBootstrapped<TCreateQueueSchemaActorV2>::PassAway(); } TDeleteQueueSchemaActorV2::TDeleteQueueSchemaActorV2(const TQueuePath& path, - const TActorId& sender, + const TActorId& sender, const TString& requestId, TIntrusivePtr<TUserCounters> userCounters) : QueuePath_(path) @@ -1251,7 +1251,7 @@ TDeleteQueueSchemaActorV2::TDeleteQueueSchemaActorV2(const TQueuePath& path, } TDeleteQueueSchemaActorV2::TDeleteQueueSchemaActorV2(const TQueuePath& path, - const TActorId& sender, + const TActorId& sender, const TString& requestId, TIntrusivePtr<TUserCounters> userCounters, const ui64 advisedQueueVersion, @@ -1493,7 +1493,7 @@ void TDeleteQueueSchemaActorV2::DeleteRPSQuota() { } void TDeleteQueueSchemaActorV2::HandleDeleteQuoterResource(NKesus::TEvKesus::TEvDeleteQuoterResourceResult::TPtr& ev) { - DeleteQuoterResourceActor_ = TActorId(); + DeleteQuoterResourceActor_ = TActorId(); auto status = ev->Get()->Record.GetError().GetStatus(); if (status == Ydb::StatusIds::SUCCESS || status == Ydb::StatusIds::NOT_FOUND) { RLOG_SQS_DEBUG("Successfully deleted quoter resource"); @@ -1511,7 +1511,7 @@ void TDeleteQueueSchemaActorV2::HandleDeleteQuoterResource(NKesus::TEvKesus::TEv void TDeleteQueueSchemaActorV2::PassAway() { if (DeleteQuoterResourceActor_) { Send(DeleteQuoterResourceActor_, new TEvPoisonPill()); - DeleteQuoterResourceActor_ = TActorId(); + DeleteQuoterResourceActor_ = TActorId(); } TActorBootstrapped<TDeleteQueueSchemaActorV2>::PassAway(); } diff --git a/ydb/core/ymq/actor/queue_schema.h b/ydb/core/ymq/actor/queue_schema.h index 06a344de322..793ba1c06ad 100644 --- a/ydb/core/ymq/actor/queue_schema.h +++ b/ydb/core/ymq/actor/queue_schema.h @@ -18,7 +18,7 @@ class TCreateQueueSchemaActorV2 public: TCreateQueueSchemaActorV2(const TQueuePath& path, const TCreateQueueRequest& req, - const TActorId& sender, + const TActorId& sender, const TString& requestId, const TString& customQueueName, const TString& folderId, @@ -104,7 +104,7 @@ private: const TQueuePath QueuePath_; const TCreateQueueRequest Request_; - const TActorId Sender_; + const TActorId Sender_; const TString CustomQueueName_; const TString FolderId_; const TString RequestId_; @@ -136,7 +136,7 @@ private: ECreateComponentsStep CurrentCreationStep_ = ECreateComponentsStep::GetTablesFormatSetting; - TActorId AddQuoterResourceActor_; + TActorId AddQuoterResourceActor_; }; class TDeleteQueueSchemaActorV2 @@ -144,12 +144,12 @@ class TDeleteQueueSchemaActorV2 { public: TDeleteQueueSchemaActorV2(const TQueuePath& path, - const TActorId& sender, + const TActorId& sender, const TString& requestId, TIntrusivePtr<TUserCounters> userCounters); TDeleteQueueSchemaActorV2(const TQueuePath& path, - const TActorId& sender, + const TActorId& sender, const TString& requestId, TIntrusivePtr<TUserCounters> userCounters, const ui64 advisedQueueVersion, @@ -196,14 +196,14 @@ private: }; const TQueuePath QueuePath_; - const TActorId Sender_; + const TActorId Sender_; TVector<TTable> Tables_; TVector<int> Shards_; ui32 SI_; const TString RequestId_; TIntrusivePtr<TUserCounters> UserCounters_; ui64 Version_ = 0; - TActorId DeleteQuoterResourceActor_; + TActorId DeleteQuoterResourceActor_; }; } // namespace NKikimr::NSQS diff --git a/ydb/core/ymq/actor/queues_list_reader.h b/ydb/core/ymq/actor/queues_list_reader.h index 04e69c03a80..6562ad4036f 100644 --- a/ydb/core/ymq/actor/queues_list_reader.h +++ b/ydb/core/ymq/actor/queues_list_reader.h @@ -42,7 +42,7 @@ private: TString CurrentQueue; THolder<TSqsEvents::TEvQueuesList> Result; bool ListingQueues = false; - THashSet<TActorId> Recipients; + THashSet<TActorId> Recipients; }; } // namespace NKikimr::NSQS diff --git a/ydb/core/ymq/actor/receive_message.cpp b/ydb/core/ymq/actor/receive_message.cpp index ca7a86fd432..2b5bf0e5174 100644 --- a/ydb/core/ymq/actor/receive_message.cpp +++ b/ydb/core/ymq/actor/receive_message.cpp @@ -12,7 +12,7 @@ #include <ydb/public/lib/value/value.h> #include <library/cpp/digest/md5/md5.h> -#include <library/cpp/monlib/metrics/histogram_collector.h> +#include <library/cpp/monlib/metrics/histogram_collector.h> using NKikimr::NClient::TValue; diff --git a/ydb/core/ymq/actor/schema.cpp b/ydb/core/ymq/actor/schema.cpp index 74f6188a7e9..20e84581fe5 100644 --- a/ydb/core/ymq/actor/schema.cpp +++ b/ydb/core/ymq/actor/schema.cpp @@ -249,7 +249,7 @@ THolder<TEvTxUserProxy::TEvProposeTransaction> TCreateUserSchemaActor::TCreateUserSchemaActor(const TString& root, const TString& userName, - const TActorId& sender, + const TActorId& sender, const TString& requestId, TIntrusivePtr<TUserCounters> userCounters) : Root_(root) @@ -359,7 +359,7 @@ void TCreateUserSchemaActor::HandleExecuted(TSqsEvents::TEvExecuted::TPtr& ev) { } void TCreateUserSchemaActor::HandleAddQuoterResource(NKesus::TEvKesus::TEvAddQuoterResourceResult::TPtr& ev) { - AddQuoterResourceActor_ = TActorId(); + AddQuoterResourceActor_ = TActorId(); auto status = ev->Get()->Record.GetError().GetStatus(); if (status == Ydb::StatusIds::SUCCESS || status == Ydb::StatusIds::ALREADY_EXISTS) { RLOG_SQS_DEBUG("Successfully added quoter resource. Id: " << ev->Get()->Record.GetResourceId()); @@ -382,14 +382,14 @@ void TCreateUserSchemaActor::AddRPSQuota() { void TCreateUserSchemaActor::PassAway() { if (AddQuoterResourceActor_) { Send(AddQuoterResourceActor_, new TEvPoisonPill()); - AddQuoterResourceActor_ = TActorId(); + AddQuoterResourceActor_ = TActorId(); } TActorBootstrapped<TCreateUserSchemaActor>::PassAway(); } TDeleteUserSchemaActor::TDeleteUserSchemaActor(const TString& root, const TString& name, - const TActorId& sender, + const TActorId& sender, const TString& requestId, TIntrusivePtr<TUserCounters> userCounters) : Root_(root) @@ -458,7 +458,7 @@ void TDeleteUserSchemaActor::HandleExecuted(TSqsEvents::TEvExecuted::TPtr& ev) { } } -TAtomicCounterActor::TAtomicCounterActor(const TActorId& sender, const TString& rootPath, const TString& requestId) +TAtomicCounterActor::TAtomicCounterActor(const TActorId& sender, const TString& rootPath, const TString& requestId) : Sender_(sender) , RootPath_(rootPath) , RequestId_(requestId) @@ -497,7 +497,7 @@ public: ui64 quoterPathId, const typename TEvCmd::ProtoRecordType& cmd, const TString& requestId, - const TActorId& parent + const TActorId& parent ) : QuoterSchemeShardId(quoterSchemeShardId) , QuoterPathId(quoterPathId) @@ -511,7 +511,7 @@ public: const TString& quoterPath, const typename TEvCmd::ProtoRecordType& cmd, const TString& requestId, - const TActorId& parent + const TActorId& parent ) : QuoterPath(quoterPath) , Cmd(cmd) @@ -579,7 +579,7 @@ private: void PassAway() override { if (PipeClient) { NTabletPipe::CloseClient(this->SelfId(), PipeClient); - PipeClient = TActorId(); + PipeClient = TActorId(); } TActorBootstrapped<TQuoterCmdRunner>::PassAway(); } @@ -620,12 +620,12 @@ private: const TString QuoterPath; const typename TEvCmd::ProtoRecordType Cmd; const TString RequestId_; - const TActorId Parent; + const TActorId Parent; ui64 QuoterTabletId = 0; - TActorId PipeClient; + TActorId PipeClient; }; -TActorId RunAddQuoterResource(ui64 quoterSchemeShardId, ui64 quoterPathId, const NKikimrKesus::TEvAddQuoterResource& cmd, const TString& requestId) { +TActorId RunAddQuoterResource(ui64 quoterSchemeShardId, ui64 quoterPathId, const NKikimrKesus::TEvAddQuoterResource& cmd, const TString& requestId) { return TActivationContext::Register( new TQuoterCmdRunner<NKesus::TEvKesus::TEvAddQuoterResource, NKesus::TEvKesus::TEvAddQuoterResourceResult>( quoterSchemeShardId, quoterPathId, cmd, requestId, TActivationContext::AsActorContext().SelfID @@ -633,7 +633,7 @@ TActorId RunAddQuoterResource(ui64 quoterSchemeShardId, ui64 quoterPathId, const ); } -TActorId RunAddQuoterResource(const TString& quoterPath, const NKikimrKesus::TEvAddQuoterResource& cmd, const TString& requestId) { +TActorId RunAddQuoterResource(const TString& quoterPath, const NKikimrKesus::TEvAddQuoterResource& cmd, const TString& requestId) { return TActivationContext::Register( new TQuoterCmdRunner<NKesus::TEvKesus::TEvAddQuoterResource, NKesus::TEvKesus::TEvAddQuoterResourceResult>( quoterPath, cmd, requestId, TActivationContext::AsActorContext().SelfID @@ -641,7 +641,7 @@ TActorId RunAddQuoterResource(const TString& quoterPath, const NKikimrKesus::TEv ); } -TActorId RunDeleteQuoterResource(const TString& quoterPath, const NKikimrKesus::TEvDeleteQuoterResource& cmd, const TString& requestId) { +TActorId RunDeleteQuoterResource(const TString& quoterPath, const NKikimrKesus::TEvDeleteQuoterResource& cmd, const TString& requestId) { return TActivationContext::Register( new TQuoterCmdRunner<NKesus::TEvKesus::TEvDeleteQuoterResource, NKesus::TEvKesus::TEvDeleteQuoterResourceResult>( quoterPath, cmd, requestId, TActivationContext::AsActorContext().SelfID diff --git a/ydb/core/ymq/actor/schema.h b/ydb/core/ymq/actor/schema.h index 5619d6a793c..c88face921b 100644 --- a/ydb/core/ymq/actor/schema.h +++ b/ydb/core/ymq/actor/schema.h @@ -37,9 +37,9 @@ THolder<TEvTxUserProxy::TEvProposeTransaction> MakeRemoveDirectoryEvent(const TString& root, const TString& name); // Create actor that calls AddQuoterResource and handles pipe errors and retries -TActorId RunAddQuoterResource(ui64 quoterSchemeShardId, ui64 quoterPathId, const NKikimrKesus::TEvAddQuoterResource& cmd, const TString& requestId); -TActorId RunAddQuoterResource(const TString& quoterPath, const NKikimrKesus::TEvAddQuoterResource& cmd, const TString& requestId); -TActorId RunDeleteQuoterResource(const TString& quoterPath, const NKikimrKesus::TEvDeleteQuoterResource& cmd, const TString& requestId); +TActorId RunAddQuoterResource(ui64 quoterSchemeShardId, ui64 quoterPathId, const NKikimrKesus::TEvAddQuoterResource& cmd, const TString& requestId); +TActorId RunAddQuoterResource(const TString& quoterPath, const NKikimrKesus::TEvAddQuoterResource& cmd, const TString& requestId); +TActorId RunDeleteQuoterResource(const TString& quoterPath, const NKikimrKesus::TEvDeleteQuoterResource& cmd, const TString& requestId); inline TIntrusivePtr<TTransactionCounters> GetTransactionCounters(const TIntrusivePtr<TUserCounters>& userCounters) { if (userCounters) { @@ -52,7 +52,7 @@ class TCreateUserSchemaActor : public TActorBootstrapped<TCreateUserSchemaActor> { public: - TCreateUserSchemaActor(const TString& root, const TString& userName, const TActorId& sender, const TString& requestId, TIntrusivePtr<TUserCounters> userCounters); + TCreateUserSchemaActor(const TString& root, const TString& userName, const TActorId& sender, const TString& requestId, TIntrusivePtr<TUserCounters> userCounters); ~TCreateUserSchemaActor(); void Bootstrap(); @@ -95,20 +95,20 @@ private: const TString Root_; const TString UserName_; - const TActorId Sender_; + const TActorId Sender_; int SI_; const TString RequestId_; bool CreateRootSqsDirAttemptWasMade_ = false; TIntrusivePtr<TUserCounters> UserCounters_; std::pair<ui64, ui64> KesusPathId_ = {}; // SchemeShardTableId, PathId for quoter kesus - TActorId AddQuoterResourceActor_; + TActorId AddQuoterResourceActor_; }; class TDeleteUserSchemaActor : public TActorBootstrapped<TDeleteUserSchemaActor> { public: - TDeleteUserSchemaActor(const TString& root, const TString& name, const TActorId& sender, const TString& requestId, TIntrusivePtr<TUserCounters> userCounters); + TDeleteUserSchemaActor(const TString& root, const TString& name, const TActorId& sender, const TString& requestId, TIntrusivePtr<TUserCounters> userCounters); ~TDeleteUserSchemaActor(); void Bootstrap(); @@ -142,7 +142,7 @@ private: const TString Root_; const TString Name_; - const TActorId Sender_; + const TActorId Sender_; ui32 SI_; const TString RequestId_; TIntrusivePtr<TUserCounters> UserCounters_; @@ -152,7 +152,7 @@ class TAtomicCounterActor : public TActorBootstrapped<TAtomicCounterActor> { public: - TAtomicCounterActor(const TActorId& sender, const TString& rootPath, const TString& requestId); + TAtomicCounterActor(const TActorId& sender, const TString& rootPath, const TString& requestId); ~TAtomicCounterActor(); void Bootstrap(); @@ -171,7 +171,7 @@ private: void HandleExecuted(TSqsEvents::TEvExecuted::TPtr& ev); private: - const TActorId Sender_; + const TActorId Sender_; const TString RootPath_; const TString RequestId_; }; diff --git a/ydb/core/ymq/actor/service.cpp b/ydb/core/ymq/actor/service.cpp index 76926cd0291..34c9a4ae46b 100644 --- a/ydb/core/ymq/actor/service.cpp +++ b/ydb/core/ymq/actor/service.cpp @@ -188,10 +188,10 @@ struct TSqsService::TQueueInfo : public TAtomicRefCount<TQueueInfo> { ui64 LeaderTabletId_ = 0; TIntrusivePtr<TQueueCounters> Counters_; TIntrusivePtr<TUserCounters> UserCounters_; - TActorId PipeClient_; + TActorId PipeClient_; TActorId LeaderPipeServer_; TActorId LocalLeader_; - TActorId SchemeCache_; + TActorId SchemeCache_; ui64 LocalLeaderRefCount_ = 0; TIntrusivePtr<TSqsEvents::TQuoterResourcesForActions> QuoterResourcesForUser_; diff --git a/ydb/core/ymq/actor/service.h b/ydb/core/ymq/actor/service.h index c63a417f223..caf091c24b8 100644 --- a/ydb/core/ymq/actor/service.h +++ b/ydb/core/ymq/actor/service.h @@ -140,8 +140,8 @@ private: TUsersMap Users_; THashMap<ui64, TQueueInfoPtr> LeaderTabletIdToQueue_; THashMap<TActorId, TQueueInfoPtr> LocalLeaderRefs_; // referer -> queue info - TActorId SchemeCache_; - TActorId QueuesListReader_; + TActorId SchemeCache_; + TActorId QueuesListReader_; // State machine bool RequestingUsersList_ = false; diff --git a/ydb/core/ymq/actor/serviceid.h b/ydb/core/ymq/actor/serviceid.h index 976d2bd7963..7219d918ba0 100644 --- a/ydb/core/ymq/actor/serviceid.h +++ b/ydb/core/ymq/actor/serviceid.h @@ -9,25 +9,25 @@ namespace NKikimr::NSQS { -inline TActorId MakeSqsServiceID(ui32 nodeId) { +inline TActorId MakeSqsServiceID(ui32 nodeId) { Y_VERIFY(nodeId != 0); return TActorId(nodeId, TStringBuf("SQS_SERVICE")); } -inline TActorId MakeSqsProxyServiceID(ui32 nodeId) { +inline TActorId MakeSqsProxyServiceID(ui32 nodeId) { Y_VERIFY(nodeId != 0); return TActorId(nodeId, TStringBuf("SQS_PROXY")); } -inline TActorId MakeSqsAccessServiceID() { +inline TActorId MakeSqsAccessServiceID() { return TActorId(0, TStringBuf("SQS_ACCESS")); } -inline TActorId MakeSqsFolderServiceID() { +inline TActorId MakeSqsFolderServiceID() { return TActorId(0, TStringBuf("SQS_FOLDER")); } -inline TActorId MakeSqsMeteringServiceID() { +inline TActorId MakeSqsMeteringServiceID() { return TActorId(0, TStringBuf("SQS_METER")); } diff --git a/ydb/core/ymq/actor/ya.make b/ydb/core/ymq/actor/ya.make index d0980c76569..070653bb577 100644 --- a/ydb/core/ymq/actor/ya.make +++ b/ydb/core/ymq/actor/ya.make @@ -62,8 +62,8 @@ PEERDIR( library/cpp/grpc/client library/cpp/logger library/cpp/lwtrace/mon - library/cpp/monlib/dynamic_counters - library/cpp/monlib/service/pages + library/cpp/monlib/dynamic_counters + library/cpp/monlib/service/pages library/cpp/scheme ydb/core/base ydb/core/client/minikql_compile diff --git a/ydb/core/ymq/actor/yc_search_ut/index_events_processor_ut.cpp b/ydb/core/ymq/actor/yc_search_ut/index_events_processor_ut.cpp index 96b12a86e2c..e484f117be1 100644 --- a/ydb/core/ymq/actor/yc_search_ut/index_events_processor_ut.cpp +++ b/ydb/core/ymq/actor/yc_search_ut/index_events_processor_ut.cpp @@ -13,7 +13,7 @@ using namespace NYdb; class TIndexProcesorTests : public TTestBase { using EEvType = TSearchEventsProcessor::EQueueEventType; - + public: TIndexProcesorTests() { TPortManager portManager; @@ -22,7 +22,7 @@ public: auto settings = TServerSettings(mbusPort); settings.SetDomainName("Root"); Server = MakeHolder<TServer>(settings); - Server->EnableGRpc(NGrpc::TServerOptions().SetHost("localhost").SetPort(grpcPort)); + Server->EnableGRpc(NGrpc::TServerOptions().SetHost("localhost").SetPort(grpcPort)); auto driverConfig = TDriverConfig().SetEndpoint(TStringBuilder() << "localhost:" << grpcPort); Driver = MakeHolder<TDriver>(driverConfig); @@ -239,7 +239,7 @@ private: client.InitRootScheme(); client.MkDir("/Root", "SQS"); } - + private: THolder<TDriver> Driver; TSimpleSharedPtr<NYdb::NTable::TTableClient> TableClient; @@ -254,7 +254,7 @@ private: UNIT_TEST(TestManyMessages) UNIT_TEST(TestOver1000Queues) UNIT_TEST_SUITE_END(); - + void CheckEventsLine( const TString& line, EEvType type, const TString& queueName = TString()) { @@ -294,7 +294,7 @@ private: void TestCreateIndexProcessor() { TTestRunner("CreateIndexProcessor", this); } - + void TestSingleCreateQueueEvent() { TTestRunner runner{"SingleCreateQueueEvent", this}; runner.AddEvent( "cloud1", "queue1", EEvType::Created); diff --git a/ydb/core/ymq/base/counters.h b/ydb/core/ymq/base/counters.h index e6f17791649..3c2667b2410 100644 --- a/ydb/core/ymq/base/counters.h +++ b/ydb/core/ymq/base/counters.h @@ -9,10 +9,10 @@ #include <ydb/library/yql/minikql/aligned_page_pool.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <util/system/spinlock.h> -#include <util/generic/hash.h> +#include <util/generic/hash.h> #include <atomic> #include <utility> diff --git a/ydb/core/ymq/base/ya.make b/ydb/core/ymq/base/ya.make index 5e768bc94fc..14cd4e72a10 100644 --- a/ydb/core/ymq/base/ya.make +++ b/ydb/core/ymq/base/ya.make @@ -28,7 +28,7 @@ PEERDIR( library/cpp/cgiparam library/cpp/ipmath library/cpp/lwtrace - library/cpp/monlib/dynamic_counters + library/cpp/monlib/dynamic_counters library/cpp/scheme library/cpp/string_utils/base64 ydb/core/base diff --git a/ydb/library/schlab/mon/mon.cpp b/ydb/library/schlab/mon/mon.cpp index 84b2254e09a..fc06b412ae0 100644 --- a/ydb/library/schlab/mon/mon.cpp +++ b/ydb/library/schlab/mon/mon.cpp @@ -1,9 +1,9 @@ #include "mon.h" #include <ydb/library/schlab/schemu/schemu.h> -#include <library/cpp/monlib/service/pages/mon_page.h> -#include <library/cpp/monlib/service/pages/resource_mon_page.h> -#include <library/cpp/monlib/service/pages/templates.h> +#include <library/cpp/monlib/service/pages/mon_page.h> +#include <library/cpp/monlib/service/pages/resource_mon_page.h> +#include <library/cpp/monlib/service/pages/templates.h> #include <library/cpp/resource/resource.h> #include <library/cpp/html/pcdata/pcdata.h> #include <util/stream/str.h> diff --git a/ydb/library/schlab/mon/mon.h b/ydb/library/schlab/mon/mon.h index c2034b56335..82d17d334c2 100644 --- a/ydb/library/schlab/mon/mon.h +++ b/ydb/library/schlab/mon/mon.h @@ -1,6 +1,6 @@ #pragma once -#include <library/cpp/monlib/service/monservice.h> +#include <library/cpp/monlib/service/monservice.h> namespace NKikimr { diff --git a/ydb/library/schlab/mon/test/test.cpp b/ydb/library/schlab/mon/test/test.cpp index c52d50e1a7b..64bf719d9a8 100644 --- a/ydb/library/schlab/mon/test/test.cpp +++ b/ydb/library/schlab/mon/test/test.cpp @@ -1,7 +1,7 @@ #include <ydb/library/schlab/mon/mon.h> #include <library/cpp/getopt/last_getopt.h> -#include <library/cpp/monlib/service/monservice.h> -#include <library/cpp/monlib/service/pages/resource_mon_page.h> +#include <library/cpp/monlib/service/monservice.h> +#include <library/cpp/monlib/service/pages/resource_mon_page.h> #include <library/cpp/resource/resource.h> namespace { diff --git a/ydb/library/schlab/mon/test/ya.make b/ydb/library/schlab/mon/test/ya.make index a90cf5bf18c..bc282bfcf41 100644 --- a/ydb/library/schlab/mon/test/ya.make +++ b/ydb/library/schlab/mon/test/ya.make @@ -15,7 +15,7 @@ SRCS( PEERDIR( library/cpp/getopt - library/cpp/monlib/dynamic_counters + library/cpp/monlib/dynamic_counters ydb/library/schlab/mon ) diff --git a/ydb/library/schlab/mon/ya.make b/ydb/library/schlab/mon/ya.make index 4396e3d2da2..ca05b8cdc34 100644 --- a/ydb/library/schlab/mon/ya.make +++ b/ydb/library/schlab/mon/ya.make @@ -25,7 +25,7 @@ SRCS( PEERDIR( library/cpp/html/pcdata - library/cpp/monlib/dynamic_counters + library/cpp/monlib/dynamic_counters library/cpp/resource ydb/library/schlab/schemu ) diff --git a/ydb/library/yql/ast/ya.make b/ydb/library/yql/ast/ya.make index d5cad895ffb..2d55860c898 100644 --- a/ydb/library/yql/ast/ya.make +++ b/ydb/library/yql/ast/ya.make @@ -14,8 +14,8 @@ SRCS( yql_constraint.h yql_ast_annotation.cpp yql_ast_annotation.h - yql_ast_escaping.cpp - yql_ast_escaping.h + yql_ast_escaping.cpp + yql_ast_escaping.h yql_errors.cpp yql_errors.h yql_expr.cpp diff --git a/ydb/library/yql/ast/yql_ast.cpp b/ydb/library/yql/ast/yql_ast.cpp index 94d6f15975a..16e521d9010 100644 --- a/ydb/library/yql/ast/yql_ast.cpp +++ b/ydb/library/yql/ast/yql_ast.cpp @@ -1,5 +1,5 @@ #include "yql_ast.h" -#include "yql_ast_escaping.h" +#include "yql_ast_escaping.h" #include <util/string/builder.h> #include <util/system/compiler.h> @@ -306,38 +306,38 @@ namespace { } // hex string starts with 'x"' else if (IsHexStringStart(c)) { - Ctx_.Next(); // skip 'x' + Ctx_.Next(); // skip 'x' if (Ctx_.AtEnd()) break; if (!IsStringStart(Ctx_.Peek())) { continue; } - Ctx_.Next(); // skip first '"' - - size_t readBytes = 0; - TStringStream ss; - TStringBuf atom = Ctx_.Str().SubStr(Ctx_.Offset()); - EUnescapeResult unescapeResult = UnescapeBinaryAtom( - atom, '"', &ss, &readBytes); - - // advance position - while (readBytes-- != 0) { - Ctx_.Next(); - } - + Ctx_.Next(); // skip first '"' + + size_t readBytes = 0; + TStringStream ss; + TStringBuf atom = Ctx_.Str().SubStr(Ctx_.Offset()); + EUnescapeResult unescapeResult = UnescapeBinaryAtom( + atom, '"', &ss, &readBytes); + + // advance position + while (readBytes-- != 0) { + Ctx_.Next(); + } + if (unescapeResult != EUnescapeResult::OK) { AddError(TString(UnescapeResultToString(unescapeResult))); return nullptr; } - Ctx_.Next(); // skip last '"' + Ctx_.Next(); // skip last '"' if (!Ctx_.IsAtomEnded()) { AddError("Unexpected end of \""); return nullptr; } - return TAstNode::NewAtom(resPosition, ss.Str(), Ctx_.Pool(), TNodeFlags::BinaryContent); + return TAstNode::NewAtom(resPosition, ss.Str(), Ctx_.Pool(), TNodeFlags::BinaryContent); } else if (IsStringStart(c)) { if (Ctx_.Offset() != atomStart) { @@ -345,19 +345,19 @@ namespace { return nullptr; } - Ctx_.Next(); // skip first '"' - - size_t readBytes = 0; - TStringStream ss; - TStringBuf atom = Ctx_.Str().SubStr(Ctx_.Offset()); - EUnescapeResult unescapeResult = UnescapeArbitraryAtom( - atom, '"', &ss, &readBytes); - - // advance position - while (readBytes-- != 0) { - Ctx_.Next(); - } - + Ctx_.Next(); // skip first '"' + + size_t readBytes = 0; + TStringStream ss; + TStringBuf atom = Ctx_.Str().SubStr(Ctx_.Offset()); + EUnescapeResult unescapeResult = UnescapeArbitraryAtom( + atom, '"', &ss, &readBytes); + + // advance position + while (readBytes-- != 0) { + Ctx_.Next(); + } + if (unescapeResult != EUnescapeResult::OK) { AddError(TString(UnescapeResultToString(unescapeResult))); return nullptr; @@ -368,7 +368,7 @@ namespace { return nullptr; } - return TAstNode::NewAtom(resPosition, ss.Str(), Ctx_.Pool(), TNodeFlags::ArbitraryContent); + return TAstNode::NewAtom(resPosition, ss.Str(), Ctx_.Pool(), TNodeFlags::ArbitraryContent); } } @@ -472,9 +472,9 @@ namespace { void PrintNode(IOutputStream& out, const TAstNode& node) { if (node.GetType() == TAstNode::Atom) { if (node.GetFlags() & TNodeFlags::ArbitraryContent) { - EscapeArbitraryAtom(node.GetContent(), '"', &out); + EscapeArbitraryAtom(node.GetContent(), '"', &out); } else if (node.GetFlags() & TNodeFlags::BinaryContent) { - EscapeBinaryAtom(node.GetContent(), '"', &out); + EscapeBinaryAtom(node.GetContent(), '"', &out); } else if (node.GetFlags() & TNodeFlags::MultilineContent) { MultilineAtomPrint(out, node.GetContent()); } else if (node.GetContent().empty()) { @@ -520,17 +520,17 @@ namespace { { out << node.GetContent(); } else { - EscapeArbitraryAtom(node.GetContent(), '"', &out); + EscapeArbitraryAtom(node.GetContent(), '"', &out); } } else if (node.GetFlags() & TNodeFlags::BinaryContent) { - EscapeBinaryAtom(node.GetContent(), '"', &out); + EscapeBinaryAtom(node.GetContent(), '"', &out); } else if (node.GetFlags() & TNodeFlags::MultilineContent) { MultilineAtomPrint(out, node.GetContent()); } else { if (((flags & TAstPrintFlags::AdaptArbitraryContent) && NeedEscaping(node.GetContent())) || node.GetContent().empty()) { - EscapeArbitraryAtom(node.GetContent(), '"', &out); + EscapeArbitraryAtom(node.GetContent(), '"', &out); } else { out << node.GetContent(); } diff --git a/ydb/library/yql/ast/yql_ast_escaping.cpp b/ydb/library/yql/ast/yql_ast_escaping.cpp index d73bfafb226..56aee8f5964 100644 --- a/ydb/library/yql/ast/yql_ast_escaping.cpp +++ b/ydb/library/yql/ast/yql_ast_escaping.cpp @@ -1,112 +1,112 @@ -#include "yql_ast_escaping.h" - -#include <util/charset/wide.h> -#include <util/stream/output.h> -#include <util/string/hex.h> - - -namespace NYql { - -static char HexDigit(char c) -{ - return (c < 10 ? '0' + c : 'A' + (c - 10)); -} - +#include "yql_ast_escaping.h" + +#include <util/charset/wide.h> +#include <util/stream/output.h> +#include <util/string/hex.h> + + +namespace NYql { + +static char HexDigit(char c) +{ + return (c < 10 ? '0' + c : 'A' + (c - 10)); +} + static void EscapedPrintChar(ui8 c, IOutputStream* out) -{ - switch (c) { - case '\\': out->Write("\\\\", 2); break; - case '"' : out->Write("\\\"", 2); break; - case '\t': out->Write("\\t", 2); break; - case '\n': out->Write("\\n", 2); break; - case '\r': out->Write("\\r", 2); break; - case '\b': out->Write("\\b", 2); break; - case '\f': out->Write("\\f", 2); break; - case '\a': out->Write("\\a", 2); break; - case '\v': out->Write("\\v", 2); break; - default: { - if (isprint(c)) out->Write(static_cast<char>(c)); - else { - char buf[4] = { "\\x" }; - buf[2] = HexDigit((c & 0xf0) >> 4); - buf[3] = HexDigit((c & 0x0f)); - out->Write(buf, 4); - } - } - } -} - +{ + switch (c) { + case '\\': out->Write("\\\\", 2); break; + case '"' : out->Write("\\\"", 2); break; + case '\t': out->Write("\\t", 2); break; + case '\n': out->Write("\\n", 2); break; + case '\r': out->Write("\\r", 2); break; + case '\b': out->Write("\\b", 2); break; + case '\f': out->Write("\\f", 2); break; + case '\a': out->Write("\\a", 2); break; + case '\v': out->Write("\\v", 2); break; + default: { + if (isprint(c)) out->Write(static_cast<char>(c)); + else { + char buf[4] = { "\\x" }; + buf[2] = HexDigit((c & 0xf0) >> 4); + buf[3] = HexDigit((c & 0x0f)); + out->Write(buf, 4); + } + } + } +} + static void EscapedPrintUnicode(wchar32 rune, IOutputStream* out) -{ - static const int MAX_ESCAPE_LEN = 10; - - if (rune < 0x80) { - EscapedPrintChar(static_cast<ui8>(rune & 0xff), out); - } else { - int i = 0; - char buf[MAX_ESCAPE_LEN]; - - if (rune < 0x10000) { - buf[i++] = '\\'; - buf[i++] = 'u'; - } else { - buf[i++] = '\\'; - buf[i++] = 'U'; - buf[i++] = HexDigit((rune & 0xf0000000) >> 28); - buf[i++] = HexDigit((rune & 0x0f000000) >> 24); - buf[i++] = HexDigit((rune & 0x00f00000) >> 20); - buf[i++] = HexDigit((rune & 0x000f0000) >> 16); - } - - buf[i++] = HexDigit((rune & 0xf000) >> 12); - buf[i++] = HexDigit((rune & 0x0f00) >> 8); - buf[i++] = HexDigit((rune & 0x00f0) >> 4); - buf[i++] = HexDigit((rune & 0x000f)); - - out->Write(buf, i); - } -} - -static bool TryParseOctal(const char*& p, const char* e, int maxlen, wchar32* value) -{ - while (maxlen-- && p != e) { - if (*value > 255) return false; - - char ch = *p++; - if (ch >= '0' && ch <= '7') { - *value = *value * 8 + (ch - '0'); - continue; - } - - break; - } - - return (maxlen == -1); -} - -static bool TryParseHex(const char*& p, const char* e, int maxlen, wchar32* value) -{ - while (maxlen-- > 0 && p != e) { - char ch = *p++; - if (ch >= '0' && ch <= '9') { - *value = *value * 16 + (ch - '0'); - continue; - } - - // to lower case - ch |= 0x20; - - if (ch >= 'a' && ch <= 'f') { - *value = *value * 16 + (ch - 'a') + 10; - continue; - } - - break; - } - - return (maxlen == -1); -} - +{ + static const int MAX_ESCAPE_LEN = 10; + + if (rune < 0x80) { + EscapedPrintChar(static_cast<ui8>(rune & 0xff), out); + } else { + int i = 0; + char buf[MAX_ESCAPE_LEN]; + + if (rune < 0x10000) { + buf[i++] = '\\'; + buf[i++] = 'u'; + } else { + buf[i++] = '\\'; + buf[i++] = 'U'; + buf[i++] = HexDigit((rune & 0xf0000000) >> 28); + buf[i++] = HexDigit((rune & 0x0f000000) >> 24); + buf[i++] = HexDigit((rune & 0x00f00000) >> 20); + buf[i++] = HexDigit((rune & 0x000f0000) >> 16); + } + + buf[i++] = HexDigit((rune & 0xf000) >> 12); + buf[i++] = HexDigit((rune & 0x0f00) >> 8); + buf[i++] = HexDigit((rune & 0x00f0) >> 4); + buf[i++] = HexDigit((rune & 0x000f)); + + out->Write(buf, i); + } +} + +static bool TryParseOctal(const char*& p, const char* e, int maxlen, wchar32* value) +{ + while (maxlen-- && p != e) { + if (*value > 255) return false; + + char ch = *p++; + if (ch >= '0' && ch <= '7') { + *value = *value * 8 + (ch - '0'); + continue; + } + + break; + } + + return (maxlen == -1); +} + +static bool TryParseHex(const char*& p, const char* e, int maxlen, wchar32* value) +{ + while (maxlen-- > 0 && p != e) { + char ch = *p++; + if (ch >= '0' && ch <= '9') { + *value = *value * 16 + (ch - '0'); + continue; + } + + // to lower case + ch |= 0x20; + + if (ch >= 'a' && ch <= 'f') { + *value = *value * 16 + (ch - 'a') + 10; + continue; + } + + break; + } + + return (maxlen == -1); +} + static bool IsValidUtf8Rune(wchar32 value) { return value <= 0x10ffff && (value < 0xd800 || value > 0xdfff); } @@ -133,85 +133,85 @@ TStringBuf UnescapeResultToString(EUnescapeResult result) } void EscapeArbitraryAtom(TStringBuf atom, char quoteChar, IOutputStream* out) -{ - out->Write(quoteChar); - const ui8 *p = reinterpret_cast<const ui8*>(atom.begin()), - *e = reinterpret_cast<const ui8*>(atom.end()); - while (p != e) { - wchar32 rune = 0; +{ + out->Write(quoteChar); + const ui8 *p = reinterpret_cast<const ui8*>(atom.begin()), + *e = reinterpret_cast<const ui8*>(atom.end()); + while (p != e) { + wchar32 rune = 0; size_t rune_len = 0; if (SafeReadUTF8Char(rune, rune_len, p, e) == RECODE_RESULT::RECODE_OK && IsValidUtf8Rune(rune)) { - EscapedPrintUnicode(rune, out); + EscapedPrintUnicode(rune, out); p += rune_len; - } else { - EscapedPrintChar(*p++, out); - } - } - out->Write(quoteChar); -} - -EUnescapeResult UnescapeArbitraryAtom( + } else { + EscapedPrintChar(*p++, out); + } + } + out->Write(quoteChar); +} + +EUnescapeResult UnescapeArbitraryAtom( TStringBuf atom, char endChar, IOutputStream* out, size_t* readBytes) -{ - const char *p = atom.begin(), - *e = atom.end(); - - while (p != e) { - char current = *p++; - - // C-style escape sequences - if (current == '\\') { - if (p == e) { - *readBytes = p - atom.begin(); - return EUnescapeResult::INVALID_ESCAPE_SEQUENCE; - } - - char next = *p++; - switch (next) { - case 't': current = '\t'; break; - case 'n': current = '\n'; break; - case 'r': current = '\r'; break; - case 'b': current = '\b'; break; - case 'f': current = '\f'; break; - case 'a': current = '\a'; break; - case 'v': current = '\v'; break; - case '0': case '1': case '2': case '3': { - wchar32 value = (next - '0'); - if (!TryParseOctal(p, e, 2, &value)) { - *readBytes = p - atom.begin(); - return EUnescapeResult::INVALID_OCTAL; - } - current = value & 0xff; - break; - } - case 'x': { - wchar32 value = 0; - if (!TryParseHex(p, e, 2, &value)) { - *readBytes = p - atom.begin(); - return EUnescapeResult::INVALID_HEXADECIMAL; - } - current = value & 0xff; - break; - } - case 'u': - case 'U': { - wchar32 value = 0; - int len = (next == 'u' ? 4 : 8); +{ + const char *p = atom.begin(), + *e = atom.end(); + + while (p != e) { + char current = *p++; + + // C-style escape sequences + if (current == '\\') { + if (p == e) { + *readBytes = p - atom.begin(); + return EUnescapeResult::INVALID_ESCAPE_SEQUENCE; + } + + char next = *p++; + switch (next) { + case 't': current = '\t'; break; + case 'n': current = '\n'; break; + case 'r': current = '\r'; break; + case 'b': current = '\b'; break; + case 'f': current = '\f'; break; + case 'a': current = '\a'; break; + case 'v': current = '\v'; break; + case '0': case '1': case '2': case '3': { + wchar32 value = (next - '0'); + if (!TryParseOctal(p, e, 2, &value)) { + *readBytes = p - atom.begin(); + return EUnescapeResult::INVALID_OCTAL; + } + current = value & 0xff; + break; + } + case 'x': { + wchar32 value = 0; + if (!TryParseHex(p, e, 2, &value)) { + *readBytes = p - atom.begin(); + return EUnescapeResult::INVALID_HEXADECIMAL; + } + current = value & 0xff; + break; + } + case 'u': + case 'U': { + wchar32 value = 0; + int len = (next == 'u' ? 4 : 8); if (!TryParseHex(p, e, len, &value) || !IsValidUtf8Rune(value)) { - *readBytes = p - atom.begin(); - return EUnescapeResult::INVALID_UNICODE; - } - size_t written = 0; - char buf[4]; - WideToUTF8(&value, 1, buf, written); - out->Write(buf, written); - continue; - } + *readBytes = p - atom.begin(); + return EUnescapeResult::INVALID_UNICODE; + } + size_t written = 0; + char buf[4]; + WideToUTF8(&value, 1, buf, written); + out->Write(buf, written); + continue; + } default: { current = next; } - } + } } else if (endChar == '`') { if (current == '`') { if (p == e) { @@ -226,50 +226,50 @@ EUnescapeResult UnescapeArbitraryAtom( } } } - } else if (current == endChar) { - *readBytes = p - atom.begin(); - return EUnescapeResult::OK; - } - - out->Write(current); - } - - *readBytes = p - atom.begin(); - return EUnescapeResult::INVALID_END; -} - + } else if (current == endChar) { + *readBytes = p - atom.begin(); + return EUnescapeResult::OK; + } + + out->Write(current); + } + + *readBytes = p - atom.begin(); + return EUnescapeResult::INVALID_END; +} + void EscapeBinaryAtom(TStringBuf atom, char quoteChar, IOutputStream* out) -{ - char prefix[] = { 'x', quoteChar }; - out->Write(prefix, 2); +{ + char prefix[] = { 'x', quoteChar }; + out->Write(prefix, 2); out->Write(HexEncode(atom.data(), atom.size())); - out->Write(quoteChar); -} - -EUnescapeResult UnescapeBinaryAtom( + out->Write(quoteChar); +} + +EUnescapeResult UnescapeBinaryAtom( TStringBuf atom, char endChar, IOutputStream* out, size_t* readBytes) -{ - const char *p = atom.begin(), - *e = atom.end(); - - while (p != e) { - char current = *p; - if (current == endChar) { - *readBytes = p - atom.begin(); - return EUnescapeResult::OK; - } - - wchar32 byte = 0; - if (!TryParseHex(p, e, 2, &byte)) { - *readBytes = p - atom.begin(); - return EUnescapeResult::INVALID_BINARY; - } - - out->Write(byte & 0xff); - } - - *readBytes = p - atom.begin(); - return EUnescapeResult::INVALID_END; -} - -} // namspace NYql +{ + const char *p = atom.begin(), + *e = atom.end(); + + while (p != e) { + char current = *p; + if (current == endChar) { + *readBytes = p - atom.begin(); + return EUnescapeResult::OK; + } + + wchar32 byte = 0; + if (!TryParseHex(p, e, 2, &byte)) { + *readBytes = p - atom.begin(); + return EUnescapeResult::INVALID_BINARY; + } + + out->Write(byte & 0xff); + } + + *readBytes = p - atom.begin(); + return EUnescapeResult::INVALID_END; +} + +} // namspace NYql diff --git a/ydb/library/yql/ast/yql_ast_escaping.h b/ydb/library/yql/ast/yql_ast_escaping.h index 04fc6af232e..744ab985161 100644 --- a/ydb/library/yql/ast/yql_ast_escaping.h +++ b/ydb/library/yql/ast/yql_ast_escaping.h @@ -1,35 +1,35 @@ -#pragma once - -#include <util/generic/fwd.h> -#include <util/system/types.h> +#pragma once + +#include <util/generic/fwd.h> +#include <util/system/types.h> #include <util/generic/strbuf.h> - - + + class IOutputStream; - -namespace NYql { - -enum class EUnescapeResult -{ - OK, - INVALID_ESCAPE_SEQUENCE, - INVALID_BINARY, - INVALID_OCTAL, - INVALID_HEXADECIMAL, - INVALID_UNICODE, - INVALID_END, -}; - + +namespace NYql { + +enum class EUnescapeResult +{ + OK, + INVALID_ESCAPE_SEQUENCE, + INVALID_BINARY, + INVALID_OCTAL, + INVALID_HEXADECIMAL, + INVALID_UNICODE, + INVALID_END, +}; + TStringBuf UnescapeResultToString(EUnescapeResult result); void EscapeArbitraryAtom(TStringBuf atom, char quoteChar, IOutputStream* out); - -EUnescapeResult UnescapeArbitraryAtom( + +EUnescapeResult UnescapeArbitraryAtom( TStringBuf atom, char endChar, IOutputStream* out, size_t* readBytes); - + void EscapeBinaryAtom(TStringBuf atom, char quoteChar, IOutputStream* out); - -EUnescapeResult UnescapeBinaryAtom( + +EUnescapeResult UnescapeBinaryAtom( TStringBuf atom, char endChar, IOutputStream* out, size_t* readBytes); - -} // namspace NYql + +} // namspace NYql diff --git a/ydb/library/yql/ast/yql_ast_ut.cpp b/ydb/library/yql/ast/yql_ast_ut.cpp index 851240f160a..23e140653d7 100644 --- a/ydb/library/yql/ast/yql_ast_ut.cpp +++ b/ydb/library/yql/ast/yql_ast_ut.cpp @@ -224,9 +224,9 @@ Y_UNIT_TEST_SUITE(TParseYqlAst) { TestBadArbitraryAtom("(\"\\U0000d800\")", "Invalid unicode value"); TestBadArbitraryAtom("(\"\\U0000dfff\")", "Invalid unicode value"); - TestBadArbitraryAtom("(x\"ag\")", "Invalid binary value"); - TestBadArbitraryAtom("(x\"abc\")", "Invalid binary value"); - TestBadArbitraryAtom("(x\"abcd)", "Invalid binary value"); + TestBadArbitraryAtom("(x\"ag\")", "Invalid binary value"); + TestBadArbitraryAtom("(x\"abc\")", "Invalid binary value"); + TestBadArbitraryAtom("(x\"abcd)", "Invalid binary value"); TestBadArbitraryAtom("(x\"abcd", "Unexpected end of atom"); } @@ -376,10 +376,10 @@ Y_UNIT_TEST_SUITE(TParseYqlAst) { TString printResult = ast.Root->ToString(); UNIT_ASSERT_STRINGS_EQUAL(s5, printResult); } - + Y_UNIT_TEST(UnicodePrettyPrint) { ParseAndAdaptPrint("(\"абв αβγ ﬡ\")", "(\"\\u0430\\u0431\\u0432 \\u03B1\\u03B2\\u03B3 \\uFB21\")"); - } + } Y_UNIT_TEST(SerializeQuotedEmptyAtom) { TMemoryPool pool(1024); diff --git a/ydb/library/yql/ast/yql_expr.cpp b/ydb/library/yql/ast/yql_expr.cpp index 0ac4750a5b3..82f6c5812ac 100644 --- a/ydb/library/yql/ast/yql_expr.cpp +++ b/ydb/library/yql/ast/yql_expr.cpp @@ -467,15 +467,15 @@ namespace { return Expr.MakeType<TResourceExprType>(TString(node.GetChild(1)->GetContent())); } else if (content == TStringBuf("Tagged")) { - if (node.GetChildrenCount() != 3 || !node.GetChild(2)->IsAtom()) { - AddError(node, "Bad tagged type annotation"); - return nullptr; - } - - auto type = CompileTypeAnnotationNode(*node.GetChild(1)); - if (!type) - return nullptr; - + if (node.GetChildrenCount() != 3 || !node.GetChild(2)->IsAtom()) { + AddError(node, "Bad tagged type annotation"); + return nullptr; + } + + auto type = CompileTypeAnnotationNode(*node.GetChild(1)); + if (!type) + return nullptr; + TString tag(node.GetChild(2)->GetContent()); auto ann = Expr.MakeType<TTaggedExprType>(type, tag); if (!ann->Validate(node.GetPosition(), Expr)) { diff --git a/ydb/library/yql/ast/yql_expr.h b/ydb/library/yql/ast/yql_expr.h index de5bbda0af9..d6eb544f9af 100644 --- a/ydb/library/yql/ast/yql_expr.h +++ b/ydb/library/yql/ast/yql_expr.h @@ -886,12 +886,12 @@ public: } }; -struct TArgumentFlags { - enum { - AutoMap = 0x01, - }; -}; - +struct TArgumentFlags { + enum { + AutoMap = 0x01, + }; +}; + class TCallableExprType : public TTypeAnnotationNode { public: static constexpr ETypeAnnotationKind KindValue = ETypeAnnotationKind::Callable; diff --git a/ydb/library/yql/ast/yql_expr_ut.cpp b/ydb/library/yql/ast/yql_expr_ut.cpp index 3d220b8ef03..06550f2507d 100644 --- a/ydb/library/yql/ast/yql_expr_ut.cpp +++ b/ydb/library/yql/ast/yql_expr_ut.cpp @@ -320,7 +320,7 @@ Y_UNIT_TEST_SUITE(TCompareExprTrees) { const TExprNode* diffTwo = rootTwo.Get(); if (diffPositions) { - UNIT_ASSERT(!CompareExprTrees(diffOne, diffTwo)); + UNIT_ASSERT(!CompareExprTrees(diffOne, diffTwo)); UNIT_ASSERT_EQUAL(ctxOne.GetPosition(diffOne->Pos()), diffPositions->first); UNIT_ASSERT_EQUAL(ctxTwo.GetPosition(diffTwo->Pos()), diffPositions->second); } else diff --git a/ydb/library/yql/ast/yql_type_string.cpp b/ydb/library/yql/ast/yql_type_string.cpp index 4178d577eba..af9af93259d 100644 --- a/ydb/library/yql/ast/yql_type_string.cpp +++ b/ydb/library/yql/ast/yql_type_string.cpp @@ -1,6 +1,6 @@ #include "yql_type_string.h" #include "yql_expr.h" -#include "yql_ast_escaping.h" +#include "yql_ast_escaping.h" #include <library/cpp/containers/stack_vector/stack_vec.h> @@ -10,48 +10,48 @@ #include <library/cpp/deprecated/enum_codegen/enum_codegen.h> -#define EXPECT_AND_SKIP_TOKEN_IMPL(token, message, result) \ +#define EXPECT_AND_SKIP_TOKEN_IMPL(token, message, result) \ do { \ - if (Y_LIKELY(Token == token)) { \ - GetNextToken(); \ - } else { \ - AddError(message); \ - return result; \ + if (Y_LIKELY(Token == token)) { \ + GetNextToken(); \ + } else { \ + AddError(message); \ + return result; \ } \ } while (0); -#define EXPECT_AND_SKIP_TOKEN(token, result) \ - EXPECT_AND_SKIP_TOKEN_IMPL(token, "Expected " #token, result) +#define EXPECT_AND_SKIP_TOKEN(token, result) \ + EXPECT_AND_SKIP_TOKEN_IMPL(token, "Expected " #token, result) + - namespace NYql { namespace { -enum EToken -{ - TOKEN_EOF = -1, - - // type keywords - TOKEN_TYPE_MIN = -2, - TOKEN_STRING = -3, - TOKEN_BOOL = -4, - TOKEN_INT32 = -6, - TOKEN_UINT32 = -7, - TOKEN_INT64 = -8, - TOKEN_UINT64 = -9, - TOKEN_FLOAT = -10, - TOKEN_DOUBLE = -11, - TOKEN_LIST = -12, - TOKEN_OPTIONAL = -13, - TOKEN_DICT = -14, - TOKEN_TUPLE = -15, - TOKEN_STRUCT = -16, - TOKEN_RESOURCE = -17, - TOKEN_VOID = -18, - TOKEN_CALLABLE = -19, - TOKEN_TAGGED = -20, - TOKEN_YSON = -21, - TOKEN_UTF8 = -22, +enum EToken +{ + TOKEN_EOF = -1, + + // type keywords + TOKEN_TYPE_MIN = -2, + TOKEN_STRING = -3, + TOKEN_BOOL = -4, + TOKEN_INT32 = -6, + TOKEN_UINT32 = -7, + TOKEN_INT64 = -8, + TOKEN_UINT64 = -9, + TOKEN_FLOAT = -10, + TOKEN_DOUBLE = -11, + TOKEN_LIST = -12, + TOKEN_OPTIONAL = -13, + TOKEN_DICT = -14, + TOKEN_TUPLE = -15, + TOKEN_STRUCT = -16, + TOKEN_RESOURCE = -17, + TOKEN_VOID = -18, + TOKEN_CALLABLE = -19, + TOKEN_TAGGED = -20, + TOKEN_YSON = -21, + TOKEN_UTF8 = -22, TOKEN_VARIANT = -23, TOKEN_UNIT = -24, TOKEN_STREAM = -25, @@ -79,22 +79,22 @@ enum EToken TOKEN_TYPE_MAX = -47, TOKEN_JSON_DOCUMENT = -48, TOKEN_DYNUMBER = -49, - - // identifiers + + // identifiers TOKEN_IDENTIFIER = -100, TOKEN_ESCAPED_IDENTIFIER = -101, - - // special + + // special TOKEN_ARROW = -200, -}; - -bool IsTypeKeyword(int token) -{ - return token < TOKEN_TYPE_MIN && token > TOKEN_TYPE_MAX; -} - -EToken TokenTypeFromStr(TStringBuf str) -{ +}; + +bool IsTypeKeyword(int token) +{ + return token < TOKEN_TYPE_MIN && token > TOKEN_TYPE_MAX; +} + +EToken TokenTypeFromStr(TStringBuf str) +{ static const THashMap<TStringBuf, EToken> map = { { TStringBuf("String"), TOKEN_STRING }, { TStringBuf("Bool"), TOKEN_BOOL }, @@ -141,17 +141,17 @@ EToken TokenTypeFromStr(TStringBuf str) { TStringBuf("EmptyDict"), TOKEN_EMPTYDICT }, { TStringBuf("JsonDocument"), TOKEN_JSON_DOCUMENT }, { TStringBuf("DyNumber"), TOKEN_DYNUMBER }, - }; - - auto it = map.find(str); - if (it != map.end()) { - return it->second; - } - - return TOKEN_IDENTIFIER; -} - - + }; + + auto it = map.find(str); + if (it != map.end()) { + return it->second; + } + + return TOKEN_IDENTIFIER; +} + + ////////////////////////////////////////////////////////////////////////////// // TTypeParser ////////////////////////////////////////////////////////////////////////////// @@ -166,40 +166,40 @@ public: , Position(position) , Index(0) , Pool(pool) - { - GetNextToken(); - } - - TAstNode* ParseTopLevelType() { - TAstNode* type = ParseType(); - if (type) { - EXPECT_AND_SKIP_TOKEN_IMPL( - TOKEN_EOF, "Expected end of string", nullptr); - } - return type; - } - -private: + { + GetNextToken(); + } + + TAstNode* ParseTopLevelType() { + TAstNode* type = ParseType(); + if (type) { + EXPECT_AND_SKIP_TOKEN_IMPL( + TOKEN_EOF, "Expected end of string", nullptr); + } + return type; + } + +private: TAstNode* ParseType() { TAstNode* type = nullptr; - - switch (Token) { - case '(': return ParseCallableType(); - case TOKEN_STRING: - case TOKEN_BOOL: + switch (Token) { + case '(': return ParseCallableType(); + + case TOKEN_STRING: + case TOKEN_BOOL: case TOKEN_INT8: case TOKEN_UINT8: case TOKEN_INT16: case TOKEN_UINT16: - case TOKEN_INT32: - case TOKEN_UINT32: - case TOKEN_INT64: - case TOKEN_UINT64: - case TOKEN_FLOAT: - case TOKEN_DOUBLE: - case TOKEN_YSON: - case TOKEN_UTF8: + case TOKEN_INT32: + case TOKEN_UINT32: + case TOKEN_INT64: + case TOKEN_UINT64: + case TOKEN_FLOAT: + case TOKEN_DOUBLE: + case TOKEN_YSON: + case TOKEN_UTF8: case TOKEN_JSON: case TOKEN_DATE: case TOKEN_DATETIME: @@ -211,43 +211,43 @@ private: case TOKEN_UUID: case TOKEN_JSON_DOCUMENT: case TOKEN_DYNUMBER: - type = MakeDataType(Identifier); - GetNextToken(); - break; + type = MakeDataType(Identifier); + GetNextToken(); + break; case TOKEN_DECIMAL: type = ParseDecimalType(); break; - case TOKEN_LIST: - type = ParseListType(); - break; - - case TOKEN_OPTIONAL: - type = ParseOptionalType(); - break; - - case TOKEN_DICT: - type = ParseDictType(); - break; - - case TOKEN_TUPLE: - type = ParseTupleType(); - break; - - case TOKEN_STRUCT: - type = ParseStructType(); - break; - - case TOKEN_RESOURCE: - type = ParseResourceType(); - break; - - case TOKEN_VOID: - type = MakeVoidType(); - GetNextToken(); - break; - + case TOKEN_LIST: + type = ParseListType(); + break; + + case TOKEN_OPTIONAL: + type = ParseOptionalType(); + break; + + case TOKEN_DICT: + type = ParseDictType(); + break; + + case TOKEN_TUPLE: + type = ParseTupleType(); + break; + + case TOKEN_STRUCT: + type = ParseStructType(); + break; + + case TOKEN_RESOURCE: + type = ParseResourceType(); + break; + + case TOKEN_VOID: + type = MakeVoidType(); + GetNextToken(); + break; + case TOKEN_NULL: type = MakeNullType(); GetNextToken(); @@ -263,14 +263,14 @@ private: GetNextToken(); break; - case TOKEN_CALLABLE: - type = ParseCallableTypeWithKeyword(); - break; - - case TOKEN_TAGGED: - type = ParseTaggedType(); - break; - + case TOKEN_CALLABLE: + type = ParseCallableTypeWithKeyword(); + break; + + case TOKEN_TAGGED: + type = ParseTaggedType(); + break; + case TOKEN_VARIANT: type = ParseVariantType(); break; @@ -301,204 +301,204 @@ private: type = ParseEnumType(); break; - default: + default: if (Identifier.empty()) { - return AddError("Expected type"); - } + return AddError("Expected type"); + } return AddError(TString("Unknown type: '") + Identifier + "\'"); - } - - if (type) { - while (Token == '?') { - type = MakeOptionalType(type); - GetNextToken(); - } - } + } + + if (type) { + while (Token == '?') { + type = MakeOptionalType(type); + GetNextToken(); + } + } return type; } - char LookaheadNonSpaceChar() { - size_t i = Index; + char LookaheadNonSpaceChar() { + size_t i = Index; while (i < Str.size() && isspace(Str[i])) { - i++; - } + i++; + } return (i < Str.size()) ? Str[i] : -1; - } - - int GetNextToken() { - return Token = ReadNextToken(); } - int ReadNextToken() { - // skip spaces - while (!AtEnd() && isspace(Get())) { - Move(); + int GetNextToken() { + return Token = ReadNextToken(); + } + + int ReadNextToken() { + // skip spaces + while (!AtEnd() && isspace(Get())) { + Move(); } - - TokenBegin = Position; - if (AtEnd()) { - return TOKEN_EOF; + + TokenBegin = Position; + if (AtEnd()) { + return TOKEN_EOF; } - // clear last readed indentifier - Identifier = {}; - - char lastChar = Get(); + // clear last readed indentifier + Identifier = {}; + + char lastChar = Get(); if (lastChar == '_' || isalnum(lastChar)) { // identifier - size_t start = Index; - while (!AtEnd()) { - lastChar = Get(); - if (lastChar == '_' || isalnum(lastChar)) Move(); - else break; - } - - Identifier = Str.SubString(start, Index - start); - return TokenTypeFromStr(Identifier); - } else if (lastChar == '\'') { // escaped identifier - Move(); // skip '\'' - if (AtEnd()) return TOKEN_EOF; - - UnescapedIdentifier.clear(); - TStringOutput sout(UnescapedIdentifier); - TStringBuf atom = Str.SubStr(Index); - size_t readBytes = 0; - EUnescapeResult unescapeResunt = - UnescapeArbitraryAtom(atom, '\'', &sout, &readBytes); - - if (unescapeResunt != EUnescapeResult::OK) return TOKEN_EOF; - - // skip already readed chars - while (readBytes-- != 0) { - Move(); - } - - if (AtEnd()) return TOKEN_EOF; - - Identifier = UnescapedIdentifier; - return TOKEN_ESCAPED_IDENTIFIER; - } else { - Move(); // skip last char - if (lastChar == '-' && !AtEnd() && Get() == '>') { - Move(); // skip '>' - return TOKEN_ARROW; - } - // otherwise, just return the last character as its ascii value - return lastChar; - } + size_t start = Index; + while (!AtEnd()) { + lastChar = Get(); + if (lastChar == '_' || isalnum(lastChar)) Move(); + else break; + } + + Identifier = Str.SubString(start, Index - start); + return TokenTypeFromStr(Identifier); + } else if (lastChar == '\'') { // escaped identifier + Move(); // skip '\'' + if (AtEnd()) return TOKEN_EOF; + + UnescapedIdentifier.clear(); + TStringOutput sout(UnescapedIdentifier); + TStringBuf atom = Str.SubStr(Index); + size_t readBytes = 0; + EUnescapeResult unescapeResunt = + UnescapeArbitraryAtom(atom, '\'', &sout, &readBytes); + + if (unescapeResunt != EUnescapeResult::OK) return TOKEN_EOF; + + // skip already readed chars + while (readBytes-- != 0) { + Move(); + } + + if (AtEnd()) return TOKEN_EOF; + + Identifier = UnescapedIdentifier; + return TOKEN_ESCAPED_IDENTIFIER; + } else { + Move(); // skip last char + if (lastChar == '-' && !AtEnd() && Get() == '>') { + Move(); // skip '>' + return TOKEN_ARROW; + } + // otherwise, just return the last character as its ascii value + return lastChar; + } } TAstNode* ParseCallableType() { - EXPECT_AND_SKIP_TOKEN('(', nullptr); + EXPECT_AND_SKIP_TOKEN('(', nullptr); TSmallVec<TAstNode*> args; args.push_back(nullptr); // CallableType Atom + settings + return type args.push_back(nullptr); args.push_back(nullptr); bool optArgsStarted = false; - bool namedArgsStarted = false; + bool namedArgsStarted = false; ui32 optArgsCount = 0; - bool lastWasTypeStatement = false; + bool lastWasTypeStatement = false; // (1) parse argements for (;;) { - if (Token == TOKEN_EOF) { + if (Token == TOKEN_EOF) { if (optArgsStarted) { return AddError("Expected ']'"); } return AddError("Expected ')'"); } - if (Token == ']' || Token == ')') { + if (Token == ']' || Token == ')') { break; } - if (lastWasTypeStatement) { - EXPECT_AND_SKIP_TOKEN(',', nullptr); - lastWasTypeStatement = false; + if (lastWasTypeStatement) { + EXPECT_AND_SKIP_TOKEN(',', nullptr); + lastWasTypeStatement = false; } - if (Token == '[') { + if (Token == '[') { optArgsStarted = true; - GetNextToken(); // eat '[' - } else if (Token == ':') { - return AddError("Expected non empty argument name"); - } else if (IsTypeKeyword(Token) || Token == '(' || // '(' - begin of callable type - Token == TOKEN_IDENTIFIER || - Token == TOKEN_ESCAPED_IDENTIFIER) - { - TStringBuf argName; - ui32 argNameFlags = TNodeFlags::Default; - - if (LookaheadNonSpaceChar() == ':') { - namedArgsStarted = true; - argName = Identifier; - - if (Token == TOKEN_ESCAPED_IDENTIFIER) { - argNameFlags = TNodeFlags::ArbitraryContent; - } - - GetNextToken(); // eat name - EXPECT_AND_SKIP_TOKEN(':', nullptr); - - if (Token == TOKEN_EOF) { - return AddError("Expected type of named argument"); - } - } else { - if (namedArgsStarted) { - return AddError("Expected named argument, because of " - "previous argument(s) was named"); - } - } - - auto argType = ParseType(); - if (!argType) { - return nullptr; + GetNextToken(); // eat '[' + } else if (Token == ':') { + return AddError("Expected non empty argument name"); + } else if (IsTypeKeyword(Token) || Token == '(' || // '(' - begin of callable type + Token == TOKEN_IDENTIFIER || + Token == TOKEN_ESCAPED_IDENTIFIER) + { + TStringBuf argName; + ui32 argNameFlags = TNodeFlags::Default; + + if (LookaheadNonSpaceChar() == ':') { + namedArgsStarted = true; + argName = Identifier; + + if (Token == TOKEN_ESCAPED_IDENTIFIER) { + argNameFlags = TNodeFlags::ArbitraryContent; + } + + GetNextToken(); // eat name + EXPECT_AND_SKIP_TOKEN(':', nullptr); + + if (Token == TOKEN_EOF) { + return AddError("Expected type of named argument"); + } + } else { + if (namedArgsStarted) { + return AddError("Expected named argument, because of " + "previous argument(s) was named"); + } } - lastWasTypeStatement = true; - if (optArgsStarted) { - if (!argType->IsList() || argType->GetChildrenCount() == 0 || - !argType->GetChild(0)->IsAtom() || + auto argType = ParseType(); + if (!argType) { + return nullptr; + } + lastWasTypeStatement = true; + + if (optArgsStarted) { + if (!argType->IsList() || argType->GetChildrenCount() == 0 || + !argType->GetChild(0)->IsAtom() || argType->GetChild(0)->GetContent() != TStringBuf("OptionalType")) - { - return AddError("Optionals are only allowed in the optional arguments"); - } - optArgsCount++; - } - - ui32 argFlags = 0; - if (Token == '{') { - if (!ParseCallableArgFlags(argFlags)) return nullptr; + { + return AddError("Optionals are only allowed in the optional arguments"); + } + optArgsCount++; + } + + ui32 argFlags = 0; + if (Token == '{') { + if (!ParseCallableArgFlags(argFlags)) return nullptr; } - - TSmallVec<TAstNode*> argSettings; - argSettings.push_back(argType); + + TSmallVec<TAstNode*> argSettings; + argSettings.push_back(argType); if (!argName.empty()) { - argSettings.push_back(MakeQuotedAtom(argName, argNameFlags)); - } - if (argFlags) { + argSettings.push_back(MakeQuotedAtom(argName, argNameFlags)); + } + if (argFlags) { if (argName.empty()) { auto atom = MakeQuotedLiteralAtom(TStringBuf(""), TNodeFlags::ArbitraryContent); - argSettings.push_back(atom); - } - argSettings.push_back(MakeQuotedAtom(ToString(argFlags))); - } - args.push_back(MakeQuote( - MakeList(argSettings.data(), argSettings.size()))); - } else { - return AddError("Expected type or argument name"); - } + argSettings.push_back(atom); + } + argSettings.push_back(MakeQuotedAtom(ToString(argFlags))); + } + args.push_back(MakeQuote( + MakeList(argSettings.data(), argSettings.size()))); + } else { + return AddError("Expected type or argument name"); + } } if (optArgsStarted) { - EXPECT_AND_SKIP_TOKEN(']', nullptr); + EXPECT_AND_SKIP_TOKEN(']', nullptr); } - EXPECT_AND_SKIP_TOKEN(')', nullptr); + EXPECT_AND_SKIP_TOKEN(')', nullptr); // (2) expect '->' after arguments - EXPECT_AND_SKIP_TOKEN_IMPL( - TOKEN_ARROW, "Expected '->' after arguments", nullptr); + EXPECT_AND_SKIP_TOKEN_IMPL( + TOKEN_ARROW, "Expected '->' after arguments", nullptr); // (3) parse return type TAstNode* returnType = ParseType(); @@ -506,96 +506,96 @@ private: return nullptr; } - // (4) parse payload - TStringBuf payload; - if (Token == '{') { - if (!ParseCallablePayload(payload)) return nullptr; - } - - return MakeCallableType(args, optArgsCount, returnType, payload); + // (4) parse payload + TStringBuf payload; + if (Token == '{') { + if (!ParseCallablePayload(payload)) return nullptr; + } + + return MakeCallableType(args, optArgsCount, returnType, payload); } - // { Flags: f1 | f2 | f3 } - bool ParseCallableArgFlags(ui32& argFlags) { - GetNextToken(); // eat '{' - + // { Flags: f1 | f2 | f3 } + bool ParseCallableArgFlags(ui32& argFlags) { + GetNextToken(); // eat '{' + if (Token != TOKEN_IDENTIFIER || Identifier != TStringBuf("Flags")) { - AddError("Expected Flags field"); - return false; - } - - GetNextToken(); // eat 'Flags' - EXPECT_AND_SKIP_TOKEN(':', false); - - for (;;) { - if (Token == TOKEN_IDENTIFIER) { + AddError("Expected Flags field"); + return false; + } + + GetNextToken(); // eat 'Flags' + EXPECT_AND_SKIP_TOKEN(':', false); + + for (;;) { + if (Token == TOKEN_IDENTIFIER) { if (Identifier == TStringBuf("AutoMap")) { - argFlags |= TArgumentFlags::AutoMap; - } else { + argFlags |= TArgumentFlags::AutoMap; + } else { AddError(TString("Unknown flag name: ") + Identifier); - return false; - } - GetNextToken(); // eat flag name - } else { - AddError("Expected flag name"); - return false; - } - - if (Token == '}') { - break; - } else if (Token == '|') { - GetNextToken(); // eat '|' - } else { - AddError("Expected '}' or '|'"); - } - } - - GetNextToken(); // eat '}' - return true; - } - - bool ParseCallablePayload(TStringBuf& payload) { - GetNextToken(); // eat '{' - + return false; + } + GetNextToken(); // eat flag name + } else { + AddError("Expected flag name"); + return false; + } + + if (Token == '}') { + break; + } else if (Token == '|') { + GetNextToken(); // eat '|' + } else { + AddError("Expected '}' or '|'"); + } + } + + GetNextToken(); // eat '}' + return true; + } + + bool ParseCallablePayload(TStringBuf& payload) { + GetNextToken(); // eat '{' + if (Token != TOKEN_IDENTIFIER && Identifier != TStringBuf("Payload")) { - AddError("Expected Payload field"); - return false; - } - - GetNextToken(); // eat 'Payload' - EXPECT_AND_SKIP_TOKEN(':', false); - - if (Token == TOKEN_IDENTIFIER || Token == TOKEN_ESCAPED_IDENTIFIER) { - payload = Identifier; - GetNextToken(); // eat payload data - } else { - AddError("Expected payload data"); - return false; - } - - EXPECT_AND_SKIP_TOKEN('}', false); - return true; - } - + AddError("Expected Payload field"); + return false; + } + + GetNextToken(); // eat 'Payload' + EXPECT_AND_SKIP_TOKEN(':', false); + + if (Token == TOKEN_IDENTIFIER || Token == TOKEN_ESCAPED_IDENTIFIER) { + payload = Identifier; + GetNextToken(); // eat payload data + } else { + AddError("Expected payload data"); + return false; + } + + EXPECT_AND_SKIP_TOKEN('}', false); + return true; + } + TAstNode* ParseCallableTypeWithKeyword() { - GetNextToken(); // eat keyword - EXPECT_AND_SKIP_TOKEN('<', nullptr); + GetNextToken(); // eat keyword + EXPECT_AND_SKIP_TOKEN('<', nullptr); auto type = ParseCallableType(); if (!type) return nullptr; - EXPECT_AND_SKIP_TOKEN('>', nullptr); + EXPECT_AND_SKIP_TOKEN('>', nullptr); return type; } TAstNode* ParseListType() { - GetNextToken(); // eat keyword - EXPECT_AND_SKIP_TOKEN('<', nullptr); + GetNextToken(); // eat keyword + EXPECT_AND_SKIP_TOKEN('<', nullptr); auto itemType = ParseType(); if (!itemType) return nullptr; - EXPECT_AND_SKIP_TOKEN('>', nullptr); + EXPECT_AND_SKIP_TOKEN('>', nullptr); return MakeListType(itemType); } @@ -639,29 +639,29 @@ private: } TAstNode* ParseOptionalType() { - GetNextToken(); // eat keyword - EXPECT_AND_SKIP_TOKEN('<', nullptr); + GetNextToken(); // eat keyword + EXPECT_AND_SKIP_TOKEN('<', nullptr); auto itemType = ParseType(); if (!itemType) return nullptr; - EXPECT_AND_SKIP_TOKEN('>', nullptr); + EXPECT_AND_SKIP_TOKEN('>', nullptr); return MakeOptionalType(itemType); } TAstNode* ParseDictType() { - GetNextToken(); // eat keyword - EXPECT_AND_SKIP_TOKEN('<', nullptr); + GetNextToken(); // eat keyword + EXPECT_AND_SKIP_TOKEN('<', nullptr); auto keyType = ParseType(); if (!keyType) return nullptr; - EXPECT_AND_SKIP_TOKEN(',', nullptr); + EXPECT_AND_SKIP_TOKEN(',', nullptr); auto valueType = ParseType(); if (!valueType) return nullptr; - EXPECT_AND_SKIP_TOKEN('>', nullptr); + EXPECT_AND_SKIP_TOKEN('>', nullptr); return MakeDictType(keyType, valueType); } @@ -676,7 +676,7 @@ private: return MakeDictType(keyType, MakeVoidType()); } - TAstNode* ParseTupleTypeImpl() { + TAstNode* ParseTupleTypeImpl() { TSmallVec<TAstNode*> items; items.push_back(nullptr); // reserve for TupleType @@ -700,17 +700,17 @@ private: return MakeTupleType(items); } - TAstNode* ParseTupleType() { - GetNextToken(); // eat keyword - EXPECT_AND_SKIP_TOKEN('<', nullptr); - TAstNode* tupleType = ParseTupleTypeImpl(); - if (tupleType) { - EXPECT_AND_SKIP_TOKEN('>', nullptr); - } - return tupleType; - } + TAstNode* ParseTupleType() { + GetNextToken(); // eat keyword + EXPECT_AND_SKIP_TOKEN('<', nullptr); + TAstNode* tupleType = ParseTupleTypeImpl(); + if (tupleType) { + EXPECT_AND_SKIP_TOKEN('>', nullptr); + } + return tupleType; + } - TAstNode* ParseStructTypeImpl() { + TAstNode* ParseStructTypeImpl() { TMap<TString, TAstNode*> members; if (Token != '>') { for (;;) { @@ -728,7 +728,7 @@ private: } else if (members.contains(name)) { return AddError("Member name duplication"); } - + GetNextToken(); // eat member name EXPECT_AND_SKIP_TOKEN(':', nullptr); @@ -747,36 +747,36 @@ private: } } - return MakeStructType(members); - } - - TAstNode* ParseStructType() { - GetNextToken(); // eat keyword - EXPECT_AND_SKIP_TOKEN('<', nullptr); - TAstNode* structType = ParseStructTypeImpl(); - if (structType) { - EXPECT_AND_SKIP_TOKEN('>', nullptr); - } - return structType; - } - - TAstNode* ParseVariantType() { - GetNextToken(); // eat keyword - EXPECT_AND_SKIP_TOKEN('<', nullptr); - - TAstNode* underlyingType = nullptr; - if (Token == TOKEN_IDENTIFIER || Token == TOKEN_ESCAPED_IDENTIFIER) { - underlyingType = ParseStructTypeImpl(); - } else if (IsTypeKeyword(Token) || Token == '(') { - underlyingType = ParseTupleTypeImpl(); - } else { - return AddError("Expected type"); - } - - if (!underlyingType) return nullptr; - - EXPECT_AND_SKIP_TOKEN('>', nullptr); - return MakeVariantType(underlyingType); + return MakeStructType(members); + } + + TAstNode* ParseStructType() { + GetNextToken(); // eat keyword + EXPECT_AND_SKIP_TOKEN('<', nullptr); + TAstNode* structType = ParseStructTypeImpl(); + if (structType) { + EXPECT_AND_SKIP_TOKEN('>', nullptr); + } + return structType; + } + + TAstNode* ParseVariantType() { + GetNextToken(); // eat keyword + EXPECT_AND_SKIP_TOKEN('<', nullptr); + + TAstNode* underlyingType = nullptr; + if (Token == TOKEN_IDENTIFIER || Token == TOKEN_ESCAPED_IDENTIFIER) { + underlyingType = ParseStructTypeImpl(); + } else if (IsTypeKeyword(Token) || Token == '(') { + underlyingType = ParseTupleTypeImpl(); + } else { + return AddError("Expected type"); + } + + if (!underlyingType) return nullptr; + + EXPECT_AND_SKIP_TOKEN('>', nullptr); + return MakeVariantType(underlyingType); } TAstNode* ParseEnumType() { @@ -819,7 +819,7 @@ private: TAstNode* MakeCallableType( TSmallVec<TAstNode*>& args, size_t optionalArgsCount, - TAstNode* returnType, TStringBuf payload) + TAstNode* returnType, TStringBuf payload) { args[0] = MakeLiteralAtom(TStringBuf("CallableType")); TSmallVec<TAstNode*> mainSettings; @@ -904,47 +904,47 @@ private: } TAstNode* ParseResourceType() { - GetNextToken(); // eat keyword - EXPECT_AND_SKIP_TOKEN('<', nullptr); + GetNextToken(); // eat keyword + EXPECT_AND_SKIP_TOKEN('<', nullptr); - if (Token != TOKEN_IDENTIFIER && Token != TOKEN_ESCAPED_IDENTIFIER) { - return AddError("Expected resource tag"); - } + if (Token != TOKEN_IDENTIFIER && Token != TOKEN_ESCAPED_IDENTIFIER) { + return AddError("Expected resource tag"); + } - TStringBuf tag = Identifier; + TStringBuf tag = Identifier; if (tag.empty()) { - return AddError("Expected non empty resource tag"); - } - - GetNextToken(); // eat tag - EXPECT_AND_SKIP_TOKEN('>', nullptr); - return MakeResourceType(tag); + return AddError("Expected non empty resource tag"); + } + + GetNextToken(); // eat tag + EXPECT_AND_SKIP_TOKEN('>', nullptr); + return MakeResourceType(tag); } TAstNode* ParseTaggedType() { - GetNextToken(); // eat keyword - EXPECT_AND_SKIP_TOKEN('<', nullptr); + GetNextToken(); // eat keyword + EXPECT_AND_SKIP_TOKEN('<', nullptr); auto baseType = ParseType(); if (!baseType) return nullptr; - EXPECT_AND_SKIP_TOKEN(',', nullptr); - - if (Token != TOKEN_IDENTIFIER && Token != TOKEN_ESCAPED_IDENTIFIER) { - return AddError("Expected tag of type"); - } + EXPECT_AND_SKIP_TOKEN(',', nullptr); + + if (Token != TOKEN_IDENTIFIER && Token != TOKEN_ESCAPED_IDENTIFIER) { + return AddError("Expected tag of type"); + } - TStringBuf tag = Identifier; + TStringBuf tag = Identifier; if (tag.empty()) { - return AddError("Expected non empty tag of type"); + return AddError("Expected non empty tag of type"); } - GetNextToken(); // eat tag - EXPECT_AND_SKIP_TOKEN('>', nullptr); - return MakeTaggedType(baseType, tag); + GetNextToken(); // eat tag + EXPECT_AND_SKIP_TOKEN('>', nullptr); + return MakeTaggedType(baseType, tag); } - TAstNode* MakeResourceType(TStringBuf tag) { + TAstNode* MakeResourceType(TStringBuf tag) { TAstNode* items[] = { MakeLiteralAtom(TStringBuf("ResourceType")), MakeQuotedAtom(tag), @@ -994,7 +994,7 @@ private: return MakeList(items, Y_ARRAY_SIZE(items)); } - TAstNode* MakeTaggedType(TAstNode* baseType, TStringBuf tag) { + TAstNode* MakeTaggedType(TAstNode* baseType, TStringBuf tag) { TAstNode* items[] = { MakeLiteralAtom(TStringBuf("TaggedType")), baseType, @@ -1004,10 +1004,10 @@ private: return MakeList(items, Y_ARRAY_SIZE(items)); } - TAstNode* MakeDataType(TStringBuf type) { + TAstNode* MakeDataType(TStringBuf type) { TAstNode* items[] = { MakeLiteralAtom(TStringBuf("DataType")), - MakeQuotedAtom(type), + MakeQuotedAtom(type), }; return MakeList(items, Y_ARRAY_SIZE(items)); } @@ -1030,11 +1030,11 @@ private: return MakeList(items, Y_ARRAY_SIZE(items)); } - TAstNode* MakeAtom(TStringBuf content, ui32 flags = TNodeFlags::Default) { + TAstNode* MakeAtom(TStringBuf content, ui32 flags = TNodeFlags::Default) { return TAstNode::NewAtom(Position, content, Pool, flags); } - TAstNode* MakeLiteralAtom(TStringBuf content, ui32 flags = TNodeFlags::Default) { + TAstNode* MakeLiteralAtom(TStringBuf content, ui32 flags = TNodeFlags::Default) { return TAstNode::NewLiteralAtom(Position, content, Pool, flags); } @@ -1046,11 +1046,11 @@ private: return MakeList(items, Y_ARRAY_SIZE(items)); } - TAstNode* MakeQuotedAtom(TStringBuf content, ui32 flags = TNodeFlags::Default) { + TAstNode* MakeQuotedAtom(TStringBuf content, ui32 flags = TNodeFlags::Default) { return MakeQuote(MakeAtom(content, flags)); } - TAstNode* MakeQuotedLiteralAtom(TStringBuf content, ui32 flags = TNodeFlags::Default) { + TAstNode* MakeQuotedLiteralAtom(TStringBuf content, ui32 flags = TNodeFlags::Default) { return MakeQuote(MakeLiteralAtom(content, flags)); } @@ -1067,13 +1067,13 @@ private: } void Move() { - if (AtEnd()) return; - + if (AtEnd()) return; + ++Index; ++Position.Column; - - if (!AtEnd() && Str[Index] == '\n') { - Position.Row++; + + if (!AtEnd() && Str[Index] == '\n') { + Position.Row++; Position.Column = 1; } } @@ -1086,11 +1086,11 @@ private: private: TStringBuf Str; TIssues& Issues; - TPosition TokenBegin, Position; + TPosition TokenBegin, Position; size_t Index; - int Token; + int Token; TString UnescapedIdentifier; - TStringBuf Identifier; + TStringBuf Identifier; TMemoryPool& Pool; }; @@ -1148,8 +1148,8 @@ private: } void Visit(const TItemExprType& type) final { - EscapeArbitraryAtom(type.GetName(), '\'', &Out_); - Out_ << ':'; + EscapeArbitraryAtom(type.GetName(), '\'', &Out_); + Out_ << ':'; type.GetItemType()->Accept(*this); } @@ -1209,19 +1209,19 @@ private: if (i == argsCount - optArgsCount) { Out_ << '['; } - const TCallableExprType::TArgumentInfo& argInfo = args[i]; + const TCallableExprType::TArgumentInfo& argInfo = args[i]; if (!argInfo.Name.empty()) { - EscapeArbitraryAtom(argInfo.Name, '\'', &Out_); - Out_ << ':'; - } - argInfo.Type->Accept(*this); - if (argInfo.Flags) { + EscapeArbitraryAtom(argInfo.Name, '\'', &Out_); + Out_ << ':'; + } + argInfo.Type->Accept(*this); + if (argInfo.Flags) { Out_ << TStringBuf("{Flags:"); - if (argInfo.Flags & TArgumentFlags::AutoMap) { + if (argInfo.Flags & TArgumentFlags::AutoMap) { Out_ << TStringBuf("AutoMap"); - } - Out_ << '}'; - } + } + Out_ << '}'; + } } if (optArgsCount > 0) { @@ -1232,14 +1232,14 @@ private: type.GetReturnType()->Accept(*this); if (!type.GetPayload().empty()) { Out_ << TStringBuf("{Payload:") << type.GetPayload() << '}'; - } + } Out_ << '>'; } void Visit(const TResourceExprType& type) final { Out_ << TStringBuf("Resource<"); - EscapeArbitraryAtom(type.GetTag(), '\'', &Out_); - Out_ << '>'; + EscapeArbitraryAtom(type.GetTag(), '\'', &Out_); + Out_ << '>'; } void Visit(const TTypeExprType& type) final { @@ -1290,9 +1290,9 @@ private: void Visit(const TTaggedExprType& type) final { Out_ << TStringBuf("Tagged<"); type.GetBaseType()->Accept(*this); - Out_ << ','; - EscapeArbitraryAtom(type.GetTag(), '\'', &Out_); - Out_ << '>'; + Out_ << ','; + EscapeArbitraryAtom(type.GetTag(), '\'', &Out_); + Out_ << '>'; } void Visit(const TErrorExprType& type) final { @@ -1356,7 +1356,7 @@ TAstNode* ParseType(TStringBuf str, TMemoryPool& pool, TIssues& issues, TPosition position /* = TPosition(1, 1) */) { TTypeParser parser(str, issues, position, pool); - return parser.ParseTopLevelType(); + return parser.ParseTopLevelType(); } TString FormatType(const TTypeAnnotationNode* typeNode) diff --git a/ydb/library/yql/ast/yql_type_string_ut.cpp b/ydb/library/yql/ast/yql_type_string_ut.cpp index 05c69093dc9..2f22c42258a 100644 --- a/ydb/library/yql/ast/yql_type_string_ut.cpp +++ b/ydb/library/yql/ast/yql_type_string_ut.cpp @@ -35,20 +35,20 @@ Y_UNIT_TEST_SUITE(TTypeString) } Y_UNIT_TEST(ParseDataTypes) { - TestOk("String", "(DataType 'String)"); - TestOk("Bool", "(DataType 'Bool)"); + TestOk("String", "(DataType 'String)"); + TestOk("Bool", "(DataType 'Bool)"); TestOk("Uint8", "(DataType 'Uint8)"); TestOk("Int8", "(DataType 'Int8)"); TestOk("Uint16", "(DataType 'Uint16)"); TestOk("Int16", "(DataType 'Int16)"); - TestOk("Int32", "(DataType 'Int32)"); - TestOk("Uint32", "(DataType 'Uint32)"); - TestOk("Int64", "(DataType 'Int64)"); - TestOk("Uint64", "(DataType 'Uint64)"); - TestOk("Float", "(DataType 'Float)"); - TestOk("Double", "(DataType 'Double)"); - TestOk("Yson", "(DataType 'Yson)"); - TestOk("Utf8", "(DataType 'Utf8)"); + TestOk("Int32", "(DataType 'Int32)"); + TestOk("Uint32", "(DataType 'Uint32)"); + TestOk("Int64", "(DataType 'Int64)"); + TestOk("Uint64", "(DataType 'Uint64)"); + TestOk("Float", "(DataType 'Float)"); + TestOk("Double", "(DataType 'Double)"); + TestOk("Yson", "(DataType 'Yson)"); + TestOk("Utf8", "(DataType 'Utf8)"); TestOk("Json", "(DataType 'Json)"); TestOk("Date", "(DataType 'Date)"); TestOk("Datetime", "(DataType 'Datetime)"); @@ -59,29 +59,29 @@ Y_UNIT_TEST_SUITE(TTypeString) TestOk("TzTimestamp", "(DataType 'TzTimestamp)"); TestOk("Uuid", "(DataType 'Uuid)"); TestOk("Decimal(10,2)", "(DataType 'Decimal '10 '2)"); - } - + } + Y_UNIT_TEST(Multiline) { - TestOk(R"(Struct - < - name : String - , - age : Uint32 - >)", "(StructType " - "'('\"age\" (DataType 'Uint32)) " - "'('\"name\" (DataType 'String)))"); - } - + TestOk(R"(Struct + < + name : String + , + age : Uint32 + >)", "(StructType " + "'('\"age\" (DataType 'Uint32)) " + "'('\"name\" (DataType 'String)))"); + } + Y_UNIT_TEST(ParseNoArgsWithStringResult) { TestOk("()->String", "(CallableType '() '((DataType 'String)))"); - TestOk("()->Utf8", "(CallableType '() '((DataType 'Utf8)))"); + TestOk("()->Utf8", "(CallableType '() '((DataType 'Utf8)))"); } Y_UNIT_TEST(ParseNoArgsWithOptionalStringResult) { TestOk("()->String?", "(CallableType '() '((OptionalType (DataType 'String))))"); - TestOk("()->Yson?", - "(CallableType '() '((OptionalType (DataType 'Yson))))"); + TestOk("()->Yson?", + "(CallableType '() '((OptionalType (DataType 'Yson))))"); } Y_UNIT_TEST(ParseOneArgWithDoubleResult) { @@ -89,14 +89,14 @@ Y_UNIT_TEST_SUITE(TTypeString) "(CallableType '() '((DataType 'Double)) " "'((DataType 'Int32))" ")"); - TestOk("(Yson)->Double", - "(CallableType '() '((DataType 'Double)) " - "'((DataType 'Yson))" - ")"); - TestOk("(Utf8)->Double", - "(CallableType '() '((DataType 'Double)) " - "'((DataType 'Utf8))" - ")"); + TestOk("(Yson)->Double", + "(CallableType '() '((DataType 'Double)) " + "'((DataType 'Yson))" + ")"); + TestOk("(Utf8)->Double", + "(CallableType '() '((DataType 'Double)) " + "'((DataType 'Utf8))" + ")"); } Y_UNIT_TEST(ParseTwoArgsWithOptionalByteResult) { @@ -158,60 +158,60 @@ Y_UNIT_TEST_SUITE(TTypeString) Y_UNIT_TEST(ParseCallableWithNamedArgs) { TestOk("(a:Uint8)->Double", - "(CallableType '() '((DataType 'Double)) " + "(CallableType '() '((DataType 'Double)) " "'((DataType 'Uint8) 'a)" - ")"); + ")"); TestOk("(List:Uint8)->Double", - "(CallableType '() '((DataType 'Double)) " + "(CallableType '() '((DataType 'Double)) " "'((DataType 'Uint8) 'List)" - ")"); + ")"); TestOk("('Dict':Uint8)->Double", - "(CallableType '() '((DataType 'Double)) " + "(CallableType '() '((DataType 'Double)) " "'((DataType 'Uint8) '\"Dict\")" - ")"); + ")"); TestOk("(a:Uint8,[b:Int32?])->Double", - "(CallableType '('1) '((DataType 'Double)) " + "(CallableType '('1) '((DataType 'Double)) " "'((DataType 'Uint8) 'a) " - "'((OptionalType (DataType 'Int32)) 'b)" - ")"); + "'((OptionalType (DataType 'Int32)) 'b)" + ")"); TestOk("(Uint8,[b:Int32?])->Double", - "(CallableType '('1) '((DataType 'Double)) " + "(CallableType '('1) '((DataType 'Double)) " "'((DataType 'Uint8)) " - "'((OptionalType (DataType 'Int32)) 'b)" - ")"); - } - + "'((OptionalType (DataType 'Int32)) 'b)" + ")"); + } + Y_UNIT_TEST(ParseCallableWithArgFlags) { - TestOk("(Int32{Flags:AutoMap})->Double", - "(CallableType '() '((DataType 'Double)) " - "'((DataType 'Int32) '\"\" '1)" - ")"); - TestOk("(Int32?{Flags:AutoMap})->Double", - "(CallableType '() '((DataType 'Double)) " - "'((OptionalType (DataType 'Int32)) '\"\" '1)" - ")"); - TestOk("(x:Int32{Flags:AutoMap})->Double", - "(CallableType '() '((DataType 'Double)) " - "'((DataType 'Int32) 'x '1)" - ")"); - TestOk("(x:Int32{Flags:AutoMap}, [y:Uint32?{Flags: AutoMap}])->Double", - "(CallableType '('1) '((DataType 'Double)) " - "'((DataType 'Int32) 'x '1) " - "'((OptionalType (DataType 'Uint32)) 'y '1)" - ")"); - TestOk("(x:Int32{Flags: AutoMap | AutoMap})->Double", - "(CallableType '() '((DataType 'Double)) " - "'((DataType 'Int32) 'x '1)" - ")"); - } - + TestOk("(Int32{Flags:AutoMap})->Double", + "(CallableType '() '((DataType 'Double)) " + "'((DataType 'Int32) '\"\" '1)" + ")"); + TestOk("(Int32?{Flags:AutoMap})->Double", + "(CallableType '() '((DataType 'Double)) " + "'((OptionalType (DataType 'Int32)) '\"\" '1)" + ")"); + TestOk("(x:Int32{Flags:AutoMap})->Double", + "(CallableType '() '((DataType 'Double)) " + "'((DataType 'Int32) 'x '1)" + ")"); + TestOk("(x:Int32{Flags:AutoMap}, [y:Uint32?{Flags: AutoMap}])->Double", + "(CallableType '('1) '((DataType 'Double)) " + "'((DataType 'Int32) 'x '1) " + "'((OptionalType (DataType 'Uint32)) 'y '1)" + ")"); + TestOk("(x:Int32{Flags: AutoMap | AutoMap})->Double", + "(CallableType '() '((DataType 'Double)) " + "'((DataType 'Int32) 'x '1)" + ")"); + } + Y_UNIT_TEST(ParseCallableWithPayload) { - TestOk("(Int32)->Double{Payload:MyFunction}", - "(CallableType '('0 '\"MyFunction\") '((DataType 'Double)) " - "'((DataType 'Int32))" - ")"); - } - + TestOk("(Int32)->Double{Payload:MyFunction}", + "(CallableType '('0 '\"MyFunction\") '((DataType 'Double)) " + "'((DataType 'Int32))" + ")"); + } + Y_UNIT_TEST(ParseOptional) { TestOk("Uint32?", "(OptionalType (DataType 'Uint32))"); TestOk("Optional<Uint32>", "(OptionalType (DataType 'Uint32))"); @@ -223,37 +223,37 @@ Y_UNIT_TEST_SUITE(TTypeString) Y_UNIT_TEST(ParseCallableComplete) { TestFail("(Uint32)->", 11, "Expected type"); - TestFail("(,)->", 2, "Expected type or argument name"); - TestFail("(Int32 Int32)->Int32", 8, "Expected ','"); + TestFail("(,)->", 2, "Expected type or argument name"); + TestFail("(Int32 Int32)->Int32", 8, "Expected ','"); TestFail("([],)->Uint32", 4, "Expected ')'"); TestFail("([)->Uint32", 3, "Expected ']'"); TestFail("(])->Uint32", 2, "Expected ')'"); - TestFail("([,)->Uint32", 3, "Expected type or argument name"); - TestFail("([,])->Uint32", 3, "Expected type or argument name"); - TestFail("(->Uint32", 2, "Expected type or argument name"); + TestFail("([,)->Uint32", 3, "Expected type or argument name"); + TestFail("([,])->Uint32", 3, "Expected type or argument name"); + TestFail("(->Uint32", 2, "Expected type or argument name"); TestFail("([Uint32],Uint8)->Uint32", 9, "Optionals are only allowed in the optional arguments"); TestFail("([Uint32?],Uint8)->Uint32", 11, "Expected ')'"); - TestFail("Callable<()>", 12, "Expected '->' after arguments"); + TestFail("Callable<()>", 12, "Expected '->' after arguments"); TestFail("Callable<()->", 14, "Expected type"); TestFail("Callable<()->Uint32", 20, "Expected '>'"); - TestFail("(:Uint32)->Uint32", 2, "Expected non empty argument name"); - TestFail("(a:)->Uint32", 4, "Expected type"); - TestFail("(:)->Uint32", 2, "Expected non empty argument name"); - TestFail("(a:Uint32,Uint32)->Uint32", 11, "Expected named argument, because of previous argument(s) was named"); - TestFail("(Uint32{)->Uint32", 9, "Expected Flags field"); - TestFail("(Uint32})->Uint32", 8, "Expected ','"); - TestFail("(Uint32{})->Uint32", 9, "Expected Flags field"); - TestFail("(Uint32{Flags})->Uint32", 14, "Expected ':'"); - TestFail("(Uint32{Flags:})->Uint32", 15, "Expected flag name"); - TestFail("(Uint32{Flags:Map})->Uint32", 15, "Unknown flag name: Map"); - TestFail("(Uint32{Flags:|})->Uint32", 15, "Expected flag name"); - TestFail("(Uint32{Flags:AutoMap|})->Uint32", 23, "Expected flag name"); - TestFail("(Uint32{NonFlags:AutoMap})->Uint32", 9, "Expected Flags field"); - TestFail("(Uint32)->Uint32{", 18, "Expected Payload field"); - TestFail("(Uint32)->Uint32{}", 18, "Expected Payload field"); - TestFail("(Uint32)->Uint32}", 17, "Expected end of string"); - TestFail("(Uint32)->Uint32{Payload}", 25, "Expected ':'"); - TestFail("(Uint32)->Uint32{Payload:}", 26, "Expected payload data"); + TestFail("(:Uint32)->Uint32", 2, "Expected non empty argument name"); + TestFail("(a:)->Uint32", 4, "Expected type"); + TestFail("(:)->Uint32", 2, "Expected non empty argument name"); + TestFail("(a:Uint32,Uint32)->Uint32", 11, "Expected named argument, because of previous argument(s) was named"); + TestFail("(Uint32{)->Uint32", 9, "Expected Flags field"); + TestFail("(Uint32})->Uint32", 8, "Expected ','"); + TestFail("(Uint32{})->Uint32", 9, "Expected Flags field"); + TestFail("(Uint32{Flags})->Uint32", 14, "Expected ':'"); + TestFail("(Uint32{Flags:})->Uint32", 15, "Expected flag name"); + TestFail("(Uint32{Flags:Map})->Uint32", 15, "Unknown flag name: Map"); + TestFail("(Uint32{Flags:|})->Uint32", 15, "Expected flag name"); + TestFail("(Uint32{Flags:AutoMap|})->Uint32", 23, "Expected flag name"); + TestFail("(Uint32{NonFlags:AutoMap})->Uint32", 9, "Expected Flags field"); + TestFail("(Uint32)->Uint32{", 18, "Expected Payload field"); + TestFail("(Uint32)->Uint32{}", 18, "Expected Payload field"); + TestFail("(Uint32)->Uint32}", 17, "Expected end of string"); + TestFail("(Uint32)->Uint32{Payload}", 25, "Expected ':'"); + TestFail("(Uint32)->Uint32{Payload:}", 26, "Expected payload data"); } Y_UNIT_TEST(ParseCallableWithKeyword) { @@ -285,25 +285,25 @@ Y_UNIT_TEST_SUITE(TTypeString) } Y_UNIT_TEST(ParseVariantType) { - TestOk("Variant<String>", - "(VariantType (TupleType " - "(DataType 'String)" - "))"); + TestOk("Variant<String>", + "(VariantType (TupleType " + "(DataType 'String)" + "))"); TestOk("Variant<String, Uint8>", - "(VariantType (TupleType " - "(DataType 'String) " + "(VariantType (TupleType " + "(DataType 'String) " "(DataType 'Uint8)" - "))"); - TestOk("Variant<Name: String, Age: Int32>", - "(VariantType (StructType " - "'('\"Age\" (DataType 'Int32)) " - "'('\"Name\" (DataType 'String))" - "))"); - TestOk("Variant<'Some Name': String, 'Age': Int32>", - "(VariantType (StructType " - "'('\"Age\" (DataType 'Int32)) " - "'('\"Some Name\" (DataType 'String))" - "))"); + "))"); + TestOk("Variant<Name: String, Age: Int32>", + "(VariantType (StructType " + "'('\"Age\" (DataType 'Int32)) " + "'('\"Name\" (DataType 'String))" + "))"); + TestOk("Variant<'Some Name': String, 'Age': Int32>", + "(VariantType (StructType " + "'('\"Age\" (DataType 'Int32)) " + "'('\"Some Name\" (DataType 'String))" + "))"); } Y_UNIT_TEST(ParseEnumType) { @@ -380,13 +380,13 @@ Y_UNIT_TEST_SUITE(TTypeString) } Y_UNIT_TEST(ParseVariantComplete) { - TestFail("Variant<>", 9, "Expected type"); - TestFail("Variant<Uint32,>", 16, "Expected type"); - TestFail("Variant<Uint32", 15, "Expected '>' or ','"); - - TestFail("Variant<name:>", 14, "Expected type"); - TestFail("Variant<name:String,>", 21, "Expected struct member name"); - TestFail("Variant<name:String", 20, "Expected '>' or ','"); + TestFail("Variant<>", 9, "Expected type"); + TestFail("Variant<Uint32,>", 16, "Expected type"); + TestFail("Variant<Uint32", 15, "Expected '>' or ','"); + + TestFail("Variant<name:>", 14, "Expected type"); + TestFail("Variant<name:String,>", 21, "Expected struct member name"); + TestFail("Variant<name:String", 20, "Expected '>' or ','"); } Y_UNIT_TEST(ParseDictOfDictTypes) { @@ -444,7 +444,7 @@ Y_UNIT_TEST_SUITE(TTypeString) Y_UNIT_TEST(ParseStructComplete) { TestFail("(Struct<name>)->Uint32", 13, "Expected ':'"); TestFail("(Struct<name:>)->Uint32", 14, "Expected type"); - TestFail("(Struct<name:String,>)->Uint32", 21, "Expected struct member name"); + TestFail("(Struct<name:String,>)->Uint32", 21, "Expected struct member name"); TestFail("(Struct<name:String)->Uint32", 20, "Expected '>' or ','"); } @@ -457,14 +457,14 @@ Y_UNIT_TEST_SUITE(TTypeString) } Y_UNIT_TEST(ParseVoid) { - TestOk("Void", "(VoidType)"); - TestOk("Void?", "(OptionalType (VoidType))"); - TestOk("(Void?)->Void", - "(CallableType '() '((VoidType)) " - "'((OptionalType (VoidType)))" - ")"); - } - + TestOk("Void", "(VoidType)"); + TestOk("Void?", "(OptionalType (VoidType))"); + TestOk("(Void?)->Void", + "(CallableType '() '((VoidType)) " + "'((OptionalType (VoidType)))" + ")"); + } + Y_UNIT_TEST(ParseNull) { TestOk("Null", "(NullType)"); TestOk("Null?", "(OptionalType (NullType))"); @@ -493,10 +493,10 @@ Y_UNIT_TEST_SUITE(TTypeString) } Y_UNIT_TEST(UnknownType) { - TestFail("(Yson2)->String", 2, "Unknown type: 'Yson2'"); - TestFail("()->", 5, "Expected type"); - } - + TestFail("(Yson2)->String", 2, "Unknown type: 'Yson2'"); + TestFail("()->", 5, "Expected type"); + } + Y_UNIT_TEST(ParseTagged) { TestOk("Tagged<Uint32, IdTag>", "(TaggedType (DataType 'Uint32) 'IdTag)"); } @@ -632,38 +632,38 @@ Y_UNIT_TEST_SUITE(TTypeString) } Y_UNIT_TEST(FormatCallableWithNamedArgs) { - TestFormat("((Callable () " - " ((Data String))" - " ((Data Uint32) x)" + TestFormat("((Callable () " + " ((Data String))" + " ((Data Uint32) x)" " ((Data Uint8) y)" - "))", + "))", "Callable<('x':Uint32,'y':Uint8)->String>"); - TestFormat("((Callable () " - " ((Data String))" + TestFormat("((Callable () " + " ((Data String))" " ((Optional (Data Uint8)) a 1)" - "))", + "))", "Callable<('a':Uint8?{Flags:AutoMap})->String>"); - } - + } + Y_UNIT_TEST(FormatCallableWithPayload) { - TestFormat("((Callable (0 MyFunction) " - " ((Data String))" + TestFormat("((Callable (0 MyFunction) " + " ((Data String))" " ((Optional (Data Uint8)))" - "))", + "))", "Callable<(Uint8?)->String{Payload:MyFunction}>"); - } - + } + Y_UNIT_TEST(FormatResource) { TestFormat("((Resource aaa))", "Resource<'aaa'>"); - TestFormat("((Resource \"a b\"))", "Resource<'a b'>"); - TestFormat("((Resource \"a\\t\\n\\x01b\"))", "Resource<'a\\t\\n\\x01b'>"); + TestFormat("((Resource \"a b\"))", "Resource<'a b'>"); + TestFormat("((Resource \"a\\t\\n\\x01b\"))", "Resource<'a\\t\\n\\x01b'>"); TestFormat("((Optional (Resource aaa)))", "Resource<'aaa'>?"); } - + Y_UNIT_TEST(FormatTagged) { TestFormat("((Tagged (Data String) aaa))", "Tagged<String,'aaa'>"); TestFormat("((Tagged (Data String) \"a b\"))", "Tagged<String,'a b'>"); TestFormat("((Tagged (Data String) \"a\\t\\n\\x01b\"))", "Tagged<String,'a\\t\\n\\x01b'>"); - } - + } + } diff --git a/ydb/library/yql/core/facade/ya.make b/ydb/library/yql/core/facade/ya.make index 0b70423970a..c3b51889809 100644 --- a/ydb/library/yql/core/facade/ya.make +++ b/ydb/library/yql/core/facade/ya.make @@ -1,12 +1,12 @@ -LIBRARY() - +LIBRARY() + OWNER(g:yql) - -SRCS( - yql_facade.cpp -) - -PEERDIR( + +SRCS( + yql_facade.cpp +) + +PEERDIR( library/cpp/deprecated/split library/cpp/random_provider library/cpp/string_utils/base64 @@ -27,8 +27,8 @@ PEERDIR( ydb/library/yql/providers/common/udf_resolve ydb/library/yql/providers/config ydb/library/yql/providers/result/provider -) - +) + YQL_LAST_ABI_VERSION() -END() +END() diff --git a/ydb/library/yql/core/facade/yql_facade.cpp b/ydb/library/yql/core/facade/yql_facade.cpp index 77f53f13e64..b74abc1c34c 100644 --- a/ydb/library/yql/core/facade/yql_facade.cpp +++ b/ydb/library/yql/core/facade/yql_facade.cpp @@ -1,5 +1,5 @@ -#include "yql_facade.h" - +#include "yql_facade.h" + #include <ydb/library/yql/core/yql_execution.h> #include <ydb/library/yql/core/yql_expr_csee.h> #include <ydb/library/yql/core/yql_expr_optimize.h> @@ -22,26 +22,26 @@ #include <ydb/library/yql/providers/common/proto/gateways_config.pb.h> #include <ydb/library/yql/providers/common/config/yql_setting.h> #include <ydb/library/yql/public/issue/yql_issue.h> - + #include <library/cpp/yson/node/node_io.h> #include <library/cpp/deprecated/split/split_iterator.h> #include <library/cpp/yson/writer.h> #include <library/cpp/string_utils/base64/base64.h> - -#include <util/stream/file.h> + +#include <util/stream/file.h> #include <util/stream/null.h> #include <util/string/split.h> -#include <util/generic/guid.h> +#include <util/generic/guid.h> #include <util/system/rusage.h> #include <util/generic/yexception.h> - - -using namespace NThreading; - -namespace NYql { - -namespace { - + + +using namespace NThreading; + +namespace NYql { + +namespace { + const size_t DEFAULT_AST_BUF_SIZE = 1024; const size_t DEFAULT_PLAN_BUF_SIZE = 1024; @@ -60,32 +60,32 @@ private: TFileStoragePtr Storage_; }; -template <typename... Params1, typename... Params2> -TProgram::TStatus SyncExecution( - TProgram* program, - TProgram::TFutureStatus (TProgram::*method)(Params1...), +template <typename... Params1, typename... Params2> +TProgram::TStatus SyncExecution( + TProgram* program, + TProgram::TFutureStatus (TProgram::*method)(Params1...), Params2&&... params) { - TProgram::TFutureStatus future = + TProgram::TFutureStatus future = (program->*method)(std::forward<Params2>(params)...); YQL_ENSURE(future.Initialized()); - future.Wait(); + future.Wait(); YQL_ENSURE(!future.HasException()); - - TProgram::TStatus status = future.GetValue(); - while (status == TProgram::TStatus::Async) { - auto continueFuture = program->ContinueAsync(); - continueFuture.Wait(); + + TProgram::TStatus status = future.GetValue(); + while (status == TProgram::TStatus::Async) { + auto continueFuture = program->ContinueAsync(); + continueFuture.Wait(); YQL_ENSURE(!continueFuture.HasException()); - status = continueFuture.GetValue(); - } - + status = continueFuture.GetValue(); + } + if (status == TProgram::TStatus::Error) { program->Print(program->ExprStream(), program->PlanStream()); } - return status; -} - + return status; +} + std::function<TMaybe<TString>(const TString&)> BuildCompositeTokenResolver(TVector<std::function<TMaybe<TString>(const TString&)>>&& children) { if (children.empty()) { return {}; @@ -106,14 +106,14 @@ std::function<TMaybe<TString>(const TString&)> BuildCompositeTokenResolver(TVect }; } -} // namspace - -/////////////////////////////////////////////////////////////////////////////// -// TProgramFactory -/////////////////////////////////////////////////////////////////////////////// +} // namspace + +/////////////////////////////////////////////////////////////////////////////// +// TProgramFactory +/////////////////////////////////////////////////////////////////////////////// TProgramFactory::TProgramFactory( bool useRepeatableRandomAndTimeProviders, - const NKikimr::NMiniKQL::IFunctionRegistry* functionRegistry, + const NKikimr::NMiniKQL::IFunctionRegistry* functionRegistry, ui64 nextUniqueId, const TVector<TDataProviderInitializer>& dataProvidersInit, const TString& runner) @@ -123,10 +123,10 @@ TProgramFactory::TProgramFactory( , DataProvidersInit_(dataProvidersInit) , GatewaysConfig_(nullptr) , Runner_(runner) -{ +{ AddCredentialsTable(std::make_shared<TCredentialTable>()); -} - +} + void TProgramFactory::UnrepeatableRandom() { UseUnrepeatableRandom = true; } @@ -172,19 +172,19 @@ void TProgramFactory::SetFileStorage(TFileStoragePtr fileStorage) { FileStorage_ = fileStorage; } -TProgramPtr TProgramFactory::Create( - const TFile& file, +TProgramPtr TProgramFactory::Create( + const TFile& file, const TString& sessionId) -{ +{ TString sourceCode = TFileInput(file).ReadAll(); - return Create(file.GetName(), sourceCode, sessionId); -} - -TProgramPtr TProgramFactory::Create( + return Create(file.GetName(), sourceCode, sessionId); +} + +TProgramPtr TProgramFactory::Create( const TString& filename, const TString& sourceCode, const TString& sessionId) -{ +{ auto randomProvider = UseRepeatableRandomAndTimeProviders_ && !UseUnrepeatableRandom ? CreateDeterministicRandomProvider(1) : CreateDefaultRandomProvider(); auto timeProvider = UseRepeatableRandomAndTimeProviders_ ? @@ -199,13 +199,13 @@ TProgramPtr TProgramFactory::Create( return new TProgram(FunctionRegistry_, randomProvider, timeProvider, NextUniqueId_, DataProvidersInit_, UserDataTable_, CredentialTables_, UserCredentials_, moduleResolver, udfResolver, udfIndex, udfIndexPackageSet, FileStorage_, GatewaysConfig_, filename, sourceCode, sessionId, Runner_, EnableRangeComputeFor_); -} - -/////////////////////////////////////////////////////////////////////////////// -// TProgram -/////////////////////////////////////////////////////////////////////////////// -TProgram::TProgram( - const NKikimr::NMiniKQL::IFunctionRegistry* functionRegistry, +} + +/////////////////////////////////////////////////////////////////////////////// +// TProgram +/////////////////////////////////////////////////////////////////////////////// +TProgram::TProgram( + const NKikimr::NMiniKQL::IFunctionRegistry* functionRegistry, const TIntrusivePtr<IRandomProvider> randomProvider, const TIntrusivePtr<ITimeProvider> timeProvider, ui64 nextUniqueId, @@ -238,18 +238,18 @@ TProgram::TProgram( , FileStorage_(fileStorage) , UserDataStorage_(MakeIntrusive<TUserDataStorage>(fileStorage, userDataTable, udfResolver, udfIndex)) , GatewaysConfig_(gatewaysConfig) - , Filename_(filename) - , SourceCode_(sourceCode) + , Filename_(filename) + , SourceCode_(sourceCode) , SourceSyntax_(ESourceSyntax::Unknown) , SyntaxVersion_(0) , AstRoot_(nullptr) , Modules_(modules) - , ExprRoot_(nullptr) - , SessionId_(sessionId) + , ExprRoot_(nullptr) + , SessionId_(sessionId) , ResultFormat_(NYson::EYsonFormat::Binary) , OutputFormat_(NYson::EYsonFormat::Pretty) , EnableRangeComputeFor_(enableRangeComputeFor) -{ +{ if (SessionId_.empty()) { SessionId_ = CreateGuidAsString(); } @@ -259,18 +259,18 @@ TProgram::TProgram( modules->SetUrlLoader(new TUrlLoader(FileStorage_)); } OperationOptions_.Runner = runner; -} - +} + TProgram::~TProgram() { - try { - CloseLastSession(); - // stop all non complete execution before deleting TExprCtx + try { + CloseLastSession(); + // stop all non complete execution before deleting TExprCtx DataProviders_.clear(); - } catch (...) { - Cerr << CurrentExceptionMessage() << Endl; - } -} - + } catch (...) { + Cerr << CurrentExceptionMessage() << Endl; + } +} + void TProgram::ConfigureYsonResultFormat(NYson::EYsonFormat format) { ResultFormat_ = format; OutputFormat_ = format; @@ -299,7 +299,7 @@ TTypeAnnotationContextPtr TProgram::GetAnnotationContext() const { TTypeAnnotationContextPtr TProgram::ProvideAnnotationContext(const TString& username) { if (!TypeCtx_) { TypeCtx_ = BuildTypeAnnotationContext(username); - TypeCtx_->OperationOptions = OperationOptions_; + TypeCtx_->OperationOptions = OperationOptions_; TypeCtx_->ValidateMode = ValidateMode_; TypeCtx_->DisableNativeUdfSupport = DisableNativeUdfSupport_; TypeCtx_->UseTableMetaFromGraph = UseTableMetaFromGraph_; @@ -366,14 +366,14 @@ bool TProgram::FillParseResult(NYql::TAstParseResult&& astRes, NYql::TWarningRul } iManager.LeaveScope(); } - if (!astRes.IsOk()) { - return false; - } - AstRoot_ = astRes.Root; + if (!astRes.IsOk()) { + return false; + } + AstRoot_ = astRes.Root; AstPool_ = std::move(astRes.Pool); - return true; -} - + return true; +} + TString TProgram::GetSessionId() const { with_lock(SessionIdLock_) { return SessionId_; @@ -396,29 +396,29 @@ bool TProgram::ParseYql() { } bool TProgram::ParseSql() { - YQL_PROFILE_FUNC(TRACE); - + YQL_PROFILE_FUNC(TRACE); + static const THashMap<TString, TString> clusters = { { "plato", TString(YtProviderName) } - }; + }; NSQLTranslation::TTranslationSettings settings; settings.ClusterMapping = clusters; return ParseSql(settings); -} - +} + bool TProgram::ParseSql(const NSQLTranslation::TTranslationSettings& settings) -{ - YQL_PROFILE_FUNC(TRACE); +{ + YQL_PROFILE_FUNC(TRACE); YQL_ENSURE(SourceSyntax_ == ESourceSyntax::Unknown); SourceSyntax_ = ESourceSyntax::Sql; SyntaxVersion_ = settings.SyntaxVersion; NYql::TWarningRules warningRules; return FillParseResult(SqlToYql(SourceCode_, settings, &warningRules), &warningRules); -} - +} + bool TProgram::Compile(const TString& username) { - YQL_PROFILE_FUNC(TRACE); + YQL_PROFILE_FUNC(TRACE); Y_ENSURE(AstRoot_, "Program not parsed yet"); if (!ExprCtx_) { @@ -443,8 +443,8 @@ bool TProgram::Compile(const TString& username) { } return true; -} - +} + bool TProgram::CollectUsedClusters() { using namespace NNodes; @@ -545,17 +545,17 @@ TProgram::TFutureStatus TProgram::DiscoverAsync(const TString& username) { } TProgram::TStatus TProgram::Validate(const TString& username, IOutputStream* exprOut, bool withTypes) { - YQL_PROFILE_FUNC(TRACE); - auto m = &TProgram::ValidateAsync; + YQL_PROFILE_FUNC(TRACE); + auto m = &TProgram::ValidateAsync; return SyncExecution(this, m, username, exprOut, withTypes); -} - +} + TProgram::TFutureStatus TProgram::ValidateAsync(const TString& username, IOutputStream* exprOut, bool withTypes) { if (!ProvideAnnotationContext(username)->Initialize(*ExprCtx_) || !CollectUsedClusters()) { return NThreading::MakeFuture<TStatus>(IGraphTransformer::TStatus::Error); - } + } TypeCtx_->IsReadOnly = true; - + for (const auto& dp : DataProviders_) { if (!dp.RemoteClusterProvider || !dp.RemoteValidate) { continue; @@ -577,13 +577,13 @@ TProgram::TFutureStatus TProgram::ValidateAsync(const TString& username, IOutput .AddIOAnnotation() .AddTypeAnnotation() .Add(TExprOutputTransformer::Sync(ExprRoot_, exprOut, withTypes), "AstOutput") - .Build(); - - TFuture<void> openSession = OpenSession(username); + .Build(); + + TFuture<void> openSession = OpenSession(username); if (!openSession.Initialized()) { return NThreading::MakeFuture<TStatus>(IGraphTransformer::TStatus::Error); } - + return openSession.Apply([this](const TFuture<void>& f) { YQL_LOG_CTX_ROOT_SCOPE(GetSessionId()); try { @@ -594,33 +594,33 @@ TProgram::TFutureStatus TProgram::ValidateAsync(const TString& username, IOutput return NThreading::MakeFuture<TStatus>(IGraphTransformer::TStatus::Error); } return AsyncTransformWithFallback(false); - }); -} - -TProgram::TStatus TProgram::Optimize( + }); +} + +TProgram::TStatus TProgram::Optimize( const TString& username, IOutputStream* traceOut, IOutputStream* tracePlan, IOutputStream* exprOut, - bool withTypes) -{ - YQL_PROFILE_FUNC(TRACE); - auto m = &TProgram::OptimizeAsync; + bool withTypes) +{ + YQL_PROFILE_FUNC(TRACE); + auto m = &TProgram::OptimizeAsync; return SyncExecution(this, m, username, traceOut, tracePlan, exprOut, withTypes); -} - -TProgram::TFutureStatus TProgram::OptimizeAsync( +} + +TProgram::TFutureStatus TProgram::OptimizeAsync( const TString& username, IOutputStream* traceOut, IOutputStream* tracePlan, IOutputStream* exprOut, - bool withTypes) -{ + bool withTypes) +{ if (!ProvideAnnotationContext(username)->Initialize(*ExprCtx_) || !CollectUsedClusters()) { return NThreading::MakeFuture<TStatus>(IGraphTransformer::TStatus::Error); - } + } TypeCtx_->IsReadOnly = true; - + for (const auto& dp : DataProviders_) { if (!dp.RemoteClusterProvider || !dp.RemoteOptimize) { continue; @@ -650,12 +650,12 @@ TProgram::TFutureStatus TProgram::OptimizeAsync( .Add(CreatePlanInfoTransformer(*TypeCtx_), "PlanInfo") .Add(TExprOutputTransformer::Sync(ExprRoot_, exprOut, withTypes), "AstOutput") .Add(TPlanOutputTransformer::Sync(tracePlan, GetPlanBuilder(), OutputFormat_), "PlanOutput") - .Build(); - - TFuture<void> openSession = OpenSession(username); - if (!openSession.Initialized()) + .Build(); + + TFuture<void> openSession = OpenSession(username); + if (!openSession.Initialized()) return NThreading::MakeFuture<TStatus>(IGraphTransformer::TStatus::Error); - + return openSession.Apply([this](const TFuture<void>& f) { YQL_LOG_CTX_ROOT_SCOPE(GetSessionId()); try { @@ -666,25 +666,25 @@ TProgram::TFutureStatus TProgram::OptimizeAsync( return NThreading::MakeFuture<TStatus>(IGraphTransformer::TStatus::Error); } return AsyncTransformWithFallback(false); - }); -} - -TProgram::TStatus TProgram::OptimizeWithConfig( + }); +} + +TProgram::TStatus TProgram::OptimizeWithConfig( const TString& username, const IPipelineConfigurator& pipelineConf) -{ - YQL_PROFILE_FUNC(TRACE); - auto m = &TProgram::OptimizeAsyncWithConfig; - return SyncExecution(this, m, username, pipelineConf); -} - -TProgram::TFutureStatus TProgram::OptimizeAsyncWithConfig( +{ + YQL_PROFILE_FUNC(TRACE); + auto m = &TProgram::OptimizeAsyncWithConfig; + return SyncExecution(this, m, username, pipelineConf); +} + +TProgram::TFutureStatus TProgram::OptimizeAsyncWithConfig( const TString& username, const IPipelineConfigurator& pipelineConf) -{ +{ if (!ProvideAnnotationContext(username)->Initialize(*ExprCtx_) || !CollectUsedClusters()) { - return NThreading::MakeFuture<TStatus>(IGraphTransformer::TStatus::Error); - } + return NThreading::MakeFuture<TStatus>(IGraphTransformer::TStatus::Error); + } TypeCtx_->IsReadOnly = true; - + for (const auto& dp : DataProviders_) { if (!dp.RemoteClusterProvider || !dp.RemoteOptimize) { continue; @@ -700,7 +700,7 @@ TProgram::TFutureStatus TProgram::OptimizeAsyncWithConfig( Y_ENSURE(ExprRoot_, "Program not compiled yet"); TTransformationPipeline pipeline(TypeCtx_); - pipelineConf.AfterCreate(&pipeline); + pipelineConf.AfterCreate(&pipeline); pipeline.AddServiceTransformers(); pipeline.AddParametersEvaluation(*FunctionRegistry_); pipeline.AddPreTypeAnnotation(); @@ -708,8 +708,8 @@ TProgram::TFutureStatus TProgram::OptimizeAsyncWithConfig( pipeline.AddIOAnnotation(); pipeline.AddTypeAnnotation(); pipeline.AddPostTypeAnnotation(); - pipelineConf.AfterTypeAnnotation(&pipeline); - + pipelineConf.AfterTypeAnnotation(&pipeline); + pipeline.AddOptimization(); if (EnableRangeComputeFor_) { pipeline.Add(MakeExpandRangeComputeForTransformer(pipeline.GetTypeAnnotationContext()), @@ -717,14 +717,14 @@ TProgram::TFutureStatus TProgram::OptimizeAsyncWithConfig( } pipeline.Add(CreatePlanInfoTransformer(*TypeCtx_), "PlanInfo"); - pipelineConf.AfterOptimize(&pipeline); - - Transformer_ = pipeline.Build(); - - TFuture<void> openSession = OpenSession(username); - if (!openSession.Initialized()) - return NThreading::MakeFuture<TStatus>(IGraphTransformer::TStatus::Error); - + pipelineConf.AfterOptimize(&pipeline); + + Transformer_ = pipeline.Build(); + + TFuture<void> openSession = OpenSession(username); + if (!openSession.Initialized()) + return NThreading::MakeFuture<TStatus>(IGraphTransformer::TStatus::Error); + return openSession.Apply([this](const TFuture<void>& f) { YQL_LOG_CTX_ROOT_SCOPE(GetSessionId()); try { @@ -735,33 +735,33 @@ TProgram::TFutureStatus TProgram::OptimizeAsyncWithConfig( return NThreading::MakeFuture<TStatus>(IGraphTransformer::TStatus::Error); } return AsyncTransformWithFallback(false); - }); -} - -TProgram::TStatus TProgram::Run( + }); +} + +TProgram::TStatus TProgram::Run( const TString& username, IOutputStream* traceOut, IOutputStream* tracePlan, IOutputStream* exprOut, - bool withTypes) -{ - YQL_PROFILE_FUNC(TRACE); - auto m = &TProgram::RunAsync; + bool withTypes) +{ + YQL_PROFILE_FUNC(TRACE); + auto m = &TProgram::RunAsync; return SyncExecution(this, m, username, traceOut, tracePlan, exprOut, withTypes); -} - -TProgram::TFutureStatus TProgram::RunAsync( +} + +TProgram::TFutureStatus TProgram::RunAsync( const TString& username, IOutputStream* traceOut, IOutputStream* tracePlan, IOutputStream* exprOut, - bool withTypes) -{ + bool withTypes) +{ if (!ProvideAnnotationContext(username)->Initialize(*ExprCtx_) || !CollectUsedClusters()) { return NThreading::MakeFuture<TStatus>(IGraphTransformer::TStatus::Error); - } + } TypeCtx_->IsReadOnly = false; - + for (const auto& dp : DataProviders_) { if (!dp.RemoteClusterProvider || !dp.RemoteRun) { continue; @@ -779,7 +779,7 @@ TProgram::TFutureStatus TProgram::RunAsync( ExprStream_ = exprOut; PlanStream_ = tracePlan; - + TTransformationPipeline pipeline(TypeCtx_); pipeline.AddServiceTransformers(); pipeline.AddParametersEvaluation(*FunctionRegistry_); @@ -800,7 +800,7 @@ TProgram::TFutureStatus TProgram::RunAsync( Transformer_ = pipeline.Build(); - TFuture<void> openSession = OpenSession(username); + TFuture<void> openSession = OpenSession(username); if (!openSession.Initialized()) { return NThreading::MakeFuture<TStatus>(IGraphTransformer::TStatus::Error); } @@ -817,25 +817,25 @@ TProgram::TFutureStatus TProgram::RunAsync( return NThreading::MakeFuture<TStatus>(IGraphTransformer::TStatus::Error); } return AsyncTransformWithFallback(false); - }); -} - -TProgram::TStatus TProgram::RunWithConfig( + }); +} + +TProgram::TStatus TProgram::RunWithConfig( const TString& username, const IPipelineConfigurator& pipelineConf) -{ - YQL_PROFILE_FUNC(TRACE); - auto m = &TProgram::RunAsyncWithConfig; - return SyncExecution(this, m, username, pipelineConf); -} - -TProgram::TFutureStatus TProgram::RunAsyncWithConfig( +{ + YQL_PROFILE_FUNC(TRACE); + auto m = &TProgram::RunAsyncWithConfig; + return SyncExecution(this, m, username, pipelineConf); +} + +TProgram::TFutureStatus TProgram::RunAsyncWithConfig( const TString& username, const IPipelineConfigurator& pipelineConf) -{ +{ if (!ProvideAnnotationContext(username)->Initialize(*ExprCtx_) || !CollectUsedClusters()) { - return NThreading::MakeFuture<TStatus>(IGraphTransformer::TStatus::Error); - } + return NThreading::MakeFuture<TStatus>(IGraphTransformer::TStatus::Error); + } TypeCtx_->IsReadOnly = false; - + for (const auto& dp : DataProviders_) { if (!dp.RemoteClusterProvider || !dp.RemoteRun) { continue; @@ -852,7 +852,7 @@ TProgram::TFutureStatus TProgram::RunAsyncWithConfig( Y_ENSURE(ExprRoot_, "Program not compiled yet"); TTransformationPipeline pipeline(TypeCtx_); - pipelineConf.AfterCreate(&pipeline); + pipelineConf.AfterCreate(&pipeline); pipeline.AddServiceTransformers(); pipeline.AddParametersEvaluation(*FunctionRegistry_); pipeline.AddPreTypeAnnotation(); @@ -860,23 +860,23 @@ TProgram::TFutureStatus TProgram::RunAsyncWithConfig( pipeline.AddIOAnnotation(); pipeline.AddTypeAnnotation(); pipeline.AddPostTypeAnnotation(); - pipelineConf.AfterTypeAnnotation(&pipeline); - + pipelineConf.AfterTypeAnnotation(&pipeline); + pipeline.AddOptimization(); if (EnableRangeComputeFor_) { pipeline.Add(MakeExpandRangeComputeForTransformer(pipeline.GetTypeAnnotationContext()), "ExpandRangeComputeFor", TIssuesIds::CORE_EXEC); } - pipelineConf.AfterOptimize(&pipeline); + pipelineConf.AfterOptimize(&pipeline); pipeline.AddRun(ProgressWriter_); - - Transformer_ = pipeline.Build(); - - TFuture<void> openSession = OpenSession(username); - if (!openSession.Initialized()) { - return NThreading::MakeFuture<TStatus>(IGraphTransformer::TStatus::Error); - } - + + Transformer_ = pipeline.Build(); + + TFuture<void> openSession = OpenSession(username); + if (!openSession.Initialized()) { + return NThreading::MakeFuture<TStatus>(IGraphTransformer::TStatus::Error); + } + SavedExprRoot_ = ExprRoot_; return openSession.Apply([this](const TFuture<void>& f) { @@ -889,9 +889,9 @@ TProgram::TFutureStatus TProgram::RunAsyncWithConfig( return NThreading::MakeFuture<TStatus>(IGraphTransformer::TStatus::Error); } return AsyncTransformWithFallback(false); - }); -} - + }); +} + TFuture<IGraphTransformer::TStatus> TProgram::AsyncTransformWithFallback(bool applyAsyncChanges) { return AsyncTransform(*Transformer_, ExprRoot_, *ExprCtx_, applyAsyncChanges).Apply([this](const TFuture<IGraphTransformer::TStatus>& res) { @@ -1192,13 +1192,13 @@ TMaybe<TString> TProgram::GetDiscoveredData() { TProgram::TFutureStatus TProgram::ContinueAsync() { YQL_LOG_CTX_ROOT_SCOPE(GetSessionId()); return AsyncTransformWithFallback(true); -} - -void TProgram::Abort() -{ - CloseLastSession(); -} - +} + +void TProgram::Abort() +{ + CloseLastSession(); +} + void TProgram::CleanupLastSession() { YQL_LOG_CTX_ROOT_SCOPE(GetSessionId()); @@ -1216,36 +1216,36 @@ void TProgram::CleanupLastSession() { void TProgram::CloseLastSession() { YQL_LOG_CTX_ROOT_SCOPE(GetSessionId()); - + TString sessionId = TakeSessionId(); if (sessionId.empty()) { - return; - } - + return; + } + for (const auto& dp : DataProviders_) { if (dp.CloseSession) { dp.CloseSession(sessionId); } - } -} - + } +} + TString TProgram::ResultsAsString() const { - if (!ResultProviderConfig_) - return ""; - - TStringStream resultOut; + if (!ResultProviderConfig_) + return ""; + + TStringStream resultOut; NYson::TYsonWriter yson(&resultOut, OutputFormat_); - yson.OnBeginList(); - for (const auto& result: Results()) { - yson.OnListItem(); - yson.OnRaw(result); - } - yson.OnEndList(); - return resultOut.Str(); -} - + yson.OnBeginList(); + for (const auto& result: Results()) { + yson.OnListItem(); + yson.OnRaw(result); + } + yson.OnEndList(); + return resultOut.Str(); +} + TTypeAnnotationContextPtr TProgram::BuildTypeAnnotationContext(const TString& username) { - auto typeAnnotationContext = MakeIntrusive<TTypeAnnotationContext>(); + auto typeAnnotationContext = MakeIntrusive<TTypeAnnotationContext>(); typeAnnotationContext->UserDataStorage = UserDataStorage_; typeAnnotationContext->Credentials = CredentialTables_; @@ -1281,7 +1281,7 @@ TTypeAnnotationContextPtr TProgram::BuildTypeAnnotationContext(const TString& us ProgressWriter_, OperationOptions_ ); - + providerNames.insert(dp.Names.begin(), dp.Names.end()); DataProviders_.emplace_back(dp); if (dp.Source) { @@ -1332,8 +1332,8 @@ TTypeAnnotationContextPtr TProgram::BuildTypeAnnotationContext(const TString& us auto resultProvider = CreateResultProvider(ResultProviderConfig_); typeAnnotationContext->AddDataSink(ResultProviderName, resultProvider); typeAnnotationContext->AvailablePureResultDataSources = resultProviderDataSources; - } - + } + if (!fullResultDataSinks.empty()) { typeAnnotationContext->FullResultDataSink = fullResultDataSinks.front(); } @@ -1344,11 +1344,11 @@ TTypeAnnotationContextPtr TProgram::BuildTypeAnnotationContext(const TString& us } typeAnnotationContext->UserDataStorage->SetTokenResolver(BuildCompositeTokenResolver(std::move(tokenResolvers))); - return typeAnnotationContext; -} - + return typeAnnotationContext; +} + TFuture<void> TProgram::OpenSession(const TString& username) -{ +{ TVector<TFuture<void>> openFutures; for (const auto& dp : DataProviders_) { if (dp.OpenSession) { @@ -1356,11 +1356,11 @@ TFuture<void> TProgram::OpenSession(const TString& username) RandomProvider_, TimeProvider_); openFutures.push_back(future); } - } - + } + return WaitExceptionOrAll(openFutures); -} - +} + void TProgram::Print(IOutputStream* exprOut, IOutputStream* planOut, bool cleanPlan) { TVector<TTransformStage> printTransformers; const auto issueCode = TIssuesIds::DEFAULT_ERROR; @@ -1393,4 +1393,4 @@ bool TProgram::HasActiveProcesses() { return false; } -} // namespace NYql +} // namespace NYql diff --git a/ydb/library/yql/core/facade/yql_facade.h b/ydb/library/yql/core/facade/yql_facade.h index ffcc884c65f..1f8f47681da 100644 --- a/ydb/library/yql/core/facade/yql_facade.h +++ b/ydb/library/yql/core/facade/yql_facade.h @@ -1,5 +1,5 @@ -#pragma once - +#pragma once + #include <ydb/library/yql/core/services/yql_plan.h> #include <ydb/library/yql/core/services/yql_transform_pipeline.h> #include <ydb/library/yql/core/file_storage/file_storage.h> @@ -11,35 +11,35 @@ #include <library/cpp/random_provider/random_provider.h> #include <library/cpp/time_provider/time_provider.h> #include <library/cpp/threading/future/future.h> - -#include <util/system/file.h> + +#include <util/system/file.h> #include <util/generic/ptr.h> - + #include <functional> - + namespace NKikimr { namespace NMiniKQL { class IFunctionRegistry; } } -namespace NYql { - +namespace NYql { + class TGatewaysConfig; -class TProgram; +class TProgram; using TProgramPtr = TIntrusivePtr<TProgram>; class TProgramFactory; using TProgramFactoryPtr = TIntrusivePtr<TProgramFactory>; - -/////////////////////////////////////////////////////////////////////////////// -// TProgramFactory -/////////////////////////////////////////////////////////////////////////////// + +/////////////////////////////////////////////////////////////////////////////// +// TProgramFactory +/////////////////////////////////////////////////////////////////////////////// class TProgramFactory: public TThrRefBase, private TMoveOnly -{ -public: +{ +public: TProgramFactory( bool useRepeatableRandomAndTimeProviders, - const NKikimr::NMiniKQL::IFunctionRegistry* functionRegistry, + const NKikimr::NMiniKQL::IFunctionRegistry* functionRegistry, ui64 nextUniqueId, const TVector<TDataProviderInitializer>& dataProvidersInit, const TString& runner); @@ -53,21 +53,21 @@ public: void SetUdfIndex(TUdfIndex::TPtr udfIndex, TUdfIndexPackageSet::TPtr udfIndexPackageSet); void SetFileStorage(TFileStoragePtr fileStorage); void EnableRangeComputeFor(); - - TProgramPtr Create( - const TFile& file, + + TProgramPtr Create( + const TFile& file, const TString& sessionId = TString()); - - TProgramPtr Create( + + TProgramPtr Create( const TString& filename, const TString& sourceCode, const TString& sessionId = TString()); - + void UnrepeatableRandom(); -private: +private: const bool UseRepeatableRandomAndTimeProviders_; bool UseUnrepeatableRandom = false; - const NKikimr::NMiniKQL::IFunctionRegistry* FunctionRegistry_; + const NKikimr::NMiniKQL::IFunctionRegistry* FunctionRegistry_; const ui64 NextUniqueId_; TVector<TDataProviderInitializer> DataProvidersInit_; TUserDataTable UserDataTable_; @@ -81,93 +81,93 @@ private: TFileStoragePtr FileStorage_; TString Runner_; bool EnableRangeComputeFor_ = false; -}; - -/////////////////////////////////////////////////////////////////////////////// -// TProgram -/////////////////////////////////////////////////////////////////////////////// +}; + +/////////////////////////////////////////////////////////////////////////////// +// TProgram +/////////////////////////////////////////////////////////////////////////////// class TProgram: public TThrRefBase, private TNonCopyable -{ -public: - friend TProgramFactory; - using TStatus = IGraphTransformer::TStatus; - using TFutureStatus = NThreading::TFuture<TStatus>; - -public: - ~TProgram(); - - bool ParseYql(); +{ +public: + friend TProgramFactory; + using TStatus = IGraphTransformer::TStatus; + using TFutureStatus = NThreading::TFuture<TStatus>; + +public: + ~TProgram(); + + bool ParseYql(); bool ParseSql(); bool ParseSql(const NSQLTranslation::TTranslationSettings& settings); - + bool Compile(const TString& username); - + TStatus Discover(const TString& username); TFutureStatus DiscoverAsync(const TString& username); TStatus Validate(const TString& username, IOutputStream* exprOut = nullptr, bool withTypes = false); - + TFutureStatus ValidateAsync(const TString& username, IOutputStream* exprOut = nullptr, bool withTypes = false); - - TStatus Optimize( + + TStatus Optimize( const TString& username, IOutputStream* traceOut = nullptr, IOutputStream* tracePlan = nullptr, IOutputStream* exprOut = nullptr, - bool withTypes = false); - - TFutureStatus OptimizeAsync( + bool withTypes = false); + + TFutureStatus OptimizeAsync( const TString& username, IOutputStream* traceOut = nullptr, IOutputStream* tracePlan = nullptr, IOutputStream* exprOut = nullptr, - bool withTypes = false); - - TStatus Run( + bool withTypes = false); + + TStatus Run( const TString& username, IOutputStream* traceOut = nullptr, IOutputStream* tracePlan = nullptr, IOutputStream* exprOut = nullptr, - bool withTypes = false); - - TFutureStatus RunAsync( + bool withTypes = false); + + TFutureStatus RunAsync( const TString& username, IOutputStream* traceOut = nullptr, IOutputStream* tracePlan = nullptr, IOutputStream* exprOut = nullptr, - bool withTypes = false); - - TStatus OptimizeWithConfig( + bool withTypes = false); + + TStatus OptimizeWithConfig( const TString& username, - const IPipelineConfigurator& pipelineConf); - - TFutureStatus OptimizeAsyncWithConfig( + const IPipelineConfigurator& pipelineConf); + + TFutureStatus OptimizeAsyncWithConfig( const TString& username, - const IPipelineConfigurator& pipelineConf); - - TStatus RunWithConfig( + const IPipelineConfigurator& pipelineConf); + + TStatus RunWithConfig( const TString& username, - const IPipelineConfigurator& pipelineConf); - - TFutureStatus RunAsyncWithConfig( + const IPipelineConfigurator& pipelineConf); + + TFutureStatus RunAsyncWithConfig( const TString& username, - const IPipelineConfigurator& pipelineConf); - - TFutureStatus ContinueAsync(); - + const IPipelineConfigurator& pipelineConf); + + TFutureStatus ContinueAsync(); + bool HasActiveProcesses(); - void Abort(); - + void Abort(); + inline TIssues Issues() { if (ExprCtx_) { return ExprCtx_->IssueManager.GetIssues(); } else { return {}; } - } - + } + inline TIssues CompletedIssues() const { if (ExprCtx_) { return ExprCtx_->IssueManager.GetCompletedIssues(); @@ -182,29 +182,29 @@ public: if (ExprCtx_) { ExprCtx_->IssueManager.GetIssues().PrintWithProgramTo(out, Filename_, SourceCode_); } - } - - inline const TAstNode* AstRoot() const { + } + + inline const TAstNode* AstRoot() const { return AstRoot_; - } - + } + inline const TExprNode::TPtr& ExprRoot() const { - return ExprRoot_; - } - + return ExprRoot_; + } + inline TExprContext& ExprCtx() const { return *ExprCtx_; } - inline bool HasResults() const { - return ResultProviderConfig_ && - !ResultProviderConfig_->CommittedResults.empty(); - } - + inline bool HasResults() const { + return ResultProviderConfig_ && + !ResultProviderConfig_->CommittedResults.empty(); + } + inline const TVector<TString>& Results() const { - return ResultProviderConfig_->CommittedResults; - } - + return ResultProviderConfig_->CommittedResults; + } + TMaybe<TString> GetQueryAst(); TMaybe<TString> GetQueryPlan(); @@ -223,13 +223,13 @@ public: TString ResultsAsString() const; void ConfigureYsonResultFormat(NYson::EYsonFormat format); - + inline IOutputStream* ExprStream() const { return ExprStream_; } inline IOutputStream* PlanStream() const { return PlanStream_; } NYson::EYsonFormat GetResultFormat() const { return ResultFormat_; } NYson::EYsonFormat GetOutputFormat() const { return OutputFormat_; } - + void SetValidateOptions(NUdf::EValidateMode validateMode); void SetDisableNativeUdfSupport(bool disable); void SetUseTableMetaFromGraph(bool use); @@ -237,8 +237,8 @@ public: void SetProgressWriter(TOperationProgressWriter writer) { Y_ENSURE(!TypeCtx_, "TypeCtx_ already created"); ProgressWriter_ = ThreadSafeProgressWriter(writer); - } - + } + void SetAuthenticatedUser(const TString& user) { Y_ENSURE(!TypeCtx_, "TypeCtx_ already created"); OperationOptions_.AuthenticatedUser = user; @@ -246,9 +246,9 @@ public: void SetOperationId(const TString& id) { Y_ENSURE(!TypeCtx_, "TypeCtx_ already created"); - OperationOptions_.Id = id; - } - + OperationOptions_.Id = id; + } + void SetSharedOperationId(const TString& id) { Y_ENSURE(!TypeCtx_, "TypeCtx_ already created"); OperationOptions_.SharedId = id; @@ -260,9 +260,9 @@ public: ythrow yexception() << "Please mention YQL in the title '" << title << "'"; } - OperationOptions_.Title = title; - } - + OperationOptions_.Title = title; + } + void SetOperationUrl(const TString& url) { Y_ENSURE(!TypeCtx_, "TypeCtx_ already created"); OperationOptions_.Url = url; @@ -275,9 +275,9 @@ public: void SetOperationAttrsYson(const TString& attrs) { Y_ENSURE(!TypeCtx_, "TypeCtx_ already created"); - OperationOptions_.AttrsYson = attrs; - } - + OperationOptions_.AttrsYson = attrs; + } + void SetParametersYson(const TString& parameters); // should be used after Compile phase bool ExtractQueryParametersMetadata(); @@ -292,9 +292,9 @@ public: IPlanBuilder& GetPlanBuilder(); -private: - TProgram( - const NKikimr::NMiniKQL::IFunctionRegistry* functionRegistry, +private: + TProgram( + const NKikimr::NMiniKQL::IFunctionRegistry* functionRegistry, const TIntrusivePtr<IRandomProvider> randomProvider, const TIntrusivePtr<ITimeProvider> timeProvider, ui64 nextUniqueId, @@ -313,17 +313,17 @@ private: const TString& sessionId, const TString& runner, bool enableRangeComputeFor); - + TTypeAnnotationContextPtr BuildTypeAnnotationContext(const TString& username); TTypeAnnotationContextPtr GetAnnotationContext() const; TTypeAnnotationContextPtr ProvideAnnotationContext(const TString& username); bool CollectUsedClusters(); NThreading::TFuture<void> OpenSession(const TString& username); - + void CleanupLastSession(); - void CloseLastSession(); - + void CloseLastSession(); + TFutureStatus RemoteKikimrValidate(const TString& cluster); TFutureStatus RemoteKikimrOptimize(const TString& cluster, const IPipelineConfigurator* pipelineConf); TFutureStatus RemoteKikimrRun(const TString& cluster, const IPipelineConfigurator* pipelineConf); @@ -335,13 +335,13 @@ private: NThreading::TFuture<IGraphTransformer::TStatus> AsyncTransformWithFallback(bool applyAsyncChanges); private: - const NKikimr::NMiniKQL::IFunctionRegistry* FunctionRegistry_; + const NKikimr::NMiniKQL::IFunctionRegistry* FunctionRegistry_; const TIntrusivePtr<IRandomProvider> RandomProvider_; const TIntrusivePtr<ITimeProvider> TimeProvider_; const ui64 NextUniqueId_; TVector<TDataProviderInitializer> DataProvidersInit_; TVector<TDataProviderInfo> DataProviders_; - TYqlOperationOptions OperationOptions_; + TYqlOperationOptions OperationOptions_; TVector<TCredentialTablePtr> CredentialTables_; TUserCredentials UserCredentials_; const IUdfResolver::TPtr UdfResolver_; @@ -354,7 +354,7 @@ private: TString SourceCode_; ESourceSyntax SourceSyntax_; ui16 SyntaxVersion_; - + TAstNode* AstRoot_; std::unique_ptr<TMemoryPool> AstPool_; TAutoPtr<TExprContext> ExprCtx_; @@ -363,10 +363,10 @@ private: TExprNode::TPtr SavedExprRoot_; mutable TAdaptiveLock SessionIdLock_; TString SessionId_; - TTypeAnnotationContextPtr TypeCtx_; + TTypeAnnotationContextPtr TypeCtx_; TAutoPtr<IPlanBuilder> PlanBuilder_; - TAutoPtr<IGraphTransformer> Transformer_; - TIntrusivePtr<TResultProviderConfig> ResultProviderConfig_; + TAutoPtr<IGraphTransformer> Transformer_; + TIntrusivePtr<TResultProviderConfig> ResultProviderConfig_; bool SupportsResultPosition_ = false; NYson::EYsonFormat ResultFormat_; NYson::EYsonFormat OutputFormat_; @@ -385,6 +385,6 @@ private: TOperationProgressWriter ProgressWriter_ = [](const TOperationProgress&) {}; TString ExtractedQueryParametersMetadataYson_; const bool EnableRangeComputeFor_; -}; - -} // namspace NYql +}; + +} // namspace NYql diff --git a/ydb/library/yql/core/file_storage/file_storage.cpp b/ydb/library/yql/core/file_storage/file_storage.cpp index b7a0b040dc9..631adbc688f 100644 --- a/ydb/library/yql/core/file_storage/file_storage.cpp +++ b/ydb/library/yql/core/file_storage/file_storage.cpp @@ -299,7 +299,7 @@ private: TMultiResourceLock MultiResourceLock; }; -TFileStoragePtr CreateFileStorage(const TFileStorageConfig& params) { +TFileStoragePtr CreateFileStorage(const TFileStorageConfig& params) { Y_ENSURE(0 != params.GetMaxFiles(), "FileStorage: MaxFiles must be greater than 0"); Y_ENSURE(0 != params.GetMaxSizeMb(), "FileStorage: MaxSizeMb must be greater than 0"); return new TFileStorageImpl(params); diff --git a/ydb/library/yql/core/file_storage/file_storage.h b/ydb/library/yql/core/file_storage/file_storage.h index addff506e3e..9133d57b1b3 100644 --- a/ydb/library/yql/core/file_storage/file_storage.h +++ b/ydb/library/yql/core/file_storage/file_storage.h @@ -1,7 +1,7 @@ #pragma once -#include "storage.h" - +#include "storage.h" + #include <library/cpp/threading/future/future.h> #include <library/cpp/uri/http_url.h> @@ -24,7 +24,7 @@ struct IFileStorage: public TThrRefBase { }; using IDownloaderPtr = TIntrusivePtr<IDownloader>; - virtual ~IFileStorage() = default; + virtual ~IFileStorage() = default; virtual void AddDownloader(IDownloaderPtr downloader) = 0; virtual TFileLinkPtr PutFile(const TString& file, const TString& outFileName = {}) = 0; virtual TFileLinkPtr PutFileStripped(const TString& file, const TString& originalMd5 = {}) = 0; @@ -43,6 +43,6 @@ struct IFileStorage: public TThrRefBase { using TFileStoragePtr = TIntrusivePtr<IFileStorage>; // Will use auto-cleaned temporary directory if storagePath is empty -TFileStoragePtr CreateFileStorage(const TFileStorageConfig& params); +TFileStoragePtr CreateFileStorage(const TFileStorageConfig& params); } // NYql diff --git a/ydb/library/yql/core/file_storage/proto/file_storage.proto b/ydb/library/yql/core/file_storage/proto/file_storage.proto index d52aabf3a8a..c8fa642932c 100644 --- a/ydb/library/yql/core/file_storage/proto/file_storage.proto +++ b/ydb/library/yql/core/file_storage/proto/file_storage.proto @@ -5,7 +5,7 @@ message TSchemeTranslate { required string TargetUrl = 2; // replacement string for target URL } -message TFileStorageConfig { +message TFileStorageConfig { optional string Path = 1; // Path to file storage. An auto-cleaned temp directory is used for empty value optional uint32 MaxFiles = 2 [default = 1000]; // Maximum number of files in the storage optional uint32 MaxSizeMb = 3 [default = 100]; // Maximum total size of all files in the storage diff --git a/ydb/library/yql/core/file_storage/storage.cpp b/ydb/library/yql/core/file_storage/storage.cpp index 6d91621dc0b..247bff52422 100644 --- a/ydb/library/yql/core/file_storage/storage.cpp +++ b/ydb/library/yql/core/file_storage/storage.cpp @@ -6,18 +6,18 @@ #include <library/cpp/digest/md5/md5.h> -#include <util/folder/dirut.h> -#include <util/generic/algorithm.h> -#include <util/generic/vector.h> -#include <util/generic/yexception.h> +#include <util/folder/dirut.h> +#include <util/generic/algorithm.h> +#include <util/generic/vector.h> +#include <util/generic/yexception.h> #include <util/generic/ptr.h> #include <util/generic/utility.h> #include <util/system/file.h> #include <util/system/file_lock.h> -#include <util/system/fs.h> +#include <util/system/fs.h> #include <util/system/maxlen.h> -#include <util/system/mutex.h> -#include <util/system/utime.h> +#include <util/system/mutex.h> +#include <util/system/utime.h> #include <util/system/thread.h> #include <functional> @@ -26,31 +26,31 @@ #include <pthread.h> #endif -#include <errno.h> - - +#include <errno.h> + + namespace NYql { namespace { -struct TFileObject { +struct TFileObject { TString Name; - time_t MTime; - ui64 Size; -}; - + time_t MTime; + ui64 Size; +}; + TFsPath ToFilePath(const TString& path) -{ +{ if (path.empty()) { - char tempDir[MAX_PATH]; - if (MakeTempDir(tempDir, nullptr) != 0) - ythrow yexception() << "FileStorage: Can't create temporary directory " << tempDir; - return tempDir; - } - return path; + char tempDir[MAX_PATH]; + if (MakeTempDir(tempDir, nullptr) != 0) + ythrow yexception() << "FileStorage: Can't create temporary directory " << tempDir; + return tempDir; + } + return path; } -} // namespace +} // namespace TFileLink::TFileLink(const TFsPath& path, const TString& storageFileName, ui64 size, const TString& md5, bool deleteOnDestroy) : Path(path) @@ -150,15 +150,15 @@ public: }; TImpl(size_t maxFiles, ui64 maxSize, const TString& storagePath) - : StorageDir(ToFilePath(storagePath)) - , ProcessTempDir(StorageDir / ToString(GetPID())) // must be subfolder for fast hardlinking + : StorageDir(ToFilePath(storagePath)) + , ProcessTempDir(StorageDir / ToString(GetPID())) // must be subfolder for fast hardlinking , IsTemp(storagePath.empty()) - , MaxFiles(maxFiles) - , MaxSize(maxSize) + , MaxFiles(maxFiles) + , MaxSize(maxSize) { // TFsPath is not thread safe. It can initialize internal Split at any time. Force do it right now - StorageDir.PathSplit(); - ProcessTempDir.PathSplit(); + StorageDir.PathSplit(); + ProcessTempDir.PathSplit(); StorageDir.MkDirs(MODE0711); ProcessTempDir.MkDirs(MODE0711); @@ -167,31 +167,31 @@ public: ProcessTempDirLock->Acquire(); // We never explicitly release this lock. It will be released when all file handles (including those in child processes) will be closed #endif - - if (!IsTemp) { - LoadStats(); - } + + if (!IsTemp) { + LoadStats(); + } TAtforkReinit::Get().Register(this); - YQL_LOG(INFO) << "FileStorage initialized in " << StorageDir.GetPath().Quote() + YQL_LOG(INFO) << "FileStorage initialized in " << StorageDir.GetPath().Quote() << ", temporary dir: " << ProcessTempDir.GetPath().Quote() << ", files: " << CurrentFiles - << ", total size: " << CurrentSize; + << ", total size: " << CurrentSize; } - ~TImpl() { + ~TImpl() { TAtforkReinit::Get().Unregister(this); try { - ProcessTempDir.ForceDelete(); + ProcessTempDir.ForceDelete(); if (IsTemp) { - StorageDir.ForceDelete(); + StorageDir.ForceDelete(); } } catch (...) { - YQL_LOG(ERROR) << CurrentExceptionMessage(); + YQL_LOG(ERROR) << CurrentExceptionMessage(); } } - const TFsPath& GetRoot() const { - return StorageDir; + const TFsPath& GetRoot() const { + return StorageDir; } const TFsPath& GetTemp() const { @@ -199,13 +199,13 @@ public: } TFileLinkPtr Put(const TString& storageFileName, const TString& outFileName, const TString& md5, const TStorage::TDataPuller& puller) { - bool newFileAdded = false; + bool newFileAdded = false; TFileLinkPtr result = HardlinkFromStorage(storageFileName, md5, outFileName); if (!result) { TFsPath storageFile = StorageDir / storageFileName; TFsPath hardlinkFile = ProcessTempDir / (outFileName ? outFileName : GetTempName()); Y_ENSURE(!hardlinkFile.Exists(), "FileStorage: temporary file " << hardlinkFile.GetPath().Quote() << " already exists"); - + ui64 fileSize = 0; TString pullerMd5; // overrides input arg 'md5' try { @@ -217,7 +217,7 @@ public: } Y_ENSURE(hardlinkFile.Exists(), "FileStorage: cannot put not existing temporary path"); Y_ENSURE(hardlinkFile.IsFile(), "FileStorage: cannot put non-file temporary path"); - + SetCacheFilePermissionsNoThrow(hardlinkFile); if (NFs::HardLink(hardlinkFile, storageFile)) { @@ -226,18 +226,18 @@ public: } // Ignore HardLink fail. Another process managed to download before us TouchFile(storageFile.c_str()); - + newFileAdded = true; result = MakeIntrusive<TFileLink>(hardlinkFile, storageFileName, fileSize, pullerMd5); - } - + } + YQL_LOG(INFO) << "Using " << (newFileAdded ? "new" : "existing") << " storage file " << result->GetStorageFileName().Quote() << ", temp path: " << result->GetPath().GetPath().Quote() << ", size: " << result->GetSize(); if (newFileAdded) { - Cleanup(); - } + Cleanup(); + } return result; } @@ -306,12 +306,12 @@ public: return result; } - ui64 GetOccupiedSize() const { - return AtomicGet(CurrentSize); + ui64 GetOccupiedSize() const { + return AtomicGet(CurrentSize); } - size_t GetCount() const { - return AtomicGet(CurrentFiles); + size_t GetCount() const { + return AtomicGet(CurrentFiles); } TString GetTempName() { @@ -320,25 +320,25 @@ public: } } -private: - void LoadStats() { +private: + void LoadStats() { TVector<TString> names; - StorageDir.ListNames(names); - - ui64 actualFiles = 0; - ui64 actualSize = 0; - - ui32 oldPid; - + StorageDir.ListNames(names); + + ui64 actualFiles = 0; + ui64 actualSize = 0; + + ui32 oldPid; + for (const TString& name: names) { - TFsPath childPath(StorageDir / name); - TFileStat stat(childPath, true); - if (stat.IsFile()) { + TFsPath childPath(StorageDir / name); + TFileStat stat(childPath, true); + if (stat.IsFile()) { ++actualFiles; - actualSize += stat.Size; - } else if (stat.IsDir() && TryFromString(name, oldPid)) { + actualSize += stat.Size; + } else if (stat.IsDir() && TryFromString(name, oldPid)) { if (!IsProcessAlive(oldPid)) { - // cleanup of previously not cleaned hardlinks directory + // cleanup of previously not cleaned hardlinks directory try { #ifdef _linux_ TFileLock childLock(childPath / ".lockfile"); @@ -356,73 +356,73 @@ private: YQL_LOG(WARN) << "Error cleaning dead process dir " << childPath << ": " << CurrentExceptionMessage(); } - } - } - } - - CurrentFiles = actualFiles; - CurrentSize = actualSize; + } + } + } + + CurrentFiles = actualFiles; + CurrentSize = actualSize; } - bool NeedToCleanup() { - return static_cast<ui64>(AtomicGet(CurrentFiles)) > MaxFiles || - static_cast<ui64>(AtomicGet(CurrentSize)) > MaxSize; + bool NeedToCleanup() { + return static_cast<ui64>(AtomicGet(CurrentFiles)) > MaxFiles || + static_cast<ui64>(AtomicGet(CurrentSize)) > MaxSize; } - void Cleanup() { + void Cleanup() { if (!NeedToCleanup()) { return; } - with_lock (CleanupLock) { + with_lock (CleanupLock) { TVector<TString> names; - StorageDir.ListNames(names); - + StorageDir.ListNames(names); + TVector<TFileObject> files; - files.reserve(names.size()); - - ui64 actualFiles = 0; - ui64 actualSize = 0; - + files.reserve(names.size()); + + ui64 actualFiles = 0; + ui64 actualSize = 0; + for (const TString& name: names) { - TFsPath childPath(StorageDir / name); - TFileStat stat(childPath, true); - if (stat.IsFile()) { - files.push_back(TFileObject{name, stat.MTime, stat.Size}); + TFsPath childPath(StorageDir / name); + TFileStat stat(childPath, true); + if (stat.IsFile()) { + files.push_back(TFileObject{name, stat.MTime, stat.Size}); ++actualFiles; - actualSize += stat.Size; + actualSize += stat.Size; } } - - // sort files to get older files first - Sort(files, [](const TFileObject& f1, const TFileObject& f2) { - if (f1.MTime == f2.MTime) { + + // sort files to get older files first + Sort(files, [](const TFileObject& f1, const TFileObject& f2) { + if (f1.MTime == f2.MTime) { return f1.Name.compare(f2.Name) < 0; - } - return f1.MTime < f2.MTime; - }); - - ui64 filesThreshold = MaxFiles / 2; - ui64 sizeThreshold = MaxSize / 2; - - for (const TFileObject& f: files) { - if (actualFiles <= filesThreshold && actualSize <= sizeThreshold) { - break; - } - - YQL_LOG(INFO) << "Removing file from cache (name: " << f.Name - << ", size: " << f.Size - << ", mtime: " << f.MTime << ")"; + } + return f1.MTime < f2.MTime; + }); + + ui64 filesThreshold = MaxFiles / 2; + ui64 sizeThreshold = MaxSize / 2; + + for (const TFileObject& f: files) { + if (actualFiles <= filesThreshold && actualSize <= sizeThreshold) { + break; + } + + YQL_LOG(INFO) << "Removing file from cache (name: " << f.Name + << ", size: " << f.Size + << ", mtime: " << f.MTime << ")"; if (!NFs::Remove(StorageDir / f.Name)) { YQL_LOG(WARN) << "Failed to remove file " << f.Name.Quote() << ": " << LastSystemErrorText(); } else { --actualFiles; actualSize -= f.Size; } - } - - AtomicSet(CurrentFiles, actualFiles); - AtomicSet(CurrentSize, actualSize); + } + + AtomicSet(CurrentFiles, actualFiles); + AtomicSet(CurrentSize, actualSize); } } @@ -433,33 +433,33 @@ private: } private: - TMutex CleanupLock; - const TFsPath StorageDir; - const TFsPath ProcessTempDir; + TMutex CleanupLock; + const TFsPath StorageDir; + const TFsPath ProcessTempDir; THolder<TFileLock> ProcessTempDirLock; - const bool IsTemp; - const ui64 MaxFiles; - const ui64 MaxSize; - TAtomic CurrentFiles = 0; - TAtomic CurrentSize = 0; + const bool IsTemp; + const ui64 MaxFiles; + const ui64 MaxSize; + TAtomic CurrentFiles = 0; + TAtomic CurrentSize = 0; TMutex RndLock; TRandGuid Rnd; }; TStorage::TStorage(size_t maxFiles, ui64 maxSize, const TString& storagePath) - : Impl(new TImpl(maxFiles, maxSize, storagePath)) + : Impl(new TImpl(maxFiles, maxSize, storagePath)) { } TStorage::~TStorage() -{ -} - +{ +} + TFsPath TStorage::GetRoot() const { return Impl->GetRoot(); } - + TFsPath TStorage::GetTemp() const { return Impl->GetTemp(); @@ -469,7 +469,7 @@ TFileLinkPtr TStorage::Put(const TString& storageFileName, const TString& outFil { return Impl->Put(storageFileName, outFileName, md5, puller); } - + TFileLinkPtr TStorage::HardlinkFromStorage(const TString& existingStorageFileName, const TString& storageFileMd5, const TString& outFileName) { return Impl->HardlinkFromStorage(existingStorageFileName, storageFileMd5, outFileName); diff --git a/ydb/library/yql/core/file_storage/storage.h b/ydb/library/yql/core/file_storage/storage.h index 46c6cec711a..8ab42a69a11 100644 --- a/ydb/library/yql/core/file_storage/storage.h +++ b/ydb/library/yql/core/file_storage/storage.h @@ -9,45 +9,45 @@ namespace NYql { -class TFileLink: public TThrRefBase { -public: +class TFileLink: public TThrRefBase { +public: TFileLink(const TFsPath& path, const TString& storageFileName, ui64 size, const TString& md5, bool deleteOnDestroy = true); ~TFileLink(); - + const TString& GetStorageFileName() const { return StorageFileName; } - const TFsPath& GetPath() const { return Path; } - ui64 GetSize() const { return Size; } + const TFsPath& GetPath() const { return Path; } + ui64 GetSize() const { return Size; } const TString& GetMd5() const { return Md5; } - -private: + +private: const TFsPath Path; const TString StorageFileName; const ui64 Size; const TString Md5; const bool DeleteOnDestroy; -}; - -using TFileLinkPtr = TIntrusivePtr<TFileLink>; - +}; + +using TFileLinkPtr = TIntrusivePtr<TFileLink>; + TFileLinkPtr CreateFakeFileLink(const TFsPath& path, const TString& md5 = "", bool deleteOnDestroy = false); /* File cache with limits by count and by occupied size. - The same file can be added multiple times. - It is safe to add the same file from multiple threads/processes simultaneously. - Only first thread will do the actual work. Other threads will wait and reuse the result. + The same file can be added multiple times. + It is safe to add the same file from multiple threads/processes simultaneously. + Only first thread will do the actual work. Other threads will wait and reuse the result. */ class TStorage { public: // This function is called by the storage to transfer user data to the provided temporary file path // Returns content size and md5 using TDataPuller = std::function<std::pair<ui64, TString> (const TFsPath& dstPath)>; - + /* Constructs the storage with the specified limits. storagePath can be empty - a temporary directory will be used in this case. */ TStorage(size_t maxFiles, ui64 maxSize, const TString& storagePath = {}); ~TStorage(); - + // Returns root storage directory TFsPath GetRoot() const; // Returns temp storage directory @@ -68,7 +68,7 @@ public: TString GetTempName(); private: - class TImpl; + class TImpl; THolder<TImpl> Impl; }; diff --git a/ydb/library/yql/core/file_storage/storage_ut.cpp b/ydb/library/yql/core/file_storage/storage_ut.cpp index 7c7fc72c89d..4229abd2199 100644 --- a/ydb/library/yql/core/file_storage/storage_ut.cpp +++ b/ydb/library/yql/core/file_storage/storage_ut.cpp @@ -3,12 +3,12 @@ #include <library/cpp/testing/unittest/registar.h> #include <library/cpp/threading/future/async.h> -#include <util/datetime/base.h> +#include <util/datetime/base.h> #include <util/folder/path.h> -#include <util/generic/algorithm.h> +#include <util/generic/algorithm.h> #include <util/generic/yexception.h> #include <util/stream/file.h> -#include <util/stream/input.h> +#include <util/stream/input.h> #include <util/stream/str.h> using namespace NYql; @@ -59,16 +59,16 @@ Y_UNIT_TEST_SUITE(TStorageTests) { Y_UNIT_TEST(Put) { THolder<TStorage> storage = MakeHolder<TStorage>(10, 100); TFileLinkPtr fileInStorage = PutFile("test.file", *storage, "somename"); - TFsPath rootPath(fileInStorage->GetPath()); - UNIT_ASSERT(rootPath.Exists()); + TFsPath rootPath(fileInStorage->GetPath()); + UNIT_ASSERT(rootPath.Exists()); UNIT_ASSERT_VALUES_EQUAL(rootPath.GetName(), "somename"); - UNIT_ASSERT_VALUES_EQUAL(fileInStorage->GetStorageFileName(), "test.file"); - UNIT_ASSERT(fileInStorage->GetPath().IsSubpathOf(storage->GetRoot().Fix())); - UNIT_ASSERT_VALUES_EQUAL(TIFStream(fileInStorage->GetPath()).ReadAll(), DATA); - UNIT_ASSERT_EQUAL(storage->GetCount(), 1); - UNIT_ASSERT_EQUAL(storage->GetOccupiedSize(), DATA.size()); - fileInStorage.Reset(); - UNIT_ASSERT(false == rootPath.Exists()); + UNIT_ASSERT_VALUES_EQUAL(fileInStorage->GetStorageFileName(), "test.file"); + UNIT_ASSERT(fileInStorage->GetPath().IsSubpathOf(storage->GetRoot().Fix())); + UNIT_ASSERT_VALUES_EQUAL(TIFStream(fileInStorage->GetPath()).ReadAll(), DATA); + UNIT_ASSERT_EQUAL(storage->GetCount(), 1); + UNIT_ASSERT_EQUAL(storage->GetOccupiedSize(), DATA.size()); + fileInStorage.Reset(); + UNIT_ASSERT(false == rootPath.Exists()); } Y_UNIT_TEST(ParallelPut) { @@ -79,105 +79,105 @@ Y_UNIT_TEST_SUITE(TStorageTests) { TVector<TFuture<TFileLinkPtr>> res; res.reserve(10); for (size_t i = 0; i < 10; ++i) { - res.emplace_back(Async([&]() { - return PutFileWithSleep("test.file", *storage); - }, queue)); + res.emplace_back(Async([&]() { + return PutFileWithSleep("test.file", *storage); + }, queue)); } for (auto& f: res) { f.Wait(); - TFileLinkPtr fileInStorage = f.GetValue(); - UNIT_ASSERT(fileInStorage->GetPath().Exists()); - UNIT_ASSERT_VALUES_EQUAL(fileInStorage->GetStorageFileName(), "test.file"); - UNIT_ASSERT(fileInStorage->GetPath().IsSubpathOf(storage->GetRoot().Fix())); - UNIT_ASSERT_VALUES_EQUAL(TIFStream(fileInStorage->GetPath()).ReadAll(), DATA); - UNIT_ASSERT_EQUAL(storage->GetCount(), 1); - UNIT_ASSERT_EQUAL(storage->GetOccupiedSize(), DATA.size()); + TFileLinkPtr fileInStorage = f.GetValue(); + UNIT_ASSERT(fileInStorage->GetPath().Exists()); + UNIT_ASSERT_VALUES_EQUAL(fileInStorage->GetStorageFileName(), "test.file"); + UNIT_ASSERT(fileInStorage->GetPath().IsSubpathOf(storage->GetRoot().Fix())); + UNIT_ASSERT_VALUES_EQUAL(TIFStream(fileInStorage->GetPath()).ReadAll(), DATA); + UNIT_ASSERT_EQUAL(storage->GetCount(), 1); + UNIT_ASSERT_EQUAL(storage->GetOccupiedSize(), DATA.size()); } } Y_UNIT_TEST(CleanUp) { THolder<TStorage> storage = MakeHolder<TStorage>(10, 100); - TFileLinkPtr fileInStorage = PutFile("test.file", *storage); - UNIT_ASSERT(fileInStorage->GetPath().Exists()); + TFileLinkPtr fileInStorage = PutFile("test.file", *storage); + UNIT_ASSERT(fileInStorage->GetPath().Exists()); auto rootPath = storage->GetRoot(); storage.Destroy(); - UNIT_ASSERT(!fileInStorage->GetPath().Exists()); + UNIT_ASSERT(!fileInStorage->GetPath().Exists()); UNIT_ASSERT(!rootPath.Exists()); } Y_UNIT_TEST(DisplaceByCount) { - ui64 maxCount = 2; + ui64 maxCount = 2; THolder<TStorage> storage = MakeHolder<TStorage>(maxCount, 100); - TFileLinkPtr file1 = PutFile("test1.file", *storage); - TFileLinkPtr file2 = PutFile("test2.file", *storage); + TFileLinkPtr file1 = PutFile("test1.file", *storage); + TFileLinkPtr file2 = PutFile("test2.file", *storage); UNIT_ASSERT_VALUES_EQUAL(storage->GetCount(), 2); - UNIT_ASSERT_VALUES_EQUAL(storage->GetOccupiedSize(), 2 * DATA.size()); - UNIT_ASSERT(file1->GetPath().Exists()); - UNIT_ASSERT(file2->GetPath().Exists()); - - TFileLinkPtr file3 = PutFileWithSleep("test3.file", *storage); - - // after putting 3rd file we exceeded the limit by count - // and cleaned up half of the original maxCount files - UNIT_ASSERT_VALUES_EQUAL(storage->GetCount(), maxCount / 2); - UNIT_ASSERT_VALUES_EQUAL(storage->GetOccupiedSize(), (maxCount / 2) * DATA.size()); - - // despite of file1 and file2 are displaced from storage, we still - // have temporary hardlinks to it - UNIT_ASSERT(file1->GetPath().Exists()); - UNIT_ASSERT(file2->GetPath().Exists()); - UNIT_ASSERT(file3->GetPath().Exists()); - + UNIT_ASSERT_VALUES_EQUAL(storage->GetOccupiedSize(), 2 * DATA.size()); + UNIT_ASSERT(file1->GetPath().Exists()); + UNIT_ASSERT(file2->GetPath().Exists()); + + TFileLinkPtr file3 = PutFileWithSleep("test3.file", *storage); + + // after putting 3rd file we exceeded the limit by count + // and cleaned up half of the original maxCount files + UNIT_ASSERT_VALUES_EQUAL(storage->GetCount(), maxCount / 2); + UNIT_ASSERT_VALUES_EQUAL(storage->GetOccupiedSize(), (maxCount / 2) * DATA.size()); + + // despite of file1 and file2 are displaced from storage, we still + // have temporary hardlinks to it + UNIT_ASSERT(file1->GetPath().Exists()); + UNIT_ASSERT(file2->GetPath().Exists()); + UNIT_ASSERT(file3->GetPath().Exists()); + TVector<TString> filesInStorage; - storage->GetRoot().ListNames(filesInStorage); - UNIT_ASSERT_EQUAL(filesInStorage.size(), 2); // 1 file + 1 hardlink directory - - auto beg = filesInStorage.begin(), - end = filesInStorage.end(); - - // file1 and file2 were displaced - UNIT_ASSERT(Find(beg, end, file1->GetStorageFileName()) == end); - UNIT_ASSERT(Find(beg, end, file2->GetStorageFileName()) == end); - UNIT_ASSERT(Find(beg, end, file3->GetStorageFileName()) != end); + storage->GetRoot().ListNames(filesInStorage); + UNIT_ASSERT_EQUAL(filesInStorage.size(), 2); // 1 file + 1 hardlink directory + + auto beg = filesInStorage.begin(), + end = filesInStorage.end(); + + // file1 and file2 were displaced + UNIT_ASSERT(Find(beg, end, file1->GetStorageFileName()) == end); + UNIT_ASSERT(Find(beg, end, file2->GetStorageFileName()) == end); + UNIT_ASSERT(Find(beg, end, file3->GetStorageFileName()) != end); } Y_UNIT_TEST(DisplaceBySize) { - ui64 maxSize = 25; + ui64 maxSize = 25; THolder<TStorage> storage = MakeHolder<TStorage>(10, maxSize); - TFileLinkPtr file1 = PutFile("test1.file", *storage); - TFileLinkPtr file2 = PutFile("test2.file", *storage); + TFileLinkPtr file1 = PutFile("test1.file", *storage); + TFileLinkPtr file2 = PutFile("test2.file", *storage); UNIT_ASSERT_VALUES_EQUAL(storage->GetCount(), 2); - UNIT_ASSERT_VALUES_EQUAL(storage->GetOccupiedSize(), 2 * DATA.size()); - UNIT_ASSERT(file1->GetPath().Exists()); - UNIT_ASSERT(file2->GetPath().Exists()); + UNIT_ASSERT_VALUES_EQUAL(storage->GetOccupiedSize(), 2 * DATA.size()); + UNIT_ASSERT(file1->GetPath().Exists()); + UNIT_ASSERT(file2->GetPath().Exists()); - TFileLinkPtr file3 = PutFileWithSleep("test3.file", *storage); + TFileLinkPtr file3 = PutFileWithSleep("test3.file", *storage); - // after putting 3rd file we exceeded the limit by size - // and cleaned up files till we get half of the original maxSize - UNIT_ASSERT_VALUES_EQUAL(storage->GetCount(), 1); - UNIT_ASSERT_VALUES_EQUAL(storage->GetOccupiedSize(), DATA.size()); + // after putting 3rd file we exceeded the limit by size + // and cleaned up files till we get half of the original maxSize + UNIT_ASSERT_VALUES_EQUAL(storage->GetCount(), 1); + UNIT_ASSERT_VALUES_EQUAL(storage->GetOccupiedSize(), DATA.size()); - // despite of file1 and file2 are displaced from storage, we still - // have temporary hardlinks to it - UNIT_ASSERT(file1->GetPath().Exists()); - UNIT_ASSERT(file2->GetPath().Exists()); - UNIT_ASSERT(file3->GetPath().Exists()); + // despite of file1 and file2 are displaced from storage, we still + // have temporary hardlinks to it + UNIT_ASSERT(file1->GetPath().Exists()); + UNIT_ASSERT(file2->GetPath().Exists()); + UNIT_ASSERT(file3->GetPath().Exists()); TVector<TString> filesInStorage; - storage->GetRoot().ListNames(filesInStorage); - UNIT_ASSERT_EQUAL(filesInStorage.size(), 2); // 1 file + 1 hardlink directory + storage->GetRoot().ListNames(filesInStorage); + UNIT_ASSERT_EQUAL(filesInStorage.size(), 2); // 1 file + 1 hardlink directory - auto beg = filesInStorage.begin(), - end = filesInStorage.end(); + auto beg = filesInStorage.begin(), + end = filesInStorage.end(); - // file1 and file2 were displaced - UNIT_ASSERT(Find(beg, end, file1->GetStorageFileName()) == end); - UNIT_ASSERT(Find(beg, end, file2->GetStorageFileName()) == end); - UNIT_ASSERT(Find(beg, end, file3->GetStorageFileName()) != end); + // file1 and file2 were displaced + UNIT_ASSERT(Find(beg, end, file1->GetStorageFileName()) == end); + UNIT_ASSERT(Find(beg, end, file2->GetStorageFileName()) == end); + UNIT_ASSERT(Find(beg, end, file3->GetStorageFileName()) != end); } Y_UNIT_TEST(PersistStorage) { @@ -185,17 +185,17 @@ Y_UNIT_TEST_SUITE(TStorageTests) { THolder<TStorage> storage = MakeHolder<TStorage>(100, 100, dir.GetFsPath()); auto rootPath = storage->GetRoot(); - TFileLinkPtr fileInStorage = PutFile("test.file", *storage); - UNIT_ASSERT_EQUAL(storage->GetCount(), 1); - UNIT_ASSERT_EQUAL(storage->GetOccupiedSize(), DATA.size()); - + TFileLinkPtr fileInStorage = PutFile("test.file", *storage); + UNIT_ASSERT_EQUAL(storage->GetCount(), 1); + UNIT_ASSERT_EQUAL(storage->GetOccupiedSize(), DATA.size()); + storage.Destroy(); - UNIT_ASSERT(!fileInStorage->GetPath().Exists()); // hardlink was deleted + UNIT_ASSERT(!fileInStorage->GetPath().Exists()); // hardlink was deleted UNIT_ASSERT(rootPath.Exists()); - UNIT_ASSERT((rootPath / fileInStorage->GetStorageFileName()).Exists()); + UNIT_ASSERT((rootPath / fileInStorage->GetStorageFileName()).Exists()); storage.Reset(new TStorage(100, 100, dir.GetFsPath())); - UNIT_ASSERT_EQUAL(storage->GetCount(), 1); - UNIT_ASSERT_EQUAL(storage->GetOccupiedSize(), DATA.size()); + UNIT_ASSERT_EQUAL(storage->GetCount(), 1); + UNIT_ASSERT_EQUAL(storage->GetOccupiedSize(), DATA.size()); } } diff --git a/ydb/library/yql/core/type_ann/ya.make b/ydb/library/yql/core/type_ann/ya.make index fbcfcfbe690..be843b5abc1 100644 --- a/ydb/library/yql/core/type_ann/ya.make +++ b/ydb/library/yql/core/type_ann/ya.make @@ -1,4 +1,4 @@ -LIBRARY() +LIBRARY() OWNER( vvvv diff --git a/ydb/library/yql/core/ya.make b/ydb/library/yql/core/ya.make index 0dd2b7fc62d..29169996f59 100644 --- a/ydb/library/yql/core/ya.make +++ b/ydb/library/yql/core/ya.make @@ -10,8 +10,8 @@ SRCS( yql_atom_enums.h yql_callable_transform.cpp yql_callable_transform.h - yql_csv.cpp - yql_csv.h + yql_csv.cpp + yql_csv.h yql_data_provider.h yql_execution.cpp yql_execution.h diff --git a/ydb/library/yql/core/yql_csv.cpp b/ydb/library/yql/core/yql_csv.cpp index f15d509d7b9..259b6144f1b 100644 --- a/ydb/library/yql/core/yql_csv.cpp +++ b/ydb/library/yql/core/yql_csv.cpp @@ -1,120 +1,120 @@ #include "yql_csv.h" - + #include <util/string/split.h> -#include <util/string/escape.h> - -namespace { - -const char QUOTE_CH = '"'; -const char ESCAPE_CH = '\\'; +#include <util/string/escape.h> + +namespace { + +const char QUOTE_CH = '"'; +const char ESCAPE_CH = '\\'; const char NULL_CH = '\0'; - -class TCsvLineParser { -public: + +class TCsvLineParser { +public: TCsvLineParser(const TStringBuf& line, char delimiter) - : Cur_(line.begin()) - , End_(line.end()) + : Cur_(line.begin()) + , End_(line.end()) , Delim_(delimiter) , Prev_(NULL_CH) /* must differ from ESCAPE_CH */ - { - } - - bool Next(TStringBuf& token) { - if (Cur_ > End_) return false; - - const char* tokenStart = Cur_; + { + } + + bool Next(TStringBuf& token) { + if (Cur_ > End_) return false; + + const char* tokenStart = Cur_; while (Cur_ < End_ && (Prev_ == ESCAPE_CH || *Cur_ != QUOTE_CH) && *Cur_ != Delim_) Prev_ = *Cur_++; - + if (Cur_ == End_) { - token = { tokenStart, Cur_ }; + token = { tokenStart, Cur_ }; Prev_ = NULL_CH; - Cur_++; - return true; + Cur_++; + return true; } if (Prev_ != ESCAPE_CH && *Cur_ == QUOTE_CH) { Prev_ = *Cur_++; tokenStart = Cur_; - while (Cur_ < End_) { - // find non escaped quote char + while (Cur_ < End_) { + // find non escaped quote char if (*Cur_ == QUOTE_CH && Prev_ != ESCAPE_CH) break; Prev_ = *Cur_++; - } - - if (Cur_ == End_) - ythrow yexception() << "expected closing \""; - - token = { tokenStart, Cur_ }; + } + + if (Cur_ == End_) + ythrow yexception() << "expected closing \""; + + token = { tokenStart, Cur_ }; Prev_ = *Cur_++; // skip closing quote char - + if (Cur_ != End_ && *Cur_ != Delim_) - ythrow yexception() << "expected end of line or delimiter"; - + ythrow yexception() << "expected end of line or delimiter"; + Prev_ = *Cur_++; // move out of buffer or skip delimiter - return true; + return true; } else if (*Cur_ == Delim_) { - token = { tokenStart, Cur_ }; + token = { tokenStart, Cur_ }; Prev_ = *Cur_++; - return true; - } - - return false; - } - -private: - const char* Cur_; - const char* End_; + return true; + } + + return false; + } + +private: + const char* Cur_; + const char* End_; const char Delim_; char Prev_; -}; - -} // namspace - -namespace NYql { -namespace NUtils { - -/////////////////////////////////////////////////////////////////////////////// -// TCsvInputStream -/////////////////////////////////////////////////////////////////////////////// +}; + +} // namspace + +namespace NYql { +namespace NUtils { + +/////////////////////////////////////////////////////////////////////////////// +// TCsvInputStream +/////////////////////////////////////////////////////////////////////////////// TCsvInputStream::TCsvInputStream(IInputStream& slave, char delimiter) - : Slave_(slave) + : Slave_(slave) , Delim_(delimiter) -{ -} - +{ +} + TVector<TString> TCsvInputStream::ReadLine() -{ +{ TVector<TString> parts; TString line; - - if (Slave_.ReadLine(line)) { + + if (Slave_.ReadLine(line)) { TCsvLineParser lineParser(line, Delim_); TStringBuf token; while (lineParser.Next(token)) { parts.push_back(UnescapeC(token.data(), token.size())); - } - } - - return parts; -} - + } + } + + return parts; +} + TVector<TString> TCsvInputStream::ReadLineWithEscaping() -{ +{ TVector<TString> parts; TString line; - - if (Slave_.ReadLine(line)) { + + if (Slave_.ReadLine(line)) { TCsvLineParser lineParser(line, Delim_); - TStringBuf token; - while (lineParser.Next(token)) { + TStringBuf token; + while (lineParser.Next(token)) { parts.push_back(TString(token)); - } - } - - return parts; -} - -/////////////////////////////////////////////////////////////////////////////// + } + } + + return parts; +} + +/////////////////////////////////////////////////////////////////////////////// // TCsvInputStream /////////////////////////////////////////////////////////////////////////////// TCsvInputBuffer::TCsvInputBuffer(const TStringBuf& buffer, char delimiter) @@ -157,38 +157,38 @@ TVector<TString> TCsvInputBuffer::ReadLineWithEscaping() /////////////////////////////////////////////////////////////////////////////// -// TCsvOutputStream -/////////////////////////////////////////////////////////////////////////////// +// TCsvOutputStream +/////////////////////////////////////////////////////////////////////////////// TCsvOutputStream::TCsvOutputStream(IOutputStream& slave, char delimiter, bool quoteItems) - : Slave_(slave) - , WasNL_(true) + : Slave_(slave) + , WasNL_(true) , Delim_(delimiter) , QuoteItems_(quoteItems) -{ -} - -void TCsvOutputStream::DoWrite(const void* buf, size_t len) -{ - TStringBuf charBuf(reinterpret_cast<const char*>(buf), len); +{ +} + +void TCsvOutputStream::DoWrite(const void* buf, size_t len) +{ + TStringBuf charBuf(reinterpret_cast<const char*>(buf), len); if (charBuf == TStringBuf("\n")) { - WasNL_ = true; - Slave_.Write(buf, len); - } else { + WasNL_ = true; + Slave_.Write(buf, len); + } else { if (!WasNL_) Slave_.Write(Delim_); - WasNL_ = false; + WasNL_ = false; if (QuoteItems_) { Slave_.Write(QUOTE_CH); } - Slave_.Write(EscapeC(charBuf)); + Slave_.Write(EscapeC(charBuf)); if (QuoteItems_) { Slave_.Write(QUOTE_CH); } - } -} - + } +} + void TCsvOutputStream::DoFlush() { Slave_.Flush(); } -} // namspace NUtils -} // namspace NYql +} // namspace NUtils +} // namspace NYql diff --git a/ydb/library/yql/core/yql_csv.h b/ydb/library/yql/core/yql_csv.h index b68533c33ee..6e3e48b816f 100644 --- a/ydb/library/yql/core/yql_csv.h +++ b/ydb/library/yql/core/yql_csv.h @@ -1,29 +1,29 @@ -#pragma once - -#include <util/stream/input.h> -#include <util/stream/output.h> -#include <util/generic/vector.h> - -namespace NYql { -namespace NUtils { - -/////////////////////////////////////////////////////////////////////////////// -// TCsvInputStream -/////////////////////////////////////////////////////////////////////////////// -class TCsvInputStream { -public: +#pragma once + +#include <util/stream/input.h> +#include <util/stream/output.h> +#include <util/generic/vector.h> + +namespace NYql { +namespace NUtils { + +/////////////////////////////////////////////////////////////////////////////// +// TCsvInputStream +/////////////////////////////////////////////////////////////////////////////// +class TCsvInputStream { +public: TCsvInputStream(IInputStream& slave, char delimiter = ';'); - + TVector<TString> ReadLine(); TVector<TString> ReadLineWithEscaping(); - -private: + +private: IInputStream& Slave_; const char Delim_; -}; - +}; -/////////////////////////////////////////////////////////////////////////////// + +/////////////////////////////////////////////////////////////////////////////// // TCsvInputBuffer /////////////////////////////////////////////////////////////////////////////// class TCsvInputBuffer { @@ -40,24 +40,24 @@ private: /////////////////////////////////////////////////////////////////////////////// -// TCsvOutputStream -/////////////////////////////////////////////////////////////////////////////// +// TCsvOutputStream +/////////////////////////////////////////////////////////////////////////////// class TCsvOutputStream: public IOutputStream { -public: +public: TCsvOutputStream(IOutputStream& slave, char delimiter = ';', bool quoteItems = true); - - // hack for output empty values + + // hack for output empty values inline TCsvOutputStream& operator<<(const TString& value) { DoWrite(value.data(), value.size()); - return *this; - } - - // hack for output empty values - inline TCsvOutputStream& operator<<(const char* value) { - DoWrite(value, strlen(value)); - return *this; - } - + return *this; + } + + // hack for output empty values + inline TCsvOutputStream& operator<<(const char* value) { + DoWrite(value, strlen(value)); + return *this; + } + inline TCsvOutputStream& operator<<(const TVector<TString>& values) { for(const TString& value: values) { (*this) << value; @@ -66,16 +66,16 @@ public: return *this; } -private: - void DoWrite(const void* buf, size_t len) override final; +private: + void DoWrite(const void* buf, size_t len) override final; void DoFlush() override; - -private: + +private: IOutputStream& Slave_; - bool WasNL_; + bool WasNL_; const char Delim_; const bool QuoteItems_; -}; - -} // namspace NUtils -} // namspace NYql +}; + +} // namspace NUtils +} // namspace NYql diff --git a/ydb/library/yql/core/yql_execution.cpp b/ydb/library/yql/core/yql_execution.cpp index 654af1db651..eb138b1b4f3 100644 --- a/ydb/library/yql/core/yql_execution.cpp +++ b/ydb/library/yql/core/yql_execution.cpp @@ -1,15 +1,15 @@ #include "yql_execution.h" #include "yql_expr_optimize.h" #include "yql_opt_proposed_by_data.h" - + #include <ydb/library/yql/utils/log/log.h> #include <ydb/library/yql/utils/yql_panic.h> - + #include <util/string/builder.h> #include <util/string/join.h> #include <util/generic/queue.h> - + namespace NYql { namespace { @@ -48,7 +48,7 @@ public: if (FinalizingTransformer) { YQL_CLOG(INFO, CoreExecution) << "FinalizingTransformer, root #" << input->UniqueId(); auto status = FinalizingTransformer->Transform(input, output, ctx); - YQL_CLOG(INFO, CoreExecution) << "FinalizingTransformer done, output #" << output->UniqueId() << ", status: " << status; + YQL_CLOG(INFO, CoreExecution) << "FinalizingTransformer done, output #" << output->UniqueId() << ", status: " << status; return status; } @@ -124,7 +124,7 @@ public: continue; } - YQL_CLOG(INFO, CoreExecution) << "Completed async execution for node #" << item.Node->UniqueId(); + YQL_CLOG(INFO, CoreExecution) << "Completed async execution for node #" << item.Node->UniqueId(); TExprNode::TPtr callableOutput; auto status = item.DataProvider->GetCallableExecutionTransformer().ApplyAsyncChanges(item.Node, callableOutput, ctx); Y_VERIFY(callableOutput); @@ -148,7 +148,7 @@ public: if (item.Node->GetState() == TExprNode::EState::ExecutionComplete || item.Node->GetState() == TExprNode::EState::Error) - { + { FinishNode(item.DataProvider->GetName(), *item.Node, *callableOutput); } } @@ -393,7 +393,7 @@ public: } else { if (output->GetState() == TExprNode::EState::ExecutionComplete || output->GetState() == TExprNode::EState::Error) - { + { StartNode(category, *node); FinishNode(category, *node, *output); } @@ -441,7 +441,7 @@ public: } else { if (output->GetState() == TExprNode::EState::ExecutionComplete || output->GetState() == TExprNode::EState::Error) - { + { StartNode(dataProvider->GetName(), *node); FinishNode(dataProvider->GetName(), *node, *output); } @@ -451,8 +451,8 @@ public: } void AddCallable(const TExprNode::TPtr& node, IDataProvider* dataProvider, TExprContext& ctx) { - Y_UNUSED(ctx); - YQL_CLOG(INFO, CoreExecution) << "Register async execution for node #" << node->UniqueId(); + Y_UNUSED(ctx); + YQL_CLOG(INFO, CoreExecution) << "Register async execution for node #" << node->UniqueId(); auto future = dataProvider->GetCallableExecutionTransformer().GetAsyncFuture(*node); SubscribeAsyncFuture(node, dataProvider, future); } @@ -499,14 +499,14 @@ public: if (newNode.UniqueId() != node.UniqueId()) { Types.NodeToOperationId[newNode.UniqueId()] = *publicId; } - + auto progIt = Progresses.find(*publicId); YQL_ENSURE(progIt != Progresses.end()); auto newState = (node.GetState() == TExprNode::EState::ExecutionComplete) - ? TOperationProgress::EState::Finished - : TOperationProgress::EState::Failed; - + ? TOperationProgress::EState::Finished + : TOperationProgress::EState::Failed; + if (progIt->second.State != newState) { TString stage = progIt->second.Stage.first; progIt->second = TOperationProgress(TString(category), *publicId, newState, stage); diff --git a/ydb/library/yql/core/yql_execution.h b/ydb/library/yql/core/yql_execution.h index cada494a511..cb38d913fa6 100644 --- a/ydb/library/yql/core/yql_execution.h +++ b/ydb/library/yql/core/yql_execution.h @@ -12,7 +12,7 @@ namespace NYql { xx(InProgress, 1) \ xx(Finished, 2) \ xx(Failed, 3) \ - xx(Aborted, 4) + xx(Aborted, 4) enum class EState { YQL_OPERATION_PROGRESS_STATE_MAP(ENUM_VALUE_GEN) diff --git a/ydb/library/yql/core/yql_type_annotation.h b/ydb/library/yql/core/yql_type_annotation.h index a6f647a96f9..1a1f1fb4766 100644 --- a/ydb/library/yql/core/yql_type_annotation.h +++ b/ydb/library/yql/core/yql_type_annotation.h @@ -136,7 +136,7 @@ bool SplitUdfName(TStringBuf name, TStringBuf& moduleName, TStringBuf& funcName) using TUdfModulesTable = THashMap<TString, TString>; // external module name -> alias of file -struct TYqlOperationOptions { +struct TYqlOperationOptions { TString Runner; TMaybe<TString> AuthenticatedUser; TMaybe<TString> Id; @@ -146,8 +146,8 @@ struct TYqlOperationOptions { TMaybe<TString> Url; TMaybe<TString> AttrsYson; TMaybe<NYT::TNode> ParametersYson; -}; - +}; + using TColumnOrder = TVector<TString>; TString FormatColumnOrder(const TMaybe<TColumnOrder>& columnOrder); ui64 AddColumnOrderHash(const TMaybe<TColumnOrder>& columnOrder, ui64 hash); @@ -189,7 +189,7 @@ struct TTypeAnnotationContext: public TThrRefBase { TString FullResultDataSink; TUserDataStorage::TPtr UserDataStorage; TUserDataTable UserDataStorageCrutches; - TYqlOperationOptions OperationOptions; + TYqlOperationOptions OperationOptions; TVector<TCredentialTablePtr> Credentials; TUserCredentials UserCredentials; IModuleResolver::TPtr Modules; diff --git a/ydb/library/yql/dq/actors/compute/dq_compute_actor_channels.cpp b/ydb/library/yql/dq/actors/compute/dq_compute_actor_channels.cpp index 3541e44e80d..1a1611ec81e 100644 --- a/ydb/library/yql/dq/actors/compute/dq_compute_actor_channels.cpp +++ b/ydb/library/yql/dq/actors/compute/dq_compute_actor_channels.cpp @@ -61,7 +61,7 @@ TDqComputeActorChannels::TDqComputeActorChannels(TActorId owner, const TTxId& tx outputChannel.PeerState.PeerFreeSpace = channelBufferSize; if (channel.GetDstEndpoint().HasActorId()) { - outputChannel.Peer = ActorIdFromProto(channel.GetDstEndpoint().GetActorId()); + outputChannel.Peer = ActorIdFromProto(channel.GetDstEndpoint().GetActorId()); } if (Y_UNLIKELY(statsMode >= NDqProto::DQ_STATS_MODE_PROFILE)) { diff --git a/ydb/library/yql/dq/actors/protos/dq_events.proto b/ydb/library/yql/dq/actors/protos/dq_events.proto index f0d26520418..0e38ab3bbc1 100644 --- a/ydb/library/yql/dq/actors/protos/dq_events.proto +++ b/ydb/library/yql/dq/actors/protos/dq_events.proto @@ -43,7 +43,7 @@ enum EComputeState { }; message TComputeActorStats { - optional NActorsProto.TActorId ActorId = 1; + optional NActorsProto.TActorId ActorId = 1; reserved 2; //optional NKqpProto.TKqpStatsRun LegacyStats = 2; }; diff --git a/ydb/library/yql/dq/proto/dq_tasks.proto b/ydb/library/yql/dq/proto/dq_tasks.proto index 1df1e033556..e29e64ef2c5 100644 --- a/ydb/library/yql/dq/proto/dq_tasks.proto +++ b/ydb/library/yql/dq/proto/dq_tasks.proto @@ -40,7 +40,7 @@ message TDqUri { message TEndpoint { oneof EndpointType { - NActorsProto.TActorId ActorId = 1; + NActorsProto.TActorId ActorId = 1; fixed64 TabletId = 2; TDqUri Uri = 3; } diff --git a/ydb/library/yql/dq/tasks/dq_tasks_graph.h b/ydb/library/yql/dq/tasks/dq_tasks_graph.h index fcdafc56c73..e5c772310bf 100644 --- a/ydb/library/yql/dq/tasks/dq_tasks_graph.h +++ b/ydb/library/yql/dq/tasks/dq_tasks_graph.h @@ -173,7 +173,7 @@ struct TTask { TStageId StageId; TVector<TInputType> Inputs; TVector<TOutputType> Outputs; - NActors::TActorId ComputeActorId; + NActors::TActorId ComputeActorId; TTaskMeta Meta; NDqProto::ECheckpointingMode CheckpointingMode = NDqProto::CHECKPOINTING_MODE_DEFAULT; TTransform OutputTransform; diff --git a/ydb/library/yql/minikql/aligned_page_pool.h b/ydb/library/yql/minikql/aligned_page_pool.h index f6915b5e669..2ef28786554 100644 --- a/ydb/library/yql/minikql/aligned_page_pool.h +++ b/ydb/library/yql/minikql/aligned_page_pool.h @@ -5,7 +5,7 @@ #include <util/system/yassert.h> #include <util/system/defaults.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <type_traits> #include <stack> diff --git a/ydb/library/yql/minikql/comp_nodes/mkql_apply.cpp b/ydb/library/yql/minikql/comp_nodes/mkql_apply.cpp index 7aa8931414d..a613c9dcd92 100644 --- a/ydb/library/yql/minikql/comp_nodes/mkql_apply.cpp +++ b/ydb/library/yql/minikql/comp_nodes/mkql_apply.cpp @@ -1,5 +1,5 @@ #include "mkql_apply.h" - + #include <ydb/library/yql/minikql/computation/mkql_computation_node_codegen.h> #include <ydb/library/yql/minikql/computation/mkql_computation_node_holders.h> #include <ydb/library/yql/minikql/mkql_node_cast.h> diff --git a/ydb/library/yql/minikql/comp_nodes/mkql_extend.cpp b/ydb/library/yql/minikql/comp_nodes/mkql_extend.cpp index e49ae712e8a..36fd03b0a4c 100644 --- a/ydb/library/yql/minikql/comp_nodes/mkql_extend.cpp +++ b/ydb/library/yql/minikql/comp_nodes/mkql_extend.cpp @@ -4,8 +4,8 @@ #include <ydb/library/yql/minikql/computation/mkql_custom_list.h> #include <ydb/library/yql/minikql/mkql_node_cast.h> -#include <util/string/cast.h> - +#include <util/string/cast.h> + namespace NKikimr { namespace NMiniKQL { diff --git a/ydb/library/yql/minikql/comp_nodes/mkql_join.cpp b/ydb/library/yql/minikql/comp_nodes/mkql_join.cpp index 9d3bed8a91f..d2182bbc236 100644 --- a/ydb/library/yql/minikql/comp_nodes/mkql_join.cpp +++ b/ydb/library/yql/minikql/comp_nodes/mkql_join.cpp @@ -21,7 +21,7 @@ const ui64 DEFAULT_STACK_ITEMS = 16; static const TStatKey Join_Spill_Count("Join_Spill_Count", true); static const TStatKey Join_Spill_MaxFileSize("Join_Spill_MaxFileSize", false); static const TStatKey Join_Spill_MaxRowsCount("Join_Spill_MaxRowsCount", false); - + enum class EOutputMode { Unknown, LeftNull, @@ -1455,10 +1455,10 @@ public: Cerr << "Spill finished at " << Count << " items" << Endl; FileState->Output.reset(); Cerr << "File size: " << GetFileLength(FileState->File.GetName()) << ", expected: " << FileState->TotalSize << Endl; - - MKQL_INC_STAT(Ctx->Stats, Join_Spill_Count); - MKQL_SET_MAX_STAT(Ctx->Stats, Join_Spill_MaxFileSize, static_cast<i64>(FileState->TotalSize)); - MKQL_SET_MAX_STAT(Ctx->Stats, Join_Spill_MaxRowsCount, static_cast<i64>(Count)); + + MKQL_INC_STAT(Ctx->Stats, Join_Spill_Count); + MKQL_SET_MAX_STAT(Ctx->Stats, Join_Spill_MaxFileSize, static_cast<i64>(FileState->TotalSize)); + MKQL_SET_MAX_STAT(Ctx->Stats, Join_Spill_MaxRowsCount, static_cast<i64>(Count)); } } diff --git a/ydb/library/yql/minikql/comp_nodes/mkql_map.cpp b/ydb/library/yql/minikql/comp_nodes/mkql_map.cpp index 2c9ebdc874c..04997bd1f1d 100644 --- a/ydb/library/yql/minikql/comp_nodes/mkql_map.cpp +++ b/ydb/library/yql/minikql/comp_nodes/mkql_map.cpp @@ -127,8 +127,8 @@ protected: } bool HasFastListLength() const final { - return List.HasFastListLength(); - } + return List.HasFastListLength(); + } TComputationContext& CompCtx; const NUdf::TUnboxedValue List; diff --git a/ydb/library/yql/minikql/comp_nodes/mkql_map_join.cpp b/ydb/library/yql/minikql/comp_nodes/mkql_map_join.cpp index cdc86facaf2..613958f74b2 100644 --- a/ydb/library/yql/minikql/comp_nodes/mkql_map_join.cpp +++ b/ydb/library/yql/minikql/comp_nodes/mkql_map_join.cpp @@ -6,8 +6,8 @@ #include <ydb/library/yql/minikql/mkql_program_builder.h> #include <ydb/library/yql/minikql/invoke_builtins/mkql_builtins.h> -#include <util/string/cast.h> - +#include <util/string/cast.h> + namespace NKikimr { namespace NMiniKQL { diff --git a/ydb/library/yql/minikql/comp_nodes/mkql_multimap.cpp b/ydb/library/yql/minikql/comp_nodes/mkql_multimap.cpp index a43c633ef6c..e9e93aca672 100644 --- a/ydb/library/yql/minikql/comp_nodes/mkql_multimap.cpp +++ b/ydb/library/yql/minikql/comp_nodes/mkql_multimap.cpp @@ -4,8 +4,8 @@ #include <ydb/library/yql/minikql/mkql_node_cast.h> #include <ydb/library/yql/utils/cast.h> -#include <util/string/cast.h> - +#include <util/string/cast.h> + namespace NKikimr { namespace NMiniKQL { diff --git a/ydb/library/yql/minikql/comp_nodes/mkql_switch.cpp b/ydb/library/yql/minikql/comp_nodes/mkql_switch.cpp index 12faf98a3c1..4571141d807 100644 --- a/ydb/library/yql/minikql/comp_nodes/mkql_switch.cpp +++ b/ydb/library/yql/minikql/comp_nodes/mkql_switch.cpp @@ -4,8 +4,8 @@ #include <ydb/library/yql/minikql/mkql_stats_registry.h> #include <ydb/library/yql/utils/cast.h> -#include <util/string/cast.h> - +#include <util/string/cast.h> + namespace NKikimr { namespace NMiniKQL { diff --git a/ydb/library/yql/minikql/comp_nodes/mkql_timezone.cpp b/ydb/library/yql/minikql/comp_nodes/mkql_timezone.cpp index bd08831500c..3d2129e53b1 100644 --- a/ydb/library/yql/minikql/comp_nodes/mkql_timezone.cpp +++ b/ydb/library/yql/minikql/comp_nodes/mkql_timezone.cpp @@ -4,8 +4,8 @@ #include <ydb/library/yql/minikql/mkql_string_util.h> #include <ydb/library/yql/minikql/mkql_type_ops.h> -#include <util/string/cast.h> - +#include <util/string/cast.h> + namespace NKikimr { namespace NMiniKQL { diff --git a/ydb/library/yql/minikql/comp_nodes/mkql_way.cpp b/ydb/library/yql/minikql/comp_nodes/mkql_way.cpp index 9db144d1b1a..1071b8b7800 100644 --- a/ydb/library/yql/minikql/comp_nodes/mkql_way.cpp +++ b/ydb/library/yql/minikql/comp_nodes/mkql_way.cpp @@ -5,8 +5,8 @@ #include <ydb/library/yql/minikql/mkql_node_builder.h> #include <ydb/library/yql/minikql/mkql_string_util.h> -#include <util/string/cast.h> - +#include <util/string/cast.h> + namespace NKikimr { namespace NMiniKQL { diff --git a/ydb/library/yql/minikql/comp_nodes/mkql_wide_combine.cpp b/ydb/library/yql/minikql/comp_nodes/mkql_wide_combine.cpp index ed337ff265b..1fe1734a156 100644 --- a/ydb/library/yql/minikql/comp_nodes/mkql_wide_combine.cpp +++ b/ydb/library/yql/minikql/comp_nodes/mkql_wide_combine.cpp @@ -7,8 +7,8 @@ #include <ydb/library/yql/minikql/defs.h> #include <ydb/library/yql/utils/cast.h> -#include <util/string/cast.h> - +#include <util/string/cast.h> + namespace NKikimr { namespace NMiniKQL { diff --git a/ydb/library/yql/minikql/comp_nodes/ut/mkql_chain_map_ut.cpp b/ydb/library/yql/minikql/comp_nodes/ut/mkql_chain_map_ut.cpp index b8d0140260f..b41a0527de8 100644 --- a/ydb/library/yql/minikql/comp_nodes/ut/mkql_chain_map_ut.cpp +++ b/ydb/library/yql/minikql/comp_nodes/ut/mkql_chain_map_ut.cpp @@ -2,7 +2,7 @@ #include <ydb/library/yql/minikql/mkql_node_cast.h> #include <ydb/library/yql/minikql/mkql_string_util.h> - + namespace NKikimr { namespace NMiniKQL { diff --git a/ydb/library/yql/minikql/comp_nodes/ut/mkql_compare_ut.cpp b/ydb/library/yql/minikql/comp_nodes/ut/mkql_compare_ut.cpp index 18b8abeffb6..8f51d66bc0d 100644 --- a/ydb/library/yql/minikql/comp_nodes/ut/mkql_compare_ut.cpp +++ b/ydb/library/yql/minikql/comp_nodes/ut/mkql_compare_ut.cpp @@ -7,13 +7,13 @@ #include <ydb/library/yql/minikql/computation/mkql_computation_node_impl.h> #include <ydb/library/yql/minikql/computation/mkql_computation_list_adapter.h> #include <ydb/library/yql/minikql/computation/mkql_computation_node_holders.h> - + #include <cfloat> #include <utility> - + namespace NKikimr { namespace NMiniKQL { - + Y_UNIT_TEST_SUITE(TMiniKQLCompareTest) { Y_UNIT_TEST_LLVM(SqlString) { TSetup<LLVM> setup; @@ -867,7 +867,7 @@ Y_UNIT_TEST_SUITE(TMiniKQLCompareTest) { UNIT_ASSERT(!iterator.Next(item)); UNIT_ASSERT(!iterator.Next(item)); - } + } Y_UNIT_TEST_LLVM(TzMin) { TSetup<LLVM> setup; diff --git a/ydb/library/yql/minikql/comp_nodes/ut/mkql_computation_node_ut.cpp b/ydb/library/yql/minikql/comp_nodes/ut/mkql_computation_node_ut.cpp index 44682a365f8..19f82aae4ed 100644 --- a/ydb/library/yql/minikql/comp_nodes/ut/mkql_computation_node_ut.cpp +++ b/ydb/library/yql/minikql/comp_nodes/ut/mkql_computation_node_ut.cpp @@ -6,16 +6,16 @@ #include <ydb/library/yql/minikql/computation/mkql_computation_node_impl.h> #include <ydb/library/yql/minikql/computation/mkql_computation_node_holders.h> - + #include <cfloat> #include <utility> #include <random> - + namespace NKikimr { namespace NMiniKQL { - + namespace { - + ui64 g_Yield = std::numeric_limits<ui64>::max(); ui64 g_TestStreamData[] = {0, 0, 1, 0, 0, 0, 1, 2, 3}; @@ -58,7 +58,7 @@ public: NUdf::TUnboxedValuePod DoCalculate(TComputationContext& ctx) const { return ctx.HolderFactory.Create<TStreamValue>(Count); } - + private: void RegisterDependencies() const final {} @@ -1851,7 +1851,7 @@ Y_UNIT_TEST_SUITE(TMiniKQLComputationNodeTest) { Y_UNIT_TEST_LLVM(TestDivZero) { TSetup<LLVM> setup; TProgramBuilder& pb = *setup.PgmBuilder; - + const auto data1 = pb.NewDataLiteral<ui32>(17); const auto data2 = pb.NewDataLiteral<ui32>(0); const auto pgmReturn = pb.Div(data1, data2); @@ -3957,12 +3957,12 @@ Y_UNIT_TEST_SUITE(TMiniKQLComputationNodeTest) { pb.NewStruct(map3) }); - { + { const auto pgmReturn = pb.Sort(list, pb.NewDataLiteral(asc), - [&](TRuntimeNode item) { + [&](TRuntimeNode item) { return pb.Member(item, "key"); }); - + if (asc) { // ascending sort const auto graph = setup.BuildGraph(pgmReturn); @@ -3998,64 +3998,64 @@ Y_UNIT_TEST_SUITE(TMiniKQLComputationNodeTest) { UNIT_ASSERT(!iterator.Next(item)); UNIT_ASSERT(!iterator.Next(item)); } - } + } } - + Y_UNIT_TEST_LLVM(TestSort) { TestSortImpl<LLVM>(true); TestSortImpl<LLVM>(false); } using TTriple = std::tuple<ui32, ui32, ui32>; - + TRuntimeNode TupleOrder(TProgramBuilder& pb, bool asc1, bool asc2, bool asc3) { TVector<TRuntimeNode> ascending(3); ascending[0] = pb.NewDataLiteral(asc1); ascending[1] = pb.NewDataLiteral(asc2); ascending[2] = pb.NewDataLiteral(asc3); - + TVector<TType*> tupleTypes(3); tupleTypes[0] = pb.NewDataType(NUdf::TDataType<bool>::Id); tupleTypes[1] = pb.NewDataType(NUdf::TDataType<bool>::Id); tupleTypes[2] = pb.NewDataType(NUdf::TDataType<bool>::Id); - + return pb.NewTuple(pb.NewTupleType(tupleTypes), ascending); - } - + } + template<bool LLVM> TVector<TTriple> SortTuples(TSetup<LLVM>& setup, TRuntimeNode list, TRuntimeNode order) - { + { auto& pb = *setup.PgmBuilder; const auto pgmReturn = pb.Sort(list, order, [](TRuntimeNode item) { return item; }); const auto graph = setup.BuildGraph(pgmReturn); const auto iterator = graph->GetValue().GetListIterator(); - + TVector<TTriple> result; for (NUdf::TUnboxedValue value; iterator.Next(value);) { ui32 first = value.GetElement(0).template Get<ui32>(); ui32 second = value.GetElement(1).template Get<ui32>(); ui32 third = value.GetElement(2).template Get<ui32>(); - result.push_back(TTriple{ first, second, third }); - } + result.push_back(TTriple{ first, second, third }); + } UNIT_ASSERT(!iterator.Skip()); - return result; - } - + return result; + } + Y_UNIT_TEST_LLVM(TestSortTuples) { TSetup<LLVM> setup; TProgramBuilder& pb = *setup.PgmBuilder; const auto listMaker = [&]() - { - TTriple testData[] = { - { 1, 1, 1 }, - { 1, 1, 2 }, - { 1, 2, 3 }, - { 1, 3, 0 }, - { 2, 0, 1 }, - { 2, 1, 0 }, - }; - + { + TTriple testData[] = { + { 1, 1, 1 }, + { 1, 1, 2 }, + { 1, 2, 3 }, + { 1, 3, 0 }, + { 2, 0, 1 }, + { 2, 1, 0 }, + }; + TVector<TRuntimeNode> tuplesList; for (ui32 i = 0; i < Y_ARRAY_SIZE(testData); i++) { TVector<TRuntimeNode> elements(3); @@ -4063,77 +4063,77 @@ Y_UNIT_TEST_SUITE(TMiniKQLComputationNodeTest) { elements[1] = pb.NewDataLiteral(std::get<1>(testData[i])); elements[2] = pb.NewDataLiteral(std::get<2>(testData[i])); tuplesList.push_back(pb.NewTuple(elements)); - } - + } + TVector<TType*> tupleTypes(3); tupleTypes[0] = pb.NewDataType(NUdf::TDataType<ui32>::Id); tupleTypes[1] = pb.NewDataType(NUdf::TDataType<ui32>::Id); tupleTypes[2] = pb.NewDataType(NUdf::TDataType<ui32>::Id); - + return pb.NewList(pb.NewTupleType(tupleTypes), tuplesList); }; - - { + + { TRuntimeNode order = TupleOrder(pb, true, true, true); - TTriple expectedData[] = { - { 1, 1, 1 }, - { 1, 1, 2 }, - { 1, 2, 3 }, - { 1, 3, 0 }, - { 2, 0, 1 }, - { 2, 1, 0 }, - }; + TTriple expectedData[] = { + { 1, 1, 1 }, + { 1, 1, 2 }, + { 1, 2, 3 }, + { 1, 3, 0 }, + { 2, 0, 1 }, + { 2, 1, 0 }, + }; TVector<TTriple> expected(expectedData, expectedData + sizeof(expectedData) / sizeof(*expectedData)); TVector<TTriple> result = SortTuples<LLVM>(setup, listMaker(), order); - UNIT_ASSERT_EQUAL(result, expected); - } - - { + UNIT_ASSERT_EQUAL(result, expected); + } + + { TRuntimeNode order = TupleOrder(pb, false, false, false); - TTriple expectedData[] = { - { 2, 1, 0 }, - { 2, 0, 1 }, - { 1, 3, 0 }, - { 1, 2, 3 }, - { 1, 1, 2 }, - { 1, 1, 1 }, - }; + TTriple expectedData[] = { + { 2, 1, 0 }, + { 2, 0, 1 }, + { 1, 3, 0 }, + { 1, 2, 3 }, + { 1, 1, 2 }, + { 1, 1, 1 }, + }; TVector<TTriple> expected(expectedData, expectedData + sizeof(expectedData) / sizeof(*expectedData)); TVector<TTriple> result = SortTuples<LLVM>(setup, listMaker(), order); - UNIT_ASSERT_EQUAL(result, expected); - } - - { + UNIT_ASSERT_EQUAL(result, expected); + } + + { TRuntimeNode order = TupleOrder(pb, true, false, true); - TTriple expectedData[] = { - { 1, 3, 0 }, - { 1, 2, 3 }, - { 1, 1, 1 }, - { 1, 1, 2 }, - { 2, 1, 0 }, - { 2, 0, 1 }, - }; + TTriple expectedData[] = { + { 1, 3, 0 }, + { 1, 2, 3 }, + { 1, 1, 1 }, + { 1, 1, 2 }, + { 2, 1, 0 }, + { 2, 0, 1 }, + }; TVector<TTriple> expected(expectedData, expectedData + sizeof(expectedData) / sizeof(*expectedData)); TVector<TTriple> result = SortTuples<LLVM>(setup, listMaker(), order); - UNIT_ASSERT_EQUAL(result, expected); - } - - { + UNIT_ASSERT_EQUAL(result, expected); + } + + { TRuntimeNode order = TupleOrder(pb, false, true, false); - TTriple expectedData[] = { - { 2, 0, 1 }, - { 2, 1, 0 }, - { 1, 1, 2 }, - { 1, 1, 1 }, - { 1, 2, 3 }, - { 1, 3, 0 }, - }; + TTriple expectedData[] = { + { 2, 0, 1 }, + { 2, 1, 0 }, + { 1, 1, 2 }, + { 1, 1, 1 }, + { 1, 2, 3 }, + { 1, 3, 0 }, + }; TVector<TTriple> expected(expectedData, expectedData + sizeof(expectedData) / sizeof(*expectedData)); TVector<TTriple> result = SortTuples<LLVM>(setup, listMaker(), order); - UNIT_ASSERT_EQUAL(result, expected); - } - } - + UNIT_ASSERT_EQUAL(result, expected); + } + } + Y_UNIT_TEST_LLVM(TestAsList) { TSetup<LLVM> setup; TProgramBuilder& pb = *setup.PgmBuilder; @@ -4189,7 +4189,7 @@ Y_UNIT_TEST_SUITE(TMiniKQLComputationNodeTest) { const auto res = graph->GetValue().template Get<ui32>(); UNIT_ASSERT_VALUES_EQUAL(res, 56); } - + Y_UNIT_TEST_LLVM(NonDeterministicEnv) { TSetup<LLVM> setup; TProgramBuilder& pb = *setup.PgmBuilder; @@ -4202,7 +4202,7 @@ Y_UNIT_TEST_SUITE(TMiniKQLComputationNodeTest) { pb.NewDataLiteral(false), [](TRuntimeNode item) { return item; } ); - + const auto graph = setup.BuildGraph(pgmReturn); const auto iterator = graph->GetValue().GetListIterator(); NUdf::TUnboxedValue item; @@ -4215,8 +4215,8 @@ Y_UNIT_TEST_SUITE(TMiniKQLComputationNodeTest) { UNIT_ASSERT(std::abs(item.template Get<float>() - 1.0) < 0.001); UNIT_ASSERT(false == iterator.Next(item)); UNIT_ASSERT(false == iterator.Next(item)); - } - + } + Y_UNIT_TEST_LLVM(TestIndexDictContains) { TSetup<LLVM> setup; TProgramBuilder& pb = *setup.PgmBuilder; diff --git a/ydb/library/yql/minikql/comp_nodes/ut/mkql_computation_node_ut.h b/ydb/library/yql/minikql/comp_nodes/ut/mkql_computation_node_ut.h index 116be966c96..db79fc7ab85 100644 --- a/ydb/library/yql/minikql/comp_nodes/ut/mkql_computation_node_ut.h +++ b/ydb/library/yql/minikql/comp_nodes/ut/mkql_computation_node_ut.h @@ -6,19 +6,19 @@ #include <ydb/library/yql/minikql/mkql_function_registry.h> #include <ydb/library/yql/minikql/mkql_terminator.h> #include "../mkql_factories.h" - + #include <library/cpp/testing/unittest/registar.h> -#define UNBOXED_VALUE_STR_EQUAL(unboxed, expected) \ - do { \ +#define UNBOXED_VALUE_STR_EQUAL(unboxed, expected) \ + do { \ const auto v = (unboxed); \ - if (!(v.AsStringRef() == (expected))) { \ - UNIT_FAIL_IMPL( \ - "equal assertion failed", \ - Sprintf("%s == %s", #unboxed, #expected)); \ - } \ - } while (0) - + if (!(v.AsStringRef() == (expected))) { \ + UNIT_FAIL_IMPL( \ + "equal assertion failed", \ + Sprintf("%s == %s", #unboxed, #expected)); \ + } \ + } while (0) + #if defined(_msan_enabled_) || defined(_ubsan_enabled_) || defined(WITH_VALGRIND) diff --git a/ydb/library/yql/minikql/comp_nodes/ut/mkql_condense_ut.cpp b/ydb/library/yql/minikql/comp_nodes/ut/mkql_condense_ut.cpp index 9b22c49986d..653d6c0cc08 100644 --- a/ydb/library/yql/minikql/comp_nodes/ut/mkql_condense_ut.cpp +++ b/ydb/library/yql/minikql/comp_nodes/ut/mkql_condense_ut.cpp @@ -2,7 +2,7 @@ #include <ydb/library/yql/minikql/mkql_node_cast.h> #include <ydb/library/yql/minikql/mkql_string_util.h> - + namespace NKikimr { namespace NMiniKQL { diff --git a/ydb/library/yql/minikql/comp_nodes/ut/mkql_decimal_ut.cpp b/ydb/library/yql/minikql/comp_nodes/ut/mkql_decimal_ut.cpp index 41adc8ff695..eb5ff4df71b 100644 --- a/ydb/library/yql/minikql/comp_nodes/ut/mkql_decimal_ut.cpp +++ b/ydb/library/yql/minikql/comp_nodes/ut/mkql_decimal_ut.cpp @@ -5,7 +5,7 @@ namespace NKikimr { namespace NMiniKQL { - + Y_UNIT_TEST_SUITE(TMiniKQLDecimalTest) { Y_UNIT_TEST_LLVM(TestNanvl) { TSetup<LLVM> setup; diff --git a/ydb/library/yql/minikql/comp_nodes/ut/mkql_dict_ut.cpp b/ydb/library/yql/minikql/comp_nodes/ut/mkql_dict_ut.cpp index 5a604b6e9bf..ac6d4c356b2 100644 --- a/ydb/library/yql/minikql/comp_nodes/ut/mkql_dict_ut.cpp +++ b/ydb/library/yql/minikql/comp_nodes/ut/mkql_dict_ut.cpp @@ -5,7 +5,7 @@ namespace NKikimr { namespace NMiniKQL { - + Y_UNIT_TEST_SUITE(TMiniKQLDictRelatedNodesTest) { Y_UNIT_TEST_LLVM(TestDictLength) { TSetup<LLVM> setup; diff --git a/ydb/library/yql/minikql/comp_nodes/ut/mkql_filters_ut.cpp b/ydb/library/yql/minikql/comp_nodes/ut/mkql_filters_ut.cpp index c4083aabe9e..e2c1c4dc690 100644 --- a/ydb/library/yql/minikql/comp_nodes/ut/mkql_filters_ut.cpp +++ b/ydb/library/yql/minikql/comp_nodes/ut/mkql_filters_ut.cpp @@ -6,7 +6,7 @@ namespace NKikimr { namespace NMiniKQL { - + Y_UNIT_TEST_SUITE(TMiniKQLFiltersTest) { Y_UNIT_TEST_LLVM(TestSkipNullMembers) { TSetup<LLVM> setup; diff --git a/ydb/library/yql/minikql/comp_nodes/ut/mkql_flatmap_ut.cpp b/ydb/library/yql/minikql/comp_nodes/ut/mkql_flatmap_ut.cpp index cf186e2170d..8ad155473d5 100644 --- a/ydb/library/yql/minikql/comp_nodes/ut/mkql_flatmap_ut.cpp +++ b/ydb/library/yql/minikql/comp_nodes/ut/mkql_flatmap_ut.cpp @@ -3,7 +3,7 @@ namespace NKikimr { namespace NMiniKQL { - + Y_UNIT_TEST_SUITE(TMiniKQLFlatMapTest) { Y_UNIT_TEST_LLVM(TestOverListAndPartialLists) { TSetup<LLVM> setup; diff --git a/ydb/library/yql/minikql/comp_nodes/ut/mkql_fold_ut.cpp b/ydb/library/yql/minikql/comp_nodes/ut/mkql_fold_ut.cpp index 326e4a6712a..87c596e094c 100644 --- a/ydb/library/yql/minikql/comp_nodes/ut/mkql_fold_ut.cpp +++ b/ydb/library/yql/minikql/comp_nodes/ut/mkql_fold_ut.cpp @@ -3,7 +3,7 @@ #include <ydb/library/yql/minikql/mkql_node_cast.h> #include <ydb/library/yql/minikql/mkql_string_util.h> #include <ydb/library/yql/minikql/computation/mkql_computation_node_holders.h> - + #include <random> #include <ctime> #include <algorithm> @@ -46,7 +46,7 @@ Y_UNIT_TEST_SUITE(TMiniKQLFoldNodeTest) { }); auto graph = setup.BuildGraph(pgmReturn); - auto value = graph->GetValue(); + auto value = graph->GetValue(); UNIT_ASSERT(!value); } @@ -67,7 +67,7 @@ Y_UNIT_TEST_SUITE(TMiniKQLFoldNodeTest) { }); auto graph = setup.BuildGraph(pgmReturn); - auto value = graph->GetValue(); + auto value = graph->GetValue(); UNIT_ASSERT(value); UNIT_ASSERT_VALUES_EQUAL(value.template Get<ui32>(), 2); } @@ -89,7 +89,7 @@ Y_UNIT_TEST_SUITE(TMiniKQLFoldNodeTest) { }); auto graph = setup.BuildGraph(pgmReturn); - auto value = graph->GetValue(); + auto value = graph->GetValue(); UNIT_ASSERT(value); UNIT_ASSERT_VALUES_EQUAL(value.template Get<ui32>(), 4); } @@ -684,7 +684,7 @@ Y_UNIT_TEST_SUITE(TMiniKQLFoldNodeTest) { }); auto graph = setup.BuildGraph(pgmReturn); - auto res = graph->GetValue(); + auto res = graph->GetValue(); UNBOXED_VALUE_STR_EQUAL(res, "Xaabbbzzzz"); } @@ -811,7 +811,7 @@ Y_UNIT_TEST_SUITE(TMiniKQLFoldNodeTest) { const auto listType = pb.NewListType(pb.NewDataType(NUdf::TDataType<double>::Id)); const auto list = TCallableBuilder(pb.GetTypeEnvironment(), "TestList", listType).Build(); - + const auto pgmReturn = pb.Fold1(pb.Collect(TRuntimeNode(list, false)), [&](TRuntimeNode item) { return item; }, [&](TRuntimeNode item, TRuntimeNode state) { return pb.AggrAdd(state, item); } diff --git a/ydb/library/yql/minikql/comp_nodes/ut/mkql_join_dict_ut.cpp b/ydb/library/yql/minikql/comp_nodes/ut/mkql_join_dict_ut.cpp index e5509efb452..893722216d9 100644 --- a/ydb/library/yql/minikql/comp_nodes/ut/mkql_join_dict_ut.cpp +++ b/ydb/library/yql/minikql/comp_nodes/ut/mkql_join_dict_ut.cpp @@ -2,7 +2,7 @@ namespace NKikimr { namespace NMiniKQL { - + Y_UNIT_TEST_SUITE(TMiniKQLJoinDictNodeTest) { Y_UNIT_TEST_LLVM(TestInner) { for (ui32 pass = 0; pass < 1; ++pass) { diff --git a/ydb/library/yql/minikql/comp_nodes/ut/mkql_variant_ut.cpp b/ydb/library/yql/minikql/comp_nodes/ut/mkql_variant_ut.cpp index 6ce096913b3..2a52f73ef33 100644 --- a/ydb/library/yql/minikql/comp_nodes/ut/mkql_variant_ut.cpp +++ b/ydb/library/yql/minikql/comp_nodes/ut/mkql_variant_ut.cpp @@ -5,7 +5,7 @@ namespace NKikimr { namespace NMiniKQL { - + Y_UNIT_TEST_SUITE(TMiniKQLVariantTest) { Y_UNIT_TEST_LLVM(TestGuessTuple) { TSetup<LLVM> setup; diff --git a/ydb/library/yql/minikql/computation/mkql_computation_list_adapter.h b/ydb/library/yql/minikql/computation/mkql_computation_list_adapter.h index 667f8473649..2b3b7fb8865 100644 --- a/ydb/library/yql/minikql/computation/mkql_computation_list_adapter.h +++ b/ydb/library/yql/minikql/computation/mkql_computation_list_adapter.h @@ -97,14 +97,14 @@ public: ui64 Index; }; - TVectorListAdapter( - TMemoryUsageInfo* memInfo, + TVectorListAdapter( + TMemoryUsageInfo* memInfo, const TVectorType& list, - TItemFactory itemFactory, - ui64 start, ui64 finish, - bool reversed) - : TBase(memInfo) - , List(list) + TItemFactory itemFactory, + ui64 start, ui64 finish, + bool reversed) + : TBase(memInfo) + , List(list) , ItemFactory(itemFactory) , Start(start) , Finish(finish) @@ -130,12 +130,12 @@ private: return Finish != Start; } - NUdf::IBoxedValuePtr ReverseListImpl(const NUdf::IValueBuilder& builder) const override { + NUdf::IBoxedValuePtr ReverseListImpl(const NUdf::IValueBuilder& builder) const override { Y_UNUSED(builder); return new TSelf(this->GetMemInfo(), List, ItemFactory, Start, Finish, !Reversed); } - NUdf::IBoxedValuePtr SkipListImpl(const NUdf::IValueBuilder& builder, ui64 count) const override { + NUdf::IBoxedValuePtr SkipListImpl(const NUdf::IValueBuilder& builder, ui64 count) const override { Y_UNUSED(builder); const ui64 newStart = Min(Start + count, Finish); if (newStart == Start) { @@ -145,7 +145,7 @@ private: return new TSelf(this->GetMemInfo(), List, ItemFactory, newStart, Finish, Reversed); } - NUdf::IBoxedValuePtr TakeListImpl(const NUdf::IValueBuilder& builder, ui64 count) const override { + NUdf::IBoxedValuePtr TakeListImpl(const NUdf::IValueBuilder& builder, ui64 count) const override { Y_UNUSED(builder); const ui64 newFinish = Min(Start + count, Finish); if (newFinish == Finish) { @@ -155,7 +155,7 @@ private: return new TSelf(this->GetMemInfo(), List, ItemFactory, Start, newFinish, Reversed); } - NUdf::IBoxedValuePtr ToIndexDictImpl(const NUdf::IValueBuilder& builder) const override { + NUdf::IBoxedValuePtr ToIndexDictImpl(const NUdf::IValueBuilder& builder) const override { Y_UNUSED(builder); return const_cast<TVectorListAdapter*>(this); } @@ -243,5 +243,5 @@ NUdf::TUnboxedValue CreateOwningVectorListAdapter( &memInfo, std::forward<TVectorType>(list), itemFactory, start, finish, reversed)); } -} +} } diff --git a/ydb/library/yql/minikql/computation/mkql_computation_node.h b/ydb/library/yql/minikql/computation/mkql_computation_node.h index 0eb51cad41b..a5d11b180b0 100644 --- a/ydb/library/yql/minikql/computation/mkql_computation_node.h +++ b/ydb/library/yql/minikql/computation/mkql_computation_node.h @@ -1,5 +1,5 @@ #pragma once - + #include "mkql_computation_node_list.h" #include <ydb/library/yql/minikql/defs.h> @@ -10,19 +10,19 @@ #include <ydb/library/yql/minikql/mkql_alloc.h> #include <ydb/library/yql/minikql/mkql_stats_registry.h> #include <ydb/library/yql/minikql/mkql_terminator.h> - + #include <ydb/library/yql/public/udf/udf_value.h> #include <ydb/library/yql/public/udf/udf_validate.h> #include <ydb/library/yql/public/udf/udf_value_builder.h> #include <library/cpp/random_provider/random_provider.h> #include <library/cpp/time_provider/time_provider.h> - + #include <map> #include <unordered_set> #include <unordered_map> #include <vector> - + namespace NKikimr { namespace NMiniKQL { @@ -69,7 +69,7 @@ class THolderFactory; // Do not reorder: used in LLVM! struct TComputationContextLLVM { - const THolderFactory& HolderFactory; + const THolderFactory& HolderFactory; IStatsRegistry *const Stats; const std::unique_ptr<NUdf::TUnboxedValue[]> MutableValues; const NUdf::IValueBuilder *const Builder; @@ -135,7 +135,7 @@ public: virtual void PrepareStageTwo() = 0; virtual TString DebugString() const = 0; - + virtual void Ref() = 0; virtual void UnRef() = 0; virtual ui32 RefCount() const = 0; @@ -187,8 +187,8 @@ public: virtual IComputationExternalNode* GetEntryPoint(size_t index, bool require) = 0; virtual const TComputationNodePtrDeque& GetNodes() const = 0; virtual void Invalidate() = 0; - virtual TMemoryUsageInfo& GetMemInfo() const = 0; - virtual const THolderFactory& GetHolderFactory() const = 0; + virtual TMemoryUsageInfo& GetMemInfo() const = 0; + virtual const THolderFactory& GetHolderFactory() const = 0; virtual ITerminator* GetTerminator() const = 0; virtual bool SetExecuteLLVM(bool value) = 0; virtual TString SaveGraphState() = 0; @@ -201,12 +201,12 @@ typedef std::function<void (IComputationNode*)> TNodePushBack; struct TComputationNodeFactoryContext { TNodeLocator NodeLocator; - const IFunctionRegistry& FunctionRegistry; + const IFunctionRegistry& FunctionRegistry; const TTypeEnvironment& Env; NUdf::ITypeInfoHelper::TPtr TypeInfoHelper; NUdf::ICountersProvider* CountersProvider; const NUdf::ISecureParamsProvider* SecureParamsProvider; - const TNodeFactory& NodeFactory; + const TNodeFactory& NodeFactory; const THolderFactory& HolderFactory; const NUdf::IValueBuilder *const Builder; NUdf::EValidateMode ValidateMode; @@ -216,10 +216,10 @@ struct TComputationNodeFactoryContext { TComputationNodeOnNodeMap& ElementsCache; const TNodePushBack NodePushBack; - TComputationNodeFactoryContext( - const TNodeLocator& nodeLocator, - const IFunctionRegistry& functionRegistry, - const TTypeEnvironment& env, + TComputationNodeFactoryContext( + const TNodeLocator& nodeLocator, + const IFunctionRegistry& functionRegistry, + const TTypeEnvironment& env, NUdf::ITypeInfoHelper::TPtr typeInfoHelper, NUdf::ICountersProvider* countersProvider, const NUdf::ISecureParamsProvider* secureParamsProvider, @@ -234,12 +234,12 @@ struct TComputationNodeFactoryContext { TNodePushBack&& nodePushBack ) : NodeLocator(nodeLocator) - , FunctionRegistry(functionRegistry) + , FunctionRegistry(functionRegistry) , Env(env) , TypeInfoHelper(typeInfoHelper) , CountersProvider(countersProvider) , SecureParamsProvider(secureParamsProvider) - , NodeFactory(nodeFactory) + , NodeFactory(nodeFactory) , HolderFactory(holderFactory) , Builder(builder) , ValidateMode(validateMode) @@ -253,7 +253,7 @@ struct TComputationNodeFactoryContext { using TComputationNodeFactory = std::function<IComputationNode* (TCallable&, const TComputationNodeFactoryContext&)>; using TStreamEmitter = std::function<void(NUdf::TUnboxedValue&&)>; - + struct TComputationPatternOpts { TComputationPatternOpts(const std::shared_ptr<TInjectedAlloc>& cacheAlloc, const std::shared_ptr<TTypeEnvironment>& cacheEnv) : CacheAlloc(std::move(cacheAlloc)) @@ -269,8 +269,8 @@ struct TComputationPatternOpts { TComputationPatternOpts( TAllocState& allocState, - const TTypeEnvironment& env, - TComputationNodeFactory factory, + const TTypeEnvironment& env, + TComputationNodeFactory factory, const IFunctionRegistry* functionRegistry, NUdf::EValidateMode validateMode, NUdf::EValidatePolicy validatePolicy, @@ -325,7 +325,7 @@ struct TComputationPatternOpts { return TComputationOptsFull(Stats, allocStatePtr ? *allocStatePtr : AllocState, randomProvider, timeProvider, ValidatePolicy, SecureParamsProvider); } }; - + class IComputationPattern: public TAtomicRefCount<IComputationPattern> { public: typedef TIntrusivePtr<IComputationPattern> TPtr; @@ -357,5 +357,5 @@ public: std::unique_ptr<NUdf::ISecureParamsProvider> MakeSimpleSecureParamsProvider(const THashMap<TString, TString>& secureParams); -} // namespace NMiniKQL -} // namespace NKikimr +} // namespace NMiniKQL +} // namespace NKikimr diff --git a/ydb/library/yql/minikql/computation/mkql_computation_node_codegen.cpp b/ydb/library/yql/minikql/computation/mkql_computation_node_codegen.cpp index f31db9f92fa..563f820bf61 100644 --- a/ydb/library/yql/minikql/computation/mkql_computation_node_codegen.cpp +++ b/ydb/library/yql/minikql/computation/mkql_computation_node_codegen.cpp @@ -4,8 +4,8 @@ #include <ydb/library/yql/minikql/codegen/codegen.h> #include <ydb/library/yql/public/decimal/yql_decimal.h> -#include <util/string/cast.h> - +#include <util/string/cast.h> + #ifndef MKQL_DISABLE_CODEGEN extern "C" void DeleteBoxed(NKikimr::NUdf::IBoxedValue *const boxed) { diff --git a/ydb/library/yql/minikql/computation/mkql_computation_node_graph.cpp b/ydb/library/yql/minikql/computation/mkql_computation_node_graph.cpp index 939bc9ccef6..f2b188ae47c 100644 --- a/ydb/library/yql/minikql/computation/mkql_computation_node_graph.cpp +++ b/ydb/library/yql/minikql/computation/mkql_computation_node_graph.cpp @@ -1,4 +1,4 @@ -#include "mkql_computation_node_holders.h" +#include "mkql_computation_node_holders.h" #include "mkql_value_builder.h" #include "mkql_computation_node_codegen.h" #include <ydb/library/yql/minikql/arrow/mkql_memory_pool.h> @@ -9,7 +9,7 @@ #include <util/system/env.h> #include <util/system/mutex.h> #include <util/digest/city.h> - + #ifndef MKQL_DISABLE_CODEGEN #include <llvm/Support/raw_ostream.h> #endif @@ -37,7 +37,7 @@ const static TStatKey PagePool_PageHitCount("PagePool_PageHitCount", true); const static TStatKey PagePool_PageMissCount("PagePool_PageMissCount", true); const static TStatKey PagePool_OffloadedAllocCount("PagePool_OffloadedAllocCount", true); const static TStatKey PagePool_OffloadedBytes("PagePool_OffloadedBytes", true); - + const static TStatKey CodeGen_FullTime("CodeGen_FullTime", true); const static TStatKey CodeGen_GenerateTime("CodeGen_GenerateTime", true); const static TStatKey CodeGen_CompileTime("CodeGen_CompileTime", true); @@ -46,7 +46,7 @@ const static TStatKey CodeGen_TotalInstructions("CodeGen_TotalInstructions", tru const static TStatKey CodeGen_MaxFunctionInstructions("CodeGen_MaxFunctionInstructions", false); const static TStatKey CodeGen_ModulePassTime("CodeGen_ModulePassTime", true); const static TStatKey CodeGen_FinalizeTime("CodeGen_FinalizeTime", true); - + const static TStatKey Mkql_TotalNodes("Mkql_TotalNodes", true); const static TStatKey Mkql_CodegenFunctions("Mkql_CodegenFunctions", true); @@ -205,10 +205,10 @@ private: TComputationNodeOnNodeMap ElementsCache; }; -class TComputationGraphBuildingVisitor: - public INodeVisitor, - private TNonCopyable -{ +class TComputationGraphBuildingVisitor: + public INodeVisitor, + private TNonCopyable +{ public: TComputationGraphBuildingVisitor(const TComputationPatternOpts& opts) : Env(opts.Env) @@ -236,10 +236,10 @@ public: return Env; } - const IFunctionRegistry& GetFunctionRegistry() const { - return FunctionRegistry; - } - + const IFunctionRegistry& GetFunctionRegistry() const { + return FunctionRegistry; + } + private: template <typename T> void VisitType(T& node) { @@ -382,7 +382,7 @@ private: void Visit(TOptionalLiteral& node) override { auto item = node.HasItem() ? GetComputationNode(node.GetItem().GetNode()) : nullptr; - AddNode(node, NodeFactory->CreateOptionalNode(item)); + AddNode(node, NodeFactory->CreateOptionalNode(item)); } void Visit(TDictLiteral& node) override { @@ -414,10 +414,10 @@ private: return; } - TNodeLocator nodeLocator = [this](TNode* dependentNode, bool pop) { - return GetComputationNode(dependentNode, pop); - }; - TComputationNodeFactoryContext ctx( + TNodeLocator nodeLocator = [this](TNode* dependentNode, bool pop) { + return GetComputationNode(dependentNode, pop); + }; + TComputationNodeFactoryContext ctx( nodeLocator, FunctionRegistry, Env, @@ -481,12 +481,12 @@ public: TPatternNodes::TPtr GetPatternNodes() { return PatternNodes; - } - + } + const TComputationNodePtrDeque& GetNodes() const { return PatternNodes->GetNodes(); - } - + } + void PreserveRoot(IComputationNode* rootNode) { PatternNodes->RootNode = rootNode; } @@ -512,9 +512,9 @@ private: NUdf::ICountersProvider* CountersProvider; const NUdf::ISecureParamsProvider* SecureParamsProvider; const TComputationNodeFactory Factory; - const IFunctionRegistry& FunctionRegistry; + const IFunctionRegistry& FunctionRegistry; TIntrusivePtr<TMemoryUsageInfo> MemInfo; - THolder<TNodeFactory> NodeFactory; + THolder<TNodeFactory> NodeFactory; NUdf::EValidateMode ValidateMode; NUdf::EValidatePolicy ValidatePolicy; EGraphPerProcess GraphPerProcess; @@ -587,16 +587,16 @@ public: const TComputationNodePtrDeque& GetNodes() const override { return PatternNodes->GetNodes(); - } - - TMemoryUsageInfo& GetMemInfo() const override { + } + + TMemoryUsageInfo& GetMemInfo() const override { return *MemInfo; - } - + } + const THolderFactory& GetHolderFactory() const override { return *HolderFactory; - } - + } + ITerminator* GetTerminator() const override { return ValueBuilder.Get(); } @@ -656,7 +656,7 @@ private: bool IsPrepared = false; }; -} // namespace +} // namespace class TComputationPatternImpl : public IComputationPattern { public: diff --git a/ydb/library/yql/minikql/computation/mkql_computation_node_holders.cpp b/ydb/library/yql/minikql/computation/mkql_computation_node_holders.cpp index 717c2c153a1..67a7cfb3028 100644 --- a/ydb/library/yql/minikql/computation/mkql_computation_node_holders.cpp +++ b/ydb/library/yql/minikql/computation/mkql_computation_node_holders.cpp @@ -1,8 +1,8 @@ -#include "mkql_computation_node_holders.h" +#include "mkql_computation_node_holders.h" #include "mkql_computation_node_codegen.h" #include "mkql_computation_node_pack.h" #include "mkql_custom_list.h" -#include "mkql_value_builder.h" +#include "mkql_value_builder.h" #include "presort.h" #include <ydb/library/yql/minikql/mkql_node_builder.h> @@ -12,26 +12,26 @@ #include <ydb/library/yql/minikql/mkql_string_util.h> #include <ydb/library/yql/public/udf/udf_value.h> - + #include <library/cpp/containers/stack_vector/stack_vec.h> - + namespace NKikimr { namespace NMiniKQL { namespace { -class TValueDataHolder: public TComputationValue<TValueDataHolder> { -public: - TValueDataHolder(TMemoryUsageInfo* memInfo, NUdf::TUnboxedValue&& value) - : TComputationValue(memInfo) +class TValueDataHolder: public TComputationValue<TValueDataHolder> { +public: + TValueDataHolder(TMemoryUsageInfo* memInfo, NUdf::TUnboxedValue&& value) + : TComputationValue(memInfo) , Value(std::move(value)) {} - -private: + +private: const NUdf::TUnboxedValue Value; -}; - -class TDirectListHolder: public TComputationValue<TDirectListHolder> { +}; + +class TDirectListHolder: public TComputationValue<TDirectListHolder> { public: class TIterator: public TComputationValue<TIterator> { public: @@ -111,7 +111,7 @@ public: }; TDirectListHolder(TMemoryUsageInfo* memInfo, TDefaultListRepresentation&& items) - : TComputationValue(memInfo) + : TComputationValue(memInfo) , Items(std::move(items)) {} @@ -203,7 +203,7 @@ private: return new TDirectListHolder(GetMemInfo(), std::move(result)); } - NUdf::IBoxedValuePtr ToIndexDictImpl(const NUdf::IValueBuilder& builder) const override { + NUdf::IBoxedValuePtr ToIndexDictImpl(const NUdf::IValueBuilder& builder) const override { Y_UNUSED(builder); return const_cast<TDirectListHolder*>(this); } @@ -728,8 +728,8 @@ private: Y_UNUSED(builder); Y_UNUSED(count); return const_cast<TEmptyContainerHolder*>(this); - } - + } + NUdf::IBoxedValuePtr ToIndexDictImpl(const NUdf::IValueBuilder& builder) const override { Y_UNUSED(builder); return const_cast<TEmptyContainerHolder*>(this); @@ -951,7 +951,7 @@ private: mutable std::optional<TGenericPresortEncoder> Packer; }; -class TSortedDictHolder: public TComputationValue<TSortedDictHolder> { +class TSortedDictHolder: public TComputationValue<TSortedDictHolder> { public: typedef TKeyPayloadPairVector TItems; @@ -1013,17 +1013,17 @@ public: bool AtStart; }; - TSortedDictHolder( - TMemoryUsageInfo* memInfo, - TSortedDictFiller filler, + TSortedDictHolder( + TMemoryUsageInfo* memInfo, + TSortedDictFiller filler, const TKeyTypes& types, bool isTuple, - EDictSortMode mode, + EDictSortMode mode, bool eagerFill, TType* encodedType, const THolderFactory& holderFactory) - : TComputationValue(memInfo) - , Filler(filler) + : TComputationValue(memInfo) + , Filler(filler) , Types(types) , IsTuple(isTuple) , Mode(mode) @@ -1116,9 +1116,9 @@ private: Y_VERIFY_DEBUG(IsSortedUnique()); IsBuilt = true; - + if (!Items.empty()) { - MKQL_MEM_TAKE(GetMemInfo(), &Items, Items.capacity() * sizeof(TItems::value_type)); + MKQL_MEM_TAKE(GetMemInfo(), &Items, Items.capacity() * sizeof(TItems::value_type)); } } @@ -1892,7 +1892,7 @@ private: TComputationContext& CompCtx; }; -class THashedDictHolder: public TComputationValue<THashedDictHolder> { +class THashedDictHolder: public TComputationValue<THashedDictHolder> { public: template <bool NoSwap> class TIterator: public TComputationValue<TIterator<NoSwap>> { @@ -1956,8 +1956,8 @@ public: THashedDictHolder(TMemoryUsageInfo* memInfo, THashedDictFiller filler, const TKeyTypes& types, bool isTuple, bool eagerFill, TType* encodedType, const THolderFactory& holderFactory) - : TComputationValue(memInfo) - , Filler(filler) + : TComputationValue(memInfo) + , Filler(filler) , Types(types) , Map(0, TValueHasher(Types, isTuple), TValueEqual(Types, isTuple)) , IsBuilt(false) @@ -2422,7 +2422,7 @@ private: TComputationContext* Ctx; }; -class TDictNode: public TMutableComputationNode<TDictNode> { +class TDictNode: public TMutableComputationNode<TDictNode> { typedef TMutableComputationNode<TDictNode> TBaseComputation; public: TDictNode(TComputationMutables& mutables, @@ -2458,7 +2458,7 @@ public: map.emplace(std::move(key), std::move(value.second)); } }; - + return ctx.HolderFactory.CreateDirectHashedDictHolder( filler, Types, IsTuple, true, EncodedType); } @@ -2546,7 +2546,7 @@ private: const NUdf::TUnboxedValue Iter; }; -class TLimitedList: public TComputationValue<TLimitedList> { +class TLimitedList: public TComputationValue<TLimitedList> { public: class TIterator: public TComputationValue<TIterator> { public: @@ -2627,8 +2627,8 @@ public: }; TLimitedList(TMemoryUsageInfo* memInfo, NUdf::TRefCountedPtr<NUdf::IBoxedValue> parent, TMaybe<ui64> skip, TMaybe<ui64> take) - : TComputationValue(memInfo) - , Parent(parent) + : TComputationValue(memInfo) + , Parent(parent) , Skip(skip) , Take(take) { @@ -2682,7 +2682,7 @@ private: return NUdf::TUnboxedValuePod(new TIterator(GetMemInfo(), NUdf::TBoxedValueAccessor::GetListIterator(*Parent), Skip, Take)); } - NUdf::IBoxedValuePtr SkipListImpl(const NUdf::IValueBuilder& builder, ui64 count) const override { + NUdf::IBoxedValuePtr SkipListImpl(const NUdf::IValueBuilder& builder, ui64 count) const override { if (!count) { return const_cast<TLimitedList*>(this); } @@ -2711,7 +2711,7 @@ private: return new TLimitedList(GetMemInfo(), Parent, newSkip, newTake); } - NUdf::IBoxedValuePtr TakeListImpl(const NUdf::IValueBuilder& builder, ui64 count) const override { + NUdf::IBoxedValuePtr TakeListImpl(const NUdf::IValueBuilder& builder, ui64 count) const override { if (!count) { return builder.NewEmptyList().Release().AsBoxed(); } @@ -2825,7 +2825,7 @@ private: const NUdf::IBoxedValuePtr List; }; -} // namespace +} // namespace /////////////////////////////////////////////////////////////////////////////// // TDictValueBuilder @@ -2847,7 +2847,7 @@ public: { Items_.reserve(10); } - + NUdf::IDictValueBuilder& Add(NUdf::TUnboxedValue&& key, NUdf::TUnboxedValue&& value) override { Items_.emplace_back(std::move(key), std::move(value)); @@ -2990,9 +2990,9 @@ private: }; -////////////////////////////////////////////////////////////////////////////// -// THolderFactory -////////////////////////////////////////////////////////////////////////////// +////////////////////////////////////////////////////////////////////////////// +// THolderFactory +////////////////////////////////////////////////////////////////////////////// THolderFactory::THolderFactory( TAllocState& allocState, TMemoryUsageInfo& memInfo, @@ -3004,7 +3004,7 @@ THolderFactory::THolderFactory( {} NUdf::TUnboxedValuePod THolderFactory::CreateTypeHolder(TType* type) const -{ +{ return NUdf::TUnboxedValuePod(AllocateOn<TTypeHolder>(CurrentAllocState, &MemInfo, type)); } @@ -3104,9 +3104,9 @@ NUdf::TUnboxedValuePod THolderFactory::Reversed(const NUdf::TUnboxedValuePod& it NUdf::TUnboxedValuePod THolderFactory::CreateLimitedList( NUdf::IBoxedValuePtr&& parent, - TMaybe<ui64> skip, TMaybe<ui64> take, - TMaybe<ui64> knownLength) const -{ + TMaybe<ui64> skip, TMaybe<ui64> take, + TMaybe<ui64> knownLength) const +{ if (take && !take.GetRef()) { return GetEmptyContainer(); } @@ -3370,27 +3370,27 @@ NUdf::TUnboxedValuePod THolderFactory::CreateDirectSortedSetHolder( } NUdf::TUnboxedValuePod THolderFactory::CreateDirectSortedDictHolder( - TSortedDictFiller filler, + TSortedDictFiller filler, const TKeyTypes& types, bool isTuple, - EDictSortMode mode, + EDictSortMode mode, bool eagerFill, TType* encodedType) const -{ +{ return NUdf::TUnboxedValuePod(AllocateOn<TSortedDictHolder>(CurrentAllocState, &MemInfo, filler, types, isTuple, mode, eagerFill, encodedType, *this)); -} +} NUdf::TUnboxedValuePod THolderFactory::CreateDirectHashedDictHolder( - THashedDictFiller filler, + THashedDictFiller filler, const TKeyTypes& types, bool isTuple, bool eagerFill, TType* encodedType) const -{ +{ return NUdf::TUnboxedValuePod(AllocateOn<THashedDictHolder>(CurrentAllocState, &MemInfo, filler, types, isTuple, eagerFill, encodedType, *this)); -} +} NUdf::TUnboxedValuePod THolderFactory::CreateDirectHashedSetHolder( THashedSetFiller filler, @@ -3501,9 +3501,9 @@ KNOWN_PRIMITIVE_VALUE_TYPES(DEFINE_HASHED_SINGLE_FIXED_COMPACT_MAP) KNOWN_PRIMITIVE_VALUE_TYPES(DEFINE_HASHED_SINGLE_FIXED_COMPACT_MULTI_MAP) #undef DEFINE_HASHED_SINGLE_FIXED_COMPACT_MULTI_MAP -////////////////////////////////////////////////////////////////////////////// -// TNodeFactory -////////////////////////////////////////////////////////////////////////////// +////////////////////////////////////////////////////////////////////////////// +// TNodeFactory +////////////////////////////////////////////////////////////////////////////// TNodeFactory::TNodeFactory(TMemoryUsageInfo& memInfo, TComputationMutables& mutables) : MemInfo(memInfo) , Mutables(mutables) @@ -3511,35 +3511,35 @@ TNodeFactory::TNodeFactory(TMemoryUsageInfo& memInfo, TComputationMutables& muta } IComputationNode* TNodeFactory::CreateEmptyNode() const -{ +{ return new TEmptyNode(Mutables); } -IComputationNode* TNodeFactory::CreateOptionalNode(IComputationNode* item) const -{ +IComputationNode* TNodeFactory::CreateOptionalNode(IComputationNode* item) const +{ return item ? new TOptionalNode(item) : CreateImmutableNode(NUdf::TUnboxedValuePod()); } IComputationNode* TNodeFactory::CreateArrayNode(TComputationNodePtrVector&& values) const -{ +{ if (values.empty()) { return new TEmptyNode(Mutables); } return new TArrayNode(Mutables, std::move(values)); } - -IComputationNode* TNodeFactory::CreateDictNode( + +IComputationNode* TNodeFactory::CreateDictNode( std::vector<std::pair<IComputationNode*, IComputationNode*>>&& items, const TKeyTypes& types, bool isTuple, TType* encodedType) const -{ +{ if (items.empty()) { return new TEmptyNode(Mutables); } return new TDictNode(Mutables, std::move(items), types, isTuple, encodedType); -} - +} + IComputationNode* TNodeFactory::CreateVariantNode(IComputationNode* item, ui32 index) const { return new TVariantNode(Mutables, item, index); } @@ -3778,5 +3778,5 @@ NUdf::TUnboxedValuePod TPlainContainerCache::NewArray(const THolderFactory& fact return static_cast<const NUdf::TUnboxedValuePod&>(Cached[CacheIndex]); } -} // namespace NMiniKQL -} // namespace NKikimr +} // namespace NMiniKQL +} // namespace NKikimr diff --git a/ydb/library/yql/minikql/computation/mkql_computation_node_holders.h b/ydb/library/yql/minikql/computation/mkql_computation_node_holders.h index 110963b982c..4486d6f43a1 100644 --- a/ydb/library/yql/minikql/computation/mkql_computation_node_holders.h +++ b/ydb/library/yql/minikql/computation/mkql_computation_node_holders.h @@ -1,14 +1,14 @@ -#pragma once - +#pragma once + #include <ydb/library/yql/utils/hash.h> #include "mkql_computation_node_impl.h" #include "mkql_computation_node_list.h" - + #include <ydb/library/yql/minikql/aligned_page_pool.h> #include <ydb/library/yql/minikql/compact_hash.h> #include <ydb/library/yql/minikql/mkql_type_ops.h> - + #include <contrib/libs/apache/arrow/cpp/src/arrow/datum.h> #include <util/generic/maybe.h> @@ -17,7 +17,7 @@ #include <functional> #include <unordered_map> #include <unordered_set> - + #ifndef MKQL_DISABLE_CODEGEN namespace llvm { class Value; @@ -25,11 +25,11 @@ namespace llvm { } #endif -namespace NKikimr { -namespace NMiniKQL { - -class TMemoryUsageInfo; - +namespace NKikimr { +namespace NMiniKQL { + +class TMemoryUsageInfo; + #ifndef MKQL_DISABLE_CODEGEN struct TCodegenContext; #endif @@ -122,16 +122,16 @@ struct TValueEqual { TValueEqual(const TKeyTypes& types, bool isTuple) : Types(&types) , IsTuple(isTuple) - {} - + {} + bool operator()(const NUdf::TUnboxedValue& left, const NUdf::TUnboxedValue& right) const { return CompareKeys(left, right, *Types, IsTuple) == 0; - } - + } + const TKeyTypes* Types; bool IsTuple; -}; - +}; + struct TValueLess { TValueLess(const TKeyTypes& types, bool isTuple) : Types(&types) @@ -148,12 +148,12 @@ struct TValueLess { constexpr NUdf::THashType HashOfNull = ~0ULL; -struct TValueHasher { +struct TValueHasher { TValueHasher(const TKeyTypes& types, bool isTuple) : Types(&types) , IsTuple(isTuple) - {} - + {} + NUdf::THashType operator()(const NUdf::TUnboxedValuePod& value) const { if (!value) return HashOfNull; @@ -177,12 +177,12 @@ struct TValueHasher { return hash; } return NUdf::GetValueHash((*Types).front().first, value); - } - + } + const TKeyTypes* Types; bool IsTuple; -}; - +}; + template<typename T> struct TFloatHash : private std::hash<T> { std::size_t operator()(T value) const { @@ -204,7 +204,7 @@ template <typename T> using TMyEquals = std::conditional_t<std::is_floating_point<T>::value, TFloatEquals<T>, std::equal_to<T>>; constexpr float COMPACT_HASH_MAX_LOAD_FACTOR = 1.2f; - + using TValuesDictHashMap = std::unordered_map< NUdf::TUnboxedValue, NUdf::TUnboxedValue, NYql::TVaryingHash<NUdf::TUnboxedValue, TValueHasher>, TValueEqual, @@ -295,32 +295,32 @@ using THashedDictFiller = std::function<void(TValuesDictHashMap&)>; using THashedSetFiller = std::function<void(TValuesDictHashSet&)>; using TSortedDictFiller = std::function<void(TKeyPayloadPairVector&)>; using TSortedSetFiller = std::function<void(TUnboxedValueVector&)>; - -enum class EDictSortMode { - RequiresSorting, - SortedUniqueAscending, - SortedUniqueDescening -}; - -class TTypeHolder: public TComputationValue<TTypeHolder> { -public: - TTypeHolder(TMemoryUsageInfo* memInfo, TType* type) - : TComputationValue(memInfo) - , Type(type) + +enum class EDictSortMode { + RequiresSorting, + SortedUniqueAscending, + SortedUniqueDescening +}; + +class TTypeHolder: public TComputationValue<TTypeHolder> { +public: + TTypeHolder(TMemoryUsageInfo* memInfo, TType* type) + : TComputationValue(memInfo) + , Type(type) {} - + NUdf::TStringRef GetResourceTag() const override { return NUdf::TStringRef::Of("TypeHolder"); } void* GetResource() override { - return Type; - } - + return Type; + } + private: - TType* const Type; -}; - + TType* const Type; +}; + class TArrowBlock: public TComputationValue<TArrowBlock> { public: explicit TArrowBlock(TMemoryUsageInfo* memInfo, arrow::Datum&& datum) @@ -349,26 +349,26 @@ private: arrow::Datum Datum_; }; -////////////////////////////////////////////////////////////////////////////// -// THolderFactory -////////////////////////////////////////////////////////////////////////////// -class THolderFactory: private TNonCopyable -{ -public: +////////////////////////////////////////////////////////////////////////////// +// THolderFactory +////////////////////////////////////////////////////////////////////////////// +class THolderFactory: private TNonCopyable +{ +public: THolderFactory( TAllocState& allocState, TMemoryUsageInfo& memInfo, const IFunctionRegistry* functionRegistry = nullptr); - - template <typename T, typename... TArgs> + + template <typename T, typename... TArgs> NUdf::TUnboxedValuePod Create(TArgs&&... args) const { return NUdf::TUnboxedValuePod(AllocateOn<T>(CurrentAllocState, &MemInfo, std::forward<TArgs>(args)...)); - } - + } + NUdf::TUnboxedValuePod CreateTypeHolder(TType* type) const; - + NUdf::TUnboxedValuePod CreateDirectListHolder(TDefaultListRepresentation&& items) const; - + NUdf::TUnboxedValuePod CreateDirectArrayHolder(ui32 size, NUdf::TUnboxedValue*& itemsPtr) const; NUdf::TFlatDataBlockPtr CreateFlatDataBlock(ui32 initialSize, ui32 initialCapacity) const; @@ -404,20 +404,20 @@ public: TType* encodedType) const; NUdf::TUnboxedValuePod CreateDirectSortedDictHolder( - TSortedDictFiller filler, + TSortedDictFiller filler, const TKeyTypes& types, bool isTuple, - EDictSortMode mode, + EDictSortMode mode, bool eagerFill, TType* encodedType) const; - + NUdf::TUnboxedValuePod CreateDirectHashedDictHolder( - THashedDictFiller filler, + THashedDictFiller filler, const TKeyTypes& types, bool isTuple, bool eagerFill, TType* encodedType) const; - + NUdf::TUnboxedValuePod CreateDirectHashedSetHolder( THashedSetFiller filler, const TKeyTypes& types, @@ -462,12 +462,12 @@ public: NUdf::TUnboxedValuePod Cloned(const NUdf::TUnboxedValuePod& it) const; NUdf::TUnboxedValuePod Reversed(const NUdf::TUnboxedValuePod& it) const; - + NUdf::TUnboxedValuePod CreateLimitedList( NUdf::IBoxedValuePtr&& parent, - TMaybe<ui64> skip, TMaybe<ui64> take, - TMaybe<ui64> knownLength) const; - + TMaybe<ui64> skip, TMaybe<ui64> take, + TMaybe<ui64> knownLength) const; + NUdf::TUnboxedValuePod ReverseList(const NUdf::IValueBuilder* builder, const NUdf::TUnboxedValuePod list) const; NUdf::TUnboxedValuePod SkipList(const NUdf::IValueBuilder* builder, const NUdf::TUnboxedValuePod list, ui64 count) const; NUdf::TUnboxedValuePod TakeList(const NUdf::IValueBuilder* builder, const NUdf::TUnboxedValuePod list, ui64 count) const; @@ -488,10 +488,10 @@ public: NUdf::TUnboxedValuePod CloneArray(const NUdf::TUnboxedValuePod list, NUdf::TUnboxedValue*& itemsPtr) const; - TMemoryUsageInfo& GetMemInfo() const { - return MemInfo; - } - + TMemoryUsageInfo& GetMemInfo() const { + return MemInfo; + } + NUdf::TUnboxedValuePod GetEmptyContainer() const { return static_cast<const NUdf::TUnboxedValuePod&>(EmptyContainer); } @@ -518,13 +518,13 @@ public: NUdf::TUnboxedValuePod ExtendList(NUdf::TUnboxedValue* data, ui64 size) const; NUdf::TUnboxedValuePod ExtendStream(NUdf::TUnboxedValue* data, ui64 size) const; -private: +private: TAllocState* const CurrentAllocState; - TMemoryUsageInfo& MemInfo; + TMemoryUsageInfo& MemInfo; const IFunctionRegistry* const FunctionRegistry; const NUdf::TUnboxedValue EmptyContainer; -}; - +}; + constexpr const ui32 STEP_FOR_RSS_CHECK = 100U; // Returns true if current usage delta exceeds the memory limit @@ -542,35 +542,35 @@ inline bool TComputationContext::CheckAdjustedMemLimit(ui64 memLimit, ui64 initM return currentMemUsage * UsageAdjustor >= initMemUsage + memLimit; } -////////////////////////////////////////////////////////////////////////////// -// TNodeFactory -////////////////////////////////////////////////////////////////////////////// -class TNodeFactory: private TNonCopyable -{ -public: +////////////////////////////////////////////////////////////////////////////// +// TNodeFactory +////////////////////////////////////////////////////////////////////////////// +class TNodeFactory: private TNonCopyable +{ +public: TNodeFactory(TMemoryUsageInfo& memInfo, TComputationMutables& mutables); - + IComputationNode* CreateTypeNode(TType* type) const; - + IComputationNode* CreateImmutableNode(NUdf::TUnboxedValue&& value) const; - + IComputationNode* CreateEmptyNode() const; - + IComputationNode* CreateArrayNode(TComputationNodePtrVector&& items) const; - IComputationNode* CreateOptionalNode(IComputationNode* item) const; - - IComputationNode* CreateDictNode( + IComputationNode* CreateOptionalNode(IComputationNode* item) const; + + IComputationNode* CreateDictNode( std::vector<std::pair<IComputationNode*, IComputationNode*>>&& items, const TKeyTypes& types, bool isTuple, TType* encodedType) const; - + IComputationNode* CreateVariantNode(IComputationNode* item, ui32 index) const; - -private: - TMemoryUsageInfo& MemInfo; + +private: + TMemoryUsageInfo& MemInfo; TComputationMutables& Mutables; -}; - +}; + void GetDictionaryKeyTypes(TType* keyType, TKeyTypes& types, bool& isTuple, bool& encoded); struct TContainerCacheOnContext : private TNonCopyable { diff --git a/ydb/library/yql/minikql/computation/mkql_computation_node_impl.cpp b/ydb/library/yql/minikql/computation/mkql_computation_node_impl.cpp index e035d54374c..15777c32880 100644 --- a/ydb/library/yql/minikql/computation/mkql_computation_node_impl.cpp +++ b/ydb/library/yql/minikql/computation/mkql_computation_node_impl.cpp @@ -1,5 +1,5 @@ #include "mkql_computation_node_impl.h" - + #include "ydb/library/yql/minikql/mkql_string_util.h" namespace NKikimr { @@ -30,14 +30,14 @@ TUnboxedImmutableComputationNode::TUnboxedImmutableComputationNode(TMemoryUsageI : MemInfo(memInfo) , UnboxedValue(std::move(value)) , RepresentationKind(UnboxedValue.HasValue() ? (UnboxedValue.IsBoxed() ? EValueRepresentation::Boxed : (UnboxedValue.IsString() ? EValueRepresentation::String : EValueRepresentation::Embedded)) : EValueRepresentation::Embedded) -{ +{ MKQL_MEM_TAKE(MemInfo, this, sizeof(*this), "TOwnerType: " << DebugString()); } TUnboxedImmutableComputationNode::~TUnboxedImmutableComputationNode() { MKQL_MEM_RETURN(MemInfo, this, sizeof(*this)); } - + NUdf::TUnboxedValue TUnboxedImmutableComputationNode::GetValue(TComputationContext& compCtx) const { Y_UNUSED(compCtx); if (RepresentationKind == EValueRepresentation::String) { diff --git a/ydb/library/yql/minikql/computation/mkql_computation_node_impl.h b/ydb/library/yql/minikql/computation/mkql_computation_node_impl.h index bfe160231c4..0a63ca9d349 100644 --- a/ydb/library/yql/minikql/computation/mkql_computation_node_impl.h +++ b/ydb/library/yql/minikql/computation/mkql_computation_node_impl.h @@ -1,10 +1,10 @@ #pragma once - + #include "mkql_computation_node.h" #include <ydb/library/yql/minikql/mkql_alloc.h> #include <ydb/library/yql/public/udf/udf_value.h> - + #include <util/system/type_name.h> namespace NKikimr { @@ -35,12 +35,12 @@ private: }; class TUnboxedImmutableComputationNode: public TRefCountedComputationNode<IComputationNode> -{ +{ public: TUnboxedImmutableComputationNode(TMemoryUsageInfo* memInfo, NUdf::TUnboxedValue&& value); ~TUnboxedImmutableComputationNode(); - + private: void InitNode(TComputationContext&) const override {} @@ -103,7 +103,7 @@ private: }; class TExternalComputationNode: public TStatefulComputationNode<IComputationExternalNode> -{ +{ public: TExternalComputationNode(TComputationMutables& mutables, EValueRepresentation kind = EValueRepresentation::Any); @@ -143,7 +143,7 @@ protected: template <typename TDerived, bool SerializableState = false> class TStatefulSourceComputationNode: public TStatefulComputationNode<IComputationNode, SerializableState> -{ +{ using TStatefulComputationNode = TStatefulComputationNode<IComputationNode, SerializableState>; private: bool IsTemporaryValue() const final { @@ -197,7 +197,7 @@ protected: } TString DebugString() const override { - return TypeName<TDerived>(); + return TypeName<TDerived>(); } mutable std::unordered_set<const IComputationNode*> Sources; // TODO: remove const and mutable. @@ -791,21 +791,21 @@ protected: template <typename TDerived, typename TBase = NUdf::IBoxedValue> class TComputationValue: public TBase, public TWithMiniKQLAlloc -{ +{ public: template <typename... Args> TComputationValue(TMemoryUsageInfo* memInfo, Args&&... args) : TBase(std::forward<Args>(args)...) { - M_.MemInfo = memInfo; - MKQL_MEM_TAKE(memInfo, this, sizeof(TDerived), - "TDerived: " << DebugString()); - } - - ~TComputationValue() { + M_.MemInfo = memInfo; + MKQL_MEM_TAKE(memInfo, this, sizeof(TDerived), + "TDerived: " << DebugString()); + } + + ~TComputationValue() { MKQL_MEM_RETURN(GetMemInfo(), this, sizeof(TDerived)); - } - + } + private: bool HasFastListLength() const override { ThrowNotSupported(__func__); @@ -814,7 +814,7 @@ private: ui64 GetListLength() const override { ThrowNotSupported(__func__); - return 0; + return 0; } ui64 GetEstimatedListLength() const override { @@ -828,7 +828,7 @@ private: } const NUdf::TOpaqueListRepresentation* GetListRepresentation() const override { - return nullptr; + return nullptr; } NUdf::IBoxedValuePtr ReverseListImpl(const NUdf::IValueBuilder& builder) const override { @@ -855,12 +855,12 @@ private: ui64 GetDictLength() const override { ThrowNotSupported(__func__); - return 0; + return 0; } bool HasDictItems() const override { ThrowNotSupported(__func__); - return false; + return false; } NUdf::TStringRef GetResourceTag() const override { @@ -920,14 +920,14 @@ private: } NUdf::TUnboxedValue Run( - const NUdf::IValueBuilder* valueBuilder, + const NUdf::IValueBuilder* valueBuilder, const NUdf::TUnboxedValuePod* args) const override - { + { Y_UNUSED(valueBuilder); Y_UNUSED(args); ThrowNotSupported(__func__); - return {}; - } + return {}; + } bool Skip() override { NUdf::TUnboxedValue stub; @@ -1021,14 +1021,14 @@ private: public: TString DebugString() const { - return TypeName<TDerived>(); - } - -protected: - inline TMemoryUsageInfo* GetMemInfo() const { - return static_cast<TMemoryUsageInfo*>(M_.MemInfo); - } - + return TypeName<TDerived>(); + } + +protected: + inline TMemoryUsageInfo* GetMemInfo() const { + return static_cast<TMemoryUsageInfo*>(M_.MemInfo); + } + [[noreturn]] void ThrowNotSupported(const char* func) const { THROW yexception() << "Unsupported access to '" << func << "' method of: " << TypeName(*this); } diff --git a/ydb/library/yql/minikql/computation/mkql_computation_node_list.h b/ydb/library/yql/minikql/computation/mkql_computation_node_list.h index f7f5f9999a6..d87fe871bf0 100644 --- a/ydb/library/yql/minikql/computation/mkql_computation_node_list.h +++ b/ydb/library/yql/minikql/computation/mkql_computation_node_list.h @@ -1,11 +1,11 @@ #pragma once - + #include <ydb/library/yql/minikql/defs.h> #include <ydb/library/yql/minikql/mkql_alloc.h> #include <ydb/library/yql/public/udf/udf_value.h> - - + + namespace NKikimr { namespace NMiniKQL { template <typename T> diff --git a/ydb/library/yql/minikql/computation/mkql_value_builder.cpp b/ydb/library/yql/minikql/computation/mkql_value_builder.cpp index eff0577861d..ad476617158 100644 --- a/ydb/library/yql/minikql/computation/mkql_value_builder.cpp +++ b/ydb/library/yql/minikql/computation/mkql_value_builder.cpp @@ -1,18 +1,18 @@ -#include "mkql_value_builder.h" +#include "mkql_value_builder.h" #include "mkql_validate.h" - + #include <ydb/library/yql/minikql/mkql_node_cast.h> #include <ydb/library/yql/minikql/mkql_string_util.h> #include <library/cpp/yson/node/node_io.h> #include <util/system/env.h> -namespace NKikimr { -namespace NMiniKQL { - -/////////////////////////////////////////////////////////////////////////////// -// TDefaultValueBuilder -/////////////////////////////////////////////////////////////////////////////// +namespace NKikimr { +namespace NMiniKQL { + +/////////////////////////////////////////////////////////////////////////////// +// TDefaultValueBuilder +/////////////////////////////////////////////////////////////////////////////// TDefaultValueBuilder::TDefaultValueBuilder(const THolderFactory& holderFactory, NUdf::EValidatePolicy policy) : HolderFactory_(holderFactory) , Policy_(policy) @@ -125,7 +125,7 @@ NUdf::TUnboxedValue TDefaultValueBuilder::NewArray(ui32 count, NUdf::TUnboxedVal NUdf::TUnboxedValue TDefaultValueBuilder::NewVariant(ui32 index, NUdf::TUnboxedValue&& value) const { return HolderFactory_.CreateVariantHolder(value.Release(), index); } - + NUdf::IDictValueBuilder::TPtr TDefaultValueBuilder::NewDict(const NUdf::TType* dictType, ui32 flags) const { return HolderFactory_.NewDict(dictType, flags); diff --git a/ydb/library/yql/minikql/computation/mkql_value_builder.h b/ydb/library/yql/minikql/computation/mkql_value_builder.h index 1d05e87890b..b293b263a3d 100644 --- a/ydb/library/yql/minikql/computation/mkql_value_builder.h +++ b/ydb/library/yql/minikql/computation/mkql_value_builder.h @@ -1,26 +1,26 @@ -#pragma once - +#pragma once + #include "mkql_computation_node_holders.h" #include <ydb/library/yql/public/udf/udf_value_builder.h> #include <ydb/library/yql/public/udf/udf_validate.h> - -#include <util/generic/noncopyable.h> + +#include <util/generic/noncopyable.h> #include <util/memory/pool.h> - + #include <ydb/library/yql/minikql/mkql_node.h> #include <ydb/library/yql/minikql/mkql_terminator.h> - -namespace NKikimr { -namespace NMiniKQL { - -/////////////////////////////////////////////////////////////////////////////// -// TDefaultValueBuilder -/////////////////////////////////////////////////////////////////////////////// + +namespace NKikimr { +namespace NMiniKQL { + +/////////////////////////////////////////////////////////////////////////////// +// TDefaultValueBuilder +/////////////////////////////////////////////////////////////////////////////// class TDefaultValueBuilder final: public NUdf::IValueBuilder, private TNonCopyable, public ITerminator, public NUdf::IDateBuilder -{ -public: +{ +public: TDefaultValueBuilder(const THolderFactory& holderFactory, NUdf::EValidatePolicy policy = NUdf::EValidatePolicy::Fail); void SetSecureParamsProvider(const NUdf::ISecureParamsProvider* provider); @@ -93,6 +93,6 @@ private: const NUdf::TSourcePosition** CalleePositionPtr_ = nullptr; mutable bool Rethrow_ = false; }; - + } // namespace NMiniKQL } // namespace Nkikimr diff --git a/ydb/library/yql/minikql/computation/ya.make b/ydb/library/yql/minikql/computation/ya.make index 990e97979c3..ae419b3b232 100644 --- a/ydb/library/yql/minikql/computation/ya.make +++ b/ydb/library/yql/minikql/computation/ya.make @@ -8,7 +8,7 @@ OWNER( ) SRCS( - mkql_computation_node.cpp + mkql_computation_node.cpp mkql_computation_node.h mkql_computation_node_codegen.h mkql_computation_node_codegen.cpp @@ -19,14 +19,14 @@ SRCS( mkql_computation_node_impl.h mkql_computation_node_impl.cpp mkql_computation_node_list.h - mkql_computation_node_pack.cpp + mkql_computation_node_pack.cpp mkql_computation_node_pack.h mkql_custom_list.cpp mkql_custom_list.h mkql_validate.cpp mkql_validate.h - mkql_value_builder.cpp - mkql_value_builder.h + mkql_value_builder.cpp + mkql_value_builder.h presort.h presort.cpp ) diff --git a/ydb/library/yql/minikql/defs.h b/ydb/library/yql/minikql/defs.h index ba2df2660c7..3e332363c6c 100644 --- a/ydb/library/yql/minikql/defs.h +++ b/ydb/library/yql/minikql/defs.h @@ -1,28 +1,28 @@ #pragma once - + // unique tag to fix pragma once gcc glueing: ./ydb/library/yql/minikql/defs.h -#include <util/system/compiler.h> +#include <util/system/compiler.h> #include <util/generic/array_ref.h> -#include <util/generic/yexception.h> - -#define THROW ::NKikimr::TThrowable() , __LOCATION__ + - +#include <util/generic/yexception.h> + +#define THROW ::NKikimr::TThrowable() , __LOCATION__ + + #define MKQL_ENSURE(condition, message) \ do { \ if (Y_UNLIKELY(!(condition))) { \ (THROW yexception() << __FUNCTION__ << "(): requirement " \ << #condition << " failed. " << message); \ } \ - } while (0) - + } while (0) + #define MKQL_ENSURE_WITH_LOC(location, condition, message) \ do { \ if (Y_UNLIKELY(!(condition))) { \ ThrowException(location + yexception() << message); \ } \ - } while (0) - + } while (0) + #define MKQL_ENSURE_S(condition, ...) \ do { \ if (Y_UNLIKELY(!(condition))) { \ @@ -32,21 +32,21 @@ } while (0) namespace NKikimr { - -template <typename T> + +template <typename T> [[noreturn]] void ThrowException(T&& e) -{ - throw e; +{ + throw e; } - -struct TThrowable -{ - template <typename T> + +struct TThrowable +{ + template <typename T> [[noreturn]] void operator,(T&& e) { - ThrowException(e); - } -}; - + ThrowException(e); + } +}; + typedef #ifdef _win_ struct { ui64 Data, Meta; } @@ -55,4 +55,4 @@ unsigned __int128 #endif TRawUV; -} +} diff --git a/ydb/library/yql/minikql/invoke_builtins/mkql_builtins.cpp b/ydb/library/yql/minikql/invoke_builtins/mkql_builtins.cpp index 7317611d695..ab991842f1a 100644 --- a/ydb/library/yql/minikql/invoke_builtins/mkql_builtins.cpp +++ b/ydb/library/yql/minikql/invoke_builtins/mkql_builtins.cpp @@ -1,17 +1,17 @@ #include "mkql_builtins_impl.h" #include "mkql_builtins_compare.h" -#include <util/digest/murmur.h> +#include <util/digest/murmur.h> #include <util/generic/yexception.h> #include <util/generic/maybe.h> - + #include <algorithm> -namespace NKikimr { -namespace NMiniKQL { - -namespace { - +namespace NKikimr { +namespace NMiniKQL { + +namespace { + void RegisterDefaultOperations(IBuiltinFunctionRegistry& registry) { RegisterAdd(registry); RegisterAggrAdd(registry); @@ -52,68 +52,68 @@ void RegisterDefaultOperations(IBuiltinFunctionRegistry& registry) { RegisterLessOrEqual(registry); RegisterGreater(registry); RegisterGreaterOrEqual(registry); -} - +} + void PrintType(NUdf::TDataTypeId schemeType, bool isOptional, IOutputStream& out) -{ +{ const auto slot = NUdf::FindDataSlot(schemeType); out << (slot ? NUdf::GetDataTypeInfo(*slot).Name : "unknown"); - - if (isOptional) { + + if (isOptional) { out << '?'; - } -} - -void PrintFunctionSignature( + } +} + +void PrintFunctionSignature( const std::string_view& funcName, - const TFunctionDescriptor& desc, + const TFunctionDescriptor& desc, IOutputStream& out) -{ +{ const auto* param = desc.ResultAndArgs; - out << '\t'; - - // print results type + out << '\t'; + + // print results type PrintType(param->SchemeType, param->IsNullable(), out); - ++param; - - // print function name and args types - out << ' ' << funcName << '('; - while (param->SchemeType != 0) { + ++param; + + // print function name and args types + out << ' ' << funcName << '('; + while (param->SchemeType != 0) { PrintType(param->SchemeType, param->IsNullable(), out); - ++param; - if (param->SchemeType != 0) { - out << ", "; - } - } + ++param; + if (param->SchemeType != 0) { + out << ", "; + } + } out << ')'; -} - -bool IsArgumentsMatch( - const TFunctionParamMetadata* paramsMetadata, +} + +bool IsArgumentsMatch( + const TFunctionParamMetadata* paramsMetadata, const std::pair<NUdf::TDataTypeId, bool>* argTypes, size_t argTypesCount) -{ - size_t index = 0; - - while (paramsMetadata->SchemeType) { - if (index >= argTypesCount) { - return false; - } - +{ + size_t index = 0; + + while (paramsMetadata->SchemeType) { + if (index >= argTypesCount) { + return false; + } + if (argTypes[index].first != paramsMetadata->SchemeType) { - return false; - } - + return false; + } + if (argTypes[index].second != paramsMetadata->IsNullable()) { return false; } - ++paramsMetadata; - ++index; - } - - return index == argTypesCount; -} - + ++paramsMetadata; + ++index; + } + + return index == argTypesCount; +} + ////////////////////////////////////////////////////////////////////////////// // TBuiltinFunctionRegistry ////////////////////////////////////////////////////////////////////////////// @@ -121,7 +121,7 @@ class TBuiltinFunctionRegistry: public IBuiltinFunctionRegistry { public: TBuiltinFunctionRegistry(); - + private: TFunctionDescriptor GetBuiltin(const std::string_view& name, const std::pair<NUdf::TDataTypeId, bool>* argTypes, size_t argTypesCount) const final; @@ -150,23 +150,23 @@ private: }; TBuiltinFunctionRegistry::TBuiltinFunctionRegistry() -{ - RegisterDefaultOperations(*this); - CalculateMetadataEtag(); -} - +{ + RegisterDefaultOperations(*this); + CalculateMetadataEtag(); +} + void TBuiltinFunctionRegistry::Register(const std::string_view& name, const TFunctionDescriptor& description) -{ +{ Functions[TString(name)].push_back(description); -} - +} + void TBuiltinFunctionRegistry::RegisterAll(TFunctionsMap&& functions, TFunctionParamMetadataList&& arguments) -{ - Functions = std::move(functions); - ArgumentsMetadata = std::move(arguments); - CalculateMetadataEtag(); -} - +{ + Functions = std::move(functions); + ArgumentsMetadata = std::move(arguments); + CalculateMetadataEtag(); +} + const TFunctionsMap& TBuiltinFunctionRegistry::GetFunctions() const { return Functions; @@ -177,27 +177,27 @@ const TDescriptionList& TBuiltinFunctionRegistry::FindCandidates(const std::stri return it->second; ythrow yexception() << "Not found builtin function: '" << name << "' in " << Functions.size() << " total."; -} - +} + std::optional<TFunctionDescriptor> TBuiltinFunctionRegistry::FindBuiltin(const std::string_view& name, const std::pair<NUdf::TDataTypeId, bool>* argTypes, size_t argTypesCount) const -{ +{ for (const auto& desc: FindCandidates(name)) { if (IsArgumentsMatch(desc.ResultAndArgs, argTypes, argTypesCount)) { - return desc; - } - } - + return desc; + } + } + return std::nullopt; -} - +} + TFunctionDescriptor TBuiltinFunctionRegistry::GetBuiltin(const std::string_view& name, const std::pair<NUdf::TDataTypeId, bool>* argTypes, size_t argTypesCount) const -{ +{ if (const auto desc = FindBuiltin(name, argTypes, argTypesCount)) { - return *desc; - } - - TStringStream ss; + return *desc; + } + + TStringStream ss; PrintType(argTypes[0].first, argTypes[0].second, ss); ss << ' ' << name << '('; for (size_t i = 1U; i < argTypesCount; i++) { @@ -205,77 +205,77 @@ TFunctionDescriptor TBuiltinFunctionRegistry::GetBuiltin(const std::string_view& ss << ", "; } PrintType(argTypes[i].first, argTypes[i].second, ss); - } - ss << ").\nCandidates are: [\n"; + } + ss << ").\nCandidates are: [\n"; ui32 i = 0; - for (const TFunctionDescriptor& desc: FindCandidates(name)) { + for (const TFunctionDescriptor& desc: FindCandidates(name)) { PrintFunctionSignature(name, desc, ss); - ss << '\n'; + ss << '\n'; if (++i > 32) { ss << "\t...\n"; break; } - } - ss << ']'; - + } + ss << ']'; + ythrow yexception() << "Unsupported builtin function: " << ss.Str(); -} - +} + bool TBuiltinFunctionRegistry::HasBuiltin(const std::string_view& name) const -{ +{ return Functions.find(TString(name)) != Functions.cend(); -} - -void TBuiltinFunctionRegistry::CalculateMetadataEtag() { +} + +void TBuiltinFunctionRegistry::CalculateMetadataEtag() { using TFunctionPair = std::pair<std::string_view, const TDescriptionList*>; - + std::vector<TFunctionPair> operations; for (const auto& func : Functions) { operations.emplace_back(func.first, &func.second); - } - + } + std::sort(operations.begin(), operations.end(), [](const TFunctionPair& x, const TFunctionPair& y) { return x.first < y.first; - }); - - ui64 hash = 0; - - for (const auto& op : operations) { + }); + + ui64 hash = 0; + + for (const auto& op : operations) { const ui64 nameLength = op.first.size(); - hash = MurmurHash<ui64>(&nameLength, sizeof(nameLength), hash); + hash = MurmurHash<ui64>(&nameLength, sizeof(nameLength), hash); hash = MurmurHash<ui64>(op.first.data(), op.first.size(), hash); const auto& descriptions = *op.second; - const ui64 descriptionCount = descriptions.size(); - hash = MurmurHash<ui64>(&descriptionCount, sizeof(descriptionCount), hash); - for (const auto& description : descriptions) { + const ui64 descriptionCount = descriptions.size(); + hash = MurmurHash<ui64>(&descriptionCount, sizeof(descriptionCount), hash); + for (const auto& description : descriptions) { for (const auto* args = description.ResultAndArgs; args->SchemeType; ++args) { - hash = MurmurHash<ui64>(args, sizeof(*args), hash); - } - } - } - - MetadataEtag = hash; -} - -ui64 TBuiltinFunctionRegistry::GetMetadataEtag() const -{ - return *MetadataEtag; -} - + hash = MurmurHash<ui64>(args, sizeof(*args), hash); + } + } + } + + MetadataEtag = hash; +} + +ui64 TBuiltinFunctionRegistry::GetMetadataEtag() const +{ + return *MetadataEtag; +} + void TBuiltinFunctionRegistry::PrintInfoTo(IOutputStream& out) const -{ - for (const auto& f: Functions) { - out << f.first << ": [\n"; - - for (const TFunctionDescriptor& desc: f.second) { +{ + for (const auto& f: Functions) { + out << f.first << ": [\n"; + + for (const TFunctionDescriptor& desc: f.second) { PrintFunctionSignature(f.first, desc, out); - out << '\n'; - } - - out << "]\n\n"; - } -} - + out << '\n'; + } + + out << "]\n\n"; + } +} + } // namespace IBuiltinFunctionRegistry::TPtr CreateBuiltinRegistry() { @@ -283,5 +283,5 @@ IBuiltinFunctionRegistry::TPtr CreateBuiltinRegistry() { } -} // namespace NMiniKQL -} // namespace NKikimr +} // namespace NMiniKQL +} // namespace NKikimr diff --git a/ydb/library/yql/minikql/invoke_builtins/mkql_builtins.h b/ydb/library/yql/minikql/invoke_builtins/mkql_builtins.h index fc84401faf2..3cec6d80c18 100644 --- a/ydb/library/yql/minikql/invoke_builtins/mkql_builtins.h +++ b/ydb/library/yql/minikql/invoke_builtins/mkql_builtins.h @@ -1,11 +1,11 @@ -#pragma once - +#pragma once + #include <ydb/library/yql/minikql/mkql_function_metadata.h> - -namespace NKikimr { -namespace NMiniKQL { - + +namespace NKikimr { +namespace NMiniKQL { + IBuiltinFunctionRegistry::TPtr CreateBuiltinRegistry(); - -} // namspace NMiniKQL -} // namspace NKikimr + +} // namspace NMiniKQL +} // namspace NKikimr diff --git a/ydb/library/yql/minikql/invoke_builtins/mkql_builtins_ut.cpp b/ydb/library/yql/minikql/invoke_builtins/mkql_builtins_ut.cpp index b7e2526e274..9420223b74e 100644 --- a/ydb/library/yql/minikql/invoke_builtins/mkql_builtins_ut.cpp +++ b/ydb/library/yql/minikql/invoke_builtins/mkql_builtins_ut.cpp @@ -1,15 +1,15 @@ -#include "mkql_builtins.h" +#include "mkql_builtins.h" #include <ydb/library/yql/public/udf/udf_value.h> - + #include <library/cpp/testing/unittest/registar.h> #include <array> namespace NKikimr { -namespace NMiniKQL { +namespace NMiniKQL { -static TFunctionParamMetadata AddUi32Metadata[] = { +static TFunctionParamMetadata AddUi32Metadata[] = { { NUdf::TDataType<ui32>::Id, 0 }, // result { NUdf::TDataType<ui32>::Id, 0 }, // first arg { NUdf::TDataType<ui32>::Id, 0 }, // second arg @@ -17,9 +17,9 @@ static TFunctionParamMetadata AddUi32Metadata[] = { }; static NUdf::TUnboxedValuePod AddUi32(const NUdf::TUnboxedValuePod* args) -{ - const ui32 first = args[0].Get<ui32>(); - const ui32 second = args[1].Get<ui32>(); +{ + const ui32 first = args[0].Get<ui32>(); + const ui32 second = args[1].Get<ui32>(); return NUdf::TUnboxedValuePod(first + second); } @@ -27,7 +27,7 @@ Y_UNIT_TEST_SUITE(TFunctionRegistryTest) { Y_UNIT_TEST(TestRegistration) { const auto functionRegistry = CreateBuiltinRegistry(); functionRegistry->Register("MyAdd", TFunctionDescriptor(AddUi32Metadata, &AddUi32)); - + const std::array<TArgType, 3U> argTypes ={{{ NUdf::TDataType<ui32>::Id, false }, { NUdf::TDataType<ui32>::Id, false }, { NUdf::TDataType<ui32>::Id, false }}}; auto op = functionRegistry->GetBuiltin("MyAdd", argTypes.data(), argTypes.size()); UNIT_ASSERT_EQUAL(op.Function, &AddUi32); @@ -38,14 +38,14 @@ Y_UNIT_TEST_SUITE(TFunctionRegistryTest) { UNIT_ASSERT_EQUAL(op.ResultAndArgs[2].SchemeType, NUdf::TDataType<ui32>::Id); UNIT_ASSERT_EQUAL(op.ResultAndArgs[2].Flags, 0); UNIT_ASSERT_EQUAL(op.ResultAndArgs[3].SchemeType, 0); - + const NUdf::TUnboxedValuePod args[2] = {NUdf::TUnboxedValuePod(ui32(2)), NUdf::TUnboxedValuePod(ui32(3))}; - + auto result = op.Function(&args[0]); - UNIT_ASSERT_EQUAL(result.Get<ui32>(), 5); + UNIT_ASSERT_EQUAL(result.Get<ui32>(), 5); } } -} // namespace NMiniKQL +} // namespace NMiniKQL -} // namespace NKikimr +} // namespace NKikimr diff --git a/ydb/library/yql/minikql/mkql_function_registry.cpp b/ydb/library/yql/minikql/mkql_function_registry.cpp index 6f9e1a5fe5d..d3545ad138d 100644 --- a/ydb/library/yql/minikql/mkql_function_registry.cpp +++ b/ydb/library/yql/minikql/mkql_function_registry.cpp @@ -1,135 +1,135 @@ -#include "mkql_function_registry.h" -#include "mkql_utils.h" -#include "mkql_type_builder.h" +#include "mkql_function_registry.h" +#include "mkql_utils.h" +#include "mkql_type_builder.h" #include <ydb/library/yql/public/udf/udf_static_registry.h> - -#include <util/folder/iterator.h> -#include <util/folder/dirut.h> + +#include <util/folder/iterator.h> +#include <util/folder/dirut.h> #include <util/folder/path.h> -#include <util/system/dynlib.h> -#include <util/stream/str.h> -#include <util/string/builder.h> +#include <util/system/dynlib.h> +#include <util/stream/str.h> +#include <util/string/builder.h> #include <util/string/split.h> -namespace { +namespace { + +using namespace NKikimr; +using namespace NMiniKQL; -using namespace NKikimr; -using namespace NMiniKQL; - -const char MODULE_NAME_DELIMITER = '.'; -const char* RegisterFuncName = "Register"; -const char* AbiVersionFuncName = "AbiVersion"; +const char MODULE_NAME_DELIMITER = '.'; +const char* RegisterFuncName = "Register"; +const char* AbiVersionFuncName = "AbiVersion"; #if defined(_win_) || defined(_darwin_) const char* BindSymbolsFuncName = "BindSymbols"; #endif const char* SetBackTraceCallbackName = "SetBackTraceCallback"; - -////////////////////////////////////////////////////////////////////////////// -// TMutableFunctionRegistry -////////////////////////////////////////////////////////////////////////////// -class TMutableFunctionRegistry: public IMutableFunctionRegistry -{ - struct TUdfModule { + +////////////////////////////////////////////////////////////////////////////// +// TMutableFunctionRegistry +////////////////////////////////////////////////////////////////////////////// +class TMutableFunctionRegistry: public IMutableFunctionRegistry +{ + struct TUdfModule { TString LibraryPath; std::shared_ptr<NUdf::IUdfModule> Impl; - }; - + }; + using TUdfModulesMap = THashMap<TString, TUdfModule>; - - struct TUdfLibrary: public TThrRefBase { + + struct TUdfLibrary: public TThrRefBase { ui32 AbiVersion = 0; - TDynamicLibrary Lib; + TDynamicLibrary Lib; TUdfLibrary() { } - }; - using TUdfLibraryPtr = TIntrusivePtr<TUdfLibrary>; - - class TUdfModuleLoader: public NUdf::IRegistrator { - public: - TUdfModuleLoader( - TUdfModulesMap& modulesMap, + }; + using TUdfLibraryPtr = TIntrusivePtr<TUdfLibrary>; + + class TUdfModuleLoader: public NUdf::IRegistrator { + public: + TUdfModuleLoader( + TUdfModulesMap& modulesMap, const TString& libraryPath, - const TUdfModuleRemappings& remappings, - ui32 abiVersion) - : ModulesMap(modulesMap) - , LibraryPath(libraryPath) - , Remappings(remappings) - , AbiVersion(NUdf::AbiVersionToStr(abiVersion)) - { - } - - void AddModule( - const NUdf::TStringRef& name, - NUdf::TUniquePtr<NUdf::IUdfModule> module) override - { + const TUdfModuleRemappings& remappings, + ui32 abiVersion) + : ModulesMap(modulesMap) + , LibraryPath(libraryPath) + , Remappings(remappings) + , AbiVersion(NUdf::AbiVersionToStr(abiVersion)) + { + } + + void AddModule( + const NUdf::TStringRef& name, + NUdf::TUniquePtr<NUdf::IUdfModule> module) override + { Y_VERIFY_DEBUG(module, "Module is empty"); - - if (!HasError()) { - TUdfModule m; - m.LibraryPath = LibraryPath; + + if (!HasError()) { + TUdfModule m; + m.LibraryPath = LibraryPath; m.Impl.reset(module.Release()); - - auto it = Remappings.find(name); + + auto it = Remappings.find(name); const TString& newName = (it == Remappings.end()) ? TString(name) - : it->second; - - auto i = ModulesMap.insert({ newName, std::move(m) }); - if (!i.second) { - TUdfModule* oldModule = ModulesMap.FindPtr(newName); + : it->second; + + auto i = ModulesMap.insert({ newName, std::move(m) }); + if (!i.second) { + TUdfModule* oldModule = ModulesMap.FindPtr(newName); Y_VERIFY_DEBUG(oldModule != nullptr); - Error = (TStringBuilder() - << "UDF module duplication: name " << TStringBuf(name) - << ", already loaded from " << oldModule->LibraryPath - << ", trying to load from " << LibraryPath); - } - } - } - + Error = (TStringBuilder() + << "UDF module duplication: name " << TStringBuf(name) + << ", already loaded from " << oldModule->LibraryPath + << ", trying to load from " << LibraryPath); + } + } + } + const TString& GetError() const { return Error; } bool HasError() const { return !Error.empty(); } - - private: - TUdfModulesMap& ModulesMap; + + private: + TUdfModulesMap& ModulesMap; const TString LibraryPath; - const TUdfModuleRemappings& Remappings; + const TUdfModuleRemappings& Remappings; const TString AbiVersion; TString Error; - }; - -public: + }; + +public: TMutableFunctionRegistry(IBuiltinFunctionRegistry::TPtr builtins) : Builtins_(std::move(builtins)) - { - } - - TMutableFunctionRegistry(const TMutableFunctionRegistry& rhs) - : Builtins_(rhs.Builtins_) - , LoadedLibraries_(rhs.LoadedLibraries_) - , UdfModules_(rhs.UdfModules_) + { + } + + TMutableFunctionRegistry(const TMutableFunctionRegistry& rhs) + : Builtins_(rhs.Builtins_) + , LoadedLibraries_(rhs.LoadedLibraries_) + , UdfModules_(rhs.UdfModules_) , SupportsSizedAllocators_(rhs.SupportsSizedAllocators_) - { - } - + { + } + ~TMutableFunctionRegistry() { } void AllowUdfPatch() override { } - void LoadUdfs( + void LoadUdfs( const TString& libraryPath, - const TUdfModuleRemappings& remmapings, - ui32 flags /* = 0 */) override - { - TUdfLibraryPtr lib; - - auto libIt = LoadedLibraries_.find(libraryPath); - if (libIt != LoadedLibraries_.end()) { + const TUdfModuleRemappings& remmapings, + ui32 flags /* = 0 */) override + { + TUdfLibraryPtr lib; + + auto libIt = LoadedLibraries_.find(libraryPath); + if (libIt != LoadedLibraries_.end()) { return; - } else { - lib = MakeIntrusive<TUdfLibrary>(); + } else { + lib = MakeIntrusive<TUdfLibrary>(); #ifdef _win32_ ui32 loadFlags = 0; #else @@ -143,11 +143,11 @@ public: lib->Lib.Open(absPath.data(), loadFlags); lib->Lib.SetUnloadable(false); - - // (1) check ABI version - auto abiVersionFunc = reinterpret_cast<NUdf::TAbiVersionFunctionPtr>( - lib->Lib.Sym(AbiVersionFuncName)); - ui32 version = abiVersionFunc(); + + // (1) check ABI version + auto abiVersionFunc = reinterpret_cast<NUdf::TAbiVersionFunctionPtr>( + lib->Lib.Sym(AbiVersionFuncName)); + ui32 version = abiVersionFunc(); Y_ENSURE(NUdf::IsAbiCompatible(version), "Non compatible ABI version of UDF library " << libraryPath << ", expected up to " << NUdf::AbiVersionToStr(NUdf::CurrentCompatibilityAbiVersion() * 100) @@ -155,11 +155,11 @@ public: << "; try to re-compile library using " << "YQL_ABI_VERSION(" << UDF_ABI_VERSION_MAJOR << " " << UDF_ABI_VERSION_MINOR << " 0) macro in ya.make"); - lib->AbiVersion = version; + lib->AbiVersion = version; if (version < NUdf::MakeAbiVersion(2, 8, 0)) { SupportsSizedAllocators_ = false; } - + #if defined(_win_) || defined(_darwin_) auto bindSymbolsFunc = reinterpret_cast<NUdf::TBindSymbolsFunctionPtr>(lib->Lib.Sym(BindSymbolsFuncName)); bindSymbolsFunc(NUdf::GetStaticSymbols()); @@ -173,76 +173,76 @@ public: } libIt = LoadedLibraries_.insert({ libraryPath, lib }).first; - } - - // (2) ensure that Register() func is present - auto registerFunc = reinterpret_cast<NUdf::TRegisterFunctionPtr>( - lib->Lib.Sym(RegisterFuncName)); - - // (3) do load - TUdfModuleLoader loader( + } + + // (2) ensure that Register() func is present + auto registerFunc = reinterpret_cast<NUdf::TRegisterFunctionPtr>( + lib->Lib.Sym(RegisterFuncName)); + + // (3) do load + TUdfModuleLoader loader( UdfModules_, libraryPath, remmapings, lib->AbiVersion); - registerFunc(loader, flags); + registerFunc(loader, flags); Y_ENSURE(!loader.HasError(), loader.GetError()); - } - - void AddModule( - const TStringBuf& libraryPath, - const TStringBuf& moduleName, - NUdf::TUniquePtr<NUdf::IUdfModule> module) override - { + } + + void AddModule( + const TStringBuf& libraryPath, + const TStringBuf& moduleName, + NUdf::TUniquePtr<NUdf::IUdfModule> module) override + { TString libraryPathStr(libraryPath); auto inserted = LoadedLibraries_.insert({ libraryPathStr, nullptr }); - if (!inserted.second) { - return; - } - - TUdfModuleRemappings remappings; - TUdfModuleLoader loader( + if (!inserted.second) { + return; + } + + TUdfModuleRemappings remappings; + TUdfModuleLoader loader( UdfModules_, libraryPathStr, - remappings, NUdf::CurrentAbiVersion()); + remappings, NUdf::CurrentAbiVersion()); loader.AddModule(moduleName, std::move(module)); - + Y_ENSURE(!loader.HasError(), loader.GetError()); - } - - void SetSystemModulePaths(const TUdfModulePathsMap& paths) override { - SystemModulePaths_ = paths; - } - + } + + void SetSystemModulePaths(const TUdfModulePathsMap& paths) override { + SystemModulePaths_ = paths; + } + const IBuiltinFunctionRegistry::TPtr& GetBuiltins() const override { return Builtins_; } - TStatus FindFunctionTypeInfo( - const TTypeEnvironment& env, + TStatus FindFunctionTypeInfo( + const TTypeEnvironment& env, NUdf::ITypeInfoHelper::TPtr typeInfoHelper, NUdf::ICountersProvider* countersProvider, - const TStringBuf& name, - TType* userType, - const TStringBuf& typeConfig, - ui32 flags, + const TStringBuf& name, + TType* userType, + const TStringBuf& typeConfig, + ui32 flags, const NUdf::TSourcePosition& pos, const NUdf::ISecureParamsProvider* secureParamsProvider, - TFunctionTypeInfo* funcInfo) const override - { - TStringBuf moduleName, funcName; - if (name.TrySplit(MODULE_NAME_DELIMITER, moduleName, funcName)) { - auto it = UdfModules_.find(moduleName); - if (it != UdfModules_.end()) { + TFunctionTypeInfo* funcInfo) const override + { + TStringBuf moduleName, funcName; + if (name.TrySplit(MODULE_NAME_DELIMITER, moduleName, funcName)) { + auto it = UdfModules_.find(moduleName); + if (it != UdfModules_.end()) { TFunctionTypeInfoBuilder typeInfoBuilder(env, typeInfoHelper, moduleName, (flags & NUdf::IUdfModule::TFlags::TypesOnly) ? nullptr : countersProvider, pos, secureParamsProvider); const auto& module = *it->second.Impl; module.BuildFunctionTypeInfo( - funcName, userType, typeConfig, flags, typeInfoBuilder); - - if (typeInfoBuilder.HasError()) { - return TStatus::Error() - << "Module: " << moduleName - << ", function: " << funcName - << ", error: " << typeInfoBuilder.GetError(); - } - + funcName, userType, typeConfig, flags, typeInfoBuilder); + + if (typeInfoBuilder.HasError()) { + return TStatus::Error() + << "Module: " << moduleName + << ", function: " << funcName + << ", error: " << typeInfoBuilder.GetError(); + } + try { typeInfoBuilder.Build(funcInfo); } @@ -252,66 +252,66 @@ public: << ", function: " << funcName << ", error: " << e.what(); } - - if ((flags & NUdf::IRegistrator::TFlags::TypesOnly) && - !funcInfo->FunctionType) - { - return TStatus::Error() - << "Module: " << moduleName - << ", function: " << funcName - << ", function not found"; - } - + + if ((flags & NUdf::IRegistrator::TFlags::TypesOnly) && + !funcInfo->FunctionType) + { + return TStatus::Error() + << "Module: " << moduleName + << ", function: " << funcName + << ", function not found"; + } + if (funcInfo->ModuleIRUniqID) { funcInfo->ModuleIRUniqID.prepend(moduleName); } - return TStatus::Ok(); - } - - return TStatus::Error() - << "Module " << moduleName << " is not registered"; - } - - return TStatus::Error() - << "Function name must be in <module>.<func_name> scheme. " - << "But get " << name; - } - + return TStatus::Ok(); + } + + return TStatus::Error() + << "Module " << moduleName << " is not registered"; + } + + return TStatus::Error() + << "Function name must be in <module>.<func_name> scheme. " + << "But get " << name; + } + TMaybe<TString> FindUdfPath(const TStringBuf& moduleName) const override { - if (const TUdfModule* udf = UdfModules_.FindPtr(moduleName)) { - return udf->LibraryPath; - } - + if (const TUdfModule* udf = UdfModules_.FindPtr(moduleName)) { + return udf->LibraryPath; + } + if (const TString* path = SystemModulePaths_.FindPtr(moduleName)) { - return *path; - } - - return Nothing(); - } - - bool IsLoadedUdfModule(const TStringBuf& moduleName) const override { + return *path; + } + + return Nothing(); + } + + bool IsLoadedUdfModule(const TStringBuf& moduleName) const override { return UdfModules_.contains(moduleName); - } - + } + THashSet<TString> GetAllModuleNames() const override { THashSet<TString> names; names.reserve(UdfModules_.size()); - for (const auto& module: UdfModules_) { - names.insert(module.first); - } - return names; - } - + for (const auto& module: UdfModules_) { + names.insert(module.first); + } + return names; + } + TFunctionsMap GetModuleFunctions(const TStringBuf& moduleName) const override { - struct TFunctionNamesSink: public NUdf::IFunctionNamesSink { + struct TFunctionNamesSink: public NUdf::IFunctionNamesSink { TFunctionsMap Functions; class TFuncDescriptor : public NUdf::IFunctionDescriptor { public: TFuncDescriptor(TFunctionProperties& properties) : Properties(properties) {} - + private: void SetTypeAwareness() final { Properties.IsTypeAwareness = true; @@ -323,69 +323,69 @@ public: NUdf::IFunctionDescriptor::TPtr Add(const NUdf::TStringRef& name) final { const auto it = Functions.emplace(name, TFunctionProperties{}); return new TFuncDescriptor(it.first->second); - } - } sink; - - + } + } sink; + + const auto it = UdfModules_.find(moduleName); if (UdfModules_.cend() == it) return TFunctionsMap(); it->second.Impl->GetAllFunctions(sink); return sink.Functions; - } - + } + bool SupportsSizedAllocators() const override { return SupportsSizedAllocators_; } void PrintInfoTo(IOutputStream& out) const override { - Builtins_->PrintInfoTo(out); - } - + Builtins_->PrintInfoTo(out); + } + void CleanupModulesOnTerminate() const override { for (const auto& module : UdfModules_) { module.second.Impl->CleanupOnTerminate(); } } - TIntrusivePtr<IMutableFunctionRegistry> Clone() const override { - return new TMutableFunctionRegistry(*this); + TIntrusivePtr<IMutableFunctionRegistry> Clone() const override { + return new TMutableFunctionRegistry(*this); } void SetBackTraceCallback(NUdf::TBackTraceCallback callback) override { BackTraceCallback_ = callback; } -private: +private: const IBuiltinFunctionRegistry::TPtr Builtins_; THashMap<TString, TUdfLibraryPtr> LoadedLibraries_; - TUdfModulesMap UdfModules_; + TUdfModulesMap UdfModules_; THolder<TMemoryUsageInfo> UdfMemoryInfo_; - TUdfModulePathsMap SystemModulePaths_; + TUdfModulePathsMap SystemModulePaths_; NUdf::TBackTraceCallback BackTraceCallback_ = nullptr; bool SupportsSizedAllocators_ = true; -}; - -////////////////////////////////////////////////////////////////////////////// -// TBuiltinsWrapper -////////////////////////////////////////////////////////////////////////////// -class TBuiltinsWrapper: public IFunctionRegistry -{ -public: +}; + +////////////////////////////////////////////////////////////////////////////// +// TBuiltinsWrapper +////////////////////////////////////////////////////////////////////////////// +class TBuiltinsWrapper: public IFunctionRegistry +{ +public: TBuiltinsWrapper(IBuiltinFunctionRegistry::TPtr&& builtins) : Builtins_(std::move(builtins)) - { - } - + { + } + const IBuiltinFunctionRegistry::TPtr& GetBuiltins() const override { return Builtins_; - } - + } + void AllowUdfPatch() override { } - TStatus FindFunctionTypeInfo( + TStatus FindFunctionTypeInfo( const TTypeEnvironment& env, NUdf::ITypeInfoHelper::TPtr typeInfoHelper, NUdf::ICountersProvider* countersProvider, @@ -396,7 +396,7 @@ public: const NUdf::TSourcePosition& pos, const NUdf::ISecureParamsProvider* secureParamsProvider, TFunctionTypeInfo* funcInfo) const override - { + { Y_UNUSED(env); Y_UNUSED(typeInfoHelper); Y_UNUSED(countersProvider); @@ -408,72 +408,72 @@ public: Y_UNUSED(secureParamsProvider); Y_UNUSED(funcInfo); return TStatus::Error(TStringBuf("Unsupported access to builtins registry")); - } - + } + TMaybe<TString> FindUdfPath( - const TStringBuf& /* moduleName */) const override - { - return{}; - } - - bool IsLoadedUdfModule(const TStringBuf& /* moduleName */) const override { + const TStringBuf& /* moduleName */) const override + { + return{}; + } + + bool IsLoadedUdfModule(const TStringBuf& /* moduleName */) const override { return false; - } - + } + THashSet<TString> GetAllModuleNames() const override { - return {}; - } - + return {}; + } + TFunctionsMap GetModuleFunctions(const TStringBuf&) const override { return TFunctionsMap(); - } - + } + bool SupportsSizedAllocators() const override { return true; } void PrintInfoTo(IOutputStream& out) const override { - Builtins_->PrintInfoTo(out); - } - + Builtins_->PrintInfoTo(out); + } + void CleanupModulesOnTerminate() const override { } - TIntrusivePtr<IMutableFunctionRegistry> Clone() const override { + TIntrusivePtr<IMutableFunctionRegistry> Clone() const override { return new TMutableFunctionRegistry(Builtins_); - } - -private: + } + +private: const IBuiltinFunctionRegistry::TPtr Builtins_; -}; - -} // namespace - -namespace NKikimr { -namespace NMiniKQL { - +}; + +} // namespace + +namespace NKikimr { +namespace NMiniKQL { + void FindUdfsInDir(const TString& dirPath, TVector<TString>* paths) -{ +{ static const TStringBuf libPrefix = TStringBuf(MKQL_UDF_LIB_PREFIX); static const TStringBuf libSuffix = TStringBuf(MKQL_UDF_LIB_SUFFIX); - + if (!dirPath.empty()) { std::vector<TString> dirs; StringSplitter(dirPath).Split(';').AddTo(&dirs); - + for (auto d : dirs) { TDirIterator dir(d, TDirIterator::TOptions(FTS_LOGICAL).SetMaxLevel(10)); - + for (auto file = dir.begin(), end = dir.end(); file != end; ++file) { // skip entries with empty name, and all non-files // all valid symlinks are already dereferenced, provided by FTS_LOGICAL if (file->fts_pathlen == file->fts_namelen || file->fts_info != FTS_F) { continue; } - + TString path(file->fts_path); TString fileName = GetBaseName(path); - + // skip non shared libraries if (!fileName.StartsWith(libPrefix) || !fileName.EndsWith(libSuffix)) @@ -489,52 +489,52 @@ void FindUdfsInDir(const TString& dirPath, TVector<TString>* paths) paths->push_back(std::move(path)); } - } - } -} - -bool SplitModuleAndFuncName(const TStringBuf& name, TStringBuf& module, TStringBuf& func) -{ - return name.TrySplit(MODULE_NAME_DELIMITER, module, func); -} - + } + } +} + +bool SplitModuleAndFuncName(const TStringBuf& name, TStringBuf& module, TStringBuf& func) +{ + return name.TrySplit(MODULE_NAME_DELIMITER, module, func); +} + TString FullName(const TStringBuf& module, const TStringBuf& func) -{ +{ TString fullName; - fullName.reserve(module.size() + func.size() + 1); - fullName.append(module); - fullName.append(MODULE_NAME_DELIMITER); - fullName.append(func); - return fullName; -} - + fullName.reserve(module.size() + func.size() + 1); + fullName.append(module); + fullName.append(MODULE_NAME_DELIMITER); + fullName.append(func); + return fullName; +} + TIntrusivePtr<IFunctionRegistry> CreateFunctionRegistry(IBuiltinFunctionRegistry::TPtr&& builtins) -{ - return new TBuiltinsWrapper(std::move(builtins)); +{ + return new TBuiltinsWrapper(std::move(builtins)); } -TIntrusivePtr<IFunctionRegistry> CreateFunctionRegistry( +TIntrusivePtr<IFunctionRegistry> CreateFunctionRegistry( NKikimr::NUdf::TBackTraceCallback backtraceCallback, IBuiltinFunctionRegistry::TPtr&& builtins, bool allowUdfPatch, const TVector<TString>& udfsPaths, ui32 flags /* = 0 */) -{ +{ auto registry = MakeHolder<TMutableFunctionRegistry>(std::move(builtins)); if (allowUdfPatch) { registry->AllowUdfPatch(); } registry->SetBackTraceCallback(backtraceCallback); - - // system UDFs loaded with default names - TUdfModuleRemappings remappings; + + // system UDFs loaded with default names + TUdfModuleRemappings remappings; for (const TString& udfPath: udfsPaths) { - registry->LoadUdfs(udfPath, remappings, flags); - } - - return registry.Release(); -} - + registry->LoadUdfs(udfPath, remappings, flags); + } + + return registry.Release(); +} + void FillStaticModules(IMutableFunctionRegistry& registry) { for (const auto& wrapper : NUdf::GetStaticUdfModuleWrapperList()) { auto [name, ptr] = wrapper(); diff --git a/ydb/library/yql/minikql/mkql_function_registry.h b/ydb/library/yql/minikql/mkql_function_registry.h index 309bb994c34..f18152cb24e 100644 --- a/ydb/library/yql/minikql/mkql_function_registry.h +++ b/ydb/library/yql/minikql/mkql_function_registry.h @@ -5,83 +5,83 @@ #include <ydb/library/yql/public/udf/udf_counter.h> #include <ydb/library/yql/public/udf/udf_registrator.h> #include <ydb/library/yql/public/udf/udf_type_builder.h> - -#include <util/generic/ptr.h> + +#include <util/generic/ptr.h> #include <util/generic/vector.h> -#include <util/generic/maybe.h> -#include <util/generic/hash.h> -#include <util/generic/hash_set.h> +#include <util/generic/maybe.h> +#include <util/generic/hash.h> +#include <util/generic/hash_set.h> #include <map> -#ifdef _win_ -# define MKQL_UDF_LIB_PREFIX "" -# define MKQL_UDF_LIB_SUFFIX ".dll" -#elif defined(_darwin_) -# define MKQL_UDF_LIB_PREFIX "lib" -# define MKQL_UDF_LIB_SUFFIX ".dylib" -#else -# define MKQL_UDF_LIB_PREFIX "lib" -# define MKQL_UDF_LIB_SUFFIX ".so" -#endif - -////////////////////////////////////////////////////////////////////////////// -// forward declarations -////////////////////////////////////////////////////////////////////////////// +#ifdef _win_ +# define MKQL_UDF_LIB_PREFIX "" +# define MKQL_UDF_LIB_SUFFIX ".dll" +#elif defined(_darwin_) +# define MKQL_UDF_LIB_PREFIX "lib" +# define MKQL_UDF_LIB_SUFFIX ".dylib" +#else +# define MKQL_UDF_LIB_PREFIX "lib" +# define MKQL_UDF_LIB_SUFFIX ".so" +#endif + +////////////////////////////////////////////////////////////////////////////// +// forward declarations +////////////////////////////////////////////////////////////////////////////// class IOutputStream; - + namespace NKikimr { - namespace NMiniKQL { - class IMutableFunctionRegistry; - class TTypeEnvironment; + namespace NMiniKQL { + class IMutableFunctionRegistry; + class TTypeEnvironment; struct TFunctionTypeInfo; - class TStatus; + class TStatus; class TType; - } -} - -namespace NKikimr { -namespace NMiniKQL { - + } +} + +namespace NKikimr { +namespace NMiniKQL { + using TUdfModuleRemappings = THashMap<TString, TString>; // old => new using TUdfModulePathsMap = THashMap<TString, TString>; // module name => udf path - -////////////////////////////////////////////////////////////////////////////// -// IFunctionRegistry -////////////////////////////////////////////////////////////////////////////// -class IFunctionRegistry: public TThrRefBase -{ -public: + +////////////////////////////////////////////////////////////////////////////// +// IFunctionRegistry +////////////////////////////////////////////////////////////////////////////// +class IFunctionRegistry: public TThrRefBase +{ +public: typedef TIntrusivePtr<IFunctionRegistry> TPtr; - virtual ~IFunctionRegistry() = default; - + virtual ~IFunctionRegistry() = default; + virtual const IBuiltinFunctionRegistry::TPtr& GetBuiltins() const = 0; - + virtual void AllowUdfPatch() = 0; - virtual TStatus FindFunctionTypeInfo( - const TTypeEnvironment& env, + virtual TStatus FindFunctionTypeInfo( + const TTypeEnvironment& env, NUdf::ITypeInfoHelper::TPtr typeInfoHelper, NUdf::ICountersProvider* countersProvider, - const TStringBuf& name, - TType* userType, - const TStringBuf& typeConfig, - ui32 flags, + const TStringBuf& name, + TType* userType, + const TStringBuf& typeConfig, + ui32 flags, const NUdf::TSourcePosition& pos, const NUdf::ISecureParamsProvider* secureParamsProvider, - TFunctionTypeInfo* funcInfo) const = 0; - + TFunctionTypeInfo* funcInfo) const = 0; + virtual TMaybe<TString> FindUdfPath(const TStringBuf& moduleName) const = 0; - - virtual bool IsLoadedUdfModule(const TStringBuf& moduleName) const = 0; - + + virtual bool IsLoadedUdfModule(const TStringBuf& moduleName) const = 0; + virtual THashSet<TString> GetAllModuleNames() const = 0; - + virtual void PrintInfoTo(IOutputStream& out) const = 0; - + virtual void CleanupModulesOnTerminate() const = 0; - virtual TIntrusivePtr<IMutableFunctionRegistry> Clone() const = 0; + virtual TIntrusivePtr<IMutableFunctionRegistry> Clone() const = 0; struct TFunctionProperties { bool IsTypeAwareness = false; }; @@ -92,59 +92,59 @@ public: virtual bool SupportsSizedAllocators() const = 0; }; -////////////////////////////////////////////////////////////////////////////// -// IMutableFunctionRegistry -////////////////////////////////////////////////////////////////////////////// -class IMutableFunctionRegistry: public IFunctionRegistry -{ -public: +////////////////////////////////////////////////////////////////////////////// +// IMutableFunctionRegistry +////////////////////////////////////////////////////////////////////////////// +class IMutableFunctionRegistry: public IFunctionRegistry +{ +public: virtual void SetBackTraceCallback(NUdf::TBackTraceCallback callback) = 0; - virtual void LoadUdfs( + virtual void LoadUdfs( const TString& libraryPath, - const TUdfModuleRemappings& remmapings, - ui32 flags = 0) = 0; - - virtual void AddModule( - const TStringBuf& libraryPath, - const TStringBuf& moduleName, - NUdf::TUniquePtr<NUdf::IUdfModule> module) = 0; - - virtual void SetSystemModulePaths(const TUdfModulePathsMap& paths) = 0; -}; - -////////////////////////////////////////////////////////////////////////////// -// factories -////////////////////////////////////////////////////////////////////////////// + const TUdfModuleRemappings& remmapings, + ui32 flags = 0) = 0; + + virtual void AddModule( + const TStringBuf& libraryPath, + const TStringBuf& moduleName, + NUdf::TUniquePtr<NUdf::IUdfModule> module) = 0; + + virtual void SetSystemModulePaths(const TUdfModulePathsMap& paths) = 0; +}; + +////////////////////////////////////////////////////////////////////////////// +// factories +////////////////////////////////////////////////////////////////////////////// TIntrusivePtr<IFunctionRegistry> CreateFunctionRegistry(IBuiltinFunctionRegistry::TPtr&& builtins); - -TIntrusivePtr<IFunctionRegistry> CreateFunctionRegistry( + +TIntrusivePtr<IFunctionRegistry> CreateFunctionRegistry( NKikimr::NUdf::TBackTraceCallback backtraceCallback, IBuiltinFunctionRegistry::TPtr&& builtins, bool allowUdfPatch, const TVector<TString>& udfsPaths, - ui32 flags = 0); // see NUdf::IRegistrator::TFlags - -////////////////////////////////////////////////////////////////////////////// -// helper functions -////////////////////////////////////////////////////////////////////////////// + ui32 flags = 0); // see NUdf::IRegistrator::TFlags + +////////////////////////////////////////////////////////////////////////////// +// helper functions +////////////////////////////////////////////////////////////////////////////// void FindUdfsInDir(const TString& dirPath, TVector<TString>* paths); - -bool SplitModuleAndFuncName( - const TStringBuf& name, TStringBuf& module, TStringBuf& func); + +bool SplitModuleAndFuncName( + const TStringBuf& name, TStringBuf& module, TStringBuf& func); TString FullName(const TStringBuf& module, const TStringBuf& func); - -inline TStringBuf ModuleName(const TStringBuf& name) { - TStringBuf moduleName, _; - if (SplitModuleAndFuncName(name, moduleName, _)) { - return moduleName; - } - return name; -} - + +inline TStringBuf ModuleName(const TStringBuf& name) { + TStringBuf moduleName, _; + if (SplitModuleAndFuncName(name, moduleName, _)) { + return moduleName; + } + return name; +} + const TStringBuf StaticModulePrefix(TStringBuf("<static>::")); void FillStaticModules(IMutableFunctionRegistry& registry); } // namespace NMiniKQL -} // namespace NKikimr +} // namespace NKikimr diff --git a/ydb/library/yql/minikql/mkql_mem_info.h b/ydb/library/yql/minikql/mkql_mem_info.h index 7b261dafb2f..8e880d1db30 100644 --- a/ydb/library/yql/minikql/mkql_mem_info.h +++ b/ydb/library/yql/minikql/mkql_mem_info.h @@ -1,60 +1,60 @@ -#pragma once - -#include <util/stream/output.h> -#include <util/string/builder.h> +#pragma once + +#include <util/stream/output.h> +#include <util/string/builder.h> #include <unordered_map> - -#ifndef NDEBUG -# define MKQL_MEM_TAKE3(MemInfo, Mem, Size) \ - ::NKikimr::NMiniKQL::Take(MemInfo, (Mem), (Size), TStringBuilder() << __LOCATION__) -# define MKQL_MEM_TAKE4(MemInfo, Mem, Size, Message) \ - ::NKikimr::NMiniKQL::Take(MemInfo, (Mem), (Size), TStringBuilder() << __LOCATION__ << ", " << Message) + +#ifndef NDEBUG +# define MKQL_MEM_TAKE3(MemInfo, Mem, Size) \ + ::NKikimr::NMiniKQL::Take(MemInfo, (Mem), (Size), TStringBuilder() << __LOCATION__) +# define MKQL_MEM_TAKE4(MemInfo, Mem, Size, Message) \ + ::NKikimr::NMiniKQL::Take(MemInfo, (Mem), (Size), TStringBuilder() << __LOCATION__ << ", " << Message) # define MKQL_MEM_RETURN(MemInfo, Mem, Size) \ ::NKikimr::NMiniKQL::Return(MemInfo, (Mem), (Size)) # define MKQL_MEM_RETURN_PTR(MemInfo, Mem) \ ::NKikimr::NMiniKQL::Return(MemInfo, (Mem)) -#else -# define MKQL_MEM_TAKE3(MemInfo, Mem, Size) \ +#else +# define MKQL_MEM_TAKE3(MemInfo, Mem, Size) \ Y_UNUSED(MemInfo); Y_UNUSED(Mem); Y_UNUSED(Size); -# define MKQL_MEM_TAKE4(MemInfo, Mem, Size, Message) \ +# define MKQL_MEM_TAKE4(MemInfo, Mem, Size, Message) \ Y_UNUSED(MemInfo); Y_UNUSED(Mem); Y_UNUSED(Size); # define MKQL_MEM_RETURN(MemInfo, Mem, Size) \ Y_UNUSED(MemInfo); Y_UNUSED(Mem); Y_UNUSED(Size); # define MKQL_MEM_RETURN_PTR(MemInfo, Mem) \ Y_UNUSED(MemInfo); Y_UNUSED(Mem); -#endif - -#define GET_MKQL_MEM_TAKE(_1, _2, _3, _4, IMPL, ...) IMPL +#endif + +#define GET_MKQL_MEM_TAKE(_1, _2, _3, _4, IMPL, ...) IMPL #define MKQL_MEM_TAKE(...) Y_PASS_VA_ARGS(GET_MKQL_MEM_TAKE(__VA_ARGS__, MKQL_MEM_TAKE4, MKQL_MEM_TAKE3)(__VA_ARGS__)) - - -namespace NKikimr { -namespace NMiniKQL { - + + +namespace NKikimr { +namespace NMiniKQL { + class TMemoryUsageInfo : public TThrRefBase -{ - struct TAllocationInfo { - ui64 Size; +{ + struct TAllocationInfo { + ui64 Size; TString Location; bool IsDeleted; - }; -public: - inline explicit TMemoryUsageInfo(const TStringBuf& title) - : Title_(title) - , Allocated_(0) - , Freed_(0) + }; +public: + inline explicit TMemoryUsageInfo(const TStringBuf& title) + : Title_(title) + , Allocated_(0) + , Freed_(0) , Peak_(0) , AllowMissing_(false) , CheckOnExit_(true) - { - } - - inline ~TMemoryUsageInfo() { + { + } + + inline ~TMemoryUsageInfo() { if (CheckOnExit_ && !UncaughtException()) { VerifyDebug(); } - } - + } + void AllowMissing() { AllowMissing_ = true; } @@ -65,7 +65,7 @@ public: #ifndef NDEBUG inline void Take(const void* mem, ui64 size, TString location) { - Allocated_ += size; + Allocated_ += size; Peak_ = Max(Peak_, Allocated_ - Freed_); if (size == 0) { return; @@ -77,20 +77,20 @@ public: } } auto res = AllocationsMap_.insert({mem, { size, std::move(location), false }}); - Y_VERIFY_DEBUG(res.second, "Duplicate allocation at: %p, " + Y_VERIFY_DEBUG(res.second, "Duplicate allocation at: %p, " "already allocated at: %s", mem, res.first->second.Location.data()); //Clog << Title_ << " take: " << size << " -> " << mem << " " << AllocationsMap_.size() << Endl; } -#endif - +#endif + #ifndef NDEBUG inline void Return(const void* mem, ui64 size) { - Freed_ += size; + Freed_ += size; if (size == 0) { return; } //Clog << Title_ << " free: " << size << " -> " << mem << " " << AllocationsMap_.size() << Endl; - auto it = AllocationsMap_.find(mem); + auto it = AllocationsMap_.find(mem); if (AllowMissing_ && it == AllocationsMap_.end()) { return; } @@ -101,7 +101,7 @@ public: Y_VERIFY_DEBUG(it != AllocationsMap_.end(), "Double free at: %p", mem); } - Y_VERIFY_DEBUG(size == it->second.Size, + Y_VERIFY_DEBUG(size == it->second.Size, "Deallocating wrong size at: %p, " "allocated at: %s", mem, it->second.Location.data()); if (AllowMissing_) { @@ -110,8 +110,8 @@ public: AllocationsMap_.erase(it); } } -#endif - +#endif + #ifndef NDEBUG inline void Return(const void* mem) { //Clog << Title_ << " free: " << size << " -> " << mem << " " << AllocationsMap_.size() << Endl; @@ -135,24 +135,24 @@ public: } #endif - inline i64 GetUsage() const { - return static_cast<i64>(Allocated_) - static_cast<i64>(Freed_); - } - - inline ui64 GetAllocated() const { return Allocated_; } - inline ui64 GetFreed() const { return Freed_; } + inline i64 GetUsage() const { + return static_cast<i64>(Allocated_) - static_cast<i64>(Freed_); + } + + inline ui64 GetAllocated() const { return Allocated_; } + inline ui64 GetFreed() const { return Freed_; } inline ui64 GetPeak() const { return Peak_; } - + inline void PrintTo(IOutputStream& out) const { out << Title_ << TStringBuf(": usage=") << GetUsage() << TStringBuf(" (allocated=") << GetAllocated() << TStringBuf(", freed=") << GetFreed() << TStringBuf(", peak=") << GetPeak() - << ')'; - } - - inline void VerifyDebug() const { -#ifndef NDEBUG + << ')'; + } + + inline void VerifyDebug() const { +#ifndef NDEBUG size_t leakCount = 0; for (const auto& it: AllocationsMap_) { if (it.second.IsDeleted) { @@ -164,40 +164,40 @@ public: << TStringBuf(", location: ") << it.second.Location << Endl; } - + if (!AllowMissing_) { Y_VERIFY_DEBUG(GetUsage() == 0, "Allocated: %ld, Freed: %ld, Peak: %ld", GetAllocated(), GetFreed(), GetPeak()); } Y_VERIFY_DEBUG(!leakCount, "Has no freed memory"); -#endif - } - -private: +#endif + } + +private: const TString Title_; - ui64 Allocated_; - ui64 Freed_; + ui64 Allocated_; + ui64 Freed_; ui64 Peak_; bool AllowMissing_; bool CheckOnExit_; - -#ifndef NDEBUG + +#ifndef NDEBUG std::unordered_map<const void*, TAllocationInfo> AllocationsMap_; -#endif -}; - +#endif +}; + #ifndef NDEBUG inline void Take(TMemoryUsageInfo& memInfo, const void* mem, ui64 size, TString location) -{ - memInfo.Take(mem, size, std::move(location)); -} - +{ + memInfo.Take(mem, size, std::move(location)); +} + inline void Take(TMemoryUsageInfo* memInfo, const void* mem, ui64 size, TString location) -{ - memInfo->Take(mem, size, std::move(location)); -} - +{ + memInfo->Take(mem, size, std::move(location)); +} + inline void Return(TMemoryUsageInfo& memInfo, const void* mem, ui64 size) { memInfo.Return(mem, size); @@ -221,11 +221,11 @@ inline void Return(TMemoryUsageInfo* memInfo, const void* mem) } // namespace NMiniKQL } // namespace NKikimr - -template <> -inline void Out<NKikimr::NMiniKQL::TMemoryUsageInfo>( + +template <> +inline void Out<NKikimr::NMiniKQL::TMemoryUsageInfo>( IOutputStream& out, - const NKikimr::NMiniKQL::TMemoryUsageInfo& memInfo) -{ - memInfo.PrintTo(out); -} + const NKikimr::NMiniKQL::TMemoryUsageInfo& memInfo) +{ + memInfo.PrintTo(out); +} diff --git a/ydb/library/yql/minikql/mkql_node.cpp b/ydb/library/yql/minikql/mkql_node.cpp index 0fb29e7508c..4c6f3e45ad4 100644 --- a/ydb/library/yql/minikql/mkql_node.cpp +++ b/ydb/library/yql/minikql/mkql_node.cpp @@ -36,7 +36,7 @@ TTypeEnvironment::TTypeEnvironment(TScopedAlloc& alloc) Ui64 = TDataType::Create(NUdf::TDataType<ui64>::Id, *this); AnyType = TAnyType::Create(TypeOfType, *this); EmptyStruct = TStructLiteral::Create(0, nullptr, TStructType::Create(0, nullptr, *this), *this); - EmptyTuple = TTupleLiteral::Create(0, nullptr, TTupleType::Create(0, nullptr, *this), *this); + EmptyTuple = TTupleLiteral::Create(0, nullptr, TTupleType::Create(0, nullptr, *this), *this); ListOfVoid = TListLiteral::Create(nullptr, 0, TListType::Create(Void->GetGenericType(), *this), *this); } @@ -181,11 +181,11 @@ bool TNode::IsMergeable() const { TStringBuf TType::KindAsStr(EKind kind) { switch (static_cast<int>(kind)) { MKQL_TYPE_KINDS(MKQL_SWITCH_ENUM_TYPE_TO_STR) - } - + } + return TStringBuf("unknown"); -} - +} + TStringBuf TType::GetKindAsStr() const { return KindAsStr(Kind); } @@ -307,18 +307,18 @@ bool TType::IsConvertableTo(const TType& typeToCompare, bool ignoreTagged) const } switch (self->Kind) { -#define APPLY(kind, type) \ - case EKind::kind: \ +#define APPLY(kind, type) \ + case EKind::kind: \ return static_cast<const type&>(*self).IsConvertableTo(static_cast<const type&>(*other), ignoreTagged); - - TYPES_LIST(APPLY) - -#undef APPLY - default: + + TYPES_LIST(APPLY) + +#undef APPLY + default: Y_FAIL(); - } -} - + } +} + TTypeType* TTypeType::Create(const TTypeEnvironment& env) { return ::new(env.Allocate<TTypeType>()) TTypeType(); } @@ -330,9 +330,9 @@ bool TTypeType::IsSameType(const TTypeType& typeToCompare) const { bool TTypeType::IsConvertableTo(const TTypeType& typeToCompare, bool ignoreTagged) const { Y_UNUSED(ignoreTagged); - return IsSameType(typeToCompare); -} - + return IsSameType(typeToCompare); +} + void TTypeType::DoUpdateLinks(const THashMap<TNode*, TNode*>& links) { Y_UNUSED(links); } @@ -369,9 +369,9 @@ bool TDataType::IsSameType(const TDataType& typeToCompare) const { bool TDataType::IsConvertableTo(const TDataType& typeToCompare, bool ignoreTagged) const { Y_UNUSED(ignoreTagged); - return IsSameType(typeToCompare); -} - + return IsSameType(typeToCompare); +} + void TDataType::DoUpdateLinks(const THashMap<TNode*, TNode*>& links) { Y_UNUSED(links); } @@ -525,22 +525,22 @@ bool TStructType::IsSameType(const TStructType& typeToCompare) const { } bool TStructType::IsConvertableTo(const TStructType& typeToCompare, bool ignoreTagged) const { - if (this == &typeToCompare) - return true; - + if (this == &typeToCompare) + return true; + if (MembersCount != typeToCompare.MembersCount) - return false; - - for (size_t index = 0; index < MembersCount; ++index) { - if (Members[index].first != typeToCompare.Members[index].first) - return false; + return false; + + for (size_t index = 0; index < MembersCount; ++index) { + if (Members[index].first != typeToCompare.Members[index].first) + return false; if (!Members[index].second->IsConvertableTo(*typeToCompare.Members[index].second, ignoreTagged)) - return false; - } - - return true; -} - + return false; + } + + return true; +} + void TStructType::DoUpdateLinks(const THashMap<TNode*, TNode*>& links) { for (ui32 i = 0; i < MembersCount; ++i) { auto& member = Members[i]; @@ -626,7 +626,7 @@ TStructLiteral::TStructLiteral(TRuntimeNode* values, TStructType* type, bool val MKQL_ENSURE(!type->GetMemberName(index).empty(), "Empty struct member name is not allowed"); auto& value = Values[index]; - MKQL_ENSURE(value.GetStaticType()->IsSameType(*type->GetMemberType(index)), "Wrong type of member"); + MKQL_ENSURE(value.GetStaticType()->IsSameType(*type->GetMemberType(index)), "Wrong type of member"); value.Freeze(); } @@ -737,8 +737,8 @@ bool TListType::IsSameType(const TListType& typeToCompare) const { bool TListType::IsConvertableTo(const TListType& typeToCompare, bool ignoreTagged) const { return GetItemType()->IsConvertableTo(*typeToCompare.GetItemType(), ignoreTagged); -} - +} + void TListType::DoUpdateLinks(const THashMap<TNode*, TNode*>& links) { auto itemTypeIt = links.find(GetItemType()); if (itemTypeIt != links.end()) { @@ -776,7 +776,7 @@ TListLiteral::TListLiteral(TRuntimeNode* items, ui32 count, TListType* type, con for (ui32 i = 0; i < Count; ++i) { auto& item = Items[i]; - MKQL_ENSURE(item.GetStaticType()->IsSameType(*type->GetItemType()), "Wrong type of item"); + MKQL_ENSURE(item.GetStaticType()->IsSameType(*type->GetItemType()), "Wrong type of item"); } TListLiteral::DoFreeze(env); @@ -988,8 +988,8 @@ bool TOptionalType::IsSameType(const TOptionalType& typeToCompare) const { bool TOptionalType::IsConvertableTo(const TOptionalType& typeToCompare, bool ignoreTagged) const { return GetItemType()->IsConvertableTo(*typeToCompare.GetItemType(), ignoreTagged); -} - +} + void TOptionalType::DoUpdateLinks(const THashMap<TNode*, TNode*>& links) { auto itemTypeIt = links.find(GetItemType()); if (itemTypeIt != links.end()) { @@ -1068,7 +1068,7 @@ TOptionalLiteral::TOptionalLiteral(TRuntimeNode item, TOptionalType* type, bool Y_VERIFY_DEBUG(Item.GetNode()); - MKQL_ENSURE(Item.GetStaticType()->IsSameType(*type->GetItemType()), "Wrong type of item"); + MKQL_ENSURE(Item.GetStaticType()->IsSameType(*type->GetItemType()), "Wrong type of item"); Item.Freeze(); } @@ -1142,8 +1142,8 @@ bool TDictType::IsSameType(const TDictType& typeToCompare) const { bool TDictType::IsConvertableTo(const TDictType& typeToCompare, bool ignoreTagged) const { return KeyType->IsConvertableTo(*typeToCompare.KeyType, ignoreTagged) && PayloadType->IsConvertableTo(*typeToCompare.PayloadType, ignoreTagged); -} - +} + void TDictType::DoUpdateLinks(const THashMap<TNode*, TNode*>& links) { auto keyTypeIt = links.find(KeyType); if (keyTypeIt != links.end()) { @@ -1392,35 +1392,35 @@ bool TCallableType::IsSameType(const TCallableType& typeToCompare) const { } bool TCallableType::IsConvertableTo(const TCallableType& typeToCompare, bool ignoreTagged) const { - // do not check callable name here - - if (this == &typeToCompare) - return true; - + // do not check callable name here + + if (this == &typeToCompare) + return true; + if (IsMergeDisabled0 != typeToCompare.IsMergeDisabled0) - return false; - - if (ArgumentsCount != typeToCompare.ArgumentsCount) - return false; - - // function with fewer optional args can't be converted to function - // with more optional args - if (OptionalArgs < typeToCompare.OptionalArgs) - return false; - - for (size_t index = 0; index < ArgumentsCount; ++index) { + return false; + + if (ArgumentsCount != typeToCompare.ArgumentsCount) + return false; + + // function with fewer optional args can't be converted to function + // with more optional args + if (OptionalArgs < typeToCompare.OptionalArgs) + return false; + + for (size_t index = 0; index < ArgumentsCount; ++index) { const auto arg = Arguments[index]; const auto otherArg = typeToCompare.Arguments[index]; if (!arg->IsConvertableTo(*otherArg, ignoreTagged)) - return false; - } - + return false; + } + if (!ReturnType->IsConvertableTo(*typeToCompare.ReturnType, ignoreTagged)) - return false; - - return !Payload || Payload->Equals(*typeToCompare.Payload); -} - + return false; + + return !Payload || Payload->Equals(*typeToCompare.Payload); +} + void TCallableType::SetOptionalArgumentsCount(ui32 count) { MKQL_ENSURE(count <= ArgumentsCount, "Wrong optional arguments count: " << count << ", function has only " << ArgumentsCount << " arguments"); @@ -1512,7 +1512,7 @@ TCallable::TCallable(ui32 inputsCount, TRuntimeNode* inputs, TCallableType* type return; } - MKQL_ENSURE(inputsCount == type->GetArgumentsCount(), "Wrong count of inputs"); + MKQL_ENSURE(inputsCount == type->GetArgumentsCount(), "Wrong count of inputs"); for (size_t index = 0; index < inputsCount; ++index) { auto& node = Inputs[index]; @@ -1660,9 +1660,9 @@ bool TCallable::Equals(const TCallable& nodeToCompare) const { void TCallable::SetResult(TRuntimeNode result, const TTypeEnvironment& env) { Y_UNUSED(env); - MKQL_ENSURE(!Result.GetNode(), "result is already set"); + MKQL_ENSURE(!Result.GetNode(), "result is already set"); - MKQL_ENSURE(result.GetStaticType()->IsSameType(*GetType()->GetReturnType()), + MKQL_ENSURE(result.GetStaticType()->IsSameType(*GetType()->GetReturnType()), "incorrect result type of function " << GetType()->GetName() << ", left: " << PrintNode(result.GetStaticType(), true) << ", right: " << PrintNode(GetType()->GetReturnType(), true)); @@ -1679,7 +1679,7 @@ bool TRuntimeNode::HasValue() const { if (current.IsImmediate()) return true; - MKQL_ENSURE(current.GetNode()->GetType()->IsCallable(), "Wrong type"); + MKQL_ENSURE(current.GetNode()->GetType()->IsCallable(), "Wrong type"); const auto& callable = static_cast<const TCallable&>(*current.GetNode()); if (!callable.HasResult()) @@ -1695,7 +1695,7 @@ TNode* TRuntimeNode::GetValue() const { if (current.IsImmediate()) return current.GetNode(); - MKQL_ENSURE(current.GetNode()->GetType()->IsCallable(), "Wrong type"); + MKQL_ENSURE(current.GetNode()->GetType()->IsCallable(), "Wrong type"); const auto& callable = static_cast<const TCallable&>(*current.GetNode()); current = callable.GetResult(); @@ -1704,7 +1704,7 @@ TNode* TRuntimeNode::GetValue() const { void TRuntimeNode::Freeze() { while (!IsImmediate()) { - MKQL_ENSURE(GetNode()->GetType()->IsCallable(), "Wrong type"); + MKQL_ENSURE(GetNode()->GetType()->IsCallable(), "Wrong type"); const auto& callable = static_cast<const TCallable&>(*GetNode()); if (!callable.HasResult()) @@ -1721,9 +1721,9 @@ bool TAnyType::IsSameType(const TAnyType& typeToCompare) const { bool TAnyType::IsConvertableTo(const TAnyType& typeToCompare, bool ignoreTagged) const { Y_UNUSED(ignoreTagged); - return IsSameType(typeToCompare); -} - + return IsSameType(typeToCompare); +} + void TAnyType::DoUpdateLinks(const THashMap<TNode*, TNode*>& links) { Y_UNUSED(links); } @@ -1831,20 +1831,20 @@ bool TTupleType::IsSameType(const TTupleType& typeToCompare) const { } bool TTupleType::IsConvertableTo(const TTupleType& typeToCompare, bool ignoreTagged) const { - if (this == &typeToCompare) - return true; - + if (this == &typeToCompare) + return true; + if (ElementsCount != typeToCompare.ElementsCount) - return false; - - for (size_t index = 0; index < ElementsCount; ++index) { + return false; + + for (size_t index = 0; index < ElementsCount; ++index) { if (!Elements[index]->IsConvertableTo(*typeToCompare.Elements[index], ignoreTagged)) - return false; - } - - return true; -} - + return false; + } + + return true; +} + void TTupleType::DoUpdateLinks(const THashMap<TNode*, TNode*>& links) { for (ui32 i = 0; i < ElementsCount; ++i) { auto& element = Elements[i]; @@ -2002,9 +2002,9 @@ bool TResourceType::IsSameType(const TResourceType& typeToCompare) const { bool TResourceType::IsConvertableTo(const TResourceType& typeToCompare, bool ignoreTagged) const { Y_UNUSED(ignoreTagged); - return IsSameType(typeToCompare); -} - + return IsSameType(typeToCompare); +} + void TResourceType::DoUpdateLinks(const THashMap<TNode*, TNode*>& links) { Y_UNUSED(links); } diff --git a/ydb/library/yql/minikql/mkql_node.h b/ydb/library/yql/minikql/mkql_node.h index 971be0a91fe..ee74455ea69 100644 --- a/ydb/library/yql/minikql/mkql_node.h +++ b/ydb/library/yql/minikql/mkql_node.h @@ -1,17 +1,17 @@ #pragma once #include "defs.h" -#include "mkql_mem_info.h" +#include "mkql_mem_info.h" #include "mkql_alloc.h" - + #include <ydb/library/yql/public/udf/udf_value.h> - + #include <util/generic/hash.h> #include <util/generic/hash_set.h> #include <util/generic/maybe.h> #include <library/cpp/deprecated/enum_codegen/enum_codegen.h> #include <stack> - + namespace NKikimr { namespace NMiniKQL { @@ -128,17 +128,17 @@ class TTypeType; class TTypeEnvironment; // if kind is above 0xf, it must ends with 0x02 pattern (like Void) -#define MKQL_TYPE_KINDS(XX) \ - XX(Type, 0) \ +#define MKQL_TYPE_KINDS(XX) \ + XX(Type, 0) \ XX(Variant, 1) \ - XX(Void, 2) \ - XX(Data, 3) \ + XX(Void, 2) \ + XX(Data, 3) \ XX(Stream, 4) \ - XX(Struct, 5) \ - XX(List, 6) \ - XX(Optional, 7) \ - XX(Dict, 8) \ - XX(Callable, 9) \ + XX(Struct, 5) \ + XX(List, 6) \ + XX(Optional, 7) \ + XX(Dict, 8) \ + XX(Callable, 9) \ XX(Any, 10) \ XX(Tuple, 11) \ XX(Resource, 12) \ @@ -149,27 +149,27 @@ class TTypeEnvironment; XX(EmptyDict, 32 + 2) \ XX(Tagged, 48 + 7) \ XX(Block, 16 + 13) - + class TType : public TNode { public: enum class EKind : ui8 { - MKQL_TYPE_KINDS(ENUM_VALUE_GEN) + MKQL_TYPE_KINDS(ENUM_VALUE_GEN) }; - inline EKind GetKind() const { + inline EKind GetKind() const { return Kind; } static TStringBuf KindAsStr(EKind kind); - TStringBuf GetKindAsStr() const; - -#define MKQL_KIND_ACCESSOR(name, value) \ - inline bool Is##name() const { return Kind == EKind::name; } - - MKQL_TYPE_KINDS(MKQL_KIND_ACCESSOR) - -#undef MKQL_KIND_ACCESSOR - + TStringBuf GetKindAsStr() const; + +#define MKQL_KIND_ACCESSOR(name, value) \ + inline bool Is##name() const { return Kind == EKind::name; } + + MKQL_TYPE_KINDS(MKQL_KIND_ACCESSOR) + +#undef MKQL_KIND_ACCESSOR + bool IsSameType(const TType& typeToCompare) const; bool IsConvertableTo(const TType& typeToCompare, bool ignoreTagged = false) const; void Accept(INodeVisitor& visitor); @@ -196,9 +196,9 @@ public: using TType::IsSameType; bool IsSameType(const TTypeType& typeToCompare) const; - using TType::IsConvertableTo; + using TType::IsConvertableTo; bool IsConvertableTo(const TTypeType& typeToCompare, bool ignoreTagged = false) const; - + private: TTypeType() : TType() @@ -220,9 +220,9 @@ public: using TType::IsSameType; bool IsSameType(const TSingularType<SingularKind>& typeToCompare) const; - using TType::IsConvertableTo; + using TType::IsConvertableTo; bool IsConvertableTo(const TSingularType<SingularKind>& typeToCompare, bool ignoreTagged = false) const; - + private: TSingularType(TTypeType* type) : TType(SingularKind, type) @@ -486,10 +486,10 @@ private: TEmptyDictType* TypeOfEmptyDict; TEmptyDict* EmptyDict; TDataType* Ui32; - TDataType* Ui64; + TDataType* Ui64; TAnyType* AnyType; TStructLiteral* EmptyStruct; - TTupleLiteral* EmptyTuple; + TTupleLiteral* EmptyTuple; TListLiteral* ListOfVoid; }; @@ -517,13 +517,13 @@ class TDataType : public TType { friend class TType; public: static TDataType* Create(NUdf::TDataTypeId schemeType, const TTypeEnvironment& env); - + using TType::IsSameType; bool IsSameType(const TDataType& typeToCompare) const; - - using TType::IsConvertableTo; + + using TType::IsConvertableTo; bool IsConvertableTo(const TDataType& typeToCompare, bool ignoreTagged = false) const; - + NUdf::TDataTypeId GetSchemeType() const { return SchemeType; } @@ -565,7 +565,7 @@ class TDataLiteral : public TNode, private NUdf::TUnboxedValuePod { friend class TNode; public: static TDataLiteral* Create(const NUdf::TUnboxedValuePod& value, TDataType* type, const TTypeEnvironment& env); - + TDataType* GetType() const { return static_cast<TDataType*>(GetGenericType()); } @@ -584,41 +584,41 @@ private: void DoFreeze(const TTypeEnvironment& env); }; -struct TStructMember { - TStructMember() - : Type(nullptr) - , Index(nullptr) - { - } - - TStructMember(const TStringBuf& name, TType* type, ui32* index = nullptr) - : Name(name) - , Type(type) - , Index(index) - { - } - - bool operator<(const TStructMember& rhs) const { - return Name < rhs.Name; - } - - TStringBuf Name; - TType* Type; - ui32* Index; -}; - +struct TStructMember { + TStructMember() + : Type(nullptr) + , Index(nullptr) + { + } + + TStructMember(const TStringBuf& name, TType* type, ui32* index = nullptr) + : Name(name) + , Type(type) + , Index(index) + { + } + + bool operator<(const TStructMember& rhs) const { + return Name < rhs.Name; + } + + TStringBuf Name; + TType* Type; + ui32* Index; +}; + class TStructType : public TType { friend class TType; public: static TStructType* Create(const std::pair<TString, TType*>* members, ui32 membersCount, const TTypeEnvironment& env); static TStructType* Create(ui32 membersCount, const TStructMember* members, const TTypeEnvironment& env); - - using TType::IsSameType; + + using TType::IsSameType; bool IsSameType(const TStructType& typeToCompare) const; - - using TType::IsConvertableTo; + + using TType::IsConvertableTo; bool IsConvertableTo(const TStructType& typeToCompare, bool ignoreTagged = false) const; - + ui32 GetMembersCount() const { return MembersCount; } @@ -687,13 +687,13 @@ class TListType : public TType { friend class TType; public: static TListType* Create(TType* itemType, const TTypeEnvironment& env); - + using TType::IsSameType; bool IsSameType(const TListType& typeToCompare) const; - - using TType::IsConvertableTo; + + using TType::IsConvertableTo; bool IsConvertableTo(const TListType& typeToCompare, bool ignoreTagged = false) const; - + TType* GetItemType() const { return Data; } @@ -801,13 +801,13 @@ class TOptionalType : public TType { friend class TType; public: static TOptionalType* Create(TType* itemType, const TTypeEnvironment& env); - + using TType::IsSameType; bool IsSameType(const TOptionalType& typeToCompare) const; - - using TType::IsConvertableTo; + + using TType::IsConvertableTo; bool IsConvertableTo(const TOptionalType& typeToCompare, bool ignoreTagged = false) const; - + TType* GetItemType() const { return Data; } @@ -828,7 +828,7 @@ friend class TNode; public: static TOptionalLiteral* Create(TRuntimeNode item, TOptionalType* type, const TTypeEnvironment& env); static TOptionalLiteral* Create(TOptionalType* type, const TTypeEnvironment& env); - + TOptionalType* GetType() const { return static_cast<TOptionalType*>(GetGenericType()); } @@ -860,13 +860,13 @@ class TDictType : public TType { friend class TType; public: static TDictType* Create(TType* keyType, TType* payloadType, const TTypeEnvironment& env); - + using TType::IsSameType; bool IsSameType(const TDictType& typeToCompare) const; - - using TType::IsConvertableTo; + + using TType::IsConvertableTo; bool IsConvertableTo(const TDictType& typeToCompare, bool ignoreTagged = false) const; - + TType* GetKeyType() const { return KeyType; } @@ -932,12 +932,12 @@ public: return OptionalArgs; } - using TType::IsSameType; + using TType::IsSameType; bool IsSameType(const TCallableType& typeToCompare) const; - - using TType::IsConvertableTo; + + using TType::IsConvertableTo; bool IsConvertableTo(const TCallableType& typeToCompare, bool ignoreTagged = false) const; - + TStringBuf GetName() const { return Name.Str(); } @@ -1054,7 +1054,7 @@ inline TType* TRuntimeNode::GetStaticType() const { if (IsImmediate()) { return GetNode()->GetGenericType(); } else { - MKQL_ENSURE(GetNode()->GetType()->IsCallable(), "Wrong type"); + MKQL_ENSURE(GetNode()->GetType()->IsCallable(), "Wrong type"); const auto& callable = static_cast<const TCallable&>(*GetNode()); return callable.GetType()->GetReturnType(); @@ -1120,13 +1120,13 @@ class TTupleType : public TType { friend class TType; public: static TTupleType* Create(ui32 elementsCount, TType* const* elements, const TTypeEnvironment& env); - + using TType::IsSameType; bool IsSameType(const TTupleType& typeToCompare) const; - - using TType::IsConvertableTo; + + using TType::IsConvertableTo; bool IsConvertableTo(const TTupleType& typeToCompare, bool ignoreTagged = false) const; - + ui32 GetElementsCount() const { return ElementsCount; } @@ -1185,9 +1185,9 @@ public: using TType::IsSameType; bool IsSameType(const TResourceType& typeToCompare) const; - using TType::IsConvertableTo; + using TType::IsConvertableTo; bool IsConvertableTo(const TResourceType& typeToCompare, bool ignoreTagged = false) const; - + TStringBuf GetTag() const { return Tag.Str(); } diff --git a/ydb/library/yql/minikql/mkql_node_builder.cpp b/ydb/library/yql/minikql/mkql_node_builder.cpp index e3fac43e076..34466b89446 100644 --- a/ydb/library/yql/minikql/mkql_node_builder.cpp +++ b/ydb/library/yql/minikql/mkql_node_builder.cpp @@ -1,8 +1,8 @@ #include "mkql_node_builder.h" #include "mkql_node_printer.h" -#include <util/generic/algorithm.h> - +#include <util/generic/algorithm.h> + namespace NKikimr { namespace NMiniKQL { @@ -74,8 +74,8 @@ TType* UnpackOptional(TRuntimeNode data, bool& isOptional) { } TType* UnpackOptional(TType* type, bool& isOptional) { - isOptional = type->IsOptional(); - if (!isOptional) + isOptional = type->IsOptional(); + if (!isOptional) return type; auto optType = static_cast<TOptionalType*>(type); @@ -103,7 +103,7 @@ void TStructTypeBuilder::Reserve(ui32 size) { Members.reserve(size); } -TStructTypeBuilder& TStructTypeBuilder::Add(const TStringBuf& name, TType* type, ui32* index) { +TStructTypeBuilder& TStructTypeBuilder::Add(const TStringBuf& name, TType* type, ui32* index) { Members.push_back(TStructMember(Env->InternName(name).Str(), type, index)); return *this; } @@ -112,19 +112,19 @@ TStructType* TStructTypeBuilder::Build() { if (Members.empty()) return Env->GetEmptyStruct()->GetType(); - Sort(Members.begin(), Members.end()); + Sort(Members.begin(), Members.end()); return TStructType::Create(Members.size(), Members.data(), *Env); } -void TStructTypeBuilder::FillIndexes() { - ui32 index = 0; - for (const TStructMember& member: Members) { - if (member.Index) { - *(member.Index) = index++; - } - } -} - +void TStructTypeBuilder::FillIndexes() { + ui32 index = 0; + for (const TStructMember& member: Members) { + if (member.Index) { + *(member.Index) = index++; + } + } +} + void TStructTypeBuilder::Clear() { Members.clear(); } diff --git a/ydb/library/yql/minikql/mkql_node_builder.h b/ydb/library/yql/minikql/mkql_node_builder.h index cc8dcfc94e3..46a43dc3529 100644 --- a/ydb/library/yql/minikql/mkql_node_builder.h +++ b/ydb/library/yql/minikql/mkql_node_builder.h @@ -29,9 +29,9 @@ public: TStructTypeBuilder(const TStructTypeBuilder&) = default; TStructTypeBuilder& operator=(const TStructTypeBuilder&) = default; void Reserve(ui32 size); - TStructTypeBuilder& Add(const TStringBuf& name, TType* type, ui32* index = nullptr); + TStructTypeBuilder& Add(const TStringBuf& name, TType* type, ui32* index = nullptr); TStructType* Build(); - void FillIndexes(); + void FillIndexes(); void Clear(); private: diff --git a/ydb/library/yql/minikql/mkql_node_cast.cpp b/ydb/library/yql/minikql/mkql_node_cast.cpp index 6216563cbcd..0a62a7d6156 100644 --- a/ydb/library/yql/minikql/mkql_node_cast.cpp +++ b/ydb/library/yql/minikql/mkql_node_cast.cpp @@ -1,25 +1,25 @@ -#include "mkql_node_cast.h" - -#define MKQL_AS_VALUE(name, suffix) \ - template <> \ - T##name##suffix* \ - AsValue(TRuntimeNode node, const TSourceLocation& location) { \ - MKQL_ENSURE_WITH_LOC( \ - location, \ - node.HasValue() && node.GetStaticType()->Is##name(), \ - "Expected value of T" #name #suffix << \ - " but got " << node.GetStaticType()->GetKindAsStr()); \ - return static_cast<T##name##suffix*>(node.GetValue()); \ - } - -#define MKQL_AS_TYPE(name) \ - template <> \ - T##name##Type* \ +#include "mkql_node_cast.h" + +#define MKQL_AS_VALUE(name, suffix) \ + template <> \ + T##name##suffix* \ + AsValue(TRuntimeNode node, const TSourceLocation& location) { \ + MKQL_ENSURE_WITH_LOC( \ + location, \ + node.HasValue() && node.GetStaticType()->Is##name(), \ + "Expected value of T" #name #suffix << \ + " but got " << node.GetStaticType()->GetKindAsStr()); \ + return static_cast<T##name##suffix*>(node.GetValue()); \ + } + +#define MKQL_AS_TYPE(name) \ + template <> \ + T##name##Type* \ AsType(TType* type, const TSourceLocation& location) { \ - MKQL_ENSURE_WITH_LOC( \ - location, \ + MKQL_ENSURE_WITH_LOC( \ + location, \ type->Is##name(), \ - "Expected type of T" #name "Type" \ + "Expected type of T" #name "Type" \ " but got " << type->GetKindAsStr()); \ return static_cast<T##name##Type*>(type); \ } \ @@ -32,66 +32,66 @@ "Expected type of T" #name "Type" \ " but got " << type->GetKindAsStr()); \ return static_cast<const T##name##Type*>(type); \ - } - -namespace NKikimr { -namespace NMiniKQL { - + } + +namespace NKikimr { +namespace NMiniKQL { + MKQL_AS_TYPE(Any) -MKQL_AS_TYPE(Callable) -MKQL_AS_TYPE(Data) -MKQL_AS_TYPE(Dict) -MKQL_AS_TYPE(List) -MKQL_AS_TYPE(Optional) -MKQL_AS_TYPE(Struct) -MKQL_AS_TYPE(Tuple) -MKQL_AS_TYPE(Type) -MKQL_AS_TYPE(Void) +MKQL_AS_TYPE(Callable) +MKQL_AS_TYPE(Data) +MKQL_AS_TYPE(Dict) +MKQL_AS_TYPE(List) +MKQL_AS_TYPE(Optional) +MKQL_AS_TYPE(Struct) +MKQL_AS_TYPE(Tuple) +MKQL_AS_TYPE(Type) +MKQL_AS_TYPE(Void) MKQL_AS_TYPE(Resource) MKQL_AS_TYPE(Variant) MKQL_AS_TYPE(Stream) MKQL_AS_TYPE(Flow) MKQL_AS_TYPE(Tagged) MKQL_AS_TYPE(Block) - + MKQL_AS_VALUE(Any, Type) -MKQL_AS_VALUE(Callable, Type) -MKQL_AS_VALUE(Data, Type) -MKQL_AS_VALUE(Dict, Type) -MKQL_AS_VALUE(List, Type) -MKQL_AS_VALUE(Optional, Type) -MKQL_AS_VALUE(Struct, Type) -MKQL_AS_VALUE(Tuple, Type) -MKQL_AS_VALUE(Type, Type) -MKQL_AS_VALUE(Void, Type) +MKQL_AS_VALUE(Callable, Type) +MKQL_AS_VALUE(Data, Type) +MKQL_AS_VALUE(Dict, Type) +MKQL_AS_VALUE(List, Type) +MKQL_AS_VALUE(Optional, Type) +MKQL_AS_VALUE(Struct, Type) +MKQL_AS_VALUE(Tuple, Type) +MKQL_AS_VALUE(Type, Type) +MKQL_AS_VALUE(Void, Type) MKQL_AS_VALUE(Variant, Type) MKQL_AS_VALUE(Stream, Type) MKQL_AS_VALUE(Flow, Type) - -MKQL_AS_VALUE(Data, Literal) -MKQL_AS_VALUE(Dict, Literal) -MKQL_AS_VALUE(List, Literal) -MKQL_AS_VALUE(Optional, Literal) -MKQL_AS_VALUE(Struct, Literal) -MKQL_AS_VALUE(Tuple, Literal) + +MKQL_AS_VALUE(Data, Literal) +MKQL_AS_VALUE(Dict, Literal) +MKQL_AS_VALUE(List, Literal) +MKQL_AS_VALUE(Optional, Literal) +MKQL_AS_VALUE(Struct, Literal) +MKQL_AS_VALUE(Tuple, Literal) MKQL_AS_VALUE(Variant, Literal) - -TCallable* AsCallable( - const TStringBuf& name, - TRuntimeNode node, - const TSourceLocation& location) -{ - MKQL_ENSURE_WITH_LOC(location, - !node.IsImmediate() && node.GetNode()->GetType()->IsCallable(), - "Expected callable " << name); - - auto callable = static_cast<TCallable*>(node.GetNode()); - MKQL_ENSURE_WITH_LOC(location, - callable->GetType()->GetName() == name, - "Expected callable " << name); - - return callable; -} - -} // namespace NMiniKQL -} // namespace NKikimr + +TCallable* AsCallable( + const TStringBuf& name, + TRuntimeNode node, + const TSourceLocation& location) +{ + MKQL_ENSURE_WITH_LOC(location, + !node.IsImmediate() && node.GetNode()->GetType()->IsCallable(), + "Expected callable " << name); + + auto callable = static_cast<TCallable*>(node.GetNode()); + MKQL_ENSURE_WITH_LOC(location, + callable->GetType()->GetName() == name, + "Expected callable " << name); + + return callable; +} + +} // namespace NMiniKQL +} // namespace NKikimr diff --git a/ydb/library/yql/minikql/mkql_node_cast.h b/ydb/library/yql/minikql/mkql_node_cast.h index 95c220d4d52..9fd7dec896f 100644 --- a/ydb/library/yql/minikql/mkql_node_cast.h +++ b/ydb/library/yql/minikql/mkql_node_cast.h @@ -1,24 +1,24 @@ -#pragma once - -#include "mkql_node.h" - -#include <util/system/src_location.h> - - -#define AS_VALUE(type, node) ::NKikimr::NMiniKQL::AsValue<type>((node), __LOCATION__) -#define AS_TYPE(type, node) ::NKikimr::NMiniKQL::AsType<type>((node), __LOCATION__) +#pragma once + +#include "mkql_node.h" + +#include <util/system/src_location.h> + + +#define AS_VALUE(type, node) ::NKikimr::NMiniKQL::AsValue<type>((node), __LOCATION__) +#define AS_TYPE(type, node) ::NKikimr::NMiniKQL::AsType<type>((node), __LOCATION__) #define AS_CALLABLE(name, node) ::NKikimr::NMiniKQL::AsCallable(TStringBuf(name), node, __LOCATION__) - - -namespace NKikimr { -namespace NMiniKQL { - -template <typename T> -T* AsValue(TRuntimeNode node, const TSourceLocation& location); - -template <typename T> + + +namespace NKikimr { +namespace NMiniKQL { + +template <typename T> +T* AsValue(TRuntimeNode node, const TSourceLocation& location); + +template <typename T> T* AsType(TType* type, const TSourceLocation& location); - + template <typename T> const T* AsType(const TType* type, const TSourceLocation& location); @@ -28,10 +28,10 @@ T* AsType(TRuntimeNode node, const TSourceLocation& location) { return AsType<T>(node.GetStaticType(), location); } -TCallable* AsCallable( - const TStringBuf& name, - TRuntimeNode node, - const TSourceLocation& location); - -} // namespace NMiniKQL -} // namespace NKikimr +TCallable* AsCallable( + const TStringBuf& name, + TRuntimeNode node, + const TSourceLocation& location); + +} // namespace NMiniKQL +} // namespace NKikimr diff --git a/ydb/library/yql/minikql/mkql_node_cast_ut.cpp b/ydb/library/yql/minikql/mkql_node_cast_ut.cpp index b35ea2373fc..0ae19fa5e45 100644 --- a/ydb/library/yql/minikql/mkql_node_cast_ut.cpp +++ b/ydb/library/yql/minikql/mkql_node_cast_ut.cpp @@ -1,72 +1,72 @@ -#include "mkql_node.h" -#include "mkql_node_cast.h" +#include "mkql_node.h" +#include "mkql_node_cast.h" #include "mkql_node_builder.h" - + #include <library/cpp/testing/unittest/registar.h> - - -namespace NKikimr { -namespace NMiniKQL { - -class TMiniKQLNodeCast: public TTestBase -{ - UNIT_TEST_SUITE(TMiniKQLNodeCast); - UNIT_TEST(AsTypeTest); - UNIT_TEST_EXCEPTION(BadAsTypeTest, yexception); - UNIT_TEST(AsValueTest); - UNIT_TEST_EXCEPTION(BadAsValueTest, yexception); - UNIT_TEST_SUITE_END(); - - void AsTypeTest() { - TRuntimeNode node = Uint32AsNode(123); - TDataType* type = AS_TYPE(TDataType, node); - UNIT_ASSERT_EQUAL(type, node.GetStaticType()); - UNIT_ASSERT_EQUAL(type, node.GetNode()->GetType()); - } - - void BadAsTypeTest() { - TRuntimeNode node = Uint32AsNode(123); - TCallableType* type = AS_TYPE(TCallableType, node); + + +namespace NKikimr { +namespace NMiniKQL { + +class TMiniKQLNodeCast: public TTestBase +{ + UNIT_TEST_SUITE(TMiniKQLNodeCast); + UNIT_TEST(AsTypeTest); + UNIT_TEST_EXCEPTION(BadAsTypeTest, yexception); + UNIT_TEST(AsValueTest); + UNIT_TEST_EXCEPTION(BadAsValueTest, yexception); + UNIT_TEST_SUITE_END(); + + void AsTypeTest() { + TRuntimeNode node = Uint32AsNode(123); + TDataType* type = AS_TYPE(TDataType, node); + UNIT_ASSERT_EQUAL(type, node.GetStaticType()); + UNIT_ASSERT_EQUAL(type, node.GetNode()->GetType()); + } + + void BadAsTypeTest() { + TRuntimeNode node = Uint32AsNode(123); + TCallableType* type = AS_TYPE(TCallableType, node); Y_UNUSED(type); - } - - void AsValueTest() { - TRuntimeNode dataNode = Uint32AsNode(123); - - TCallableType* ctype = TCallableType::Create( - "callable", dataNode.GetStaticType(), + } + + void AsValueTest() { + TRuntimeNode dataNode = Uint32AsNode(123); + + TCallableType* ctype = TCallableType::Create( + "callable", dataNode.GetStaticType(), 0, nullptr, nullptr, Env); - - TCallable* callable = TCallable::Create(dataNode, ctype, Env); - - TRuntimeNode node(callable, false); - node.Freeze(); - - TDataLiteral* value = AS_VALUE(TDataLiteral, node); - UNIT_ASSERT_EQUAL(value, dataNode.GetNode()); - } - - void BadAsValueTest() { - TRuntimeNode node = Uint32AsNode(123); - TListLiteral* list = AS_VALUE(TListLiteral, node); + + TCallable* callable = TCallable::Create(dataNode, ctype, Env); + + TRuntimeNode node(callable, false); + node.Freeze(); + + TDataLiteral* value = AS_VALUE(TDataLiteral, node); + UNIT_ASSERT_EQUAL(value, dataNode.GetNode()); + } + + void BadAsValueTest() { + TRuntimeNode node = Uint32AsNode(123); + TListLiteral* list = AS_VALUE(TListLiteral, node); Y_UNUSED(list); - } - + } + TMiniKQLNodeCast() : Env(Alloc) {} -private: - TRuntimeNode Uint32AsNode(ui32 value) { +private: + TRuntimeNode Uint32AsNode(ui32 value) { return TRuntimeNode(BuildDataLiteral(NUdf::TUnboxedValuePod(value), NUdf::EDataSlot::Uint32, Env), true); - } - -private: + } + +private: TScopedAlloc Alloc; - TTypeEnvironment Env; -}; - -UNIT_TEST_SUITE_REGISTRATION(TMiniKQLNodeCast); - + TTypeEnvironment Env; +}; + +UNIT_TEST_SUITE_REGISTRATION(TMiniKQLNodeCast); + } // namespace NMiniKQL } // namespace NKikimr diff --git a/ydb/library/yql/minikql/mkql_node_serialization.cpp b/ydb/library/yql/minikql/mkql_node_serialization.cpp index 6806bb58fe3..e314f38cd61 100644 --- a/ydb/library/yql/minikql/mkql_node_serialization.cpp +++ b/ydb/library/yql/minikql/mkql_node_serialization.cpp @@ -8,9 +8,9 @@ #include <library/cpp/containers/stack_vector/stack_vec.h> #include <library/cpp/packedtypes/zigzag.h> -#include <util/generic/algorithm.h> +#include <util/generic/algorithm.h> #include <util/system/unaligned_mem.h> - + namespace NKikimr { namespace NMiniKQL { @@ -1258,16 +1258,16 @@ namespace { TNode* ReadStructType() { const ui32 membersCount = ReadVar32(); - TStackVec<TStructMember> members(membersCount); + TStackVec<TStructMember> members(membersCount); for (ui32 i = 0; i < membersCount; ++i) { auto memberTypeNode = PopNode(); if (memberTypeNode->GetType()->GetKind() != TType::EKind::Type) ThrowCorrupted(); - members[i].Type = static_cast<TType*>(memberTypeNode); + members[i].Type = static_cast<TType*>(memberTypeNode); } for (ui32 i = 0; i < membersCount; ++i) { - members[i].Name = ReadName(); + members[i].Name = ReadName(); } auto node = TStructType::Create(membersCount, members.data(), Env); diff --git a/ydb/library/yql/minikql/mkql_node_ut.cpp b/ydb/library/yql/minikql/mkql_node_ut.cpp index b946247908a..16126272462 100644 --- a/ydb/library/yql/minikql/mkql_node_ut.cpp +++ b/ydb/library/yql/minikql/mkql_node_ut.cpp @@ -1,11 +1,11 @@ #include "mkql_node.h" #include "mkql_node_visitor.h" - + #include <library/cpp/testing/unittest/registar.h> -#include <util/generic/algorithm.h> - - +#include <util/generic/algorithm.h> + + namespace NKikimr { namespace NMiniKQL { @@ -15,7 +15,7 @@ Y_UNIT_TEST_SUITE(TMiniKQLNodeTest) { TTypeEnvironment env(alloc); { auto type = env.GetTypeOfType(); - UNIT_ASSERT(type->IsType()); + UNIT_ASSERT(type->IsType()); UNIT_ASSERT(type->GetType() == type); UNIT_ASSERT(type->IsSameType(*type)); } @@ -26,7 +26,7 @@ Y_UNIT_TEST_SUITE(TMiniKQLNodeTest) { TTypeEnvironment env(alloc); { auto type = env.GetTypeOfVoid(); - UNIT_ASSERT(type->IsVoid()); + UNIT_ASSERT(type->IsVoid()); UNIT_ASSERT(type->GetType() == env.GetTypeOfType()); UNIT_ASSERT(type->IsSameType(*type)); } @@ -36,7 +36,7 @@ Y_UNIT_TEST_SUITE(TMiniKQLNodeTest) { TScopedAlloc alloc; TTypeEnvironment env(alloc); TVoid* voidObj = env.GetVoid(); - UNIT_ASSERT(voidObj->GetType()->IsVoid()); + UNIT_ASSERT(voidObj->GetType()->IsVoid()); UNIT_ASSERT(voidObj->GetType() == env.GetTypeOfVoid()); } @@ -292,13 +292,13 @@ Y_UNIT_TEST_SUITE(TMiniKQLNodeTest) { UNIT_ASSERT(ctype2withPayload2clone->IsSameType(*ctype2withPayload2)); TCallableType* ctype2optArg = TCallableType::Create("c2", env.GetVoid()->GetGenericType(), types.size(), types.data(), nullptr, env); - + ctype2optArg->SetOptionalArgumentsCount(0); UNIT_ASSERT(ctype2optArg->IsSameType(*ctype2)); - UNIT_ASSERT(ctype2optArg->IsConvertableTo(*ctype2)); - UNIT_ASSERT(ctype2->IsSameType(*ctype2optArg)); - UNIT_ASSERT(ctype2->IsConvertableTo(*ctype2optArg)); - + UNIT_ASSERT(ctype2optArg->IsConvertableTo(*ctype2)); + UNIT_ASSERT(ctype2->IsSameType(*ctype2optArg)); + UNIT_ASSERT(ctype2->IsConvertableTo(*ctype2optArg)); + UNIT_ASSERT_EXCEPTION(ctype2optArg->SetOptionalArgumentsCount(1), yexception); UNIT_ASSERT_EXCEPTION(ctype2optArg->SetOptionalArgumentsCount(3), yexception); } diff --git a/ydb/library/yql/minikql/mkql_node_visitor.cpp b/ydb/library/yql/minikql/mkql_node_visitor.cpp index 49f58433634..5a4ac8e4798 100644 --- a/ydb/library/yql/minikql/mkql_node_visitor.cpp +++ b/ydb/library/yql/minikql/mkql_node_visitor.cpp @@ -556,12 +556,12 @@ const TExploringNodeVisitor::TNodesVec& TExploringNodeVisitor::GetConsumerNodes( template <bool InPlace> TRuntimeNode SinglePassVisitCallablesImpl(TRuntimeNode root, TExploringNodeVisitor& explorer, const TCallableVisitFuncProvider& funcProvider, const TTypeEnvironment& env, bool& wereChanges) -{ +{ auto& nodes = explorer.GetNodes(); wereChanges = false; - for (TNode* exploredNode : nodes) { + for (TNode* exploredNode : nodes) { TNode* node; if (!InPlace) { node = exploredNode->CloneOnCallableWrite(env); @@ -570,7 +570,7 @@ TRuntimeNode SinglePassVisitCallablesImpl(TRuntimeNode root, TExploringNodeVisit node->Freeze(env); } - if (node->GetType()->IsCallable()) { + if (node->GetType()->IsCallable()) { auto& callable = static_cast<TCallable&>(*node); if (!callable.HasResult()) { const auto& callableType = callable.GetType(); diff --git a/ydb/library/yql/minikql/mkql_opt_literal.cpp b/ydb/library/yql/minikql/mkql_opt_literal.cpp index 5a7ffd2eb76..832a87ea80c 100644 --- a/ydb/library/yql/minikql/mkql_opt_literal.cpp +++ b/ydb/library/yql/minikql/mkql_opt_literal.cpp @@ -1,15 +1,15 @@ -#include "mkql_opt_literal.h" -#include "mkql_node_cast.h" -#include "mkql_node_builder.h" +#include "mkql_opt_literal.h" +#include "mkql_node_cast.h" +#include "mkql_node_builder.h" #include "mkql_node_visitor.h" #include "mkql_program_builder.h" #include "mkql_node_printer.h" - + #include <library/cpp/containers/stack_vector/stack_vec.h> - + #include <util/generic/singleton.h> - + namespace NKikimr { namespace NMiniKQL { @@ -17,16 +17,16 @@ using namespace NDetail; namespace { -TNode* LiteralAddMember( - const TStructLiteral& oldStruct, - const TStructType& newStructType, - TRuntimeNode newMember, - TRuntimeNode position, - const TTypeEnvironment& env) -{ +TNode* LiteralAddMember( + const TStructLiteral& oldStruct, + const TStructType& newStructType, + TRuntimeNode newMember, + TRuntimeNode position, + const TTypeEnvironment& env) +{ TStructLiteralBuilder resultBuilder(env); - TDataLiteral* positionData = AS_VALUE(TDataLiteral, position); + TDataLiteral* positionData = AS_VALUE(TDataLiteral, position); const ui32 positionValue = positionData->AsValue().Get<ui32>(); MKQL_ENSURE(positionValue <= oldStruct.GetType()->GetMembersCount(), "Bad member index"); @@ -66,13 +66,13 @@ TNode* LiteralRemoveMember( TRuntimeNode OptimizeIf(TCallable& callable, const TTypeEnvironment& env) { Y_UNUSED(env); - MKQL_ENSURE(callable.GetInputsCount() == 3, "Expected 3 arguments"); + MKQL_ENSURE(callable.GetInputsCount() == 3, "Expected 3 arguments"); auto predicateInput = callable.GetInput(0); auto thenInput = callable.GetInput(1); auto elseInput = callable.GetInput(2); if (predicateInput.HasValue()) { - TDataLiteral* data = AS_VALUE(TDataLiteral, predicateInput); + TDataLiteral* data = AS_VALUE(TDataLiteral, predicateInput); const bool predicateValue = data->AsValue().Get<bool>(); return predicateValue ? thenInput : elseInput; } @@ -85,11 +85,11 @@ TRuntimeNode OptimizeIf(TCallable& callable, const TTypeEnvironment& env) { } TRuntimeNode OptimizeSize(TCallable& callable, const TTypeEnvironment& env) { - MKQL_ENSURE(callable.GetInputsCount() == 1, "Expected 1 arguments"); + MKQL_ENSURE(callable.GetInputsCount() == 1, "Expected 1 arguments"); auto dataInput = callable.GetInput(0); if (dataInput.HasValue()) { - TDataLiteral* value = AS_VALUE(TDataLiteral, dataInput); + TDataLiteral* value = AS_VALUE(TDataLiteral, dataInput); return TRuntimeNode(BuildDataLiteral(NUdf::TUnboxedValuePod((ui32)value->AsValue().AsStringRef().Size()), NUdf::EDataSlot::Uint32, env), true); } @@ -97,7 +97,7 @@ TRuntimeNode OptimizeSize(TCallable& callable, const TTypeEnvironment& env) { } TRuntimeNode OptimizeLength(TCallable& callable, const TTypeEnvironment& env) { - MKQL_ENSURE(callable.GetInputsCount() == 1, "Expected 1 arguments"); + MKQL_ENSURE(callable.GetInputsCount() == 1, "Expected 1 arguments"); auto listOrDictInput = callable.GetInput(0); if (listOrDictInput.HasValue()) { @@ -111,15 +111,15 @@ TRuntimeNode OptimizeLength(TCallable& callable, const TTypeEnvironment& env) { } TRuntimeNode OptimizeAddMember(TCallable& callable, const TTypeEnvironment& env) { - MKQL_ENSURE(callable.GetInputsCount() == 3, "Expected 3 arguments"); - + MKQL_ENSURE(callable.GetInputsCount() == 3, "Expected 3 arguments"); + auto callableReturnType = callable.GetType()->GetReturnType(); - MKQL_ENSURE(callableReturnType->IsStruct(), "Expected struct"); + MKQL_ENSURE(callableReturnType->IsStruct(), "Expected struct"); const auto& newType = static_cast<TStructType&>(*callableReturnType); auto structInput = callable.GetInput(0); if (structInput.HasValue()) { - TStructLiteral* value = AS_VALUE(TStructLiteral, structInput); + TStructLiteral* value = AS_VALUE(TStructLiteral, structInput); return TRuntimeNode(LiteralAddMember(*value, newType, callable.GetInput(1), callable.GetInput(2), env), true); } @@ -142,18 +142,18 @@ TRuntimeNode OptimizeRemoveMember(TCallable& callable, const TTypeEnvironment& e TRuntimeNode OptimizeMember(TCallable& callable, const TTypeEnvironment& env) { Y_UNUSED(env); - MKQL_ENSURE(callable.GetInputsCount() == 2, "Expected 2 arguments"); + MKQL_ENSURE(callable.GetInputsCount() == 2, "Expected 2 arguments"); auto structInput = callable.GetInput(0); if (structInput.HasValue() && structInput.GetStaticType()->IsStruct()) { - TStructLiteral* value = AS_VALUE(TStructLiteral, structInput); + TStructLiteral* value = AS_VALUE(TStructLiteral, structInput); auto position = callable.GetInput(1); - TDataLiteral* positionData = AS_VALUE(TDataLiteral, position); + TDataLiteral* positionData = AS_VALUE(TDataLiteral, position); const ui32 positionValue = positionData->AsValue().Get<ui32>(); MKQL_ENSURE(positionValue < value->GetValuesCount(), "Bad member index"); - return value->GetValue(positionValue); + return value->GetValue(positionValue); } return TRuntimeNode(&callable, false); @@ -195,7 +195,7 @@ TRuntimeNode OptimizeMap(TCallable& callable, const TTypeEnvironment& env) { auto listType = static_cast<TListType*>(returnType); auto newItemInput = callable.GetInput(2); - if (listType->GetItemType()->IsVoid() && newItemInput.HasValue()) { + if (listType->GetItemType()->IsVoid() && newItemInput.HasValue()) { return TRuntimeNode(env.GetListOfVoid(), true); } @@ -212,7 +212,7 @@ TRuntimeNode OptimizeFlatMap(TCallable& callable, const TTypeEnvironment& env) { const auto listType = static_cast<TListType*>(returnType); const auto newItemInput = callable.GetInput(2); - if (listType->GetItemType()->IsVoid() && newItemInput.HasValue()) { + if (listType->GetItemType()->IsVoid() && newItemInput.HasValue()) { if (newItemInput.GetStaticType()->IsList()) { TListLiteral* list = AS_VALUE(TListLiteral, newItemInput); if (list->GetItemsCount() == 0) { @@ -231,16 +231,16 @@ TRuntimeNode OptimizeFlatMap(TCallable& callable, const TTypeEnvironment& env) { TRuntimeNode OptimizeCoalesce(TCallable& callable, const TTypeEnvironment& env) { Y_UNUSED(env); - MKQL_ENSURE(callable.GetInputsCount() == 2, "Expected 2 arguments"); + MKQL_ENSURE(callable.GetInputsCount() == 2, "Expected 2 arguments"); auto optionalInput = callable.GetInput(0); auto defaultInput = callable.GetInput(1); bool isDefaultOptional; UnpackOptional(defaultInput, isDefaultOptional); if (optionalInput.HasValue()) { - auto optionalData = AS_VALUE(TOptionalLiteral, optionalInput); - if (optionalData->HasItem()) { - return isDefaultOptional ? optionalInput : optionalData->GetItem(); + auto optionalData = AS_VALUE(TOptionalLiteral, optionalInput); + if (optionalData->HasItem()) { + return isDefaultOptional ? optionalInput : optionalData->GetItem(); } else { return defaultInput; } @@ -250,7 +250,7 @@ TRuntimeNode OptimizeCoalesce(TCallable& callable, const TTypeEnvironment& env) } TRuntimeNode OptimizeExists(TCallable& callable, const TTypeEnvironment& env) { - MKQL_ENSURE(callable.GetInputsCount() == 1, "Expected 1 arguments"); + MKQL_ENSURE(callable.GetInputsCount() == 1, "Expected 1 arguments"); auto optionalInput = callable.GetInput(0); if (optionalInput.HasValue()) { @@ -268,7 +268,7 @@ TRuntimeNode OptimizeNth(TCallable& callable, const TTypeEnvironment& env) { auto tupleInput = callable.GetInput(0); if (tupleInput.HasValue() && tupleInput.GetStaticType()->IsTuple()) { auto tuple = tupleInput.GetValue(); - auto indexData = AS_VALUE(TDataLiteral, callable.GetInput(1)); + auto indexData = AS_VALUE(TDataLiteral, callable.GetInput(1)); const ui32 index = indexData->AsValue().Get<ui32>(); const auto& value = static_cast<const TTupleLiteral&>(*tuple); @@ -340,7 +340,7 @@ struct TOptimizationFuncMapFiller { } }; -} // namespace +} // namespace TCallableVisitFuncProvider GetLiteralPropagationOptimizationFuncProvider() { return Singleton<TOptimizationFuncMapFiller>()->Provider; @@ -353,5 +353,5 @@ TRuntimeNode LiteralPropagationOptimization(TRuntimeNode root, const TTypeEnviro return SinglePassVisitCallables(root, explorer, GetLiteralPropagationOptimizationFuncProvider(), env, inPlace, wereChanges); } -} // namespace NMiniKQL -} // namespace NKikimr +} // namespace NMiniKQL +} // namespace NKikimr diff --git a/ydb/library/yql/minikql/mkql_opt_literal_ut.cpp b/ydb/library/yql/minikql/mkql_opt_literal_ut.cpp index 3a72f9672a3..c1c0a5b79da 100644 --- a/ydb/library/yql/minikql/mkql_opt_literal_ut.cpp +++ b/ydb/library/yql/minikql/mkql_opt_literal_ut.cpp @@ -1,7 +1,7 @@ -#include "mkql_opt_literal.h" +#include "mkql_opt_literal.h" #include "mkql_program_builder.h" #include "mkql_node_printer.h" -#include "mkql_function_registry.h" +#include "mkql_function_registry.h" #include <library/cpp/testing/unittest/registar.h> diff --git a/ydb/library/yql/minikql/mkql_program_builder.cpp b/ydb/library/yql/minikql/mkql_program_builder.cpp index 76bed6b52b9..47ae6e4c8fc 100644 --- a/ydb/library/yql/minikql/mkql_program_builder.cpp +++ b/ydb/library/yql/minikql/mkql_program_builder.cpp @@ -1,13 +1,13 @@ #include "mkql_program_builder.h" -#include "mkql_opt_literal.h" +#include "mkql_opt_literal.h" #include "mkql_node_visitor.h" -#include "mkql_node_cast.h" +#include "mkql_node_cast.h" #include "mkql_runtime_version.h" #include "ydb/library/yql/minikql/mkql_node_printer.h" #include "ydb/library/yql/minikql/mkql_function_registry.h" #include "ydb/library/yql/minikql/mkql_utils.h" #include "ydb/library/yql/minikql/mkql_type_builder.h" - + #include <util/string/cast.h> #include <util/string/printf.h> #include <array> @@ -17,46 +17,46 @@ using namespace std::string_view_literals; namespace NKikimr { namespace NMiniKQL { -namespace { - -struct TDataFunctionFlags { - enum { - HasBooleanResult = 0x01, - RequiresBooleanArgs = 0x02, - HasOptionalResult = 0x04, - AllowOptionalArgs = 0x08, - HasUi32Result = 0x10, - RequiresCompare = 0x20, - HasStringResult = 0x40, - RequiresStringArgs = 0x80, - RequiresHash = 0x100, +namespace { + +struct TDataFunctionFlags { + enum { + HasBooleanResult = 0x01, + RequiresBooleanArgs = 0x02, + HasOptionalResult = 0x04, + AllowOptionalArgs = 0x08, + HasUi32Result = 0x10, + RequiresCompare = 0x20, + HasStringResult = 0x40, + RequiresStringArgs = 0x80, + RequiresHash = 0x100, RequiresEquals = 0x200, AllowNull = 0x400, CommonOptionalResult = 0x800, SupportsTuple = 0x1000, SameOptionalArgs = 0x2000, - Default = 0x00 - }; -}; - -#define MKQL_BAD_TYPE_VISIT(NodeType, ScriptName) \ - void Visit(NodeType& node) override { \ + Default = 0x00 + }; +}; + +#define MKQL_BAD_TYPE_VISIT(NodeType, ScriptName) \ + void Visit(NodeType& node) override { \ Y_UNUSED(node); \ - MKQL_ENSURE(false, "Can't convert " #NodeType " to " ScriptName " object"); \ - } + MKQL_ENSURE(false, "Can't convert " #NodeType " to " ScriptName " object"); \ + } class TPythonTypeChecker : public TExploringNodeVisitor { using TExploringNodeVisitor::Visit; MKQL_BAD_TYPE_VISIT(TAnyType, "Python"); }; - + class TLuaTypeChecker : public TExploringNodeVisitor { using TExploringNodeVisitor::Visit; MKQL_BAD_TYPE_VISIT(TVoidType, "Lua"); MKQL_BAD_TYPE_VISIT(TAnyType, "Lua"); MKQL_BAD_TYPE_VISIT(TVariantType, "Lua"); }; - + class TJavascriptTypeChecker : public TExploringNodeVisitor { using TExploringNodeVisitor::Visit; MKQL_BAD_TYPE_VISIT(TAnyType, "Javascript"); @@ -85,9 +85,9 @@ void EnsureScriptSpecificTypes( return TJavascriptTypeChecker().Walk(funcType, env); default: MKQL_ENSURE(false, "Unknown script type " << static_cast<ui32>(scriptType)); - } -} - + } +} + ui32 GetNumericSchemeTypeLevel(NUdf::TDataTypeId typeId) { switch (typeId) { case NUdf::TDataType<ui8>::Id: @@ -226,32 +226,32 @@ bool ReduceOptionalElements(const TType* type, const TArrayRef<const ui32>& test } } // namespace - + std::string_view ScriptTypeAsStr(EScriptType type) { - switch (type) { + switch (type) { #define MKQL_SCRIPT_TYPE_CASE(name, value, ...) \ case EScriptType::name: return std::string_view(#name); - MKQL_SCRIPT_TYPES(MKQL_SCRIPT_TYPE_CASE) - -#undef MKQL_SCRIPT_TYPE_CASE - } // switch - + MKQL_SCRIPT_TYPES(MKQL_SCRIPT_TYPE_CASE) + +#undef MKQL_SCRIPT_TYPE_CASE + } // switch + return std::string_view("Unknown"); -} - +} + EScriptType ScriptTypeFromStr(std::string_view str) { TString lowerStr = TString(str); lowerStr.to_lower(); #define MKQL_SCRIPT_TYPE_FROM_STR(name, value, lowerName) \ if (lowerStr == #lowerName) return EScriptType::name; - - MKQL_SCRIPT_TYPES(MKQL_SCRIPT_TYPE_FROM_STR) -#undef MKQL_SCRIPT_TYPE_FROM_STR - - return EScriptType::Unknown; -} - + + MKQL_SCRIPT_TYPES(MKQL_SCRIPT_TYPE_FROM_STR) +#undef MKQL_SCRIPT_TYPE_FROM_STR + + return EScriptType::Unknown; +} + bool IsCustomPython(EScriptType type) { return type == EScriptType::CustomPython || type == EScriptType::CustomPython2 || @@ -284,8 +284,8 @@ const TTypeEnvironment& TProgramBuilder::GetTypeEnvironment() const { return Env; } -const IFunctionRegistry& TProgramBuilder::GetFunctionRegistry() const { - return FunctionRegistry; +const IFunctionRegistry& TProgramBuilder::GetFunctionRegistry() const { + return FunctionRegistry; } TType* TProgramBuilder::ChooseCommonType(TType* type1, TType* type2) { @@ -364,7 +364,7 @@ TRuntimeNode TProgramBuilder::Element(TRuntimeNode structObj, const std::string_ TRuntimeNode TProgramBuilder::AddMember(TRuntimeNode structObj, const std::string_view& memberName, TRuntimeNode memberValue) { auto oldType = structObj.GetStaticType(); - MKQL_ENSURE(oldType->IsStruct(), "Expected struct"); + MKQL_ENSURE(oldType->IsStruct(), "Expected struct"); const auto& oldTypeDetailed = static_cast<const TStructType&>(*oldType); TStructTypeBuilder newTypeBuilder(Env); @@ -503,7 +503,7 @@ TRuntimeNode TProgramBuilder::Fold(TRuntimeNode list, TRuntimeNode state, const const auto stateNodeArg = Arg(state.GetStaticType()); const auto itemArg = Arg(itemType); const auto newState = handler(itemArg, stateNodeArg); - MKQL_ENSURE(newState.GetStaticType()->IsSameType(*state.GetStaticType()), "State type is changed by the handler"); + MKQL_ENSURE(newState.GetStaticType()->IsSameType(*state.GetStaticType()), "State type is changed by the handler"); TCallableBuilder callableBuilder(Env, __func__, state.GetStaticType()); callableBuilder.Add(list); @@ -550,13 +550,13 @@ TRuntimeNode TProgramBuilder::Reduce(TRuntimeNode list, TRuntimeNode state1, const auto state3NodeArg = Arg(state3.GetStaticType()); const auto itemArg = Arg(itemType); const auto newState1 = handler1(itemArg, state1NodeArg); - MKQL_ENSURE(newState1.GetStaticType()->IsSameType(*state1.GetStaticType()), "State 1 type is changed by the handler"); + MKQL_ENSURE(newState1.GetStaticType()->IsSameType(*state1.GetStaticType()), "State 1 type is changed by the handler"); const auto newState2 = handler2(state1NodeArg); TRuntimeNode itemState2Arg = Arg(newState2.GetStaticType()); const auto newState3 = handler3(itemState2Arg, state3NodeArg); - MKQL_ENSURE(newState3.GetStaticType()->IsSameType(*state3.GetStaticType()), "State 3 type is changed by the handler"); + MKQL_ENSURE(newState3.GetStaticType()->IsSameType(*state3.GetStaticType()), "State 3 type is changed by the handler"); TCallableBuilder callableBuilder(Env, __func__, newState3.GetStaticType()); callableBuilder.Add(list); @@ -858,7 +858,7 @@ TRuntimeNode TProgramBuilder::Chain1Map(TRuntimeNode list, const TUnarySplitLamb TRuntimeNode TProgramBuilder::ToList(TRuntimeNode optional) { const auto optionalType = optional.GetStaticType(); - MKQL_ENSURE(optionalType->IsOptional(), "Expected optional"); + MKQL_ENSURE(optionalType->IsOptional(), "Expected optional"); const auto& optionalDetailedType = static_cast<const TOptionalType&>(*optionalType); const auto itemType = optionalDetailedType.GetItemType(); @@ -1792,7 +1792,7 @@ TRuntimeNode TProgramBuilder::Append(TRuntimeNode list, TRuntimeNode item) { const auto& listDetailedType = static_cast<const TListType&>(*listType); auto itemType = item.GetStaticType(); - MKQL_ENSURE(itemType->IsSameType(*listDetailedType.GetItemType()), "Types of list and item are different"); + MKQL_ENSURE(itemType->IsSameType(*listDetailedType.GetItemType()), "Types of list and item are different"); TCallableBuilder callableBuilder(Env, __func__, listType); callableBuilder.Add(list); @@ -1806,7 +1806,7 @@ TRuntimeNode TProgramBuilder::Prepend(TRuntimeNode item, TRuntimeNode list) { const auto& listDetailedType = static_cast<const TListType&>(*listType); auto itemType = item.GetStaticType(); - MKQL_ENSURE(itemType->IsSameType(*listDetailedType.GetItemType()), "Types of list and item are different"); + MKQL_ENSURE(itemType->IsSameType(*listDetailedType.GetItemType()), "Types of list and item are different"); TCallableBuilder callableBuilder(Env, __func__, listType); callableBuilder.Add(item); @@ -1920,7 +1920,7 @@ TRuntimeNode TProgramBuilder::NewEmptyListOfVoid() { } TRuntimeNode TProgramBuilder::NewEmptyOptional(TType* optionalType) { - MKQL_ENSURE(optionalType->IsOptional(), "Expected optional type"); + MKQL_ENSURE(optionalType->IsOptional(), "Expected optional type"); return TRuntimeNode(TOptionalLiteral::Create(static_cast<TOptionalType*>(optionalType), Env), true); } @@ -1938,7 +1938,7 @@ TType* TProgramBuilder::NewEmptyStructType() { } TType* TProgramBuilder::NewStructType(TType* baseStructType, const std::string_view& memberName, TType* memberType) { - MKQL_ENSURE(baseStructType->IsStruct(), "Expected struct type"); + MKQL_ENSURE(baseStructType->IsStruct(), "Expected struct type"); const auto& detailedBaseStructType = static_cast<const TStructType&>(*baseStructType); TStructTypeBuilder builder(Env); @@ -2051,7 +2051,7 @@ TType* TProgramBuilder::NewDictType(TType* keyType, TType* payloadType, bool mul } TRuntimeNode TProgramBuilder::NewDict(TType* dictType, const TArrayRef<const std::pair<TRuntimeNode, TRuntimeNode>>& items) { - MKQL_ENSURE(dictType->IsDict(), "Expected dict type"); + MKQL_ENSURE(dictType->IsDict(), "Expected dict type"); return TRuntimeNode(TDictLiteral::Create(items.size(), items.data(), static_cast<TDictType*>(dictType), Env), true); } @@ -2120,7 +2120,7 @@ TRuntimeNode TProgramBuilder::Coalesce(TRuntimeNode data, TRuntimeNode defaultDa if (!dataType->IsSameType(*defaultData.GetStaticType())) { bool isOptionalDefault; const auto defaultDataType = UnpackOptional(defaultData, isOptionalDefault); - MKQL_ENSURE(dataType->IsSameType(*defaultDataType), "Mismatch operand types"); + MKQL_ENSURE(dataType->IsSameType(*defaultDataType), "Mismatch operand types"); } TCallableBuilder callableBuilder(Env, __func__, defaultData.GetStaticType()); @@ -3037,15 +3037,15 @@ TRuntimeNode TProgramBuilder::UnaryDataFunction(TRuntimeNode data, const std::st bool isOptional; auto type = UnpackOptionalData(data, isOptional); if (!(flags & TDataFunctionFlags::AllowOptionalArgs)) { - MKQL_ENSURE(!isOptional, "Optional data is not allowed"); + MKQL_ENSURE(!isOptional, "Optional data is not allowed"); } auto schemeType = type->GetSchemeType(); - if (flags & TDataFunctionFlags::RequiresBooleanArgs) { + if (flags & TDataFunctionFlags::RequiresBooleanArgs) { MKQL_ENSURE(schemeType == NUdf::TDataType<bool>::Id, "Boolean data is required"); - } else if (flags & TDataFunctionFlags::RequiresStringArgs) { + } else if (flags & TDataFunctionFlags::RequiresStringArgs) { MKQL_ENSURE(schemeType == NUdf::TDataType<char*>::Id, "String data is required"); - } + } if (!schemeType) { MKQL_ENSURE((flags & TDataFunctionFlags::AllowNull) != 0, "Null is not allowed"); @@ -3075,7 +3075,7 @@ TRuntimeNode TProgramBuilder::UnaryDataFunction(TRuntimeNode data, const std::st TRuntimeNode TProgramBuilder::ToDict(TRuntimeNode list, bool multi, const TUnaryLambda& keySelector, const TUnaryLambda& payloadSelector, std::string_view callableName, bool isCompact, ui64 itemsCountHint) -{ +{ bool isOptional; const auto type = UnpackOptional(list, isOptional); MKQL_ENSURE(type->IsList(), "Expected list."); @@ -3695,26 +3695,26 @@ TRuntimeNode TProgramBuilder::Invoke(const std::string_view& funcName, TType* re return TRuntimeNode(callableBuilder.Build(), false); } -TRuntimeNode TProgramBuilder::Udf( +TRuntimeNode TProgramBuilder::Udf( const std::string_view& funcName, TRuntimeNode runConfig, TType* userType, const std::string_view& typeConfig ) -{ +{ TRuntimeNode userTypeNode = userType ? TRuntimeNode(userType, true) : TRuntimeNode(Env.GetVoid()->GetType(), true); const ui32 flags = NUdf::IUdfModule::TFlags::TypesOnly; - + if (!TypeInfoHelper) { TypeInfoHelper = new TTypeInfoHelper(); } - TFunctionTypeInfo funcInfo; - TStatus status = FunctionRegistry.FindFunctionTypeInfo( + TFunctionTypeInfo funcInfo; + TStatus status = FunctionRegistry.FindFunctionTypeInfo( Env, TypeInfoHelper, nullptr, funcName, userType, typeConfig, flags, {}, nullptr, &funcInfo); - MKQL_ENSURE(status.IsOk(), status.GetError()); - - auto runConfigType = funcInfo.RunConfigType; + MKQL_ENSURE(status.IsOk(), status.GetError()); + + auto runConfigType = funcInfo.RunConfigType; if (runConfig) { bool typesMatch = runConfigType->IsSameType(*runConfig.GetStaticType()); MKQL_ENSURE(typesMatch, "RunConfig type mismatch"); @@ -3726,19 +3726,19 @@ TRuntimeNode TProgramBuilder::Udf( runConfig = NewEmptyOptional(const_cast<TType*>(runConfigType)); } } - + auto funNameNode = NewDataLiteral<NUdf::EDataSlot::String>(funcName); auto typeConfigNode = NewDataLiteral<NUdf::EDataSlot::String>(typeConfig); - + TCallableBuilder callableBuilder(Env, __func__, funcInfo.FunctionType); - callableBuilder.Add(funNameNode); + callableBuilder.Add(funNameNode); callableBuilder.Add(userTypeNode); - callableBuilder.Add(typeConfigNode); + callableBuilder.Add(typeConfigNode); callableBuilder.Add(runConfig); - - return TRuntimeNode(callableBuilder.Build(), false); -} - + + return TRuntimeNode(callableBuilder.Build(), false); +} + TRuntimeNode TProgramBuilder::TypedUdf( const std::string_view& funcName, TType* funcType, @@ -3765,80 +3765,80 @@ TRuntimeNode TProgramBuilder::TypedUdf( return TRuntimeNode(callableBuilder.Build(), false); } -TRuntimeNode TProgramBuilder::ScriptUdf( - EScriptType scriptType, +TRuntimeNode TProgramBuilder::ScriptUdf( + EScriptType scriptType, const std::string_view& funcName, - TType* funcType, + TType* funcType, TRuntimeNode script, const std::string_view& file, ui32 row, ui32 column) -{ +{ MKQL_ENSURE(funcType, "UDF callable type must not be empty"); - MKQL_ENSURE(funcType->IsCallable(), "type must be callable"); - EnsureScriptSpecificTypes(scriptType, static_cast<TCallableType*>(funcType), Env); - + MKQL_ENSURE(funcType->IsCallable(), "type must be callable"); + EnsureScriptSpecificTypes(scriptType, static_cast<TCallableType*>(funcType), Env); + auto scriptTypeStr = ScriptTypeAsStr(CanonizeScriptType(scriptType)); - - TStringBuilder name; + + TStringBuilder name; name.reserve(scriptTypeStr.size() + funcName.size() + 1); - name << scriptTypeStr << '.' << funcName; + name << scriptTypeStr << '.' << funcName; auto funcNameNode = NewDataLiteral<NUdf::EDataSlot::String>(name); TRuntimeNode userTypeNode(funcType, true); auto typeConfigNode = NewDataLiteral<NUdf::EDataSlot::String>(""); - + TCallableBuilder callableBuilder(Env, __func__, funcType); - callableBuilder.Add(funcNameNode); + callableBuilder.Add(funcNameNode); callableBuilder.Add(userTypeNode); - callableBuilder.Add(typeConfigNode); - callableBuilder.Add(script); + callableBuilder.Add(typeConfigNode); + callableBuilder.Add(script); callableBuilder.Add(NewDataLiteral<NUdf::EDataSlot::String>(file)); callableBuilder.Add(NewDataLiteral(row)); callableBuilder.Add(NewDataLiteral(column)); - return TRuntimeNode(callableBuilder.Build(), false); -} - + return TRuntimeNode(callableBuilder.Build(), false); +} + TRuntimeNode TProgramBuilder::Apply(TRuntimeNode callableNode, const TArrayRef<const TRuntimeNode>& args, const std::string_view& file, ui32 row, ui32 column, ui32 dependentCount) { MKQL_ENSURE(dependentCount <= args.size(), "Too many dependent nodes"); ui32 usedArgs = args.size() - dependentCount; - MKQL_ENSURE(!callableNode.IsImmediate() && callableNode.GetNode()->GetType()->IsCallable(), - "Expected callable"); - - auto callable = static_cast<TCallable*>(callableNode.GetNode()); - TType* returnType = callable->GetType()->GetReturnType(); - MKQL_ENSURE(returnType->IsCallable(), "Expected callable as return type"); - - auto callableType = static_cast<TCallableType*>(returnType); + MKQL_ENSURE(!callableNode.IsImmediate() && callableNode.GetNode()->GetType()->IsCallable(), + "Expected callable"); + + auto callable = static_cast<TCallable*>(callableNode.GetNode()); + TType* returnType = callable->GetType()->GetReturnType(); + MKQL_ENSURE(returnType->IsCallable(), "Expected callable as return type"); + + auto callableType = static_cast<TCallableType*>(returnType); MKQL_ENSURE(usedArgs <= callableType->GetArgumentsCount(), "Too many arguments"); MKQL_ENSURE(usedArgs >= callableType->GetArgumentsCount() - callableType->GetOptionalArgumentsCount(), "Too few arguments"); - + for (ui32 i = 0; i < usedArgs; i++) { TType* argType = callableType->GetArgumentType(i); - TRuntimeNode arg = args[i]; + TRuntimeNode arg = args[i]; MKQL_ENSURE(arg.GetStaticType()->IsConvertableTo(*argType), "Argument type mismatch for argument " << i << ": runtime " << argType->GetKindAsStr() << " with static " << arg.GetStaticType()->GetKindAsStr()); - } - + } + TCallableBuilder callableBuilder(Env, RuntimeVersion >= 8 ? "Apply2" : "Apply", callableType->GetReturnType()); - callableBuilder.Add(callableNode); + callableBuilder.Add(callableNode); callableBuilder.Add(NewDataLiteral<ui32>(dependentCount)); - + if constexpr (RuntimeVersion >= 8) { callableBuilder.Add(NewDataLiteral<NUdf::EDataSlot::String>(file)); callableBuilder.Add(NewDataLiteral(row)); callableBuilder.Add(NewDataLiteral(column)); } - for (const auto& arg: args) { - callableBuilder.Add(arg); - } - - return TRuntimeNode(callableBuilder.Build(), false); -} - + for (const auto& arg: args) { + callableBuilder.Add(arg); + } + + return TRuntimeNode(callableBuilder.Build(), false); +} + TRuntimeNode TProgramBuilder::Apply( TRuntimeNode callableNode, const TArrayRef<const TRuntimeNode>& args, diff --git a/ydb/library/yql/minikql/mkql_program_builder.h b/ydb/library/yql/minikql/mkql_program_builder.h index a3b00676a8d..10f1ad7ccf1 100644 --- a/ydb/library/yql/minikql/mkql_program_builder.h +++ b/ydb/library/yql/minikql/mkql_program_builder.h @@ -1,17 +1,17 @@ #pragma once - + #include "defs.h" #include "mkql_node.h" #include "mkql_node_builder.h" #include <ydb/library/yql/public/udf/udf_value.h> - + #include <functional> namespace NKikimr { -namespace NMiniKQL { +namespace NMiniKQL { -class IFunctionRegistry; -class TBuiltinFunctionRegistry; +class IFunctionRegistry; +class TBuiltinFunctionRegistry; constexpr std::string_view RandomMTResource = "MTRand"; constexpr std::string_view ResourceQueuePrefix = "TResourceQueue:"; @@ -84,7 +84,7 @@ inline void AddAnyJoinSide(EAnyJoinSettings& combined, EAnyJoinSettings value) { combined = (EAnyJoinSettings)combinedVal; } -#define MKQL_SCRIPT_TYPES(xx) \ +#define MKQL_SCRIPT_TYPES(xx) \ xx(Unknown, 0, unknown) \ xx(Python, 1, python) \ xx(Lua, 2, lua) \ @@ -99,16 +99,16 @@ inline void AddAnyJoinSide(EAnyJoinSettings& combined, EAnyJoinSettings value) { xx(CustomPython3, 11, custompython3) \ xx(SystemPython2, 12, systempython2) \ xx(SystemPython3, 13, systempython3) \ - -enum class EScriptType { - MKQL_SCRIPT_TYPES(ENUM_VALUE_GEN) -}; - + +enum class EScriptType { + MKQL_SCRIPT_TYPES(ENUM_VALUE_GEN) +}; + std::string_view ScriptTypeAsStr(EScriptType type); EScriptType ScriptTypeFromStr(std::string_view str); bool IsCustomPython(EScriptType type); EScriptType CanonizeScriptType(EScriptType type); - + struct TSwitchInput { std::vector<ui32> Indicies; TType* InputType = nullptr; @@ -118,9 +118,9 @@ struct TSwitchInput { class TProgramBuilder : private TNonCopyable { public: TProgramBuilder(const TTypeEnvironment& env, const IFunctionRegistry& functionRegistry, bool voidWithEffects = false); - + const TTypeEnvironment& GetTypeEnvironment() const; - const IFunctionRegistry& GetFunctionRegistry() const; + const IFunctionRegistry& GetFunctionRegistry() const; TRuntimeNode Arg(TType* type) const; TRuntimeNode WideFlowArg(TType* type) const; @@ -239,13 +239,13 @@ public: TRuntimeNode BlockAdd(TRuntimeNode data1, TRuntimeNode data2); - // udfs - TRuntimeNode Udf( + // udfs + TRuntimeNode Udf( const std::string_view& funcName, TRuntimeNode runConfig = TRuntimeNode(), TType* userType = nullptr, const std::string_view& typeConfig = std::string_view("")); - + TRuntimeNode TypedUdf( const std::string_view& funcName, TType* funcType, @@ -254,13 +254,13 @@ public: const std::string_view& typeConfig = std::string_view(""), const std::string_view& file = std::string_view(""), ui32 row = 0, ui32 column = 0); - TRuntimeNode ScriptUdf( - EScriptType scriptType, + TRuntimeNode ScriptUdf( + EScriptType scriptType, const std::string_view& funcName, - TType* funcType, + TType* funcType, TRuntimeNode script, const std::string_view& file = std::string_view(""), ui32 row = 0, ui32 column = 0); - + typedef std::function<TRuntimeNode ()> TZeroLambda; typedef std::function<TRuntimeNode (TRuntimeNode)> TUnaryLambda; typedef std::function<TRuntimeNode (TRuntimeNode, TRuntimeNode)> TBinaryLambda; @@ -282,7 +282,7 @@ public: TRuntimeNode Apply(TRuntimeNode callableNode, const TArrayRef<const TRuntimeNode>& args, const std::string_view& file, ui32 row, ui32 column, ui32 dependentCount = 0); TRuntimeNode Callable(TType* callableType, const TArrayLambda& handler); - + //-- struct functions TRuntimeNode Member(TRuntimeNode structObj, const std::string_view& memberName); TRuntimeNode Element(TRuntimeNode tuple, const std::string_view& memberName); @@ -621,12 +621,12 @@ public: typedef TRuntimeNode (TProgramBuilder::*ProcessFunctionMethod)(TRuntimeNode, const TUnaryLambda&); typedef TRuntimeNode (TProgramBuilder::*NarrowFunctionMethod)(TRuntimeNode, const TNarrowLambda&); -protected: +protected: TRuntimeNode Invoke(const std::string_view& funcName, TType* resultType, const TArrayRef<const TRuntimeNode>& args); TRuntimeNode IfPresent(TRuntimeNode optional, const TUnaryLambda& thenBranch, TRuntimeNode elseBranch); - void ThrowIfListOfVoid(TType* type); - + void ThrowIfListOfVoid(TType* type); + template <typename ResultType> TRuntimeNode BuildContainerProperty(const std::string_view& callableName, TRuntimeNode listOrDict); TRuntimeNode Filter(TRuntimeNode list, const TUnaryLambda& handler, TType* resultType); @@ -662,7 +662,7 @@ private: const TNarrowLambda& payloadSelector, std::string_view callableName, bool isCompact, ui64 itemsCountHint); TRuntimeNode UnaryDataFunction(TRuntimeNode data, const std::string_view& callableName, ui32 flags); - + template<bool IsFilter, bool OnStruct> TRuntimeNode BuildFilterNulls(TRuntimeNode list); template<bool IsFilter, bool OnStruct> @@ -729,9 +729,9 @@ private: TType* BuildArithmeticCommonType(TType* type1, TType* type2); bool IsNull(TRuntimeNode arg); -protected: +protected: const TTypeEnvironment& Env; - const IFunctionRegistry& FunctionRegistry; + const IFunctionRegistry& FunctionRegistry; const bool VoidWithEffects; NUdf::ITypeInfoHelper::TPtr TypeInfoHelper; bool UseNullType = true; diff --git a/ydb/library/yql/minikql/mkql_stats_registry.cpp b/ydb/library/yql/minikql/mkql_stats_registry.cpp index 8e9585f3467..431737731ac 100644 --- a/ydb/library/yql/minikql/mkql_stats_registry.cpp +++ b/ydb/library/yql/minikql/mkql_stats_registry.cpp @@ -1,89 +1,89 @@ -#include "mkql_stats_registry.h" - -#include <util/generic/yexception.h> -#include <util/generic/vector.h> -#include <util/generic/hash_set.h> -#include <util/generic/singleton.h> - - -namespace NKikimr { -namespace NMiniKQL { -namespace { - -/** - * vector based implementation of stats registry - */ -class TDefaultStatsRegistry: public IStatsRegistry { -public: - i64 GetStat(const TStatKey& key) const override { - return key.GetId() < Values_.size() ? Values_[key.GetId()] : 0; - } - - void SetStat(const TStatKey& key, i64 value) override { - EnsureSize(key); - Values_[key.GetId()] = value; - } - - void SetMaxStat(const TStatKey& key, i64 value) override { - EnsureSize(key); - i64& oldValue = Values_[key.GetId()]; - oldValue = Max(value, oldValue); - } - - void SetMinStat(const TStatKey& key, i64 value) override { - EnsureSize(key); - i64& oldValue = Values_[key.GetId()]; - oldValue = Min(value, oldValue); - } - - void AddStat(const TStatKey& key, i64 value) override { - EnsureSize(key); - Values_[key.GetId()] += value; - } - -private: - void EnsureSize(const TStatKey& key) { - if (Y_UNLIKELY(Values_.size() <= key.GetId())) { - Values_.resize(key.GetId() + 10); // try to avoid too often resizing - } - } - -private: +#include "mkql_stats_registry.h" + +#include <util/generic/yexception.h> +#include <util/generic/vector.h> +#include <util/generic/hash_set.h> +#include <util/generic/singleton.h> + + +namespace NKikimr { +namespace NMiniKQL { +namespace { + +/** + * vector based implementation of stats registry + */ +class TDefaultStatsRegistry: public IStatsRegistry { +public: + i64 GetStat(const TStatKey& key) const override { + return key.GetId() < Values_.size() ? Values_[key.GetId()] : 0; + } + + void SetStat(const TStatKey& key, i64 value) override { + EnsureSize(key); + Values_[key.GetId()] = value; + } + + void SetMaxStat(const TStatKey& key, i64 value) override { + EnsureSize(key); + i64& oldValue = Values_[key.GetId()]; + oldValue = Max(value, oldValue); + } + + void SetMinStat(const TStatKey& key, i64 value) override { + EnsureSize(key); + i64& oldValue = Values_[key.GetId()]; + oldValue = Min(value, oldValue); + } + + void AddStat(const TStatKey& key, i64 value) override { + EnsureSize(key); + Values_[key.GetId()] += value; + } + +private: + void EnsureSize(const TStatKey& key) { + if (Y_UNLIKELY(Values_.size() <= key.GetId())) { + Values_.resize(key.GetId() + 10); // try to avoid too often resizing + } + } + +private: TVector<i64> Values_; -}; - -/** - * Key names set to prevent keys accidental duplication. - */ +}; + +/** + * Key names set to prevent keys accidental duplication. + */ class TKeyNames: private THashSet<TStringBuf> { -public: - bool AddIfNotExists(TStringBuf name) { - return insert(name).second; - } -}; - +public: + bool AddIfNotExists(TStringBuf name) { + return insert(name).second; + } +}; + } // namespace - - -ui32 TStatKey::IdSequence_ = 0; -TStatKey* TStatKey::KeysChain_ = nullptr; - - + + +ui32 TStatKey::IdSequence_ = 0; +TStatKey* TStatKey::KeysChain_ = nullptr; + + TStatKey::TStatKey(TStringBuf name, bool deriv) - : Name_(name) + : Name_(name) , Deriv_(deriv) - , Id_(IdSequence_++) -{ - bool newOne = Singleton<TKeyNames>()->AddIfNotExists(name); - Y_ENSURE(newOne, "duplicated stat key: " << name); - - SetNext(KeysChain_); - KeysChain_ = this; -} - -IStatsRegistryPtr CreateDefaultStatsRegistry() { - return new TDefaultStatsRegistry; -} - + , Id_(IdSequence_++) +{ + bool newOne = Singleton<TKeyNames>()->AddIfNotExists(name); + Y_ENSURE(newOne, "duplicated stat key: " << name); + + SetNext(KeysChain_); + KeysChain_ = this; +} + +IStatsRegistryPtr CreateDefaultStatsRegistry() { + return new TDefaultStatsRegistry; +} + } // namespace NMiniKQL } // namespace NKikimr diff --git a/ydb/library/yql/minikql/mkql_stats_registry.h b/ydb/library/yql/minikql/mkql_stats_registry.h index 673361cef2c..1b888153e48 100644 --- a/ydb/library/yql/minikql/mkql_stats_registry.h +++ b/ydb/library/yql/minikql/mkql_stats_registry.h @@ -1,36 +1,36 @@ -#pragma once - -#include <util/generic/strbuf.h> -#include <util/generic/intrlist.h> +#pragma once + +#include <util/generic/strbuf.h> +#include <util/generic/intrlist.h> #include <util/system/datetime.h> #include <util/system/hp_timer.h> - -namespace NKikimr { -namespace NMiniKQL { - -struct IStatsRegistry; - -/** - * @brief Statistic unique named key. - * - * NOTE: Instances of this class must be static objects. - */ -class TStatKey: public TIntrusiveSListItem<TStatKey> { - friend struct IStatsRegistry; - -// -- static part -- - static ui32 IdSequence_; - static TStatKey* KeysChain_; - -// -- instance part -- -public: + +namespace NKikimr { +namespace NMiniKQL { + +struct IStatsRegistry; + +/** + * @brief Statistic unique named key. + * + * NOTE: Instances of this class must be static objects. + */ +class TStatKey: public TIntrusiveSListItem<TStatKey> { + friend struct IStatsRegistry; + +// -- static part -- + static ui32 IdSequence_; + static TStatKey* KeysChain_; + +// -- instance part -- +public: TStatKey(TStringBuf name, bool deriv); - - TStringBuf GetName() const noexcept { return Name_; } + + TStringBuf GetName() const noexcept { return Name_; } bool IsDeriv() const noexcept { return Deriv_; } - ui32 GetId() const noexcept { return Id_; } + ui32 GetId() const noexcept { return Id_; } static ui32 GetMaxId() { return IdSequence_; } - + template <typename F> static void ForEach(F f) { for (TStatKey* p = KeysChain_; p;) { @@ -40,69 +40,69 @@ public: } } -private: - TStringBuf Name_; +private: + TStringBuf Name_; bool Deriv_; - ui32 Id_; -}; - -/** - * @brief Sase interface for statistics registry implementations. - */ -struct IStatsRegistry { - virtual ~IStatsRegistry() = default; - - virtual i64 GetStat(const TStatKey& key) const = 0; - - virtual void SetStat(const TStatKey& key, i64 value) = 0; - virtual void SetMaxStat(const TStatKey& key, i64 value) = 0; - virtual void SetMinStat(const TStatKey& key, i64 value) = 0; - - virtual void AddStat(const TStatKey& key, i64 value) = 0; - - void IncStat(const TStatKey& key) { - AddStat(key, 1); - } - - void DecStat(const TStatKey& key) { - AddStat(key, -1); - } - - // TConsumer = void(const TStatKey& key, i64 value) - template <typename TConsumer> - void ForEachStat(TConsumer c) const { + ui32 Id_; +}; + +/** + * @brief Sase interface for statistics registry implementations. + */ +struct IStatsRegistry { + virtual ~IStatsRegistry() = default; + + virtual i64 GetStat(const TStatKey& key) const = 0; + + virtual void SetStat(const TStatKey& key, i64 value) = 0; + virtual void SetMaxStat(const TStatKey& key, i64 value) = 0; + virtual void SetMinStat(const TStatKey& key, i64 value) = 0; + + virtual void AddStat(const TStatKey& key, i64 value) = 0; + + void IncStat(const TStatKey& key) { + AddStat(key, 1); + } + + void DecStat(const TStatKey& key) { + AddStat(key, -1); + } + + // TConsumer = void(const TStatKey& key, i64 value) + template <typename TConsumer> + void ForEachStat(TConsumer c) const { TStatKey::ForEach([this, &c](const auto& key) { c(key, GetStat(key)); }); - } -}; - -using IStatsRegistryPtr = TAutoPtr<IStatsRegistry>; - -IStatsRegistryPtr CreateDefaultStatsRegistry(); - - -#define MKQL_STAT_MOD_IMPL(method, nullableStatsRegistry, key, value) \ - do { \ - if (nullableStatsRegistry) \ - nullableStatsRegistry->method(key, value); \ - } while (0) - -#define MKQL_SET_STAT(statRegistry, key, value) \ - MKQL_STAT_MOD_IMPL(SetStat, statRegistry, key, value) - -#define MKQL_SET_MIN_STAT(statRegistry, key, value) \ - MKQL_STAT_MOD_IMPL(SetMinStat, statRegistry, key, value) - -#define MKQL_SET_MAX_STAT(statRegistry, key, value) \ - MKQL_STAT_MOD_IMPL(SetMaxStat, statRegistry, key, value) - -#define MKQL_ADD_STAT(statRegistry, key, value) \ - MKQL_STAT_MOD_IMPL(AddStat, statRegistry, key, value) - -#define MKQL_INC_STAT(statRegistry, key) MKQL_ADD_STAT(statRegistry, key, 1) -#define MKQL_DEC_STAT(statRegistry, key) MKQL_ADD_STAT(statRegistry, key, -1) - + } +}; + +using IStatsRegistryPtr = TAutoPtr<IStatsRegistry>; + +IStatsRegistryPtr CreateDefaultStatsRegistry(); + + +#define MKQL_STAT_MOD_IMPL(method, nullableStatsRegistry, key, value) \ + do { \ + if (nullableStatsRegistry) \ + nullableStatsRegistry->method(key, value); \ + } while (0) + +#define MKQL_SET_STAT(statRegistry, key, value) \ + MKQL_STAT_MOD_IMPL(SetStat, statRegistry, key, value) + +#define MKQL_SET_MIN_STAT(statRegistry, key, value) \ + MKQL_STAT_MOD_IMPL(SetMinStat, statRegistry, key, value) + +#define MKQL_SET_MAX_STAT(statRegistry, key, value) \ + MKQL_STAT_MOD_IMPL(SetMaxStat, statRegistry, key, value) + +#define MKQL_ADD_STAT(statRegistry, key, value) \ + MKQL_STAT_MOD_IMPL(AddStat, statRegistry, key, value) + +#define MKQL_INC_STAT(statRegistry, key) MKQL_ADD_STAT(statRegistry, key, 1) +#define MKQL_DEC_STAT(statRegistry, key) MKQL_ADD_STAT(statRegistry, key, -1) + class TStatTimer { public: TStatTimer(const TStatKey& key) diff --git a/ydb/library/yql/minikql/mkql_stats_registry_ut.cpp b/ydb/library/yql/minikql/mkql_stats_registry_ut.cpp index c2b013ccf6a..0504fb2986b 100644 --- a/ydb/library/yql/minikql/mkql_stats_registry_ut.cpp +++ b/ydb/library/yql/minikql/mkql_stats_registry_ut.cpp @@ -1,74 +1,74 @@ -#include "mkql_stats_registry.h" - +#include "mkql_stats_registry.h" + #include <library/cpp/testing/unittest/registar.h> - - -using namespace NKikimr; -using namespace NMiniKQL; - + + +using namespace NKikimr; +using namespace NMiniKQL; + Y_UNIT_TEST_SUITE(TStatsRegistryTest) { - + Y_UNIT_TEST(Stats) { static TStatKey key("key", false); - auto stats = CreateDefaultStatsRegistry(); - - stats->SetStat(key, 37); - UNIT_ASSERT_EQUAL(stats->GetStat(key), 37); - - stats->AddStat(key, 3); - UNIT_ASSERT_EQUAL(stats->GetStat(key), 40); - - stats->IncStat(key); - UNIT_ASSERT_EQUAL(stats->GetStat(key), 41); - - stats->DecStat(key); - UNIT_ASSERT_EQUAL(stats->GetStat(key), 40); - - stats->SetMaxStat(key, 3); - UNIT_ASSERT_EQUAL(stats->GetStat(key), 40); - - stats->SetMaxStat(key, 43); - UNIT_ASSERT_EQUAL(stats->GetStat(key), 43); - - stats->SetMinStat(key, 57); - UNIT_ASSERT_EQUAL(stats->GetStat(key), 43); - - stats->SetMinStat(key, 34); - UNIT_ASSERT_EQUAL(stats->GetStat(key), 34); - } - + auto stats = CreateDefaultStatsRegistry(); + + stats->SetStat(key, 37); + UNIT_ASSERT_EQUAL(stats->GetStat(key), 37); + + stats->AddStat(key, 3); + UNIT_ASSERT_EQUAL(stats->GetStat(key), 40); + + stats->IncStat(key); + UNIT_ASSERT_EQUAL(stats->GetStat(key), 41); + + stats->DecStat(key); + UNIT_ASSERT_EQUAL(stats->GetStat(key), 40); + + stats->SetMaxStat(key, 3); + UNIT_ASSERT_EQUAL(stats->GetStat(key), 40); + + stats->SetMaxStat(key, 43); + UNIT_ASSERT_EQUAL(stats->GetStat(key), 43); + + stats->SetMinStat(key, 57); + UNIT_ASSERT_EQUAL(stats->GetStat(key), 43); + + stats->SetMinStat(key, 34); + UNIT_ASSERT_EQUAL(stats->GetStat(key), 34); + } + Y_UNIT_TEST(ForEach) { static TStatKey key1("key1", false), key2("key2", true); UNIT_ASSERT(!key1.IsDeriv()); UNIT_ASSERT(key2.IsDeriv()); - - auto stats = CreateDefaultStatsRegistry(); - stats->SetStat(key1, 42); - stats->SetStat(key2, 37); - - stats->ForEachStat([](const TStatKey& key, i64 value) { - if (key.GetName() == "key1") { - UNIT_ASSERT_EQUAL(value, 42); - } else if (key.GetName() == "key2") { - UNIT_ASSERT_EQUAL(value, 37); - } else { - // depending on how to run tests other keys can be linked - // together and appears here - } - }); - } - + + auto stats = CreateDefaultStatsRegistry(); + stats->SetStat(key1, 42); + stats->SetStat(key2, 37); + + stats->ForEachStat([](const TStatKey& key, i64 value) { + if (key.GetName() == "key1") { + UNIT_ASSERT_EQUAL(value, 42); + } else if (key.GetName() == "key2") { + UNIT_ASSERT_EQUAL(value, 37); + } else { + // depending on how to run tests other keys can be linked + // together and appears here + } + }); + } + Y_UNIT_TEST(DuplicateKeys) { TString error; - try { + try { static TStatKey key("my_key", false), sameKey("my_key", false); - // avoid variables elimitation - Cerr << key.GetId() << ": " << key.GetName() << '\n' - << sameKey.GetId() << ": " << sameKey.GetName() << Endl; - } catch (const yexception& e) { - error = e.AsStrBuf(); - } - - UNIT_ASSERT(error.Contains("duplicated stat key: my_key")); - } -} + // avoid variables elimitation + Cerr << key.GetId() << ": " << key.GetName() << '\n' + << sameKey.GetId() << ": " << sameKey.GetName() << Endl; + } catch (const yexception& e) { + error = e.AsStrBuf(); + } + + UNIT_ASSERT(error.Contains("duplicated stat key: my_key")); + } +} diff --git a/ydb/library/yql/minikql/mkql_type_builder.cpp b/ydb/library/yql/minikql/mkql_type_builder.cpp index e2f91a34e39..4aecb564bf7 100644 --- a/ydb/library/yql/minikql/mkql_type_builder.cpp +++ b/ydb/library/yql/minikql/mkql_type_builder.cpp @@ -1,20 +1,20 @@ -#include "mkql_type_builder.h" -#include "mkql_node_cast.h" -#include "mkql_node_builder.h" - +#include "mkql_type_builder.h" +#include "mkql_node_cast.h" +#include "mkql_node_builder.h" + #include <ydb/library/yql/public/udf/udf_type_ops.h> #include <library/cpp/containers/stack_vector/stack_vec.h> #include <ydb/library/yql/minikql/computation/mkql_computation_node_impl.h> - -// TODO: remove const_casts - -namespace NKikimr { - -namespace { - + +// TODO: remove const_casts + +namespace NKikimr { + +namespace { + static const TString UdfName("UDF"); - + class TCallablePayload : public NUdf::ICallablePayload { public: TCallablePayload(NMiniKQL::TNode* node) @@ -36,7 +36,7 @@ public: ArgsFlags_.push_back(AS_VALUE(NMiniKQL::TDataLiteral, arg->GetValue(flagsIndex))->AsValue().Get<ui64>()); } } - + NUdf::TStringRef GetPayload() const override { return Payload_; } @@ -56,83 +56,83 @@ private: }; ///////////////////////////////////////////////////////////////////////////// -// TOptionalTypeBuilder -////////////////////////////////////////////////////////////////////////////// -class TOptionalTypeBuilder: public NUdf::IOptionalTypeBuilder -{ -public: - TOptionalTypeBuilder(const NMiniKQL::TFunctionTypeInfoBuilder& parent) - : Parent_(parent) - { - } - +// TOptionalTypeBuilder +////////////////////////////////////////////////////////////////////////////// +class TOptionalTypeBuilder: public NUdf::IOptionalTypeBuilder +{ +public: + TOptionalTypeBuilder(const NMiniKQL::TFunctionTypeInfoBuilder& parent) + : Parent_(parent) + { + } + NUdf::IOptionalTypeBuilder& Item(NUdf::TDataTypeId typeId) override { - ItemType_ = NMiniKQL::TDataType::Create(typeId, Parent_.Env()); - return *this; - } - - NUdf::IOptionalTypeBuilder& Item(const NUdf::TType* type) override { - ItemType_ = static_cast<const NMiniKQL::TType*>(type); - return *this; - } - - NUdf::IOptionalTypeBuilder& Item( - const NUdf::ITypeBuilder& typeBuilder) override - { - ItemType_ = static_cast<NMiniKQL::TType*>(typeBuilder.Build()); - return *this; - } - - NUdf::TType* Build() const override { - return NMiniKQL::TOptionalType::Create( - const_cast<NMiniKQL::TType*>(ItemType_), + ItemType_ = NMiniKQL::TDataType::Create(typeId, Parent_.Env()); + return *this; + } + + NUdf::IOptionalTypeBuilder& Item(const NUdf::TType* type) override { + ItemType_ = static_cast<const NMiniKQL::TType*>(type); + return *this; + } + + NUdf::IOptionalTypeBuilder& Item( + const NUdf::ITypeBuilder& typeBuilder) override + { + ItemType_ = static_cast<NMiniKQL::TType*>(typeBuilder.Build()); + return *this; + } + + NUdf::TType* Build() const override { + return NMiniKQL::TOptionalType::Create( + const_cast<NMiniKQL::TType*>(ItemType_), Parent_.Env()); - } - -private: - const NMiniKQL::TFunctionTypeInfoBuilder& Parent_; + } + +private: + const NMiniKQL::TFunctionTypeInfoBuilder& Parent_; const NMiniKQL::TType* ItemType_ = nullptr; -}; - -////////////////////////////////////////////////////////////////////////////// -// TListTypeBuilder -////////////////////////////////////////////////////////////////////////////// -class TListTypeBuilder: public NUdf::IListTypeBuilder -{ -public: - TListTypeBuilder(const NMiniKQL::TFunctionTypeInfoBuilder& parent) - : Parent_(parent) - { - } - +}; + +////////////////////////////////////////////////////////////////////////////// +// TListTypeBuilder +////////////////////////////////////////////////////////////////////////////// +class TListTypeBuilder: public NUdf::IListTypeBuilder +{ +public: + TListTypeBuilder(const NMiniKQL::TFunctionTypeInfoBuilder& parent) + : Parent_(parent) + { + } + NUdf::IListTypeBuilder& Item(NUdf::TDataTypeId typeId) override { - ItemType_ = NMiniKQL::TDataType::Create(typeId, Parent_.Env()); - return *this; - } - - NUdf::IListTypeBuilder& Item(const NUdf::TType* type) override { - ItemType_ = static_cast<const NMiniKQL::TType*>(type); - return *this; - } - - NUdf::IListTypeBuilder& Item( - const NUdf::ITypeBuilder& typeBuilder) override - { - ItemType_ = static_cast<NMiniKQL::TType*>(typeBuilder.Build()); - return *this; - } - - NUdf::TType* Build() const override { - return NMiniKQL::TListType::Create( + ItemType_ = NMiniKQL::TDataType::Create(typeId, Parent_.Env()); + return *this; + } + + NUdf::IListTypeBuilder& Item(const NUdf::TType* type) override { + ItemType_ = static_cast<const NMiniKQL::TType*>(type); + return *this; + } + + NUdf::IListTypeBuilder& Item( + const NUdf::ITypeBuilder& typeBuilder) override + { + ItemType_ = static_cast<NMiniKQL::TType*>(typeBuilder.Build()); + return *this; + } + + NUdf::TType* Build() const override { + return NMiniKQL::TListType::Create( const_cast<NMiniKQL::TType*>(ItemType_), Parent_.Env()); - } - -private: - const NMiniKQL::TFunctionTypeInfoBuilder& Parent_; + } + +private: + const NMiniKQL::TFunctionTypeInfoBuilder& Parent_; const NMiniKQL::TType* ItemType_ = nullptr; -}; - -////////////////////////////////////////////////////////////////////////////// +}; + +////////////////////////////////////////////////////////////////////////////// // TStreamTypeBuilder ////////////////////////////////////////////////////////////////////////////// class TStreamTypeBuilder : public NUdf::IStreamTypeBuilder @@ -171,79 +171,79 @@ private: }; ////////////////////////////////////////////////////////////////////////////// -// TDictTypeBuilder -////////////////////////////////////////////////////////////////////////////// -class TDictTypeBuilder: public NUdf::IDictTypeBuilder -{ -public: - TDictTypeBuilder(const NMiniKQL::TFunctionTypeInfoBuilder& parent) - : Parent_(parent) - { - } - +// TDictTypeBuilder +////////////////////////////////////////////////////////////////////////////// +class TDictTypeBuilder: public NUdf::IDictTypeBuilder +{ +public: + TDictTypeBuilder(const NMiniKQL::TFunctionTypeInfoBuilder& parent) + : Parent_(parent) + { + } + NUdf::IDictTypeBuilder& Key(NUdf::TDataTypeId typeId) override { auto slot = NUdf::FindDataSlot(typeId); Y_VERIFY(slot, "unknown type: %d", (int)typeId); Y_VERIFY(NUdf::GetDataTypeInfo(*slot).Features & NUdf::CanCompare, "key type is not comparable"); - KeyType_ = NMiniKQL::TDataType::Create(typeId, Parent_.Env()); - return *this; - } - - NUdf::IDictTypeBuilder& Key(const NUdf::TType* type) override { - KeyType_ = static_cast<const NMiniKQL::TType*>(type); - CheckKeyType(); - return *this; - } - - NUdf::IDictTypeBuilder& Key( - const NUdf::ITypeBuilder& typeBuilder) override - { - KeyType_ = static_cast<NMiniKQL::TType*>(typeBuilder.Build()); - CheckKeyType(); - return *this; - } - + KeyType_ = NMiniKQL::TDataType::Create(typeId, Parent_.Env()); + return *this; + } + + NUdf::IDictTypeBuilder& Key(const NUdf::TType* type) override { + KeyType_ = static_cast<const NMiniKQL::TType*>(type); + CheckKeyType(); + return *this; + } + + NUdf::IDictTypeBuilder& Key( + const NUdf::ITypeBuilder& typeBuilder) override + { + KeyType_ = static_cast<NMiniKQL::TType*>(typeBuilder.Build()); + CheckKeyType(); + return *this; + } + NUdf::IDictTypeBuilder& Value(NUdf::TDataTypeId typeId) override { - ValueType_ = NMiniKQL::TDataType::Create(typeId, Parent_.Env()); - return *this; - } - - NUdf::IDictTypeBuilder& Value(const NUdf::TType* type) override { - ValueType_ = static_cast<const NMiniKQL::TType*>(type); - return *this; - } - - NUdf::IDictTypeBuilder& Value( - const NUdf::ITypeBuilder& typeBuilder) override - { - ValueType_ = static_cast<NMiniKQL::TType*>(typeBuilder.Build()); - return *this; - } - - NUdf::TType* Build() const override { - return NMiniKQL::TDictType::Create( - const_cast<NMiniKQL::TType*>(KeyType_), + ValueType_ = NMiniKQL::TDataType::Create(typeId, Parent_.Env()); + return *this; + } + + NUdf::IDictTypeBuilder& Value(const NUdf::TType* type) override { + ValueType_ = static_cast<const NMiniKQL::TType*>(type); + return *this; + } + + NUdf::IDictTypeBuilder& Value( + const NUdf::ITypeBuilder& typeBuilder) override + { + ValueType_ = static_cast<NMiniKQL::TType*>(typeBuilder.Build()); + return *this; + } + + NUdf::TType* Build() const override { + return NMiniKQL::TDictType::Create( + const_cast<NMiniKQL::TType*>(KeyType_), const_cast<NMiniKQL::TType*>(ValueType_), Parent_.Env()); - } - -private: - void CheckKeyType() const { - auto t = AS_TYPE(NMiniKQL::TDataType, const_cast<NMiniKQL::TType*>(KeyType_)); + } + +private: + void CheckKeyType() const { + auto t = AS_TYPE(NMiniKQL::TDataType, const_cast<NMiniKQL::TType*>(KeyType_)); auto keySchemeType = t->GetSchemeType(); auto slot = NUdf::FindDataSlot(keySchemeType); Y_VERIFY(slot, "unknown type: %d", (int)keySchemeType); Y_VERIFY(NUdf::GetDataTypeInfo(*slot).Features & NUdf::CanCompare, "key type is not comparable"); - } - -private: - const NMiniKQL::TFunctionTypeInfoBuilder& Parent_; + } + +private: + const NMiniKQL::TFunctionTypeInfoBuilder& Parent_; const NMiniKQL::TType* KeyType_ = nullptr; const NMiniKQL::TType* ValueType_ = nullptr; -}; - -////////////////////////////////////////////////////////////////////////////// +}; + +////////////////////////////////////////////////////////////////////////////// // TSetTypeBuilder -////////////////////////////////////////////////////////////////////////////// +////////////////////////////////////////////////////////////////////////////// class TSetTypeBuilder : public NUdf::ISetTypeBuilder { public: @@ -297,60 +297,60 @@ private: ////////////////////////////////////////////////////////////////////////////// // TStructTypeBuilder ////////////////////////////////////////////////////////////////////////////// -class TStructTypeBuilder: public NUdf::IStructTypeBuilder -{ -public: - TStructTypeBuilder( - const NMiniKQL::TFunctionTypeInfoBuilder& parent, - ui32 itemsCount) - : Parent_(parent) +class TStructTypeBuilder: public NUdf::IStructTypeBuilder +{ +public: + TStructTypeBuilder( + const NMiniKQL::TFunctionTypeInfoBuilder& parent, + ui32 itemsCount) + : Parent_(parent) , StructBuilder_(Parent_.Env()) - { - StructBuilder_.Reserve(itemsCount); - } - - NUdf::IStructTypeBuilder& AddField( - const NUdf::TStringRef& name, + { + StructBuilder_.Reserve(itemsCount); + } + + NUdf::IStructTypeBuilder& AddField( + const NUdf::TStringRef& name, NUdf::TDataTypeId typeId, - ui32* index) override - { - auto type = NMiniKQL::TDataType::Create(typeId, Parent_.Env()); - StructBuilder_.Add(name, type, index); - return *this; - } - - NUdf::IStructTypeBuilder& AddField( - const NUdf::TStringRef& name, - const NUdf::TType* type, - ui32* index) override - { - auto mkqlType = static_cast<const NMiniKQL::TType*>(type); - StructBuilder_.Add(name, const_cast<NMiniKQL::TType*>(mkqlType), index); - return *this; - } - - NUdf::IStructTypeBuilder& AddField( - const NUdf::TStringRef& name, - const NUdf::ITypeBuilder& typeBuilder, - ui32* index) override - { - auto type = static_cast<NMiniKQL::TType*>(typeBuilder.Build()); - StructBuilder_.Add(name, type, index); - return *this; - } - - NUdf::TType* Build() const override { - auto structType = StructBuilder_.Build(); - StructBuilder_.FillIndexes(); - return structType; - } - -private: - const NMiniKQL::TFunctionTypeInfoBuilder& Parent_; - mutable NMiniKQL::TStructTypeBuilder StructBuilder_; -}; - -////////////////////////////////////////////////////////////////////////////// + ui32* index) override + { + auto type = NMiniKQL::TDataType::Create(typeId, Parent_.Env()); + StructBuilder_.Add(name, type, index); + return *this; + } + + NUdf::IStructTypeBuilder& AddField( + const NUdf::TStringRef& name, + const NUdf::TType* type, + ui32* index) override + { + auto mkqlType = static_cast<const NMiniKQL::TType*>(type); + StructBuilder_.Add(name, const_cast<NMiniKQL::TType*>(mkqlType), index); + return *this; + } + + NUdf::IStructTypeBuilder& AddField( + const NUdf::TStringRef& name, + const NUdf::ITypeBuilder& typeBuilder, + ui32* index) override + { + auto type = static_cast<NMiniKQL::TType*>(typeBuilder.Build()); + StructBuilder_.Add(name, type, index); + return *this; + } + + NUdf::TType* Build() const override { + auto structType = StructBuilder_.Build(); + StructBuilder_.FillIndexes(); + return structType; + } + +private: + const NMiniKQL::TFunctionTypeInfoBuilder& Parent_; + mutable NMiniKQL::TStructTypeBuilder StructBuilder_; +}; + +////////////////////////////////////////////////////////////////////////////// // TEnumTypeBuilder ////////////////////////////////////////////////////////////////////////////// class TEnumTypeBuilder : public NUdf::IEnumTypeBuilder @@ -385,53 +385,53 @@ private: }; ////////////////////////////////////////////////////////////////////////////// -// TTupleTypeBuilder -////////////////////////////////////////////////////////////////////////////// -class TTupleTypeBuilder: public NUdf::ITupleTypeBuilder -{ -public: - TTupleTypeBuilder( - const NMiniKQL::TFunctionTypeInfoBuilder& parent, - ui32 itemsCount) - : Parent_(parent) - { - ElementTypes_.reserve(itemsCount); - } - +// TTupleTypeBuilder +////////////////////////////////////////////////////////////////////////////// +class TTupleTypeBuilder: public NUdf::ITupleTypeBuilder +{ +public: + TTupleTypeBuilder( + const NMiniKQL::TFunctionTypeInfoBuilder& parent, + ui32 itemsCount) + : Parent_(parent) + { + ElementTypes_.reserve(itemsCount); + } + NUdf::ITupleTypeBuilder& Add(NUdf::TDataTypeId typeId) override { - auto type = NMiniKQL::TDataType::Create(typeId, Parent_.Env()); - ElementTypes_.push_back(type); - return *this; - } - - NUdf::ITupleTypeBuilder& Add(const NUdf::TType* type) override { - auto mkqlType = static_cast<const NMiniKQL::TType*>(type); - ElementTypes_.push_back(const_cast<NMiniKQL::TType*>(mkqlType)); - return *this; - } - - NUdf::ITupleTypeBuilder& Add( - const NUdf::ITypeBuilder& typeBuilder) override - { - auto type = static_cast<NMiniKQL::TType*>(typeBuilder.Build()); - ElementTypes_.push_back(type); - return *this; - } - - NUdf::TType* Build() const override { - return NMiniKQL::TTupleType::Create( + auto type = NMiniKQL::TDataType::Create(typeId, Parent_.Env()); + ElementTypes_.push_back(type); + return *this; + } + + NUdf::ITupleTypeBuilder& Add(const NUdf::TType* type) override { + auto mkqlType = static_cast<const NMiniKQL::TType*>(type); + ElementTypes_.push_back(const_cast<NMiniKQL::TType*>(mkqlType)); + return *this; + } + + NUdf::ITupleTypeBuilder& Add( + const NUdf::ITypeBuilder& typeBuilder) override + { + auto type = static_cast<NMiniKQL::TType*>(typeBuilder.Build()); + ElementTypes_.push_back(type); + return *this; + } + + NUdf::TType* Build() const override { + return NMiniKQL::TTupleType::Create( ElementTypes_.size(), ElementTypes_.data(), - Parent_.Env()); - } - -private: - const NMiniKQL::TFunctionTypeInfoBuilder& Parent_; + Parent_.Env()); + } + +private: + const NMiniKQL::TFunctionTypeInfoBuilder& Parent_; TVector<NMiniKQL::TType*> ElementTypes_; -}; - +}; + ///////////////////////////////////////////////////////////////////////////// // TVariantTypeBuilder -////////////////////////////////////////////////////////////////////////////// +////////////////////////////////////////////////////////////////////////////// class TVariantTypeBuilder : public NUdf::IVariantTypeBuilder { public: @@ -464,125 +464,125 @@ private: }; ////////////////////////////////////////////////////////////////////////////// -// TCallableTypeBuilder -////////////////////////////////////////////////////////////////////////////// -class TCallableTypeBuilder: public NUdf::ICallableTypeBuilder -{ -public: - TCallableTypeBuilder( - const NMiniKQL::TTypeEnvironment& env, ui32 argsCount) - : Env_(env) - , ReturnType_(nullptr) - , OptionalArgs_(0) - { - ArgsTypes_.reserve(argsCount); - } - - NUdf::ICallableTypeBuilder& Returns( +// TCallableTypeBuilder +////////////////////////////////////////////////////////////////////////////// +class TCallableTypeBuilder: public NUdf::ICallableTypeBuilder +{ +public: + TCallableTypeBuilder( + const NMiniKQL::TTypeEnvironment& env, ui32 argsCount) + : Env_(env) + , ReturnType_(nullptr) + , OptionalArgs_(0) + { + ArgsTypes_.reserve(argsCount); + } + + NUdf::ICallableTypeBuilder& Returns( NUdf::TDataTypeId typeId) override - { - ReturnType_ = NMiniKQL::TDataType::Create(typeId, Env_); - return *this; - } - - NUdf::ICallableTypeBuilder& Returns( + { + ReturnType_ = NMiniKQL::TDataType::Create(typeId, Env_); + return *this; + } + + NUdf::ICallableTypeBuilder& Returns( const NUdf::TType* type) override - { - ReturnType_ = const_cast<NMiniKQL::TType*>( - static_cast<const NMiniKQL::TType*>(type)); - return *this; - } - - NUdf::ICallableTypeBuilder& Returns( + { + ReturnType_ = const_cast<NMiniKQL::TType*>( + static_cast<const NMiniKQL::TType*>(type)); + return *this; + } + + NUdf::ICallableTypeBuilder& Returns( const NUdf::ITypeBuilder& typeBuilder) override - { - ReturnType_ = static_cast<NMiniKQL::TType*>(typeBuilder.Build()); - return *this; - } - + { + ReturnType_ = static_cast<NMiniKQL::TType*>(typeBuilder.Build()); + return *this; + } + NUdf::ICallableTypeBuilder& Arg(NUdf::TDataTypeId typeId) override { - auto type = NMiniKQL::TDataType::Create(typeId, Env_); + auto type = NMiniKQL::TDataType::Create(typeId, Env_); ArgsTypes_.push_back(type); - return *this; - } - - NUdf::ICallableTypeBuilder& Arg(const NUdf::TType* type) override { + return *this; + } + + NUdf::ICallableTypeBuilder& Arg(const NUdf::TType* type) override { auto mkqlType = const_cast<NMiniKQL::TType*>(static_cast<const NMiniKQL::TType*>(type)); ArgsTypes_.push_back(mkqlType); - return *this; - } - - NUdf::ICallableTypeBuilder& Arg( - const NUdf::ITypeBuilder& typeBuilder) override - { - auto type = static_cast<NMiniKQL::TType*>(typeBuilder.Build()); + return *this; + } + + NUdf::ICallableTypeBuilder& Arg( + const NUdf::ITypeBuilder& typeBuilder) override + { + auto type = static_cast<NMiniKQL::TType*>(typeBuilder.Build()); ArgsTypes_.push_back(type); - return *this; - } - - ICallableTypeBuilder& OptionalArgs(ui32 optionalArgs) override { - OptionalArgs_ = optionalArgs; - return *this; - } - - NUdf::TType* Build() const override { + return *this; + } + + ICallableTypeBuilder& OptionalArgs(ui32 optionalArgs) override { + OptionalArgs_ = optionalArgs; + return *this; + } + + NUdf::TType* Build() const override { Y_VERIFY(ReturnType_, "callable returns type is not configured"); - - NMiniKQL::TNode* payload = nullptr; - - auto callableType = NMiniKQL::TCallableType::Create( + + NMiniKQL::TNode* payload = nullptr; + + auto callableType = NMiniKQL::TCallableType::Create( UdfName, ReturnType_, ArgsTypes_.size(), const_cast<NMiniKQL::TType**>(ArgsTypes_.data()), - payload, Env_); - callableType->SetOptionalArgumentsCount(OptionalArgs_); - return callableType; - } - -private: - const NMiniKQL::TTypeEnvironment& Env_; - NMiniKQL::TType* ReturnType_; + payload, Env_); + callableType->SetOptionalArgumentsCount(OptionalArgs_); + return callableType; + } + +private: + const NMiniKQL::TTypeEnvironment& Env_; + NMiniKQL::TType* ReturnType_; TVector<NMiniKQL::TType*> ArgsTypes_; - ui32 OptionalArgs_; -}; - -////////////////////////////////////////////////////////////////////////////// -// TFunctionArgTypesBuilder -////////////////////////////////////////////////////////////////////////////// -class TFunctionArgTypesBuilder: public NUdf::IFunctionArgTypesBuilder -{ -public: - explicit TFunctionArgTypesBuilder( - NMiniKQL::TFunctionTypeInfoBuilder& parent, + ui32 OptionalArgs_; +}; + +////////////////////////////////////////////////////////////////////////////// +// TFunctionArgTypesBuilder +////////////////////////////////////////////////////////////////////////////// +class TFunctionArgTypesBuilder: public NUdf::IFunctionArgTypesBuilder +{ +public: + explicit TFunctionArgTypesBuilder( + NMiniKQL::TFunctionTypeInfoBuilder& parent, TVector<NMiniKQL::TArgInfo>& args) - : NUdf::IFunctionArgTypesBuilder(parent) - , Env_(parent.Env()) + : NUdf::IFunctionArgTypesBuilder(parent) + , Env_(parent.Env()) , Args_(args) - { - } - + { + } + NUdf::IFunctionArgTypesBuilder& Add(NUdf::TDataTypeId typeId) override { - auto type = NMiniKQL::TDataType::Create(typeId, Env_); + auto type = NMiniKQL::TDataType::Create(typeId, Env_); Args_.emplace_back(); Args_.back().Type_ = type; - return *this; - } - - NUdf::IFunctionArgTypesBuilder& Add(const NUdf::TType* type) override { - auto mkqlType = static_cast<const NMiniKQL::TType*>(type); + return *this; + } + + NUdf::IFunctionArgTypesBuilder& Add(const NUdf::TType* type) override { + auto mkqlType = static_cast<const NMiniKQL::TType*>(type); Args_.emplace_back(); Args_.back().Type_ = const_cast<NMiniKQL::TType*>(mkqlType); - return *this; - } - - NUdf::IFunctionArgTypesBuilder& Add( - const NUdf::ITypeBuilder& typeBuilder) override - { - auto type = static_cast<NMiniKQL::TType*>(typeBuilder.Build()); + return *this; + } + + NUdf::IFunctionArgTypesBuilder& Add( + const NUdf::ITypeBuilder& typeBuilder) override + { + auto type = static_cast<NMiniKQL::TType*>(typeBuilder.Build()); Args_.emplace_back(); Args_.back().Type_ = type; - return *this; - } - + return *this; + } + NUdf::IFunctionArgTypesBuilder& Name(const NUdf::TStringRef& name) override { Args_.back().Name_ = Env_.InternName(name); return *this; @@ -593,11 +593,11 @@ public: return *this; } -private: - const NMiniKQL::TTypeEnvironment& Env_; +private: + const NMiniKQL::TTypeEnvironment& Env_; TVector<NMiniKQL::TArgInfo>& Args_; -}; - +}; + ////////////////////////////////////////////////////////////////////////////// // THash ////////////////////////////////////////////////////////////////////////////// @@ -1233,39 +1233,39 @@ private: const NUdf::ICompare::TPtr Compare_; }; -} // namespace - -namespace NMiniKQL { - -////////////////////////////////////////////////////////////////////////////// -// TFunctionTypeInfoBuilder -////////////////////////////////////////////////////////////////////////////// -TFunctionTypeInfoBuilder::TFunctionTypeInfoBuilder( - const TTypeEnvironment& env, +} // namespace + +namespace NMiniKQL { + +////////////////////////////////////////////////////////////////////////////// +// TFunctionTypeInfoBuilder +////////////////////////////////////////////////////////////////////////////// +TFunctionTypeInfoBuilder::TFunctionTypeInfoBuilder( + const TTypeEnvironment& env, NUdf::ITypeInfoHelper::TPtr typeInfoHelper, const TStringBuf& moduleName, NUdf::ICountersProvider* countersProvider, const NUdf::TSourcePosition& pos, const NUdf::ISecureParamsProvider* provider) - : Env_(env) - , ReturnType_(nullptr) - , RunConfigType_(Env_.GetTypeOfVoid()) + : Env_(env) + , ReturnType_(nullptr) + , RunConfigType_(Env_.GetTypeOfVoid()) , UserType_(Env_.GetTypeOfVoid()) , TypeInfoHelper_(typeInfoHelper) , ModuleName_(moduleName) , CountersProvider_(countersProvider) , Pos_(pos) , SecureParamsProvider_(provider) -{ -} - +{ +} + NUdf::IFunctionTypeInfoBuilder1& TFunctionTypeInfoBuilder::ImplementationImpl( - NUdf::TUniquePtr<NUdf::IBoxedValue> impl) -{ + NUdf::TUniquePtr<NUdf::IBoxedValue> impl) +{ Implementation_ = std::move(impl); - return *this; -} - + return *this; +} + NUdf::IFunctionTypeInfoBuilder7& TFunctionTypeInfoBuilder::IRImplementationImpl( const NUdf::TStringRef& moduleIR, const NUdf::TStringRef& moduleIRUniqId, @@ -1316,32 +1316,32 @@ bool TFunctionTypeInfoBuilder::GetSecureParam(NUdf::TStringRef key, NUdf::TStrin NUdf::IFunctionTypeInfoBuilder1& TFunctionTypeInfoBuilder::ReturnsImpl( NUdf::TDataTypeId typeId) -{ - ReturnType_ = TDataType::Create(typeId, Env_); - return *this; -} - +{ + ReturnType_ = TDataType::Create(typeId, Env_); + return *this; +} + NUdf::IFunctionTypeInfoBuilder1& TFunctionTypeInfoBuilder::ReturnsImpl( - const NUdf::TType* type) -{ - ReturnType_ = static_cast<const NMiniKQL::TType*>(type); - return *this; -} - + const NUdf::TType* type) +{ + ReturnType_ = static_cast<const NMiniKQL::TType*>(type); + return *this; +} + NUdf::IFunctionTypeInfoBuilder1& TFunctionTypeInfoBuilder::ReturnsImpl( - const NUdf::ITypeBuilder& typeBuilder) -{ - ReturnType_ = static_cast<TType*>(typeBuilder.Build()); - return *this; -} - -NUdf::IFunctionArgTypesBuilder::TPtr TFunctionTypeInfoBuilder::Args( - ui32 expectedItem) -{ + const NUdf::ITypeBuilder& typeBuilder) +{ + ReturnType_ = static_cast<TType*>(typeBuilder.Build()); + return *this; +} + +NUdf::IFunctionArgTypesBuilder::TPtr TFunctionTypeInfoBuilder::Args( + ui32 expectedItem) +{ Args_.reserve(expectedItem); return new TFunctionArgTypesBuilder(*this, Args_); -} - +} + NUdf::IFunctionTypeInfoBuilder1& TFunctionTypeInfoBuilder::OptionalArgsImpl(ui32 optionalArgs) { OptionalArgs_ = optionalArgs; return *this; @@ -1354,25 +1354,25 @@ NUdf::IFunctionTypeInfoBuilder1& TFunctionTypeInfoBuilder::PayloadImpl(const NUd NUdf::IFunctionTypeInfoBuilder1& TFunctionTypeInfoBuilder::RunConfigImpl( NUdf::TDataTypeId typeId) -{ - RunConfigType_ = TDataType::Create(typeId, Env_); - return *this; -} - +{ + RunConfigType_ = TDataType::Create(typeId, Env_); + return *this; +} + NUdf::IFunctionTypeInfoBuilder1& TFunctionTypeInfoBuilder::RunConfigImpl( - const NUdf::TType* type) -{ - RunConfigType_ = static_cast<const NMiniKQL::TType*>(type); - return *this; -} - + const NUdf::TType* type) +{ + RunConfigType_ = static_cast<const NMiniKQL::TType*>(type); + return *this; +} + NUdf::IFunctionTypeInfoBuilder1& TFunctionTypeInfoBuilder::RunConfigImpl( - const NUdf::ITypeBuilder& typeBuilder) -{ - RunConfigType_ = static_cast<TType*>(typeBuilder.Build()); - return *this; -} - + const NUdf::ITypeBuilder& typeBuilder) +{ + RunConfigType_ = static_cast<TType*>(typeBuilder.Build()); + return *this; +} + NUdf::IFunctionTypeInfoBuilder1& TFunctionTypeInfoBuilder::UserTypeImpl( NUdf::TDataTypeId typeId) { @@ -1394,16 +1394,16 @@ NUdf::IFunctionTypeInfoBuilder1& TFunctionTypeInfoBuilder::UserTypeImpl( return *this; } -void TFunctionTypeInfoBuilder::SetError(const NUdf::TStringRef& error) -{ +void TFunctionTypeInfoBuilder::SetError(const NUdf::TStringRef& error) +{ if (!Error_) { Error_ = error; } -} - -void TFunctionTypeInfoBuilder::Build(TFunctionTypeInfo* funcInfo) -{ - if (ReturnType_) { +} + +void TFunctionTypeInfoBuilder::Build(TFunctionTypeInfo* funcInfo) +{ + if (ReturnType_) { TCallableTypeBuilder builder(Env_, UdfName, const_cast<NMiniKQL::TType*>(ReturnType_)); for (const auto& arg : Args_) { builder.Add(arg.Type_); @@ -1422,65 +1422,65 @@ void TFunctionTypeInfoBuilder::Build(TFunctionTypeInfo* funcInfo) builder.SetOptionalArgs(OptionalArgs_); funcInfo->FunctionType = builder.Build(); - } - - funcInfo->RunConfigType = RunConfigType_; + } + + funcInfo->RunConfigType = RunConfigType_; funcInfo->UserType = UserType_; funcInfo->Implementation = std::move(Implementation_); funcInfo->ModuleIR = std::move(ModuleIR_); funcInfo->ModuleIRUniqID = std::move(ModuleIRUniqID_); funcInfo->IRFunctionName = std::move(IRFunctionName_); funcInfo->BlockImplementation = std::move(BlockImplementation_); -} - +} + NUdf::TType* TFunctionTypeInfoBuilder::Primitive(NUdf::TDataTypeId typeId) const -{ - return TDataType::Create(typeId, Env_); -} - +{ + return TDataType::Create(typeId, Env_); +} + NUdf::TType* TFunctionTypeInfoBuilder::Decimal(ui8 precision, ui8 scale) const { return TDataDecimalType::Create(precision, scale, Env_); } -NUdf::IOptionalTypeBuilder::TPtr TFunctionTypeInfoBuilder::Optional() const -{ - return new TOptionalTypeBuilder(*this); -} - -NUdf::IListTypeBuilder::TPtr TFunctionTypeInfoBuilder::List() const -{ - return new TListTypeBuilder(*this); -} - -NUdf::IDictTypeBuilder::TPtr TFunctionTypeInfoBuilder::Dict() const -{ - return new TDictTypeBuilder(*this); -} - -NUdf::IStructTypeBuilder::TPtr TFunctionTypeInfoBuilder::Struct( - ui32 expectedItems) const -{ - return new NKikimr::TStructTypeBuilder(*this, expectedItems); -} - -NUdf::ITupleTypeBuilder::TPtr TFunctionTypeInfoBuilder::Tuple( - ui32 expectedItems) const -{ - return new TTupleTypeBuilder(*this, expectedItems); -} - -NUdf::ICallableTypeBuilder::TPtr TFunctionTypeInfoBuilder::Callable( - ui32 expectedArgs) const -{ - return new NKikimr::TCallableTypeBuilder(Env_, expectedArgs); -} - -NUdf::TType* TFunctionTypeInfoBuilder::Void() const -{ - return Env_.GetTypeOfVoid(); -} - +NUdf::IOptionalTypeBuilder::TPtr TFunctionTypeInfoBuilder::Optional() const +{ + return new TOptionalTypeBuilder(*this); +} + +NUdf::IListTypeBuilder::TPtr TFunctionTypeInfoBuilder::List() const +{ + return new TListTypeBuilder(*this); +} + +NUdf::IDictTypeBuilder::TPtr TFunctionTypeInfoBuilder::Dict() const +{ + return new TDictTypeBuilder(*this); +} + +NUdf::IStructTypeBuilder::TPtr TFunctionTypeInfoBuilder::Struct( + ui32 expectedItems) const +{ + return new NKikimr::TStructTypeBuilder(*this, expectedItems); +} + +NUdf::ITupleTypeBuilder::TPtr TFunctionTypeInfoBuilder::Tuple( + ui32 expectedItems) const +{ + return new TTupleTypeBuilder(*this, expectedItems); +} + +NUdf::ICallableTypeBuilder::TPtr TFunctionTypeInfoBuilder::Callable( + ui32 expectedArgs) const +{ + return new NKikimr::TCallableTypeBuilder(Env_, expectedArgs); +} + +NUdf::TType* TFunctionTypeInfoBuilder::Void() const +{ + return Env_.GetTypeOfVoid(); +} + NUdf::TType* TFunctionTypeInfoBuilder::Resource(const NUdf::TStringRef& tag) const { return TResourceType::Create(tag, Env_); } @@ -1493,10 +1493,10 @@ NUdf::IStreamTypeBuilder::TPtr TFunctionTypeInfoBuilder::Stream() const { return new TStreamTypeBuilder(*this); } -NUdf::ITypeInfoHelper::TPtr TFunctionTypeInfoBuilder::TypeInfoHelper() const { +NUdf::ITypeInfoHelper::TPtr TFunctionTypeInfoBuilder::TypeInfoHelper() const { return TypeInfoHelper_; -} - +} + NUdf::TCounter TFunctionTypeInfoBuilder::GetCounter(const NUdf::TStringRef& name, bool deriv) { if (CountersProvider_) { return CountersProvider_->GetCounter(ModuleName_, name, deriv); diff --git a/ydb/library/yql/minikql/mkql_type_builder.h b/ydb/library/yql/minikql/mkql_type_builder.h index 6136aa0a914..a93fb41cee3 100644 --- a/ydb/library/yql/minikql/mkql_type_builder.h +++ b/ydb/library/yql/minikql/mkql_type_builder.h @@ -1,29 +1,29 @@ -#pragma once - -#include "mkql_node.h" - +#pragma once + +#include "mkql_node.h" + #include <ydb/library/yql/public/udf/udf_type_builder.h> - -namespace NKikimr { -namespace NMiniKQL { - -////////////////////////////////////////////////////////////////////////////// -// TFunctionTypeInfo -////////////////////////////////////////////////////////////////////////////// -struct TFunctionTypeInfo -{ - TCallableType* FunctionType = nullptr; - const TType* RunConfigType = nullptr; + +namespace NKikimr { +namespace NMiniKQL { + +////////////////////////////////////////////////////////////////////////////// +// TFunctionTypeInfo +////////////////////////////////////////////////////////////////////////////// +struct TFunctionTypeInfo +{ + TCallableType* FunctionType = nullptr; + const TType* RunConfigType = nullptr; const TType* UserType = nullptr; - NUdf::TUniquePtr<NUdf::IBoxedValue> Implementation; + NUdf::TUniquePtr<NUdf::IBoxedValue> Implementation; NUdf::TUniquePtr<NUdf::IBoxedValue> BlockImplementation; TString ModuleIR; TString ModuleIRUniqID; TString IRFunctionName; - bool Deterministic = true; -}; - -////////////////////////////////////////////////////////////////////////////// + bool Deterministic = true; +}; + +////////////////////////////////////////////////////////////////////////////// // TArgInfo ////////////////////////////////////////////////////////////////////////////// struct TArgInfo { @@ -33,64 +33,64 @@ struct TArgInfo { }; ////////////////////////////////////////////////////////////////////////////// -// TFunctionTypeInfoBuilder -////////////////////////////////////////////////////////////////////////////// -class TFunctionTypeInfoBuilder: public NUdf::IFunctionTypeInfoBuilder -{ -public: - TFunctionTypeInfoBuilder( - const TTypeEnvironment& env, +// TFunctionTypeInfoBuilder +////////////////////////////////////////////////////////////////////////////// +class TFunctionTypeInfoBuilder: public NUdf::IFunctionTypeInfoBuilder +{ +public: + TFunctionTypeInfoBuilder( + const TTypeEnvironment& env, NUdf::ITypeInfoHelper::TPtr typeInfoHelper, const TStringBuf& moduleName, NUdf::ICountersProvider* countersProvider, const NUdf::TSourcePosition& pos, const NUdf::ISecureParamsProvider* provider = nullptr); - + NUdf::IFunctionTypeInfoBuilder1& ImplementationImpl( - NUdf::TUniquePtr<NUdf::IBoxedValue> impl) override; - + NUdf::TUniquePtr<NUdf::IBoxedValue> impl) override; + NUdf::IFunctionTypeInfoBuilder1& ReturnsImpl(NUdf::TDataTypeId typeId) override; NUdf::IFunctionTypeInfoBuilder1& ReturnsImpl(const NUdf::TType* type) override; NUdf::IFunctionTypeInfoBuilder1& ReturnsImpl( - const NUdf::ITypeBuilder& typeBuilder) override; - - NUdf::IFunctionArgTypesBuilder::TPtr Args(ui32 expectedItem) override; + const NUdf::ITypeBuilder& typeBuilder) override; + + NUdf::IFunctionArgTypesBuilder::TPtr Args(ui32 expectedItem) override; NUdf::IFunctionTypeInfoBuilder1& OptionalArgsImpl(ui32 optionalArgs) override; NUdf::IFunctionTypeInfoBuilder1& PayloadImpl(const NUdf::TStringRef& payload) override; - + NUdf::IFunctionTypeInfoBuilder1& RunConfigImpl(NUdf::TDataTypeId typeId) override; NUdf::IFunctionTypeInfoBuilder1& RunConfigImpl(const NUdf::TType* type) override; NUdf::IFunctionTypeInfoBuilder1& RunConfigImpl( - const NUdf::ITypeBuilder& typeBuilder) override; - + const NUdf::ITypeBuilder& typeBuilder) override; + NUdf::IFunctionTypeInfoBuilder1& UserTypeImpl(NUdf::TDataTypeId typeId) override; NUdf::IFunctionTypeInfoBuilder1& UserTypeImpl(const NUdf::TType* type) override; NUdf::IFunctionTypeInfoBuilder1& UserTypeImpl(const NUdf::ITypeBuilder& typeBuilder) override; - void SetError(const NUdf::TStringRef& error) override; + void SetError(const NUdf::TStringRef& error) override; inline bool HasError() const { return !Error_.empty(); } inline const TString& GetError() const { return Error_; } - - void Build(TFunctionTypeInfo* funcInfo); - + + void Build(TFunctionTypeInfo* funcInfo); + NUdf::TType* Primitive(NUdf::TDataTypeId typeId) const override; - NUdf::IOptionalTypeBuilder::TPtr Optional() const override; - NUdf::IListTypeBuilder::TPtr List() const override; - NUdf::IDictTypeBuilder::TPtr Dict() const override; - NUdf::IStructTypeBuilder::TPtr Struct(ui32 expectedItems) const override; - NUdf::ITupleTypeBuilder::TPtr Tuple(ui32 expectedItems) const override; - NUdf::ICallableTypeBuilder::TPtr Callable(ui32 expectedArgs) const override; - - NUdf::TType* Void() const override; + NUdf::IOptionalTypeBuilder::TPtr Optional() const override; + NUdf::IListTypeBuilder::TPtr List() const override; + NUdf::IDictTypeBuilder::TPtr Dict() const override; + NUdf::IStructTypeBuilder::TPtr Struct(ui32 expectedItems) const override; + NUdf::ITupleTypeBuilder::TPtr Tuple(ui32 expectedItems) const override; + NUdf::ICallableTypeBuilder::TPtr Callable(ui32 expectedArgs) const override; + + NUdf::TType* Void() const override; NUdf::TType* Resource(const NUdf::TStringRef& tag) const override; NUdf::IVariantTypeBuilder::TPtr Variant() const override; - + NUdf::IStreamTypeBuilder::TPtr Stream() const override; - NUdf::ITypeInfoHelper::TPtr TypeInfoHelper() const override; - - const TTypeEnvironment& Env() const { return Env_; } - + NUdf::ITypeInfoHelper::TPtr TypeInfoHelper() const override; + + const TTypeEnvironment& Env() const { return Env_; } + NUdf::TCounter GetCounter(const NUdf::TStringRef& name, bool deriv) override; NUdf::TScopedProbe GetScopedProbe(const NUdf::TStringRef& name) override; NUdf::TSourcePosition GetSourcePosition() override; @@ -119,12 +119,12 @@ public: NUdf::TType* Tagged(const NUdf::TType* baseType, const NUdf::TStringRef& tag) const override; bool GetSecureParam(NUdf::TStringRef key, NUdf::TStringRef& value) const override; -private: - const TTypeEnvironment& Env_; - NUdf::TUniquePtr<NUdf::IBoxedValue> Implementation_; +private: + const TTypeEnvironment& Env_; + NUdf::TUniquePtr<NUdf::IBoxedValue> Implementation_; NUdf::TUniquePtr<NUdf::IBoxedValue> BlockImplementation_; - const TType* ReturnType_; - const TType* RunConfigType_; + const TType* ReturnType_; + const TType* RunConfigType_; const TType* UserType_; TVector<TArgInfo> Args_; TString Error_; @@ -138,8 +138,8 @@ private: TString ModuleIR_; TString ModuleIRUniqID_; TString IRFunctionName_; -}; - +}; + class TTypeInfoHelper : public NUdf::ITypeInfoHelper { public: diff --git a/ydb/library/yql/minikql/mkql_utils.h b/ydb/library/yql/minikql/mkql_utils.h index 3e5e23526fc..0c22d2857c5 100644 --- a/ydb/library/yql/minikql/mkql_utils.h +++ b/ydb/library/yql/minikql/mkql_utils.h @@ -1,57 +1,57 @@ -#pragma once - -#include <util/generic/strbuf.h> - -namespace NKikimr { -namespace NMiniKQL { - -// TODO: move this class to beter place -class TStatus -{ -public: - inline static const TStatus Ok() { - return TStatus(); - } - - inline static TStatus Error(const TStringBuf& error) { +#pragma once + +#include <util/generic/strbuf.h> + +namespace NKikimr { +namespace NMiniKQL { + +// TODO: move this class to beter place +class TStatus +{ +public: + inline static const TStatus Ok() { + return TStatus(); + } + + inline static TStatus Error(const TStringBuf& error) { Y_VERIFY_DEBUG(!error.empty()); return TStatus(TString(error)); - } - + } + inline static TStatus Error(TString&& error) { Y_VERIFY_DEBUG(!error.empty()); return TStatus(std::move(error)); - } - - inline static TStatus Error() { + } + + inline static TStatus Error() { return TStatus(TString(TStringBuf("Error: "))); - } - - template <class T> - inline TStatus& operator<<(const T& t) { - Error_.append(t); - return *this; - } - - inline bool IsOk() const { + } + + template <class T> + inline TStatus& operator<<(const T& t) { + Error_.append(t); + return *this; + } + + inline bool IsOk() const { return Error_.empty(); - } - + } + inline const TString& GetError() const { - return Error_; - } - -private: - inline TStatus() = default; - + return Error_; + } + +private: + inline TStatus() = default; + inline TStatus(TString&& error) : Error_(std::move(error)) - { - } - -private: + { + } + +private: TString Error_; -}; - +}; + } // namespace NMiniKQL } // namespace NKikimr diff --git a/ydb/library/yql/minikql/ut/ya.make b/ydb/library/yql/minikql/ut/ya.make index ccbb386b489..098a77d0c60 100644 --- a/ydb/library/yql/minikql/ut/ya.make +++ b/ydb/library/yql/minikql/ut/ya.make @@ -20,11 +20,11 @@ SRCS( compact_hash_ut.cpp mkql_alloc_ut.cpp mkql_node_builder_ut.cpp - mkql_node_cast_ut.cpp + mkql_node_cast_ut.cpp mkql_node_printer_ut.cpp mkql_node_ut.cpp - mkql_opt_literal_ut.cpp - mkql_stats_registry_ut.cpp + mkql_opt_literal_ut.cpp + mkql_stats_registry_ut.cpp mkql_type_ops_ut.cpp mkql_string_util_ut.cpp pack_num_ut.cpp diff --git a/ydb/library/yql/minikql/ya.make b/ydb/library/yql/minikql/ya.make index 4e3f3688b43..c4b608128d8 100644 --- a/ydb/library/yql/minikql/ya.make +++ b/ydb/library/yql/minikql/ya.make @@ -21,8 +21,8 @@ SRCS( mkql_node.h mkql_node_builder.cpp mkql_node_builder.h - mkql_node_cast.cpp - mkql_node_cast.h + mkql_node_cast.cpp + mkql_node_cast.h mkql_node_printer.cpp mkql_node_printer.h mkql_node_serialization.cpp @@ -35,13 +35,13 @@ SRCS( mkql_program_builder.h mkql_runtime_version.cpp mkql_runtime_version.h - mkql_stats_registry.cpp + mkql_stats_registry.cpp mkql_string_util.cpp mkql_string_util.h mkql_terminator.cpp mkql_terminator.h - mkql_type_builder.cpp - mkql_type_builder.h + mkql_type_builder.cpp + mkql_type_builder.h mkql_type_ops.cpp mkql_type_ops.h mkql_unboxed_value_stream.cpp @@ -58,7 +58,7 @@ PEERDIR( library/cpp/actors/util library/cpp/deprecated/enum_codegen library/cpp/enumbitset - library/cpp/monlib/dynamic_counters + library/cpp/monlib/dynamic_counters library/cpp/packedtypes library/cpp/resource library/cpp/yson diff --git a/ydb/library/yql/providers/common/codec/yql_codec.cpp b/ydb/library/yql/providers/common/codec/yql_codec.cpp index ba8bffad802..7587e188fab 100644 --- a/ydb/library/yql/providers/common/codec/yql_codec.cpp +++ b/ydb/library/yql/providers/common/codec/yql_codec.cpp @@ -93,7 +93,7 @@ void WriteYsonValueImpl(TYsonResultWriter& writer, const NUdf::TUnboxedValuePod& case NUdf::TDataType<char*>::Id: case NUdf::TDataType<NUdf::TUuid>::Id: case NUdf::TDataType<NUdf::TDyNumber>::Id: - writer.OnStringScalar(value.AsStringRef()); + writer.OnStringScalar(value.AsStringRef()); return; case NUdf::TDataType<NUdf::TDecimal>::Id: { @@ -255,12 +255,12 @@ TString WriteYsonValue(const NUdf::TUnboxedValuePod& value, TType* type, const T TCodecContext::TCodecContext( const TTypeEnvironment& env, - const IFunctionRegistry& functionRegistry, - const NKikimr::NMiniKQL::THolderFactory* holderFactory /* = nullptr */ + const IFunctionRegistry& functionRegistry, + const NKikimr::NMiniKQL::THolderFactory* holderFactory /* = nullptr */ ) : Env(env) , Builder(Env, functionRegistry) - , HolderFactory(holderFactory) + , HolderFactory(holderFactory) { } diff --git a/ydb/library/yql/providers/common/codec/yql_codec.h b/ydb/library/yql/providers/common/codec/yql_codec.h index 812373ba859..512e01224c4 100644 --- a/ydb/library/yql/providers/common/codec/yql_codec.h +++ b/ydb/library/yql/providers/common/codec/yql_codec.h @@ -49,13 +49,13 @@ TExprNode::TPtr NodeToExprLiteral(TPositionHandle pos, const TTypeAnnotationNode struct TCodecContext { const NKikimr::NMiniKQL::TTypeEnvironment& Env; NKikimr::NMiniKQL::TProgramBuilder Builder; - const NKikimr::NMiniKQL::THolderFactory* HolderFactory; // lazy initialized + const NKikimr::NMiniKQL::THolderFactory* HolderFactory; // lazy initialized std::list<std::vector<size_t>> StructReorders; TCodecContext( const NKikimr::NMiniKQL::TTypeEnvironment& env, - const NKikimr::NMiniKQL::IFunctionRegistry& functionRegistry, - const NKikimr::NMiniKQL::THolderFactory* holderFactory = nullptr + const NKikimr::NMiniKQL::IFunctionRegistry& functionRegistry, + const NKikimr::NMiniKQL::THolderFactory* holderFactory = nullptr ); }; diff --git a/ydb/library/yql/providers/common/metrics/metrics_registry.cpp b/ydb/library/yql/providers/common/metrics/metrics_registry.cpp index 6d369796861..1fb7bdcbba7 100644 --- a/ydb/library/yql/providers/common/metrics/metrics_registry.cpp +++ b/ydb/library/yql/providers/common/metrics/metrics_registry.cpp @@ -1,51 +1,51 @@ -#include "metrics_registry.h" - +#include "metrics_registry.h" + #include <ydb/library/yql/providers/common/metrics/protos/metrics_registry.pb.h> -#include <util/generic/hash.h> -#include <util/generic/maybe.h> -#include <util/generic/stack.h> - - -namespace NYql { -namespace { - -////////////////////////////////////////////////////////////////////////////// -// TCountersPhotographer -////////////////////////////////////////////////////////////////////////////// -class TCountersPhotographer: public NMonitoring::ICountableConsumer { -public: +#include <util/generic/hash.h> +#include <util/generic/maybe.h> +#include <util/generic/stack.h> + + +namespace NYql { +namespace { + +////////////////////////////////////////////////////////////////////////////// +// TCountersPhotographer +////////////////////////////////////////////////////////////////////////////// +class TCountersPhotographer: public NMonitoring::ICountableConsumer { +public: TCountersPhotographer(NProto::TCounterGroup* groupProto, bool invalidate) - : HasAnyCounters_(false) + : HasAnyCounters_(false) , Invalidate_(invalidate) - { - GroupsProto_.push(groupProto); - } - - bool HasAnyCounters() const { - return HasAnyCounters_; - } - -private: - void OnCounter( + { + GroupsProto_.push(groupProto); + } + + bool HasAnyCounters() const { + return HasAnyCounters_; + } + +private: + void OnCounter( const TString& labelName, const TString& labelValue, - const TSensorCounter* counter) override - { - HasAnyCounters_ = true; - - auto* counterProto = GroupsProto_.top()->AddCounters(); - counterProto->SetDerivative(counter->ForDerivative()); - auto counterVal = counter->Val(); - counterProto->SetValue(counterVal); + const TSensorCounter* counter) override + { + HasAnyCounters_ = true; + + auto* counterProto = GroupsProto_.top()->AddCounters(); + counterProto->SetDerivative(counter->ForDerivative()); + auto counterVal = counter->Val(); + counterProto->SetValue(counterVal); if (Invalidate_) { - const_cast<TSensorCounter*>(counter)->Sub(counterVal); + const_cast<TSensorCounter*>(counter)->Sub(counterVal); } - - auto* label = counterProto->MutableLabel(); - label->SetName(labelName); - label->SetValue(labelValue); - } - + + auto* label = counterProto->MutableLabel(); + label->SetName(labelName); + label->SetValue(labelValue); + } + void OnHistogram(const TString& labelName, const TString& labelValue, NMonitoring::IHistogramSnapshotPtr snapshot, bool) override { if (Invalidate_) { return; @@ -64,48 +64,48 @@ private: bucket->SetUpperBound(upperBound); bucket->SetValue(value); } - } - - void OnGroupBegin( + } + + void OnGroupBegin( const TString& labelName, const TString& labelValue, - const TSensorsGroup*) override - { + const TSensorsGroup*) override + { if (labelName.empty() && labelValue.empty()) { - // root group is alrady present - return; - } - - auto* groupProto = GroupsProto_.top()->AddGroups(); - auto* label = groupProto->MutableLabel(); - label->SetName(labelName); - label->SetValue(labelValue); - - GroupsProto_.push(groupProto); - } - - void OnGroupEnd(const TString&, const TString&, const TSensorsGroup*) override { - GroupsProto_.pop(); - } - -private: + // root group is alrady present + return; + } + + auto* groupProto = GroupsProto_.top()->AddGroups(); + auto* label = groupProto->MutableLabel(); + label->SetName(labelName); + label->SetValue(labelValue); + + GroupsProto_.push(groupProto); + } + + void OnGroupEnd(const TString&, const TString&, const TSensorsGroup*) override { + GroupsProto_.pop(); + } + +private: TStack<NProto::TCounterGroup*> GroupsProto_; - bool HasAnyCounters_; + bool HasAnyCounters_; bool Invalidate_; -}; - -////////////////////////////////////////////////////////////////////////////// -// TMetricsRegistryImpl -////////////////////////////////////////////////////////////////////////////// -class TMetricsRegistryImpl final: public IMetricsRegistry { -public: - TMetricsRegistryImpl( - TSensorsGroupPtr sensors, +}; + +////////////////////////////////////////////////////////////////////////////// +// TMetricsRegistryImpl +////////////////////////////////////////////////////////////////////////////// +class TMetricsRegistryImpl final: public IMetricsRegistry { +public: + TMetricsRegistryImpl( + TSensorsGroupPtr sensors, TMaybe<TString> userName) - : Sensors_(std::move(sensors)) - , UserName_(std::move(userName)) - { - } - + : Sensors_(std::move(sensors)) + , UserName_(std::move(userName)) + { + } + void SetCounter( const TString& labelName, const TString& labelValue, @@ -129,27 +129,27 @@ public: } } - void IncCounter( + void IncCounter( const TString& labelName, const TString& labelValue, - bool derivative) override - { - // total aggregate counter - auto totalCnt = GetCounter(labelName, labelValue, nullptr, derivative); - if (totalCnt) { - totalCnt->Inc(); - } - - if (UserName_) { - // per user counter - auto userCnt = GetCounter(labelName, labelValue, UserName_.Get(), - derivative); - if (userCnt) { - userCnt->Inc(); - } - } - } - + bool derivative) override + { + // total aggregate counter + auto totalCnt = GetCounter(labelName, labelValue, nullptr, derivative); + if (totalCnt) { + totalCnt->Inc(); + } + + if (UserName_) { + // per user counter + auto userCnt = GetCounter(labelName, labelValue, UserName_.Get(), + derivative); + if (userCnt) { + userCnt->Inc(); + } + } + } + void AddCounter( const TString& labelName, const TString& labelValue, @@ -172,19 +172,19 @@ public: } } - bool TakeSnapshot(NProto::TMetricsRegistrySnapshot* snapshot) const override { - bool hasRootGroupBefore = snapshot->HasRootGroup(); + bool TakeSnapshot(NProto::TMetricsRegistrySnapshot* snapshot) const override { + bool hasRootGroupBefore = snapshot->HasRootGroup(); TCountersPhotographer photographer(snapshot->MutableRootGroup(), snapshot->GetDontIncrement() == false); Sensors_->Accept(TString(), TString(), photographer); - if (!photographer.HasAnyCounters() && !hasRootGroupBefore) { - // remove prematurely allocated group - snapshot->ClearRootGroup(); - return false; - } - return true; - } - - void MergeSnapshot(const NProto::TMetricsRegistrySnapshot& snapshot) override { + if (!photographer.HasAnyCounters() && !hasRootGroupBefore) { + // remove prematurely allocated group + snapshot->ClearRootGroup(); + return false; + } + return true; + } + + void MergeSnapshot(const NProto::TMetricsRegistrySnapshot& snapshot) override { MergeFromGroupProto( snapshot.HasMergeToRoot() ? GetSensorsRootGroup().Get() @@ -193,42 +193,42 @@ public: snapshot.HasDontIncrement() ? snapshot.GetDontIncrement() : false); - } - + } + IMetricsRegistryPtr Personalized(const TString& userName) const override { - return new TMetricsRegistryImpl(Sensors_, MakeMaybe(userName)); - } - - void Flush() override { - // do nothing - } - + return new TMetricsRegistryImpl(Sensors_, MakeMaybe(userName)); + } + + void Flush() override { + // do nothing + } + TSensorsGroupPtr GetSensors() override { return Sensors_.Get(); } -private: - TSensorCounterPtr GetCounter( +private: + TSensorCounterPtr GetCounter( const TString& labelName, const TString& labelValue, const TString* userName, - bool derivative) - { + bool derivative) + { static const TString USER("user"); static const TString USER_ABSOLUTE("user_absolute"); static const TString TOTAL("total"); - + const TString& userGroup = derivative ? USER : USER_ABSOLUTE; - return Sensors_ - ->GetSubgroup(userGroup, userName ? *userName : TOTAL) - ->GetNamedCounter(labelName, labelValue, derivative); - } - - void MergeFromGroupProto( + return Sensors_ + ->GetSubgroup(userGroup, userName ? *userName : TOTAL) + ->GetNamedCounter(labelName, labelValue, derivative); + } + + void MergeFromGroupProto( TSensorsGroup* group, const NProto::TCounterGroup& groupProto, bool asIs) - { - for (const auto& counterProto: groupProto.GetCounters()) { - const auto& label = counterProto.GetLabel(); + { + for (const auto& counterProto: groupProto.GetCounters()) { + const auto& label = counterProto.GetLabel(); if (!counterProto.GetBucket().empty()) { NMonitoring::TBucketBounds bounds; @@ -261,28 +261,28 @@ private: *counter += counterProto.GetValue(); } } - } - - for (const auto& subGroupProto: groupProto.GetGroups()) { - const auto& label = subGroupProto.GetLabel(); - auto subGroup = group->GetSubgroup( - label.GetName(), label.GetValue()); + } + + for (const auto& subGroupProto: groupProto.GetGroups()) { + const auto& label = subGroupProto.GetLabel(); + auto subGroup = group->GetSubgroup( + label.GetName(), label.GetValue()); MergeFromGroupProto(subGroup.Get(), subGroupProto, asIs); - } - } - -private: - TSensorsGroupPtr Sensors_; + } + } + +private: + TSensorsGroupPtr Sensors_; const TMaybe<TString> UserName_; THashMap<NMonitoring::THistogramPtr, NMonitoring::IHistogramCollector*> Histograms; -}; - -} // namespace - - -IMetricsRegistryPtr CreateMetricsRegistry(TSensorsGroupPtr sensors) { - return new TMetricsRegistryImpl(std::move(sensors), Nothing()); -} - -} // namespace NYql +}; + +} // namespace + + +IMetricsRegistryPtr CreateMetricsRegistry(TSensorsGroupPtr sensors) { + return new TMetricsRegistryImpl(std::move(sensors), Nothing()); +} + +} // namespace NYql diff --git a/ydb/library/yql/providers/common/metrics/metrics_registry.h b/ydb/library/yql/providers/common/metrics/metrics_registry.h index 4058053632c..d344ba9dd98 100644 --- a/ydb/library/yql/providers/common/metrics/metrics_registry.h +++ b/ydb/library/yql/providers/common/metrics/metrics_registry.h @@ -1,61 +1,61 @@ -#pragma once - -#include "sensors_group.h" - +#pragma once + +#include "sensors_group.h" + #include <util/generic/yexception.h> - -namespace NYql { - + +namespace NYql { + namespace NProto { class TMetricsRegistrySnapshot; } -struct IMetricsRegistry; -using IMetricsRegistryPtr = TIntrusivePtr<IMetricsRegistry>; - -using TMetricsDecorator = std::function<IMetricsRegistryPtr( +struct IMetricsRegistry; +using IMetricsRegistryPtr = TIntrusivePtr<IMetricsRegistry>; + +using TMetricsDecorator = std::function<IMetricsRegistryPtr( const IMetricsRegistryPtr& old, const TString& username)>; - -////////////////////////////////////////////////////////////////////////////// -// IYqlMetricsRegistry -////////////////////////////////////////////////////////////////////////////// -struct IMetricsRegistry: public TThrRefBase { - + +////////////////////////////////////////////////////////////////////////////// +// IYqlMetricsRegistry +////////////////////////////////////////////////////////////////////////////// +struct IMetricsRegistry: public TThrRefBase { + virtual void SetCounter( const TString& labelName, const TString& labelValue, i64 value, bool derivative = false) = 0; - virtual void IncCounter( + virtual void IncCounter( const TString& labelName, const TString& labelValue, - bool derivative = true) = 0; - + bool derivative = true) = 0; + virtual void AddCounter( const TString& labelName, const TString& labelValue, i64 value, bool derivative = true) = 0; - // will invalidate all counters - virtual bool TakeSnapshot( - NProto::TMetricsRegistrySnapshot* snapshot) const = 0; - - virtual void MergeSnapshot( - const NProto::TMetricsRegistrySnapshot& snapshot) = 0; - - virtual IMetricsRegistryPtr Personalized( + // will invalidate all counters + virtual bool TakeSnapshot( + NProto::TMetricsRegistrySnapshot* snapshot) const = 0; + + virtual void MergeSnapshot( + const NProto::TMetricsRegistrySnapshot& snapshot) = 0; + + virtual IMetricsRegistryPtr Personalized( const TString& userName) const = 0; - - virtual void Flush() = 0; + + virtual void Flush() = 0; virtual TSensorsGroupPtr GetSensors() { ythrow yexception() << "Not implemented"; } -}; - - -IMetricsRegistryPtr CreateMetricsRegistry(TSensorsGroupPtr sensors); - -} // namespace NYql +}; + + +IMetricsRegistryPtr CreateMetricsRegistry(TSensorsGroupPtr sensors); + +} // namespace NYql diff --git a/ydb/library/yql/providers/common/metrics/protos/metrics_registry.proto b/ydb/library/yql/providers/common/metrics/protos/metrics_registry.proto index 07d97d0d90f..07144462582 100644 --- a/ydb/library/yql/providers/common/metrics/protos/metrics_registry.proto +++ b/ydb/library/yql/providers/common/metrics/protos/metrics_registry.proto @@ -1,31 +1,31 @@ -package NYql.NProto; - - -message TLabel { - optional string Name = 1; - optional string Value = 2; -} - +package NYql.NProto; + + +message TLabel { + optional string Name = 1; + optional string Value = 2; +} + message TBucket { optional double UpperBound = 1; optional int64 Value = 2; } -message TCounter { - optional TLabel Label = 1; - optional int64 Value = 2; - optional bool Derivative = 3; +message TCounter { + optional TLabel Label = 1; + optional int64 Value = 2; + optional bool Derivative = 3; repeated TBucket Bucket = 4; -} - -message TCounterGroup { - optional TLabel Label = 1; - repeated TCounter Counters = 2; - repeated TCounterGroup Groups = 3; -} - -message TMetricsRegistrySnapshot { - optional TCounterGroup RootGroup = 1; +} + +message TCounterGroup { + optional TLabel Label = 1; + repeated TCounter Counters = 2; + repeated TCounterGroup Groups = 3; +} + +message TMetricsRegistrySnapshot { + optional TCounterGroup RootGroup = 1; optional bool DontIncrement = 2; optional bool MergeToRoot = 3; -} +} diff --git a/ydb/library/yql/providers/common/metrics/sensors_group.cpp b/ydb/library/yql/providers/common/metrics/sensors_group.cpp index cb9817ce06d..37f8d573087 100644 --- a/ydb/library/yql/providers/common/metrics/sensors_group.cpp +++ b/ydb/library/yql/providers/common/metrics/sensors_group.cpp @@ -1,21 +1,21 @@ -#include "sensors_group.h" - -#include <util/generic/singleton.h> - - -namespace NYql { - -struct TSensorsRootGroup { - TSensorsGroupPtr Value; - - TSensorsRootGroup() - : Value(new TSensorsGroup) - { - } -}; - -TSensorsGroupPtr GetSensorsRootGroup() { - return Singleton<TSensorsRootGroup>()->Value; -} - -} // namspace NYql +#include "sensors_group.h" + +#include <util/generic/singleton.h> + + +namespace NYql { + +struct TSensorsRootGroup { + TSensorsGroupPtr Value; + + TSensorsRootGroup() + : Value(new TSensorsGroup) + { + } +}; + +TSensorsGroupPtr GetSensorsRootGroup() { + return Singleton<TSensorsRootGroup>()->Value; +} + +} // namspace NYql diff --git a/ydb/library/yql/providers/common/metrics/sensors_group.h b/ydb/library/yql/providers/common/metrics/sensors_group.h index 065c51db762..956a0c6fe22 100644 --- a/ydb/library/yql/providers/common/metrics/sensors_group.h +++ b/ydb/library/yql/providers/common/metrics/sensors_group.h @@ -1,30 +1,30 @@ -#pragma once - -#include <library/cpp/monlib/dynamic_counters/counters.h> - - -namespace NYql { -namespace NSensorComponent { +#pragma once + +#include <library/cpp/monlib/dynamic_counters/counters.h> + + +namespace NYql { +namespace NSensorComponent { static const TString kExecutor = "executor"; static const TString kWorkerServer = "worker_server"; static const TString kDataServer = "data_server"; static const TString kInspectorClient = "inspector_client"; static const TString kDq = "dq"; -} // namspace NSensorComponent - - -using TSensorsGroup = NMonitoring::TDynamicCounters; -using TSensorsGroupPtr = TIntrusivePtr<TSensorsGroup>; - -using TSensorCounter = NMonitoring::TCounterForPtr; -using TSensorCounterPtr = TIntrusivePtr<TSensorCounter>; - - -TSensorsGroupPtr GetSensorsRootGroup(); - +} // namspace NSensorComponent + + +using TSensorsGroup = NMonitoring::TDynamicCounters; +using TSensorsGroupPtr = TIntrusivePtr<TSensorsGroup>; + +using TSensorCounter = NMonitoring::TCounterForPtr; +using TSensorCounterPtr = TIntrusivePtr<TSensorCounter>; + + +TSensorsGroupPtr GetSensorsRootGroup(); + inline TSensorsGroupPtr GetSensorsGroupFor(const TString& compName) { static TString compLabel("component"); - return GetSensorsRootGroup()->GetSubgroup(compLabel, compName); -} - -} // namspace NYql + return GetSensorsRootGroup()->GetSubgroup(compLabel, compName); +} + +} // namspace NYql diff --git a/ydb/library/yql/providers/common/metrics/ya.make b/ydb/library/yql/providers/common/metrics/ya.make index fdbe77b7652..19f8832060a 100644 --- a/ydb/library/yql/providers/common/metrics/ya.make +++ b/ydb/library/yql/providers/common/metrics/ya.make @@ -11,7 +11,7 @@ SRCS( PEERDIR( library/cpp/logger/global - library/cpp/monlib/dynamic_counters + library/cpp/monlib/dynamic_counters ydb/library/yql/providers/common/metrics/protos ) diff --git a/ydb/library/yql/providers/common/mkql/yql_provider_mkql.h b/ydb/library/yql/providers/common/mkql/yql_provider_mkql.h index 7dfa777249f..a25c03b488e 100644 --- a/ydb/library/yql/providers/common/mkql/yql_provider_mkql.h +++ b/ydb/library/yql/providers/common/mkql/yql_provider_mkql.h @@ -14,7 +14,7 @@ struct TMkqlBuildContext { using TMemoizedNodesMap = TArgumentsMap; const IMkqlCallableCompiler& MkqlCompiler; - NKikimr::NMiniKQL::TProgramBuilder& ProgramBuilder; + NKikimr::NMiniKQL::TProgramBuilder& ProgramBuilder; TExprContext& ExprCtx; TMemoizedNodesMap Memoization; TMkqlBuildContext *const ParentCtx = nullptr; diff --git a/ydb/library/yql/providers/common/proto/gateways_config.proto b/ydb/library/yql/providers/common/proto/gateways_config.proto index 64d9d215cee..8561fe25368 100644 --- a/ydb/library/yql/providers/common/proto/gateways_config.proto +++ b/ydb/library/yql/providers/common/proto/gateways_config.proto @@ -1,6 +1,6 @@ -package NYql; -option java_package = "ru.yandex.yql.proto"; - +package NYql; +option java_package = "ru.yandex.yql.proto"; + import "ydb/library/yql/protos/clickhouse.proto"; /////////////////////////////// common /////////////////////////////// @@ -18,12 +18,12 @@ message TActivationPercentage { optional bool ExcludeRobots = 5 [default = true]; } -message TAttr { - required string Name = 1; - required string Value = 2; +message TAttr { + required string Name = 1; + required string Value = 2; optional TActivationPercentage Activation = 3; -} - +} + message TRemoteFilePattern { required string Pattern = 1; // regexp to match required string Cluster = 2; // replacement string for YT server @@ -60,15 +60,15 @@ message THttpGatewayConfig { /////////////////////////////// YT /////////////////////////////// message TYtClusterConfig { - optional string Name = 1; // Short cluster name - optional string Cluster = 2; // Real cluster server name with port + optional string Name = 1; // Short cluster name + optional string Cluster = 2; // Real cluster server name with port optional bool Default = 3 [default = false]; // Default cluster optional string YTToken = 4; // token to use at YT servers optional string YTName = 5; // YT cluster name repeated TAttr Settings = 100; -} - +} + message TYtGatewayConfig { optional uint32 GatewayThreads = 1 [default = 0]; // Number of gateway MtpQueue threads. The adaptive MtpQueue will be used for 0 value optional EYtLogLevel YtLogLevel = 2 [default = YL_ERROR]; // The verbosity level of YT log @@ -83,20 +83,20 @@ message TYtGatewayConfig { optional uint64 YtDebugLogSize = 11 [default = 0]; // Max YT debug log size optional bool YtDebugLogAlwaysWrite = 12 [default = false]; // Write YT debug log always or on error only optional string LocalChainFile = 14; // File to dump table contet in test mode - + repeated TRemoteFilePattern RemoteFilePatterns = 100; repeated TYtClusterConfig ClusterMapping = 101; // Accepted cluster shortcuts. All names not from this list will be rejected repeated TAttr DefaultSettings = 102; -} - +} + /////////////////////////////// Kikimr /////////////////////////////// -message THostPort { - required string Host = 1; - required uint32 Port = 2; +message THostPort { + required string Host = 1; + required uint32 Port = 2; optional uint64 TotalTimeoutMs = 3; -} - +} + message TKikimrGrpcData { repeated string Locators = 1; optional uint64 TimeoutMs = 2; @@ -106,10 +106,10 @@ message TKikimrGrpcData { optional bool UseLegacyApi = 6; } -message TKikimrClusterConfig { - optional string Name = 1; - optional bool Default = 2 [default = false]; - repeated THostPort MessageBus = 3; +message TKikimrClusterConfig { + optional string Name = 1; + optional bool Default = 2 [default = false]; + repeated THostPort MessageBus = 3; optional TKikimrGrpcData Grpc = 4; optional string Database = 5; optional uint32 TvmId = 6 [default = 0]; @@ -119,13 +119,13 @@ message TKikimrClusterConfig { optional bool IsFromMvp = 10; optional ETokenType TokenType = 11; repeated TAttr Settings = 101; -} - -message TKikimrGatewayConfig { - repeated TKikimrClusterConfig ClusterMapping = 1; +} + +message TKikimrGatewayConfig { + repeated TKikimrClusterConfig ClusterMapping = 1; repeated TAttr DefaultSettings = 2; -} - +} + /////////////////////////////// Kikimr MVP /////////////////////////////// message TKikimrMvpProxyConfig { @@ -435,7 +435,7 @@ message TSqlCoreConfig { /////////////////////////////// Root /////////////////////////////// -message TGatewaysConfig { +message TGatewaysConfig { optional TYtGatewayConfig Yt = 1; optional TKikimrGatewayConfig Kikimr = 2; optional TClickHouseGatewayConfig ClickHouse = 3; @@ -454,4 +454,4 @@ message TGatewaysConfig { optional TPqGatewayConfig Pq = 16; optional TS3GatewayConfig S3 = 17; optional THttpGatewayConfig HttpGateway = 18; -} +} diff --git a/ydb/library/yql/providers/common/proto/python/ya.make b/ydb/library/yql/providers/common/proto/python/ya.make index c4034b0014e..0fd7873cf7a 100644 --- a/ydb/library/yql/providers/common/proto/python/ya.make +++ b/ydb/library/yql/providers/common/proto/python/ya.make @@ -1,3 +1,3 @@ PY_PROTOS_FOR(ydb/library/yql/providers/common/proto) - + OWNER(g:yql) diff --git a/ydb/library/yql/providers/common/proto/ya.make b/ydb/library/yql/providers/common/proto/ya.make index b26857c7711..5b79b7dd12d 100644 --- a/ydb/library/yql/providers/common/proto/ya.make +++ b/ydb/library/yql/providers/common/proto/ya.make @@ -1,12 +1,12 @@ OWNER(g:yql) - + PROTO_LIBRARY() - -SRCS( - gateways_config.proto + +SRCS( + gateways_config.proto udf_resolver.proto -) - +) + PEERDIR( ydb/library/yql/protos ) @@ -15,4 +15,4 @@ IF (NOT PY_PROTOS_FOR) EXCLUDE_TAGS(GO_PROTO) ENDIF() -END() +END() diff --git a/ydb/library/yql/providers/common/udf_resolve/yql_outproc_udf_resolver.cpp b/ydb/library/yql/providers/common/udf_resolve/yql_outproc_udf_resolver.cpp index 6dc60f022bb..dc31e0ef300 100644 --- a/ydb/library/yql/providers/common/udf_resolve/yql_outproc_udf_resolver.cpp +++ b/ydb/library/yql/providers/common/udf_resolve/yql_outproc_udf_resolver.cpp @@ -23,7 +23,7 @@ #include <util/string/split.h> #include <regex> - + namespace NYql { namespace NCommon { @@ -105,14 +105,14 @@ TString ExtractSharedObjectNameFromErrorMessage(const char* message) { class TOutProcUdfResolver : public IUdfResolver { public: - TOutProcUdfResolver(const NKikimr::NMiniKQL::IFunctionRegistry* functionRegistry, + TOutProcUdfResolver(const NKikimr::NMiniKQL::IFunctionRegistry* functionRegistry, const TFileStoragePtr& fileStorage, const TString& resolverPath, const TString& user, const TString& group, bool filterSyscalls, const TString& udfDependencyStubPath, const TMap<TString, TString>& path2md5) - : FunctionRegistry_(functionRegistry) + : FunctionRegistry_(functionRegistry) , TypeInfoHelper_(new TTypeInfoHelper) , FileStorage_(fileStorage) - , ResolverPath_(resolverPath) + , ResolverPath_(resolverPath) , UdfDependencyStubPath_(udfDependencyStubPath) , Path2Md5_(path2md5) { @@ -198,7 +198,7 @@ public: } for (auto& module : requiredExternalModules) { - if (auto path = FunctionRegistry_->FindUdfPath(module)) { + if (auto path = FunctionRegistry_->FindUdfPath(module)) { auto importRequest = request.AddImports(); const TString hiddenPath = filesBox->MakeLinkFrom(*path); importRequest->SetFileAlias(hiddenPath); @@ -370,7 +370,7 @@ private: } private: - const NKikimr::NMiniKQL::IFunctionRegistry* FunctionRegistry_; + const NKikimr::NMiniKQL::IFunctionRegistry* FunctionRegistry_; NUdf::ITypeInfoHelper::TPtr TypeInfoHelper_; TFileStoragePtr FileStorage_; const TString ResolverPath_; @@ -379,16 +379,16 @@ private: const TMap<TString, TString> Path2Md5_; }; -void LoadSystemModulePaths( +void LoadSystemModulePaths( const TString& resolverPath, const TString& dir, - TUdfModulePathsMap* paths) -{ + TUdfModulePathsMap* paths) +{ const TList<TString> args = { TString("--list"), dir }; RunResolver(resolverPath, args, nullptr, [&](const TString& output) { // output format is: // {{module_name}}\t{{module_path}}\n - + for (const auto& it : StringSplitter(output).Split('\n')) { TStringBuf moduleName, modulePath; const TStringBuf& line = it.Token(); @@ -396,13 +396,13 @@ void LoadSystemModulePaths( line.Split('\t', moduleName, modulePath); paths->emplace(moduleName, modulePath); } - } + } }); -} - +} + IUdfResolver::TPtr CreateOutProcUdfResolver( - const NKikimr::NMiniKQL::IFunctionRegistry* functionRegistry, - const TFileStoragePtr& fileStorage, + const NKikimr::NMiniKQL::IFunctionRegistry* functionRegistry, + const TFileStoragePtr& fileStorage, const TString& resolverPath, const TString& user, const TString& group, diff --git a/ydb/library/yql/providers/common/udf_resolve/yql_outproc_udf_resolver.h b/ydb/library/yql/providers/common/udf_resolve/yql_outproc_udf_resolver.h index 91244d672b8..5659a833e41 100644 --- a/ydb/library/yql/providers/common/udf_resolve/yql_outproc_udf_resolver.h +++ b/ydb/library/yql/providers/common/udf_resolve/yql_outproc_udf_resolver.h @@ -11,14 +11,14 @@ namespace NYql { namespace NCommon { -void LoadSystemModulePaths( +void LoadSystemModulePaths( const TString& resolverPath, const TString& dir, - NKikimr::NMiniKQL::TUdfModulePathsMap* paths); - + NKikimr::NMiniKQL::TUdfModulePathsMap* paths); + IUdfResolver::TPtr CreateOutProcUdfResolver( - const NKikimr::NMiniKQL::IFunctionRegistry* functionRegistry, - const TFileStoragePtr& fileStorage, + const NKikimr::NMiniKQL::IFunctionRegistry* functionRegistry, + const TFileStoragePtr& fileStorage, const TString& resolverPath, const TString& user, const TString& group, diff --git a/ydb/library/yql/providers/common/ya.make b/ydb/library/yql/providers/common/ya.make index 13b1a2e24b6..66778b0bcf8 100644 --- a/ydb/library/yql/providers/common/ya.make +++ b/ydb/library/yql/providers/common/ya.make @@ -7,7 +7,7 @@ RECURSE( metrics mkql comp_nodes - proto + proto provider schema schema/expr diff --git a/ydb/library/yql/providers/config/yql_config_provider.cpp b/ydb/library/yql/providers/config/yql_config_provider.cpp index 59923fcb57d..6952e96cf07 100644 --- a/ydb/library/yql/providers/config/yql_config_provider.cpp +++ b/ydb/library/yql/providers/config/yql_config_provider.cpp @@ -107,9 +107,9 @@ namespace { input->SetState(TExprNode::EState::ExecutionComplete); input->SetResult(ctx.NewWorld(input->Pos())); - return TStatus::Ok; - } - + return TStatus::Ok; + } + ctx.AddError(TIssue(ctx.GetPosition(input->Pos()), TStringBuilder() << "Failed to execute node: " << input->Content())); return TStatus::Error; } @@ -129,7 +129,7 @@ namespace { }; TConfigProvider(TTypeAnnotationContext& types, const TGatewaysConfig* config, const TAllowSettingPolicy& policy) - : Types(types) + : Types(types) , CoreConfig(config && config->HasYqlCore() ? &config->GetYqlCore() : nullptr) , Policy(policy) {} @@ -163,32 +163,32 @@ namespace { return true; } - bool MatchCategory(const TExprNode& node) { + bool MatchCategory(const TExprNode& node) { return (node.Child(1)->Child(0)->Content() == ConfigProviderName); - } - + } + bool CanParse(const TExprNode& node) override { if (ConfigProviderFunctions().contains(node.Content()) || node.Content() == ConfigureName) - { - return MatchCategory(node); - } - - return false; + { + return MatchCategory(node); + } + + return false; } IGraphTransformer& GetConfigurationTransformer() override { if (ConfigurationTransformer) { return *ConfigurationTransformer; - } - + } + ConfigurationTransformer = CreateFunctorTransformer( [this](const TExprNode::TPtr& input, TExprNode::TPtr& output, TExprContext& ctx) -> IGraphTransformer::TStatus { output = input; if (ctx.Step.IsDone(TExprStep::Configure)) { return IGraphTransformer::TStatus::Ok; } - + bool hasPendingEvaluations = false; TOptimizeExprSettings settings(nullptr); settings.VisitChanges = true; @@ -197,12 +197,12 @@ namespace { if (!hasPendingEvaluations && node->Content() == ConfigureName) { if (!EnsureMinArgsCount(*node, 2, ctx)) { return {}; - } - + } + if (!node->Child(1)->IsCallable("DataSource")) { return node; - } - + } + if (node->Child(1)->Child(0)->Content() != ConfigProviderName) { return node; } @@ -237,11 +237,11 @@ namespace { res = ctx.ChangeChild(*node, 3, ctx.RenameNode(*node->Child(3), Types.PureResultDataSource)); } } - } - + } + return res; }, ctx, settings); - + return status; }); @@ -327,13 +327,13 @@ namespace { } else if (input->Content() == ConfigureName) { if (!EnsureWorldType(*input->Child(0), ctx)) { - return IGraphTransformer::TStatus::Error; - } - + return IGraphTransformer::TStatus::Error; + } + input->SetTypeAnn(input->Child(0)->GetTypeAnn()); - return IGraphTransformer::TStatus::Ok; - } - + return IGraphTransformer::TStatus::Ok; + } + ctx.AddError(TIssue(ctx.GetPosition(input->Pos()), TStringBuilder() << "(Config) Unsupported function: " << input->Content())); return IGraphTransformer::TStatus::Error; }); @@ -352,7 +352,7 @@ namespace { YQL_ENSURE(false, "Expected Read!"); } - YQL_CLOG(INFO, ProviderConfig) << "RewriteIO"; + YQL_CLOG(INFO, ProviderConfig) << "RewriteIO"; auto newRead = ctx.RenameNode(*read, newName); auto retChildren = node->ChildrenList(); retChildren[0] = newRead; @@ -375,8 +375,8 @@ namespace { bool CanExecute(const TExprNode& node) override { if (ConfigProviderFunctions().contains(node.Content()) || node.Content() == ConfigureName) - { - return MatchCategory(node); + { + return MatchCategory(node); } return false; diff --git a/ydb/library/yql/providers/config/yql_config_provider.h b/ydb/library/yql/providers/config/yql_config_provider.h index 8785c3706cd..ea1f47508ac 100644 --- a/ydb/library/yql/providers/config/yql_config_provider.h +++ b/ydb/library/yql/providers/config/yql_config_provider.h @@ -1,5 +1,5 @@ #pragma once - + #include <ydb/library/yql/core/yql_type_annotation.h> #include <functional> @@ -17,4 +17,4 @@ TIntrusivePtr<IDataProvider> CreateConfigProvider(TTypeAnnotationContext& types, const THashSet<TStringBuf>& ConfigProviderFunctions(); -} // namespace NYql +} // namespace NYql diff --git a/ydb/library/yql/providers/dq/actors/actor_helpers.h b/ydb/library/yql/providers/dq/actors/actor_helpers.h index 18cf07dbddc..b92603ab0fb 100644 --- a/ydb/library/yql/providers/dq/actors/actor_helpers.h +++ b/ydb/library/yql/providers/dq/actors/actor_helpers.h @@ -63,7 +63,7 @@ private: this->PassAway(); } - TAutoPtr<NActors::IEventHandle> AfterRegister(const NActors::TActorId& self, const NActors::TActorId& parentId) override { + TAutoPtr<NActors::IEventHandle> AfterRegister(const NActors::TActorId& self, const NActors::TActorId& parentId) override { return new NActors::IEventHandle(self, parentId, new NActors::TEvents::TEvBootstrap, 0); } diff --git a/ydb/library/yql/providers/dq/actors/events.cpp b/ydb/library/yql/providers/dq/actors/events.cpp index 6499c29f807..892c5cccd70 100644 --- a/ydb/library/yql/providers/dq/actors/events.cpp +++ b/ydb/library/yql/providers/dq/actors/events.cpp @@ -21,14 +21,14 @@ namespace NYql::NDqs { { *Record.MutableRequest() = request; NActors::ActorIdToProto(controlId, Record.MutableControlId()); - NActors::ActorIdToProto(resultId, Record.MutableResultId()); + NActors::ActorIdToProto(resultId, Record.MutableResultId()); if (checkPointCoordinatorId) { NActors::ActorIdToProto(checkPointCoordinatorId, Record.MutableCheckPointCoordinatorId()); } } - TEvReadyState::TEvReadyState(NActors::TActorId sourceId, TString type) { - NActors::ActorIdToProto(sourceId, Record.MutableSourceId()); + TEvReadyState::TEvReadyState(NActors::TActorId sourceId, TString type) { + NActors::ActorIdToProto(sourceId, Record.MutableSourceId()); *Record.MutableResultType() = std::move(type); } diff --git a/ydb/library/yql/providers/dq/actors/events.h b/ydb/library/yql/providers/dq/actors/events.h index 70d91ab8843..594921560ae 100644 --- a/ydb/library/yql/providers/dq/actors/events.h +++ b/ydb/library/yql/providers/dq/actors/events.h @@ -37,7 +37,7 @@ namespace NYql::NDqs { struct TEvReadyState : NActors::TEventPB<TEvReadyState, NDqProto::TReadyState, TDqExecuterEvents::ES_READY_TO_PULL> { TEvReadyState() = default; - TEvReadyState(NActors::TActorId sourceId, TString type); + TEvReadyState(NActors::TActorId sourceId, TString type); explicit TEvReadyState(NDqProto::TReadyState&& proto); }; diff --git a/ydb/library/yql/providers/dq/actors/executer_actor.cpp b/ydb/library/yql/providers/dq/actors/executer_actor.cpp index b4706838caa..91f1caee748 100644 --- a/ydb/library/yql/providers/dq/actors/executer_actor.cpp +++ b/ydb/library/yql/providers/dq/actors/executer_actor.cpp @@ -133,7 +133,7 @@ private: Y_VERIFY(!ResultId); YQL_LOG(DEBUG) << "TDqExecuter::OnGraph"; ControlId = NActors::ActorIdFromProto(ev->Get()->Record.GetControlId()); - ResultId = NActors::ActorIdFromProto(ev->Get()->Record.GetResultId()); + ResultId = NActors::ActorIdFromProto(ev->Get()->Record.GetResultId()); CheckPointCoordinatorId = NActors::ActorIdFromProto(ev->Get()->Record.GetCheckPointCoordinatorId()); // These actors will be killed at exit. AddChild(ControlId); @@ -324,9 +324,9 @@ private: auto& workerGroup = response.GetWorkers(); ResourceId = workerGroup.GetResourceId(); YQL_LOG(DEBUG) << "Allocated resource " << ResourceId; - TVector<NActors::TActorId> workers; + TVector<NActors::TActorId> workers; for (const auto& actorIdProto : workerGroup.GetWorkerActor()) { - workers.emplace_back(NActors::ActorIdFromProto(actorIdProto)); + workers.emplace_back(NActors::ActorIdFromProto(actorIdProto)); } auto tasks = ExecutionPlanner->GetTasks(workers); @@ -447,7 +447,7 @@ private: TDqConfiguration::TPtr Settings; NActors::TActorId ControlId; - NActors::TActorId ResultId; + NActors::TActorId ResultId; NActors::TActorId CheckPointCoordinatorId; TExprNode::TPtr ExprRoot; THolder<IDqsExecutionPlanner> ExecutionPlanner; diff --git a/ydb/library/yql/providers/dq/actors/resource_allocator.cpp b/ydb/library/yql/providers/dq/actors/resource_allocator.cpp index e98a406f4ea..6acfb1277e0 100644 --- a/ydb/library/yql/providers/dq/actors/resource_allocator.cpp +++ b/ydb/library/yql/providers/dq/actors/resource_allocator.cpp @@ -188,10 +188,10 @@ private: } if (AllocatedCount == RequestedCount) { - TVector<NActors::TActorId> workerIds; + TVector<NActors::TActorId> workerIds; for (auto& group : AllocatedWorkers) { for (const auto& actorIdProto : group.GetWorkerActor()) { - workerIds.emplace_back(NActors::ActorIdFromProto(actorIdProto)); + workerIds.emplace_back(NActors::ActorIdFromProto(actorIdProto)); } } @@ -217,7 +217,7 @@ private: { for (const auto& group : AllocatedWorkers) { for (const auto& actorIdProto : group.GetWorkerActor()) { - auto actorNode = NActors::ActorIdFromProto(actorIdProto).NodeId(); + auto actorNode = NActors::ActorIdFromProto(actorIdProto).NodeId(); auto request = MakeHolder<TEvFreeWorkersNotify>(group.GetResourceId()); request->Record.SetTraceId(TraceId); Send(MakeWorkerManagerActorID(actorNode), request.Release()); @@ -297,7 +297,7 @@ private: } const TActorId GwmActor; - const TActorId SenderId; + const TActorId SenderId; const TActorId ControlId; THashMap<ui64, TRequestInfo> RequestedNodes; diff --git a/ydb/library/yql/providers/dq/actors/resource_allocator.h b/ydb/library/yql/providers/dq/actors/resource_allocator.h index 2af39cac787..ab5db2df0e7 100644 --- a/ydb/library/yql/providers/dq/actors/resource_allocator.h +++ b/ydb/library/yql/providers/dq/actors/resource_allocator.h @@ -9,7 +9,7 @@ namespace NYql { NActors::IActor* CreateResourceAllocator( NActors::TActorId gwmActor, - NActors::TActorId senderId, + NActors::TActorId senderId, NActors::TActorId resultId, ui32 workerCount, const TString& traceId, diff --git a/ydb/library/yql/providers/dq/actors/worker_actor.cpp b/ydb/library/yql/providers/dq/actors/worker_actor.cpp index f3d554eccac..f4503a60b2e 100644 --- a/ydb/library/yql/providers/dq/actors/worker_actor.cpp +++ b/ydb/library/yql/providers/dq/actors/worker_actor.cpp @@ -524,7 +524,7 @@ private: NActors::TActorId ResolveEndpoint(const TEndpoint& ep) { switch (ep.GetEndpointTypeCase()) { case TEndpoint::kActorId: - return NActors::ActorIdFromProto(ep.GetActorId()); + return NActors::ActorIdFromProto(ep.GetActorId()); case TEndpoint::kUri: Y_ENSURE(false, "kUri is not supported"); case TEndpoint::kTabletId: diff --git a/ydb/library/yql/providers/dq/api/protos/dqs.proto b/ydb/library/yql/providers/dq/api/protos/dqs.proto index f2dc35830a9..09aecfcf923 100644 --- a/ydb/library/yql/providers/dq/api/protos/dqs.proto +++ b/ydb/library/yql/providers/dq/api/protos/dqs.proto @@ -40,7 +40,7 @@ message TAllocateWorkersRequest { message TWorkerGroup { uint64 ResourceId = 1; - repeated NActorsProto.TActorId WorkerActor = 2; + repeated NActorsProto.TActorId WorkerActor = 2; repeated Yql.DqsProto.TWorkerInfo Worker = 3; } @@ -198,7 +198,7 @@ message TDqTaskRequest { } message TReadyState { - NActorsProto.TActorId SourceId = 1; + NActorsProto.TActorId SourceId = 1; string ResultType = 2; repeated TMetric Metric = 3; repeated NYql.NDqProto.TDqTask Task = 4; // used for compute actor diff --git a/ydb/library/yql/providers/dq/planner/execution_planner.cpp b/ydb/library/yql/providers/dq/planner/execution_planner.cpp index e90ec696587..ec404c0a6dc 100644 --- a/ydb/library/yql/providers/dq/planner/execution_planner.cpp +++ b/ydb/library/yql/providers/dq/planner/execution_planner.cpp @@ -86,7 +86,7 @@ namespace NYql::NDqs { const NKikimr::NMiniKQL::IFunctionRegistry* functionRegistry, NYql::TExprNode::TPtr dqExprRoot, NActors::TActorId executerID, - NActors::TActorId resultID) + NActors::TActorId resultID) : TypeContext(std::move(typeContext)) , ExprContext(exprContext) , FunctionRegistry(functionRegistry) @@ -303,9 +303,9 @@ namespace NYql::NDqs { TVector<TDqTask>& TDqsExecutionPlanner::GetTasks() { if (Tasks.empty()) { auto workerCount = TasksGraph.GetTasks().size(); - TVector<NActors::TActorId> workers; + TVector<NActors::TActorId> workers; for (unsigned int i = 0; i < workerCount; ++i) { - NActors::TActorId fakeActorId(i+1, 0, 0, 0); + NActors::TActorId fakeActorId(i+1, 0, 0, 0); workers.emplace_back(fakeActorId); } Tasks = GetTasks(workers); @@ -333,7 +333,7 @@ namespace NYql::NDqs { Yql::DqsProto::TTaskMeta taskMeta; TDqTask taskDesc; taskDesc.SetId(task.Id); - NActors::ActorIdToProto(ExecuterID, taskDesc.MutableExecuter()->MutableActorId()); + NActors::ActorIdToProto(ExecuterID, taskDesc.MutableExecuter()->MutableActorId()); auto& taskParams = *taskMeta.MutableTaskParams(); for (const auto& [k, v]: task.Meta.TaskParams) { taskParams[k] = v; @@ -379,7 +379,7 @@ namespace NYql::NDqs { return plan; } - NActors::TActorId TDqsExecutionPlanner::GetSourceID() const { + NActors::TActorId TDqsExecutionPlanner::GetSourceID() const { if (SourceID) { return *SourceID; } else { @@ -567,18 +567,18 @@ namespace NYql::NDqs { channelDesc.SetCheckpointingMode(channel.CheckpointingMode); if (channel.SrcTask) { - NActors::ActorIdToProto(TasksGraph.GetTask(channel.SrcTask).ComputeActorId, + NActors::ActorIdToProto(TasksGraph.GetTask(channel.SrcTask).ComputeActorId, channelDesc.MutableSrcEndpoint()->MutableActorId()); } if (channel.DstTask) { - NActors::ActorIdToProto(TasksGraph.GetTask(channel.DstTask).ComputeActorId, + NActors::ActorIdToProto(TasksGraph.GetTask(channel.DstTask).ComputeActorId, channelDesc.MutableDstEndpoint()->MutableActorId()); } else { auto& stageInfo = TasksGraph.GetStageInfo(TasksGraph.GetTask(channel.SrcTask).StageId); YQL_ENSURE(stageInfo.Tasks.size() == 1); YQL_ENSURE(!SourceID); - ActorIdToProto(ResultID, channelDesc.MutableDstEndpoint()->MutableActorId()); + ActorIdToProto(ResultID, channelDesc.MutableDstEndpoint()->MutableActorId()); SourceID = TasksGraph.GetTask(channel.SrcTask).ComputeActorId; } } @@ -689,7 +689,7 @@ namespace NYql::NDqs { task.SetStageId(1); task.MutableMeta()->PackFrom(taskMeta); - NActors::ActorIdToProto(ExecuterID, task.MutableExecuter()->MutableActorId()); + NActors::ActorIdToProto(ExecuterID, task.MutableExecuter()->MutableActorId()); auto& program = *task.MutableProgram(); program.SetRuntimeVersion(NYql::NDqProto::ERuntimeVersion::RUNTIME_VERSION_YQL_1_0); program.SetRaw(Program); @@ -702,15 +702,15 @@ namespace NYql::NDqs { channelDesc->SetSrcTaskId(2); channelDesc->SetDstTaskId(1); - NActors::ActorIdToProto(ExecuterID, channelDesc->MutableSrcEndpoint()->MutableActorId()); - NActors::ActorIdToProto(ResultID, channelDesc->MutableDstEndpoint()->MutableActorId()); + NActors::ActorIdToProto(ExecuterID, channelDesc->MutableSrcEndpoint()->MutableActorId()); + NActors::ActorIdToProto(ResultID, channelDesc->MutableDstEndpoint()->MutableActorId()); SourceID = worker; return {task}; } - NActors::TActorId TDqsSingleExecutionPlanner::GetSourceID() const + NActors::TActorId TDqsSingleExecutionPlanner::GetSourceID() const { if (SourceID) { return *SourceID; @@ -742,8 +742,8 @@ namespace NYql::NDqs { const TVector<TDqTask>& tasks, ui64 sourceId, const TString& resultType, - NActors::TActorId executerID, - NActors::TActorId resultID) + NActors::TActorId executerID, + NActors::TActorId resultID) : Tasks(tasks) , SourceId(sourceId) , ResultType(resultType) @@ -761,18 +761,18 @@ namespace NYql::NDqs { auto setActorId = [&](NYql::NDqProto::TEndpoint* endpoint) { if (endpoint->GetEndpointTypeCase() == NYql::NDqProto::TEndpoint::kActorId) { - NActors::TActorId fakeId = NActors::ActorIdFromProto(endpoint->GetActorId()); + NActors::TActorId fakeId = NActors::ActorIdFromProto(endpoint->GetActorId()); if (fakeId.NodeId() > 0) { - NActors::TActorId realId = fakeId.LocalId() == 0 + NActors::TActorId realId = fakeId.LocalId() == 0 ? workers[fakeId.NodeId()-1] : ResultID; - NActors::ActorIdToProto(realId, endpoint->MutableActorId()); + NActors::ActorIdToProto(realId, endpoint->MutableActorId()); } } }; for (auto& taskDesc : Tasks) { - NActors::ActorIdToProto(ExecuterID, taskDesc.MutableExecuter()->MutableActorId()); + NActors::ActorIdToProto(ExecuterID, taskDesc.MutableExecuter()->MutableActorId()); for (auto& inputDesc : *taskDesc.MutableInputs()) { for (auto& channelDesc : *inputDesc.MutableChannels()) { @@ -792,7 +792,7 @@ namespace NYql::NDqs { return Tasks; } - NActors::TActorId TGraphExecutionPlanner::GetSourceID() const + NActors::TActorId TGraphExecutionPlanner::GetSourceID() const { if (SourceID) { return *SourceID; diff --git a/ydb/library/yql/providers/dq/planner/execution_planner.h b/ydb/library/yql/providers/dq/planner/execution_planner.h index 534c11d26a1..280600c8f7b 100644 --- a/ydb/library/yql/providers/dq/planner/execution_planner.h +++ b/ydb/library/yql/providers/dq/planner/execution_planner.h @@ -19,7 +19,7 @@ namespace NYql::NDqs { virtual ~IDqsExecutionPlanner() = default; virtual TVector<NDqProto::TDqTask> GetTasks(const TVector<NActors::TActorId>& workers) = 0; virtual TVector<NDqProto::TDqTask>& GetTasks() = 0; - virtual NActors::TActorId GetSourceID() const = 0; + virtual NActors::TActorId GetSourceID() const = 0; virtual TString GetResultType(bool withTagged = false) const = 0; }; @@ -29,8 +29,8 @@ namespace NYql::NDqs { NYql::TExprContext& exprContext, const NKikimr::NMiniKQL::IFunctionRegistry* functionRegistry, NYql::TExprNode::TPtr dqExprRoot, - NActors::TActorId executerID = NActors::TActorId(), - NActors::TActorId resultID = NActors::TActorId(1, 0, 1, 0)); + NActors::TActorId executerID = NActors::TActorId(), + NActors::TActorId resultID = NActors::TActorId(1, 0, 1, 0)); void Clear(); bool CanFallback(); @@ -42,7 +42,7 @@ namespace NYql::NDqs { TVector<NDqProto::TDqTask> GetTasks(const TVector<NActors::TActorId>& workers) override; TVector<NDqProto::TDqTask>& GetTasks() override; - NActors::TActorId GetSourceID() const override; + NActors::TActorId GetSourceID() const override; TString GetResultType(bool withTagged = false) const override; void SetPublicIds(const THashMap<ui64, ui32>& publicIds) { @@ -67,9 +67,9 @@ namespace NYql::NDqs { const NKikimr::NMiniKQL::IFunctionRegistry* FunctionRegistry; NYql::TExprNode::TPtr DqExprRoot; TVector<const TTypeAnnotationNode*> InputType; - NActors::TActorId ExecuterID; - NActors::TActorId ResultID; - TMaybe<NActors::TActorId> SourceID = {}; + NActors::TActorId ExecuterID; + NActors::TActorId ResultID; + TMaybe<NActors::TActorId> SourceID = {}; ui64 SourceTaskID = 0; ui64 _MaxDataSizePerJob = 0; @@ -91,15 +91,15 @@ namespace NYql::NDqs { TVector<NDqProto::TDqTask>& GetTasks() override; TVector<NDqProto::TDqTask> GetTasks(const TVector<NActors::TActorId>& workers) override; - NActors::TActorId GetSourceID() const override; + NActors::TActorId GetSourceID() const override; TString GetResultType(bool withTagged = false) const override; private: TString Program; - NActors::TActorId ExecuterID; - NActors::TActorId ResultID; + NActors::TActorId ExecuterID; + NActors::TActorId ResultID; - TMaybe<NActors::TActorId> SourceID = {}; + TMaybe<NActors::TActorId> SourceID = {}; TVector<NDqProto::TDqTask> Tasks; const NKikimr::NMiniKQL::IFunctionRegistry* FunctionRegistry; const TTypeAnnotationNode* TypeAnn; @@ -112,14 +112,14 @@ namespace NYql::NDqs { const TVector<NDqProto::TDqTask>& tasks, ui64 sourceId, const TString& resultType, - NActors::TActorId executerID, - NActors::TActorId resultID); + NActors::TActorId executerID, + NActors::TActorId resultID); TVector<NDqProto::TDqTask>& GetTasks() override { ythrow yexception() << "unimplemented"; } TVector<NDqProto::TDqTask> GetTasks(const TVector<NActors::TActorId>& workers) override; - NActors::TActorId GetSourceID() const override; + NActors::TActorId GetSourceID() const override; TString GetResultType(bool withTagged = false) const override; private: @@ -127,9 +127,9 @@ namespace NYql::NDqs { ui64 SourceId = 0; TString ResultType; - NActors::TActorId ExecuterID; - NActors::TActorId ResultID; + NActors::TActorId ExecuterID; + NActors::TActorId ResultID; - TMaybe<NActors::TActorId> SourceID = {}; + TMaybe<NActors::TActorId> SourceID = {}; }; } diff --git a/ydb/library/yql/providers/dq/service/grpc_service.cpp b/ydb/library/yql/providers/dq/service/grpc_service.cpp index d0fe6dcc774..65a9e500d9d 100644 --- a/ydb/library/yql/providers/dq/service/grpc_service.cpp +++ b/ydb/library/yql/providers/dq/service/grpc_service.cpp @@ -21,7 +21,7 @@ #include <ydb/library/yql/minikql/invoke_builtins/mkql_builtins.h> -#include <library/cpp/grpc/server/grpc_counters.h> +#include <library/cpp/grpc/server/grpc_counters.h> #include <ydb/public/api/protos/ydb_status_codes.pb.h> #include <library/cpp/actors/interconnect/interconnect.h> @@ -40,9 +40,9 @@ namespace NYql::NDqs { using namespace NActors; namespace { - NGrpc::ICounterBlockPtr BuildCB(TIntrusivePtr<NMonitoring::TDynamicCounters>& counters, const TString& name) { + NGrpc::ICounterBlockPtr BuildCB(TIntrusivePtr<NMonitoring::TDynamicCounters>& counters, const TString& name) { auto grpcCB = counters->GetSubgroup("rpc_name", name); - return MakeIntrusive<NGrpc::TCounterBlock>( + return MakeIntrusive<NGrpc::TCounterBlock>( grpcCB->GetCounter("total", true), grpcCB->GetCounter("infly", true), grpcCB->GetCounter("notOkReq", true), @@ -61,7 +61,7 @@ namespace NYql::NDqs { static constexpr char ActorName[] = "SERVICE_PROXY"; explicit TServiceProxyActor( - NGrpc::IRequestContextBase* ctx, + NGrpc::IRequestContextBase* ctx, const TIntrusivePtr<NMonitoring::TDynamicCounters>& counters, const TString& traceId, const TString& username) : TSynchronizableRichActor<TServiceProxyActor<RequestType, ResponseType>>(&TServiceProxyActor::Handler) @@ -90,7 +90,7 @@ namespace NYql::NDqs { SFunc(TEvents::TEvBootstrap, DoBootstrap) }) - TAutoPtr<IEventHandle> AfterRegister(const TActorId& self, const TActorId& parentId) override { + TAutoPtr<IEventHandle> AfterRegister(const TActorId& self, const TActorId& parentId) override { return new IEventHandle(self, parentId, new TEvents::TEvBootstrap(), 0); } @@ -110,9 +110,9 @@ namespace NYql::NDqs { if (!CtxSubscribed) { auto selfId = ctx.SelfID; auto* actorSystem = ctx.ExecutorThread.ActorSystem; - Ctx->GetFinishFuture().Subscribe([selfId, actorSystem](const NGrpc::IRequestContextBase::TAsyncFinishResult& future) { + Ctx->GetFinishFuture().Subscribe([selfId, actorSystem](const NGrpc::IRequestContextBase::TAsyncFinishResult& future) { Y_VERIFY(future.HasValue()); - if (future.GetValue() == NGrpc::IRequestContextBase::EFinishStatus::CANCEL) { + if (future.GetValue() == NGrpc::IRequestContextBase::EFinishStatus::CANCEL) { actorSystem->Send(selfId, new TEvents::TEvPoison()); } }); @@ -219,7 +219,7 @@ namespace NYql::NDqs { } private: - NGrpc::IRequestContextBase* Ctx; + NGrpc::IRequestContextBase* Ctx; bool CtxSubscribed = false; ResponseType ResponseBuffer; @@ -254,7 +254,7 @@ namespace NYql::NDqs { class TExecuteGraphProxyActor: public TServiceProxyActor<Yql::DqsProto::ExecuteGraphRequest, Yql::DqsProto::ExecuteGraphResponse> { public: using TBase = TServiceProxyActor<Yql::DqsProto::ExecuteGraphRequest, Yql::DqsProto::ExecuteGraphResponse>; - TExecuteGraphProxyActor(NGrpc::IRequestContextBase* ctx, + TExecuteGraphProxyActor(NGrpc::IRequestContextBase* ctx, const TIntrusivePtr<NMonitoring::TDynamicCounters>& counters, const TString& traceId, const TString& username, const NActors::TActorId& graphExecutionEventsActorId) @@ -454,7 +454,7 @@ namespace NYql::NDqs { ->Run(); \ } while (0) - void TDqsGrpcService::InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) { + void TDqsGrpcService::InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) { using namespace google::protobuf; CQ = cq; @@ -603,7 +603,7 @@ namespace NYql::NDqs { }, TDuration::MilliSeconds(2000)); - TActorId callbackId = ActorSystem.Register(callback.Release()); + TActorId callbackId = ActorSystem.Register(callback.Release()); ActorSystem.Send(new IEventHandle(MakeWorkerManagerActorID(ActorSystem.NodeId), callbackId, ev.Release(), IEventHandle::FlagTrackDelivery)); }); @@ -680,7 +680,7 @@ namespace NYql::NDqs { }, TDuration::MilliSeconds(5000)); - TActorId callbackId = ActorSystem.Register(callback.Release()); + TActorId callbackId = ActorSystem.Register(callback.Release()); ActorSystem.Send(new IEventHandle(MakeWorkerManagerActorID(ActorSystem.NodeId), callbackId, ev.Release(), IEventHandle::FlagTrackDelivery)); }); @@ -698,7 +698,7 @@ namespace NYql::NDqs { ctx->Reply(result, Ydb::StatusIds::SUCCESS); }); - TActorId callbackId = ActorSystem.Register(callback.Release()); + TActorId callbackId = ActorSystem.Register(callback.Release()); ActorSystem.Send(new IEventHandle(MakeWorkerManagerActorID(ActorSystem.NodeId), callbackId, requestEvent.Release())); }); @@ -716,7 +716,7 @@ namespace NYql::NDqs { ctx->Reply(result, Ydb::StatusIds::SUCCESS); }); - TActorId callbackId = ActorSystem.Register(callback.Release()); + TActorId callbackId = ActorSystem.Register(callback.Release()); ActorSystem.Send(new IEventHandle(MakeWorkerManagerActorID(ActorSystem.NodeId), callbackId, requestEvent.Release())); }); @@ -799,7 +799,7 @@ namespace NYql::NDqs { */ } - void TDqsGrpcService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { + void TDqsGrpcService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { Limiter = limiter; } diff --git a/ydb/library/yql/providers/dq/service/grpc_service.h b/ydb/library/yql/providers/dq/service/grpc_service.h index ccaf4373637..fa2a835c8c6 100644 --- a/ydb/library/yql/providers/dq/service/grpc_service.h +++ b/ydb/library/yql/providers/dq/service/grpc_service.h @@ -7,13 +7,13 @@ #include <ydb/library/yql/minikql/mkql_function_registry.h> -#include <library/cpp/grpc/server/grpc_request.h> -#include <library/cpp/grpc/server/grpc_server.h> +#include <library/cpp/grpc/server/grpc_request.h> +#include <library/cpp/grpc/server/grpc_server.h> #include <library/cpp/actors/core/actorsystem.h> #include <library/cpp/actors/core/event_local.h> #include <library/cpp/actors/core/events.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <library/cpp/threading/future/future.h> #include "grpc_session.h" @@ -21,14 +21,14 @@ namespace NYql::NDqs { class TDatabaseManager; - class TDqsGrpcService: public NGrpc::TGrpcServiceBase<Yql::DqsProto::DqService> { + class TDqsGrpcService: public NGrpc::TGrpcServiceBase<Yql::DqsProto::DqService> { public: TDqsGrpcService(NActors::TActorSystem& system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, const TDqTaskPreprocessorFactoryCollection& dqTaskPreprocessorFactories); - void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; - void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; + void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; + void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; bool IncRequest(); void DecRequest(); @@ -38,7 +38,7 @@ namespace NYql::NDqs { private: NActors::TActorSystem& ActorSystem; grpc::ServerCompletionQueue* CQ = nullptr; - NGrpc::TGlobalLimiter* Limiter = nullptr; + NGrpc::TGlobalLimiter* Limiter = nullptr; TIntrusivePtr<NMonitoring::TDynamicCounters> Counters; TDqTaskPreprocessorFactoryCollection DqTaskPreprocessorFactories; diff --git a/ydb/library/yql/providers/dq/service/interconnect_helpers.cpp b/ydb/library/yql/providers/dq/service/interconnect_helpers.cpp index 43c1b5fde94..bef368a961f 100644 --- a/ydb/library/yql/providers/dq/service/interconnect_helpers.cpp +++ b/ydb/library/yql/providers/dq/service/interconnect_helpers.cpp @@ -28,7 +28,7 @@ namespace NYql::NDqs { using namespace NActors; using namespace NActors::NDnsResolver; - using namespace NGrpc; + using namespace NGrpc; class TYqlLogBackend: public TLogBackend { void WriteData(const TLogRecord& rec) override { @@ -156,7 +156,7 @@ namespace NYql::NDqs { InitSelfPingActor(setup.Get(), counters); TIntrusivePtr<TInterconnectProxyCommon> icCommon = new TInterconnectProxyCommon(); - icCommon->NameserviceId = GetNameserviceActorId(); + icCommon->NameserviceId = GetNameserviceActorId(); Y_UNUSED(counters); //icCommon->MonCounters = counters->GetSubgroup("counters", "interconnect"); icCommon->MonCounters = MakeIntrusive<NMonitoring::TDynamicCounters>(); diff --git a/ydb/library/yql/providers/dq/service/service_node.cpp b/ydb/library/yql/providers/dq/service/service_node.cpp index f2d4b9fc349..cda24ec6cf7 100644 --- a/ydb/library/yql/providers/dq/service/service_node.cpp +++ b/ydb/library/yql/providers/dq/service/service_node.cpp @@ -6,13 +6,13 @@ #include <ydb/library/yql/providers/dq/actors/execution_helpers.h> -#include <library/cpp/grpc/server/actors/logger.h> - +#include <library/cpp/grpc/server/actors/logger.h> + #include <utility> namespace NYql { using namespace NActors; - using namespace NGrpc; + using namespace NGrpc; using namespace NYql::NDqs; class TGrpcExternalListener: public IExternalListener { @@ -95,7 +95,7 @@ namespace NYql { Config.ICSettings); } - void TServiceNode::AddLocalService(TActorId actorId, const TActorSetupCmd& service) { + void TServiceNode::AddLocalService(TActorId actorId, const TActorSetupCmd& service) { YQL_ENSURE(!ActorSystem); Setup->LocalServices.emplace_back(actorId, service); } @@ -145,8 +145,8 @@ namespace NYql { .SetKeepAliveProbeIntervalSec(1) .SetServerBuilderMutator([](grpc::ServerBuilder& builder) { builder.SetOption(std::make_unique<TCustomOption>()); - }) - .SetLogger(CreateActorSystemLogger(*ActorSystem, 413)); // 413 - NKikimrServices::GRPC_SERVER + }) + .SetLogger(CreateActorSystemLogger(*ActorSystem, 413)); // 413 - NKikimrServices::GRPC_SERVER Server = MakeHolder<TGRpcServer>(options); Service = TIntrusivePtr<IGRpcService>(new TDqsGrpcService(*ActorSystem, MetricsRegistry->GetSensors(), dqTaskPreprocessorFactories)); diff --git a/ydb/library/yql/providers/dq/service/service_node.h b/ydb/library/yql/providers/dq/service/service_node.h index a86f65ab46c..32f904901dc 100644 --- a/ydb/library/yql/providers/dq/service/service_node.h +++ b/ydb/library/yql/providers/dq/service/service_node.h @@ -24,7 +24,7 @@ namespace NYql { ui32 threads, IMetricsRegistryPtr metricsRegistry); - void AddLocalService(NActors::TActorId actorId, const NActors::TActorSetupCmd& service); + void AddLocalService(NActors::TActorId actorId, const NActors::TActorSetupCmd& service); NActors::TActorSystem* StartActorSystem(void* appData = nullptr); void StartService(const TDqTaskPreprocessorFactoryCollection& dqTaskPreprocessorFactories); @@ -41,8 +41,8 @@ namespace NYql { THolder<NActors::TActorSystemSetup> Setup; TIntrusivePtr<NActors::NLog::TSettings> LogSettings; THolder<NActors::TActorSystem> ActorSystem; - TVector<NActors::TActorId> ActorIds; - THolder<NGrpc::TGRpcServer> Server; - TIntrusivePtr<NGrpc::IGRpcService> Service; + TVector<NActors::TActorId> ActorIds; + THolder<NGrpc::TGRpcServer> Server; + TIntrusivePtr<NGrpc::IGRpcService> Service; }; } diff --git a/ydb/library/yql/providers/dq/worker_manager/interface/events.cpp b/ydb/library/yql/providers/dq/worker_manager/interface/events.cpp index ffde459b327..9044dc47e30 100644 --- a/ydb/library/yql/providers/dq/worker_manager/interface/events.cpp +++ b/ydb/library/yql/providers/dq/worker_manager/interface/events.cpp @@ -28,11 +28,11 @@ namespace NYql::NDqs { Record.MutableError()->SetErrorCode(code); } - TEvAllocateWorkersResponse::TEvAllocateWorkersResponse(ui64 resourceId, const TVector<NActors::TActorId>& ids) { + TEvAllocateWorkersResponse::TEvAllocateWorkersResponse(ui64 resourceId, const TVector<NActors::TActorId>& ids) { auto& group = *Record.MutableWorkers(); group.SetResourceId(resourceId); for (const auto& actorId : ids) { - NActors::ActorIdToProto(actorId, group.AddWorkerActor()); + NActors::ActorIdToProto(actorId, group.AddWorkerActor()); } } diff --git a/ydb/library/yql/providers/dq/worker_manager/interface/events.h b/ydb/library/yql/providers/dq/worker_manager/interface/events.h index 47f2dd54868..a2d0c9f5409 100644 --- a/ydb/library/yql/providers/dq/worker_manager/interface/events.h +++ b/ydb/library/yql/providers/dq/worker_manager/interface/events.h @@ -32,7 +32,7 @@ using TDqResManEvents = NDq::TBaseDqResManEvents<NActors::TEvents::EEventSpace:: TDqResManEvents::ES_ALLOCATE_WORKERS_RESPONSE> { TEvAllocateWorkersResponse(); explicit TEvAllocateWorkersResponse(const TString& error, NYql::NDqProto::EErrorCode code = NYql::NDqProto::EUNKNOWN); - explicit TEvAllocateWorkersResponse(ui64 resourceId, const TVector<NActors::TActorId>& ids); + explicit TEvAllocateWorkersResponse(ui64 resourceId, const TVector<NActors::TActorId>& ids); explicit TEvAllocateWorkersResponse(ui64 resourceId, const TVector<TWorkerInfo::TPtr>& workerInfos); explicit TEvAllocateWorkersResponse(ui64 resourceId, const TVector<ui32>& nodes); }; @@ -152,9 +152,9 @@ using TDqResManEvents = NDq::TBaseDqResManEvents<NActors::TEvents::EEventSpace:: TEvConfigureFailureInjectorResponse() = default; }; - inline NActors::TActorId MakeWorkerManagerActorID(ui32 nodeId) { + inline NActors::TActorId MakeWorkerManagerActorID(ui32 nodeId) { char x[12] = {'r', 'e', 's', 'm', 'a', 'n'}; memcpy(x + 7, &nodeId, sizeof(ui32)); - return NActors::TActorId(nodeId, TStringBuf(x, 12)); + return NActors::TActorId(nodeId, TStringBuf(x, 12)); } } diff --git a/ydb/library/yql/providers/dq/worker_manager/interface/worker_info.h b/ydb/library/yql/providers/dq/worker_manager/interface/worker_info.h index 9f7c336d556..5281a844f4c 100644 --- a/ydb/library/yql/providers/dq/worker_manager/interface/worker_info.h +++ b/ydb/library/yql/providers/dq/worker_manager/interface/worker_info.h @@ -4,7 +4,7 @@ #include <util/generic/hash.h> #include <util/generic/guid.h> #include <util/generic/hash_set.h> -#include <util/generic/maybe.h> +#include <util/generic/maybe.h> #include <ydb/library/yql/providers/common/metrics/sensors_group.h> #include <ydb/library/yql/providers/dq/api/grpc/api.grpc.pb.h> diff --git a/ydb/library/yql/providers/dq/worker_manager/local_worker_manager.cpp b/ydb/library/yql/providers/dq/worker_manager/local_worker_manager.cpp index f030e82e2ab..5ecf0a4a852 100644 --- a/ydb/library/yql/providers/dq/worker_manager/local_worker_manager.cpp +++ b/ydb/library/yql/providers/dq/worker_manager/local_worker_manager.cpp @@ -81,7 +81,7 @@ private: hFunc(TEvQueryStatus, OnQueryStatus) }) - TAutoPtr<IEventHandle> AfterRegister(const TActorId& self, const TActorId& parentId) override { + TAutoPtr<IEventHandle> AfterRegister(const TActorId& self, const TActorId& parentId) override { return new IEventHandle(self, parentId, new TEvents::TEvBootstrap(), 0); } diff --git a/ydb/library/yql/providers/result/provider/yql_result_provider.cpp b/ydb/library/yql/providers/result/provider/yql_result_provider.cpp index a1df25d2c6d..13bfd5e0f74 100644 --- a/ydb/library/yql/providers/result/provider/yql_result_provider.cpp +++ b/ydb/library/yql/providers/result/provider/yql_result_provider.cpp @@ -1335,7 +1335,7 @@ namespace { YQL_ENSURE(false, "Expected Write!"); } - YQL_CLOG(INFO, ProviderResult) << "RewriteIO"; + YQL_CLOG(INFO, ProviderResult) << "RewriteIO"; return ret; } @@ -1470,7 +1470,7 @@ TIntrusivePtr<IDataProvider> CreateResultProvider(const TIntrusivePtr<TResultPro } const THashSet<TStringBuf>& ResultProviderFunctions() { - return Singleton<TResultProvider::TFunctions>()->Names; + return Singleton<TResultProvider::TFunctions>()->Names; +} + } - -} diff --git a/ydb/library/yql/public/decimal/ut/yql_decimal_ut.cpp b/ydb/library/yql/public/decimal/ut/yql_decimal_ut.cpp index 4fa9cfdbd23..9c263c2b0b8 100644 --- a/ydb/library/yql/public/decimal/ut/yql_decimal_ut.cpp +++ b/ydb/library/yql/public/decimal/ut/yql_decimal_ut.cpp @@ -244,20 +244,20 @@ Y_UNIT_TEST_SUITE(TYqlDecimalTest) { UNIT_ASSERT(IsInf(FromString("+INf", MaxPrecision, 6))); UNIT_ASSERT(IsInf(FromString("-inF", 4, 2))); } - - Y_UNIT_TEST(TestToStringOfNonNormal) { - // above Inf - for (TInt128 i = Inf() + 2, end = Inf() + 100; i < end; i++) { - UNIT_ASSERT(!IsNormal(i)); - UNIT_ASSERT(ToString(i, MaxPrecision, 0) == nullptr); - } - - // below -Inf - for (TInt128 i = -Inf() - 2, end = -Inf() - 100; i < end; i--) { - UNIT_ASSERT(!IsNormal(i)); - UNIT_ASSERT(ToString(i, MaxPrecision, 0) == nullptr); - } - } + + Y_UNIT_TEST(TestToStringOfNonNormal) { + // above Inf + for (TInt128 i = Inf() + 2, end = Inf() + 100; i < end; i++) { + UNIT_ASSERT(!IsNormal(i)); + UNIT_ASSERT(ToString(i, MaxPrecision, 0) == nullptr); + } + + // below -Inf + for (TInt128 i = -Inf() - 2, end = -Inf() - 100; i < end; i--) { + UNIT_ASSERT(!IsNormal(i)); + UNIT_ASSERT(ToString(i, MaxPrecision, 0) == nullptr); + } + } Y_UNIT_TEST(TestSerializeAndDeserialize) { SimpleSerializeAndDeserialize(-Nan(), 1U); diff --git a/ydb/library/yql/public/issue/protos/issue_message.proto b/ydb/library/yql/public/issue/protos/issue_message.proto index 55c2a788028..d581a5d8f07 100644 --- a/ydb/library/yql/public/issue/protos/issue_message.proto +++ b/ydb/library/yql/public/issue/protos/issue_message.proto @@ -1,5 +1,5 @@ package NYql.NIssue.NProto; -option java_package = "com.yandex.yql.issue.proto"; +option java_package = "com.yandex.yql.issue.proto"; message IssueMessage { message Position { diff --git a/ydb/library/yql/public/issue/protos/issue_severity.proto b/ydb/library/yql/public/issue/protos/issue_severity.proto index 46a886a4e42..70fd61f5d9a 100644 --- a/ydb/library/yql/public/issue/protos/issue_severity.proto +++ b/ydb/library/yql/public/issue/protos/issue_severity.proto @@ -2,7 +2,7 @@ syntax = "proto3"; package NYql; -option java_package = "com.yandex.yql.proto"; +option java_package = "com.yandex.yql.proto"; message TSeverityIds { enum ESeverityId { diff --git a/ydb/library/yql/public/udf/udf_allocator.h b/ydb/library/yql/public/udf/udf_allocator.h index 3132163637a..48a94ba6a1d 100644 --- a/ydb/library/yql/public/udf/udf_allocator.h +++ b/ydb/library/yql/public/udf/udf_allocator.h @@ -1,10 +1,10 @@ -#pragma once +#pragma once #include "udf_version.h" #include <util/system/types.h> #include <new> #include <cstddef> #include <limits> - + #if UDF_ABI_COMPATIBILITY_VERSION_CURRENT >= UDF_ABI_COMPATIBILITY_VERSION(2, 8) extern "C" void* UdfAllocateWithSize(ui64 size); extern "C" void UdfFreeWithSize(const void* mem, ui64 size); @@ -22,8 +22,8 @@ extern "C" void UdfFree(const void* mem); #endif namespace NYql { -namespace NUdf { - +namespace NUdf { + template <typename Type> struct TStdAllocatorForUdf { @@ -99,7 +99,7 @@ struct TWithUdfAllocator { } #endif }; - -} // namespace NUdf + +} // namespace NUdf } // namespace NYql diff --git a/ydb/library/yql/public/udf/udf_ptr.h b/ydb/library/yql/public/udf/udf_ptr.h index 04dd73b15c6..44debb7856f 100644 --- a/ydb/library/yql/public/udf/udf_ptr.h +++ b/ydb/library/yql/public/udf/udf_ptr.h @@ -1,173 +1,173 @@ -#pragma once - +#pragma once + #include <util/system/yassert.h> // Y_ASSERT - - + + namespace NYql { -namespace NUdf { - +namespace NUdf { + namespace NDetails { -struct TDelete { - template <typename T> - static void DoDelete(T* ptr) { - delete ptr; - } -}; +struct TDelete { + template <typename T> + static void DoDelete(T* ptr) { + delete ptr; + } +}; } - -/////////////////////////////////////////////////////////////////////////////// -// TUniquePtr -/////////////////////////////////////////////////////////////////////////////// + +/////////////////////////////////////////////////////////////////////////////// +// TUniquePtr +/////////////////////////////////////////////////////////////////////////////// template <typename T, typename D = NDetails::TDelete> -class TUniquePtr -{ -public: - inline TUniquePtr(T* ptr = nullptr) - : Ptr_(ptr) - { - } - - inline TUniquePtr(TUniquePtr&& rhs) - : Ptr_(rhs.Release()) - { - } - - inline TUniquePtr& operator=(TUniquePtr&& rhs) { - if (this != &rhs) { - Reset(rhs.Release()); - } - - return *this; - } - - TUniquePtr(const TUniquePtr& rhs) = delete; - TUniquePtr& operator=(const TUniquePtr& rhs) = delete; - - inline ~TUniquePtr() { - DoDestroy(); - } - - inline T* Release() { - T* tmp = Ptr_; - Ptr_ = nullptr; - return tmp; - } - - inline void Reset(T* ptr = nullptr) { - if (Ptr_ != ptr) { - DoDestroy(); - Ptr_ = ptr; - } - } - - inline void Swap(TUniquePtr& rhs) { - T* tmp = Ptr_; - Ptr_ = rhs.Ptr_; - rhs.Ptr_ = tmp; - } - - inline T* Get() const { return Ptr_; } - inline T& operator*() const { return *Ptr_; } - inline T* operator->() const { return Ptr_; } - inline explicit operator bool() const { return Ptr_ != nullptr; } - -private: - inline void DoDestroy() { - if (Ptr_) +class TUniquePtr +{ +public: + inline TUniquePtr(T* ptr = nullptr) + : Ptr_(ptr) + { + } + + inline TUniquePtr(TUniquePtr&& rhs) + : Ptr_(rhs.Release()) + { + } + + inline TUniquePtr& operator=(TUniquePtr&& rhs) { + if (this != &rhs) { + Reset(rhs.Release()); + } + + return *this; + } + + TUniquePtr(const TUniquePtr& rhs) = delete; + TUniquePtr& operator=(const TUniquePtr& rhs) = delete; + + inline ~TUniquePtr() { + DoDestroy(); + } + + inline T* Release() { + T* tmp = Ptr_; + Ptr_ = nullptr; + return tmp; + } + + inline void Reset(T* ptr = nullptr) { + if (Ptr_ != ptr) { + DoDestroy(); + Ptr_ = ptr; + } + } + + inline void Swap(TUniquePtr& rhs) { + T* tmp = Ptr_; + Ptr_ = rhs.Ptr_; + rhs.Ptr_ = tmp; + } + + inline T* Get() const { return Ptr_; } + inline T& operator*() const { return *Ptr_; } + inline T* operator->() const { return Ptr_; } + inline explicit operator bool() const { return Ptr_ != nullptr; } + +private: + inline void DoDestroy() { + if (Ptr_) D::DoDelete(Ptr_); - } - -private: - T* Ptr_; -}; - -/////////////////////////////////////////////////////////////////////////////// -// TRefCountedPtr -/////////////////////////////////////////////////////////////////////////////// -template <class T> -class TDefaultRefCountedPtrOps -{ -public: - static inline void Ref(T* t) { + } + +private: + T* Ptr_; +}; + +/////////////////////////////////////////////////////////////////////////////// +// TRefCountedPtr +/////////////////////////////////////////////////////////////////////////////// +template <class T> +class TDefaultRefCountedPtrOps +{ +public: + static inline void Ref(T* t) { Y_ASSERT(t); - t->Ref(); - } - - static inline void UnRef(T* t) { + t->Ref(); + } + + static inline void UnRef(T* t) { Y_ASSERT(t); - t->UnRef(); - } - - static inline ui32 RefCount(const T* t) { + t->UnRef(); + } + + static inline ui32 RefCount(const T* t) { Y_ASSERT(t); - return t->RefCount(); - } -}; - -template <typename T, typename Ops = TDefaultRefCountedPtrOps<T>> -class TRefCountedPtr -{ -public: - enum AddRef { ADD_REF }; - enum StealRef { STEAL_REF }; - -public: - inline TRefCountedPtr(T* ptr = nullptr) - : Ptr_(ptr) - { - Ref(); - } - - inline TRefCountedPtr(T* ptr, StealRef) - : Ptr_(ptr) - { - // do not call Ref() on new pointer - } - - inline TRefCountedPtr(const TRefCountedPtr& rhs) - : Ptr_(rhs.Ptr_) - { - Ref(); - } - - inline TRefCountedPtr(TRefCountedPtr&& rhs) - : Ptr_(rhs.Ptr_) - { - rhs.Ptr_ = nullptr; - } - - inline TRefCountedPtr& operator=(const TRefCountedPtr& rhs) { - if (this != &rhs) { - UnRef(); - Ptr_ = rhs.Ptr_; - Ref(); - } - - return *this; - } - - inline TRefCountedPtr& operator=(TRefCountedPtr&& rhs) { - if (this != &rhs) { - UnRef(); - Ptr_ = rhs.Ptr_; - rhs.Ptr_ = nullptr; - } - - return *this; - } - - inline ~TRefCountedPtr() { - UnRef(); - } - - inline void Reset(T* ptr = nullptr) { - if (Ptr_ != ptr) { - UnRef(); - Ptr_ = ptr; - Ref(); - } - } - + return t->RefCount(); + } +}; + +template <typename T, typename Ops = TDefaultRefCountedPtrOps<T>> +class TRefCountedPtr +{ +public: + enum AddRef { ADD_REF }; + enum StealRef { STEAL_REF }; + +public: + inline TRefCountedPtr(T* ptr = nullptr) + : Ptr_(ptr) + { + Ref(); + } + + inline TRefCountedPtr(T* ptr, StealRef) + : Ptr_(ptr) + { + // do not call Ref() on new pointer + } + + inline TRefCountedPtr(const TRefCountedPtr& rhs) + : Ptr_(rhs.Ptr_) + { + Ref(); + } + + inline TRefCountedPtr(TRefCountedPtr&& rhs) + : Ptr_(rhs.Ptr_) + { + rhs.Ptr_ = nullptr; + } + + inline TRefCountedPtr& operator=(const TRefCountedPtr& rhs) { + if (this != &rhs) { + UnRef(); + Ptr_ = rhs.Ptr_; + Ref(); + } + + return *this; + } + + inline TRefCountedPtr& operator=(TRefCountedPtr&& rhs) { + if (this != &rhs) { + UnRef(); + Ptr_ = rhs.Ptr_; + rhs.Ptr_ = nullptr; + } + + return *this; + } + + inline ~TRefCountedPtr() { + UnRef(); + } + + inline void Reset(T* ptr = nullptr) { + if (Ptr_ != ptr) { + UnRef(); + Ptr_ = ptr; + Ref(); + } + } + inline void Reset(T* ptr, StealRef) { if (Ptr_ != ptr) { UnRef(); @@ -175,45 +175,45 @@ public: } } - inline void Swap(TRefCountedPtr& rhs) { - T* tmp = Ptr_; - Ptr_ = rhs.Ptr_; - rhs.Ptr_ = tmp; - } - - inline T* Release() { - // do not decrement ref counter here. just send ownership to caller - T* tmp = Ptr_; - Ptr_ = nullptr; - return tmp; - } - - inline T* Get() const { return Ptr_; } - inline T& operator*() const { return *Ptr_; } - inline T* operator->() const { return Ptr_; } - inline explicit operator bool() const { return Ptr_ != nullptr; } - - inline ui32 RefCount() const { - return Ptr_ ? Ops::RefCount(Ptr_) : 0; - } - -private: - inline void Ref() { - if (Ptr_){ - Ops::Ref(Ptr_); - } - } - - inline void UnRef() { - if (Ptr_) { - Ops::UnRef(Ptr_); - Ptr_ = nullptr; - } - } - -private: - T* Ptr_; -}; - -} // namspace NUdf + inline void Swap(TRefCountedPtr& rhs) { + T* tmp = Ptr_; + Ptr_ = rhs.Ptr_; + rhs.Ptr_ = tmp; + } + + inline T* Release() { + // do not decrement ref counter here. just send ownership to caller + T* tmp = Ptr_; + Ptr_ = nullptr; + return tmp; + } + + inline T* Get() const { return Ptr_; } + inline T& operator*() const { return *Ptr_; } + inline T* operator->() const { return Ptr_; } + inline explicit operator bool() const { return Ptr_ != nullptr; } + + inline ui32 RefCount() const { + return Ptr_ ? Ops::RefCount(Ptr_) : 0; + } + +private: + inline void Ref() { + if (Ptr_){ + Ops::Ref(Ptr_); + } + } + + inline void UnRef() { + if (Ptr_) { + Ops::UnRef(Ptr_); + Ptr_ = nullptr; + } + } + +private: + T* Ptr_; +}; + +} // namspace NUdf } // namspace NYql diff --git a/ydb/library/yql/public/udf/udf_registrator.h b/ydb/library/yql/public/udf/udf_registrator.h index c2c966905f0..4d195d7eda6 100644 --- a/ydb/library/yql/public/udf/udf_registrator.h +++ b/ydb/library/yql/public/udf/udf_registrator.h @@ -1,11 +1,11 @@ -#pragma once - -#include "udf_version.h" -#include "udf_types.h" -#include "udf_ptr.h" -#include "udf_string.h" +#pragma once + +#include "udf_version.h" +#include "udf_types.h" +#include "udf_ptr.h" +#include "udf_string.h" #include "udf_value.h" - + #include <functional> #include <stdarg.h> @@ -19,20 +19,20 @@ #else # define UDF_API __attribute__ ((visibility("default"))) #endif - + #ifdef BUILD_UDF -#define REGISTER_MODULES(...) \ - extern "C" UDF_API void Register( \ +#define REGISTER_MODULES(...) \ + extern "C" UDF_API void Register( \ ::NYql::NUdf::IRegistrator& registrator, ui32 flags) { \ Y_UNUSED(flags); \ ::NYql::NUdf::RegisterHelper<__VA_ARGS__>(registrator); \ - } \ - extern "C" UDF_API ui32 AbiVersion() { \ + } \ + extern "C" UDF_API ui32 AbiVersion() { \ return ::NYql::NUdf::CurrentAbiVersion(); \ }\ extern "C" UDF_API void SetBackTraceCallback(::NYql::NUdf::TBackTraceCallback callback) { \ ::NYql::NUdf::SetBackTraceCallbackImpl(callback); \ - } + } #else #define REGISTER_MODULES(...) \ namespace { \ @@ -45,10 +45,10 @@ #endif namespace NYql { -namespace NUdf { - -class IFunctionTypeInfoBuilder; - +namespace NUdf { + +class IFunctionTypeInfoBuilder; + struct TStaticSymbols { void* (*UdfAllocateFunc)(ui64 size); void (*UdfFreeFunc)(const void* mem); @@ -69,9 +69,9 @@ UDF_ASSERT_TYPE_SIZE(TStaticSymbols, 40); inline TStaticSymbols GetStaticSymbols(); -////////////////////////////////////////////////////////////////////////////// -// IFunctionNamesSink -////////////////////////////////////////////////////////////////////////////// +////////////////////////////////////////////////////////////////////////////// +// IFunctionNamesSink +////////////////////////////////////////////////////////////////////////////// class IFunctionDescriptor { public: @@ -98,65 +98,65 @@ public: UDF_ASSERT_TYPE_SIZE(IFunctionsSink, 8); typedef IFunctionsSink IFunctionNamesSink; - -////////////////////////////////////////////////////////////////////////////// -// IUdfModule -////////////////////////////////////////////////////////////////////////////// -class IUdfModule -{ -public: - struct TFlags { - enum { + +////////////////////////////////////////////////////////////////////////////// +// IUdfModule +////////////////////////////////////////////////////////////////////////////// +class IUdfModule +{ +public: + struct TFlags { + enum { TypesOnly = 0x01 - }; - }; - -public: - virtual ~IUdfModule() = default; - + }; + }; + +public: + virtual ~IUdfModule() = default; + virtual void GetAllFunctions(IFunctionsSink& sink) const = 0; - - virtual void BuildFunctionTypeInfo( - const TStringRef& name, + + virtual void BuildFunctionTypeInfo( + const TStringRef& name, TType* userType, - const TStringRef& typeConfig, - ui32 flags, - IFunctionTypeInfoBuilder& builder) const = 0; + const TStringRef& typeConfig, + ui32 flags, + IFunctionTypeInfoBuilder& builder) const = 0; virtual void CleanupOnTerminate() const = 0; -}; - -UDF_ASSERT_TYPE_SIZE(IUdfModule, 8); - -////////////////////////////////////////////////////////////////////////////// -// TRegistrator -////////////////////////////////////////////////////////////////////////////// -class IRegistrator -{ -public: - struct TFlags { - enum { - TypesOnly = 0x01, - }; - }; - -public: - virtual ~IRegistrator() = default; - - virtual void AddModule( - const TStringRef& name, - TUniquePtr<IUdfModule> module) = 0; -}; - -UDF_ASSERT_TYPE_SIZE(IRegistrator, 8); - +}; + +UDF_ASSERT_TYPE_SIZE(IUdfModule, 8); + +////////////////////////////////////////////////////////////////////////////// +// TRegistrator +////////////////////////////////////////////////////////////////////////////// +class IRegistrator +{ +public: + struct TFlags { + enum { + TypesOnly = 0x01, + }; + }; + +public: + virtual ~IRegistrator() = default; + + virtual void AddModule( + const TStringRef& name, + TUniquePtr<IUdfModule> module) = 0; +}; + +UDF_ASSERT_TYPE_SIZE(IRegistrator, 8); + typedef void(*TBackTraceCallback)(); -using TRegisterFunctionPtr = void (*)(IRegistrator& registrator, ui32 flags); -using TAbiVersionFunctionPtr = ui32 (*)(); +using TRegisterFunctionPtr = void (*)(IRegistrator& registrator, ui32 flags); +using TAbiVersionFunctionPtr = ui32 (*)(); using TBindSymbolsFunctionPtr = void (*)(const TStaticSymbols& symbols); using TSetBackTraceCallbackPtr = void(*)(TBackTraceCallback callback); - + template<typename TModule> static inline void RegisterHelper(IRegistrator& registrator) { TUniquePtr<TModule> ptr(new TModule()); @@ -193,11 +193,11 @@ static inline void AddToStaticUdfRegistry() { } -} // namspace NUdf +} // namspace NUdf } // namspace NYql extern "C" UDF_API void Register(NYql::NUdf::IRegistrator& registrator, ui32 flags); -extern "C" UDF_API ui32 AbiVersion(); +extern "C" UDF_API ui32 AbiVersion(); #if defined(_win_) || defined(_darwin_) extern "C" UDF_API void BindSymbols(const NYql::NUdf::TStaticSymbols& symbols); #endif diff --git a/ydb/library/yql/public/udf/udf_string.h b/ydb/library/yql/public/udf/udf_string.h index 22bc90cbe43..7c687ffbf48 100644 --- a/ydb/library/yql/public/udf/udf_string.h +++ b/ydb/library/yql/public/udf/udf_string.h @@ -1,28 +1,28 @@ -#pragma once - -#include "udf_types.h" +#pragma once + +#include "udf_types.h" #include "udf_allocator.h" #include "udf_string_ref.h" - -#include <util/generic/strbuf.h> + +#include <util/generic/strbuf.h> #include <util/system/align.h> - + #include <new> namespace NYql { -namespace NUdf { - -////////////////////////////////////////////////////////////////////////////// -// TStringValue -////////////////////////////////////////////////////////////////////////////// -class TStringValue -{ - friend class TUnboxedValue; +namespace NUdf { + +////////////////////////////////////////////////////////////////////////////// +// TStringValue +////////////////////////////////////////////////////////////////////////////// +class TStringValue +{ + friend class TUnboxedValue; friend class TUnboxedValuePod; - - class TData { - friend class TStringValue; - public: + + class TData { + friend class TStringValue; + public: inline TData(ui32 size, ui32 cap) : Size_(size) , Refs_(1) @@ -208,7 +208,7 @@ public: TData* Data_; }; -UDF_ASSERT_TYPE_SIZE(TStringValue, 8); - -} // namspace NUdf +UDF_ASSERT_TYPE_SIZE(TStringValue, 8); + +} // namspace NUdf } // namspace NYql diff --git a/ydb/library/yql/public/udf/udf_string_ref.h b/ydb/library/yql/public/udf/udf_string_ref.h index 8d23b2425f3..75a82b81e6a 100644 --- a/ydb/library/yql/public/udf/udf_string_ref.h +++ b/ydb/library/yql/public/udf/udf_string_ref.h @@ -1,33 +1,33 @@ -#pragma once - -#include "udf_types.h" - -#include <util/generic/strbuf.h> - +#pragma once + +#include "udf_types.h" + +#include <util/generic/strbuf.h> + #include <algorithm> #include <string_view> #include <type_traits> namespace NYql { -namespace NUdf { - -////////////////////////////////////////////////////////////////////////////// +namespace NUdf { + +////////////////////////////////////////////////////////////////////////////// // TStringRefBase -////////////////////////////////////////////////////////////////////////////// +////////////////////////////////////////////////////////////////////////////// template<bool Const> class TStringRefBase -{ -public: +{ +public: typedef std::conditional_t<Const, const char*, char*> TDataType; - + protected: inline constexpr TStringRefBase() noexcept = default; - + inline constexpr TStringRefBase(TDataType data, ui32 size) noexcept - : Data_(data) - , Size_(size) + : Data_(data) + , Size_(size) {} - + public: inline constexpr operator std::string_view() const noexcept { return { Data_, Size_ }; } inline constexpr operator TStringBuf() const noexcept { return { Data_, Size_ }; } @@ -83,37 +83,37 @@ public: inline constexpr TStringRef(const TStringType& buf) noexcept : TBase(TGetData<TStringType>::Get(buf), TGetSize<TStringType>::Get(buf)) {} - - template <size_t size> + + template <size_t size> inline static constexpr TStringRef Of(const char(&str)[size]) noexcept { return TStringRef(str); - } - + } + inline constexpr TStringRef& Trunc(ui32 len) noexcept { - if (Size_ > len) { - Size_ = len; - } - return *this; - } - + if (Size_ > len) { + Size_ = len; + } + return *this; + } + inline constexpr TStringRef Substring(ui32 start, ui32 count) const noexcept { start = std::min(start, Size_); count = std::min(count, Size_ - start); - return TStringRef(Data_ + start, count); - } - + return TStringRef(Data_ + start, count); + } + inline constexpr bool operator==(const TStringRef& rhs) const noexcept { - return Compare(*this, rhs) == 0; - } - + return Compare(*this, rhs) == 0; + } + inline constexpr bool operator!=(const TStringRef& rhs) const noexcept { return Compare(*this, rhs) != 0; } inline constexpr bool operator<(const TStringRef& rhs) const noexcept { - return Compare(*this, rhs) < 0; - } - + return Compare(*this, rhs) < 0; + } + inline constexpr bool operator<=(const TStringRef& rhs) const noexcept { return Compare(*this, rhs) <= 0; } @@ -126,13 +126,13 @@ public: return Compare(*this, rhs) >= 0; } -private: +private: inline static constexpr int Compare(const TStringRef& s1, const TStringRef& s2) noexcept { auto minSize = std::min(s1.Size(), s2.Size()); if (const auto result = minSize > 0 ? std::memcmp(s1.Data(), s2.Data(), minSize) : 0) return result; return int(s1.Size()) - int(s2.Size()); - } + } Y_HAS_MEMBER(Data); Y_HAS_MEMBER(Size); @@ -170,9 +170,9 @@ private: template<typename TStringType> using TGetSize = std::conditional_t<THasSize<TStringType>::value, TBySize<TStringType>, TBysize<TStringType>>; -}; - -UDF_ASSERT_TYPE_SIZE(TStringRef, 16); - -} // namspace NUdf +}; + +UDF_ASSERT_TYPE_SIZE(TStringRef, 16); + +} // namspace NUdf } // namspace NYql diff --git a/ydb/library/yql/public/udf/udf_type_builder.h b/ydb/library/yql/public/udf/udf_type_builder.h index d96881091e1..8bfc73a449d 100644 --- a/ydb/library/yql/public/udf/udf_type_builder.h +++ b/ydb/library/yql/public/udf/udf_type_builder.h @@ -1,19 +1,19 @@ -#pragma once - +#pragma once + #include "udf_counter.h" -#include "udf_types.h" -#include "udf_ptr.h" +#include "udf_types.h" +#include "udf_ptr.h" #include "udf_string_ref.h" #include "udf_value.h" - + #include <type_traits> - + namespace NYql { -namespace NUdf { - -class TStringRef; -class IBoxedValue; - +namespace NUdf { + +class TStringRef; +class IBoxedValue; + /////////////////////////////////////////////////////////////////////////////// // ISecureParamsProvider /////////////////////////////////////////////////////////////////////////////// @@ -24,95 +24,95 @@ public: virtual bool GetSecureParam(NUdf::TStringRef key, NUdf::TStringRef& value) const = 0; }; -template <typename T> +template <typename T> struct TAutoMap { using ItemType = T; }; template <typename T> -struct TOptional { using ItemType = T; }; - +struct TOptional { using ItemType = T; }; + template <typename T, const char* Name> struct TNamedArg { using ItemType = T; }; -template <typename T> +template <typename T> struct TListType { using ItemType = T; }; - -template <typename TKey, typename TValue> -struct TDict { - using KeyType = TKey; - using ValueType = TValue; -}; - -template <typename... TArgs> -struct TTuple; - + +template <typename TKey, typename TValue> +struct TDict { + using KeyType = TKey; + using ValueType = TValue; +}; + +template <typename... TArgs> +struct TTuple; + template <const char* Tag> struct TResource {}; template <typename... TArgs> struct TVariant; - + template <typename T> struct TStream { using ItemType = T; }; - + template <typename T, const char* Tag> struct TTagged { using BaseType = T; }; -////////////////////////////////////////////////////////////////////////////// -// ITypeBuilder -////////////////////////////////////////////////////////////////////////////// -class ITypeBuilder -{ -public: - virtual ~ITypeBuilder() = default; - - virtual TType* Build() const = 0; -}; - -UDF_ASSERT_TYPE_SIZE(ITypeBuilder, 8); - -////////////////////////////////////////////////////////////////////////////// -// IOptionalTypeBuilder -////////////////////////////////////////////////////////////////////////////// -class IOptionalTypeBuilder: public ITypeBuilder -{ -public: - using TPtr = TUniquePtr<IOptionalTypeBuilder>; - -public: +////////////////////////////////////////////////////////////////////////////// +// ITypeBuilder +////////////////////////////////////////////////////////////////////////////// +class ITypeBuilder +{ +public: + virtual ~ITypeBuilder() = default; + + virtual TType* Build() const = 0; +}; + +UDF_ASSERT_TYPE_SIZE(ITypeBuilder, 8); + +////////////////////////////////////////////////////////////////////////////// +// IOptionalTypeBuilder +////////////////////////////////////////////////////////////////////////////// +class IOptionalTypeBuilder: public ITypeBuilder +{ +public: + using TPtr = TUniquePtr<IOptionalTypeBuilder>; + +public: template <typename T, typename = std::enable_if_t<TKnownDataType<T>::Result>> - inline IOptionalTypeBuilder& Item() { + inline IOptionalTypeBuilder& Item() { return Item(TDataType<T>::Id); - } - + } + virtual IOptionalTypeBuilder& Item(TDataTypeId type) = 0; - virtual IOptionalTypeBuilder& Item(const TType* type) = 0; - virtual IOptionalTypeBuilder& Item(const ITypeBuilder& type) = 0; -}; - -UDF_ASSERT_TYPE_SIZE(IOptionalTypeBuilder, 8); - -////////////////////////////////////////////////////////////////////////////// -// IListTypeBuilder -////////////////////////////////////////////////////////////////////////////// -class IListTypeBuilder: public ITypeBuilder -{ -public: - using TPtr = TUniquePtr<IListTypeBuilder>; - -public: + virtual IOptionalTypeBuilder& Item(const TType* type) = 0; + virtual IOptionalTypeBuilder& Item(const ITypeBuilder& type) = 0; +}; + +UDF_ASSERT_TYPE_SIZE(IOptionalTypeBuilder, 8); + +////////////////////////////////////////////////////////////////////////////// +// IListTypeBuilder +////////////////////////////////////////////////////////////////////////////// +class IListTypeBuilder: public ITypeBuilder +{ +public: + using TPtr = TUniquePtr<IListTypeBuilder>; + +public: template <typename T, typename = std::enable_if_t<TKnownDataType<T>::Result>> - inline IListTypeBuilder& Item() { + inline IListTypeBuilder& Item() { return Item(TDataType<T>::Id); - } - + } + virtual IListTypeBuilder& Item(TDataTypeId type) = 0; - virtual IListTypeBuilder& Item(const TType* type) = 0; - virtual IListTypeBuilder& Item(const ITypeBuilder& type) = 0; -}; - -UDF_ASSERT_TYPE_SIZE(IListTypeBuilder, 8); - -////////////////////////////////////////////////////////////////////////////// + virtual IListTypeBuilder& Item(const TType* type) = 0; + virtual IListTypeBuilder& Item(const ITypeBuilder& type) = 0; +}; + +UDF_ASSERT_TYPE_SIZE(IListTypeBuilder, 8); + +////////////////////////////////////////////////////////////////////////////// // IVariantTypeBuilder ////////////////////////////////////////////////////////////////////////////// class IVariantTypeBuilder : public ITypeBuilder @@ -121,7 +121,7 @@ public: using TPtr = TUniquePtr<IVariantTypeBuilder>; public: - // type must be either tuple or struct + // type must be either tuple or struct virtual IVariantTypeBuilder& Over(const TType* type) = 0; virtual IVariantTypeBuilder& Over(const ITypeBuilder& type) = 0; }; @@ -150,36 +150,36 @@ public: UDF_ASSERT_TYPE_SIZE(IStreamTypeBuilder, 8); ////////////////////////////////////////////////////////////////////////////// -// IDictTypeBuilder -////////////////////////////////////////////////////////////////////////////// -class IDictTypeBuilder: public ITypeBuilder -{ -public: - using TPtr = TUniquePtr<IDictTypeBuilder>; - -public: +// IDictTypeBuilder +////////////////////////////////////////////////////////////////////////////// +class IDictTypeBuilder: public ITypeBuilder +{ +public: + using TPtr = TUniquePtr<IDictTypeBuilder>; + +public: template <typename T, typename = std::enable_if_t<TKnownDataType<T>::Result>> - inline IDictTypeBuilder& Key() { + inline IDictTypeBuilder& Key() { return Key(TDataType<T>::Id); - } - + } + virtual IDictTypeBuilder& Key(TDataTypeId type) = 0; - virtual IDictTypeBuilder& Key(const TType* type) = 0; - virtual IDictTypeBuilder& Key(const ITypeBuilder& type) = 0; - + virtual IDictTypeBuilder& Key(const TType* type) = 0; + virtual IDictTypeBuilder& Key(const ITypeBuilder& type) = 0; + template <typename T, typename = std::enable_if_t<TKnownDataType<T>::Result>> - inline IDictTypeBuilder& Value() { + inline IDictTypeBuilder& Value() { return Value(TDataType<T>::Id); - } - + } + virtual IDictTypeBuilder& Value(TDataTypeId type) = 0; - virtual IDictTypeBuilder& Value(const TType* type) = 0; - virtual IDictTypeBuilder& Value(const ITypeBuilder& type) = 0; -}; - -UDF_ASSERT_TYPE_SIZE(IDictTypeBuilder, 8); - -////////////////////////////////////////////////////////////////////////////// + virtual IDictTypeBuilder& Value(const TType* type) = 0; + virtual IDictTypeBuilder& Value(const ITypeBuilder& type) = 0; +}; + +UDF_ASSERT_TYPE_SIZE(IDictTypeBuilder, 8); + +////////////////////////////////////////////////////////////////////////////// // ISetTypeBuilder ////////////////////////////////////////////////////////////////////////////// class ISetTypeBuilder: public ITypeBuilder @@ -201,36 +201,36 @@ public: UDF_ASSERT_TYPE_SIZE(ISetTypeBuilder, 8); ////////////////////////////////////////////////////////////////////////////// -// IStructTypeBuilder -////////////////////////////////////////////////////////////////////////////// -class IStructTypeBuilder: public ITypeBuilder -{ -public: - using TPtr = TUniquePtr<IStructTypeBuilder>; - -public: +// IStructTypeBuilder +////////////////////////////////////////////////////////////////////////////// +class IStructTypeBuilder: public ITypeBuilder +{ +public: + using TPtr = TUniquePtr<IStructTypeBuilder>; + +public: template <typename T, typename = std::enable_if_t<TKnownDataType<T>::Result>> - inline IStructTypeBuilder& AddField( - const TStringRef& name, ui32* index) - { + inline IStructTypeBuilder& AddField( + const TStringRef& name, ui32* index) + { return AddField(name, TDataType<T>::Id, index); - } - - virtual IStructTypeBuilder& AddField( + } + + virtual IStructTypeBuilder& AddField( const TStringRef& name, TDataTypeId type, ui32* index) = 0; - - virtual IStructTypeBuilder& AddField( - const TStringRef& name, const TType* type, ui32* index) = 0; - - virtual IStructTypeBuilder& AddField( - const TStringRef& name, - const ITypeBuilder& typeBuilder, - ui32* index) = 0; -}; - -UDF_ASSERT_TYPE_SIZE(IStructTypeBuilder, 8); - -////////////////////////////////////////////////////////////////////////////// + + virtual IStructTypeBuilder& AddField( + const TStringRef& name, const TType* type, ui32* index) = 0; + + virtual IStructTypeBuilder& AddField( + const TStringRef& name, + const ITypeBuilder& typeBuilder, + ui32* index) = 0; +}; + +UDF_ASSERT_TYPE_SIZE(IStructTypeBuilder, 8); + +////////////////////////////////////////////////////////////////////////////// // IEnumTypeBuilder ////////////////////////////////////////////////////////////////////////////// class IEnumTypeBuilder: public ITypeBuilder @@ -246,102 +246,102 @@ public: UDF_ASSERT_TYPE_SIZE(IEnumTypeBuilder, 8); ////////////////////////////////////////////////////////////////////////////// -// ITupleTypeBuilder -////////////////////////////////////////////////////////////////////////////// -class ITupleTypeBuilder: public ITypeBuilder -{ -public: - using TPtr = TUniquePtr<ITupleTypeBuilder>; - -public: +// ITupleTypeBuilder +////////////////////////////////////////////////////////////////////////////// +class ITupleTypeBuilder: public ITypeBuilder +{ +public: + using TPtr = TUniquePtr<ITupleTypeBuilder>; + +public: template <typename T, typename = std::enable_if_t<TKnownDataType<T>::Result>> - ITupleTypeBuilder& Add() { + ITupleTypeBuilder& Add() { return Add(TDataType<T>::Id); - } - + } + virtual ITupleTypeBuilder& Add(TDataTypeId type) = 0; - virtual ITupleTypeBuilder& Add(const TType* type) = 0; - virtual ITupleTypeBuilder& Add(const ITypeBuilder& typeBuilder) = 0; -}; - -UDF_ASSERT_TYPE_SIZE(ITupleTypeBuilder, 8); - -////////////////////////////////////////////////////////////////////////////// -// ICallableTypeBuilder -////////////////////////////////////////////////////////////////////////////// -class ICallableTypeBuilder: public ITypeBuilder -{ -public: - using TPtr = TUniquePtr<ICallableTypeBuilder>; - -public: + virtual ITupleTypeBuilder& Add(const TType* type) = 0; + virtual ITupleTypeBuilder& Add(const ITypeBuilder& typeBuilder) = 0; +}; + +UDF_ASSERT_TYPE_SIZE(ITupleTypeBuilder, 8); + +////////////////////////////////////////////////////////////////////////////// +// ICallableTypeBuilder +////////////////////////////////////////////////////////////////////////////// +class ICallableTypeBuilder: public ITypeBuilder +{ +public: + using TPtr = TUniquePtr<ICallableTypeBuilder>; + +public: template <typename T, typename = std::enable_if_t<TKnownDataType<T>::Result>> - ICallableTypeBuilder& Returns() { + ICallableTypeBuilder& Returns() { return Returns(TDataType<T>::Id); - } - + } + virtual ICallableTypeBuilder& Returns(TDataTypeId type) = 0; - virtual ICallableTypeBuilder& Returns(const TType* type) = 0; - virtual ICallableTypeBuilder& Returns(const ITypeBuilder& typeBuilder) = 0; - + virtual ICallableTypeBuilder& Returns(const TType* type) = 0; + virtual ICallableTypeBuilder& Returns(const ITypeBuilder& typeBuilder) = 0; + template <typename T, typename = std::enable_if_t<TKnownDataType<T>::Result>> - ICallableTypeBuilder& Arg() { + ICallableTypeBuilder& Arg() { return Arg(TDataType<T>::Id); - } - + } + virtual ICallableTypeBuilder& Arg(TDataTypeId type) = 0; - virtual ICallableTypeBuilder& Arg(const TType* type) = 0; - virtual ICallableTypeBuilder& Arg(const ITypeBuilder& typeBuilder) = 0; - - virtual ICallableTypeBuilder& OptionalArgs(ui32 optionalArgs) = 0; -}; - -UDF_ASSERT_TYPE_SIZE(ICallableTypeBuilder, 8); - -class IFunctionTypeInfoBuilder; - -////////////////////////////////////////////////////////////////////////////// -// IFunctionArgTypesBuilder -////////////////////////////////////////////////////////////////////////////// -class IFunctionArgTypesBuilder -{ -public: - using TPtr = TUniquePtr<IFunctionArgTypesBuilder>; - -public: - IFunctionArgTypesBuilder(IFunctionTypeInfoBuilder& parent) - : Parent_(parent) - { - } - - virtual ~IFunctionArgTypesBuilder() = default; - + virtual ICallableTypeBuilder& Arg(const TType* type) = 0; + virtual ICallableTypeBuilder& Arg(const ITypeBuilder& typeBuilder) = 0; + + virtual ICallableTypeBuilder& OptionalArgs(ui32 optionalArgs) = 0; +}; + +UDF_ASSERT_TYPE_SIZE(ICallableTypeBuilder, 8); + +class IFunctionTypeInfoBuilder; + +////////////////////////////////////////////////////////////////////////////// +// IFunctionArgTypesBuilder +////////////////////////////////////////////////////////////////////////////// +class IFunctionArgTypesBuilder +{ +public: + using TPtr = TUniquePtr<IFunctionArgTypesBuilder>; + +public: + IFunctionArgTypesBuilder(IFunctionTypeInfoBuilder& parent) + : Parent_(parent) + { + } + + virtual ~IFunctionArgTypesBuilder() = default; + template <typename T> inline IFunctionArgTypesBuilder& Add(); - + virtual IFunctionArgTypesBuilder& Add(TDataTypeId type) = 0; - virtual IFunctionArgTypesBuilder& Add(const TType* type) = 0; - virtual IFunctionArgTypesBuilder& Add(const ITypeBuilder& typeBuilder) = 0; + virtual IFunctionArgTypesBuilder& Add(const TType* type) = 0; + virtual IFunctionArgTypesBuilder& Add(const ITypeBuilder& typeBuilder) = 0; // sets name for the last added argument virtual IFunctionArgTypesBuilder& Name(const TStringRef& name) = 0; // sets flags for the last added argument, see ICallablePayload::TArgumentFlags virtual IFunctionArgTypesBuilder& Flags(ui64 flags) = 0; - - inline IFunctionTypeInfoBuilder& Done() const { - return Parent_; - } - - inline IFunctionTypeInfoBuilder& Parent() const { - return Parent_; - } - -private: - IFunctionTypeInfoBuilder& Parent_; -}; - -UDF_ASSERT_TYPE_SIZE(IFunctionArgTypesBuilder, 16); - -////////////////////////////////////////////////////////////////////////////// + + inline IFunctionTypeInfoBuilder& Done() const { + return Parent_; + } + + inline IFunctionTypeInfoBuilder& Parent() const { + return Parent_; + } + +private: + IFunctionTypeInfoBuilder& Parent_; +}; + +UDF_ASSERT_TYPE_SIZE(IFunctionArgTypesBuilder, 16); + +////////////////////////////////////////////////////////////////////////////// // IRefCounted ////////////////////////////////////////////////////////////////////////////// class IRefCounted { @@ -409,18 +409,18 @@ public: UDF_ASSERT_TYPE_SIZE(ICompare, 16); ////////////////////////////////////////////////////////////////////////////// -// IFunctionTypeInfoBuilder -////////////////////////////////////////////////////////////////////////////// -namespace NImpl { - -template <typename T> struct TSimpleSignatureHelper; -template <typename T> struct TTypeBuilderHelper; -template <typename... TArgs> struct TArgsHelper; -template <typename... TArgs> struct TTupleHelper; -template <typename... TArgs> struct TCallableArgsHelper; - -} // namspace NImpl - +// IFunctionTypeInfoBuilder +////////////////////////////////////////////////////////////////////////////// +namespace NImpl { + +template <typename T> struct TSimpleSignatureHelper; +template <typename T> struct TTypeBuilderHelper; +template <typename... TArgs> struct TArgsHelper; +template <typename... TArgs> struct TTupleHelper; +template <typename... TArgs> struct TCallableArgsHelper; + +} // namspace NImpl + struct TSourcePosition { TSourcePosition(ui32 row = 0, ui32 column = 0, TStringRef file = {}) : Row_(row) @@ -441,46 +441,46 @@ inline IOutputStream& operator<<(IOutputStream& os, const TSourcePosition& pos) } class IFunctionTypeInfoBuilder1 -{ -public: +{ +public: virtual ~IFunctionTypeInfoBuilder1() = default; - - // function implementation + + // function implementation virtual IFunctionTypeInfoBuilder1& ImplementationImpl( - TUniquePtr<IBoxedValue> impl) = 0; - + TUniquePtr<IBoxedValue> impl) = 0; + virtual IFunctionTypeInfoBuilder1& ReturnsImpl(TDataTypeId type) = 0; virtual IFunctionTypeInfoBuilder1& ReturnsImpl(const TType* type) = 0; virtual IFunctionTypeInfoBuilder1& ReturnsImpl(const ITypeBuilder& typeBuilder) = 0; - - // function argument types - virtual TUniquePtr<IFunctionArgTypesBuilder> Args(ui32 expectedItem = 10) = 0; + + // function argument types + virtual TUniquePtr<IFunctionArgTypesBuilder> Args(ui32 expectedItem = 10) = 0; virtual IFunctionTypeInfoBuilder1& OptionalArgsImpl(ui32 optionalArgs) = 0; - + virtual IFunctionTypeInfoBuilder1& RunConfigImpl(TDataTypeId type) = 0; virtual IFunctionTypeInfoBuilder1& RunConfigImpl(const TType* type) = 0; virtual IFunctionTypeInfoBuilder1& RunConfigImpl(const ITypeBuilder& typeBuilder) = 0; - - // errors - virtual void SetError(const TStringRef& error) = 0; - - // primitive types + + // errors + virtual void SetError(const TStringRef& error) = 0; + + // primitive types virtual TType* Primitive(TDataTypeId typeId) const = 0; - - // complex types builders - virtual IOptionalTypeBuilder::TPtr Optional() const = 0; - virtual IListTypeBuilder::TPtr List() const = 0; - virtual IDictTypeBuilder::TPtr Dict() const = 0; - virtual IStructTypeBuilder::TPtr Struct(ui32 expectedItems = 10) const = 0; - virtual ITupleTypeBuilder::TPtr Tuple(ui32 expectedItems = 10) const = 0; - virtual ICallableTypeBuilder::TPtr Callable(ui32 expectedArgs = 10) const = 0; - - // special types - virtual TType* Void() const = 0; + + // complex types builders + virtual IOptionalTypeBuilder::TPtr Optional() const = 0; + virtual IListTypeBuilder::TPtr List() const = 0; + virtual IDictTypeBuilder::TPtr Dict() const = 0; + virtual IStructTypeBuilder::TPtr Struct(ui32 expectedItems = 10) const = 0; + virtual ITupleTypeBuilder::TPtr Tuple(ui32 expectedItems = 10) const = 0; + virtual ICallableTypeBuilder::TPtr Callable(ui32 expectedArgs = 10) const = 0; + + // special types + virtual TType* Void() const = 0; virtual TType* Resource(const TStringRef& tag) const = 0; - - // type info - virtual ITypeInfoHelper::TPtr TypeInfoHelper() const = 0; + + // type info + virtual ITypeInfoHelper::TPtr TypeInfoHelper() const = 0; // 1.2 virtual IFunctionTypeInfoBuilder1& PayloadImpl(const TStringRef& payload) = 0; @@ -721,31 +721,31 @@ public: return *this; } #endif -}; - -UDF_ASSERT_TYPE_SIZE(IFunctionTypeInfoBuilder, 8); - +}; + +UDF_ASSERT_TYPE_SIZE(IFunctionTypeInfoBuilder, 8); + using IFunctionTypeInfoBuilderPtr = TUniquePtr<IFunctionTypeInfoBuilder>; -namespace NImpl { - -template <typename T> -struct TTypeBuilderHelper { - static TType* Build( - const IFunctionTypeInfoBuilder& builder, +namespace NImpl { + +template <typename T> +struct TTypeBuilderHelper { + static TType* Build( + const IFunctionTypeInfoBuilder& builder, std::enable_if_t<TKnownDataType<T>::Result>* = nullptr) - { + { return builder.Primitive(TDataType<T>::Id); - } -}; - -template <> -struct TTypeBuilderHelper<void> { - static TType* Build(const IFunctionTypeInfoBuilder& builder) { - return builder.Void(); - } -}; - + } +}; + +template <> +struct TTypeBuilderHelper<void> { + static TType* Build(const IFunctionTypeInfoBuilder& builder) { + return builder.Void(); + } +}; + #if UDF_ABI_COMPATIBILITY_VERSION_CURRENT >= UDF_ABI_COMPATIBILITY_VERSION(2, 13) template <ui8 Precision, ui8 Scale> struct TTypeBuilderHelper<TDecimalDataType<Precision, Scale>> { @@ -762,34 +762,34 @@ struct TTypeBuilderHelper<TResource<Tag>> { } }; -template <typename T> +template <typename T> struct TTypeBuilderHelper<TListType<T>> { - static TType* Build(const IFunctionTypeInfoBuilder& builder) { - return builder.List()-> - Item(TTypeBuilderHelper<T>::Build(builder)) - .Build(); - } -}; - -template <typename T> -struct TTypeBuilderHelper<TOptional<T>> { - static TType* Build(const IFunctionTypeInfoBuilder& builder) { - return builder.Optional()-> - Item(TTypeBuilderHelper<T>::Build(builder)) - .Build(); - } -}; - -template <typename TKey, typename TValue> -struct TTypeBuilderHelper<TDict<TKey, TValue>> { - static TType* Build(const IFunctionTypeInfoBuilder& builder) { - return builder.Dict()-> - Key(TTypeBuilderHelper<TKey>::Build(builder)) - .Value(TTypeBuilderHelper<TValue>::Build(builder)) - .Build(); - } -}; - + static TType* Build(const IFunctionTypeInfoBuilder& builder) { + return builder.List()-> + Item(TTypeBuilderHelper<T>::Build(builder)) + .Build(); + } +}; + +template <typename T> +struct TTypeBuilderHelper<TOptional<T>> { + static TType* Build(const IFunctionTypeInfoBuilder& builder) { + return builder.Optional()-> + Item(TTypeBuilderHelper<T>::Build(builder)) + .Build(); + } +}; + +template <typename TKey, typename TValue> +struct TTypeBuilderHelper<TDict<TKey, TValue>> { + static TType* Build(const IFunctionTypeInfoBuilder& builder) { + return builder.Dict()-> + Key(TTypeBuilderHelper<TKey>::Build(builder)) + .Value(TTypeBuilderHelper<TValue>::Build(builder)) + .Build(); + } +}; + template <typename T> struct TTypeBuilderHelper<TStream<T>> { static TType* Build(const IFunctionTypeInfoBuilder& builder) { @@ -798,7 +798,7 @@ struct TTypeBuilderHelper<TStream<T>> { .Build(); } }; - + #if UDF_ABI_COMPATIBILITY_VERSION_CURRENT >= UDF_ABI_COMPATIBILITY_VERSION(2, 21) template <typename T, const char* Tag> struct TTypeBuilderHelper<TTagged<T, Tag>> { @@ -809,67 +809,67 @@ struct TTypeBuilderHelper<TTagged<T, Tag>> { }; #endif -template <> -struct TCallableArgsHelper<> { - static void Arg( - ICallableTypeBuilder& callableBuilder, - const IFunctionTypeInfoBuilder& builder) - { +template <> +struct TCallableArgsHelper<> { + static void Arg( + ICallableTypeBuilder& callableBuilder, + const IFunctionTypeInfoBuilder& builder) + { Y_UNUSED(callableBuilder); Y_UNUSED(builder); - } -}; - -template <typename TArg, typename... TArgs> -struct TCallableArgsHelper<TArg, TArgs...> { - static void Arg( - ICallableTypeBuilder& callableBuilder, - const IFunctionTypeInfoBuilder& builder) - { - callableBuilder.Arg(TTypeBuilderHelper<TArg>::Build(builder)); - TCallableArgsHelper<TArgs...>::Arg(callableBuilder, builder); - } -}; - -template <typename TReturn, typename... TArgs> -struct TTypeBuilderHelper<TReturn(*)(TArgs...)> { - static TType* Build(const IFunctionTypeInfoBuilder& builder) { - auto callableBuilder = builder.Callable(sizeof...(TArgs)); - callableBuilder->Returns(TTypeBuilderHelper<TReturn>::Build(builder)); - TCallableArgsHelper<TArgs...>::Arg(*callableBuilder, builder); - return callableBuilder->Build(); - } -}; - -template <typename TArg, typename... TArgs> -struct TTupleHelper<TArg, TArgs...> { - static void Add( - ITupleTypeBuilder& tupleBuilder, - const IFunctionTypeInfoBuilder& builder) - { - tupleBuilder.Add(TTypeBuilderHelper<TArg>::Build(builder)); - TTupleHelper<TArgs...>::Add(tupleBuilder, builder); - } -}; - -template <> -struct TTupleHelper<> { - static void Add( - ITupleTypeBuilder& tupleBuilder, - const IFunctionTypeInfoBuilder& builder) - { + } +}; + +template <typename TArg, typename... TArgs> +struct TCallableArgsHelper<TArg, TArgs...> { + static void Arg( + ICallableTypeBuilder& callableBuilder, + const IFunctionTypeInfoBuilder& builder) + { + callableBuilder.Arg(TTypeBuilderHelper<TArg>::Build(builder)); + TCallableArgsHelper<TArgs...>::Arg(callableBuilder, builder); + } +}; + +template <typename TReturn, typename... TArgs> +struct TTypeBuilderHelper<TReturn(*)(TArgs...)> { + static TType* Build(const IFunctionTypeInfoBuilder& builder) { + auto callableBuilder = builder.Callable(sizeof...(TArgs)); + callableBuilder->Returns(TTypeBuilderHelper<TReturn>::Build(builder)); + TCallableArgsHelper<TArgs...>::Arg(*callableBuilder, builder); + return callableBuilder->Build(); + } +}; + +template <typename TArg, typename... TArgs> +struct TTupleHelper<TArg, TArgs...> { + static void Add( + ITupleTypeBuilder& tupleBuilder, + const IFunctionTypeInfoBuilder& builder) + { + tupleBuilder.Add(TTypeBuilderHelper<TArg>::Build(builder)); + TTupleHelper<TArgs...>::Add(tupleBuilder, builder); + } +}; + +template <> +struct TTupleHelper<> { + static void Add( + ITupleTypeBuilder& tupleBuilder, + const IFunctionTypeInfoBuilder& builder) + { Y_UNUSED(tupleBuilder); Y_UNUSED(builder); - } -}; - -template <typename... TArgs> -struct TTypeBuilderHelper<TTuple<TArgs...>> { - static TType* Build(const IFunctionTypeInfoBuilder& builder) { - auto tupleBuilder = builder.Tuple(sizeof...(TArgs)); - TTupleHelper<TArgs...>::Add(*tupleBuilder, builder); - return tupleBuilder->Build(); - } -}; - + } +}; + +template <typename... TArgs> +struct TTypeBuilderHelper<TTuple<TArgs...>> { + static TType* Build(const IFunctionTypeInfoBuilder& builder) { + auto tupleBuilder = builder.Tuple(sizeof...(TArgs)); + TTupleHelper<TArgs...>::Add(*tupleBuilder, builder); + return tupleBuilder->Build(); + } +}; + template <typename... TArgs> struct TTypeBuilderHelper<NUdf::TVariant<TArgs...>> { static TType* Build(const IFunctionTypeInfoBuilder& builder) { @@ -878,17 +878,17 @@ struct TTypeBuilderHelper<NUdf::TVariant<TArgs...>> { return builder.Variant()->Over(*tupleBuilder).Build(); } }; - -template <> -struct TArgsHelper<> { + +template <> +struct TArgsHelper<> { static void Add(IFunctionArgTypesBuilder& builder, const char* name = nullptr, ui64 flags = 0) { Y_UNUSED(builder); Y_UNUSED(name); Y_UNUSED(flags); - } -}; - -template <typename TArg, typename... TArgs> + } +}; + +template <typename TArg, typename... TArgs> struct TArgsHelper<TAutoMap<TArg>, TArgs...> { static void Add(IFunctionArgTypesBuilder& builder, const char* name = nullptr, ui64 flags = 0) { TArgsHelper<TArg>::Add(builder, name, flags | ICallablePayload::TArgumentFlags::AutoMap); @@ -906,26 +906,26 @@ struct TArgsHelper<TNamedArg<TArg, Name>, TArgs...> { }; template <typename TArg, typename... TArgs> -struct TArgsHelper<TArg, TArgs...> { +struct TArgsHelper<TArg, TArgs...> { static void Add(IFunctionArgTypesBuilder& builder, const char* name = nullptr, ui64 flags = 0) { builder.Add(TTypeBuilderHelper<TArg>::Build(builder.Parent())).Flags(flags); if (name) { builder.Name(TStringRef(name, std::strlen(name))); } - TArgsHelper<TArgs...>::Add(builder); - } -}; - -template <typename TReturn, typename... TArgs> -struct TSimpleSignatureHelper<TReturn(TArgs...)> { - static void Register(IFunctionTypeInfoBuilder& builder) { - builder.Returns(TTypeBuilderHelper<TReturn>::Build(builder)); - TArgsHelper<TArgs...>::Add(*builder.Args()); - } -}; - -} // namspace NImpl - + TArgsHelper<TArgs...>::Add(builder); + } +}; + +template <typename TReturn, typename... TArgs> +struct TSimpleSignatureHelper<TReturn(TArgs...)> { + static void Register(IFunctionTypeInfoBuilder& builder) { + builder.Returns(TTypeBuilderHelper<TReturn>::Build(builder)); + TArgsHelper<TArgs...>::Add(*builder.Args()); + } +}; + +} // namspace NImpl + template <typename T> inline IFunctionArgTypesBuilder& IFunctionArgTypesBuilder::Add() { @@ -933,5 +933,5 @@ inline IFunctionArgTypesBuilder& IFunctionArgTypesBuilder::Add() return *this; } -} // namespace NUdf +} // namespace NUdf } // namespace NYql diff --git a/ydb/library/yql/public/udf/udf_type_inspection.cpp b/ydb/library/yql/public/udf/udf_type_inspection.cpp index d0329cc573d..46978c060ec 100644 --- a/ydb/library/yql/public/udf/udf_type_inspection.cpp +++ b/ydb/library/yql/public/udf/udf_type_inspection.cpp @@ -1 +1 @@ -#include "udf_type_inspection.h" +#include "udf_type_inspection.h" diff --git a/ydb/library/yql/public/udf/udf_type_inspection.h b/ydb/library/yql/public/udf/udf_type_inspection.h index 02ff8f55e1f..f11ea0ff072 100644 --- a/ydb/library/yql/public/udf/udf_type_inspection.h +++ b/ydb/library/yql/public/udf/udf_type_inspection.h @@ -1,36 +1,36 @@ -#pragma once - +#pragma once + #include "udf_string_ref.h" -#include "udf_types.h" - +#include "udf_types.h" + #include <library/cpp/containers/stack_vector/stack_vec.h> // TSmallVec #include <util/generic/algorithm.h> - -#include <util/system/yassert.h> // FAIL, VERIFY_DEBUG -#include <util/generic/ylimits.h> // Max - - + +#include <util/system/yassert.h> // FAIL, VERIFY_DEBUG +#include <util/generic/ylimits.h> // Max + + namespace NYql { -namespace NUdf { - -////////////////////////////////////////////////////////////////////////////// -// TStubTypeVisitor -////////////////////////////////////////////////////////////////////////////// +namespace NUdf { + +////////////////////////////////////////////////////////////////////////////// +// TStubTypeVisitor +////////////////////////////////////////////////////////////////////////////// class TStubTypeVisitor1: public ITypeVisitor -{ -private: +{ +private: void OnDataType(TDataTypeId typeId) override; - void OnStruct( - ui32 membersCount, - TStringRef* membersNames, - const TType** membersTypes) override; - void OnList(const TType* itemType) override; - void OnOptional(const TType* itemType) override; - void OnTuple(ui32 elementsCount, const TType** elementsTypes) override; - void OnDict(const TType* keyType, const TType* valueType) override; - void OnCallable( - const TType* returnType, - ui32 argsCount, const TType** argsTypes, + void OnStruct( + ui32 membersCount, + TStringRef* membersNames, + const TType** membersTypes) override; + void OnList(const TType* itemType) override; + void OnOptional(const TType* itemType) override; + void OnTuple(ui32 elementsCount, const TType** elementsTypes) override; + void OnDict(const TType* keyType, const TType* valueType) override; + void OnCallable( + const TType* returnType, + ui32 argsCount, const TType** argsTypes, ui32 optionalArgsCount, const ICallablePayload* payload) override; void OnVariant(const TType* underlyingType) override; @@ -58,7 +58,7 @@ public: void OnTagged(const TType* baseType, TStringRef tag) override; }; #endif - + #if UDF_ABI_COMPATIBILITY_VERSION_CURRENT >= UDF_ABI_COMPATIBILITY_VERSION(2, 21) using TStubTypeVisitor = TStubTypeVisitor4; #elif UDF_ABI_COMPATIBILITY_VERSION_CURRENT >= UDF_ABI_COMPATIBILITY_VERSION(2, 15) @@ -69,29 +69,29 @@ using TStubTypeVisitor = TStubTypeVisitor2; using TStubTypeVisitor = TStubTypeVisitor1; #endif -////////////////////////////////////////////////////////////////////////////// -// TDataTypeInspector -////////////////////////////////////////////////////////////////////////////// -class TDataTypeInspector: public TStubTypeVisitor -{ -public: - TDataTypeInspector(const ITypeInfoHelper& typeHelper, const TType* type) { - if (typeHelper.GetTypeKind(type) == ETypeKind::Data) { - typeHelper.VisitType(type, this); - } - } - - explicit operator bool() const { return TypeId_ != 0; } +////////////////////////////////////////////////////////////////////////////// +// TDataTypeInspector +////////////////////////////////////////////////////////////////////////////// +class TDataTypeInspector: public TStubTypeVisitor +{ +public: + TDataTypeInspector(const ITypeInfoHelper& typeHelper, const TType* type) { + if (typeHelper.GetTypeKind(type) == ETypeKind::Data) { + typeHelper.VisitType(type, this); + } + } + + explicit operator bool() const { return TypeId_ != 0; } TDataTypeId GetTypeId() const { return TypeId_; } -private: +private: void OnDataType(TDataTypeId typeId) override { - TypeId_ = typeId; - } + TypeId_ = typeId; + } #if UDF_ABI_COMPATIBILITY_VERSION_CURRENT >= UDF_ABI_COMPATIBILITY_VERSION(2, 13) void OnDecimal(ui8, ui8) override {} #endif TDataTypeId TypeId_ = 0; -}; +}; #if UDF_ABI_COMPATIBILITY_VERSION_CURRENT >= UDF_ABI_COMPATIBILITY_VERSION(2, 13) ////////////////////////////////////////////////////////////////////////////// // TDataAndDecimalTypeInspector @@ -104,7 +104,7 @@ public: typeHelper.VisitType(type, this); } } - + explicit operator bool() const { return TypeId_ != 0; } TDataTypeId GetTypeId() const { return TypeId_; } ui8 GetPrecision() const { @@ -129,197 +129,197 @@ private: ui8 Precision_ = 0, Scale_ = 0; }; #endif -////////////////////////////////////////////////////////////////////////////// -// TStructTypeInspector -////////////////////////////////////////////////////////////////////////////// -class TStructTypeInspector: public TStubTypeVisitor -{ -public: - TStructTypeInspector(const ITypeInfoHelper& typeHelper, const TType* type) { - if (typeHelper.GetTypeKind(type) == ETypeKind::Struct) { - typeHelper.VisitType(type, this); - } - } - - explicit operator bool() const { return MembersCount_ != Max<ui32>(); } - ui32 GetMembersCount() const { return MembersCount_; } - ui32 GetMemberIndex(TStringRef name) const { - ui32 index = 0; - for (TStringRef memberName: MembersNames_) { - if (memberName == name) { - return index; - } - index++; - } - return Max<ui32>(); - } - const TStringRef& GetMemberName(ui32 i) const { return MembersNames_[i]; } - const TType* GetMemberType(ui32 i) const { return MembersTypes_[i]; } - -private: - void OnStruct( - ui32 membersCount, - TStringRef* membersNames, - const TType** membersTypes) override - { - MembersCount_ = membersCount; - MembersNames_.reserve(membersCount); - MembersTypes_.reserve(membersCount); - - for (ui32 i = 0; i < membersCount; i++) { - MembersNames_.push_back(membersNames[i]); - MembersTypes_.push_back(membersTypes[i]); - } - } - -private: - ui32 MembersCount_ = Max<ui32>(); - TSmallVec<TStringRef> MembersNames_; - TSmallVec<const TType*> MembersTypes_; -}; - -////////////////////////////////////////////////////////////////////////////// -// TListTypeInspector -////////////////////////////////////////////////////////////////////////////// -class TListTypeInspector: public TStubTypeVisitor -{ -public: - TListTypeInspector(const ITypeInfoHelper& typeHelper, const TType* type) { - if (typeHelper.GetTypeKind(type) == ETypeKind::List) { - typeHelper.VisitType(type, this); - } - } - - explicit operator bool() const { return ItemType_ != nullptr; } - const TType* GetItemType() const { return ItemType_; } - -private: - void OnList(const TType* itemType) override { - ItemType_ = itemType; - } - -private: - const TType* ItemType_ = nullptr; -}; - -////////////////////////////////////////////////////////////////////////////// -// TOptionalTypeInspector -////////////////////////////////////////////////////////////////////////////// -class TOptionalTypeInspector: public TStubTypeVisitor -{ -public: - TOptionalTypeInspector(const ITypeInfoHelper& typeHelper, const TType* type) { - if (typeHelper.GetTypeKind(type) == ETypeKind::Optional) { - typeHelper.VisitType(type, this); - } - } - - explicit operator bool() const { return ItemType_ != nullptr; } - const TType* GetItemType() const { return ItemType_; } - -private: - void OnOptional(const TType* itemType) override { - ItemType_ = itemType; - } - -private: - const TType* ItemType_ = nullptr; -}; - -////////////////////////////////////////////////////////////////////////////// -// TTupleTypeInspector -////////////////////////////////////////////////////////////////////////////// -class TTupleTypeInspector: public TStubTypeVisitor -{ -public: - TTupleTypeInspector(const ITypeInfoHelper& typeHelper, const TType* type) { - if (typeHelper.GetTypeKind(type) == ETypeKind::Tuple) { - typeHelper.VisitType(type, this); - } - } - - explicit operator bool() const { return ElementsCount_ != Max<ui32>(); } - ui32 GetElementsCount() const { return ElementsCount_; } - const TType* GetElementType(ui32 i) const { return ElementsTypes_[i]; } - -private: - void OnTuple(ui32 elementsCount, const TType** elementsTypes) override { - ElementsCount_ = elementsCount; - ElementsTypes_.reserve(elementsCount); - - for (ui32 i = 0; i < elementsCount; i++) { - ElementsTypes_.push_back(elementsTypes[i]); - } - } - -private: - ui32 ElementsCount_ = Max<ui32>(); - TSmallVec<const TType*> ElementsTypes_; -}; - -////////////////////////////////////////////////////////////////////////////// -// TDictTypeInspector -////////////////////////////////////////////////////////////////////////////// -class TDictTypeInspector: public TStubTypeVisitor -{ -public: - TDictTypeInspector(const ITypeInfoHelper& typeHelper, const TType* type) { - if (typeHelper.GetTypeKind(type) == ETypeKind::Dict) { - typeHelper.VisitType(type, this); - } - } - - explicit operator bool() const { return KeyType_ != nullptr; } - const TType* GetKeyType() const { return KeyType_; } - const TType* GetValueType() const { return ValueType_; } - -private: - void OnDict(const TType* keyType, const TType* valueType) override { - KeyType_ = keyType; - ValueType_ = valueType; - } - -private: - const TType* KeyType_ = nullptr; - const TType* ValueType_ = nullptr; -}; - -////////////////////////////////////////////////////////////////////////////// -// TCallableTypeInspector -////////////////////////////////////////////////////////////////////////////// -class TCallableTypeInspector: public TStubTypeVisitor -{ -public: - TCallableTypeInspector(const ITypeInfoHelper& typeHelper, const TType* type) { - if (typeHelper.GetTypeKind(type) == ETypeKind::Callable) { - typeHelper.VisitType(type, this); - } - } - - explicit operator bool() const { return ReturnType_ != nullptr; } - const TType* GetReturnType() const { return ReturnType_; } - ui32 GetArgsCount() const { return ArgsCount_; } - const TType* GetArgType(ui32 i) const { return ArgsTypes_[i]; } - ui32 GetOptionalArgsCount() const { return OptionalArgsCount_; } +////////////////////////////////////////////////////////////////////////////// +// TStructTypeInspector +////////////////////////////////////////////////////////////////////////////// +class TStructTypeInspector: public TStubTypeVisitor +{ +public: + TStructTypeInspector(const ITypeInfoHelper& typeHelper, const TType* type) { + if (typeHelper.GetTypeKind(type) == ETypeKind::Struct) { + typeHelper.VisitType(type, this); + } + } + + explicit operator bool() const { return MembersCount_ != Max<ui32>(); } + ui32 GetMembersCount() const { return MembersCount_; } + ui32 GetMemberIndex(TStringRef name) const { + ui32 index = 0; + for (TStringRef memberName: MembersNames_) { + if (memberName == name) { + return index; + } + index++; + } + return Max<ui32>(); + } + const TStringRef& GetMemberName(ui32 i) const { return MembersNames_[i]; } + const TType* GetMemberType(ui32 i) const { return MembersTypes_[i]; } + +private: + void OnStruct( + ui32 membersCount, + TStringRef* membersNames, + const TType** membersTypes) override + { + MembersCount_ = membersCount; + MembersNames_.reserve(membersCount); + MembersTypes_.reserve(membersCount); + + for (ui32 i = 0; i < membersCount; i++) { + MembersNames_.push_back(membersNames[i]); + MembersTypes_.push_back(membersTypes[i]); + } + } + +private: + ui32 MembersCount_ = Max<ui32>(); + TSmallVec<TStringRef> MembersNames_; + TSmallVec<const TType*> MembersTypes_; +}; + +////////////////////////////////////////////////////////////////////////////// +// TListTypeInspector +////////////////////////////////////////////////////////////////////////////// +class TListTypeInspector: public TStubTypeVisitor +{ +public: + TListTypeInspector(const ITypeInfoHelper& typeHelper, const TType* type) { + if (typeHelper.GetTypeKind(type) == ETypeKind::List) { + typeHelper.VisitType(type, this); + } + } + + explicit operator bool() const { return ItemType_ != nullptr; } + const TType* GetItemType() const { return ItemType_; } + +private: + void OnList(const TType* itemType) override { + ItemType_ = itemType; + } + +private: + const TType* ItemType_ = nullptr; +}; + +////////////////////////////////////////////////////////////////////////////// +// TOptionalTypeInspector +////////////////////////////////////////////////////////////////////////////// +class TOptionalTypeInspector: public TStubTypeVisitor +{ +public: + TOptionalTypeInspector(const ITypeInfoHelper& typeHelper, const TType* type) { + if (typeHelper.GetTypeKind(type) == ETypeKind::Optional) { + typeHelper.VisitType(type, this); + } + } + + explicit operator bool() const { return ItemType_ != nullptr; } + const TType* GetItemType() const { return ItemType_; } + +private: + void OnOptional(const TType* itemType) override { + ItemType_ = itemType; + } + +private: + const TType* ItemType_ = nullptr; +}; + +////////////////////////////////////////////////////////////////////////////// +// TTupleTypeInspector +////////////////////////////////////////////////////////////////////////////// +class TTupleTypeInspector: public TStubTypeVisitor +{ +public: + TTupleTypeInspector(const ITypeInfoHelper& typeHelper, const TType* type) { + if (typeHelper.GetTypeKind(type) == ETypeKind::Tuple) { + typeHelper.VisitType(type, this); + } + } + + explicit operator bool() const { return ElementsCount_ != Max<ui32>(); } + ui32 GetElementsCount() const { return ElementsCount_; } + const TType* GetElementType(ui32 i) const { return ElementsTypes_[i]; } + +private: + void OnTuple(ui32 elementsCount, const TType** elementsTypes) override { + ElementsCount_ = elementsCount; + ElementsTypes_.reserve(elementsCount); + + for (ui32 i = 0; i < elementsCount; i++) { + ElementsTypes_.push_back(elementsTypes[i]); + } + } + +private: + ui32 ElementsCount_ = Max<ui32>(); + TSmallVec<const TType*> ElementsTypes_; +}; + +////////////////////////////////////////////////////////////////////////////// +// TDictTypeInspector +////////////////////////////////////////////////////////////////////////////// +class TDictTypeInspector: public TStubTypeVisitor +{ +public: + TDictTypeInspector(const ITypeInfoHelper& typeHelper, const TType* type) { + if (typeHelper.GetTypeKind(type) == ETypeKind::Dict) { + typeHelper.VisitType(type, this); + } + } + + explicit operator bool() const { return KeyType_ != nullptr; } + const TType* GetKeyType() const { return KeyType_; } + const TType* GetValueType() const { return ValueType_; } + +private: + void OnDict(const TType* keyType, const TType* valueType) override { + KeyType_ = keyType; + ValueType_ = valueType; + } + +private: + const TType* KeyType_ = nullptr; + const TType* ValueType_ = nullptr; +}; + +////////////////////////////////////////////////////////////////////////////// +// TCallableTypeInspector +////////////////////////////////////////////////////////////////////////////// +class TCallableTypeInspector: public TStubTypeVisitor +{ +public: + TCallableTypeInspector(const ITypeInfoHelper& typeHelper, const TType* type) { + if (typeHelper.GetTypeKind(type) == ETypeKind::Callable) { + typeHelper.VisitType(type, this); + } + } + + explicit operator bool() const { return ReturnType_ != nullptr; } + const TType* GetReturnType() const { return ReturnType_; } + ui32 GetArgsCount() const { return ArgsCount_; } + const TType* GetArgType(ui32 i) const { return ArgsTypes_[i]; } + ui32 GetOptionalArgsCount() const { return OptionalArgsCount_; } TStringRef GetPayload() const { return HasPayload_ ? Payload_ : TStringRef(); } TStringRef GetArgumentName(ui32 i) const { return HasPayload_ ? ArgsNames_[i] : TStringRef(); } ui64 GetArgumentFlags(ui32 i) const { return HasPayload_ ? ArgsFlags_[i] : 0; } - -private: - void OnCallable( - const TType* returnType, - ui32 argsCount, const TType** argsTypes, + +private: + void OnCallable( + const TType* returnType, + ui32 argsCount, const TType** argsTypes, ui32 optionalArgsCount, const ICallablePayload* payload) override - { - ReturnType_ = returnType; - ArgsCount_ = argsCount; - OptionalArgsCount_ = optionalArgsCount; - ArgsTypes_.reserve(argsCount); - - for (ui32 i = 0; i < argsCount; i++) { - ArgsTypes_.push_back(argsTypes[i]); - } + { + ReturnType_ = returnType; + ArgsCount_ = argsCount; + OptionalArgsCount_ = optionalArgsCount; + ArgsTypes_.reserve(argsCount); + + for (ui32 i = 0; i < argsCount; i++) { + ArgsTypes_.push_back(argsTypes[i]); + } HasPayload_ = (payload != nullptr); if (HasPayload_) { @@ -331,19 +331,19 @@ private: ArgsFlags_.push_back(payload->GetArgumentFlags(i)); } } - } - -private: - const TType* ReturnType_ = nullptr; - ui32 ArgsCount_ = ~0; - TSmallVec<const TType*> ArgsTypes_; - ui32 OptionalArgsCount_ = ~0; + } + +private: + const TType* ReturnType_ = nullptr; + ui32 ArgsCount_ = ~0; + TSmallVec<const TType*> ArgsTypes_; + ui32 OptionalArgsCount_ = ~0; bool HasPayload_ = false; TStringRef Payload_; TSmallVec<TStringRef> ArgsNames_; TSmallVec<ui64> ArgsFlags_; -}; - +}; + ////////////////////////////////////////////////////////////////////////////// // TStreamTypeInspector ////////////////////////////////////////////////////////////////////////////// @@ -521,5 +521,5 @@ inline void TStubTypeVisitor4::OnTagged(const TType*, TStringRef) { } #endif -} // namspace NUdf +} // namspace NUdf } // namspace NYql diff --git a/ydb/library/yql/public/udf/udf_types.h b/ydb/library/yql/public/udf/udf_types.h index 4b072c2708f..c2b67c7cbbc 100644 --- a/ydb/library/yql/public/udf/udf_types.h +++ b/ydb/library/yql/public/udf/udf_types.h @@ -1,17 +1,17 @@ -#pragma once - -#include "udf_ptr.h" +#pragma once + +#include "udf_ptr.h" #include "udf_data_type.h" #include "udf_version.h" - + namespace NYql { -namespace NUdf { - -class TStringRef; - -// opaque type info -using TType = void; - +namespace NUdf { + +class TStringRef; + +// opaque type info +using TType = void; + #if UDF_ABI_COMPATIBILITY_VERSION_CURRENT >= UDF_ABI_COMPATIBILITY_VERSION(2, 21) #define UDF_TYPE_KIND_MAP(XX) \ @@ -69,18 +69,18 @@ using TType = void; #endif enum ETypeKind -{ +{ UDF_TYPE_KIND_MAP(ENUM_VALUE_GEN_NO_VALUE) -}; - +}; + ENUM_TO_STRING(ETypeKind, UDF_TYPE_KIND_MAP) -#define UDF_ASSERT_TYPE_SIZE(type, expectedSize) \ - static_assert(sizeof(type) == (expectedSize), \ - "Size of " #type " exceeds expected size. " \ - "Expected size is " #expectedSize) - -////////////////////////////////////////////////////////////////////////////// +#define UDF_ASSERT_TYPE_SIZE(type, expectedSize) \ + static_assert(sizeof(type) == (expectedSize), \ + "Size of " #type " exceeds expected size. " \ + "Expected size is " #expectedSize) + +////////////////////////////////////////////////////////////////////////////// // ICallablePayload ////////////////////////////////////////////////////////////////////////////// class ICallablePayload @@ -102,29 +102,29 @@ public: UDF_ASSERT_TYPE_SIZE(ICallablePayload, 8); ////////////////////////////////////////////////////////////////////////////// -// ITypeVisitor -////////////////////////////////////////////////////////////////////////////// +// ITypeVisitor +////////////////////////////////////////////////////////////////////////////// class ITypeVisitor1 -{ -public: +{ +public: inline bool IsCompatibleTo(ui16 compatibilityVersion) const { return AbiCompatibility_ >= compatibilityVersion; } virtual ~ITypeVisitor1() = default; - + virtual void OnDataType(TDataTypeId typeId) = 0; - virtual void OnStruct( - ui32 membersCount, - TStringRef* membersNames, - const TType** membersTypes) = 0; - virtual void OnList(const TType* itemType) = 0; - virtual void OnOptional(const TType* itemType) = 0; - virtual void OnTuple(ui32 elementsCount, const TType** elementsTypes) = 0; - virtual void OnDict(const TType* keyType, const TType* valueType) = 0; - virtual void OnCallable( - const TType* returnType, - ui32 argsCount, const TType** argsTypes, + virtual void OnStruct( + ui32 membersCount, + TStringRef* membersNames, + const TType** membersTypes) = 0; + virtual void OnList(const TType* itemType) = 0; + virtual void OnOptional(const TType* itemType) = 0; + virtual void OnTuple(ui32 elementsCount, const TType** elementsTypes) = 0; + virtual void OnDict(const TType* keyType, const TType* valueType) = 0; + virtual void OnCallable( + const TType* returnType, + ui32 argsCount, const TType** argsTypes, ui32 optionalArgsCount, const ICallablePayload* payload) = 0; virtual void OnVariant(const TType* underlyingType) = 0; virtual void OnStream(const TType* itemType) = 0; @@ -137,8 +137,8 @@ private: Y_UNUSED(Reserved1_); Y_UNUSED(Reserved2_); } -}; - +}; + #if UDF_ABI_COMPATIBILITY_VERSION_CURRENT >= UDF_ABI_COMPATIBILITY_VERSION(2, 13) class ITypeVisitor2: public ITypeVisitor1 { public: @@ -171,20 +171,20 @@ using ITypeVisitor = ITypeVisitor1; #endif UDF_ASSERT_TYPE_SIZE(ITypeVisitor, 16); - -////////////////////////////////////////////////////////////////////////////// -// ITypeInfoHelper -////////////////////////////////////////////////////////////////////////////// -class ITypeInfoHelper -{ -public: + +////////////////////////////////////////////////////////////////////////////// +// ITypeInfoHelper +////////////////////////////////////////////////////////////////////////////// +class ITypeInfoHelper +{ +public: using TPtr = TRefCountedPtr<ITypeInfoHelper>; - -public: - virtual ~ITypeInfoHelper() = default; - - virtual ETypeKind GetTypeKind(const TType* type) const = 0; - virtual void VisitType(const TType* type, ITypeVisitor* visitor) const = 0; + +public: + virtual ~ITypeInfoHelper() = default; + + virtual ETypeKind GetTypeKind(const TType* type) const = 0; + virtual void VisitType(const TType* type, ITypeVisitor* visitor) const = 0; virtual bool IsSameType(const TType* type1, const TType* type2) const = 0; // reference counting @@ -210,10 +210,10 @@ private: void UnusedPrivates() { Y_UNUSED(Reserved_); } -}; - +}; + UDF_ASSERT_TYPE_SIZE(ITypeInfoHelper, 16); -UDF_ASSERT_TYPE_SIZE(ITypeInfoHelper::TPtr, 8); - -} // namspace NUdf +UDF_ASSERT_TYPE_SIZE(ITypeInfoHelper::TPtr, 8); + +} // namspace NUdf } // namspace NYql diff --git a/ydb/library/yql/public/udf/udf_ut_helpers.h b/ydb/library/yql/public/udf/udf_ut_helpers.h index daac63fb826..e748ebf8985 100644 --- a/ydb/library/yql/public/udf/udf_ut_helpers.h +++ b/ydb/library/yql/public/udf/udf_ut_helpers.h @@ -82,10 +82,10 @@ public: } private: - bool HasFastListLength() const override { + bool HasFastListLength() const override { return HasLength; - } - + } + ui64 GetListLength() const override { if (HasLength) return Abs(To_ - From_); diff --git a/ydb/library/yql/public/udf/udf_value.h b/ydb/library/yql/public/udf/udf_value.h index b6cffeffea9..705ac411333 100644 --- a/ydb/library/yql/public/udf/udf_value.h +++ b/ydb/library/yql/public/udf/udf_value.h @@ -1,30 +1,30 @@ -#pragma once - +#pragma once + #include "udf_allocator.h" -#include "udf_string.h" +#include "udf_string.h" #include "udf_terminator.h" #include "udf_version.h" - + #include <ydb/library/yql/public/decimal/yql_decimal.h> -#include <util/system/yassert.h> // FAIL, VERIFY_DEBUG -#include <util/generic/utility.h> // Min, Max +#include <util/system/yassert.h> // FAIL, VERIFY_DEBUG +#include <util/generic/utility.h> // Min, Max #include <util/generic/yexception.h> // Y_ENSURE #include <util/system/compiler.h> // Y_FORCE_INLINE #include <algorithm> #include <type_traits> - + class IOutputStream; namespace NYql { -namespace NUdf { - -class TUnboxedValue; +namespace NUdf { + +class TUnboxedValue; class TUnboxedValuePod; class TOpaqueListRepresentation; class IValueBuilder; - + enum class EFetchStatus : ui32 { Ok, Finish, @@ -41,7 +41,7 @@ constexpr ui32 PreferredBlockRows = 65536; constexpr ui32 PreferredBlockBytes = 1000000; #endif -/////////////////////////////////////////////////////////////////////////////// +/////////////////////////////////////////////////////////////////////////////// // IApplyContext /////////////////////////////////////////////////////////////////////////////// class IApplyContext { @@ -61,22 +61,22 @@ typedef bool(*TBlockCallback)(TBlock& block, void* context); #endif /////////////////////////////////////////////////////////////////////////////// -// IBoxedValue -/////////////////////////////////////////////////////////////////////////////// +// IBoxedValue +/////////////////////////////////////////////////////////////////////////////// class IBoxedValue; using IBoxedValuePtr = TRefCountedPtr<IBoxedValue>; class IBoxedValue1 -{ +{ friend struct TBoxedValueAccessor; friend class TBlock; -public: +public: inline bool IsCompatibleTo(ui16 compatibilityVersion) const { return AbiCompatibility_ >= compatibilityVersion; } virtual ~IBoxedValue1() = default; - + private: // List accessors virtual bool HasFastListLength() const = 0; @@ -1111,11 +1111,11 @@ private: TResourceData ResourceData_; }; -#define INCLUDE_UDF_VALUE_INL_H -#include "udf_value_inl.h" -#undef INCLUDE_UDF_VALUE_INL_H - -} // namespace NUdf +#define INCLUDE_UDF_VALUE_INL_H +#include "udf_value_inl.h" +#undef INCLUDE_UDF_VALUE_INL_H + +} // namespace NUdf } // namespace NYql template<> diff --git a/ydb/library/yql/public/udf/udf_value_builder.h b/ydb/library/yql/public/udf/udf_value_builder.h index 092bdead97f..096f735a2b1 100644 --- a/ydb/library/yql/public/udf/udf_value_builder.h +++ b/ydb/library/yql/public/udf/udf_value_builder.h @@ -1,35 +1,35 @@ -#pragma once - -#include "udf_ptr.h" -#include "udf_types.h" +#pragma once + +#include "udf_ptr.h" +#include "udf_types.h" #include "udf_type_builder.h" #include "udf_string.h" #include "udf_value.h" - + #include <array> namespace NYql { -namespace NUdf { - -/////////////////////////////////////////////////////////////////////////////// -// IDictValueBuilder -/////////////////////////////////////////////////////////////////////////////// -struct TDictFlags { - enum EDictKind: ui32 { - Sorted = 0x01, - Hashed = 0x02, - Multi = 0x04, - }; -}; - -class IDictValueBuilder -{ -public: - using TPtr = TUniquePtr<IDictValueBuilder>; - -public: - virtual ~IDictValueBuilder() = default; - +namespace NUdf { + +/////////////////////////////////////////////////////////////////////////////// +// IDictValueBuilder +/////////////////////////////////////////////////////////////////////////////// +struct TDictFlags { + enum EDictKind: ui32 { + Sorted = 0x01, + Hashed = 0x02, + Multi = 0x04, + }; +}; + +class IDictValueBuilder +{ +public: + using TPtr = TUniquePtr<IDictValueBuilder>; + +public: + virtual ~IDictValueBuilder() = default; + virtual IDictValueBuilder& Add(TUnboxedValue&& key, TUnboxedValue&& value) = 0; virtual TUnboxedValue Build() = 0; @@ -37,7 +37,7 @@ public: UDF_ASSERT_TYPE_SIZE(IDictValueBuilder, 8); -/////////////////////////////////////////////////////////////////////////////// +/////////////////////////////////////////////////////////////////////////////// // IDateBuilder /////////////////////////////////////////////////////////////////////////////// class IDateBuilder1 @@ -103,8 +103,8 @@ class IDateBuilder: public IDateBuilder1 {}; UDF_ASSERT_TYPE_SIZE(IDateBuilder, 8); /////////////////////////////////////////////////////////////////////////////// -// IValueBuilder -/////////////////////////////////////////////////////////////////////////////// +// IValueBuilder +/////////////////////////////////////////////////////////////////////////////// class IValueBuilder1 { public: @@ -190,8 +190,8 @@ class IValueBuilder: public IValueBuilder2 {}; class IValueBuilder: public IValueBuilder1 {}; #endif -UDF_ASSERT_TYPE_SIZE(IValueBuilder, 8); - +UDF_ASSERT_TYPE_SIZE(IValueBuilder, 8); + class TPlainArrayCache { private: const ui32 Size; @@ -227,5 +227,5 @@ public: } }; -} // namespace NUdf +} // namespace NUdf } // namespace NYql diff --git a/ydb/library/yql/public/udf/udf_value_inl.h b/ydb/library/yql/public/udf/udf_value_inl.h index 87adc7556b9..eb2db28e18a 100644 --- a/ydb/library/yql/public/udf/udf_value_inl.h +++ b/ydb/library/yql/public/udf/udf_value_inl.h @@ -1,9 +1,9 @@ -#pragma once - -#if !defined(INCLUDE_UDF_VALUE_INL_H) -#error "you should never include udf_value_inl.h directly" -#endif // INCLUDE_UDF_VALUE_INL_H - +#pragma once + +#if !defined(INCLUDE_UDF_VALUE_INL_H) +#error "you should never include udf_value_inl.h directly" +#endif // INCLUDE_UDF_VALUE_INL_H + #ifdef LLVM_BC #define UDF_VERIFY(expr, ...) \ @@ -22,21 +22,21 @@ #define UDF_ALWAYS_INLINE Y_FORCE_INLINE #endif - -////////////////////////////////////////////////////////////////////////////// -// IBoxedValue -////////////////////////////////////////////////////////////////////////////// + +////////////////////////////////////////////////////////////////////////////// +// IBoxedValue +////////////////////////////////////////////////////////////////////////////// inline void IBoxedValue1::Ref() noexcept -{ +{ #if UDF_ABI_COMPATIBILITY_VERSION_CURRENT >= UDF_ABI_COMPATIBILITY_VERSION(2, 4) if (Refs_ < 0) return; #endif ++Refs_; -} - +} + inline void IBoxedValue1::UnRef() noexcept -{ +{ #if UDF_ABI_COMPATIBILITY_VERSION_CURRENT >= UDF_ABI_COMPATIBILITY_VERSION(2, 4) if (Refs_ < 0) return; diff --git a/ydb/library/yql/public/udf/udf_value_ut.cpp b/ydb/library/yql/public/udf/udf_value_ut.cpp index 82a18f5e623..8d9708c7b19 100644 --- a/ydb/library/yql/public/udf/udf_value_ut.cpp +++ b/ydb/library/yql/public/udf/udf_value_ut.cpp @@ -1,51 +1,51 @@ -#include "udf_value.h" +#include "udf_value.h" #include "udf_ut_helpers.h" - + #include <ydb/library/yql/minikql/mkql_alloc.h> #include <library/cpp/testing/unittest/registar.h> - + using namespace NYql::NUdf; - + Y_UNIT_TEST_SUITE(TUdfValue) { - + Y_UNIT_TEST(TestOptional) { TUnboxedValuePod foo((ui32) 42); UNIT_ASSERT(foo); - UNIT_ASSERT(42 == foo.Get<ui32>()); - + UNIT_ASSERT(42 == foo.Get<ui32>()); + auto optFoo = foo.MakeOptional(); UNIT_ASSERT(optFoo); UNIT_ASSERT(optFoo.HasValue()); - + auto bar = optFoo.GetOptionalValue(); - UNIT_ASSERT(42 == bar.Get<ui32>()); - } - + UNIT_ASSERT(42 == bar.Get<ui32>()); + } + Y_UNIT_TEST(TestOptional2) { auto valueOpt = TUnboxedValuePod((ui32) 42); UNIT_ASSERT(valueOpt); UNIT_ASSERT(valueOpt.HasValue()); - + auto value = valueOpt.GetOptionalValue(); - UNIT_ASSERT(42 == value.Get<ui32>()); - } - + UNIT_ASSERT(42 == value.Get<ui32>()); + } + Y_UNIT_TEST(TestEmptyOptional) { auto optEmpty = TUnboxedValuePod(); UNIT_ASSERT(!optEmpty); UNIT_ASSERT(!optEmpty.HasValue()); - + auto optOptEmpty = optEmpty.MakeOptional(); UNIT_ASSERT(optOptEmpty); UNIT_ASSERT(!optOptEmpty.HasValue()); - + auto optOptOptEmpty = optOptEmpty.MakeOptional(); UNIT_ASSERT(optOptOptEmpty); UNIT_ASSERT(!optOptOptEmpty.HasValue()); auto v = optOptEmpty.GetOptionalValue(); UNIT_ASSERT(!v); - } + } Y_UNIT_TEST(TestVariant) { TUnboxedValuePod foo((ui64) 42); @@ -152,4 +152,4 @@ Y_UNIT_TEST_SUITE(TUdfValue) { #endif #undef METHOD_INDEX } -} +} diff --git a/ydb/library/yql/public/udf/udf_version.cpp b/ydb/library/yql/public/udf/udf_version.cpp index e9786d52b58..f900f99b4b7 100644 --- a/ydb/library/yql/public/udf/udf_version.cpp +++ b/ydb/library/yql/public/udf/udf_version.cpp @@ -1,20 +1,20 @@ -#include "udf_version.h" - -#include <util/string/builder.h> - - +#include "udf_version.h" + +#include <util/string/builder.h> + + namespace NYql { -namespace NUdf { - +namespace NUdf { + TString AbiVersionToStr(ui32 version) -{ - TStringBuilder sb; - sb << (version / 10000) << '.' - << (version / 100) % 100 << '.' - << (version % 100); - - return sb; -} - -} // namespace NUdf +{ + TStringBuilder sb; + sb << (version / 10000) << '.' + << (version / 100) % 100 << '.' + << (version % 100); + + return sb; +} + +} // namespace NUdf } // namespace NYql diff --git a/ydb/library/yql/public/udf/udf_version.h b/ydb/library/yql/public/udf/udf_version.h index 8e23e78846c..e31e518d44f 100644 --- a/ydb/library/yql/public/udf/udf_version.h +++ b/ydb/library/yql/public/udf/udf_version.h @@ -1,11 +1,11 @@ -#pragma once - +#pragma once + #include <util/generic/fwd.h> -#include <util/system/types.h> - +#include <util/system/types.h> + namespace NYql { -namespace NUdf { - +namespace NUdf { + #define CURRENT_UDF_ABI_VERSION_MAJOR 2 #define CURRENT_UDF_ABI_VERSION_MINOR 24 #define CURRENT_UDF_ABI_VERSION_PATCH 0 @@ -19,20 +19,20 @@ namespace NUdf { #error Please use UDF_ABI_VERSION macro to define ABI version #endif #endif - -inline const char* CurrentAbiVersionStr() -{ -#define str(s) #s -#define xstr(s) str(s) - - return xstr(UDF_ABI_VERSION_MAJOR) "." - xstr(UDF_ABI_VERSION_MINOR) "." - xstr(UDF_ABI_VERSION_PATCH); - -#undef str -#undef xstr -} - + +inline const char* CurrentAbiVersionStr() +{ +#define str(s) #s +#define xstr(s) str(s) + + return xstr(UDF_ABI_VERSION_MAJOR) "." + xstr(UDF_ABI_VERSION_MINOR) "." + xstr(UDF_ABI_VERSION_PATCH); + +#undef str +#undef xstr +} + #define UDF_ABI_COMPATIBILITY_VERSION(MAJOR, MINOR) ((MAJOR) * 100 + (MINOR)) #define UDF_ABI_COMPATIBILITY_VERSION_CURRENT UDF_ABI_COMPATIBILITY_VERSION(UDF_ABI_VERSION_MAJOR, UDF_ABI_VERSION_MINOR) @@ -42,12 +42,12 @@ static_assert(UDF_ABI_COMPATIBILITY_VERSION_CURRENT <= " is above " Y_STRINGIZE(CURRENT_UDF_ABI_VERSION_MAJOR) "." Y_STRINGIZE(CURRENT_UDF_ABI_VERSION_MINOR)); constexpr ui32 MakeAbiVersion(ui8 major, ui8 minor, ui8 patch) -{ +{ return major * 10000 + minor * 100 + patch; -} - +} + constexpr ui16 MakeAbiCompatibilityVersion(ui8 major, ui8 minor) -{ +{ return major * 100 + minor; } @@ -63,14 +63,14 @@ constexpr ui32 CurrentCompatibilityAbiVersion() constexpr bool IsAbiCompatible(ui32 version) { - // backward compatibility in greater minor versions of host - return version / 10000 == UDF_ABI_VERSION_MAJOR && - (version / 100) % 100 <= UDF_ABI_VERSION_MINOR; -} - + // backward compatibility in greater minor versions of host + return version / 10000 == UDF_ABI_VERSION_MAJOR && + (version / 100) % 100 <= UDF_ABI_VERSION_MINOR; +} + TString AbiVersionToStr(ui32 version); - -} // namspace NUdf + +} // namspace NUdf } // namspace NYql namespace NKikimr { namespace NUdf = ::NYql::NUdf; } diff --git a/ydb/library/yql/public/udf/udfs_exports.symlist b/ydb/library/yql/public/udf/udfs_exports.symlist index 04ca9c2538f..99ebc369f89 100644 --- a/ydb/library/yql/public/udf/udfs_exports.symlist +++ b/ydb/library/yql/public/udf/udfs_exports.symlist @@ -1,4 +1,4 @@ -{ +{ global: Register; AbiVersion; SetBackTraceCallback; BindSymbols; -local: *; -}; +local: *; +}; diff --git a/ydb/library/yql/public/udf/ut/ya.make b/ydb/library/yql/public/udf/ut/ya.make index 99834586498..e51fd164807 100644 --- a/ydb/library/yql/public/udf/ut/ya.make +++ b/ydb/library/yql/public/udf/ut/ya.make @@ -1,17 +1,17 @@ UNITTEST_FOR(ydb/library/yql/public/udf) - + OWNER(g:yql) - -SRCS( + +SRCS( udf_counter_ut.cpp - udf_value_ut.cpp + udf_value_ut.cpp udf_value_builder_ut.cpp -) - +) + YQL_LAST_ABI_VERSION() PEERDIR( ydb/library/yql/public/udf/service/exception_policy ) -END() +END() diff --git a/ydb/library/yql/public/udf/ya.make b/ydb/library/yql/public/udf/ya.make index 0e42aa4a4b6..57a34d00b41 100644 --- a/ydb/library/yql/public/udf/ya.make +++ b/ydb/library/yql/public/udf/ya.make @@ -1,11 +1,11 @@ -LIBRARY() - +LIBRARY() + OWNER( g:yql g:yql_ydb_core ) - -SRCS( + +SRCS( udf_allocator.cpp udf_allocator.h udf_counter.cpp @@ -19,10 +19,10 @@ SRCS( udf_registrator.h udf_static_registry.cpp udf_static_registry.h - udf_string.cpp + udf_string.cpp udf_string.h udf_type_builder.h - udf_type_inspection.cpp + udf_type_inspection.cpp udf_type_inspection.h udf_type_ops.h udf_type_printer.cpp @@ -36,10 +36,10 @@ SRCS( udf_value.h udf_value_builder.h udf_value_inl.h - udf_version.cpp + udf_version.cpp udf_version.h -) - +) + PEERDIR( library/cpp/deprecated/enum_codegen library/cpp/resource @@ -49,7 +49,7 @@ PEERDIR( YQL_LAST_ABI_VERSION() -END() +END() RECURSE_FOR_TESTS( ut diff --git a/ydb/library/yql/sql/v0/aggregation.cpp b/ydb/library/yql/sql/v0/aggregation.cpp index da0defc47d8..64406048811 100644 --- a/ydb/library/yql/sql/v0/aggregation.cpp +++ b/ydb/library/yql/sql/v0/aggregation.cpp @@ -2,7 +2,7 @@ #include "context.h" #include <ydb/library/yql/ast/yql_type_string.h> - + #include <library/cpp/charset/ci_string.h> #include <util/string/builder.h> #include <util/string/cast.h> diff --git a/ydb/library/yql/sql/v0/builtin.cpp b/ydb/library/yql/sql/v0/builtin.cpp index 42ba1c694cb..a0945f9d05e 100644 --- a/ydb/library/yql/sql/v0/builtin.cpp +++ b/ydb/library/yql/sql/v0/builtin.cpp @@ -9,7 +9,7 @@ #include <ydb/library/yql/minikql/mkql_program_builder.h> #include <ydb/library/yql/minikql/mkql_type_ops.h> #include <ydb/library/yql/public/issue/yql_issue_id.h> - + #include <library/cpp/charset/ci_string.h> #include <util/string/builder.h> #include <util/string/cast.h> diff --git a/ydb/library/yql/sql/v0/list_builtin.h b/ydb/library/yql/sql/v0/list_builtin.h index 1539eb6d113..9fc4f22ce62 100644 --- a/ydb/library/yql/sql/v0/list_builtin.h +++ b/ydb/library/yql/sql/v0/list_builtin.h @@ -4,7 +4,7 @@ #include "context.h" #include <ydb/library/yql/ast/yql_type_string.h> - + #include <library/cpp/charset/ci_string.h> #include <util/string/builder.h> #include <util/string/cast.h> diff --git a/ydb/library/yql/sql/v0/node.cpp b/ydb/library/yql/sql/v0/node.cpp index b23e55d7590..57f0c6a561f 100644 --- a/ydb/library/yql/sql/v0/node.cpp +++ b/ydb/library/yql/sql/v0/node.cpp @@ -5,7 +5,7 @@ #include <ydb/library/yql/ast/yql_expr.h> #include <ydb/library/yql/minikql/mkql_type_ops.h> #include <ydb/library/yql/utils/yql_panic.h> - + #include <library/cpp/containers/stack_vector/stack_vec.h> #include <library/cpp/charset/ci_string.h> #include <util/generic/hash_set.h> @@ -1758,10 +1758,10 @@ bool TryStringContent(const TString& str, TString& result, ui32& flags, TString& bool doubleQuoted = (str.StartsWith('"') && str.EndsWith('"')); bool singleQuoted = !doubleQuoted && (str.StartsWith('\'') && str.EndsWith('\'')); - - if (str.size() >= 2 && (doubleQuoted || singleQuoted)) { + + if (str.size() >= 2 && (doubleQuoted || singleQuoted)) { flags = TNodeFlags::ArbitraryContent; - char quoteChar = doubleQuoted ? '"' : '\''; + char quoteChar = doubleQuoted ? '"' : '\''; size_t readBytes = 0; TStringBuf atom(str); TStringOutput sout(result); @@ -1780,8 +1780,8 @@ bool TryStringContent(const TString& str, TString& result, ui32& flags, TString& TString s = str.substr(2, str.length() - 4); SubstGlobal(s, "@@@@", "@@"); result.swap(s); - } else { - flags = TNodeFlags::Default; + } else { + flags = TNodeFlags::Default; result = str; } return true; diff --git a/ydb/library/yql/sql/v1/aggregation.cpp b/ydb/library/yql/sql/v1/aggregation.cpp index e11db30fdc4..850e0e5056e 100644 --- a/ydb/library/yql/sql/v1/aggregation.cpp +++ b/ydb/library/yql/sql/v1/aggregation.cpp @@ -2,7 +2,7 @@ #include "context.h" #include <ydb/library/yql/ast/yql_type_string.h> - + #include <library/cpp/charset/ci_string.h> #include <util/string/builder.h> #include <util/string/cast.h> diff --git a/ydb/library/yql/sql/v1/builtin.cpp b/ydb/library/yql/sql/v1/builtin.cpp index 029e39ec79b..43ef1243f10 100644 --- a/ydb/library/yql/sql/v1/builtin.cpp +++ b/ydb/library/yql/sql/v1/builtin.cpp @@ -10,7 +10,7 @@ #include <ydb/library/yql/minikql/mkql_program_builder.h> #include <ydb/library/yql/minikql/mkql_type_ops.h> #include <ydb/library/yql/public/issue/yql_issue_id.h> - + #include <library/cpp/charset/ci_string.h> #include <library/cpp/yson/node/node_io.h> #include <util/string/builder.h> diff --git a/ydb/library/yql/sql/v1/list_builtin.h b/ydb/library/yql/sql/v1/list_builtin.h index 7837a65c387..a8077b464a2 100644 --- a/ydb/library/yql/sql/v1/list_builtin.h +++ b/ydb/library/yql/sql/v1/list_builtin.h @@ -4,7 +4,7 @@ #include "context.h" #include <ydb/library/yql/ast/yql_type_string.h> - + #include <library/cpp/charset/ci_string.h> #include <util/string/builder.h> #include <util/string/cast.h> diff --git a/ydb/library/yql/sql/v1/node.cpp b/ydb/library/yql/sql/v1/node.cpp index 58acc29a2d7..870bbd9cc7f 100644 --- a/ydb/library/yql/sql/v1/node.cpp +++ b/ydb/library/yql/sql/v1/node.cpp @@ -6,7 +6,7 @@ #include <ydb/library/yql/core/sql_types/simple_types.h> #include <ydb/library/yql/minikql/mkql_type_ops.h> #include <ydb/library/yql/utils/yql_panic.h> - + #include <library/cpp/containers/stack_vector/stack_vec.h> #include <library/cpp/charset/ci_string.h> #include <util/generic/hash_set.h> @@ -2176,8 +2176,8 @@ StringContentInternal(TContext& ctx, TPosition pos, const TString& input, EStrin bool doubleQuoted = (str.StartsWith('"') && str.EndsWith('"')); bool singleQuoted = !doubleQuoted && (str.StartsWith('\'') && str.EndsWith('\'')); - - if (str.size() >= 2 && (doubleQuoted || singleQuoted)) { + + if (str.size() >= 2 && (doubleQuoted || singleQuoted)) { result.Flags = NYql::TNodeFlags::ArbitraryContent; if (ctx.Settings.AnsiLexer) { YQL_ENSURE(singleQuoted); @@ -2194,7 +2194,7 @@ StringContentInternal(TContext& ctx, TPosition pos, const TString& input, EStrin TString s = str.substr(2, str.length() - 4); SubstGlobal(s, "@@@@", "@@"); result.Content.swap(s); - } else { + } else { ctx.Error(pos) << "Invalid string literal: " << EscapeC(str); return {}; } diff --git a/ydb/library/yql/udfs/common/datetime/datetime_udf.cpp b/ydb/library/yql/udfs/common/datetime/datetime_udf.cpp index b82c95a65bc..01567c0b1b1 100644 --- a/ydb/library/yql/udfs/common/datetime/datetime_udf.cpp +++ b/ydb/library/yql/udfs/common/datetime/datetime_udf.cpp @@ -1,24 +1,24 @@ #include <ydb/library/yql/public/udf/udf_helpers.h> - + #include <util/datetime/base.h> #include <util/draft/datetime.h> #include <util/string/cast.h> - + #include <library/cpp/timezone_conversion/convert.h> #include <unordered_map> -using namespace NKikimr; -using namespace NUdf; +using namespace NKikimr; +using namespace NUdf; using namespace NDatetime; - + namespace { SIMPLE_UDF(TToString, char*(TAutoMap<TTimestamp>)) { const auto input = args[0].Get<ui64>(); TInstant instant = TInstant::MicroSeconds(input); return valueBuilder->NewString(instant.ToString()); } - + SIMPLE_UDF(TToStringUpToSeconds, char*(TAutoMap<TTimestamp>)) { const auto input = args[0].Get<ui64>(); TInstant instant = TInstant::MicroSeconds(input); @@ -107,7 +107,7 @@ namespace { if (!success) { \ success = TryStrptime(input, format, result); \ } - + TUnboxedValue FromStringImpl(const TUnboxedValuePod* args) { EMPTY_RESULT_ON_EMPTY_ARG(0); TString input(args[0].AsStringRef()); @@ -226,7 +226,7 @@ namespace { const auto input = args[0].Get<ui64>(); \ return TUnboxedValuePod(TInstant::unit(input).MicroSeconds()); \ } - + #define DATETIME_TIMESTAMP_FROM_UDF(unit) \ SIMPLE_UDF(TTimestampFrom##unit, TOptional<TTimestamp>(TOptional<ui64>)) { \ Y_UNUSED(valueBuilder); \ @@ -317,7 +317,7 @@ namespace { #define DATETIME_GET_REGISTER_UDF(udfName, ...) udfName, #define DATETIME_GET_REGISTER_STRING_UDF(udfName, ...) udfName, #define DATETIME_START_REGISTER_UDF(udfName, ...) udfName, - + #define DATETIME_FROM_UDF_MAP(XX) \ XX(MicroSeconds) \ XX(MilliSeconds) \ @@ -360,7 +360,7 @@ namespace { instant.MilliSecondsOfSecond()) \ XX(TGetMicrosecondOfSecond, ui32, \ instant.MicroSecondsOfSecond()) - + #define DATETIME_GET_UDF_STRING_MAP(XX) \ XX(TGetMonthName, tm.ToString("%B")) \ XX(TGetDayOfWeekName, tm.ToString("%A")) @@ -401,7 +401,7 @@ namespace { DATETIME_GET_UDF_MAP(DATETIME_GET_UDF) DATETIME_GET_UDF_STRING_MAP(DATETIME_GET_STRING_UDF) DATETIME_START_UDF_MAP(DATETIME_START_UDF) - + SIMPLE_MODULE(TDateTimeModule, DATETIME_TO_UDF_MAP(DATETIME_TO_REGISTER_UDF) DATETIME_INTERVAL_TO_UDF_MAP(DATETIME_TO_INTERVAL_REGISTER_UDF) @@ -420,5 +420,5 @@ namespace { TIsWeekend, TDateStartOfDay, TDatetimeStartOfDay, TTimestampStartOfDay, TGetTimeOfDay) } - + REGISTER_MODULES(TDateTimeModule) diff --git a/ydb/library/yql/udfs/common/datetime/ya.make b/ydb/library/yql/udfs/common/datetime/ya.make index 6d353ee358a..400415d407f 100644 --- a/ydb/library/yql/udfs/common/datetime/ya.make +++ b/ydb/library/yql/udfs/common/datetime/ya.make @@ -1,20 +1,20 @@ YQL_UDF(datetime_udf) - + YQL_ABI_VERSION( 2 9 0 ) - + OWNER(g:yql g:yql_ydb_core) -SRCS( +SRCS( datetime_udf.cpp -) - +) + PEERDIR( library/cpp/timezone_conversion util/draft ) -END() +END() diff --git a/ydb/library/yql/udfs/common/digest/digest_udf.cpp b/ydb/library/yql/udfs/common/digest/digest_udf.cpp index 90b763d7c0c..7487a078dde 100644 --- a/ydb/library/yql/udfs/common/digest/digest_udf.cpp +++ b/ydb/library/yql/udfs/common/digest/digest_udf.cpp @@ -1,6 +1,6 @@ #include <ydb/library/yql/public/udf/udf_helpers.h> #include <ydb/library/yql/public/udf/udf_value_builder.h> - + #include <util/digest/murmur.h> #include <util/digest/city.h> #include <util/digest/numeric.h> @@ -12,7 +12,7 @@ #include <library/cpp/digest/md5/md5.h> #include <library/cpp/digest/old_crc/crc.h> #include <library/cpp/digest/sfh/sfh.h> - + #include <contrib/libs/highwayhash/highwayhash/c_bindings.h> #include <contrib/libs/highwayhash/highwayhash/sip_hash.h> @@ -21,9 +21,9 @@ #include <contrib/libs/openssl/include/openssl/sha.h> -using namespace NKikimr; -using namespace NUdf; - +using namespace NKikimr; +using namespace NUdf; + namespace { SIMPLE_UDF(TCrc32c, ui32(TAutoMap<char*>)) { Y_UNUSED(valueBuilder); @@ -31,7 +31,7 @@ namespace { ui32 hash = Crc32c(inputRef.Data(), inputRef.Size()); return TUnboxedValuePod(hash); } - + SIMPLE_UDF(TCrc64, ui64(TAutoMap<char*>)) { Y_UNUSED(valueBuilder); const auto& inputRef = args[0].AsStringRef(); @@ -136,7 +136,7 @@ namespace { const TString& hash = md5.CalcRaw(inputRef); return valueBuilder->NewString(hash); } - + SIMPLE_UDF(TMd5HalfMix, ui64(TAutoMap<char*>)) { Y_UNUSED(valueBuilder); return TUnboxedValuePod(MD5::CalcHalfMix(args[0].AsStringRef())); @@ -381,5 +381,5 @@ namespace { ) } - + REGISTER_MODULES(TDigestModule) diff --git a/ydb/library/yql/udfs/common/digest/ya.make b/ydb/library/yql/udfs/common/digest/ya.make index 52d1ed4311e..53ee51c8d1a 100644 --- a/ydb/library/yql/udfs/common/digest/ya.make +++ b/ydb/library/yql/udfs/common/digest/ya.make @@ -1,5 +1,5 @@ YQL_UDF(digest_udf) - + YQL_ABI_VERSION( 2 9 @@ -7,11 +7,11 @@ YQL_ABI_VERSION( ) OWNER(g:yql g:yql_ydb_core) - -SRCS( + +SRCS( digest_udf.cpp -) - +) + PEERDIR( contrib/libs/farmhash contrib/libs/highwayhash @@ -26,4 +26,4 @@ PEERDIR( ADDINCL(contrib/libs/highwayhash) -END() +END() diff --git a/ydb/library/yql/udfs/common/json/json_udf.cpp b/ydb/library/yql/udfs/common/json/json_udf.cpp index 34804b804bc..e994e589bda 100644 --- a/ydb/library/yql/udfs/common/json/json_udf.cpp +++ b/ydb/library/yql/udfs/common/json/json_udf.cpp @@ -1,10 +1,10 @@ #include <ydb/library/yql/public/udf/udf_helpers.h> - + #include <library/cpp/json/easy_parse/json_easy_parser.h> -using namespace NKikimr; -using namespace NUdf; - +using namespace NKikimr; +using namespace NUdf; + namespace { class TGetField: public TBoxedValue { public: diff --git a/ydb/library/yql/udfs/common/json/ya.make b/ydb/library/yql/udfs/common/json/ya.make index 10e58e2f1f8..e95c3f42e03 100644 --- a/ydb/library/yql/udfs/common/json/ya.make +++ b/ydb/library/yql/udfs/common/json/ya.make @@ -1,19 +1,19 @@ YQL_UDF(json_udf) - + YQL_ABI_VERSION( 2 9 0 ) - + OWNER(g:yql g:yql_ydb_core) -SRCS( +SRCS( json_udf.cpp -) - +) + PEERDIR( library/cpp/json/easy_parse ) -END() +END() diff --git a/ydb/library/yql/udfs/common/math/math_udf.cpp b/ydb/library/yql/udfs/common/math/math_udf.cpp index 0db6ae1e9a6..04aaa29feae 100644 --- a/ydb/library/yql/udfs/common/math/math_udf.cpp +++ b/ydb/library/yql/udfs/common/math/math_udf.cpp @@ -2,7 +2,7 @@ #include <ydb/library/yql/public/udf/udf_helpers.h> - + #define MATH_UDF_MAP(XX, XXL) \ XX(Pi, double(), ;) \ XX(E, double(), ;) \ @@ -46,7 +46,7 @@ XX(Mod, TOptional<i64>(TAutoMap<i64>, i64), ;) \ XX(Rem, TOptional<i64>(TAutoMap<i64>, i64), ;) \ XXL(Round, double(TAutoMap<double>, TPrecision), builder.OptionalArgs(1)) - + #define MATH_UDF_IMPL(name, signature, options) \ UDF_IMPL(T##name, builder.SimpleSignature<signature>(); options;, ;, ;, "/llvm_bc/Math", #name "IR") { \ TUnboxedValuePod res; \ @@ -57,9 +57,9 @@ #define REGISTER_MATH_UDF(udfName, ...) T##udfName, #define REGISTER_MATH_UDF_LAST(udfName, ...) T##udfName -using namespace NKikimr; -using namespace NUdf; - +using namespace NKikimr; +using namespace NUdf; + namespace { extern const char epsilon[] = "Epsilon"; using TEpsilon = TNamedArg<double, epsilon>; diff --git a/ydb/library/yql/udfs/common/math/ya.make b/ydb/library/yql/udfs/common/math/ya.make index b63b160d353..7aaedf9619e 100644 --- a/ydb/library/yql/udfs/common/math/ya.make +++ b/ydb/library/yql/udfs/common/math/ya.make @@ -1,17 +1,17 @@ YQL_UDF(math_udf) - + YQL_ABI_VERSION( 2 16 0 ) - + OWNER(g:yql g:yql_ydb_core) -SRCS( +SRCS( math_udf.cpp -) - +) + LLVM_BC( math_ir.cpp NAME Math @@ -64,4 +64,4 @@ PEERDIR( ydb/library/yql/udfs/common/math/lib ) -END() +END() diff --git a/ydb/library/yql/udfs/common/pire/pire_udf.cpp b/ydb/library/yql/udfs/common/pire/pire_udf.cpp index 48030fd0a9a..95929596839 100644 --- a/ydb/library/yql/udfs/common/pire/pire_udf.cpp +++ b/ydb/library/yql/udfs/common/pire/pire_udf.cpp @@ -3,16 +3,16 @@ #include <ydb/library/yql/public/udf/udf_registrator.h> #include <ydb/library/yql/public/udf/udf_value_builder.h> #include <ydb/library/yql/public/udf/udf_terminator.h> - + #include <library/cpp/regex/pire/regexp.h> #include <library/cpp/regex/pire/pcre2pire.h> - + #include <util/string/builder.h> -using namespace NRegExp; -using namespace NKikimr; -using namespace NUdf; - +using namespace NRegExp; +using namespace NKikimr; +using namespace NUdf; + namespace { class TPireUdfBase: public TBoxedValue { protected: @@ -67,7 +67,7 @@ namespace { bool MultiMode; size_t RegexpsCount; }; - + static const TStringRef& Name(bool surroundMode, bool multiMode) { static auto match = TStringRef::Of("Match"); static auto grep = TStringRef::Of("Grep"); @@ -79,7 +79,7 @@ namespace { return multiMode ? multiMatch : match; } } - + TPireMatch( const IValueBuilder* valueBuilder, const TUnboxedValuePod& runConfig, @@ -119,10 +119,10 @@ namespace { UdfTerminate((TStringBuilder() << Pos_ << " " << e.what()).data()); } } - + private: TUnboxedValue Run( - const IValueBuilder* valueBuilder, + const IValueBuilder* valueBuilder, const TUnboxedValuePod* args) const final try { TUnboxedValue* items = nullptr; TUnboxedValue tuple; @@ -161,14 +161,14 @@ namespace { } catch (const std::exception& e) { UdfTerminate((TStringBuilder() << Pos_ << " " << e.what()).data()); } - + private: TUniquePtr<TFsm> Fsm_; bool MultiMode; size_t RegexpsCount; bool SurroundMode; }; - + class TPireCapture: public TPireUdfBase { public: class TFactory: public TPireUdfBase { @@ -188,8 +188,8 @@ namespace { static const TStringRef& Name() { static auto name = TStringRef::Of("Capture"); return name; - } - + } + TPireCapture(const TUnboxedValuePod& runConfig, TSourcePosition pos) : TPireUdfBase(pos) { @@ -201,14 +201,14 @@ namespace { private: TUnboxedValue Run( - const IValueBuilder* valueBuilder, + const IValueBuilder* valueBuilder, const TUnboxedValuePod* args) const final try { if (args[0]) { const std::string_view input = args[0].AsStringRef(); - + TSlowSearcher searcher(*Fsm_); searcher.Search(input.data(), input.size()); - + if (searcher.Captured()) { const auto& captured = searcher.GetCaptured(); return valueBuilder->SubString(args[0], std::distance(input.begin(), captured.begin()), captured.length()); @@ -218,11 +218,11 @@ namespace { return TUnboxedValue(); } catch (const std::exception& e) { UdfTerminate((TStringBuilder() << Pos_ << " " << e.what()).data()); - } - + } + TUniquePtr<TSlowCapturingFsm> Fsm_; }; - + class TPireReplace: public TPireUdfBase { public: class TFactory: public TPireUdfBase { @@ -298,23 +298,23 @@ namespace { sink.Add(TPireCapture::Name()); sink.Add(TPireReplace::Name()); } - + void BuildFunctionTypeInfo( - const TStringRef& name, + const TStringRef& name, TType*, - const TStringRef& typeConfig, - ui32 flags, + const TStringRef& typeConfig, + ui32 flags, IFunctionTypeInfoBuilder& builder) const final try { const bool typesOnly = (flags & TFlags::TypesOnly); const bool isMatch = (TPireMatch::Name(false, false) == name); const bool isGrep = (TPireMatch::Name(true, false) == name); const bool isMultiMatch = (TPireMatch::Name(false, true) == name); const bool isMultiGrep = (TPireMatch::Name(true, true) == name); - + if (isMatch || isGrep) { builder.SimpleSignature<bool(TOptional<char*>)>() .RunConfig<const char*>(); - + if (!typesOnly) { builder.Implementation(new TPireMatch::TFactory(isGrep, false, builder.GetSourcePosition())); } @@ -329,7 +329,7 @@ namespace { } const auto tupleType = tuple->Build(); builder.Args(1)->Add(optionalStringType).Done().Returns(tupleType).RunConfig<char*>(); - + if (!typesOnly) { builder.Implementation(new TPireMatch::TFactory(isMultiGrep, true, builder.GetSourcePosition(), regexpCount)); } @@ -343,16 +343,16 @@ namespace { } else if (TPireReplace::Name() == name) { builder.SimpleSignature<TOptional<char*>(TOptional<char*>, char*)>() .RunConfig<char*>(); - + if (!typesOnly) { builder.Implementation(new TPireReplace::TFactory(builder.GetSourcePosition())); } } } catch (const std::exception& e) { builder.SetError(CurrentExceptionMessage()); - } + } }; - + } - + REGISTER_MODULES(TPireModule) diff --git a/ydb/library/yql/udfs/common/pire/ya.make b/ydb/library/yql/udfs/common/pire/ya.make index 5b5a1cb5fc7..ba65dd6d107 100644 --- a/ydb/library/yql/udfs/common/pire/ya.make +++ b/ydb/library/yql/udfs/common/pire/ya.make @@ -1,5 +1,5 @@ YQL_UDF(pire_udf) - + YQL_ABI_VERSION( 2 23 @@ -7,13 +7,13 @@ YQL_ABI_VERSION( ) OWNER(g:yql g:yql_ydb_core) - -SRCS( + +SRCS( pire_udf.cpp -) - -PEERDIR( +) + +PEERDIR( library/cpp/regex/pire -) - -END() +) + +END() diff --git a/ydb/library/yql/udfs/common/re2/re2_udf.cpp b/ydb/library/yql/udfs/common/re2/re2_udf.cpp index 8769cb4db7d..b478675fb60 100644 --- a/ydb/library/yql/udfs/common/re2/re2_udf.cpp +++ b/ydb/library/yql/udfs/common/re2/re2_udf.cpp @@ -1,17 +1,17 @@ #include <ydb/library/yql/public/udf/udf_helpers.h> #include <ydb/library/yql/public/udf/udf_value_builder.h> - + #include <contrib/libs/re2/re2/re2.h> - + #include <util/charset/utf8.h> #include <util/string/cast.h> using namespace re2; -using namespace NKikimr; -using namespace NUdf; - +using namespace NKikimr; +using namespace NUdf; + namespace { - + template <typename T> T Id(T x) { return x; @@ -122,7 +122,7 @@ namespace { } Y_FAIL("Unexpected mode"); } - + TRe2Udf( const IValueBuilder*, const TUnboxedValuePod& runConfig, @@ -166,10 +166,10 @@ namespace { UdfTerminate((TStringBuilder() << Pos_ << " " << e.what()).data()); } } - + private: TUnboxedValue Run( - const IValueBuilder* valueBuilder, + const IValueBuilder* valueBuilder, const TUnboxedValuePod* args) const final try { RE2::Anchor anchor = RE2::UNANCHORED; if (args[0]) { @@ -241,7 +241,7 @@ namespace { } } catch (const std::exception& e) { UdfTerminate((TStringBuilder() << Pos_ << " " << e.what()).data()); - } + } std::unique_ptr<RE2> Regexp; const TRegexpGroups RegexpGroups; @@ -255,7 +255,7 @@ namespace { return valueBuilder->NewArray(RegexpGroups.Names.size(), items); } }; - + SIMPLE_UDF(TEscape, char*(char*)) { const std::string_view input(args[0].AsStringRef()); const auto& result = RE2::QuoteMeta(StringPiece(input.data(), input.size())); @@ -420,17 +420,17 @@ namespace { sink.Add(TPatternFromLike::Name()); sink.Add(TOptions::Name()); } - + void BuildFunctionTypeInfo( - const TStringRef& name, + const TStringRef& name, TType* userType, - const TStringRef& typeConfig, - ui32 flags, + const TStringRef& typeConfig, + ui32 flags, IFunctionTypeInfoBuilder& builder) const final try { Y_UNUSED(userType); TOptionsSchema optionsSchema = MakeOptionsSchema(builder); auto optOptionsStructType = builder.Optional()->Item(optionsSchema.StructType).Build(); - + bool typesOnly = (flags & TFlags::TypesOnly); bool isMatch = (TRe2Udf::Name(TRe2Udf::EMode::MATCH) == name); bool isGrep = (TRe2Udf::Name(TRe2Udf::EMode::GREP) == name); @@ -438,11 +438,11 @@ namespace { bool isReplace = (TRe2Udf::Name(TRe2Udf::EMode::REPLACE) == name); bool isCount = (TRe2Udf::Name(TRe2Udf::EMode::COUNT) == name); bool isFindAndConsume = (TRe2Udf::Name(TRe2Udf::FIND_AND_CONSUME) == name); - + if (isMatch || isGrep) { builder.SimpleSignature<bool(TOptional<char*>)>() .RunConfig(MakeRunConfigType(builder, optOptionsStructType)); - + if (!typesOnly) { const auto mode = isMatch ? TRe2Udf::EMode::MATCH : TRe2Udf::EMode::GREP; builder.Implementation(new TRe2Udf::TFactory<posix>(mode, optionsSchema, builder.GetSourcePosition())); @@ -514,11 +514,11 @@ namespace { } } catch (const std::exception& e) { builder.SetError(CurrentExceptionMessage()); - } + } }; - + } - + REGISTER_MODULES( TRe2Module<false>, TRe2Module<true>) diff --git a/ydb/library/yql/udfs/common/re2/ya.make b/ydb/library/yql/udfs/common/re2/ya.make index c82dc74e156..64d14c652b5 100644 --- a/ydb/library/yql/udfs/common/re2/ya.make +++ b/ydb/library/yql/udfs/common/re2/ya.make @@ -1,20 +1,20 @@ YQL_UDF(re2_udf) - + YQL_ABI_VERSION( 2 9 0 ) - + OWNER(g:yql g:yql_ydb_core) -SRCS( +SRCS( re2_udf.cpp -) - -PEERDIR( +) + +PEERDIR( contrib/libs/re2 library/cpp/deprecated/enum_codegen -) - -END() +) + +END() diff --git a/ydb/library/yql/udfs/common/string/string_udf.cpp b/ydb/library/yql/udfs/common/string/string_udf.cpp index e682c9b5cc0..c468215090b 100644 --- a/ydb/library/yql/udfs/common/string/string_udf.cpp +++ b/ydb/library/yql/udfs/common/string/string_udf.cpp @@ -1,7 +1,7 @@ #include <ydb/library/yql/public/udf/udf_allocator.h> #include <ydb/library/yql/public/udf/udf_helpers.h> #include <ydb/library/yql/public/udf/udf_value_builder.h> - + #include <library/cpp/charset/codepage.h> #include <library/cpp/deprecated/split/split_iterator.h> #include <library/cpp/html/pcdata/pcdata.h> @@ -23,9 +23,9 @@ #include <util/string/util.h> #include <util/string/vector.h> -using namespace NKikimr; -using namespace NUdf; - +using namespace NKikimr; +using namespace NUdf; + namespace { #define STRING_UDF(udfName, function) \ SIMPLE_UDF(T##udfName, char*(TAutoMap<char*>)) { \ @@ -141,7 +141,7 @@ namespace { XX(Base64Decode, Base64Decode) \ XX(Base64StrictDecode, Base64StrictDecode) \ XX(HexDecode, HexDecode) - + #define STROKA_CASE_UDF_MAP(XX) \ XX(ToLower, ToLower) \ XX(ToUpper, ToUpper) \ diff --git a/ydb/library/yql/udfs/common/string/ya.make b/ydb/library/yql/udfs/common/string/ya.make index ed79b0e95c0..ca6da4e5865 100644 --- a/ydb/library/yql/udfs/common/string/ya.make +++ b/ydb/library/yql/udfs/common/string/ya.make @@ -1,17 +1,17 @@ YQL_UDF(string_udf) - + YQL_ABI_VERSION( 2 9 0 ) - + OWNER(g:yql g:yql_ydb_core) -SRCS( +SRCS( string_udf.cpp -) - +) + PEERDIR( library/cpp/charset library/cpp/deprecated/split @@ -25,4 +25,4 @@ TIMEOUT(300) SIZE(MEDIUM) -END() +END() diff --git a/ydb/library/yql/udfs/common/unicode_base/ya.make b/ydb/library/yql/udfs/common/unicode_base/ya.make index 921157e11fd..ad722ed0ee8 100644 --- a/ydb/library/yql/udfs/common/unicode_base/ya.make +++ b/ydb/library/yql/udfs/common/unicode_base/ya.make @@ -1,19 +1,19 @@ YQL_UDF(unicode_udf) - + YQL_ABI_VERSION( 2 9 0 ) - + OWNER(g:yql g:yql_ydb_core) -SRCS( +SRCS( unicode_base.cpp -) - +) + PEERDIR( ydb/library/yql/udfs/common/unicode_base/lib ) -END() +END() diff --git a/ydb/library/yql/udfs/common/url_base/ya.make b/ydb/library/yql/udfs/common/url_base/ya.make index d12c09b9afd..9a2117fb3cb 100644 --- a/ydb/library/yql/udfs/common/url_base/ya.make +++ b/ydb/library/yql/udfs/common/url_base/ya.make @@ -1,23 +1,23 @@ YQL_UDF(url_udf) - + YQL_ABI_VERSION( 2 23 0 ) - + OWNER( g:yql g:yql_ydb_core ) -SRCS( +SRCS( url_base.cpp -) - +) + PEERDIR( ydb/library/yql/public/udf ydb/library/yql/udfs/common/url_base/lib ) -END() +END() diff --git a/ydb/library/yql/udfs/logs/dsv/dsv_udf.cpp b/ydb/library/yql/udfs/logs/dsv/dsv_udf.cpp index 2d669e99a9d..b74bf80b185 100644 --- a/ydb/library/yql/udfs/logs/dsv/dsv_udf.cpp +++ b/ydb/library/yql/udfs/logs/dsv/dsv_udf.cpp @@ -96,7 +96,7 @@ private: auto keyData = args[0].GetElement(KsvIndexes_.key); auto subkeyData = args[0].GetElement(KsvIndexes_.subkey); auto valueData = args[0].GetElement(KsvIndexes_.value); - + auto dict = valueBuilder->NewDict(ResultIndexes_.DictType, 0); @@ -213,13 +213,13 @@ public: const TStringRef& name, TType* userType, const TStringRef& typeConfig, - ui32 flags, + ui32 flags, IFunctionTypeInfoBuilder& builder) const final try { Y_UNUSED(typeConfig); bool typesOnly = (flags & TFlags::TypesOnly); - + if (TStringRef::Of("ReadRecord") == name) { TKsvIndexes ksvIndexes; auto recordType = builder.Struct(3U)-> @@ -235,11 +235,11 @@ public: .AddField<char*>("subkey", &resultIndexes.subkey) .AddField("dict", resultIndexes.DictType, &resultIndexes.dict) .Build(); - + builder.Returns(structType) .Args()->Add(recordType).Done() .RunConfig<TOptional<char*>>(); - + if (!typesOnly) { builder.Implementation(new TDsvReadRecord::TFactory( resultIndexes, ksvIndexes)); diff --git a/ydb/library/yql/utils/backtrace/backtrace.cpp b/ydb/library/yql/utils/backtrace/backtrace.cpp index c97bddb4a2a..f9487f66717 100644 --- a/ydb/library/yql/utils/backtrace/backtrace.cpp +++ b/ydb/library/yql/utils/backtrace/backtrace.cpp @@ -1,5 +1,5 @@ #include "backtrace.h" - + #include <llvm/DebugInfo/Symbolize/Symbolize.h> #include <llvm/DebugInfo/Symbolize/DIPrinter.h> #include <llvm/Support/raw_ostream.h> @@ -10,22 +10,22 @@ #include <util/generic/xrange.h> #include <util/generic/yexception.h> #include <util/stream/format.h> -#include <util/stream/output.h> -#include <util/system/backtrace.h> +#include <util/stream/output.h> +#include <util/system/backtrace.h> #include <util/system/type_name.h> #include <util/system/execpath.h> #include <util/system/platform.h> #include <util/system/mlock.h> - + #ifdef _linux_ #include <dlfcn.h> #include <link.h> #endif #ifndef _win_ - -namespace { - + +namespace { + bool SetSignalHandler(int signo, void (*handler)(int)) { struct sigaction sa; memset(&sa, 0, sizeof(sa)); @@ -34,11 +34,11 @@ bool SetSignalHandler(int signo, void (*handler)(int)) { sigfillset(&sa.sa_mask); return sigaction(signo, &sa, nullptr) != -1; } - -} // namespace - + +} // namespace + #endif // _win_ - + void KikimrBackTrace() { KikimrBacktraceFormatImpl(&Cerr); } @@ -146,31 +146,31 @@ void KikimrBacktraceFormatImpl(IOutputStream* out, void* const* stack, size_t st } void PrintBacktraceToStderr(int signum) -{ +{ if (!NMalloc::IsAllocatorCorrupted) { /* we want memory allocation for backtrace printing */ KikimrBacktraceFormatImpl(&Cerr); } - /* Now reraise the signal. We reactivate the signal’s default handling, - which is to terminate the process. We could just call exit or abort, - but reraising the signal sets the return status from the process - correctly. */ - raise(signum); -} - -void EnableKikimrBacktraceFormat() -{ + /* Now reraise the signal. We reactivate the signal’s default handling, + which is to terminate the process. We could just call exit or abort, + but reraising the signal sets the return status from the process + correctly. */ + raise(signum); +} + +void EnableKikimrBacktraceFormat() +{ SetFormatBackTraceFn(KikimrBacktraceFormatImpl); -} - +} + void SetFatalSignalHandler(void (*handler)(int)) -{ +{ Y_UNUSED(handler); -#ifndef _win_ +#ifndef _win_ for (int signo: {SIGSEGV, SIGILL, SIGABRT, SIGFPE}) { if (!SetSignalHandler(signo, handler)) { ythrow TSystemError() << "Cannot set handler for signal " << strsignal(signo); } - } -#endif -} + } +#endif +} diff --git a/ydb/library/yql/utils/backtrace/backtrace.h b/ydb/library/yql/utils/backtrace/backtrace.h index d218db21f35..487547e63b8 100644 --- a/ydb/library/yql/utils/backtrace/backtrace.h +++ b/ydb/library/yql/utils/backtrace/backtrace.h @@ -1,13 +1,13 @@ -#pragma once - +#pragma once + #include <util/system/types.h> class IOutputStream; - + void KikimrBacktraceFormatImpl(IOutputStream* out); void KikimrBacktraceFormatImpl(IOutputStream* out, void* const* stack, size_t stackSize); void KikimrBackTrace(); -void EnableKikimrBacktraceFormat(); +void EnableKikimrBacktraceFormat(); void PrintBacktraceToStderr(int signum); void SetFatalSignalHandler(void (*handler)(int)); diff --git a/ydb/library/yql/utils/backtrace/ya.make b/ydb/library/yql/utils/backtrace/ya.make index 0dbde943363..990bc74cc55 100644 --- a/ydb/library/yql/utils/backtrace/ya.make +++ b/ydb/library/yql/utils/backtrace/ya.make @@ -1,17 +1,17 @@ -LIBRARY() - +LIBRARY() + OWNER( g:kikimr g:yql g:yql_ydb_core ) - -SRCS( - backtrace.cpp -) - + +SRCS( + backtrace.cpp +) + PEERDIR( contrib/libs/llvm12/lib/DebugInfo/Symbolize ) -END() +END() diff --git a/ydb/library/yql/utils/debug_info.cpp b/ydb/library/yql/utils/debug_info.cpp index c90e334fd93..ff56166695e 100644 --- a/ydb/library/yql/utils/debug_info.cpp +++ b/ydb/library/yql/utils/debug_info.cpp @@ -1,50 +1,50 @@ -#include "debug_info.h" - -#include <util/system/thread.h> -#include <util/system/tls.h> +#include "debug_info.h" + +#include <util/system/thread.h> +#include <util/system/tls.h> #include <util/stream/file.h> #include <util/generic/string.h> - + #include <string.h> - - -namespace NYql { - -static const size_t OPERATION_ID_MAX_LENGTH = 24; -static const size_t THREAD_NAME_MAX_LENGTH = 16; - - -struct TDebugInfo { - char OperationId[OPERATION_ID_MAX_LENGTH + 1]; -}; - -Y_POD_THREAD(TDebugInfo) TlsDebugInfo; - - -void SetCurrentOperationId(const char* operationId) { - size_t len = strlcpy( - (&TlsDebugInfo)->OperationId, - operationId, - OPERATION_ID_MAX_LENGTH); - - const char* threadName = nullptr; - if (len > THREAD_NAME_MAX_LENGTH) { - threadName = operationId + (len - THREAD_NAME_MAX_LENGTH + 1); + + +namespace NYql { + +static const size_t OPERATION_ID_MAX_LENGTH = 24; +static const size_t THREAD_NAME_MAX_LENGTH = 16; + + +struct TDebugInfo { + char OperationId[OPERATION_ID_MAX_LENGTH + 1]; +}; + +Y_POD_THREAD(TDebugInfo) TlsDebugInfo; + + +void SetCurrentOperationId(const char* operationId) { + size_t len = strlcpy( + (&TlsDebugInfo)->OperationId, + operationId, + OPERATION_ID_MAX_LENGTH); + + const char* threadName = nullptr; + if (len > THREAD_NAME_MAX_LENGTH) { + threadName = operationId + (len - THREAD_NAME_MAX_LENGTH + 1); } else { threadName = operationId; - } + } TThread::SetCurrentThreadName(threadName); -} - -long GetRunnigThreadsCount() { +} + +long GetRunnigThreadsCount() { TString procStat = TFileInput("/proc/self/stat").ReadAll(); long num_threads = -2; // Number of threads in this process (since Linux 2.6) - + int n = sscanf(procStat.data(), "%*d %*s %*c %*d %*d %*d %*d %*d %*u %*u %*u %*u %*u %*u %*u %*d %*d %*d %*d %ld", &num_threads); - + return n == 1 ? num_threads : -2; -} - -} // namespace NYql +} + +} // namespace NYql diff --git a/ydb/library/yql/utils/debug_info.h b/ydb/library/yql/utils/debug_info.h index 2ea55155a7a..3e2a55140b1 100644 --- a/ydb/library/yql/utils/debug_info.h +++ b/ydb/library/yql/utils/debug_info.h @@ -1,10 +1,10 @@ -#pragma once - - -namespace NYql { - -void SetCurrentOperationId(const char* operationId); - -long GetRunnigThreadsCount(); - -} // namespace NYql +#pragma once + + +namespace NYql { + +void SetCurrentOperationId(const char* operationId); + +long GetRunnigThreadsCount(); + +} // namespace NYql diff --git a/ydb/library/yql/utils/log/context.cpp b/ydb/library/yql/utils/log/context.cpp index 3e7c128bfd8..b591c42a779 100644 --- a/ydb/library/yql/utils/log/context.cpp +++ b/ydb/library/yql/utils/log/context.cpp @@ -1,85 +1,85 @@ #include "context.h" -#include "log.h" +#include "log.h" + +#include <util/thread/singleton.h> -#include <util/thread/singleton.h> - namespace NYql { -namespace NLog { -namespace { +namespace NLog { +namespace { -struct TThrowedLogContext { +struct TThrowedLogContext { TString LocationWithLogContext; // separated with ': ' -}; - +}; + } // namspace void OutputLogCtx(IOutputStream* out, bool withBraces) { - const NImpl::TLogContextListItem* ctxList = NImpl::GetLogContextList(); - - if (ctxList->HasNext()) { - if (withBraces) { - (*out) << '{'; - } - - // skip header stub element - NImpl::TLogContextListItem* ctxItem = ctxList->Next; - - { // special case for outputing the first element + const NImpl::TLogContextListItem* ctxList = NImpl::GetLogContextList(); + + if (ctxList->HasNext()) { + if (withBraces) { + (*out) << '{'; + } + + // skip header stub element + NImpl::TLogContextListItem* ctxItem = ctxList->Next; + + { // special case for outputing the first element const TString* it = ctxItem->begin(); const TString* end = ctxItem->end(); - - (*out) << *it++; - for (; it != end; ++it) { - (*out) << '/' << *it; - } - - ctxItem = ctxItem->Next; - } - - // output remaining elements - while (ctxItem != ctxList) { + + (*out) << *it++; + for (; it != end; ++it) { + (*out) << '/' << *it; + } + + ctxItem = ctxItem->Next; + } + + // output remaining elements + while (ctxItem != ctxList) { for (const TString& name: *ctxItem) { - (*out) << '/' << name; - } - ctxItem = ctxItem->Next; - } - - if (withBraces) { + (*out) << '/' << name; + } + ctxItem = ctxItem->Next; + } + + if (withBraces) { (*out) << TStringBuf("} "); - } - } + } + } } -NImpl::TLogContextListItem* NImpl::GetLogContextList() { - return FastTlsSingleton<NImpl::TLogContextListItem>(); +NImpl::TLogContextListItem* NImpl::GetLogContextList() { + return FastTlsSingleton<NImpl::TLogContextListItem>(); } TString CurrentLogContextPath() { - TStringStream ss; - OutputLogCtx(&ss, false); - return ss.Str(); -} - + TStringStream ss; + OutputLogCtx(&ss, false); + return ss.Str(); +} + TString ThrowedLogContextPath() { - TThrowedLogContext* tlc = FastTlsSingleton<TThrowedLogContext>(); - return std::move(tlc->LocationWithLogContext); -} - -TAutoPtr<TLogElement> TContextPreprocessor::Preprocess( - TAutoPtr<TLogElement> element) -{ - OutputLogCtx(element.Get(), true); - return element; + TThrowedLogContext* tlc = FastTlsSingleton<TThrowedLogContext>(); + return std::move(tlc->LocationWithLogContext); +} + +TAutoPtr<TLogElement> TContextPreprocessor::Preprocess( + TAutoPtr<TLogElement> element) +{ + OutputLogCtx(element.Get(), true); + return element; } -void TYqlLogContextLocation::SetThrowedLogContextPath() const { - TStringStream ss; +void TYqlLogContextLocation::SetThrowedLogContextPath() const { + TStringStream ss; ss << Location_ << TStringBuf(": "); - OutputLogCtx(&ss, true); - TThrowedLogContext* tlc = FastTlsSingleton<TThrowedLogContext>(); - tlc->LocationWithLogContext = ss.Str(); + OutputLogCtx(&ss, true); + TThrowedLogContext* tlc = FastTlsSingleton<TThrowedLogContext>(); + tlc->LocationWithLogContext = ss.Str(); } -} // namespace NLog +} // namespace NLog } // namespace NYql diff --git a/ydb/library/yql/utils/log/context.h b/ydb/library/yql/utils/log/context.h index 8f9dd3caf66..ab6e9430e50 100644 --- a/ydb/library/yql/utils/log/context.h +++ b/ydb/library/yql/utils/log/context.h @@ -1,219 +1,219 @@ #pragma once #include <util/system/defaults.h> -#include <util/generic/strbuf.h> -#include <util/stream/output.h> -#include <util/system/src_location.h> -#include <util/system/yassert.h> - - -// continues existing contexts chain - -#define YQL_LOG_CTX_SCOPE(...) \ - auto Y_CAT(c, __LINE__) = ::NYql::NLog::MakeCtx(__VA_ARGS__); \ - Y_UNUSED(Y_CAT(c, __LINE__)) - -#define YQL_LOG_CTX_BLOCK(...) \ - if (auto Y_GENERATE_UNIQUE_ID(c) = ::NYql::NLog::MakeCtx(__VA_ARGS__)) { \ - goto Y_CAT(YQL_LOG_CTX_LABEL, __LINE__); \ - } else Y_CAT(YQL_LOG_CTX_LABEL, __LINE__): - - -// starts new contexts chain, after leaving current scope restores -// previous contexts chain - -#define YQL_LOG_CTX_ROOT_SCOPE(...) \ - auto Y_CAT(c, __LINE__) = ::NYql::NLog::MakeRootCtx(__VA_ARGS__); \ - Y_UNUSED(Y_CAT(c, __LINE__)) - -#define YQL_LOG_CTX_ROOT_BLOCK(...) \ - if (auto Y_GENERATE_UNIQUE_ID(c) = ::NYql::NLog::MakeRootCtx(__VA_ARGS__)) { \ - goto Y_CAT(YQL_LOG_CTX_LABEL, __LINE__); \ - } else Y_CAT(YQL_LOG_CTX_LABEL, __LINE__): - - -// adds current contexts path to exception message before throwing it - -#define YQL_LOG_CTX_THROW throw ::NYql::NLog::TYqlLogContextLocation(__LOCATION__) + - -class TLogElement; - +#include <util/generic/strbuf.h> +#include <util/stream/output.h> +#include <util/system/src_location.h> +#include <util/system/yassert.h> + + +// continues existing contexts chain + +#define YQL_LOG_CTX_SCOPE(...) \ + auto Y_CAT(c, __LINE__) = ::NYql::NLog::MakeCtx(__VA_ARGS__); \ + Y_UNUSED(Y_CAT(c, __LINE__)) + +#define YQL_LOG_CTX_BLOCK(...) \ + if (auto Y_GENERATE_UNIQUE_ID(c) = ::NYql::NLog::MakeCtx(__VA_ARGS__)) { \ + goto Y_CAT(YQL_LOG_CTX_LABEL, __LINE__); \ + } else Y_CAT(YQL_LOG_CTX_LABEL, __LINE__): + + +// starts new contexts chain, after leaving current scope restores +// previous contexts chain + +#define YQL_LOG_CTX_ROOT_SCOPE(...) \ + auto Y_CAT(c, __LINE__) = ::NYql::NLog::MakeRootCtx(__VA_ARGS__); \ + Y_UNUSED(Y_CAT(c, __LINE__)) + +#define YQL_LOG_CTX_ROOT_BLOCK(...) \ + if (auto Y_GENERATE_UNIQUE_ID(c) = ::NYql::NLog::MakeRootCtx(__VA_ARGS__)) { \ + goto Y_CAT(YQL_LOG_CTX_LABEL, __LINE__); \ + } else Y_CAT(YQL_LOG_CTX_LABEL, __LINE__): + + +// adds current contexts path to exception message before throwing it + +#define YQL_LOG_CTX_THROW throw ::NYql::NLog::TYqlLogContextLocation(__LOCATION__) + + +class TLogElement; + namespace NYql { -namespace NLog { -namespace NImpl { - -/** - * @brief Represents item of logging context list. - */ -struct TLogContextListItem { - TLogContextListItem* Next; - TLogContextListItem* Prev; - size_t NamesCount; - - explicit TLogContextListItem(size_t namesCount = 0) - : Next(this) - , Prev(this) - , NamesCount(namesCount) - { - } - +namespace NLog { +namespace NImpl { + +/** + * @brief Represents item of logging context list. + */ +struct TLogContextListItem { + TLogContextListItem* Next; + TLogContextListItem* Prev; + size_t NamesCount; + + explicit TLogContextListItem(size_t namesCount = 0) + : Next(this) + , Prev(this) + , NamesCount(namesCount) + { + } + const TString* begin() const { - auto* ptr = reinterpret_cast<const ui8*>(this); + auto* ptr = reinterpret_cast<const ui8*>(this); return reinterpret_cast<const TString*>(ptr + sizeof(*this)); - } - + } + const TString* end() const { - return begin() + NamesCount; - } - - bool HasNext() const { - return Next != this; - } - - void LinkBefore(TLogContextListItem* item) { - Y_VERIFY_DEBUG(!HasNext()); - Next = item; - Prev = item->Prev; - Prev->Next = this; - Next->Prev = this; - } - - void Unlink() { - if (!HasNext()) return; - - Prev->Next = Next; - Next->Prev = Prev; - Next = Prev = this; - } -}; - -/** - * @brief Returns pointer to thread local log context list. - */ -TLogContextListItem* GetLogContextList(); - -} // namspace NImpl - -/** - * @brief YQL logger context element. Each element can contains several names. - */ -template <size_t Size> -class TLogContext: public NImpl::TLogContextListItem { + return begin() + NamesCount; + } + + bool HasNext() const { + return Next != this; + } + + void LinkBefore(TLogContextListItem* item) { + Y_VERIFY_DEBUG(!HasNext()); + Next = item; + Prev = item->Prev; + Prev->Next = this; + Next->Prev = this; + } + + void Unlink() { + if (!HasNext()) return; + + Prev->Next = Next; + Next->Prev = Prev; + Next = Prev = this; + } +}; + +/** + * @brief Returns pointer to thread local log context list. + */ +TLogContextListItem* GetLogContextList(); + +} // namspace NImpl + +/** + * @brief YQL logger context element. Each element can contains several names. + */ +template <size_t Size> +class TLogContext: public NImpl::TLogContextListItem { public: - template <typename... TArgs> - TLogContext(TArgs... args) - : TLogContextListItem(Size) + template <typename... TArgs> + TLogContext(TArgs... args) + : TLogContextListItem(Size) , Names_{{ TString{std::forward<TArgs>(args)}... }} - { - LinkBefore(NImpl::GetLogContextList()); - } - - ~TLogContext() { - Unlink(); - } - + { + LinkBefore(NImpl::GetLogContextList()); + } + + ~TLogContext() { + Unlink(); + } + explicit inline operator bool() const noexcept { return true; } - + private: std::array<TString, Size> Names_; }; -/** - * @brief Special Root context elements which replaces previous log context - * list head by itself and restores previous one on destruction. - */ -template <size_t Size> -class TRootLogContext: public NImpl::TLogContextListItem { -public: - template <typename... TArgs> - TRootLogContext(TArgs... args) - : TLogContextListItem(Size) +/** + * @brief Special Root context elements which replaces previous log context + * list head by itself and restores previous one on destruction. + */ +template <size_t Size> +class TRootLogContext: public NImpl::TLogContextListItem { +public: + template <typename... TArgs> + TRootLogContext(TArgs... args) + : TLogContextListItem(Size) , Names_{{ TString{std::forward<TArgs>(args)}... }} - { - NImpl::TLogContextListItem* ctxList = NImpl::GetLogContextList(); - PrevLogContextHead_.Prev = ctxList->Prev; - PrevLogContextHead_.Next = ctxList->Next; - ctxList->Next = ctxList->Prev = ctxList; - LinkBefore(ctxList); - } - - ~TRootLogContext() { - Unlink(); - NImpl::TLogContextListItem* ctxList = NImpl::GetLogContextList(); - ctxList->Prev = PrevLogContextHead_.Prev; - ctxList->Next = PrevLogContextHead_.Next; - } - - explicit inline operator bool() const noexcept { - return true; - } - -private: + { + NImpl::TLogContextListItem* ctxList = NImpl::GetLogContextList(); + PrevLogContextHead_.Prev = ctxList->Prev; + PrevLogContextHead_.Next = ctxList->Next; + ctxList->Next = ctxList->Prev = ctxList; + LinkBefore(ctxList); + } + + ~TRootLogContext() { + Unlink(); + NImpl::TLogContextListItem* ctxList = NImpl::GetLogContextList(); + ctxList->Prev = PrevLogContextHead_.Prev; + ctxList->Next = PrevLogContextHead_.Next; + } + + explicit inline operator bool() const noexcept { + return true; + } + +private: std::array<TString, Size> Names_; - NImpl::TLogContextListItem PrevLogContextHead_; -}; - -/** - * @brief Helper function to construct TLogContext from variable - * arguments list. - */ -template <typename... TArgs> -auto MakeCtx(TArgs&&... args) -> TLogContext<sizeof...(args)> { - return TLogContext<sizeof...(args)>(std::forward<TArgs>(args)...); -} - -template <typename... TArgs> -auto MakeRootCtx(TArgs&&... args) -> TRootLogContext<sizeof...(args)> { - return TRootLogContext<sizeof...(args)>(std::forward<TArgs>(args)...); -} - -/** - * @brief Returns current logger contexts path as string. Each element - * is separated with '/'. - */ + NImpl::TLogContextListItem PrevLogContextHead_; +}; + +/** + * @brief Helper function to construct TLogContext from variable + * arguments list. + */ +template <typename... TArgs> +auto MakeCtx(TArgs&&... args) -> TLogContext<sizeof...(args)> { + return TLogContext<sizeof...(args)>(std::forward<TArgs>(args)...); +} + +template <typename... TArgs> +auto MakeRootCtx(TArgs&&... args) -> TRootLogContext<sizeof...(args)> { + return TRootLogContext<sizeof...(args)>(std::forward<TArgs>(args)...); +} + +/** + * @brief Returns current logger contexts path as string. Each element + * is separated with '/'. + */ TString CurrentLogContextPath(); - -/** - * @brief If last throwing exception was performed with YQL_LOG_CTX_THROW - * macro this function returns location and context of that throw point. - */ + +/** + * @brief If last throwing exception was performed with YQL_LOG_CTX_THROW + * macro this function returns location and context of that throw point. + */ TString ThrowedLogContextPath(); - -/** - * @brief Adds context preffix before logging message. - */ -struct TContextPreprocessor { - static TAutoPtr<TLogElement> Preprocess(TAutoPtr<TLogElement> element); + +/** + * @brief Adds context preffix before logging message. + */ +struct TContextPreprocessor { + static TAutoPtr<TLogElement> Preprocess(TAutoPtr<TLogElement> element); }; -/** +/** * @brief Outputs current logger context into stream */ void OutputLogCtx(IOutputStream* out, bool withBraces); /** - * @brief Outputs current logger context into exception message. - */ -class TYqlLogContextLocation { -public: - TYqlLogContextLocation(const TSourceLocation& location) - : Location_(location.File, location.Line) - { - } - - void SetThrowedLogContextPath() const; - - template <class T> - inline T&& operator+(T&& t) { - SetThrowedLogContextPath(); - return std::forward<T>(t); - } - -private: - TSourceLocation Location_; -}; - -} // namespace NLog -} // namespace NYql + * @brief Outputs current logger context into exception message. + */ +class TYqlLogContextLocation { +public: + TYqlLogContextLocation(const TSourceLocation& location) + : Location_(location.File, location.Line) + { + } + + void SetThrowedLogContextPath() const; + + template <class T> + inline T&& operator+(T&& t) { + SetThrowedLogContextPath(); + return std::forward<T>(t); + } + +private: + TSourceLocation Location_; +}; + +} // namespace NLog +} // namespace NYql diff --git a/ydb/library/yql/utils/log/log.cpp b/ydb/library/yql/utils/log/log.cpp index 2909e6d1da0..b62bfeb907a 100644 --- a/ydb/library/yql/utils/log/log.cpp +++ b/ydb/library/yql/utils/log/log.cpp @@ -3,37 +3,37 @@ #include <library/cpp/logger/stream.h> #include <library/cpp/logger/system.h> -#include <util/datetime/systime.h> -#include <util/generic/strbuf.h> -#include <util/stream/format.h> -#include <util/system/getpid.h> +#include <util/datetime/systime.h> +#include <util/generic/strbuf.h> +#include <util/stream/format.h> +#include <util/system/getpid.h> #include <util/system/mutex.h> #include <util/system/progname.h> -#include <util/system/thread.i> +#include <util/system/thread.i> + +#include <stdio.h> +#include <time.h> -#include <stdio.h> -#include <time.h> - static TMutex g_InitLoggerMutex; static int g_LoggerInitialized = 0; - -namespace { - + +namespace { + void WriteLocalTime(IOutputStream* out) { - struct timeval now; - gettimeofday(&now, nullptr); - - struct tm tm; - time_t seconds = static_cast<time_t>(now.tv_sec); - localtime_r(&seconds, &tm); - - char buf[sizeof("2016-01-02 03:04:05.006")]; - int n = strftime(buf, sizeof(buf), "%Y-%m-%d %H:%M:%S.", &tm); - snprintf(buf + n, sizeof(buf) - n, "%03" PRIu32, static_cast<ui32>(now.tv_usec) / 1000); - - out->Write(buf, sizeof(buf) - 1); -} - + struct timeval now; + gettimeofday(&now, nullptr); + + struct tm tm; + time_t seconds = static_cast<time_t>(now.tv_sec); + localtime_r(&seconds, &tm); + + char buf[sizeof("2016-01-02 03:04:05.006")]; + int n = strftime(buf, sizeof(buf), "%Y-%m-%d %H:%M:%S.", &tm); + snprintf(buf + n, sizeof(buf) - n, "%03" PRIu32, static_cast<ui32>(now.tv_usec) / 1000); + + out->Write(buf, sizeof(buf) - 1); +} + class TLimitedLogBackend final : public TLogBackend { public: TLimitedLogBackend(TAutoPtr<TLogBackend> b, TAtomic& flag, ui64 limit) noexcept @@ -68,27 +68,27 @@ private: }; -} // namspace - +} // namspace + namespace NYql { -namespace NLog { - -/** - * TYqlLogElement - * automaticaly adds new line char - */ -class TYqlLogElement: public TLogElement { -public: - TYqlLogElement(const TLog* parent, ELevel level) - : TLogElement(parent, ELevelHelpers::ToLogPriority(level)) - { - } - - ~TYqlLogElement() { - *this << '\n'; - } -}; - +namespace NLog { + +/** + * TYqlLogElement + * automaticaly adds new line char + */ +class TYqlLogElement: public TLogElement { +public: + TYqlLogElement(const TLog* parent, ELevel level) + : TLogElement(parent, ELevelHelpers::ToLogPriority(level)) + { + } + + ~TYqlLogElement() { + *this << '\n'; + } +}; + TYqlLog::TYqlLog() : TLog() , ProcName_() @@ -96,36 +96,36 @@ TYqlLog::TYqlLog() , WriteTruncMsg_(0) {} TYqlLog::TYqlLog(const TString& logType, const TComponentLevels& levels) - : TLog(logType) - , ProcName_(GetProgramName()) - , ProcId_(GetPID()) + : TLog(logType) + , ProcName_(GetProgramName()) + , ProcId_(GetPID()) , WriteTruncMsg_(0) -{ +{ for (size_t component = 0; component < levels.size(); ++component) { SetComponentLevel(EComponentHelpers::FromInt(component), levels[component]); } -} - -TYqlLog::TYqlLog(TAutoPtr<TLogBackend> backend, const TComponentLevels& levels) - : TLog(backend) - , ProcName_(GetProgramName()) - , ProcId_(GetPID()) +} + +TYqlLog::TYqlLog(TAutoPtr<TLogBackend> backend, const TComponentLevels& levels) + : TLog(backend) + , ProcName_(GetProgramName()) + , ProcId_(GetPID()) , WriteTruncMsg_(0) -{ +{ for (size_t component = 0; component < levels.size(); ++component) { SetComponentLevel(EComponentHelpers::FromInt(component), levels[component]); } -} - +} + void TYqlLog::UpdateProcInfo(const TString& procName) { - ProcName_ = procName; - ProcId_ = GetPID(); -} - -TAutoPtr<TLogElement> TYqlLog::CreateLogElement( - EComponent component, ELevel level, + ProcName_ = procName; + ProcId_ = GetPID(); +} + +TAutoPtr<TLogElement> TYqlLog::CreateLogElement( + EComponent component, ELevel level, TStringBuf file, int line) const -{ +{ const bool writeMsg = AtomicCas(&WriteTruncMsg_, 0, 1); auto element = MakeHolder<TYqlLogElement>(this, writeMsg ? ELevel::FATAL : level); if (writeMsg) { @@ -139,40 +139,40 @@ TAutoPtr<TLogElement> TYqlLog::CreateLogElement( } void TYqlLog::WriteLogPrefix(IOutputStream* out, EComponent component, ELevel level, TStringBuf file, int line) const { - // LOG FORMAT: - // {datetime} {level} {procname}(pid={pid}, tid={tid}) [{component}] {source_location}: {message}\n - // - + // LOG FORMAT: + // {datetime} {level} {procname}(pid={pid}, tid={tid}) [{component}] {source_location}: {message}\n + // + WriteLocalTime(out); *out << ' ' - << ELevelHelpers::ToString(level) << ' ' + << ELevelHelpers::ToString(level) << ' ' << ProcName_ << TStringBuf("(pid=") << ProcId_ << TStringBuf(", tid=") -#ifdef _unix_ - << Hex(SystemCurrentThreadIdImpl()) -#else - << SystemCurrentThreadIdImpl() -#endif +#ifdef _unix_ + << Hex(SystemCurrentThreadIdImpl()) +#else + << SystemCurrentThreadIdImpl() +#endif << TStringBuf(") [") << EComponentHelpers::ToString(component) << TStringBuf("] ") << file.RAfter(LOCSLASH_C) << ':' << line << TStringBuf(": "); } - + void TYqlLog::SetMaxLogLimit(ui64 limit) { auto backend = TLog::ReleaseBackend(); TLog::ResetBackend(THolder(new TLimitedLogBackend(backend, WriteTruncMsg_, limit))); -} - +} + void InitLogger(const TString& logType, bool startAsDaemon) { with_lock(g_InitLoggerMutex) { ++g_LoggerInitialized; if (g_LoggerInitialized > 1) { return; } - + TComponentLevels levels; levels.fill(ELevel::INFO); - + if (startAsDaemon && ( TStringBuf("console") == logType || TStringBuf("cout") == logType || @@ -193,7 +193,7 @@ void InitLogger(const TString& logType, bool startAsDaemon) { } } -void InitLogger(TAutoPtr<TLogBackend> backend) { +void InitLogger(TAutoPtr<TLogBackend> backend) { with_lock(g_InitLoggerMutex) { ++g_LoggerInitialized; if (g_LoggerInitialized > 1) { @@ -204,12 +204,12 @@ void InitLogger(TAutoPtr<TLogBackend> backend) { levels.fill(ELevel::INFO); TLoggerOperator<TYqlLog>::Set(new TYqlLog(backend, levels)); } -} - +} + void InitLogger(IOutputStream* out) { - InitLogger(new TStreamLogBackend(out)); -} - + InitLogger(new TStreamLogBackend(out)); +} + void CleanupLogger() { with_lock(g_InitLoggerMutex) { --g_LoggerInitialized; @@ -221,15 +221,15 @@ void CleanupLogger() { } } -} // namespace NLog +} // namespace NLog } // namespace NYql - -/** - * creates default YQL logger writing to /dev/null - */ -template <> -NYql::NLog::TYqlLog* CreateDefaultLogger<NYql::NLog::TYqlLog>() { - NYql::NLog::TComponentLevels levels; - levels.fill(NYql::NLog::ELevel::INFO); - return new NYql::NLog::TYqlLog("null", levels); -} + +/** + * creates default YQL logger writing to /dev/null + */ +template <> +NYql::NLog::TYqlLog* CreateDefaultLogger<NYql::NLog::TYqlLog>() { + NYql::NLog::TComponentLevels levels; + levels.fill(NYql::NLog::ELevel::INFO); + return new NYql::NLog::TYqlLog("null", levels); +} diff --git a/ydb/library/yql/utils/log/log.h b/ydb/library/yql/utils/log/log.h index a6390fa19c8..8095fdd4952 100644 --- a/ydb/library/yql/utils/log/log.h +++ b/ydb/library/yql/utils/log/log.h @@ -1,43 +1,43 @@ #pragma once -#include "log_component.h" -#include "log_level.h" -#include "context.h" -#include "profile.h" +#include "log_component.h" +#include "log_level.h" +#include "context.h" +#include "profile.h" #include <library/cpp/logger/global/common.h> - + #include <util/system/atomic.h> #include <util/stream/output.h> #include <util/generic/strbuf.h> -#include <array> - - -#define YQL_LOG_IMPL(logger, component, level, preprocessor, file, line) \ - logger.NeedToLog(component, level) && NPrivateGlobalLogger::TEatStream() | \ - (*preprocessor::Preprocess(logger.CreateLogElement(component, level, file, line))) - -#define YQL_LOG_IF_IMPL(logger, component, level, preprocessor, condition, file, line) \ - logger.NeedToLog(component, level) && (condition) && NPrivateGlobalLogger::TEatStream() | \ - (*preprocessor::Preprocess(logger.CreateLogElement(component, level, file, line))) - -// with component logger - -#define YQL_CLOG_PREP(level, component, preprocessor) YQL_LOG_IMPL(\ - ::NYql::NLog::YqlLogger(), \ - ::NYql::NLog::EComponent::component, \ - ::NYql::NLog::ELevel::level, \ - preprocessor, \ - __FILE__, __LINE__) - -#define YQL_CLOG(level, component) \ - YQL_CLOG_PREP(level, component, ::NYql::NLog::TContextPreprocessor) - -#define YQL_CLOG_ACTIVE(level, component) ::NYql::NLog::YqlLogger().NeedToLog( \ - ::NYql::NLog::EComponent::component, \ - ::NYql::NLog::ELevel::level) - +#include <array> + + +#define YQL_LOG_IMPL(logger, component, level, preprocessor, file, line) \ + logger.NeedToLog(component, level) && NPrivateGlobalLogger::TEatStream() | \ + (*preprocessor::Preprocess(logger.CreateLogElement(component, level, file, line))) + +#define YQL_LOG_IF_IMPL(logger, component, level, preprocessor, condition, file, line) \ + logger.NeedToLog(component, level) && (condition) && NPrivateGlobalLogger::TEatStream() | \ + (*preprocessor::Preprocess(logger.CreateLogElement(component, level, file, line))) + +// with component logger + +#define YQL_CLOG_PREP(level, component, preprocessor) YQL_LOG_IMPL(\ + ::NYql::NLog::YqlLogger(), \ + ::NYql::NLog::EComponent::component, \ + ::NYql::NLog::ELevel::level, \ + preprocessor, \ + __FILE__, __LINE__) + +#define YQL_CLOG(level, component) \ + YQL_CLOG_PREP(level, component, ::NYql::NLog::TContextPreprocessor) + +#define YQL_CLOG_ACTIVE(level, component) ::NYql::NLog::YqlLogger().NeedToLog( \ + ::NYql::NLog::EComponent::component, \ + ::NYql::NLog::ELevel::level) + // with component/level values logger #define YQL_CVLOG_PREP(level, component, preprocessor) YQL_LOG_IMPL(\ @@ -54,115 +54,115 @@ component, \ level) -// default logger - -#define YQL_LOG_PREP(level, preprocessor) \ - YQL_CLOG_PREP(level, Default, preprocessor) - -#define YQL_LOG(level) \ - YQL_LOG_PREP(level, ::NYql::NLog::TContextPreprocessor) - -#define YQL_LOG_ACTIVE(level) YQL_CLOG_ACTIVE(level, Default) - -// conditional logger - -#define YQL_CLOG_PREP_IF(level, component, preprocessor, condition) YQL_LOG_IF_IMPL(\ - ::NYql::NLog::YqlLogger(), \ - ::NYql::NLog::EComponent::component, \ - ::NYql::NLog::ELevel::level, \ - preprocessor, \ - condition, \ - __FILE__, __LINE__) - -#define YQL_CLOG_IF(level, component, condition) \ - YQL_CLOG_PREP_IF(level, component, ::NYql::NLog::TContextPreprocessor, condition) - -#define YQL_LOG_PREP_IF(level, preprocessor, condition) \ - YQL_CLOG_PREP_IF(level, Default, preprocessor, condition) - -#define YQL_LOG_IF(level, condition) \ - YQL_LOG_PREP_IF(level, ::NYql::NLog::TContextPreprocessor, condition) - - +// default logger + +#define YQL_LOG_PREP(level, preprocessor) \ + YQL_CLOG_PREP(level, Default, preprocessor) + +#define YQL_LOG(level) \ + YQL_LOG_PREP(level, ::NYql::NLog::TContextPreprocessor) + +#define YQL_LOG_ACTIVE(level) YQL_CLOG_ACTIVE(level, Default) + +// conditional logger + +#define YQL_CLOG_PREP_IF(level, component, preprocessor, condition) YQL_LOG_IF_IMPL(\ + ::NYql::NLog::YqlLogger(), \ + ::NYql::NLog::EComponent::component, \ + ::NYql::NLog::ELevel::level, \ + preprocessor, \ + condition, \ + __FILE__, __LINE__) + +#define YQL_CLOG_IF(level, component, condition) \ + YQL_CLOG_PREP_IF(level, component, ::NYql::NLog::TContextPreprocessor, condition) + +#define YQL_LOG_PREP_IF(level, preprocessor, condition) \ + YQL_CLOG_PREP_IF(level, Default, preprocessor, condition) + +#define YQL_LOG_IF(level, condition) \ + YQL_LOG_PREP_IF(level, ::NYql::NLog::TContextPreprocessor, condition) + + namespace NYql { -namespace NLog { +namespace NLog { -using TComponentLevels = - std::array<ELevel, EComponentHelpers::ToInt(EComponent::MaxValue)>; +using TComponentLevels = + std::array<ELevel, EComponentHelpers::ToInt(EComponent::MaxValue)>; -/** - * @brief Component based logger frontend. - */ -class TYqlLog: public TLog { -public: +/** + * @brief Component based logger frontend. + */ +class TYqlLog: public TLog { +public: TYqlLog(); TYqlLog(const TString& logType, const TComponentLevels& levels); - TYqlLog(TAutoPtr<TLogBackend> backend, const TComponentLevels& levels); - - // XXX: not thread-safe + TYqlLog(TAutoPtr<TLogBackend> backend, const TComponentLevels& levels); + + // XXX: not thread-safe void UpdateProcInfo(const TString& procName); - - ELevel GetComponentLevel(EComponent component) const { + + ELevel GetComponentLevel(EComponent component) const { return ELevelHelpers::FromInt(AtomicGet(ComponentLevels_[EComponentHelpers::ToInt(component)])); - } - - void SetComponentLevel(EComponent component, ELevel level) { + } + + void SetComponentLevel(EComponent component, ELevel level) { AtomicSet(ComponentLevels_[EComponentHelpers::ToInt(component)], ELevelHelpers::ToInt(level)); - } - - bool NeedToLog(EComponent component, ELevel level) const { - return ELevelHelpers::Lte(level, GetComponentLevel(component)); - } - + } + + bool NeedToLog(EComponent component, ELevel level) const { + return ELevelHelpers::Lte(level, GetComponentLevel(component)); + } + void SetMaxLogLimit(ui64 limit); - + TAutoPtr<TLogElement> CreateLogElement(EComponent component, ELevel level, TStringBuf file, int line) const; void WriteLogPrefix(IOutputStream* out, EComponent component, ELevel level, TStringBuf file, int line) const; private: TString ProcName_; - pid_t ProcId_; + pid_t ProcId_; std::array<TAtomic, EComponentHelpers::ToInt(EComponent::MaxValue)> ComponentLevels_{0}; mutable TAtomic WriteTruncMsg_; -}; - -/** - * @brief returns reference to YQL logger instance. - */ -inline TYqlLog& YqlLogger() { - return static_cast<TYqlLog&>(TLoggerOperator<TYqlLog>::Log()); -} - -/** - * @brief returns true it YQL logger already initialized. - */ -inline bool IsYqlLoggerInitialized() { - return TLoggerOperator<TYqlLog>::Usage(); -} - -/** - * @brief Initialize logger with selected backend type. - * - * @param log - one of { syslog, console, cout, cerr, null, /path/to/file } - * @param startAsDaemon - true if process is demonized - */ +}; + +/** + * @brief returns reference to YQL logger instance. + */ +inline TYqlLog& YqlLogger() { + return static_cast<TYqlLog&>(TLoggerOperator<TYqlLog>::Log()); +} + +/** + * @brief returns true it YQL logger already initialized. + */ +inline bool IsYqlLoggerInitialized() { + return TLoggerOperator<TYqlLog>::Usage(); +} + +/** + * @brief Initialize logger with selected backend type. + * + * @param log - one of { syslog, console, cout, cerr, null, /path/to/file } + * @param startAsDaemon - true if process is demonized + */ void InitLogger(const TString& log, bool startAsDaemon = false); - -/** - * @brief Initialize logger with concrete backend. - * - * @param backend - logger backend - */ -void InitLogger(TAutoPtr<TLogBackend> backend); - -/** - * @brief Initialize logger with concrete output stream. - * - * @param out - output stream - */ + +/** + * @brief Initialize logger with concrete backend. + * + * @param backend - logger backend + */ +void InitLogger(TAutoPtr<TLogBackend> backend); + +/** + * @brief Initialize logger with concrete output stream. + * + * @param out - output stream + */ void InitLogger(IOutputStream* out); - + void CleanupLogger(); class YqlLoggerScope { @@ -174,8 +174,8 @@ public: ~YqlLoggerScope() { CleanupLogger(); } }; -} // namespace NLog +} // namespace NLog } // namespace NYql - -template <> -NYql::NLog::TYqlLog* CreateDefaultLogger<NYql::NLog::TYqlLog>(); + +template <> +NYql::NLog::TYqlLog* CreateDefaultLogger<NYql::NLog::TYqlLog>(); diff --git a/ydb/library/yql/utils/log/log_component.h b/ydb/library/yql/utils/log/log_component.h index e49e3ebd944..d7a1e10ca16 100644 --- a/ydb/library/yql/utils/log/log_component.h +++ b/ydb/library/yql/utils/log/log_component.h @@ -1,27 +1,27 @@ -#pragma once - -#include <util/generic/strbuf.h> -#include <util/generic/yexception.h> - - -namespace NYql { -namespace NLog { - -// keep this enum in sync with simmilar enum from yqlworker config.proto -enum class EComponent { - Default = 0, - Core, - CoreExecution, - Sql, - ProviderCommon, - ProviderConfig, - ProviderResult, - ProviderYt, - ProviderKikimr, - ProviderKqp, - ProviderRtmr, - Performance, Perf = Performance, - Net, +#pragma once + +#include <util/generic/strbuf.h> +#include <util/generic/yexception.h> + + +namespace NYql { +namespace NLog { + +// keep this enum in sync with simmilar enum from yqlworker config.proto +enum class EComponent { + Default = 0, + Core, + CoreExecution, + Sql, + ProviderCommon, + ProviderConfig, + ProviderResult, + ProviderYt, + ProviderKikimr, + ProviderKqp, + ProviderRtmr, + Performance, Perf = Performance, + Net, ProviderStat, ProviderSolomon, CoreEval, @@ -32,24 +32,24 @@ enum class EComponent { ProviderPq, ProviderS3, CoreDq, - // <--- put other log components here - MaxValue -}; - -struct EComponentHelpers { - static constexpr int ToInt(EComponent component) { - return static_cast<int>(component); - } - - static constexpr EComponent FromInt(int component) { - return (component >= ToInt(EComponent::Default) && - component < ToInt(EComponent::MaxValue)) - ? static_cast<EComponent>(component) - : EComponent::Default; - } - - static TStringBuf ToString(EComponent component) { - switch (component) { + // <--- put other log components here + MaxValue +}; + +struct EComponentHelpers { + static constexpr int ToInt(EComponent component) { + return static_cast<int>(component); + } + + static constexpr EComponent FromInt(int component) { + return (component >= ToInt(EComponent::Default) && + component < ToInt(EComponent::MaxValue)) + ? static_cast<EComponent>(component) + : EComponent::Default; + } + + static TStringBuf ToString(EComponent component) { + switch (component) { case EComponent::Default: return TStringBuf("default"); case EComponent::Core: return TStringBuf("core"); case EComponent::CoreEval: return TStringBuf("core eval"); @@ -73,13 +73,13 @@ struct EComponentHelpers { case EComponent::ProviderPq: return TStringBuf("PQ"); case EComponent::ProviderS3: return TStringBuf("S3"); case EComponent::CoreDq: return TStringBuf("core dq"); - default: - ythrow yexception() << "invalid log component value: " - << ToInt(component); - } - } - - static EComponent FromString(TStringBuf str) { + default: + ythrow yexception() << "invalid log component value: " + << ToInt(component); + } + } + + static EComponent FromString(TStringBuf str) { if (str == TStringBuf("default")) return EComponent::Default; if (str == TStringBuf("core")) return EComponent::Core; if (str == TStringBuf("core eval")) return EComponent::CoreEval; @@ -103,19 +103,19 @@ struct EComponentHelpers { if (str == TStringBuf("PQ")) return EComponent::ProviderPq; if (str == TStringBuf("S3")) return EComponent::ProviderS3; if (str == TStringBuf("core dq")) return EComponent::CoreDq; - ythrow yexception() << "unknown log component: '" << str << '\''; - } - - template <typename TFunctor> - static void ForEach(TFunctor&& f) { - static const int minValue = ToInt(EComponent::Default); - static const int maxValue = ToInt(EComponent::MaxValue); - - for (int c = minValue; c < maxValue; c++) { - f(FromInt(c)); - } - } -}; - + ythrow yexception() << "unknown log component: '" << str << '\''; + } + + template <typename TFunctor> + static void ForEach(TFunctor&& f) { + static const int minValue = ToInt(EComponent::Default); + static const int maxValue = ToInt(EComponent::MaxValue); + + for (int c = minValue; c < maxValue; c++) { + f(FromInt(c)); + } + } +}; + } // namespace NLog } // namespace NYql diff --git a/ydb/library/yql/utils/log/log_level.h b/ydb/library/yql/utils/log/log_level.h index 6775043529c..ccb12e4690b 100644 --- a/ydb/library/yql/utils/log/log_level.h +++ b/ydb/library/yql/utils/log/log_level.h @@ -1,64 +1,64 @@ -#pragma once - +#pragma once + #include <library/cpp/logger/priority.h> - -#include <util/generic/strbuf.h> -#include <util/generic/yexception.h> - - -namespace NYql { -namespace NLog { - -enum class ELevel { - FATAL = TLOG_EMERG, - ERROR = TLOG_ERR, - WARN = TLOG_WARNING, + +#include <util/generic/strbuf.h> +#include <util/generic/yexception.h> + + +namespace NYql { +namespace NLog { + +enum class ELevel { + FATAL = TLOG_EMERG, + ERROR = TLOG_ERR, + WARN = TLOG_WARNING, NOTICE = TLOG_NOTICE, - INFO = TLOG_INFO, - DEBUG = TLOG_DEBUG, - TRACE = TLOG_RESOURCES, -}; - -struct ELevelHelpers { - static constexpr bool Lte(ELevel l1, ELevel l2) { - return ToInt(l1) <= ToInt(l2); - } - + INFO = TLOG_INFO, + DEBUG = TLOG_DEBUG, + TRACE = TLOG_RESOURCES, +}; + +struct ELevelHelpers { + static constexpr bool Lte(ELevel l1, ELevel l2) { + return ToInt(l1) <= ToInt(l2); + } + static constexpr ELogPriority ToLogPriority(ELevel level) { return static_cast<ELogPriority>(ToInt(level)); - } - + } + static ELevel FromLogPriority(ELogPriority priority) { - return FromInt(static_cast<int>(priority)); - } - - static constexpr int ToInt(ELevel level) { - return static_cast<int>(level); - } - - static ELevel FromInt(int level) { - switch (level) { - case TLOG_EMERG: - case TLOG_ALERT: - case TLOG_CRIT: return ELevel::FATAL; - - case TLOG_ERR: return ELevel::ERROR; - case TLOG_WARNING: return ELevel::WARN; - - case TLOG_NOTICE: - case TLOG_INFO: return ELevel::INFO; - - case TLOG_DEBUG: return ELevel::DEBUG; - case TLOG_RESOURCES: return ELevel::TRACE; - - default: - return ELevel::INFO; - } - } - - static TStringBuf ToString(ELevel level) { - // aligned 5-letters string - switch (level) { + return FromInt(static_cast<int>(priority)); + } + + static constexpr int ToInt(ELevel level) { + return static_cast<int>(level); + } + + static ELevel FromInt(int level) { + switch (level) { + case TLOG_EMERG: + case TLOG_ALERT: + case TLOG_CRIT: return ELevel::FATAL; + + case TLOG_ERR: return ELevel::ERROR; + case TLOG_WARNING: return ELevel::WARN; + + case TLOG_NOTICE: + case TLOG_INFO: return ELevel::INFO; + + case TLOG_DEBUG: return ELevel::DEBUG; + case TLOG_RESOURCES: return ELevel::TRACE; + + default: + return ELevel::INFO; + } + } + + static TStringBuf ToString(ELevel level) { + // aligned 5-letters string + switch (level) { case ELevel::FATAL: return TStringBuf("FATAL"); case ELevel::ERROR: return TStringBuf("ERROR"); case ELevel::WARN: return TStringBuf("WARN "); @@ -66,12 +66,12 @@ struct ELevelHelpers { case ELevel::INFO: return TStringBuf("INFO "); case ELevel::DEBUG: return TStringBuf("DEBUG"); case ELevel::TRACE: return TStringBuf("TRACE"); - } - ythrow yexception() << "unknown log level: " << ToInt(level); - } - - static ELevel FromString(TStringBuf str) { - // aligned 5-letters string + } + ythrow yexception() << "unknown log level: " << ToInt(level); + } + + static ELevel FromString(TStringBuf str) { + // aligned 5-letters string if (str == TStringBuf("FATAL")) return ELevel::FATAL; if (str == TStringBuf("ERROR")) return ELevel::ERROR; if (str == TStringBuf("WARN ")) return ELevel::WARN; @@ -79,20 +79,20 @@ struct ELevelHelpers { if (str == TStringBuf("INFO ")) return ELevel::INFO; if (str == TStringBuf("DEBUG")) return ELevel::DEBUG; if (str == TStringBuf("TRACE")) return ELevel::TRACE; - ythrow yexception() << "unknown log level: " << str; - } - - template <typename TFunctor> - static void ForEach(TFunctor&& f) { - static const int minValue = ToInt(ELevel::FATAL); - static const int maxValue = ToInt(ELevel::TRACE); - - for (int l = minValue; l <= maxValue; l++) { - f(FromInt(l)); - } - } -}; - - -} // namspace NLog -} // namspace NYql + ythrow yexception() << "unknown log level: " << str; + } + + template <typename TFunctor> + static void ForEach(TFunctor&& f) { + static const int minValue = ToInt(ELevel::FATAL); + static const int maxValue = ToInt(ELevel::TRACE); + + for (int l = minValue; l <= maxValue; l++) { + f(FromInt(l)); + } + } +}; + + +} // namspace NLog +} // namspace NYql diff --git a/ydb/library/yql/utils/log/log_ut.cpp b/ydb/library/yql/utils/log/log_ut.cpp index 973b542b737..bf8b71476c0 100644 --- a/ydb/library/yql/utils/log/log_ut.cpp +++ b/ydb/library/yql/utils/log/log_ut.cpp @@ -1,451 +1,451 @@ -#include "log.h" -#include "context.h" -#include "profile.h" +#include "log.h" +#include "context.h" +#include "profile.h" #include <ydb/library/yql/utils/log/ut/log_parser.h> - + #include <library/cpp/testing/unittest/registar.h> #include <library/cpp/logger/stream.h> - -#include <util/datetime/base.h> -#include <util/generic/yexception.h> -#include <util/system/getpid.h> -#include <util/string/split.h> -#include <util/string/cast.h> + +#include <util/datetime/base.h> +#include <util/generic/yexception.h> +#include <util/system/getpid.h> +#include <util/string/split.h> +#include <util/string/cast.h> #include <util/string/subst.h> - -#include <regex> - - -using namespace NYql; -using namespace NLog; - + +#include <regex> + + +using namespace NYql; +using namespace NLog; + Y_UNIT_TEST_SUITE(TLogTest) -{ +{ Y_UNIT_TEST(Format) { - TStringStream out; + TStringStream out; YqlLoggerScope logger(&out); - YqlLogger().UpdateProcInfo("my_proc"); - + YqlLogger().UpdateProcInfo("my_proc"); + TString message = "some performance info"; - YQL_LOG(INFO) << message; - - TLogRow logRow = ParseLogRow(out.Str()); - - TDuration elapsed(logRow.Time - TInstant::Now()); - UNIT_ASSERT(elapsed < TDuration::MilliSeconds(5)); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_STRINGS_EQUAL(logRow.ProcName, "my_proc"); - UNIT_ASSERT_EQUAL(logRow.ProcId, GetPID()); - UNIT_ASSERT(logRow.ThreadId > 0); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); - UNIT_ASSERT_STRINGS_EQUAL( - logRow.FileName, - TStringBuf(__FILE__).RNextTok(LOCSLASH_C)); - UNIT_ASSERT(logRow.LineNumber != 0); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, message); - } - + YQL_LOG(INFO) << message; + + TLogRow logRow = ParseLogRow(out.Str()); + + TDuration elapsed(logRow.Time - TInstant::Now()); + UNIT_ASSERT(elapsed < TDuration::MilliSeconds(5)); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_STRINGS_EQUAL(logRow.ProcName, "my_proc"); + UNIT_ASSERT_EQUAL(logRow.ProcId, GetPID()); + UNIT_ASSERT(logRow.ThreadId > 0); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); + UNIT_ASSERT_STRINGS_EQUAL( + logRow.FileName, + TStringBuf(__FILE__).RNextTok(LOCSLASH_C)); + UNIT_ASSERT(logRow.LineNumber != 0); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, message); + } + Y_UNIT_TEST(Levels) { - TStringStream out; + TStringStream out; YqlLoggerScope logger(&out); // default log level INFO - - YQL_LOG(FATAL) << "fatal message"; - YQL_LOG(ERROR) << "error message"; - YQL_LOG(WARN) << "warning message"; - YQL_LOG(INFO) << "info message"; - YQL_LOG(DEBUG) << "debug message"; - YQL_LOG(TRACE) << "trace message"; - + + YQL_LOG(FATAL) << "fatal message"; + YQL_LOG(ERROR) << "error message"; + YQL_LOG(WARN) << "warning message"; + YQL_LOG(INFO) << "info message"; + YQL_LOG(DEBUG) << "debug message"; + YQL_LOG(TRACE) << "trace message"; + TString fatalStr, errorStr, warnStr, infoStr, _; - Split(out.Str(), '\n', fatalStr, errorStr, warnStr, infoStr, _); - - { - TLogRow logRow = ParseLogRow(fatalStr); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::FATAL); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "fatal message"); - } - { - TLogRow logRow = ParseLogRow(errorStr); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::ERROR); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "error message"); - } - { - TLogRow logRow = ParseLogRow(warnStr); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::WARN); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "warning message"); - } - { - TLogRow logRow = ParseLogRow(infoStr); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "info message"); - } - } - + Split(out.Str(), '\n', fatalStr, errorStr, warnStr, infoStr, _); + + { + TLogRow logRow = ParseLogRow(fatalStr); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::FATAL); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "fatal message"); + } + { + TLogRow logRow = ParseLogRow(errorStr); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::ERROR); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "error message"); + } + { + TLogRow logRow = ParseLogRow(warnStr); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::WARN); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "warning message"); + } + { + TLogRow logRow = ParseLogRow(infoStr); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "info message"); + } + } + Y_UNIT_TEST(Components) { - TStringStream out; + TStringStream out; YqlLoggerScope logger(&out); - - YQL_CLOG(INFO, Default) << "default message"; - YQL_CLOG(INFO, Core) << "core message"; - YQL_CLOG(INFO, Sql) << "sql message"; - YQL_CLOG(INFO, ProviderCommon) << "common message"; - YQL_CLOG(INFO, ProviderYt) << "yt message"; - YQL_CLOG(INFO, ProviderKikimr) << "kikimr message"; - YQL_CLOG(INFO, ProviderRtmr) << "rtmr message"; - YQL_CLOG(INFO, Performance) << "performance message"; - YQL_CLOG(INFO, Perf) << "perf message"; - + + YQL_CLOG(INFO, Default) << "default message"; + YQL_CLOG(INFO, Core) << "core message"; + YQL_CLOG(INFO, Sql) << "sql message"; + YQL_CLOG(INFO, ProviderCommon) << "common message"; + YQL_CLOG(INFO, ProviderYt) << "yt message"; + YQL_CLOG(INFO, ProviderKikimr) << "kikimr message"; + YQL_CLOG(INFO, ProviderRtmr) << "rtmr message"; + YQL_CLOG(INFO, Performance) << "performance message"; + YQL_CLOG(INFO, Perf) << "perf message"; + TString defaultStr, coreStr, sqlStr, commonStr, ytStr, - kikimrStr, rtmrStr, performanceStr, perfStr, _; - Split(out.Str(), '\n', defaultStr, coreStr, sqlStr, + kikimrStr, rtmrStr, performanceStr, perfStr, _; + Split(out.Str(), '\n', defaultStr, coreStr, sqlStr, commonStr, ytStr, - kikimrStr, rtmrStr, - performanceStr, perfStr, _); - - { - TLogRow logRow = ParseLogRow(defaultStr); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "default message"); - } - { - TLogRow logRow = ParseLogRow(coreStr); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Core); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "core message"); - } - { - TLogRow logRow = ParseLogRow(sqlStr); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Sql); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "sql message"); - } - { - TLogRow logRow = ParseLogRow(commonStr); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::ProviderCommon); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "common message"); - } - { - TLogRow logRow = ParseLogRow(ytStr); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::ProviderYt); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "yt message"); - } - { - TLogRow logRow = ParseLogRow(kikimrStr); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::ProviderKikimr); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "kikimr message"); - } - { - TLogRow logRow = ParseLogRow(rtmrStr); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::ProviderRtmr); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "rtmr message"); - } - { - TLogRow logRow = ParseLogRow(performanceStr); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Performance); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Perf); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "performance message"); - } - { - TLogRow logRow = ParseLogRow(perfStr); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Perf); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Performance); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "perf message"); - } - } - + kikimrStr, rtmrStr, + performanceStr, perfStr, _); + + { + TLogRow logRow = ParseLogRow(defaultStr); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "default message"); + } + { + TLogRow logRow = ParseLogRow(coreStr); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Core); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "core message"); + } + { + TLogRow logRow = ParseLogRow(sqlStr); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Sql); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "sql message"); + } + { + TLogRow logRow = ParseLogRow(commonStr); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::ProviderCommon); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "common message"); + } + { + TLogRow logRow = ParseLogRow(ytStr); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::ProviderYt); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "yt message"); + } + { + TLogRow logRow = ParseLogRow(kikimrStr); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::ProviderKikimr); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "kikimr message"); + } + { + TLogRow logRow = ParseLogRow(rtmrStr); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::ProviderRtmr); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "rtmr message"); + } + { + TLogRow logRow = ParseLogRow(performanceStr); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Performance); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Perf); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "performance message"); + } + { + TLogRow logRow = ParseLogRow(perfStr); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Perf); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Performance); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "perf message"); + } + } + Y_UNIT_TEST(Conditional) { - TStringStream out; + TStringStream out; YqlLoggerScope logger(&out); - - YQL_LOG_IF(INFO, true) << "default info message"; - YQL_LOG_IF(INFO, false) << "must not be logged"; - - YQL_CLOG_IF(INFO, Perf, true) << "perf info message"; - YQL_CLOG_IF(INFO, Perf, false) << "perf info message"; - + + YQL_LOG_IF(INFO, true) << "default info message"; + YQL_LOG_IF(INFO, false) << "must not be logged"; + + YQL_CLOG_IF(INFO, Perf, true) << "perf info message"; + YQL_CLOG_IF(INFO, Perf, false) << "perf info message"; + TString defaultStr, perfStr, _; - Split(out.Str(), '\n', defaultStr, perfStr, _); - - { - TLogRow logRow = ParseLogRow(defaultStr); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "default info message"); - } - { - TLogRow logRow = ParseLogRow(perfStr); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Perf); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "perf info message"); - } - } - + Split(out.Str(), '\n', defaultStr, perfStr, _); + + { + TLogRow logRow = ParseLogRow(defaultStr); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "default info message"); + } + { + TLogRow logRow = ParseLogRow(perfStr); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Perf); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "perf info message"); + } + } + Y_UNIT_TEST(Contexts) { - TStringStream out; + TStringStream out; YqlLoggerScope logger(&out); - - UNIT_ASSERT_STRINGS_EQUAL(CurrentLogContextPath(), ""); - YQL_LOG(INFO) << "level0 - begin"; - { - YQL_LOG_CTX_SCOPE("ctx1"); - UNIT_ASSERT_STRINGS_EQUAL(CurrentLogContextPath(), "ctx1"); - YQL_LOG(INFO) << "level1 - begin"; - + + UNIT_ASSERT_STRINGS_EQUAL(CurrentLogContextPath(), ""); + YQL_LOG(INFO) << "level0 - begin"; + { + YQL_LOG_CTX_SCOPE("ctx1"); + UNIT_ASSERT_STRINGS_EQUAL(CurrentLogContextPath(), "ctx1"); + YQL_LOG(INFO) << "level1 - begin"; + YQL_LOG_CTX_BLOCK(TStringBuf("ctx2")) { - UNIT_ASSERT_STRINGS_EQUAL(CurrentLogContextPath(), "ctx1/ctx2"); - YQL_LOG(WARN) << "level2"; - } - - UNIT_ASSERT_STRINGS_EQUAL(CurrentLogContextPath(), "ctx1"); - YQL_LOG(INFO) << "level1 - end"; - } - UNIT_ASSERT_STRINGS_EQUAL(CurrentLogContextPath(), ""); - YQL_LOG(INFO) << "level0 - end"; - + UNIT_ASSERT_STRINGS_EQUAL(CurrentLogContextPath(), "ctx1/ctx2"); + YQL_LOG(WARN) << "level2"; + } + + UNIT_ASSERT_STRINGS_EQUAL(CurrentLogContextPath(), "ctx1"); + YQL_LOG(INFO) << "level1 - end"; + } + UNIT_ASSERT_STRINGS_EQUAL(CurrentLogContextPath(), ""); + YQL_LOG(INFO) << "level0 - end"; + TString row1Str, row2Str, row3Str, row4Str, row5Str, _; - Split(out.Str(), '\n', row1Str, row2Str, row3Str, row4Str, row5Str, _); - - { - TLogRow logRow = ParseLogRow(row1Str); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "level0 - begin"); - } - { - TLogRow logRow = ParseLogRow(row2Str); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "{ctx1} level1 - begin"); - } - { - TLogRow logRow = ParseLogRow(row3Str); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::WARN); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "{ctx1/ctx2} level2"); - } - { - TLogRow logRow = ParseLogRow(row4Str); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "{ctx1} level1 - end"); - } - { - TLogRow logRow = ParseLogRow(row5Str); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "level0 - end"); - } - } - + Split(out.Str(), '\n', row1Str, row2Str, row3Str, row4Str, row5Str, _); + + { + TLogRow logRow = ParseLogRow(row1Str); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "level0 - begin"); + } + { + TLogRow logRow = ParseLogRow(row2Str); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "{ctx1} level1 - begin"); + } + { + TLogRow logRow = ParseLogRow(row3Str); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::WARN); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "{ctx1/ctx2} level2"); + } + { + TLogRow logRow = ParseLogRow(row4Str); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "{ctx1} level1 - end"); + } + { + TLogRow logRow = ParseLogRow(row5Str); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "level0 - end"); + } + } + Y_UNIT_TEST(ThrowWithContext) { - bool isThrown = false; - YQL_LOG_CTX_SCOPE("first"); - try { - YQL_LOG_CTX_SCOPE("second"); - YQL_LOG_CTX_THROW yexception() << "some message"; - } catch (const yexception& e) { - isThrown = true; - - UNIT_ASSERT_STRINGS_EQUAL(e.AsStrBuf(), "some message"); - + bool isThrown = false; + YQL_LOG_CTX_SCOPE("first"); + try { + YQL_LOG_CTX_SCOPE("second"); + YQL_LOG_CTX_THROW yexception() << "some message"; + } catch (const yexception& e) { + isThrown = true; + + UNIT_ASSERT_STRINGS_EQUAL(e.AsStrBuf(), "some message"); + TString throwedLogCtx = ThrowedLogContextPath(); - TStringBuf file, line, context; + TStringBuf file, line, context; TStringBuf(throwedLogCtx).Split(".cpp:", file, line); - line.Split(':', line, context); - + line.Split(':', line, context); + TString expectedFile(__LOCATION__.File); SubstGlobal(expectedFile, LOCSLASH_C, '/'); UNIT_ASSERT_STRINGS_EQUAL(TString(file)+".cpp", expectedFile); - int lineNumber; - UNIT_ASSERT(TryFromString<int>(line, lineNumber)); - UNIT_ASSERT(lineNumber > 0); - UNIT_ASSERT_STRINGS_EQUAL(context, " {first/second} "); - - // second call without throw returns empty string - throwedLogCtx = ThrowedLogContextPath(); + int lineNumber; + UNIT_ASSERT(TryFromString<int>(line, lineNumber)); + UNIT_ASSERT(lineNumber > 0); + UNIT_ASSERT_STRINGS_EQUAL(context, " {first/second} "); + + // second call without throw returns empty string + throwedLogCtx = ThrowedLogContextPath(); UNIT_ASSERT(throwedLogCtx.empty()); - } - - UNIT_ASSERT_C(isThrown, "exception was not thrown"); - } - + } + + UNIT_ASSERT_C(isThrown, "exception was not thrown"); + } + Y_UNIT_TEST(ContextOverride) { - TStringStream out; + TStringStream out; YqlLoggerScope logger(&out); - - UNIT_ASSERT_STRINGS_EQUAL(CurrentLogContextPath(), ""); - { - YQL_LOG_CTX_SCOPE("ctx1"); - UNIT_ASSERT_STRINGS_EQUAL(CurrentLogContextPath(), "ctx1"); - YQL_LOG(INFO) << "level1 - begin"; - + + UNIT_ASSERT_STRINGS_EQUAL(CurrentLogContextPath(), ""); + { + YQL_LOG_CTX_SCOPE("ctx1"); + UNIT_ASSERT_STRINGS_EQUAL(CurrentLogContextPath(), "ctx1"); + YQL_LOG(INFO) << "level1 - begin"; + YQL_LOG_CTX_BLOCK(TStringBuf("ctx2")) { - UNIT_ASSERT_STRINGS_EQUAL(CurrentLogContextPath(), "ctx1/ctx2"); - YQL_LOG(WARN) << "level2 - begin"; - - { - YQL_LOG_CTX_ROOT_SCOPE("ctx3"); - UNIT_ASSERT_STRINGS_EQUAL(CurrentLogContextPath(), "ctx3"); - YQL_LOG(ERROR) << "level3"; - } - - UNIT_ASSERT_STRINGS_EQUAL(CurrentLogContextPath(), "ctx1/ctx2"); - YQL_LOG(WARN) << "level2 - end"; - } - - UNIT_ASSERT_STRINGS_EQUAL(CurrentLogContextPath(), "ctx1"); - YQL_LOG(INFO) << "level1 - end"; - } - UNIT_ASSERT_STRINGS_EQUAL(CurrentLogContextPath(), ""); - + UNIT_ASSERT_STRINGS_EQUAL(CurrentLogContextPath(), "ctx1/ctx2"); + YQL_LOG(WARN) << "level2 - begin"; + + { + YQL_LOG_CTX_ROOT_SCOPE("ctx3"); + UNIT_ASSERT_STRINGS_EQUAL(CurrentLogContextPath(), "ctx3"); + YQL_LOG(ERROR) << "level3"; + } + + UNIT_ASSERT_STRINGS_EQUAL(CurrentLogContextPath(), "ctx1/ctx2"); + YQL_LOG(WARN) << "level2 - end"; + } + + UNIT_ASSERT_STRINGS_EQUAL(CurrentLogContextPath(), "ctx1"); + YQL_LOG(INFO) << "level1 - end"; + } + UNIT_ASSERT_STRINGS_EQUAL(CurrentLogContextPath(), ""); + TString row1Str, row2Str, row3Str, row4Str, row5Str, _; - Split(out.Str(), '\n', row1Str, row2Str, row3Str, row4Str, row5Str, _); - - { - TLogRow logRow = ParseLogRow(row1Str); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "{ctx1} level1 - begin"); - } - { - TLogRow logRow = ParseLogRow(row2Str); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::WARN); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "{ctx1/ctx2} level2 - begin"); - } - { - TLogRow logRow = ParseLogRow(row3Str); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::ERROR); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "{ctx3} level3"); - } - { - TLogRow logRow = ParseLogRow(row4Str); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::WARN); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "{ctx1/ctx2} level2 - end"); - } - { - TLogRow logRow = ParseLogRow(row5Str); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "{ctx1} level1 - end"); - } - } - + Split(out.Str(), '\n', row1Str, row2Str, row3Str, row4Str, row5Str, _); + + { + TLogRow logRow = ParseLogRow(row1Str); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "{ctx1} level1 - begin"); + } + { + TLogRow logRow = ParseLogRow(row2Str); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::WARN); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "{ctx1/ctx2} level2 - begin"); + } + { + TLogRow logRow = ParseLogRow(row3Str); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::ERROR); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "{ctx3} level3"); + } + { + TLogRow logRow = ParseLogRow(row4Str); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::WARN); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "{ctx1/ctx2} level2 - end"); + } + { + TLogRow logRow = ParseLogRow(row5Str); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "{ctx1} level1 - end"); + } + } + Y_UNIT_TEST(Profiling) { - TStringStream out; + TStringStream out; YqlLoggerScope logger(&out); - - { - YQL_PROFILE_SCOPE(INFO, "scope1"); - } - - YQL_PROFILE_BLOCK(WARN, "block1") { - Sleep(TDuration::MilliSeconds(2)); - } - - YQL_PROFILE_BLOCK(ERROR, "block2") { - Sleep(TDuration::MilliSeconds(1200)); - } - - bool isExecuted = false; - YQL_PROFILE_BLOCK(TRACE, "block3") { // log will be filtered out - isExecuted = true; - } - UNIT_ASSERT(isExecuted); - + + { + YQL_PROFILE_SCOPE(INFO, "scope1"); + } + + YQL_PROFILE_BLOCK(WARN, "block1") { + Sleep(TDuration::MilliSeconds(2)); + } + + YQL_PROFILE_BLOCK(ERROR, "block2") { + Sleep(TDuration::MilliSeconds(1200)); + } + + bool isExecuted = false; + YQL_PROFILE_BLOCK(TRACE, "block3") { // log will be filtered out + isExecuted = true; + } + UNIT_ASSERT(isExecuted); + TString row1Str, row2Str, row3Str, _; - Split(out.Str(), '\n', row1Str, row2Str, row3Str, _); - - { - TLogRow logRow = ParseLogRow(row1Str); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Perf); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Performance); + Split(out.Str(), '\n', row1Str, row2Str, row3Str, _); + + { + TLogRow logRow = ParseLogRow(row1Str); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Perf); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Performance); std::regex re("Execution of \\[scope1\\] took [0-9\\.]+us"); - bool isMatch = std::regex_match(logRow.Message.c_str(), re); - UNIT_ASSERT_C(isMatch, "Unexpected message: " << logRow.Message); - } - { - TLogRow logRow = ParseLogRow(row2Str); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::WARN); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Perf); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Performance); + bool isMatch = std::regex_match(logRow.Message.c_str(), re); + UNIT_ASSERT_C(isMatch, "Unexpected message: " << logRow.Message); + } + { + TLogRow logRow = ParseLogRow(row2Str); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::WARN); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Perf); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Performance); std::regex re("Execution of \\[block1\\] took [0-9\\.]+ms"); - bool isMatch = std::regex_match(logRow.Message.c_str(), re); - UNIT_ASSERT_C(isMatch, "Unexpected message: " << logRow.Message); - } - { - TLogRow logRow = ParseLogRow(row3Str); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::ERROR); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Perf); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Performance); + bool isMatch = std::regex_match(logRow.Message.c_str(), re); + UNIT_ASSERT_C(isMatch, "Unexpected message: " << logRow.Message); + } + { + TLogRow logRow = ParseLogRow(row3Str); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::ERROR); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Perf); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Performance); std::regex re("Execution of \\[block2\\] took [0-9\\.]+s"); - bool isMatch = std::regex_match(logRow.Message.c_str(), re); - UNIT_ASSERT_C(isMatch, "Unexpected message: " << logRow.Message); - } - } - - - int Func1(int a, char b) { - YQL_PROFILE_FUNC(INFO); - return a + b; - } - - int Func2(int a, char b) { - YQL_PROFILE_FUNCSIG(WARN); - return a + b; - } - + bool isMatch = std::regex_match(logRow.Message.c_str(), re); + UNIT_ASSERT_C(isMatch, "Unexpected message: " << logRow.Message); + } + } + + + int Func1(int a, char b) { + YQL_PROFILE_FUNC(INFO); + return a + b; + } + + int Func2(int a, char b) { + YQL_PROFILE_FUNCSIG(WARN); + return a + b; + } + Y_UNIT_TEST(ProfilingFuncs) { - TStringStream out; + TStringStream out; YqlLoggerScope logger(&out); - - Func1(1, 2); - Func2(1, 2); - + + Func1(1, 2); + Func2(1, 2); + TString row1Str, row2Str, _; - Split(out.Str(), '\n', row1Str, row2Str, _); - - { - TLogRow logRow = ParseLogRow(row1Str); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Perf); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Performance); + Split(out.Str(), '\n', row1Str, row2Str, _); + + { + TLogRow logRow = ParseLogRow(row1Str); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Perf); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Performance); #ifdef _win_ std::regex re("Execution of \\[NTestSuiteTLogTest::Func1\\] took [0-9\\.]+us"); #else std::regex re("Execution of \\[Func1\\] took [0-9\\.]+us"); #endif - bool isMatch = std::regex_match(logRow.Message.c_str(), re); - UNIT_ASSERT_C(isMatch, "Unexpected message: " << logRow.Message); - } - { - TLogRow logRow = ParseLogRow(row2Str); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::WARN); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Perf); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Performance); + bool isMatch = std::regex_match(logRow.Message.c_str(), re); + UNIT_ASSERT_C(isMatch, "Unexpected message: " << logRow.Message); + } + { + TLogRow logRow = ParseLogRow(row2Str); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::WARN); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Perf); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Performance); #ifdef _win_ std::regex re("Execution of \\[int __cdecl NTestSuiteTLogTest::Func2\\(int,char\\)\\] took [0-9\\.]+us"); #else std::regex re("Execution of \\[int NTestSuiteTLogTest::Func2\\(int, char\\)\\] took [0-9\\.]+us"); #endif - bool isMatch = std::regex_match(logRow.Message.c_str(), re); - UNIT_ASSERT_C(isMatch, "Unexpected message: " << logRow.Message); - } - } + bool isMatch = std::regex_match(logRow.Message.c_str(), re); + UNIT_ASSERT_C(isMatch, "Unexpected message: " << logRow.Message); + } + } Y_UNIT_TEST(Limit1) { size_t limit = 0; @@ -536,4 +536,4 @@ Y_UNIT_TEST_SUITE(TLogTest) UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "message3"); } } -} +} diff --git a/ydb/library/yql/utils/log/profile.cpp b/ydb/library/yql/utils/log/profile.cpp index 85562006ed4..ba9556498fd 100644 --- a/ydb/library/yql/utils/log/profile.cpp +++ b/ydb/library/yql/utils/log/profile.cpp @@ -1,45 +1,45 @@ -#include "profile.h" -#include "log.h" - -#include <util/stream/format.h> - - -#define YQL_PERF_LOG(level, file, line) YQL_LOG_IMPL( \ - ::NYql::NLog::YqlLogger(), ::NYql::NLog::EComponent::Perf, level, \ - ::NYql::NLog::TContextPreprocessor, file, line) - - -namespace NYql { -namespace NLog { - -TProfilingScope::~TProfilingScope() { - if (Name_ == nullptr) { - return; - } - - double elapsed = static_cast<double>(::MicroSeconds() - StartedAt_); - TStringBuf unit("us"); - if (elapsed > 1000000) { - elapsed /= 1000000; +#include "profile.h" +#include "log.h" + +#include <util/stream/format.h> + + +#define YQL_PERF_LOG(level, file, line) YQL_LOG_IMPL( \ + ::NYql::NLog::YqlLogger(), ::NYql::NLog::EComponent::Perf, level, \ + ::NYql::NLog::TContextPreprocessor, file, line) + + +namespace NYql { +namespace NLog { + +TProfilingScope::~TProfilingScope() { + if (Name_ == nullptr) { + return; + } + + double elapsed = static_cast<double>(::MicroSeconds() - StartedAt_); + TStringBuf unit("us"); + if (elapsed > 1000000) { + elapsed /= 1000000; unit = TStringBuf("s"); - } else if (elapsed > 1000) { - elapsed /= 1000; + } else if (elapsed > 1000) { + elapsed /= 1000; unit = TStringBuf("ms"); - } - - auto doLog = [&]() { - YQL_PERF_LOG(Level_, File_, Line_) + } + + auto doLog = [&]() { + YQL_PERF_LOG(Level_, File_, Line_) << TStringBuf("Execution of [") << Name_ << TStringBuf("] took ") << Prec(elapsed, 3) << unit; - }; - + }; + if (!LogCtxPath_.empty()) { - YQL_LOG_CTX_ROOT_SCOPE(LogCtxPath_); - doLog(); - } else { - doLog(); - } -} - -} // namspace NLog -} // namspace NYql + YQL_LOG_CTX_ROOT_SCOPE(LogCtxPath_); + doLog(); + } else { + doLog(); + } +} + +} // namspace NLog +} // namspace NYql diff --git a/ydb/library/yql/utils/log/profile.h b/ydb/library/yql/utils/log/profile.h index cda887f6e17..656dd550880 100644 --- a/ydb/library/yql/utils/log/profile.h +++ b/ydb/library/yql/utils/log/profile.h @@ -1,74 +1,74 @@ -#pragma once - -#include "log_level.h" -#include "context.h" - -#include <util/system/datetime.h> - - -#define YQL_PROFILE_SCOPE(level, name) \ - ::NYql::NLog::TProfilingScope Y_GENERATE_UNIQUE_ID(ps)( \ - name, ::NYql::NLog::ELevel::level, __FILE__, __LINE__) - -#define YQL_PROFILE_BLOCK_IMPL(level, name) \ - ::NYql::NLog::TProfilingScope( \ - name, ::NYql::NLog::ELevel::level, __FILE__, __LINE__) - -#define YQL_PROFILE_SCOPE_VAL(level, name) \ - TAutoPtr<::NYql::NLog::TProfilingScope>(new ::NYql::NLog::TProfilingScope(\ - name, ::NYql::NLog::ELevel::level, __FILE__, __LINE__, \ - ::NYql::NLog::CurrentLogContextPath())) - -#define YQL_PROFILE_BIND_VAL(future, scopeVal) \ - future.Apply([scopeVal](const decltype(future)& f) { \ - return f.GetValue(); \ - }); - -#define YQL_PROFILE_BLOCK(level, name) \ - if (auto Y_GENERATE_UNIQUE_ID(t) = YQL_PROFILE_SCOPE_VAL(level, name)) { \ - goto Y_CAT(YQL_LOG_CTX_LABEL, __LINE__); \ - } else Y_CAT(YQL_LOG_CTX_LABEL, __LINE__): - -#define YQL_PROFILE_FUNC(level) YQL_PROFILE_SCOPE(level, __FUNCTION__) -#define YQL_PROFILE_FUNCSIG(level) YQL_PROFILE_SCOPE(level, Y_FUNC_SIGNATURE) - -#define YQL_PROFILE_FUNC_VAL(level) YQL_PROFILE_SCOPE_VAL(level, __FUNCTION__) -#define YQL_PROFILE_FUNCSIG_VAL(level) YQL_PROFILE_SCOPE_VAL(level, Y_FUNC_SIGNATURE) - - -namespace NYql { -namespace NLog { - -/** - * @brief Adds elapsed execution time to log when goes outside of scope. - */ -class TProfilingScope { -public: - TProfilingScope(const char* name, ELevel level, const char* file, int line, +#pragma once + +#include "log_level.h" +#include "context.h" + +#include <util/system/datetime.h> + + +#define YQL_PROFILE_SCOPE(level, name) \ + ::NYql::NLog::TProfilingScope Y_GENERATE_UNIQUE_ID(ps)( \ + name, ::NYql::NLog::ELevel::level, __FILE__, __LINE__) + +#define YQL_PROFILE_BLOCK_IMPL(level, name) \ + ::NYql::NLog::TProfilingScope( \ + name, ::NYql::NLog::ELevel::level, __FILE__, __LINE__) + +#define YQL_PROFILE_SCOPE_VAL(level, name) \ + TAutoPtr<::NYql::NLog::TProfilingScope>(new ::NYql::NLog::TProfilingScope(\ + name, ::NYql::NLog::ELevel::level, __FILE__, __LINE__, \ + ::NYql::NLog::CurrentLogContextPath())) + +#define YQL_PROFILE_BIND_VAL(future, scopeVal) \ + future.Apply([scopeVal](const decltype(future)& f) { \ + return f.GetValue(); \ + }); + +#define YQL_PROFILE_BLOCK(level, name) \ + if (auto Y_GENERATE_UNIQUE_ID(t) = YQL_PROFILE_SCOPE_VAL(level, name)) { \ + goto Y_CAT(YQL_LOG_CTX_LABEL, __LINE__); \ + } else Y_CAT(YQL_LOG_CTX_LABEL, __LINE__): + +#define YQL_PROFILE_FUNC(level) YQL_PROFILE_SCOPE(level, __FUNCTION__) +#define YQL_PROFILE_FUNCSIG(level) YQL_PROFILE_SCOPE(level, Y_FUNC_SIGNATURE) + +#define YQL_PROFILE_FUNC_VAL(level) YQL_PROFILE_SCOPE_VAL(level, __FUNCTION__) +#define YQL_PROFILE_FUNCSIG_VAL(level) YQL_PROFILE_SCOPE_VAL(level, Y_FUNC_SIGNATURE) + + +namespace NYql { +namespace NLog { + +/** + * @brief Adds elapsed execution time to log when goes outside of scope. + */ +class TProfilingScope { +public: + TProfilingScope(const char* name, ELevel level, const char* file, int line, TString logCtxPath = TString()) - : Name_(name) - , Level_(level) - , File_(file) - , Line_(line) - , StartedAt_(::MicroSeconds()) - , LogCtxPath_(std::move(logCtxPath)) - { - } - - ~TProfilingScope(); - - explicit inline operator bool() const noexcept { - return true; - } - -private: - const char* Name_; - ELevel Level_; - const char* File_; - int Line_; - ui64 StartedAt_; + : Name_(name) + , Level_(level) + , File_(file) + , Line_(line) + , StartedAt_(::MicroSeconds()) + , LogCtxPath_(std::move(logCtxPath)) + { + } + + ~TProfilingScope(); + + explicit inline operator bool() const noexcept { + return true; + } + +private: + const char* Name_; + ELevel Level_; + const char* File_; + int Line_; + ui64 StartedAt_; TString LogCtxPath_; -}; - -} // namspace NLog -} // namspace NYql +}; + +} // namspace NLog +} // namspace NYql diff --git a/ydb/library/yql/utils/log/tls_backend.cpp b/ydb/library/yql/utils/log/tls_backend.cpp index d8a0a56f810..a92f123c9bf 100644 --- a/ydb/library/yql/utils/log/tls_backend.cpp +++ b/ydb/library/yql/utils/log/tls_backend.cpp @@ -1,49 +1,49 @@ -#include "tls_backend.h" - -#include <util/system/tls.h> - - -namespace NYql { -namespace NLog { -namespace { - -Y_POD_STATIC_THREAD(TLogBackend*) CurrentBackend; - -} // namspace - -TLogBackend* SetLogBackendForCurrentThread(TLogBackend* backend) { - TLogBackend* prev = *(&CurrentBackend); - *(&CurrentBackend) = backend; - return prev; -} - -void TTlsLogBackend::WriteData(const TLogRecord& rec) { - TLogBackend* backend = *(&CurrentBackend); - if (backend) { - backend->WriteData(rec); - } else { - DefaultBackend_->WriteData(rec); - } -} - -void TTlsLogBackend::ReopenLog() { - TLogBackend* backend = *(&CurrentBackend); - if (backend) { - backend->ReopenLog(); - } else { - DefaultBackend_->ReopenLog(); - } -} - +#include "tls_backend.h" + +#include <util/system/tls.h> + + +namespace NYql { +namespace NLog { +namespace { + +Y_POD_STATIC_THREAD(TLogBackend*) CurrentBackend; + +} // namspace + +TLogBackend* SetLogBackendForCurrentThread(TLogBackend* backend) { + TLogBackend* prev = *(&CurrentBackend); + *(&CurrentBackend) = backend; + return prev; +} + +void TTlsLogBackend::WriteData(const TLogRecord& rec) { + TLogBackend* backend = *(&CurrentBackend); + if (backend) { + backend->WriteData(rec); + } else { + DefaultBackend_->WriteData(rec); + } +} + +void TTlsLogBackend::ReopenLog() { + TLogBackend* backend = *(&CurrentBackend); + if (backend) { + backend->ReopenLog(); + } else { + DefaultBackend_->ReopenLog(); + } +} + ELogPriority TTlsLogBackend::FiltrationLevel() const { - TLogBackend* backend = *(&CurrentBackend); - if (backend) { - return backend->FiltrationLevel(); - } else { - return DefaultBackend_->FiltrationLevel(); - } - return LOG_MAX_PRIORITY; -} - -} // namspace NLog -} // namspace NYql + TLogBackend* backend = *(&CurrentBackend); + if (backend) { + return backend->FiltrationLevel(); + } else { + return DefaultBackend_->FiltrationLevel(); + } + return LOG_MAX_PRIORITY; +} + +} // namspace NLog +} // namspace NYql diff --git a/ydb/library/yql/utils/log/tls_backend.h b/ydb/library/yql/utils/log/tls_backend.h index 1498159ad35..69a0955432c 100644 --- a/ydb/library/yql/utils/log/tls_backend.h +++ b/ydb/library/yql/utils/log/tls_backend.h @@ -1,67 +1,67 @@ -#pragma once - +#pragma once + #include <library/cpp/logger/backend.h> - -#include <util/generic/ptr.h> - -#include <utility> - - -namespace NYql { -namespace NLog { - -/** - * @brief Dispatches all invocations to default logger backend configured - * for current thread. Must be used in conjunction with - * SetLogBackendForCurrentThread() function or TScopedBackend class. - */ -class TTlsLogBackend: public TLogBackend { -public: - TTlsLogBackend(TAutoPtr<TLogBackend> defaultBackend) - : DefaultBackend_(defaultBackend) - { - Y_VERIFY_DEBUG(DefaultBackend_, "default backend is not set"); - } - - void WriteData(const TLogRecord& rec) override; - void ReopenLog() override; + +#include <util/generic/ptr.h> + +#include <utility> + + +namespace NYql { +namespace NLog { + +/** + * @brief Dispatches all invocations to default logger backend configured + * for current thread. Must be used in conjunction with + * SetLogBackendForCurrentThread() function or TScopedBackend class. + */ +class TTlsLogBackend: public TLogBackend { +public: + TTlsLogBackend(TAutoPtr<TLogBackend> defaultBackend) + : DefaultBackend_(defaultBackend) + { + Y_VERIFY_DEBUG(DefaultBackend_, "default backend is not set"); + } + + void WriteData(const TLogRecord& rec) override; + void ReopenLog() override; ELogPriority FiltrationLevel() const override; - -private: - TAutoPtr<TLogBackend> DefaultBackend_; -}; - -/** - * @brief Sets given backend as default for current thread. Must be used in - * conjunction with TTlsLogBackend. - * - * @param backend - pointer to logger backend - * @return previous default logger backend - */ -TLogBackend* SetLogBackendForCurrentThread(TLogBackend* backend); - -/** - * @brief Sets itself as default for current thread on instantiation - * and restores previous one on destruction. Must be used in - * conjunction with TTlsLogBackend. - */ -template <typename TBackend> -class TScopedBackend: public TBackend { -public: - template <typename... TArgs> + +private: + TAutoPtr<TLogBackend> DefaultBackend_; +}; + +/** + * @brief Sets given backend as default for current thread. Must be used in + * conjunction with TTlsLogBackend. + * + * @param backend - pointer to logger backend + * @return previous default logger backend + */ +TLogBackend* SetLogBackendForCurrentThread(TLogBackend* backend); + +/** + * @brief Sets itself as default for current thread on instantiation + * and restores previous one on destruction. Must be used in + * conjunction with TTlsLogBackend. + */ +template <typename TBackend> +class TScopedBackend: public TBackend { +public: + template <typename... TArgs> TScopedBackend(TArgs&&... args) - : TBackend(std::forward<TArgs>(args)...) - , PrevBacked_(SetLogBackendForCurrentThread(this)) - { - } - - ~TScopedBackend() { - SetLogBackendForCurrentThread(PrevBacked_); - } - -private: - TLogBackend* PrevBacked_; -}; - -} // namspace NLog -} // namspace NYql + : TBackend(std::forward<TArgs>(args)...) + , PrevBacked_(SetLogBackendForCurrentThread(this)) + { + } + + ~TScopedBackend() { + SetLogBackendForCurrentThread(PrevBacked_); + } + +private: + TLogBackend* PrevBacked_; +}; + +} // namspace NLog +} // namspace NYql diff --git a/ydb/library/yql/utils/log/tls_backend_ut.cpp b/ydb/library/yql/utils/log/tls_backend_ut.cpp index e55bc630cea..6f4147fed6b 100644 --- a/ydb/library/yql/utils/log/tls_backend_ut.cpp +++ b/ydb/library/yql/utils/log/tls_backend_ut.cpp @@ -1,121 +1,121 @@ -#include "tls_backend.h" -#include "log.h" +#include "tls_backend.h" +#include "log.h" #include <ydb/library/yql/utils/log/ut/log_parser.h> - + #include <library/cpp/testing/unittest/registar.h> #include <library/cpp/logger/stream.h> #include <library/cpp/logger/null.h> - -#include <util/system/thread.h> -#include <util/string/split.h> - -#include <thread> -#include <chrono> - - -using namespace NYql; -using namespace NLog; - -class TRunnable { -public: - TRunnable(TStringBuf name, int count) - : Name_(name) - , Count_(count) - { - } - - void operator()() { - using namespace std::chrono_literals; - - YQL_LOG(INFO) << "this message will be missed"; - { - TScopedBackend<TStreamLogBackend> logBackend(&Logs_); - for (int i = 0; i < Count_; i++) { - YQL_LOG(INFO) << Name_; - std::this_thread::sleep_for(20ms); - } - } - YQL_LOG(INFO) << "this message will be missed"; - } - + +#include <util/system/thread.h> +#include <util/string/split.h> + +#include <thread> +#include <chrono> + + +using namespace NYql; +using namespace NLog; + +class TRunnable { +public: + TRunnable(TStringBuf name, int count) + : Name_(name) + , Count_(count) + { + } + + void operator()() { + using namespace std::chrono_literals; + + YQL_LOG(INFO) << "this message will be missed"; + { + TScopedBackend<TStreamLogBackend> logBackend(&Logs_); + for (int i = 0; i < Count_; i++) { + YQL_LOG(INFO) << Name_; + std::this_thread::sleep_for(20ms); + } + } + YQL_LOG(INFO) << "this message will be missed"; + } + const TString& GetLogs() const { - return Logs_.Str(); - } - -private: + return Logs_.Str(); + } + +private: TString Name_; - int Count_; - TStringStream Logs_; -}; - + int Count_; + TStringStream Logs_; +}; + Y_UNIT_TEST_SUITE(TTlsLogBackendTest) -{ +{ Y_UNIT_TEST(CaptureOutputs) { YqlLoggerScope logger(new TTlsLogBackend(new TNullLogBackend)); - - YQL_LOG(INFO) << "this message will be missed"; - - TRunnable r1("t1", 3); - std::thread t1(std::ref(r1)); - - TRunnable r2("t2", 2); - std::thread t2(std::ref(r2)); - - t1.join(); - t2.join(); - -// Cout << "--[t1 logs]-----------------\n" << r1.GetLogs() << Endl; -// Cout << "--[t2 logs]-----------------\n" << r2.GetLogs() << Endl; - - { // t1 + + YQL_LOG(INFO) << "this message will be missed"; + + TRunnable r1("t1", 3); + std::thread t1(std::ref(r1)); + + TRunnable r2("t2", 2); + std::thread t2(std::ref(r2)); + + t1.join(); + t2.join(); + +// Cout << "--[t1 logs]-----------------\n" << r1.GetLogs() << Endl; +// Cout << "--[t2 logs]-----------------\n" << r2.GetLogs() << Endl; + + { // t1 TString row1Str, row2Str, row3Str, _; - Split(r1.GetLogs(), '\n', row1Str, row2Str, row3Str, _); - - ui64 threadId = 0; - { - TLogRow logRow = ParseLogRow(row1Str); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); - UNIT_ASSERT(logRow.ThreadId > 0); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "t1"); - threadId = logRow.ThreadId; - } - { - TLogRow logRow = ParseLogRow(row2Str); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); - UNIT_ASSERT_EQUAL(logRow.ThreadId, threadId); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "t1"); - } - { - TLogRow logRow = ParseLogRow(row3Str); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); - UNIT_ASSERT_EQUAL(logRow.ThreadId, threadId); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "t1"); - } - } - - { // t2 + Split(r1.GetLogs(), '\n', row1Str, row2Str, row3Str, _); + + ui64 threadId = 0; + { + TLogRow logRow = ParseLogRow(row1Str); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); + UNIT_ASSERT(logRow.ThreadId > 0); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "t1"); + threadId = logRow.ThreadId; + } + { + TLogRow logRow = ParseLogRow(row2Str); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); + UNIT_ASSERT_EQUAL(logRow.ThreadId, threadId); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "t1"); + } + { + TLogRow logRow = ParseLogRow(row3Str); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); + UNIT_ASSERT_EQUAL(logRow.ThreadId, threadId); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "t1"); + } + } + + { // t2 TString row1Str, row2Str, _; - Split(r2.GetLogs(), '\n', row1Str, row2Str, _); - - ui64 threadId = 0; - { - TLogRow logRow = ParseLogRow(row1Str); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); - UNIT_ASSERT(logRow.ThreadId > 0); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "t2"); - threadId = logRow.ThreadId; - } - { - TLogRow logRow = ParseLogRow(row2Str); - UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); - UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); - UNIT_ASSERT_EQUAL(logRow.ThreadId, threadId); - UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "t2"); - } - } - } -} + Split(r2.GetLogs(), '\n', row1Str, row2Str, _); + + ui64 threadId = 0; + { + TLogRow logRow = ParseLogRow(row1Str); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); + UNIT_ASSERT(logRow.ThreadId > 0); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "t2"); + threadId = logRow.ThreadId; + } + { + TLogRow logRow = ParseLogRow(row2Str); + UNIT_ASSERT_EQUAL(logRow.Level, ELevel::INFO); + UNIT_ASSERT_EQUAL(logRow.Component, EComponent::Default); + UNIT_ASSERT_EQUAL(logRow.ThreadId, threadId); + UNIT_ASSERT_STRINGS_EQUAL(logRow.Message, "t2"); + } + } + } +} diff --git a/ydb/library/yql/utils/log/ut/log_parser.h b/ydb/library/yql/utils/log/ut/log_parser.h index 433bf989a40..98d796f0077 100644 --- a/ydb/library/yql/utils/log/ut/log_parser.h +++ b/ydb/library/yql/utils/log/ut/log_parser.h @@ -1,62 +1,62 @@ -#pragma once - +#pragma once + #include <ydb/library/yql/utils/log/log.h> - + #include <library/cpp/testing/unittest/registar.h> - -#include <util/datetime/base.h> - -#include <regex> - - -namespace NYql { -namespace NLog { - -struct TLogRow { - TInstant Time; - ELevel Level; + +#include <util/datetime/base.h> + +#include <regex> + + +namespace NYql { +namespace NLog { + +struct TLogRow { + TInstant Time; + ELevel Level; TString ProcName; - pid_t ProcId; - ui64 ThreadId; - EComponent Component; + pid_t ProcId; + ui64 ThreadId; + EComponent Component; TString FileName; - ui32 LineNumber; + ui32 LineNumber; TString Message; -}; - +}; + static TLogRow ParseLogRow(const TString& str) { - static std::regex rowRe( - "^([0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}\\.[0-9]{3}) " // (1) time - "([A-Z ]{5}) " // (2) level - "([a-zA-Z0-9_\\.-]+)" // (3) process name - ".pid=([0-9]+)," // (4) process id + static std::regex rowRe( + "^([0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}\\.[0-9]{3}) " // (1) time + "([A-Z ]{5}) " // (2) level + "([a-zA-Z0-9_\\.-]+)" // (3) process name + ".pid=([0-9]+)," // (4) process id " tid=(0?x?[0-9a-fA-F]+). " // (5) thread id - ".([a-zA-Z0-9_\\. ]+). " // (6) component name - "([a-z0-9_\\.]+):" // (7) file name - "([0-9]+): " // (8) line number - "([^\n]*)\n?$" // (9) message - , std::regex_constants::extended); - - std::cmatch match; - bool isMatch = std::regex_match(str.c_str(), match, rowRe); - - UNIT_ASSERT_C(isMatch, "log row does not match format: '" << str << '\''); - UNIT_ASSERT_EQUAL_C(match.size(), 10, "expected 10 groups in log row: '" << str << '\''); - - TLogRow logRow; + ".([a-zA-Z0-9_\\. ]+). " // (6) component name + "([a-z0-9_\\.]+):" // (7) file name + "([0-9]+): " // (8) line number + "([^\n]*)\n?$" // (9) message + , std::regex_constants::extended); + + std::cmatch match; + bool isMatch = std::regex_match(str.c_str(), match, rowRe); + + UNIT_ASSERT_C(isMatch, "log row does not match format: '" << str << '\''); + UNIT_ASSERT_EQUAL_C(match.size(), 10, "expected 10 groups in log row: '" << str << '\''); + + TLogRow logRow; logRow.Time = TInstant::ParseIso8601(match[1].str()) - TDuration::Hours(4); - logRow.Level = ELevelHelpers::FromString(match[2].str()); - logRow.ProcName = match[3].str(); - logRow.ProcId = FromString<pid_t>(match[4].str()); + logRow.Level = ELevelHelpers::FromString(match[2].str()); + logRow.ProcName = match[3].str(); + logRow.ProcId = FromString<pid_t>(match[4].str()); logRow.ThreadId = match[5].str().substr(0, 2) == "0x" ? IntFromString<ui64, 16, TStringBuf>(match[5].str().substr(2)) : IntFromString<ui64, 10, TStringBuf>(match[5].str()); - logRow.Component = EComponentHelpers::FromString(match[6].str()); - logRow.FileName = match[7].str(); - logRow.LineNumber = FromString<ui32>(match[8].str()); - logRow.Message = match[9].str(); - return logRow; -} - -} // namspace NLog -} // namspace NYql + logRow.Component = EComponentHelpers::FromString(match[6].str()); + logRow.FileName = match[7].str(); + logRow.LineNumber = FromString<ui32>(match[8].str()); + logRow.Message = match[9].str(); + return logRow; +} + +} // namspace NLog +} // namspace NYql diff --git a/ydb/library/yql/utils/log/ut/ya.make b/ydb/library/yql/utils/log/ut/ya.make index a6b8eb5a82f..b9ec906f056 100644 --- a/ydb/library/yql/utils/log/ut/ya.make +++ b/ydb/library/yql/utils/log/ut/ya.make @@ -1,10 +1,10 @@ UNITTEST_FOR(ydb/library/yql/utils/log) - + OWNER(g:yql) - -SRCS( - log_ut.cpp - tls_backend_ut.cpp -) - -END() + +SRCS( + log_ut.cpp + tls_backend_ut.cpp +) + +END() diff --git a/ydb/library/yql/utils/log/ya.make b/ydb/library/yql/utils/log/ya.make index 22e5218b5b6..e83464fe9fb 100644 --- a/ydb/library/yql/utils/log/ya.make +++ b/ydb/library/yql/utils/log/ya.make @@ -6,10 +6,10 @@ OWNER( ) SRCS( - context.cpp + context.cpp log.cpp - profile.cpp - tls_backend.cpp + profile.cpp + tls_backend.cpp ) PEERDIR( diff --git a/ydb/library/yql/utils/ya.make b/ydb/library/yql/utils/ya.make index bb46750f4dc..afba70be535 100644 --- a/ydb/library/yql/utils/ya.make +++ b/ydb/library/yql/utils/ya.make @@ -1,15 +1,15 @@ -LIBRARY() - +LIBRARY() + OWNER( g:yql g:yql_ydb_core ) - -SRCS( + +SRCS( bind_in_range.cpp bind_in_range.h cast.h - debug_info.cpp + debug_info.cpp debug_info.h future_action.cpp future_action.h @@ -35,15 +35,15 @@ SRCS( yql_paths.cpp yql_paths.h utf8.cpp -) - +) + PEERDIR( library/cpp/digest/md5 library/cpp/threading/future library/cpp/messagebus ) -END() +END() RECURSE( actors diff --git a/ydb/public/api/grpc/draft/ya.make b/ydb/public/api/grpc/draft/ya.make index 43c790a729b..f63be96521c 100644 --- a/ydb/public/api/grpc/draft/ya.make +++ b/ydb/public/api/grpc/draft/ya.make @@ -1,7 +1,7 @@ -PROTO_LIBRARY(api-grpc-draft) +PROTO_LIBRARY(api-grpc-draft) + +MAVEN_GROUP_ID(com.yandex.ydb) -MAVEN_GROUP_ID(com.yandex.ydb) - GRPC() OWNER( diff --git a/ydb/public/api/grpc/draft/ydb_experimental_v1.proto b/ydb/public/api/grpc/draft/ydb_experimental_v1.proto index 06378bf0ea5..003ab6d671a 100644 --- a/ydb/public/api/grpc/draft/ydb_experimental_v1.proto +++ b/ydb/public/api/grpc/draft/ydb_experimental_v1.proto @@ -1,7 +1,7 @@ syntax = "proto3"; package Ydb.Experimental.V1; -option java_package = "com.yandex.ydb.experimental.v1"; +option java_package = "com.yandex.ydb.experimental.v1"; import "ydb/public/api/protos/ydb_experimental.proto"; diff --git a/ydb/public/api/grpc/draft/ydb_persqueue_v1.proto b/ydb/public/api/grpc/draft/ydb_persqueue_v1.proto index 1f2103f0573..873ee38b3cf 100644 --- a/ydb/public/api/grpc/draft/ydb_persqueue_v1.proto +++ b/ydb/public/api/grpc/draft/ydb_persqueue_v1.proto @@ -3,7 +3,7 @@ option cc_enable_arenas = true; package Ydb.PersQueue.V1; -option java_package = "com.yandex.ydb.persqueue.v1"; +option java_package = "com.yandex.ydb.persqueue.v1"; import "ydb/public/api/protos/ydb_persqueue_cluster_discovery.proto"; import "ydb/public/api/protos/ydb_persqueue_v1.proto"; diff --git a/ydb/public/api/grpc/draft/ydb_s3_internal_v1.proto b/ydb/public/api/grpc/draft/ydb_s3_internal_v1.proto index d5a526ffc52..2553c91d37f 100644 --- a/ydb/public/api/grpc/draft/ydb_s3_internal_v1.proto +++ b/ydb/public/api/grpc/draft/ydb_s3_internal_v1.proto @@ -1,7 +1,7 @@ syntax = "proto3"; package Ydb.S3Internal.V1; -option java_package = "com.yandex.ydb.s3_internal.v1"; +option java_package = "com.yandex.ydb.s3_internal.v1"; import "ydb/public/api/protos/ydb_s3_internal.proto"; diff --git a/ydb/public/api/grpc/ya.make b/ydb/public/api/grpc/ya.make index 691e0d5833b..24173a0264d 100644 --- a/ydb/public/api/grpc/ya.make +++ b/ydb/public/api/grpc/ya.make @@ -1,7 +1,7 @@ -PROTO_LIBRARY(api-grpc) +PROTO_LIBRARY(api-grpc) + +MAVEN_GROUP_ID(com.yandex.ydb) -MAVEN_GROUP_ID(com.yandex.ydb) - GRPC() OWNER( diff --git a/ydb/public/api/grpc/ydb_cms_v1.proto b/ydb/public/api/grpc/ydb_cms_v1.proto index 25d9b3f61fc..11e4517381f 100644 --- a/ydb/public/api/grpc/ydb_cms_v1.proto +++ b/ydb/public/api/grpc/ydb_cms_v1.proto @@ -1,7 +1,7 @@ syntax = "proto3"; package Ydb.Cms.V1; -option java_package = "com.yandex.ydb.cms.v1"; +option java_package = "com.yandex.ydb.cms.v1"; import "ydb/public/api/protos/ydb_cms.proto"; diff --git a/ydb/public/api/grpc/ydb_coordination_v1.proto b/ydb/public/api/grpc/ydb_coordination_v1.proto index cbd5c9d23f4..c4120a2cd02 100644 --- a/ydb/public/api/grpc/ydb_coordination_v1.proto +++ b/ydb/public/api/grpc/ydb_coordination_v1.proto @@ -2,7 +2,7 @@ syntax = "proto3"; package Ydb.Coordination.V1; -option java_package = "com.yandex.ydb.coordination.v1"; +option java_package = "com.yandex.ydb.coordination.v1"; option java_outer_classname = "CoordinationGrpc"; option java_multiple_files = true; diff --git a/ydb/public/api/grpc/ydb_discovery_v1.proto b/ydb/public/api/grpc/ydb_discovery_v1.proto index 15fb6b8f9a1..dc06a4f6788 100644 --- a/ydb/public/api/grpc/ydb_discovery_v1.proto +++ b/ydb/public/api/grpc/ydb_discovery_v1.proto @@ -1,7 +1,7 @@ syntax = "proto3"; package Ydb.Discovery.V1; -option java_package = "com.yandex.ydb.discovery.v1"; +option java_package = "com.yandex.ydb.discovery.v1"; import "ydb/public/api/protos/ydb_discovery.proto"; diff --git a/ydb/public/api/grpc/ydb_operation_v1.proto b/ydb/public/api/grpc/ydb_operation_v1.proto index e5056470248..8202d084843 100644 --- a/ydb/public/api/grpc/ydb_operation_v1.proto +++ b/ydb/public/api/grpc/ydb_operation_v1.proto @@ -1,7 +1,7 @@ syntax = "proto3"; package Ydb.Operation.V1; -option java_package = "com.yandex.ydb.operation.v1"; +option java_package = "com.yandex.ydb.operation.v1"; import "ydb/public/api/protos/ydb_operation.proto"; diff --git a/ydb/public/api/grpc/ydb_scheme_v1.proto b/ydb/public/api/grpc/ydb_scheme_v1.proto index acda5995249..634b16e8493 100644 --- a/ydb/public/api/grpc/ydb_scheme_v1.proto +++ b/ydb/public/api/grpc/ydb_scheme_v1.proto @@ -1,7 +1,7 @@ syntax = "proto3"; package Ydb.Scheme.V1; -option java_package = "com.yandex.ydb.scheme.v1"; +option java_package = "com.yandex.ydb.scheme.v1"; import "ydb/public/api/protos/ydb_scheme.proto"; diff --git a/ydb/public/api/grpc/ydb_scripting_v1.proto b/ydb/public/api/grpc/ydb_scripting_v1.proto index b38dd7a28dd..c8743a07de3 100644 --- a/ydb/public/api/grpc/ydb_scripting_v1.proto +++ b/ydb/public/api/grpc/ydb_scripting_v1.proto @@ -1,7 +1,7 @@ syntax = "proto3"; package Ydb.Scripting.V1; -option java_package = "com.yandex.ydb.scripting.v1"; +option java_package = "com.yandex.ydb.scripting.v1"; import "ydb/public/api/protos/ydb_scripting.proto"; diff --git a/ydb/public/api/grpc/ydb_table_v1.proto b/ydb/public/api/grpc/ydb_table_v1.proto index f9b64c350ae..dbbf62c5c8e 100644 --- a/ydb/public/api/grpc/ydb_table_v1.proto +++ b/ydb/public/api/grpc/ydb_table_v1.proto @@ -1,7 +1,7 @@ syntax = "proto3"; package Ydb.Table.V1; -option java_package = "com.yandex.ydb.table.v1"; +option java_package = "com.yandex.ydb.table.v1"; import "ydb/public/api/protos/ydb_table.proto"; diff --git a/ydb/public/api/protos/draft/persqueue_error_codes.proto b/ydb/public/api/protos/draft/persqueue_error_codes.proto index d09af6b616a..8b2a0986314 100644 --- a/ydb/public/api/protos/draft/persqueue_error_codes.proto +++ b/ydb/public/api/protos/draft/persqueue_error_codes.proto @@ -1,7 +1,7 @@ syntax = "proto3"; package NPersQueue.NErrorCode; -option java_package = "com.yandex.ydb.persqueue"; +option java_package = "com.yandex.ydb.persqueue"; enum EErrorCode { OK = 0; diff --git a/ydb/public/api/protos/persqueue_error_codes_v1.proto b/ydb/public/api/protos/persqueue_error_codes_v1.proto index 476c9c3f993..c6658305c2e 100644 --- a/ydb/public/api/protos/persqueue_error_codes_v1.proto +++ b/ydb/public/api/protos/persqueue_error_codes_v1.proto @@ -1,7 +1,7 @@ syntax = "proto3"; package Ydb.PersQueue.ErrorCode; -option java_package = "com.yandex.ydb.persqueue.errorcode"; +option java_package = "com.yandex.ydb.persqueue.errorcode"; //500000..500999 enum ErrorCode { diff --git a/ydb/public/api/protos/ya.make b/ydb/public/api/protos/ya.make index 5c5895d7855..8a156403a3d 100644 --- a/ydb/public/api/protos/ya.make +++ b/ydb/public/api/protos/ya.make @@ -1,6 +1,6 @@ -PROTO_LIBRARY(api-protos) +PROTO_LIBRARY(api-protos) -MAVEN_GROUP_ID(com.yandex.ydb) +MAVEN_GROUP_ID(com.yandex.ydb) OWNER( dcherednik diff --git a/ydb/public/api/protos/ydb_cms.proto b/ydb/public/api/protos/ydb_cms.proto index 0568c20d4e4..1a569c70812 100644 --- a/ydb/public/api/protos/ydb_cms.proto +++ b/ydb/public/api/protos/ydb_cms.proto @@ -2,7 +2,7 @@ syntax = "proto3"; option cc_enable_arenas = true; package Ydb.Cms; -option java_package = "com.yandex.ydb.cms"; +option java_package = "com.yandex.ydb.cms"; import "ydb/public/api/protos/ydb_operation.proto"; diff --git a/ydb/public/api/protos/ydb_common.proto b/ydb/public/api/protos/ydb_common.proto index 3fd2de93c4e..a63e202f821 100644 --- a/ydb/public/api/protos/ydb_common.proto +++ b/ydb/public/api/protos/ydb_common.proto @@ -2,7 +2,7 @@ syntax = "proto3"; option cc_enable_arenas = true; package Ydb; -option java_package = "com.yandex.ydb.common"; +option java_package = "com.yandex.ydb.common"; option java_outer_classname = "CommonProtos"; message FeatureFlag { diff --git a/ydb/public/api/protos/ydb_coordination.proto b/ydb/public/api/protos/ydb_coordination.proto index b6a013c7511..378045e78e8 100644 --- a/ydb/public/api/protos/ydb_coordination.proto +++ b/ydb/public/api/protos/ydb_coordination.proto @@ -3,7 +3,7 @@ option cc_enable_arenas = true; package Ydb.Coordination; -option java_package = "com.yandex.ydb.coordination"; +option java_package = "com.yandex.ydb.coordination"; option java_outer_classname = "CoordinationProtos"; option java_multiple_files = true; diff --git a/ydb/public/api/protos/ydb_discovery.proto b/ydb/public/api/protos/ydb_discovery.proto index 1dcf62b1ef8..5577dafe36d 100644 --- a/ydb/public/api/protos/ydb_discovery.proto +++ b/ydb/public/api/protos/ydb_discovery.proto @@ -2,7 +2,7 @@ syntax = "proto3"; option cc_enable_arenas = true; package Ydb.Discovery; -option java_package = "com.yandex.ydb.discovery"; +option java_package = "com.yandex.ydb.discovery"; option java_outer_classname = "DiscoveryProtos"; import "ydb/public/api/protos/ydb_operation.proto"; diff --git a/ydb/public/api/protos/ydb_experimental.proto b/ydb/public/api/protos/ydb_experimental.proto index 8262b788060..94eed9671b7 100644 --- a/ydb/public/api/protos/ydb_experimental.proto +++ b/ydb/public/api/protos/ydb_experimental.proto @@ -2,7 +2,7 @@ syntax = "proto3"; option cc_enable_arenas = true; package Ydb.Experimental; -option java_package = "com.yandex.ydb.experimental"; +option java_package = "com.yandex.ydb.experimental"; option java_outer_classname = "ExperimentalProtos"; import "ydb/public/api/protos/ydb_issue_message.proto"; diff --git a/ydb/public/api/protos/ydb_issue_message.proto b/ydb/public/api/protos/ydb_issue_message.proto index b0af8a84f89..1c294d7c593 100644 --- a/ydb/public/api/protos/ydb_issue_message.proto +++ b/ydb/public/api/protos/ydb_issue_message.proto @@ -2,7 +2,7 @@ syntax = "proto3"; option cc_enable_arenas = true; package Ydb.Issue; -option java_package = "com.yandex.ydb"; +option java_package = "com.yandex.ydb"; // IssueMessage is a transport format for ydb/library/yql/public/issue library message IssueMessage { diff --git a/ydb/public/api/protos/ydb_operation.proto b/ydb/public/api/protos/ydb_operation.proto index c940303f07b..30ddd002a2e 100644 --- a/ydb/public/api/protos/ydb_operation.proto +++ b/ydb/public/api/protos/ydb_operation.proto @@ -11,7 +11,7 @@ import "ydb/public/api/protos/ydb_status_codes.proto"; package Ydb.Operations; -option java_package = "com.yandex.ydb"; +option java_package = "com.yandex.ydb"; option java_outer_classname = "OperationProtos"; message OperationParams { diff --git a/ydb/public/api/protos/ydb_persqueue_v1.proto b/ydb/public/api/protos/ydb_persqueue_v1.proto index 60f8cb56e82..93a7fb6c79d 100644 --- a/ydb/public/api/protos/ydb_persqueue_v1.proto +++ b/ydb/public/api/protos/ydb_persqueue_v1.proto @@ -7,7 +7,7 @@ import "ydb/public/api/protos/annotations/validation.proto"; package Ydb.PersQueue.V1; -option java_package = "com.yandex.ydb.persqueue"; +option java_package = "com.yandex.ydb.persqueue"; option cc_enable_arenas = true; diff --git a/ydb/public/api/protos/ydb_query_stats.proto b/ydb/public/api/protos/ydb_query_stats.proto index 3a02f8e55d8..300d5d9837c 100644 --- a/ydb/public/api/protos/ydb_query_stats.proto +++ b/ydb/public/api/protos/ydb_query_stats.proto @@ -2,7 +2,7 @@ syntax = "proto3"; option cc_enable_arenas = true; package Ydb.TableStats; -option java_package = "com.yandex.ydb"; +option java_package = "com.yandex.ydb"; // Describes select, update (insert, upsert, replace) and delete operations message OperationStats { diff --git a/ydb/public/api/protos/ydb_s3_internal.proto b/ydb/public/api/protos/ydb_s3_internal.proto index f54381ce2b0..0f12fe63028 100644 --- a/ydb/public/api/protos/ydb_s3_internal.proto +++ b/ydb/public/api/protos/ydb_s3_internal.proto @@ -2,7 +2,7 @@ syntax = "proto3"; option cc_enable_arenas = true; package Ydb.S3Internal; -option java_package = "com.yandex.ydb.s3_internal"; +option java_package = "com.yandex.ydb.s3_internal"; option java_outer_classname = "S3InternalProtos"; import "ydb/public/api/protos/ydb_operation.proto"; diff --git a/ydb/public/api/protos/ydb_scheme.proto b/ydb/public/api/protos/ydb_scheme.proto index 381c9e868ec..d66b765725c 100644 --- a/ydb/public/api/protos/ydb_scheme.proto +++ b/ydb/public/api/protos/ydb_scheme.proto @@ -2,7 +2,7 @@ syntax = "proto3"; option cc_enable_arenas = true; package Ydb.Scheme; -option java_package = "com.yandex.ydb.scheme"; +option java_package = "com.yandex.ydb.scheme"; option java_outer_classname = "SchemeOperationProtos"; import "ydb/public/api/protos/ydb_operation.proto"; diff --git a/ydb/public/api/protos/ydb_scripting.proto b/ydb/public/api/protos/ydb_scripting.proto index 497b95842d3..4c1a2954185 100644 --- a/ydb/public/api/protos/ydb_scripting.proto +++ b/ydb/public/api/protos/ydb_scripting.proto @@ -2,7 +2,7 @@ syntax = "proto3"; option cc_enable_arenas = true; package Ydb.Scripting; -option java_package = "com.yandex.ydb.scripting"; +option java_package = "com.yandex.ydb.scripting"; option java_outer_classname = "ScriptingProtos"; import "ydb/public/api/protos/ydb_operation.proto"; diff --git a/ydb/public/api/protos/ydb_status_codes.proto b/ydb/public/api/protos/ydb_status_codes.proto index 40974a7586c..4c500dc43ac 100644 --- a/ydb/public/api/protos/ydb_status_codes.proto +++ b/ydb/public/api/protos/ydb_status_codes.proto @@ -1,7 +1,7 @@ syntax = "proto3"; package Ydb; -option java_package = "com.yandex.ydb"; +option java_package = "com.yandex.ydb"; option java_outer_classname = "StatusCodesProtos"; message StatusIds { diff --git a/ydb/public/api/protos/ydb_table.proto b/ydb/public/api/protos/ydb_table.proto index 52a4681eebe..c8063d9475c 100644 --- a/ydb/public/api/protos/ydb_table.proto +++ b/ydb/public/api/protos/ydb_table.proto @@ -15,7 +15,7 @@ import "google/protobuf/empty.proto"; import "google/protobuf/timestamp.proto"; package Ydb.Table; -option java_package = "com.yandex.ydb.table"; +option java_package = "com.yandex.ydb.table"; // Create new session message CreateSessionRequest { diff --git a/ydb/public/api/protos/ydb_value.proto b/ydb/public/api/protos/ydb_value.proto index a85d0e997ea..d885d73f8e1 100644 --- a/ydb/public/api/protos/ydb_value.proto +++ b/ydb/public/api/protos/ydb_value.proto @@ -4,7 +4,7 @@ option cc_enable_arenas = true; import "google/protobuf/struct.proto"; package Ydb; -option java_package = "com.yandex.ydb"; +option java_package = "com.yandex.ydb"; option java_outer_classname = "ValueProtos"; message DecimalType { diff --git a/ydb/public/lib/base/msgbus.h b/ydb/public/lib/base/msgbus.h index 1a8a74e8d64..f11bd9771f1 100644 --- a/ydb/public/lib/base/msgbus.h +++ b/ydb/public/lib/base/msgbus.h @@ -274,8 +274,8 @@ IMessageBusServer* CreateMsgBusTracingServer( ui32 bindPort = TProtocol::DefaultPort ); -inline NActors::TActorId CreateMsgBusProxyId() { - return NActors::TActorId(0, "MsgBusProxy"); +inline NActors::TActorId CreateMsgBusProxyId() { + return NActors::TActorId(0, "MsgBusProxy"); } } diff --git a/ydb/public/lib/deprecated/kicli/cpp_ut.cpp b/ydb/public/lib/deprecated/kicli/cpp_ut.cpp index 15429e67eda..cb7dc3ed6d9 100644 --- a/ydb/public/lib/deprecated/kicli/cpp_ut.cpp +++ b/ydb/public/lib/deprecated/kicli/cpp_ut.cpp @@ -963,11 +963,11 @@ Y_UNIT_TEST_SUITE(ClientLib) { auto begin = shardRange.MutableType()->MutableStruct()->AddMember(); begin->SetName("Begin"); begin->MutableType()->SetKind(NKikimrMiniKQL::ETypeKind::Data); - begin->MutableType()->MutableData()->SetScheme(NScheme::NTypeIds::Uint32); + begin->MutableType()->MutableData()->SetScheme(NScheme::NTypeIds::Uint32); auto end = shardRange.MutableType()->MutableStruct()->AddMember(); end->SetName("End"); end->MutableType()->SetKind(NKikimrMiniKQL::ETypeKind::Data); - end->MutableType()->MutableData()->SetScheme(NScheme::NTypeIds::Uint32); + end->MutableType()->MutableData()->SetScheme(NScheme::NTypeIds::Uint32); } { //set value diff --git a/ydb/public/lib/deprecated/kicli/schema.cpp b/ydb/public/lib/deprecated/kicli/schema.cpp index 0dc8b1e4cdc..d1ee0384f8b 100644 --- a/ydb/public/lib/deprecated/kicli/schema.cpp +++ b/ydb/public/lib/deprecated/kicli/schema.cpp @@ -40,13 +40,13 @@ TType::TType(NScheme::TTypeId typeId) , TypeId(typeId) {} -const TType TType::Int64(NScheme::NTypeIds::Int64); -const TType TType::Uint64(NScheme::NTypeIds::Uint64); -const TType TType::Int32(NScheme::NTypeIds::Int32); -const TType TType::Uint32(NScheme::NTypeIds::Uint32); -const TType TType::Bool(NScheme::NTypeIds::Bool); -const TType TType::Double(NScheme::NTypeIds::Double); -const TType TType::Float(NScheme::NTypeIds::Float); +const TType TType::Int64(NScheme::NTypeIds::Int64); +const TType TType::Uint64(NScheme::NTypeIds::Uint64); +const TType TType::Int32(NScheme::NTypeIds::Int32); +const TType TType::Uint32(NScheme::NTypeIds::Uint32); +const TType TType::Bool(NScheme::NTypeIds::Bool); +const TType TType::Double(NScheme::NTypeIds::Double); +const TType TType::Float(NScheme::NTypeIds::Float); const TType TType::Utf8(NScheme::NTypeIds::Utf8); const TType TType::String(NScheme::NTypeIds::String); const TType TType::String4k(NScheme::NTypeIds::String4k); diff --git a/ydb/public/lib/deprecated/ya.make b/ydb/public/lib/deprecated/ya.make index 685439aa8ea..84dab1af07b 100644 --- a/ydb/public/lib/deprecated/ya.make +++ b/ydb/public/lib/deprecated/ya.make @@ -1,4 +1,4 @@ RECURSE( - client + client kicli ) diff --git a/ydb/public/lib/scheme_types/scheme_type_id.h b/ydb/public/lib/scheme_types/scheme_type_id.h index 87ad2b9be5a..22fada2a7da 100644 --- a/ydb/public/lib/scheme_types/scheme_type_id.h +++ b/ydb/public/lib/scheme_types/scheme_type_id.h @@ -1,20 +1,20 @@ -#pragma once - -#include <util/system/types.h> +#pragma once + +#include <util/system/types.h> #include <util/generic/array_size.h> #include <util/generic/strbuf.h> #include <ydb/library/yql/public/types/yql_types.pb.h> - -namespace NKikimr { -namespace NScheme { - + +namespace NKikimr { +namespace NScheme { + constexpr ui32 DECIMAL_PRECISION = 22; constexpr ui32 DECIMAL_SCALE = 9; -using TTypeId = ui16; - -namespace NTypeIds { - +using TTypeId = ui16; + +namespace NTypeIds { + static constexpr TTypeId Int32 = NYql::NProto::Int32; static constexpr TTypeId Uint32 = NYql::NProto::Uint32; static constexpr TTypeId Int64 = NYql::NProto::Int64; @@ -25,25 +25,25 @@ static constexpr TTypeId Int8 = NYql::NProto::Int8; static constexpr TTypeId Uint8 = NYql::NProto::Uint8; static constexpr TTypeId Int16 = NYql::NProto::Int16; static constexpr TTypeId Uint16 = NYql::NProto::Uint16; - + static constexpr TTypeId Double = NYql::NProto::Double; static constexpr TTypeId Float = NYql::NProto::Float; - + static constexpr TTypeId Date = NYql::NProto::Date; // days since 1970 static constexpr TTypeId Datetime = NYql::NProto::Datetime; // seconds since 1970 static constexpr TTypeId Timestamp = NYql::NProto::Timestamp; // microseconds since 1970 aka TInstant static constexpr TTypeId Interval = NYql::NProto::Interval; // microseconds aka TDuration, signed static constexpr TTypeId PairUi64Ui64 = 0x101; // DEPRECATED, don't use - + static constexpr TTypeId String = NYql::NProto::String; static constexpr TTypeId String4k = 0x1011; static constexpr TTypeId String2m = 0x1012; - + static constexpr TTypeId Utf8 = NYql::NProto::Utf8; static constexpr TTypeId Yson = NYql::NProto::Yson; static constexpr TTypeId Json = NYql::NProto::Json; - + static constexpr TTypeId JsonDocument = NYql::NProto::JsonDocument; static constexpr TTypeId DyNumber = NYql::NProto::DyNumber; @@ -121,5 +121,5 @@ const char *TypeName(TTypeId typeId) { } } -} // namspace NScheme -} // namspace NKikimr +} // namspace NScheme +} // namspace NKikimr diff --git a/ydb/public/lib/value/value.cpp b/ydb/public/lib/value/value.cpp index 67f8c5d9075..0ed8b826895 100644 --- a/ydb/public/lib/value/value.cpp +++ b/ydb/public/lib/value/value.cpp @@ -251,7 +251,7 @@ TWriteValue& TWriteValue::Interval(i64 value) { TWriteValue& TWriteValue::operator =(bool value) { Type.SetKind(NKikimrMiniKQL::ETypeKind::Data); - Type.MutableData()->SetScheme(NScheme::NTypeIds::Bool); + Type.MutableData()->SetScheme(NScheme::NTypeIds::Bool); Value.SetBool(value); return *this; } @@ -286,21 +286,21 @@ TWriteValue& TWriteValue::operator =(i16 value) { TWriteValue& TWriteValue::operator =(ui64 value) { Type.SetKind(NKikimrMiniKQL::ETypeKind::Data); - Type.MutableData()->SetScheme(NScheme::NTypeIds::Uint64); + Type.MutableData()->SetScheme(NScheme::NTypeIds::Uint64); Value.SetUint64(value); return *this; } TWriteValue& TWriteValue::operator =(i64 value) { Type.SetKind(NKikimrMiniKQL::ETypeKind::Data); - Type.MutableData()->SetScheme(NScheme::NTypeIds::Int64); + Type.MutableData()->SetScheme(NScheme::NTypeIds::Int64); Value.SetInt64(value); return *this; } TWriteValue& TWriteValue::operator =(ui32 value) { Type.SetKind(NKikimrMiniKQL::ETypeKind::Data); - Type.MutableData()->SetScheme(NScheme::NTypeIds::Uint32); + Type.MutableData()->SetScheme(NScheme::NTypeIds::Uint32); Value.SetUint32(value); return *this; } @@ -314,14 +314,14 @@ TWriteValue& TWriteValue::operator =(i32 value) { TWriteValue& TWriteValue::operator =(double value) { Type.SetKind(NKikimrMiniKQL::ETypeKind::Data); - Type.MutableData()->SetScheme(NScheme::NTypeIds::Double); + Type.MutableData()->SetScheme(NScheme::NTypeIds::Double); Value.SetDouble(value); return *this; } TWriteValue& TWriteValue::operator =(float value) { Type.SetKind(NKikimrMiniKQL::ETypeKind::Data); - Type.MutableData()->SetScheme(NScheme::NTypeIds::Float); + Type.MutableData()->SetScheme(NScheme::NTypeIds::Float); Value.SetFloat(value); return *this; } @@ -377,23 +377,23 @@ NScheme::TTypeId TValue::GetDataType() const { TString TValue::GetDataText() const { Y_ASSERT(Type.GetKind() == NKikimrMiniKQL::ETypeKind::Data); switch (Type.GetData().GetScheme()) { - case NScheme::NTypeIds::Bool: + case NScheme::NTypeIds::Bool: return Value.GetBool() ? "true" : "false"; - case NScheme::NTypeIds::Uint64: + case NScheme::NTypeIds::Uint64: return ToString(Value.GetUint64()); - case NScheme::NTypeIds::Int64: + case NScheme::NTypeIds::Int64: return ToString(Value.GetInt64()); - case NScheme::NTypeIds::Uint32: + case NScheme::NTypeIds::Uint32: case NScheme::NTypeIds::Uint16: case NScheme::NTypeIds::Uint8: return ToString(Value.GetUint32()); - case NScheme::NTypeIds::Int32: + case NScheme::NTypeIds::Int32: case NScheme::NTypeIds::Int16: case NScheme::NTypeIds::Int8: return ToString(Value.GetInt32()); - case NScheme::NTypeIds::Double: + case NScheme::NTypeIds::Double: return ToString(Value.GetDouble()); - case NScheme::NTypeIds::Float: + case NScheme::NTypeIds::Float: return ToString(Value.GetFloat()); case NScheme::NTypeIds::Utf8: case NScheme::NTypeIds::Json: @@ -874,43 +874,43 @@ TValue TValue::EatOptional() const { } TValue::operator ui64() const { - Y_ASSERT(Type.GetData().GetScheme() == NScheme::NTypeIds::Uint64); + Y_ASSERT(Type.GetData().GetScheme() == NScheme::NTypeIds::Uint64); Y_ASSERT(Value.HasUint64()); return Value.GetUint64(); } TValue::operator i64() const { - Y_ASSERT(Type.GetData().GetScheme() == NScheme::NTypeIds::Int64); + Y_ASSERT(Type.GetData().GetScheme() == NScheme::NTypeIds::Int64); Y_ASSERT(Value.HasInt64()); return Value.GetInt64(); } TValue::operator ui32() const { - Y_ASSERT(Type.GetData().GetScheme() == NScheme::NTypeIds::Uint32); + Y_ASSERT(Type.GetData().GetScheme() == NScheme::NTypeIds::Uint32); Y_ASSERT(Value.HasUint32()); return Value.GetUint32(); } TValue::operator i32() const { - Y_ASSERT(Type.GetData().GetScheme() == NScheme::NTypeIds::Int32); + Y_ASSERT(Type.GetData().GetScheme() == NScheme::NTypeIds::Int32); Y_ASSERT(Value.HasInt32()); return Value.GetInt32(); } TValue::operator double() const { - Y_ASSERT(Type.GetData().GetScheme() == NScheme::NTypeIds::Double); + Y_ASSERT(Type.GetData().GetScheme() == NScheme::NTypeIds::Double); Y_ASSERT(Value.HasDouble()); return Value.GetDouble(); } TValue::operator float() const { - Y_ASSERT(Type.GetData().GetScheme() == NScheme::NTypeIds::Float); + Y_ASSERT(Type.GetData().GetScheme() == NScheme::NTypeIds::Float); Y_ASSERT(Value.HasFloat()); return Value.GetFloat(); } TValue::operator bool() const { - Y_ASSERT(Type.GetData().GetScheme() == NScheme::NTypeIds::Bool); + Y_ASSERT(Type.GetData().GetScheme() == NScheme::NTypeIds::Bool); Y_ASSERT(Value.HasBool()); return Value.GetBool(); } @@ -944,7 +944,7 @@ TValue::operator TString() const { || Type.GetData().GetScheme() == NScheme::NTypeIds::String || Type.GetData().GetScheme() == NScheme::NTypeIds::String4k || Type.GetData().GetScheme() == NScheme::NTypeIds::String2m - || Type.GetData().GetScheme() == NScheme::NTypeIds::Yson + || Type.GetData().GetScheme() == NScheme::NTypeIds::Yson || Type.GetData().GetScheme() == NScheme::NTypeIds::Json ); Y_ASSERT(Value.HasText() || Value.HasBytes()); diff --git a/ydb/public/lib/ydb_cli/commands/ydb_command.cpp b/ydb/public/lib/ydb_cli/commands/ydb_command.cpp index e99feba81db..e1b8fab3e6f 100644 --- a/ydb/public/lib/ydb_cli/commands/ydb_command.cpp +++ b/ydb/public/lib/ydb_cli/commands/ydb_command.cpp @@ -11,7 +11,7 @@ TYdbCommand::TYdbCommand(const TString& name, const std::initializer_list<TStrin TDriver TYdbCommand::CreateDriver(TConfig& config) { auto driverConfig = TDriverConfig() .SetEndpoint(config.Address) - .SetDatabase(config.Database) + .SetDatabase(config.Database) .SetCredentialsProviderFactory(config.CredentialsGetter(config)); if (config.EnableSsl) { driverConfig.UseSecureConnection(config.CaCerts); diff --git a/ydb/public/lib/ydb_cli/common/command.h b/ydb/public/lib/ydb_cli/common/command.h index 605f568e3fd..441206cdf7b 100644 --- a/ydb/public/lib/ydb_cli/common/command.h +++ b/ydb/public/lib/ydb_cli/common/command.h @@ -3,7 +3,7 @@ #include "common.h" #include <ydb/public/sdk/cpp/client/ydb_types/credentials/credentials.h> - + #include <library/cpp/getopt/last_getopt.h> #include <library/cpp/colorizer/colors.h> #include <util/generic/strbuf.h> @@ -94,9 +94,9 @@ public: TLoginCredentialsParams StaticCredentials; TString YCToken; - bool UseMetadataCredentials = false; + bool UseMetadataCredentials = false; TString SaKeyFile; - TString IamEndpoint; + TString IamEndpoint; TString YScope; TString YdbDir; diff --git a/ydb/public/sdk/cpp/client/extensions/solomon_stats/README.md b/ydb/public/sdk/cpp/client/extensions/solomon_stats/README.md index 674a415b428..869678d8dd7 100644 --- a/ydb/public/sdk/cpp/client/extensions/solomon_stats/README.md +++ b/ydb/public/sdk/cpp/client/extensions/solomon_stats/README.md @@ -69,9 +69,9 @@ Select a method which is right for you: ... -void AddMetricRegistry(NYdb::TDriver& driver, NMonitoring::IMetricRegistry* ptr); -void AddMetricRegistry(NYdb::TDriver& driver, std::shared_ptr<NMonitoring::IMetricRegistry> ptr); -void AddMetricRegistry(NYdb::TDriver& driver, TAtomicSharedPtr<NMonitoring::IMetricRegistry> ptr); +void AddMetricRegistry(NYdb::TDriver& driver, NMonitoring::IMetricRegistry* ptr); +void AddMetricRegistry(NYdb::TDriver& driver, std::shared_ptr<NMonitoring::IMetricRegistry> ptr); +void AddMetricRegistry(NYdb::TDriver& driver, TAtomicSharedPtr<NMonitoring::IMetricRegistry> ptr); ``` If you provide a raw pointer, it's your responsibility to delete the registry. You must shutdown the SDK driver before destroying the registry. diff --git a/ydb/public/sdk/cpp/client/extensions/solomon_stats/pull_client.cpp b/ydb/public/sdk/cpp/client/extensions/solomon_stats/pull_client.cpp index 426f043155c..ec7b0f0db68 100644 --- a/ydb/public/sdk/cpp/client/extensions/solomon_stats/pull_client.cpp +++ b/ydb/public/sdk/cpp/client/extensions/solomon_stats/pull_client.cpp @@ -34,9 +34,9 @@ void TSolomonStatPullExtension::TSolomonStatPage::Output(NMonitoring::IMonHttpRe } TSolomonStatPullExtension::TSolomonStatPullExtension(const TSolomonStatPullExtension::TParams& params, IApi* api) - : MetricRegistry_(new NMonitoring::TMetricRegistry(params.GetLabels())) + : MetricRegistry_(new NMonitoring::TMetricRegistry(params.GetLabels())) , MonService_(params.Port_, params.Host_, 0), Page_( new TSolomonStatPage("stats", "Statistics", api) ) { - api->SetMetricRegistry(MetricRegistry_.get()); + api->SetMetricRegistry(MetricRegistry_.get()); MonService_.Register(Page_); MonService_.StartOrThrow(); } diff --git a/ydb/public/sdk/cpp/client/extensions/solomon_stats/pull_client.h b/ydb/public/sdk/cpp/client/extensions/solomon_stats/pull_client.h index d3ee3444be2..8d410c2fa2e 100644 --- a/ydb/public/sdk/cpp/client/extensions/solomon_stats/pull_client.h +++ b/ydb/public/sdk/cpp/client/extensions/solomon_stats/pull_client.h @@ -3,11 +3,11 @@ #include <ydb/public/sdk/cpp/client/ydb_extension/extension.h> #include <library/cpp/http/server/response.h> -#include <library/cpp/monlib/metrics/metric_consumer.h> -#include <library/cpp/monlib/encode/json/json.h> -#include <library/cpp/monlib/metrics/metric_registry.h> -#include <library/cpp/monlib/service/pages/mon_page.h> -#include <library/cpp/monlib/service/monservice.h> +#include <library/cpp/monlib/metrics/metric_consumer.h> +#include <library/cpp/monlib/encode/json/json.h> +#include <library/cpp/monlib/metrics/metric_registry.h> +#include <library/cpp/monlib/service/pages/mon_page.h> +#include <library/cpp/monlib/service/monservice.h> #include <util/generic/vector.h> @@ -52,7 +52,7 @@ private: }; private: - std::shared_ptr<NMonitoring::TMetricRegistry> MetricRegistry_; + std::shared_ptr<NMonitoring::TMetricRegistry> MetricRegistry_; NMonitoring::TMonService2 MonService_; TIntrusivePtr<TSolomonStatPage> Page_; }; diff --git a/ydb/public/sdk/cpp/client/extensions/solomon_stats/pull_connector.h b/ydb/public/sdk/cpp/client/extensions/solomon_stats/pull_connector.h index b939afda8bf..24f5f30069d 100644 --- a/ydb/public/sdk/cpp/client/extensions/solomon_stats/pull_connector.h +++ b/ydb/public/sdk/cpp/client/extensions/solomon_stats/pull_connector.h @@ -2,23 +2,23 @@ #include <ydb/public/sdk/cpp/client/ydb_extension/extension.h> -#include <library/cpp/monlib/metrics/metric_registry.h> +#include <library/cpp/monlib/metrics/metric_registry.h> #include <util/generic/ptr.h> namespace NSolomonStatExtension { -template <typename TMetricRegistryPtr> -class TMetricRegistryConnector: public NYdb::IExtension { - static NMonitoring::IMetricRegistry* ToRawPtr(NMonitoring::IMetricRegistry* p) { +template <typename TMetricRegistryPtr> +class TMetricRegistryConnector: public NYdb::IExtension { + static NMonitoring::IMetricRegistry* ToRawPtr(NMonitoring::IMetricRegistry* p) { return p; } - static NMonitoring::IMetricRegistry* ToRawPtr(std::shared_ptr<NMonitoring::IMetricRegistry> p) { + static NMonitoring::IMetricRegistry* ToRawPtr(std::shared_ptr<NMonitoring::IMetricRegistry> p) { return p.get(); } - static NMonitoring::IMetricRegistry* ToRawPtr(TAtomicSharedPtr<NMonitoring::IMetricRegistry> p) { + static NMonitoring::IMetricRegistry* ToRawPtr(TAtomicSharedPtr<NMonitoring::IMetricRegistry> p) { return p.Get(); } @@ -26,46 +26,46 @@ public: using IApi = NYdb::NSdkStats::IStatApi; class TParams : public TNonCopyable { - friend class TMetricRegistryConnector; + friend class TMetricRegistryConnector; public: - TParams(TMetricRegistryPtr registry) + TParams(TMetricRegistryPtr registry) : Registry(registry) {} private: - TMetricRegistryPtr Registry; + TMetricRegistryPtr Registry; }; - TMetricRegistryConnector(const TParams& params, IApi* api) - : MetricRegistry_(params.Registry) + TMetricRegistryConnector(const TParams& params, IApi* api) + : MetricRegistry_(params.Registry) { - api->SetMetricRegistry(ToRawPtr(MetricRegistry_)); + api->SetMetricRegistry(ToRawPtr(MetricRegistry_)); } private: - TMetricRegistryPtr MetricRegistry_; + TMetricRegistryPtr MetricRegistry_; }; -inline void AddMetricRegistry(NYdb::TDriver& driver, NMonitoring::IMetricRegistry* ptr) { +inline void AddMetricRegistry(NYdb::TDriver& driver, NMonitoring::IMetricRegistry* ptr) { if (!ptr) return; - using TConnector = TMetricRegistryConnector<NMonitoring::IMetricRegistry*>; + using TConnector = TMetricRegistryConnector<NMonitoring::IMetricRegistry*>; driver.AddExtension<TConnector>(TConnector::TParams(ptr)); } -inline void AddMetricRegistry(NYdb::TDriver& driver, std::shared_ptr<NMonitoring::IMetricRegistry> ptr) { +inline void AddMetricRegistry(NYdb::TDriver& driver, std::shared_ptr<NMonitoring::IMetricRegistry> ptr) { if (!ptr) return; - using TConnector = TMetricRegistryConnector<std::shared_ptr<NMonitoring::IMetricRegistry>>; + using TConnector = TMetricRegistryConnector<std::shared_ptr<NMonitoring::IMetricRegistry>>; driver.AddExtension<TConnector>(TConnector::TParams(ptr)); } -inline void AddMetricRegistry(NYdb::TDriver& driver, TAtomicSharedPtr<NMonitoring::IMetricRegistry> ptr) { +inline void AddMetricRegistry(NYdb::TDriver& driver, TAtomicSharedPtr<NMonitoring::IMetricRegistry> ptr) { if (!ptr) return; - using TConnector = TMetricRegistryConnector<TAtomicSharedPtr<NMonitoring::IMetricRegistry>>; + using TConnector = TMetricRegistryConnector<TAtomicSharedPtr<NMonitoring::IMetricRegistry>>; driver.AddExtension<TConnector>(TConnector::TParams(ptr)); } diff --git a/ydb/public/sdk/cpp/client/extensions/solomon_stats/ya.make b/ydb/public/sdk/cpp/client/extensions/solomon_stats/ya.make index 87ba9ee3a99..18fa0b584fd 100644 --- a/ydb/public/sdk/cpp/client/extensions/solomon_stats/ya.make +++ b/ydb/public/sdk/cpp/client/extensions/solomon_stats/ya.make @@ -12,9 +12,9 @@ SRCS( PEERDIR( library/cpp/monlib/encode/json - library/cpp/monlib/metrics - library/cpp/monlib/service - library/cpp/monlib/service/pages + library/cpp/monlib/metrics + library/cpp/monlib/service + library/cpp/monlib/service/pages ydb/public/sdk/cpp/client/ydb_extension ) diff --git a/ydb/public/sdk/cpp/client/impl/ydb_endpoints/endpoints.cpp b/ydb/public/sdk/cpp/client/impl/ydb_endpoints/endpoints.cpp index 32c1e1696aa..b6d03cc50a9 100644 --- a/ydb/public/sdk/cpp/client/impl/ydb_endpoints/endpoints.cpp +++ b/ydb/public/sdk/cpp/client/impl/ydb_endpoints/endpoints.cpp @@ -1,6 +1,6 @@ #include "endpoints.h" -#include <library/cpp/monlib/metrics/metric_registry.h> +#include <library/cpp/monlib/metrics/metric_registry.h> #include <library/cpp/string_utils/quote/quote.h> #include <util/random/random.h> diff --git a/ydb/public/sdk/cpp/client/impl/ydb_endpoints/ya.make b/ydb/public/sdk/cpp/client/impl/ydb_endpoints/ya.make index cfe7e59d629..da6a27f707a 100644 --- a/ydb/public/sdk/cpp/client/impl/ydb_endpoints/ya.make +++ b/ydb/public/sdk/cpp/client/impl/ydb_endpoints/ya.make @@ -10,7 +10,7 @@ SRCS( ) PEERDIR( - library/cpp/monlib/metrics + library/cpp/monlib/metrics ydb/public/api/grpc ) diff --git a/ydb/public/sdk/cpp/client/impl/ydb_stats/stats.cpp b/ydb/public/sdk/cpp/client/impl/ydb_stats/stats.cpp index a7f46763571..21a58b33cb7 100644 --- a/ydb/public/sdk/cpp/client/impl/ydb_stats/stats.cpp +++ b/ydb/public/sdk/cpp/client/impl/ydb_stats/stats.cpp @@ -8,46 +8,46 @@ const NMonitoring::TLabel TRANSPORT_ERRORS_BY_HOST_LABEL = NMonitoring::TLabel { const NMonitoring::TLabel GRPC_INFLIGHT_BY_HOST_LABEL = NMonitoring::TLabel {"sensor", "Grpc/InFlightByYdbHost"}; void TStatCollector::IncSessionsOnHost(const TStringType& host) { - if (TMetricRegistry* ptr = MetricRegistryPtr_.Get()) { + if (TMetricRegistry* ptr = MetricRegistryPtr_.Get()) { ptr->IntGauge({ DatabaseLabel_, SESSIONS_ON_KQP_HOST_LABEL, {"YdbHost", host} })->Inc(); } } void TStatCollector::DecSessionsOnHost(const TStringType& host) { - if (TMetricRegistry* ptr = MetricRegistryPtr_.Get()) { + if (TMetricRegistry* ptr = MetricRegistryPtr_.Get()) { ptr->IntGauge({ DatabaseLabel_, SESSIONS_ON_KQP_HOST_LABEL, {"YdbHost", host} })->Dec(); } } void TStatCollector::IncTransportErrorsByHost(const TStringType& host) { - if (TMetricRegistry* ptr = MetricRegistryPtr_.Get()) { + if (TMetricRegistry* ptr = MetricRegistryPtr_.Get()) { ptr->Rate({ DatabaseLabel_, TRANSPORT_ERRORS_BY_HOST_LABEL, {"YdbHost", host} })->Inc(); } } void TStatCollector::IncGRpcInFlightByHost(const TStringType& host) { - if (TMetricRegistry* ptr = MetricRegistryPtr_.Get()) { + if (TMetricRegistry* ptr = MetricRegistryPtr_.Get()) { ptr->IntGauge({ DatabaseLabel_, GRPC_INFLIGHT_BY_HOST_LABEL, {"YdbHost", host} })->Inc(); } } void TStatCollector::DecGRpcInFlightByHost(const TStringType& host) { - if (TMetricRegistry* ptr = MetricRegistryPtr_.Get()) { + if (TMetricRegistry* ptr = MetricRegistryPtr_.Get()) { ptr->IntGauge({ DatabaseLabel_, GRPC_INFLIGHT_BY_HOST_LABEL, {"YdbHost", host} })->Dec(); } } void TStatCollector::DeleteHost(const TStringType& host) { - if (TMetricRegistry* ptr = MetricRegistryPtr_.Get()) { + if (TMetricRegistry* ptr = MetricRegistryPtr_.Get()) { const NMonitoring::TLabel hostLabel = NMonitoring::TLabel {"YdbHost", host}; { NMonitoring::TLabels label({ DatabaseLabel_, SESSIONS_ON_KQP_HOST_LABEL, hostLabel }); - ptr->RemoveMetric(label); + ptr->RemoveMetric(label); } { NMonitoring::TLabels label({ DatabaseLabel_, TRANSPORT_ERRORS_BY_HOST_LABEL, hostLabel }); - ptr->RemoveMetric(label); + ptr->RemoveMetric(label); } } } diff --git a/ydb/public/sdk/cpp/client/impl/ydb_stats/stats.h b/ydb/public/sdk/cpp/client/impl/ydb_stats/stats.h index bf93e3a01e4..cae40996824 100644 --- a/ydb/public/sdk/cpp/client/impl/ydb_stats/stats.h +++ b/ydb/public/sdk/cpp/client/impl/ydb_stats/stats.h @@ -4,8 +4,8 @@ #include <ydb/public/sdk/cpp/client/impl/ydb_internal/common/type_switcher.h> #include <library/cpp/grpc/client/grpc_client_low.h> -#include <library/cpp/monlib/metrics/metric_registry.h> -#include <library/cpp/monlib/metrics/histogram_collector.h> +#include <library/cpp/monlib/metrics/metric_registry.h> +#include <library/cpp/monlib/metrics/histogram_collector.h> #include <util/string/builder.h> @@ -161,7 +161,7 @@ private: }; struct TStatCollector { - using TMetricRegistry = NMonitoring::TMetricRegistry; + using TMetricRegistry = NMonitoring::TMetricRegistry; public: @@ -200,14 +200,14 @@ public: struct TClientRetryOperationStatCollector { - TClientRetryOperationStatCollector() : MetricRegistry_(), Database_() {} + TClientRetryOperationStatCollector() : MetricRegistry_(), Database_() {} TClientRetryOperationStatCollector(NMonitoring::TMetricRegistry* registry, const TStringType& database) - : MetricRegistry_(registry), Database_(database) + : MetricRegistry_(registry), Database_(database) { } void IncSyncRetryOperation(const EStatus& status) { - if (auto registry = MetricRegistry_.Get()) { + if (auto registry = MetricRegistry_.Get()) { TString statusName = TStringBuilder() << status; TString sensor = TStringBuilder() << "RetryOperation/" << UnderscoreToUpperCamel(statusName); registry->Rate({ {"database", Database_}, {"sensor", sensor} })->Inc(); @@ -215,7 +215,7 @@ public: } void IncAsyncRetryOperation(const EStatus& status) { - if (auto registry = MetricRegistry_.Get()) { + if (auto registry = MetricRegistry_.Get()) { TString statusName = TStringBuilder() << status; TString sensor = TStringBuilder() << "RetryOperation/" << UnderscoreToUpperCamel(statusName); registry->Rate({ {"database", Database_}, {"sensor", sensor} })->Inc(); @@ -223,7 +223,7 @@ public: } private: - TAtomicPointer<NMonitoring::TMetricRegistry> MetricRegistry_; + TAtomicPointer<NMonitoring::TMetricRegistry> MetricRegistry_; TStringType Database_; }; @@ -256,13 +256,13 @@ public: , DatabaseLabel_({"database", database}) { if (sensorsRegistry) { - SetMetricRegistry(sensorsRegistry); + SetMetricRegistry(sensorsRegistry); } } - void SetMetricRegistry(TMetricRegistry* sensorsRegistry) { - Y_VERIFY(sensorsRegistry, "TMetricRegistry is null in stats collector."); - MetricRegistryPtr_.Set(sensorsRegistry); + void SetMetricRegistry(TMetricRegistry* sensorsRegistry) { + Y_VERIFY(sensorsRegistry, "TMetricRegistry is null in stats collector."); + MetricRegistryPtr_.Set(sensorsRegistry); DiscoveryDuePessimization_.Set(sensorsRegistry->Rate({ DatabaseLabel_, {"sensor", "Discovery/TooManyBadEndpoints"} })); DiscoveryDueExpiration_.Set(sensorsRegistry->Rate({ DatabaseLabel_, {"sensor", "Discovery/Regular"} })); DiscoveryFailDueTransportError_.Set(sensorsRegistry->Rate({ DatabaseLabel_, {"sensor", "Discovery/FailedTransportError"} })); @@ -322,7 +322,7 @@ public: } void IncCounter(const TStringType& sensor) { - if (auto registry = MetricRegistryPtr_.Get()) { + if (auto registry = MetricRegistryPtr_.Get()) { registry->Counter({ {"database", Database_}, {"sensor", sensor} })->Inc(); } } @@ -340,7 +340,7 @@ public: } TEndpointElectorStatCollector GetEndpointElectorStatCollector() { - if (auto registry = MetricRegistryPtr_.Get()) { + if (auto registry = MetricRegistryPtr_.Get()) { auto endpointCoint = registry->IntGauge({ {"database", Database_}, {"sensor", "Endpoints/Total"} }); auto pessimizationRatio = registry->IntGauge({ {"database", Database_}, {"sensor", "Endpoints/BadRatio"} }); auto activeEndpoints = registry->IntGauge({ {"database", Database_}, {"sensor", "Endpoints/Good"} }); @@ -370,14 +370,14 @@ public: if (IsCollecting()) { return TClientStatCollector(CacheMiss_.Get(), QuerySize_.Get(), ParamsSize_.Get(), SessionRemovedDueBalancing_.Get(), RequestMigrated_.Get(), - TClientRetryOperationStatCollector(MetricRegistryPtr_.Get(), Database_)); + TClientRetryOperationStatCollector(MetricRegistryPtr_.Get(), Database_)); } else { return TClientStatCollector(); } } bool IsCollecting() { - return MetricRegistryPtr_.Get() != nullptr; + return MetricRegistryPtr_.Get() != nullptr; } void IncSessionsOnHost(const TStringType& host); @@ -392,7 +392,7 @@ public: private: const TStringType Database_; const NMonitoring::TLabel DatabaseLabel_; - TAtomicPointer<TMetricRegistry> MetricRegistryPtr_; + TAtomicPointer<TMetricRegistry> MetricRegistryPtr_; TAtomicCounter<NMonitoring::TRate> DiscoveryDuePessimization_; TAtomicCounter<NMonitoring::TRate> DiscoveryDueExpiration_; TAtomicCounter<NMonitoring::TRate> RequestFailDueQueueOverflow_; diff --git a/ydb/public/sdk/cpp/client/ydb_extension/extension.h b/ydb/public/sdk/cpp/client/ydb_extension/extension.h index 462bf1f281a..3371bffddc5 100644 --- a/ydb/public/sdk/cpp/client/ydb_extension/extension.h +++ b/ydb/public/sdk/cpp/client/ydb_extension/extension.h @@ -2,7 +2,7 @@ #include <ydb/public/sdk/cpp/client/ydb_driver/driver.h> -#include <library/cpp/monlib/metrics/metric_registry.h> +#include <library/cpp/monlib/metrics/metric_registry.h> namespace Ydb { namespace Discovery { @@ -37,8 +37,8 @@ class IStatApi: public IExtensionApi { public: static IStatApi* Create(TDriver driver); public: - virtual void SetMetricRegistry(NMonitoring::IMetricRegistry* sensorsRegistry) = 0; - virtual void Accept(NMonitoring::IMetricConsumer* consumer) const = 0; + virtual void SetMetricRegistry(NMonitoring::IMetricRegistry* sensorsRegistry) = 0; + virtual void Accept(NMonitoring::IMetricConsumer* consumer) const = 0; }; class DestroyedClientException: public yexception {}; diff --git a/ydb/public/sdk/cpp/client/ydb_persqueue_core/ut/ut_utils/sdk_test_setup.h b/ydb/public/sdk/cpp/client/ydb_persqueue_core/ut/ut_utils/sdk_test_setup.h index f2631292aa2..677eb8c03df 100644 --- a/ydb/public/sdk/cpp/client/ydb_persqueue_core/ut/ut_utils/sdk_test_setup.h +++ b/ydb/public/sdk/cpp/client/ydb_persqueue_core/ut/ut_utils/sdk_test_setup.h @@ -91,7 +91,7 @@ public: return Server.GrpcPort; } - NGrpc::TServerOptions& GetGrpcServerOptions() { + NGrpc::TServerOptions& GetGrpcServerOptions() { return Server.GrpcServerOptions; } diff --git a/ydb/public/sdk/cpp/client/ydb_persqueue_core/ut/ut_utils/test_server.h b/ydb/public/sdk/cpp/client/ydb_persqueue_core/ut/ut_utils/test_server.h index 5a7e90848fa..250ce03620c 100644 --- a/ydb/public/sdk/cpp/client/ydb_persqueue_core/ut/ut_utils/test_server.h +++ b/ydb/public/sdk/cpp/client/ydb_persqueue_core/ut/ut_utils/test_server.h @@ -2,7 +2,7 @@ #include <ydb/core/testlib/test_pq_client.h> #include <ydb/public/sdk/cpp/client/ydb_persqueue_core/persqueue.h> -#include <library/cpp/grpc/server/grpc_server.h> +#include <library/cpp/grpc/server/grpc_server.h> #include <library/cpp/testing/unittest/registar.h> #include <library/cpp/testing/unittest/tests_data.h> @@ -20,7 +20,7 @@ public: , Port(PortManager->GetPort(2134)) , GrpcPort(PortManager->GetPort(2135)) , ServerSettings(NKikimr::NPersQueueTests::PQSettings(Port).SetGrpcPort(GrpcPort)) - , GrpcServerOptions(NGrpc::TServerOptions().SetHost("[::1]").SetPort(GrpcPort)) + , GrpcServerOptions(NGrpc::TServerOptions().SetHost("[::1]").SetPort(GrpcPort)) { if (start) { StartServer(); @@ -100,7 +100,7 @@ public: THolder<NKikimr::Tests::TServer> CleverServer; NKikimr::Tests::TServerSettings ServerSettings; - NGrpc::TServerOptions GrpcServerOptions; + NGrpc::TServerOptions GrpcServerOptions; THolder<TTempFileHandle> NetDataFile; THolder<NKikimr::NPersQueueTests::TFlatMsgBusPQClient> AnnoyingClient; diff --git a/ydb/public/sdk/cpp/client/ydb_result/result.cpp b/ydb/public/sdk/cpp/client/ydb_result/result.cpp index a3890a32afe..dc7eb09c0ac 100644 --- a/ydb/public/sdk/cpp/client/ydb_result/result.cpp +++ b/ydb/public/sdk/cpp/client/ydb_result/result.cpp @@ -108,10 +108,10 @@ public: return ResultSet_.ColumnsCount(); } - size_t RowsCount() const { - return ResultSet_.RowsCount(); - } - + size_t RowsCount() const { + return ResultSet_.RowsCount(); + } + bool TryNextRow() { if (RowIndex_ == ResultSet_.RowsCount()) { return false; @@ -126,11 +126,11 @@ public: return true; } - ssize_t ColumnIndex(const TString& columnName) { - auto idx = ColumnIndexMap.FindPtr(columnName); - return idx ? static_cast<ssize_t>(*idx) : -1; - } - + ssize_t ColumnIndex(const TString& columnName) { + auto idx = ColumnIndexMap.FindPtr(columnName); + return idx ? static_cast<ssize_t>(*idx) : -1; + } + TValueParser& ColumnParser(size_t columnIndex) { if (columnIndex >= ColumnParsers.size()) { FatalError(TStringBuilder() << "Column index out of bounds: " << columnIndex); @@ -196,18 +196,18 @@ size_t TResultSetParser::ColumnsCount() const { return Impl_->ColumnsCount(); } -size_t TResultSetParser::RowsCount() const { - return Impl_->RowsCount(); -} - +size_t TResultSetParser::RowsCount() const { + return Impl_->RowsCount(); +} + bool TResultSetParser::TryNextRow() { return Impl_->TryNextRow(); } -ssize_t TResultSetParser::ColumnIndex(const TString& columnName) { - return Impl_->ColumnIndex(columnName); -} - +ssize_t TResultSetParser::ColumnIndex(const TString& columnName) { + return Impl_->ColumnIndex(columnName); +} + TValueParser& TResultSetParser::ColumnParser(size_t columnIndex) { return Impl_->ColumnParser(columnIndex); } diff --git a/ydb/public/sdk/cpp/client/ydb_result/result.h b/ydb/public/sdk/cpp/client/ydb_result/result.h index 305c1c3cbdb..287c8078be0 100644 --- a/ydb/public/sdk/cpp/client/ydb_result/result.h +++ b/ydb/public/sdk/cpp/client/ydb_result/result.h @@ -67,18 +67,18 @@ public: //! Returns number of columns size_t ColumnsCount() const; - //! Returns number of rows - size_t RowsCount() const; - + //! Returns number of rows + size_t RowsCount() const; + //! Set iterator to the next result row. //! On success TryNextRow will reset all column parsers to the values in next row. //! Column parsers are invalid before the first TryNextRow call. bool TryNextRow(); - //! Returns index for column with specified name. - //! If there is no column with such name, then -1 is returned. - ssize_t ColumnIndex(const TString& columnName); - + //! Returns index for column with specified name. + //! If there is no column with such name, then -1 is returned. + ssize_t ColumnIndex(const TString& columnName); + //! Returns column value parser for column with specified index. //! State of the parser is preserved until next TryNextRow call. TValueParser& ColumnParser(size_t columnIndex); diff --git a/ydb/public/sdk/cpp/client/ydb_table/table.cpp b/ydb/public/sdk/cpp/client/ydb_table/table.cpp index 76412d3b00c..610fec38847 100644 --- a/ydb/public/sdk/cpp/client/ydb_table/table.cpp +++ b/ydb/public/sdk/cpp/client/ydb_table/table.cpp @@ -1928,14 +1928,14 @@ public: return SessionPool_.GetActiveSessions(); } - i64 GetActiveSessionsLimit() const { - return SessionPool_.GetActiveSessionsLimit(); - } - - i64 GetCurrentPoolSize() const { - return SessionPool_.GetCurrentPoolSize(); - } - + i64 GetActiveSessionsLimit() const { + return SessionPool_.GetActiveSessionsLimit(); + } + + i64 GetCurrentPoolSize() const { + return SessionPool_.GetCurrentPoolSize(); + } + TAsyncCreateSessionResult CreateSession(const TCreateSessionSettings& settings, bool standalone, TString preferedLocation = TString()) { @@ -3204,14 +3204,14 @@ i64 TTableClient::GetActiveSessionCount() const { return Impl_->GetActiveSessionCount(); } -i64 TTableClient::GetActiveSessionsLimit() const { - return Impl_->GetActiveSessionsLimit(); -} - -i64 TTableClient::GetCurrentPoolSize() const { - return Impl_->GetCurrentPoolSize(); -} - +i64 TTableClient::GetActiveSessionsLimit() const { + return Impl_->GetActiveSessionsLimit(); +} + +i64 TTableClient::GetCurrentPoolSize() const { + return Impl_->GetCurrentPoolSize(); +} + TTableBuilder TTableClient::GetTableBuilder() { return TTableBuilder(); } @@ -3332,10 +3332,10 @@ protected: return self->Promise.SetValue(status); } } - - static void HandleException(TRetryContextPtr self, std::exception_ptr e) { - self->Promise.SetException(e); - } + + static void HandleException(TRetryContextPtr self, std::exception_ptr e) { + self->Promise.SetException(e); + } }; class TRetryOperationWithSession : public TRetryOperationContext { @@ -3358,17 +3358,17 @@ public: TableClient.GetSession( TCreateSessionSettings().ClientTimeout(Settings.GetSessionClientTimeout_)).Subscribe( [self](const TAsyncCreateSessionResult& resultFuture) { - try { - auto& result = resultFuture.GetValue(); - if (!result.IsSuccess()) { - return HandleStatus(self, result); - } - - auto* myself = dynamic_cast<TRetryOperationWithSession*>(self.Get()); - myself->Session = result.GetSession(); - myself->DoRunOp(self); - } catch (...) { - return HandleException(self, std::current_exception()); + try { + auto& result = resultFuture.GetValue(); + if (!result.IsSuccess()) { + return HandleStatus(self, result); + } + + auto* myself = dynamic_cast<TRetryOperationWithSession*>(self.Get()); + myself->Session = result.GetSession(); + myself->DoRunOp(self); + } catch (...) { + return HandleException(self, std::current_exception()); } }); } else { @@ -3383,11 +3383,11 @@ private: void DoRunOp(TRetryContextPtr self) { Operation(Session.GetRef()).Subscribe([self](const TAsyncStatus& result) { - try { - return HandleStatus(self, result.GetValue()); - } catch (...) { - return HandleException(self, std::current_exception()); - } + try { + return HandleStatus(self, result.GetValue()); + } catch (...) { + return HandleException(self, std::current_exception()); + } }); } }; @@ -3414,11 +3414,11 @@ public: void Execute() override { TRetryContextPtr self(this); Operation(TableClient).Subscribe([self](const TAsyncStatus& result) { - try { - return HandleStatus(self, result.GetValue()); - } catch (...) { - return HandleException(self, std::current_exception()); - } + try { + return HandleStatus(self, result.GetValue()); + } catch (...) { + return HandleException(self, std::current_exception()); + } }); } }; diff --git a/ydb/public/sdk/cpp/client/ydb_table/table.h b/ydb/public/sdk/cpp/client/ydb_table/table.h index 1fe9e4db082..42d21d2fec7 100644 --- a/ydb/public/sdk/cpp/client/ydb_table/table.h +++ b/ydb/public/sdk/cpp/client/ydb_table/table.h @@ -932,12 +932,12 @@ public: //! Returns number of active sessions given via session pool i64 GetActiveSessionCount() const; - //! Returns the maximum number of sessions in session pool - i64 GetActiveSessionsLimit() const; - - //! Returns the size of session pool - i64 GetCurrentPoolSize() const; - + //! Returns the maximum number of sessions in session pool + i64 GetActiveSessionsLimit() const; + + //! Returns the size of session pool + i64 GetCurrentPoolSize() const; + //! Returns new table builder TTableBuilder GetTableBuilder(); //! Returns new params builder diff --git a/ydb/public/sdk/cpp/client/ydb_value/value.h b/ydb/public/sdk/cpp/client/ydb_value/value.h index 49125b9ee62..4a163b92206 100644 --- a/ydb/public/sdk/cpp/client/ydb_value/value.h +++ b/ydb/public/sdk/cpp/client/ydb_value/value.h @@ -64,7 +64,7 @@ struct TDecimalType { ui8 Precision; ui8 Scale; - TDecimalType(ui8 precision, ui8 scale) + TDecimalType(ui8 precision, ui8 scale) : Precision(precision) , Scale(scale) {} }; diff --git a/ydb/services/cms/grpc_service.cpp b/ydb/services/cms/grpc_service.cpp index 7d1cb84bcc1..33a7569982c 100644 --- a/ydb/services/cms/grpc_service.cpp +++ b/ydb/services/cms/grpc_service.cpp @@ -9,19 +9,19 @@ namespace NGRpcService { TGRpcCmsService::TGRpcCmsService(NActors::TActorSystem *system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, - NActors::TActorId id) + NActors::TActorId id) : ActorSystem_(system) , Counters_(counters) , GRpcRequestProxyId_(id) { } -void TGRpcCmsService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { +void TGRpcCmsService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { CQ_ = cq; - SetupIncomingRequests(std::move(logger)); + SetupIncomingRequests(std::move(logger)); } -void TGRpcCmsService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) +void TGRpcCmsService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { Limiter_ = limiter; } @@ -35,18 +35,18 @@ void TGRpcCmsService::DecRequest() { Y_ASSERT(Limiter_->GetCurrentInFlight() >= 0); } -void TGRpcCmsService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { +void TGRpcCmsService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { auto getCounterBlock = CreateCounterCb(Counters_, ActorSystem_); #ifdef ADD_REQUEST #error ADD_REQUEST macro already defined #endif #define ADD_REQUEST(NAME, IN, OUT, ACTION) \ MakeIntrusive<TGRpcRequest<Ydb::Cms::IN, Ydb::Cms::OUT, TGRpcCmsService>>(this, &Service_, CQ_, \ - [this](NGrpc::IRequestContextBase *ctx) { \ + [this](NGrpc::IRequestContextBase *ctx) { \ ReportGrpcReqToMon(*ActorSystem_, ctx->GetPeer()); \ ACTION; \ }, &Ydb::Cms::V1::CmsService::AsyncService::Request ## NAME, \ - #NAME, logger, getCounterBlock("cms", #NAME))->Run(); + #NAME, logger, getCounterBlock("cms", #NAME))->Run(); ADD_REQUEST(CreateDatabase, CreateDatabaseRequest, CreateDatabaseResponse, { ActorSystem_->Send(GRpcRequestProxyId_, new TEvCreateTenantRequest(ctx)); diff --git a/ydb/services/cms/grpc_service.h b/ydb/services/cms/grpc_service.h index abc33a7f203..80f3bbe9283 100644 --- a/ydb/services/cms/grpc_service.h +++ b/ydb/services/cms/grpc_service.h @@ -9,27 +9,27 @@ namespace NKikimr { namespace NGRpcService { class TGRpcCmsService - : public NGrpc::TGrpcServiceBase<Ydb::Cms::V1::CmsService> + : public NGrpc::TGrpcServiceBase<Ydb::Cms::V1::CmsService> { public: TGRpcCmsService(NActors::TActorSystem* system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, - NActors::TActorId id); + NActors::TActorId id); - void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; - void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; + void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; + void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; bool IncRequest(); void DecRequest(); private: - void SetupIncomingRequests(NGrpc::TLoggerPtr logger); + void SetupIncomingRequests(NGrpc::TLoggerPtr logger); NActors::TActorSystem* ActorSystem_; grpc::ServerCompletionQueue* CQ_ = nullptr; TIntrusivePtr<NMonitoring::TDynamicCounters> Counters_; - NActors::TActorId GRpcRequestProxyId_; + NActors::TActorId GRpcRequestProxyId_; NGrpc::TGlobalLimiter* Limiter_ = nullptr; }; diff --git a/ydb/services/datastreams/datastreams_proxy.h b/ydb/services/datastreams/datastreams_proxy.h index 2aab51cfcf9..1712e171eea 100644 --- a/ydb/services/datastreams/datastreams_proxy.h +++ b/ydb/services/datastreams/datastreams_proxy.h @@ -5,7 +5,7 @@ #include <ydb/core/client/server/grpc_base.h> #include <ydb/core/grpc_services/rpc_calls.h> -#include <library/cpp/grpc/server/grpc_server.h> +#include <library/cpp/grpc/server/grpc_server.h> #include <library/cpp/actors/core/actor_bootstrapped.h> #include <library/cpp/actors/core/actorsystem.h> diff --git a/ydb/services/datastreams/grpc_service.cpp b/ydb/services/datastreams/grpc_service.cpp index d52f0ba0b47..7a18cd4dc67 100644 --- a/ydb/services/datastreams/grpc_service.cpp +++ b/ydb/services/datastreams/grpc_service.cpp @@ -20,7 +20,7 @@ TGRpcDataStreamsService::TGRpcDataStreamsService(NActors::TActorSystem *system, { } -void TGRpcDataStreamsService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) +void TGRpcDataStreamsService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { CQ_ = cq; @@ -32,7 +32,7 @@ void TGRpcDataStreamsService::InitService(grpc::ServerCompletionQueue *cq, NGrpc SetupIncomingRequests(logger); } -void TGRpcDataStreamsService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter *limiter) { +void TGRpcDataStreamsService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter *limiter) { Limiter_ = limiter; } @@ -53,7 +53,7 @@ void TGRpcDataStreamsService::InitNewSchemeCache() { TMailboxType::HTSwap, ActorSystem_->AppData<TAppData>()->UserPoolId); } -void TGRpcDataStreamsService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) +void TGRpcDataStreamsService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { auto getCounterBlock = CreateCounterCb(Counters_, ActorSystem_); #ifdef ADD_REQUEST @@ -61,7 +61,7 @@ void TGRpcDataStreamsService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) #endif #define ADD_REQUEST(NAME, IN, OUT, ACTION) \ MakeIntrusive<TGRpcRequest<Ydb::DataStreams::V1::IN, Ydb::DataStreams::V1::OUT, TGRpcDataStreamsService>>(this, &Service_, CQ_, \ - [this](NGrpc::IRequestContextBase *ctx) { \ + [this](NGrpc::IRequestContextBase *ctx) { \ ReportGrpcReqToMon(*ActorSystem_, ctx->GetPeer()); \ ACTION; \ }, &Ydb::DataStreams::V1::DataStreamsService::AsyncService::Request ## NAME, \ diff --git a/ydb/services/datastreams/grpc_service.h b/ydb/services/datastreams/grpc_service.h index 6cf2c09abd4..d8ed51df522 100644 --- a/ydb/services/datastreams/grpc_service.h +++ b/ydb/services/datastreams/grpc_service.h @@ -1,26 +1,26 @@ #pragma once #include <library/cpp/actors/core/actorsystem.h> -#include <library/cpp/grpc/server/grpc_server.h> +#include <library/cpp/grpc/server/grpc_server.h> #include <ydb/public/api/grpc/draft/ydb_datastreams_v1.grpc.pb.h> namespace NKikimr::NGRpcService { - class TGRpcDataStreamsService : public NGrpc::TGrpcServiceBase<Ydb::DataStreams::V1::DataStreamsService> + class TGRpcDataStreamsService : public NGrpc::TGrpcServiceBase<Ydb::DataStreams::V1::DataStreamsService> { public: TGRpcDataStreamsService(NActors::TActorSystem* system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId id); - void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; - void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; + void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; + void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; bool IncRequest(); void DecRequest(); private: - void SetupIncomingRequests(NGrpc::TLoggerPtr logger); + void SetupIncomingRequests(NGrpc::TLoggerPtr logger); void InitNewSchemeCache(); NActors::TActorSystem* ActorSystem_; diff --git a/ydb/services/discovery/grpc_service.cpp b/ydb/services/discovery/grpc_service.cpp index d96f8d7e1d4..010322b5d9b 100644 --- a/ydb/services/discovery/grpc_service.cpp +++ b/ydb/services/discovery/grpc_service.cpp @@ -7,29 +7,29 @@ namespace NKikimr { namespace NGRpcService { -static TString GetSdkBuildInfo(NGrpc::IRequestContextBase* reqCtx) { +static TString GetSdkBuildInfo(NGrpc::IRequestContextBase* reqCtx) { const auto& res = reqCtx->GetPeerMetaValues(NYdb::YDB_SDK_BUILD_INFO_HEADER); if (res.empty()) { return {}; } - return TString{res[0]}; + return TString{res[0]}; } TGRpcDiscoveryService::TGRpcDiscoveryService(NActors::TActorSystem *system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, - NActors::TActorId id) + NActors::TActorId id) : ActorSystem_(system) , Counters_(counters) , GRpcRequestProxyId_(id) { } - void TGRpcDiscoveryService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { + void TGRpcDiscoveryService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { CQ_ = cq; - SetupIncomingRequests(std::move(logger)); + SetupIncomingRequests(std::move(logger)); } - void TGRpcDiscoveryService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter *limiter) { + void TGRpcDiscoveryService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter *limiter) { Limiter_ = limiter; } @@ -42,18 +42,18 @@ TGRpcDiscoveryService::TGRpcDiscoveryService(NActors::TActorSystem *system, Y_ASSERT(Limiter_->GetCurrentInFlight() >= 0); } - void TGRpcDiscoveryService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { + void TGRpcDiscoveryService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { auto getCounterBlock = CreateCounterCb(Counters_, ActorSystem_); #ifdef ADD_REQUEST #error ADD_REQUEST macro already defined #endif #define ADD_REQUEST(NAME, IN, OUT, ACTION) \ MakeIntrusive<TGRpcRequest<Ydb::Discovery::IN, Ydb::Discovery::OUT, TGRpcDiscoveryService>>(this, &Service_, CQ_, \ - [this](NGrpc::IRequestContextBase *reqCtx) { \ + [this](NGrpc::IRequestContextBase *reqCtx) { \ NGRpcService::ReportGrpcReqToMon(*ActorSystem_, reqCtx->GetPeer(), GetSdkBuildInfo(reqCtx)); \ ACTION; \ }, &Ydb::Discovery::V1::DiscoveryService::AsyncService::Request ## NAME, \ - #NAME, logger, getCounterBlock("discovery", #NAME))->Run(); + #NAME, logger, getCounterBlock("discovery", #NAME))->Run(); ADD_REQUEST(ListEndpoints, ListEndpointsRequest, ListEndpointsResponse, { ActorSystem_->Send(GRpcRequestProxyId_, new TEvListEndpointsRequest(reqCtx)); diff --git a/ydb/services/discovery/grpc_service.h b/ydb/services/discovery/grpc_service.h index 50ac330a049..c6cb25f0fd9 100644 --- a/ydb/services/discovery/grpc_service.h +++ b/ydb/services/discovery/grpc_service.h @@ -4,33 +4,33 @@ #include <ydb/public/api/grpc/ydb_discovery_v1.grpc.pb.h> -#include <library/cpp/grpc/server/grpc_server.h> +#include <library/cpp/grpc/server/grpc_server.h> namespace NKikimr { namespace NGRpcService { class TGRpcDiscoveryService - : public NGrpc::TGrpcServiceBase<Ydb::Discovery::V1::DiscoveryService> + : public NGrpc::TGrpcServiceBase<Ydb::Discovery::V1::DiscoveryService> { public: TGRpcDiscoveryService(NActors::TActorSystem* system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, - NActors::TActorId id); + NActors::TActorId id); - void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; - void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; + void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; + void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; bool IncRequest(); void DecRequest(); private: - void SetupIncomingRequests(NGrpc::TLoggerPtr logger); + void SetupIncomingRequests(NGrpc::TLoggerPtr logger); NActors::TActorSystem* ActorSystem_; grpc::ServerCompletionQueue* CQ_ = nullptr; TIntrusivePtr<NMonitoring::TDynamicCounters> Counters_; - NActors::TActorId GRpcRequestProxyId_; + NActors::TActorId GRpcRequestProxyId_; NGrpc::TGlobalLimiter* Limiter_ = nullptr; }; diff --git a/ydb/services/kesus/grpc_service.cpp b/ydb/services/kesus/grpc_service.cpp index 3ea10ab8b9f..c5826b66274 100644 --- a/ydb/services/kesus/grpc_service.cpp +++ b/ydb/services/kesus/grpc_service.cpp @@ -11,8 +11,8 @@ #include <ydb/core/grpc_streaming/grpc_streaming.h> #include <ydb/core/base/ticket_parser.h> -#include <library/cpp/grpc/server/event_callback.h> -#include <library/cpp/grpc/server/grpc_async_ctx_base.h> +#include <library/cpp/grpc/server/event_callback.h> +#include <library/cpp/grpc/server/grpc_async_ctx_base.h> #include <ydb/public/sdk/cpp/client/resources/ydb_resources.h> #include <library/cpp/actors/core/actor_bootstrapped.h> @@ -593,7 +593,7 @@ private: THolder<IContext::TEvReadFinished> StartRequest; TString KesusPath; - TActorId ProxyActor; + TActorId ProxyActor; bool ReadAllowed = false; bool WriteAllowed = false; bool EraseAllowed = false; @@ -609,7 +609,7 @@ private: TKesusGRpcService::TKesusGRpcService( NActors::TActorSystem* actorSystem, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, - NActors::TActorId id) + NActors::TActorId id) : ActorSystem(actorSystem) , Counters(counters) , GRpcRequestProxyId(id) @@ -619,12 +619,12 @@ TKesusGRpcService::~TKesusGRpcService() { // empty } -void TKesusGRpcService::InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) { +void TKesusGRpcService::InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) { CQ = cq; - SetupIncomingRequests(std::move(logger)); + SetupIncomingRequests(std::move(logger)); } -void TKesusGRpcService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { +void TKesusGRpcService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { Limiter = limiter; } @@ -637,7 +637,7 @@ void TKesusGRpcService::DecRequest() { Y_ASSERT(Limiter->GetCurrentInFlight() >= 0); } -void TKesusGRpcService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { +void TKesusGRpcService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { auto getCounterBlock = NGRpcService::CreateCounterCb(Counters, ActorSystem); #ifdef ADD_REQUEST @@ -649,13 +649,13 @@ void TKesusGRpcService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { this, \ &Service_, \ CQ, \ - [this](NGrpc::IRequestContextBase* reqCtx) { \ + [this](NGrpc::IRequestContextBase* reqCtx) { \ NGRpcService::ReportGrpcReqToMon(*ActorSystem, reqCtx->GetPeer()); \ ACTION; \ }, \ &Ydb::Coordination::V1::CoordinationService::AsyncService::Request ## NAME, \ - "Coordination/" #NAME, \ - logger, \ + "Coordination/" #NAME, \ + logger, \ getCounterBlock("coordination", #NAME))->Run(); ADD_REQUEST(CreateNode, CreateNodeRequest, CreateNodeResponse, { diff --git a/ydb/services/kesus/grpc_service.h b/ydb/services/kesus/grpc_service.h index 1761a60e68c..b5f597b7d9b 100644 --- a/ydb/services/kesus/grpc_service.h +++ b/ydb/services/kesus/grpc_service.h @@ -2,7 +2,7 @@ #include <ydb/public/api/grpc/ydb_coordination_v1.grpc.pb.h> -#include <library/cpp/grpc/server/grpc_server.h> +#include <library/cpp/grpc/server/grpc_server.h> #include <library/cpp/actors/core/actorsystem.h> @@ -12,7 +12,7 @@ namespace NKikimr { namespace NKesus { class TKesusGRpcService - : public NGrpc::TGrpcServiceBase<Ydb::Coordination::V1::CoordinationService> + : public NGrpc::TGrpcServiceBase<Ydb::Coordination::V1::CoordinationService> { class TContextBase; class TSessionContext; @@ -21,25 +21,25 @@ public: TKesusGRpcService( NActors::TActorSystem* actorSystem, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, - NActors::TActorId id); + NActors::TActorId id); ~TKesusGRpcService(); - void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; - void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; + void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; + void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; bool IncRequest(); void DecRequest(); private: - void SetupIncomingRequests(NGrpc::TLoggerPtr logger); + void SetupIncomingRequests(NGrpc::TLoggerPtr logger); private: NActors::TActorSystem* ActorSystem; TIntrusivePtr<NMonitoring::TDynamicCounters> Counters; - NActors::TActorId GRpcRequestProxyId; + NActors::TActorId GRpcRequestProxyId; grpc::ServerCompletionQueue* CQ = nullptr; - NGrpc::TGlobalLimiter* Limiter = nullptr; + NGrpc::TGlobalLimiter* Limiter = nullptr; }; } diff --git a/ydb/services/monitoring/grpc_service.cpp b/ydb/services/monitoring/grpc_service.cpp index 5162db43b2e..d09fe4f3a4f 100644 --- a/ydb/services/monitoring/grpc_service.cpp +++ b/ydb/services/monitoring/grpc_service.cpp @@ -6,29 +6,29 @@ namespace NKikimr { namespace NGRpcService { -static TString GetSdkBuildInfo(NGrpc::IRequestContextBase* reqCtx) { +static TString GetSdkBuildInfo(NGrpc::IRequestContextBase* reqCtx) { const auto& res = reqCtx->GetPeerMetaValues(NYdb::YDB_SDK_BUILD_INFO_HEADER); if (res.empty()) { return {}; } - return TString{res[0]}; + return TString{res[0]}; } TGRpcMonitoringService::TGRpcMonitoringService(NActors::TActorSystem* system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, - NActors::TActorId id) + NActors::TActorId id) : ActorSystem_(system) , Counters_(counters) , GRpcRequestProxyId_(id) { } -void TGRpcMonitoringService::InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) { +void TGRpcMonitoringService::InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) { CQ_ = cq; - SetupIncomingRequests(std::move(logger)); + SetupIncomingRequests(std::move(logger)); } -void TGRpcMonitoringService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { +void TGRpcMonitoringService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { Limiter_ = limiter; } @@ -41,18 +41,18 @@ void TGRpcMonitoringService::DecRequest() { Y_ASSERT(Limiter_->GetCurrentInFlight() >= 0); } -void TGRpcMonitoringService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { +void TGRpcMonitoringService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { auto getCounterBlock = CreateCounterCb(Counters_, ActorSystem_); #ifdef ADD_REQUEST #error ADD_REQUEST macro already defined #endif #define ADD_REQUEST(NAME, IN, OUT, ACTION) \ MakeIntrusive<TGRpcRequest<Ydb::Monitoring::IN, Ydb::Monitoring::OUT, TGRpcMonitoringService>>(this, &Service_, CQ_, \ - [this](NGrpc::IRequestContextBase* reqCtx) { \ + [this](NGrpc::IRequestContextBase* reqCtx) { \ NGRpcService::ReportGrpcReqToMon(*ActorSystem_, reqCtx->GetPeer(), GetSdkBuildInfo(reqCtx)); \ ACTION; \ }, &Ydb::Monitoring::V1::MonitoringService::AsyncService::Request ## NAME, \ - #NAME, logger, getCounterBlock("monitoring", #NAME))->Run(); + #NAME, logger, getCounterBlock("monitoring", #NAME))->Run(); ADD_REQUEST(SelfCheck, SelfCheckRequest, SelfCheckResponse, { ActorSystem_->Send(GRpcRequestProxyId_, new TEvSelfCheckRequest(reqCtx)); diff --git a/ydb/services/monitoring/grpc_service.h b/ydb/services/monitoring/grpc_service.h index c0b0fd27e84..3c324424fdd 100644 --- a/ydb/services/monitoring/grpc_service.h +++ b/ydb/services/monitoring/grpc_service.h @@ -4,32 +4,32 @@ #include <ydb/public/api/grpc/ydb_monitoring_v1.grpc.pb.h> -#include <library/cpp/grpc/server/grpc_server.h> +#include <library/cpp/grpc/server/grpc_server.h> namespace NKikimr { namespace NGRpcService { - class TGRpcMonitoringService : public NGrpc::TGrpcServiceBase<Ydb::Monitoring::V1::MonitoringService> + class TGRpcMonitoringService : public NGrpc::TGrpcServiceBase<Ydb::Monitoring::V1::MonitoringService> { public: TGRpcMonitoringService(NActors::TActorSystem* system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, - NActors::TActorId id); + NActors::TActorId id); - void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; - void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; + void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; + void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; bool IncRequest(); void DecRequest(); private: - void SetupIncomingRequests(NGrpc::TLoggerPtr logger); + void SetupIncomingRequests(NGrpc::TLoggerPtr logger); NActors::TActorSystem* ActorSystem_; grpc::ServerCompletionQueue* CQ_ = nullptr; TIntrusivePtr<NMonitoring::TDynamicCounters> Counters_; - NActors::TActorId GRpcRequestProxyId_; + NActors::TActorId GRpcRequestProxyId_; NGrpc::TGlobalLimiter* Limiter_ = nullptr; }; diff --git a/ydb/services/persqueue_cluster_discovery/cluster_discovery_service.cpp b/ydb/services/persqueue_cluster_discovery/cluster_discovery_service.cpp index 1b3ccbd1ff8..aabd11d4cac 100644 --- a/ydb/services/persqueue_cluster_discovery/cluster_discovery_service.cpp +++ b/ydb/services/persqueue_cluster_discovery/cluster_discovery_service.cpp @@ -273,8 +273,8 @@ private: } private: - TActorId NetClassifierActorId; - TActorId ClusterTrackerActorId; + TActorId NetClassifierActorId; + TActorId ClusterTrackerActorId; NAddressClassifier::TLabeledAddressClassifier::TConstPtr DatacenterClassifier; NAddressClassifier::TLabeledAddressClassifier::TConstPtr CloudNetworksClassifier; TMaybe<TInstant> NetDataUpdateTs; diff --git a/ydb/services/persqueue_cluster_discovery/cluster_discovery_service.h b/ydb/services/persqueue_cluster_discovery/cluster_discovery_service.h index 533d31de8cc..fec5708c152 100644 --- a/ydb/services/persqueue_cluster_discovery/cluster_discovery_service.h +++ b/ydb/services/persqueue_cluster_discovery/cluster_discovery_service.h @@ -7,14 +7,14 @@ #include <library/cpp/actors/core/defs.h> #include <library/cpp/actors/core/event_local.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> namespace NKikimr::NPQ::NClusterDiscovery { - using NActors::TActorId; + using NActors::TActorId; - inline TActorId MakeClusterDiscoveryServiceID() { - const char x[TActorId::MaxServiceIDLength] = "pq_discosvc"; - return TActorId(0, TStringBuf(x, TActorId::MaxServiceIDLength)); + inline TActorId MakeClusterDiscoveryServiceID() { + const char x[TActorId::MaxServiceIDLength] = "pq_discosvc"; + return TActorId(0, TStringBuf(x, TActorId::MaxServiceIDLength)); } NActors::IActor* CreateClusterDiscoveryService(TIntrusivePtr<NMonitoring::TDynamicCounters> counters); diff --git a/ydb/services/persqueue_cluster_discovery/cluster_discovery_service_ut.cpp b/ydb/services/persqueue_cluster_discovery/cluster_discovery_service_ut.cpp index 1572f687e07..34721e262de 100644 --- a/ydb/services/persqueue_cluster_discovery/cluster_discovery_service_ut.cpp +++ b/ydb/services/persqueue_cluster_discovery/cluster_discovery_service_ut.cpp @@ -285,7 +285,7 @@ public: void Run() { Server_ = MakeHolder<TServer>(Settings_); - Server_->EnableGRpc(NGrpc::TServerOptions().SetHost("localhost").SetPort(GrpcPort_)); + Server_->EnableGRpc(NGrpc::TServerOptions().SetHost("localhost").SetPort(GrpcPort_)); } NPersQueueTests::TFlatMsgBusPQClient& PQClient() { @@ -313,7 +313,7 @@ public: HttpProxyId_ = ActorSystem().Register(proxy); } - TActorId clientId = ActorSystem().AllocateEdgeActor(); + TActorId clientId = ActorSystem().AllocateEdgeActor(); NHttp::THttpOutgoingRequestPtr httpRequest = NHttp::THttpOutgoingRequest::CreateRequestGet("http://[::1]:" + ToString(MonPort()) + path); ActorSystem().Send(new NActors::IEventHandle(HttpProxyId_, clientId, new NHttp::TEvHttpProxy::TEvHttpOutgoingRequest(httpRequest)), 0, true); @@ -379,11 +379,11 @@ private: ui16 GrpcPort_; TServerSettings Settings_; - NMonitoring::TMetricRegistry Sensors_; + NMonitoring::TMetricRegistry Sensors_; THolder<TServer> Server_; THolder<NPersQueueTests::TFlatMsgBusPQClient> PQClient_; - TActorId HttpProxyId_; + TActorId HttpProxyId_; THolder<TTempFileHandle> NetDataFile; }; @@ -404,7 +404,7 @@ Y_UNIT_TEST_SUITE(TPQCDTest) { auto& actorSystem = server.ActorSystem(); // check that NetClassifier is up and running { - const TActorId sender = actorSystem.AllocateEdgeActor(); + const TActorId sender = actorSystem.AllocateEdgeActor(); actorSystem.Send( new IEventHandle(NNetClassifier::MakeNetClassifierID(), sender, @@ -433,7 +433,7 @@ Y_UNIT_TEST_SUITE(TPQCDTest) { // check that ClusterTracker is up and running { - const TActorId sender = actorSystem.AllocateEdgeActor(); + const TActorId sender = actorSystem.AllocateEdgeActor(); actorSystem.Send( new IEventHandle(NPQ::NClusterTracker::MakeClusterTrackerID(), sender, @@ -601,7 +601,7 @@ Y_UNIT_TEST_SUITE(TPQCDTest) { auto& actorSystem = server.ActorSystem(); // check that NetClassifier instance is initialized with null { - const TActorId sender = actorSystem.AllocateEdgeActor(); + const TActorId sender = actorSystem.AllocateEdgeActor(); actorSystem.Send( new IEventHandle(NNetClassifier::MakeNetClassifierID(), sender, diff --git a/ydb/services/persqueue_cluster_discovery/counters.h b/ydb/services/persqueue_cluster_discovery/counters.h index 5d24cab1cad..495b2e6939c 100644 --- a/ydb/services/persqueue_cluster_discovery/counters.h +++ b/ydb/services/persqueue_cluster_discovery/counters.h @@ -5,7 +5,7 @@ #include <ydb/core/persqueue/cluster_tracker.h> #include <ydb/core/util/address_classifier.h> -#include <library/cpp/monlib/dynamic_counters/counters.h> +#include <library/cpp/monlib/dynamic_counters/counters.h> #include <util/generic/hash.h> #include <util/generic/ptr.h> diff --git a/ydb/services/persqueue_cluster_discovery/grpc_service.cpp b/ydb/services/persqueue_cluster_discovery/grpc_service.cpp index 27f2cdd9119..086d162443b 100644 --- a/ydb/services/persqueue_cluster_discovery/grpc_service.cpp +++ b/ydb/services/persqueue_cluster_discovery/grpc_service.cpp @@ -8,7 +8,7 @@ #include <ydb/core/grpc_services/grpc_request_proxy.h> #include <ydb/core/grpc_services/rpc_calls.h> -#include <library/cpp/grpc/server/grpc_request.h> +#include <library/cpp/grpc/server/grpc_request.h> namespace NKikimr::NGRpcService { @@ -26,19 +26,19 @@ TGRpcPQClusterDiscoveryService::TGRpcPQClusterDiscoveryService( } } -void TGRpcPQClusterDiscoveryService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { +void TGRpcPQClusterDiscoveryService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { CQ_ = cq; if (ActorSystem_->AppData<TAppData>()->PQClusterDiscoveryConfig.GetEnabled()) { IActor* actor = NPQ::NClusterDiscovery::CreateClusterDiscoveryService(GetServiceCounters(Counters_, "persqueue")->GetSubgroup("subsystem", "cluster_discovery")); - TActorId clusterDiscoveryServiceId = ActorSystem_->Register(actor, TMailboxType::HTSwap, ActorSystem_->AppData<TAppData>()->UserPoolId); + TActorId clusterDiscoveryServiceId = ActorSystem_->Register(actor, TMailboxType::HTSwap, ActorSystem_->AppData<TAppData>()->UserPoolId); ActorSystem_->RegisterLocalService(NPQ::NClusterDiscovery::MakeClusterDiscoveryServiceID(), clusterDiscoveryServiceId); - SetupIncomingRequests(std::move(logger)); + SetupIncomingRequests(std::move(logger)); } } -void TGRpcPQClusterDiscoveryService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter*) { +void TGRpcPQClusterDiscoveryService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter*) { } bool TGRpcPQClusterDiscoveryService::IncRequest() { @@ -54,7 +54,7 @@ void TGRpcPQClusterDiscoveryService::DecRequest() { } } -void TGRpcPQClusterDiscoveryService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { +void TGRpcPQClusterDiscoveryService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { auto getCounterBlock = NGRpcService::CreateCounterCb(Counters_, ActorSystem_); #ifdef ADD_REQUEST @@ -62,11 +62,11 @@ void TGRpcPQClusterDiscoveryService::SetupIncomingRequests(NGrpc::TLoggerPtr log #endif #define ADD_REQUEST(NAME, IN, OUT, ACTION) \ MakeIntrusive<TGRpcRequest<Ydb::PersQueue::ClusterDiscovery::IN, Ydb::PersQueue::ClusterDiscovery::OUT, TGRpcPQClusterDiscoveryService>>(this, &Service_, CQ_, \ - [this](NGrpc::IRequestContextBase* ctx) { \ + [this](NGrpc::IRequestContextBase* ctx) { \ NGRpcService::ReportGrpcReqToMon(*ActorSystem_, ctx->GetPeer()); \ ACTION; \ }, &Ydb::PersQueue::V1::ClusterDiscoveryService::AsyncService::Request ## NAME, \ - #NAME, logger, getCounterBlock("pq_cluster_discovery", #NAME))->Run(); + #NAME, logger, getCounterBlock("pq_cluster_discovery", #NAME))->Run(); ADD_REQUEST(DiscoverClusters, DiscoverClustersRequest, DiscoverClustersResponse, { ActorSystem_->Send(GRpcRequestProxyId_, new TEvDiscoverPQClustersRequest(ctx)); diff --git a/ydb/services/persqueue_cluster_discovery/grpc_service.h b/ydb/services/persqueue_cluster_discovery/grpc_service.h index 0828291e50a..7bc17ba2f5e 100644 --- a/ydb/services/persqueue_cluster_discovery/grpc_service.h +++ b/ydb/services/persqueue_cluster_discovery/grpc_service.h @@ -2,31 +2,31 @@ #include <library/cpp/actors/core/actorsystem.h> #include <ydb/public/api/grpc/draft/ydb_persqueue_v1.grpc.pb.h> -#include <library/cpp/grpc/server/grpc_server.h> +#include <library/cpp/grpc/server/grpc_server.h> namespace NKikimr::NGRpcService { class TGRpcPQClusterDiscoveryService - : public NGrpc::TGrpcServiceBase<Ydb::PersQueue::V1::ClusterDiscoveryService> { + : public NGrpc::TGrpcServiceBase<Ydb::PersQueue::V1::ClusterDiscoveryService> { public: TGRpcPQClusterDiscoveryService(NActors::TActorSystem* system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId id, const TMaybe<ui64>& requestsInflightLimit = Nothing()); - void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; - void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; + void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; + void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; bool IncRequest(); void DecRequest(); void StopService() noexcept override; - using NGrpc::TGrpcServiceBase<Ydb::PersQueue::V1::ClusterDiscoveryService>::GetService; + using NGrpc::TGrpcServiceBase<Ydb::PersQueue::V1::ClusterDiscoveryService>::GetService; private: - void SetupIncomingRequests(NGrpc::TLoggerPtr logger); + void SetupIncomingRequests(NGrpc::TLoggerPtr logger); NActors::TActorSystem* ActorSystem_; grpc::ServerCompletionQueue* CQ_ = nullptr; TIntrusivePtr<NMonitoring::TDynamicCounters> Counters_; - NActors::TActorId GRpcRequestProxyId_; + NActors::TActorId GRpcRequestProxyId_; THolder<NGrpc::TGlobalLimiter> Limiter; }; diff --git a/ydb/services/persqueue_v1/grpc_pq_actor.h b/ydb/services/persqueue_v1/grpc_pq_actor.h index 7518acba18d..65c8aca1db2 100644 --- a/ydb/services/persqueue_v1/grpc_pq_actor.h +++ b/ydb/services/persqueue_v1/grpc_pq_actor.h @@ -570,7 +570,7 @@ private: }; EState State; - TActorId SchemeCache; + TActorId SchemeCache; TActorId NewSchemeCache; TActorId Writer; @@ -649,7 +649,7 @@ private: TInstant LogSessionDeadline; ui64 BalancerTabletId; - TActorId PipeToBalancer; + TActorId PipeToBalancer; // PQ tablet configuration that we get at the time of session initialization NKikimrPQ::TPQTabletConfig InitialPQTabletConfig; @@ -720,12 +720,12 @@ private: const TActorContext& ctx); private: - const TActorId ParentId; + const TActorId ParentId; const ui64 Cookie; const TString Session; const TActorId MetaCacheId; - const TActorId NewSchemeCache; + const TActorId NewSchemeCache; const TString ClientId; const TString ClientPath; @@ -867,12 +867,12 @@ private: void ProcessAnswer(const NActors::TActorContext& ctx, TIntrusivePtr<TFormedReadResponse> formedResponse); // returns false if actor died void RegisterSessions(const NActors::TActorContext& ctx); - void RegisterSession(const TActorId& pipe, const TString& topic, const TVector<ui32>& groups, const TActorContext& ctx); + void RegisterSession(const TActorId& pipe, const TString& topic, const TVector<ui32>& groups, const TActorContext& ctx); struct TPartitionActorInfo; void DropPartition(THashMap<ui64, TPartitionActorInfo>::iterator it, const TActorContext& ctx); - bool ActualPartitionActor(const TActorId& part); + bool ActualPartitionActor(const TActorId& part); void ReleasePartition(const THashMap<ui64, TPartitionActorInfo>::iterator& it, bool couldBeReads, const TActorContext& ctx); // returns false if actor died @@ -890,10 +890,10 @@ private: const TInstant StartTimestamp; - TActorId SchemeCache; - TActorId NewSchemeCache; + TActorId SchemeCache; + TActorId NewSchemeCache; - TActorId AuthInitActor; + TActorId AuthInitActor; TIntrusivePtr<NACLib::TUserToken> Token; TString ClientId; @@ -919,7 +919,7 @@ private: TInstant LastACLCheckTimestamp; struct TPartitionActorInfo { - TActorId Actor; + TActorId Actor; const TPartitionId Partition; std::deque<ui64> Commits; bool Reading; @@ -937,7 +937,7 @@ private: TInstant AssignTimestamp; - TPartitionActorInfo(const TActorId& actor, const TPartitionId& partition, const TActorContext& ctx) + TPartitionActorInfo(const TActorId& actor, const TPartitionId& partition, const TActorContext& ctx) : Actor(actor) , Partition(partition) , Reading(false) @@ -955,7 +955,7 @@ private: }; - THashSet<TActorId> ActualPartitionActors; + THashSet<TActorId> ActualPartitionActors; THashMap<ui64, std::pair<ui32, ui64>> BalancerGeneration; ui64 NextAssignId; THashMap<ui64, TPartitionActorInfo> Partitions; //assignId -> info @@ -999,7 +999,7 @@ private: //returns byteSize diff i64 ApplyResponse(PersQueue::V1::MigrationStreamingReadServerMessage&& resp); - THashSet<TActorId> PartitionsTookPartInRead; + THashSet<TActorId> PartitionsTookPartInRead; TSet<TPartitionId> PartitionsTookPartInControlMessages; TSet<TPartitionInfo> PartitionsBecameAvailable; // Partitions that became available during this read request execution. @@ -1012,7 +1012,7 @@ private: TDuration WaitQuotaTime; }; - THashMap<TActorId, TFormedReadResponse::TPtr> PartitionToReadResponse; // Partition actor -> TFormedReadResponse answer that has this partition. + THashMap<TActorId, TFormedReadResponse::TPtr> PartitionToReadResponse; // Partition actor -> TFormedReadResponse answer that has this partition. // PartitionsTookPartInRead in formed read response contain this actor id. struct TControlMessages { @@ -1112,10 +1112,10 @@ private: void ProcessAnswers(const TActorContext& ctx); private: - TActorId SchemeCache; - TActorId NewSchemeCache; + TActorId SchemeCache; + TActorId NewSchemeCache; - TActorId AuthInitActor; + TActorId AuthInitActor; TTopicTabletsPairs TopicAndTablets; diff --git a/ydb/services/persqueue_v1/grpc_pq_read.h b/ydb/services/persqueue_v1/grpc_pq_read.h index 90132a8be0d..558063ed131 100644 --- a/ydb/services/persqueue_v1/grpc_pq_read.h +++ b/ydb/services/persqueue_v1/grpc_pq_read.h @@ -19,8 +19,8 @@ namespace V1 { -inline TActorId GetPQReadServiceActorID() { - return TActorId(0, "PQReadSvc"); +inline TActorId GetPQReadServiceActorID() { + return TActorId(0, "PQReadSvc"); } IActor* CreatePQReadService(const NActors::TActorId& schemeCache, const NActors::TActorId& newSchemeCache, @@ -60,12 +60,12 @@ private: void Handle(TEvPQProxy::TEvSessionDead::TPtr& ev, const TActorContext& ctx); - NActors::TActorId SchemeCache; - NActors::TActorId NewSchemeCache; + NActors::TActorId SchemeCache; + NActors::TActorId NewSchemeCache; TAtomic LastCookie = 0; - THashMap<ui64, TActorId> Sessions; + THashMap<ui64, TActorId> Sessions; TIntrusivePtr<NMonitoring::TDynamicCounters> Counters; diff --git a/ydb/services/persqueue_v1/grpc_pq_read_actor.cpp b/ydb/services/persqueue_v1/grpc_pq_read_actor.cpp index bbb689ea6f6..654edfcfcd4 100644 --- a/ydb/services/persqueue_v1/grpc_pq_read_actor.cpp +++ b/ydb/services/persqueue_v1/grpc_pq_read_actor.cpp @@ -106,7 +106,7 @@ bool RemoveEmptyMessages(MigrationStreamingReadServerMessage::DataBatch& data) { class TPartitionActor : public NActors::TActorBootstrapped<TPartitionActor> { public: - TPartitionActor(const TActorId& parentId, const TString& clientId, const TString& clientPath, const ui64 cookie, const TString& session, const TPartitionId& partition, ui32 generation, ui32 step, + TPartitionActor(const TActorId& parentId, const TString& clientId, const TString& clientPath, const ui64 cookie, const TString& session, const TPartitionId& partition, ui32 generation, ui32 step, const ui64 tabletID, const TReadSessionActor::TTopicCounters& counters, const bool commitsDisabled, const TString& clientDC); ~TPartitionActor(); @@ -171,7 +171,7 @@ private: private: - const TActorId ParentId; + const TActorId ParentId; const TString ClientId; const TString ClientPath; const ui64 Cookie; @@ -203,7 +203,7 @@ private: bool StartReading; bool AllPrepareInited; bool FirstInit; - TActorId PipeClient; + TActorId PipeClient; ui32 PipeGeneration; bool RequestInfly; NKikimrClient::TPersQueueRequest CurrentRequest; @@ -786,7 +786,7 @@ void TReadSessionActor::Handle(TEvPQProxy::TEvReadInit::TPtr& ev, const TActorCo } -void TReadSessionActor::RegisterSession(const TActorId& pipe, const TString& topic, const TVector<ui32>& groups, const TActorContext& ctx) +void TReadSessionActor::RegisterSession(const TActorId& pipe, const TString& topic, const TVector<ui32>& groups, const TActorContext& ctx) { LOG_INFO_S(ctx, NKikimrServices::PQ_READ_PROXY, PQ_LOG_PREFIX << " register session to " << topic); @@ -795,7 +795,7 @@ void TReadSessionActor::RegisterSession(const TActorId& pipe, const TString& top auto& req = request->Record; req.SetSession(Session); req.SetClientNode(PeerName); - ActorIdToProto(pipe, req.MutablePipeClient()); + ActorIdToProto(pipe, req.MutablePipeClient()); req.SetClientId(ClientId); for (ui32 i = 0; i < groups.size(); ++i) { @@ -893,7 +893,7 @@ void TReadSessionActor::Handle(TEvPQProxy::TEvAuthResultOk::TPtr& ev, const TAct PQ_LOG_PREFIX << " auth ok, got " << ev->Get()->TopicAndTablets.size() << " topics, init done " << InitDone ); - AuthInitActor = TActorId(); + AuthInitActor = TActorId(); if (!InitDone) { ui32 initBorder = AppData(ctx)->PQConfig.GetReadInitLatencyBigMs(); @@ -980,7 +980,7 @@ void TReadSessionActor::Handle(TEvPersQueue::TEvLockPartition::TPtr& ev, const T Y_VERIFY(record.GetSession() == Session); Y_VERIFY(record.GetClientId() == ClientId); - TActorId pipe = ActorIdFromProto(record.GetPipeClient()); + TActorId pipe = ActorIdFromProto(record.GetPipeClient()); auto converterIter = FullPathToConverter.find(NPersQueue::NormalizeFullPath(record.GetPath())); if (converterIter.IsEnd()) { @@ -1019,7 +1019,7 @@ void TReadSessionActor::Handle(TEvPersQueue::TEvLockPartition::TPtr& ev, const T IActor* partitionActor = new TPartitionActor(ctx.SelfID, ClientId, ClientPath, Cookie, Session, partitionId, record.GetGeneration(), record.GetStep(), record.GetTabletId(), it->second, CommitsDisabled, ClientDC); - TActorId actorId = ctx.Register(partitionActor); + TActorId actorId = ctx.Register(partitionActor); PartsPerSession.DecFor(Partitions.size(), 1); Y_VERIFY(record.GetGeneration() > 0); auto pp = Partitions.insert(std::make_pair(assignId, TPartitionActorInfo{actorId, partitionId, ctx})); @@ -1156,7 +1156,7 @@ void TReadSessionActor::Handle(TEvPersQueue::TEvReleasePartition::TPtr& ev, cons Y_VERIFY(!it.IsEnd()); auto& converter = it->second.TopicNameConverter; - TActorId pipe = ActorIdFromProto(record.GetPipeClient()); + TActorId pipe = ActorIdFromProto(record.GetPipeClient()); if (pipe != it->second.PipeClient) { //this is message from old version of pipe return; @@ -1165,7 +1165,7 @@ void TReadSessionActor::Handle(TEvPersQueue::TEvReleasePartition::TPtr& ev, cons for (ui32 c = 0; c < record.GetCount(); ++c) { Y_VERIFY(!Partitions.empty()); - TActorId actorId = TActorId{}; + TActorId actorId = TActorId{}; auto jt = Partitions.begin(); ui32 i = 0; for (auto it = Partitions.begin(); it != Partitions.end(); ++it) { @@ -1223,7 +1223,7 @@ void TReadSessionActor::InformBalancerAboutRelease(const THashMap<ui64, TPartiti Y_VERIFY(jt != Topics.end()); req.SetSession(Session); - ActorIdToProto(jt->second.PipeClient, req.MutablePipeClient()); + ActorIdToProto(jt->second.PipeClient, req.MutablePipeClient()); req.SetClientId(ClientId); req.SetTopic(converter->GetPrimaryPath()); req.SetPartition(it->second.Partition.Partition); @@ -1291,7 +1291,7 @@ void TReadSessionActor::Handle(TEvTabletPipe::TEvClientConnected::TPtr& ev, cons } } -bool TReadSessionActor::ActualPartitionActor(const TActorId& part) { +bool TReadSessionActor::ActualPartitionActor(const TActorId& part) { return ActualPartitionActors.contains(part); } @@ -1421,7 +1421,7 @@ i64 TReadSessionActor::TFormedReadResponse::ApplyResponse(MigrationStreamingRead } void TReadSessionActor::Handle(TEvPQProxy::TEvReadResponse::TPtr& ev, const TActorContext& ctx) { - TActorId sender = ev->Sender; + TActorId sender = ev->Sender; if (!ActualPartitionActor(sender)) return; @@ -1515,7 +1515,7 @@ void TReadSessionActor::ProcessAnswer(const TActorContext& ctx, TFormedReadRespo } } - for (const TActorId& p : formedResponse->PartitionsTookPartInRead) { + for (const TActorId& p : formedResponse->PartitionsTookPartInRead) { PartitionToReadResponse.erase(p); } @@ -1669,7 +1669,7 @@ void TReadSessionActor::HandleWakeup(const TActorContext& ctx) { ////////////////// PARTITION ACTOR -TPartitionActor::TPartitionActor(const TActorId& parentId, const TString& clientId, const TString& clientPath, const ui64 cookie, const TString& session, +TPartitionActor::TPartitionActor(const TActorId& parentId, const TString& clientId, const TString& clientPath, const ui64 cookie, const TString& session, const TPartitionId& partition, const ui32 generation, const ui32 step, const ui64 tabletID, const TReadSessionActor::TTopicCounters& counters, bool commitsDisabled, const TString& clientDC) : ParentId(parentId) @@ -1817,7 +1817,7 @@ void TPartitionActor::SendCommit(const ui64 readId, const ui64 offset, const TAc Y_VERIFY(PipeClient); - ActorIdToProto(PipeClient, request.MutablePartitionRequest()->MutablePipeClient()); + ActorIdToProto(PipeClient, request.MutablePartitionRequest()->MutablePipeClient()); auto commit = request.MutablePartitionRequest()->MutableCmdSetClientOffset(); commit->SetClientId(ClientId); commit->SetOffset(offset); @@ -1842,7 +1842,7 @@ void TPartitionActor::RestartPipe(const TActorContext& ctx, const TString& reaso Counters.Errors.Inc(); NTabletPipe::CloseClient(ctx, PipeClient); - PipeClient = TActorId{}; + PipeClient = TActorId{}; if (errorCode != NPersQueue::NErrorCode::OVERLOAD) ++PipeGeneration; @@ -1884,7 +1884,7 @@ void TPartitionActor::Handle(const TEvPQProxy::TEvRestartPipe::TPtr&, const TAct TAutoPtr<TEvPersQueue::TEvRequest> event(new TEvPersQueue::TEvRequest); event->Record = CurrentRequest; - ActorIdToProto(PipeClient, event->Record.MutablePartitionRequest()->MutablePipeClient()); + ActorIdToProto(PipeClient, event->Record.MutablePartitionRequest()->MutablePipeClient()); NTabletPipe::SendData(ctx, PipeClient, event.Release()); } @@ -2322,7 +2322,7 @@ void TPartitionActor::InitLockPartition(const TActorContext& ctx) { request.MutablePartitionRequest()->SetPartition(Partition.Partition); request.MutablePartitionRequest()->SetCookie(INIT_COOKIE); - ActorIdToProto(PipeClient, request.MutablePartitionRequest()->MutablePipeClient()); + ActorIdToProto(PipeClient, request.MutablePartitionRequest()->MutablePipeClient()); auto cmd = request.MutablePartitionRequest()->MutableCmdCreateSession(); cmd->SetClientId(ClientId); @@ -2443,7 +2443,7 @@ void TPartitionActor::Handle(TEvPQProxy::TEvRead::TPtr& ev, const TActorContext& request.MutablePartitionRequest()->SetPartition(Partition.Partition); request.MutablePartitionRequest()->SetCookie((ui64)ReadOffset); - ActorIdToProto(PipeClient, request.MutablePartitionRequest()->MutablePipeClient()); + ActorIdToProto(PipeClient, request.MutablePartitionRequest()->MutablePipeClient()); auto read = request.MutablePartitionRequest()->MutableCmdRead(); read->SetClientId(ClientId); read->SetClientDC(ClientDC); diff --git a/ydb/services/persqueue_v1/grpc_pq_schema.cpp b/ydb/services/persqueue_v1/grpc_pq_schema.cpp index 2cd761c51fb..7756adc797b 100644 --- a/ydb/services/persqueue_v1/grpc_pq_schema.cpp +++ b/ydb/services/persqueue_v1/grpc_pq_schema.cpp @@ -30,7 +30,7 @@ IActor* CreatePQSchemaService(const TActorId& schemeCache, TIntrusivePtr<NMonito -TPQSchemaService::TPQSchemaService(const TActorId& schemeCache, +TPQSchemaService::TPQSchemaService(const TActorId& schemeCache, TIntrusivePtr<NMonitoring::TDynamicCounters> counters) : SchemeCache(schemeCache) , Counters(counters) diff --git a/ydb/services/persqueue_v1/grpc_pq_schema.h b/ydb/services/persqueue_v1/grpc_pq_schema.h index f8a10d8d421..2ef8c503e4d 100644 --- a/ydb/services/persqueue_v1/grpc_pq_schema.h +++ b/ydb/services/persqueue_v1/grpc_pq_schema.h @@ -16,15 +16,15 @@ namespace NKikimr::NGRpcProxy::V1 { static const i64 DEFAULT_MAX_DATABASE_MESSAGEGROUP_SEQNO_RETENTION_PERIOD = 16*24*60*60*1000; -inline TActorId GetPQSchemaServiceActorID() { - return TActorId(0, "PQSchmSvc"); +inline TActorId GetPQSchemaServiceActorID() { + return TActorId(0, "PQSchmSvc"); } -IActor* CreatePQSchemaService(const NActors::TActorId& schemeCache, TIntrusivePtr<NMonitoring::TDynamicCounters> counters); +IActor* CreatePQSchemaService(const NActors::TActorId& schemeCache, TIntrusivePtr<NMonitoring::TDynamicCounters> counters); class TPQSchemaService : public NActors::TActorBootstrapped<TPQSchemaService> { public: - TPQSchemaService(const NActors::TActorId& schemeCache, TIntrusivePtr<NMonitoring::TDynamicCounters> counters); + TPQSchemaService(const NActors::TActorId& schemeCache, TIntrusivePtr<NMonitoring::TDynamicCounters> counters); void Bootstrap(const TActorContext& ctx); @@ -54,7 +54,7 @@ private: void Handle(NPQ::NClusterTracker::TEvClusterTracker::TEvClustersUpdate::TPtr& ev); - NActors::TActorId SchemeCache; + NActors::TActorId SchemeCache; TIntrusivePtr<NMonitoring::TDynamicCounters> Counters; diff --git a/ydb/services/persqueue_v1/grpc_pq_write.h b/ydb/services/persqueue_v1/grpc_pq_write.h index f6001fa4a4d..73148388b29 100644 --- a/ydb/services/persqueue_v1/grpc_pq_write.h +++ b/ydb/services/persqueue_v1/grpc_pq_write.h @@ -17,8 +17,8 @@ namespace NKikimr { namespace NGRpcProxy { namespace V1 { -inline TActorId GetPQWriteServiceActorID() { - return TActorId(0, "PQWriteSvc"); +inline TActorId GetPQWriteServiceActorID() { + return TActorId(0, "PQWriteSvc"); } IActor* CreatePQWriteService(const NActors::TActorId& schemeCache, const NActors::TActorId& newSchemeCache, @@ -59,12 +59,12 @@ private: void Handle(TEvPQProxy::TEvSessionSetPreferredCluster::TPtr& ev, const TActorContext& ctx); void Handle(TEvPQProxy::TEvSessionDead::TPtr& ev, const TActorContext& ctx); - NActors::TActorId SchemeCache; + NActors::TActorId SchemeCache; NActors::TActorId NewSchemeCache; TAtomic LastCookie = 0; - THashMap<ui64, TActorId> Sessions; + THashMap<ui64, TActorId> Sessions; // Created at by session cookie map by remote preferred cluster name THashMap<TString, THashMap<ui64, TInstant>> SessionsByRemotePreferredCluster; THashMap<ui64, TString> RemotePreferredClusterBySessionCookie; diff --git a/ydb/services/persqueue_v1/persqueue.cpp b/ydb/services/persqueue_v1/persqueue.cpp index 8efb6707a0c..ccacc4d43af 100644 --- a/ydb/services/persqueue_v1/persqueue.cpp +++ b/ydb/services/persqueue_v1/persqueue.cpp @@ -24,14 +24,14 @@ TGRpcPersQueueService::TGRpcPersQueueService(NActors::TActorSystem *system, TInt , GRpcRequestProxy(grpcRequestProxy) { } -void TGRpcPersQueueService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { +void TGRpcPersQueueService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { CQ = cq; InitNewSchemeCacheActor(); if (ActorSystem->AppData<TAppData>()->PQConfig.GetEnabled()) { IActor* writeSvc = NGRpcProxy::V1::CreatePQWriteService(SchemeCache, NewSchemeCache,Counters, PersQueueWriteSessionsMaxCount); - TActorId actorId = ActorSystem->Register(writeSvc, TMailboxType::HTSwap, ActorSystem->AppData<TAppData>()->UserPoolId); + TActorId actorId = ActorSystem->Register(writeSvc, TMailboxType::HTSwap, ActorSystem->AppData<TAppData>()->UserPoolId); ActorSystem->RegisterLocalService(NGRpcProxy::V1::GetPQWriteServiceActorID(), actorId); IActor* readSvc = NGRpcProxy::V1::CreatePQReadService(SchemeCache, NewSchemeCache, Counters, PersQueueReadSessionsMaxCount); @@ -42,11 +42,11 @@ void TGRpcPersQueueService::InitService(grpc::ServerCompletionQueue *cq, NGrpc:: actorId = ActorSystem->Register(schemaSvc, TMailboxType::HTSwap, ActorSystem->AppData<TAppData>()->UserPoolId); ActorSystem->RegisterLocalService(NGRpcProxy::V1::GetPQSchemaServiceActorID(), actorId); - SetupIncomingRequests(std::move(logger)); + SetupIncomingRequests(std::move(logger)); } } -void TGRpcPersQueueService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { +void TGRpcPersQueueService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { Limiter = limiter; } @@ -66,7 +66,7 @@ void TGRpcPersQueueService::InitNewSchemeCacheActor() { TMailboxType::HTSwap, ActorSystem->AppData<TAppData>()->UserPoolId); } -void TGRpcPersQueueService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { +void TGRpcPersQueueService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { auto getCounterBlock = NKikimr::NGRpcService::CreateCounterCb(Counters, ActorSystem); @@ -115,7 +115,7 @@ void TGRpcPersQueueService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { #endif #define ADD_REQUEST(NAME, SVC, IN, OUT, ACTION) \ MakeIntrusive<TGRpcRequest<Ydb::PersQueue::V1::IN, Ydb::PersQueue::V1::OUT, NGRpcService::V1::TGRpcPersQueueService>>(this, this->GetService(), CQ, \ - [this](NGrpc::IRequestContextBase *ctx) { \ + [this](NGrpc::IRequestContextBase *ctx) { \ NGRpcService::ReportGrpcReqToMon(*ActorSystem, ctx->GetPeer()); \ ACTION; \ }, &Ydb::PersQueue::V1::SVC::AsyncService::Request ## NAME, \ diff --git a/ydb/services/persqueue_v1/persqueue.h b/ydb/services/persqueue_v1/persqueue.h index 98d43993887..eefe1ee4c2b 100644 --- a/ydb/services/persqueue_v1/persqueue.h +++ b/ydb/services/persqueue_v1/persqueue.h @@ -4,7 +4,7 @@ #include <ydb/public/api/grpc/draft/ydb_persqueue_v1.grpc.pb.h> -#include <library/cpp/grpc/server/grpc_server.h> +#include <library/cpp/grpc/server/grpc_server.h> namespace NKikimr { @@ -13,22 +13,22 @@ namespace NGRpcService { namespace V1 { class TGRpcPersQueueService - : public NGrpc::TGrpcServiceBase<Ydb::PersQueue::V1::PersQueueService> + : public NGrpc::TGrpcServiceBase<Ydb::PersQueue::V1::PersQueueService> { public: - TGRpcPersQueueService(NActors::TActorSystem* system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, const NActors::TActorId& schemeCache, const NActors::TActorId& grpcRequestProxy); + TGRpcPersQueueService(NActors::TActorSystem* system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, const NActors::TActorId& schemeCache, const NActors::TActorId& grpcRequestProxy); - void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; - void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; + void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; + void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; void StopService() noexcept override; - using NGrpc::TGrpcServiceBase<Ydb::PersQueue::V1::PersQueueService>::GetService; + using NGrpc::TGrpcServiceBase<Ydb::PersQueue::V1::PersQueueService>::GetService; bool IncRequest(); void DecRequest(); private: - void SetupIncomingRequests(NGrpc::TLoggerPtr logger); + void SetupIncomingRequests(NGrpc::TLoggerPtr logger); void InitNewSchemeCacheActor(); @@ -37,9 +37,9 @@ private: TIntrusivePtr<NMonitoring::TDynamicCounters> Counters; NGrpc::TGlobalLimiter* Limiter = nullptr; - NActors::TActorId SchemeCache; + NActors::TActorId SchemeCache; NActors::TActorId NewSchemeCache; - NActors::TActorId GRpcRequestProxy; + NActors::TActorId GRpcRequestProxy; }; } // namespace V1 diff --git a/ydb/services/persqueue_v1/persqueue_ut.cpp b/ydb/services/persqueue_v1/persqueue_ut.cpp index 419d2a391a0..3bc9b217285 100644 --- a/ydb/services/persqueue_v1/persqueue_ut.cpp +++ b/ydb/services/persqueue_v1/persqueue_ut.cpp @@ -20,7 +20,7 @@ #include <library/cpp/testing/unittest/tests_data.h> #include <library/cpp/testing/unittest/registar.h> #include <library/cpp/json/json_reader.h> -#include <library/cpp/monlib/dynamic_counters/encode.h> +#include <library/cpp/monlib/dynamic_counters/encode.h> #include <google/protobuf/text_format.h> #include <google/protobuf/util/message_differencer.h> @@ -818,9 +818,9 @@ namespace { ui32 fromDisk = info0.BlobsFromDisk + info16.BlobsFromDisk; ui32 fromCache = info0.BlobsFromCache + info16.BlobsFromCache; - UNIT_ASSERT(fromDisk > 0); + UNIT_ASSERT(fromDisk > 0); UNIT_ASSERT(fromDisk < 5); - UNIT_ASSERT(fromCache > 0); + UNIT_ASSERT(fromCache > 0); UNIT_ASSERT(fromCache < 5); } diff --git a/ydb/services/persqueue_v1/ut/persqueue_test_fixture.h b/ydb/services/persqueue_v1/ut/persqueue_test_fixture.h index 5a68b295b28..2933d1f3a17 100644 --- a/ydb/services/persqueue_v1/ut/persqueue_test_fixture.h +++ b/ydb/services/persqueue_v1/ut/persqueue_test_fixture.h @@ -265,5 +265,5 @@ namespace NKikimr::NPersQueueTests { } */ }; -} - +} + diff --git a/ydb/services/persqueue_v1/ut/rate_limiter_test_setup.cpp b/ydb/services/persqueue_v1/ut/rate_limiter_test_setup.cpp index 40c3ff793fe..779d60e7785 100644 --- a/ydb/services/persqueue_v1/ut/rate_limiter_test_setup.cpp +++ b/ydb/services/persqueue_v1/ut/rate_limiter_test_setup.cpp @@ -51,10 +51,10 @@ void TRateLimiterTestSetup::CreateKesus(const TString& account) { UNIT_ASSERT_C(createKesusResult == NMsgBusProxy::MSTATUS_OK, createKesusResult); const TString kesusPath = TStringBuilder() << QuotersRootPath << "/" << account; - + Cerr << "Creating kesus with path=" << kesusPath << Endl; auto setAccountQuota = [&](const TString& quotaPrefix, double value) { - Cerr << "Adding quota for account kesus=" << kesusPath << " quota-path=" << quotaPrefix << " value=" << value << Endl; + Cerr << "Adding quota for account kesus=" << kesusPath << " quota-path=" << quotaPrefix << " value=" << value << Endl; const auto statusCode = Server->AnnoyingClient->AddQuoterResource( Server->CleverServer->GetRuntime(), kesusPath, quotaPrefix, value ); @@ -74,7 +74,7 @@ void TRateLimiterTestSetup::CreateQuotaResources(const TString& path, const TStr } TStringBuilder prefixPath; prefixPath << quotaPrefix; - + auto firstIt = pathComponents.begin() + 1; // resource path must be without account auto lastIt = pathComponents.end() - (excludeLastComponent ? 1 : 0); diff --git a/ydb/services/persqueue_v1/ut/test_utils.h b/ydb/services/persqueue_v1/ut/test_utils.h index 5f52c4b6d86..c02308b0d7d 100644 --- a/ydb/services/persqueue_v1/ut/test_utils.h +++ b/ydb/services/persqueue_v1/ut/test_utils.h @@ -31,7 +31,7 @@ inline void WaitACLModification() { const ui16 grpc = pm.GetPort(2135);\ TServerSettings settings = PQSettings(port);\ TServer server = TServer(settings);\ - server.EnableGRpc(NGrpc::TServerOptions().SetHost("localhost").SetPort(grpc));\ + server.EnableGRpc(NGrpc::TServerOptions().SetHost("localhost").SetPort(grpc));\ TFlatMsgBusPQClient client(settings, grpc);\ client.FullInit();\ client.CreateTopic("rt3.dc1--" + TString(topic), 1);\ diff --git a/ydb/services/rate_limiter/grpc_service.cpp b/ydb/services/rate_limiter/grpc_service.cpp index 61d8f545a9d..480045d29ac 100644 --- a/ydb/services/rate_limiter/grpc_service.cpp +++ b/ydb/services/rate_limiter/grpc_service.cpp @@ -6,7 +6,7 @@ namespace NKikimr::NQuoter { -TRateLimiterGRpcService::TRateLimiterGRpcService(NActors::TActorSystem* actorSystem, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId grpcRequestProxyId) +TRateLimiterGRpcService::TRateLimiterGRpcService(NActors::TActorSystem* actorSystem, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId grpcRequestProxyId) : ActorSystem(actorSystem) , Counters(std::move(counters)) , GRpcRequestProxyId(grpcRequestProxyId) @@ -15,12 +15,12 @@ TRateLimiterGRpcService::TRateLimiterGRpcService(NActors::TActorSystem* actorSys TRateLimiterGRpcService::~TRateLimiterGRpcService() = default; -void TRateLimiterGRpcService::InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) { +void TRateLimiterGRpcService::InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) { CQ = cq; - SetupIncomingRequests(std::move(logger)); + SetupIncomingRequests(std::move(logger)); } -void TRateLimiterGRpcService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { +void TRateLimiterGRpcService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { Limiter = limiter; } @@ -32,7 +32,7 @@ void TRateLimiterGRpcService::DecRequest() { Limiter->Dec(); } -void TRateLimiterGRpcService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { +void TRateLimiterGRpcService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { auto getCounterBlock = NGRpcService::CreateCounterCb(Counters, ActorSystem); #ifdef SETUP_METHOD @@ -48,13 +48,13 @@ void TRateLimiterGRpcService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { this, \ &Service_, \ CQ, \ - [this](NGrpc::IRequestContextBase* reqCtx) { \ + [this](NGrpc::IRequestContextBase* reqCtx) { \ NGRpcService::ReportGrpcReqToMon(*ActorSystem, reqCtx->GetPeer()); \ ActorSystem->Send(GRpcRequestProxyId, new NGRpcService::event(reqCtx)); \ }, \ &Ydb::RateLimiter::V1::RateLimiterService::AsyncService::Y_CAT(Request, methodName), \ "RateLimiter/" Y_STRINGIZE(methodName), \ - logger, \ + logger, \ getCounterBlock("rate_limiter", Y_STRINGIZE(methodName)) \ )->Run() diff --git a/ydb/services/rate_limiter/grpc_service.h b/ydb/services/rate_limiter/grpc_service.h index 69d5f616214..82ab6f91f53 100644 --- a/ydb/services/rate_limiter/grpc_service.h +++ b/ydb/services/rate_limiter/grpc_service.h @@ -1,34 +1,34 @@ #pragma once #include <library/cpp/actors/core/actorsystem.h> -#include <library/cpp/grpc/server/grpc_server.h> +#include <library/cpp/grpc/server/grpc_server.h> #include <ydb/public/api/grpc/ydb_rate_limiter_v1.grpc.pb.h> namespace NKikimr::NQuoter { class TRateLimiterGRpcService - : public NGrpc::TGrpcServiceBase<Ydb::RateLimiter::V1::RateLimiterService> + : public NGrpc::TGrpcServiceBase<Ydb::RateLimiter::V1::RateLimiterService> { public: - TRateLimiterGRpcService(NActors::TActorSystem* actorSystem, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId grpcRequestProxyId); + TRateLimiterGRpcService(NActors::TActorSystem* actorSystem, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId grpcRequestProxyId); ~TRateLimiterGRpcService(); - void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; - void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; + void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; + void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; bool IncRequest(); void DecRequest(); private: - void SetupIncomingRequests(NGrpc::TLoggerPtr logger); + void SetupIncomingRequests(NGrpc::TLoggerPtr logger); private: NActors::TActorSystem* ActorSystem = nullptr; TIntrusivePtr<NMonitoring::TDynamicCounters> Counters; - NActors::TActorId GRpcRequestProxyId; + NActors::TActorId GRpcRequestProxyId; grpc::ServerCompletionQueue* CQ = nullptr; - NGrpc::TGlobalLimiter* Limiter = nullptr; + NGrpc::TGlobalLimiter* Limiter = nullptr; }; } // namespace NKikimr::NQuoter diff --git a/ydb/services/rate_limiter/ya.make b/ydb/services/rate_limiter/ya.make index 75695eccdc3..134624e8844 100644 --- a/ydb/services/rate_limiter/ya.make +++ b/ydb/services/rate_limiter/ya.make @@ -10,7 +10,7 @@ SRCS( ) PEERDIR( - library/cpp/grpc/server + library/cpp/grpc/server ydb/core/grpc_services ydb/core/kesus/tablet ydb/public/api/grpc diff --git a/ydb/services/ydb/ydb_clickhouse_internal.cpp b/ydb/services/ydb/ydb_clickhouse_internal.cpp index 5b955b7a4a2..543bcd98ee1 100644 --- a/ydb/services/ydb/ydb_clickhouse_internal.cpp +++ b/ydb/services/ydb/ydb_clickhouse_internal.cpp @@ -10,18 +10,18 @@ namespace NGRpcService { TGRpcYdbClickhouseInternalService::TGRpcYdbClickhouseInternalService(NActors::TActorSystem *system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, TIntrusivePtr<TInFlightLimiterRegistry> inFlightLimiterRegistry, - NActors::TActorId id) + NActors::TActorId id) : ActorSystem_(system) , Counters_(counters) , LimiterRegistry_(inFlightLimiterRegistry) , GRpcRequestProxyId_(id) {} -void TGRpcYdbClickhouseInternalService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { +void TGRpcYdbClickhouseInternalService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { CQ_ = cq; - SetupIncomingRequests(std::move(logger)); + SetupIncomingRequests(std::move(logger)); } -void TGRpcYdbClickhouseInternalService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { +void TGRpcYdbClickhouseInternalService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { Limiter_ = limiter; } @@ -34,7 +34,7 @@ void TGRpcYdbClickhouseInternalService::DecRequest() { Y_ASSERT(Limiter_->GetCurrentInFlight() >= 0); } -void TGRpcYdbClickhouseInternalService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { +void TGRpcYdbClickhouseInternalService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { auto getCounterBlock = CreateCounterCb(Counters_, ActorSystem_); auto getLimiter = CreateLimiterCb(LimiterRegistry_); @@ -43,11 +43,11 @@ void TGRpcYdbClickhouseInternalService::SetupIncomingRequests(NGrpc::TLoggerPtr #endif #define ADD_REQUEST(NAME, IN, OUT, ACTION) \ MakeIntrusive<TGRpcRequest<Ydb::ClickhouseInternal::IN, Ydb::ClickhouseInternal::OUT, TGRpcYdbClickhouseInternalService>>(this, &Service_, CQ_, \ - [this](NGrpc::IRequestContextBase *ctx) { \ + [this](NGrpc::IRequestContextBase *ctx) { \ NGRpcService::ReportGrpcReqToMon(*ActorSystem_, ctx->GetPeer()); \ ACTION; \ }, &Ydb::ClickhouseInternal::V1::ClickhouseInternalService::AsyncService::Request ## NAME, \ - #NAME, logger, getCounterBlock("clickhouse_internal", #NAME), getLimiter("ClickhouseInternal", #NAME, DEFAULT_MAX_IN_FLIGHT))->Run(); + #NAME, logger, getCounterBlock("clickhouse_internal", #NAME), getLimiter("ClickhouseInternal", #NAME, DEFAULT_MAX_IN_FLIGHT))->Run(); ADD_REQUEST(Scan, ScanRequest, ScanResponse, { ActorSystem_->Send(GRpcRequestProxyId_, new TEvReadColumnsRequest(ctx)); diff --git a/ydb/services/ydb/ydb_clickhouse_internal.h b/ydb/services/ydb/ydb_clickhouse_internal.h index cb3cd81ff40..a0ff8863d10 100644 --- a/ydb/services/ydb/ydb_clickhouse_internal.h +++ b/ydb/services/ydb/ydb_clickhouse_internal.h @@ -3,7 +3,7 @@ #include <ydb/public/api/grpc/draft/ydb_clickhouse_internal_v1.grpc.pb.h> #include <ydb/core/grpc_services/grpc_helper.h> -#include <library/cpp/grpc/server/grpc_server.h> +#include <library/cpp/grpc/server/grpc_server.h> #include <library/cpp/actors/core/actorsystem.h> @@ -11,29 +11,29 @@ namespace NKikimr { namespace NGRpcService { class TGRpcYdbClickhouseInternalService - : public NGrpc::TGrpcServiceBase<Ydb::ClickhouseInternal::V1::ClickhouseInternalService> + : public NGrpc::TGrpcServiceBase<Ydb::ClickhouseInternal::V1::ClickhouseInternalService> { private: constexpr static i64 DEFAULT_MAX_IN_FLIGHT = 200; public: TGRpcYdbClickhouseInternalService(NActors::TActorSystem* system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, - TIntrusivePtr<NGRpcService::TInFlightLimiterRegistry> inFlightLimiterRegistry, NActors::TActorId id); + TIntrusivePtr<NGRpcService::TInFlightLimiterRegistry> inFlightLimiterRegistry, NActors::TActorId id); - void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; - void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; + void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; + void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; bool IncRequest(); void DecRequest(); private: - void SetupIncomingRequests(NGrpc::TLoggerPtr logger); + void SetupIncomingRequests(NGrpc::TLoggerPtr logger); NActors::TActorSystem* ActorSystem_; grpc::ServerCompletionQueue* CQ_ = nullptr; TIntrusivePtr<NMonitoring::TDynamicCounters> Counters_; TIntrusivePtr<NGRpcService::TInFlightLimiterRegistry> LimiterRegistry_; - NActors::TActorId GRpcRequestProxyId_; + NActors::TActorId GRpcRequestProxyId_; NGrpc::TGlobalLimiter* Limiter_ = nullptr; }; diff --git a/ydb/services/ydb/ydb_common_ut.h b/ydb/services/ydb/ydb_common_ut.h index 211e84e678f..c067928c785 100644 --- a/ydb/services/ydb/ydb_common_ut.h +++ b/ydb/services/ydb/ydb_common_ut.h @@ -105,13 +105,13 @@ public: Server_->GetRuntime()->SetLogPriority(NKikimrServices::KQP_COMPUTE, NActors::NLog::PRI_DEBUG); } - NGrpc::TServerOptions grpcOption; + NGrpc::TServerOptions grpcOption; if (TestSettings::AUTH) { grpcOption.SetUseAuth(true); } grpcOption.SetPort(grpc); if (TestSettings::SSL) { - NGrpc::TSslData sslData{NYdbSslTestData::ServerCrt, + NGrpc::TSslData sslData{NYdbSslTestData::ServerCrt, NYdbSslTestData::ServerKey, NYdbSslTestData::CaCrt}; grpcOption.SetSslData(sslData); diff --git a/ydb/services/ydb/ydb_dummy.cpp b/ydb/services/ydb/ydb_dummy.cpp index 933e20c9189..c628b0b3e4a 100644 --- a/ydb/services/ydb/ydb_dummy.cpp +++ b/ydb/services/ydb/ydb_dummy.cpp @@ -16,7 +16,7 @@ using namespace Draft::Dummy; using TEvInfiniteRequest = TGRpcRequestWrapper<0, InfiniteRequest, InfiniteResponse, true>; -static void HandlePing(NGrpc::IRequestContextBase* ctx) { +static void HandlePing(NGrpc::IRequestContextBase* ctx) { auto req = static_cast<const PingRequest*>(ctx->GetRequest()); auto resp = google::protobuf::Arena::CreateMessage<PingResponse>(ctx->GetArena()); if (req->copy()) { @@ -59,7 +59,7 @@ private: class TBiStreamPingRequestRPC : public TActorBootstrapped<TBiStreamPingRequestRPC> { using TBase = TActorBootstrapped<TBiStreamPingRequestRPC>; - using IContext = NGRpcServer::IGRpcStreamingContext< + using IContext = NGRpcServer::IGRpcStreamingContext< TEvBiStreamPingRequest::TRequest, TEvBiStreamPingRequest::TResponse>; @@ -144,11 +144,11 @@ void TGRpcYdbDummyService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { #endif #define ADD_REQUEST(NAME, IN, OUT, ACTION) \ MakeIntrusive<TGRpcRequest<Draft::Dummy::IN, Draft::Dummy::OUT, TGRpcYdbDummyService>>(this, &Service_, CQ_, \ - [this](NGrpc::IRequestContextBase *ctx) { \ + [this](NGrpc::IRequestContextBase *ctx) { \ NGRpcService::ReportGrpcReqToMon(*ActorSystem_, ctx->GetPeer()); \ ACTION; \ }, &Draft::Dummy::DummyService::AsyncService::Request ## NAME, \ - #NAME, logger, getCounterBlock("dummy", #NAME))->Run(); + #NAME, logger, getCounterBlock("dummy", #NAME))->Run(); ADD_REQUEST(Ping, PingRequest, PingResponse, { HandlePing(ctx); diff --git a/ydb/services/ydb/ydb_dummy.h b/ydb/services/ydb/ydb_dummy.h index 4c6de5c6901..7e3e74c506c 100644 --- a/ydb/services/ydb/ydb_dummy.h +++ b/ydb/services/ydb/ydb_dummy.h @@ -1,8 +1,8 @@ #pragma once -#include <library/cpp/grpc/server/grpc_server.h> +#include <library/cpp/grpc/server/grpc_server.h> #include <library/cpp/actors/core/actorsystem.h> -#include <library/cpp/monlib/counters/counters.h> +#include <library/cpp/monlib/counters/counters.h> #include <ydb/public/api/grpc/draft/dummy.grpc.pb.h> namespace NKikimr { diff --git a/ydb/services/ydb/ydb_experimental.cpp b/ydb/services/ydb/ydb_experimental.cpp index 89bd068e17f..1ca0009770d 100644 --- a/ydb/services/ydb/ydb_experimental.cpp +++ b/ydb/services/ydb/ydb_experimental.cpp @@ -8,17 +8,17 @@ namespace NKikimr { namespace NGRpcService { TGRpcYdbExperimentalService::TGRpcYdbExperimentalService(NActors::TActorSystem *system, - TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId id) + TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId id) : ActorSystem_(system) , Counters_(counters) , GRpcRequestProxyId_(id) {} -void TGRpcYdbExperimentalService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { +void TGRpcYdbExperimentalService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { CQ_ = cq; - SetupIncomingRequests(std::move(logger)); + SetupIncomingRequests(std::move(logger)); } -void TGRpcYdbExperimentalService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { +void TGRpcYdbExperimentalService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { Limiter_ = limiter; } @@ -31,7 +31,7 @@ void TGRpcYdbExperimentalService::DecRequest() { Y_ASSERT(Limiter_->GetCurrentInFlight() >= 0); } -void TGRpcYdbExperimentalService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { +void TGRpcYdbExperimentalService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { auto getCounterBlock = CreateCounterCb(Counters_, ActorSystem_); #ifdef ADD_REQUEST @@ -39,11 +39,11 @@ void TGRpcYdbExperimentalService::SetupIncomingRequests(NGrpc::TLoggerPtr logger #endif #define ADD_REQUEST(NAME, IN, OUT, ACTION) \ MakeIntrusive<TGRpcRequest<Ydb::Experimental::IN, Ydb::Experimental::OUT, TGRpcYdbExperimentalService>>(this, &Service_, CQ_, \ - [this](NGrpc::IRequestContextBase *ctx) { \ + [this](NGrpc::IRequestContextBase *ctx) { \ NGRpcService::ReportGrpcReqToMon(*ActorSystem_, ctx->GetPeer()); \ ACTION; \ }, &Ydb::Experimental::V1::ExperimentalService::AsyncService::Request ## NAME, \ - #NAME, logger, getCounterBlock("experimental", #NAME))->Run(); + #NAME, logger, getCounterBlock("experimental", #NAME))->Run(); ADD_REQUEST(ExecuteStreamQuery, ExecuteStreamQueryRequest, ExecuteStreamQueryResponse, { ActorSystem_->Send(GRpcRequestProxyId_, new TEvExperimentalStreamQueryRequest(ctx)); diff --git a/ydb/services/ydb/ydb_experimental.h b/ydb/services/ydb/ydb_experimental.h index 4c3bce329ac..c312a18cacb 100644 --- a/ydb/services/ydb/ydb_experimental.h +++ b/ydb/services/ydb/ydb_experimental.h @@ -2,7 +2,7 @@ #include <library/cpp/actors/core/actorsystem.h> -#include <library/cpp/grpc/server/grpc_server.h> +#include <library/cpp/grpc/server/grpc_server.h> #include <ydb/public/api/grpc/draft/ydb_experimental_v1.grpc.pb.h> @@ -10,25 +10,25 @@ namespace NKikimr { namespace NGRpcService { class TGRpcYdbExperimentalService - : public NGrpc::TGrpcServiceBase<Ydb::Experimental::V1::ExperimentalService> + : public NGrpc::TGrpcServiceBase<Ydb::Experimental::V1::ExperimentalService> { public: TGRpcYdbExperimentalService(NActors::TActorSystem* system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, - NActors::TActorId id); + NActors::TActorId id); - void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; - void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; + void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; + void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; bool IncRequest(); void DecRequest(); private: - void SetupIncomingRequests(NGrpc::TLoggerPtr logger); + void SetupIncomingRequests(NGrpc::TLoggerPtr logger); NActors::TActorSystem* ActorSystem_; grpc::ServerCompletionQueue* CQ_ = nullptr; TIntrusivePtr<NMonitoring::TDynamicCounters> Counters_; - NActors::TActorId GRpcRequestProxyId_; + NActors::TActorId GRpcRequestProxyId_; NGrpc::TGlobalLimiter* Limiter_ = nullptr; }; diff --git a/ydb/services/ydb/ydb_export.cpp b/ydb/services/ydb/ydb_export.cpp index 6082bc029fe..7a0933ce555 100644 --- a/ydb/services/ydb/ydb_export.cpp +++ b/ydb/services/ydb/ydb_export.cpp @@ -7,18 +7,18 @@ namespace NKikimr { namespace NGRpcService { -TGRpcYdbExportService::TGRpcYdbExportService(NActors::TActorSystem *system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId id) +TGRpcYdbExportService::TGRpcYdbExportService(NActors::TActorSystem *system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId id) : ActorSystem_(system) , Counters_(counters) , GRpcRequestProxyId_(id) { } -void TGRpcYdbExportService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { +void TGRpcYdbExportService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { CQ_ = cq; - SetupIncomingRequests(std::move(logger)); + SetupIncomingRequests(std::move(logger)); } -void TGRpcYdbExportService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { +void TGRpcYdbExportService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { Limiter_ = limiter; } @@ -31,7 +31,7 @@ void TGRpcYdbExportService::DecRequest() { Y_ASSERT(Limiter_->GetCurrentInFlight() >= 0); } -void TGRpcYdbExportService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { +void TGRpcYdbExportService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { auto getCounterBlock = CreateCounterCb(Counters_, ActorSystem_); #ifdef ADD_REQUEST @@ -39,11 +39,11 @@ void TGRpcYdbExportService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { #endif #define ADD_REQUEST(NAME, IN, OUT, ACTION) \ MakeIntrusive<TGRpcRequest<Ydb::Export::IN, Ydb::Export::OUT, TGRpcYdbExportService>>(this, &Service_, CQ_, \ - [this](NGrpc::IRequestContextBase *ctx) { \ + [this](NGrpc::IRequestContextBase *ctx) { \ NGRpcService::ReportGrpcReqToMon(*ActorSystem_, ctx->GetPeer()); \ ACTION; \ }, &Ydb::Export::V1::ExportService::AsyncService::Request ## NAME, \ - #NAME, logger, getCounterBlock("export", #NAME))->Run(); + #NAME, logger, getCounterBlock("export", #NAME))->Run(); ADD_REQUEST(ExportToYt, ExportToYtRequest, ExportToYtResponse, { ActorSystem_->Send(GRpcRequestProxyId_, new TEvExportToYtRequest(ctx)); diff --git a/ydb/services/ydb/ydb_export.h b/ydb/services/ydb/ydb_export.h index 65903c7d2a2..6049462ca22 100644 --- a/ydb/services/ydb/ydb_export.h +++ b/ydb/services/ydb/ydb_export.h @@ -1,31 +1,31 @@ #pragma once #include <library/cpp/actors/core/actorsystem.h> -#include <library/cpp/grpc/server/grpc_server.h> +#include <library/cpp/grpc/server/grpc_server.h> #include <ydb/public/api/grpc/ydb_export_v1.grpc.pb.h> namespace NKikimr { namespace NGRpcService { class TGRpcYdbExportService - : public NGrpc::TGrpcServiceBase<Ydb::Export::V1::ExportService> + : public NGrpc::TGrpcServiceBase<Ydb::Export::V1::ExportService> { public: - TGRpcYdbExportService(NActors::TActorSystem* system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId id); + TGRpcYdbExportService(NActors::TActorSystem* system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId id); - void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; - void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; + void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; + void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; bool IncRequest(); void DecRequest(); private: - void SetupIncomingRequests(NGrpc::TLoggerPtr logger); + void SetupIncomingRequests(NGrpc::TLoggerPtr logger); NActors::TActorSystem* ActorSystem_; grpc::ServerCompletionQueue* CQ_ = nullptr; TIntrusivePtr<NMonitoring::TDynamicCounters> Counters_; - NActors::TActorId GRpcRequestProxyId_; + NActors::TActorId GRpcRequestProxyId_; NGrpc::TGlobalLimiter* Limiter_ = nullptr; }; diff --git a/ydb/services/ydb/ydb_import.cpp b/ydb/services/ydb/ydb_import.cpp index 959c8602003..c27578329d4 100644 --- a/ydb/services/ydb/ydb_import.cpp +++ b/ydb/services/ydb/ydb_import.cpp @@ -13,12 +13,12 @@ TGRpcYdbImportService::TGRpcYdbImportService(NActors::TActorSystem *system, TInt , GRpcRequestProxyId_(id) { } -void TGRpcYdbImportService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { +void TGRpcYdbImportService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { CQ_ = cq; - SetupIncomingRequests(std::move(logger)); + SetupIncomingRequests(std::move(logger)); } -void TGRpcYdbImportService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { +void TGRpcYdbImportService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { Limiter_ = limiter; } @@ -31,7 +31,7 @@ void TGRpcYdbImportService::DecRequest() { Y_ASSERT(Limiter_->GetCurrentInFlight() >= 0); } -void TGRpcYdbImportService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { +void TGRpcYdbImportService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { auto getCounterBlock = CreateCounterCb(Counters_, ActorSystem_); #ifdef ADD_REQUEST @@ -39,11 +39,11 @@ void TGRpcYdbImportService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { #endif #define ADD_REQUEST(NAME, IN, OUT, ACTION) \ MakeIntrusive<TGRpcRequest<Ydb::Import::IN, Ydb::Import::OUT, TGRpcYdbImportService>>(this, &Service_, CQ_, \ - [this](NGrpc::IRequestContextBase *ctx) { \ + [this](NGrpc::IRequestContextBase *ctx) { \ NGRpcService::ReportGrpcReqToMon(*ActorSystem_, ctx->GetPeer()); \ ACTION; \ }, &Ydb::Import::V1::ImportService::AsyncService::Request ## NAME, \ - #NAME, logger, getCounterBlock("import", #NAME))->Run(); + #NAME, logger, getCounterBlock("import", #NAME))->Run(); ADD_REQUEST(ImportFromS3, ImportFromS3Request, ImportFromS3Response, { ActorSystem_->Send(GRpcRequestProxyId_, new TEvImportFromS3Request(ctx)); diff --git a/ydb/services/ydb/ydb_import.h b/ydb/services/ydb/ydb_import.h index 27814bc7963..a7db52853a5 100644 --- a/ydb/services/ydb/ydb_import.h +++ b/ydb/services/ydb/ydb_import.h @@ -1,25 +1,25 @@ #pragma once #include <library/cpp/actors/core/actorsystem.h> -#include <library/cpp/grpc/server/grpc_server.h> +#include <library/cpp/grpc/server/grpc_server.h> #include <ydb/public/api/grpc/ydb_import_v1.grpc.pb.h> namespace NKikimr { namespace NGRpcService { class TGRpcYdbImportService - : public NGrpc::TGrpcServiceBase<Ydb::Import::V1::ImportService> + : public NGrpc::TGrpcServiceBase<Ydb::Import::V1::ImportService> { public: TGRpcYdbImportService(NActors::TActorSystem* system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId id); - void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; - void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; + void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; + void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; bool IncRequest(); void DecRequest(); private: - void SetupIncomingRequests(NGrpc::TLoggerPtr logger); + void SetupIncomingRequests(NGrpc::TLoggerPtr logger); NActors::TActorSystem* ActorSystem_; grpc::ServerCompletionQueue* CQ_ = nullptr; diff --git a/ydb/services/ydb/ydb_long_tx.cpp b/ydb/services/ydb/ydb_long_tx.cpp index 12ac572190a..908c0fd332d 100644 --- a/ydb/services/ydb/ydb_long_tx.cpp +++ b/ydb/services/ydb/ydb_long_tx.cpp @@ -16,12 +16,12 @@ TGRpcYdbLongTxService::TGRpcYdbLongTxService(NActors::TActorSystem* system, , GRpcRequestProxyId_(id) {} -void TGRpcYdbLongTxService::InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) { +void TGRpcYdbLongTxService::InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) { CQ_ = cq; - SetupIncomingRequests(std::move(logger)); + SetupIncomingRequests(std::move(logger)); } -void TGRpcYdbLongTxService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { +void TGRpcYdbLongTxService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { Limiter_ = limiter; } @@ -34,7 +34,7 @@ void TGRpcYdbLongTxService::DecRequest() { Y_ASSERT(Limiter_->GetCurrentInFlight() >= 0); } -void TGRpcYdbLongTxService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { +void TGRpcYdbLongTxService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { auto getCounterBlock = CreateCounterCb(Counters_, ActorSystem_); #ifdef ADD_REQUEST @@ -42,11 +42,11 @@ void TGRpcYdbLongTxService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { #endif #define ADD_REQUEST(NAME, IN, OUT, ACTION) \ MakeIntrusive<TGRpcRequest<Ydb::LongTx::IN, Ydb::LongTx::OUT, TGRpcYdbLongTxService>>(this, &Service_, CQ_, \ - [this](NGrpc::IRequestContextBase *ctx) { \ + [this](NGrpc::IRequestContextBase *ctx) { \ NGRpcService::ReportGrpcReqToMon(*ActorSystem_, ctx->GetPeer()); \ ACTION; \ }, &Ydb::LongTx::V1::LongTxService::AsyncService::Request ## NAME, \ - #NAME, logger, getCounterBlock("long_tx", #NAME))->Run(); + #NAME, logger, getCounterBlock("long_tx", #NAME))->Run(); ADD_REQUEST(BeginTx, BeginTransactionRequest, BeginTransactionResponse, { ActorSystem_->Send(GRpcRequestProxyId_, new TEvLongTxBeginRequest(ctx)); diff --git a/ydb/services/ydb/ydb_long_tx.h b/ydb/services/ydb/ydb_long_tx.h index 35248321c54..1788da91c55 100644 --- a/ydb/services/ydb/ydb_long_tx.h +++ b/ydb/services/ydb/ydb_long_tx.h @@ -1,26 +1,26 @@ #pragma once #include <library/cpp/actors/core/actorsystem.h> -#include <library/cpp/grpc/server/grpc_server.h> +#include <library/cpp/grpc/server/grpc_server.h> #include <ydb/public/api/grpc/draft/ydb_long_tx_v1.grpc.pb.h> namespace NKikimr { namespace NGRpcService { class TGRpcYdbLongTxService - : public NGrpc::TGrpcServiceBase<Ydb::LongTx::V1::LongTxService> + : public NGrpc::TGrpcServiceBase<Ydb::LongTx::V1::LongTxService> { public: TGRpcYdbLongTxService(NActors::TActorSystem* system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId id); - void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; - void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; + void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; + void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; bool IncRequest(); void DecRequest(); private: - void SetupIncomingRequests(NGrpc::TLoggerPtr logger); + void SetupIncomingRequests(NGrpc::TLoggerPtr logger); NActors::TActorSystem* ActorSystem_; grpc::ServerCompletionQueue* CQ_ = nullptr; diff --git a/ydb/services/ydb/ydb_operation.cpp b/ydb/services/ydb/ydb_operation.cpp index 025e1152cb5..602980027cb 100644 --- a/ydb/services/ydb/ydb_operation.cpp +++ b/ydb/services/ydb/ydb_operation.cpp @@ -7,18 +7,18 @@ namespace NKikimr { namespace NGRpcService { -TGRpcOperationService::TGRpcOperationService(NActors::TActorSystem *system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId id) +TGRpcOperationService::TGRpcOperationService(NActors::TActorSystem *system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId id) : ActorSystem_(system) , Counters_(counters) , GRpcRequestProxyId_(id) { } -void TGRpcOperationService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { +void TGRpcOperationService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { CQ_ = cq; - SetupIncomingRequests(std::move(logger)); + SetupIncomingRequests(std::move(logger)); } -void TGRpcOperationService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter *limiter) { +void TGRpcOperationService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter *limiter) { Limiter_ = limiter; } @@ -31,18 +31,18 @@ void TGRpcOperationService::DecRequest() { Y_ASSERT(Limiter_->GetCurrentInFlight() >= 0); } -void TGRpcOperationService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { +void TGRpcOperationService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { auto getCounterBlock = CreateCounterCb(Counters_, ActorSystem_); #ifdef ADD_REQUEST #error ADD_REQUEST macro already defined #endif #define ADD_REQUEST(NAME, IN, OUT, ACTION) \ MakeIntrusive<TGRpcRequest<Ydb::Operations::IN, Ydb::Operations::OUT, TGRpcOperationService>>(this, &Service_, CQ_, \ - [this](NGrpc::IRequestContextBase *ctx) { \ + [this](NGrpc::IRequestContextBase *ctx) { \ NGRpcService::ReportGrpcReqToMon(*ActorSystem_, ctx->GetPeer()); \ ACTION; \ }, &Ydb::Operation::V1::OperationService::AsyncService::Request ## NAME, \ - #NAME, logger, getCounterBlock("operation", #NAME))->Run(); + #NAME, logger, getCounterBlock("operation", #NAME))->Run(); ADD_REQUEST(GetOperation, GetOperationRequest, GetOperationResponse, { ActorSystem_->Send(GRpcRequestProxyId_, new TEvGetOperationRequest(ctx)); diff --git a/ydb/services/ydb/ydb_operation.h b/ydb/services/ydb/ydb_operation.h index 4c0b0dffe1f..9f7e8646b1d 100644 --- a/ydb/services/ydb/ydb_operation.h +++ b/ydb/services/ydb/ydb_operation.h @@ -2,30 +2,30 @@ #include <library/cpp/actors/core/actorsystem.h> #include <ydb/public/api/grpc/ydb_operation_v1.grpc.pb.h> -#include <library/cpp/grpc/server/grpc_server.h> +#include <library/cpp/grpc/server/grpc_server.h> namespace NKikimr { namespace NGRpcService { class TGRpcOperationService - : public NGrpc::TGrpcServiceBase<Ydb::Operation::V1::OperationService> + : public NGrpc::TGrpcServiceBase<Ydb::Operation::V1::OperationService> { public: - TGRpcOperationService(NActors::TActorSystem* system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId id); + TGRpcOperationService(NActors::TActorSystem* system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId id); - void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; - void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; + void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; + void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; bool IncRequest(); void DecRequest(); private: - void SetupIncomingRequests(NGrpc::TLoggerPtr logger); + void SetupIncomingRequests(NGrpc::TLoggerPtr logger); NActors::TActorSystem* ActorSystem_; grpc::ServerCompletionQueue* CQ_ = nullptr; TIntrusivePtr<NMonitoring::TDynamicCounters> Counters_; - NActors::TActorId GRpcRequestProxyId_; + NActors::TActorId GRpcRequestProxyId_; NGrpc::TGlobalLimiter* Limiter_ = nullptr; }; diff --git a/ydb/services/ydb/ydb_s3_internal.cpp b/ydb/services/ydb/ydb_s3_internal.cpp index ec5dd1e778b..5dca2c2fd95 100644 --- a/ydb/services/ydb/ydb_s3_internal.cpp +++ b/ydb/services/ydb/ydb_s3_internal.cpp @@ -8,17 +8,17 @@ namespace NKikimr { namespace NGRpcService { TGRpcYdbS3InternalService::TGRpcYdbS3InternalService(NActors::TActorSystem *system, - TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId id) + TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId id) : ActorSystem_(system) , Counters_(counters) , GRpcRequestProxyId_(id) {} -void TGRpcYdbS3InternalService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { +void TGRpcYdbS3InternalService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { CQ_ = cq; - SetupIncomingRequests(std::move(logger)); + SetupIncomingRequests(std::move(logger)); } -void TGRpcYdbS3InternalService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { +void TGRpcYdbS3InternalService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { Limiter_ = limiter; } @@ -31,7 +31,7 @@ void TGRpcYdbS3InternalService::DecRequest() { Y_ASSERT(Limiter_->GetCurrentInFlight() >= 0); } -void TGRpcYdbS3InternalService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { +void TGRpcYdbS3InternalService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { auto getCounterBlock = CreateCounterCb(Counters_, ActorSystem_); #ifdef ADD_REQUEST @@ -39,11 +39,11 @@ void TGRpcYdbS3InternalService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) #endif #define ADD_REQUEST(NAME, IN, OUT, ACTION) \ MakeIntrusive<TGRpcRequest<Ydb::S3Internal::IN, Ydb::S3Internal::OUT, TGRpcYdbS3InternalService>>(this, &Service_, CQ_, \ - [this](NGrpc::IRequestContextBase *ctx) { \ + [this](NGrpc::IRequestContextBase *ctx) { \ NGRpcService::ReportGrpcReqToMon(*ActorSystem_, ctx->GetPeer()); \ ACTION; \ }, &Ydb::S3Internal::V1::S3InternalService::AsyncService::Request ## NAME, \ - #NAME, logger, getCounterBlock("s3_internal", #NAME))->Run(); + #NAME, logger, getCounterBlock("s3_internal", #NAME))->Run(); ADD_REQUEST(S3Listing, S3ListingRequest, S3ListingResponse, { ActorSystem_->Send(GRpcRequestProxyId_, new TEvS3ListingRequest(ctx)); diff --git a/ydb/services/ydb/ydb_s3_internal.h b/ydb/services/ydb/ydb_s3_internal.h index 0881a8ca942..e947a9b131f 100644 --- a/ydb/services/ydb/ydb_s3_internal.h +++ b/ydb/services/ydb/ydb_s3_internal.h @@ -1,32 +1,32 @@ #pragma once #include <library/cpp/actors/core/actorsystem.h> -#include <library/cpp/grpc/server/grpc_server.h> +#include <library/cpp/grpc/server/grpc_server.h> #include <ydb/public/api/grpc/draft/ydb_s3_internal_v1.grpc.pb.h> namespace NKikimr { namespace NGRpcService { class TGRpcYdbS3InternalService - : public NGrpc::TGrpcServiceBase<Ydb::S3Internal::V1::S3InternalService> + : public NGrpc::TGrpcServiceBase<Ydb::S3Internal::V1::S3InternalService> { public: TGRpcYdbS3InternalService(NActors::TActorSystem* system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, - NActors::TActorId id); + NActors::TActorId id); - void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; - void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; + void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; + void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; bool IncRequest(); void DecRequest(); private: - void SetupIncomingRequests(NGrpc::TLoggerPtr logger); + void SetupIncomingRequests(NGrpc::TLoggerPtr logger); NActors::TActorSystem* ActorSystem_; grpc::ServerCompletionQueue* CQ_ = nullptr; TIntrusivePtr<NMonitoring::TDynamicCounters> Counters_; - NActors::TActorId GRpcRequestProxyId_; + NActors::TActorId GRpcRequestProxyId_; NGrpc::TGlobalLimiter* Limiter_ = nullptr; }; diff --git a/ydb/services/ydb/ydb_scheme.cpp b/ydb/services/ydb/ydb_scheme.cpp index fd05edfabd2..ac339f54ab2 100644 --- a/ydb/services/ydb/ydb_scheme.cpp +++ b/ydb/services/ydb/ydb_scheme.cpp @@ -8,18 +8,18 @@ namespace NKikimr { namespace NGRpcService { -TGRpcYdbSchemeService::TGRpcYdbSchemeService(NActors::TActorSystem *system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId id) +TGRpcYdbSchemeService::TGRpcYdbSchemeService(NActors::TActorSystem *system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId id) : ActorSystem_(system) , Counters_(counters) , GRpcRequestProxyId_(id) { } -void TGRpcYdbSchemeService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { +void TGRpcYdbSchemeService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { CQ_ = cq; - SetupIncomingRequests(std::move(logger)); + SetupIncomingRequests(std::move(logger)); } -void TGRpcYdbSchemeService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { +void TGRpcYdbSchemeService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { Limiter_ = limiter; } @@ -32,7 +32,7 @@ void TGRpcYdbSchemeService::DecRequest() { Y_ASSERT(Limiter_->GetCurrentInFlight() >= 0); } -void TGRpcYdbSchemeService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { +void TGRpcYdbSchemeService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { auto getCounterBlock = CreateCounterCb(Counters_, ActorSystem_); #ifdef ADD_REQUEST diff --git a/ydb/services/ydb/ydb_scheme.h b/ydb/services/ydb/ydb_scheme.h index 177923b21dd..a510e5f5920 100644 --- a/ydb/services/ydb/ydb_scheme.h +++ b/ydb/services/ydb/ydb_scheme.h @@ -1,31 +1,31 @@ #pragma once #include <library/cpp/actors/core/actorsystem.h> -#include <library/cpp/grpc/server/grpc_server.h> +#include <library/cpp/grpc/server/grpc_server.h> #include <ydb/public/api/grpc/ydb_scheme_v1.grpc.pb.h> namespace NKikimr { namespace NGRpcService { class TGRpcYdbSchemeService - : public NGrpc::TGrpcServiceBase<Ydb::Scheme::V1::SchemeService> + : public NGrpc::TGrpcServiceBase<Ydb::Scheme::V1::SchemeService> { public: - TGRpcYdbSchemeService(NActors::TActorSystem* system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId id); + TGRpcYdbSchemeService(NActors::TActorSystem* system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId id); - void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; - void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; + void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; + void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; bool IncRequest(); void DecRequest(); private: - void SetupIncomingRequests(NGrpc::TLoggerPtr logger); + void SetupIncomingRequests(NGrpc::TLoggerPtr logger); NActors::TActorSystem* ActorSystem_; grpc::ServerCompletionQueue* CQ_ = nullptr; TIntrusivePtr<NMonitoring::TDynamicCounters> Counters_; - NActors::TActorId GRpcRequestProxyId_; + NActors::TActorId GRpcRequestProxyId_; NGrpc::TGlobalLimiter* Limiter_ = nullptr; }; diff --git a/ydb/services/ydb/ydb_scripting.cpp b/ydb/services/ydb/ydb_scripting.cpp index fb4be09a29d..3596b17fd94 100644 --- a/ydb/services/ydb/ydb_scripting.cpp +++ b/ydb/services/ydb/ydb_scripting.cpp @@ -9,17 +9,17 @@ namespace NKikimr { namespace NGRpcService { TGRpcYdbScriptingService::TGRpcYdbScriptingService(NActors::TActorSystem *system, - TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId id) + TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId id) : ActorSystem_(system) , Counters_(counters) , GRpcRequestProxyId_(id) {} -void TGRpcYdbScriptingService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { +void TGRpcYdbScriptingService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { CQ_ = cq; - SetupIncomingRequests(std::move(logger)); + SetupIncomingRequests(std::move(logger)); } -void TGRpcYdbScriptingService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { +void TGRpcYdbScriptingService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { Limiter_ = limiter; } @@ -32,7 +32,7 @@ void TGRpcYdbScriptingService::DecRequest() { Y_ASSERT(Limiter_->GetCurrentInFlight() >= 0); } -void TGRpcYdbScriptingService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { +void TGRpcYdbScriptingService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { using Ydb::Scripting::ExecuteYqlRequest; using Ydb::Scripting::ExecuteYqlResponse; using Ydb::Scripting::ExecuteYqlPartialResponse; @@ -46,11 +46,11 @@ void TGRpcYdbScriptingService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { #endif #define ADD_REQUEST(NAME, IN, OUT, ACTION) \ MakeIntrusive<TGRpcRequest<Ydb::Scripting::IN, Ydb::Scripting::OUT, TGRpcYdbScriptingService>>(this, &Service_, CQ_, \ - [this](NGrpc::IRequestContextBase *ctx) { \ + [this](NGrpc::IRequestContextBase *ctx) { \ NGRpcService::ReportGrpcReqToMon(*ActorSystem_, ctx->GetPeer()); \ ACTION; \ }, &Ydb::Scripting::V1::ScriptingService::AsyncService::Request ## NAME, \ - #NAME, logger, getCounterBlock("scripting", #NAME))->Run(); + #NAME, logger, getCounterBlock("scripting", #NAME))->Run(); ADD_REQUEST(ExecuteYql, ExecuteYqlRequest, ExecuteYqlResponse, { ActorSystem_->Send(GRpcRequestProxyId_, diff --git a/ydb/services/ydb/ydb_scripting.h b/ydb/services/ydb/ydb_scripting.h index f89f271afdd..9566cadeeea 100644 --- a/ydb/services/ydb/ydb_scripting.h +++ b/ydb/services/ydb/ydb_scripting.h @@ -1,32 +1,32 @@ #pragma once #include <library/cpp/actors/core/actorsystem.h> -#include <library/cpp/grpc/server/grpc_server.h> +#include <library/cpp/grpc/server/grpc_server.h> #include <ydb/public/api/grpc/ydb_scripting_v1.grpc.pb.h> namespace NKikimr { namespace NGRpcService { class TGRpcYdbScriptingService - : public NGrpc::TGrpcServiceBase<Ydb::Scripting::V1::ScriptingService> + : public NGrpc::TGrpcServiceBase<Ydb::Scripting::V1::ScriptingService> { public: TGRpcYdbScriptingService(NActors::TActorSystem* system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, - NActors::TActorId id); + NActors::TActorId id); - void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; - void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; + void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; + void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; bool IncRequest(); void DecRequest(); private: - void SetupIncomingRequests(NGrpc::TLoggerPtr logger); + void SetupIncomingRequests(NGrpc::TLoggerPtr logger); NActors::TActorSystem* ActorSystem_; grpc::ServerCompletionQueue* CQ_ = nullptr; TIntrusivePtr<NMonitoring::TDynamicCounters> Counters_; - NActors::TActorId GRpcRequestProxyId_; + NActors::TActorId GRpcRequestProxyId_; NGrpc::TGlobalLimiter* Limiter_ = nullptr; }; diff --git a/ydb/services/ydb/ydb_stats_ut.cpp b/ydb/services/ydb/ydb_stats_ut.cpp index f2d7fa0b14c..b40e22fc42d 100644 --- a/ydb/services/ydb/ydb_stats_ut.cpp +++ b/ydb/services/ydb/ydb_stats_ut.cpp @@ -7,7 +7,7 @@ #include <library/cpp/testing/unittest/registar.h> #include <library/cpp/testing/unittest/tests_data.h> #include <library/cpp/monlib/encode/encoder.h> -#include <library/cpp/monlib/encode/json/json.h> +#include <library/cpp/monlib/encode/json/json.h> #include <util/generic/ptr.h> #include <util/system/valgrind.h> @@ -32,7 +32,7 @@ struct TStatCounters { ui64 RetryOperationDueAborted = 0; }; -class TMetricEncoder: public NMonitoring::IMetricEncoder { +class TMetricEncoder: public NMonitoring::IMetricEncoder { public: enum class ECounterType: size_t { @@ -66,8 +66,8 @@ public: void OnCommonTime(TInstant) override { } - void OnMetricBegin(NMonitoring::EMetricType) override { } - void OnMetricEnd() override { } + void OnMetricBegin(NMonitoring::EMetricType) override { } + void OnMetricEnd() override { } void OnLabelsBegin() override { } void OnLabelsEnd() override { } @@ -178,13 +178,13 @@ public: TCountersExtractExtension(const TParams& params, IApi* api); TStatCounters Pull() { - TMetricEncoder extractor; + TMetricEncoder extractor; Api_->Accept(&extractor); return extractor.Counters; } private: - std::shared_ptr<NMonitoring::TMetricRegistry> MetricRegistry_; + std::shared_ptr<NMonitoring::TMetricRegistry> MetricRegistry_; IApi* Api_ = nullptr; }; @@ -203,10 +203,10 @@ private: }; TCountersExtractExtension::TCountersExtractExtension(const TParams& params, IApi* api) - : MetricRegistry_(new NMonitoring::TMetricRegistry()) + : MetricRegistry_(new NMonitoring::TMetricRegistry()) , Api_(api) { - api->SetMetricRegistry(MetricRegistry_.get()); + api->SetMetricRegistry(MetricRegistry_.get()); params.Extractor->Register(this); } @@ -334,14 +334,14 @@ Y_UNIT_TEST_SUITE(ClientStatsCollector) { driver.Stop(true); } - Y_UNIT_TEST(ExternalMetricRegistryByRawPtr) { - NMonitoring::TMetricRegistry sensorsRegistry; + Y_UNIT_TEST(ExternalMetricRegistryByRawPtr) { + NMonitoring::TMetricRegistry sensorsRegistry; NYdb::TKikimrWithGrpcAndRootSchema server; auto endpoint = TStringBuilder() << "localhost:" << server.GetPort(); NYdb::TDriver driver(NYdb::TDriverConfig().SetEndpoint(endpoint)); - NSolomonStatExtension::AddMetricRegistry(driver, &sensorsRegistry); + NSolomonStatExtension::AddMetricRegistry(driver, &sensorsRegistry); { NYdb::NTable::TTableClient client(driver); @@ -350,21 +350,21 @@ Y_UNIT_TEST_SUITE(ClientStatsCollector) { UNIT_ASSERT(SimpleSelect(session, "SELECT 1;").IsSuccess()); TStringStream out; - NMonitoring::IMetricEncoderPtr encoder = NMonitoring::EncoderJson(&out); + NMonitoring::IMetricEncoderPtr encoder = NMonitoring::EncoderJson(&out); sensorsRegistry.Accept(TInstant::Zero(), encoder.Get()); } driver.Stop(true); } template<template<typename...> class TPointer> - void TestExternalMetricRegistry() { - TPointer<NMonitoring::TMetricRegistry> sensorsRegistry(new NMonitoring::TMetricRegistry()); + void TestExternalMetricRegistry() { + TPointer<NMonitoring::TMetricRegistry> sensorsRegistry(new NMonitoring::TMetricRegistry()); NYdb::TKikimrWithGrpcAndRootSchema server; auto endpoint = TStringBuilder() << "localhost:" << server.GetPort(); NYdb::TDriver driver(NYdb::TDriverConfig().SetEndpoint(endpoint)); - NSolomonStatExtension::AddMetricRegistry(driver, sensorsRegistry); + NSolomonStatExtension::AddMetricRegistry(driver, sensorsRegistry); { NYdb::NTable::TTableClient client(driver); @@ -373,13 +373,13 @@ Y_UNIT_TEST_SUITE(ClientStatsCollector) { UNIT_ASSERT(SimpleSelect(session, "SELECT 1;").IsSuccess()); TStringStream out; - NMonitoring::IMetricEncoderPtr encoder = NMonitoring::EncoderJson(&out); + NMonitoring::IMetricEncoderPtr encoder = NMonitoring::EncoderJson(&out); sensorsRegistry->Accept(TInstant::Zero(), encoder.Get()); } driver.Stop(true); } - Y_UNIT_TEST(ExternalMetricRegistryStdSharedPtr) { - TestExternalMetricRegistry<std::shared_ptr>(); + Y_UNIT_TEST(ExternalMetricRegistryStdSharedPtr) { + TestExternalMetricRegistry<std::shared_ptr>(); } } diff --git a/ydb/services/ydb/ydb_table.cpp b/ydb/services/ydb/ydb_table.cpp index 63c42b6c6b4..7189614b8e8 100644 --- a/ydb/services/ydb/ydb_table.cpp +++ b/ydb/services/ydb/ydb_table.cpp @@ -7,18 +7,18 @@ namespace NKikimr { namespace NGRpcService { -TGRpcYdbTableService::TGRpcYdbTableService(NActors::TActorSystem *system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId id) +TGRpcYdbTableService::TGRpcYdbTableService(NActors::TActorSystem *system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, NActors::TActorId id) : ActorSystem_(system) , Counters_(counters) , GRpcRequestProxyId_(id) { } -void TGRpcYdbTableService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { +void TGRpcYdbTableService::InitService(grpc::ServerCompletionQueue *cq, NGrpc::TLoggerPtr logger) { CQ_ = cq; - SetupIncomingRequests(std::move(logger)); + SetupIncomingRequests(std::move(logger)); } -void TGRpcYdbTableService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { +void TGRpcYdbTableService::SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) { Limiter_ = limiter; } @@ -31,26 +31,26 @@ void TGRpcYdbTableService::DecRequest() { Y_ASSERT(Limiter_->GetCurrentInFlight() >= 0); } -void TGRpcYdbTableService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { +void TGRpcYdbTableService::SetupIncomingRequests(NGrpc::TLoggerPtr logger) { auto getCounterBlock = CreateCounterCb(Counters_, ActorSystem_); #ifdef ADD_REQUEST #error ADD_REQUEST macro already defined #endif #define ADD_REQUEST(NAME, IN, OUT, ACTION) \ MakeIntrusive<TGRpcRequest<Ydb::Table::IN, Ydb::Table::OUT, TGRpcYdbTableService>>(this, &Service_, CQ_, \ - [this](NGrpc::IRequestContextBase *ctx) { \ + [this](NGrpc::IRequestContextBase *ctx) { \ NGRpcService::ReportGrpcReqToMon(*ActorSystem_, ctx->GetPeer()); \ ACTION; \ }, &Ydb::Table::V1::TableService::AsyncService::Request ## NAME, \ - #NAME, logger, getCounterBlock("table", #NAME))->Run(); + #NAME, logger, getCounterBlock("table", #NAME))->Run(); #define ADD_BYTES_REQUEST(NAME, IN, OUT, ACTION) \ MakeIntrusive<TGRpcRequest<Ydb::Table::IN, Ydb::Table::OUT, TGRpcYdbTableService>>(this, &Service_, CQ_, \ - [this](NGrpc::IRequestContextBase *ctx) { \ + [this](NGrpc::IRequestContextBase *ctx) { \ NGRpcService::ReportGrpcReqToMon(*ActorSystem_, ctx->GetPeer()); \ ACTION; \ }, &Ydb::Table::V1::TableService::AsyncService::Request ## NAME, \ - #NAME, logger, getCounterBlock("table", #NAME))->Run(); + #NAME, logger, getCounterBlock("table", #NAME))->Run(); ADD_REQUEST(CreateSession, CreateSessionRequest, CreateSessionResponse, { ActorSystem_->Send(GRpcRequestProxyId_, new TEvCreateSessionRequest(ctx)); diff --git a/ydb/services/ydb/ydb_table.h b/ydb/services/ydb/ydb_table.h index be7d0ae24b4..77756bf023d 100644 --- a/ydb/services/ydb/ydb_table.h +++ b/ydb/services/ydb/ydb_table.h @@ -9,25 +9,25 @@ namespace NKikimr { namespace NGRpcService { class TGRpcYdbTableService - : public NGrpc::TGrpcServiceBase<Ydb::Table::V1::TableService> + : public NGrpc::TGrpcServiceBase<Ydb::Table::V1::TableService> { public: TGRpcYdbTableService(NActors::TActorSystem* system, TIntrusivePtr<NMonitoring::TDynamicCounters> counters, - NActors::TActorId id); + NActors::TActorId id); - void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; - void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; + void InitService(grpc::ServerCompletionQueue* cq, NGrpc::TLoggerPtr logger) override; + void SetGlobalLimiterHandle(NGrpc::TGlobalLimiter* limiter) override; bool IncRequest(); void DecRequest(); private: - void SetupIncomingRequests(NGrpc::TLoggerPtr logger); + void SetupIncomingRequests(NGrpc::TLoggerPtr logger); NActors::TActorSystem* ActorSystem_; grpc::ServerCompletionQueue* CQ_ = nullptr; TIntrusivePtr<NMonitoring::TDynamicCounters> Counters_; - NActors::TActorId GRpcRequestProxyId_; + NActors::TActorId GRpcRequestProxyId_; NGrpc::TGlobalLimiter* Limiter_ = nullptr; }; diff --git a/ydb/services/ydb/ydb_table_ut.cpp b/ydb/services/ydb/ydb_table_ut.cpp index 9211e8fdc27..4861c06b92e 100644 --- a/ydb/services/ydb/ydb_table_ut.cpp +++ b/ydb/services/ydb/ydb_table_ut.cpp @@ -2309,7 +2309,7 @@ R"___(<main>: Error: Transaction not found: , code: 2015 request.set_session_id(sessionId); Ydb::Table::KeepAliveResponse response; auto status = stub->KeepAlive(&context, request, &response); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); auto deferred = response.operation(); UNIT_ASSERT(deferred.ready() == true); UNIT_ASSERT_VALUES_EQUAL(deferred.status(), Ydb::StatusIds::BAD_SESSION); @@ -2367,7 +2367,7 @@ R"___(<main>: Error: Transaction not found: , code: 2015 request.set_session_id(sessionId); Ydb::Table::KeepAliveResponse response; auto status = stub->KeepAlive(&context, request, &response); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); auto deferred = response.operation(); UNIT_ASSERT(deferred.ready() == true); UNIT_ASSERT(deferred.status() == Ydb::StatusIds::BAD_SESSION); @@ -2413,7 +2413,7 @@ R"___(<main>: Error: Transaction not found: , code: 2015 request.set_session_id(sessionId); Ydb::Table::KeepAliveResponse response; auto status = stub->KeepAlive(&context, request, &response); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); auto deferred = response.operation(); UNIT_ASSERT(deferred.ready() == true); UNIT_ASSERT(deferred.status() == Ydb::StatusIds::BAD_SESSION); @@ -2459,7 +2459,7 @@ R"___(<main>: Error: Transaction not found: , code: 2015 request.set_session_id(sessionId); Ydb::Table::KeepAliveResponse response; auto status = stub->KeepAlive(&context, request, &response); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); auto deferred = response.operation(); UNIT_ASSERT(deferred.ready() == true); UNIT_ASSERT(deferred.status() == Ydb::StatusIds::BAD_SESSION); diff --git a/ydb/services/ydb/ydb_ut.cpp b/ydb/services/ydb/ydb_ut.cpp index bd14a888641..2d370630799 100644 --- a/ydb/services/ydb/ydb_ut.cpp +++ b/ydb/services/ydb/ydb_ut.cpp @@ -1745,7 +1745,7 @@ partitioning_settings { request.mutable_query()->set_yql_text("SELECT 1 as a, 'qwerty' as b, 43.5 as c UNION ALL SELECT 11 as a, 'asdfgg' as b, Null as c;"); Ydb::Table::ExecuteDataQueryResponse response; auto status = Stub_->ExecuteDataQuery(&context, request, &response); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); auto deferred = response.operation(); UNIT_ASSERT(deferred.ready() == true); UNIT_ASSERT(deferred.status() == Ydb::StatusIds::BAD_REQUEST); @@ -1784,7 +1784,7 @@ partitioning_settings { request.set_session_id(sessionId); Ydb::Table::KeepAliveResponse response; auto status = Stub_->KeepAlive(&context, request, &response); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); auto deferred = response.operation(); UNIT_ASSERT(deferred.ready() == true); UNIT_ASSERT(deferred.status() == Ydb::StatusIds::SUCCESS); @@ -1801,7 +1801,7 @@ partitioning_settings { request.mutable_tx_control()->set_commit_tx(true); Ydb::Table::ExecuteDataQueryResponse response; auto status = Stub_->ExecuteDataQuery(&context, request, &response); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); auto deferred = response.operation(); UNIT_ASSERT(deferred.ready() == true); UNIT_ASSERT(deferred.status() == Ydb::StatusIds::SUCCESS); @@ -1925,7 +1925,7 @@ partitioning_settings { request.mutable_tx_control()->set_commit_tx(true); Ydb::Table::ExecuteDataQueryResponse response; auto status = Stub_->ExecuteDataQuery(&context, request, &response); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); auto deferred = response.operation(); UNIT_ASSERT(deferred.ready() == true); UNIT_ASSERT(deferred.status() == Ydb::StatusIds::SUCCESS); @@ -2011,7 +2011,7 @@ tx_meta { Ydb::Table::ExecuteDataQueryResponse response; grpc::ClientContext context; auto status = Stub_->ExecuteDataQuery(&context, request, &response); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); auto deferred = response.operation(); UNIT_ASSERT(deferred.ready() == true); NYql::TIssues issues; @@ -2045,7 +2045,7 @@ tx_meta { Ydb::Table::ExecuteDataQueryResponse response; grpc::ClientContext context; auto status = Stub_->ExecuteDataQuery(&context, request, &response); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); auto deferred = response.operation(); UNIT_ASSERT(deferred.ready() == true); NYql::TIssues issues; @@ -2127,7 +2127,7 @@ tx_meta { request.set_session_id(sessionId); Ydb::Table::ExecuteDataQueryResponse response; auto status = Stub_->ExecuteDataQuery(&context, request, &response); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); auto deferred = response.operation(); UNIT_ASSERT(deferred.ready() == true); UNIT_ASSERT(deferred.status() == Ydb::StatusIds::SUCCESS); @@ -2180,7 +2180,7 @@ tx_meta { request.set_session_id(sessionId); Ydb::Table::ExecuteDataQueryResponse response; auto status = Stub_->ExecuteDataQuery(&context, request, &response); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); auto deferred = response.operation(); UNIT_ASSERT(deferred.ready() == true); UNIT_ASSERT(deferred.status() == Ydb::StatusIds::SUCCESS); @@ -2255,7 +2255,7 @@ tx_meta { Ydb::Table::ExecuteSchemeQueryResponse response; auto status = Stub_->ExecuteSchemeQuery(&context, request, &response); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); auto deferred = response.operation(); UNIT_ASSERT(deferred.ready() == true); @@ -2276,7 +2276,7 @@ tx_meta { Ydb::Table::ExplainDataQueryResponse response; auto status = Stub_->ExplainDataQuery(&context, request, &response); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); auto deferred = response.operation(); UNIT_ASSERT(deferred.ready() == true); @@ -2301,7 +2301,7 @@ tx_meta { Ydb::Table::ExecuteDataQueryResponse response; auto status = Stub_->ExecuteDataQuery(&context, request, &response); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); auto deferred = response.operation(); UNIT_ASSERT(deferred.ready() == true); @@ -2332,7 +2332,7 @@ tx_meta { Ydb::Table::ExecuteDataQueryResponse response; auto status = Stub_->ExecuteDataQuery(&context, request, &response); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); auto deferred = response.operation(); UNIT_ASSERT(deferred.ready() == true); @@ -2392,7 +2392,7 @@ tx_meta { Ydb::Table::ExecuteDataQueryResponse response; auto status = Stub_->ExecuteDataQuery(&context, request, &response); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); auto deferred = response.operation(); UNIT_ASSERT(deferred.ready() == true); @@ -2441,7 +2441,7 @@ tx_meta { Ydb::Table::ExecuteDataQueryResponse response; auto status = Stub_->ExecuteDataQuery(&context, request, &response); auto deferred = response.operation(); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); UNIT_ASSERT(deferred.ready() == true); } { @@ -2456,7 +2456,7 @@ tx_meta { Ydb::Table::ExecuteDataQueryResponse response; auto status = Stub_->ExecuteDataQuery(&context, request, &response); auto deferred = response.operation(); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); UNIT_ASSERT(deferred.ready() == true); UNIT_ASSERT(deferred.status() == Ydb::StatusIds::SCHEME_ERROR); NYql::TIssues issues; @@ -2609,14 +2609,14 @@ tx_meta { Ydb::Table::ExecuteDataQueryResponse response; auto status = Stub_->ExecuteDataQuery(&context, request, &response); auto deferred = response.operation(); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); UNIT_ASSERT(deferred.ready() == true); Ydb::Table::ExecuteQueryResult result; deferred.result().UnpackTo(&result); UNIT_ASSERT(result.has_query_meta()); queryId = result.query_meta().id(); - UNIT_ASSERT(!queryId.empty()); + UNIT_ASSERT(!queryId.empty()); UNIT_ASSERT(!result.query_meta().parameters_types().empty()); } @@ -2715,7 +2715,7 @@ tx_meta { request.mutable_tx_control()->set_commit_tx(true); Ydb::Table::ExecuteDataQueryResponse response; auto status = Stub_->ExecuteDataQuery(&context, request, &response); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); auto deferred = response.operation(); UNIT_ASSERT(deferred.ready() == true); UNIT_ASSERT(deferred.status() == Ydb::StatusIds::SUCCESS); @@ -2730,7 +2730,7 @@ tx_meta { request.set_yql_text("SELECT COUNT(*) FROM [Root/TheTable];"); Ydb::Table::ExplainDataQueryResponse response; auto status = Stub_->ExplainDataQuery(&context, request, &response); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); auto deferred = response.operation(); UNIT_ASSERT(deferred.ready() == true); UNIT_ASSERT(deferred.status() == Ydb::StatusIds::SUCCESS); @@ -2840,7 +2840,7 @@ tx_meta { request.mutable_tx_control()->set_commit_tx(true); Ydb::Table::ExecuteDataQueryResponse response; auto status = Stub_->ExecuteDataQuery(&context, request, &response); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); auto deferred = response.operation(); UNIT_ASSERT(deferred.ready() == true); UNIT_ASSERT(deferred.status() == Ydb::StatusIds::SUCCESS); @@ -2856,7 +2856,7 @@ tx_meta { request.mutable_tx_control()->set_commit_tx(true); Ydb::Table::ExecuteDataQueryResponse response; auto status = Stub_->ExecuteDataQuery(&context, request, &response); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); auto deferred = response.operation(); UNIT_ASSERT(deferred.ready() == true); UNIT_ASSERT(deferred.status() == Ydb::StatusIds::SUCCESS); @@ -2940,7 +2940,7 @@ tx_meta { request.mutable_tx_control()->set_commit_tx(true); Ydb::Table::ExecuteDataQueryResponse response; auto status = Stub_->ExecuteDataQuery(&context, request, &response); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); auto deferred = response.operation(); UNIT_ASSERT(deferred.ready() == true); UNIT_ASSERT(deferred.status() == Ydb::StatusIds::SUCCESS); @@ -3133,7 +3133,7 @@ tx_meta { request.set_session_id(sessionId); Ydb::Table::KeepAliveResponse response; auto status = Stub_->KeepAlive(&context, request, &response); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); auto deferred = response.operation(); UNIT_ASSERT(deferred.ready() == true); UNIT_ASSERT(deferred.status() == Ydb::StatusIds::SUCCESS); @@ -4737,7 +4737,7 @@ void ExecSchemeYql(std::shared_ptr<grpc::Channel> channel, const TString &sessio Ydb::Table::ExecuteSchemeQueryResponse response; auto status = stub->ExecuteSchemeQuery(&context, request, &response); - UNIT_ASSERT(status.ok()); + UNIT_ASSERT(status.ok()); auto deferred = response.operation(); UNIT_ASSERT(deferred.ready() == true); |