From c8ea4c941264f44f5d5cd63fb2f7de11744cc48a Mon Sep 17 00:00:00 2001 From: Nikolay Shestakov Date: Mon, 20 Jul 2026 14:11:23 +0500 Subject: Codestyle for aggents (#46019) --- AGENTS.md | 5 +- ydb/agents/CODESTYLE.md | 337 ++++++++++++++++++++++++++++++++++++ ydb/agents/GUIDE.md | 51 ++++++ ydb/agents/TESTS.md | 53 ++++++ ydb/core/kafka_proxy/AGENTS.md | 17 ++ ydb/core/persqueue/AGENTS.md | 41 +++++ ydb/core/persqueue/README.md | 13 -- ydb/services/datastreams/AGENTS.md | 19 ++ ydb/services/persqueue_v1/AGENTS.md | 16 ++ ydb/services/sqs_topic/AGENTS.md | 20 +++ 10 files changed, 557 insertions(+), 15 deletions(-) create mode 100644 ydb/agents/CODESTYLE.md create mode 100644 ydb/agents/GUIDE.md create mode 100644 ydb/agents/TESTS.md create mode 100644 ydb/core/kafka_proxy/AGENTS.md create mode 100644 ydb/core/persqueue/AGENTS.md delete mode 100644 ydb/core/persqueue/README.md create mode 100644 ydb/services/datastreams/AGENTS.md create mode 100644 ydb/services/persqueue_v1/AGENTS.md create mode 100644 ydb/services/sqs_topic/AGENTS.md diff --git a/AGENTS.md b/AGENTS.md index 130a458ab37..b02b27e0198 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -11,6 +11,9 @@ # Run specific test ./ya make --build relwithdebinfo -tA -F *test-filter* + +# Run tests repeatedly (e.g. to catch flakes) +./ya make --build relwithdebinfo -tA -F *test-filter* --test-retries N ``` - Tests include build @@ -21,5 +24,3 @@ ## C++ - Use C++20 or earlier - - diff --git a/ydb/agents/CODESTYLE.md b/ydb/agents/CODESTYLE.md new file mode 100644 index 00000000000..ca8b67db35f --- /dev/null +++ b/ydb/agents/CODESTYLE.md @@ -0,0 +1,337 @@ +# YDB Coding Style + +Follow the coding style of the file you are modifying. If the style is unclear +(e.g. a new file), apply the rules below. + +## Names + +A name should reflect the data, type, or action it represents. Use only common +abbreviations. Single-letter names (`i`, `j`, `k`) are allowed only for loop +counters and iterators. + +Structs follow the same rules as classes unless stated otherwise. + +### Variables and identifiers + +- Local and global variables: lowercase. +- Functions and class methods: PascalCase (leading capital letter), no suffix. +- Function arguments, lambda captures, and function-pointer variables: lowercase. +- Class data members: PascalCase; private and protected members end with `_`, + public members have no suffix. +- Class and typedef names: prefix `T` + PascalCase (e.g. `TVector`). +- Virtual interfaces: prefix `I` + PascalCase. A virtual interface has at least + one pure virtual method (including inherited) and no data members (including + inherited). +- Global constants and macros: `UPPER_CASE` with underscores between tokens. +- Compound names: capitalize the first letter of each token without underscores + (e.g. `GetQueueSize`). In `UPPER_CASE` constants, separate tokens with `_`. +- Do not start a name with `_`. +- Do not use Hungarian notation. + +```cpp +class TClass { +public: + int Size; + int GetSize() { + return Size; + } +}; + +TClass object; +int GetValue(); +void SetValue(int val); +``` + +**Interop exception**: when interfacing with external code, add required methods +or free functions even if they break local naming (e.g. STL iterator protocol +`.begin()` / `.end()`, buffer access `.data()` / `.size()`, external `swap()`). + +### Macros + +If you must use a macro, make it unique (e.g. reflect the directory path). +Public API macros use the `Y_` prefix (see `util/system/compiler.h`). + +### Enumerations + +- Global enums: prefix `E` + PascalCase (same rules as classes). +- Global enum members: `UPPER_CASE`, prefixed by enum initials + (e.g. `EFetchType` → `FT_SIMPLE`). +- Class enum members: same rules as other class members (PascalCase). +- Unnamed enums are allowed only as class members. +- `enum class` follows the same rules as class enums. + +```cpp +enum EFetchType { + FT_SIMPLE, + FT_RELFORM_DEBUG, +}; + +class TIndicator { +public: + enum EStatus { + Created, + Running, + Finished + }; +}; + +enum class EStatus { + Created, + Running, + Finished +}; +``` + +Do not hand-roll enum ↔ `TString` conversion. Use `GENERATE_ENUM_SERIALIZATION` +or `GENERATE_ENUM_SERIALIZATION_WITH_HEADER` in `ya.make` (see +`tools/enum_parser/enum_serialization_runtime/README.md`). + +## Formatting + +### Indentation and whitespace + +- Indent with **4 spaces**. Never use tab characters for indentation. +- No trailing whitespace at end of line. +- Files must end with a single newline. + +### Braces + +Use [1TBS / K&R style](https://en.wikipedia.org/wiki/Indentation_style#Variant:_1TBS) +for block statements. Pick one function-brace style per file and stay consistent: + +```cpp +if (something) { + One(); + Two(); +} else { + Three(); +} + +for (int i = 0; i < N; ++i) { + // ... +} + +// Either style is OK within a file: +void Func1(int a, int b) { +} + +void Func2(int a, int b) +{ +} +``` + +Multi-line conditions break before the opening brace: + +```cpp +if (a && b && c && + d && e) +{ + Op(); +} +``` + +Even one-line statement bodies start on a new line and use braces: + +```cpp +if (something) { + A(); +} +``` + +Do not write empty loop bodies (`for (...);` or `for (...) ;`). + +Do not put multiple statements on one line. + +Leave blank lines between logical blocks of code. + +### Operators and spacing + +- Binary operators and assignment: spaces on both sides (`a = b`, `x += 3`). +- Unary operators and member access: no extra spaces. +- `noexcept` attaches to the declaration: `void F() noexcept { }`. +- Pointer and reference markers attach to the **type**: `const T& value`, `int* p`. +- No space between function name and `(`: `Func(a, b, c)`. +- Space between keyword and `(`: `if (`, `for (`, `while (`. +- No spaces inside `()`: `(a + b)`. +- Template angle brackets without spaces: `TVector>`. + +### Lambdas + +Single-line lambdas are allowed only at the point of use and only if they do not +violate other rules (one statement per line, braces for bodies): + +```cpp +Sort(a.begin(), a.end(), [](int x, int y) { return x < y; }); // OK + +auto cmp = [](int x, int y) { + return x < y; +}; +Sort(a.begin(), a.end(), cmp); +``` + +## Variables and classes + +### Variable declarations + +Prefer one declaration per line. Multiple variables of the same type on one line +are acceptable. Do not mix arrays, pointers, references, and scalars in one +declaration. Do not break a declaration across lines. + +```cpp +int level; // preferred +int size; + +int level, size; // OK + +int level, // not OK: line break + size; + +int level, array[16], *p; // not OK: mixed kinds +``` + +### Class layout + +- `struct` may contain only public members; omit `public:`. + If it has anything beyond members plus ctor/dtor, prefer `class`. +- Access specifiers align with the class declaration column. Always declare + `private:` / `protected:` explicitly, including the first private section. +- Do not mix data members and methods in the same access section; repeat the + access specifier to separate them. Use the minimum number of access sections. +- Within one access section: constructors → destructor → overridden operators → + other methods. +- Public methods precede `protected` / `private` methods. +- Data members at the beginning or end of the class; nested types may precede + data members. +- Prefer explicit default member initializers (`= nullptr`, `= 0`) over `= {}` + when possible; match the style already used in the file. +- Start `template` on its own line. + +```cpp +class TClass { +public: + TClass(); + ~TClass(); + +private: + int Member_ = 0; + int* OtherMember_ = nullptr; +}; +``` + +### Constructors + +```cpp +TClass::TClass() + : FieldA(1) + , FieldB("value") + , FieldC(true) +{ +} +``` + +Use either in-class initializers or constructor initializer lists consistently — +do not mix both styles for the same members. + +## Namespaces + +Namespace names start with `N` + PascalCase. Closing brace gets a comment with +the namespace name (`// namespace` for anonymous namespaces). Fix incorrect +comments when editing. + +```cpp +namespace NStl { + namespace NPrivate { + } // namespace NPrivate + + class TVector { + }; +} // namespace NStl + +namespace { +} // namespace +``` + +## Includes + +Remove unnecessary `#include` directives in both `.cpp` and `.h` files. + +Header files must compile standalone. For unknown types in a header: +- standard types → include the minimal standard header (``, ``); +- class/struct/enum used by pointer or reference → forward-declare in the header; +- otherwise → include the declaring header. + +Do not use `using namespace` in headers except for standard-library literals. + +Use `#pragma once` for include guards. + +### Include order + +Break includes into groups (less general before more general). Separate groups +with a blank line. Sort alphabetically within each group. + +1. Paired header (`.cpp` only): `#include "same_name.h"` first if present. +2. Local module headers in quotes: `#include "..."`. +3. `ydb/` +4. `yql/` +5. `library/` +6. `util/` +7. Other external includes (e.g. `contrib/`, `yt/`), excluding the standard library. +8. Standard library: `#include `, `#include `, etc. + +Non-local paths use angle brackets. + +```cpp +#include "sqs.h" +#include "ymq.h" + +#include +#include + +#include + +#include + +#include + +#include +``` + +## Comments + +Comments explain non-obvious code. Do not comment out dead code — delete it and +rely on version control. + +Single-line `//` comments: one space after `//`. + +```cpp +// Single comment + +void Function() { // trailing comment +} +``` + +Write comments in English (preferred) or Russian (UTF-8). Doxygen-style comments +are welcome. + +TODO format: + +```cpp +// TODO(username): fix me later +// TODO(PROJECT-1234): fix me later +``` + +## Files and preprocessor + +- File names: lowercase only. C++ extensions: `.cpp`, `.h`. +- Preprocessor directives use the same 4-space indent as code; `#` moves with + the indent level. +- Mid-file `#ifdef` starts at column 0 only when it wraps a top-level block; + nested conditions indent with code. +- Avoid preprocessor when templates, `if constexpr`, or virtual functions suffice. + +## C++ features + +- Use `nullptr`, not `0` or `NULL`. +- Prefer `using` over `typedef`. +- In derived classes use `override` without repeating `virtual`. +- Use trailing return type syntax (`auto f() -> T`) only when necessary. diff --git a/ydb/agents/GUIDE.md b/ydb/agents/GUIDE.md new file mode 100644 index 00000000000..481661d373a --- /dev/null +++ b/ydb/agents/GUIDE.md @@ -0,0 +1,51 @@ +# YDB Agent Guide + +Detailed instructions for AI agents working in the YDB monorepo. +Quick reference: root [`AGENTS.md`](../../AGENTS.md). + +## Architecture boundaries + +Respect layer boundaries when adding dependencies (`PEERDIR` in `ya.make`): + +- `ydb/public/` — code for external consumers (SDK, CLI). +- `ydb/library/` — internal shared YDB libraries. +- `ydb/core/` — server internals; must not be depended on by CLI or public SDK. + +## Nested agent instructions + +Read the nearest `AGENTS.md` when working in that tree. +Add area-specific rules in a local `AGENTS.md` rather than growing this file. + +## Build & Test + +Run `./ya` from the repository root. See [`BUILD.md`](../../BUILD.md) and +[Yatool docs](https://ydb.tech/docs/en/development/build-ya). + +```bash +./ya make --build relwithdebinfo +./ya make ydb/apps/ydbd --build relwithdebinfo +./ya make ydb/apps/ydb --build relwithdebinfo +./ya make --build relwithdebinfo -tA +./ya make --build relwithdebinfo -tA -F *test-filter* +./ya make --build relwithdebinfo -tA -F *test-filter* --test-retries N +``` + +- Prefer `--build relwithdebinfo`. No `-j`. No force rebuild (`-r`, `-R`, …). +- Use `2>&1 | tail` for test output. +- Build and test the smallest relevant folder. + +Test types and frameworks: [`TESTS.md`](TESTS.md). + +## Languages + +C++20 or earlier. Style: [`CODESTYLE.md`](CODESTYLE.md). Tests: [`TESTS.md`](TESTS.md). +Python via `ya`, packages from `contrib/python`. + +## Agent workflow + +- Make the smallest correct change; do not edit `contrib/` or `vendor/` unless required. +- Match surrounding code style ([`CODESTYLE.md`](CODESTYLE.md)). +- Search for existing code; check for a local `AGENTS.md`. +- For non-trivial changes, check [`CONTRIBUTING.md`](../../CONTRIBUTING.md). +- Run `./ya make --build relwithdebinfo -tA ` for the area you changed. +- Do not create commits or push unless explicitly asked. diff --git a/ydb/agents/TESTS.md b/ydb/agents/TESTS.md new file mode 100644 index 00000000000..f9cf5533f9e --- /dev/null +++ b/ydb/agents/TESTS.md @@ -0,0 +1,53 @@ +# YDB Tests + +Follow the testing style of the module you are modifying. For new C++ unit tests, +prefer **UNITTEST** over **GTEST**. + +## C++ unit tests + +- Place tests in a `ut/` subdirectory next to the code under test. +- Declare the test module in `ya.make` with `UNITTEST_FOR(...)` (or `UNITTEST()` + for standalone test targets). +- Register C++ test sources in `SRCS()`. +- Write tests with `Y_UNIT_TEST_SUITE` / `Y_UNIT_TEST` from + `library/cpp/testing/unittest/registar.h`. + +Use `GTEST()` only when the surrounding code already does (e.g. some +`ydb/public/sdk/cpp/tests/` targets). Do not introduce new GTest-based tests +where UNITTEST is used in the same area. + +## Python tests + +- Use the `unittest` framework (`unittest.IsolatedAsyncioTestCase` for async code). +- Register Python test files in `TEST_SRCS()` in the module's `ya.make`. +- Third-party packages come from `contrib/python`. +- See [`ydb/tools/mnc/AGENTS.md`](../tools/mnc/AGENTS.md) for MNC-specific + Python test conventions. + +## Functional tests + +End-to-end tests that run against a YDB cluster live under `ydb/tests/functional/`. + +- Declare targets with `PY3TEST()` in `ya.make`. +- List test files in `TEST_SRCS()` (paths are relative to the repository root). +- Follow existing harness patterns in the same area (`INCLUDE(...harness_dep.inc)`, + `ENV(...)`, suite `.inc` files). + +## Compatibility tests + +Upgrade/downgrade tests live under `ydb/tests/compatibility/`. + +- Place each test module in the folder matching the feature area. +- Use existing fixtures and follow the layout described in + [`ydb/tests/compatibility/README.md`](../tests/compatibility/README.md). + +## Running tests + +```bash +./ya make --build relwithdebinfo -tA +./ya make --build relwithdebinfo -tA -F *test-filter* +``` + +CLI tests (`ya make -tA`) run only on Linux. + +Build rules: [`GUIDE.md`](GUIDE.md). diff --git a/ydb/core/kafka_proxy/AGENTS.md b/ydb/core/kafka_proxy/AGENTS.md new file mode 100644 index 00000000000..ed8ef943740 --- /dev/null +++ b/ydb/core/kafka_proxy/AGENTS.md @@ -0,0 +1,17 @@ +# Kafka proxy + +**Kafka-compatible API on top of YDB Topics**. +Accepts **Kafka wire-protocol** connections (`kafka_connection.cpp`). +Core: [`ydb/core/persqueue/AGENTS.md`](../persqueue/AGENTS.md). + +## Layout + +* Root — connections, message codecs (`kafka_messages.*`), transactions. +* `actors/` — produce, fetch, metadata, offsets, consumer groups, SASL, … + +Depends on `persqueue/public/`, not `pqtablet/` / `pqrb/` internals. + +Tests: `./ya make --build relwithdebinfo -tA ydb/core/kafka_proxy` + +Style/workflow: [`agents/CODESTYLE.md`](../../agents/CODESTYLE.md) · +[`agents/GUIDE.md`](../../agents/GUIDE.md) · [`agents/TESTS.md`](../../agents/TESTS.md) diff --git a/ydb/core/persqueue/AGENTS.md b/ydb/core/persqueue/AGENTS.md new file mode 100644 index 00000000000..07d3bd29b74 --- /dev/null +++ b/ydb/core/persqueue/AGENTS.md @@ -0,0 +1,41 @@ +# YDB Topics + +Core implementation of YDB topics (persistent queues). + +## Layout + +One root directory per tablet or service, plus `common/`, `public/`, `events/`. +In `public/` and `common/`, nested dirs by **purpose**; in `pqrb/` and +`pqtablet/` — by **sub-actor or component**. + +* **`public/`** — APIs for code outside persqueue (`schema/`, `fetcher/`, …). +* **`common/`** — shared tablet internals only. +* **`events/`** — internal events and protos. +* **`pqrb/`** — whole-topic tablet (balancing, stats, autopartitioning). +* **`pqtablet/`** — one or more partitions (reads, writes, batching, …). +* **`writer/`**, **`dread_cache_service/`**, **`deferred_publish/`** — writer, + direct-read cache, deferred publish. + +Protocol layers (each has its own `AGENTS.md`): + +* [`persqueue_v1`](../../services/persqueue_v1/AGENTS.md) · + [`kafka_proxy`](../kafka_proxy/AGENTS.md) · + [`datastreams`](../../services/datastreams/AGENTS.md) · + [`sqs_topic`](../../services/sqs_topic/AGENTS.md) + +## Guidelines + +* External code uses `public/` (and `events/` where needed), not `pqtablet/` / + `pqrb/` internals. Layering: `public` → tablets. +* In `pqrb` / `pqtablet`: batch when persisting; minimize persists and + inter-actor messages; extract large logic into separate actors or classes. +* `*_fwd.h` for forward declarations; `.pb.h` only when needed. + Config helpers: [`public/config.h`](public/config.h). +* Put method implementations in `.cpp` files, not in headers — except + template functions and template classes. + +## Tests + +`./ya make --build relwithdebinfo -tA ydb/core/persqueue` (or a narrower `ut/`). +Monorepo rules: [`agents/CODESTYLE.md`](../../agents/CODESTYLE.md) · +[`agents/GUIDE.md`](../../agents/GUIDE.md) · [`agents/TESTS.md`](../../agents/TESTS.md). diff --git a/ydb/core/persqueue/README.md b/ydb/core/persqueue/README.md deleted file mode 100644 index b44af7fa536..00000000000 --- a/ydb/core/persqueue/README.md +++ /dev/null @@ -1,13 +0,0 @@ -# YDB Topics - -This directory contains the implementation of YDB topics. - -* public - these are utilitarian methods and actors that are supposed to be worked with outside this directory. -* common - common utilitarian methods and actors that are a closed part of the implementation. Their use is acceptable in the implementation of tablets inside persqueue. -* pqrb is a tablet that controls the topic as a whole. Implements the following functions: - * balancing of reading sessions (partitioning by readers); - * aggregation of statistics for all partitions; - * coordination of topic autopartitioning; - * and others. -* pqtablet - a tablet responsible for the operation of one or more partitions (depending on the cluster configuration). -* dread_cache_service is a service for transmitting topic messages during direct reading. diff --git a/ydb/services/datastreams/AGENTS.md b/ydb/services/datastreams/AGENTS.md new file mode 100644 index 00000000000..9e89e20a9e7 --- /dev/null +++ b/ydb/services/datastreams/AGENTS.md @@ -0,0 +1,19 @@ +# Data Streams (Kinesis) + +**Kinesis-compatible API on top of YDB Topics**. +HTTP entry: [`http_proxy`](../../core/http_proxy/) (`datastreams.cpp`) → this service. +Core: [`ydb/core/persqueue/AGENTS.md`](../../core/persqueue/AGENTS.md). + +## Layout + +* Root — `datastreams_proxy.cpp`, `grpc_service.cpp`, shard iterators, record writers. +* `codes/` — error codes. +* `ut/` — unit tests. + +Depends on `persqueue/public/`, not `pqtablet/` / `pqrb/` internals. + +Tests: `./ya make --build relwithdebinfo -tA ydb/services/datastreams` +HTTP integration: `ydb/core/http_proxy/ut/kinesis_ut.cpp` + +Style/workflow: [`agents/CODESTYLE.md`](../../agents/CODESTYLE.md) · +[`agents/GUIDE.md`](../../agents/GUIDE.md) · [`agents/TESTS.md`](../../agents/TESTS.md) diff --git a/ydb/services/persqueue_v1/AGENTS.md b/ydb/services/persqueue_v1/AGENTS.md new file mode 100644 index 00000000000..2cfe6034ebb --- /dev/null +++ b/ydb/services/persqueue_v1/AGENTS.md @@ -0,0 +1,16 @@ +# PersQueue v1 + +gRPC layer for **Topic API** and **PersQueue v1 (PQv1)**. +Core: [`ydb/core/persqueue/AGENTS.md`](../../core/persqueue/AGENTS.md). + +## Layout + +* Root — entry points (`topic.cpp`, `grpc_pq_*.cpp`, `persqueue.cpp`, init). +* `actors/` — read, write, schema, PQv1 handlers. + +Depends on `persqueue/public/` and `events/`, not `pqtablet/` / `pqrb/` internals. + +Tests: `./ya make --build relwithdebinfo -tA ydb/services/persqueue_v1` + +Style/workflow: [`agents/CODESTYLE.md`](../../agents/CODESTYLE.md) · +[`agents/GUIDE.md`](../../agents/GUIDE.md) · [`agents/TESTS.md`](../../agents/TESTS.md) diff --git a/ydb/services/sqs_topic/AGENTS.md b/ydb/services/sqs_topic/AGENTS.md new file mode 100644 index 00000000000..9c567644699 --- /dev/null +++ b/ydb/services/sqs_topic/AGENTS.md @@ -0,0 +1,20 @@ +# SQS over Topics + +**SQS API on top of YDB Topics**. +HTTP entry: [`http_proxy`](../../core/http_proxy/) (`sqs.cpp`) → this service. +Core: [`ydb/core/persqueue/AGENTS.md`](../../core/persqueue/AGENTS.md). + +## Layout + +* Root — API handlers and `sqs_topic_proxy.cpp`. +* `queue_url/` — URL parsing (`arn`, `consumer`, `holder/`). +* `protos/` — SQS protos (e.g. receipt). +* `ut/` — unit tests. + +Depends on `persqueue/public/`, not `pqtablet/` / `pqrb/` internals. + +Tests: `./ya make --build relwithdebinfo -tA ydb/services/sqs_topic` +Functional: `ydb/tests/functional/sqs/` + +Style/workflow: [`agents/CODESTYLE.md`](../../agents/CODESTYLE.md) · +[`agents/GUIDE.md`](../../agents/GUIDE.md) · [`agents/TESTS.md`](../../agents/TESTS.md) -- cgit v1.3