summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
-rw-r--r--AGENTS.md5
-rw-r--r--ydb/agents/CODESTYLE.md337
-rw-r--r--ydb/agents/GUIDE.md51
-rw-r--r--ydb/agents/TESTS.md53
-rw-r--r--ydb/core/kafka_proxy/AGENTS.md17
-rw-r--r--ydb/core/persqueue/AGENTS.md41
-rw-r--r--ydb/core/persqueue/README.md13
-rw-r--r--ydb/services/datastreams/AGENTS.md19
-rw-r--r--ydb/services/persqueue_v1/AGENTS.md16
-rw-r--r--ydb/services/sqs_topic/AGENTS.md20
10 files changed, 557 insertions, 15 deletions
diff --git a/AGENTS.md b/AGENTS.md
index 130a458ab37..b02b27e0198 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -11,6 +11,9 @@
# Run specific test
./ya make --build relwithdebinfo -tA <folder> -F *test-filter*
+
+# Run tests repeatedly (e.g. to catch flakes)
+./ya make --build relwithdebinfo -tA <folder> -F *test-filter* --test-retries N
```
- Tests include build
@@ -21,5 +24,3 @@
## C++
- Use C++20 or earlier
-
-
diff --git a/ydb/agents/CODESTYLE.md b/ydb/agents/CODESTYLE.md
new file mode 100644
index 00000000000..ca8b67db35f
--- /dev/null
+++ b/ydb/agents/CODESTYLE.md
@@ -0,0 +1,337 @@
+# YDB Coding Style
+
+Follow the coding style of the file you are modifying. If the style is unclear
+(e.g. a new file), apply the rules below.
+
+## Names
+
+A name should reflect the data, type, or action it represents. Use only common
+abbreviations. Single-letter names (`i`, `j`, `k`) are allowed only for loop
+counters and iterators.
+
+Structs follow the same rules as classes unless stated otherwise.
+
+### Variables and identifiers
+
+- Local and global variables: lowercase.
+- Functions and class methods: PascalCase (leading capital letter), no suffix.
+- Function arguments, lambda captures, and function-pointer variables: lowercase.
+- Class data members: PascalCase; private and protected members end with `_`,
+ public members have no suffix.
+- Class and typedef names: prefix `T` + PascalCase (e.g. `TVector`).
+- Virtual interfaces: prefix `I` + PascalCase. A virtual interface has at least
+ one pure virtual method (including inherited) and no data members (including
+ inherited).
+- Global constants and macros: `UPPER_CASE` with underscores between tokens.
+- Compound names: capitalize the first letter of each token without underscores
+ (e.g. `GetQueueSize`). In `UPPER_CASE` constants, separate tokens with `_`.
+- Do not start a name with `_`.
+- Do not use Hungarian notation.
+
+```cpp
+class TClass {
+public:
+ int Size;
+ int GetSize() {
+ return Size;
+ }
+};
+
+TClass object;
+int GetValue();
+void SetValue(int val);
+```
+
+**Interop exception**: when interfacing with external code, add required methods
+or free functions even if they break local naming (e.g. STL iterator protocol
+`.begin()` / `.end()`, buffer access `.data()` / `.size()`, external `swap()`).
+
+### Macros
+
+If you must use a macro, make it unique (e.g. reflect the directory path).
+Public API macros use the `Y_` prefix (see `util/system/compiler.h`).
+
+### Enumerations
+
+- Global enums: prefix `E` + PascalCase (same rules as classes).
+- Global enum members: `UPPER_CASE`, prefixed by enum initials
+ (e.g. `EFetchType` → `FT_SIMPLE`).
+- Class enum members: same rules as other class members (PascalCase).
+- Unnamed enums are allowed only as class members.
+- `enum class` follows the same rules as class enums.
+
+```cpp
+enum EFetchType {
+ FT_SIMPLE,
+ FT_RELFORM_DEBUG,
+};
+
+class TIndicator {
+public:
+ enum EStatus {
+ Created,
+ Running,
+ Finished
+ };
+};
+
+enum class EStatus {
+ Created,
+ Running,
+ Finished
+};
+```
+
+Do not hand-roll enum ↔ `TString` conversion. Use `GENERATE_ENUM_SERIALIZATION`
+or `GENERATE_ENUM_SERIALIZATION_WITH_HEADER` in `ya.make` (see
+`tools/enum_parser/enum_serialization_runtime/README.md`).
+
+## Formatting
+
+### Indentation and whitespace
+
+- Indent with **4 spaces**. Never use tab characters for indentation.
+- No trailing whitespace at end of line.
+- Files must end with a single newline.
+
+### Braces
+
+Use [1TBS / K&R style](https://en.wikipedia.org/wiki/Indentation_style#Variant:_1TBS)
+for block statements. Pick one function-brace style per file and stay consistent:
+
+```cpp
+if (something) {
+ One();
+ Two();
+} else {
+ Three();
+}
+
+for (int i = 0; i < N; ++i) {
+ // ...
+}
+
+// Either style is OK within a file:
+void Func1(int a, int b) {
+}
+
+void Func2(int a, int b)
+{
+}
+```
+
+Multi-line conditions break before the opening brace:
+
+```cpp
+if (a && b && c &&
+ d && e)
+{
+ Op();
+}
+```
+
+Even one-line statement bodies start on a new line and use braces:
+
+```cpp
+if (something) {
+ A();
+}
+```
+
+Do not write empty loop bodies (`for (...);` or `for (...) ;`).
+
+Do not put multiple statements on one line.
+
+Leave blank lines between logical blocks of code.
+
+### Operators and spacing
+
+- Binary operators and assignment: spaces on both sides (`a = b`, `x += 3`).
+- Unary operators and member access: no extra spaces.
+- `noexcept` attaches to the declaration: `void F() noexcept { }`.
+- Pointer and reference markers attach to the **type**: `const T& value`, `int* p`.
+- No space between function name and `(`: `Func(a, b, c)`.
+- Space between keyword and `(`: `if (`, `for (`, `while (`.
+- No spaces inside `()`: `(a + b)`.
+- Template angle brackets without spaces: `TVector<TVector<int>>`.
+
+### Lambdas
+
+Single-line lambdas are allowed only at the point of use and only if they do not
+violate other rules (one statement per line, braces for bodies):
+
+```cpp
+Sort(a.begin(), a.end(), [](int x, int y) { return x < y; }); // OK
+
+auto cmp = [](int x, int y) {
+ return x < y;
+};
+Sort(a.begin(), a.end(), cmp);
+```
+
+## Variables and classes
+
+### Variable declarations
+
+Prefer one declaration per line. Multiple variables of the same type on one line
+are acceptable. Do not mix arrays, pointers, references, and scalars in one
+declaration. Do not break a declaration across lines.
+
+```cpp
+int level; // preferred
+int size;
+
+int level, size; // OK
+
+int level, // not OK: line break
+ size;
+
+int level, array[16], *p; // not OK: mixed kinds
+```
+
+### Class layout
+
+- `struct` may contain only public members; omit `public:`.
+ If it has anything beyond members plus ctor/dtor, prefer `class`.
+- Access specifiers align with the class declaration column. Always declare
+ `private:` / `protected:` explicitly, including the first private section.
+- Do not mix data members and methods in the same access section; repeat the
+ access specifier to separate them. Use the minimum number of access sections.
+- Within one access section: constructors → destructor → overridden operators →
+ other methods.
+- Public methods precede `protected` / `private` methods.
+- Data members at the beginning or end of the class; nested types may precede
+ data members.
+- Prefer explicit default member initializers (`= nullptr`, `= 0`) over `= {}`
+ when possible; match the style already used in the file.
+- Start `template` on its own line.
+
+```cpp
+class TClass {
+public:
+ TClass();
+ ~TClass();
+
+private:
+ int Member_ = 0;
+ int* OtherMember_ = nullptr;
+};
+```
+
+### Constructors
+
+```cpp
+TClass::TClass()
+ : FieldA(1)
+ , FieldB("value")
+ , FieldC(true)
+{
+}
+```
+
+Use either in-class initializers or constructor initializer lists consistently —
+do not mix both styles for the same members.
+
+## Namespaces
+
+Namespace names start with `N` + PascalCase. Closing brace gets a comment with
+the namespace name (`// namespace` for anonymous namespaces). Fix incorrect
+comments when editing.
+
+```cpp
+namespace NStl {
+ namespace NPrivate {
+ } // namespace NPrivate
+
+ class TVector {
+ };
+} // namespace NStl
+
+namespace {
+} // namespace
+```
+
+## Includes
+
+Remove unnecessary `#include` directives in both `.cpp` and `.h` files.
+
+Header files must compile standalone. For unknown types in a header:
+- standard types → include the minimal standard header (`<cstddef>`, `<cstdio>`);
+- class/struct/enum used by pointer or reference → forward-declare in the header;
+- otherwise → include the declaring header.
+
+Do not use `using namespace` in headers except for standard-library literals.
+
+Use `#pragma once` for include guards.
+
+### Include order
+
+Break includes into groups (less general before more general). Separate groups
+with a blank line. Sort alphabetically within each group.
+
+1. Paired header (`.cpp` only): `#include "same_name.h"` first if present.
+2. Local module headers in quotes: `#include "..."`.
+3. `ydb/`
+4. `yql/`
+5. `library/`
+6. `util/`
+7. Other external includes (e.g. `contrib/`, `yt/`), excluding the standard library.
+8. Standard library: `#include <vector>`, `#include <cstddef>`, etc.
+
+Non-local paths use angle brackets.
+
+```cpp
+#include "sqs.h"
+#include "ymq.h"
+
+#include <ydb/core/base/path.h>
+#include <ydb/library/actors/core/actor_bootstrapped.h>
+
+#include <yql/essentials/public/issue/yql_issue_message.h>
+
+#include <library/cpp/scheme/scheme.h>
+
+#include <util/string/cast.h>
+
+#include <vector>
+```
+
+## Comments
+
+Comments explain non-obvious code. Do not comment out dead code — delete it and
+rely on version control.
+
+Single-line `//` comments: one space after `//`.
+
+```cpp
+// Single comment
+
+void Function() { // trailing comment
+}
+```
+
+Write comments in English (preferred) or Russian (UTF-8). Doxygen-style comments
+are welcome.
+
+TODO format:
+
+```cpp
+// TODO(username): fix me later
+// TODO(PROJECT-1234): fix me later
+```
+
+## Files and preprocessor
+
+- File names: lowercase only. C++ extensions: `.cpp`, `.h`.
+- Preprocessor directives use the same 4-space indent as code; `#` moves with
+ the indent level.
+- Mid-file `#ifdef` starts at column 0 only when it wraps a top-level block;
+ nested conditions indent with code.
+- Avoid preprocessor when templates, `if constexpr`, or virtual functions suffice.
+
+## C++ features
+
+- Use `nullptr`, not `0` or `NULL`.
+- Prefer `using` over `typedef`.
+- In derived classes use `override` without repeating `virtual`.
+- Use trailing return type syntax (`auto f() -> T`) only when necessary.
diff --git a/ydb/agents/GUIDE.md b/ydb/agents/GUIDE.md
new file mode 100644
index 00000000000..481661d373a
--- /dev/null
+++ b/ydb/agents/GUIDE.md
@@ -0,0 +1,51 @@
+# YDB Agent Guide
+
+Detailed instructions for AI agents working in the YDB monorepo.
+Quick reference: root [`AGENTS.md`](../../AGENTS.md).
+
+## Architecture boundaries
+
+Respect layer boundaries when adding dependencies (`PEERDIR` in `ya.make`):
+
+- `ydb/public/` — code for external consumers (SDK, CLI).
+- `ydb/library/` — internal shared YDB libraries.
+- `ydb/core/` — server internals; must not be depended on by CLI or public SDK.
+
+## Nested agent instructions
+
+Read the nearest `AGENTS.md` when working in that tree.
+Add area-specific rules in a local `AGENTS.md` rather than growing this file.
+
+## Build & Test
+
+Run `./ya` from the repository root. See [`BUILD.md`](../../BUILD.md) and
+[Yatool docs](https://ydb.tech/docs/en/development/build-ya).
+
+```bash
+./ya make --build relwithdebinfo <folder>
+./ya make ydb/apps/ydbd --build relwithdebinfo
+./ya make ydb/apps/ydb --build relwithdebinfo
+./ya make --build relwithdebinfo -tA <folder>
+./ya make --build relwithdebinfo -tA <folder> -F *test-filter*
+./ya make --build relwithdebinfo -tA <folder> -F *test-filter* --test-retries N
+```
+
+- Prefer `--build relwithdebinfo`. No `-j`. No force rebuild (`-r`, `-R`, …).
+- Use `2>&1 | tail` for test output.
+- Build and test the smallest relevant folder.
+
+Test types and frameworks: [`TESTS.md`](TESTS.md).
+
+## Languages
+
+C++20 or earlier. Style: [`CODESTYLE.md`](CODESTYLE.md). Tests: [`TESTS.md`](TESTS.md).
+Python via `ya`, packages from `contrib/python`.
+
+## Agent workflow
+
+- Make the smallest correct change; do not edit `contrib/` or `vendor/` unless required.
+- Match surrounding code style ([`CODESTYLE.md`](CODESTYLE.md)).
+- Search for existing code; check for a local `AGENTS.md`.
+- For non-trivial changes, check [`CONTRIBUTING.md`](../../CONTRIBUTING.md).
+- Run `./ya make --build relwithdebinfo -tA <folder>` for the area you changed.
+- Do not create commits or push unless explicitly asked.
diff --git a/ydb/agents/TESTS.md b/ydb/agents/TESTS.md
new file mode 100644
index 00000000000..f9cf5533f9e
--- /dev/null
+++ b/ydb/agents/TESTS.md
@@ -0,0 +1,53 @@
+# YDB Tests
+
+Follow the testing style of the module you are modifying. For new C++ unit tests,
+prefer **UNITTEST** over **GTEST**.
+
+## C++ unit tests
+
+- Place tests in a `ut/` subdirectory next to the code under test.
+- Declare the test module in `ya.make` with `UNITTEST_FOR(...)` (or `UNITTEST()`
+ for standalone test targets).
+- Register C++ test sources in `SRCS()`.
+- Write tests with `Y_UNIT_TEST_SUITE` / `Y_UNIT_TEST` from
+ `library/cpp/testing/unittest/registar.h`.
+
+Use `GTEST()` only when the surrounding code already does (e.g. some
+`ydb/public/sdk/cpp/tests/` targets). Do not introduce new GTest-based tests
+where UNITTEST is used in the same area.
+
+## Python tests
+
+- Use the `unittest` framework (`unittest.IsolatedAsyncioTestCase` for async code).
+- Register Python test files in `TEST_SRCS()` in the module's `ya.make`.
+- Third-party packages come from `contrib/python`.
+- See [`ydb/tools/mnc/AGENTS.md`](../tools/mnc/AGENTS.md) for MNC-specific
+ Python test conventions.
+
+## Functional tests
+
+End-to-end tests that run against a YDB cluster live under `ydb/tests/functional/`.
+
+- Declare targets with `PY3TEST()` in `ya.make`.
+- List test files in `TEST_SRCS()` (paths are relative to the repository root).
+- Follow existing harness patterns in the same area (`INCLUDE(...harness_dep.inc)`,
+ `ENV(...)`, suite `.inc` files).
+
+## Compatibility tests
+
+Upgrade/downgrade tests live under `ydb/tests/compatibility/`.
+
+- Place each test module in the folder matching the feature area.
+- Use existing fixtures and follow the layout described in
+ [`ydb/tests/compatibility/README.md`](../tests/compatibility/README.md).
+
+## Running tests
+
+```bash
+./ya make --build relwithdebinfo -tA <folder>
+./ya make --build relwithdebinfo -tA <folder> -F *test-filter*
+```
+
+CLI tests (`ya make -tA`) run only on Linux.
+
+Build rules: [`GUIDE.md`](GUIDE.md).
diff --git a/ydb/core/kafka_proxy/AGENTS.md b/ydb/core/kafka_proxy/AGENTS.md
new file mode 100644
index 00000000000..ed8ef943740
--- /dev/null
+++ b/ydb/core/kafka_proxy/AGENTS.md
@@ -0,0 +1,17 @@
+# Kafka proxy
+
+**Kafka-compatible API on top of YDB Topics**.
+Accepts **Kafka wire-protocol** connections (`kafka_connection.cpp`).
+Core: [`ydb/core/persqueue/AGENTS.md`](../persqueue/AGENTS.md).
+
+## Layout
+
+* Root — connections, message codecs (`kafka_messages.*`), transactions.
+* `actors/` — produce, fetch, metadata, offsets, consumer groups, SASL, …
+
+Depends on `persqueue/public/`, not `pqtablet/` / `pqrb/` internals.
+
+Tests: `./ya make --build relwithdebinfo -tA ydb/core/kafka_proxy`
+
+Style/workflow: [`agents/CODESTYLE.md`](../../agents/CODESTYLE.md) ·
+[`agents/GUIDE.md`](../../agents/GUIDE.md) · [`agents/TESTS.md`](../../agents/TESTS.md)
diff --git a/ydb/core/persqueue/AGENTS.md b/ydb/core/persqueue/AGENTS.md
new file mode 100644
index 00000000000..07d3bd29b74
--- /dev/null
+++ b/ydb/core/persqueue/AGENTS.md
@@ -0,0 +1,41 @@
+# YDB Topics
+
+Core implementation of YDB topics (persistent queues).
+
+## Layout
+
+One root directory per tablet or service, plus `common/`, `public/`, `events/`.
+In `public/` and `common/`, nested dirs by **purpose**; in `pqrb/` and
+`pqtablet/` — by **sub-actor or component**.
+
+* **`public/`** — APIs for code outside persqueue (`schema/`, `fetcher/`, …).
+* **`common/`** — shared tablet internals only.
+* **`events/`** — internal events and protos.
+* **`pqrb/`** — whole-topic tablet (balancing, stats, autopartitioning).
+* **`pqtablet/`** — one or more partitions (reads, writes, batching, …).
+* **`writer/`**, **`dread_cache_service/`**, **`deferred_publish/`** — writer,
+ direct-read cache, deferred publish.
+
+Protocol layers (each has its own `AGENTS.md`):
+
+* [`persqueue_v1`](../../services/persqueue_v1/AGENTS.md) ·
+ [`kafka_proxy`](../kafka_proxy/AGENTS.md) ·
+ [`datastreams`](../../services/datastreams/AGENTS.md) ·
+ [`sqs_topic`](../../services/sqs_topic/AGENTS.md)
+
+## Guidelines
+
+* External code uses `public/` (and `events/` where needed), not `pqtablet/` /
+ `pqrb/` internals. Layering: `public` → tablets.
+* In `pqrb` / `pqtablet`: batch when persisting; minimize persists and
+ inter-actor messages; extract large logic into separate actors or classes.
+* `*_fwd.h` for forward declarations; `.pb.h` only when needed.
+ Config helpers: [`public/config.h`](public/config.h).
+* Put method implementations in `.cpp` files, not in headers — except
+ template functions and template classes.
+
+## Tests
+
+`./ya make --build relwithdebinfo -tA ydb/core/persqueue` (or a narrower `ut/`).
+Monorepo rules: [`agents/CODESTYLE.md`](../../agents/CODESTYLE.md) ·
+[`agents/GUIDE.md`](../../agents/GUIDE.md) · [`agents/TESTS.md`](../../agents/TESTS.md).
diff --git a/ydb/core/persqueue/README.md b/ydb/core/persqueue/README.md
deleted file mode 100644
index b44af7fa536..00000000000
--- a/ydb/core/persqueue/README.md
+++ /dev/null
@@ -1,13 +0,0 @@
-# YDB Topics
-
-This directory contains the implementation of YDB topics.
-
-* public - these are utilitarian methods and actors that are supposed to be worked with outside this directory.
-* common - common utilitarian methods and actors that are a closed part of the implementation. Their use is acceptable in the implementation of tablets inside persqueue.
-* pqrb is a tablet that controls the topic as a whole. Implements the following functions:
- * balancing of reading sessions (partitioning by readers);
- * aggregation of statistics for all partitions;
- * coordination of topic autopartitioning;
- * and others.
-* pqtablet - a tablet responsible for the operation of one or more partitions (depending on the cluster configuration).
-* dread_cache_service is a service for transmitting topic messages during direct reading.
diff --git a/ydb/services/datastreams/AGENTS.md b/ydb/services/datastreams/AGENTS.md
new file mode 100644
index 00000000000..9e89e20a9e7
--- /dev/null
+++ b/ydb/services/datastreams/AGENTS.md
@@ -0,0 +1,19 @@
+# Data Streams (Kinesis)
+
+**Kinesis-compatible API on top of YDB Topics**.
+HTTP entry: [`http_proxy`](../../core/http_proxy/) (`datastreams.cpp`) → this service.
+Core: [`ydb/core/persqueue/AGENTS.md`](../../core/persqueue/AGENTS.md).
+
+## Layout
+
+* Root — `datastreams_proxy.cpp`, `grpc_service.cpp`, shard iterators, record writers.
+* `codes/` — error codes.
+* `ut/` — unit tests.
+
+Depends on `persqueue/public/`, not `pqtablet/` / `pqrb/` internals.
+
+Tests: `./ya make --build relwithdebinfo -tA ydb/services/datastreams`
+HTTP integration: `ydb/core/http_proxy/ut/kinesis_ut.cpp`
+
+Style/workflow: [`agents/CODESTYLE.md`](../../agents/CODESTYLE.md) ·
+[`agents/GUIDE.md`](../../agents/GUIDE.md) · [`agents/TESTS.md`](../../agents/TESTS.md)
diff --git a/ydb/services/persqueue_v1/AGENTS.md b/ydb/services/persqueue_v1/AGENTS.md
new file mode 100644
index 00000000000..2cfe6034ebb
--- /dev/null
+++ b/ydb/services/persqueue_v1/AGENTS.md
@@ -0,0 +1,16 @@
+# PersQueue v1
+
+gRPC layer for **Topic API** and **PersQueue v1 (PQv1)**.
+Core: [`ydb/core/persqueue/AGENTS.md`](../../core/persqueue/AGENTS.md).
+
+## Layout
+
+* Root — entry points (`topic.cpp`, `grpc_pq_*.cpp`, `persqueue.cpp`, init).
+* `actors/` — read, write, schema, PQv1 handlers.
+
+Depends on `persqueue/public/` and `events/`, not `pqtablet/` / `pqrb/` internals.
+
+Tests: `./ya make --build relwithdebinfo -tA ydb/services/persqueue_v1`
+
+Style/workflow: [`agents/CODESTYLE.md`](../../agents/CODESTYLE.md) ·
+[`agents/GUIDE.md`](../../agents/GUIDE.md) · [`agents/TESTS.md`](../../agents/TESTS.md)
diff --git a/ydb/services/sqs_topic/AGENTS.md b/ydb/services/sqs_topic/AGENTS.md
new file mode 100644
index 00000000000..9c567644699
--- /dev/null
+++ b/ydb/services/sqs_topic/AGENTS.md
@@ -0,0 +1,20 @@
+# SQS over Topics
+
+**SQS API on top of YDB Topics**.
+HTTP entry: [`http_proxy`](../../core/http_proxy/) (`sqs.cpp`) → this service.
+Core: [`ydb/core/persqueue/AGENTS.md`](../../core/persqueue/AGENTS.md).
+
+## Layout
+
+* Root — API handlers and `sqs_topic_proxy.cpp`.
+* `queue_url/` — URL parsing (`arn`, `consumer`, `holder/`).
+* `protos/` — SQS protos (e.g. receipt).
+* `ut/` — unit tests.
+
+Depends on `persqueue/public/`, not `pqtablet/` / `pqrb/` internals.
+
+Tests: `./ya make --build relwithdebinfo -tA ydb/services/sqs_topic`
+Functional: `ydb/tests/functional/sqs/`
+
+Style/workflow: [`agents/CODESTYLE.md`](../../agents/CODESTYLE.md) ·
+[`agents/GUIDE.md`](../../agents/GUIDE.md) · [`agents/TESTS.md`](../../agents/TESTS.md)