diff options
| -rw-r--r-- | .cursor/rules/ydb-project.mdc | 17 | ||||
| -rw-r--r-- | .github/copilot-instructions.md | 66 | ||||
| -rw-r--r-- | .gitignore | 5 | ||||
| -rw-r--r-- | AGENTS.md | 72 |
4 files changed, 10 insertions, 150 deletions
diff --git a/.cursor/rules/ydb-project.mdc b/.cursor/rules/ydb-project.mdc deleted file mode 100644 index dc968f3e87d..00000000000 --- a/.cursor/rules/ydb-project.mdc +++ /dev/null @@ -1,17 +0,0 @@ ---- -description: YDB project structure, build, tests, and PR conventions -alwaysApply: true ---- - -# YDB Project Rules - -- **Docs:** <https://ydb.tech/docs/en/>. See also repo root [AGENTS.md](/AGENTS.md) and [.github/copilot-instructions.md](/.github/copilot-instructions.md). -- **Structure:** `/ydb/core` — core modules, `/ydb/library` — YDB libs, `/ydb/apps`, `/ydb/services`, `/ydb/tools`, `/ydb/tests` (integration/functional), `/ydb/docs`. `/library` and `/util` — do not modify unless change is explicitly required. `/yql` — YQL; `/contrib` — third-party deps. Build: `ya.make` (and `ya.make.inc`); tool: `/ya` in repo root. -- **Build:** From repo root: `./ya make path/to/project` (e.g. `./ya make ydb/core/base`, `./ya make ydb/apps/ydb`). -- **Tests:** From repo root. Default: `./ya make -tt --build "relwithdebinfo"`. Use `-tt` (small+medium); `-ttt` only when large tests needed. **SIZE(LARGE) tests do not run in PR-check.** When verification needs a large/specific test, suggest the exact command (e.g. `./ya make -ttt --build "relwithdebinfo" -F '...' path/to/target`) or workflow [add_run_tests_table.yml](.github/workflows/add_run_tests_table.yml) (workflow_dispatch + PR number). Single test: `-F 'TestNameOrPattern' path/to/test/target`. Broader: `-F ydb/`. Sanitizers: `--sanitize="address"` (or memory/thread). Do not abort test runs. **Finding targets:** C++ in `ut_*` dirs; integration under `ydb/tests/`. -- **C++:** Modern C++ only, no later than C++20. -- **Docs (in repo):** Follow <https://ydb.tech/docs/en/contributor/documentation/style-guide> and structure. -- **PRs:** Description must include changelog entry (short summary) + detailed description for reviewers. Exactly one category: New Feature, Experimental Feature, Improvement, Performance Improvement, User Interface, Bugfix, Backward-Incompatible Change, Documentation (changelog not required), Not for Changelog (changelog not required). Discuss non-trivial changes in an issue first (see [CONTRIBUTING.md](/CONTRIBUTING.md)). -- **YDB context:** YDB is complex and must be fault-tolerant. For functional changes or root-cause analysis: consider nemesis, OLAP, stability/stress, and `ydb workload` (stock, kv, clickbench, tpcc, tpch, tpcds, topic, transfer, query); suggest running the relevant test suite or workload when appropriate. See [AGENTS.md](/AGENTS.md) section "YDB context: fault tolerance, testing, and workloads". -- **Muted tests:** Tests in `.github/config/muted_ya.txt` are disabled (broken product/test/infra). If a muted test might be hiding a real bug (change touches same area), say so and suggest checking or running that test. -- **.github changes:** When editing workflows, scripts (`.github/scripts/**`), config (e.g. `muted_ya.txt`, `stable_tests_branches.json`), or actions (e.g. `test_ya`): (1) Map who uses the changed file (which workflows/jobs). (2) **Critical:** Warn if PR-check, gate, postcommit, or mute pipeline (update_muted_ya, create_issues_for_muted_tests, `muted_ya.txt` path/format) are affected. (3) **Potential:** Warn on schedule/trigger changes, reusable workflow inputs, analytics/Telegram scripts, secrets/vars usage. (4) **Backward compatibility:** If a script or shared step can run from another branch (e.g. stable-*, prestable-*), breaking changes (new required args, path/format changes) cause failures or divergent behavior on that branch; keep compatibility or plan backport and warn. (5) Suggest verification (e.g. workflow_dispatch, docs in `.github/config/mute_rules.md`, `.github/scripts/telegram/README.md`). See [.github/copilot-instructions.md](/.github/copilot-instructions.md) section "Changes to .github". diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md deleted file mode 100644 index 80701e0026f..00000000000 --- a/.github/copilot-instructions.md +++ /dev/null @@ -1,66 +0,0 @@ -# Repository instructions for GitHub Copilot - -Instructions for working with the YDB codebase. **Apply these guidelines when writing code, reviewing PRs, or changing CI/config.** For project structure, building, running tests, coding conventions, and PR guidelines use [AGENTS.md](/AGENTS.md) in the repository root. This file adds only: - -1. **How to find test targets** (below) — supplement to the tests section in AGENTS.md. -2. **Impact of changes under `.github/`** — workflows, scripts, config, actions. - -Docs: <https://ydb.tech/docs/en/>. - ---- - -## How to find test targets - -- **C++ unit tests:** Look for directories named **`ut_*`** (e.g. `ut_index_build_reboots`) under the module. The target path is the directory path: e.g. `ydb/core/tx/schemeshard/ut_index_build_reboots`. -- **`ya.make`** in such directories usually contains **`UNITTEST_FOR(...)`** and **`SIZE(SMALL|MEDIUM|LARGE)`**; `-tt` runs small+medium, `-ttt` adds large. **Tests with `SIZE(LARGE)` do not run in PR-check.** When verification needs a large or specific test, suggest the **exact command** (e.g. `./ya make -ttt --build "relwithdebinfo" -F '...' path/to/test/target`) or the workflow [add_run_tests_table.yml](.github/workflows/add_run_tests_table.yml) (workflow_dispatch with PR number) to add a run-tests table to the PR. -- **Python / integration tests:** Under `ydb/tests/` (e.g. `ydb/tests/functional`, `ydb/tests/library`). Run via `./ya make -tt ...` with the appropriate test path. - -**YDB fault tolerance and workloads:** YDB is a complex, fault-tolerant system. When making functional changes or analyzing test failures, consider **nemesis** (fault injection), **OLAP** tests (`ydb/tests/olap`), **stability/stress** (`ydb/tests/stability`, `ydb/tests/stress`), and **load via `ydb workload`** (stock, kv, clickbench, tpcc, tpch, tpcds, topic, transfer, query). Suggest running the relevant suite or `ydb workload <type>` when appropriate. Full guidance: [AGENTS.md](/AGENTS.md) section "YDB context: fault tolerance, testing, and workloads". - ---- - -## Changes to `.github` (workflows, scripts, config, actions) - -When you change anything under **`.github/`** (workflows, scripts, config files, or composite actions), **assess impact** and **warn** about consequences. Use two priority levels: **critical / real** (high impact, likely breakage) and **potential** (possible side effects, follow-up checks needed). - -### 1. Map impact before editing - -- **Workflows** (`.github/workflows/*.yml`, `*.yaml`): which jobs/steps call which scripts or actions; which use `secrets.*`, `vars.*`, or config paths. -- **Scripts** (`.github/scripts/**`): which workflows or actions invoke them; expected CLI args, env vars, and input files (e.g. `muted_ya.txt`). -- **Config** (`.github/config/*`): who reads it — e.g. `muted_ya.txt`, `stable_tests_branches.json`, `stable_nightly_branches.json` are used by multiple workflows and by the **test_ya** action (PR-check, postcommit, run_tests, update_muted_ya, create_issues_for_muted_tests). -- **Actions** (`.github/actions/**`): which workflows call them; inputs/outputs and any hardcoded paths (e.g. `.github/config/muted_ya.txt` in **test_ya**). - -### 2. Critical / real problems (warn with high priority) - -- **PR-check / gate / postcommit:** Changes to `pr_check.yml`, `gate_postcommits.yml`, `postcommit_*.yml`, or to **test_ya** / **build_and_test_ya** can break merge checks and postcommit runs. Any change to how `muted_ya.txt` is read or passed (path, format, YAML conversion in `mute_utils.py`) affects all PR and postcommit test runs. -- **Mute pipeline:** `update_muted_ya.yml` and `create_issues_for_muted_tests.yml` depend on: `create_new_muted_ya.py`, `get_muted_tests.py`, `flaky_tests_history.py`, `tests_monitor.py`, and on **`.github/config/muted_ya.txt`** path and format. Changing mute/unmute rules (in code or in `mute_rules.md`) can change which tests get muted and affect analytics and Telegram notifications. -- **Paths and interfaces:** Renaming or moving **`.github/config/muted_ya.txt`**, or changing the format expected by `mute_utils.py` / `transform_build_results.py` / `create_new_muted_ya.py`, will break workflows and the **test_ya** action unless all call sites are updated. -- **Muted tests and real bugs:** Tests in `muted_ya.txt` are disabled (broken product/test/infra). When reviewing code or investigating failures, if a **muted test could be hiding a real bug** (e.g. the change or fix touches the same area as that test), **warn explicitly** and suggest verifying (e.g. run the test locally or consider unmuting to confirm no regression). -- **Secrets and vars:** Workflows use `CI_YDB_SERVICE_ACCOUNT_KEY_FILE_CREDENTIALS`, `YDBOT_TOKEN`, `GH_PERSONAL_ACCESS_TOKEN`, `TELEGRAM_YDBOT_TOKEN`, `TELEGRAM_MUTE_CHAT_ID`, `TEAM_TO_RESPONSIBLE_TG`, etc. Changing step logic that uses these can cause auth/notification failures. - -### 3. Potential problems (warn, suggest verification) - -- **Schedules and triggers:** Changing `on.schedule` or `on.workflow_dispatch` / `on.pull_request` can change how often jobs run or when they start; document the new behavior. -- **Reusable workflows:** `run_tests.yml` is called by regression_* and run_and_debug_tests; changing its inputs (e.g. `branches_config_path`, `test_targets`) affects all callers. -- **Analytics chain:** `collect_analytics.yml`, `collect_analytics_fast.yml`, `monitoring_queries.yml`, and `create_issues_for_muted_tests` run scripts in `.github/scripts/analytics/` and write to YDB; changing query paths, table names, or script order can break dashboards and downstream jobs. -- **Telegram / alerts:** `alert_queued_jobs.py`, `parse_and_send_team_issues.py`, and RAM/alert steps depend on Telegram secrets and vars; logic or env changes can break notifications. -- **Branch/config lists:** `update_muted_ya.yml` reads branches from `BRANCHES_CONFIG_PATH` (e.g. `stable_tests_branches.json`). Changing that path or the JSON shape can change which branches get mute-update PRs. - -### 4. Backward compatibility when a component is used from another branch - -If a **script**, **composite action**, or **reusable workflow** can be run from **another branch** (e.g. `stable-*`, `prestable-*`, or any workflow that runs on a ref other than `main`), the job checks out **that branch** — so it executes **that branch’s version** of the script/action, not the one on `main`. - -**What this can lead to:** - -- **Failures on release/other branches:** A breaking change on `main` (new required CLI argument, changed config path, changed output format) is not present on the old branch. When the workflow runs there, it still uses the old script/action. If only the **workflow YAML** was backported (e.g. new step or new input), the **code** on that branch is still old — the new args or paths are passed to the old script → **runtime errors** or wrong behavior. -- **Divergent behavior:** `main` has the new logic; stable/release branches keep the old one. The “same” workflow name behaves differently depending on branch — **hard to reason about** and **support**. -- **Silent breakage after partial backport:** Someone backports a workflow change but not the script; the run may fail in CI or produce incorrect results (e.g. wrong mute list, wrong artifact). - -**What to do:** When changing a script or shared step that may be invoked from another branch (check which workflows run on which refs, e.g. `stable-*` in `on.branches` or matrix over branches), either **(1)** keep **backward compatibility** (optional args, same paths/formats, additive changes only), or **(2)** explicitly plan **backport** of the script/action to all branches that use it, and **warn** in the PR that release branches must be updated. - -### 5. What to do in the response - -- **Before applying changes:** Briefly list which workflows, actions, or scripts are affected and whether any **critical** path is touched (PR-check, gate, postcommit, mute pipeline, muted_ya path/format). -- **Warn explicitly:** e.g. “Critical: this changes how muted_ya.txt is read; PR-check and postcommit use it — ensure path/format stay compatible” or “Potential: schedule change; verify run frequency is still intended.” -- **Suggest checks:** If you changed a script or workflow, suggest running the workflow manually (workflow_dispatch) or checking dependent workflows’ docs (e.g. `.github/scripts/telegram/README.md`, `.github/config/mute_rules.md`). - diff --git a/.gitignore b/.gitignore index 6e0798ba28b..77ec532350b 100644 --- a/.gitignore +++ b/.gitignore @@ -104,9 +104,8 @@ path.bash.inc # Editor config /.editorconfig -# Cursor ignore (keep .cursor/rules/ in repo so team shares rules) -.cursor/* -!.cursor/rules +# Cursor ignore +.cursor/ .cursorignore # Harness local cluster working directory diff --git a/AGENTS.md b/AGENTS.md index 2e8ca5d16df..9a3f902f52d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -13,83 +13,29 @@ Every directory is a project (a library or a program). - `/ydb/core`: Core modules. - `/ydb/library`: YDB libraries. - `/ydb/docs`: YDB documentation. -- `/library`, `/util`: Common libraries; never change them unless the change is explicitly required. -- `/yql`: YQL (query layer); shared with YDB. -- `/contrib`: Third-party and restricted dependencies. +- `/library`: Common libraries; never change them. +- `/util`: Common libraries; never change them. ## Building and Testing Requirements for AI Agents Building and testing are performed using the Ya utility (`ya-tool`), which is located in the root of the codebase: `/ya`. -To build a project, from the **repository root** run: +To build a project, an AI agent should run: ```bash -./ya make path/to/project +cd path/to/project +/ya make ``` -Alternatively, `cd path/to/project` then run `/ya make`. - -### Running tests with ya make (recommended for C++ unit tests, gtest, py3tests) - -Use the following commands from the **repository root** (where `/ya` lives). - -**Note:** Test runs can take a long time — `ya make` may need to build the whole project or large parts of it if required by dependencies. Do not abort the run; let it complete. - -**Test size:** Use `-tt` by default — runs only **small** and **medium** tests (faster). Use `-ttt` when you need **large** tests too; prefer `-ttt` only for a narrow list of targets or a single test, as it can run much longer. - -**Tests marked `SIZE(LARGE)` in ya.make do not run in PR-check** (PR-check uses `-tt` = small+medium only). When verification **requires** a large or specific test that would cover the change, **suggest** either: (1) **the exact command** to run locally, e.g. `./ya make -ttt --build "relwithdebinfo" -F 'TestNameOrPattern' path/to/test/target`, or a short list of such commands; or (2) running the workflow **Add run tests table to PR** (`.github/workflows/add_run_tests_table.yml`) via `workflow_dispatch` with the PR number — it adds a run-tests table to the PR so important tests can be run. Prefer attaching the concrete command when suggesting a specific test. - -**Default (no sanitizers):** `./ya make -tt --build "relwithdebinfo"`. Use when there is no suspicion of memory/address/thread bugs. - -- **Single test or narrow scope** — when changes touch one test binary or a small area, filter by test name with `-F`: - - ```bash - ./ya make -tt --build "relwithdebinfo" -F 'TestNameOrPattern' path/to/test/target - ``` - - Example: - - ```bash - ./ya make -tt --build "relwithdebinfo" -F 'FulltextIndexBuildTestReboots::BaseCase[PipeResetsBucket1]' ydb/core/tx/schemeshard/ut_index_build_reboots - ``` - -- **Broader scope** — when changes affect more code: - - ```bash - ./ya make -tt --build "relwithdebinfo" -F ydb/ - ``` - -**With sanitizers** — when there are suspicions of address, memory, or thread-safety issues, add `--sanitize="address"`, `--sanitize="memory"`, or `--sanitize="thread"` (use one at a time as needed). Example: +To test a project, an AI agent should run: ```bash -./ya make -tt --build "relwithdebinfo" --sanitize="address" -F 'TestNameOrPattern' path/to/test/target +cd path/to/project +/ya make -A ``` -Choose the single-test form for one test or one binary; use `-F ydb/` when the change touches multiple modules or you need broader coverage. - More information is available at <https://ydb.tech/docs/en/contributor/build-ya>. -## YDB context: fault tolerance, testing, and workloads - -YDB is a **complex distributed system** with strict correctness and **fault-tolerance** requirements. When implementing functional changes, searching for root cause of failures, or debugging test failures, be **extra careful** and consider the following. - -### Test and load contexts - -- **Integration/functional tests** — `ydb/tests/`: `functional`, `stability`, `stress`, `olap`, `sql`, `library`, `compatibility`, `datashard`, `fq`, etc. Run via `./ya make -tt ... ydb/tests/functional` (or the relevant suite). -- **Nemesis** — fault-injection and load tests under failure conditions; results feed into analytics (e.g. `nemesis/aggregated_mart`). When changes touch replication, node failure, or cluster behavior, consider suggesting nemesis-related runs or stability/stress suites. -- **OLAP** — analytical workloads and benchmarks; tests under `ydb/tests/olap` and performance results (e.g. ClickBench, TPC-H). When touching query execution, storage, or analytics, suggest OLAP tests or workloads. -- **Workload (ydb CLI)** — load tests via `ydb workload` subcommands: **stock**, **key-value** (kv), **ClickBench**, **TPC-C**, **TPC-H**, **TPC-DS**, **topic**, **transfer**, **query**. Implementations live in `ydb/library/workload/`. Docs: `ydb/docs/.../workload` (e.g. reference in `ydb-cli/commands/workload`). When changes affect transactions, storage, or throughput, **suggest running the relevant workload** (e.g. `ydb workload stock ...` or the suite that matches the changed area). - -### What agents should do - -- **Functional changes:** Before finishing, consider impact on fault tolerance and data consistency; **suggest** running the most relevant test suite (e.g. `ydb/tests/stability`, `ydb/tests/stress`, `ydb/tests/olap`) or workload (`ydb workload <type>`). -- **Root cause / test failure:** When analyzing a failing test or bug, keep in mind nemesis, OLAP, and workload contexts; if the failure might be environment or load-related, suggest running stability/stress or a specific workload to reproduce. -- **Documentation:** Workload command reference is under `ydb/docs/.../ydb-cli/commands/workload`; link to it when suggesting load tests. - -### Muted tests (`.github/config/muted_ya.txt`) - -Some tests are **disabled** (in muted_ya.txt) because of a broken product, broken test, or broken infra. When reviewing code, investigating a failure, or making changes: **if you suspect that a muted test could be hiding a real bug** (e.g. the change touches the same component or code path that the muted test covers), **say so explicitly** and suggest checking or running that test (e.g. locally or after unmuting) to confirm there is no regression. - ## Coding Conventions for AI agents ### C++ Standards for AI Agents @@ -114,5 +60,3 @@ Some tests are **disabled** (in muted_ya.txt) because of a broken product, broke - Backward-Incompatible Change - Documentation (changelog entry is not required) - Not for Changelog (changelog entry is not required) - -For changes under **`.github/`** (workflows, scripts, config) and for **how to find test targets** (e.g. `ut_*` directories, integration tests under `ydb/tests/`), see [.github/copilot-instructions.md](.github/copilot-instructions.md). |