summaryrefslogtreecommitdiffstats
path: root/yql/essentials/sql
Commit message (Collapse)AuthorAgeFilesLines
...
* YQL-20068 backport modevvvv2025-06-101-0/+1
| | | | commit_hash:4ad0e146a3e92836d6aec7eaac00ae0225484b09
* Intermediate changesrobot-piglet2025-06-0914-0/+507
| | | | commit_hash:a5cf347a0fdbf87672444c86b9acad24d4ab956a
* YQL-19747: Extract analysis/localvityaman2025-06-0914-14/+40
| | | | | | | | | | | - Related to `YQL-19747` - Related to https://github.com/ydb-platform/ydb/issues/9056 - Related to https://github.com/vityaman/ydb/issues/52 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1321 commit_hash:ef8af2fa618e6fc316d7a6cf348b7f737bb0cfb4
* YQL-19747: Fix some TODOs and Clang Tidy checksvityaman2025-06-0624-48/+49
| | | | | | | | | | | | | | | | | | | Clang Tidy is cool, but too slow ;(. Btw, should we respect `misc-include-cleaner` check? ```bash ya dump compile-commands > /tmp/cc/compile_commands.json find . -type f \( -name "*.cpp" -o -name "*.h" \) -exec clang-tidy {} -p /tmp/cc {} -checks="misc-*" \; ``` --- - Related to `YQL-19747` - Related to https://github.com/ydb-platform/ydb/issues/9056 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1305 commit_hash:c2ecf638062dcdd0730fed3381da4a230f5352c3
* YQL-20053 fix & clarify warningvvvv2025-06-051-2/+2
| | | | commit_hash:98cd4d4370d009f6e71e69219f5b9d89ab3eca9e
* support database settings in ALTER DATABASEdeminds2025-06-0410-16/+207
| | | | | | | | The main goal is to enable setting SchemeLimits (ShardsLimit and PathsLimit) via YQL. Issue: - https://github.com/ydb-platform/ydb/issues/16742 commit_hash:e873849e3b4bb494e349ccebbb1ff6d51655c5d1
* New CHANGEFEED's option: SCHEMA_CHANGESilnaz2025-06-035-0/+29
| | | | commit_hash:23ba332ac8979f2e4e62231fb47414233ee4043e
* Intermediate changesrobot-piglet2025-06-0221-0/+709
| | | | commit_hash:6476b453be16b7db003e75789e3b630c1168f14f
* YQL-19747: Ignore unsupported statements for YDBvityaman2025-06-021-1/+5
| | | | | | | | | | | - Related to `YQL-19747` - Related to https://github.com/ydb-platform/ydb/issues/9056 - Related to https://github.com/vityaman/ydb/issues/39 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1307 commit_hash:782732c290cf2894ec466d2fbb5d0e96905f1804
* YQL-19747: Complete named expressionsvityaman2025-06-0233-69/+555
| | | | | | | | | | | | | | | | | | | - Introduced scoped name collection at global analysis - Added `BingingsNameService` constructed after global analysis and united with the base service. - Added dollar detection at local analysis. After this point design should be reexamined and module should be refactored. --- - Related to `YQL-19747` - Related to https://github.com/ydb-platform/ydb/issues/9056 - Related to https://github.com/vityaman/ydb/issues/57 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1304 commit_hash:a7bf41703f0ee846e359407b53a5d62fc05928ea
* Intermediate changesrobot-piglet2025-05-289-97/+37
| | | | commit_hash:ab24b3db82789b77ad465efa64de93c5eab43629
* YQL-19747: Support cursor shifting on completionvityaman2025-05-283-33/+34
| | | | | | | | | | | | | | | | | Completion candidates comes unclosed from right and with zeroed `CursorShift`. In future there will be an atomic patch that will close candidates and set `CursorShift` correctly. Not it is zeroed to integrate with clients (e.g. YDB CLI). --- - Related to `YQL-19747` - Related to https://github.com/vityaman/ydb/issues/53 - Related to https://github.com/ydb-platform/ydb/issues/9056 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1299 Co-authored-by: vvvv <[email protected]> commit_hash:38bcd21f0d6959275af89a981fd9b296374aea51
* YQL-19747: Fix cluster provider interpretationvityaman2025-05-273-25/+15
| | | | | | | | | | | | | | | There was a mistake thinking that provider is a part of a cluster name. Actually `yt:saurus` and `saurus` are same clusters and provider (`yt`) serves only as a hint for translation expressions such as `yt:$cluster_name`. --- - Related to `YQL-19747` - Related to https://github.com/vityaman/ydb/issues/54 - Related to https://github.com/ydb-platform/ydb/issues/9056 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1301 commit_hash:19a45cf0bb6c7dda26747b2e3371646272f79bce
* YQL-17269: add DISTINCT to UNIONkndrvt2025-05-275-12/+33
| | | | commit_hash:d002e9690bd7cbd1874fdbfe454c9f7a00256839
* YQL-19747: Support immediate string named expressionsvityaman2025-05-2713-22/+180
| | | | | | | | | | | --- - Related to `YQL-19747` - Related to https://github.com/vityaman/ydb/issues/50 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1292 commit_hash:a6d543bc18b2f9239bf0b33afd8497a7919fe4b3
* YQL-19845 changed defaults for translator/typectxvvvv2025-05-261-1/+1
| | | | commit_hash:03cd7489d5021112dd8bb61893361b17637d37d1
* YQL-19747: Refactor ISqlCompletionEngine configurationvityaman2025-05-261-13/+17
| | | | | | | | | | | | | | | | | > no such statement > sql_stmt: (EXPLAIN (QUERY PLAN)?)? sql_stmt_core; > I suggest to add it always if needed for c3 --- - Related to `YQL-19747` - Related to https://github.com/ytsaurus/ytsaurus/pull/1289 - Related to https://github.com/vityaman/ydb/issues/39 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1297 commit_hash:7b1a8947fdd5d8c6b737e781f2288632359f569d
* YQL-19747: Support statement blacklistvityaman2025-05-239-11/+128
| | | | | | | | | | | | | | I leaved the YDB configuration empty as need your feedback. --- - Related to `YQL-19747` - Related to https://github.com/vityaman/ydb/issues/39 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1289 commit_hash:408c7e0f101027fc6ca3d86d251b5639fcae07bb
* Fix parsing INDEX GLOBAL (SYNC|ASYNC)Vasily Gerasimov2025-05-212-10/+124
| | | | | | | | | | | | | | * Changelog entry Type: fix Component: yql Fix parsing unique index type. There was an error that, if not specified explicitly (SYNC or ASYNC), index type remained with the default value GLOBAL SYNC, despite that it was explicitly specified as UNIQUE in query. https://github.com/ydb-platform/ydb/issues/17885 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1287 commit_hash:60de1b723fb1229f2197087fe21db67c72ccbd13
* YQL-19747: Recover from unclosed ID_QUOTEDvityaman2025-05-205-7/+47
| | | | | | | | | | | | | | | | The simplest solution actually is an SQL injection. --- - Related to `YQL-19747` - Related to https://github.com/ydb-platform/ydb/issues/9056 - Related to https://github.com/ydb-platform/ydb/pull/18146 - Related to https://github.com/vityaman/ydb/issues/46 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1283 commit_hash:7043e7cdd1e0d378926783d31cf5f8e055393aaa
* YQL-19747: Support USE statementvityaman2025-05-2013-15/+382
| | | | | | | | | | | | | | Introduce global analysis which will be extended with more ParseTree analysis later. --- - Related to `YQL-19747` - Related to https://github.com/vityaman/ydb/issues/40 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1281 commit_hash:848fb3fdd6bc9612769c47d66198fca018ff465f
* YQL-19747: Fix unquoted object path completionvityaman2025-05-194-5/+19
| | | | | | | | | | | | | | | | On `SELECT * FROM pn` it completed to `SELECT * FROM ppnv1`, but expected ``` SELECT * FROM `pnv1` ```. --- - Related to `YQL-19747` - Related to https://github.com/ydb-platform/ydb/issues/9056 - Related to https://github.com/vityaman/ydb/issues/44 - Related to https://github.com/ydb-platform/ydb/pull/18146 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1280 commit_hash:e937239fae7bf609bde6358b9b27c2b504c8ba32
* YQL-19616: Fix TRegexLexer performancevityaman2025-05-1910-86/+144
| | | | | | | | | | | | | | | | | Fix `TRegexLexer` performance. Now it is just 2 times slower than a reference ANTLR implementation on Release mode, so merged regexes are 3 times better than scan&compare. ![image](https://github.com/user-attachments/assets/4e0cb27a-491d-4dbd-b10a-5725ffa6d902) --- - Related to `YQL-19616` - Related to https://github.com/ydb-platform/ydb/issues/15129 - Related to https://github.com/vityaman/ydb/issues/42 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1278 commit_hash:1529f641172fea13f0d33fbfd06a4827c6efde01
* Intermediate changesrobot-piglet2025-05-161-1/+1
| | | | commit_hash:6b048925e9cf8eede1e184503d43f211b3be6dbd
* Intermediate changesrobot-piglet2025-05-162-1/+17
| | | | commit_hash:2e70854d1aaae271baa81e098f791d9f4d54ddeb
* YQL-19747: Ignore type_id completions and table_keyvityaman2025-05-162-24/+2
| | | | | | | | | | | | | | Fixes superfluous candidates (`OPTIONAL`, `CALLABLE`, ...) at `SELECT * FROM #`. --- - Related to `YQL-19747` - Related to https://github.com/ydb-platform/ydb/issues/9056 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1273 commit_hash:d34d81ea86b8b6b5d339174be20d14d0371c3de6
* YQL-19966 subqueries in viewsvvvv2025-05-151-1/+1
| | | | commit_hash:d39880b60ceb7a5978a9ed1b8b95cfaf3c184373
* YQL-19610: Split OrderedColumns into 2 pragmasmrlolthe1st2025-05-134-0/+13
| | | | commit_hash:9780948e26ef2e689d26520b76fefa0794efdd9e
* YQL-19747: Complete camel case type namesvityaman2025-05-122-36/+69
| | | | | | | | | | | | --- - Related to `YQL-19747` - Related to https://github.com/vityaman/ydb/issues/8 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1268 commit_hash:76e4e2c8b225994cc842c1ee00a0ec8e6eed5c05
* YQL-19747: Fix caret token positionvityaman2025-05-125-16/+30
| | | | | | | | | | | | | | | | | | | | Outcomes: 1. Need to think more deeply about library interfaces. If you think that an interface is redundant, think again, because author knows a usecase that you do not imagine. 2. If you are lazy now to implement something robustly, think twice because the day will come and this code will fire. The problem was that on input ``` ... cluster.`/yql/t#` ``` C3 receives only a prefix ``` ... cluster.`/yql/t ``` and a token stream produced was incorrect and therefore completions too. --- - Related to `YQL-19747` - Related to https://github.com/ytsaurus/ytsaurus/pull/1257 - Related to https://github.com/ydb-platform/ydb/issues/9056 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1271 commit_hash:8bf5f2683c9e41963003da923f255f63b898aac5
* Intermediate changesrobot-piglet2025-05-122-7/+5
| | | | commit_hash:5fe1ef25f8ddc098a82ee48c8f39ac413905f659
* Intermediate changesrobot-piglet2025-05-1221-191/+1396
| | | | commit_hash:3a624a323006078de71f50747f7b2e8cadba7ccd
* YQL-19747 integration of cluster & schema name services to tornadovvvv2025-05-071-5/+6
| | | | commit_hash:13ef56a642595d0dc7336b790971a5ed9155163f
* YQL-19747 Complete folder, table and cluster namesvityaman2025-05-0657-395/+1540
| | | | | | | | | | | | | | | | --- - Related to `YQL-19747` - On top of https://github.com/ytsaurus/ytsaurus/pull/1253 - Related to https://github.com/ydb-platform/ydb/issues/9056 - Related to https://github.com/vityaman/ydb/issues/14 - Related to https://github.com/vityaman/ydb/issues/35 - Related to https://github.com/vityaman/ydb/issues/40 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1257 commit_hash:0b842abb27184c88b8177beeea29fb1ea86b7a04
* Intermediate changesrobot-piglet2025-05-051-0/+16
| | | | commit_hash:9d5644514d892de24a7e9625590c95a4549b0a46
* YQL-19747 Fallback on failing NormalizeNamevityaman2025-04-3011-40/+32
| | | | | | | | | | | | | | | We faced with a failing assertion at `NYql::NormalizeName`. This patch expects it and fallbacks to `ToLowerUTF8`. --- - Related to `YQL-19747` - Related to https://github.com/ydb-platform/ydb/issues/9056 - Related to https://github.com/vityaman/ydb/issues/38 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1255 commit_hash:946162904ce3b74f490debb7f9e0587f136b4075
* YQL-19747 Introduce Union and split Static NameServicesvityaman2025-04-3011-77/+328
| | | | | | | | | | | | | | | | | Essentially we need the `UnionNameService` to integrate dynamic name services into the `sql/v1/complete`. This decorator queries multiple name services and merges their responses. We will use it to union static `NameServices` and the `SchemaNameService` (and the `ClusterNameService`, and so on). It does not deal with exceptions and fails the whole request on some subtask failed. Client should prepare robust children (later `SwallowingNameService` will be added to return an empty `NameResponse` on errors to provide best-effort dynamic object loading). `StaticNameService` was split into micro-`NameService`s :) `NameConstraints` are extracted to provide name qualification via `Qualified` and `Unqualified` methods. This is needed because depending on a context `NameService` can return unqualified names (for example, on `PRAGMA yt.#`). As internal indexes for scanning and ranking are built on a sorted list of, then `Ranking` actually needs an ability to get fully-qualified names, so now it via the `NameConstraints`. Also this design potentially let us to improve internal indexes by using partitioning by a namespace. Other option was to make `PragmaName` and `FunctionName` more structured via adding a separate field for a namespace, but it seems to me that it will force to do more parsing from indexes results to a `Name`. Anyway this is an internal component so it can be changed if needed. I still doubt this decision because structured `PragmaName { Namespace, Identifier }` seems to be cleaner and there should be no noticeable overhead because of COW strings. --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1252 commit_hash:e8a1fbb17e2ca3ffe72aafbea943a38624d74491
* YQL-19747 inside udfsvvvv2025-04-293-6/+13
| | | | commit_hash:42980de83cc65d4d533f6b690391a0e1038fe372
* YQL-19747 Publish ParseUdfs, ParseTypes and othersvityaman2025-04-292-2/+25
| | | | | | | | | | | | | | A client might want to have completions of its own private UDFs. Then a client should make a JSON document and parse it to create a custom `TNameSet`. --- - Related to https://github.com/ydb-platform/ydb/issues/9056 - Related to https://github.com/vityaman/ydb/issues/36 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1251 commit_hash:bbee9be4a480262aa788e7b242b7abdc90882ba7
* YQL-19747 Introduce SimpleSchemaGatewayvityaman2025-04-299-36/+133
| | | | | | | | | | | | | | | | | | | Introduce the `SimpleSchemaGateway` to make it easier to implement `SchemaGateway`s. The idea is that actually existing schema providers really do not support filtration such as by name and type, so in practice they return us the whole list and we need to filter it by hand. The `SimpleSchemaGateway` to `SchemaGateway` adapter does this for us -- we only need to implement a path splitting and folder listing. The other and important feature of the `SimpleSchemaGateway` is that it is simple to implement a caching decorator for it -- just store a mapping `Path -> [FolderEntry]`, while caching a `SchemaGateway` with filters is soooo not trivial. I also added string constants for known folder entry types, because they should be documented somewhere. --- - Related to `YQL-19747` - Related to https://github.com/vityaman/ydb/issues/14 - Related to https://github.com/vityaman/ydb/issues/34 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1245 commit_hash:dda6dcac544ca95d5e8e08f1e7de9de6b5770f25
* YQL-19747 Enable custom NameSet and FrequencyDatavityaman2025-04-2817-140/+196
| | | | | | | | | | | | | | | | | | | | | | Clients might want to use custom `NameSet` and `FrequencyData` in their environment, for example, to get their private UDFs and to have more relevant ranking that includes their private UDF and respect their usage pattern. To achieve this goal I decided to load pure `NameSet` and `FrequencyData` and provide functions for pruning. Also I checked defaults and decided that it is more common for a client to create a `StaticNameService` from pure `NameSet` and `FrequencyData` to keep their pruning consistent. I also extracted a separate module `ranking`. It will be needed when I will implement `UnionNameService` to union `StaticNameService` and `SchemaNameService`. `UnionNameService` will load `Limit` names from each child and then crop them to sorted prefix of length `Limit` using `IRanking`. --- - Related to `YQL-19747` - Related to https://github.com/ydb-platform/ydb/issues/9056 - Related to https://github.com/vityaman/ydb/issues/14 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1246 Co-authored-by: vvvv <[email protected]> Co-authored-by: vvvv <[email protected]> commit_hash:cdca301a58a34e56d537a447b4ff779cd70faea6
* YQL-19747 Use TMaybevityaman2025-04-285-18/+22
| | | | | | | | | | | | | | Just replaced `std::optional` usages with `TMaybe` to prevent this refactoring noise in future PRs. --- - Related to `YQL-19747` - Related to https://github.com/ydb-platform/ydb/issues/9056 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1247 commit_hash:dca8a6849e5ba9cb614d8350996f9423a1dc2373
* YQL-19747: Refactor sql/v1/complete usage of Future and Ptrvityaman2025-04-2531-330/+262
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - [x] Chained futures, add `CompleteAsync` method (then will migrate the YDB CLI on it). - [x] Removed deadlined and fallback NameService as unused - [x] Annotate thread-safe methods with `const` and use `AtomicSharedPtr` for them. - [x] Move `name` to `name/service` with backward compatibility with the YDB CLI. --- `CompletionEngine` is left thread-unsafe because of the dependency chain `CompletionEngine -> LocalSyntaxAnalysis -> C3Engine` which is thread-unsafe, but readonly indexed data structures such as `Ranking` and `NameService` are annotated with const and distributed via shared pointers. I removed deadlined and fallback name services because the first is stupid the second is ahead of its time and is better to be added later to keep interfaces as minimal as possible. --- The migration on async complete plan: 1. Introduce CompleteAsync 2. Migrate clients on CompleteAsync 3. Make Complete to return Future 4. Migrate clients on Complete 5. Remove CompleteAsync --- - Related to https://github.com/ydb-platform/ydb/issues/9056 - Related to https://github.com/vityaman/ydb/issues/33 - Related to https://github.com/vityaman/ydb/issues/31 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1241 Co-authored-by: vvvv <[email protected]> Co-authored-by: vvvv <[email protected]> commit_hash:497cc081ab78bebf7354e0acfaa418d936cc8240
* Primitives for case insensitive simple pattern matchzverevgeny2025-04-255-8/+25
| | | | commit_hash:5f4bdb090c2f60459073e3e95ccd39ec58b95232
* YQL-19747 Detect a token at the caret positionvityaman2025-04-2310-115/+302
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When I tried to implement a folder and object names completion at `ID_QUOTED` I faced with a problem, that I actually can't detect, that cursor is at `ID_QUOTED` token because `TCompletionInput::Text` it was cut until the `TCompletionInput::CursorPosition`, therefore at input ``` SELECT * FROM `#` ``` prefix was ``` SELECT * FROM `# ``` and then lexer failed. While we actually want tokenize the whole current statement, `C3` still needs to receive a prefix as input. I tried to tokenize the whole statement and then on input `SELECT Optional<#>` got nothing because `<>` is solid token in the `SQL`. The only way to fix it I found is to cut a query to prefix until the cursor position. BTW, current implementation is not so efficient as we tokenize the input multiple times. Especially `SplitQueryToStatemnts` seems heavy. In future we anyway will parse the whole input so will need to design APIs to receive ready token streams to do statements splitting, for example, just not to do the work twice. ![image](https://github.com/user-attachments/assets/114804d3-f311-4a46-be84-8ed4650bc9dd) So I introduce you the following changes - [x] Select the whole current statement, not just prefix. - [x] Find the token at caret and output no candidates when caret is at `STRING_VALUE`, `DIGIGTS` and so on. - [x] Change `C3` wrapper interface to take `TCompletionInput` to hide an implementation detail that it runs on cut prefix. - [x] `#` annotated queries in unit tests. - [x] Detect `CaretTokenPosition` -- if is it enclosed with a token or between two. - [x] Ensure that `maxErrors` in `ILexer::Tokenize` is positive. Just a tiny bugfix. --- - Related to https://github.com/ytsaurus/ytsaurus/pull/1209 - Related to https://github.com/ydb-platform/ydb/issues/9056 - Related to https://github.com/vityaman/ydb/issues/14 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1225 commit_hash:a434b9888ec8a7356247d63d9f1420e256ae4fca
* YQL-19747 Normalize names for ranking and filteringvityaman2025-04-2210-22/+183
| | | | | | | | | | | | | | I was lazy to search for a most frequent used name among equivalent by the relation `(a ~ b) iff (NormalizeName(a) = NormalizeName(b))`. Because it seems that names we receive from JSONs are canonized and therefore in a preferable style by the opinion of the YQL language designers. But because of duplicates at `statements_opensource.json` we have, for example, both `IGNORETYPEV3` and `IGNORE_TYPE_V3` in candidates list. I think that we should just remove `IGNORETYPEV3` from the JSON. --- - Related to https://github.com/ydb-platform/ydb/issues/9056 - Related to https://github.com/vityaman/ydb/issues/21 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1229 commit_hash:fe73374ae27df1fcacb0adccda930ec98ed1d7a6
* Intermediate changesrobot-piglet2025-04-229-0/+293
| | | | commit_hash:a1ac242ea5619c1e83dd638a82653e1d5c385821
* YQL-19845 CurrentLanguageVersion funcvvvv2025-04-221-0/+1
| | | | commit_hash:2af511a18740c931b471dc1f2ff36a8b4ce573a8
* YQL-19864 docsvvvv2025-04-184-5/+16
| | | | commit_hash:cc82995d04c8bd3b7ca4d6fe69e91edc092e1b32
* YQL-19864 sql flag + test with explicit flag & by versionvvvv2025-04-174-3/+21
| | | | commit_hash:902cfa0c1b574c1addb5df96a4b38c792ae82258
generated by cgit v1.3 (git 2.53.0) at 2026-07-26 17:18:23 +0000