summaryrefslogtreecommitdiffstats
path: root/yql/essentials/sql
Commit message (Collapse)AuthorAgeFilesLines
...
* YQL-19845 support of lang version checking inside facadevvvv2025-04-162-0/+3
| | | | commit_hash:5cfb2a0aa2904106df4ae69b9311bcc5a695928d
* YQL-19747 Remove default completion engine factoryvityaman2025-04-153-25/+0
| | | | | | | | | | | | | This is to decouple from the `sql/v1/lexer` implementation. - Related to https://github.com/ydb-platform/ydb/issues/9056 - Related to https://github.com/vityaman/ydb/issues/18 - Following https://github.com/ydb-platform/ydb/pull/16820 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1196 commit_hash:4995f3a7cb14b0b735a48e211111764f65be8033
* YQL-19616 refactor test lexers from sql2yql, supported facade run toolsvvvv2025-04-144-0/+113
| | | | commit_hash:fb1727dda2b8c7d2ff42d4436c54cb7aa1ce4bc2
* YQL-19747 Rank keywords just by plain usagesvityaman2025-04-1419-91/+198
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - [x] Rank keywords just by plain usages - [x] `LocalSyntaxAnalysis` now returns a mapping `:: Keyword -> [Following Keywords]`. - [x] Extracted keyword sequence formatting from `syntax/local` to `syntax/format`. - [x] Extracted token display logic from `syntax/local` to `antlr4/vocabulary` as it is ANTLR dependent. --- Example ```python $ ./yql_complete <<< "select " [Keyword] CAST( [Keyword] NULL [Keyword] NOT [FunctionName] If( [FunctionName] Yson::ConvertToString( [FunctionName] Count( [FunctionName] Sum( [FunctionName] Unwrap( [FunctionName] Coalesce( [Keyword] DISTINCT [Keyword] ALL [Keyword] CASE [FunctionName] Max( [Keyword] FALSE [FunctionName] Some( ``` --- - Related to https://github.com/ydb-platform/ydb/issues/9056 - Related to https://github.com/vityaman/ydb/issues/17 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1197 commit_hash:f42cb4aaffe6de7c9137069c4d9c635ee110a805
* Intermediate changesrobot-piglet2025-04-144-3/+50
| | | | commit_hash:b6187f8eba6e8debc23f1928b2e44a396f3511ad
* YQL-19616 Fix lexer/regex STRING_VALUE and TSKIP recognitionvityaman2025-04-113-10/+40
| | | | | | | | | | - Related to https://github.com/ydb-platform/ydb/issues/15129 - Related to https://github.com/vityaman/ydb/issues/11 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1201 commit_hash:53ef677a35649a6dc77d8c4269a8aceefcd15026
* YQL-19790 allow distinct over keysvvvv2025-04-104-1/+9
| | | | commit_hash:5f778a5600a05b527c9ff0b07dcf55e207782165
* YQL-19747 Complete read hints on PROCESS and REDUCEvityaman2025-04-104-13/+33
| | | | | | | | | | | | Forgot to support it. - Related to https://github.com/ydb-platform/ydb/issues/9056 - Related to https://github.com/vityaman/ydb/issues/24 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1195 commit_hash:a2f5937dbca5712f3ecbfccdf66662ce99e70619
* YQL-19747 Complete select and insert hintsvityaman2025-04-0920-28/+220
| | | | | | | | | | - Related to https://github.com/ydb-platform/ydb/issues/9056 - Related to https://github.com/vityaman/ydb/issues/19 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1189 commit_hash:7f1cb1dcf0617aa2c94c3f2188fc9bd481380252
* YQL-19616 Fix regex lexervityaman2025-04-096-9/+27
| | | | | | | | | | | | | | | | | | Fixed regex lexer issues: - `TSKIP` token recognition - `HEXGIGITS` number recognition - `EOF` token content --- - Related to https://github.com/ydb-platform/ydb/issues/15129 - Related to https://github.com/vityaman/ydb/issues/11 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1190 commit_hash:497c39efcbbe4e387da523b5e2c8abaa6485d93b
* YQL-19747 Complete after PRAGMA and multi-token namesvityaman2025-04-0815-30/+286
| | | | | | | | | | | | - [x] Complete after PRAGMA - [x] Complete multi-token names correctly, for example, `yt.` returns only `DisableStrict`, not `yt.DisableStrict` and `DateTime::` returns `Split`, not `DateTime::Split`. I tried to implement it using `CompletedToken` edition, but not all completion environments support candidates with various `contextLen` (`Replxx` does not). So I decided that completions should rewrite only the current token, not sequences. For example, on `DateTime::Spl` rewrite only `Spl`. It makes sense as multi-token names have some namespace separated by a punctuation, so used types only namespace and gets names inside of it. --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1181 commit_hash:9d8967ac43b9348f6dbb53837d92a9dcc9b51f48
* Fix keeping aggregate columns in case of distinct aggregation over windowziganshinmr2025-04-081-11/+11
| | | | commit_hash:6aa8a8297542455d107d7debbfaac3f30f48d885
* YQL-19747 Improve yql_complete tool and add input validationvityaman2025-04-082-9/+32
| | | | | | | | | No description --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1185 commit_hash:1def5874ff6a9a5b3dcdd0ad285d2e64b16c9306
* YQL-19747 statements (hints etc)vvvv2025-04-072-0/+12
| | | | commit_hash:1288e94c1f35aed35f40ac5e9b59e708b7cfafad
* Intermediate changesrobot-piglet2025-04-078-14/+34
| | | | commit_hash:6768768ea3a3962231d3fabdffb2ce0db44e9347
* YQL-19747 Complete type name as a function argumentvityaman2025-04-074-6/+52
| | | | | | | | | | | | | | | | | As I understand, type name should not be completed at `SELECT |`, so I added a check that we are at `invoke_expr` context. Currently composite type keywords are suggested at `SELECT |` and also are uppercased. I will fix it separately when this merged during - https://github.com/users/vityaman/projects/5?pane=issue&itemId=105056723&issue=vityaman%7Cydb%7C8 --- - Related to https://github.com/ydb-platform/ydb/issues/9056 - Related to https://github.com/users/vityaman/projects/5/views/1?pane=issue&itemId=105056423&issue=vityaman%7Cydb%7C7 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1182 commit_hash:e87565867cf9fa82d9ac49a88d59b293d6686fe7
* YQL-19747 pragmasvvvv2025-04-034-0/+73
| | | | commit_hash:7aaa06cd58cc9563a1656a7118c14a461e7f4e2d
* YQL-19747 Complete token sequencesvityaman2025-04-027-82/+120
| | | | | | | | | | | | | Token sequences plan - [x] [Easy] Support `GROUP BY`, `ORDER BY`. - [x] [Easy] Support `Optional<`, `List<`, `Dict<`. - [x] [Easy] Support `Avg(`, `Sum(`. --- Co-authored-by: Victor Smirnov [[email protected]] Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1173 commit_hash:a443dec666c486fef7f891be04d68a786be83049
* YQL-19747 Introduce types and functions rankingvityaman2025-04-0218-147/+526
| | | | | | | | | | | | - [x] Fix bug with incorrect no-case sorting. - [x] Get names from `sql_functions.json` and `types.json`. - [x] Add types and functions ranking according to `rules_corr_basic.json` data via a `PartialSort`. - [x] Add benchmark workspace. --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1167 commit_hash:84d93265fb69bf5651f905d6af038056657e9a16
* Intermediate changesrobot-piglet2025-04-022-5/+7
| | | | commit_hash:28e9e4bd6b02e8914d82b2aafe9f341b5492421f
* Fix DISTINCT over window over joinziganshinmr2025-04-011-0/+4
| | | | commit_hash:d7101ec6fbc95dde360e2a18ac52159dd4535764
* Intermediate changesrobot-piglet2025-04-018-84/+325
| | | | commit_hash:e57b3e95787cc8037f200f1b6b6073e35403b27e
* view: support show create in parserdeminds2025-03-317-11/+51
| | | | commit_hash:1cf0e84327c47568687a689e091a6efbc8286bed
* Error for batch operations with RETURNINGditimizhev2025-03-312-4/+28
| | | | commit_hash:23ea6a6a3224e161a1998aceb2162dfe84744831
* YQL-19747 Complete Function Namesvityaman2025-03-2832-291/+517
| | | | | | | | | | | - Function names are suggested now - Changed the module structure - Checking ruleIndex independence on mode (ansi | default) via unit tests --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1163 commit_hash:1b1a27d2cff8db663c5c7e8efb57896476823315
* YQL-19747 sql functionsvvvv2025-03-282-374/+417
| | | | commit_hash:9f628fe1894ee7dcdcbdd161855b668ca6e7380f
* YQL-19616 Convert YQL lexer grammar to regexesvityaman2025-03-2820-64/+1263
| | | | | | | | | | | | | | | | | - [x] Parse YQL grammar to extract lexer grammar into `TLexerGrammar`. - [x] Translate `TLexerGrammar` into regexes. - [x] Implement a lexer via regexes `TRegexLexer` to test generated regexes validity. - [x] Test on `Default` syntax mode. - [x] Test on `ANSI` syntax mode. --- - Related to https://github.com/ydb-platform/ydb/issues/15129 - Requirement for https://github.com/ytsaurus/ytsaurus/pull/1112 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1127 commit_hash:03ffffe81cdafe7f93a4d3fd9a3212fe67f1c72d
* YQL-19747 Complete a simple type namevityaman2025-03-2819-38/+544
| | | | | | | | | - Related to https://github.com/ydb-platform/ydb/issues/9056 --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1151 commit_hash:6e1e429a2ea805016bf00a1e60b501b7fc8dc8de
* YQL-19747 Split statementsvityaman2025-03-279-32/+127
| | | | | | | | | | | | When we run completion engine on multi-statement query, where preceding statements are syntactically incorrect, `antlr4-c3` does not return candidates. Running engine only on a current statement provides a best-effort attempt to provide candidates. - Related to https://github.com/ydb-platform/ydb/issues/9056 - Depends on https://github.com/ytsaurus/ytsaurus/pull/1127 (`ELexerFlavor`) --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1144 commit_hash:0ced9443a9712191f5420246531f781ca4bc5f42
* YQL-19716: Fix Block Readermrlolthe1st2025-03-211-1/+6
| | | | commit_hash:61891dc030f4c526542b5e7e070d1660880e6c08
* YQL-19746 fixvvvv2025-03-204-6/+17
| | | | commit_hash:49e5e33ed38f7fd623cef80d2765d464a353ff9c
* YQL-19616 Implement ILexer via antlr_astVictor Smirnov2025-03-1917-23/+172
| | | | | | | | | | | | | - [x] Added `antlr_ast/antlr4` module and moved `TLexerTokensCollector4` there from `proto_ast/antlr4`. - [x] Moved stuff around back and forth. Ready for a review. --- Co-authored-by: vityaman [[email protected]] Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1128 commit_hash:e08785c3408ef813505bdc7511560e9536f4ab79
* Added settings of create transfer statement: flush_interval, batch_size, ↵tesseract2025-03-191-1/+12
| | | | | | | consumer_name <https://github.com/ydb-platform/ydb/pull/15770> commit_hash:f9e7a01a29938b4bec4aacdaa4e116101326f7da
* YQL fix complete logicvityaman2025-03-182-23/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ## Why these changes are proposed? I figured out that all this time I did not understand how `antlr4-c3` really works. It is configured with ```ts ignoredTokens: Set<number> preferredRules: Set<number> ``` On completion, it returns ```ts class CandidatesCollection { tokens: Map<number, TokenList> rules: Map<number, ICandidateRule> } class ICandidateRule { startTokenIndex: number ruleList: RuleList } type TokenList = number[] type RuleList = number[] ``` I thought that `rules` is a mapping from a `TokenId` to a `ParserCallStack`, but it totally is not true, as `rules` key is in fact a `RuleId`. The documentation says > Whenever the c3 engine hits a __lexer token__ ... it will check the call stack for it and, if that contains any of the **preferred rules**, will select that **instead** of the __lexer token__. > [Preferred] Rules which replace any candidate token they contain. So when we add the rule `X` to `preferredRules`, then when `C3` hits a lexer token with a `Parser Call Stack` containing the rule `X`, it will not add it to `CandidatesCollection::tokens`, but instead it will add an entry to `CandidatesCollection::rules` with a `Parser Call Stack`. It used when we have `ID_PLAIN` in a grammar, but this `ID_PLAIN` has different meaning depending on the context (`Parser Call Stack`), e.g. it can be a Table Name, Column Name and so on... So we can ask C3 not to include `ID_PLAIN` in `CandidatesCollection::tokens`, but instead emit `rules` like `table_id`, `column_id` and so on into the `CandidatesCollection::rules`. So we have a `Parser Call Stack` for `rules`, but not for `tokens`. ## How it works correctly now then? Described in the comments section. ## How to live on? - [BAD ] Make a rule for each token to capture keywords at `CandidatesCollection::rules`. - [GOOD] Extend `antlr4-c3` to include `Parser Call Stack` for `CandidatesCollection::tokens`. --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1131 commit_hash:1a954f55098b9c090ab87e88f8bee61d9ff319ed
* YQL-19712 RuntimeLogLevel setting, mrjob loggervvvv2025-03-183-0/+23
| | | | | init commit_hash:6178c9e20a737d499b13f1b38fdfb621f2d8db2f
* YQL-19701 linter extension for unknown clustersvvvv2025-03-132-3/+7
| | | | commit_hash:79c042af0cf2c51389b5a22bd866cd211b6acf64
* view: use parent context for query AST buildingdeminds2025-03-114-69/+70
| | | | | | | | | | | | | | | | | | Issues: * <HIDDEN_URL> * <https://github.com/ydb-platform/ydb/issues/14709> Previously it was impossible to create a simple view like the following: ```sql CREATE VIEW demo_view WITH (security_invoker = TRUE) AS SELECT "bbb" LIKE Unwrap("aaa") ``` The problem was caused by the fact that a separate parsing context was used to build a view select AST for later validation. The list of UDFs is global for the whole parsing process to optimize the calculations. This global list was lost in the separate context and the view validation failed in such cases. We fix the issue by using the same context for the view query validation in which the `CREATE VIEW` statement is executed. Moreover, we have started capturing all the statements in the view query text that can affect the compilation (see the `NeedUseForAllStatements` function). commit_hash:f84d54ff1688fb43af7170a4db35f6729f2c4f10
* YQL-19212 more fixes to translator for case insensitive udf namesvvvv2025-03-101-12/+18
| | | | commit_hash:b73ddecc5bdf4eb52cb62f9ba551c72e2d2d3e73
* Fix bug in ALTER USER with set null passwordflown4qqqq2025-03-064-24/+106
| | | | | До этого, при любом запросе ALTER USER без опции PASSWORD выставлялось значение Null Password как дефолтовое - это баг, потому что при модификации пользователя не надо выставлять вообще никакого значения. commit_hash:3cb3d5bb3e9487439787deb9102cd0105b6b9e82
* ignore wildcard warning policy during replayvvvv2025-03-032-0/+2
| | | | commit_hash:2632b064e7883e55cb1876761b7404ace9e77590
* Change the text of error for create user with password.flown4qqqq2025-03-024-20/+78
| | | | commit_hash:6631b600dda026c616983f0b08daa912beb0f5db
* Intermediate changesrobot-piglet2025-02-2816-0/+991
| | | | commit_hash:5fc851d2c72810067fe0d407b66535b17de63129
* Intermediate changesrobot-piglet2025-02-284-16320/+8363
| | | | commit_hash:6de10ee070edbce022e46ebcc3c5740ed2f84b10
* refine Antlr depsvvvv2025-02-2720-201/+83
| | | | | refine Antlr deps commit_hash:71b99424edb283bf56be6e79c9c9a19d805ce2ce
* Switch to library/cpp/charset/lite to avoid dependency on libiconvaneporada2025-02-272-2/+2
| | | | commit_hash:787f4898632b0109fb15fa3f6bbd993430f47616
* YQL-19626 reexport of symbols, better issues from librariesvvvv2025-02-251-4/+0
| | | | commit_hash:051b3854a032e0f7803ad23c03dfbf7f71590cc2
* YQL-18303: Introduce Parse64 functionimunkin2025-02-251-2/+0
| | | | commit_hash:f2068081a3ddfb16f531239f3e0dcc923084d3b0
* YQL-19129 soft WITHOUTvvvv2025-02-216-11/+22
| | | | commit_hash:563aebc32ae87bc18e5e3e38babc034e5e89e2c1
* YQL-19594 Explicit lexers & parsers 2vvvv2025-02-2112-24/+124
| | | | commit_hash:2aa733486a55a04c39ff8bb1c7eaca3a85719647
* Drop internal v0 depsvvvv2025-02-203-63/+4
| | | | commit_hash:d5e60311a1b50570d1c9eb4a7a1620e9881c1a4e
generated by cgit v1.3 (git 2.53.0) at 2026-07-26 18:31:50 +0000