summaryrefslogtreecommitdiffstats
path: root/yql/essentials/sql/v1
Commit message (Collapse)AuthorAgeFilesLines
* YQL-20436: Translate `SELECT .. FROM VALUES ..` to `YqlSelect`vitya-smirnov7 days12-3/+871
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Current `SQLv1` translation produces relatively low level `YQLs` constructions that are hard to match during later optimizations, for example, subqueries unnesting. Also it assumes that expressions are only depend on corresponding source row, which is not true for correlated subqueries. Both limitations blocks currelated subqueries implementation. Although the problem exists for `SQLv1`, it is already solved for `PG` syntax. There PostgreSQL-produced AST is converted to special `YQLs` "bulk select" node called `PgSelect`. It is more declarative and expanded later than translation. This fact helped to support correlated subqueries, which with `PgSelect` are type-checkable and decorrelatable (not generally, but heuristically). This patch is the first step forward to "bulk select" translation for `SQLv1`. As there are a lot of code already written for `PgSelect` and `PG` and `SQLv1` relatively similar, I decided to rebrand `PgSelect` into more general `SqlSelect` node. It seems that for some near future goals `PgSelect` should be enough. There was 2 problems solved: 1. `PgSelect` comes with `OrderedColumns` by default and its implementation tightly coupled with it. 2. `PgSelect` does Pg type casts. This patch contains following changes: - Added pragma `YqlSelect = 'disable' | 'auto' | 'force'` - Added `YqlSelect` (`PgSelect` alias) translation - Changed `PgSelect` wrappers to support `YqlSelect` - Changed `PgSelect` expanders to support `YqlSelect` commit_hash:8a55d63e06c22592b2029dd260bbd259194e92dc
* YQL-20504: Fix formatting check in QPlayersvitya-smirnov8 days4-0/+134
| | | | commit_hash:cbb6bbe39c72e2322d57a38ded86cda07c68a419
* YQL-20086 invert flagvvvv10 days8-16/+0
| | | | commit_hash:a6b640bc576263b62884a0b4187ba79c893e13d9
* YQL-20072: Init pragma Layer and yt.LayerCachesmrlolthe1st11 days4-5/+27
| | | | commit_hash:9199da15c695cf5c8a492750a8bcfbb0f31c0b34
* YT-25914: [cbo] use column DataWeight + update cost function + add version ↵lucius12 days3-0/+22
| | | | | | | | | | | | | pragma Новая кост-функция для cbo. Под флагом чтобы удобнее сравнивать со старой. 2 изменения: - Для оценки размера входных таблиц используется поколоночный dataweight (если есть), а не таблица целиком - В самой кост-функции убран outputSize, чтобы он не учитывался дважды: он неявно учтен в каждом следующем джоине как левый либо правый inputSize, а размер результата последнего джоина не зависит от перестановки джоинов commit_hash:d130848393114b1b4383035381dba7767aac62fb
* YQL-20496: Improve YQL highlightingvitya-smirnov12 days6-15/+58
| | | | commit_hash:edf72fd8386c41f858f677152612bd58680506a6
* Implement partitions() table functionaneporada12 days3-0/+44
| | | | commit_hash:19bbcd26d46a4ca8d18f0dbae605b48ac823c614
* YQL-20086 sql/v1vitya-smirnov14 days223-23903/+23654
| | | | commit_hash:55bc611cdaa0d8a0fc3c4c7708ed9f17cc4976cf
* support consumers' availability_period setting in the yqlswarmer2025-10-065-26/+72
| | | | | KIKIMR-24054 commit_hash:b7c840d6f25ec732f818f8760e8ce8819393901e
* YQL-20116: Introduce ANTLR4 ambiguity detectionvitya-smirnov2025-10-067-14/+69
| | | | | | - Ignored 2 known ambiguities. - Tested tools. commit_hash:9e29bb2f876dabc68293b3e5c26a470d373506ae
* YQL-20086: Format yql/essentials/sql/v1/highlightvitya-smirnov2025-10-029-423/+425
| | | | | | | | | | | | | | | | This patch adds a style check for `yql/essentials/sql/v1`. To format use `ya style`. To test use `ya test ya test --test-type clang_format`. To enable style checking in a module, need to add: `ENABLE(YQL_STYLE_CPP)`. So in a such way we gradually will format all YQL modules, expect those, where fluent YQL-builders are used. Then we will revert activation macro, so switch from `ENABLE(YQL_STYLE_CPP)` to`ENABLE(YQL_STYLE_CPP_XFAIL)`. Guide: <https://nda.ya.ru/t/XzkoRIGg7KjXmq>. commit_hash:fa758a214e094c74821fe896184f30483dd18c55
* YQL-20307: Support inline subqueriesvitya-smirnov2025-10-0122-203/+952
| | | | | | | | | | | - Alter grammar to support inline subqueries. - Support inline subqueries in `sql/v1` (translator). - Introduce `sql/v1/proto_ast/parse_tree.h` for reusable parse tree predicates. - Support inline subqueries in `sql/v1/format`. - Support inline subqueries in `sql/v1/complete`. - Add some SQL-tests. - Pass all tests. commit_hash:075b2240778d071e1c7542f912d3cc83019ef849
* Implement partition_list table functionaneporada2025-09-293-0/+33
| | | | commit_hash:dba6cae7baee068a70013da4dc3377a10f2d4d22
* YQL-19616: Optimize regexes for Monarchvitya-smirnov2025-09-292-84/+86
| | | | | | | Monarch regex engine is so slow on regexes like `([A-Z]|[a-z]|_|[0-9])*`, but works well on `([A-Za-z_0-9])*`. So such optimization is added. commit_hash:b749e59818a20d7fd1d1ba21b233e1467e9d6ced
* YQL-20461: Normalize table function namesvitya-smirnov2025-09-292-8/+20
| | | | | So now both `AsTable` and `AS_TABLE` work. commit_hash:ae8c33f44e7b10de02a9d979373f4b0602ff0b12
* Fix -Wunused-but-set-variable in yqlmikhnenko2025-09-264-6/+6
| | | | | | | If you think that this pr has broken something for you, roll it back If the problem is not trivial, come to [support](https://nda.ya.ru/t/a1JIzEbo78mtfQ) Original errors you can see [here](https://nda.ya.ru/t/pFJQM0ho7KSaTN commit_hash:77d426cfbc646733517e14e69ac61a1afedaf82f
* YQL support fulltext index typekungasc2025-09-233-3/+12
| | | | | part of https://github.com/ydb-platform/ydb/issues/23384 commit_hash:dc3c331d4054a7739ca632ddce312c985d4b8a13
* The ability to specify not null, default, family in any orderflown4qqqq2025-09-239-38/+432
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Проблема в следующем. Есть запрос: ```sql CREATE TABLE series_with_families ( series_id Uint64, title Utf8, series_info Utf8  NOT NULL FAMILY family_large, release_date Uint64, PRIMARY KEY (series_id), FAMILY default ( DATA = "ssd", COMPRESSION = "off" ), FAMILY family_large ( DATA = "rot", COMPRESSION = "lz4" ) ); ``` Такой запрос является на данный момент неверным на уровне грамматики. Ругается на строку \``series_info Utf8  NOT NULL FAMILY family_large`\` Однако запрос, в котором NOT NULL и FAMILY поменяны местами, отрабатывает корректно: ```sql CREATE TABLE series_with_families ( series_id Uint64, title Utf8, series_info Utf8 FAMILY family_large NOT NULL, release_date Uint64, PRIMARY KEY (series_id), FAMILY default ( DATA = "ssd", COMPRESSION = "off" ), FAMILY family_large ( DATA = "rot", COMPRESSION = "lz4" ) ); ``` В этом Pull Request'е я меняю грамматику таким образом, чтобы была возможность задавать "опции" колонки (DEFAULT, NOT NULL, FAMILY). Особое внимание стоит обратить внимание на тест `CreateTableDefaultAndNotNullInOrderSpace`: в нем показывается, что можно использовать синтаксис ```sql CREATE TABLE tbl ( k Uint64, v Bool DEFAULT false NOT NULL, PRIMARY KEY (k) ); ``` И такое поведение означает именно DEFAULT (false NOT NULL), то есть DEFAULT true. Дело в том, что выражение "false NOT NULL" эквивалентно "false IS NOT NULL", что является, безусловно, истиной. Ровно для того, чтобы была вообще возможность одновременно указать и default, и not null, было добавлено расширение синтаксиса: теперь можно указывать скобки для указания опций столбца, а сами опции внутри них можно перечислять через запятую: ```sql CREATE TABLE tbl ( k Uint64, v Bool (DEFAULT false, NOT NULL), PRIMARY KEY (k) ); ``` Запрос выше означает, что столбец должен по умолчанию заполняться false'ами, но при этом он должен быть NOT NULL. commit_hash:67cc27a5fe6ae73f8725b051b232c2ca0d1ec54f
* YQL-20339 expr high level funcsvvvv2025-09-221-0/+5
| | | | | init commit_hash:479f16e9cd255b616573d811910403290f9ef656
* Do not use DependsOn in TablePath/TableRecordziganshinmr2025-09-226-3/+21
| | | | commit_hash:0241dcec17f0a9b9cf2abfc955a4af169084c387
* YQL-20339 expr low level funcsvvvv2025-09-223-0/+92
| | | | | init commit_hash:ededd246fa929931de6e89fd5f809157d9fe4d16
* YQL-20339 block function for scoped argsvvvv2025-09-191-0/+1
| | | | | без langver, т.к. может быть использована для починки оптимизаторов, где при раскрытии List-ов из воздуха рождается Iterator а потом все заканчивается Collect commit_hash:21cb9b9454cb255b476d50bdb8c5db150cd773fd
* SeqMode supportudovichenko-r2025-09-181-1/+1
| | | | commit_hash:3d0e8bfb46470e6ac36b0d198cfa1723c665f643
* YQL-20339 expr types & reflectionvvvv2025-09-181-0/+5
| | | | | init commit_hash:1c72053b3785a26cfde418f28a9d054b5a624627
* Add new ASYNC REPLICATION's & TRANSFER's optionsilnaz2025-09-171-0/+8
| | | | | | | Related to - https://github.com/ydb-platform/ydb/pull/25090 - https://github.com/ydb-platform/ydb/issues/21470 commit_hash:f97a6414f95b706c34a086d4431367909dfd35a4
* Pass index settings as iskungasc2025-09-155-160/+46
| | | | | | | It seems that there is no much sence in parsing (vector) index settings, and it is much easier to pass them as is It will much help in adding fulltext index with dozens of parameters commit_hash:ba3d7b32d60f54dd6c8f0aba116a10450455d68b
* YQL-20410 not nullvvvv2025-09-104-11/+48
| | | | commit_hash:95bd6eff4013fca11cfe9e941b6175d93c310bdd
* YQL-20258 sql syntaxvvvv2025-09-081-0/+37
| | | | commit_hash:c2440ce49cc8c7637134ddf8081b32251ff75fd4
* YQL-20381: Infer columns from indirect named nodevitya-smirnov2025-09-082-40/+92
| | | | commit_hash:05d53dee40119ad1d756d90775e6056fbebd99d2
* YQL-20379: Fix columns at subqueryvitya-smirnov2025-09-082-6/+32
| | | | | | | Thet top-level source was always selected for column inference, even when the cursor is at a subquery. commit_hash:c0eb00b98b23d52079aea5eb8d646887ff2fbbdd
* YQL-20297: Make generated regexes more stablevitya-smirnov2025-09-051-1/+5
| | | | commit_hash:b3270397329599800d4e7b1f92b8e0f18e94cfd6
* Move vector index settings validation out of YQL (ydb/issues/22345)kungasc2025-09-056-86/+91
| | | | | | | перенос валидации параметров векторного индекса из YQL в YDB (https://github.com/ydb-platform/ydb/pull/23204), потому что аналогичные создания индекса могут происходить и из SDK, и делать бы две разных валидации было бы странно соответственно тут остался только фикс параметров не того типа, а валидация убрана совсем (все параметры опциональные, провалидируются дальше) commit_hash:5418e97f75e30ea8acdfe99243c0a96ef79be5d1
* YQL-20234: Remove predicate argument on middle aggregation phasesvitya-smirnov2025-09-054-2/+48
| | | | | | | | | | | Also added tests for: - max_by - agg_list - some - avg_if - sum_if - count_if commit_hash:0da4f3b9b1767850e65c914b727bc362cf4cc125
* KIKIMR-23992: Add secret SQL operations – create, alter, dropyurikiselev2025-09-0513-0/+889
| | | | commit_hash:3b74a89e67fd29fdced780847365861db726b687
* YQL-20375: Fix emit of SqlCall external typesimunkin2025-09-044-29/+28
| | | | | Follows up ydb-platform/ydb@34fcd78a2d18d9846fd68339c8e240702857e1e5 commit_hash:08520ae8b837c69f6b92e575882da4786aa441fd
* cleared world dependency for streaming queriesgrigoriypisar2025-09-046-16/+36
| | | | | | | Cleared world dependency for streaming queries Supported top level pragmas for streaming queries commit_hash:2bf7256ee9ff7bba724c76acf7086db014a96afd
* YQL-20367: Add DisableExceptIntersectBefore202503 feature flagvitya-smirnov2025-09-045-3/+29
| | | | commit_hash:a8645f7215087df3309048de31a7f0b3dc802a11
* YQL-20368: Fix unsupported literal error positionvitya-smirnov2025-09-041-8/+8
| | | | commit_hash:7007950c8dbbf4d1c8edd5c275eaea723e29e56b
* fixed parsing for BEGIN / END in streaming queriesgrigoriypisar2025-09-045-29/+135
| | | | | | | Добавлена возможность отключения отложенного применения '\\n' и '\\r' в TextWalker, чтобы позиции генерируемые им были в точности равны позициям токенов от antlr4 лексера: <https://nda.ya.ru/t/hmKq_iWN7JVCGe> commit_hash:15049d23b9ac1232b9e1d281d86d6b51d5822f85
* YQL-20365 ConvertTo, warn for Ensurevvvv2025-09-031-1/+1
| | | | commit_hash:39b07bbc84e57d9df02c839273366c28442b789f
* YQL-20189: Track warnings as errorsvitya-smirnov2025-09-0314-105/+346
| | | | | | There was an issue that a query with warnings as errors passed translation. commit_hash:890d18853380b5ad669e9684553cdb6991827cff
* YQL-20368: Improve unimplemented literal errorvitya-smirnov2025-09-031-5/+21
| | | | commit_hash:27dc40f323e5d24af0dae4f3917a63248f8f8386
* YQL-20345: Disable EXCEPT/INTERSECT/DISTINCT before 2025.03vitya-smirnov2025-09-014-3/+39
| | | | | | Added absent checks for `langVer` at `EXCEPT/INTERSECT/DISTINCT` translation. In next PR will get rid of `.sqlx` file extensions. commit_hash:699cc8f6084d1d2fc19ecd53c35a56496dc842f4
* added sql syntax for CREATE / ALTER DROP STREAMING QUERYgrigoriypisar2025-08-2816-106/+1054
| | | | | | | | | | | | | | | | | | | | | | | Добавлен синтаксис (переиспользован код для inline action): ``` CREATE [OR REPLACE] STREAMING QUERY [IF NOT EXISTS] query_name [WITH ( key = value ... )] AS DO BEGIN ... END DO; ALTER STREAMING QUERY [IF EXISTS] query_name [SET( key = value ... )] [AS DO BEGIN ... END DO]; DROP STREAMING QUERY [IF EXISTS] query_name; ``` commit_hash:29fa6aa7e61ecf45112480fe3c1df8fab542354e
* YQL-19747: Replicate unambiguous columns and fix rankingvitya-smirnov2025-08-268-40/+263
| | | | | | | | | | | | | | This is an improvement for column suggestions. Even when user made a source aliased, he still is able to reference a column with both unqualified and qualified name, so we should provide both candidates, but with unqualified more relevant. As this patch required non trivial `NameResponse` modifications at the `CompletionEngine`, ranking should be used properly, so a way to pass it to the engine was introduced. commit_hash:1f2b90e2f6fbb7d33d9fc8479f43349a7f08c320
* YQL-20332: Fix error messagemaxkovalev2025-08-261-1/+1
| | | | commit_hash:2fc8e4b885a4426726d34aa7a80bb62e747462e2
* Add external_data_channels_count create table feature (SQL part)Anton Romanov2025-08-224-1/+35
| | | | | | | | | | | | | | | Can I use your `soroka` for push YQL part of a new feature for YDB, please? [There](https://github.com/ydb-platform/ydb/pull/21672) is the original PR approved by Fomichev an Puchin which can't be committed directly to YDB github. Type: feature Component: yql --- Co-authored-by: Tony-Romanov [[email protected]] Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1414 commit_hash:c98a7b656a050b09976f22f1470ba0ab5bbb52c2
* Add new option CACHE_MODE in FAMILY settingsivannik2025-08-225-3/+115
| | | | | | RFC: https://nda.ya.ru/t/vPsncOoC7HttdF GitHub Issue: https://github.com/ydb-platform/ydb/issues/18695 commit_hash:96445a9aef7b1fede372b9e4c60b30eb9ef5fdc1
* YQL-19747: Fix YQL autocomplete configurationvitya-smirnov2025-08-191-8/+10
| | | | | | | Fixed an absence of candidates at `$x = (sel#)` as of `subselect_stmt` ignorance at the `YQLConfiguration`. commit_hash:5a841d63fad104599a791953f2f89b8791816115
* YQL-19747: Fix some autocomplete issuesvitya-smirnov2025-08-193-15/+85
| | | | | | - Collect only named node definitions - Fix qualified column after WHERE commit_hash:641977921329199e645c3ac45ae8053a5d0e8b13
generated by cgit v1.2.3 (git 2.25.1) at 2025-10-20 23:50:57 +0000