summaryrefslogtreecommitdiffstats
path: root/yql/essentials/tests/sql/sql2yql
Commit message (Collapse)AuthorAgeFilesLines
...
* YQL-20751: Stabilize AsList type annotation errorvitya-smirnov2025-12-112-2/+2
| | | | | | | | | | | | | Sometimes the message is: `List items types isn't same: Tuple<Int32,Null> and Tuple<Int32,Int32>` Sometimes the message is: `List items types isn't same: Tuple<Int32,Int32> and Tuple<Int32,Null>` The problem was at `AsListWrapper`. It used `std::set<T*>` to find conflicting item types. The problem is that addresses are not stable. The `EnsureNoItemTypeConflicts` algorithm was introduced to find first type mismatch in an enumeration order, so the message stable now. commit_hash:aa98195e9dfdb1e2b8c7549d2c14cb686af313c9
* YQL-20436: Translate HAVING to YqlSelectvitya-smirnov2025-12-104-0/+103
| | | | commit_hash:11d0f62274141849a198999a9aed9af33b30ffdf
* YQL-20749 fixvvvv2025-12-093-0/+68
| | | | commit_hash:407d90acf0d13a45ae61adfe0f4948ba515ac64e
* YQL-20436: Translate COMMA/LEFT JOINs to YqlSelectvitya-smirnov2025-12-089-0/+303
| | | | | | | The translation for the RIGHT JOIN works also, but somewhy fails on type annotation. I propose to postpone this and hope for the best, because there is no RIGHT JOIN usages in the TPCH suite. commit_hash:4bdc08881e11c5a5885f04573e559c9dbbe320ad
* YQL-20436: Translate GROUP BY to YqlSelectvitya-smirnov2025-12-0819-0/+689
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds a support for a basic aggregation on `YqlSelect` and includes translation, type annotation, expansion changes. There are `count`, `min`, `max`, `sum`, `avg` and other aggregations without parameters supported. `HAVING` support will be added in a future PRs. Support for (2+)-arg aggregation is postponed, as it is not required for TPCH queries. ### Translator The `std::expected` was passed though the function call hierarchy to gracefully do not support some aggregation functions. The translation unit `select_yql_aggregation` was introduced. It implements an alternative aggregation translation to `YqlAggFactory` and `YqlAgg` callables. It reuses an `extractor` body and `aggregation_traits_factory` from a legacy code, via dirty hack with a `friend` keyword, as I decided to make minimal changes on an existing code. For a query fragment: ```yql ... Sum(body) ... ``` Resulting YQLs AST looks like this: ```yqls (YqlAggApply (YqlAggFactory '"sum") '() # <-- Options (Void) # <-- Result type stub body') ``` ### Type Annotation I followed a plan described at the https://nda.ya.ru/t/psA5Gaji7PDe9R. Besides, new `PgXXX` to `YqlXXX` renames, there were introduced `YqlAggFactoryWrapper` and `YqlAggWrapper`. First just checks argument count and types, but the second does more sofisticated work. `YqlAggWrapper` acts in 2 stages. On the first stage it will write an expression with a `TypeOf` of a `Result` to a result type stub and then calls for a transformation pipeline repetition. On the second stage result type stub is typed and so it just defines its type to be equal to a type of the stub. The tricky thing here is to contruct this expression with `Result` type. To do this it needs to instantiate an aggregation traits factory with a correct `list_type` and an `extractor`. The `list_type` is just a `(ListType (TypeOf row))`, where `row` is an `Argument` from an enclosing `lambda` at `YqlResultItem`. An `extractor` is just a `(lambda (row) body')`. The traits are imported from the `mount/lib/aggregate.yqls`, then they are beta-reduced with constructed `list_type` and `extractor` expressions. Then the `state` method is extracted and beta-reduced with a `body`. Then the `finish` method is extracted and beta-reduced with a some `state`-expression. So the resulting expression is constructed. ### Expansion The `YqlAgg` is expanded at `BuildAggregationTraits` from `yql_co_pgselect`, that is nicely integrated into `PgSelect` expansion infrastructure. There it is sufficient just to import an aggregation traits factory and beta-reduce it with `list_type` and `extractor`, that are provided by the `PgSelect`-related code. ### Refactoring Loading an `ExprNode`s from a module, so I extracted a logic to `ImportReadonly` and `ImportDeeplyCopied` at `yql_module_helpers`. commit_hash:9303f00567dd423bedc334e7c7514584f5bd8cff
* YQL-20713 better FlattenMembersvvvv2025-11-272-0/+24
| | | | commit_hash:00baeecf15bee9cd2ae069392b690f0f40bfe7fe
* YQL-20436: Support YqlSelect subqueriesvitya-smirnov2025-11-2618-5/+649
| | | | commit_hash:b1b865945f4632cf1e201e3a72026774ff151fb0
* YQL-20522 case insensitive mode for named argsvvvv2025-11-252-0/+17
| | | | commit_hash:59f56a93a03eb0bcddbf3fcebbdebe60b887b78b
* YQL-20645 fix pending files scanvvvv2025-11-252-0/+28
| | | | commit_hash:255a34401a5949af770f899fb3c291dd8f639fad
* YQL-20673 Graph cycle detectorvvvv2025-11-255-0/+78
| | | | commit_hash:a8a7fef19b3cc3445a177192748b987ff6b80786
* YQL-20436: Fix YqlSelect anon sourcevitya-smirnov2025-11-243-9/+59
| | | | | | | | | | | | | | The problem was that in `type_ann_pg::PgSetItemWrapper` when `optionName == "sort"`, a new input with an empty name was added, so it conflicts with an existing empty-named input, so somewhy `ValidateSort` was always detecting sort changes. Added regression tests. Warning. Actually this is a non-trivial bug in `PgSelect` type annotation and this patch hides it. See https://nda.ya.ru/t/2VPHrfH97NEK6s. commit_hash:83a9398af9bab838de3df56dc61cabb2c7dac5cf
* YQL-20436: Fix YqlSelect alias collision detectionvitya-smirnov2025-11-243-3/+19
| | | | | | Fixed the test yql/essentials/tests/sql/suites/ select_yql/projection_alias_collision.yql. commit_hash:e868b9ae79cb41378a1a563c1217ba633013c079
* [Optimizers] Add sqlin with nothing YQL-20627deniskhalikov2025-11-193-0/+92
| | | | commit_hash:8371a21ae0b8f907eb411c0fbe706f4cefd424b4
* YQL-20436: Translate VALUES statement to YqlSelectvitya-smirnov2025-11-142-4/+22
| | | | commit_hash:e2c8bcd9d8fd5ceb294d1fb3c6b1d15993dbb8f9
* YQL-20436: Add YqlSelect to differential testingvitya-smirnov2025-11-138-42/+218
| | | | | | | | | | | | | | | | | | | | | | Added `AutoYqlSelect` differential testing target for `minirun` and `yt_file`. It helped to explore at least the following issues: 1. Not set `DeriveColumnOrder` (fixed) 2. Absent `Unordered` decorator (fixed) 3. Unsupported not fully-qualified `VALUES` (fixed) 4. Premature cluster check leading to error instead of an unsupported status (fixed) 5. Invalid column name generation (fixed) 6. Absent top-level select column order (postponed) 7. Absent error on column name collision (postponed) 8. Other issues unexplored due to noise from bullet 6. As tests are failing `AutoYqlSelect` is commented out until `YqlSelect` is ready. Also added some missing tests for `LIMIT`/`OFFSET` and `WHERE`. commit_hash:70424260914acff8275f3544e2ad7a99789f7620
* Intermediate changesrobot-piglet2025-11-133-8/+8
| | | | commit_hash:aaaacc50ee0fa3722dcc5dd84f0a4fbcefd05bde
* YQL-20436: Translate JOIN to YqlSelectvitya-smirnov2025-11-065-3/+150
| | | | | | | | | The support is actually limited because asterisk is not supported, because of non-trivial to implement YQL semantics where same-named columns at ON clause are deduplicated, will create an task for it. Supported JOIN, LEFT/RIGHT JOIN, CROSS JOIN. commit_hash:789a09fe45f40cb8171badd22ac6c06b366bdd03
* YQL-19673: Realizationmrlolthe1st2025-11-025-0/+95
| | | | commit_hash:68fcce49bf7355e49569a96ff7ad2faa537c563e
* YQL-20564 fixed singular types availabilityvvvv2025-10-242-0/+18
| | | | commit_hash:59fbd02c0cb455f6670e3e85ea9f5436f4e4f7c8
* YQL-20498 more pg & missing functionsvvvv2025-10-234-3/+48
| | | | commit_hash:18698e72d2e575212150eb234da6f22dc69a9d96
* YQL-20498 more pg & missing functionsvvvv2025-10-213-0/+37
| | | | commit_hash:add2f391888f47fb6a57a3a985cb0c391d9054ac
* YQL-20436: Extend YqlSelect translation supportvitya-smirnov2025-10-214-21/+91
| | | | | | | | | | | New supported SELECT-features: - `SELECT * FROM ...` - `SELECT ... FROM cluster.table` - `SELECT ... FROM (SELECT ...)` - `SELECT ... FROM ... WHERE expr` - `SELECT ... FROM ... ORDER BY sort+` - `SELECT ... FROM ... LIMIT expr OFFSET expr` commit_hash:9c6981a26050b52ed1943a325476164673806256
* YQL-20555 nullifvvvv2025-10-212-0/+16
| | | | commit_hash:4a2e71c935ef62a45d8fd13f180e81464a446441
* YQL-20498: SimplePg infrastructurevvvv2025-10-205-0/+98
| | | | commit_hash:309ca86c90175be5a1e016c3e6a5160ba04034e6
* YQL-20520 concat (w/o runtime)vvvv2025-10-202-0/+20
| | | | commit_hash:0b092481cfb4813c26c165c17f6d05a9f1f88481
* YQL-20339 composite types in grammarvvvv2025-10-172-3/+5
| | | | commit_hash:2fc2878449f90c948e0cd75a3c1d59f07d660620
* YQL-20436: Translate `SELECT .. FROM VALUES ..` to `YqlSelect`vitya-smirnov2025-10-148-0/+160
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Current `SQLv1` translation produces relatively low level `YQLs` constructions that are hard to match during later optimizations, for example, subqueries unnesting. Also it assumes that expressions are only depend on corresponding source row, which is not true for correlated subqueries. Both limitations blocks currelated subqueries implementation. Although the problem exists for `SQLv1`, it is already solved for `PG` syntax. There PostgreSQL-produced AST is converted to special `YQLs` "bulk select" node called `PgSelect`. It is more declarative and expanded later than translation. This fact helped to support correlated subqueries, which with `PgSelect` are type-checkable and decorrelatable (not generally, but heuristically). This patch is the first step forward to "bulk select" translation for `SQLv1`. As there are a lot of code already written for `PgSelect` and `PG` and `SQLv1` relatively similar, I decided to rebrand `PgSelect` into more general `SqlSelect` node. It seems that for some near future goals `PgSelect` should be enough. There was 2 problems solved: 1. `PgSelect` comes with `OrderedColumns` by default and its implementation tightly coupled with it. 2. `PgSelect` does Pg type casts. This patch contains following changes: - Added pragma `YqlSelect = 'disable' | 'auto' | 'force'` - Added `YqlSelect` (`PgSelect` alias) translation - Changed `PgSelect` wrappers to support `YqlSelect` - Changed `PgSelect` expanders to support `YqlSelect` commit_hash:8a55d63e06c22592b2029dd260bbd259194e92dc
* YQL-20072: Init pragma Layer and yt.LayerCachesmrlolthe1st2025-10-102-0/+27
| | | | commit_hash:9199da15c695cf5c8a492750a8bcfbb0f31c0b34
* YQL-20116: Introduce ANTLR4 ambiguity detectionvitya-smirnov2025-10-061-0/+1
| | | | | | - Ignored 2 known ambiguities. - Tested tools. commit_hash:9e29bb2f876dabc68293b3e5c26a470d373506ae
* Intermediate changesrobot-piglet2025-10-031-136/+136
| | | | commit_hash:c3c929f4005b33d2fddb492bbc2c553125d9ad40
* YQL-20307: Support inline subqueriesvitya-smirnov2025-10-0113-2/+1169
| | | | | | | | | | | - Alter grammar to support inline subqueries. - Support inline subqueries in `sql/v1` (translator). - Introduce `sql/v1/proto_ast/parse_tree.h` for reusable parse tree predicates. - Support inline subqueries in `sql/v1/format`. - Support inline subqueries in `sql/v1/complete`. - Add some SQL-tests. - Pass all tests. commit_hash:075b2240778d071e1c7542f912d3cc83019ef849
* Intermediate changesrobot-piglet2025-10-014-10/+40
| | | | commit_hash:a9a24e2156ccebef387f026b91efcd5628389146
* YQL-20339 linear checkervvvv2025-09-2914-19/+223
| | | | | | | usage types commit_hash:6214c6a10f057439898414592a7b70a178ef14a1
* YQL-20461: Normalize table function namesvitya-smirnov2025-09-291-1/+1
| | | | | So now both `AsTable` and `AS_TABLE` work. commit_hash:ae8c33f44e7b10de02a9d979373f4b0602ff0b12
* YQL-20339 merge blocks optimizervvvv2025-09-265-0/+63
| | | | | init commit_hash:fee9a86a1f95e9271bf49205dad4b6eb0b934dec
* YQL-20226: Wrap unpickle with try catchatarasov52025-09-262-0/+18
| | | | commit_hash:96e8a09d23befb758e60a6e01f691868331af968
* YQL-20339 expr high level funcsvvvv2025-09-225-0/+68
| | | | | init commit_hash:479f16e9cd255b616573d811910403290f9ef656
* YQL-20339 expr low level funcsvvvv2025-09-2216-0/+360
| | | | | init commit_hash:ededd246fa929931de6e89fd5f809157d9fe4d16
* YQL-20339 block function for scoped argsvvvv2025-09-192-0/+15
| | | | | без langver, т.к. может быть использована для починки оптимизаторов, где при раскрытии List-ов из воздуха рождается Iterator а потом все заканчивается Collect commit_hash:21cb9b9454cb255b476d50bdb8c5db150cd773fd
* SeqMode supportudovichenko-r2025-09-186-15/+20
| | | | commit_hash:3d0e8bfb46470e6ac36b0d198cfa1723c665f643
* YQL-20339 expr types & reflectionvvvv2025-09-183-0/+38
| | | | | init commit_hash:1c72053b3785a26cfde418f28a9d054b5a624627
* YQL-20423 pending files should postpone all outer evaluationsvvvv2025-09-154-0/+61
| | | | commit_hash:70c91e881d5f8126d3d1c68a5816f440295330da
* Fix NormalizeEqualityFilterOverJoin with optionalsaneporada2025-09-152-0/+32
| | | | commit_hash:dbf1a2fd200d2699cbc984fd4da357fb79ca3bd9
* Improve optimizer for logical ops over Justaneporada2025-09-092-0/+29
| | | | commit_hash:f29316d37138b6555b2d09a4c67aeef3b447ea60
* Fix 'Distributive law for OR' optimizeraneporada2025-09-092-0/+30
| | | | commit_hash:832d6963d7f12d5ff8da09d883801e65857acaca
* YQL-20258 sql syntaxvvvv2025-09-0824-142/+100
| | | | commit_hash:c2440ce49cc8c7637134ddf8081b32251ff75fd4
* YQL-20258 more tests & refactorvvvv2025-09-082-0/+20
| | | | commit_hash:031b3f5c4215bc9c66d0cea9c8dd8d69a1ec1627
* YQL-20234: Remove predicate argument on middle aggregation phasesvitya-smirnov2025-09-0520-0/+2887
| | | | | | | | | | | Also added tests for: - max_by - agg_list - some - avg_if - sum_if - count_if commit_hash:0da4f3b9b1767850e65c914b727bc362cf4cc125
* YQL-20375: Fix emit of SqlCall external typesimunkin2025-09-041-9/+9
| | | | | Follows up ydb-platform/ydb@34fcd78a2d18d9846fd68339c8e240702857e1e5 commit_hash:08520ae8b837c69f6b92e575882da4786aa441fd
* YQL-20367: Add DisableExceptIntersectBefore202503 feature flagvitya-smirnov2025-09-044-17/+57
| | | | commit_hash:a8645f7215087df3309048de31a7f0b3dc802a11
generated by cgit v1.3 (git 2.53.0) at 2026-06-10 22:35:45 +0000