summaryrefslogtreecommitdiffstats
path: root/yql/essentials/sql
Commit message (Collapse)AuthorAgeFilesLines
...
* Intermediate changesrobot-piglet2026-01-294-54/+58
| | | | commit_hash:3fc8faa25000417fd21875d20df0ff007764c7ea
* YQL-20837: Respect original newlines at formatterlukashevich842026-01-262-13/+110
| | | | commit_hash:862362f17e02a0d50b3a568aedc0c490c05ea9a7
* YQL-20095: Enable google-explicit-constructorvitya-smirnov2026-01-2326-51/+68
| | | | commit_hash:4d77ad10fd4db303459ec4e45e139967c7fc8196
* YDB fultext indexes: fulltext_plain & fulltext_relevanceazevaykin2026-01-223-7/+14
| | | | commit_hash:e8895f6fdbcfb589ae1947d278fa06c937bc7674
* support streaming disposition in streaming queriesgrigoriypisar2026-01-219-102/+201
| | | | commit_hash:4a122e23fcc9c4b652ec7eae7cb546501a33748c
* add fulltext builtin functions KIKIMR-24587gvit2026-01-201-0/+6
| | | | commit_hash:0450af366e38b009b621d0922033d753049da104
* YQL-13448: Introduce window RANGE logicatarasov52026-01-196-6/+19
| | | | | | | | Introduce window RANGE logic. Зеркальный пр с включение фичи путем инлайнинга в коде <https://nda.ya.ru/t/p2qAEoNq7SNwUJ> Зеркальный пр где я провожу perf измерения <https://nda.ya.ru/t/7UdlI38n7SNwUL> - в нем я описал результаты commit_hash:2626d7d6b77f1ccb31e395d974a2beaa60f27a97
* Support DROP ... IF EXISTS & CREATE ... IF NOT EXISTS features.Anton Romanov2026-01-162-8/+57
| | | | | | | | | | | | | | | Type: feature Component: query-tracker --- Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1544 Co-authored-by: aneporada <[email protected]> Co-authored-by: aneporada <[email protected]> Co-authored-by: aneporada <[email protected]> Co-authored-by: Tony-Romanov <[email protected]> commit_hash:f6cb810269ec24554c570ebcd0d3e5a94fb991ac
* YQL-20095: Enable modernize-use-overridevitya-smirnov2026-01-1317-91/+91
| | | | | | | | | | | | | | | | | | | The check `modernize-use-override` is useful as it: 1. Removes redundant `override` when it is used with a `final`. 2. Removes redundant `virtual` or replace it with a `override`. So it is more clean for a reader, that overriding happens. It is enabled not out of order, as it is just an alias to `cppcoreguidelines-explicit-virtual-functions`. I also decided to switch a strategy of enabling checks. Now I will enable only a single rule with a single PR and prefer rules with a non-breaking autofix. In the new year with new linter checks! 🎄 commit_hash:e6e233baa90b31e5f65e11837546690c47f71ab5
* YQL-20530: Fix a fatal table alias ending with '/'vitya-smirnov2025-12-302-2/+50
| | | | | | | | | | | I found a case, where the completion engine dies, when a table alias ends with a `/` character. The general problem is that the huge part of NameService stuff is built on a fact that `/` is used as path delimeter. So when it is used at a name, that will be unsafely contatenated somewhere with an other path, strange things can happen. commit_hash:69be32117450bfebf9609326f3a9ccef12985d61
* YQL-20436: Fix free names on YqlSelect named nodevitya-smirnov2025-12-304-6/+25
| | | | | | | The only place, where external column references are unavailable is a source (after `FROM`), so to disable it was decided to wrap a named node content with a `(SELECT * FROM ...)`. commit_hash:d491739c02e5e18de3294f2d6df1f93060b26fe8
* YQL-20436: Support Named Nodes on YqlSelectvitya-smirnov2025-12-2712-98/+464
| | | | | | | | | | | | | | | | This patch unlocks the last TPC-H query Q15, but has a couple of known issues. The first issue is that a named select with external column references will work at a substitution place. To fix this I will introduce a new `SqlSelect` option called `unknowns_disallowed`. I guess there is no such issue in a pgSQL syntax, because there `WITH CTE` subqueries can be referenced only from a `FROM` at a syntax level (CTE is not a column reference). The second issue is that `AutoYqlSelect` is broken, when `YqlSelect` is used as a legacy `ISource`. commit_hash:2c83192921d1f50899063b3b22461b1bf111b47e
* multihopping: add FarFuture statistics and limitsyumkam72025-12-263-17/+163
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ~~TODO:~~ [X] uncomment RuntimeVersion checks [X] rethink interval parsing (switch to ISO/"PT10S") [X] remove debug prints [X] rfc: при задании любого из параметров в HoppingWindow, для неуказанных параметров используются умолчания; возможно, лучше брать из `config.flags`? [X] rfc: вообще, нужны ли `config.flags`? -> дропаем [X] добавить тесты (ut) нового синтаксиса `HoppingWindow()` yql/essentials/sql/v1/sql_ut_common.h Лимиты и политику превышения можно задавать опциональные именованные параметры в HoppingWindow: ``` SELECT ... GROUP BY HoppingWindow(`time`, 'PT1S', 'PT10S', 10 AS SizeLimit, -- также допускается указание специального значения "max" -- лимит при этом выключается "PT1H" AS TimeLimit, -- также допускается указание специального значения "max" -- лимит при этом выключается "adjust" AS LatePolicy -- возможно указывать также "drop" "drop" AS EarlyPolicy) -- возможно указывать также "close" и "adjust" ``` Подробнее см. тикет Изменение contrib/ydb можно тащить отдельно (без него опции не будут работать, но ничего не сломается) commit_hash:fe165d77aa3aa923770accb805177c6b1eba4cb0
* YQL-20436: Support trailing comma at VALUES and ASvitya-smirnov2025-12-263-5/+24
| | | | commit_hash:ac7d78346879d05bd141d6b9b207cba179a87165
* YQL-20436: Set top-level column order on YqlSelectvitya-smirnov2025-12-231-0/+34
| | | | | | | | | | | This patch fixes `YqlSelect` top-level `SELECT` statement column order. Despite a column order is not specified in the YQL, users could depend on it, so we need to keep an order specified literally. @vvvv suggested to infer this column order on a YQLs type annotation, but it seems so much harder and as we have not so much time, I decided to keep things simple. commit_hash:1f8cadabf772edd13e7044af7878097d88a5c1cd
* YQL-20436: Support anonymous table on YqlSelectvitya-smirnov2025-12-234-8/+47
| | | | | | | As a temporary workaround for TPC-H Q15 it was decided to materialize a result to a temporary table. In future named nodes will be implemented. commit_hash:090b1b2263e24a2b898f48a15e71208e641db9ab
* YQL-20436: Fix column disambiguation on YqlSelectvitya-smirnov2025-12-222-6/+29
| | | | commit_hash:3777f5e967bf5e2ee1cd67db3ab6d6a2f9225934
* YQL-20436: Fix column name on YqlSelect CROSS JOINvitya-smirnov2025-12-192-1/+21
| | | | commit_hash:35370785d743212d174a7b3607d8a74faf2239e9
* Add transfer metrics_level optionkomels2025-12-172-0/+102
| | | | commit_hash:c9edfe6605d611af37ac9d15c55a8eab57659df8
* YQL-20349: Improve errors for ALTER TABLE and CREATE TABLE IF NOT EXISTSlucius2025-12-152-12/+32
| | | | | | | | | | | | | Было: * `Unsupported Write! mode: alter` * `Unsupported Write! mode: create_if_not_exists` Стало: * `ALTER TABLE is not supported for yt provider` * `CREATE TABLE IF NOT EXISTS is not supported for yt provider` commit_hash:c5423d23289ebbc3f256db8853ca1f188e29e75f
* YQL-20436: Support Agg(DISTINCT x) for YqlSelectvitya-smirnov2025-12-145-3/+36
| | | | | TODO commit_hash:c1d59638bbbf9ce20c8e1bcc08da82fecdb2fe17
* YQL-20436: Fix IN on a tuple on YqlSelectvitya-smirnov2025-12-133-2/+27
| | | | | | Before this patch only subquery after IN was handled properly. Now there can be a list or tuple, for example. This unlocks TPC-H Q12. commit_hash:34145d8f935ab7e5c5119d2b0e433da4846323ef
* YQL-20436: Check for a mandatory AS on YqlSelectvitya-smirnov2025-12-132-7/+61
| | | | commit_hash:c1ca375cc6fff02162cff68793b2169ac18adb56
* YQL-20436: Fix CAST (x AS T) on YqlSelectvitya-smirnov2025-12-132-10/+22
| | | | | | | Fixes an error `Name not found: row` of TPC-H Q07, Q08, Q09, but there is an other error with some other reason. The problem was that a `Member` was emitted instead of `YqlColumnRef`. commit_hash:c7dbdbe2d526bb513330d50fc7c607389fb0a23e
* Add warnings for invalid placed DISCARDsanely-d2025-12-124-4/+106
| | | | | | | Add warnings if discard is not on the top level for YDB DISCARD SELECT support. Issue on github: [#26008](https://github.com/ydb-platform/ydb/issues/26008) commit_hash:37cb74e18ee458e6bf5984be0013ac825e2e4644
* Require 2025.05 for YT DROP VIEWaneporada2025-12-101-0/+8
| | | | commit_hash:8e6917a4f59e8c4007e24acb20736e6fbd069e7a
* YQL-19017: Remove TTranslationSettings::Antlr4Parservitya-smirnov2025-12-1028-274/+225
| | | | | Then I will remove an `Antlr4Parser` usages from the YDB. commit_hash:0183482e46de023f71698e03db9a0c51ef10fe98
* Require 2025.05 for YT CREATE VIEWaneporada2025-12-103-12/+59
| | | | commit_hash:50871c6094dcbdbfba6ea59c691157e623fe7f4e
* YQL-20436: Translate HAVING to YqlSelectvitya-smirnov2025-12-104-2/+53
| | | | commit_hash:11d0f62274141849a198999a9aed9af33b30ffdf
* YQL-20436: Translate COMMA/LEFT JOINs to YqlSelectvitya-smirnov2025-12-087-21/+228
| | | | | | | The translation for the RIGHT JOIN works also, but somewhy fails on type annotation. I propose to postpone this and hope for the best, because there is no RIGHT JOIN usages in the TPCH suite. commit_hash:4bdc08881e11c5a5885f04573e559c9dbbe320ad
* YQL-20436: Translate GROUP BY to YqlSelectvitya-smirnov2025-12-0823-318/+770
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds a support for a basic aggregation on `YqlSelect` and includes translation, type annotation, expansion changes. There are `count`, `min`, `max`, `sum`, `avg` and other aggregations without parameters supported. `HAVING` support will be added in a future PRs. Support for (2+)-arg aggregation is postponed, as it is not required for TPCH queries. ### Translator The `std::expected` was passed though the function call hierarchy to gracefully do not support some aggregation functions. The translation unit `select_yql_aggregation` was introduced. It implements an alternative aggregation translation to `YqlAggFactory` and `YqlAgg` callables. It reuses an `extractor` body and `aggregation_traits_factory` from a legacy code, via dirty hack with a `friend` keyword, as I decided to make minimal changes on an existing code. For a query fragment: ```yql ... Sum(body) ... ``` Resulting YQLs AST looks like this: ```yqls (YqlAggApply (YqlAggFactory '"sum") '() # <-- Options (Void) # <-- Result type stub body') ``` ### Type Annotation I followed a plan described at the https://nda.ya.ru/t/psA5Gaji7PDe9R. Besides, new `PgXXX` to `YqlXXX` renames, there were introduced `YqlAggFactoryWrapper` and `YqlAggWrapper`. First just checks argument count and types, but the second does more sofisticated work. `YqlAggWrapper` acts in 2 stages. On the first stage it will write an expression with a `TypeOf` of a `Result` to a result type stub and then calls for a transformation pipeline repetition. On the second stage result type stub is typed and so it just defines its type to be equal to a type of the stub. The tricky thing here is to contruct this expression with `Result` type. To do this it needs to instantiate an aggregation traits factory with a correct `list_type` and an `extractor`. The `list_type` is just a `(ListType (TypeOf row))`, where `row` is an `Argument` from an enclosing `lambda` at `YqlResultItem`. An `extractor` is just a `(lambda (row) body')`. The traits are imported from the `mount/lib/aggregate.yqls`, then they are beta-reduced with constructed `list_type` and `extractor` expressions. Then the `state` method is extracted and beta-reduced with a `body`. Then the `finish` method is extracted and beta-reduced with a some `state`-expression. So the resulting expression is constructed. ### Expansion The `YqlAgg` is expanded at `BuildAggregationTraits` from `yql_co_pgselect`, that is nicely integrated into `PgSelect` expansion infrastructure. There it is sufficient just to import an aggregation traits factory and beta-reduce it with `list_type` and `extractor`, that are provided by the `PgSelect`-related code. ### Refactoring Loading an `ExprNode`s from a module, so I extracted a logic to `ImportReadonly` and `ImportDeeplyCopied` at `yql_module_helpers`. commit_hash:9303f00567dd423bedc334e7c7514584f5bd8cff
* Rename Upsert to Replacempereskokova2025-12-063-27/+18
| | | | | То, что было `UPSERT`-ом на самом деле работает как `REPLACE` - возвращаю справедливость. commit_hash:3b4cdd5bcf3957f68be4ab648e399e8dc923d48f
* YQL-20519: Fix multiline ID_QUOTED highlightingvitya-smirnov2025-12-031-1/+5
| | | | | | | | There were patches to fix multiline `STRING_VALUE`s, but `ID_QUOTED`, was untouched, but actually is able be multiline, so this change fixes it. Now unclosed `ID_QUOTED` highlightes the whole suffix of a source until the end or closing `BACKTICK`. commit_hash:d1262172c798fd4b06cc39980cceb74681d5dd40
* YQL-20728 new lang vervvvv2025-12-031-2/+2
| | | | commit_hash:535480c8aa341720faec02421d8efdb7b698eda9
* New statement TRUNCATE TABLEflown4qqqq2025-12-0210-1/+168
| | | | commit_hash:f6dfab4a7add0379cacefb943c702ae065750654
* KIKIMR-24294: Use TablePrefixPath at ALTER EXTERNAL DATA SOURCEyurikiselev2025-12-022-1/+41
| | | | | TablePrefixPath pragma value will be used in `ALTER EXTERNAL DATA SOURCE` command (the altered object itself and paths of secrets) commit_hash:bc6cf96f2ca431639beacff596d39f5ce92220c9
* YQL-20436: Fix YqlSelect LangVersionvitya-smirnov2025-12-014-30/+43
| | | | | | | | Before YqlSelect LangVersion was hardcoded to 2025.04, but should be set to Max until release. Also I tested that YqlSelect is unavailable on 2025.04 when 2025.05 is added to yql_langver_list.inc. commit_hash:0e5e880b8f4ad5a54dc98b5fe21450e98902aa07
* YQL-20714 linting for available udf functionsvvvv2025-11-282-0/+17
| | | | commit_hash:7e79078f8b77378282f9c77735204bb5452daa85
* YQL-20713 better FlattenMembersvvvv2025-11-271-2/+4
| | | | commit_hash:00baeecf15bee9cd2ae069392b690f0f40bfe7fe
* KIKIMR-24294: Add new (*_PATH) settings for secrets in SQL commandsyurikiselev2025-11-277-172/+871
| | | | | | | | | Secrets are being reworked in YDB. Previously, they were used in async replication, transfer, and external dat source commands via settings like `..._NAME`. The new secrets will work with the new settings (for more details, see [here](https://nda.ya.ru/t/jqkP2-PX7NLLWp "title")). Here, we add their support when parsing the corresponding SQL commands. 1. Add new secrets paired settings with the `..._PATH` prefix to all old settings. 2. Add parameter usage checks: they must be used in a mutually exclusive manner, meaning that only old settings or only new ones can be used in one SQL command. 3. Add TablePathPrefix pragma values to the new parameter values, since the new settings name schema objects. commit_hash:d7db87d3b07e1b3ccf8981b241f0489db3506924
* YQL-20664: Improve Udf error messagevitya-smirnov2025-11-262-4/+4
| | | | | | A return value of `Udf` callable can not be used as an expression. At an YQLs it should be passed to `NamedApply`. commit_hash:202130399b528ca370a587f12f6e44b671590e6a
* YQL-20436: Support YqlSelect subqueriesvitya-smirnov2025-11-268-63/+427
| | | | commit_hash:b1b865945f4632cf1e201e3a72026774ff151fb0
* Fix "alter topic reset"qyryq2025-11-261-1/+1
| | | | commit_hash:e1e3549ea2039278d517397eab567e8e6c9b7ac8
* YQL-20086: Enable arcadia and bugprone checksvitya-smirnov2025-11-257-21/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch enables more Clang Tidy checks for the `yql/essentials` project. I chosen the policy to enable all checks by default because - It will keep us updated when new checks are become available in the DevTools whitelist. - You never know what check will be useful, even if it was not fired now. Why some checks from the https://nda.ya.ru/t/8QnKCVIO7NWhBD were disabled? - `arcadia-taxi-` checks are `userver`-specific. - `bugprone-bool-pointer-implicit-conversion` is not applicable to our codebase as we heavily used to it. - `bugprone-branch-clone` triggered on a code that is not obviously rewrittable and not so poor. - `bugprone-dynamic-static-initializers` are heavily used in our codebase especially in the translator. Static variables with dynamic initialization are so common, so refactor them is too hard and requires huge code rewritting to deliver some hardcoded configuration values via ctors, having them in a one place. - `bugprone-exception-escape` is useful check for programs where the care of exception safety is taken, but we a lot of places where exception can be thrown from the destructor or move-operator. I left a TODO in the `.clang-tidy` to fix this. Maybe will create a ticket. - `bugprone-lambda-function-name` appears when we throw an exception from the lambda. I think, that this is okay and do not see any options to fix it. - `bugprone-macro-parentheses` had false positives at `YQL_ENSURE` second argument and also broke some code, by surrounding `A` at `template <class A>`. - `bugprone-misplaced-widening-cast`, `bugprone-narrowing-conversions`, `bugprone-signed-char-misuse` are disabled because checking arithmetics correctness is hard and non-trivial. I am not sure that it worth spending a lot of time on it as everything seems to work well :o) - `bugprone-reserved-identifier` was triggered on some harmless identifier in computation, so I decided just to turn this off, rather than argue with someone. - `bugprone-suspicious-enum-usage` happend on Arcadia HTTP client usage. Do not think that it is worth to argue with Arcadia HTTP client code owners. - `bugprone-suspicious-memory-comparison` happened in some compuration test. I just decided to trust low-level code authors. - `bugprone-too-small-loop-variable` seems not really a problem and lets you safe some symbols on `static_cast`s. - `bugprone-sizeof-expression` had so many false positives at computation. - `bugprone-unchecked-optional-access` a good check, but optionals are dereferenced everywhere in our codebase. Of course, in a perfect world near each of such dereference should be an assertion with comment why it is a valid action, because when static analysis can not infer that the value is non-null, then it is not an obvious thing for a human also. But we a too far from such code quality now. - `bugprone-undefined-memory-manipulation` is interesting. It argued on some `memcpy` at computation. There was some `UnboxedValues` array and this objects have some logic of memory marking and releasing under an `NDEBUG`. I added a TODO to check this. Maybe will create a ticket. - `bugprone-unhandled-self-assignment` has false positives on non-owning objects such as iterators. commit_hash:c0c0de6b157bf095eb681bdea55ada47845de814
* YQL-20436: Fix YqlSelect anon sourcevitya-smirnov2025-11-241-2/+3
| | | | | | | | | | | | | | The problem was that in `type_ann_pg::PgSetItemWrapper` when `optionName == "sort"`, a new input with an empty name was added, so it conflicts with an existing empty-named input, so somewhy `ValidateSort` was always detecting sort changes. Added regression tests. Warning. Actually this is a non-trivial bug in `PgSelect` type annotation and this patch hides it. See https://nda.ya.ru/t/2VPHrfH97NEK6s. commit_hash:83a9398af9bab838de3df56dc61cabb2c7dac5cf
* YQL-20666: Fix failed sql/v1 buildvitya-smirnov2025-11-211-2/+2
| | | | commit_hash:ac8dcb47057ba9dd9a09c07fac3dfa35c325b376
* YQL-20436: Use std::expected at sql_expressionvitya-smirnov2025-11-2117-392/+615
| | | | | | | | | | | | | | | | | | | | | | | | This patch introduces a new `TSQLV1Result` for error-handling in YQL translator. A nullable `TNodePtr` with 2 states was sufficient for error reporting, but `YqlSelect` feature fallback mode requires one more state `YqlSelectUnsupported`. `TSQLV1Result` is used at `YqlSelect` sub-translator and `sql_expression`. The last is needed to support `YqlSelect`-subqueries. For an error-handling `std::expected` is used with `TNonNull` pointer decorator to prevent implicit casts and ensure safety. For a legacy code interoperability `Wrap` and `Unwrap` methods where added. They convert `nullptr` to `ESQLV1Error::Basic` Unwrap is used in places, where `TSqlExpression` is called without `YqlSelect` configuration, so `YqlSelectUnsupported` is impossible. Also I noticed some strange error ignorance at JSON API translation and added `FIXME`s. commit_hash:47c1e05ef02574bfd3af4712bd8bfa652ab7868b
* Watermarks: YQL: watermarks sql 2.0vokayndzop2025-11-214-9/+62
| | | | commit_hash:2248d808136868c425ce9ceb05681c37fd108bb8
* YQL-20086: Add yql/essentials Clang Tidy configvitya-smirnov2025-11-2126-112/+118
| | | | | | | | | This patch introduces an extension configuration for the Clang Tidy. It is merged with Arcadia Clang Tidy configuration. To begin with, more checks for identifiers naming are added. Documentation: https://nda.ya.ru/t/AhbDZbiF7MKe3M. commit_hash:3481da4c8df0a4d23a991d4a660ae050d2dc5d33
* YQL-10945: Insert into dyntable supportmpereskokova2025-11-153-2/+22
| | | | commit_hash:9f9d825b0f0ce0eeca9a3f94faf62d1c0ba9f544
generated by cgit v1.3 (git 2.53.0) at 2026-07-26 09:57:22 +0000