YQL-19747 Detect a token at the caret position

When I tried to implement a folder and object names completion at `ID_QUOTED` I faced with a problem, that I actually can't detect, that cursor is at `ID_QUOTED` token because `TCompletionInput::Text` it was cut until the `TCompletionInput::CursorPosition`, therefore at input ``` SELECT * FROM `#` ``` prefix was ``` SELECT * FROM `# ``` and then lexer failed.

While we actually want tokenize the whole current statement, `C3` still needs to receive a prefix as input. I tried to tokenize the whole statement and then on input `SELECT Optional<#>` got nothing because `<>` is solid token in the `SQL`. The only way to fix it I found is to cut a query to prefix until the cursor position.

BTW, current implementation is not so efficient as we tokenize the input multiple times. Especially `SplitQueryToStatemnts` seems heavy. In future we anyway will parse the whole input so will need to design APIs to receive ready token streams to do statements splitting, for example, just not to do the work twice.

![image](https://github.com/user-attachments/assets/114804d3-f311-4a46-be84-8ed4650bc9dd)

So I introduce you the following changes

- [x] Select the whole current statement, not just prefix.
- [x] Find the token at caret and output no candidates when caret is at `STRING_VALUE`, `DIGIGTS` and so on.
- [x] Change `C3` wrapper interface to take `TCompletionInput` to hide an implementation detail that it runs on cut prefix.
- [x] `#` annotated queries in unit tests.
- [x] Detect `CaretTokenPosition` -- if is it enclosed with a token or between two.
- [x] Ensure that `maxErrors` in `ILexer::Tokenize` is positive. Just a tiny bugfix.

---

- Related to https://github.com/ytsaurus/ytsaurus/pull/1209
- Related to https://github.com/ydb-platform/ydb/issues/9056
- Related to https://github.com/vityaman/ydb/issues/14

---

Pull Request resolved: https://github.com/ytsaurus/ytsaurus/pull/1225
commit_hash:a434b9888ec8a7356247d63d9f1420e256ae4fca

0 files changed, 0 insertions, 0 deletions