YQ Connector:Use docker-compose in integrational tests

1 files changed, 924 insertions, 0 deletions

diff --git a/contrib/libs/pcre2/README b/contrib/libs/pcre2/README
new file mode 100644
index 0000000000..c88acff8ba
--- /dev/null
+++ b/contrib/libs/pcre2/README
@@ -0,0 +1,924 @@
+README file for PCRE2 (Perl-compatible regular expression library)
+------------------------------------------------------------------
+
+PCRE2 is a re-working of the original PCRE1 library to provide an entirely new
+API. Since its initial release in 2015, there has been further development of
+the code and it now differs from PCRE1 in more than just the API. There are new
+features, and the internals have been improved. The original PCRE1 library is
+now obsolete and no longer maintained. The latest release of PCRE2 is available
+in .tar.gz, tar.bz2, or .zip form from this GitHub repository:
+
+https://github.com/PCRE2Project/pcre2/releases
+
+There is a mailing list for discussion about the development of PCRE2 at
+pcre2-dev@googlegroups.com. You can subscribe by sending an email to
+pcre2-dev+subscribe@googlegroups.com.
+
+You can access the archives and also subscribe or manage your subscription
+here:
+
+https://groups.google.com/g/pcre2-dev
+
+Please read the NEWS file if you are upgrading from a previous release. The
+contents of this README file are:
+
+  The PCRE2 APIs
+  Documentation for PCRE2
+  Contributions by users of PCRE2
+  Building PCRE2 on non-Unix-like systems
+  Building PCRE2 without using autotools
+  Building PCRE2 using autotools
+  Retrieving configuration information
+  Shared libraries
+  Cross-compiling using autotools
+  Making new tarballs
+  Testing PCRE2
+  Character tables
+  File manifest
+
+
+The PCRE2 APIs
+--------------
+
+PCRE2 is written in C, and it has its own API. There are three sets of
+functions, one for the 8-bit library, which processes strings of bytes, one for
+the 16-bit library, which processes strings of 16-bit values, and one for the
+32-bit library, which processes strings of 32-bit values. Unlike PCRE1, there
+are no C++ wrappers.
+
+The distribution does contain a set of C wrapper functions for the 8-bit
+library that are based on the POSIX regular expression API (see the pcre2posix
+man page). These are built into a library called libpcre2-posix. Note that this
+just provides a POSIX calling interface to PCRE2; the regular expressions
+themselves still follow Perl syntax and semantics. The POSIX API is restricted,
+and does not give full access to all of PCRE2's facilities.
+
+The header file for the POSIX-style functions is called pcre2posix.h. The
+official POSIX name is regex.h, but I did not want to risk possible problems
+with existing files of that name by distributing it that way. To use PCRE2 with
+an existing program that uses the POSIX API, pcre2posix.h will have to be
+renamed or pointed at by a link (or the program modified, of course). See the
+pcre2posix documentation for more details.
+
+
+Documentation for PCRE2
+-----------------------
+
+If you install PCRE2 in the normal way on a Unix-like system, you will end up
+with a set of man pages whose names all start with "pcre2". The one that is
+just called "pcre2" lists all the others. In addition to these man pages, the
+PCRE2 documentation is supplied in two other forms:
+
+  1. There are files called doc/pcre2.txt, doc/pcre2grep.txt, and
+     doc/pcre2test.txt in the source distribution. The first of these is a
+     concatenation of the text forms of all the section 3 man pages except the
+     listing of pcre2demo.c and those that summarize individual functions. The
+     other two are the text forms of the section 1 man pages for the pcre2grep
+     and pcre2test commands. These text forms are provided for ease of scanning
+     with text editors or similar tools. They are installed in
+     <prefix>/share/doc/pcre2, where <prefix> is the installation prefix
+     (defaulting to /usr/local).
+
+  2. A set of files containing all the documentation in HTML form, hyperlinked
+     in various ways, and rooted in a file called index.html, is distributed in
+     doc/html and installed in <prefix>/share/doc/pcre2/html.
+
+
+Building PCRE2 on non-Unix-like systems
+---------------------------------------
+
+For a non-Unix-like system, please read the file NON-AUTOTOOLS-BUILD, though if
+your system supports the use of "configure" and "make" you may be able to build
+PCRE2 using autotools in the same way as for many Unix-like systems.
+
+PCRE2 can also be configured using CMake, which can be run in various ways
+(command line, GUI, etc). This creates Makefiles, solution files, etc. The file
+NON-AUTOTOOLS-BUILD has information about CMake.
+
+PCRE2 has been compiled on many different operating systems. It should be
+straightforward to build PCRE2 on any system that has a Standard C compiler and
+library, because it uses only Standard C functions.
+
+
+Building PCRE2 without using autotools
+--------------------------------------
+
+The use of autotools (in particular, libtool) is problematic in some
+environments, even some that are Unix or Unix-like. See the NON-AUTOTOOLS-BUILD
+file for ways of building PCRE2 without using autotools.
+
+
+Building PCRE2 using autotools
+------------------------------
+
+The following instructions assume the use of the widely used "configure; make;
+make install" (autotools) process.
+
+If you have downloaded and unpacked a PCRE2 release tarball, run the
+"configure" command from the PCRE2 directory, with your current directory set
+to the directory where you want the files to be created. This command is a
+standard GNU "autoconf" configuration script, for which generic instructions
+are supplied in the file INSTALL.
+
+The files in the GitHub repository do not contain "configure". If you have
+downloaded the PCRE2 source files from GitHub, before you can run "configure"
+you must run the shell script called autogen.sh. This runs a number of
+autotools to create a "configure" script (you must of course have the autotools
+commands installed in order to do this).
+
+Most commonly, people build PCRE2 within its own distribution directory, and in
+this case, on many systems, just running "./configure" is sufficient. However,
+the usual methods of changing standard defaults are available. For example:
+
+CFLAGS='-O2 -Wall' ./configure --prefix=/opt/local
+
+This command specifies that the C compiler should be run with the flags '-O2
+-Wall' instead of the default, and that "make install" should install PCRE2
+under /opt/local instead of the default /usr/local.
+
+If you want to build in a different directory, just run "configure" with that
+directory as current. For example, suppose you have unpacked the PCRE2 source
+into /source/pcre2/pcre2-xxx, but you want to build it in
+/build/pcre2/pcre2-xxx:
+
+cd /build/pcre2/pcre2-xxx
+/source/pcre2/pcre2-xxx/configure
+
+PCRE2 is written in C and is normally compiled as a C library. However, it is
+possible to build it as a C++ library, though the provided building apparatus
+does not have any features to support this.
+
+There are some optional features that can be included or omitted from the PCRE2
+library. They are also documented in the pcre2build man page.
+
+. By default, both shared and static libraries are built. You can change this
+  by adding one of these options to the "configure" command:
+
+  --disable-shared
+  --disable-static
+
+  (See also "Shared libraries on Unix-like systems" below.)
+
+. By default, only the 8-bit library is built. If you add --enable-pcre2-16 to
+  the "configure" command, the 16-bit library is also built. If you add
+  --enable-pcre2-32 to the "configure" command, the 32-bit library is also
+  built. If you want only the 16-bit or 32-bit library, use --disable-pcre2-8
+  to disable building the 8-bit library.
+
+. If you want to include support for just-in-time (JIT) compiling, which can
+  give large performance improvements on certain platforms, add --enable-jit to
+  the "configure" command. This support is available only for certain hardware
+  architectures. If you try to enable it on an unsupported architecture, there
+  will be a compile time error. If in doubt, use --enable-jit=auto, which
+  enables JIT only if the current hardware is supported.
+
+. If you are enabling JIT under SELinux environment you may also want to add
+  --enable-jit-sealloc, which enables the use of an executable memory allocator
+  that is compatible with SELinux. Warning: this allocator is experimental!
+  It does not support fork() operation and may crash when no disk space is
+  available. This option has no effect if JIT is disabled.
+
+. If you do not want to make use of the default support for UTF-8 Unicode
+  character strings in the 8-bit library, UTF-16 Unicode character strings in
+  the 16-bit library, or UTF-32 Unicode character strings in the 32-bit
+  library, you can add --disable-unicode to the "configure" command. This
+  reduces the size of the libraries. It is not possible to configure one
+  library with Unicode support, and another without, in the same configuration.
+  It is also not possible to use --enable-ebcdic (see below) with Unicode
+  support, so if this option is set, you must also use --disable-unicode.
+
+  When Unicode support is available, the use of a UTF encoding still has to be
+  enabled by setting the PCRE2_UTF option at run time or starting a pattern
+  with (*UTF). When PCRE2 is compiled with Unicode support, its input can only
+  either be ASCII or UTF-8/16/32, even when running on EBCDIC platforms.
+
+  As well as supporting UTF strings, Unicode support includes support for the
+  \P, \p, and \X sequences that recognize Unicode character properties.
+  However, only a subset of Unicode properties are supported; see the
+  pcre2pattern man page for details. Escape sequences such as \d and \w in
+  patterns do not by default make use of Unicode properties, but can be made to
+  do so by setting the PCRE2_UCP option or starting a pattern with (*UCP).
+
+. You can build PCRE2 to recognize either CR or LF or the sequence CRLF, or any
+  of the preceding, or any of the Unicode newline sequences, or the NUL (zero)
+  character as indicating the end of a line. Whatever you specify at build time
+  is the default; the caller of PCRE2 can change the selection at run time. The
+  default newline indicator is a single LF character (the Unix standard). You
+  can specify the default newline indicator by adding --enable-newline-is-cr,
+  --enable-newline-is-lf, --enable-newline-is-crlf,
+  --enable-newline-is-anycrlf, --enable-newline-is-any, or
+  --enable-newline-is-nul to the "configure" command, respectively.
+
+. By default, the sequence \R in a pattern matches any Unicode line ending
+  sequence. This is independent of the option specifying what PCRE2 considers
+  to be the end of a line (see above). However, the caller of PCRE2 can
+  restrict \R to match only CR, LF, or CRLF. You can make this the default by
+  adding --enable-bsr-anycrlf to the "configure" command (bsr = "backslash R").
+
+. In a pattern, the escape sequence \C matches a single code unit, even in a
+  UTF mode. This can be dangerous because it breaks up multi-code-unit
+  characters. You can build PCRE2 with the use of \C permanently locked out by
+  adding --enable-never-backslash-C (note the upper case C) to the "configure"
+  command. When \C is allowed by the library, individual applications can lock
+  it out by calling pcre2_compile() with the PCRE2_NEVER_BACKSLASH_C option.
+
+. PCRE2 has a counter that limits the depth of nesting of parentheses in a
+  pattern. This limits the amount of system stack that a pattern uses when it
+  is compiled. The default is 250, but you can change it by setting, for
+  example,
+
+  --with-parens-nest-limit=500
+
+. PCRE2 has a counter that can be set to limit the amount of computing resource
+  it uses when matching a pattern. If the limit is exceeded during a match, the
+  match fails. The default is ten million. You can change the default by
+  setting, for example,
+
+  --with-match-limit=500000
+
+  on the "configure" command. This is just the default; individual calls to
+  pcre2_match() or pcre2_dfa_match() can supply their own value. There is more
+  discussion in the pcre2api man page (search for pcre2_set_match_limit).
+
+. There is a separate counter that limits the depth of nested backtracking
+  (pcre2_match()) or nested function calls (pcre2_dfa_match()) during a
+  matching process, which indirectly limits the amount of heap memory that is
+  used, and in the case of pcre2_dfa_match() the amount of stack as well. This
+  counter also has a default of ten million, which is essentially "unlimited".
+  You can change the default by setting, for example,
+
+  --with-match-limit-depth=5000
+
+  There is more discussion in the pcre2api man page (search for
+  pcre2_set_depth_limit).
+
+. You can also set an explicit limit on the amount of heap memory used by
+  the pcre2_match() and pcre2_dfa_match() interpreters:
+
+  --with-heap-limit=500
+
+  The units are kibibytes (units of 1024 bytes). This limit does not apply when
+  the JIT optimization (which has its own memory control features) is used.
+  There is more discussion on the pcre2api man page (search for
+  pcre2_set_heap_limit).
+
+. In the 8-bit library, the default maximum compiled pattern size is around
+  64 kibibytes. You can increase this by adding --with-link-size=3 to the
+  "configure" command. PCRE2 then uses three bytes instead of two for offsets
+  to different parts of the compiled pattern. In the 16-bit library,
+  --with-link-size=3 is the same as --with-link-size=4, which (in both
+  libraries) uses four-byte offsets. Increasing the internal link size reduces
+  performance in the 8-bit and 16-bit libraries. In the 32-bit library, the
+  link size setting is ignored, as 4-byte offsets are always used.
+
+. For speed, PCRE2 uses four tables for manipulating and identifying characters
+  whose code point values are less than 256. By default, it uses a set of
+  tables for ASCII encoding that is part of the distribution. If you specify
+
+  --enable-rebuild-chartables
+
+  a program called pcre2_dftables is compiled and run in the default C locale
+  when you obey "make". It builds a source file called pcre2_chartables.c. If
+  you do not specify this option, pcre2_chartables.c is created as a copy of
+  pcre2_chartables.c.dist. See "Character tables" below for further
+  information.
+
+. It is possible to compile PCRE2 for use on systems that use EBCDIC as their
+  character code (as opposed to ASCII/Unicode) by specifying
+
+  --enable-ebcdic --disable-unicode
+
+  This automatically implies --enable-rebuild-chartables (see above). However,
+  when PCRE2 is built this way, it always operates in EBCDIC. It cannot support
+  both EBCDIC and UTF-8/16/32. There is a second option, --enable-ebcdic-nl25,
+  which specifies that the code value for the EBCDIC NL character is 0x25
+  instead of the default 0x15.
+
+. If you specify --enable-debug, additional debugging code is included in the
+  build. This option is intended for use by the PCRE2 maintainers.
+
+. In environments where valgrind is installed, if you specify
+
+  --enable-valgrind
+
+  PCRE2 will use valgrind annotations to mark certain memory regions as
+  unaddressable. This allows it to detect invalid memory accesses, and is
+  mostly useful for debugging PCRE2 itself.
+
+. In environments where the gcc compiler is used and lcov is installed, if you
+  specify
+
+  --enable-coverage
+
+  the build process implements a code coverage report for the test suite. The
+  report is generated by running "make coverage". If ccache is installed on
+  your system, it must be disabled when building PCRE2 for coverage reporting.
+  You can do this by setting the environment variable CCACHE_DISABLE=1 before
+  running "make" to build PCRE2. There is more information about coverage
+  reporting in the "pcre2build" documentation.
+
+. When JIT support is enabled, pcre2grep automatically makes use of it, unless
+  you add --disable-pcre2grep-jit to the "configure" command.
+
+. There is support for calling external programs during matching in the
+  pcre2grep command, using PCRE2's callout facility with string arguments. This
+  support can be disabled by adding --disable-pcre2grep-callout to the
+  "configure" command. There are two kinds of callout: one that generates
+  output from inbuilt code, and another that calls an external program. The
+  latter has special support for Windows and VMS; otherwise it assumes the
+  existence of the fork() function. This facility can be disabled by adding
+  --disable-pcre2grep-callout-fork to the "configure" command.
+
+. The pcre2grep program currently supports only 8-bit data files, and so
+  requires the 8-bit PCRE2 library. It is possible to compile pcre2grep to use
+  libz and/or libbz2, in order to read .gz and .bz2 files (respectively), by
+  specifying one or both of
+
+  --enable-pcre2grep-libz
+  --enable-pcre2grep-libbz2
+
+  Of course, the relevant libraries must be installed on your system.
+
+. The default starting size (in bytes) of the internal buffer used by pcre2grep
+  can be set by, for example:
+
+  --with-pcre2grep-bufsize=51200
+
+  The value must be a plain integer. The default is 20480. The amount of memory
+  used by pcre2grep is actually three times this number, to allow for "before"
+  and "after" lines. If very long lines are encountered, the buffer is
+  automatically enlarged, up to a fixed maximum size.
+
+. The default maximum size of pcre2grep's internal buffer can be set by, for
+  example:
+
+  --with-pcre2grep-max-bufsize=2097152
+
+  The default is either 1048576 or the value of --with-pcre2grep-bufsize,
+  whichever is the larger.
+
+. It is possible to compile pcre2test so that it links with the libreadline
+  or libedit libraries, by specifying, respectively,
+
+  --enable-pcre2test-libreadline or --enable-pcre2test-libedit
+
+  If this is done, when pcre2test's input is from a terminal, it reads it using
+  the readline() function. This provides line-editing and history facilities.
+  Note that libreadline is GPL-licenced, so if you distribute a binary of
+  pcre2test linked in this way, there may be licensing issues. These can be
+  avoided by linking with libedit (which has a BSD licence) instead.
+
+  Enabling libreadline causes the -lreadline option to be added to the
+  pcre2test build. In many operating environments with a sytem-installed
+  readline library this is sufficient. However, in some environments (e.g. if
+  an unmodified distribution version of readline is in use), it may be
+  necessary to specify something like LIBS="-lncurses" as well. This is
+  because, to quote the readline INSTALL, "Readline uses the termcap functions,
+  but does not link with the termcap or curses library itself, allowing
+  applications which link with readline the option to choose an appropriate
+  library." If you get error messages about missing functions tgetstr, tgetent,
+  tputs, tgetflag, or tgoto, this is the problem, and linking with the ncurses
+  library should fix it.
+
+. The C99 standard defines formatting modifiers z and t for size_t and
+  ptrdiff_t values, respectively. By default, PCRE2 uses these modifiers in
+  environments other than Microsoft Visual Studio versions earlier than 2013
+  when __STDC_VERSION__ is defined and has a value greater than or equal to
+  199901L (indicating C99). However, there is at least one environment that
+  claims to be C99 but does not support these modifiers. If
+  --disable-percent-zt is specified, no use is made of the z or t modifiers.
+  Instead of %td or %zu, %lu is used, with a cast for size_t values.
+
+. There is a special option called --enable-fuzz-support for use by people who
+  want to run fuzzing tests on PCRE2. At present this applies only to the 8-bit
+  library. If set, it causes an extra library called libpcre2-fuzzsupport.a to
+  be built, but not installed. This contains a single function called
+  LLVMFuzzerTestOneInput() whose arguments are a pointer to a string and the
+  length of the string. When called, this function tries to compile the string
+  as a pattern, and if that succeeds, to match it. This is done both with no
+  options and with some random options bits that are generated from the string.
+  Setting --enable-fuzz-support also causes a binary called pcre2fuzzcheck to
+  be created. This is normally run under valgrind or used when PCRE2 is
+  compiled with address sanitizing enabled. It calls the fuzzing function and
+  outputs information about what it is doing. The input strings are specified
+  by arguments: if an argument starts with "=" the rest of it is a literal
+  input string. Otherwise, it is assumed to be a file name, and the contents
+  of the file are the test string.
+
+. Releases before 10.30 could be compiled with --disable-stack-for-recursion,
+  which caused pcre2_match() to use individual blocks on the heap for
+  backtracking instead of recursive function calls (which use the stack). This
+  is now obsolete because pcre2_match() was refactored always to use the heap
+  (in a much more efficient way than before). This option is retained for
+  backwards compatibility, but has no effect other than to output a warning.
+
+The "configure" script builds the following files for the basic C library:
+
+. Makefile             the makefile that builds the library
+. src/config.h         build-time configuration options for the library
+. src/pcre2.h          the public PCRE2 header file
+. pcre2-config         script that shows the building settings such as CFLAGS
+                         that were set for "configure"
+. libpcre2-8.pc        )
+. libpcre2-16.pc       ) data for the pkg-config command
+. libpcre2-32.pc       )
+. libpcre2-posix.pc    )
+. libtool              script that builds shared and/or static libraries
+
+Versions of config.h and pcre2.h are distributed in the src directory of PCRE2
+tarballs under the names config.h.generic and pcre2.h.generic. These are
+provided for those who have to build PCRE2 without using "configure" or CMake.
+If you use "configure" or CMake, the .generic versions are not used.
+
+The "configure" script also creates config.status, which is an executable
+script that can be run to recreate the configuration, and config.log, which
+contains compiler output from tests that "configure" runs.
+
+Once "configure" has run, you can run "make". This builds whichever of the
+libraries libpcre2-8, libpcre2-16 and libpcre2-32 are configured, and a test
+program called pcre2test. If you enabled JIT support with --enable-jit, another
+test program called pcre2_jit_test is built as well. If the 8-bit library is
+built, libpcre2-posix, pcre2posix_test, and the pcre2grep command are also
+built. Running "make" with the -j option may speed up compilation on
+multiprocessor systems.
+
+The command "make check" runs all the appropriate tests. Details of the PCRE2
+tests are given below in a separate section of this document. The -j option of
+"make" can also be used when running the tests.
+
+You can use "make install" to install PCRE2 into live directories on your
+system. The following are installed (file names are all relative to the
+<prefix> that is set when "configure" is run):
+
+  Commands (bin):
+    pcre2test
+    pcre2grep (if 8-bit support is enabled)
+    pcre2-config
+
+  Libraries (lib):
+    libpcre2-8      (if 8-bit support is enabled)
+    libpcre2-16     (if 16-bit support is enabled)
+    libpcre2-32     (if 32-bit support is enabled)
+    libpcre2-posix  (if 8-bit support is enabled)
+
+  Configuration information (lib/pkgconfig):
+    libpcre2-8.pc
+    libpcre2-16.pc
+    libpcre2-32.pc
+    libpcre2-posix.pc
+
+  Header files (include):
+    pcre2.h
+    pcre2posix.h
+
+  Man pages (share/man/man{1,3}):
+    pcre2grep.1
+    pcre2test.1
+    pcre2-config.1
+    pcre2.3
+    pcre2*.3 (lots more pages, all starting "pcre2")
+
+  HTML documentation (share/doc/pcre2/html):
+    index.html
+    *.html (lots more pages, hyperlinked from index.html)
+
+  Text file documentation (share/doc/pcre2):
+    AUTHORS
+    COPYING
+    ChangeLog
+    LICENCE
+    NEWS
+    README
+    pcre2.txt         (a concatenation of the man(3) pages)
+    pcre2test.txt     the pcre2test man page
+    pcre2grep.txt     the pcre2grep man page
+    pcre2-config.txt  the pcre2-config man page
+
+If you want to remove PCRE2 from your system, you can run "make uninstall".
+This removes all the files that "make install" installed. However, it does not
+remove any directories, because these are often shared with other programs.
+
+
+Retrieving configuration information
+------------------------------------
+
+Running "make install" installs the command pcre2-config, which can be used to
+recall information about the PCRE2 configuration and installation. For example:
+
+  pcre2-config --version
+
+prints the version number, and
+
+  pcre2-config --libs8
+
+outputs information about where the 8-bit library is installed. This command
+can be included in makefiles for programs that use PCRE2, saving the programmer
+from having to remember too many details. Run pcre2-config with no arguments to
+obtain a list of possible arguments.
+
+The pkg-config command is another system for saving and retrieving information
+about installed libraries. Instead of separate commands for each library, a
+single command is used. For example:
+
+  pkg-config --libs libpcre2-16
+
+The data is held in *.pc files that are installed in a directory called
+<prefix>/lib/pkgconfig.
+
+
+Shared libraries
+----------------
+
+The default distribution builds PCRE2 as shared libraries and static libraries,
+as long as the operating system supports shared libraries. Shared library
+support relies on the "libtool" script which is built as part of the
+"configure" process.
+
+The libtool script is used to compile and link both shared and static
+libraries. They are placed in a subdirectory called .libs when they are newly
+built. The programs pcre2test and pcre2grep are built to use these uninstalled
+libraries (by means of wrapper scripts in the case of shared libraries). When
+you use "make install" to install shared libraries, pcre2grep and pcre2test are
+automatically re-built to use the newly installed shared libraries before being
+installed themselves. However, the versions left in the build directory still
+use the uninstalled libraries.
+
+To build PCRE2 using static libraries only you must use --disable-shared when
+configuring it. For example:
+
+./configure --prefix=/usr/gnu --disable-shared
+
+Then run "make" in the usual way. Similarly, you can use --disable-static to
+build only shared libraries.
+
+
+Cross-compiling using autotools
+-------------------------------
+
+You can specify CC and CFLAGS in the normal way to the "configure" command, in
+order to cross-compile PCRE2 for some other host. However, you should NOT
+specify --enable-rebuild-chartables, because if you do, the pcre2_dftables.c
+source file is compiled and run on the local host, in order to generate the
+inbuilt character tables (the pcre2_chartables.c file). This will probably not
+work, because pcre2_dftables.c needs to be compiled with the local compiler,
+not the cross compiler.
+
+When --enable-rebuild-chartables is not specified, pcre2_chartables.c is
+created by making a copy of pcre2_chartables.c.dist, which is a default set of
+tables that assumes ASCII code. Cross-compiling with the default tables should
+not be a problem.
+
+If you need to modify the character tables when cross-compiling, you should
+move pcre2_chartables.c.dist out of the way, then compile pcre2_dftables.c by
+hand and run it on the local host to make a new version of
+pcre2_chartables.c.dist. See the pcre2build section "Creating character tables
+at build time" for more details.
+
+
+Making new tarballs
+-------------------
+
+The command "make dist" creates three PCRE2 tarballs, in tar.gz, tar.bz2, and
+zip formats. The command "make distcheck" does the same, but then does a trial
+build of the new distribution to ensure that it works.
+
+If you have modified any of the man page sources in the doc directory, you
+should first run the PrepareRelease script before making a distribution. This
+script creates the .txt and HTML forms of the documentation from the man pages.
+
+
+Testing PCRE2
+-------------
+
+To test the basic PCRE2 library on a Unix-like system, run the RunTest script.
+There is another script called RunGrepTest that tests the pcre2grep command.
+When the 8-bit library is built, a test program for the POSIX wrapper, called
+pcre2posix_test, is compiled, and when JIT support is enabled, a test program
+called pcre2_jit_test is built. The scripts and the program tests are all run
+when you obey "make check". For other environments, see the instructions in
+NON-AUTOTOOLS-BUILD.
+
+The RunTest script runs the pcre2test test program (which is documented in its
+own man page) on each of the relevant testinput files in the testdata
+directory, and compares the output with the contents of the corresponding
+testoutput files. RunTest uses a file called testtry to hold the main output
+from pcre2test. Other files whose names begin with "test" are used as working
+files in some tests.
+
+Some tests are relevant only when certain build-time options were selected. For
+example, the tests for UTF-8/16/32 features are run only when Unicode support
+is available. RunTest outputs a comment when it skips a test.
+
+Many (but not all) of the tests that are not skipped are run twice if JIT
+support is available. On the second run, JIT compilation is forced. This
+testing can be suppressed by putting "-nojit" on the RunTest command line.
+
+The entire set of tests is run once for each of the 8-bit, 16-bit and 32-bit
+libraries that are enabled. If you want to run just one set of tests, call
+RunTest with either the -8, -16 or -32 option.
+
+If valgrind is installed, you can run the tests under it by putting "-valgrind"
+on the RunTest command line. To run pcre2test on just one or more specific test
+files, give their numbers as arguments to RunTest, for example:
+
+  RunTest 2 7 11
+
+You can also specify ranges of tests such as 3-6 or 3- (meaning 3 to the
+end), or a number preceded by ~ to exclude a test. For example:
+
+  Runtest 3-15 ~10
+
+This runs tests 3 to 15, excluding test 10, and just ~13 runs all the tests
+except test 13. Whatever order the arguments are in, the tests are always run
+in numerical order.
+
+You can also call RunTest with the single argument "list" to cause it to output
+a list of tests.
+
+The test sequence starts with "test 0", which is a special test that has no
+input file, and whose output is not checked. This is because it will be
+different on different hardware and with different configurations. The test
+exists in order to exercise some of pcre2test's code that would not otherwise
+be run.
+
+Tests 1 and 2 can always be run, as they expect only plain text strings (not
+UTF) and make no use of Unicode properties. The first test file can be fed
+directly into the perltest.sh script to check that Perl gives the same results.
+The only difference you should see is in the first few lines, where the Perl
+version is given instead of the PCRE2 version. The second set of tests check
+auxiliary functions, error detection, and run-time flags that are specific to
+PCRE2. It also uses the debugging flags to check some of the internals of
+pcre2_compile().
+
+If you build PCRE2 with a locale setting that is not the standard C locale, the
+character tables may be different (see next paragraph). In some cases, this may
+cause failures in the second set of tests. For example, in a locale where the
+isprint() function yields TRUE for characters in the range 128-255, the use of
+[:isascii:] inside a character class defines a different set of characters, and
+this shows up in this test as a difference in the compiled code, which is being
+listed for checking. For example, where the comparison test output contains
+[\x00-\x7f] the test might contain [\x00-\xff], and similarly in some other
+cases. This is not a bug in PCRE2.
+
+Test 3 checks pcre2_maketables(), the facility for building a set of character
+tables for a specific locale and using them instead of the default tables. The
+script uses the "locale" command to check for the availability of the "fr_FR",
+"french", or "fr" locale, and uses the first one that it finds. If the "locale"
+command fails, or if its output doesn't include "fr_FR", "french", or "fr" in
+the list of available locales, the third test cannot be run, and a comment is
+output to say why. If running this test produces an error like this:
+
+  ** Failed to set locale "fr_FR"
+
+it means that the given locale is not available on your system, despite being
+listed by "locale". This does not mean that PCRE2 is broken. There are three
+alternative output files for the third test, because three different versions
+of the French locale have been encountered. The test passes if its output
+matches any one of them.
+
+Tests 4 and 5 check UTF and Unicode property support, test 4 being compatible
+with the perltest.sh script, and test 5 checking PCRE2-specific things.
+
+Tests 6 and 7 check the pcre2_dfa_match() alternative matching function, in
+non-UTF mode and UTF-mode with Unicode property support, respectively.
+
+Test 8 checks some internal offsets and code size features, but it is run only
+when Unicode support is enabled. The output is different in 8-bit, 16-bit, and
+32-bit modes and for different link sizes, so there are different output files
+for each mode and link size.
+
+Tests 9 and 10 are run only in 8-bit mode, and tests 11 and 12 are run only in
+16-bit and 32-bit modes. These are tests that generate different output in
+8-bit mode. Each pair are for general cases and Unicode support, respectively.
+
+Test 13 checks the handling of non-UTF characters greater than 255 by
+pcre2_dfa_match() in 16-bit and 32-bit modes.
+
+Test 14 contains some special UTF and UCP tests that give different output for
+different code unit widths.
+
+Test 15 contains a number of tests that must not be run with JIT. They check,
+among other non-JIT things, the match-limiting features of the interpretive
+matcher.
+
+Test 16 is run only when JIT support is not available. It checks that an
+attempt to use JIT has the expected behaviour.
+
+Test 17 is run only when JIT support is available. It checks JIT complete and
+partial modes, match-limiting under JIT, and other JIT-specific features.
+
+Tests 18 and 19 are run only in 8-bit mode. They check the POSIX interface to
+the 8-bit library, without and with Unicode support, respectively.
+
+Test 20 checks the serialization functions by writing a set of compiled
+patterns to a file, and then reloading and checking them.
+
+Tests 21 and 22 test \C support when the use of \C is not locked out, without
+and with UTF support, respectively. Test 23 tests \C when it is locked out.
+
+Tests 24 and 25 test the experimental pattern conversion functions, without and
+with UTF support, respectively.
+
+Test 26 checks Unicode property support using tests that are generated
+automatically from the Unicode data tables.
+
+
+Character tables
+----------------
+
+For speed, PCRE2 uses four tables for manipulating and identifying characters
+whose code point values are less than 256. By default, a set of tables that is
+built into the library is used. The pcre2_maketables() function can be called
+by an application to create a new set of tables in the current locale. This are
+passed to PCRE2 by calling pcre2_set_character_tables() to put a pointer into a
+compile context.
+
+The source file called pcre2_chartables.c contains the default set of tables.
+By default, this is created as a copy of pcre2_chartables.c.dist, which
+contains tables for ASCII coding. However, if --enable-rebuild-chartables is
+specified for ./configure, a new version of pcre2_chartables.c is built by the
+program pcre2_dftables (compiled from pcre2_dftables.c), which uses the ANSI C
+character handling functions such as isalnum(), isalpha(), isupper(),
+islower(), etc. to build the table sources. This means that the default C
+locale that is set for your system will control the contents of these default
+tables. You can change the default tables by editing pcre2_chartables.c and
+then re-building PCRE2. If you do this, you should take care to ensure that the
+file does not get automatically re-generated. The best way to do this is to
+move pcre2_chartables.c.dist out of the way and replace it with your customized
+tables.
+
+When the pcre2_dftables program is run as a result of specifying
+--enable-rebuild-chartables, it uses the default C locale that is set on your
+system. It does not pay attention to the LC_xxx environment variables. In other
+words, it uses the system's default locale rather than whatever the compiling
+user happens to have set. If you really do want to build a source set of
+character tables in a locale that is specified by the LC_xxx variables, you can
+run the pcre2_dftables program by hand with the -L option. For example:
+
+  ./pcre2_dftables -L pcre2_chartables.c.special
+
+The second argument names the file where the source code for the tables is
+written. The first two 256-byte tables provide lower casing and case flipping
+functions, respectively. The next table consists of a number of 32-byte bit
+maps which identify certain character classes such as digits, "word"
+characters, white space, etc. These are used when building 32-byte bit maps
+that represent character classes for code points less than 256. The final
+256-byte table has bits indicating various character types, as follows:
+
+    1   white space character
+    2   letter
+    4   lower case letter
+    8   decimal digit
+   16   alphanumeric or '_'
+
+You can also specify -b (with or without -L) when running pcre2_dftables. This
+causes the tables to be written in binary instead of as source code. A set of
+binary tables can be loaded into memory by an application and passed to
+pcre2_compile() in the same way as tables created dynamically by calling
+pcre2_maketables(). The tables are just a string of bytes, independent of
+hardware characteristics such as endianness. This means they can be bundled
+with an application that runs in different environments, to ensure consistent
+behaviour.
+
+See also the pcre2build section "Creating character tables at build time".
+
+
+File manifest
+-------------
+
+The distribution should contain the files listed below.
+
+(A) Source files for the PCRE2 library functions and their headers are found in
+    the src directory:
+
+  src/pcre2_dftables.c     auxiliary program for building pcre2_chartables.c
+                           when --enable-rebuild-chartables is specified
+
+  src/pcre2_chartables.c.dist  a default set of character tables that assume
+                           ASCII coding; unless --enable-rebuild-chartables is
+                           specified, used by copying to pcre2_chartables.c
+
+  src/pcre2posix.c         )
+  src/pcre2_auto_possess.c )
+  src/pcre2_compile.c      )
+  src/pcre2_config.c       )
+  src/pcre2_context.c      )
+  src/pcre2_convert.c      )
+  src/pcre2_dfa_match.c    )
+  src/pcre2_error.c        )
+  src/pcre2_extuni.c       )
+  src/pcre2_find_bracket.c )
+  src/pcre2_jit_compile.c  )
+  src/pcre2_jit_match.c    ) sources for the functions in the library,
+  src/pcre2_jit_misc.c     )   and some internal functions that they use
+  src/pcre2_maketables.c   )
+  src/pcre2_match.c        )
+  src/pcre2_match_data.c   )
+  src/pcre2_newline.c      )
+  src/pcre2_ord2utf.c      )
+  src/pcre2_pattern_info.c )
+  src/pcre2_script_run.c   )
+  src/pcre2_serialize.c    )
+  src/pcre2_string_utils.c )
+  src/pcre2_study.c        )
+  src/pcre2_substitute.c   )
+  src/pcre2_substring.c    )
+  src/pcre2_tables.c       )
+  src/pcre2_ucd.c          )
+  src/pcre2_ucptables.c    )
+  src/pcre2_valid_utf.c    )
+  src/pcre2_xclass.c       )
+
+  src/pcre2_printint.c     debugging function that is used by pcre2test,
+  src/pcre2_fuzzsupport.c  function for (optional) fuzzing support
+
+  src/config.h.in          template for config.h, when built by "configure"
+  src/pcre2.h.in           template for pcre2.h when built by "configure"
+  src/pcre2posix.h         header for the external POSIX wrapper API
+  src/pcre2_internal.h     header for internal use
+  src/pcre2_intmodedep.h   a mode-specific internal header
+  src/pcre2_jit_neon_inc.h header used by JIT
+  src/pcre2_jit_simd_inc.h header used by JIT
+  src/pcre2_ucp.h          header for Unicode property handling
+
+  sljit/*                  source files for the JIT compiler
+
+(B) Source files for programs that use PCRE2:
+
+  src/pcre2demo.c          simple demonstration of coding calls to PCRE2
+  src/pcre2grep.c          source of a grep utility that uses PCRE2
+  src/pcre2test.c          comprehensive test program
+  src/pcre2_jit_test.c     JIT test program
+  src/pcre2posix_test.c    POSIX wrapper API test program
+
+(C) Auxiliary files:
+
+  132html                  script to turn "man" pages into HTML
+  AUTHORS                  information about the author of PCRE2
+  ChangeLog                log of changes to the code
+  CleanTxt                 script to clean nroff output for txt man pages
+  Detrail                  script to remove trailing spaces
+  HACKING                  some notes about the internals of PCRE2
+  INSTALL                  generic installation instructions
+  LICENCE                  conditions for the use of PCRE2
+  COPYING                  the same, using GNU's standard name
+  Makefile.in              ) template for Unix Makefile, which is built by
+                           )   "configure"
+  Makefile.am              ) the automake input that was used to create
+                           )   Makefile.in
+  NEWS                     important changes in this release
+  NON-AUTOTOOLS-BUILD      notes on building PCRE2 without using autotools
+  PrepareRelease           script to make preparations for "make dist"
+  README                   this file
+  RunTest                  a Unix shell script for running tests
+  RunGrepTest              a Unix shell script for pcre2grep tests
+  aclocal.m4               m4 macros (generated by "aclocal")
+  config.guess             ) files used by libtool,
+  config.sub               )   used only when building a shared library
+  configure                a configuring shell script (built by autoconf)
+  configure.ac             ) the autoconf input that was used to build
+                           )   "configure" and config.h
+  depcomp                  ) script to find program dependencies, generated by
+                           )   automake
+  doc/*.3                  man page sources for PCRE2
+  doc/*.1                  man page sources for pcre2grep and pcre2test
+  doc/index.html.src       the base HTML page
+  doc/html/*               HTML documentation
+  doc/pcre2.txt            plain text version of the man pages
+  doc/pcre2test.txt        plain text documentation of test program
+  install-sh               a shell script for installing files
+  libpcre2-8.pc.in         template for libpcre2-8.pc for pkg-config
+  libpcre2-16.pc.in        template for libpcre2-16.pc for pkg-config
+  libpcre2-32.pc.in        template for libpcre2-32.pc for pkg-config
+  libpcre2-posix.pc.in     template for libpcre2-posix.pc for pkg-config
+  ltmain.sh                file used to build a libtool script
+  missing                  ) common stub for a few missing GNU programs while
+                           )   installing, generated by automake
+  mkinstalldirs            script for making install directories
+  perltest.sh              Script for running a Perl test program
+  pcre2-config.in          source of script which retains PCRE2 information
+  testdata/testinput*      test data for main library tests
+  testdata/testoutput*     expected test results
+  testdata/grep*           input and output for pcre2grep tests
+  testdata/*               other supporting test files
+
+(D) Auxiliary files for cmake support
+
+  cmake/COPYING-CMAKE-SCRIPTS
+  cmake/FindPackageHandleStandardArgs.cmake
+  cmake/FindEditline.cmake
+  cmake/FindReadline.cmake
+  CMakeLists.txt
+  config-cmake.h.in
+
+(E) Auxiliary files for building PCRE2 "by hand"
+
+  src/pcre2.h.generic     ) a version of the public PCRE2 header file
+                          )   for use in non-"configure" environments
+  src/config.h.generic    ) a version of config.h for use in non-"configure"
+                          )   environments
+
+Philip Hazel
+Email local part: Philip.Hazel
+Email domain: gmail.com
+Last updated: 10 December 2022